MonoFlow: A Unified Generative Modeling Framework for GAN Variants
Authors: Mingxuan Yi
Zhanxing Zhu
Song Liu
Venue: Submitted to ICLR 2023 (2023)

Abstract: Generative adversarial networks (GANs) play a minmax two-player game via adversarial training. The conventional understanding of adversarial training is that the discriminator is trained to estimate a divergence and the generator learns to minimize this divergence. We argue that despite the fact that many variants of GANs are developed following this paradigm, the existing theoretical understanding of GANs and the practical algorithms are inconsistent. In order to gain deeper theoretical insights and algorithmic inspiration for these GAN variants, we leverage Wasserstein gradient flows which characterize the evolution of particles in the sample space. Based on this, we introduce a unified generative modeling framework – MonoFlow: the particle evolution is rescaled via an arbitrary monotonically increasing mapping. Under our framework, adversarial training can be viewed as a procedure first obtaining MonoFlow's vector field via the discriminator and then the generator learns to parameterize the flow defined by the corresponding vector field. We also reveal the fundamental difference between variational divergence minimization and adversarial training. These analysis help us to identify what types of generator loss functions can lead to the successful training of GANs and suggest that GANs may have more loss designs beyond those developed in the literature, e.g., non-saturated loss, as long as they realize MonoFlow. Consistent empirical studies are also included to validate the effectiveness of our framework.

Under review as a conference paper at ICLR 2023
MONOFLOW: A UNIFIED GENERATIVE MODELING FRAMEWORK FOR DIVERGENCE GANS
Anonymous authors Paper under double-blind review
ABSTRACT
Generative adversarial networks (GANs) play minmax two-player game via ad- versarial training. The conventional understanding of adversarial training is that the discriminator is trained to estimate divergence and the generator learns to minimize this divergence. We argue that despite the fact that many variants of GANs are developed following this paradigm, the existing theoretical understand- ing of GANs and the practical algorithms are inconsistent. In order to gain deeper theoretical insights and algorithmic inspiration for these GAN variants, lever- age Wasserstein gradient flows which characterize the evolution of particles the sample space. Based on this, introduce unified generative modeling framework - MonoFlow: the particle evolution is rescaled via an arbitrary mono- tonically increasing mapping. Under our framework, adversarial training viewed as a procedure first obtaining MonoFlow's vector field via the discrimi- nator and then the generator learns to parameterize the flow defined by the corre- sponding vector field. We also reveal the fundamental difference between varia- tional divergence minimization and adversarial training. This analysis helps us to identify what types of generator loss functions can lead to the successful training GANs and suggest that GANs may have more loss designs beyond those devel- oped in the literature, e.g., non-saturated loss, as long as they realize MonoFlow. Consistent empirical studies are also included to validate the effectiveness of our framework.
be
INTRODUCTION
Generative adversarial nets (GANs) (Goodfellow et al. 2014 Jabbar et al. 2021) are a powerful generative modeling framework that has gained tremendous attention in recent years. GANs have achieved significant successes in applications, especially in high-dimensional image processing such
as high-fidelity image generation (Brock et al. 2018 Karras et al., 2019), super-resolution (Ledig let al. 2017) and domain adaption (Zhang et al. 2017
In the GAN framework. discriminator d and a generator play a minmax game. The discriminator is trained distinguish real and fake samples and the generator is trained to generate fake samples to fool the discriminator. The equilibrium of the vanilla GAN is defined by
min max V(g, d) Ex-puan log o [[d(x)) + Ex-p, flog (1 - [(d(g()))))
d
The elementary optimization approach to solve the minmax game is adversarial training. Previous perspectives explained it as first estimating Jensen- Shannon divergence then the generator learns to minimize this divergence. Several variants of GANs have been developed based this point of view for other probability divergences, e.g., x² divergence (Mao et al. 2017]. Kullback-Leinn (KL) divergence (Arbel al. (2021) and general /-divergences (Nowozin al. 2016 Uehara al.) 2016), while others are developed with Integral Probability Metrics (Arjovsky et al. 2017] Dziugaite al. 2015 MMoueh et al., 20186). However, we emphasize that the traditional understanding over GANs is incomplete and here we present three non negligglie facts which are commonly associated with adversarial training, making it different from the standard variational divergence minimization (VDM) problem (Blei et al. (017):
We use slightly different notation: d(x) is the logit output of the classifier and a(-) is the Sigmoid activation.

---

Under review as a conference paper at ICLR 2023
1. The estimated divergence is computed from the discriminator d(x). It is function only depending on samples x and cannot capture the variability of the generator's 's distribution Pg- However, the optimal discriminator in Proposition 1 by Goodfellow et al. (2014) requires Pg to be a functional variable as well. This issue was also raised in (Metz et al. 2017
Franceschi et al. 2022)..
2. The generator typically minimizes divergence with missing term, e.g.. the vanilla GAN only minimizes the second term of Jensen-Shannon divergence -Ezz- lolog = where - log -=d(g(z)) is a monotonically in- creasing function of d(g(z)).
3. Practical algorithms
inconsistent with the theory, a heuristic trick "nton-aturated loss" is commonly adopted to mitigate the gradient vanishing problem, but it still lacks rigorous mathematical understanding. For example, the generator can minimize Ezzp, {log dd([g)))]] where log a [d[[(())) is also a monotonically increasing function of d(g(z)).
is known the logit output d(x) of binary classification problem in Eq. ! is the logarithm density ratio estimator between two distributions (Sugiyama et al. 2012). To gain a deeper understanding of adversarial training of GANs, we study the Wasserstein gradient flow of the KL divergence which characterizes a deterministic evolution of particles described by an ordinary differential equation (ODE). This ODE is Euclidean gradient flow of time-dependent log density ratio. Based on this ODE, we propose the MonoFlow framework transforming the log density ratio by monotoni- cally increasing mapping such that the vector field of the gradient flow is rescaled along the same direction. Consequently, approximating and learning to parameterize MonoFlow is synonymous with adversarial training. Under our framework, we gain comprehensive understanding of training dynamics over GANs: the discriminator obtains a bijection of the log density ratio that suggests the vector field and the generator learns to parameterize the particles MonoPlow. All variants of divergence GANs are subclass of our framework. Finally, we reveal that the discriminator and generator loss do not need to follow the same objective. The discriminator maximizes an objective to obtain a bijection of the log density ratio. Then the generator loss be any monotonically increasing mapping of this log density ratio. Our contributions are summarized as follows:
A novel generative modeling framework unifies divergence GANs, providing a new theoret- ically and practically consistent understanding of the underlying mechanism of the training dynamics over GANs.
We reveal the fundamental difference between VDM and adversarial training, which in- dicates that the previous analysis of GANs based on the perspective of VDM might not provide benefits and instead we should treat GANs as a particle flow method.
An analysis of what types of generator loss functions can lead to the success of training GAN. Our framework explains why and how non-saturated loss works.
An algorithmic inspiration where GANs may have more variants of loss designs than have already known.
2 WASSERSTEIN GRADIENT FLOWS
In this section, we review the definition of gradient flows in Wasserstein space (P(R"), W2), the space of Borel probability measures P(R") defined on R with finite second moments and Eq.uipped with the Wasserstein-2 metric. An absolutely continuous curve of probability measures (gcho>0 E ((R") is a Wasserstein gradient flow if it satisfies the following continuity equation (Ambrosio et al. 2008),
Ög: div (gt V W. F())).
H.
where Vw,F(u is called the Wasserstein gradient of the functional F: P(R") I. The Wasser- stein gradient is defined as Vx &7 i.e. the Fuclidean gradient of the functional's first variation Se.
sF(a)) Specifically, for the functional Flg:) f log it dg: as the KL divergence where p is fixed öge
p target probability measure, we have 8F(())) log şt + 1. Hence, the Wasserstein gradient flow of
p
2

---

Under review as a conference paper at ICLR 2023
Wasserstein space:
Euclidean space:
Xt St
V F(a))
Ut
Uj
6f Vx ögg
(x)
P
go
W2 (20. P)
Figure I: The illustration of a Wasserstein gradient flow and its particle evolution. In Wasserstein space, the blue curve is gradient flow and the red dotted line a geodesic. qe evolves along a curve whose tangent vector is given by V F(qe) such that the functional is always decreasing with time. Correspondingly, particles evolve Euclidean space towards the target measure p with the vector field Vx &2 (x). Note that directly minimizing the Wasserstein-2 metric W2(1t, p) instead óg: yields path {1x 120 along the geodesic connecting Yo and p over Wasserstein space.
the KL divergence reads the Fokker-Planne Planck equation,
dq: = div ( Vx log de Vx log
As t - oo, the stationary probability measure of qr is the target p. Denote the Euclidean path of random variables as fxt h20 E R" with the initial condition Xy ~ qu. we can define an ordinary differential equation (ODE) which describes the evolution of particles in R".
dx, (Vx logp(x,) Vx log u (xx)))(( v, (x,)dl,, Xo qu-
(4)
where the vector field v, of these particles is the negative Euclidean gradient of the functional's first variation. As shown in Figure Wasserstein gradient flows establish connection between the probability evolution in Wasserstein space and its associated particle evolution in Euclidean space.
Applying Itô integral to Langevin dynamics dx; Vx log p(x,)(ttt.........) )dt + /2dw where dw is Wiener process, we obtain the same Fokker-Pland Planck equation in Eq. (3). This indicates that the deterministic particle evolution by the ODE can be approximated via a stochastic differential equation (SDE). Langevin dynamics admits the same marginal probability measure 4: as Eq. 44), this relation of SDE and its corresponding ODE were also studied in score-based diffusion models (Song et al.) 202 Langevin dynamics was first interpreted as the Wasserstein gradient flow of the KL divergence by Jordan et al. (1998;; Otto 2001). It plays important role in generative modeling scheme. In order to transform noises into the target data distribution by Langevin dynamics, an
sampling
essential step is to fit the data distribution using mvergy-based models (Song & Kingma 2021) or
to directly estimate its scores with score-matching techniques (Hyvärinen & Dayan 2005 Vincent
2011 Song & Ermon
2019)
MoNOFL.OW: A UNIFIED GENERATIVE MODELING FRAMEWORK
This section presents our main contribution that connects gradient flows and divergence GANs into a unified framework. We first introduce MonoFlow where the ODE evolution rescaled via a monotonically increasing function. Consequentially, learning to parameterize the rescaled flow by neural network recovers the bi-level optimization dynamics of training GANs. This gives us a novel understanding of the hidden mechanism of adversarial training.
3.1 MONOFLoW
We consider the ODE in Eq. 44) with a fixed target measure p, e.g., a data distribution in a generative modeling scenario. Assume that have time-dependent log density ratio function as logr.(x) p(x) the ODE can be rewritten as
log (x))
dx, Vx log r (xx)),t Xo qo-
(5)
This is gradient flow in Euclidean space where its vector field is the gradient of the log density ratio. With a monotonically increasing (strict) mapping h: R + where is first-order differentiable, we can define another ODE:
dx, Vxh( log (xx)) N' W'ooo Volg) lx))d, Xo go
3

---

Under review as a conference paper at ICLR 2023
By transforming the time-dependent log density ratio under the mapping h. its first-order deriva- tive rescales the vector field of the original gradient flows defined in Eq. We call Eq. (6) as MonoFlow.
MonoFlow defines a different family of vector fields (ve for the particle evolution where (x())) = h' (1ogrr( Vx log ri (xt). Conversely, the vector fields fer also determine an absolutely continuous curve {4: 120 in Wasserstein space by the continuity equation, see Theorem 4.6 in (Ambrosio et al. 2008
Dq. -iviv(gev))
under mild regularity conditions. Hence the probability evolution of MonoFlow is described by
ōq. div div(D,Vxx)) div q Vx logp))
(8)
Ot
where D. =
Eq. (8) is special case of convection-diffusion equations where D,
called the diffusion coefficient and G-= is called mobility. MonoFlow defines positive diffusion coefficient. This has physical interpretation that particles diffuse to spread probability mass over the target measure other than concentrate. Next, we study the properties of MonoFlow.
Theorem 3.1. If h is strictly increasing, ie., 11(.) > divergence F(g)) = log 9t dq, satisfies
0, the dissipation rate 0F(g+) for the KI.
8t
p
0F(g:)
<0
Öt
the equality is achieved if and only if qt = and the marginal probability of MonoFlow evolves p OC.
Theorem shows that MonoFlow does not disturb the equilibrium of Eq. and the convergence the target probability measure p is guaranteed. The negative dissipation rate ensures that the gradient flow curve (0:h2 of MonoFlow always decreases the KL divergence with time.
MonoFlow is obtained by transforming the log density ratio which arises from the Wasserstein gradient flow of KI. divergence. We can also obtain other forms of deterministic particle evolution by considering Wasserstein gradient flows of general f-divergences,
= =
Dy(ll =
dg,
(10)
where f: R+ + IR is convex function and f(1) = 0.
Theorem 3.2. The Wasserstein gradient flow of an f-divergence characterizes the evolution of particles in R by
dx: =x==)(xx)3 (r(x))) log (x:) Xo ~ Qo
(1I)
similar result can also be derived with the reversed f f-divergences (q((p)) used by (Gao al, 2019 Ansari et al. 2021). Theorem 3.2 shows that the particle evolution of the Wasserstein gradient flow of /-diverggences is special instance of MonoFlow where h'(logr) = (rr) > because is convex function. It indicates once curve (gerr2 evolves with the time l in Wasserstein space to decrease an fddivergence, it simultancously decreases the KL divergence as well since the dissipation rate of MonoFlow is negative.
3.2 PRACTICAL APPROXIMATIONS DENSITY RATIOS
We first discretize the ODE in Eq. 66) by the forward Euler method such that we obtain standard gradient ascent iterations with step size a and the index of the discretized time step &
X = X + V logrx (xx)), tx+1 = tx + a.
(12)
Therefore, we can sample initial particles mating the density ratio rx)
and perform gradient ascent iterations by esti- x(x) using samples from qk and p. In order enable practical
<x(x
For the sake of simplicity, we hriefly replace t by its index though it is not rigorous.
4

---

Under review as a conference paper at ICLR 2023
Table 1: Different types of divergence GANs. is a convex function and f is the convex conjugate
by f(d) f {rd - f(r)}.
((d)
dx(()
lr(
Vanilla GAN
log (ogo(d) log (d) d
log(1 o(d)
log r(x)
log(1 -o(d) logo(d)
Non-saturated GAN
log(1 - od))
log (x)
f-GAN
-f(d)
f'(r(x)
d
6-GAN
f'(d
f(d) - (d)
r(x)
dd) f(d)
Least-square GAN - (d 1)²
-df
r(x)
-(d - 1)²
I+r(x)
Generalized EBM (KL)
- -dd+ X)
exp(-d - A)
log r
exp(-d X)
algorithm to obtain the time- deme-depnndett density ratio, we introduce a general framework that solves the following optimization problem, similar to Moustakides Basioti (2019).
max
[Ex~ [o(d(x))] + Ex-qs [+x((())]] },
(13)
dE
where d: IR" + IR is a discriminator and is class of measurable functions. and vi are scalar functions upon design later.
Lemma 3.3. Solving Eq. 4.3. the optimal d' satisfies
p(x) r(x) (x(x)'
d' dxxx=
(14)
with T(d(x)) :=
x' Note that the resulting mapping T must be a bijection such that its
(d(x))
inverse exists. Additional conditions on the hypothesis of & and sb can be found in Appendix A.3
Remark: Note that two-sample density ratio estimations discard the density information from Te- The functions d(x), r(x) only depend on x and they cannot capture the variability gN-
train d solve the optimization problem in Eq. (3) and the density ratio is approximated by (d(x)). For example, in standard binary classification problem where we can design (dd) log (dd) and (bdd) log(1 a (d)). we have d (x) log r(x) (Sugiyama al., 2012). Other types of density ratio estimation can be found in Table [ as they have been already used in GAN variants where db refers the generator's distribution Py. practice, since the change of X is sufficiently small at every step k, we can use a single discriminator d(x) and perform a few gradient updates to solve Eq. (13) per iteration to approximate the time-dependent density ratio (x), which is identical to GAN training.
To this end,
3.3 PARAMETERIZATION OF THE DISCRETIZED MONOFLOW
The previous method directly pushes particles in the Euclidean space towards the target distribution. We neural network generator to mimic the distribution of these particles, i.e., the generator learns to draw samples. we parameterize particles with a neural network generator that takes as input random noises z Px and output particles X(a,x) (z), the infinitesimal change of parameters of the generator is
dd;
(ge,(z) vxh (logr:(x;)) where x, ge, (z),
(15)
dt
00,
where 8ga (x) 20,
is the Jacobian of the neural network generator. Apply the forward Buler method with the step size 3, have
0x+1 = 0₂ P2 [h( log Tx (90, (z)))]-
(16)
Eq. [5) can be regarded as amortizing particles of the gradient flow into a neural network gener-
ator by approximately solving 0x+1 arg ming (x>p21lgo(z) x 1 where X go, (z) +
V log (go, (z))) with one-step gradient descent (Wang & Liu 2017)
Consequently, by the chain rule we have cx,,
0go. (z) d0,, replace d0, with Eq. 15 00,
dx, Exxirp [K] (2, 2 log rx:))) di
(17)
5

---

Under review as a conference paper at ICLR 2023
where (2, x²)
(-00,
(x) 8ge, (x2) the neural tangent kernel (NTK) (Jacot et al. 2018) defined
30, 20,
by the generator. Note that Eq. (17) realizes Stein Variational Gradient Descent (Liu & Wang 2016
Franceschi et al. 2022 if h is an identity mapping.
3.4 A UNIFIED FORMULATION OF DIVERGENCE GENERATIVE ADVERSARIAL NETS
Based on the above derivation, we propose general formulation for divergence GANs. We clarify that GANs be treated with different objective functions for training discriminators and genera- tors. All of these variants are algorithmic instantiations of the parameterized MonoFlow. The unified framework is summarized as: given discriminator d and a generator g, the discriminator d learns to maximize
Ex-openn [o(d(x))] Ez-px [v(dg()))))
(18)
where Piata refers to the data distribution. Next, train the generator g to minimize
-Egopz (d(g(()))))
(19)
where (d) hlog T(d) and h be any strictly increasing function. We summarize some typical GAN variants in Table T We view adversarial training maximizing Eq. (T8) to obtain the
density ratio which suggests the vector field for MonoFlow and minimizing Eq. (19) as learning to
parameterize MonoFlow corresponding to Eq. (16). Especially, we find that b-GAN (Uehara et al.)
2016) realizes parameterized Wasserstein gradient flows of f-divergences since its generator loss aligned with Theorem 3.2 for the design of h.
4 UNDERSTANDING ADVERSARIAL TRAINING VIA MONOFLOW
The dominating understanding of adversarial training GANs is that the generator learns to minimize the divergence estimated from the discriminator: However, as pointed out in Section [ the theoretical explanation of GAN and its practical algorithm inconsistent. In this section, through the lens of MonoFlow, we will cxplain why this inconsistency of adversarial training still can lead to convergence the target distribution and how it differs from variational divergence minimization (VDM) problem for generative modeling.
4.1 WHY THE ADVERSARIAL GAME WORKS?
In an adversarial game, the discriminator is trained to maximize the lower bound of f-divergences. This lower bound can be derived via the dual representation of f-divergences (Nguyen et al. 2010) between Pdata and Pg-
Pdatu (x)
Dy (pamalllg = max Exexpan [d(x)] de H
Rxx-p [f((x))]]
} r(x)
(20)
Pg (x)
lower bound
where f(d) supredom, (rd fr)) } is the convex conjugate of f(r) and H. is class of any measurable functions. Note that for binary classification problems where we design specific 6 and d. the corresponding optimization problem in Eq. (18) be translated into an equivalent formulation as the above dual representation (Nowozin et al. 2016). Since the first term of the lower bound in Eq. 20 is irrelevant to Py. the generator actually only learns to minimize the second term,
min min-Ex-p, [f(()))
(21)
Meanwhile, the generator can also alternatively minimize the heuristic non-saturated loss -Ex~p, [d(x)]. which has been proven to work well in practice. By the Fenchel duality, the optimal d' given by d' = fr) with the equality f(d") = (r) f(r). Fortunately, can be simply verified that (r) and (r) f(r are both monotonically increasing functions of the density ratio (as well as the log density ratio). Hence, adversarial training with the vanilla loss and the non-saturated loss both fall into the framework of MonoFlow which has theoretical guarantees.

---

Under review as a conference paper at ICLR 2023
Table 2: Comparisons the convergence when using different ʃ and h on three density ratio models: "V" means the generator converges to the data distribution and "x" means it does not converge. For r(x,0) 0) and r(x, 0de), the convergences means the generator parameter 0 can converge to the true value. For (x),(x), convergence means the parameter can closely approximate the true value. Visualization results included in Appendix [.1
if f convex
if h mono increases
r(x, 0)
r(x, 0sk)
TGAN
KL Forward KL
Yes
Yes
Yes
No
X
AXX
X
Chi-Square
Yes
No
V
X
Hellinger
Yes
No
X
Lensen-Shannon
Yes
No
V
X
Exp
No
Yes
4.2 DIFFERENCE BETWEEN ADVERSARIAL TRAINING AND VARIATIONAL DIVERGENCE MINIMIZATION
VDM differs from adversarial training because it requires density information from the generator, as elaborated in the following. In variational divergence minimization, the generator Go usually defines distribution via an explicit density function Py (x: 0). For example, can be reparameterized as a Gaussian random variable where 0 are its mean and scale. We are interested in minimizing an f-divergence
Dy(Psaullly Ipg)) = Ex-p,
0)) Cost(O),
(22)
where the density ratio r(x, 0) = plm(x) is function of x as well as 0 to capture the variability (x();
of Pg. Integrate out x, the f-divergence can be written cost function of A. Since is convex, such that by Jensen inequality, this cost is minimized at zero when r(x, 8) is a constant for each x, can rewrite f-divergences
meaning Pg
Pdata- Similarly under Fenchel-duulity.
Cost(O)= = Exx-pan [d' (x, 0)] Ex-p, (x)
(23)
where d' (x(x 0) fr((x, 0)). Eq. 23 is different from Eq. 20), where Cost(O) depends on the first term of the dual representation. However, in an adversarial training scenario, the density ratio estimator rxx) or its bijection d(x) are only functions of the sample x. It was also discussed by Metz al. (2017): |Franceschi et al. (2022), the smoothness between the discriminator and the variability of Pa is lost during the practical algorithm. Plugging r(x) or d(x) into the f-divergence to replace r(x, 0) or d' (x, 0), we can approximate f-divergences but the approximated divergences can never be viewed cost function of & anymore. This is the major disconnection between the theory and the practical algorithm over GANs, as Eq. (4) in (Goodfellow et al. 2014) is similar to Eq. (23).
4.3 EMPIRICAL STUDY OF GAUSSIANS
This part highlights the differences between density ratio models in terms of the convergence of generators. We start from the simplest form of generator Pg. x μ + s2, where N (0, /) and 0 = (m, 8), where ju is the mean and is the scale matrix. Let the data distribution be Plata N 50 so). By assuming the generator and data distributions Gaussians, we can define three density ratio models. The first model is r(x, 6)
(lm(x)) Ps (x:0)' where the density ratio function depends
on x and 0 simultaneously. The second model is r(x, = Pan
Pa(x:la)) where Ode means we detach
the gradient of such that the second model cannot reflect the variability of Pg. The third model is TxAN(x) where the density ratio is approximated by performing single gradient update for the classifier using samples in standard adversarial training. We train the generator to minimize the following loss function with the above three density ratio models respectively (for (x), we use the standard bi-level training).
min Ex [f()) equivalently min -Ez-pz [h(logr)]
(24)
Given frr we can rewrite it as a function of log density ratio h(logr) = -r)). Similarly, given h(logr), we write it function of density ratio f(r) h(lgr)) In this experiment, we consider five types of f-divergences (expressions summarized in Appendix B.I). In addition,
study strictly increasing function h given by h(llgr) = exp logr) 115 where its

---

Under review as a conference paper at ICLR 2023
Vanilla
C-0 IVanilla GAN)
Saturated
Logit
h(d)
-4
8
-6
-2
d
Figure 2: The plot of different generator losses as function of
Figure 3: The plots of the vanilla losses by adding different Cs.
f(r) - --1.5 is concave. The results are summarized in Table 2 which verifies our analysis that objective of VDM should be convex function of the density ratio, whereas MonoFlow only requires to be an increasing function the log density ratio. r(x. fde) can recover the true f-divergence, but minimizing this f-divergence has effects except for KL divergence.
Remark: Minimizing the KL divergence using the detached model r(x, 0sc) also leads to conver- gence. This mechanism is called "Variational Inference via Wassenstein Gradient Flows" (Lambert et al. 2022) and it also have a lower variance (Roeder et al. 2017).
5
ALGORITHIMIC INSIGHTS: ALTERNATIVES OF GENERATOR Loss
5.1 ANALYZING PRACTICAL EFFECTIVENESS OF GENERATOR LOsS
In this part, we explain what types of generator loss functions can lead to the successful training of GANs via the lens of MonoFlow. We provide study for the vanilla GAN since the logit output of the binary classifier is the log density ratio where we have d(x) = log r(x) (see Table D. We consider four generator losses which are monotonically increasing functions of the log density ratio: 1). Vanilla loss: h(d)) = - log( 1 - (d)); 2). Non-saturated (NS) loss: h(d) = log(ord))) 3).
Maximum likelihood loss: h(d) exp(d). 4). Logit loss: h(d) = d.
The plot of these functions is shown Figure 2. The vanilla loss and the MLE loss do not work well in practice (Goodfellow) 2016), since initial training steps, the generator is weak and d(x) log (x) < 0, for ~ Py x). We may observe Figure 2, the curves of the vanilla loss and the MLE loss are fairly flat on the left side, which means the derivative I²(.)) is nearly zero. According
Py(x)
to Eq. (6. such rescaling scheme yields extremely small vector fields, resulting in the generator being trapped at the initial steps as the infinitesimal change of particles dx, O. The NS and logit loss both have non-zero derivatives when d(x) < 0 despite that the NS loss is flat on the right side. This is not problem since d(x) gradually increases from negative value during the training and when d(x) = 0, it means Py = Pdata such that the generator converges.
5.2 AN EMBARRASSINGLA SIMPLE TRICK TO FIX VANILLA GAN
We have justified that GANs can work with any generator loss as long as it is a monotonically increasing mapping of the log density ratio and this mapping's derivative deviates from zero when the log r(x) & O. We show effects shifting the generator loss the vanilla GAN left by adding constant C' to the sigmoid function,
h(d) - - ((
(25)
By adding a constant, we can obtain better monotonically increasing function whose derivative deviates from zero significantly, see 3 The neural network architecture used here is DCGAN (Radford et al., 2015) and we follow the vanilla GAN framework where the log density ratio is obtained by logit output from the binary classifier and the model trained with 15 epochs. The generated samples are shown in Figure We observe that when C' = and C' = 5, the generator losses in Eq. (25) begin to work, i.e., generators output plausible fake images.
8

---

Under review as a conference paper at ICLR 2023
Q
6
4
2
4
3
5
3
6
8
1
1
9
5
7
3
4
4
3
3
1
2
9
0
8
5
9
7
4
0
4
7
6
8
9
7
4
0
0
4
4
9 6
3
7
5
4
3
5
9
8 3
8
4
9
3
2
8
*
9
3
g
(a) =
(b) C1
(c)
(=)=3 =3
(d) C =
Figure 4: Generated samples with different Cs. Experiments uses the data set MNIST.
RELATED WORKS
Wasserstein gradient flows of f-divergences have been previously studied in deep generative modeling as a refinement approach to improve sample quality (Ansari et al.) 2021). A close work to is (Gao al., 2019) where the authors proposed to gradient flows of f- divergences to refine fake samples output by the generator and the generator learns to minimize the squared distance between the refined samples and the original fake samples. However, nei- ther of the above reveals the equivalence between gradient flows and divergence GANs. Further- more, MonoFlow a more generalized framework cover existing gradient flows of f-divergences and our method also applies to traditional loss designs as well as many other types of monotoni- cally increasing functions. IPM GANs: Our framework unifies divergence GANs since estimating probability divergence is naturally related to density ratio estimation (Sugiyama et al.) 2012).
Gradient Flow:
However, some variants of GANs are developed with Integral Probability Metric (IPM) (Sripe- rumbudur et al.) 2009). For example, WGANs (Arjovsky et al. 2017 Guirajani et al.) 2017) esti- mate the Wasserstein-1 metric and then minimize this metric. While MonoFlow associated with Wasserstein-2 metric, minimizing functional in P P(R naturally decreases Wasserstein-2 metric as well. Other types of IPM GANs are MMD GAN (Dziugaite et al.) 2015) and Sobolev GAN (Mroueh et al. 2018b). Both of them have been interpreted as gradient flow approaches (Mroueh & Nguyen 2021; Mrouch et al. 201 Ba) but associated with different vector fields. |Franceschi et al. (2022) studied the NTK view on GANs given a vector field specified by loss function of IPM but lacks connections to divergence GANs. Diffusion Models: diffusion models (Ho et al. 2020 Song
al. 2021 Luo 2022) are another line of generative modeling framework. This framework first perturbs data by adding noises with different scales to create path {9r J220 interpolating the data distribution and the noise distribution. Subsequently, the generative modeling to reverse (q. h2o as denoising. The similarity of MonoFlow and diffusion models is that they both involves particle evolution associated with different paths of marginal probabilities. The difference is as follows: the vector field of MonoFlow is obtained with the log density ratio and the log density ratio must be corrected per iteration by gradient update; diffusion models directly estimate vector fields by time-dependent neural networks and they straightforward particle methods.
CONCLUSIONS
We introduce unified framework for GAN variants. Our framework provides comprehensive understanding to help us get insights into why and how adversarial training works. The mechanism adversarial as we used think. It instead simulates an ODE system,
of adversarial training the bi-level step of adversarial can be regarded as it first estimates the vector field of MonoFLow and next the generator updated to learn to draw particles guided by the vector field, aka we call it parameterizing MonoFlow. Therefore, all GAN variants discussed in this paper are equal at the methodology level. They all are different methods of estimating the bijection of the log density ratio and then mapping the log density ratio by different monotonically increasing functions. Compared to previous studies of GANs, our framework is highly theoretically and practically consistent. The limitation of this paper is that framework does not cover the variants of IPM GANs since these variants give vector field that different from the gradient of log density ratios. We will leave future work.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2008
Abdul Fatir Ansari, Ming Liang Ang, and Harold Soh. Refining deep generative models via dis- criminator gradient flow. In ICLR, 2021.
Michael Arbel, Liang Zhou, and Arthur Gretton. Generalized energy based models. ICLR, 2021.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. ICML, 2017.
David Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: review for statisti- cians. Journal of the American statistical Association, 112(518):599-8777 2017.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In ICLR, 2018.
Gintare Karolina Dziugaite, Daniel M Roy, and Zoubin Ghahramani. Training generative neural networks via maximum mean discrepancy optimization. arXiv preprint arXiv:/50S. 03990, 2015.
Jean-Y Yves Franceschi, Emmanuel De Bézenac, Ibrahim Ayed, Mickaël Chen, Sylvain Lamprier, and Patrick Gallinari. A neural tangent kernel perspective of gans, In ICML, 2022.
Yuan Gao, Yuling Jiao, Yang Wang, Yao Wang. Can Yang, and Shunkang Zhang. learning via variational gradient flow. In ICML, 2019.
Deep generative
Ian Goodfellow. Nips 2016 tutorial:
Generative adversarial networks.
arXiv preprint
1:Xiv:/701.0010 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurlPS, 2014.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans. In NeurlPS, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.
Aapo Hyvärinen and Peter Dayan, Estimation of non-normalized statistical models by score match- ing. Journal of Machine Learning Research, 6(4), 2005.
Abdul Jabbar, Xi Li, and Bourahla Omar. A survey on generative adversarial networks: Variants, applications, and training. ACM Computing Surveys (CSUR), 54(8):1 1-,, 2021.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. In NeurlPS, 2018
Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the fokker- planck equation. SIAM journal on mathematical analysis, 29(1):1-17 1998.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. 2019.
Mare Lambert, Sinho Chewi, Francis Bach, Silvère Bonnabel, and Philippe Rigollet. Variational inference via wasserstein gradient flows. In NeurIPS, 2022.
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistic single image super-resolution using generative adversarial network. 2017.
Qiang Liu and Dilin Wang. Stein variational gradient descent: general purpose bayesian inference algorithm. In NeurlPS, 2016.
10

---

Under review as a conference paper at ICLR 2023
Calvin Luo.
Understanding diffusion models: A unified perspective. arXiv:2208. 11900. 2022.
arXiv preprint
Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In In ICCV, 2017.
Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. ICLR, 2017.
George V Moustakides and Kalliopi Basioti. Training neural networks for likelihood/density ratio estimation. arXiv preprint arXiv:1911.00405 2019.
Y Mrouch, T Sercu, and A Raj. Sobolev descent: Variational transport of distributions via advection. Private communication. Apr, 2018a.
Youssef Mroueh and Truyen Nguyen. On the convergence of gradient descent in gans: Mmd gan as gradient flow. In AISTATS, 2021.
Youssef Mroueh, Chun-Liang Li, Tom Sercu, Anant Raj, and Yu Cheng. Sobolev gan. In ICLR, 2018b.
XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56 56(11):5847-5861, 2010.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In NeurIPS, 2016.
Felix Otto. The geometry dissipative evolution equations: the porous medium equation. Commu- nications in Partial Differential Equations, 26:101-174, 2001.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint
2015.
Geoffrey Roeder, Yuhuai Wu, and David K Duvenaud. Sticking the landing: Simple, lower-variance gradient estimators for variational inference. In NeurlPS, 2017.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In NeurlPS, 2019.
Yang Song and Diederik P Kingma. How to train your energy-basee models. arXiv preprint arX.v:2101.03288, 2021.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
Bharath K Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf, and Gert RG Lanckriet. On integral probability metrics, Saarriss,,ph-diveeree phiidivergenness and binary classification. arXiv preprint arXiv:0901. 22999, 2009.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning- Cambridge University Press, 2012.
Masatoshi Uehara, Issei Sato, Masahiro Suzuki, Kotaro Nakayama, and Yutaka Matsuo. Generative adversarial nets from a density ratio estimation perspective. arXiv preprint arXiv:1610.02920, 2016.
Pascal Vincent. connection between score matching and denoising autoencoders. Neural compu- tation, 2367):1661-1674 2011.
Dilin Wang and Qiang Liu.
Learning to draw samples: With application to amortized mle for generative adversarial learning. In ICLR, 2017.
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dim- itris Metaxas. Stackgan: Text to photo-realistic image synthesis with stacked generative adver- sarial networks. ICCV, 2017.
11

---

Under review as a conference paper at ICLR 2023
A PROOFS
A.1 PROOF OF THEOREM 3.1
Without loss of generality, we assume the target measure p properly normalized, i.e., fdp 1. To satisfy this assumption, p usually follows a Boltzmann distribution, i.e. & exp(-U) where the potential energy
Equilibrium of MonoFlow:
log MonoFlow defines a vector field
2t = h' log re) (Vx logp- lolog qt) such that by Theorem 4.6 in (Ambrosio et al. 2008), we can write the continuity equation
dq: div div(q:h' (logr:) (Vx log q: v lo
(26)
Ôt
At equilibrium (stationary distribution), the probability current is zero,
q:h (logr:) (Vx log
Vx log = 0,
(27)
This condition is required reflect the boundary conditions of the continuity equation. Note that the equilibrium state of general continuity equation does not necessarily indicate that the current (flux) has be zero. MonoFlow is special case where the drift force the system is generated by the potential energy (conservative force), such that achieving equilibrium is equivalent to the current being zero.
Since h' (logr:) 0, we directly have qe (Vx log 9: Vx log = O. Hence d: is the solution to the above differential equation, which is the same as finding the equilibrium of the Fokker Planck equation in Eq. (3)
The dissipation rate: For any curve {gch2o evolving according to the vector field furh2o in Wasserstein space, the dissipation rate of the functional 0F(a)) Eg. (VwzFlqa), vt), where the
Wasserstein gradient of KL divergence is Vw.F((q) = Vx log 2
Therefore, the dissipation rate of the KL. divergence under MonoFlow is
0F(4))
E.g. - -1((1ggr:)
Vx log
at p
S 0,
(28)
01
and equality achieved if and only if qı = p. Hence, MonoFlow always decreases the KL diver- gence with time.
A.2 PROOF OF THROREM 3.2
Define the functional F(g) of f-divergences as
F(g) = D, D, - (2) (x))
(29)
where f: R+ + I is convex function and we may further assume that f is twice differentiable.
Let P(RR) be arbitrary test function, the first variation (functional derivative) &F is defined as
bg
8F 5q
F(g + cσ) - -F(g)
(x)o(x)dx=
lim
6
d Flq + Eω) c=0
de
de / f 7 c
(x)(((x) + cx(x)) le=0
(30)
/ {1
(x)o(x) - (r P
(p(
(x) ((x) + cσ(x) dx
p(x)o(x)
q cΦp
l
/ {1
p
(2) p. ạ))
(x)((x)dxx
9
12

---

Under review as a conference paper at ICLR 2023
Thus,
8F -(r)-
where r=
(31)
dq
q
Recall that the Wasserstein gradient of F(g) is the Euclidean gradient of the first variation, we have
8F --
WW, Vx óg
(32)
The corresponding vector field is given by the negative Euclidean gradient, see Section the particle flow ODE of f-divergences can be written as
therefore
8F 8q
dx
Vx
(x)dt r(xx (r(x)) x log
(33)
We have h1(log r) = 2 pM (r) > O since f a convex function, concluding the proof.
A.3 PROOF OF LEMMA 3.3
This proof adapted from Lemma 1 and by Moustakides & Basioti (2019). Given the optimization problem
max Excp [<(d(x))] + Ex [d(xx))
(34)
dEH
where is a class of measurable functions. We rewrite it as
(((x) o(((x)) + +d(())) (x))
max
Excug
dEH
(35)
p(x) (d((x)) + s (d(xx)
Ex-u
max
ddEH
((xx)
we apply the interchange of maximum and integral because the integral operator indepen- dent of d. Since the maximum is holding for every fixed x, thus we let the derivative
p(x) Jd(x] 70x) ((d()) (d(x)) 0, we have the optimal d", the abbreviation of d (x),,, to sat-
isfy
p(x) q(x)
rol (d*) + (dd') 0, r(x)
(36)
Furthermore, we need to discuss under what sufficient conditions, d' is the unique maximizer for the above problem. Denote ((d) ro(d)= + (dd) in order ensure that d' is the unique maximizer, I' (d) should satisfy
(d()) 0, Ad d" and (d) <, d'.
(37)
We summarize two sufficient conditions as:
1. d, yb are concave functions and the resulting mapping T(d) == (d) का(
is a bijection.
2. d' 0 and the resulting mapping T is strictly increasing mapping (also bijection).
For condition 1, it is obvious d' = T- (r) is the root of Eq. (36) and since & and t are concave, hence ((d) is concave which satisfies Eq. (37 Therefore, d' is the unique maximizer.
For condition 2, we can write (d) [T d' (d))o' Since T is a strictly increasing mapping and d² is the maximizer, have T(d") T ((d)
for d d= and T(d") TTdd) 0 for d > d² Hence (d) satisfies the condition stated in Eq. (37).
In Table 2 b-gan satisfies condition and the rest of divergence GANs satisfy condition 1.
Some examples:
For binary classification, ((d) log (d) and (id) log(1 (dd), r(x) exp(d" (x)).
Fenchel-duality, ((d) = d, (ad)) = F(d), r(x) = (d'(xx)) where the convex conjugate
is f(d) supredom/ / (rd - f(r)}
For least-square GAN, o(d) -(d - 1)2, (dd = dd?, r(x)
d' (x)
I-d"(x)
13

---

Under review as a conference paper at ICLR 2023
B EXPERIMENTS
B.1 DETAILS FOR
Table 3: Explicit forms of and
f(r)
h(u), u = logr
KL
logr r r logr r
u
Forward KL
uexp(u)
Chi-Square
(r 1)2
(exp(u) - 1)2
Hellinger
(VF - 1)²
(Vexp(uu 1)
censen-Shannon (GAN)
r log 1+r + log 1+"
exp(u) log 1+exp(u)
2exp(u) log 1+exp(u)
Exp
exp(1.5logg)
exp(1.5u)
The generator is initialized at:
(1.0)
(1.00
1.00)
0.00
and the target distribution is
N [(00)
(1.00
0.80)
0.89)
0.80
The visualisation results follow the order: KL, Forward KL, Chi- Chi-Square, Hellinger, (GAN), Exp. (x uses a simple 2-layer discriminator with Leaky ReLU activation that has logit output the log density ratio.
14

---

Under review as a conference paper at ICLR 2023
(a) Ratio model: r(x, 0)
(b) Ratio model: r(x, 0ax)
(c) Ratio model: FGAN
catata
p_data
data
ciata
d.data
data
P_9
P_9
P_9
P_cata P.9
P_data
Pldata p_a
P.3
Pacita P_0
p_data 9_9
datata
R_citaa P.g
p_data
p_data
B.2 EXPERIMENT ON VANILLA GAN WITH AN ARBITRARY INCREASING FUNCTION
We change the generator loss to: Eez-op are sinh ((d(g(z)) - 3) 3)*) This has no meaning for
reconstructing any probability divergences, but GAN still works. We did experiment Cifar10 and Celeb-A using DCGAN architecture (Radford et al, 2015). The generated samples are shown in Figure 6
15

---

Under review as a conference paper at ICLR 2023
(a) Generated images of Cifar10
(b) Generated images of Celeb-A
Figure 6: Generated samples via arbitrary increasing function.
16


Reviews:
[{"review_id": "xrKVgsbcgn", "reviewer": "Reviewer_4htw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a theoretical framework for GANs, called MonoFlow, that challenges their usual understanding as an adversarial minimization of a distance/divergence. It interprets GAN training as the generator following a reparameterized gradient flow of the reverse KL, defined through the log density ratio estimated by the discriminator, yielding convergence results. This more accurate framework encapsulates numerous GAN models without gradient penalties and takes into account common GAN practice like non-saturating losses and discarded gradients for the generator. The validity of the developed theory is then empirically assessed on toy data and simple image datasets.", "summary_of_the_review": "The proposed framework is well motivated, relevant, interesting and novel. I believe that it could have a significant impact on our understanding of GAN optimization. However, I think that this paper is not ready for publication, because of three main issues: lack of rigor in the theoretical part, improper related work discussion and unclear developments complicating the assessment of the quality of the paper. Therefore, I would recommend to reject this submission and encourage the authors to strengthen it as it could become a great paper for a future top-tier conference such as ICLR.\n\nI am willing to change my evaluation, should any of my concerns or misunderstanding be addressed during the discussion period with the authors or the other reviewers.", "strength_and_weaknesses": "### Strengths\n\nThe proposed framework is **well motivated** in the paper, as it answers several key issues with previous interpretations of GANs highlighted in the introduction. Treating GANs as gradient flows like in MonoFlow **does solve recurring problems in GAN theory and is a relevant approach**. MonoFlow is **well presented** with a sufficient level of details for non-experts of gradient flows to be able to follow the paper. **Experiments illustrating the validity of the theory** are a nice addition supporting its relevance.\n\nTo the best of my knowledge, the key contribution of this paper is **the link between many GAN models and the gradient flow of the reverse KL**. This has **many interesting consequences**:\n - it links conflicting views of GANs and score-based generative models (Section 2);\n - it formalizes why non-saturating losses can make GANs converge, which was a known fact but only understood intuitively;\n - it gives general criteria to design GAN models that converge;\n - it better fits GAN practice than many previous theoretical approaches.\n\nThis makes the contributions **novel and potentially significant** for future studies.\n\n### Weaknesses\n\nUnfortunately, this submission is diminished by several important problems.\n\nFirst of all, I think that there are **issues in the presentation and proofs of theoretical results**. The **formulation of Lemma 3.3 is somewhat convoluted** and should be reformulated. More importantly, the proofs in the appendix of Theorem 3.1, 3.2 and Lemma 3.3 are **too informal** and lack necessary details: I suggest the authors to write complete proofs. Jointly with a **lack of rigorously formulated hypotheses**, this **casts doubt on the presented results**, which is prohibitive. In particular, I found the following problems which the authors should clarify; note that this is not exhaustive and the whole proofs should be rigorously rewritten.\n - Both the main paper and the appendix **lack hypotheses on the optimization space of functions and on manipulated objects**. For example, in Lemma 3.3, in which precise space is $d$ optimized, and what are the hypotheses on $\\psi$ and $\\phi$?\n - Could the authors explain why $\\mathcal{T}$ must be a bijection in Section A.3?\n - I do not understand how the proof of Theorem 3.1 shows that $q_t \\to p$. Indeed, as explained in Section A.1, the only equilibrium point is $q_t = p$, but nothing indicates that the gradient flow tends to this equilibrium.\n\nSecondly, the novel contributions of this paper, highlighted above, are **not clearly contextualized w.r.t. existing literature**. While the related work section mentions previous work linking GANs and gradient flows, other works have tackled this topic as well: e.g., Huang & Zhang (2022) and Franceschi et al. (2022, cited elsewhere in the paper). This should be apparent from the very beginning of the paper, either in the introduction or by moving the related work section right after the introduction: while the presented framework and results are novel, they should be contextualized w.r.t. prior work on GANs and gradient flows. Moreover, the motivation for the framework (points 1, 2, 3 in Section 1) and the explanation behind the differences between adversarial training and VDM (Section 4.2) **share similar arguments as Metz et al. (2017) and Franceschi et al. (2022)** regarding gradient computations: this should be further discussed in the paper as well. Finally, I would suggest the authors to amend the qualification of Monoflow as a \"unified generative modeling framework\", as it is restricted to some GANs, even through covering many models.\n\nThirdly, there are **some clarity issues** that make it difficult to investigate the correctness and significance of the approach, which are listed below.\n - In Section 1, authors state that \"[the discriminator] is a function only depending on samples $x$ and does not include any density information from the generator’s distribution\". This statement, that is repeated throuhout the paper, is quite obscure and should be clarified. This is especially the case in Section 4.2 where it helps understanding the differences between adversarial training and VDM. Moreover, I believe that the correct statement would be that this dependency does exist, but it is ignored when optimizing the generator (as expressed by the stop gradient operator of Section 4.3).\n - The contribution of Section 4.1 w.r.t. results of the previous sections are not immediate and should be explicated.\n - Could the authors clarify the differences between convergence requirements for $r(x, \\theta)$/$r(x, \\theta_{\\mathrm{de}})$ and $r_{\\mathrm{GAN}}(x)$?\n - The empirical illustration of Section 5.1 does not quite match the theory without explanations. If all considered losses are monotonically increasing functions of the log density ratio, how could $q_t \\not\\to p$ like in Theorem 3.1? Note that this is related to the raised doubts on this convergence result above in this review.\n - Less importantly, contrary to the statement in Section 3.2 that the discriminator is optimized by a \"one-step gradient update\", it is also common practice to perform multiple-step optimization of the discriminator between generator updates.\n\nMetz et al. Unrolled Generative Adversarial Networks. ICML 2017.\\\nHuang & Zhang. GANs as Gradient Flows that Converge. arXiv, 2022.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "L38yXIU9QAp", "reviewer": "Reviewer_Wf4w", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provides new insight to understand GANs based on log density ratios as a gradient flow in the Wasserstein space. Specifically, the bi-level step of adversarial training in GANs is regarded as it first estimates the vector field of the gradient flow (by discriminator), and next the generator is updated to learn to draw particles guided by the vector field. In particular, the particle evolution can be rescaled via an arbitrary monotonically increasing mapping. Such a new insight enables an analysis of what types of generator loss functions can lead to the success of training GAN and identifies the difference between VDM and adversarial training.", "summary_of_the_review": "I think this is a very interesting paper. The perspective of Wasserstein gradient flow indeed brings out a novel and practical insight to the GANs community, especially for the design of the generator loss. I particularly appreciate the analysis of the practical effectiveness of generator loss. However, I am not an expert on the theory of GAN, so it is possible that I did not identify the mistakes in the theoretical results. ", "strength_and_weaknesses": "Strengths:\n1. An interesting idea with nice theories and consistent practical findings. First, it is very interesting that from the perspective of the gradient flow in this paper, all log density ratio based GAN variants are methodologically equal. These GAN variants all are different methods of estimating the bijection of the log density ratio and then mapping the log density ratio by different monotonically increasing functions. Second, the analysis of generator loss, especially Fig. 2, to reveal the failure of vanilla and MLE loss and the success of NS and logit loss from the perspective of the adopted monotonically increasing mapping, really provides a novel and practical insight.\n2. The simple trick to fix vanilla GAN works well and refreshes me. The trick is also practical support to their theories.\n3. The idea is well presented in this paper.\n\nA potential improvement:\nAn algorithmic inspiration for the generator loss has been proposed. I am wondering if such a framework can provide inspiration for the discriminator loss to compare different estimations of the bijection of the log density ratio, such as JSD divergence and f-divergence. The authors should discuss this.\n\nSome typos:\n1. In the contributions: “insteand” → “instead”\n2. Under Eq. (21), the two $\\tilde{f}(d*)$ are defined differently. \n\nA further thought:\n1. In the related work, the authors somehow unify GANs and diffusion models from particle evolution. I wonder if such a framework can provide some views to discuss the advantages/disadvantages of the vector field of MonoFlow and the vector field by time-dependent neural networks of diffusion models.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "HOxVi3h6Tv", "reviewer": "Reviewer_wjv3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a new formulation for adversarial training as a special case of wasserstein gradient flow. The discriminator update is explained as approximating the vector field of a gradient flow in the Wasserstein space, and the generator update as moving particles (samples) in this vector field. This perspective unifies several GAN variants, and provides a unified insight as to why these variants work. The paper also provides some toy experiments to verify its predictions about the reasons why some variants might work or not.", "summary_of_the_review": "This paper proposes a new theory of divergence GAN that is closer to practice and can better explain the dynamics of their training and what can make/break them. Background must be discussed more substantially, and some missing connections to practice need to be clarified and discussed. Overall, I think this paper can be valuable for both understanding and improving divergence GANs.\n\n### Update post-rebuttal\nI thank the authors for their clarifications and answering my concerns. I consider the paper interesting and valuable, however, the paper is still lacking in clarity and very hard to follow (skips some underlying assumptions and discussions to better place its results wrt existing theory). Given that the paper's main contribution is theoretical, clarity and mathematical rigor is a must, and so I maintain my borderline score of 6.", "strength_and_weaknesses": "**Strengths**\n\nThe proposed theory exposes an interesting connection between Wasserstein gradient flow and divergence GANs, and can better explain why certain choices of final activations (leading to different divergences) work. This new theory is closer to practice in my opinion, and as such can be very valuable in better understanding divergence GANs and how to improve them.\n\n**Weaknesses**\n\nThe paper does not read well in my opinion, some parts can use more clear explanation and better connections to background theory, in particular the convergence of Euclidean ODE to Wasserstein PDE (convergence of the empirical measure to q) deserves more attention.\n\nAnother issue is not discussing some limitations when connecting to practice, in particular, the choice of discriminator itself (and its regularizers such as R1 R2 etc) is very consequential in practice, while the paper does not discuss this choice under the lens of its theory. The new theory assumes the ratio is estimated accurately, but this is of course not true in practice, and as a consequence the updates are not strictly following the vector field of the Wasserstein gradient flow in practice, it is important to discuss to what extent will such inconsistencies affect the theory.\n\nAnother issue is that of the three disconnects you raise about original GAN theory in page 2, I don't agree with 1 and 2. For 1, the divergence's first term derivative is uniformly zero wrt generator so how does not optimizing it matter? For 2, I'm not sure I understand your argument here, can you elaborate on this given Proposition 2 of original Goodfellow et al. theory?\n\nFinally, I think the title and abstract of the paper should mention “divergence GANs”, rather than GANs, to emphasize that IPM GANs are not covered by the proposed theory.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject