Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary

This paper proposes using gradient flows to improve samples from generative models. In practice, it uses the density ratio estimator from a GAN’s discriminator, in simulating the particle dynamics. The resulting algorithm updates latent variables based on a drift term, which is the gradient of the energy that combines the f-divergence and a negative entropy, and a diffusion term which adds a Gaussian noise. This method can be extended to other generative models such as VAE by training a density ratio corrector, similar to a discriminator.

The idea of introducing gradient flows for refining samples is theoretically very interesting, and the authors conducted a wide range of experiments to support the idea. However, I have major concerns regarding the actual algorithm and the evaluation. I would consider increasing my score if these can be well addressed.

Pros

1. Gradient flow provides a solid theory for improving samples from generative models, a direction that has attracted increasing attention recently. The idea proposed in this paper can help provide a unifying view of several recently proposed methods.
2. The paper is well-written, and it provides a good coverage of background materials.
3. It provides a wide range of experiments, from toy examples to image and text generation.

Cons

1. It is unclear to me why the proposed DGflow can outperform DDLS, which as the authors acknowledged, is equivalent to the DDLS when using the KL-divergence. More specifically how are DDLS and DGflow (KL) different in the experiments?
2. In Table 2, the FID for baseline models seem very different from numbers reported elsewhere (e.g., for SN-DCGAN(ns), the FID from Miyato et al. is 29.3, but here says 20.9).  With this discrepancy, it is difficult to evaluate the improvement from the proposed method.

Other comments:

1. The criticism of DDLS, that p_Z(z) might be undefined, seems unfair. The problem with uniform prior, e.g., can be handled by projected gradient descent. In addition, is there a similar term from entropy regularisation? 
2. Consider discussing the relationship with using gradient flows for training as in [1]
3. Consider citing [2], which achieves the state-of-the-art GAN scores from incorporating refining latent in training.

[1] Deep Generative Learning via Variational Gradient Flow. Gao et al. 2019
[2] LOGAN: Latent optimisation for generative adversarial networks. Wu et al. 2019

review_title: Review

final_score: rating: 7: Good paper, accept
review_content: Pros:
* The proposed framework seems principled and practical. Propagating generated samples to follow the data distribution by simulating the gradient flow of the f-divergence to the data distribution is reasonable, and the required quantity for simulation, i.e. the density ratio between the sample and data distributions, is readily given by the discriminator in GAN training.
* The presentation follows a clear logic flow, and related works are clearly connected. Experiment shows promising results.

Cons:
* Some statements can be made more precise and rigorous, up to my knowledge.
  - "The discriminator is trained to maximize this distance": although the discriminator is involved in a minimax optimization problem, it is the distributions-dependent optimal discriminator that defines a distance between two distributions. The minimax objective may not be a distance between two distributions given an arbitrary discriminator.
  - Eq. (1). Up to my knowledge, on a metric space, there is no formal definition of __gradient__. Even the concept of tangent vector is not defined on a metric space. Formally, a tangent vector involves a differential structure, so the space is often required to be a manifold. To define gradient, the space is further required to be a Riemannian manifold. What can be defined on a metric space is __gradient flow__, as __curves__ holding the intuition to minimize a given function steepest, and there are several formal descriptions on this intuition, e.g. the minimizing movement scheme. But the curves cannot be described using tangent vectors $x'(t)$ and gradients, as presented in Eq. (1).
  - Eq. (3) is specific to the 2-Wasserstein space.
* On Lemma 3.2.
The result is based on the rule of change of variables, Eq. (25). But if $g$ is not required to be injective, the right hand side of Eq. (25) needs to be multiplicated by the number of $z$'s that make $x = g(z)$ [Federer, 1969, "Geometric Measure Theory", Thm. 3.2.5]. So the lemma may need to be adjusted accordingly.
* It is favorable to cite the specific theorem/statement from works by Villani (2008) and Ambrosio et al. (2008) since they are huge books.
* On the method.
Although asymptotically the method guarantees that the sample distribution will converge to the data distribution, for each sample it may not converge and may traverse non-stop in the support of the data distribution. In other words, there exists nontrivial (non-zero) dynamics that keeps the data distribution stationary/invariant. Some examples in Fig. (2) already show this behavior to some extent, where all the samples along evolution seem realistic and differ in e.g. color or orientation. So is there a method to determine when to stop the evolution? Also, is it a problem that the evolution may change some attributes of the original sample?

=== EDIT: post rebuttal ===

Thanks for the response and addressing the issues for a more serious research paper.
review_title: The work presents an elegant framework to refine samples generated from a generative model.

final_score: rating: 7: Good paper, accept
review_content: In this paper, authors have proposed to use Wasserstein gradient flows to update samples from deep generative models (DGM) to closer to the empirical data distribution which is expected to produce better generated samples. The Wasserstein gradient flows are induced from the entropy-regularized f-divergence functional between data the empirical data distribution and the to-be-learned distribution. The experimental results have shown improvement in the qualitative and quantitative results on real-world datasets of images and texts. 

The writing is clear in general however the first paragraph of section 3.1 seems vague. Authors state that "experiments show
that the stale estimate ...  provides a good enough approximation for the purpose of refining samples" but then later they state "refining directly in the data-space using the stale estimate poses problems". It seems not consistent. Moreover, in the experimental sections, there is no result which refines "directly in the data-space". 

It is not clear how to combine Eq. (8) with (11) to obtain the update on Eq. (12).  It will be more convinced if authors provide rigorous proof. 

In the experimental section:
   - In Table 2, there is no result of FID for the baseline DDLS. Similarly, there is no result of IS for the baseline DOT.
   - Is it possible to use DOT/DDLS to apply for GANs with vector-valued critics (and/or VAE, Glow, etc.) by using auxiliary GDMs as with DGflow? If yes, what are their results in Table 4?

For images, KL divergence usually provides better results while  JS divergence is better for texts? Do authors have any insights?
review_title: A principled methodology to improve samples generated from deep generative models

final_score: rating: 7: Good paper, accept
review_content: # Summary
The paper introduces Wasserstein gradient flows for refining the samples of generative models. To achieve this, the authors demonstrate the gradient flow of divergences between two distribution in the Wasserstein space can be expressed as a stochastic differential equation whose dynamic mainly depends on the gradient of the divergence computed through the ratio between the two distributions. The authors observe the ratio between the generated distribution and the target distribution is provided by the discriminator and so the dynamic can be simulated via the stochastic Euler scheme. Instead of refining the distribution in the data space the authors suggest and show how to do it in the latent space. In addition, they expand their approach for non-GAN generative models, this setting requires to train (or just finetune)  another discriminator. They experimentally show that the proposed method improves the quality of the generated samples for a large panel of experimental settings. In particular, they slightly outperform other refining algorithms.
# Major comments
## Pros
I really enjoyed reading the paper, even if I had to read it twice to understand everything! The paper is well written and pleasant to read. In addition, the idea presented is very nice and is new to me. The main hypothesis made (that are not always valid) are clearly expressed and discussed. The experimental section convinced me the method is good at doing the job it is made for!

## Cons
I do not have strong concerns regarding your work. However, I am not very familiar with refining techniques. It is why I give "only" a 7/10. 
I have however the following remarks:
1) When applying your method for a model that did not use a classifier to be trained, do you really think retraining another discriminator between the two generative models is better than just training the discriminator? Or is it because you do not suppose that you have access to the dataset but only two the two generative models? This is not very clear to me why you would like to estimate the ratio indirectly. 
2) I would have liked to see the gradient flow in the 2D space fig 1. It would make a beautiful word and directly explicit how the method is working in this simple case. I don't know if this is easy to draw due to the randomness of the SDE though.
3) Why don't you compare to DDLS in table 2? I think the results could be made more readable by merging tables 2 and 3 and keeping only what is important in these tables.
4) I think discussing the intuition behind using different divergence and their impact on the refinement good also be interesting.
5) I would also like to see something about the induce "refinement" time, I suppose this is not too heavy but it would be interesting to have the information about that somewhere.
6) Experimenting on the violation of 3.2 would be interesting as well.

# Minor comments
focussed -> focused
Weiner -> Wiener
of particle -> of the particles

review_title: Refined generative models with Wasserstein gradient flows

final_score: rating: 7: Good paper, accept
review_content: Summary: 
---------------
In this paper, the authors present a technique for refining the output of generative models by using gradients from discriminator to update the latent variables from which the generator produces a sample. The technique is more versatile than previous work since it can be applied to vector-valued discriminators, and does not restrict itself to scalar discriminators. The technique itself isn’t particularly complicated, yet delivers impressive results.

Pros:
-------
This work provides some sound theoretical foundations on which to build sample refinement, which isn’t specific to a single GAN architecture, but is applicable to a wide range of generative techniques. Although I’ll admit that I’m not familiar enough with stochastic differential equations to understand every aspect of the author’s theoretical justification, the idea of leveraging gradients from the discriminator to update not the generator (which lives in a high dimensional parameter space) but the latent variable (which lives in a much lower dimensional space) seems plausible.

In addition, one sees that the authors invested much thought and effort into the experiments section. I especially liked that they did language modeling, a task that shows a generative model works on domains other than just images. All told, after reading the experiments one is satisfied with the evidence of the effectiveness of the author’s technique.

Cons:
--------
The only major concern I have is that by only sampling images generated by updated latent variables, one loses diversity of generated images. A common experiment in some GAN papers was to choose two latent vectors $z_1$ and $z_2$, and show how the generator is able to smoothly interpolates between these two generated samples $g(z_1)$ and $g(z_2)$ as one generates samples with $g(\lambda z_1 + (1-\lambda) z_2), \lambda\in[0,1]$. It would be interesting to see if the latent refinement technique presented here also allows such smooth interpolation, or if as one starts with $\lambda z_1 + (1-\lambda) z_2$ and then refines, if the generated samples fall into a few “modes” of good samples while not generating the samples between the modes.

In 5.2 and 5.3, I would appreciate it if the authors could specify, as the authors did in 5.1, how many times the authors did the train a base model and then refine pipeline in order to obtain the means and standard deviations reported.

I would also appreciate it if the authors would report the JS-6 scores in the character level language modeling experiment. One gets the scores more or less for free, if runtime concerns are an issue, I can recommend this: <https://github.com/cseward/ngram_language_model> which does the same thing as the evaluation metric the authors used (<https://github.com/igul222/improved_wgan_training>) but is much faster by using C++ under the hood (there’s a python API) and saving the n-grams in a tree structure.

One last note: In our community, to get papers published one must generally demonstrate that a method works on some standard datasets, and authors are unfortunately yet understandably hesitant to discuss situations where a method fails to produce the desired results, as such a discussion could add arrows to the quiver of unduly critical reviewers. Such situations include datasets where the method doesn’t work, sensitivity to hyperparameters and  Yet it is exactly these counter-examples which often help further understanding of the method and pave the way for further advancements. Therefore I would humbly request that, if this paper is accepted, the authors also discuss any failure cases they may have found, as these would make the paper an even more compelling read.

Room for improvement:
---------------------------------
One of your claims is that “By refining inferior samples, our technique avoids expensive sample rejection used by previous methods.” Please consider reporting the runtime (i.e. wallclock time) of your method vs the methods you’re comparing against (such a report is fine in the appendix).

It’s a fact of life that most readers don’t want to delve into the mathematical details in order to understand a method, but like a few pretty pictures that give them at least an idea of what’s going on. I think the author’s paper would gain more traction in the community if in the introduction they could come up with some “eye candy” showing how the method allows the gradient to flow all the way from a discriminator to the latent variable, resulting in a better good generated sample. 

review_title: Nice contribution in the field of sample refinement for Generative Modeling

Meta-review: This work improves deep generative models by applying Langevin dynamics to sample in the latent space. The authors test their method under different configurations (different loss functions) and various generative models (VAE, flow, besides GAN). Experimental results demonstrate the benefits of the proposed method in different generative tasks. 

I tend to accept this solid work. I just have two suggestions: 1) the authors should discuss the connections and the differences between the proposed method and the energy-based methods like (Arbel et al., 2020) in-depth; 2) it may be more suitable to replace "Wasserstein gradient flow" with "Discriminator gradient flow" in the title.
Decision: Accept (Poster)