Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes a generative model that consists of two components: a normalizing flow and an energy-based model with short-run MCMC as proposed in (Nijkamp, 2019). The whole procedure operates by pushing some simple distribution through the NF and then running the short-run chain on these samples. Both models are learned jointly as follows. The normalizing flow tries to capture the limiting distribution of the EBM by minimizing 
$$\text{KL}(K_\theta q_\alpha \Vert q_\alpha),$$ 
where $q_\alpha$ is the density of the NF with parameters $\alpha$ and $K_\theta$ is the transition kernel of the short-run chain with parameters $\theta$. Simultaneously, the EBM is learned to capture the data by minimizing 
$$\text{KL}(p_{\text{data}}\Vert p_\theta) - \text{KL}(K_\theta q_\alpha\Vert p_\theta).$$

In Section 4.2, the authors provide some analysis of the proposed procedure. Namely, the authors consider the perfect scenario (where the NF matches the EBM density and they both match data) and give some intuition for the case when the models are not expressive enough to perfectly match the data.

Finally, the authors provide an empirical study of the proposed procedure. They report metrics for image generation on CIFAR10, SVHN, Celeba (downsampled to 32x32); give several examples of image inpainting and latent space interpolation.
main_review: The paper describes a reasonable approach to generative modeling and demonstrates promising metric values. However, the main issue of the paper is that the particular choice of the generative models is not well-motivated. That is, the first question that appears in the reader's mind is why do we need to combine a flow with a short-run chain? Couldn't we use any other pair of $C_n^2$ combinations (where $n$ is the number of different approaches) to learn an efficient generative model? Finally, couldn't we just increase the size of our baseline model instead of mixing it with another approach? Unfortunately, the paper doesn't address any of these questions. To be more precise, the authors do not provide any motivation for the usage of these particular models. The main argument in favor of the proposed approach is the empirical comparison, which is, however, not extensive, since the models are not compared in terms of computational budget.

Other comments:
- The proposed objective in equation (8) closely resembles the Contrastive Divergence objective as noted by the authors. The contrastive divergence objective is notorious for its biasedness since the samples from the model ($K_\theta q_\alpha$ here) are usually treated to be independent of parameters $\theta$. I wonder if the authors follow the same way for their objective. Unfortunately, the authors neither clarify this in the paper nor provide an analysis of the bias (in the case of the biased scheme).
- The ability to model the density is not demonstrated empirically. The authors mention that their approach is able to model the density of the learned distribution (which could be a very nice motivation by the way), but they do not validate this property empirically even on a toy example.
- There is no convergence analysis. The authors neither prove nor demonstrate empirically the convergence of the procedure. Moreover, according to the last paragraph of Section 4.2, the scheme performs "an infinite looping" hence does not converge. This non-convergence property is usually considered as a significant flaw of the approach (e.g. in GANs literature).
- Some empirical results are not trustworthy. Providing comparison with other generative models the authors report numbers from different papers. However, the comparison on Celeba is hardened by the fact that many papers do not downsample the images to 32x32 resolution as the authors do. For instance, NT-EBM (Nijkamp, 2020) uses 64x64 resolution and couldn't be compared in terms of FID to the proposed model.
- Reconstruction use case is not clear. The experiments with image reconstruction require additional clarification. It is not clear what kind of a problem the authors approach there. Are these images corrupted by some noise? The choice of the intermediate representation (the output space of the flow) for the latent space is also not motivated. Why couldn't one take the initial distribution that is an input for the flow?
- Overall, the numerous applications listed after the generative modeling experiments should be measured quantitatively rather than qualitatively by several examples.
summary_of_the_review: The paper describes a reasonable approach to generative modeling and demonstrates promising metric values. However, the approach itself is a combination of two well-known models. The main issue of the paper is that the particular choice of the generative models is not well-motivated.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to train an energy-based model with a normalizing flow to achieve rapid and high-quality MCMC sampling. 
It shows the trained CoopFlow is capable of synthesizing toy data and realistic images (reasonable compared to many baseline methods in terms of common evaluation metrics).  Applications to image reconstruction and interpolation are also included. 
main_review: 
Strength
- The idea of the paper is well simple, and it seems to work well to generate realistic samples. 
- It makes progress with respect to Cooperative learning via MCMC teaching, by using the normalizing flow instead of other kind of generators. 

Weakness
- The theoretical validity of the generator is not very clear based on Section 4.2. In particular, there is no condition  or evidence to show that the generator converges to a fixed point in (8) and (9). Why there exists a fixed point? 
- If possible, the baseline in Table 1 should also include the result of Ho2019 (used in the proposed model) in the category of Flow, to see if the MCMC teaching is important. 

Questions: 
- It is not clear what is evaluated in Section 5, is it the training set or test set? Can MSE score (often used in image reconstruction) be provided for Figure 3? 
- What would happen if you use 200-step  instead of 30-step in CoopFlow (Pre) in section 5.2?

Typo
- section 4.1, in practise -> in practice
summary_of_the_review: This paper introduced a simple idea to train an energy-based model, which is very challenging in high-dimensions. The numerical result makes quite significant progress over state-of-the-art methods in terms of FID scores on natural images.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper develops a new generative model, called CoopFlow, by combining the energy-based model (EBM), normalizing flow (NF) and Langevin dynamics (LD). The CoopFlow updates all the parts in an cooperative manner: Parameters of NF are updated by using samples modified by LD, LD is performed based on the potential function provided by the EBM, and the gradient of log-likelihood of EBM are estimated according to samples generated by NF + LD. Here, LD can overcome the limitation of the model capacity of NF, and NF + LD can provide a more accurate estimation of the gradient of EBM loss. After learning, a powerful sampler and an accurate EBM can be obtained.
main_review: This paper provides a novel cooperative way to train the flow model and energy model. The idea is very attractive, and could be able to solve existing difficulties of generative models. My major concern for the CoopFlow comes from the fact that the training procedure cannot be interpreted as the minimization (or minimax) of a loss function. It is not clear whether the cooperative learning process is always convergent.

Some other comments are as follows:

1) The Langevin dynamics used in the paper is not an exact MCMC chain due to the time discretization. Is it possible to utitlize some MCMC samplers, e.g., Metropolis-adjusted Langevin algorithm in CoopFlow?

2) It is claimed that the KL div between \tilde p_\theta and p_\theta "decreases to zero monotonically". This conclusion is not widely known and requires reference.

3) Some important previous models and related references are omitted, including the deep energy model proposed by Liu within the stein variational framework, and the stochastic NF proposed by Noe consisiting of NF and MCMC.

4) Analysis in Section 4.2 is too confusing for the reader. This section is called "theoretical understanding" and there is no theorem. Can authors provide more understandable explanations?
summary_of_the_review: The article not only introduces some innovations, but also conducts a large number of experiments to demonstrate the superiority of the algorithm.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a cooperative learning framework of normalizing flow and energy-based Langevin MCMC models (CoopFlow). Specifically, the normalizing flow suggests a better initialization for the Langevin flow, then the latter generates synthetic samples via short-run MCMC. While the Langevin flow is trained with a standard MCMC likelihood toward the data distribution, the normalizing flow chases the Langevin one by maximizing the likelihood of the synthetic samples. The proposed CoopFlow seems to outperform each of its components.
main_review: This paper is strongly motivated by the short-run MCMC energy-based model [1]. The key difference between this work and [1] is the use of a normalizing flow model for a nice initialization to shorten MCMC steps even more. 

The idea is sound and well-motivated, however, my main concern on this work is that it seems both the theoretical explanation and empirical results do not support such a motivation very clearly. 

* First, the theoretical explanation reveals that the short-run MCMC CoopFlow $\pi^*$ is a moment matching estimator of $p_{data}$. However, the current version of the explanation does not clearly show that $q_{\alpha}$ initialization can yield a shorter transition path (i.e., lower MCMC steps) compared to the standard Langevin flow started from $p_{0}$ (which is the main motivation of this work). Of course, it can be deduced from the fact that $q_{\alpha}$ tries to mimic $\pi$ thus automatically $D(\pi, q_{\alpha})$ can be lower than $D(\pi, p_{0})$ for a certain discrepancy D in most cases; but it will be nice if the authors can discuss such a case in a more systematic way, with adding some intuitive comparison of Figure 1 with Figure 5 of [1].

* In addition, the current empirical results do not validate the authors’ motivation sufficiently well, because a detailed cost-vs-performance trade-off study is not presented here. Table 1 (a) only reveals that the proposed CoopFlow gets worse FID scores than, for example, NCSN++ or EBM-Diffusion. I don't think every newly proposed model should beat the best model. However, since the main advantage of CoopFlow is a fast generation (as well as training) owing to the short-run MCMC with the normalizing flow-based amortized sampler, the author should discuss such a cost perspective in detail. For example, how many iteration MCMC/SDE steps are required for other models, and how about generating samples with the same (and small) number of steps for all models? Currently, I can only conclude the CoopFlow outperforms each of its components (it is also not absolutely certain because the reference flow model, Flow++, seems to be not evaluated solely, thus the author should ), which is also a nice contribution but not greatly fascinating.

***

[1] Erik Nijkamp, Mitch Hill, Song-Chun Zhu, and Ying Nian Wu. Learning non-convergent nonpersistent short-run MCMC toward energy-based model. arXiv preprint arXiv:1904.09770, 2019.

summary_of_the_review: Overall, while the paper is interesting, the lack of justification for the authors’ motivation prevents me from recommending a clear acceptance for this paper. My current evaluation of this paper is borderline but can be changed after reading the author’s rebuttal.

***
Post-rebuttal: I appreciate the authors’ thorough responses to my questions. After reading the responses, I have tended to accept this work, thus raised the review score accordingly. However, this rating is only marginally above the borderline, thus I would not champion the paper for acceptance.

After the authors' follow-up response: I feel more comfortable accepting this work. I am sorry to keep my score as 6, mainly due to there is no review score of 7 in the system. Instead, I increased the significance score of this paper accordingly.
review_title: null

Meta-review: The paper proposes a methodological improvement in the Langevin-based training of energy-based models. The idea is to initialize the Langevin flow used to train an energy-based model with a normalizing flow which learns to mimic the Langevin flow as the energy-based model is being trained. The method is empirically evaluated on synthetic data and image benchmarks.

The reviewers are currently divided: one argues for strong rejection, two for weak acceptance, and one for strong acceptance. In summary, the reviewers have expressed two main points of criticism: (a) that the motivation is unclear or not experimentally demonstrated; (b) that the convergence properties of the algorithm are unclear. Regarding (a), I believe the authors have adequately addressed this concern, and in my judgement the method is sufficiently motivated. Regarding (b), the authors responses have mostly relied on non-rigorous argumentation and appeal to prior work, so I don't think the issue has been addressed to the reviewers' satisfaction. Having said that, in my judgement lack of clarity regarding convergence is not a sufficient reason to reject the paper, as there don't seem to be reasonable doubts that the method doesn't converge in practice.

On balance, although the paper has certain weaknesses, it proposes an interesting and potentially useful method without major technical inadequacies, so I'm leaning towards recommending acceptance.
Decision: Accept (Poster)