Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Inspired by NueTra [1], in this paper the authors propose a new approach to train deep energy-based models (EBMs). The motivation here is that EBMs are typically trained via contrastive divergence which requires MCMC sampling in a high-dimensional space and from  a mutli-modal distribution. As a result, most MCMC sampling methods do not mix well which results in a biased estimation of MLE. To address this problem, the authors propose to train an energy-based model with a (pre-trained) backbone flow; the EBM can be interpreted as a correction step to the flow model. The flow $q_\alpha(z)$ defines a latent space $z$ with a base distribution $q_0(z)$. As pointed out in [1], we can use the flow network to additionally yield the following distribution on $z$-space: $p(z) = p(x)\frac{\delta x}{\delta z}$. Similar to NeuTra [1], the authors propose to run HMC to sample from $p(z)$ which geometrically is a simpler target, and then feed $z$ to the flow to get $x$ again. The authors perform some experiments on both toy and some of the common vision datasets to show that unlike SGLD, this approach mixes way better and is able to traverse to different local models.   
main_review: **General Comment**

I very much enjoyed reading this paper. The motivation is stated clearly; the problem of having a biased gradient estimation of MLE in training EBMs is certainly a reasonable problem. The proposed approach certainly makes sense and I think is a very reasonable approach to this problem. While the work is very close to [1] which may raise some concerns regarding novelty, I think it can also be seen as a strength given  that it is showing a very good application of method for posterior inference in training EBMs. Furthermore, the addition of energy-based correction is I think, while trivial, is a very neat trick that enables one to avoid computing the determinant jacobian when running HMC unlike NeuTra. The experiments I think could use some improvements, but overall demonstrate the case that the chains in the proposed approach mix better than SGLD chains. The proposed model does have a small disadvantage in the sense that it requires to train a flow model in advance. However, as the authors show in Section 4.4, even using a small flow results in a significant improvement. Overall, I am happy to recommend acceptance. 

**Related Work**

I think this paper is missing some of the related work, namely [2,3]. I understand this work differs vastly from [2,3] but given that both of these papers also proposed a new method to address the problem of using MCMC sampling to compute the gradients of $\log p(x)$, I think this work should compare against them or at least discuss them within context. 

**Clarity**

This paper is very well-written. There were very few ambiguities. Good job!

**Experiments**

The experiments are well-designed overall. The one I think I would have liked to see (in addition to comparison against [2,3]) is a more comprehensive ablation study of HMC vs SGLD. Figure (2) already demonstrates this but if I understand correctly, Figure (2) is run on a toy dataset. Would we have seen something similar in the case of vision datasets? In other words, how much of the gain in performance is due to using HMC instead of SGLD and how much of it is due to sampling in $z$-space instead of $x$-space? Also would have been interesting to compare against standard Contrastive Divergence where instead of initializing from noise, we initialize from a flow model for a fair comparison.  

**Minor Comments**

-In the Introduction, the authors state: “Recently, Nijkamp et al. (2019) proposes to initialize short-run MCMC from a fixed noise distribution, and shows that even though the learned EBM is biased, the short-run MCMC can be considered a valid model that can generate realistic examples. This partially explains why EBM learning algorithm can synthesize high quality examples even though the MCMC does not mix.” Could you elaborate on this statement? Why does this explain that CD is able to synthesize high quality samples? Because of initialization from noise instead of data? Could you elaborate abit more on the reasoning here? Thanks


**References**

[1] Hoffman, Matthew, et al. "Neutra-lizing bad geometry in hamiltonian monte carlo using neural transport." arXiv preprint arXiv:1903.03704 (2019).

[2] Grathwohl, Will, et al. "No MCMC for me: Amortized sampling for fast and stable training of energy-based models." arXiv preprint arXiv:2010.04230 (2020).

[3] Du, Yilun, et al. "Improved contrastive divergence training of energy based models." arXiv preprint arXiv:2012.01316 (2020).


summary_of_the_review: I very much enjoyed reading this paper. The motivation is stated clearly; the problem of having a biased gradient estimation of MLE in training EBMs is certainly a reasonable problem. The proposed approach certainly makes sense and I think is a very reasonable approach to this problem. While the work is very close to [1] which may raise some concerns regarding novelty, I think it can also be seen as a strength given  that it is showing a very good application of method for posterior inference in training EBMs. Furthermore, the addition of energy-based correction is I think, while trivial, is a very neat trick that enables one to avoid computing the determinant jacobian when running HMC unlike NeuTra. The experiments I think could use some improvements, but overall demonstrate the case that the chains in the proposed approach mix better than SGLD chains. The proposed model does have a small disadvantage in the sense that it requires to train a flow model in advance. However, as the authors show in Section 4.4, even using a small flow results in a significant improvement. Overall, I am happy to recommend acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: I find that this paper is a re-submission from NeurIPS2020, for which I acted as one of the reviewers. The content almost remains the same.

This paper studies the learning of a special class of EBMs, which is a correction or an exponential tilting of a flow-based model.
An interesting observation that the resulting EBM in the latent space is of a simple form that is much more friendly to MCMC mixing.
It is said that HMC sampling of the EBM in the latent space, which is a simple special case of neural transport HMC, mixes well and traverses modes in the data space. Regarding this main claim, the authors raise a number of scientific questions, which are validated by empirical evaluations.
main_review: -Strengths

Addressing the problem of non-mixing of MCMC for sampling from an EBM is important.
The particular class of EBM model is not new, but considering the EBM in the latent space as shown in Eq.(9) seems to be new and has the benefit that the resulting EBM Eq.(9) may be much less multi-modal than the EBM Eq.(7) in the data space. 
The authors raise good questions, which are exmained by empirical evaluations.

-Weaknesses

The results are not strong. In Table 1, the FID scores over CIFAR-10 is far worse than GAN-based models [a] and EBM-based models [b]. The unusually bad FID scores hurt the quality of the paper, which makes the method, though interesting, not so convincing and useful.

Some closely related references and discussions are missed.
First, note that the EBM model in [c] is parameterized by CNNs and its MCMC sampling is mixing. Thus, the claim "To the best of our knowledge, our work is the first where MCMC sampling is mixing for EBM parametrized by modern ConvNet" in this manuscript should be revised. The idea of learning EBMs by NCE has also appeared in Wang&Ou (2018) (the reference in the paper). I suggest the authors to highlight the differences between these prior studies and this work and add relevant discussions, instead of making big claims.

[a] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv:1802.05957, 2018.

[b] Y. Song, Z. Ou. Learning Neural Random Fields with Inclusive Auxiliary Generators. arxiv 1806.00271, 2018.

[c] B. Wang and Z. Ou, “Language modeling with neural trans-dimensional random fields,” IEEE Automatic Speech Recognition and Understanding Workshop, 2017.

--update after feedback--
To acknowledge the revision and improvement of this paper, I raise the score.
summary_of_the_review: See above.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a product of EBM and FLOW and samples in latent space to improve mixing. Such a model can be learned using both MLE or NCE. Empirically, the paper shows that a single MCMC chain of such a model can traverse through different modes. 
main_review: Strengths:

This paper addresses the mixing problem of EBM to some extent. It is also well known that training with NCE is difficult, and it is interesting that NCE can work with a product of flow and EBM.

Questions:
1. Contribution. A key technique used in this paper is the neutral transport MCMC. The paper uses the technique for both training and sampling. The main difference from Hoffman et al. (2019) is to train with neutral transport MCMC. Thereby, it is important to verify that it is the training with neutral transport MCMC that solves the non-mixing problem instead of sampling with neutral transport MCMC. Perhaps the author can add an experiment to see what will happen if training with normal MCMC and sampling with neutral transport MCMC.

2. Since the model can generate different samples in a single MCMC chain, it can potentially improve the sampling speed. For example, to generate 100 samples, prior EBMs need to run 100 parallel MCMC, and the model in this paper only needs to run one MCMC and get samples from different timesteps. Thereby, it is worth adding FID results calculated by samples taken from one MCMC chain (using different intervals) and reporting the corresponding sampling speed up.

3. Despite the improvement of mixing, the sample quality is bad. It is worse than the MCMC trained EBM (Du & Mordatch, 2019). The author needs to give some explanation on this phenomenon.

4. Missing related work. [1*] uses a product of EBM and VAE, which has some similarity. It should be discussed in related work section.

5. Missing experimental details. The dataset used for Figure 2 is not specified. What is the y-axis of Figure 2 (a), (b)?

6. Typos. In Appendix A.6, the range of the index $i$ and $j$ is not consistent.

[1*] VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
summary_of_the_review: The paper is clear and easy to understanding, but there are some questions to be addressed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose to learn energy-based models with a flow-based model as "backbone". This is to utilize Neural Transport and ultimately make the Markov Chains mix well in data space. This is an important problem.
main_review: My first suggestion is that the authors explain what the consequences of non-mixing Markov chains could be. 

I was a bit worried when you said the quality of synthesis was a second issue to you. Do the authors feel that mixing is more important than good synthesis? I understand that they are related, but mixing is nice only conditioned on good synthesis.

You write on page 4, "for notation convenience". I don't see the convenience, but I see it provoking many analysts.

You should state how alpha is pretrained. Additional to this, can alpha be jointly learned with theta?

Can the authors please expand what I should conclude from Figure 2(b)?

On the same line as my second point: it seems that the authors do not think that Table 1 is the main result. If this is true, why not?

What is the cause of the missing experiments in Tables 1 and 2?

I would like the authors to tinker more about the limitations of the work. One thing I would like to know more about is the relationship of your "backbone" and what Bayesians would call a prior. Is there *some* truth in the following: are you pre-training a prior?

Minor things: 

When citing multiple papers at once, consider putting them at the end of the sentence. Else, it is hard to read the sentence.
summary_of_the_review: I like the paper, and think it is well-written in general. The main contribution is a novel combination of existing methodologies. I am not completely familiar with the literature, but if this combination is novel I believe this to be a solid contribution. However, my evaluation is dependent on adequate responses from the authors.
review_title: null

Meta-review: The authors set up a simple combination of an energy based model and a flow based model that corrects the flow based model with an energy based term. The merits of this relative only an energy based model is improved sampling to compute the gradient. The advantage over a only flow based model is that the kinds of transforms that can be used are less limited.
Decision: Accept (Poster)