Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes an improved Annealed Importance Sampling (AIS) for score-based diffusion models. The author argues that past papers on AIS have predominantly focused on modifying the intermediate distributions or the Markov kernel but have overlooked the suboptimal nature of the extended target distribution. Thus, the paper integrates a new AIS procedure in combination with recent works to approximate the extended target distribution.
strengths_and_weaknesses: Strengths:
- The paper is well motivated and theoretically proves the correctness of the proposed methods.
- Overall, the paper is well written and explained useful insight on the limitations on the Markov kernel:
  - E.g. the illustration of the suboptimality of a homogeneous MCMC chain give good intuition on the current problem of AIS.

Weaknesses:
- While experimental results show improvement, it would be more convincing if there were more real-world dataset experiments (other than binarized MNIST).

Minor Suggestions:
- Why not bold the best log Z estimate on Table 1 & 2?
questions: - Were there any experiments done on ULA-MCD and UHA-MCD on more complex distributions? I understand estimating the normalizing constant is very difficult, but a single experiment would be nice to see.
- Why do we need applications for amortized inference? Also, why not experiment on more datasets?
- I am not very familiar with AIS, are differentiable AIS and ULA and UHA the current SOTA benchmarks for likelihood estimation?
- How would you provide a solution for the numerical instability in optimizing due to the unadjusted Langevin and Hamiltonian sampler? This seems like an imperative issue that needs to be addressed.
limitations: The authors transparently addresses the limitations of their work and have a section dedicated to limitations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Annealed importance sampling is one of the most effective methods for marginal likelihood estimation. The commonly used extended target distribution is simple but sub-optimal. The paper leverage the recent progress in score-based generative modeling to approximate the optimal extended target distribution for annealed importance sampling and give the discretization of Langevin and Hamiltonian dynamics. On a number of synthetic distributions and variational auto-encoders, the proposed method obtains better performance compared to existing methods.
strengths_and_weaknesses: * The writing is of high quality. The paper gives a concise review of the annealed importance sampling and illustrates the limitations clearly via two examples. Then, the paper provides necessary details in deriving the Langevin and Hamiltonian algorithms and makes them easy to read.

* As a sampling algorithm, the effectiveness of the proposed method needs further consideration. In particular, the effectiveness of the proposed method relies on the qualities of the learned score function. However, learning the score models is not easier than estimating the partition functions. Whether the proposed method can apply well to hard distributions is still doubtful. It will be helpful if the paper can provide an ablation study that investigates the relationship between the quality of the learned score functions and the MCD samplers.

* Although I have some reservations about the effectiveness of the current algorithm due to the reason mentioned above, its contribution to further studies on extended target distributions should be recognized. For example, when the energy function is learned via a score-based model and is associated with a score function in nature, the proposed method could be very helpful.

* The evaluation in the experiment is not fair. We can run ULA and UHA directly, but we need to learn the score functions to run ULA-MCD and UHA-MCD. The paper should also report the learning time and the sampling time such that the readers can have better judgments.

questions: * Considering unadjusted algorithms are used, will the error in score estimation brings bias in the sampling? If so, is it able to quantify the bias in terms of the estimation error?

* What is the training time of the score model? How is it compared to the sampling time?
limitations: The paper mentions some limitations of the proposed methods. I think it misses two important problems:
1. The method requires learning a score model, which is time consuming and hard. This factor reduces the efficiency or even correctness of the proposed method.
2. The estimation error in the score model may lead to bias in sampling, and the paper lacks a quantification of the potential bias.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The key problem this work addresses is the suboptimality of the extended target distribution that is widely used in current annealed importance sampling (AIS) methods. The authors, building on previous literature, show that the custom choice of the backward Markov kernel of the unnormalized extended target is comfortable but suboptimal in terms of variance.
To overcome such limitations, the authors extend some key observations that have been made in the literature of diffusion processes used for generative modeling (e.g. Sohl-Dickstein et al., 2015), and draw a connection between forward/backward diffusion and forward/backward transition kernels used in importance sampling. Armed with this realization, the authors show that AIS can be studied through the lenses of forward and backward diffusion SDEs: the forward diffusion begins in a simple to sample from distribution, and gradually transforms it into the target distribution (or an extension thereof), while the backward diffusion corresponds to the (hypothetical) process of transforming the target to a simple distribution.
In addition, and as an extension to a previous paper that appeared in a recent workshop, the authors consider Hamiltoinan dynamics, and derive a similar connection to AIS, but with improved performance due to a higher fidelity representation of the paths taken by the Hamiltonian SDEs, when compared to first order dynamics.
For practical reasons, the authors produce discretized versions of the dynamics they study, and implement numerical integration schemes for the simulation of the SDEs, where necessary.
Numerical results on both static distributions and on an instance of a variational autoencoder indicate dramatic performance improvements of the proposed approach when compared to vanilla and Hamiltonian vanilla versions of AIS.
strengths_and_weaknesses: Strengths:
* This is a very important topic, and the performance gains from the proposed method are substantial
* I liked the idea of connecting the dots between AIS and (score-based) diffusion models
* The extension to Hamiltoinan dynamics is interesting, and a nice complement to [1]

Weaknesses:
* I think the exposition of this paper should be improved: the connection with diffusion models is somehow reversed, as in that literature the forward process is going from the target distribution to a simple one, and the backward process is doing the opposite, whereas in this paper it is the other way around. This could be discussed in the appendix. Also, I would study the "steady state" distribution of the forward process defined in eq 10, by means of Fokker-Plank equations, to show that indeed the forward process moves an "easy" distribution to the target one. This can be helpful for a reader familiar with diffusion models, whereby you can demonstrate that for variance preserving methods, the process converges to isotropic Gaussian.
To put things differently: a reader with familiarity in diffusion models knows that the forward process "destroys" data, whereas in this work it is the other way around.
* experiments only compare the proposed method to baseline AIS and to Hamiltonian AIS (which I appreciated a lot). However, this work could be cast under the umbrella of energy based models and density estimation. Then, would it make sense to compare it to such line of work?
* there exist a workshop paper presenting preliminary ideas that are extended in this work. The main difference here is the treatment of Hamiltonian dynamics, and the experiments with the variational autoencoder. If accepted, this paper could maybe give more space to the Hamiltonian part, and most importantly, to the excellent results obtained. This is the main reason why, in its current form, the key contribution of this paper is somehow diluted, and seem a repetition: I like this work, and I would be happy to raise the score I gave which is only tainted by this discussed weakness

[1] Tim Dockhorn and Arash Vahdat and Karsten Kreis, Score-Based Generative Modeling with Critically-Damped Langevin Diffusion, ICLR 2022

questions: Here are some questions, but these are somehow superficial as I didn't find any major flaws in the paper.

1- I think there is a typo in eq 12: the term $ 2 \nabla \log s_{\theta} ( \cdot ) $ should read $ 2 s_{\theta} ( \cdot ) $

2- I may have made mistakes in my own derivations, but isn't there a typo in eq 6 for the backward transition kernel? In the numerator you should have $q_{k+1}(x_k)$ and not $q_{k}(x_k)$? I'm not 100% sure here

3- May I suggest to improve the notation? I find it difficult to follow when $x, x'$ are used in place of $x_{k+1}, x_k$

4- Would it make sense to have an additional section in the appendix to clarify not only the similarities but also the differences w.r.t. score-based diffusion models? See my comment on the weaknesses above.

== Post Rebuttal == 
Thanks for your answers to the questions and comments above. I have modified my score, raising it to an accept/7. 
limitations: I think they did address limitations appropriately.
review_title: null

Meta-review: metareview: The submission presents a novel and interesting method using recent advances in score-based diffusion to improve the recently proposed differentiable AIS log marginal likelihood estimates. The experiments clearly show the benefit of using Monte Carlo diffusion. The writing is clear and of high quality. For these reasons, all reviewers were unanimous in recommending acceptance.

AC notes: (1) out of curiosity, how is the adjusted Langevin and Hamiltonian versions perform on the static targets?, and (2) further to the question below on the timings, would it be useful to add a comparison where the time is kept similar between U(L/H)A and that with MCD, for example, 2K intermediate densities for U(L/H)A and K for the MCD variants.
confidence: Certain
Decision: Accept