Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The present paper proposes a novel method to approximately sample from unnormalized density functions and estimate their normalizing factors. Motivated by recent successes of denoising diffusion models, the main idea is to reverse an Ornstein-Uhlenberg (OU) process which diffuses the target density into an approximate normal distribution. This requires an approximation to the score of this process which is learned using a neural network by minimizing the reverse Kullback-Leibler distance. Moreover, extensions to underdamped Langevin dynamics and the connection to Schrödinger brides and stochastic optimal control are presented. Finally, the performance of the method is demonstrated by several numerical experiments.
strength_and_weaknesses: **Strenghts:**

The paper shows how to transfer several ideas developed for denoising diffusion models to the task of sampling from unnormalized densities, where the denoising score matching objective cannot be applied. This represents an interesting research direction which is of high interest to practitioners. To this end, the paper derives a suitable objective for learning the drift of the reverse-time process using the Kullback-Leibler divergence on the path-space. Further, the forward process is extended to underdamped Langevin dynamics and the corresponding discrete-time analogous are presented.
Different from previous methods in stochastic optimal control, this leads to a reference process given by an overdamped or underdamped OU process instead of a pinned Brownian motion, which improves the numerical stability of the training procedure (especially for larger number of steps), yields better estimates for the normalizing constants, and allows to construct reverse-time ODEs.

**Weaknesses:**

Comparing the objectives of the proposed method and already existing methods (e.g., the so-called path integral sampler), there is basically only one main difference, i.e., the use of an OU-process instead of a pinned Brownian motion. Judging from the numerical experiments, this can improve training stability and estimation of normalizing constants. It remains unclear whether these benefits stem from the SDE type (variance preserving OU vs. variance exploding Brownian motion), which could also easily be adapted for the path integral sampler, or the initial Gaussian distribution instead of a fixed starting point. 

The paper also seems to lack evidence whether these changes systematically improve sample quality (very minor relative improvement in Table 1 and hardly visible differences in Figure 6) and further experiments in this direction would be valuable. Moreover, there are other methods to sample from unnormalized densities using diffusion models, see, e.g., https://arxiv.org/abs/2206.01729 (Section 3.6. and 4.5), which could be compared to the proposed method and which propose to use effective sample size as additional evaluation metric.

While the theory developed in the first sections of the paper is interesting to read, a couple of ideas actually do not benefit the numerical performance in practice, e.g., underdamped Langevin dynamics and the reverse-time ODE (continuous-time normalizing flow), see Appendix C.8 and C.9. I think this might distract the reader from the practically relevant parts and one could, for instance, move corresponding sections to the appendix.

Finally, it would be great if the paper included a discussion why the exponential type integrator is considered in the discrete-time setting. There are better theoretical guarantees as compared to the Euler-Maruyama scheme, however, the latter seems to be used for the standard implementation of the path integral sampler. Also, it would be interesting if the adjoint method could still be employed to compute memory-efficient gradients for the objectives.

**Minor issues:**
1) The proofs of the statements and additional material in the appendix should be referenced in the main paper.
2) 'Logarithmic derivatives of the intractable marginal densities': Should this rather be 'derivatives of the logarithms of the intractable marginal densities'?
3) 'As its initial state $x$ is distributed according to $\pi$': The problem is rather that we cannot sample from the density $\pi$.
4) The time-reversal is written as $(y_t)\_{t \in [0,T]}=(x_{T-t})_{t \in [0,T]}$, which could be interpreted as an equality in an almost sure sense. However, I think that this only holds in distribution.
5) The considered set of path measures should be properly defined.
6) While the chain rule for KL divergences can easily be verified for time-discrete settings, it would be beneficial to include a reference for the time-continuous case in order to see what regularity assumptions are needed.
7) Figure 1 could be described in more detail.
8) I guess the symbol $\perp$ in the beginning of Section 4 should denote the inverse. It feels counter-intuitive to detach the only part of the loss which contains information on the target density and it would be interesting to have more insights regarding this choice.
9) It would be beneficial for the reader to provide more information on the problem settings and metrics.
10) For some settings the presented box plots are difficult to compare and it would help the reader to provide, for instance, relative errors.
11) Typos:
    - 'descent' is missing after 'gradient' on page 2.
    - 'the use *of* underdamped diffusions' on page 2.
    - $\alpha_T$ should probably be $\lambda_T$ on page 2.
    - $\beta_{T-t}$ is missing in the inline equation before (5).
    - The last sentence in Section 2, the sentence starting with 'For example' in Section 3.3, and the second sentence in Appendix C.4 seem to be grammatically incorrect.
    - There is a missing full stop in the last paragraph on page 5.
    - 'At' should be 'As' in the beginning of Section 3.2.
    - Wrong references: 'Figures C.7 and C.7' in Appendix C.7. and 'Table C.7.1' in Section C.7.1. 

clarity,_quality,_novelty_and_reproducibility: The paper seems to be technically sound and generally meets the quality criteria at major ML conferences. However, the clarity of the paper and the presentation of the content could be significantly improved as outlined in the section above. Further details on the experiment settings would need to be specified for full reproducibility, especially given that corresponding code is missing. 
The method itself appears to be novel, but the changes to existing methods (i.e., path integral sampler) seem to be minor and a direct consequence of methods developed for denoising diffusion models. Several additional contributions are interesting from a theoretical viewpoint, however, do not to benefit empirical performance, see also my comments in 'Weaknesses'.
summary_of_the_review: The problem tackled in the paper is of high interest, e.g., in Bayesian statistics and computational sciences. Furthermore, it seems to be a promising direction to transfer methods from denoising diffusion models to improve methods in stochastic optimal control used for sampling from unnormalized densities. The paper provides a good starting point in terms of theoretical contributions, which, however, to a large extend directly follows from the theory developed for denoising diffusion models. The method used for the numerical experiments only differs from existing methods by using a different reference SDE (OU process vs. pinned Brownian motion). This indeed provides improved numerical stability and better estimates of normalizing constants for the considered examples. However, the presentation of the paper could be improved and a full picture of the benefits and drawbacks of the proposed method seems to be lacking.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduces Denoising Diffusion Sampler. The task at hand is to sample from an un-normalized target density with an unknown normalization constant; and also to estimate said constant. Inspired by Denoising Diffusion generative models, the paper proposes a reverse-time SDE of a process that diffuses the target distribution to the Gaussian distribution. The authors propose an approximate learning objective so that to get rid of intractable scores of the marginals of the process. The authors evaluate their method on benchmarks, comparing it to other sampling methods.
strength_and_weaknesses: Strength: the method is connected to Denoising Diffusion generative models, which gained much popularity.

Weakness: I have concerns regarding the empirical evaluation and the clarity of the presentation. The figures which are supposed to represent comparison against other sampling methods (mainly SMC and path integral sampler) are barely discernible. But it can be seen that the DDS seems to work on par with PIS. It is also not clear the training times that are mentioned to be non-negligible compared to SMC. Besides, the experiments seem to be relatively small or medium scale. 

Regarding the style of the presentation it is sometimes obscured by heavy-weight formulae, e.g. in section 3.2. I think proper formatting should remedy that. Tables do not seem to be referenced anywhere, and Figures 2, 3 neither. Sections' 3.1, 3.2 titles contain the same typo: "Ulhenbeck".
clarity,_quality,_novelty_and_reproducibility: The paper is relatively clear except sections on Underdamped Ornstein-Uhlenbeck which are hard to read due to an overabundance of in-line formulae. The figures are hard to see. The figures should also be referenced in the text, as should the tables. Empirical evaluation could see some improvements. The approach seems novel, although heavily inspired by DDPM.

summary_of_the_review: My issue with the paper lies within its empirical evaluation and comparative advantage of the proposed method as it relates to other sampling techniques; and also within the clarity of the presentation.

UPDATE: after the discussion period, I am willing to increase the score.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Authors propose novel theoretical framework based on Girsanov formula to learn sampler based on reversing diffusion process from non-normalised density function.
strength_and_weaknesses: Strength: novel theoretical approach to sampling from non-normalised density

Weakness: absent experimental comparison with Langevin Monte-Carlo methods as they aim to solve the same problem.
clarity,_quality,_novelty_and_reproducibility: The method is novel both from practical and theoretical standpoint. The algorithm should be easy to reproduce but the code is not provided.
summary_of_the_review: The proposed approach definitely interesting and I am eager to see whether it can be successfully applied for non-convex optimization. I believe that this result is worth publishing due to novel approach for sampling.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper describes sampling (MCMC parlance) as a diffusion problem. In the forward direction, the target distribution diffuses to Gaussian noise, while the reverse direction of interest allows diffusing from noise to the target distribution. As can be seen and is highlighted by the authors, this problem is classical in MCMC.
They develop a sampler which is inspired by recent advances in diffusion based models. Building a reference diffusion process that diffuses from noise to noise, they describe some particular criterion they optimize to train their sampler (eq 10), basically based on minimizing the KL divergence between the distribution it yields and the reference distribution.
The same story is then told again for different kinds of flavours, involving  augmenting the original state by a momentum variables and also studying the discrete case, which is of interest for implementations.
Experiments are numerous and mostly done on small data, but apparently matching what is usually done in the field.
strength_and_weaknesses: The paper is definitely timely, considering the hype around diffusion models, and it reads pretty rigorous. I guess it may be of interest to the very few readers able to understand it.
On the downside, I must say that I had great difficulty following it, because my math is not strong enough. As a practitionner, I am left with some frustrating feeling that I probably don't really even know how to use that sampler in practice, the whole thing being quite burried in the maths.
My guess is: this paper is very nice for a strong theoretical connection between diffusion models and samplers, but I can't really assess it.

Some attempts of comments on the go:
* below (2), I guess you mean \lambda_T, not \alpha_T
* maybe it is trivial, but I probably missed a proof for what is claimed in the first paragraph of 2.2
* between (5) and (6), isn't there a minus sign "-" mismatch ?
* "where we have use" -> "used"
* It looks like only the equation below (8) $f_\theta(t,x) = ....$ is necessary, not really (8), which looks like just a copy of (5)
* the discussion in 2.4 and what it means as a difference with the section before is difficult for me to understand
* above (14), I don't know what \pi(x; 0, \sigma^2 I) means. This section looks like some extension of section 2, and I must confess I just gave up following the overwhelming amount of maths at this point.

clarity,_quality,_novelty_and_reproducibility: Super difficult to follow for everyone that is not an expert in both diffusion models and MCMC. It is hard to me to assess originality. I bet the authors are not the first ones to attempt some connection between diffusion models and MCMC, but what is and what is not a contribution is not really discussed in the paper, and I am not knowledgeable enough to guess it myself.
summary_of_the_review: Probably correct theoretical paper about the connections between diffusion models and MCMC. Hard to follow. It doesn't look like great care was taken to help the practitionner use the approach.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This work takes inspiration from ideas developed for denoising diffusion models to sample approximately from unnormalized densities. Reviewers acknowledged contributions of this work, which paves the way for a new interesting research direction. Reviewers pointed to weaker experimental results in the submission, but the authors provided the additional data required to convince reviewers of the practical relevance of the contributions.
justification_for_why_not_higher_score: The paper could be presented as a spotlight given the practical and theoretical contributions and given that the experimental evaluation was beefed up during the rebuttal phase.
justification_for_why_not_lower_score: Authors addressed the concerns raised by the reviewers and made substantial improvement to the submitted paper.
Decision: Accept: poster