Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes an approach to sampling from distribution with deterministic Hamiltonian dynamics. 
main_review: The theory developed and results reported in the paper are not convincing to me: 

1. It is unclear how the proposed method can handle the irreducibility issue required by MCMC simulations. To be more detailed, if the proposed sampler returns to the starting point after a short time period, then this trajectory will be repeated for ever due to the nature of its deterministic dynamics.  In this case, the sampler will not be ergodic. 

2. The results are reported in graphs in the paper. This makes the reviewers very hard to accurately assess the performance of the proposed method.  The performance of the proposed method should be compared numerically with existing MCMC methods. 
limitations_and_societal_impact: Not applicable
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper proposes a novel Hamiltonian to define the sampling dynamics in the state space. The main difference with the classical Newtonian Hamiltonian is a different kinetic energy. The proposed kinetic energy is $K(v) = d/2\log(v^2/d)$, where $v$ is the momentum, and $d$ is the dimensionality of the state space. The peculiar property (which is the central point of study) of this Hamiltonian is that the velocity $\dot{x} = v/(v^2/d)$ decreases with the increase of the kinetic energy. Intuitively, this energy defines very reasonable dynamics: the particle slows down in the high-density regions and accelerates with the decrease of the density. Note that the classical HMC behaves oppositely.

For the proposed dynamics, the authors carefully develop technical details. They hypothesize that, unlike HMC, this dynamics could actually be ergodic in the state space and the resampling of the momentum is superfluous then. They also develop a specific integration scheme (based on Leap-Frog) that facilitates an efficient simulation of the dynamics. Throughout the paper, the authors claim that the accept/reject step is not necessary for the correct sampling if the integration is precise enough.

For the empirical evaluation, the authors consider several synthetic distributions and compare the algorithm against similar "physical" samplers. For the synthetic distributions, the authors report ESS and MMD. There is also a brief comparison against other samplers on the energy-based models, where the authors report the average energy of the samples and provide visual examples.
main_review: There are several major weak points in the paper. The first one is the ergodicity of the procedure. Indeed, the set $H(x,v) = c$ could be disconnected (say like an elliptic curve). Although the authors demonstrate the convergence to the target empirically, this could be achieved by averaging across many parallel chains, which authors use in the experiments. Also, the authors propose an alternative procedure formulating the particle sampling method (flow interpretation in the paper) based on this dynamics. This one doesn't require the ergodicity assumption but is not tested empirically in the paper (the only empirical evaluation is for the 1-D standard normal in the Appendix).

The second weak point is the simulation without accept/reject step. Indeed, if we simulate the dynamics precisely enough we preserve the target measure and the acceptance test always yields 1. However, this also holds for the HMC algorithm. Since no developments facilitating the removement of the accept/reject step has been made, I don't see any reason to remove it. To be more precise, the authors develop a novel integration scheme, but its development is motivated by the difficulties introduced by the novel kinetic energy. Throughout the paper (line 125, lines 264-265) the authors claim that their development avoids accept/reject, but this could be also avoided for HMC in the same manner. If this could be done, I would expect a comparison with the corresponding modification of HMC.

My final concern is the empirical study. The authors demonstrate that the Hamiltonian is approximately preserved in Fig. 12 of the appendix, but they don't report the step size for these simulations. At the same time, the step size in the experiments is quite agressive (as described by the selection procedure in Section 3.1). So I wonder whether the step size is the same for both experiments? Furthermore, the authors compare the training speed of energy models using different samplers as a subroutine. However, for the comparison, the authors rely on the visual quality of the samples. The pictures look like blobs of different colors, no specific patterns are visible. I don't think that the different colors could be interpreted as a faster convergence.

Minor comments:
- The visualization of the dynamics in Fig. 1 seems a bit weird. Does this width of the "ribbon" appear as fast oscillations of the dynamics?
- line 110. I wouldn't claim that the dynamics is actually chaotic. To classify it as chaotic one needs to estimate the Lyapunov exponent or Kolmogorov-Sinai entropy. Moreover, the proposed algorithm heavily relies on the precise simulation of the dynamics (because of the lack of accepts/rejects); hence, the chaotic property could be very unfavorable from this point of view.

Despite all my concerns, I think the paper is a good fit for NeurIPS. The question of ergodicity of deterministic dynamics seems quite challenging, and I wouldn't expect it to be solved in a single paper. The second issue could be easily solved by the recent developments for non-equilibrium sampling see [1,2,3]. I think this will actually improve the performance of the proposed algorithm. Despite some flaws in the empirical study, I think the paper is important for the MCMC field. It gives a fresh look at Hamiltonian Monte Carlo and poses interesting questions for the community.

I think my final score is a bit biased towards positive because I had the same idea before getting this paper for a review. Nevertheless, I think I wouldn't be the only one excited about this paper in the NeurIPS crowd.

- [1] Rotskoff, Grant M., and Eric Vanden-Eijnden. "Dynamical computation of the density of states and Bayes factors using nonequilibrium importance sampling." Physical review letters 122, no. 15 (2019): 150602.
- [2] Neklyudov, Kirill, and Max Welling. "Orbital MCMC." arXiv preprint arXiv:2010.08047 (2020).
- [3] Thin, Achille, Yazid Janati, Sylvain Le Corff, Charles Ollion, Arnaud Doucet, Alain Durmus, Eric Moulines, and Christian Robert. "Invertible Flow Non Equilibrium sampling." arXiv preprint arXiv:2103.10943 (2021).
limitations_and_societal_impact: The authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a new method for drawing samples from a Gibbs measure using evolution of a deterministic Hamiltonian ODE. In particular, the authors present their method as a possible alternative to Hamiltonian MCMC. The authors develop an ODE solver that is tailored to their model and present experiments learning synthetic distributions and deep energy based image models.
main_review: The EBM learning method presented in this work is very novel and the authors present an interesting ODE alternative to standard EBM learning that is powered by Langevin dynamics/HMC. The central innovation is replacing the standard kinetic energy of HMC (from a probabilistic perspective, the negative log likelihood of a Gaussian) with a different kinetic energy. The derived learning equations give an ODE which, under an assumption of ergodicity, is distributed according to the measure of the learned EBM over chaotic trajectories, allowing the authors to use deterministic instead of stochastic dynamics as is typical. This innovation is quite interesting and a strong aspect of the work.

While the overall idea is strong, there are some areas of this work that could be improved. Some questions and points for discussion:
* Figure 1 is misleading. Langevin sampling should be implemented with a much larger step size for fair comparison, and from my experience with the right step size it should be able to travel to at least a few nearby modes in a few hundred steps. In fact, Langevin sampling can sample fairly from an isotropic gaussian in a single update if epsilon is equal to the gaussian standard deviation. This is very different form what the diagram suggests due to improper tuning in the digram.
* I am a bit confused by the appearance of the data dimension $d$ in the kinetic energy formula. Actually, the inclusion of $d$ appears to lead to a minor issue with the third line of the derivation following equation (3) because I am not sure how the $E(x)/d$ term in the exponential is becomes $E(x)$ in the next step, since this exponential factor can't be absorbed into the normalizer ($d$ is a kind of temperature factor is seems). Is the inclusion of $d$ necessary in the kinetic energy?
* The results for the experiments with synthetic datasets demonstrate the usefulness of the proposed method. The EBM learning experiments are not as convincing. In my experience ULA with 50 steps can achieve good results with correct tuning on both of the tasks presented in Figure 7 and I am not sure this figure is a fair comparison. In the appendix, the authors show that ESH can generate reasonable samples on par with ULA for Cifar10 but the benefit of the new method is unclear. Nonetheless, it is good to see that the proposed method can work at a reasonable scale in practice.
* There is a subtle theoretical complication when using this method to learn an EBM. For standard MCMC-based learning, an update that follows detailed balance ensures that the samples are from the steady state even for short run distributions, as long as the initialization distribution is (approximately) equal to the steady state. While this assumption often does not hold in practical EBM training, it can be ensured to some extent with careful implementation and persistent chains as discussed in [22]. Supposing that you are using ESH to learn an EBM and the ODE initialization distribution is close to the steady state. Does the ergodic hypothesis ensure that short-run evolution under ESH from steady-state initialization has a distribution that still follows the steady state? I am not sure that using ESH to learn an EBM is entirely justified for this reason.

**Overall**: The authors present a novel and interesting deterministic sampling framework for generating samples from unnormalized densities. The experiments show improved results over baseline samplers on synthetic distributions and reasonable results on deep EBMs. There are a few points which could be improved and I am open to changing my score based on author response and revisions. 
limitations_and_societal_impact: The authors have adequately address limitations and societal impacts.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a deterministic sampling dynamic, rapid energy sampling Hamiltonian dynamics (ESH). Since there is no stochasticity, there is no rejection. The kinetic energy is formulated in a way that the dynamic stays at a state for the time proportional to the Boltzmann distribution of the (potential) energy. This paper also propose a leapfrog integrator specialized for ESH. Compared to MCMC approaches, ESH opens a way of much faster sampling from unnormalized energy based models without any auxiliary generating models.

main_review: This submission looks almost complete. I think the idea of defining a non-Newtonian momentum is very novel. This paper may bring about a paradigm shift in how the machine learning community approaches the problem of sampling from unnormalized models. Their empirical performance in mixing between modes (Fig 1, Fig 3) is very impressive. They only have one hyperparameter to tune, epsilon, which is very simple compared to other MCMC approaches although their algorithm itself is far from being simple: Langevin is hard to tune, but implementation is very simple with only a few lines. But the submitted code for time adapted leap frog sampler was daunting.


Questions: 

First, what happens to your very intuitive explanation in Fig. 2 and line 72-81, when integrating the scaled time dynamics? Your K is set to make the time spend around x as 1/\dot{x} (line 2 in Figure 2). But, now again the time is scaled  by 1/|v|.. It is kind of unclear what’s going on and what is the impact of it when we re-scale back.

For line 86, what happens if the support of p(x) is unbounded, e.g. R^d? We cannot define a uniform distribution here, and in your formulation the identity in (3) seems crucial. Even then would your method be established?

In (7), how do I obtain u(t+epsilon/2) in the third line? Is it just by applying epsilon/2 in place of the epsilon in the first line?

In 120-121, why the effective step size can vary dramatically depending on the magnitude of the velocity? Could you give me some intuitive explanation on it?

In 254, a correct label may not be Fig 13 (funnel) but Fig 17 or another? Section 3.4. did not even specify which data was used. Therefore, it adds confusion when you say “as in a first experiment” (line 666). What is the relationship between the experiments in section 3.4 and D.3. Are they separate, different experiments? It is confusing because in D.3, you have “experiment 1” and “experiment 2”, making unclear if they are additional details of the experiment in 3.4 or else separate, additional experiments.

In 272, is there any (intuitive) reason/explanation why you initialized the sampler by using Jarzynski, not by other two algorithms or leap-orig?

Which algorithm is used for the experiment? I thought you are using Algorithm 1 or a version without time adaptation, and sometimes Jarzynski for initialization (line 272). But, when addressing bias-variance trade-off, in 676-688, Jarzynski ESH sampler is mainly discussed. Then, added in 687 that you use this ignoring-the-Jarzynski-weight technique. Please identify clearly which algorithm is used in each of your experiments…


In Fig 15, how did your replay buffer images look like? If the buffer was already reasonably realistic, what is the implication of this figure?

In file samplers.py, you did not recommend to use your submitted code for anything because it works with a single chain, not parallel chains. But, for the computational cost, in line 640, only minutes or hours were taken with only a single GPU. Did the cost calculation come from your single chain code?

Is there any reason that you did not show the full trained CIFAR model by using ESH? You stopped learning at epoch 100 and only compared at that moment.


limitations_and_societal_impact: Yes
review_title: null

Meta-review: The paper proposes a sampling method based on deterministic dynamics based on a Hamiltonian with a non-standard kinetic energy. While the referees find the theory part of the manuscript under developed, in particular regarding ergodicity, they agree on the novelty of the method and find the numerical experiments convincing. In view of encouraging novel methods development, while the paper is on the boarderline, the meta-reviewer recommends acceptance as a poster. 
Decision: Accept (Poster)