Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
========

the article proposes to tune an HMC sampler by maximising E_\param[\log target(X_T)] over the parameters of the HMC sampler. Furthermore, the article studies the influence of the initial distribution. While the approach is certainly interesting, I have not found the empirical studies satisfying enough. 

Comments:
=========
1. The article considers a vector \epsilon as well as a mass matrix. Usually, the parameter epsilon is chosen as a scalar number: choosing epsilon as a vector can indeed also be seen as a particular type of preconditioning (or choice of mass matrix). I have found this part of the paper not extremely well explained.

2. It is indeed also difficult to choose L, and that is mainly what the no-U-turn method tries to automate. In practice, dynamically adapting L can make a lot of difference in high-dimensional settings and/or different parts of the state space exhibit different scales. It would have been very interesting to investigate how the proposed method can be used **in conjunction with** no-U-turn type strategies. Furthermore, it was not entirely clear to me how the \epsilon was tuned when the no-U-turn was used.

3. In the 2D example, since the authors have used rejection sampling to produce the plots, it is also easy to accurately estimate the mean/covariance of the target distribution. It would have been interesting to use these statistics [although, it is not possible to do so in more complex scenarios] and see if this leads to improved performances.

4. in the "\min \bar{p}" method, why choose a target acceptance rate of 0.25? My experience says that the number is usually chosen much higher.

5. While reporting the KSD, I think it would have been very interesting to report the ESS [or variations of it], since it is the standard measure of efficiency in the MCMC literature.

6. Finally, while the 2D examples are certainly very interesting, I am not convinced that directly going from 2D to super-difficult-target is the right approach to understand the properties of the proposed methods. There are many settings that are more difficult than these 2D distributions, but much more tractable than the DLG/molecular targets.

In summary, I think that the authors are proposing an interesting line of research, but more careful numerical investigations are necessary to really understand the worth of the methodology.
review_title: interesting line of work:numerical  studies are not entirely convincing

final_score: rating: 6: Marginally above acceptance threshold
review_content: 
### Summary:

This paper proposes a variational inference based framework to tune some of the hyper-parameters of HMC algorithms automatically. The authors drop the entropy term from regular ELBO formulation, which facilitates a gradient-based approach. However, dropping this term requires extra care for which authors offer an automated-method. Finally, the authors demonstrate empirical validation on several problems.

### Strength:

The authors do an excellent job of articulating their intuition behind the idea both (see section 3.) While dropping the entropy term from ELBO decomposition is heuristic-based, the explanations are well-formulated, and Figure 1 does an excellent job of getting the point across. 

More so, since dropping the entropy term can cause pathological behaviors, the authors propose a method to ensure wider initial distributions. I commend the authors for the non-trivial engineering that was required to make their ideas work. I also, commend the author's effort of conducting statistical tests and extensive empirical evaluations. 

### Concerns:

My main concern with the work is that it is often on-par with the competing methods--I understand that a new method doesn't need to be SOTA on every benchmark--and the SKSD enabled variants that achieve this performance are prohibitively slow (see Tables 6 and 9.) I could not help but feel concerned when no discussion was offered for an almost tenfold increase in the computational time for training DLGMs. To convince me, I will suggest offering an honest discussion on the run-times of the approaches.

I find the discussion in section B.1important, and believe it should be more formal.  Specifically, I will suggest algorithimizing what objective is used at which stage. Alternatively, authors can choose to restructure this some other way; however, it is too important to be left in its current form. 

### Updates after the rebuttal

I like the paper and found the revised version more transparent. I support the engineering approach of the paper; however, as we all know, these papers often require authors to go to greater lengths to convince. After reading the other discussion and reviews, I think the authors can consider a few additional experiments. I would suggest investing in a more involved toy-experiment to better motivate the engineering solutions. If possible, authors can also consider a more careful ablation study to establish the relevance of each component on this toy-model. Further, the authors offered explanations for the training time aberrations; if possible, authors can consider including the equally-fast-variants in the revision to be more convincing. 
review_title: The paper is well-written, and the authors do an excellent job of articulating the problem and motivate their idea well; I do have some reservations. I vote for a weak accept.

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes a method to optimize the parameters of the Hybrid Monte Carlo (HMC) algorithm (the step size and the diagonal of momentum's covariance matrix). In order to do that, the authors consider the distribution of samples q_T() obtained after T iterations of the algorithm (T accept/reject steps) starting from some distribution q_0(). Then, a reasonable objective for the optimization would be the KL-divergence between q_T() and the target density p(). However, the evaluation of the KL-divergence includes the entropy of q_T(), whose density is intractable due to numerous accept/reject steps. The proposed solution to this difficulty is to ignore the entropy term and maximize the log density of the target on samples from q_T(). To avoid the degenerate solution (due to ignorance of the entropy), the authors propose to choose q_0() carefully, e.g., to learn q_0() as a normalizing flow approximating the target p() via minimization of mass-covering alpha-divergence. The latter involves the usage of samples from the target distribution.

Major concerns:
1. The method is an engineering trick rather than a grounded approach to the optimization of sampling algorithms. Indeed, in many cases, people use MCMC methods to obtain guarantees for the sampling procedure. The proposed method removes all these guarantees by relying on the choice of the initial distribution q_0(). Moreover, the optimization of q_0() via mass-covering objectives is a notoriously hard problem since samples from the target distribution are not given in a usual setting.

2. I think the paper lacks an essential comparison with the method proposed by Titsias (Gradient-based Adaptive Markov Chain Monte Carlo, 2019). This paper proposes a more general objective for parameter optimization explicitly fostering high entropy of the proposal. Moreover, in contrast with the learning step of q_0(), it operates in an adaptive manner, not requiring any pretraining steps.

3. Given the limited theoretical novelty, I would expect the ICLR paper to demonstrate highly successful empirical results. However, it is not the case for the current submission. I'm quite confident that the results on CV tasks are out of practical interest. Also, for the molecular dynamics, the metrics' choice hinders the assessment of the practical significance.

Minor comments:
1. I don't find the comparison of marginal distributions on the 60d problem to be a convincing way to compare samplers' performance. I would suggest considering either another metric or another problem.
2. I also would suggest to include the description (at least the formula for the density) of the problem "molecular configurations." It would provide the reader with an additional intuition on its difficulty.
3. I think section 4 would benefit from the clear description of the choice of s, for instance, from the description of the variable mu, which appears there for the first time.

Additional comments:
After rereading the review, I feel that it may sound a bit harsh for the authors. Therefore, I want to say aloud that I find the paper's subject to be of great interest, consider any work in this direction valuable, and encourage the authors to continue their studies. My criticism is only an attempt to approach the review process objectively.
review_title: an engineering trick with limited practical significance

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
The paper introduces a gradient-based approach for tuning the step-size and the diagonal mass matrix of HMC together with the parameters of an initial distribution for the Markov chain. They suggest different objectives amenable for SGD: maximize the expected target log-density of the final state of the chain, but also an objective to ensure a somewhat ‘wide’ initial distribution. The approach is illustrated on 2-d toy models, deep latent Gaussian models on (Fashion) MNIST and molecular configurations.

Positives:
The submission suggests a practical approach for tuning HMC that remains a challenging problem. The combination of the different objectives is new as far as I am aware.  Empirical experiments are provided to justify the approach on standard benchmark problems, where it is seems to be competitive with state of the art methods, and a more extensive study on sampling molecular configurations. 

Negatives:
I feel that further arguments are needed to justify why the entropy of the proposed state can be ignored when adapting the hyperparameters of the sampler. The paper argues that “Since HMC, by construction, cannot collapse to such a point mass, we argue that the entropy term can be dropped provided the initial distribution of the chain has enough coverage of the target”. I am not convinced by this: take a standard normal target, then a leapfrog-integrator with 2 steps, unit mass matrix and step size of sqrt(2) proposes deterministically from a point mass distribution and this happens everywhere on the state space. While this might be an unrealistic example, it is not clear to me how such situations can be avoided in general.
It is also not clear to me why the Sliced Kernelized Stein Discrepancy objective automatically adjusts the width of the initial distribution. In equation (4) the discrepancy is between the final state and the target and I fail to see how this relates to the width of the initial density.

Recommendations:
I vote for a weak reject at the moment. The ideas proposed in the paper are indeed interesting. However, I am not yet convinced that the objectives yield HMC kernels that explore the state space well (so the HMC proposal does not become close to deterministic/completes a U-turn so that entropy comes largely from the initial distribution which is however trained with a different objective). Also the use of the Sliced Kernelized Stein Discrepancy specifically should be better motivated. I am happy to increase my score if the authors better clarify these points.

Further comments/issues:
The authors claim in the abstract that existing approaches “optimize a tractable lower bound that is too loose to be useful in practice”. Can this be backed up more concretely? I understand that such methods (such as Thin et al., 2020) use a looser bound, but not that these types of bounds are useless in practice.
In section 3.1, how do the acceptance rates compare for the narrow vs the wide initial distribution? My intuition would be that the acceptance rates for the narrow one are smaller than for the wide one. Would it then be possible to get a better exploration even in this case by including an objective to target an acceptance rate (say increase the stepsize if the acceptance rate is above 0.65)?


Minor comments:
Is it obvious that equation (6) minimizes the 1-divergence? For k=1, is this not the standard VAE/0-divergence, while for k>1 the IWAE objective can be seen as a 0-divergence on an extended space?
What are the \gamma variables simulated from N(0,I) exactly? Are they really the momentum variables? Are the initial momentum variables not from N(0,diag(m))?
In the experiments from Section 5.1, why do you target a minimum acceptance rate of 0.25 and not an average rate of 0.65, which seems a more common choice in the adaptive MCMC literature?


review_title: Official Review 2

Meta-review: This paper proposes a tuning strategy for Hamiltonian Monte Carlo (HMC). The proposed algorithm optimizes a modified variational objective over the T step distribution of an HMC chain. The proposed scheme is evaluated experimentally.

All of the reviewers agreed that this is an important problem and that the proposed methods is promising. Unfortunately, reviewers had reservations about the empirical evaluation and the theoretical properties of the scheme. Because the evaluation of the scheme is primarily empirical, I cannot recommend acceptance of the paper in its current form.

I agree with the following specific reviewer concerns. The proposed method does not come with any particular guarantees, and particularly no guarantees regarding the effect of dropping the entropy term and using an SKSD training scheme to compensate. While guarantees are not necessary for publication, the paper should make up for this with comprehensive and convincing experiments. I agree with R1 that more careful ablation studies on toy models are needed, if nothing else to reveal the strengths and weaknesses of the proposed approach. I would also recommend a more careful discussion about the computational cost of this method and how it can be fairly compared to baselines. I don't agree that "deliberately wasteful" experiments reveal much, especially if running more realistic experiments reduces the relative impact of the proposed method.
Decision: Reject