Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes to adapt the mass matrix used in HMC by maximising the speed measure introduced in [1].
Estimating the speed measure is non-trivial and it is achieved by firstly finding a lower bound of the density induced by a HMC kernel and then estimating the bound using Russian-roulette sampling.
Experiments performed on multiple targets show that the method is working in practice.

[1] Michalis Titsias and Petros Dellaportas. Gradient-based adaptive Markov chain Monte Carlo. NeurIPS, 2019.
main_review: Originality: 
Successfully using the speed measure introduced in [1] to adapt mass matrix in HMC is novel contribution to the field.
Related work has been adequately cited and discussed.

Quality: 
I appreciate the technical aspects of the method which successfully adapts the mass matrix using the speed measure from [1] via gradients,
but I found the experiments are poorly done.
In particular, the main claim of the paper is that the speed measure from [1] is a better metric to adapt in compare to expected squared jumping distance (ESJD) (or other metrics).
However, no direct comparisons to alternative metrics are done but only NUTS, a dynamics HMC method, and (in some experiments) Riemann HMC are compared.
The author(s) should compare normal HMC with mass matrix adapted using ESJD, e.g. those methods discussed in the related work section.
Besides, the setup of NUTS is weird.
- Why do you choose NUTS with only step size adaptation while comparing to the proposed method with mass matrix adaptation? Even mass matrix is adapted, NUTS is not using ESJD as a metric for adaptation.
- Why do you use a maximum depth of $10$? Shouldn't you use a more comparable value as to $L$ for the proposed method? I suspect figure 1 can change a lot if the parameter of NUTS is set differently.

Clarity: 
The method itself is easy to follow but I found the experiments section is poorly done, as I commented above.
Experiments that are designed to support the main claim are missing and the current ones tend to evaluate some characteristics of the proposed method, which are interesting but not of the top priority, e.g. why all figures focus on evaluating how ESS changes with different number of leapfrog steps?

Significance: 
The method is technically involved but the current results are limited to convince practical use of the method.

Misc:
- I quite like the outlook discussed in section 5. One thing I would add is that adapting the proposed method to multinomial HMC can be interesting as the simulation error is less of a problem there. It could make the point on coupled HMC in section 5 more interesting too, as [2] recently shows that multinomial HMC is more effective than (standard) HMC in terms of couplings.
- You should capitalise words like **M**arkov in your references. For `bibtex` used in $\LaTeX$, you can achieve so by adding `{}` to the letter to make sure it's capital.

[1] Michalis Titsias and Petros Dellaportas. Gradient-based adaptive Markov chain Monte Carlo. NeurIPS, 2019.

[2] Kai Xu, Tor Erlend Fjelde, Charles Sutton, Hong Ge. Couplings for Multinomial Hamiltonian Monte Carlo. AISTATS, 2021

---

Update after new results (other alternative metrics and NUTS):

- The comparisons against ESJD and L2HMC support the motivation of using the speed measure as the optimisation metric, which is crucial.
- The comparisons against a more common setup of NUTS (dual-averaging + diagonal mass matrix adaptation) is useful to show that the proposed methods can outperform the "standard" NUTS with more than one hyper-parameter setup (which is important to see the method is not sensitive to a particular one) and supports the empirical usefulness of the method.

I encourage the author to release the source code of the method as a ready-to-use TFP add-on as the method itself can be a bit involved to implement.
limitations_and_societal_impact: Limitations are discussed in section 5.
No societal impact is discussed but it's not appleid to the paper.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper develops a gradient-based algorithm that enables adaptation of the mass matrix by taking into account the leapfrog integration error. The adaptation strategy incorporates an approximation of the proposal entropy in comparison to previous approaches that use other techniques such as the expected squared jumping distance.
main_review: # Originality:

My main concern reading the paper was whether the authors would be able to significantly add to the work of [45]. However, the more I went through the paper, the more I could see that there was a definite contribution and that this approach enabled the direct application of the generalized speed measure to a multiple step scenario in HMC.

Given the clear writing and excellent references to existing work, I am confident that the work is placed sufficiently among previous works.

# Quality and Clarity

This paper is written to a high standard and is structured well. I feel I was able to follow the main arguments displayed in the maths due to the thoughtful structuring of the theory section. However I am unable to verify and validate every equation (not due to the author’s writing style, but due to my finite time constraints!). 

The Figures are a bit small, and perhaps the text could be a larger font size, as well as appropriately capitalising the axis labels. Furthermore, the experiments section could be a bit more focused on what the key take-aways are. I can see from the results that the adaptation works but I am not quite sure what exactly I am looking out for in particular. To me it seems previously known that a larger number of leap-frog steps leads to a better performance in higher dimensional settings. Perhaps more details of the overall take-away from the numerical experiments could be clarified further in the rebuttal? Finally, the experiment in Section 4.5 is very impressive and I actually think that the paper is too “humble” about it. Maybe it was referenced earlier in the paper, but I was surprised (in a good way) to see it pop up at the end! 

Section 5 is well-written and demonstrates good knowledge of the area and limitations of the approach.

# Comments and Questions:
1)	How does the computational complexity of estimating the Hessian compare to just performing RMHMC directly? I am assuming that the technique is to use the Hutchinson stochastic trace estimation to approximate the Hessian using the vjp? I got a bit confused about the vector $w$, how does that get included?
2)	In the previous work of [45], it is clear that they are adapting the proposal distribution (e.g. the covariance). For HMC the proposal is defined using the Hamiltonian. I just wanted to check that the parameters that are being adapted is just the lower triangular $C$ of the mass matrix in the kinetic term?
3)	I actually had a question earlier on in the paper that was subsequently answered. However, I think it would be helpful to mention the need to only perform this adaptation during the burn-in stage a bit earlier in the paper. 
4)	What is “stop_grad” in algorithm 1?
5)	Allowing NUTS to have a tree depth of $10$ leads to a maximum trajectory length of $2^{10}$. In my experience NUTS will often take advantage of the largest tree depth that is given to it (especially in high-dimensional spaces). Were other tree depths tried and is this why the Min ESS/sec is significantly lower for NUTS? In the paper the adaptive strategy only goes up to an $L=10$, maybe using a tree depth $2^5 = 32$ would be a fairer comparison?


### After Author Response:

I would like to thank the author(s) for their responses to all the reviewers (especially with the additional experiments in response to ZogC). As a result I am happy to keep my current score. 
limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: In this paper, the authors propose an adaptive Hamiltonian Monte Carlo method. In their method, they adapt the hyperparameters of HMC by maximizing an approximation of the proposal entropy. And they show the efficiency of their method by experiments.
main_review: Originality, quality, significance: In this paper, the authors propose a new HMC method. Since the reviewer is not familiar with the HMC methods, the reviewer can't judge its novelty. However, the reviewer thinks the contribution is not enough. There are not any theoretical clues or justifications showing the efficiency of their new methods. And the reviewer also thinks that its intuition is not very reasonable. 

Clarity: The reviewer thinks this paper is very unclear. There are a lot of typos. 

Overall, the reviewer thinks that a paper that only contains empirical results is not good enough to be published in a theory-focus conference. The reviewer will list questions and concerns in "Limitation". 
limitations_and_societal_impact: There are some concerns and questions listed below. Since the reviewer is not familiar with HMC, there might exist some misunderstanding:

1. The reviewer is confused about (9) and the derivations before. There are two questions:

   [1]. From line 107-114, the authors try to derive a lower bound of $\log|\text{det}(1+DS)L(v)|$. And then the authors claim we can maximize the lower bound instead. However, if we already have a Gaussian target, then $DSL$ can be explicitly written down, why do we need to lower bound it? Even we want to do an extension later, the explicit formula can still be used. Can the authors give more comments on this?

   [2]. The equation (9) is really weird. If the target is not a Gaussian, then $\Sigma^{-1}$ should be replaced by summation of $\nabla^2U$. Then why it can be simply replaced by a single term of $\nabla^2U$. The intuition that follows seems unreasonable. Why can it be approximated by a Gaussian? Can the authors give more comments or detail on this?

2. In Section 3.2 line 151, the author claims that they solve (11) by lines 151-152. The reviewer thinks this is also a really big approximation. The authors basically replace an expectation with a one-particle estimation. Can the authors give more comments on this about why this is a reasonable approximation?

3. In Algorithm 1 line 10-line 13, the reviewer thinks the author runs one gradient descent for the minimization problem, is that right? If the authors only run one gradient descent, why does this give an (approximated) minimizer?  Besides, why do we need to project $\beta,\gamma$ on a compact set?

It seems that this paper has a lot of typos that affect reading. The reviewer lists some possible typos:

1. In line 84, the authors might make this $\theta$ clear. For example, write $C$ as $C(\theta)$. The reviewer feels it's hard to find the origin of this $\theta$ later.

2. In line 90-91 (3): $\mathcal{T}_L\rightarrow\mathcal{T}_i$ ?

3. Since there are typos in (3), the reviewers hope the authors can add calculations of (2) in the appendix. 

4. In Lemma 1, what is $\|\cdot\|_p$?

5. In line 110, according to the lemma, we should have $\|C^\top\Sigma^{-1}C\|_2\leq \frac{h^2L^2}{4}$.

6. In Algorithm 1 line 5, what is (13)?
review_title: null

Meta-review: The paper introduces a gradient-based sampling algorithm with adaptive mass matrix to account for the integration error. The referees find the writing of the paper excellent and also agree upon the numerical experiments being convincing after multiple round of discussions. The paper should be accepted to the conference as a poster. 
Decision: Accept (Poster)