Venue: ICLR 2018 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper introduces a non-volume-preserving generalization of HMC whose transitions are determined by a set of neural network functions. These functions are trained to maximize expected squared jump distance.
This works because each variable (of the state space) is modified in turn, so that the resulting update is invertible, with a tractable transformation inspired by Dinh et al 2016.

Overall, I believe this paper is of good quality, clearly and carefully written, and potentially accelerates mixing in a state-of-the-art MCMC method, HMC, in many practical cases. A few downsides are commented on below.

The experimental section proves the usefulness of the method on a range of relevant test cases; in addition, an application to a latent variable model is provided sec5.2. 
Fig 1a presents results in terms of numbers of gradient evaluations, but I couldn't find much in the way of computational cost of L2HMC in the paper. I can't see where the number "124x" in sec 5.1 stems from. As a user, I would be interested in the typical computational cost of both "MCMC sampler training" and MCMC sampler usage (inference?), compared to competing methods. This is admittedly hard to quantify objectively, but just an order of magnitude would be helpful for orientation. 
Would it be relevant, in sec5.1, to compare to other methods than just HMC, eg LAHMC?

I am missing an intuition for several things: eq7, the time encoding defined in Appendix C

Appendix Fig5, I cannot quite see how the caption claim is supported by the figure (just hardly for VAE, but not for HMC).

The number "124x ESS" in sec5.1 seems at odds with the number in the abstract, "50x".

# Minor errors
- sec1: "The sampler is trained to minimize a variation": should be maximize
"as well as on a the real-world"
- sec3.2 "and 1/2 v^T v the kinetic": "energy" missing
- sec4: the acronym L2HMC is not expanded anywhere in the paper
The sentence "We will denote the complete augmented...p(d)" might be moved to after "from a uniform distribution" in the same paragraph. 
In paragraph starting "We now update x":
    - specify for clarity: "the first update, which yields x' "/ "the second update, which yields x''  "
    - "only affects $x_{\bar{m}^t}$": should be $x'_{\bar{m}^t}$  (prime missing)
    - the syntax using subscript m^t is confusing to read; wouldn't it be clearer to write this as a function, eg "mask(x',m^t)"?
    - inside zeta_2 and zeta_3, do you not mean $m^t" and $\bar{m}^t$ ?
- sec5: add reference for first mention of "A NICE MC"
- Appendix A: 
    - "Let's" -> "Let"
    - eq12 should be x''=...
- Appendix C: space missing after "Section 5.1"
- Appendix D1: "In this section is presented" : sounds odd
- Appendix D3: presumably this should consist of the figure 5 ? Maybe specify.
review_title: Good paper on fast mixing extension of HMC

final_score: rating: 7: Good paper, accept
review_content: In this work, the authors propose a procedure for tuning the parameters of an HMC algorithm (I guess, if I have understood correctly).

I think this paper has a good and strong point: this work points out the difficulties in choosing properly the parameters in a HMC method (such as the step and the number of iterations in the leapfrog integrator, for instance). In the literature, specially in machine learning, there is ``fever’’ about HMC, in my opinion, partially unjustified.

If I have understood, your method is an adaptive HMC algorithm  where the parameters are updated online; or is the training  done in advance? Please, remark and clarify this point.

However, I have other additional comments:

- Eqs. (4) and (5) are quite complicated; I think a running toy example can help the interested reader.

- I suggest to compare the proposed method to other efficient methods that do not use the gradient information (in some cases as multimodal posteriors, the use of the gradient information can be counter-productive for sampling purposes), such as Multiple Try Metropolis (MTM) schemes

L. Martino, J. Read, On the flexibility of the design of Multiple Try Metropolis schemes, Computational Statistics, Volume 28, Issue 6, Pages: 2797-2823, 2013, 

adaptive techniques, 

H. Haario, E. Saksman, and J. Tamminen. An adaptive Metropolis algorithm. Bernoulli, 7(2):223–242, April 2001,

and component-wise strategies as Gibbs Sampling, 

W. R. Gilks and P. Wild, Adaptive rejection sampling for Gibbs sampling, Appl. Statist., vol. 41, no. 2, pp. 337–348, 199. 

At least, add a brief paragraph in the introduction citing and discussing this possible alternatives.
review_title: Nice idea; but the paper could be improved.

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposed a generalized HMC by modifying the leapfrog integrator using neural networks to make the sampler to converge and mix quickly. Mixing is one of the most challenge problems for a MCMC sampler, particularly when there are many modes in a distribution. The derivations look correct to me. In the experiments, the proposed algorithm was compared to other methods, e.g., A-NICE-MC and HMC. It showed that the proposed method could mix between the modes in the posterior. Although the method could mix well when applied to those particular experiments, it lacks theoretical justifications why the method could mix well. 
review_title: The paper proposed a generalized HMC with neural networks, but it lacks theoretical justifications why it could mix with multiple modes

Meta-review: This paper presents a learned inference architecture which generalizes HMC. It defines a parameterized family of MCMC transition operators which share the volume preserving structure of HMC updates, which allows the acceptance ratio to be computed efficiently. Experiments show that the learned operators are able to mix significantly faster on some simple toy examples, and evidence is presented that it can improve posterior inference for a deep latent variable model. This paper has not quite demonstrated usefulness of the method, but it is still a good proof of concept for adaptive extensions of HMC.


Decision: Accept (Poster)