Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The paper presents a new hybrid method to unify MCMC and VI. The key idea is to interpret a ﬁnite-length MCMC/HMC chain as a parametric procedure, whose parameters can be optimized via a VI-motivated objective. Specifically, the authors propose to modify the well-known ELBO (which is now non-trivial due to the intractable entropy) to form a new constrained and tractable objective. The presented techniques are tested on synthetic datasets and with the experiments of a VAE on MNIST. 

The presented technique is interesting. However, there are several concerns of mine that should be addressed, as detailed below.

The notations of \pi and \pi^* are very confusing. I guess \pi represents the marginal distribution of the last state of the MCMC chain, while \pi^* is the target distribution. Is that right? Please clarify their meanings. 

There are related works that combine MCMC and VI, such as [1]. What are the advantages of the proposed method compared to that method? 
[1] Francisco J. R. Ruiz and Michalis K. Titsias. A Contrastive Divergence for Combining Variational Inference and MCMC. International Conference on Machine Learning (ICML). 2019.

In equation 4, given fixed P_0 and a long enough MCMC chain, P_T will decorrelate with P_0. How to prevent P_T from collapsing to a delta function? Also intuitively, there should be a weight balancing the two terms of the loss; why a weight of 1 is used?

In equation 8, the function g_{phi} is not continuous because of the indicator function 1(). How do you back-propagate through that function? In the paragraph before Section 3.3, how would you defend the adopted stop-gradient trick?

In the paragraph before Figure 1, how to choose the hyperparameter h? It might not be suitable to set h as the entropy of the prior, as in practice prior and posterior might be different dramatically.

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: The presented method is very useful to deep learning in the era of uncertainty modelling, which requires the use of Bayesian inference arguments. It's a valuable improvement upon variational inference, it's novel, and the derivations are correct. The presentation is elaborate and covers all expected aspects. The literature review is up to date. 
The experimental results are diverse enough and convincing. The authors have considered both proof of concept experiments and deep learning architectures. The comparisons are valid. 

review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper proposes a new combination of Markov chain Monte Carlo (MCMC) and variational inference (VI) for improving approximate inference. The main contribution is the optimization objective that allows improving the quality of samples obtained from the combination of VI and MCMC. Specifically, the authors minimize the "approximate" version of the Kullback-Leibler (KL) divergence between the distribution of MCMC + VI and the true distribution. The authors validate the effectiveness of their formulation through experiments on 6 synthetic benchmarks and generative modeling of MNIST (experiments on Bayesian neural networks are also provided in the appendix). 

Overall, I think the paper provides a solid contribution towards combining MCMC and VI by proposing a way to optimize the MCMC part. The experiments validate the method by showing consistent improvement over existing methods. However, I believe the justification behind the proposed formulation, i.e., Equation (4) and (5), needs to be improved before being published at the conference.

First, for Equation (4), the explanation behind "replacing" H(P_{T}) with ELBO w.r.t. P_{0} is confusing. Specifically, it is reasoned that ELBO w.r.t. P_{t} only increase after MCMC steps. This statement is misleading since the replacement was done for H(P_{T}), not the ELBO w.r.t. P_{T}. 

I also think the Equation (5) is not properly justified. it is stated that the constraint is needed for preventing P_{T} to be closer to P_{0}. However, nothing is stated about the reason on why P_{T} gets closer to \pi when Equation (5) is satisfied.  Note that even if the expected log-likelihood of the distribution is high, it does not necessarily mean that the distribution is more similar. 

Minor comments:
- I was unable to understand why the algorithm is named "ergodic" inference. Both HVI and the proposed EI rely on the ergodic property of Markov chain for improving the variational distribution. I hope the authors could better illustrate on this point. I also think the term "ergodic approximation" in page 3. is hard to understand.
- I (weakly) suggest changing y-axis of Figure 5. to log-scale for better readability. It almost seems that the brown plot does not converge in Fig 5-(a).
- The paper could have been strengthened by performing experiments on more challenging datasets, e.g., CIFAR-10 or CIFAR-100. 

review_title: Official Blind Review #2

Meta-review: This paper presents a way of adapting an HMC-based posterior inference algorithm. It's based on two approximations: replacing the entropy of the final state with the entropy of the initial state, and differentiating through the MH acceptance step. Experiments show it is able to sample from some toy distributions and achieves slightly higher log-likelihood on binarized MNIST than competing approaches.

The paper is well-written, and the experiments seem pretty reasonable.

I don't find the motivations for the aforementioned approximations very convincing. It's claimed that encouraging entropy of P_0 has a similar effect to encouraging entropy of P_T, but it seems easy to come up with situations where the algorithm could "cheat" by finding a high-entropy P_0 which leads straight downhill to an atypically high-density region. Similarly, there was some reviewer discussion about whether it's OK to differentiate through the indicator function; while we differentiate through nondifferentiable functions all the time, it makes no sense to differentiate through a discontinuous function. (This is a big part of why adaptive HMC is hard.)

This paper has some promising ideas, but overall the reviewers and I don't think this is quite ready.

Decision: Reject