Paper:
We propose a black-box algorithm called Adversarial Variational Inference and Learning (AdVIL) to perform inference and learning in a general Markov random field (MRF). AdVIL employs two variational distributions to approximately infer the latent variables and estimate the partition function of an MRF, respectively. The two variational distributions provide an estimate of the negative log-likelihood of the MRF as a minimax optimization problem, which is solved by stochastic gradient descent. AdVIL is proven convergent under certain conditions. On one hand, compared to the contrastive divergence, AdVIL requires minimal assumptions about the model structure and can deal with a broader family of MRFs. On the other hand, compared to existing black-box methods, AdVIL provides a tighter estimate of the log partition function and achieves much better empirical results.

Review:
This paper presents a black-box style learning algorithm for Markov Random Fields (MRF). The approach doubles down on the variational approach with variational approximations for both the positive phase and negative phase of the log likelihood objective function. For the negative phase, the authors use two separate variational approximations, one of which involves the modeling of the latent variable prior under the approximating distribution,
The approach is novel, as far as I know, though not particularly so, and I view this as one of the weak point of the paper. That said, it does seems like a fairly creative combination of existing approaches. As others have found in the past, a variational approximation to the partition function contribution to the loss function (i.e. the negative phase) results in the loss of the variational lower bound on log likelihood and the connection between the resulting approximation and the log likelihood becomes unclear. To deal with this issue, the authors argue (in Lemma 1) that the gradient of their approximate objective is at least in the same direction as the ELBO (lower bound) objective. The result is fairly obvious, but the conditions for validity have interesting consequences for the training algorithm, as it relates the approximation error to the norm of the gradient of the ELBO loss.
I have a minor issue with the discussion (in the last paragraph of sec. 3.2) stating that the theoretical statement of the proposed objective relies on a much weaker assumption than the nonparametric assumption made in the theoretical justification of GANs. While I agree with the statement as such, the GAN development makes a stronger statement about the nature of the learning trajectory. Specifically, it states that the generator is minimizing a Jenson-Shannon divergence which has a fixed point at the true data density. In the current development, Theorem 1 only states that the optimization process will converge to the stationary points of the approximate ELBO objective (L1 in the paper's notation).
Clarity: I found the paper to be very well written with a clear exposition of the material and sound development of the technical details. 
Relevance and Significance: This paper is highly relevant to the ICLR community and -- to the extent that one believes that training and inference in MRFs is important -- also significant. One this last point, it seems ironic to me that the proposed strategy for training the MRF is through the use of three separate directed graphical models (an encoder q(h | x),  a decoder and a VAE to model the approximate prior over the latents h). In most modeling situations, one would simply impose the directed graphical model directly and skip the formalization in terms of an MRF. I would appreciate a more forceful motivation of the relevance of MRFs rather than just stating it as a important model with applications. What is unique
about the MRF formalism that -- for practical applications -- could not be effectively captured in a directed graphical model?
I note that I am aware of the theoretical representation differences between directed and undirected models, I am wondering how these differences actually matter in practical applications at scale.
Experiments: The authors show the empirical advantages offered by the proposed method over the existing literature. I was surprised not to see how this model performs on the binarized MNIST dataset, and would like to see that result as well as CIFAR likelihood.  MNIST, in particular, is a well studied dataset that many readers will be able to easily interpret. Its absence seems like a serious omission.
What is meant by "RBM loss" in Fig. 2(d), I do not see this defined? 
I am somewhat alarmed at the use of 100 updates of the joint model q(v,h) (K1 = 100) for every update of the other parameters. For larger scale domains, I fear this could become an important obstacle to effective model training. The comparison to PCD-1 in Fig. 3 seems a bit unfair in that the learning curve ends at 8000 iterations, while PCD-1 continues to
improve NLL. I would like to see this curve extended until we start to see signs of overfitting. Perhaps PCD-1 results in performance that is far better than AdVIL. I would also like to see a comparison to CD-k, which often outperforms PCD-k. While I understand the stance taken by the authors that these methods leverage the tractability of the conditional
distributions, these strategies are sufficiently general to be considered widely applicable and a true competitor for AdVIL. 
With respect to Deep Boltzmann Machine (DBM), I would prefer to see quantitative comparisons against published results. Here again, MNIST would be a useful dataset.  It seems as though, in the application of AdVIL to the DBM, the authors are exploiting the structure of the model in how they define their sampling procedure. Is that the case? More detail for this application of AdVIL would be nice. Also, I would like to see the test estimated NLL (via AIS) learning curves for VCD and AdVIL. Given the comparison to PCD in the RBM setting, I am somewhat surprised that AdVIL is so competitive with VCD in
the case of the DBM.