Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper presents Kernel Implicit Variational Inference (KIVI), a novel class of implicit variational distributions. KIVI relies on a kernel approximation to directly estimate the density ratio. Importantly, the optimal kernel approximation in KIVI has closed-form solution, which allows for faster training since it avoids gradient ascent steps that may soon get "outdated" as the optimization over the variational distribution runs. The paper presents experiments on a variety of scenarios to show the performance of KIVI.

Up to my knowledge, the idea of estimating the density ratio using kernels is novel. I found it interesting, specially since there is a closed-form solution for this estimate. The closed form solution involves a matrix inversion, but this shouldn't be an issue, as the matrix size is controlled by the number of samples, which is a parameter that the practitioner can choose. I also found interesting the implicit MMNN architecture proposed in Section 4.

The experiments seem convincing too, although I believe the paper could probably be improved by comparing with other implicit VI methods, such as [Liu & Feng], [Tran et al.], or others.

My major criticism with the paper is the quality of the writing. I found quite a few errors in every page, which significantly affects readability. I strongly encourage the authors to carefully review the entire paper and search for typos, grammatical errors, unclear sentences, etc.

Please find below some further comments broken down by section.

Section 1: In the introduction, it is unclear to me what "protect these models" means. Also, in the second paragraph, the authors talk about "often leads to biased inference". The concept to "biased inference" is unclear. Finally, the sentence "the variational posterior we get in this way does not admit a tractable likelihood" makes no sense to me; how can a posterior admit (or not admit) a likelihood?

Section 3: The first paragraph of the KIVI section is also unclear to me. In Section 3.1, it looks like the cost function L(\hat(r)) is different from the loss in Eq. 1, so it should have a different notation. In Eq. 4, I found it confusing whether L(r)=J(r). Also, it would be nice to include a brief description of why the expectation in Eq. 4 is taken w.r.t. p(z) instead of q(z), for those readers who are less familiar with [Kanamori et al.]. Finally, the motivation behind the "reverse ratio trick" was unclear to me (the trick is clear, but I didn't fully understand why it's needed).

Section 4: The first paragraph of the example can be improved with a brief discussion of why the methods of [Mescheder et al.] and [Song et al.] "are nor applicable". Also, the paragraph above Eq. 11 ("When modeling a matrix...") was unclear to me.

Section 6: In Figure 1(a), I think there must be something wrong, because it is well-known that VI tends to cover one of the modes of the posterior only due to the form of the KL divergence (in contrast to EP, which should look like the curve in the figure). Additionally, Figure 3(a) (and the explanation in the text) was unclear to me. Finally, I disagree with the discussion regarding overfitting in Figure 3(b): that plot doesn't show overfitting because it is a plot of the training loss (and overfitting occurs on test); instead it looks like an optimization issue that makes the bound decrease.


**** EDITS AFTER AUTHORS' REBUTTAL ****

I increased the rating to 7 after reading the revised version.

review_title: Interesting and novel idea but poor writing quality

final_score: rating: 5: Marginally below acceptance threshold
review_content: Update: I read the other reviews and the authors' rebuttal. Thanks to the authors for clarifying some details. I'm still against the paper being accepted. But I don't have a strong opinion and will not argue against so if other reviewers are willing. 

------

The authors propose Kernel Implicit VI, an algorithm allowing implicit distributions as the posterior approximation by employing kernel ridge regression to estimate a density ratio. Unlike current approaches with adversarial training, the authors argue this avoids the problems of noisy ratio estimation, as well as potentially high-dimensional inputs to the discriminator.  The work has interesting ideas. Unfortunately, I'm not convinced that the method overcomes these difficulties as they argue in Sec 3.2.

An obvious difficulty with kernel ridge regression in practice is that its complete inaccuracy to estimate high-dimensional density ratios.  This is especially the case given a limited number of samples from both p and q (which is the same problem as previous methods) as well as the RBF kernel. While the RBF kernel still takes the same high-dimensional inputs and does not involve learning massive sets of parameters, it also does not scale well at all for accurate estimation. This is the same problem as related approaches with Stein variational gradient descent; namely, it avoids minimax problems as in adversarial training by implicitly integrating over the discriminator function space using the kernel trick.

This flaw has rather deep implications. For example, my understanding of the implicit VI on the Bayesian neural network in Sec 4 is that it ends up as cross-entropy minimization subject to a poorly estimated KL regularizer. I'd like to see just how much entropy the implicit approximation has instead of concnetrating toward a point; or more directly, what the implicit posterior approximation looks like compared to a true posterior inferred by, say, HMC as the ground truth. This approach also faces difficulties that the naive Gaussian approximation applied to Bayesian neural nets does not: implicit approximations cannot exploit the local reparameterization trick and are therefore limited to specific architectures that does not involve sampling very large weight matrices.

The authors report variational lower bounds, which I'm not sure is really a lower bound. Namely, the bias incurred by the ratio estimation makes it difficult to compare numbers. An obvious but very illustrative experiment I'd like to see would be the accuracy of the KL estimator on problems where we can compute it tractably, or where we can Monte Carlo estimate it very well under complicated but tractable densities. I also suggest the authors perform the experiment suggested above with HMC as ground truth on a non-toy problem such as a fairly large Bayesian neural net.
review_title: Interesting idea; not clear it scales; needs experiments on quality of ratio estimation and also posterior approximation

final_score: rating: 7: Good paper, accept
review_content: Thank you for the feedback, and I think many of my concerns have been addressed.

I think the paper should be accepted.

==== original review ====

Thank you for an interesting read. 

Approximate inference with implicit distribution has been a recent focus of the research since late 2016. I have seen several papers simultaneously proposing the density ratio estimation idea using GAN approach. This paper, although still doing density ratio estimation, uses kernel estimators instead and thus avoids the usage of discriminators. 

Furthermore, the paper proposed a new type of implicit posterior approximation which uses intuitions from matrix factorisation. I do think that another big challenge that we need to address is the construction of good implicit approximations, which is not well studied in previous literature (although this is a very new topic). This paper provides a good start in this direction.

However several points need to be clarified and improved:
1. There are other ways to do implicit posterior inference such as amortising deterministic/stochastic dynamics, and approximating the gradient updates of VI. Please check the literature.
2. For kernel based density ratio estimation methods, you probably need to cite a bunch of Sugiyama papers besides (Kanamori et al. 2009). 
3. Why do you need to introduce both regression under p and q (the reverse ratio trick)? I didn't see if you have comparisons between the two. From my perspective the reverse ratio trick version is naturally more suitable to VI.
4. Do you have any speed and numerical issues on differentiating through alpha (which requires differentiating K^{-1})?
5. For kernel methods, kernel parameters and lambda are key to performances. How did you tune them?
6. For the celebA part, can you compute some quantitative metric, e.g inception score?

review_title: A good contribution to implicit approximate posterior fitting

Meta-review: Thank you for submitting you paper to ICLR. This paper was enhanced noticeably in the rebuttal period and two of the reviewers improved their score as a result. There is a good range of experimental work on a number of different tasks. The addition of the comparison with Liu & Feng, 2016 to the appendix was sensible. Please make sure that the general conclusions drawn from this are explained in the main text and also the differences to Tran et al., 2017 (i.e. that the original model can also be implicit in this case).
Decision: Accept (Poster)