Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: In this paper, the authors proposed a new algorithm -- expected information maximization (EIM) -- for computing the I-projection of the data distribution to the model distribution, solely based on samples for general latent variable models, where the paper only focus on Gaussian mixtures models and experts. The proposed method applies a variational upper bound to the I-projection objective which is decomposable for each mixture components and the coefficients. Overall, I think the proposed technique quite sound and results are convincing. However, I do have some questions:

Questions:
-- The proposed EIM algorithm in Sec 4.1 seems to require “re-training” the discriminator every time the q function is updated. How this can be applicable to more realistic and complex models where training requires millions of gradient steps? Will the same algorithm can be applied on more general latent variable models or even implicit models like GAN does? As the paper has pointed out, the vanilla f-GAN itself can be seen as optimizing some forms of the I-Projection (reverse Kullback-Leibler divergence) objective.
-- I am a little confused about Sec 4.3. It seems that the latent variable z is not necessary for the proposed EIM? 
-- Also, the typical practise of training GAN is also iterative between the generator and the discriminator, we sometimes need to update the discriminator with more steps than the generator? Shouldn’t it be the exactly same as the proposed EIM except we have an additional regularization term of KL(q(x) || q_t(x)) which might be the true reason why training gets more stable than standard GAN?
-- Similar to the previous two questions, how to compute the regularization term KL(q(x) || q_t(x)) in EIM for normal generator which is typically implicit?

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper propose EIM an analog to EM but to perform the I-projection (i.e. reverse-KL) instead of the usual M-projection for EM. The motivation is that the reverse-KL is mode-seeking in contrast to the forward-KL which is mode-covering. The authors argue that in the case that the model is mis-specified, I-projection is sometimes desired as to avoid putting mass on very unlikely regions of the space under the target p.

The authors propose an iterative procedure that alternates between estimating likelihood ratios and proposal distribution by minimizing an upper bound on the reverse-KL. The derivations seem correct. There are some experiments, majoritarily in the robotics domain. As the author point out, likelihood shouldn't be the right metric since you are now minimizing the reverse-KL---I would have liked the authors to spend some more time on the right way to evaluate---and actually use that new metric. Finally, there has been plethora of work on different objectives and distance between distributions as well as a zoo of lower/upper bounds on how to evaluate them---it would be interesting to have more connections to prior work.

[Pros]
- clearly written
- clear motivation
- correct derivations
- interesting algorithm

[Cons]
- experiments are a little weak (and focus on a single domain)
- would have liked to see an explicit algorithm for the optimization procedure
- small lack of clarity in the presentation of Section 4.1---notation q_t is not introduced for example
- more discussion about the evaluation metric
- linking it more to prior work

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The paper presents an algorithm to match two distributions with latent variables, named expected information maximization (EIM). Specifically, EIM is based on the I-Projection, which basically is equivalent to minimizing the reverse KL divergence (i.e. min KL[p_model || p_data]); to handle latent variables, an upper-bound is derived, which is the corresponding reverse KL divergence in the joint space. To minimize that joint reverse KL, a specific procedure is developed, leading to the presented EIM. EIM variants for different applications are discussed. Fancy robot-related experiments are used to evaluate the presented algorithm.

Overall, the paper is in good shape wrt the logic and the writing. My main concerns focus on the novelty (compared to existing methods that are similar but not discussed) and the experiments. For the former, reverse KL has been exploited before, both in the marginal space [1] and the joint one [2]. Other detailed comments are listed below.

As Eq 4 is for matching two joint distributions, discussions/comparisons should be made to reveal the novelty of the presented EIM over existing methods such as [2], etc.

In Figure 2 (b), the experimental settings for adversarial learning are not fair, as the discriminator is not fixed there. 
 
In Sec 4.4, it seems EIM is highly overlapped with VIPS. So what're the advantages of EIM here?

In Figure 3, how many steps for Generator and Discriminator are used for f-GAN? Does f-GAN finally converge? It would be helpful if some results are given to demonstrate the final state of each method.

In Eq. 9, adding the denominator q(z_i) will change the optimal solution. Why only add it to the first term?

In Section 5.3 and Figure 5, “SSD” might be a typo. 

[1] Adversarial Learning of a Sampler Based on an Unnormalized Distribution. AISTATS 2018.
[2] Symmetric Variational Autoencoder and Connections to Adversarial Learning. AISTATS 2019.

review_title: Official Blind Review #2

Meta-review: The paper proposes a new algorithm called Expected Information Maximization (EIM) for learning latent variable models while computing the I-projection solely based on samples. The reviewers had several questions, which the authors sufficiently answered. The reviewers agree that the paper should be accepted. The authors should carefully read the reviewer questions and comments and use them to improve their final manuscript. 
Decision: Accept (Poster)