Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Update:
On further consideration (and reading the other reviews), I'm bumping my rating up to a 7. I think there are still some issues, but this work is both valuable and interesting, and it deserves to be published (alongside the Naesseth et al. and Maddison et al. work).

-----------

This paper proposes a version of IWAE-style training that uses SMC instead of classical importance sampling. Going beyond the several papers that proposed this simultaneously, the authors observe a key issue: the variance of the gradient of these IWAE-style bounds (w.r.t. the inference parameters) grows with their accuracy. They therefore propose using a more-biased but lower-variance bound to train the inference parameters, and the more-accurate bound to train the generative model.

Overall, I found this paper quite interesting. There are a few things I think could be cleared up, but this seems like good work (although I'm not totally up to date on the very recent literature in this area).

Some comments:

* Section 4: I found this argument extremely interesting. However, it’s worth noting that your argument implies that you could get an O(1) SNR by averaging K noisy estimates of I_K. Rainforth et al. suggest this approach, as well as the approach of averaging K^2 noisy estimates, which the theory suggests may be more appropriate if the functions involved are sufficiently smooth, which even for ReLU networks that are non-differentiable at a finite number of points I think they should be.

This paper would be stronger if it compared with Rainforth et al.’s proposed approaches. This would demonstrate the real tradeoffs between bias, variance, and computation. Of course, that involves O(K^2) or O(K^3) computation, which is a weakness. But one could use a small value of K (say, K=5).

That said, I could also imagine a scenario where there is no benefit to generating multiple noisy samples for a single example versus a single noisy sample for multiple examples. Basically, these all seem like interesting and important empirical questions that would be nice to explore in a bit more detail.

* Section 3.3: Claim 1 is an interesting observation. But Propositions 1 and 2 seem to just say that the only way to get a perfectly tight SMC ELBO is to perfectly sample from the joint posterior. I think there’s an easier way to make this argument:

Given an unbiased estimator \hat{Z} of Z, by Jensen’s inequality E[log \hat{Z}] ≤ log Z, with equality iff the variance of \hat{Z} = 0. The only way to get an SMC estimator’s variance to 0 is to drive the variance of the weights to 0. That only happens if you perfectly sample each particle from the true posterior, conditioned on all future information.

All of which is true as far as it goes, but I think it’s a bit of a distraction. The question is not “what’s it take to get to 0 variance” but “how quickly can we approach 0 variance”. In principle IS and SMC can achieve arbitrarily high accuracy by making K astronomically large. (Although [particle] MCMC is probably a better choice if one wants extremely low bias.)

* Section 3.2: The choice of how to get low-variance gradients through the ancestor-sampling choice seems seems like an important technical challenge in getting this approach to work, but there’s only a very cursory discussion in the main text. I would recommend at least summarizing the main findings of Appendix A in the main text.

* A relevant missing citation: Turner and Sahani’s “Two problems with variational expectation maximisation for time-series models” (http://www.gatsby.ucl.ac.uk/~maneesh/papers/turner-sahani-2010-ildn.pdf). They discuss in detail some examples where tighter variational bounds in state-space models lead to worse parameter estimates (though in a quite different context and with a quite different analysis).

* Figure 1: What is the x-axis here? Presumably phi is not actually 1-dimensional?

Typos etc.:

* “learn a particular series intermediate” missing “of”.

* “To do so, we generate on sequence y1:T” s/on/a/, I think?

* Equation 3: Should there be a (1/K) in Z?
review_title: Some very interesting ideas and empirical results. But there are a couple of issues.

final_score: rating: 3: Clear rejection
review_content: Overall:
I had a really hard time reading this paper because I found the writing to be quite confusing. For this reason I cannot recommend publication as I am not sure how to evaluate the paper’s contribution. 

Summary
The authors study state space models in the unsupervised learning case. We have a set of observed variables Y, we posit a latent set of variables X, the mapping from the latent to the observed variables has a parametric form and we have a prior over the parameters. We want to infer a posterior density given some data.

The authors propose an algorithm which uses sequential Monte Carlo + autoencoders. They use a REINFORCE-like algorithm to differentiate through the Monte Carlo. The contribution of this paper is to add to this a method which uses 2 different ELBOs for updating different sets of parameters.

The authors show the AESMC works better than importance weighted autoencoders and the double ELBO method works even better in some experiments. 

The proposed algorithm seems novel, but I do not understand a few points which make it hard to judge the contribution. Note that here I am assuming full technical correctness of the paper (and still cannot recommend acceptance).

Is the proposed contribution of this paper just to add the double ELBO or does it also include the AESMC (that is, should this paper subsume the anonymized pre-print mentioned in the intro)? This was very unclear to me.

The introduction/experiments section of the paper is not well motivated. What is the problem the authors are trying to solve with AESMC (over existing methods)? Is it scalability? Is it purely to improve likelihood of the fitted model (see my questions on the experiments in the next section)? 

The experiments feel lacking. There is only one experiment comparing the gains from AESMC, ALT to a simpler (?) method of IWAE. We see that they do better but the magnitude of the improvement is not obvious (should I be looking at the ELBO scores as the sole judge? Does AESMC give a better generative model?). The authors discuss the advantages of SMC and say that is scales better than other methods, it would be good to show this as an experimental result if indeed the quality of the learned representations is comparable.
review_title: Auto-encoding Sequential Monte Carlo

final_score: rating: 7: Good paper, accept
review_content: [After author feedback]
I think the approach is interesting and warrants publication. However, I think some of the counter-intuitive claims on the proposal learning are overly strong, and not supported well enough by the experiments. In the paper the authors also need to describe the differences between their work and the concurrent work of Maddison et al. and Naesseth et al. 

[Original review]
The authors propose auto-encoding sequential Monte Carlo (SMC), extending the VAE framework to a new Monte Carlo objective based on SMC. The authors show that this can be interpreted as standard variational inference on an extended space, and that the true posterior can only be obtained if we can target the true posterior marginals at each step of the SMC procedure. The authors argue that using different number of particles for learning the proposal parameters versus the model parameters can be beneficial.

The approach is interesting and the paper is well-written, however, I have some comments and questions:

- It seems clear that the AESMC bound does not in general optimize for q(x|y) to be close to p(x|y), except in the IWAE special case. This seems to mean that we should not expect for q -> p when K increases?
- Figure 1 seems inconclusive and it is a bit difficult to ascertain the claim that is made. If I'm not mistaken K=1 is regular ELBO and not IWAE/AESMC? Have you estimated the probability for positive vs. negative gradient values for  K=10? To me it looks like the probability of it being larger than zero is something like 2/3. K>10 is difficult to see from this plot alone.
- Is there a typo in the bound given by eq. (17)? Seems like there are two identical terms. Also I'm not sure about the first equality in this equatiion, is I^2 = 0 or is there a typo?
- The discussion in section 4.1 and results in the experimental section 5.2 seem a bit counter-intuitive, especially learning the proposals for SMC using IS. Have you tried this for high-dimensional models as well? Because IS suffers from collapse even in the time dimension I would expect the optimal proposal parameters learnt from a IWAE-type objective will collapse to something close to the the standard ELBO. For example have you tried learning proposals for the LG-SSM in Section 5.1 using the IS objective as proposed in 4.1? Might this be a typo in 4.1? You still propose to learn the proposal parameters using SMC but with lower number of particles? I suspect this lower number of particles might be model-dependent.

Minor comments:
- Section 1, first paragraph, last sentence, "that" -> "than"?
- Section 3.2, "... using which..." formulation in two places in the firsth and second paragraph was a bit confusing
- Page 7, second line, just "IS"?
- Perhaps you can clarify the last sentence in the second paragraph of Section 5.1 about computational graph not influencing gradient updates?
- Section 5.2, stochastic variational inference Hoffman et al. (2013) uses natural gradients and exact variational solution for local latents so I don't think K=1 reduces to this?
review_title: Interesting extension of IWAEs using SMC

Meta-review: This work develops importance weighted autoencoder-like training but with sequential Monte Carlo.  The paper is interesting, well written and the methods are very timely (there are two highly related concurrent papers -  Naesseth et al. and Maddison et al.).  Initially, the reviewers shared concerns about the technical details of the paper, but the authors appear to addressed those and two reviews have been raised accordingly.   There is one outlier review (two 7s and one 3).  The 3 is the least thorough and has the lowest confidence (2) so that review is being weighted accordingly.

This appears to be a timely and interesting paper that is interesting to the community and warrants publication at ICLR.

Pros:
- Well written and clear
- An interesting approach
- Neat technical innovations
- Generative deep models are of great interest to the community (e.g. Variational Autoencoders)

Cons:
- Could include a better treatment of recent related literature
- Leaves a variety of open questions about specific details (i.e. from the reviews)
Decision: Accept (Poster)