Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper describes an SMC algorithm to sample the posterior distribution of latent states $p_\theta(z_{1:T}|x_{1:T})$ in a latent variable models $p_\theta(x_{1:T}, z_{1:T})$. The authors consider a completely general setting (the authors assume Eq.(1) but clearly there is nothing to assume here, this the standard Bayes rule). It is well known that the vanilla SMC sampler is a good candidate for ELBO because it provides an unbiased estimator of the likelihood. But the authors prefer here to use a more sophisticated version of the  SMC algorithm, which features a partial rejection algorithm, which amounts to eliminate proposed particles which are "large enough" likelihood.
It is difficult for an expert in SMC algorithms to understand the algorithm as it is described (one must even guess the meaning of the notations). Equation 3) is rather misleading because it is not understood that one continues the acceptance/rejection sampling step until the sample is accepted [Peters' paper, a little less ambitious in its generality, is much more readable]. Also, the form of the rejection probability does not help to understand the action beyond the scene. The level of generality here is a killer: what the hyperparameter $M(i,t-1)$ means, what is the meaning of this form of rejection probability (we see something like a Barker ratio), this is very puzzling
The rejection probability modifies the mutation kernel and should be taken into account when computing the importance weights (Eqs. (4) and (5)). This implies to estimate a quantity $\alpha_t(z_{1:t}^i)$ which is not tractable. The authors suggest (similar to Peters (2012)) to use a Monte Carlo estimator of these quantities. 
To sample the ancestors variables from Eq. (7) with the $\alpha_t(z_{1:t}^i)$ defined in Eq. (6), the authors use the "Dice-Enterprise" algorithm. It does not seem necessary to appeal to such  beautiful algorithm to understand the validity of the algorithm described page 3. Here again, everything is done to frighten the reader and not much is done to explain what is being done... 

The results presented are encouraging and shows that the proposed approach outperforms IWAE and FIVO for a given calculation time. This result is clearly interesting and shows that partial rejection helps despite the additional difficulties linked with the intractability which requires an additional layer of complexity. 

I like the paper even if I have found it extremely unfriendly to read ! 
 
review_title: SMC for VI

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Review update following author discussion

I've read the author responses as well as some of the discussion with the other reviewers. Overall, this is valuable work and I've considered raising my score, but I think a weak accept is appropriate, all things considered.  I've raised the confidence score for my review, as I understand the technical details better now. I think a key strength compared to prior work is the empirical validation of the approach on a variational RNN.  However, the significance of the paper in terms of the novelty of the ideas, both conceptual and technical are overstated in my opinion, hence the weak accept. 

One other point of feedback for the final version in case of an accept:

I also share R3's concern about the paper's positioning in relation to the extremely general dice enterprise framework (or, even the bernoulli factory, for that matter) as somewhat misleading for the particular use case in Section 2.2.  The exact same multinomial sampling scheme is in fact more succinctly presented (proposed?) in BRPF [Schmon et. al, 2019] as the "Bernoulli Race" (which the authors have cited).  I would think that the current scheme is a special case of the "Bernoulli race" that uses a particular form of the acceptance probability parameterization similar to prior work (e.g. VRS [Grover et. al. 2018]). 

See Section 3, specifically e.g. Eq (10) and Proposition 2 from BRPF [Schmon et. al., 2019], which can be compared to Eq (3) and Proposition 1 in the current submission, respectively. 

Minor nit: In step 3 of the algorithm (right below Eq (8)), you use the notation $z_t$ for sampling a new variable from $q_\phi$. However, this $z_t$ has nothing to do with the latent variables that are used in computing the constants $c^i_t$. The way it's written makes it appear as if there's a circular depdendence of the $c_t$ on $z_t$ and then the $z_t$ is again resampled, which changes the $c_t$. For this reason, it maybe better to use a completely unrelated variable for the sample from $q_\phi$ in step 3. 


### Key Strengths

The paper puts together several ideas from prior works (partial rejection control/SMC, variational inference, dice-enterprise) and also evaluates these ideas for latent variable sequential state space model benchmarks. The experimental results compare favorably to prior works like FIVO and IWAE and demonstrate that using partial rejection control is beneficial in a variety of benchmarks. 

### Key Weaknesses

It seems like a key contribution in terms of novelty/theory is in fixing the bias in prior works using Partial Rejection Control in SMC (e.g. Peters et. al. 2012). More information to support this claim in terms of why the prior estimate is biased would be helpful in assessing the strengths of the paper's contributions (Peters et. al don't seem to explicitly focus on the exact bias for PRC).  Having said that, the experiments seem to show that unbiased gradients are worse than a biased version (Figure 2, left bottom), so it seems like focusing on the exact bias is not that important.  


### Additional Comments

* The proof of unbiasedness (Prop 2) says "it is easy to show that Eq (15) is an unbiased estimator" and refers to prior work by [Naesseth et. al] for details. More clarity here would be helpful (especially around the term $Q_{VSMC-PRC}$) for assessing this claim, given that this is one of the main contributions claimed in the paper (besides also commenting on where the bias in prior work is coming from). 

* The $Z()$ function first appears in Equation (9), without a prior reference/definition. You might want to introduce this around Eq (4), where the integral appears. There's also a reference to what will later be $Z$ as $p^i_t$ in Eq (8).

* Using partial forms of rejection to proposal distribution samples specifically for variational inference (rather than SMC more generally) has also been considered in prior work [R. Gummadi, "Resampled Belief Networks for Variational Inference", Advances in Approximate Bayesian Inference Workshop, 2014]. 









review_title: Official Blind Review #3

final_score: rating: 7: Good paper, accept
review_content: Summary:
The submission suggests a new variational bound for sequential latent variable models. Unlike previous work that optimize this bound using ‘standard’ particle filters with unbiased resampling, the new bound is constructed based on a partial rejection control step and uses a dice enterprise for sampling the ancestor variables. 

Positives:
The combination of partial rejection control and dice enterprise for variational inference is new and interesting. Particle filters with partial rejection control have been used before for constructing (biased) bounds based on the marginal likelihood. However, using a dice-enterprise step allows for a new unbiased bound which makes it possible to consider a lower bound on the log-likelihood via variational ideas that can be optimized with standard techniques. Empirical experiments suggest that the method outperforms previous work.

Negatives:
Does the complexity of the new bound not scale linearly with K (while K=1 for FIVO)? This seems to be not accounted for in the experiments. Choosing larger N=16 also has a better performance in the FIVO paper. 

Recommendation:
I vote for acceptance of the paper. However, I think that the experimental section should be improved.

Comments:
Variational bounds can also be constructed by targeting a smoothing distribution (Lawson et al, 2019) and particle filters with complexity N^2 based on a marginal Fisher identity have been suggested (POYIADJIS et al, 2011) for parameter estimation that avoid estimator variances scaling quadratically in time. I was wondering if there is a connection between such filters and the method suggested here, particularly for K=N?
Can you explain the connection between the variance of the estimator for the normalizing constant obtained from particle filters and the tightness of the variational bound in more details?
Are the signal-to-noise gradient issues for large N or K?
How do the methods in the experiments compare for a larger number of particles?
Is there some useful practical advice on choosing the ratio N/K and gamma?


review_title: Official Review 2

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper considers SMC to construct variational approximations. SMC methods
can be improved with partial rejection control (PRC). Then it is not obvious
that one can obtain unbiased estimators of the normalizing constant, as in
plain SMC. The authors consider a way of obtaining unbiased estimators in that
setting.  Experiments include a linear Gaussian state-space model and recurrent
neural networks on polyphonic music datasets.

---

The problem is interesting. The notation is a bit cumbersome at times, but it is 
the case for most papers on SMC. The writing is mostly clear. The experiments
include a mix of toy and more realistic examples. 

The fact that SMC with PRC can still produce unbiased estimators of the
marginal likelihood was described by Kudlicka, Murray, Schoen, Lindsten,
"Particle filter with rejection control and unbiased estimator of the marginal
likelihood". The authors do cite that paper, but I did not understand exactly
why that paper does not completely solve the problem that the authors consider.
At first glance it seems that the Kudlicka et al paper would apply "out of the
box" when replacing the prior by an arbitrary proposal in SMC, i.e. using a
proposal "$q_t$" instead of "$f_t$" and the ratio "$g_t f_t/q_t$" for the weights.
My understanding is that Kudlicka et al focus on the bootstrap particle filter
because PRC is a remarkably generic approach to improve it, applicable even
when sampling from the model prior is the only option; not because the same
reasoning would not apply for generic proposals. 

Thus it was not clear to me that there's a need for another paper showing that
SMC with PRC can still provide unbiased estimators of the marginal likelihood.
If the authors made a convincing case that the extension to general proposals
within SMC with PRC requires significant work, their contribution would be more
convincing.

Furthermore, the manuscript makes references to Bernoulli factories and dice
enterprise.  In fact, the manuscript addresses the problem of categorical
sampling with unbiased estimators of the underlying probabilities.  The problem
was addressed in "Bernoulli Race Particle Filters" by Schmon, Doucet,
Deligiannidis.  That paper is cited by the authors, but the authors do not make
it clear that their algorithm (in Section 2.2) is exactly the same as
"Algorithm 2" of that paper, their Proposition 1 is exactly "equation (12)" of
that paper, etc.  Furthermore, I don't think their algorithm is indeed a "dice
enterprise" as in the terminology of Morina et al; I believe the references to
the Bernoulli factory literature would be sending most readers in the wrong
direction.

Based on these flaws I do not think that the manuscript is suitable for
publication. Perhaps a clearer explanation of the specific shortcomings of
Kudlicka et al's work would make the paper more convincing.

---
Small comments:

- page 2: "We further assume that the joint density [...]"
in fact that decomposition always holds, it is not an assumption.
It is just p(a,b) = p(a)p(b|a). 

- page 2: A SMC sampler -> An SMC sampler

- page 2 and later: there is some inconsistency between the notation M(i,t-1) and M(t-1,i).

- The latex command \eqref seems to have been used instead of \ref, in various places.

- page 6 "utlizing the best of both worlds"

review_title: A fine paper on Sequential Monte Carlo with partial rejection control, but the question might have been tackled in the literature already 

Meta-review: This paper explores the use of partial rejection control (PRC) for improved SMC-based variational bounds. While an unbiased SMC variant with PRC has been previously introduced by Kudlicka et al. (2020), this work introduces innovations that can help apply such ideas to variational inference. These bounds result in improvements in empirical performance. 

This paper was heavily discussed, with significant engagement by both the authors and the reviewers. Most reviewers recommended acceptance of this paper, with one reviewer (R4) recommending against acceptance. R4's central concerns regard the novelty of the proposed approach and its positioning relative to the existing SMC literature. The authors argued vigorously in the comments that this paper should be judged as a contribution to the VI literature and not the SMC literature.  Unfortunately, I will recommend that this paper is rejected. It is my opinion that R4's concerns were not fully addressed.

On the one hand, I agree with the authors that there is significant value to be had in exploring variants of SMC for VI. Indeed, some prior art, like FIVO and IWAE, contributed little to the Monte Carlo literature. I believe that these were good contributions.

On the other hand, I am concerned that the current draft does not clearly circumscribe its contributions. I read the sections that disuss the works of Schmon et al. (2019) and Kudlicka et al. (2020), and the writing did not leave me with a clear enough sense of the differences. I also read the abstract and introduction of the paper. The introduction of the paper positions this work clearly within the VI literature, but does not clearly discuss prior SMC art, e.g., it does not cite Kudlicka et al. (2020). Despite citing rejection control for SMC, the writing of the abstract and introduction left me with the impression that this work was the first to introduce *unbiased, partial* rejection control for SMC. I believe that impressions matter and that the machine learning community should be generous to adjacent communities when assigning credit.

I realize that my decision is a matter of taste. I also want to say that I am confident that the authors have a clear sense of where their contribution sits, and I suspect that it is a valuable contribution. However, I cannot recommend the draft in its current form. If this is a contribution to the VI literature, as the authors argue, then the authors should not hesitate to give full credit to prior SMC art. My reading of the current draft still leaves me confused about which aspects of the SMC estimator are actual contributions.
Decision: Reject