Partial Rejection Control for Robust Variational Inference in Sequential Latent Variable Models
Authors: Rahul Sharma
Soumya Banerjee
Dootika Vats
Piyush Rai
Venue:  (2021)
Keywords: dice enterprise
partial rejection control
sequential Monte-Carlo
Bernoulli factory
variational Inference
Rejection Sampling

Abstract: Effective variational inference crucially depends on a flexible variational family of distributions. Recent work has explored sequential Monte-Carlo (SMC) methods to construct variational distributions, which can, in principle, approximate the target posterior arbitrarily well, which is especially appealing for models with inherent sequential structure. However, SMC, which represents the posterior using a weighted set of particles, often suffers from particle weight degeneracy, leading to a large variance of the resulting estimators. To address this issue, we present a novel approach that leverages the idea of \emph{partial} rejection control (PRC) for developing a robust variational inference (VI) framework. In addition to developing a superior VI bound, we propose a novel marginal likelihood estimator constructed via a dice-enterprise: a generalization of the Bernoulli factory to construct unbiased estimators for SMC-PRC. The resulting variational lower bound can be optimized efficiently with respect to the variational parameters and generalizes several existing approaches in the VI literature into a single framework. We show theoretical properties of the lower bound and report experiments on various sequential models, such as the Gaussian state-space model and variational RNN, on which our approach outperforms existing methods.

Under review as a conference paper at ICLR 2021
PARTIAL REJECTION CONTROL FOR ROBUST VARIA- TIONAL INFERENCE IN SEQUENTIAL LATENT VARI- ABLE MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Effective variational inference crucially depends on a flexible variational family of distributions. Recent work has explored sequential Monte-Carlo (SMC) methods to construct variational distributions, which can, in principle, approximate the target posterior arbitrarily well, which is especially appealing for models with inherent se- quential structure. However, SMC, which represents the posterior using a weighted set of particles, often suffers from particle weight degeneracy, leading to large variance the resulting estimators. To address this issue, we present a novel approach that leverages the idea of partial rejection control (PRC) for developing robust variational inference (VI) framework. In addition to developing a superior VI bound, we propose novel marginal likelihood estimator constructed via a dice-enterprise: a generalization of the Bernoulli factory to construct unbiased estimators for SMC-PRC. The resulting variational lower bound can be optimized efficiently with respect to the variational parameters and generalizes several exist- ing approaches in the literature into a single framework. We show theoretical properties of the lower bound and report experiments on various sequential mod- els, such as the Gaussian state-space model and variational RNN, on which our approach outperforms existing methods.
INTRODUCTION
Exact inference in latent variable models usually intractable. Markov Chain Monte-Carlo (MCMC) (Andrieu et al.] 2003) and variational inference (VI) methods (Blei et al. 2017). are commonly employed in such models to make inference tractable. While MCMC has been the tradi- tional method of choice, often with provable guarantees, optimization-basel VI methods have also enjoyed considerable recent interest due to their excellent scalability on large-scale datasets. VI is based maximizing lower bound constructed through a marginal likelihood estimator. For latent variable models with sequential structure, sequential Monte-Carlo (SMC) (Doucet & Johansen] (0099) returns much lower variance estimator of the log marginal likelihood than importance sampling (Bérard et al. 2014] Cérou et al., 201I). In this work, we focus our attention on designing a low variance, unbiased, and computationally efficient estimator of the marginal likelihood.
The performance of SMC based methods is strongly dependent on the choice of the proposal distribution. Inadequate proposal distributions propose values in low probability areas under the target, leading to particle depletion (Doucet & Johansen 2009). An effective solution is to use rejection control (Liu al., 1998 Peters et al.] 2012) which is based on an approximate rejection sampling step within SMC to reject samples with low importance weights.
In this work, we leverage the idea of partial rejection control (PRC) within the framework SMC based VI for sequential latent variable models. To this end, we construct novel lower bound, VSMC-PRC, and propose an efficient optimization strategy for selecting the variational parameters. Compared to other recent SMC based VI approaches (Naesseth al. 2017 Maddison al. 2017
Le et al. 2017). our approach consists an inbuilt accept-reject mechanism within SMC to prevent particle depletion. The use of accept-reject within SMC makes the particle weight intractable, therefore, we use a generalization of the Bernoulli factory (Asmussen et al., 1992) to construct unbiased estimators of the marginal likelihood for SMC-PRC.

---

Under review as a conference paper at ICLR 2021
Although the idea of combining VI with an inbuilt accept-reject mechanism is not new (Salimans et al. 20155] Ruiz Titsias 2019; Grover et al., 2018 Gummadi 2014), a key distinction of our approach to incorporate
accept-reject mechanism along with a resampling framework. In contrast to standard sampling algorithms that may reject the entire stream of particles, we use a partial accept-reject on the most recent update, increasing the sampling efficiency. Further, the variational framework of SMC-PRC is interesting in itself as it combines accept-reject with particle filter methods. Therefore, our proposed bound VSMC-PRC generalizes several existing approaches for example: Variational Rejection Sampling (VRS) (Grover et al. 2018). FIVO (Maddison et al. 2014). IWAE (Burda et al.) 2015). and standard variational Bayes (Blei et al. 2017].
Another key distinction is that, while existing approaches using Bernoulli factory are limited to niche one-dimensional toy examples, our proposed approach scalable. To the best of our knowledge, there is no prior work that has used Bernoulli factories for such general case like variational recurrent neural networks (VRNN); therefore, we believe this aspect be a significant contribution as well. we provide brief review on SMC, partial rejection control, and dice enterprise. In Section 3 we introduce our VSMC-PRC bound and provide new theoretical insights into the Monte-Carke estimator and design efficient ways to optimize it. Finally, we discuss related work and present experiments on the Gaussian state-space model (SSM) and VRNN.
The rest of the paper is organized as follows: In Section 2
BACKGROUND
We denote sequence of I' real-valued observations 21.7 (x1, 22 FT ) and assume that there is an associated sequence of latent variables 21.7 = (21, . 2T). We are interested in inferring the posterior distribution of the latent variables, i.c. p(2ir1( The task is, in general, intractable. For the rest of the paper we have used some common notations from SMC and VI literature where zę: 1th particle at time A ancestor variable for the 7th particle at time 8 and de model and variational parameters, respectively.
2.1 SEQUENTIAL MONTE CARLO WITH PARTIAL REJECTION CONTROL
An SMC sampler approximates a sequence of densities {po (x1.)) ) through a set of N weighted samples generated from proposal distribution. Let the proposal density be
T
(
II go go (2 21.0-1).
(1)
t=1
Consider time t-1
1 at which we have uniformly weighted samples {N -; 1 estimating
P 1x1 We want to estimate p0(21:cl (1a)) such that particles with a low importance weight are automatically rejected. PRC achieves this by using an approximate rejection sampling step (Liu
et al. 1998 Peters al. 2012). The overall procedure is follows:
1. Generate z| qs (2 A-=1 where i 1,2....,
2. Accept af with probability
M(it - 1)4s (of F1 Ai-1)
Ah-1 a0.0 (2.0(2/1:111111 x1.e)
1
(2)
4 po 2/| 2; 211-11)
where M(i.
- 1) is a hyperparameter controlling the acceptance rate (see Proposition
and Section 3.3 for more details). Note that PRC applies accept-reject only on 2 not on the entire trajectory.
3. If 2 rejected go to step 1.
4. The new incremental importance weight of the accepted sample is
av((1)a) = cc (2 :1tt),
Ai
(3)
where of is
1/
po Polwe, z 11 -1 -1 Al.
(4)
go (2f) wiee, 21:0-1 )1000) /a
( x1:t)
2

---

Under review as a conference paper at ICLR 2021
and the intractable normalization constant Z() (For simplicity of notation, have ignored the dependence of (.) on Mii (
Z 4 1 x1:1) f 00,0 (2 F1a- 3i (o 11 21t=1)
A
(5)
5. Compute Monte-Carke estimator of unnormalized weight
Ai-1 1 Ex= a (6 A T1:)) pa ( 2 -1:1-11 21--1)
(6)
A ( F1.)
1. 2 a0
where 6 ~ g (6 Ai-)) and k = 1.,, K. Note that /j would be essential for constructing unbiased estimator for pe (#1:7).
6. Generate ancestor variables A through dice-enterprise and weights w/ N-1 1 for
= 1,,....,,N:
an(3)t) Sj=1 (=(a)) Σj= ar (z
an ax
A Categorical
Σラ ar an
2.2 DICE ENTERPRISE
Simulation of ancestor variables in Eq. is non-trivial due to intractable normalization constants in the incremental importance weight (see Eq. 3p. Vanilla Monte Carlo estimation of ou yields biased samples of ancestor variables from Eq. 7 To address this issue, leverage generalization of Bernoulli factory, called dice-enterprise (Morina et al. 2019). Note that multinoulli extensions of Bernoulli factory (Dughmi al. 2017) have also been used for resampling within intractable SMC before (Schmon et al., 2019). a key distinction of our approach is to design a scalable Bernoulli factory methodology especially useful for VI applications.
Suppose we can simulate Bernoull(pp) outcomes where pi is intractable. Bernoulli factory problem simulates an event with probability f'()), where f(.) is some desired function. In our case, the intractable coin probability pl is the intractable normalization constant,
p Z Fia))
00. (2.2
1i-1 I (2. aie 1
Since pl E [0, 1] and can easily simulate this coin, obtain the dice-enterprise algorithm below.
1. Required: Constants {cf see Eq.
2. Sample C' Categorical
Ej-1 ΣN
NN
1/-11
3. If C i, generate U, ~ 10, 1] and as
(20
21--11
Af If U, < ag (2, = 11d) output
Else go to step 2
The dice-enterprise produces unbiased ancestor variables. Note that we can easily control the efficiency of the proposed dice-enterprise through the byper-parameter M (similar to Eq. (2) in contrast to existing Bernoulli factory algorithms (Schmon et al. 2019). For details on efficiency and correctness, please refer Section 3.1 and Section
Our proposed VSMC-PRC bound is constructed through marginal likelihood estimator obtained by combining the SMC sampler with a PRC step and dice-enterprise. The variance of estimators obtained through SMC-PRC particle filter is usually low (Peters et al., 2012). Therefore, we expect VSMC-PRC to be a tighter bound compared to the standard SMC based bounds used in recent works (Maddison et al. [201]] Naesseth et al.] 2017: [e et al. 2017). Algorithm ( the generative process to simulate the VSMC-PRC bound. Please see Figure | to visualize the generative process for VSMC-PRC.
3

---

Under review as a conference paper at ICLR 2021
Algorithm 1 Estimating the VSMC-PRC lower bound
1: Required: N, K, and M
17: end while
2: for 21.,..... T}
1 18: Sample {&t fk=1 qo go(2) 2ids 11=11
3: for
2..,...,NNN PRC (g. M(i, 1)) 1))
N do
19: Calculate wi from Eq.
4: =i, w]
20: Calculate oj from Eq.
Ai-1 (21:0-1)
5:
#1:1
21: return (2 Gj w))
6:
end for
22:
7:
for i {1.2..... ] do
23: DICE-ENT (fc: 2
8:
A DICE-ENT ((c), +
N
24: Sample C' Multinoulli
9: end for
10: end for
25: C== i then
i=1
return log IT (1 E w7
26: Sample U, ~ U10,11
11:
27: 2 =~00(=0(011, =(1==1
12:
13: PRC (g,P, ((it-
28: end
while sample not accepted do
29: if U, 00.0
Ai 21:-1, #1:) then
Generate 2f ( (21. 21--11 Ai-1)
30: return (i)
31: else
Accept 2 with probability Ai-1 00.0 (2f1=1/111-- 11 214)
32: return DICE-ENT ((c) =1.))
33: end if
3 PARTIAL REJECTION CONTROL BASED VI FOR SEQUENTIAL LATENT VARIABLE MODELS
We now show how to leverage PRC develop robust VI framework for sequential latent variable models. Our framework is based on the VSMC-PRC bound presented below. The complete sampling distribution of Algorithm is follows.
K
N
T IIII
QVSMC-PRC
I:N 21T
A I:N
1:7-1 1:T
(.N.IK)
IIII
10
(iih (x1)
II
go go(d
1= 1= 1=1
N
T-1 N 4o ((11))))))
90 (o(it41 11 21 A
(2841121111) 4: 21:441)
Jag
I Discrete(A((aa)
2(I1)
Z(2 x1:41)
The normalization constants 7-) in Eq. are intractable and have be estimated while calculating the weights. Therefore, we introduce an extra parameter K, denoting the number of Monte-Carlo samples used estimate 2()) The Monte-Cario estimator of VSMC-PRC bound is
LVSMC.PRC o; FIT, K)
Σ
log
Σ 1
(10)
We maximize the VSMC-PRC bound with respect to model parameters & and variational parameters 6. This requires estimating the gradient the details of which are provided Section 3.2
3.1 THEORETICAL PROPERTIES
We now present properties of the Monte-Carlo estimator LVSMC-PRC. The key variables that affect this bound are N (number of samples), byper-parameter M, and the number of Monte-Carlo samples used to compute the normalization constant Z()), i.e., K. As discussed by Bérard et al. (2014); Naesseth et al. (017). as increases, we expect the VSMC-PRC bound get tighter. Hence, we will focus our attention M and K. All the proof's can be found in the appendix.
Proposition Dice-enterprise produces unbiased ancestor variables. Further, let A, be the number of iterations required for generating one ancestor variable, then A₁ ~ Geom [E[A,]]1) where
ΣN
[[A]
ΣN ct 21,1-
4

---

Under review as a conference paper at ICLR 2021
(6) VSMC
(b) IWAE
(6) FIVO
Figure Comparison of VSMC-PRC with IWAE (Burda al. 2015 and FIVO (Maddison
2017 (a)
The blue arrows represent the resampling step, We then generate multiple samples from parametrized proposal =]|=i==1 out of which one sample is accepted via PRC, depicted via green arrows. (b) In IWAE, there is no resampling step and no PRC step (c). In FIVO, there is resampling step (blue arrows) but no PRC step.
As evident from Proposition I the computational efficiency of the dice-enterprise clearly relies on the normalization constant (.). Note that the value of (.)) could be interpreted as the average acceptance rate of PRC which depends on the hyper-parameter (i, & 1). If the average acceptance rate for PRC for all particles y, then we can express the expected number of iterations as E[A[] y -- Therefore, the computational efficiency of dice-enterprise is similar to the PRC step and depends crucially on the hyper-ppaaae paper-parameter M.
Proposition 2. For all K, exp( (vossc-rre) is unbiased, i.e., E (xp((VsMccrrc)) po(x1:)) Fur-
ther, E EVCssMC-RRcl is non-decreasing in K.
The use of Monte- Carlo estimator in place of the true value of Z() creates an inefficiency, depicted by Proposition[] The bound monotonically increases as we increase K despite the use of resampling operation. It important note that Algorithm produces unbiased estimator of the marginal likelihood for all values of K.
Proposition 3. Let the sampling distribution of the th particle (generated via PRC) time t. be ro
11: then
Ai-1 xia) (1 T8, (0,0 2 x1x) po x1xz)) s KL (0(22))
Al-1
Ai-1
Al-1 six))
KL
Proposition 3 implies that the use of the accept-reject mechanism within SMC refines the sampling distribution. Instead of accepting all samples, the PRC step ensures that only high-quality samples are accepted, leading to a tighter bound for VSMC in general (not always). We show in the appendix that when M(i, 1) - oc, the PRC step reduces to pure rejection sampling (Robert & Casella 2013). On the other hand, M(i,t - 1) >> implies that all samples are accepted from the proposal. Recall, M (i, - 1) a hyperparameter that can be tuned to control the acceptance rate. For more details on tuning M, see Section 3.3
3.2 GRADIENT ESTIMATION
For tuning the variational parameters, we use stochastic optimization. Algorithm | produces the marginal likelihood estimator by sequentially sampling the particles, ancestor variables, and particles for the normalization constant (>1:N A - 0
11:N.1:K
When the variational distribution (.).) is reparameterizable, we can make the sampling of 8tth independent of the model and variational parameters. However, the generated particles 2 are not reparametrizable due the PRC step. Finally, the ancestor variables discrete and, therefore, cannot be reparameterized. The complete gradient can be divided into three core components (assuming (.). ) is reparametrizable):
Vo, Ve LVSMcc-PRc
EQQ
Vo, Lvs d: 21,r K)
+ gPRC + GRSAMP (11)
2
E Exyawwc
V Vo.oLvsMc.PRC 0; K)
(12)
5

---

Under review as a conference paper at ICLR 2021
Note that grnc and @RSAMP denote the score gradient of PRC and resampling step, respectively. Due to high variance, we have ignored these terms for the optimization. We have derived the full gradient and explored the gradient variance issues in the appendix. Please see Figure (left) comparing the convergence of biased gradient vs. unbiased gradients on toy Gaussian SSM.
3.3 LEARNING THE M MATRIX
We use M as a hyperparameter for the PRC step which controls the acceptance rate of the sampler. The basic scheme of tuning M is as follows:
4 Define a new random variable F(2(41/210 (1++1) = log golt-1l
Draw 2/41 ~ q0(2e+1 [1++11 21:1) for = --1,, -,,,,.....
Evaluate y E 1] quantile value of (F(z/_11z, F1:++1) } In general for this case the acceptance rate would be around y for all particles.
log M(, - ((seil==d (1suil=11 (7).
(13)
If M matrix is very large then use common (M (,,t for every time-step. In general, for this configuration, the acceptance rate would be greater than equal to y for all particles:
log (..t) min {
N
(14)
F(zu41 (1)))))))
i=1
Through y: a user parameter, we can directly control the acceptance rate. Therefore, both dice- enterprise and PRC would take around (less than) y iterations to produce a sample for M value [4p. For implementation details please refer to the experiments.
learned from Eq. 13 (see Eq.
Note that a similar scheme was also employed in Grover et al. 2018 ). We update {i, 1) 12' [e] dynamically once every / epochs to save time, To learn more on setting hyper-parameter
M. see Liu et al. (1998); Peters et al. (2012).
RELATED WORK AND SPECIAL CASES
There is significant recent interest developing more expressive variational posteriors for latent variable models. There are two basic schemes for constructing tighter bounds on the log marginal likelihood: sampling-bar based methods (MCMC, rejection sampling) (Salimans et al.) 2015 Ruiz & Titsias 2019: Hoffman, 2017 Grover et al. 2018) multiple samples from VI distributions to increase the flexibility (IS, SMC) (Burda et al. 2013 Maddison et al., 2017] Lawson et al., 2018 Naesseth et al. 2015). In this work, present unified framework for combining these two approaches, utilizing the best of both worlds. Although applying sampling bampling-based methods on VI useful, the density ratio between the true posterior and the improved density is often intractable. Therefore, we cannot take advantage of variance-reducing schemes like resampling, which is crucial for sequential models. We solve this issue through dice-enterprise: extension of the Bernoulli factory.
Recently, Bernoulli factory has amassed a great interest in the area of Bayesian inference (Gonçalves) et al. 2017a Vats et al. 2020). Although Bernoulli factory is theoretically valuable, its applicability severely limited due a high rejection rate. In this paper, we have presented an approach that combines SMC with dice-enterprise for efficient implementation. A closely related work from SMC literature Schmon al. 2019)) which also utilizies Bernoulli factories to implement unbiased resampling. However, their method is not scalable and designed particularly for partially observed diffusions. Another relevant work for unbiased estimation of the marginal likelihood is that of Kudlicka al. (2020). In contrast to our approach, this method samples one additional particle and keeps track of the number of steps required by PRC for every time-step to obtain their unbiased estimator. The weights are tractable for Kudlicka et al. (2020) they do not take into account the effect of the normalization constant Z (.). On the other hand, we consider the effect of 7.) the particle's weight, making resampling operation infeasible. To fix this intractability,

---

Under review as a conference paper at ICLR 2021
1: 10,
Case d, 3, dense)
1og
VSMC
y=
7= 0.4
Case I
-18.27
-25.78
-24.80
-21.91
Case 2
-84.33
-187.5
Case 3
-33.89
-159.96-108.4
-86.36
log Pelk
log nelkg.rl vemc-prc vemc
Case 4
-443.73
-538.33
-531.89
-515.10
vsmc-prc vemkc
herations/500)
kerations/500)
05 1, dense)
Case (d. 25,d,= Saparse)
gradient hull gradient
log pelos.rl
log pelky.rl
vsmc-pre
vsmc-pre
esrric
veme
veretions(300)
kerations1500)
kerations1500)
Figure 2: (Leeft) The figures compares the bound value for VSMC-PRC with full gradient and biased gradi- ent (equation 12 a function of iterations. (Left) The Table compares the bound value for VSMC (Naesseth) al. 2017) and VSMC-PRC for 80% and 40% acceptance rate. (Right) We compare VSMC, VSMC-PRC (40% acceptance rate), and log po (21:7) as a function of iterations.
use dice-enterprise. Comparisons of our marginal likelihood estimator versus that of Kudlicka et al. (2020) would make for interesting future work.
To provide more clarity, we will consider some special cases of VSMC-PRC bound and relate it with existing work: Note that for N 1 our method reduces to special case of Gummaaii (2014) which uses constraint function C2 for every time-step and restarts the particle trajectory from A, (if C, is violated). Therefore, if use the setting C2 (Z1:)) = a( 121:t-1 (1:{) and A, / 1, our method reduces to a specific case of Gummadii (2014). For the special case of N = 1 and T = 1, our method reduces VRS (Grover al.88) For N.T T > 1, if we remove the PRC step, our bound reduces to FIVO (Maddison et al. 2017). Finally, if we remove both the PRC step and resampling, then our method effectively reduces to IWAE (Burda et al. 2015). Please refer to Figure [or more details.
EXPERIMENTS
In this section, we evaluate our proposed algorithm on synthetic as well as real-world datasets and compare them with relevant baselines. For the synthetic data experiment, we implement our method on a Gaussian SSM and compare our approach with VSMC (Naesseth et al. (2017). For the real data experiment, we train VRNN (Chung et al. 2013) on the polyphonic music dataset.
5.1 GAUSSIAN STATE SPACE MODEL.
In this experiment, we study the linear Gaussian state space model. Consider the model
21 A Cz.
2 = C2 + Ex:
where er ~ NO, 7) and 20 = O. We are interested in learning good proposal for the above model. The latent variable is denoted by 2, and the observed data by zt. the dimension of 2, be d, and dimension of x, be d,. The matrix A has the elements (A) l for 0.42. We explore different settings of dz, dz, and matrix C. A sparse version of C' matrix measures the first d. components of 2,, on the other hand a dense version of C is normally distributed i. Ct,y NN(0, 1). We consider four different configurations for the experiment. For more details please refer to Figure 2
The variational distribution is multivariate Gaussian with unknown mean vector μ {palde and diagonal covariance matrix (log 21 We set N and 7 10 for all the cases:
(==/==1) N (zx) Az-1 1 + nt, diag(o"))
7

---

Under review as a conference paper at ICLR 2021
The - matrix (see Eq. 13) for approximate rejection sampling is updated once every 10 epochs with acceptance rate y E {0.8. 0.4]. For estimating the intractable normalization constants, generate K = 3 samples. Figure 2 (left) compares the convergence of biased gradient vs unbiased gradients. Note that we get a much tighter bound as compared to VSMC (Naesseth et al.
2017).
5.2 VARIATIONAL. RNN
VRNN [Chung et al. 2015) comprises of three core components: the observation stochastic latent state 2t, and a deterministic hidden state 1x(2+-1 -1- Ft- 14-1), which modeled through RNN. For the experiments, we use single-layer LSTM for modeling the hidden state. The conditional distributions ( and qe (zxl.) are assumed to factorized Gaussians, parametrized by a single layer neural net. The output distribution (Ix l.) depends on the dataset. For fair comparison, we use the same model setting as employed in FIVO (Maddison et al. 2017). We evaluate our model four polyphonic music datasets: Not nottingtt nothingham, JSB chorales, Musedata, and Piano-midi de.
Each observation z; is represented observation distribution 9: (xx(-) by set of 88 factorized Bernoulli variables. We split all four data- sets into the standard train, validation, and test sets. For tuning the learning rate, test set. For a fair comparison, we use the same learning rate and iterations for all the models. Let the dimension of hidden state (learned by single layer LSTM) be da and dimension of latent variable be d,. We choose the setting d, dn = 64 for all the data-sets except JSB. For modeling JSB, we use d d,, 32. For VSMC-PRC we have considered N E {4, 6) Further, for each N, we consider 0.9), (1, 0.8). (3, 0.9), (3, 0.8) J. The M syper-parameter for PRC step is
binary vector of 88 dimensions. Therefore, model the
the validation
four settings (K, 7) learned from Eq. due to large size. We have updated M value once every 50 epoches. Note that in this scenario, the acceptance rate for all particles would be greater than equal to y. For more details on experiments, please refer to the appendix.
As discussed in Section 3.1 the PRC step and dice-enterprise have time complexity ((N/7) for producing samples (assuming average acceptance rate 7). Therefore, we consider [N particles for IWAE and FIVO ensure effectively the same number of particles, where N {4. 6) and y = 0.8. Note, however, that the acceptance rate is 2 y, so this adjustment actually favors the other approaches more. For FIVO, we perform resampling when ESS falls below N/2. Table 1 summarizes the results which show whether rejecting samples provide us with any benefit or not, and as the results show, our approach, even with the aforementioned adjustment, outperforms the other approaches in terms of test log- boglliilsssoods, while still having a similar computational cost.
Table 1: We report Test log-likelihood for models trained with FIVO, IWAF, ELBO, and VSMC-PRC. For VSMC-PRC N = (4, 6) and (K, 7) E (.1,. ()),),.))) (1, 0.8), (3, 0.9), (3, 0.8)) (results are in this order). The results for pianoroll data-sets in per timestep.
N
Data
ELBO
IWAE
FIVO
N
VSMC-PRC
Nott
-3.87
-3.12
-3.07
-2.96
-2.98
-2.99
-2.96
jsb
-8.69
-8.01
-7.51
-7.41
-7.28
-7.37
-7.36
Piano
-7.99
-7.97
-7.85
4
-7.82
-7.86
-7.80
-7.85
Muse
-7.48
-7.45
-6.75
-6.61
-6.63
-6.66
-6.58
N
Data
ELBO
IWAE
FIVO
N
VSMC-PRC
Nott
-3.87
-3,87
-2.99
-2.93
-2.93
-2.90
-2.91
jsb
-8.69
-8.32
-7.40
-7.29
-7.21
-7.16
-7.14
8 Piano
-7.99
-8.04
-7.80
6
-7.78
-7.77
-7.79
-7.77
Muse -7.48
-7.41
-6.67
-6.60
-6.57
-6.61
-6.60
Avg.
Rank
687 1: 0.33
6.12
0.33
4.87
0.33
2.87 1.05
2.62
1.21
2.87 1:
1.26
1.75 1: 0.66
In Sec. 3.1 discussed the effect of K and PRC rejection rate VSMC-PRC bound. We expect a performance improvement when K and the rejection rate increased. Although the results for almost the same, still get the best average ranking for (K 3. = 0.8). Overall, for most cases, VSMC-PRC bound performs better than FIVO Maddison al. 2017) and IWAE (Burda et al. 2015) for variety of configurations.
VSMC- different configurations
In VSMC-PRC, improvement in the bound value comes at the cost of estimating the normalization constant (-), i.e., K. On further inspection, we clearly see that increasing K doesn't 'itsn't provide us
8

---

Under review as a conference paper at ICLR 2021
with any substantial benefits despite the increase in computational cost. Therefore, to maintain the computational trade-off (K = 1, > 0.8) seems to be a reasonable choice for VI practitioners.
Table 1 signifies that rejecting samples with low importance weight is better instead of keeping large number of particles (at least for a reasonably high acceptance rate 7). The proposed bound uses more partieles (PRC step and dice-enterprise) than existing approaches like FIVO and IWAE due to intractability. Future work aims at designing a scalable implementation for VSMC-PRC bound that consumes fewer particles.
CONCLUSION
We introduced VSMC-PRC, novel bound that combines SMC and partial rejection sampling with VI in synergistic manner. This results in a robust VI procedure for sequential latent variable models. Instead of using standard sampling algorithms, we have employed a partial sampling scheme suitable for high dimensional sequences. Our experimental results clearly demonstrate that VSMC-PRC outperforms existing bounds like IWAE (Burda et al. (015) and standard particle filter bounds (Maddison al., 2017 Naesseth et al.) 2017,] Le al. 2017). The future work aims to explore partial versions of powerful sampling algorithms like Hamiltonian Monte Carlo (Neal et al.)
2011) instead of rejection sampling.
REFERENCES
Christophe Andrieu, Nando De Freitas, Arnaud Doucet, and Michael I Jordan. An introduction to MCMC for machine learning. Machine learning, 50(1-2):5 43: 2003.
Søren Asmussen, Peter W Glynn, and Hermann Thorisson. Stationarity detection in the initial transient problem. ACM Transactions on Modeling and Computer Simulation (TOMACS), 2(2): 130-157, 1992.
Jean Bérard, Pierre Del Moral, Arnaud Doucet, et al. lognormal central limit theorem for particle approximations of normalizing constants. Electronic Journal of Probability, 19, 2014.
David M Blei, Alp Kucukelbir, and Jon D) McAuliffe. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518):859-555 877, 2017.
Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. arXiv preprint arXiv: 1.5.9.0090000 009900059. 2015.
Frédéric Cérou, Pierre Del Moral, and Arnaud Guyader. A nonasymptotic theorem for unnormalized Feynman-K Kac particle models. In Annales de l'IHP Probabilités et statistiques, volume 47, pp. 629, 2011.
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. In Advances in neural information processing systems, pp. 2980-2999 2980,299,2200 2015.
Arnaud Doucet and Adam M Johansen. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of nonlinear filtering. 726656-704):3, 2009.
Shaddin Dughmi, Jason Hartline, Robert Kleinberg, and Rad Niazadeh. Bernoulli factories and black-box reductions in mechanism design. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pp. 158-169, 2017.
Flávio B Gonçalves, Krzysztof fatuszyáski,, Gareth Roberts, et al. Barker's algorithm for Bayesian inference with intractable likelihoods. Brazilian Journal of Probability and Statistics, 31(4): 732-745, 2017a.
Flávio B Gonçalves, Krzysztof G I Latuszyúski, and Gareth O Roberts. Exact Monte Carlo likelihood- based inference for hump-diffusion processes. arXiv preprint 177.0.1707 2017b.
Aditya Grover, Ramki Gummadi, Miguel Lazaro- Gredilla,, Dale Schuurmans, and Stefano Enmon. Variational rejection sampling. arXiv preprint arXiv: 1804.00704 2018.

---

Under review as a conference paper at ICLR 2021
Ramki Gummadi. Resampled belief networks for variational inference. In NIPS Workshop Advances in Variational Inference, 2014.
Matthew D Hoffman. Learning deep latent Gaussian models with Markov chain Monte Carlo. In Proceedings of the 34th International Conference on Machine Learning- Voluum 70, pp. 1510-1519. JMLR. org, 2017.
Jan Kudlicka, Lawrence M Murray, Thomas B Schön, and Fredrik Lindsten. Particle filter with rejection control and unbiased estimator of the marginal likelihood. In ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 5860-5864. IEEE, 2020.
Dieterich Lawson, George Tucker, Christian A Naesseth, Chris Maddison, Ryan P Adams, and Yee Whye Teh. Twisted variational sequential Monte Carlo. In Third workshop Bayesian Deep Learning (NeurIPS), 2018.
Tuan Anh Le, Maximilian Igl. Tom Rainforth, Tom Jin, and Frank Wood. Auto-encoding sequential Monte Carlo. arXiv preprint arXiv: 10306 2017.
Jun Liu, Rong Chen, and Wing Hung Wong. Rejection control and sequential importance sampling. Journal of the American Statistical Association, 93(443):1022-1033 1998.
Chris J Maddison, Daniel Tarlow, and Tom Minka. A sampling. In Advances in Neural Information Processing Systems, pp. 3086-3094, 2014.
Chris J Maddison, John Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Amnad Doucet, and Yee Teh. Filtering variational objectives. In Advances in Neural Information Processing Systems, pp. 6573-6583, 2017.
Giulio Morina, Krzysztof Latuszynski, Piotr Nayar, and Alex Wendland. From the Bernoulli factory to a dice enterprise via perfect sampling Markov chains. arXiv preprint WrXiv:191.009222 2019.
Christian A Naesseth, Fredrik Lindsten, and Thomas B Schön. Nested sequential Monte Carlo methods. arXiv preprint arXiv:1502.02533, 2015.
Christian A Naesseth, Scott W Linderman, Rajesh Ranganath, and David M Blei. sequential Monte Carlo. arXiv preprint arXiv: /1/40, 2017.
Variational
Christian A Naesseth, Fredrik Lindsten, and Thomas B Schön. Elements of sequential Monte Carlo. arXiv preprint arXiv:1903.04797 2019.
Radford M Neal et al. MCMC using Hamiltonian dynamics. Handbook of markov chain monte carlo, 2(1 1):2, 2011.
Gareth W Peters, Yanan Fan, and Scott A Sisson. On sequential Monte Carlo, partial rejection control and approximate Bayesian computation. Statistics and Computing, 22(6):1209-122 2012.
Christian Robert and George Casella. Monte Carlo statistical methods. Springer Science & Business Media, 2013.
Francisco JR Ruiz and Michalis K Titsias. A contrastive divergence for combining variational inference and MCMC. arXiv preprint wrexiv::905.044 2019.
Tim Salimans, Diederik Kingma, and Max Welling. Markov chain Monte Carlo and variational inference: Bridging the gap. In International Conference on Machine Learning, pp. 2015.
Sebastian M Schmon, Amaud Doucet, and George Deligiannidis. Bernoulli race particle filters. arXiv preprint arXiv:1903.0000 2019.
Dootika Vats, Flávio Gonçalves, Krzysztof Latuszyúski, and Gareth O Roberts. Efficient Bernoulli factory MCMC for intractable posteriors. arXiv preprint arX1v:2004.0747,, 2020.
10

---

Under review as a conference paper at ICLR 2021
A PROOF OF THEORETICAL RESULTS
Proof of proposition Dice-enterprise produces unbiased ancestor variables. Let's evaluate the probability that eite-enterprise would output as the ancestor index. Assume that the algorithm terminates after steps where E 21, ................................ x.), then the probability of output i is given follows:
Pr(output- i)
Pr(output = ilafter steps)
r-1
0
N d
Em=1 o
Σm=1
ct
j=1
c (4:-
Σj=1 (Z
easy to see that A, is geometrically distributed with success probability given by A (Z(21a-1.
Pr(getting output in loop) = Ei
Σm=
1 m
Proof of proposition Before explaining the proof, we will first introduce 7: Monte-Carlo estimator of the unknown normalization constant (.).. Since we are using K samples
A (xra 1: K)
1 K
Σ 0
4 ( Fias 211-1)- (5,
(15)
Algorithm
iis producing an unbiased estimator of the marginal likelihood. We will first integrate out from the marginal likelihood estimator.
11:N.1:K 1:K 11
EQuascenn
N/-1
11r 1/
21:1-; K) pa(ar, 1 -1.1-
A ol 11 =1.1-1)
Af
N
go (z) Fi 2
)g,
QusMC-PRe
I-N 1T-1
I.N.1:K
d2i.1 do N...N.
21:1's
1.T
Po 2 111-11
Al-1) 2 1111 21 K) IT
90 21 1
Σ ALN
[s(2) 2 - )ao, o (2f A/i=1)
QusMC.PRC
1:N (21:T 1 d21.7
1:N
po 2 -1:-1: 4/_1 ( 2 210-1)
A/-1) QUSMC-PCC (2):2 A7:N I:N
AEN
(0(2) F1: 2
Ja
21.1 211-1)
Po
2f 111-11
21.1-1)
I:N QUSMC-PCC (21:1
-1
dz
L:N
11 Ai-1)
(2.0
N
T-1 N
II
ro.0(zil
Dia (1t) d=t:N
A
1=1
T
N
=
W
(16)
T
11

---

Under review as a conference paper at ICLR 2021
Note that (zfll) is the sampling density of PRC. It is easy to that Eq. 16 is standard SMC estimator for the marginal likelihood po (T(TT:T) ). The proof is quite standard in SMC literature and can be found in (Naesseth et al. 2017; 2019). The key factor that makes our bound unbiased is the ability to produce unbiased ancestor samples despite the presence of intractable normalization constant. Using Jensen inequality likelihood.
easily show that VSMC-PRC bound is smaller then log marginal
I [CUSMC-PRCS K)
1
po 21 111. 21:8-1: K)
log
21:-1 ) ( (2/1 211-11)
4;-
(17)
JIN QUSMC-PRC
ATT 21:
11:K
1.T
dzyr d
dA S log logpo(21T).
We will show that E[[ysMccPre PRC; K] is non-decreasing with K. Let's define collection of subsets IN 12'
Hi,e ii (odiiillte C {1.2.... ........AA. K} having elements {21, i2,..., fm randomly drawn from the set {1, 2.,,.... K} having length m K. We can easily show the following expectation:
(2 K) K) = Er,,= (2nnan)))))))
a (
Ai-1
21:-11
K
k=1
(18)
Substitute the values of above expectation in equation 17 log [[X to complete the proof.
Use Jensen's inequality i.e Ellog X
LVVMM PRC: K
T
1
N
po(ar, x 11 ;
A:-1 2 [Ii.r
Al Ai-lim)
E
log E 1U
N
21: /s . (2 -1 -
4
1=1
T
po - 12 4i
1) (x Ai mm)
E
E BHI
log
N
(0(2)10111
2 )ae, ( F1:21 211-1)
A'-
1=1
E LVSMC-PRC;
Now we will see what happens when the limit K + co. Using dominated convergence theorem we can write down the estimator as follows:
lim oo [E[CvMMC-PRC)
T
po( 2/ F1st =1: 1 (1 - K)
Ai
A
lim K-+
E
log
( 11 - ; Ja, (af I
A:
A Ai
1=1
T
N
po( of | "
lim
2
2
Ai
K)
E
log
N
13
A:-1)
1,
i=1
)as
|0 24
T
N
Pa(xr: 2 ||
7
-
E
Σ
log
< log po (x(IT)
N
Ai
Ab
1=1
.14; _ )a0
=1x-11)
12

---

Under review as a conference paper at ICLR 2021
Proof of proposition
We will show that PRC step refines the learned distribution.
KT. ( 21: #1x))
Ai-1
4 r 1221221-11
f log
r( r(zol21e-1 Ai-1 (1:))
p 12212211211 01:1)
Ai-1
N أ 11t)
/ log
g ( T1: a
Ai-1
r(zel 2 xi
2 31:t) Z (+): أ
4:-1
4
< /
0 m1: )a 2 - 01:1)
log
g(=r|= = I )da
p(ze 21 1 Z 1111 2
Ai-1
((=1/2171 A: Fi ppl 2/
Mi و x.::) +
S KL
a(<1|zz:-1 1#
-
f1og log
ع( - vı )22)
2( -
KL
Ai-1 xm=e) |p( E | شي ) (((z)l
First, we will use the property of negatively correlated random variables. Note that the following two random variables
r( (1:1))
A and Y =2221212-1.
X log
Al-1 p(ac| ( .211))
are negatively correlated. We know that for negatively correlated variables following identity holds E|XY] < [[ Further we have used Jensen's inequality to show that
Ellog (z+ 2 1
x1 s log Z (x =)(-1)
Ai-
Case 1: (i, t 1) - 0 implies r r(2 (2 x11)) + q( 21a)
In this situation all samples would be accepted. Hence, we can express the sampling distribution as:
Ai 2 #1:)) = q(2g la A Ai-1),
(19)
Case 2: M(i,--11 0o implies r r= m) - p( 21 m1+)
4
In this case, the acceptance probability would reduce to standard rejection sampling. Therefore, the sampling distribution would become equal to the true posterior.
A/11)
p(=jaa ba 1, (
=1)
((11-11q(:5) - 4/-1)
A:
A$ A/-1 fq(=i|01a)
M 1:11 dz)
p(zf|e1a,21111 4-11
B GRADIENT ESTIMATION
In this section, we will derive the unbiased gradients for the Monte-Carke estimate E EV..vsMi.PRC) Note that we express the complete gradient into three core components (assuming 7() is reparametriz-
13

---

Under review as a conference paper at ICLR 2021
Sis Soer unblosed
terations5000
Figure Convergence rate for biased gradient vs. unbiased gradient on toy Gaussian SSM. compared grop (blue). gtep + SPRC (orange), and unbiased gradient (green) for optimization.
able):
Vo.
=
9tep + 9PRC + 9RSAMP
grep
=
IQQ Quscc--ic
V
Ve Ve.cVvsMcPec(O.0i 211
N T
a 222/ 211=11)
JPRC
(V (0,00 K)
log
13 Z 7
111=1)
N 7-1
ORSAMP
=
EQueuromc
E LUSMICPRC o; NAT, KVV
Σ
A log at
Note that for both gPRC and GRSAMP unbiased score gradient estimates are not available due to intractability. Therefore, we have used Monte-Carlo samples to estimate the log density for figure 3
C EXPERIMENTAL SETUP
For the real data experiment, we train a VRNN (Chung et al., 2015 on the polyphonic music dataset. Polyphonic music comprises of four datasets: Nottingham, JSB chorales, Musedata, and Piano-midi.de Each dataset was divided into standard train, validation, and test datset.
The validation data was used to tune the learning rate: picked the learning from the following set: (3 x 10-1, x 10-1, 3 x 10-5,1 x 10-5), instead of optimizing for each method, we picked the learning rate at which FIVO (Maddison al. (017) validation performance the best, Once the learning rate is decided, we ran every method for the same number of iterations to ensure uniformity, We use single-layer L.STM for modeling the hidden state having dimension d,, For length T sequence, the variational distribution and joint data likelihood are defined as follows
r(E1T(E1TT) I qe (2 (21 T1- 1. h4 1), x.)a (2, he 21-1, h4 1111),
I (he (2 Zt-1 h4 xz)
Note that (z(l))) is the acceptance probability for the PRC step and Z.(.) the intractable normal- ization constant. Similarly, we can write down the joint data likelihood as
T
p(Z1.r Firr)
II pe 1. Ize (x ( h4-1), t=1
The conditional distributions (=(l.)) and (z(.))) are assumed to be factorized Gaussians, where dimension the latent variable 2, is dz. Note that the conditional densities are parametrized by fully connected neural networks with single layer having size dy. The output distribution 9a (xxl.) modeled by a set of 88 iid Bernoulli variables. Please see Table for more details regarding implementation details. The unknown weights and biases initialized using Xavier initialization. For setting up the optimization we used a batch size of with adam optimizer. The unknown hyperparameter M is updated once every 50 epoches through Eq. 14 save time.
14

---

Under review as a conference paper at ICLR 2021
Table 2: Training parameters used for VSMC-PRC, FIVO, and IWAE.
Datasets
Parameters
Piano-mid
Muse
jsb
Nott
Optimizer Learning Rate Batch-size
Adam
Adam
Adam
Adam
0.00003
0.0001
0.00003
0.0001
4
4
4
dp
64
64
32
64
d2 num-iteration
64
32
64
50,000
1,00,000
1,00,000
1,00,000
15


Reviews:
[{"review_id": "z9QrRdvNCvq", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe submission suggests a new variational bound for sequential latent variable models. Unlike previous work that optimize this bound using ‘standard’ particle filters with unbiased resampling, the new bound is constructed based on a partial rejection control step and uses a dice enterprise for sampling the ancestor variables. \n\nPositives:\nThe combination of partial rejection control and dice enterprise for variational inference is new and interesting. Particle filters with partial rejection control have been used before for constructing (biased) bounds based on the marginal likelihood. However, using a dice-enterprise step allows for a new unbiased bound which makes it possible to consider a lower bound on the log-likelihood via variational ideas that can be optimized with standard techniques. Empirical experiments suggest that the method outperforms previous work.\n\nNegatives:\nDoes the complexity of the new bound not scale linearly with K (while K=1 for FIVO)? This seems to be not accounted for in the experiments. Choosing larger N=16 also has a better performance in the FIVO paper. \n\nRecommendation:\nI vote for acceptance of the paper. However, I think that the experimental section should be improved.\n\nComments:\nVariational bounds can also be constructed by targeting a smoothing distribution (Lawson et al, 2019) and particle filters with complexity N^2 based on a marginal Fisher identity have been suggested (POYIADJIS et al, 2011) for parameter estimation that avoid estimator variances scaling quadratically in time. I was wondering if there is a connection between such filters and the method suggested here, particularly for K=N?\nCan you explain the connection between the variance of the estimator for the normalizing constant obtained from particle filters and the tightness of the variational bound in more details?\nAre the signal-to-noise gradient issues for large N or K?\nHow do the methods in the experiments compare for a larger number of particles?\nIs there some useful practical advice on choosing the ratio N/K and gamma?\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "aEsD9IRJkyM", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Review update following author discussion\n\nI've read the author responses as well as some of the discussion with the other reviewers. Overall, this is valuable work and I've considered raising my score, but I think a weak accept is appropriate, all things considered.  I've raised the confidence score for my review, as I understand the technical details better now. I think a key strength compared to prior work is the empirical validation of the approach on a variational RNN.  However, the significance of the paper in terms of the novelty of the ideas, both conceptual and technical are overstated in my opinion, hence the weak accept. \n\nOne other point of feedback for the final version in case of an accept:\n\nI also share R3's concern about the paper's positioning in relation to the extremely general dice enterprise framework (or, even the bernoulli factory, for that matter) as somewhat misleading for the particular use case in Section 2.2.  The exact same multinomial sampling scheme is in fact more succinctly presented (proposed?) in BRPF [Schmon et. al, 2019] as the \"Bernoulli Race\" (which the authors have cited).  I would think that the current scheme is a special case of the \"Bernoulli race\" that uses a particular form of the acceptance probability parameterization similar to prior work (e.g. VRS [Grover et. al. 2018]). \n\nSee Section 3, specifically e.g. Eq (10) and Proposition 2 from BRPF [Schmon et. al., 2019], which can be compared to Eq (3) and Proposition 1 in the current submission, respectively. \n\nMinor nit: In step 3 of the algorithm (right below Eq (8)), you use the notation $z_t$ for sampling a new variable from $q_\\phi$. However, this $z_t$ has nothing to do with the latent variables that are used in computing the constants $c^i_t$. The way it's written makes it appear as if there's a circular depdendence of the $c_t$ on $z_t$ and then the $z_t$ is again resampled, which changes the $c_t$. For this reason, it maybe better to use a completely unrelated variable for the sample from $q_\\phi$ in step 3. \n\n\n### Key Strengths\n\nThe paper puts together several ideas from prior works (partial rejection control/SMC, variational inference, dice-enterprise) and also evaluates these ideas for latent variable sequential state space model benchmarks. The experimental results compare favorably to prior works like FIVO and IWAE and demonstrate that using partial rejection control is beneficial in a variety of benchmarks. \n\n### Key Weaknesses\n\nIt seems like a key contribution in terms of novelty/theory is in fixing the bias in prior works using Partial Rejection Control in SMC (e.g. Peters et. al. 2012). More information to support this claim in terms of why the prior estimate is biased would be helpful in assessing the strengths of the paper's contributions (Peters et. al don't seem to explicitly focus on the exact bias for PRC).  Having said that, the experiments seem to show that unbiased gradients are worse than a biased version (Figure 2, left bottom), so it seems like focusing on the exact bias is not that important.  \n\n\n### Additional Comments\n\n* The proof of unbiasedness (Prop 2) says \"it is easy to show that Eq (15) is an unbiased estimator\" and refers to prior work by [Naesseth et. al] for details. More clarity here would be helpful (especially around the term $Q_{VSMC-PRC}$) for assessing this claim, given that this is one of the main contributions claimed in the paper (besides also commenting on where the bias in prior work is coming from). \n\n* The $Z()$ function first appears in Equation (9), without a prior reference/definition. You might want to introduce this around Eq (4), where the integral appears. There's also a reference to what will later be $Z$ as $p^i_t$ in Eq (8).\n\n* Using partial forms of rejection to proposal distribution samples specifically for variational inference (rather than SMC more generally) has also been considered in prior work [R. Gummadi, \"Resampled Belief Networks for Variational Inference\", Advances in Approximate Bayesian Inference Workshop, 2014]. \n\n\n\n\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "MeTFUsW5069", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers SMC to construct variational approximations. SMC methods\ncan be improved with partial rejection control (PRC). Then it is not obvious\nthat one can obtain unbiased estimators of the normalizing constant, as in\nplain SMC. The authors consider a way of obtaining unbiased estimators in that\nsetting.  Experiments include a linear Gaussian state-space model and recurrent\nneural networks on polyphonic music datasets.\n\n---\n\nThe problem is interesting. The notation is a bit cumbersome at times, but it is \nthe case for most papers on SMC. The writing is mostly clear. The experiments\ninclude a mix of toy and more realistic examples. \n\nThe fact that SMC with PRC can still produce unbiased estimators of the\nmarginal likelihood was described by Kudlicka, Murray, Schoen, Lindsten,\n\"Particle filter with rejection control and unbiased estimator of the marginal\nlikelihood\". The authors do cite that paper, but I did not understand exactly\nwhy that paper does not completely solve the problem that the authors consider.\nAt first glance it seems that the Kudlicka et al paper would apply \"out of the\nbox\" when replacing the prior by an arbitrary proposal in SMC, i.e. using a\nproposal \"$q_t$\" instead of \"$f_t$\" and the ratio \"$g_t f_t/q_t$\" for the weights.\nMy understanding is that Kudlicka et al focus on the bootstrap particle filter\nbecause PRC is a remarkably generic approach to improve it, applicable even\nwhen sampling from the model prior is the only option; not because the same\nreasoning would not apply for generic proposals. \n\nThus it was not clear to me that there's a need for another paper showing that\nSMC with PRC can still provide unbiased estimators of the marginal likelihood.\nIf the authors made a convincing case that the extension to general proposals\nwithin SMC with PRC requires significant work, their contribution would be more\nconvincing.\n\nFurthermore, the manuscript makes references to Bernoulli factories and dice\nenterprise.  In fact, the manuscript addresses the problem of categorical\nsampling with unbiased estimators of the underlying probabilities.  The problem\nwas addressed in \"Bernoulli Race Particle Filters\" by Schmon, Doucet,\nDeligiannidis.  That paper is cited by the authors, but the authors do not make\nit clear that their algorithm (in Section 2.2) is exactly the same as\n\"Algorithm 2\" of that paper, their Proposition 1 is exactly \"equation (12)\" of\nthat paper, etc.  Furthermore, I don't think their algorithm is indeed a \"dice\nenterprise\" as in the terminology of Morina et al; I believe the references to\nthe Bernoulli factory literature would be sending most readers in the wrong\ndirection.\n\nBased on these flaws I do not think that the manuscript is suitable for\npublication. Perhaps a clearer explanation of the specific shortcomings of\nKudlicka et al's work would make the paper more convincing.\n\n---\nSmall comments:\n\n- page 2: \"We further assume that the joint density [...]\"\nin fact that decomposition always holds, it is not an assumption.\nIt is just p(a,b) = p(a)p(b|a). \n\n- page 2: A SMC sampler -> An SMC sampler\n\n- page 2 and later: there is some inconsistency between the notation M(i,t-1) and M(t-1,i).\n\n- The latex command \\eqref seems to have been used instead of \\ref, in various places.\n\n- page 6 \"utlizing the best of both worlds\"\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "4QayjiG_jO", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes an SMC algorithm to sample the posterior distribution of latent states $p_\\theta(z_{1:T}|x_{1:T})$ in a latent variable models $p_\\theta(x_{1:T}, z_{1:T})$. The authors consider a completely general setting (the authors assume Eq.(1) but clearly there is nothing to assume here, this the standard Bayes rule). It is well known that the vanilla SMC sampler is a good candidate for ELBO because it provides an unbiased estimator of the likelihood. But the authors prefer here to use a more sophisticated version of the  SMC algorithm, which features a partial rejection algorithm, which amounts to eliminate proposed particles which are \"large enough\" likelihood.\nIt is difficult for an expert in SMC algorithms to understand the algorithm as it is described (one must even guess the meaning of the notations). Equation 3) is rather misleading because it is not understood that one continues the acceptance/rejection sampling step until the sample is accepted [Peters' paper, a little less ambitious in its generality, is much more readable]. Also, the form of the rejection probability does not help to understand the action beyond the scene. The level of generality here is a killer: what the hyperparameter $M(i,t-1)$ means, what is the meaning of this form of rejection probability (we see something like a Barker ratio), this is very puzzling\nThe rejection probability modifies the mutation kernel and should be taken into account when computing the importance weights (Eqs. (4) and (5)). This implies to estimate a quantity $\\alpha_t(z_{1:t}^i)$ which is not tractable. The authors suggest (similar to Peters (2012)) to use a Monte Carlo estimator of these quantities. \nTo sample the ancestors variables from Eq. (7) with the $\\alpha_t(z_{1:t}^i)$ defined in Eq. (6), the authors use the \"Dice-Enterprise\" algorithm. It does not seem necessary to appeal to such  beautiful algorithm to understand the validity of the algorithm described page 3. Here again, everything is done to frighten the reader and not much is done to explain what is being done... \n\nThe results presented are encouraging and shows that the proposed approach outperforms IWAE and FIVO for a given calculation time. This result is clearly interesting and shows that partial rejection helps despite the additional difficulties linked with the intractability which requires an additional layer of complexity. \n\nI like the paper even if I have found it extremely unfriendly to read ! \n ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores the use of partial rejection control (PRC) for improved SMC-based variational bounds. While an unbiased SMC variant with PRC has been previously introduced by Kudlicka et al. (2020), this work introduces innovations that can help apply such ideas to variational inference. These bounds result in improvements in empirical performance. 

This paper was heavily discussed, with significant engagement by both the authors and the reviewers. Most reviewers recommended acceptance of this paper, with one reviewer (R4) recommending against acceptance. R4's central concerns regard the novelty of the proposed approach and its positioning relative to the existing SMC literature. The authors argued vigorously in the comments that this paper should be judged as a contribution to the VI literature and not the SMC literature.  Unfortunately, I will recommend that this paper is rejected. It is my opinion that R4's concerns were not fully addressed.

On the one hand, I agree with the authors that there is significant value to be had in exploring variants of SMC for VI. Indeed, some prior art, like FIVO and IWAE, contributed little to the Monte Carlo literature. I believe that these were good contributions.

On the other hand, I am concerned that the current draft does not clearly circumscribe its contributions. I read the sections that disuss the works of Schmon et al. (2019) and Kudlicka et al. (2020), and the writing did not leave me with a clear enough sense of the differences. I also read the abstract and introduction of the paper. The introduction of the paper positions this work clearly within the VI literature, but does not clearly discuss prior SMC art, e.g., it does not cite Kudlicka et al. (2020). Despite citing rejection control for SMC, the writing of the abstract and introduction left me with the impression that this work was the first to introduce *unbiased, partial* rejection control for SMC. I believe that impressions matter and that the machine learning community should be generous to adjacent communities when assigning credit.

I realize that my decision is a matter of taste. I also want to say that I am confident that the authors have a clear sense of where their contribution sits, and I suspect that it is a valuable contribution. However, I cannot recommend the draft in its current form. If this is a contribution to the VI literature, as the authors argue, then the authors should not hesitate to give full credit to prior SMC art. My reading of the current draft still leaves me confused about which aspects of the SMC estimator are actual contributions.
Decision: Reject