SIXO: Smoothing Inference with Twisted Objectives
Authors: Dieterich Lawson
Allan Raventos
Andrew Warrington
Scott Linderman
Venue: NeurIPS 2022 Accept (2022)
Keywords: smoothing
variational
objectives
fivo
sequential Monte Carlo
inference
twisted
time series

Abstract: Sequential Monte Carlo (SMC) is an inference algorithm for state space models that approximates the posterior by sampling from a sequence of target distributions. The target distributions are often chosen to be the filtering distributions, but these ignore information from future observations, leading to practical and theoretical limitations in inference and model learning.  We introduce SIXO, a method that instead learns target distributions that approximate the smoothing distributions, incorporating information from all observations. The key idea is to use density ratio estimation to fit functions that warp the filtering distributions into the smoothing distributions. We then use SMC with these learned targets to define a variational objective for model and proposal learning. SIXO yields provably tighter log marginal lower bounds and offers more accurate posterior inferences and parameter estimates in a variety of domains.


SIXO: Smoothing Inference with Twisted Objectives
Dieterich Lawson; Allan Raventós, Andrew Warrington; Scott Linderman
(jdlawson, aravento, awarring, scott www.neeenannnttttaforrdddd edu Stanford University
Abstract
Sequential Monte Carlo (SMC) is an inference algorithm for state space models that approximates the posterior by sampling from sequence of target distributions. The target distributions are often chosen to the filtering distributions, but these ignore information from future observations, leading to practical and theoretical limitations in inference and model learning. We introduce SIXO, method that instead learns target distributions that approximate the smoothing distributions, incorporating information from all observations. The key idea is to use density ratio estimation fit functions that warp the filtering distributions into the smoothing distributions. We then use SMC with these learned targets to define a variational objective for model and proposal learning. SIXO yields provably tighter log marginal lower bounds and offers more accurate posterior inferences and parameter estimates in a variety of domains.
Introduction
In this work we consider model learning and approximate posterior inference in probabilistic state space models. Sequential Monte Carlo (SMC) is a general-purposs purposs method for these problems [1-3] that produces unbiased estimate of the marginal likelihood as well as latent state trajectories (i.e. particles) that can be used to approximate posterior expectations. SMC can facilitate model learning via expectation onpectation-maximizztiooo also be cast in a variational framework [6, 7] as a rich family of approximate posterior distributions that can be fit using stochastic gradient ascent and modern automatic differentiation methods [8-12].
or direct maximization of the marginal likelihood estimate [4, 5]. It can
The quality of SMC's marginal likelihood and posterior estimates driven by two design decisions: the choice of proposal distributions and target distributions. The proposal distributions specify how particles propagate from one time step to the next, while the target distributions specify how those particles are weighted and which ones survive future time steps. The most common SMC variant, filtering SMC, sets the targets to the filtering distributions, the conditional distributions over latent states X1:1 (x1,.... (x,...., x1) given observations y1a (y1..... y:). The central issue is that the filtering distributions do not incorporate information from future observations Y++1.7
Figure illustrates why setting the target distributions to the filtering distributions can be problematic. In this example, the latent states follow simple Gaussian random walk, but only the last step observed. Thus, the filtering distributions reduce to the prior, a series of mean zero Gaussians, shown in Figure la. If the observation is far from the prior, the filtering distribution suddenly jumps at time T T= 10. This is a recipe for disaster in SMC: the particles time T - 1 will be distributed according to mean zero Gaussian and very few will survive to the next time step, causing the variance of the SMC estimator to explode. Even if the proposals incorporate smoothing information, using filtering targets can cause particle degeneracy by resampling away high-quality particles, as seen in Figure Ic.
* Equal contribution.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Filtering distributions
(b) Smoothing distributions
Pol ( v
pa y
Mr
10
5
10
15
-5
5
10
15
Zt
(c) FIVO particle lineages
(d) SIXO particle lineages
10
10
8
8
4
L=1
2-
+ M/
10
Timestep
Timestep
Figure 1: Theoretical and empirical target distributions for a Gaussian random walk with a single observation yr 10 and V1:T-1 Ø. (a) For t 1, ... the filtering distributions reduce to series of mean-zero Gaussians. At = T = 10, the filtering distribution incorporates the observation yr. resulting in sudden shift and particle death. (b) In contrast, the smoothing distributions steadily shift towards the observation, matching the posterior perfectly. (c) The proposal learned by a previous method, FIVO [8-10], exploits smoothing information to propose particles upwards towards the observed value. However, FIVO is based on filtering SMC which "resists" this by resampling particles back towards the prior, resulting in particle degeneracy. information, but proposed particles are preserved by the learned target distributions.
(d) SIXO's proposal also leverages smoothing
Suppose instead that the target distributions were the smoothing distributions the conditional distributions over latents X given all observations yI:7. Figure 1b shows the smoothing distributions for the simple Gaussian random walk. Unlike the filtering distributions, the smoothing distributions shift steadily toward the observation over time. These slow, smooth changes are ideal for SMC: Figure Id shows many particles surviving from one step to the next, resulting in a low-variance SMC estimator.
In practice we do not have access to the smoothing distributions if we did, there would be no need for SMC! Here, introduce Objectives. SIXO provides a unified approach for learning model and proposal parameters, as well as of nwisting functions that warp the filtering distributions into targets that better approximate the smoothing distributions [13]. Like its predecessor FIVO [8-10], SIXO uses a variational approach, deriving lower bound to the marginal likelihood. Unlike its predecessor, prove that the SIXO bound can become tight, even with finitely many particles.
method called SIXO: Smoothing Inference with Twisted
The key challenge with SIXO is learning the twisting functions. We find that simple density ratio estimation approach works best, and we propose an algorithm that interleaves twist updates with updates to the model and proposal. Thus, SIXO offers a means of jointly learning model parameters, SMC proposals, and targets for accurate posterior inference.
Finally, we give empirical evidence to support our theoretical claims. Across a range of experiments with a Gaussian diffusion, a stochastic volatility model of currency exchange rates, and a Hodgkin- Huxley model of membrane potential in neuron, SIXO consistently outperforms FIVO and related methods. We dissect these results illustrate how learning better targets enables more effective posterior inference and model learning.

---

2 Background
Consider modeling sequential data Yı.r E y" using latent variables X1:7 E X with Markovian structure, and let the joint distribution factorize
T
po NoiITT,SY.. (21(x)) xi
II (x xc-1)po xx) 2
with global parameters A E O. We further assume that the conditional distributions (x(x)) x(-1) and (y' xx) may depend nonlinearly on x and x; respectively.
The marginal likelihood and posterior for this model class are not readily available from the joint distribution due to the intractable integral over the latents X1.T, i.c.
po (y(:))) = J.r pe (yIT, XI.T) dx
cannot easily be computed due to the form the conditional distributions.
2.1 Sequential Monte Carlo
Sequential Monte Carlo is algorithm for inference in state-space models that approximates the posterior Po (xTT Y1:r) with a set of K weighted particles 1: These particles are constructed by approximately sampling from sequence of target distributions { (x1:)))==1 with the intuition that sampling from series of distributions that gradually approach the posterior easier than attempting to sample from it directly. The targets are often only available up to an unknown normalizing constant Z. so SMC uses the unnormalized targets (7r(X ] which correspond to the normalized targets via Ty (X1:1) = (x1:)) /Zz
SMC repeats three steps: First, xf-7 conditional on the current particles xi:t-1- Then, each particle is weighted using the unnormalized target "/e (x1:a) to form an empirical approximation of the normalized target distribution. Finally, particle trajectories x I:K are drawn from this approximation to the normalized target.
of latents sampled from proposal distribution go (xf
iit
Ideally the target distributions smoothly approach the posterior so that sampling from the target at time L + 1 is easy given samples from the target at time I. As long mild technical conditions are met and (r((x:7) x Po (x YL.T). SMC returns a consistent and unbiased estimate of the marginal likelihood (yr:r) and set of weighted particles approximating the posterior po (x1. Yi:r) [1-3]. For more details see Appendix B.1, and for thorough treatment of SMC see [1-3].
2.2 Filtering SMC and Model Learning
The most commonly-used SMC algorithm is filtering SMC (FSMC), which sets the normalized targets the filtering distributions, i.e. (x(x).)) = pe (x1: Vi:t) and 7 (xxx)) x
Let ZFSMC (0, (I.)) be the marginal likelihood estimator returned from running filtering SMC with proposal distributions (xo X1:-11y1:)) which may share parameters with po-
Previous work used filtering SMC fit model parameters by ascending lower bound on the log marginal likelihood called a filtering variational objective (FIVO) [8-10]. The FIVO bound is derived using Jensen's 's inequality and the unbiasedness of ZFSMC.
( (f yI.r) A E[log Zrsui (0, yI.T)] S log (1 logro(yi.T),
and is optimized using stochastic gradient ascent in [8-10, 14].
2.3 Smoothing SMC via Twisting Functions
The main disadvantage of filtering SMC is that the filtering distributions only condition on obser- vations up to the current timestep I, ignoring future observations Y:+1:7. This creates situations where future observations are highly unlikely given the current latent trajectories, which in turn causes particle death, high variance normalizing constant estimates, and poor inference and model learning [8, 13, 15]. Performing smoothing SMC would resolve this issue by choosing the smoothing
3

---

distributions targets, i.e. (x (x Yı.T) and (xx:))) po Yı.r). Unfortunately, Po (o(Xtt YI.T) is not readily available from the model and computing it is roughly as hard as the original inference problem.
However, pe (Xt:t: Y1.T) factors into the product of the filtering distributions, pe Yi.t), and the lookahead distributions, po (y(+++1:1 x²) (Appendix B.2). If the lookahead distributions can be well-approximated by series of "twisting" functions [13], {r(y++1:7 xx) then running SMC with targets " (x1:)) pe (x V1:1 (yy++r X,) would approximate smoothing SMC. In this sense, the lookahead distributions are optimal twisting functions [13, 16].
Different twisting functions yield different SMC methods such auxiliary particle filters and twisted particle filters [13, 17]. However, as long as the final unnormalized target (X1:)) proportional to (x 1:T.:::177 .i.r) and regularity conditions are met, SMC will produce unbiased estimate of the marginal likelihood, regardless of the choice of twisting functions [1. 3, 18]. Instead, the quality of the twisting functions affects the variance of SMC's marginal likelihood estimate.
3 SIXO: Smoothing Inference with Twisted Objectives
Our goal fit models by optimizing a lower bound on their log marginal likelihood constructed us- ing smoothing SMC. To construct the lower bound, fix T (xx) = and let ŽsIxO ( 2 Fi.r) be the marginal likelihood estimator returned from running SMC with unnormalized targets {pe (x1.t: V1.4) (V ())+1,,,x)))= and proposal distributions {9o(x; X1.t-1. Y Because the 7hh unnormalized target is pe (I.T). VI.T), Zsixo will be an unbiased estimator of the marginal likelihood pa (y:T)) [1.3] 3]. This implies via Jensen's 's inequality that
Lsixo (0, 20 y1.)) A E log Zsixo (0, ub V1:r)
(3)
< log E Z (0, Φb yI.T) = log
i.e. Ls 1.0....... Y1.r) a lower bound on the log marginal likelihood log (y):r) [14].
3.1 The Functional Form of the Twists
The structure of the lookahead distributions (y x:) suggests a functional form for T that accepts single latent x, and produces distributions over all future observations y++1:7. Because the twists will be evaluated once per particle and timestep in an SMC sweep, this functional form would lead to algorithm with ( complexity. To reduce the complexity, consider methods: fixed-lag twisting and backwards twisting.
Fixed-lag twisting approximates the full lookahead distribution ( xx) using a fixed window of L observations, i.e. it models po x.) [17, 19, 20]. We define the fixed-lag twisting functions (y [ as sequence of functions which accept x; X' and produce distribution over Y++te+1 yz This reduces the computational complexity to L), at the expense of only looking at L observations.
In our experiments one-step lookahead
an L 1 twist that scores the next observation by approximating the
Pe (y++1 x:) = pe (Y++1 xx+1) pe (xx+11 xx)) dx:+1
(4)
with
quadrature [21]. We refer to this as the "quadrature twist". This approach is similar to the APF [17, 22], but uses numerical quadrature in place of sample-based integration.
Backwards twisting is motivated by rewriting the lookahead distributions using Bayes' rule,
Po Po xx)) px( Y(41T) po x po(xe y++1.)) (x(x;)
(5)
Po
where we drop terms independent of x, because the twisting functions will be used to score particles in SMC. Thus, we need only approximate pe(x; N(+1.))/pef (x:). The numerator pe(x; y(+1.)

---

Algorithm SIXO-DRE
1: procedure SIXO-DRE(YL.T 8.. 40. S. N.N. K)
2: for s 1...., do
3: 4/s - ,, N)
4: 0, -MODEL-UPDATEENII. Oz 1/.. N, K)
5: return 0s. s/s
6: procedure TWIST-UPDATEG.. 4/u, N)
7: for = -....,,,. N do
8:
X1.T pe (XI:T)
9:
X1.7, (I(
LDRE (1b) - Ei (log ry (logollloy (Y1+111: xx)) + log(1 a x,)))
11: Compute ub; using the gradients of LORE evaluated at ab;-1
12:
return N
13: procedure MODEL-UPDATEIC,
Oo, K)
14:
for i = l.... N do
15:
ZsIXo (0) (MC((P) (XL+, (V+ (xe Xx-1 K)
Compute 0, using the biased gradients of 7sIxO evaluated at 0,-1
17:
return 0
procedure SMC(())(xx)))) (go(x; (xr) K) See Algorithm 2 in Appendix B.I.
the reverse of the lookahead distributions it is distribution single latent conditioned on future observations. This makes it possible to parameterize the twists using a recurrent function approximator (e.g. a recurrent neural network RNN) run backwards across the observations 1:T to produce twist values for each timestep as a function x
We define the backwards twists (1))11T.x)))) as a sequence of positive, integrable, real- valued functions y²²' X' + R, with parameters vA V. Parameterizing backward twists with recurrent function approximator results in (T) complexity (see Appendix C.3) and allows the twist condition on all future observations, making backwards twisting preferable fixed-lag twisting.
3.2 Learning Twists
Ascending the Unified Objective One way to fit the twists, proposal, and model is ascend LsIxo in the parameters of pe, qa. and rw. similar to FIVO [8-10]. The gradients of this objective include score-function terms that arise from the discrete resampling steps in SMC. We refer to ascending Lsixo with these unbiased gradients as SIXO-u. Because the resampling gradient terms have high variance, SIXO-u impractical for complex settings
[8, 11]. Lower-variane variance methods for estimating these gradients were explored by Lawson et al. [11] but found to be ineffective. We therefore seek an alternative method for training the twists. For detailed discussion and derivation of the gradient, see Appendix C.
Density Ratio Estimation Note that the optimal backwards twist is proportional to the ratio of a "backwards message" (xx yt-1:) and the latent marginal pe (xx) (Equation 5). Thus, we can learn the backwards twist using density ratio estimation (DRE) [23, 24].
DRE via classification estimates the ratio of two densities a(x)/b(x) by training a classifier to distinguish between samples from and b. If such a classifier is trained using the logit link function, then its raw output will approximate log (xx) log b(x)) up to a constant [24]. Using this approach, we interpret log Tw T (Y++1:7, xt) as the logit between samples from po (xt: Ye+rer) and po (x(x))e (x)))xx (y++1.r). which are available from the model. When trained in this way, log rw (YE41::: xx) will approximate log Po(x; y(+1-)) - log po (x:) up to constant, which can be ignored. For details see Appendix C.2 and Sugiyama et al. [24].
Bernoulli classifier, which is trained to distinguish
5

---

We use the DREE-earmed twisting functions in alternating scheme that first holds po; qo fixed and updates T using density ratio estimation, and then holds re fixed and updates po and go by ascending a biased gradient estimator (no resampling terms) of Lsixo (0, +)) O. We call the full alternating procedure for learning 0 and &b SIXO-DRE, see Algorithm 1.
3.3 The SIXO Bound Can Become Tight
Maddison et al. [8] show that the FIVO bound can only become tight in models with uncommon dependency structures. We show that the SIXO bound can become tight for any model in the class defined in Section 2.
Proposition 1. Sharpness of the SIXO bound. Let p(XI.T, be a latent variable model with Markovian structure as defined in Section 2, let Q be the set of possible sequences of proposal distributions indexed by parameters integrable twist functions indexed by parameters up E V, Assume that {p(x, x111 E Q w
O and let R be the set of possible sequences of positive,
of
and {p(y++1.T x E R. Finally, assume L0 yv.)) has the unique optimizer 0", xb" arg max Lsixo(0,U 1b. y1.)).
Then the following holds:
1. qa- (x: XI:-1, = p(x: X1a-1, for 1= 1..,...,
2. Twn Xx) x P(Y++1:77 xe) up to a constant independent of x, for t = I....,T T-1,
3. C 5 (0*, 2b" yI.r) log ( for any number of particles K
Proof. See Appendix C.5.
This is an important advantage of our work-the SIXO objective is the first to recover the true marginal likelihood with a finite number of particles while also being tailored to sequential tasks.
Related Work
Standard references for SMC include Doucet and Johansen [1]. Naesseth al. [2], and Del Moral [3] which provides a theoretical treatment of a generalization of SMC called Feynman-Kan -acc formulae. Pitt and Shephard [17] introduced the auxiliary particle filter, an early smoothing SMC method which constructs an estimate of the one-step backwards message po vy x+) using simulations from the model. Smoothing SMC general is discussed thoroughly in Briers et [15] and Del Moral et al. [25]. Later, Whiteley and Lee [13] introduced twisted particle filters, which perform SMC on a twisted model that approximates the smoothing distributions by multiplying the model's filtering distributions with "twisting" functions. Our work extends the theoretical framework in Whiteley and Lee [13] by proposing practical and effective methods for learning parametric twisting functions.
To make smoothing SMC computationally tractable, fixed-lag techniques use information from only a fixed window of future observations, as introduced in Clapp and Godsill [19] and surveyed in Lin et al. [20]. For example, Park and lonides [26] use simulations from the model to estimate fixed-lag twisting functions and Doucet al. [27] sample blocks of latents conditional on their observations via various Monte Carlo methods. These methods suffer from computational complexity that grows with the window size, and fail to take advantage of all future observations.
Other methods use twisting functions which depend on all observations. Most similar to our approach Guarniero et al. [16], Heng al. [28], and Zimmermann et al. [29] which learn parametric twists using a Bellman-lice like decomposition of the lookahead distributions p(y++1:7 x;) in terms of the same distributions one step into the future. Our method instead uses independent DRE objectives for the twist at each timestep. Del Moral and Murray [30] use Gaussian processes to approximate the twists, Lindsten et al. [18] use traditional graphical model techniques such as loopy belief propagation and expectation propagation, and Ruiz and Kappen [31] use optimal control techniques. None of these approaches consider model learning and their twist parameterizations and learning techniques are highly specialized to specific problem settings.

---

L128
10
15
FIVO SIXO-DRE
True
-3
Bound gap
10a
SIXO-U
IWAE
log ply)))
10-1
%
10-3
100
200
300
400
0
100
200
300
400
Optimization step (1000s)
Optimization step (1000s)
(a) Bound gap for different methods.
(b) Convergence twist parameters with SIXO-u.
Figure 2: Results for the Gaussian drift diffusion experiment presented in Section 5.1. The median across ten random seeds are shown. Further figures and discussion are included in Appendix D.I.
Fitting model parameters via stochastic gradient ascent on an evidence lower bound (ELBO) was introduced in Ranganath et al. [32], Hoffman et al. [33], Kingma and Welling [34]. and later gen- eralized to the Monte Carlo objectives (MCO) framework by Mnih and Rezende [14]. Since then, works have considered optimizing lower bounds defined by the normalizing constant estimators from multiple importance sampling [35], nested importance sampling, [36, 29]. rejection sampling and Hamiltonian Monte Carlo [12], filtering SMC [8-10], and smoothing SMC [11, 37, 38]. The prior work on smoothing SMC used an objective defined by forward filtering backwards smoothing [15] which suffers from the same particle degeneracy issues filtering SMC and cannot become tight. Kim et al. [39] optimize the importance weighted autoencoder (IWAE) bound [35] using a baseline derived from future likelihood estimates, but do not use SMC resampling in their bound.
5 Experiments
We experimentally explore our claims that:
1. The SIXO bound can become tight while FIVO cannot.
2. DRE-learned twists enable better posterior inference than filtering SMC.
3. Model learning with SIXO provides better parameter estimates than FIVO.
5.1 Gaussian Drift Diffusion
We first consider miedimmnsionnl Gaussian drift diffusion process with joint distribution
T
Po (Xirryyr) yr) N (yr Fr + a, oz) N (11; 2 II N (x. xt-1 ft-1+a,033
(6)
The single free model parameter is the drift R, the state is 2, R, and the observation is yr E R. Figures la and 1b show that for a = the filtering and smoothing distributions in this model quickly diverge, which can lead to poor inference for filtering methods.
compare joint model, proposal and twist learning using two variants of SIXO to variational inference with the IWAE bound [35] and FIVO with unbiased gradients [8-10]. All methods use an in- dependent proposal at each time step parameterized as qe (z; 2t-1, 8r) Wxxx; fo fe(1;111 yr), o where ft is affine function, family which contains the optimal proposal. SIXO-u uses twists parameterized as Ty (yT, 2+) N (yT: Gr (x+). o².) where 9r is an affine function, a family which contains the true lookahead distributions. The SIXO-DRE twist functions, log r. (Mr,, 22), are param- eterized as quadratic functions of ø,, where the parameters of the quadratic function are generated by neural network with inputs t). The true log density ratio will be quadratic in Tr, if the neural network is sufficiently flexible, the true log density ratio function can be obtained.
Figure 2a shows the convergence of the variational bound for each method. As expected IWAE recovers tight variational bound, whereas FIVO does not. While SIXO-u does recover tight variational bound, the high variance of the unbiased gradient estimator makes it slower to converge,

---

Table Performance FIVO and SIXO on the SVM.
Method
Train Method (as in [9])
Train CiPe
Test BPF
IWAE
6940.18 + 1.17
7019 38 2.99
3351.21 2.35
FIVO
6921.29 .9 + 1.33
7020. 14 + 2.86
3352.51 + 1.30
SIXO-q
6928.90 90 + 1.24
7019. 65 2.97
3353.10 1.58
SIXO-DRE
6931.51 = 2.08
7019.42 + 3.01
3354.08 = 1.60
and impractical for non-toy problems. Conversely, SIXO-DRE achieves tight bound while using the lower variance biased gradients. This motivates its use in more complex, non-linear settings where the unbiased FIVO gradients are not practical. Figure 2b shows that SIXO-u recovers the correct twist parameters. More figures illustrating the convergence of & and zb are included in Appendix D.1.
In Figures Ic and Id we compare particle trajectories under FIVO and SIXO-u. We see that FIVO consistently proposes particles with high likelihood under the posterior distributions (identical to the smoothing distributions in this case) which are discarded by the resampling steps in filtering SMC. In contrast, SIXO both proposes particles with high posterior likelihood and retains them through the resampling steps by properly scoring particles under the twisted target distributions. These results empirically verify the theoretical claims made in Section 3.3.
5.2 Stochastic Volatility Model
We now apply SIXO stochastic volatility model (SVM) of monthly foreign exchange rates for N = 22 currencies, over the period 9/2007 to 8/2017 [40]. The SVM generative model is defined as
x1 N (0, Q), X, /e + Φ (x,-1 11) Vos Yr 3 exp
E
with transition noise Us N (0, Q), observation noise e, N / NN) states X1:7 E R and observations YI.7 E R XN All multiplications are performed element-wise, denoted by @. The proposal, qo. is structured as go go (x)r) x II NN(x; p4; 14; Ex)po(x; x,-1) with means μ4 E R and diagonal covariance matrices Σ, E dliag(Re (R-"), ). We compare four approaches: IWAE, FIVO, SIXO with quadrature twist (SIXO-q), and SIXO with density ratio twist (SIXO-DRE). Note that the observations for this model are dense in time, so we would expect filtering-based approaches to perform well. See Appendix D.2 for more details.
Train Performance We first compare our methods in terms of their 4- 4-particle log marginal like- lihood lower bounds as in Naesseth et al. [9], e.g. for IWAE we report the IWAE bound and for FIVO we report the FIVO bound. Even though SIXO-q only scores single future observation, still obtains 7-aat improvement over FIVO. SIXO-DRE, meanwhile, conditions on all future observations and obtains 10-nat improvement over FIVO. All SMC-based methods, however, significantly underperform IWAE which seems to indicate that IWAE's unbiased gradients and use of smoothing information allow it to learn proposal which is efficient at low numbers of particles. This performance does not carry over to model learning or other high-dimensional problems, however, and is at odds with previous work on this model and dataset [8-10].
We also estimate the learned model's training set marginal likelihood by computing bootstrap particle filter's 's (BPF) log marginal lower bound with 2,048 particles, denoted 2214 one-way ANOVA [42] does not reject the null hypothesis that the training set CPPF means are all equal (p = 0.26), suggesting that the log marginal likelihoods on training data are indistinguishable and training performance has saturated. It is clear, however, that among the SMC methods SIXO performs the best inference and makes the most efficient use of particles as the L bounds are significantly higher than Ct for similar models.
<BPF [41]. Interestingly,
Test Performance We also compare methods on held-out test set to evaluate model learning. We construct this test set using the same data source as the training set, but use the period of time since Naesseth et al. [9] was published, bounds with 2,048 particles and find that SIXO-DRE outperforms IWAE, SIXO-q and FTVO. All differences are statistically significant.
extra 55 months. Again, we report BPF log marginal lower
8

---

50 BPF SIXO-bs
Resampling event
Yt
Membrane potential, mV
50
-100
15
Time, ms
Figure 3: Comparison of trajectories generated by BPF (top) and SIXO-bs sweep (bottom) on synthetic data from the Hodgkin-Hux model. Both sweeps use the true model parameters and a bootstrap proposal while SIXO-bs the twist changes the SMC weights at each model step. while the BPF only changes weights at each observation. This allows SIXO-bs resample away erroneous spikes right as they begin.
learned twist. SIXO-bs resamples more frequently because
Table 2: Hodgkin- Houllyin-Huxley inference performance for different observation intervals.
C32 Method / number of observations
Observation Interval
SIXO-sm
SIXO-bs
FIVO-fi
FIVO-bs
2ms
-1.66 + 0.054
-1.7 0.100
-22.80 + 0.457
-2.77 = 0.297
Ims
-1.18 + 0.004
-136 0.206
-11.86 0.304
-2.36 0.356
0.5ms
-1.06 + 0.013
-1.17 0.254
-
-2.17 + 0.305
0.2ms
-0.92 + 0.003
-1.00 0.2099
-2.49 + 0.106
-1.93 + 0.243
5.3 Hodgkin-Huxley Model
We conclude by comparing FIVO and SIXO on the Boodkkin-Huxler (HH) model of neural action potentials [43, 44]. A single neuron is represented with a dimensionn state-space: the instanta- neous membrane potential and the relative conductivity of three ion gates. A noise-corrupted and subsampled membrane potential can obtained using electrodes [45] or voltage imaging [46]. The state of the gates, however, is not observable, and must be inferred from the noisy potential recordings. The physiological parameters governing the time-cvolution of the system are also of interest, such as the base conductance of each of the ion channels.
We implement the HH model
four-dimensional nonlinear state space model with Gaussian transition noise [47]. The observation is single Gaussian diiiianbbedtttbeeddd value with mean equal to the instantaneous potential. Unless otherwise stated, we subsample observations by factor of 50 to simulate an acquisition frequency of 1kHz (interval of ims). For more details, see Appendix D.3.
In this model action potentials, or spikes, change in the state. Therefore, filtering-based inference is particularly disadvantageous as noisy observations may trigger erroneous spikes or "miss" true spikes.
events that happen quickly and invoke rapid
We compare four methods: SIXO with DRE twist and a learned smoothing proposal (SIXO-sm), SIXO with bootstrap proposal (SIXO-bs), FIVO with a learned filtering proposal (SIXO-fi), and FIVO with a bootstrap proposal (FIVO-bs).
we compare HH trajectories generated by a BPF and SIXO-bs. The BPF yields spurious spikes and misses the initiation of other spikes, illustrating the issue with filtering SMC for HH inference. SIXO-bs allows fewer spurious spikes to develop and fewer particles miss the onset of spikes. SIXO-bs also achieves log marginal lower bound 447.81 nats, higher than the -49.24 nats achieved by the BPF, showing that it performs more effective inference. In Table 2
Inference In Figure 3

---

104
0 absolute error
-50
Mee hed
(256
-55
10
FIVO-bs FIVO-F
SiXO-bs SIXO-am
-65
10-1
200
400
600
800
1000
200
400
600
800
1000
Optimization step (1000s)
Optimization step (1000s)
(a) Validation set Method over training.
2
(b) Relative parameter error over training.
Figure 4: Hodgkin-Huxley, model learning performance over training. Both SIXO-bs and SIXO-sm obtain better parameter estimate and more stable bound than FIVO. FIVO-fi not stable and achieves a low bound, not visible on Figure 4a.
Table 3: Hodgkin-Huxley model learning performance.
Bound
Proposal
Test 2556
Test LBP
0 Relative Error
Method
(True model)
(Bootstrap)
(N/A)
(-49.12)
(0.0)
FIVO
Bootstrap
-51332 0.73
-5.33 + 0.73
0.45 + 0.03
FIVO
Filtering
660.62 + 283.50
-115.74 + 80.12
0.78 0.20
SIXO
Bootstrap
-48.98 + 0.20
+9.40
0.10 + 0.06
SIXO
Smoothing
48.73 0.14
-49.30 = 0.71
0.05 0.02
study this effect more extensively, comparing performance across different observation frequencies. We see, as predicted, that the performance of SIXO is more consistent across observation frequencies than FIVO, supporting the claim that twists assist in inference when observations are sparse. For more results, see Table in Appendix D.3.
Model Learning We conclude by comparing FIVO and SIXO for parameter recovery in Figure 4 and Table 3. We see that FIVO-bs converges to poor parameter estimate, and recovers a poor variational bound. FIVO-fi does not converge (details and full figure Appendix D.3). The SIXO methods recover much better parameter estimates, and achieve the highest bound values, with SIXO-sm outperforming all other methods terms of final performance and training stability.
Conclusions, Limitations, and Future Work
In this work we proposed method of learning twisting functions for smoothing SMC via density ratio estimation. Our approach ascends a lower bound on the log marginal likelihood that can theoretically become tight, first for SMC objectives. We verified theoretical claims by experimentally demonstrating improvements over existing techniques in inference and model learning.
There are, however, important limitations to our approach. Training and evaluating the twist requires additional computational effort compared to FIVO, and requires further hyperparameter tuning. Although we consider SIXO to be mainly used for offline settings, extending SIXO to online settings could yield practical benefit. Finally, there is a known pathology in DRE methods where the ratio may be poorly estimated if the difference between the densities in the ratio is very large [48]. Thus, new methods for learning the twist are important topics for future work.
Acknowledgements and Disclosure of Funding We thank Matt MacKay for helpful discussions and edits, and the anonymous reviewers for their feedback during the review process. This work was supported by grants from the Simons Collaboration BRAIN Initiative (U19NS S199S113201 and ROINS1131 19). Some of the computation for this work was made possible by Microsoft Education Azure cloud credits. Dieterich Lawson supported in part by a Stanford Data Science Fellowship.
the Global Brain (SCGB 697092), the NIH
10

---

References
[1]
Arnad Doucet and Adam M. Johansen. A tutorial on particle filtering and smoothing: Fifteen years later. Dan Crisan and Boris Rozovsky, editors, The Oxford Handbook of Nonlinear Filtering, pages 656-704. Oxford University Press, 2011.
[2] Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön, et al. Elements sequential Monte Carlo. Foundations and Trends@ in Machine Learning, 12(3):307-392, 2019.
[3] Pierre Del Moral. Feynman-Kan Kac formulae: genealogical and interacting particle systems with applications, volume 88. Springer, 2004.
[4] Christophe Andrieu, Arnaud Doucet, Sumeetpal S. Singh, and Vladislav B. Tadic. Particle methods for change detection, system identification, and control. Proceedings of the IEEE, 92 13::4234338 2004.
[5] Shixiang Shane Gu, Zoubin Ghahramani, and Richard E. Turner. Neural adaptive sequential Monte Carlo. Advances in neural information processing systems, 28, 2015.
[6] David Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518) 2017.
[7] Martin Wainwright and Michael I. Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends@ in Machine Learning, 1(1-2):1-305, 2008
[8] Chris J. Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, and Yee Whye Teh. Filtering variational objectives. Advances in Neural Information Processing Systems, 30, 2017.
[9] Christian Naesseth, Scott Linderman, Rajesh Ranganath, and David Blei. Variational sequential Monte Carlo. In International Conference on Artificial Intelligence and Statistics, pages 968 977. PMLR, 2018
[10] Tuan Anh Le, Maximilian Igl, Tom Rainforth, Tom Jin, and Frank Wood. Auto-encoding sequential Monte Carlo. In 6th International Conference on Learning Representations, 2018.
[11] Dieterich Lawson, George Tucker, Christian A. Naesseth, Chris Maddison, Ryan P. Adams, and Yee Whye Teh. Twisted variational sequential Monte Carlo. In Third workshop on Bayesian Deep Learning (NeurIPS), 2018.
[12] Dieterich Lawson, George Tucker, Bo Dai, and Rajesh Ranganath. Energy-inspired models: Learning with sampler-induced distributions. Advances in Neural Information Processing Systems, 32, 2019.
[13] Nick Whiteley and Anthony Lee. Twisted particle filters. The Annals of Statistics, 42(1): 115-141, 2014.
[14] Andriy Mnih and Danilo Rezende. Variational inference for Monte Carlo objectives. In International Conference Machine Learning, pages 1188-2199 PMLR, 2016.
[15] Mark Briers, Arnaud Doucet, and Simon Maskell. Smoothing algorithms for state staeespace models. Annals of the Institute of Statistical Mathematics, 62(():61-1))) 19, 2010.
[16] Pieralberto Guarniero, Adam M. Johansen, and Anthony Lee. The iterated auxiliary particle filter. Journal ofthe the American Stafistical Association, 162656)) 2017.
[17] Michael K. Pitt and Neil Shephard. Filtering via simulation: Auxiliary particle filters. Journal of the American Statistical Association, 94(446):590-599, 1999.
[18] Fredrik Lindsten, Jouni Helske, and Matti Vihola. Graphical model inference: Sequential Monte Carlo meets deterministic approximations, Advances in Neural Information Processing Systems,
31, 2018.
11

---

[19] Tim C. Clapp and Simon J. Godsill. Fixed-lag smoothing using sequential importance sampling. In José-Miguel Bernardo, James Berger, Philip Dawid, and Adrian Smith, editors, Bayesian Statistics 6, pages 743-751. Citeseer, 1999.
[20] Ming Lin, Rong Chen, and Jun Liu. Lookahead strategies for sequential Monte Carlo. Statistical Science, 28(1):69 000999942299 2013.
[21] Milton Abramowitz and Irene Stegun. Handbook of mathematical functions with formulas, graphs, and mathematical tables, volume 55. US Government printing office, 1964.
[22] Adam M Johansen and Arnaud Doucet. A note on auxiliary particle filters. Statistics & Probability Letters, (8(12):1498-15555 2008.
[23] Shakir Mohamed and Balaji Lakshminarayanan. Learning in implicit generative models. arXiv preprint arXiv: 161.:0361000383., 2016.
[24] Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012.
[25] Pierre Del Moral, Arnaud Doucet, and Sumeetpal Singh. Forward smoothing using sequential Monte Carlo. Technical Report CUEDIF-IFTENGTTR 638, Cambridge University, 2010.
[26] Joonha Park and Edward L. Ionides, Inference on nigh-dimensionne implicit dynamic models using& guided intermediate resampling filter. Statistics and Computing, 3975):247 2020.
[27] Arnaud Doucet, Mark Briers, and Stéphane Sénécal. Efficient block sampling strategies for sequential Monte Carlo methods. Journal of Computational and Graphical Statistics, 15(3): 693-711, 2006.
[28] Jeremy Heng, Adrian N. Bishop, George Deligiannidis, and Arnaud Doucet. Controlled sequential Monte Carlo. The Annals of Statistics, (8(55):204429222 Oct 2020. ISSN 0090-5364. doi: 2914
[29] Heiko Zimmermann, Hao Wu, Babak Esmacili, and Jan-Willem van de Meent. Nested vari- ational inference. Advances in Neural Information Processing Systems, 34:20423-20435, 2021.
[30] Pierre Del Moral and Lawrence M. Murray. Sequential Monte Carlo with highly informative observations. SIAM/ASA Journal on Uncertainty Quantification, 3(1):969 00099999,900 2015.
[31] Wans-Christian Ruiz and Hilbert J. Kappen. Particle smoothing for hidden diffusion processes: Adaptive path integral smoother. IEEE Transactions on Signal Processing, 65(12):3191-322 2017.
[32] Rajesh Ranganath, Sean Gerrish, and David Blei. Black box variational inference. In Interna- tional Conference on Artificial Intelligence and Statistics, pages 814 822. PMLR, 2014.
[33] Matthew D. Hoffman, David Blei, Chong Wang, and John Paisley. Stochastic variational inference. Journal of Machine Learning Research, 2013.
[34] Diederik Kingma and Max Welling. Auto-encoding variational Bayes. In 2nd International Conference on Learning Representations, 2014.
[35] Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. In 4th International Conference on Learning Representations, 2016.
[36] Christian Naesseth, Fredrik Lindsten, and Thomas Schon. Nested sequential monte carlo methods. In International Conference on Machine Learning, pages 1292-1301. PMLR, 2015.
[37] Antonio Moretti, Zizhao Wang, Luhuan Wu, Iddo Drori, and Itsik Pe'er. Variational objectives for Markovian dynamics with backward simulation. In ECAI 2020, pages 1371-1378. IOS Press, 2020.
12

---

[38] Antonio Moretti, Zizhao Wang, Luhuan Wu, and Itsik Pe'er. Smoothing nonlinear variational objectives with sequential Monte Carlo. ICLR Workshop: Deep Generative Models for Highly Structured Data, 2019.
[39] Geon- Ccon-Hyeong Kim, Youngsoo Jang, Hongseok Yang, and Kee-Eung Kim. Variational inference for sequential data with future likelihood estimates. In International Conference Machine Learning, pages 5296-5305. PMLR, 2020.
[40] Siddhartha Chib, Yasuhiro Omori, and Manabu Asai. Multivariate stochastic volatility. In Handbook of financial time series, pages 365-400. Springer, 2009.
[41] Neil Gordon, David Salmond, and Adrian Smith. Novel approach to nonlinear/non-ggae gaussiian Bayesian state estimation. IEE Proceedings F (Radar and Signal Processing). 140(2):107-113, April 1993.
[42] Ronald Aylmer Fisher. Statistical methods for research workers. In Breakthroughs in statistics, pages 66-70. Springer, 1992.
[43] Alan L. Hodgkin and Andrew F. Huxley. A quantitative description of membrane current and its application conduction and excitation in nerve. The Journal of Physiology, 1 117(4):500, 1952.
[44] Peter Dayan and Laurence F. Abbott. Theoretical neuroscience: computational and mathemati- cal modeling of neural systems. MIT press, 2005.
[45] Justin M. Kita and R. Mark Wightman. Microelectrodes for studying neurobiology. Current Opinion in Chemical Biology. 12(5):491-496. 2008.
[46] Darcy S. Peterka, Hiroto Takahashi, and Rafael Yuste. Imaging voltage in neurons. Neuron, 69 (1):9-21, 2011.
[47] Quentin J.M. Huys and Liam Paninski. Smoothing of, and parameter estimation from, noisy biophysical recordings. PLoS computational biology, 5(5)xe1000379 2009.
[48] Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density- estimation. Advances in neural information processing systems, 33:4905-4916, 2020.
[49] Robert Price. useful theorem for nonlinear devices having Gaussian inputs. IRE Transactions on Information Theory, 4(2):69-72, 1958.
[50] Georges Bonnet. Transformations des signaux aléatoires travers les systemes non linéaires sans mémoire. In Annales des Télécommunicattonss, volume 19-9, pages 203-220. Springer, 1964.
[51] Tim Salimans and David A. Knowles. Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Analysis, 8(4):837-882, 2013.
[52] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning, pages 1278-1286. PMLR, 2014.
[53] Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine Learning, 8(3):229-25) 256. 1992.
[54] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018, URL com/ google/jax.
[55] Diederik Kingma, Jimmy Ba, Yoshua Bengio, and Yann LeCun. Adam: method for stochastic optimization. In 3rd International Conference Learning Representations, 2015.
[56] Charles W. Dunnett. Pairwise multiple comparisons the unequal variance case. Journal of the American Statistical Association, 75(372):796-80) 1980.
13

---

[57] Derek C. Sauder and Christine E. DeMars. An updated recommendation for multiple compar- isons. Advances in Methods and Practices in Psychological Science, 2(1):26-44, 2019.
[58] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz, Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[59] Marco Fraccaro, Søren Kaac Sønderby, Ulrich Paquet, and Ole Winther. Sequential neural models with stochastic layers. Advances in neural information processing systems, 29, 2016.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] We present a method build on FIVO [8] to include smoothing information.
(b) Did you describe the limitations of your work? [Yes] We especially highlight the per- sistent problem of biased gradient estimators in non-trivial models, and the dependency on the variational family used for each set of function approximators.
(c) Did you discuss any potential negative societal impacts of your work? [No] We foresee no negative social impacts arising from this work.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Sketch proofs are included in the main paper, and expanded proofs and commentaries are included in the supplement.
(b) Did you include complete proofs of all theoretical results? [Yes] See above.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] Code is available publicly at the URL listed.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Extensive written configurations included the supple- ment. Weights and Biases was used for experimental logging. Code released contains hyperparameters for reproduction of results.
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We list the types of nodes and runtimes the algorithms presented.
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "e3MWrO6SwYD", "reviewer": "Reviewer_4LSp", "summary": "Users of sequential Monte Carlo algorithms have many choices to make, including the proposal distributions and the sequence of intermediate target distributions. An appealing approach is to attempt to learn optimal proposals and intermediate targets by \nstochastic gradient descent; this can be formulated within a variational inference framework, where the SMC algorithm can be understood\neither as *being* the variational family, or as a technique for tightening an existing variational bound. Although researchers have had some success applying this approach to the problem of learning proposals, learning intermediate targets is more challenging: they affect the objective's value only through their impact on SMC's resampling steps, which are ignored by commonly used biased gradient estimators.\n\nThis paper presents a new method, SIXO, for learning the intermediate target distributions in sequential Monte Carlo (or rather, learning twisting functions that tilt filtering distributions toward the true smoothing distributions, which are the optimal intermediate targets in a state-space model). The key innovation is to recognize that the optimal twisting function in this setting is a density ratio, and to apply density ratio estimation techniques. In particular, the authors propose to train a classifier to distinguish between samples from $p(x_{t}) p(y_{t+1:T})$ and $p(x_{t}) p(y_{t+1:T} \\mid x_t)$. The logits from the trained classifier can be understood as computing, up to a constant, the log density ratio needed for twisting. The authors further propose parameterizing the classifier with a recurrent architecture, reading the sequence of $y$'s backward from the end.\n\nIn experiments, the authors show that SIXO outperforms methods that do not use twisting, and that their density-ratio estimation scheme outperforms an unbiased gradient estimator for the joint optimization of proposal, model, and twist function parameters.", "questions": "1. In Figure 3, why is resampling so much more frequent in the SIXO plot than in the BPF plot? From the trajectories I would guess that ESS is relatively high throughout the SIXO sweep, and sometimes dips quite low in the BPF sweep, which should lead to the opposite behavior (more resampling in BPF).\n\n2. The related work draws connections to APF, iAPF, cSMC, etc. and explains that SIXO is novel with respect to them (e.g., it supports model learning). But if I am not trying to learn model parameters, and am just trying to get better intermediate targets, how should I understand the relative benefits and disadvantages of these methods, compared to SIXO?", "limitations": "Yes, I think the paper adequately addresses the limitations. (However, see Weakness 2 -- I think additional context could help readers make better sense of the experimental results.)", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. This is a very nice paper that proposes an appealing solution to a long-standing and important problem. Although framed in the context of model-learning and variational inference, the technique seems like it would also be useful for improving SMC inference even when the model is not learned.\n\n2. The idea to treat the twisting function as a density ratio estimator gives rise to a simple and efficient training procedure.\n\n3. Parameterizing twisting functions as recurrent networks is a smart choice that greatly reduces computational complexity.\n\n4. The paper is very nicely written with well-made and informative figures. I especially liked Figure 2, which clearly shows the problem with FIVO: the good proposals it learns are rendered useless by resampling against inaccurate intermediate targets.\n\n\nWeaknesses:\n\n1. Because the twisting functions are trained on samples of (x, y) from the generative model, they may work less well when the true observations look quite different from the synthetic data. (This may be less of an issue in the setting where the model is learned to fit the true observations, however.)\n\n2. I don't have a great sense of how hard the inference problems from the experiments are; it might have been nice to report results from the IWAE baseline on all of them, to help readers understand to what extent resampling is even necessary on these problems.\n\n\nNot weaknesses, but minor suggestions: \n\n* The paper could do more to highlight the challenge that the DRE solution overcomes. The move from FIVO / VSMC to what the paper calls \"SIXO-u\" is quite small; FIVO/VSMC jointly optimize any parameters of the SMC algorithm, and so of course we could introduce learned parameters into the intermediate targets. The challenge is that learned targets affect the SMC algorithm only through the resampling steps, which are not differentiable; as a result, all signal for improving the intermediate targets' parameters comes from high-variance score function estimator terms, which previous papers have advocated dropping during training. This explains why there isn't a \"SIXO-biased\" that just optimizes the SIXO objective directly with a biased estimator -- the biased estimator does not enable training of the intermediate targets.\n\n* On L145-148 a reference to the appendix where the architecture is more fully explained might be nice. (A possible misunderstanding readers might have is to think X is an input to the recurrent part of the network, but of course it is important that it not be--you run the recurrent part once to get the embeddings, then score many X's using those embeddings during SMC.)\n\n* Possible connection for related work -- Nested Variational Inference (https://proceedings.neurips.cc/paper/2021/hash/ab49b208848abe14418090d95df0d590-Abstract.html) also attempts to learn intermediate targets in SMC-like algorithms, and also tries to do it using a separate objective for training them that is not understood as part of one overall variational bound. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KMXh20TCZcl", "reviewer": "Reviewer_7TcZ", "summary": "The paper proposes Smoothing Inference for Twisted Objectives (SIXO) for sequential Monte Carlo inference using approximations of the full posterior as the target distribution at each stage.  Importance weights are computed by learning approximations of the posterior (up to proportionality).  These approximations are learned by using variational backwards methods to solve for twisting functions that approximate the predictive distribution on future measurements, given the current state.  The authors propose learning the twists by casting the ratio of backwards messages and latent marginals as a density ratio estimation problem, leading to SIXO-DRE.  Empirical results are presented on small-scale simulation models and are compared to Filtering Variational Objectives (FIVO).", "questions": "The paper argues that target distributions in SMC is a design choice, yet the target distributions of SMC are uniquely defined to be the sequence of marginal or joint filter distributions (i.e. conditional on past observations only).  Under what circumstances are these targets a valid choice in algorithm development? ", "limitations": "The authors do not clearly discuss limitations of the methodology.  This reviewer sees no obvious negative societal impact of the proposed work.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**STRENGTHS**\nThe paper is well-written, well-motivated, and empirical results are sufficient for the stated claims.  The content is technically sound and appears to be without significant errors.  Figures are particularly well prepared, easily readable fonts, lines, legends, and colors.  Overall, it is a strong paper.\n\n**WEAKNESSES**\nThe argument for choosing posteriors as the target—instead of filter distributions—largely relies on a non-gradual shift in the target distributions.  The authors motivate this with the pathological case of a drift diffusion whereby a single observation is obtained at the final timestep.  However, typical dynamical systems receive measurements along with latent states at each time, and thus filters gradually shift.  In such settings as the stochastic volatility model one does not observe a strong benefit over FIVO, for example.  As a result, it is unclear to what extent SIXO is beneficial in typical settings where observations are dense in the time domain.\n\nThe consistency results in Proposition 1 are largely vacuous.  Assumptions of the propositions state that the space of twists $\\mathcal{R}$ must include the true predictive distribution, and that the space of proposals $\\mathcal{Q}$ must include the true posterior (conditioned on the previous state).  These conditions cannot be verified in general and typically hold only in trivial settings.\n\nFull posterior target distributions are impractical in the setting where data arrive in a streaming fashion with inference being updated as new data arrive.  Such a setting is the most common one for SMC and related inference for dynamical systems.  The proposed SIXO requires static data for updating backwards messages, and so cannot be applied in streaming scenarios.  This could not, for example, be used in a control scenario, where particle filters are often applied.  The authors do not discuss this limitation and instead rely on static data for experiments and model parameter learning.\n\nDetailed comments below:\n* Line 157 : Typo, change $p_{\\theta}(x_t \\mid y_{t:1:T})$ to $p_{\\theta}(x_t \\mid y_{t+1:T})$\n* The authors propose SIXO-u then state that gradient estimators are high variance and thus impractical.  Yet there exist numerous variance reduction techniques that can be applied in this setting (indeed, FIVO uses such approaches).  It seems that the authors prematurely abandon this idea.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "K2pF0otX0MS", "reviewer": "Reviewer_L368", "summary": "The paper introduces a new form of sequential Monte Carlo for particle smoothing problems. The basic idea is to learn the twisting functions that convert the filtering distributions to approximate smoothing distributions using the density estimation ratio trick. This provides a potentially tight bound on the marginal likelihood.\n", "questions": "Does the approach scale to high dimensional settings such as, for example, natural video processing?\n", "limitations": "The paper would benefit from a longer discussion section, discussing potential limitations and failure cases. I cannot see any negative societal impacts.\n", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The basic idea is solid and simple\n- The method has appealing theoretical properties. The tightness of the bound is a clear benefit over competing SMC approaches.\n- The empirical evaluation covers several interesting timeseries problems and compares performance with relevant baselines. The method clearly outperforms the baselines.\n- The paper is clear and well-written\n\nWeaknesses:\n- It would have been interesting to see the method applied to high dimensional problems\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept