Original: S MOOTHING N ONLINEAR V ARIATIONAL O BJECTIVES WITH S EQUENTIAL M ONTE C ARLO
Revised: S MOOTHING N ONLINEAR V ARIATIONAL O BJECTIVES WITH S EQUENTIAL M ONTE C ARLO

Original: A BSTRACT
Revised: A BSTRACT

Original: The task of recovering nonlinear dynamics and latent structure from a population recording is a challenging problem in statistical neuroscience motivating the development of novel techniques in time series analysis.
Revised: The task of recovering nonlinear dynamics and latent structure from a population recording is a challenging problem in statistical neuroscience motivating the development of novel techniques in time series analysis.

Original: Recent work has focused on connections between Variational Inference and Sequential Monte Carlo for performing inference and parameter estimation on sequential data.
Revised: Recent work has focused on connections between Variational Inference and Sequential Monte Carlo for performing inference and parameter estimation on sequential data.

Original: Inspired by this work, we present a framework to develop Smoothed Variational Objectives (SVOs) that condition proposal distributions on the full time-ordered sequence of observations.
Revised: Inspired by this work, we present a framework to develop Smoothed Variational Objectives (SVOs) that condition proposal distributions on the full time-ordered sequence of observations.

Original: SVO maintains both expressiveness and tractability by sharing parameters of the evolution term between the proposal and target.
Revised: SVO maintains both expressiveness and tractability by sharing parameters of the transition function between the proposal and target.

Original: We apply the method to several dimensionality reduction/expansion tasks and examine the dynamics learned with a quantitative metric.
Revised: We apply the method to several dimensionality reduction/expansion tasks and examine the dynamics learned with a quantitative metric.

Original: SVO performs favorably against the state of the art.
Revised: SVO performs favorably against the state of the art.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Conductance based models of excitable cells are widely used in neuroscience to describe the spiking activity of individual neurons.
Revised: Conductance based models of excitable cells are widely used in neuroscience to describe the spiking activity of individual neurons.

Original: It is thought that neural computation is explained by dynamics and that these dynamics often exist in a lower or higher dimensionality than that of the recorded neural populations (Paninski & Cunningham, 2017).
Revised: It is thought that neural computation is explained by dynamics and that these dynamics often exist in a lower or higher dimensionality than that of the recorded neural populations (Paninski & Cunningham, 2017).

Original: To extract the information encoded in ﬁring activity, it is paramount to develop models that allow for tractable analysis of neural data.
Revised: To extract the information encoded in ﬁring activity, it is paramount to develop models that allow for tractable analysis of neural data.

Original: For example, experimentalists who have access to a single-dimensional observation such as a voltage recording are faced with the task of recovering the multidimensional nonlinear latent dynamics and trajectories through a higher dimensional space that describe the system of interest.
Revised: For example, experimentalists who have access to a single-dimensional observation such as a voltage recording are faced with the task of recovering the multidimensional nonlinear latent dynamics and trajectories through a higher dimensional space that describe the system of interest.

Original: There is a large body of work for inferring latent trajectories for data governed by nonlinear dynamics or observations (Archer et al., 2015; Krishnan et al., 2015; Pandarinath et al., 2017; Diaz et al., 2019).
Revised: There is a large body of work for inferring latent trajectories for data governed by nonlinear dynamics or observations (Archer et al., 2015; Krishnan et al., 2015; Pandarinath et al., 2017; Diaz et al., 2019).

Original: Variational Inference (VI) and Markov Chain Monte Carlo (MCMC) are two popular approaches for performing inference in latent variable models.
Revised: Variational Inference (VI) and Markov Chain Monte Carlo (MCMC) are two popular approaches for performing inference in latent variable models.

Original: Recently connections have been established between both methods that allow for performing inference by deﬁning a ﬂexible variational family of ﬁltered distributions using Sequential Monte Carlo (SMC) (Le et al., 2018; Maddison et al., 2017; Naesseth et al., 2018a).
Revised: Recently connections have been established between both methods that allow for performing inference by deﬁning a ﬂexible variational family of ﬁltered distributions using Sequential Monte Carlo (SMC) (Le et al., 2018; Maddison et al., 2017; Naesseth et al., 2018a).

Original: Unlike ﬁltering, smoothing refers to the ability to condition states and parameters on the full time-ordered sequence of observations.
Revised: Unlike ﬁltering, smoothing refers to the ability to condition states and parameters on the full time-ordered sequence of observations.

Original: In this framework, we sketch a method to construct variational objectives using recursive backwards sampling algorithms (referred to as particle smoothers) in addition to parameterizing the encoding function on the full time ordered sequence of observations.
Revised: In this framework, we sketch a method to construct variational objectives using recursive backwards sampling algorithms (referred to as particle smoothers) in addition to parameterizing the encoding function on the full time ordered sequence of observations.

Original: Recent literature argues that tighter variational bounds may hurt the proposal learning (Rainforth et al., 2018).
Revised: Recent literature argues that tighter variational bounds may hurt the proposal learning (Rainforth et al., 2018).

Original: More speciﬁcally, in the context of the Importance Weighted AutoEncoder (IWAE), the marginal likelihood estimate becomes exact as the number of particles K → ∞ .
Revised: More speciﬁcally, in the context of the Importance Weighted AutoEncoder (IWAE), the marginal likelihood estimate becomes exact as the number of particles K → ∞ .

Original: This can lead to a degenerated gradient estimate for the proposal with the signal-to-noise ratio shrinking at the rate
Revised: This can lead to a degenerated gradient estimate for the proposal with the signal-to-noise ratio shrinking at the rate

Original: O ( (cid:112) 1 /K )
Revised: O ( (cid:112) 1 /K )

Original: (Rainforth et al., 2018).
Revised: (Rainforth et al., 2018).

Original: (Le et al., 2018) built on this idea to use different objectives for recognition and generative networks.
Revised: (Le et al., 2018) built on this idea to use different objectives for recognition and generative networks.

Original: Here we argue that the theoretical result in the IWAE setting cannot be directly extended to SMC due to the resampling process.
Revised: Here we argue that the theoretical result in the IWAE setting cannot be directly extended to SMC due to the resampling process.

Original: We present theoretical and empirical evidence to show that the degrading signal-to-noise ratio is addressed by the choice of biased gradient estimators.
Revised: We present theoretical and empirical evidence to show that the degrading signal-to-noise ratio is addressed by the choice of biased gradient estimators.

Original: The contributions of this paper are highlighted below.
Revised: The contributions of this paper are highlighted below.

Original: First, we describe a factorized proposal distribution that (i) disentangles the transition terms from the terms that represent the encoding of the data and (ii) shares variational parameters of the transition function with the target distribution.
Revised: First, we describe a factorized proposal distribution that (i) disentangles the transition terms from the terms that represent the encoding of the data and (ii) shares variational parameters of the transition function with the target distribution.

Original: We present results illustrating the positive effect of the number of particles on the signal-to-noise ratio for the transition, encoder and decoder parameters within our factorization.
Revised: We present results illustrating the positive effect of the number of particles on the signal-to-noise ratio for the transition, encoder and decoder parameters within our factorization.

Original: An important way to ascertain the quality of the learned dynamics is to use the transition terms in the generative model to propagate the system forwards without input data and then to make observation predictions.
Revised: An important way to ascertain the quality of the learned dynamics is to use the transition terms in the generative model to propagate the system forwards without input data and then to make observation predictions.

Original: We show that our smoothed proposal generates an improved estimate of the latent states as measured by the ability of the target to more accurately predict observations using the dynamics learned.
Revised: We show that our smoothed proposal generates an improved estimate of the latent states as measured by the ability of the target to more accurately predict observations using the dynamics learned.

Original: The remainder of this paper is organized as follows.
Revised: The remainder of this paper is organized as follows.

Original: Section 2 summarizes related work on VI and
Revised: Section 2 summarizes related work on VI and

Original: SMC.
Revised: SMC.

Original: Section 3 gives preliminaries and deﬁnes the proposal while analyzing the signal to noise ratio of the gradient.
Revised: Section 3 gives preliminaries and deﬁnes the proposal while analyzing the signal to noise ratio of the gradient.

Original: We then present results on both ﬁltered and smoothed proposal densities with two benchmark datasets and single cell electrophysiology data from the Allen Institute in Section 4.
Revised: We then present results on both ﬁltered and smoothed proposal densities with two benchmark datasets and single cell electrophysiology data from the Allen Institute in Section 4.

Original: Section 5 concludes.
Revised: Section 5 concludes.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: There is a large body of work on ﬁtting State Space Models (SSMs) of increasing complexity using
Revised: There is a large body of work on ﬁtting State Space Models (SSMs) of increasing complexity using

Original: VI.
Revised: VI.

Original: These methods can be separated into two classes.
Revised: These methods can be separated into two classes.

Original: Pure VI methods optimize a lower bound to the log likelihood parameterized by deﬁning a variational family.
Revised: Pure VI methods optimize a lower bound to the log likelihood parameterized by deﬁning a variational family.

Original: Hybrid methods use SMC to construct an estimate for the log marginal likelihood and optimize a surrogate lower bound.
Revised: Hybrid methods use SMC to construct an estimate for the log marginal likelihood and optimize a surrogate lower bound.

Original: We review the details of both techniques in the preliminaries section and outline the contributions of related methods below.
Revised: We review the details of both techniques in the preliminaries section and outline the contributions of related methods below.

Original: Gaussian Process Factor Analysis (Yu et al., 2009) (GPFA) is a statistical model that assumes linear time invariant transition and emission functions.
Revised: Gaussian Process Factor Analysis (Yu et al., 2009) (GPFA) is a statistical model that assumes linear time invariant transition and emission functions.

Original: GfLDS (Gao et al., 2016; Archer et al., 2015) is a VI scheme that extends the above by positing a linear dynamical system with nonlinear emission densities.
Revised: GfLDS (Gao et al., 2016; Archer et al., 2015) is a VI scheme that extends the above by positing a linear dynamical system with nonlinear emission densities.

Original: VIND (Diaz et al., 2019) is a VI scheme that permits both transition and emission densities to be represented as arbitrary nonlinearities.
Revised: VIND (Diaz et al., 2019) is a VI scheme that permits both transition and emission densities to be represented as arbitrary nonlinearities.

Original: VIND achieves reuse of the exact generative evolution structure for inference as suggested by the true posterior.
Revised: VIND achieves reuse of the exact generative evolution structure for inference as suggested by the true posterior.

Original: Tractability of the variational approximation is maintained by forming a Laplace approximation to an intractable non Gaussian parent approximate distribution.
Revised: Tractability of the variational approximation is maintained by forming a Laplace approximation to an intractable non Gaussian parent approximate distribution.

Original: The mean and the covariance of the Laplace approximation are obtained via the Fixed Point Iteration (FPI) and the intractable terms are reused within the approximation.
Revised: The mean and the covariance of the Laplace approximation are obtained via the Fixed Point Iteration (FPI) and the intractable terms are reused within the approximation.

Original: We deﬁne smoothing as a method to infer latent state z t using observations from the complete trial x 1: T , including points to the future of t < t (cid:48) ∈ { t + 1 : T } .
Revised: We deﬁne smoothing as a method to infer latent state z t using observations from the complete trial x 1: T , including points to the future of t < t (cid:48) ∈ { t + 1 : T } .

Original: GPFA, GfLDS and VIND are all smoothers in that they use the full sequence of observations to infer the current latent state.
Revised: GPFA, GfLDS and VIND are all smoothers in that they use the full sequence of observations to infer the current latent state.

Original: An approach to increasing model capacity is the development of tighter bounds than the ELBO.
Revised: An approach to increasing model capacity is the development of tighter bounds than the ELBO.

Original: Filtering Variational Objectives (Maddison et al., 2017) (FIVO) and Variational Sequential Monte
Revised: Filtering Variational Objectives (Maddison et al., 2017) (FIVO) and Variational Sequential Monte

Original: Carlo (Naesseth et al., 2018a) are related methods that use SMC to construct a ﬁltered objective function for VI.
Revised: Carlo (Naesseth et al., 2018a) are related methods that use SMC to construct a ﬁltered objective function for VI.

Original: Auto-Encoding Sequential Monte Carlo (Le et al., 2018)
Revised: Auto-Encoding Sequential Monte Carlo (Le et al., 2018)

Original: (AESMC) is a similar method to learn both proposal distributions for SMC that act as variational approximations and generative models using deep neural networks.
Revised: (AESMC) is a similar method to learn both proposal distributions for SMC that act as variational approximations and generative models using deep neural networks.

Original: Unlike GfLDS and VIND, these methods do not require inverting a block-tridiagonal matrix which mixes components of state space through the inverse covariance.
Revised: Unlike GfLDS and VIND, these methods do not require inverting a block-tridiagonal matrix which mixes components of state space through the inverse covariance.

Original: Information from the complete time ordered sequence 1 : T is not applied directly to the transition function to the future of the current time point t < T to infer the latent state.
Revised: Information from the complete time ordered sequence 1 : T is not applied directly to the transition function to the future of the current time point t < T to infer the latent state.

Original: TVSMC (Lawson et al.) is a family of variational objectives that augment the target distribution with a lookahead function to approximate smoothing SMC.
Revised: TVSMC (Lawson et al.) is a family of variational objectives that augment the target distribution with a lookahead function to approximate smoothing SMC.

Original: Temporal difference learning is used to estimate the lookahead function, however this method did not produce results comparable with ﬁltering on complex nonlinear models.
Revised: Temporal difference learning is used to estimate the lookahead function, however this method did not produce results comparable with ﬁltering on complex nonlinear models.

Original:  M ETHODS 3. P RELIMINARIES 3.1. VI AND I MPORTANCE W EIGHTED A UTO -E NCODERS
Revised:  M ETHODS 3. P RELIMINARIES 3.1. VI AND I MPORTANCE W EIGHTED A UTO -E NCODERS

Original: Inference in latent variable models requires marginalizing a generative distribution with respect to hidden variables Z .
Revised: Inference in latent variable models requires marginalizing a generative distribution with respect to hidden variables Z .

Original: [Equation]
Revised: [Equation]

Original: VI describes family of techniques for approximating a solution to the above.
Revised: VI describes family of techniques for approximating a solution to the above.

Original: The idea is to deﬁne a tractable distribution q φ ( Z | X ) and to optimize a lower bound to the log likelihood:
Revised: The idea is to deﬁne a tractable distribution q φ ( Z | X ) and to optimize a lower bound to the log likelihood:

Original: [Equation]
Revised: [Equation]

Original: Tractability and expressiveness of the variational approximation q φ ( Z | X ) are contrasting goals.
Revised: Tractability and expressiveness of the variational approximation q φ ( Z | X ) are contrasting goals.

Original: The distribution q φ ( Z | X ) must be chosen carefully; a poor choice of the variational family deﬁning q φ ( Z | X ) will affect the generative model by pulling the parameters θ towards φ .
Revised: The distribution q φ ( Z | X ) must be chosen carefully; a poor choice of the variational family deﬁning q φ ( Z | X ) will affect the generative model by pulling the parameters θ towards φ .

Original: [Equation]
Revised: [Equation]

Original: Auto Encoding Variational Bayes (Kingma & Welling, 2013) (AEVB) is method to simultaneously encode and decode the parameters { φ, θ } for both q φ ( Z | X ) and p θ ( X , Z ) .
Revised: Auto Encoding Variational Bayes (Kingma & Welling, 2013) (AEVB) is method to simultaneously encode and decode the parameters { φ, θ } for both q φ ( Z | X ) and p θ ( X , Z ) .

Original: When the densities are not in the natural exponential family, it is possible to draw samples from q φ ( Z | X ) to evaluate E q φ [log p θ ( X , Z ) − log q φ ( Z | X )] numerically.
Revised: When the densities are not in the natural exponential family, it is possible to draw samples from q φ ( Z | X ) to evaluate E q φ [log p θ ( X , Z ) − log q φ ( Z | X )] numerically.

Original: We can approximate both L ELBO terms reparameterizing in order to compute the gradients (Kingma & Welling, 2013; Rezende et al., 2014) as follows:
Revised: We can approximate both L ELBO terms reparameterizing in order to compute the gradients (Kingma & Welling, 2013; Rezende et al., 2014) as follows:

Original: [Equation]
Revised: [Equation]

Original: where Z ( i,l ) = µ
Revised: where Z ( i,l ) = µ

Original: [Equation]
Revised: [Equation]

Original: + Σ
Revised: + Σ

Original: [Equation]
Revised: [Equation]

Original: 2 φ
Revised: 2 φ

Original: [Equation]
Revised: [Equation]

Original: and (cid:15) ( i,
Revised: and (cid:15) ( i,

Original: [Equation]
Revised: [Equation]

Original: ∼ N
Revised: ∼ N

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Building upon this, the Importance Weighted Auto Encoder (Burda et al., 2015) (IWAE) constructs tighter bounds than the L AEVB through mode averaging as opposed to mode matching.
Revised: Building upon this, the Importance Weighted Auto Encoder (Burda et al., 2015) (IWAE) constructs tighter bounds than the L AEVB through mode averaging as opposed to mode matching.

Original: The idea is to estimate L AEVB by drawing K samples from a proposal distribution and to average probability ratios:
Revised: The idea is to estimate L AEVB by drawing K samples from a proposal distribution and to average probability ratios:

Original: [Equation]
Revised: [Equation]

Original: where Z ( k ) ∼ q φ ( Z | X ) (6)
Revised: where Z ( k ) ∼ q φ ( Z | X ) (6)

Original: It can be shown that log p θ ( X ) ≥ L k +1 ≥ L k ≥ L 1 .
Revised: It can be shown that log p θ ( X ) ≥ L k +1 ≥ L k ≥ L 1 .

Original: The L IWAE reduces to the L AEVB when K =and approaches the true log probability of the data as K → ∞ .
Revised: The L IWAE reduces to the L AEVB when K =and approaches the true log probability of the data as K → ∞ .

Original: 3.1. S TATE S PACE M ODELS
Revised: 3.1. S TATE S PACE M ODELS

Original: State space models (SSMs) describe a time series of observations
Revised: State space models (SSMs) describe a time series of observations

Original: X ≡ { x 1 , . . .
Revised: X ≡ { x 1 , . . .

Original: x T } , x t ∈ R d
Revised: x T } , x t ∈ R d

Original: x dependent upon a time series of latent variables Z ≡ { z 1 , . . .
Revised: x dependent upon a time series of latent variables Z ≡ { z 1 , . . .

Original: z T } , z T ∈ R
Revised: z T } , z T ∈ R

Original: d
Revised: d

Original: z that evolve according to stochastic dynamics.
Revised: z that evolve according to stochastic dynamics.

Original: We are interested in dynamical systems which exhibit the following dependency:
Revised: We are interested in dynamical systems which exhibit the following dependency:

Original: [Equation]
Revised: [Equation]

Original: where g θ is an observation model whose parameters are functions of the latent state z t .
Revised: where g θ is an observation model whose parameters are functions of the latent state z t .

Original: The normalization constant is denoted c θ and F θ denotes the latent evolution described with Markov Chain:
Revised: The normalization constant is denoted c θ and F θ denotes the latent evolution described with Markov Chain:

Original: [Equation]
Revised: [Equation]

Original: Marginalization with respect to z becomes intractable when F θ ( Z ) is deﬁned by a nonlinear function or g ( x t | z t ) is non-Gaussian.
Revised: Marginalization with respect to z becomes intractable when F θ ( Z ) is deﬁned by a nonlinear function or g ( x t | z t ) is non-Gaussian.

Original: 3.1. S EQUENTIAL M ONTE C ARLO
Revised: 3.1. S EQUENTIAL M ONTE C ARLO

Original: Sequential Monte Carlo (SMC) methods factorize an intractable distribution p θ ( X , Z ) (referred to as the target distribution) into a distribution of increasing probability spaces.
Revised: Sequential Monte Carlo (SMC) methods factorize an intractable distribution p θ ( X , Z ) (referred to as the target distribution) into a distribution of increasing probability spaces.

Original: Sequential Importance
Revised: Sequential Importance

Original: Sampling (SIS) methods perform importance sampling sequentially by deﬁning importance weights w ( k ) t at time t for sample k as follows where K samples are drawn from the proposal distribution q :
Revised: Sampling (SIS) methods perform importance sampling sequentially by deﬁning importance weights w ( k ) t at time t for sample k as follows where K samples are drawn from the proposal distribution q :

Original: [Equation]
Revised: [Equation]

Original: Various resampling schemes exist so that the samples which are referred to as particles are focused on promising regions of state space.
Revised: Various resampling schemes exist so that the samples which are referred to as particles are focused on promising regions of state space.

Original: Sequential Importance Resampling and SMC achieve this goal by resampling particles according to their importance sampling weights:
Revised: Sequential Importance Resampling and SMC achieve this goal by resampling particles according to their importance sampling weights:

Original: [Equation]
Revised: [Equation]

Original: At the last time step
Revised: At the last time step

Original: [Equation]
Revised: [Equation]

Original: the posterior can be evaluated by averaging over sample trajectories to approximate the functional integral using the empirical measure below:
Revised: the posterior can be evaluated by averaging over sample trajectories to approximate the functional integral using the empirical measure below:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: The weights can now be used to construct an unbiased estimate for the marginal likelihood:
Revised: The weights can now be used to construct an unbiased estimate for the marginal likelihood:

Original: [Equation]
Revised: [Equation]

Original: 3.1. A UTO E NCODING S EQUENTIAL M ONTE C ARLO
Revised: 3.1. A UTO E NCODING S EQUENTIAL M ONTE C ARLO

Original: An important insight of (Maddison et al., 2017; Le et al., 2018) is that the SMC algorithm is deterministic conditioning on ( Z (1: K ) 1: T , A (1: K ) 1: T − 1 ) .
Revised: An important insight of (Maddison et al., 2017; Le et al., 2018) is that the SMC algorithm is deterministic conditioning on ( Z (1: K ) 1: T , A (1: K ) 1: T − 1 ) .

Original: As a result, the importance sampling density can be reparameterized to act as a variational distribution that can be encoded:
Revised: As a result, the importance sampling density can be reparameterized to act as a variational distribution that can be encoded:

Original: [Equation]
Revised: [Equation]

Original: This gives a way of constructing a ﬁltered cost function for simultaneous model inference and learning.
Revised: This gives a way of constructing a ﬁltered cost function for simultaneous model inference and learning.

Original: The cost is constructed by running SMC to obtain an estimate of the marginal log likelihood.
Revised: The cost is constructed by running SMC to obtain an estimate of the marginal log likelihood.

Original: As with the IWAE
Revised: As with the IWAE

Original: [Equation]
Revised: [Equation]

Original: the estimate L SMC deﬁned below converges to the true log likelihood
Revised: the estimate L SMC deﬁned below converges to the true log likelihood

Original: L SMC → log p ( x 1: T ) as K → ∞ .
Revised: L SMC → log p ( x 1: T ) as K → ∞ .

Original: [Equation]
Revised: [Equation]

Original: 3. S MOOTHING V ARIATIONAL O BJECTIVES
Revised: 3. S MOOTHING V ARIATIONAL O BJECTIVES

Original: We build upon AESMC and FIVO to design a variational objective based on the marginal likelihood estimate constructed using SMC.
Revised: We build upon AESMC and FIVO to design a variational objective based on the marginal likelihood estimate constructed using SMC.

Original: One way to deﬁne a Smoothed Variational Objective (SVO) is to explicitly modify the target density by conditioning latent states on future observations.
Revised: One way to deﬁne a Smoothed Variational Objective (SVO) is to explicitly modify the target density by conditioning latent states on future observations.

Original: Our approach is to smooth the proposal distribution by conditioning on full observations thereby implicitly modifying the target through its shared factorization with the proposal.
Revised: Our approach is to smooth the proposal distribution by conditioning on full observations thereby implicitly modifying the target through its shared factorization with the proposal.

Original: 3.2. P ARAMETERIZING THE P ROPOSAL D ISTRIBUTION
Revised: 3.2. P ARAMETERIZING THE P ROPOSAL D ISTRIBUTION

Original: We begin by considering a proposal distribution of the form:
Revised: We begin by considering a proposal distribution of the form:

Original: [Equation]
Revised: [Equation]

Original: where the proposal density factorizes into separate functions for evolution of the latent dynamics and encodings of the data.
Revised: where the proposal density factorizes into separate functions for evolution of the latent dynamics and encodings of the data.

Original: [Equation]
Revised: [Equation]

Original: We take ψ : R d z → R d z and γ : R d x → R d z as nonlinear time invariant functions represented with deep neural networks.
Revised: We take ψ : R d z → R d z and γ : R d x → R d z as nonlinear time invariant functions represented with deep neural networks.

Original: The covariances Σ and Λ can be invariant trainable parameters or nonlinear functions of the latent space.
Revised: The covariances Σ and Λ can be invariant trainable parameters or nonlinear functions of the latent space.

Original: Unlike Q SMC in Eq.
Revised: Unlike Q SMC in Eq.

Original: (15), Eq.
Revised: (15), Eq.

Original: (17) models dynamics that depend on latent states instead of time.
Revised: (17) models dynamics that depend on latent states instead of time.

Original: This proposal choice is also advantageous because the transition term of the recognition model q ϕ ( z ( k ) t | z ( k ) t − 1 ) can be chosen to share the network parameters ϕ deﬁning { ψ, Σ } with the target transition term f ϕ
Revised: This proposal choice is also advantageous because the transition term of the recognition model q ϕ ( z ( k ) t | z ( k ) t − 1 ) can be chosen to share the network parameters ϕ deﬁning { ψ, Σ } with the target transition term f ϕ

Original: ( z t | z t − 1 ) of the generative model.
Revised: ( z t | z t − 1 ) of the generative model.

Original: [Equation]
Revised: [Equation]

Original: This is analogous to the bootstrap ﬁlter (Gordon et al., 1993), however the addition of a new term permits disentangling the transition function from an encoding of the data.
Revised: This is analogous to the bootstrap ﬁlter (Gordon et al., 1993), however the addition of a new term permits disentangling the transition function from an encoding of the data.

Original: As a result the evolution term of the variational posterior is exact retaining both tractability and expressiveness.
Revised: As a result the evolution term of the variational posterior is exact retaining both tractability and expressiveness.

Original: The generative evolution law is thus speciﬁed as follows:
Revised: The generative evolution law is thus speciﬁed as follows:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The decoding term is deﬁned using a deterministic nonlinear rate function υ : R d z → R d x represented with a deep network and a noise model that need not be conjugate.
Revised: The decoding term is deﬁned using a deterministic nonlinear rate function υ : R d z → R d x represented with a deep network and a noise model that need not be conjugate.

Original: Without loss of generality consider a Gaussian emission density:
Revised: Without loss of generality consider a Gaussian emission density:

Original: [Equation]
Revised: [Equation]

Original: The incremental weights are expressed using the following factorization:
Revised: The incremental weights are expressed using the following factorization:

Original: [Equation]
Revised: [Equation]

Original: 3.2. F ORWARD F ILTERING B ACKWARD S MOOTHING
Revised: 3.2. F ORWARD F ILTERING B ACKWARD S MOOTHING

Original: We consider two solutions for sampling smoothed trajectories, one is a bidirectional encoding architecture for smoothing and another is the Forward Filtering
Revised: We consider two solutions for sampling smoothed trajectories, one is a bidirectional encoding architecture for smoothing and another is the Forward Filtering

Original: Backwards Smoothing (FFBS) formula (Kitagawa, 1996).
Revised: Backwards Smoothing (FFBS) formula (Kitagawa, 1996).

Original: To smooth the latent states and ensure that q ( z t ) depends on both z t − 1 and z t +1 , we consider the following recursion:
Revised: To smooth the latent states and ensure that q ( z t ) depends on both z t − 1 and z t +1 , we consider the following recursion:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: FFBS ﬁrst iterates forward to compute the ﬁltered distribution at each time step, reweighting particles with the following backward recursion:
Revised: FFBS ﬁrst iterates forward to compute the ﬁltered distribution at each time step, reweighting particles with the following backward recursion:

Original: [Equation]
Revised: [Equation]

Original: with w ( k ) T | T = ¯ w ( k ) T .
Revised: with w ( k ) T | T = ¯ w ( k ) T .

Original: This allows us to sample from a smoothed posterior.
Revised: This allows us to sample from a smoothed posterior.

Original: When applying FFBS, we ﬁnd two drawbacks.
Revised: When applying FFBS, we ﬁnd two drawbacks.

Original: In contrast to the O ( TK ) complexity of particle ﬁlter, the complexity of FFBS is O ( TK 2 ) , scaling quadratically with the number of particles through the through the ( i, k ) pairwise interactions.
Revised: In contrast to the O ( TK ) complexity of particle ﬁlter, the complexity of FFBS is O ( TK 2 ) , scaling quadratically with the number of particles through the through the ( i, k ) pairwise interactions.

Original: Another limitation includes the fact that the backwards recursion to update weights does not change the support of the sampled particles which may be disjoint from the smoothed posterior.
Revised: Another limitation includes the fact that the backwards recursion to update weights does not change the support of the sampled particles which may be disjoint from the smoothed posterior.

Original: 3.2. E NCODING A RCHITECTURES FOR S MOOTHED V ARIATIONAL O BJECTIVES
Revised: 3.2. E NCODING A RCHITECTURES FOR S MOOTHED V ARIATIONAL O BJECTIVES

Original: We augment the encoding term of the proposal in Eq.
Revised: We augment the encoding term of the proposal in Eq.

Original: (17) using the full time ordered sequence of observations:
Revised: (17) using the full time ordered sequence of observations:

Original: [Equation]
Revised: [Equation]

Original: For the case where q t,φ ( z 1 | x 1 ) and q t,φ ( z t | x 1: t ) are factorized Gaussians with conditionals parameterized by time varying networks, simply expanding the network inputs from x t to x 1: T would be expensive.
Revised: For the case where q t,φ ( z 1 | x 1 ) and q t,φ ( z t | x 1: t ) are factorized Gaussians with conditionals parameterized by time varying networks, simply expanding the network inputs from x t to x 1: T would be expensive.

Original: As a result, we extract features from the full time series x 1: T and redeﬁne the input to our encoder function.
Revised: As a result, we extract features from the full time series x 1: T and redeﬁne the input to our encoder function.

Original: To achieve this, an RNN is run both forward and backward,
Revised: To achieve this, an RNN is run both forward and backward,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 28) where e f,t and e b,t are states containing information of x 1: t and x t : T respectively.
Revised: 28) where e f,t and e b,t are states containing information of x 1: t and x t : T respectively.

Original: Then the concatenation of [ e f,T , e b, 1 ] is used to parameterize the initial state distribution q φ ( z 1 | x 1: T ) , and [ e f,t , e b,t ] is fed into the term of the proposal deﬁning the encoder q φ ( z t | x 1: T ) .
Revised: Then the concatenation of [ e f,T , e b, 1 ] is used to parameterize the initial state distribution q φ ( z 1 | x 1: T ) , and [ e f,t , e b,t ] is fed into the term of the proposal deﬁning the encoder q φ ( z t | x 1: T ) .

Original: The nonlinear time invariant evolution function is then applied to transition between states.
Revised: The nonlinear time invariant evolution function is then applied to transition between states.

Original: [Equation]
Revised: [Equation]

Original: The cost of the feature extraction with the bidirectional RNN is O ( T ) , independent of the number of particles unlike the FFBS.
Revised: The cost of the feature extraction with the bidirectional RNN is O ( T ) , independent of the number of particles unlike the FFBS.

Original: This is similar to an approach in FIVO with the exception of a shared parameterization between proposal and target transition.
Revised: This is similar to an approach in FIVO with the exception of a shared parameterization between proposal and target transition.

Original: The objective and marginal likelihood estimate are deﬁned below.
Revised: The objective and marginal likelihood estimate are deﬁned below.

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: 3.2. N ONLINEAR E VOLUTION OF THE C OVARIANCE M ATRIX .
Revised: 3.2. N ONLINEAR E VOLUTION OF THE C OVARIANCE M ATRIX .

Original: For many nonlinear dynamical systems, the evolution of the latent state may depend on a noise term which itself may evolve stochastically.
Revised: For many nonlinear dynamical systems, the evolution of the latent state may depend on a noise term which itself may evolve stochastically.

Original: In order to model a covariance matrix that expresses nonlinear z -dependence on the latent space, we face a challenge.
Revised: In order to model a covariance matrix that expresses nonlinear z -dependence on the latent space, we face a challenge.

Original: It is important to ensure that the latent paths are smooth.
Revised: It is important to ensure that the latent paths are smooth.

Original: This is equivalent to stating that the difference between the covariance matrix and a constant matrix C is small.
Revised: This is equivalent to stating that the difference between the covariance matrix and a constant matrix C is small.

Original: [Equation]
Revised: [Equation]

Original: An expressive model should permit the covariance to undergo nonlinear evolution.
Revised: An expressive model should permit the covariance to undergo nonlinear evolution.

Original: One solution is to parameterize the covariance with a constant plus a scalar times a symmetric matrix whose components are a nonlinear function of the latent state.
Revised: One solution is to parameterize the covariance with a constant plus a scalar times a symmetric matrix whose components are a nonlinear function of the latent state.

Original: [Equation]
Revised: [Equation]

Original: In the experiments we take α = 1 e − 1 , C
Revised: In the experiments we take α = 1 e − 1 , C

Original: = I · σ 2 where σ 2 is a trainable variable and the components of Σ ( z t ) as the output of a deep network.
Revised: = I · σ 2 where σ 2 is a trainable variable and the components of Σ ( z t ) as the output of a deep network.

Original: This permits dynamic proposal distributions whose entropy evolves as the Markov chain transitions between states.
Revised: This permits dynamic proposal distributions whose entropy evolves as the Markov chain transitions between states.

Original: At different positions in latent space, the system can suppress or enhance its sensitivity to noise.
Revised: At different positions in latent space, the system can suppress or enhance its sensitivity to noise.

Original: We refer to this as a Locally
Revised: We refer to this as a Locally

Original: Linear Covariance Matrix (LLCM).
Revised: Linear Covariance Matrix (LLCM).

Original: 3. G RADIENT ESTIMATORS AND S IGNAL - TO - NOISE R ATIO
Revised: 3. G RADIENT ESTIMATORS AND S IGNAL - TO - NOISE R ATIO

Original: Because L SVO converges to the true log marginal likelihood, one would naturally think about increasing the number of particles K to get a better surrogate objective.
Revised: Because L SVO converges to the true log marginal likelihood, one would naturally think about increasing the number of particles K to get a better surrogate objective.

Original: However, (Rainforth et al., 2018) points out the detrimental effect of large K on learning the inference network by reducing the signal-to-noise-ratio of the gradient estimator.
Revised: However, (Rainforth et al., 2018) points out the detrimental effect of large K on learning the inference network by reducing the signal-to-noise-ratio of the gradient estimator.

Original: Formally, for a gradient estimator ∆ K constructed by K particles, the signal-to-noise ratio (SNR) is deﬁned as:
Revised: Formally, for a gradient estimator ∆ K constructed by K particles, the signal-to-noise ratio (SNR) is deﬁned as:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Intuitively, a vanishing SNR implies that the gradient estimator reduces to pure noise, hence providing no information to learning.
Revised: Intuitively, a vanishing SNR implies that the gradient estimator reduces to pure noise, hence providing no information to learning.

Original: In the IWAE setting, the SNR of the inference network decreases with rate O (1 / √ K )
Revised: In the IWAE setting, the SNR of the inference network decreases with rate O (1 / √ K )

Original: (Rainforth et al., 2018); (Le et al., 2018) extends the result to the SMC setting without providing the theoretical evidence.
Revised: (Rainforth et al., 2018); (Le et al., 2018) extends the result to the SMC setting without providing the theoretical evidence.

Original: Here, we argue that (Le et al., 2018) neglects that they are using a biased gradient estimator, which in fact addresses the issue of degrading SNR.
Revised: Here, we argue that (Le et al., 2018) neglects that they are using a biased gradient estimator, which in fact addresses the issue of degrading SNR.

Original: We consider the following three stochastic gradient estimators.
Revised: We consider the following three stochastic gradient estimators.

Original: A full description would be given in the Appendix.
Revised: A full description would be given in the Appendix.

Original: The unbiased estimator, denoted by ∇L K + C ATEGORICAL , which takes into account the gradient in the resampling procedure.
Revised: The unbiased estimator, denoted by ∇L K + C ATEGORICAL , which takes into account the gradient in the resampling procedure.

Original: The biased estimator without resampling gradient ∇L K 3.
Revised: The biased estimator without resampling gradient ∇L K 3.

Original: The relaxed gradient estimator, ∇L K + C ONCRETE ( λ ) , which replaces the C ATEGORICAL distribution in the resampling step with C ONCRETE distribution.
Revised: The relaxed gradient estimator, ∇L K + C ONCRETE ( λ ) , which replaces the C ATEGORICAL distribution in the resampling step with C ONCRETE distribution.

Original: Note that as λ → 0 , the CONCRETE distribution approaches the CATEGORICAL .
Revised: Note that as λ → 0 , the CONCRETE distribution approaches the CATEGORICAL .

Original: (Jang et al., 2016), (Maddison et al., 2016).
Revised: (Jang et al., 2016), (Maddison et al., 2016).

Original: For the SNR of ∇L K , we have the following proposition:
Revised: For the SNR of ∇L K , we have the following proposition:

Original: Proposition 1.
Revised: Proposition 1.

Original: Assume that the ﬁrst four moments of w (1) t and ∇ w (1) t are all ﬁnite and their variances are non-zero for t ∈ 1 : T , then the signal-to-noise ratio converges at the following rate:
Revised: Assume that the ﬁrst four moments of w (1) t and ∇ w (1) t are all ﬁnite and their variances are non-zero for t ∈ 1 : T , then the signal-to-noise ratio converges at the following rate:

Original: [Equation]
Revised: [Equation]

Revised: (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (35)

Original: where Z = p θ ( x 1: T ) and Z t = p θ ( x t | x 1: t − 1 ) for t ∈ 1 : T .
Revised: where Z = p θ ( x 1: T ) and Z t = p θ ( x t | x 1: t − 1 ) for t ∈ 1 : T .

Original: Further assuming the resampling bias (cid:80)
Revised: Further assuming the resampling bias (cid:80)

Original: Tt =2 (cid:80) Tt (cid:48) ≥ t +1 E (cid:104) ∇ w 1 t − 1 Z t − 1 · ( w 1 t (cid:48) − Z t (cid:48) ) 2 2 Z 2 t (cid:48) (cid:12)(cid:12)(cid:12) a 1 t − 1 = 1 (cid:105) =
Revised: Tt [Equation]

Original: O (1) leads to SNR K ( θ, φ, ϕ )
Revised: leads to SNR K ( [Equation] [Equation]

Original: =
Revised: [Equation]

Original:  O ( √ K ) .
Revised: = O ( √ K ) .

Original: The formal proof is provided in the Appendix.
Revised: The formal proof is provided in the Appendix.

Original: We add empirical evidence to this result in Section 4.
Revised: We add empirical evidence to this result in Section 4.

Original: For the rest of experiments, we use ∇L K to estimate the gradient.
Revised: For the rest of experiments, we use ∇L K to estimate the gradient.

Original:  E XPERIMENTAL R ESULTS
Revised:  E XPERIMENTAL R ESULTS

Original: It is important to quantify the performance of the hidden evolution in order to understand the quality of the dynamics learned.
Revised: It is important to quantify the performance of the hidden evolution in order to understand the quality of the dynamics learned.

Original: The k -step MSE and its normalized version, the R 2 k are computed by applying the transition function to the system over a rolling window k steps into the future without any input data.
Revised: The k -step MSE and its normalized version, the R 2 k are computed by applying the transition function to the system over a rolling window k steps into the future without any input data.

Original: The emission function is then used to form a predicted reconstruction ˆ x t + k and an error term using the corresponding observation window x t + k
Revised: The emission function is then used to form a predicted reconstruction ˆ x t + k and an error term using the corresponding observation window x t + k

Original: (below ¯ x k is the average of x k : T ).
Revised: (below ¯ x k is the average of x k : T ).

Original: This ensures that the evaluation criteria is dependent upon the both evolution term as well as the emission term.
Revised: This ensures that the evaluation criteria is dependent upon the both evolution term as well as the emission term.

Original: [Equation]
Revised: [Equation]

Original: We note that the ELBO is not a performance statistic that generalizes across models.
Revised: We note that the ELBO is not a performance statistic that generalizes across models.

Original: In contrast, the R 2 k provides a metric to quantify the inferred dynamics.
Revised: In contrast, the R 2 k provides a metric to quantify the inferred dynamics.

Original: Fig.
Revised: Fig.

Original: 6 in the Appendix emphasizes the above by highlighting the difference between the ELBO and the R 2 k for the IWAE and SVO.
Revised: 6 in the Appendix emphasizes the above by highlighting the difference between the ELBO and the R 2 k for the IWAE and SVO.

Original: 4. F ITZHUGH -N AGUMO
Revised: 4. F ITZHUGH -N AGUMO

Original: The Fitzhugh-Nagumo system is a two dimensional reduction of the Hodgkin-Huxley model.
Revised: The Fitzhugh-Nagumo system is a two dimensional reduction of the Hodgkin-Huxley model.

Original: It is described by two independent variables with cubic and linear functions.
Revised: It is described by two independent variables with cubic and linear functions.

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: The system was integrated over 200 time points with I ext = 1 held constant and a = 0 .
Revised: The system was integrated over 200 time points with I ext = 1 held constant and a = 0 .

Original: 7 , b = 0 .
Revised: 7 , b = 0 .

Original: 8 , c = 0 .
Revised: 8 , c = 0 .

Original: The initial state was sampled uniformly over [ − 3 , 3] 2 and 100 trials were generated with 66 for training, 17 for validation and 17 for testing.
Revised: The initial state was sampled uniformly over [ − 3 , 3] 2 and 100 trials were generated with 66 for training, 17 for validation and 17 for testing.

Original: One dimensional Gaussian observations were produced with a linear observation model acting on a single variable.
Revised: One dimensional Gaussian observations were produced with a linear observation model acting on a single variable.

Original: Dimensionality expansion is intrinsically harder than dimensionality reduction due to the loss of information.
Revised: Dimensionality expansion is intrinsically harder than dimensionality reduction due to the loss of information.

Original: For this reason we choose z t =
Revised: For this reason we choose z t =

Original: ( V t , W t ) and x t =
Revised: ( V t , W t ) and x t =

Original: N
Revised: N

Original: ( V t , σ 2 ) where σ 2 = 0 . 01 .
Revised: ( V t , σ 2 ) where σ 2 = 0 . 01 .

Original: Fig.
Revised: Fig.

Original: 1 shows the results of the Fitzhugh-Nagumo experiment.
Revised: 1 shows the results of the Fitzhugh-Nagumo experiment.

Original: The top left panel displays the phase space and trajectories of the original system.
Revised: The top left panel displays the phase space and trajectories of the original system.

Original: The bottom left panel displays latent variables from a single trial in the plane.
Revised: The bottom left panel displays latent variables from a single trial in the plane.

Original: The top center panel displays the inferred two-dimensional dynamics and trajectories.
Revised: The top center panel displays the inferred two-dimensional dynamics and trajectories.

Original: We ﬁnd that the limit cycle is recovered correctly and the topology of the space is preserved.
Revised: We ﬁnd that the limit cycle is recovered correctly and the topology of the space is preserved.

Original: Initial points located both inside and outside of the cycle in the original system are invariant in the reconstruction.
Revised: Initial points located both inside and outside of the cycle in the original system are invariant in the reconstruction.

Original: The bottom center panel displays an observation from a single trial.
Revised: The bottom center panel displays an observation from a single trial.

Original: The right panel shows the R 2 k comparison across models.
Revised: The right panel shows the R 2 k comparison across models.

Original: 4. ELBO C ONVERGENCE WITH S HARED E VOLUTION
Revised: 4. ELBO C ONVERGENCE WITH S HARED E VOLUTION

Original: We explore the effect of sharing the evolution parameters between the proposal and target distribution on model convergence by examining the ELBO.
Revised: We explore the effect of sharing the evolution parameters between the proposal and target distribution on model convergence by examining the ELBO.

Original: Fig.
Revised: Fig.

Original: 2 displays the ELBO convergence across epochs as we increase the number of particles ( K ).
Revised: 2 displays the ELBO convergence across epochs as we increase the number of particles ( K ).

Original: The left panel illustrates the ELBO convergence sharing network parameters ϕ
Revised: The left panel illustrates the ELBO convergence sharing network parameters ϕ

Original: = { ψ, Σ } between proposal and target.
Revised: = { ψ, Σ } between proposal and target.

Original: Larger values of K result in a faster convergence and lower stochastic gradient noise.
Revised: Larger values of K result in a faster convergence and lower stochastic gradient noise.

Original: The center panel displays the ELBO convergence for separate evolution networks for the proposal and the target.
Revised: The center panel displays the ELBO convergence for separate evolution networks for the proposal and the target.

Original: Note that for the same value of K , separate evolution networks require a larger number of epochs to converge and the ELBO obtains a lower value with larger stochastic gradient noise.
Revised: Note that for the same value of K , separate evolution networks require a larger number of epochs to converge and the ELBO obtains a lower value with larger stochastic gradient noise.

Original: The right panel displays both shared and independent evolution terms for K = 16 particles highlighting the difference between the two parameterizations.
Revised: The right panel displays both shared and independent evolution terms for K = 16 particles highlighting the difference between the two parameterizations.

Original: 4. L ORENZ A TTRACTOR
Revised: 4. L ORENZ A TTRACTOR

Original: The Lorenz attractor is a chaotic nonlinear dynamical system deﬁned with 3 independent variables.
Revised: The Lorenz attractor is a chaotic nonlinear dynamical system deﬁned with 3 independent variables.

Original: [Equation]
Revised: [Equation]

Original: The system of equations is integrated over 250 time points using σ = 10 , ρ = 28 , β = 8 / 3 by generating randomized initial states in [ − 10 , 10] 3 .
Revised: The system of equations is integrated over 250 time points using σ = 10 , ρ = 28 , β = 8 / 3 by generating randomized initial states in [ − 10 , 10] 3 .

Original: Ten dimensional Gaussian observations were produced with a mean speciﬁed by a z -dependent neural network.
Revised: Ten dimensional Gaussian observations were produced with a mean speciﬁed by a z -dependent neural network.

Original: The ﬁnal dataset consists oftrials with 66 for training, 17 for validation and 17 for testing.
Revised: The ﬁnal dataset consists oftrials with 66 for training, 17 for validation and 17 for testing.

Original: The results of the Lorenz experiment are shown in Fig.
Revised: The results of the Lorenz experiment are shown in Fig.

Original: The inferred latent paths are put together in the left panel illustrating the two cycles.
Revised: The inferred latent paths are put together in the left panel illustrating the two cycles.

Original: The center panel displays how R 2 k =10 scales as the latent dimension d z increases.
Revised: The center panel displays how R 2 k =10 scales as the latent dimension d z increases.

Original: The R 2 k stops improving once the correct number of independent variables in the system is reached.
Revised: The R 2 k stops improving once the correct number of independent variables in the system is reached.

Original: The right panel displays the R 2 k comparison with d z = 3 .
Revised: The right panel displays the R 2 k comparison with d z = 3 .

Original: 4. S INGLE C ELL E LECTROPHYSIOLOGY D ATA
Revised: 4. S INGLE C ELL E LECTROPHYSIOLOGY D ATA

Original: Electrophysiology data was downloaded from the Allen Brain Atlas (Jones et al., 2009).
Revised: Electrophysiology data was downloaded from the Allen Brain Atlas (Jones et al., 2009).

Original: Intracellular voltage recordings were collected from primary Visual Cortex of the mouse, area layer 4.
Revised: Intracellular voltage recordings were collected from primary Visual Cortex of the mouse, area layer 4.

Original: The dataset consists of 40 trials from 5 different cells.
Revised: The dataset consists of 40 trials from 5 different cells.

Original: The input current in the experiments is a step-function with an amplitude between 80 and 151pA. There were 30 trials reserved for training and 10 for validation.
Revised: The input current in the experiments is a step-function with an amplitude between 80 and 151pA. There were 30 trials reserved for training and 10 for validation.

Original: Each trial was divided into ﬁve parts and down-sampled from 10,000 time bins to 1, in equal-time intervals.
Revised: Each trial was divided into ﬁve parts and down-sampled from 10,000 time bins to 1, in equal-time intervals.

Original: Trials were normalized dividing each by its maximal value.
Revised: Trials were normalized dividing each by its maximal value.

Original: Fig.
Revised: Fig.

Original: 4 displays the result of the Allen experiment.
Revised: 4 displays the result of the Allen experiment.

Original: The left panel illustrates two trials of 1D observations from the training set.
Revised: The left panel illustrates two trials of 1D observations from the training set.

Original: We ﬁnd that three latent variables provides the best ﬁt.
Revised: We ﬁnd that three latent variables provides the best ﬁt.

Original: The center panel displays the predicted observation value using the dynamics learned over a rolling window ten steps ahead.
Revised: The center panel displays the predicted observation value using the dynamics learned over a rolling window ten steps ahead.

Original: Hyperpolarization and depolarization nonlinearities are accurately reconstructed by appying the inferred dynamics.
Revised: Hyperpolarization and depolarization nonlinearities are accurately reconstructed by appying the inferred dynamics.

Original: The right panel displays the R 2 k comparison with d z = 3 .
Revised: The right panel displays the R 2 k comparison with d z = 3 .

Original: 4. S IGNAL - TO - NOISE RATIO OF GRADIENT ESTIMATORS
Revised: 4. S IGNAL - TO - NOISE RATIO OF GRADIENT ESTIMATORS

Original: In this section, we focus on SNRs of gradient estimators in the encoder network ( φ ) , evolution network ( ϕ ) and decoder network ( θ ) , where the gradient is taken with respect to φ , ϕ and θ correspondingly.
Revised: In this section, we focus on SNRs of gradient estimators in the encoder network ( φ ) , evolution network ( ϕ ) and decoder network ( θ ) , where the gradient is taken with respect to φ , ϕ and θ correspondingly.

Original: Fig.
Revised: Fig.

Original: 5 presents the empirical results for SNRs of four different gradient estimators.
Revised: 5 presents the empirical results for SNRs of four different gradient estimators.

Original: The expectation and variance in the SNR are calculated using N = 100 gradient samples, which are collected in the middle training stage of running SVO on Fitzhugh-Nagumo data.
Revised: The expectation and variance in the SNR are calculated using N = 100 gradient samples, which are collected in the middle training stage of running SVO on Fitzhugh-Nagumo data.

Original: The l 2 norm is used to compute a scalar quantity to deﬁne the SNR.
Revised: The l 2 norm is used to compute a scalar quantity to deﬁne the SNR.

Original: We ﬁnd that the three panels share similar patterns: (1) The gradient estimator without resampling and the relaxed one with temperature λ = 0 .possess an SNR of convergence rate O ( √ K ) , which aligns with the theoretical result.
Revised: We ﬁnd that the three panels share similar patterns: (1) The gradient estimator without resampling and the relaxed one with temperature λ = 0 .possess an SNR of convergence rate O ( √ K ) , which aligns with the theoretical result.

Original: Although ignoring or relaxing the gradient from the resampling procedure adds a bias to the gradient estimator, it produces a positive effect of K on SNR; (2) The unbiased CATEGORIAL resampling gradient and relaxed gradient with small bias ( λ =
Revised: Although ignoring or relaxing the gradient from the resampling procedure adds a bias to the gradient estimator, it produces a positive effect of K on SNR; (2) The unbiased CATEGORIAL resampling gradient and relaxed gradient with small bias ( λ =

Original: ( K − 1) − 1 ) suffer from large variance which increases at a rate of O ( √ K ) , leading to a relatively low and even decreasing SNR for increasing K .
Revised: ( K − 1) − 1 ) suffer from large variance which increases at a rate of O ( √ K ) , leading to a relatively low and even decreasing SNR for increasing K .

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have sketched a method to construct Smoothed Variational Objectives (SVOs) using Sequential
Revised: We have sketched a method to construct Smoothed Variational Objectives (SVOs) using Sequential

Original: Monte Carlo to perform both inference and parameter estimation in nonlinear dynamical systems.
Revised: Monte Carlo to perform both inference and parameter estimation in nonlinear dynamical systems.

Original: By sharing parameters of the transition function between the proposal and target, SVO forgoes exchanging expressiveness for tractability.
Revised: By sharing parameters of the transition function between the proposal and target, SVO forgoes exchanging expressiveness for tractability.

Original: We ﬁnd that the use of a biased gradient estimate can help address the issue of a degraded signal-to-noise ratio.
Revised: We ﬁnd that the use of a biased gradient estimate can help address the issue of a degraded signal-to-noise ratio.

Original: Using the smoothed proposal density improves the computational complexity with a cost independent of the number of particles, and the prediction performance, quantiﬁed by the k-step reconstruction error across dimensionality reduction and expansion tasks.
Revised: Using the smoothed proposal density improves the computational complexity with a cost independent of the number of particles, and the prediction performance, quantiﬁed by the k-step reconstruction error across dimensionality reduction and expansion tasks.

Original: Future work includes parameterizing an input term to model non-autonomous dynamical systems.
Revised: Future work includes parameterizing an input term to model non-autonomous dynamical systems.

Original: B. G RADIENT ESTIMATORS
Revised: B. G RADIENT ESTIMATORS

Original: The unbiased gradient estimator,
Revised: The unbiased gradient estimator,

Original: [Equation]
Revised: [Equation]

Original: The derivation

Original: could

Original:  be found in (Le et al.,  or (Maddison et al., 
Revised: The derivation could be found in (Le et al., 2018) or (Maddison et al., 2017)

Original: The biased gradient estimator without resampling ∇L K is implemented by simply taking the gradient of the estimated variational objective, ˆ Z SVO .
Revised: The biased gradient estimator without resampling ∇L K is implemented by simply taking the gradient of the estimated variational objective, ˆ Z SVO .

Original: For the relaxed gradient estimator ∇L K + C ONCRETE ( λ ) , we use the C ONCRETE distribution to resample particles, and then directly evaluate the gradient of the objective.
Revised: For the relaxed gradient estimator, [Equation] We use the C ONCRETE distribution to resample particles, and then directly evaluate the gradient of the objective.

Original: C. P ROOF OF P ROPOSITION
Revised: C. P ROOF OF P ROPOSITION

Original: Proof.
Revised: Proof.

Original: It sufﬁces to show the convergence rate of expectation and variance of gradient estimate with respect to K .
Revised: It sufﬁces to show the convergence rate of expectation and variance of gradient estimate with respect to K .

Original: Throughout the analysis, we will extensively apply the result from (Rainforth et al., 2018), and exploit the factorization of the SVO objective: ˆ Z = (cid:81)
Revised: Throughout the analysis, we will extensively apply the result from (Rainforth et al., 2018), and exploit the factorization of the SVO objective: ˆ Z = (cid:81)

Original: Tt
Revised: Tt

Original: =

Revised: =

Original: ˆ Z t where ˆ Z t = 1 K (cid:80)
Revised: ˆ Z t where ˆ Z t = 1 K (cid:80)

Original: Kk =1 w kt .
Revised: Kk =1 w kt .

Original: Assume that z 1: K 1: T are obtained by passing the Guassian noise (cid:15) 1: K 1: T through the reparameterization function.
Revised: Assume that z 1: K 1: T are obtained by passing the Guassian noise (cid:15) 1: K 1: T through the reparameterization function.

Original: 1. Expectation.
Revised: 1. Expectation.

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The expectation decomposes into two terms, where the convergence rate for the ﬁrst directly follows the result from (Rainforth et al., 2018):
Revised: The expectation decomposes into two terms, where the convergence rate for the ﬁrst directly follows the result from (Rainforth et al., 2018):

Original: [Equation]
Revised: [Equation]

Original: For the remaining term that includes the resampling gradient
Revised: For the remaining term that includes the resampling gradient

Original: [Equation]
Revised: [Equation]

Original: we apply a thorough analysis as follows.
Revised: we apply a thorough analysis as follows.

Original: [Equation]
Revised: [Equation]

Original: Taylor expand log ˆ Z t (cid:48) about Z t (cid:48) :
Revised: Taylor expand log ˆ Z t (cid:48) about Z t (cid:48) :

Original: [Equation]
Revised: [Equation]

Original: where R 3 ( ˆ Z t (cid:48) ) denotes the remainder in the Taylor expansion of log ˆ Z t (cid:48) about Z t (cid:48) .
Revised: where R 3 ( ˆ Z t (cid:48) ) denotes the remainder in the Taylor expansion of log ˆ Z t (cid:48) about Z t (cid:48) .

Original: For t (cid:48) ≤ t − 1 , we have
Revised: For t (cid:48) ≤ t − 1 , we have

Original: [Equation]
Revised: [Equation]

Original: For t (cid:48) ≥ t
Revised: For t (cid:48) ≥ t

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: Hence
Revised: Hence

Original: [Equation]
Revised: [Equation]

Original: it sufﬁces to compute the convergence rate of the following:
Revised: it sufﬁces to compute the convergence rate of the following:

Original: [Equation]
Revised: [Equation]

Original: Note that when t (cid:48) ≤ t − 1 , we obtain similar results as Eq.
Revised: Note that when t (cid:48) ≤ t − 1 , we obtain similar results as Eq.

Original: Thus, we turn to the case when t (cid:48) ≥ t .
Revised: Thus, we turn to the case when t (cid:48) ≥ t .

Original: For t (cid:48) ≥ t + 1 , each w kt (cid:48) has dependence on a 1 t − 1 , hence
Revised: For t (cid:48) ≥ t + 1 , each w kt (cid:48) has dependence on a 1 t − 1 , hence

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Applying the Taylor expansion of 1ˆ Z t − 1 around Z t − 1 : 1ˆ Z t − 1 =
Revised: Applying the Taylor expansion of 1ˆ Z t − 1 around Z t − 1 : 1ˆ Z t − 1 =

Original: [Equation]
Revised: [Equation]

Original: For t (cid:48)
Revised: For t (cid:48)

Original: =
Revised: =

Original: t , only w 1 t depends on a 1 t − 1 , consequently we have
Revised: t , only w 1 t depends on a 1 t − 1 , consequently we have

Original: K · 
Revised: K · E

Original: [Equation]
Revised: [Equation]

Revised: ATEGORICAL

Revised: [Equation]

Original: 2. Variance.
Revised: 2. Variance.

Original: Var

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Decomposing the variance into the sum of variance at each time points, and the pairwise covariance across different time point, we will show that both terms are O ( 1 / K ) .
Revised: Decomposing the variance into the sum of variance at each time points [Equation] and the pairwise covariance across different time point, we will show that both terms are O ( 1 / K ) .

Original: (1) Variance at each time step.
Revised: (1) Variance at each time step.

Original: ∀ t = 1 : T ,
Revised: ∀ t = 1 : T ,

Original: [Equation]
Revised: [Equation]

Original: (2) Covariance between different time steps.
Revised: (2) Covariance between different time steps.

Original: For t (cid:54) =
Revised: For t (cid:54) =

Original: t (cid:48) ∈ 1 : T
Revised: t (cid:48) ∈ 1 : T

Original: [Equation]
Revised: [Equation]

Original: we ﬁrst apply Taylor theorem to log ˆ Z t around Z t
Revised: we ﬁrst apply Taylor theorem to log ˆ Z t around Z t

Original: [Equation]
Revised: [Equation]

Original: and then exploit the fact that ˆ Z t is an unbiased estimation of Z t
Revised: and then exploit the fact that ˆ Z t is an unbiased estimation of Z t

Original: [Equation]
Revised: [Equation]

Original: and exploit the deﬁnition of covariance to expand and collapse terms, as follows:
Revised: and exploit the deﬁnition of covariance to expand and collapse terms, as follows:

Original: [Equation]
Revised: [Equation]

Original: (i) For the ﬁrst term in Eq.
Revised: (i) For the ﬁrst term in Eq.

Original: (62), since z ( k ) t are i.i
Revised: (62), since z ( k ) t are i.i

Original: d. for ﬁxed t , we have
Revised: d. for ﬁxed t , we have

Original: [Equation]
Revised: [Equation]

Original: Without loss of generality, we assume t (cid:48) > t .
Revised: Without loss of generality, we assume t (cid:48) > t .

Original: First, when t (cid:48)
Revised: First, when t (cid:48)

Original: =
Revised: =

Original: t + 1 ,
Revised: t + 1 ,

Original: Pr (cid:0) z
Revised: Pr (cid:0) z

Original: 1 t +1 depends on z 1 t (cid:1)
Revised: 1 t +1 depends on z 1 t (cid:1)

Original: [Equation]
Revised: [Equation]

Original: When t (cid:48) > t + 1 , using chain rule and by induction we also have,
Revised: When t (cid:48) > t + 1 , using chain rule and by induction we also have,

Original: Pr( z 1 t (cid:48) depends on z 1 t )
Revised: Pr( z 1 t (cid:48) depends on z 1 t )

Original: = 1 K (68)
Revised: = 1 K (68)

Original: Hence,
Revised: Hence,

Original: Cov
Revised: Cov

Original: [Equation]
Revised: [Equation]

Original: depends on z 1 t
Revised: depends on z 1 t

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (70) (ii)
Revised: (70) (ii)

Original: For the second and third term in Eq.
Revised: For the second and third term in Eq.

Original: (62), without loss of generality, we analyze the second term E (cid:2) ∇ (cid:0) ˆ Z t (cid:48) /
Revised: (62), without loss of generality, we analyze the second term E (cid:2) ∇ (cid:0) ˆ Z t (cid:48) /

Original: Z t (cid:48) (cid:1) · ∇ R 1 ( Z t ) (cid:3) , and assume t (cid:48) > t .
Revised: Z t (cid:48) (cid:1) · ∇ R 1 ( Z t ) (cid:3) , and assume t (cid:48) > t .

Original: Using the i.i.d.
Revised: Using the i.i.d.

Original: property of particles at ﬁxed time step, we have
Revised: property of particles at ﬁxed time step, we have

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: Similar to the previous analysis on covariance, we can show that
Revised: Similar to the previous analysis on covariance, we can show that

Original: [Equation]
Revised: [Equation]

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: (iii)
Revised: (iii)

Original: For the last term in Eq.
Revised: For the last term in Eq.

Original: (62), note that | Cov( A, B ) | ≤ (cid:112) Var( A )
Revised: (62), note that | Cov( A, B ) | ≤ (cid:112) Var( A )

Original: Var( B ) , and
Revised: Var( B ) , and

Original: Var[ ∇ R 1 ( ˆ Z t )]
Revised: Var[ ∇ R 1 ( ˆ Z t )]

Original: = O ( 1 / K 2 ) , hence we obtain
Revised: = O ( 1 / K 2 ) , hence we obtain

Original: [Equation]
Revised: [Equation]

Original: Substituting Eq.
Revised: Substituting Eq.

Original: (59), Eq.
Revised: (59), Eq.

Original: (62) and Eq.
Revised: (62) and Eq.

Original: (66) into Eq.
Revised: (66) into Eq.

Original: (57), we arrive at the ﬁnal expression for the variance of gradient estimate:
Revised: (57), we arrive at the ﬁnal expression for the variance of gradient estimate:

Original: [Equation]
Revised: [Equation]