Particle-Based Score Estimation for State Space Model Learning in Autonomous Driving
Authors: Angad Singh
Omar Makhlouf
Maximilian Igl
Joao Messias
Arnaud Doucet
Shimon Whiteson
Venue: CoRL 2022 Poster (2022)
Keywords: Autonomous Driving
Particle Filtering
Self-supervised Learning

Abstract: Multi-object state estimation is a fundamental problem for robotic applications where a robot must interact with other moving objects. Typically, other objects' relevant state features are not directly observable, and must instead be inferred from observations. Particle filtering can perform such inference given approximate transition and observation models. However, these models are often unknown a priori, yielding a difficult parameter estimation problem since observations jointly carry transition and observation noise. In this work, we consider learning maximum-likelihood parameters using particle methods. Recent methods addressing this problem typically differentiate through time in a particle filter, which requires workarounds to the non-differentiable resampling step, that yield biased or high variance gradient estimates. By contrast, we exploit Fisher's identity to obtain a particle-based approximation of the score function (the gradient of the log likelihood) that yields a low variance estimate while only requiring stepwise differentiation through the transition and observation models. We apply our method to real data collected from autonomous vehicles (AVs) and show that it learns better models than existing techniques and is more stable in training, yielding an effective smoother for tracking the trajectories of vehicles around an AV.

Particle-Based Score Estimation for State Space Model Learning in Autonomous Driving
Angad Singh'
Omar Makhlouf"
Maximilian Igl Waymo Research nigl@waymo. com
Waymo Research singhangadewaymo.cm
Waymo Research makhl @waymo com
Joao Messias Waymo Research messiasj @waymo com
Arnaud Doucet DeepMind arauddooccee&&deppiiidd.d com
Shimon Whiteson Waymo Research
shimonx@vaymo.com com
Abstract: Multi-object state estimation is fundamental problem for robotic ap- plications where robot must interact with other moving objects. Typically, other objects' relevant state features are not directly observable, and must instead be inferred from observations. Particle filtering can perform such inference given approximate transition and observation models. However, these models are often unknown priori, yielding difficult parameter estimation problem since obser- vations jointly carry transition and observation noise. In this work, we consider learning maximum-likelihoon parameters using particle methods. Recent meth- ods addressing this problem typically differentiate through time in particle filter, which requires workarounds to the non-differentiable resampling step, that yield biased high variance gradient estimates. By contrast, exploit Fisher's 's iden- tity to obtain a particle-based approximation of the score function (the gradient the log likelihood) that yields low variance estimate while only requiring step- wise differentiation through the transition and observation models. We apply our method to real data collected from autonomous vehicles (AVs) and show that it learns better models than existing techniques and is more stable in training, yield- ing an effective smoother for tracking the trajectories of vehicles around an AV.
Keywords: Autonomous Driving, Particle Filtering, Self-supervised Learning
Introduction
Multi-object state estimation a fundamental problem settings where robot must interact with other moving objects, since their state is directly relevant for decision making. Typically, other objects' relevant state features are not directly observable. Instead, the robot must infer them from stream of observations it receives via a perception system. For example, an autonomous vehicle (AV) selects actions based on the state of nearby road users. However, such road users are only partially observed, owing to limited field of view, occlusions, and imperfections in the AV's sensors and perception systems. Such partial observability negatively affects many downstream tasks robot's behavioural stack that depend on observations, c.g., action planning.
Addressing partial observability requires sequential state estimation, which Bayesian filtering of- fers a generic probabilistic approach. In particular, sequential Monte Carlo methods, also known as particle filtering, have been successfully applied state estimation in many robotics applications [1]. However, Bayesian filters require models that reasonably approximate the transition and observation models of a state-space model (SSM). some special cases, these models can be derived analyti- cally from first principles, e.g., when the physical dynamics are well understood, many real-world applications, however, these models cannot be specified analytically. For example, the transition model may encode complicated motion dynamics and environmental physics. In multi-agent settings, other agents' behaviour must also be modelled.
by modeling
sensor's 's physical characteristics.
These authors contributed equally to this work.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

Modelling observations is also difficult. Modern perception systems often involve multiple stages and combine information from multiple sensors. making observation models practically impossible to specify by hand. By contrast, collecting observations from robotic system is relatively easy and cheap. We are interested, therefore, algorithms that can leverage such observations to learn tran- sition and observation models in a self-supervised fashion, and yield an effective particle smoother. Learned transition and observation models can also be independently useful for other applications, such as the evaluation of AVs by simulating realistic observations.
In this work, we propose Particle Patterin--Based Score Estimation using Fisher's Identity (PF- SEFI), a method for jointly learning observation models, that is applicable to a wide class of SSMs. Unlike many recently proposed methods [2, 3, 4, 5, 6, 7], our approach avoids differentiable approximations of the resampling step. We achieve this by revisiting a methodology originally proposed in statistics [8, 9] that relies on a particle approximation of the score, i.e., the gradient of the log likelihood of observation sequences, obtained through Fisher's identity. This only requires differentiating through the transition and observation models. Unfortunately, a direct particle approximation of this identity provides a high variance estimate of the score. While [8] propose an alternative low variance estimate, it admits a O(N2) cost, where N is the number of particles. Furthermore, these methods compute and store the gradient of the marginal log- likelihodd with respect to model parameters for each particle. This requires computing Jacobian matrices, which are slow to compute using automatic differentiation tools such as TensorFlow and PyTorch [10, 11] which rely on Aacobian-vector products. This makes these methods impractical for large models. By contrast, PF-SEFI is a simple scalable (IV) variant with only negligible bias. PF-SERI marginalises over particles before computing gradients, allowing automatic differentiation tools to make use of efficient Jacobian product operations, making it significantly faster and allowing us to scale to larger models. To the best of our knowledge, previous particle methods estimating the score have been limited SSMs with few parameters, whereas we apply PF-SEFI to neural network models with thousands of parameters.
parameters of both the transition and
We apply PF-SEFI to jointly learn transition and observation models for tracking multiple objects around an AV, using a large set of noisy trajectories. containing almost 10 hours of road-use user tra- jectories observed by AV. We show that PF-SEFI learns an SSM that yields an effective object tracker as measured by average displacement and yaw errors. We compare the learned observation model one trained through supervised learning on dataset of manually labelled trajectories, and show that PF-SEFI yields a better model (as measured by log-likelln log-likelihood on ground-truth labels) even though it requires no labels for training. Finally, we compare PF-SEFI to number of existing particle methods for jointly learning transition and observation models and show that it learns better models and is more stable to train.
2 Related Work
Particle filters are widely used for state estimation in non-linear non-Gaussian SSMs where no closed form solution is available; see e.g.. [12] for survey. The original bootstrap particle filter [13] sam- ples at each time step using the transition density particles that are then reweighted according to their conditional likelihood, which measures their "fitness" w.r.t. the available observation. Parti- cles with low weights are then eliminated while particles with high weights are replicated to focus computational efforts into regions of high probability mass. Compared to many newer methods, such as the auxiliary particle filter [14], the bootstrap particle filter only requires sampling from the transition density, not its evaluation arbitrary values, which is not possible for the compositional transition density used in this work.
In most practical applications, the SSM has unknown parameters that must be estimated together with the latent state posterior (see [9] for a review). Simply extending the latent space to include the unknown parameters suffers from insufficient parameter space exploration [15]. While particle filters can estimate consistently the likelihood for fixed model parameters, a core challenge is that the such estimated likelihood function is discontinuous in the model parameters due to the resampling step, hence complicating its optimization; see e.g. [6, Figure 1] for an illustration.
Instead, the score vector can be computed using Fisher's identity [8]. However, shown in [8], performance degrades quickly for longer sequences if a standard particle filter is used, due to the
path degeneracy problem: repeated resampling of particles and their ancestors will leave few even
2

---

just one remaining ancestor path for earlier timesteps, resulting in unbiased, but very high variance estimates. Methods for overcoming this limitation exist [8. 16, 17], but with requirements making them unsuitable in this work. Poyiadjis et al. [8] store gradients separately for each particle, making this approach infeasible for all but the smallest neural networks. Scibior and Wood [17] propose an improved implementation with lower memory requirements by smartly using automatic differen- tiation. However, their approach still requires storing computation graph whose size scales with O(N2) as the transition density for each particle pair must be evaluated during the forward pass. Both previous methods' computational complexity also scales quadratically with the number of par- ticles, N. which problematic for costly gradient backpropagation through large neural networks. Lastly, Olsson and Westerborn [16] require evaluation of the transition density for arbitrary values, which our compositional transition model does not allow. Instead, in this work, we show that fixed- lag smoothing [18, 19] is viable alternative to compute the score function of large neural network models in the context of extended object tracking.
There is extensive literature on combining particle filters with learning complex models such as neural networks [2, 4, 5, 6, 20, 21, 22, 23, 24). In contrast to our work, they make use of learned, data-dependent proposal distribution. However, for parameter estimation, they rely on dif- ferentiation of an estimated lower bound (ELBO). Due to the non-differentiable resampling step, this gradient estimation has either extremely high variance or is biased if the high variance terms are simply dropped, as in [2. 3, 4]. As we show in Section 5, this degrades performance noticeably. A second line of work proposes soft resampling [5, 20, 21], which interpolates between regular and uniform sampling. thereby allowing to trade off variance reduction through resampling with the bias introduced by ignoring the non-differentiable component of resampling. Lastly, Corenflos et al. [6] make the resampling step differentiable by using entropy- regulaa-eee optimal transport, also inducing bias and O(N2) cost.
Extended object tracking [25] considers how to track objects which, contrast to "small" objects [26). generate multiple sensor measurements per timestep. Unlike in our work, transition and mea- surement models are assumed to be known or depend on only few learnable parameters. Similar to work, the measurement model proposed in [27] assumes measurement sources lying rectangular shape. However, our model is more flexible, for example, allowing non-zero probability on all four sides simultaneously.
3 State-Space Models and Particle Filtering
3.1 State-Space Models
A SSM is partially observed discrete-time Markov process with initial density, zo ~ p(-), transi- tion density ze| ze|2;-1 ~ fo(-1xt-1), and observation density yt Ie ~ g)(-) (x)), where 2; is the latent state at time and ye the corresponding observation. The joint density of Torr Bour satisfies:
Pa Vo.T) (/ (vol (mo)
II So [a (900((/ 1=1
(1)
Given this model, we are typically interested inferring the states from the data by computing the filtering and one-step ahead prediction distributions, bo(=x/lo::)heo..... and (e:44ll {p(xo:d (oo. heEo.... satisfying
respectively, and more generally the joint distributions
po(2o:1ooa) Pe Po Po()0:c)
Po()or) -
po ( your )dror.
(2)
Additionally, to estimate parameters, we would also like to compute the marginal log likelihood:
(0) log pe (vo.T) po())))))))))))))) + Σ log logpo(uelso..1)
(3)
where (yo) = f go (vol (No) and po(1xloo=-1) = go ( (or))oo 1))x, for /
1.
For non-linear non-Gaussian SSMs, these posterior distributions and the corresponding marginal likelihood cannot be computed in closed form.
3

---

While the resulting estimates are consistent N + Do for any fixed time t [28], this does not guarantee good practical performance. Fortunately, under regularity conditions the approximation error for the estimate Po (ate [yo+)) and more generally po (loo)))) for fixed lag L 1 as well as log po (Woot)))/ does not increase with for fixed N. However, this is not the case for the joint 3.2 Particle Filtering
Particle methods provide non- non-parann con-parametri and consistent approximations of these quantities. They rely on the combination of importance sampling and resampling steps of set of N weighted parti- cles (zt, w:), where zi denotes the values of the 1th particle at time and wt is corresponding weight satisfying Σi=1 / = 1. We focus on the bootstrap particle filter, shown in Algorithm 1, which samples particles according the transition density. Let & ~ Cat(a),.... O:N) denote the categori-
Algorithm Bootstrap Particle Filter
Sample Xo lld. ua-)) for E [M] and set (o( log N Ein go(volz8)
For f = T....
Compute weights wi_1 go(ve-11 with Σ Wi =
2. Sample d/-1
1 then 2 fo (-12(-1)) for i [N].
3. Set xhe t 1, a/) for i E [M] and (0) + (0) + log (N E (o(
cal distribution with N categories, where the probability of the taking the 2th category is ag At any time f, this algorithm produces particle approximations
T
(a (oo:))
Σ wpom mis. (mo:t); i=1
(
Σ log
E g0 (welz:) N
(4)
of po (row (o.:)) and (x( -lo pa (Vo:)), where 8, is the Dirac delta distribution centred at a. Step 2 resamples, discarding particles with small weights while replicating those with large weights before evolving according to the transition density. This focuses computational effort on the "promising" regions of the state space. Unfortunately, resampling involves sampling discrete random variables at each time step and as such produces estimates of the log likelihood that are not differentiable w.r.t. as illustrated in [6, Figure I].
smoothing approximation because successive resampling means that ýo (roo (ooz) is eventually ap- proximated by single unique particle for large enough t, a phenomenon known as path degeneracy; see c.g. [12, Section 4.3].
4 Score Estimation using Particle Methods
To estimate the parameters of a given SSM (1) along with dataset of observations yo:T, we want to maximise via gradient ascent the marginal log likelihood in (3). However, the gradient of the marginal log likelihood, i.e., the score function, is intractable. As explained in Section 2, automatic differentiation through the filter is difficult due to the non-differentiable resampling step.
4.1 Score Function Using Fisher's Identity
We leverage here instead Fisher's identity [8] for the score to completely side-step the non- differentiability problem. This identity shows that
(o) Vo log po (Zo.r, Borr) (T your )dzorr,
(5)
ie., the score is the expectation of V log (o Yorr) under the joint smoothing distribution Po(ForrBorr) Plugging in (1), the score function can be simplified to
(o)
Σ
Ve log go(v) (o
T
Va log fo(xz xx- pa la Wo.T )
(6)

---

4.2 Particle Score Approximation
The identity (6) shows that we can simply estimate the score by plugging particle approximations of the marginal smoothing distributions -1 (o.r) into (6). This identity makes differentiating through time superfluous and thereby renders the use of differentiable approximations of resampling unnecessary. However, as discussed in Section 3.2, naive particle approximations of the smoothing distribution's marginals, Po (fe())o:) and po 1:+ Vo:T), suffer from path degeneracy. To bypass this problem, [8, 17] propose an ((N2) method inspired by dynamic programming. We propose here a simpler and computationally cheaper method that relies on the following fixed-lag approxi- mation of the fixed-interval smoothing distribution, which states that for L 1 large enough,
pe ( = pe (z1-1:t (t+
This approximation simply assumes that observations after time t + L do not bring further infor- mation about the states 24, This is satisfied for most models and the resulting approximation error decreases geometrically fast with L [19]. The benefit of this approximation is that the particle approximation of pe (It-1:d (:o-:: (t..,,))+ I.TT) does not suffer from path degeneracy and is a simple byproduct of the bootstrap particle filtering of Algorithm 1; e.g., for t + L < T we consider the particle approximation Po (Fozt+ [1o++ = Σm= " A (#b++ obtained at time ++ L and use its corresponding marginals in It-1, Ty and z, integrate respectively Vg log (0(x)//11))) and Vo log go (o (z).. Fort t + L 2 T, we just consider the marginals in 2t-1: Lt and 2, of po (Fo.r (80:r). So finally, we consider the estimate,
T
Vol = Σ
/
Ve e log (V)l (100)//mmmm((t4..77 drr
t=
T
/
+
Vo log fo(xz) po Vel.aeeemmmttttte L.T) )d
(8)
4.3 Score Estimation with Deterministic, Differentiable, Injective Motion Models
We have described a generic method approximate the score using particle filtering techniques. For many applications, however, the transition density function, fo (zy (I2-1). is the composition of policy, ne =e(d((=)_1) which characterises the action distribution conditioned on the state, and a poten- tially complex but deterministic, differentiable, and injective motion model, T : R x R" - R", where na < ng which characterises kinematic constraints such that z; = T a+) = (at). Under such a composition, the transition density function on the induced manifold M (Fa.., (at) at E "IK"> is thus obtained by marginalising out the latent action variable, i.e.,
fo (x-1)) I(x, E MZ..i) f 5(1z (a))((a)) to (1-1)d
It is easy to sample from this density but it is intractable analytically if the motion model is only available through a complex simulator or if not invertible. This precludes the use of sophis- ticated proposal distributions within the particle filter. Additionally, even if it were known, one cannot use the O(N²) smoothing type algorithms developed in [8, 16] as the density is concen- trated a low-dimensional manifold [29]. This setting common in mobile robotics, in which controllers factor into policies that select actions and motion models that determine the next state. Indeed, this is precisely the case in our application (see Section 5). Learning the corresponding SSM reduces to learning the parameters f of the policy, Ty aa xT-1), and the observation model, go (ueeerr.) Thankfully, even if the explicit form of the motion model is unknown, we can still com- pute V log fo fo(xx((e-1) as required by the score estimate (8).
*R>> where Th T, be smooth and injective mapping. Then, for any fixed It-1 and F4 E -M._,,, the gradient of the transition log density, i.e., V log fo(xe(2==1), reduces to the gradient of the policy log density, i.e., V log no (dr( where a, is the unique action that takes It-1 Z₁.
Lemma 4.1. For any x E R== let Ts : RT +
Proof. For Φ;-1 and 2, E M we denote by J[z (x:) E R= Xna the rectangular Jacobian matrix and write ae 7-11 (II)), i.c. this is the unique action such (as) 21. By
5

---

Observations from the real data.
(b) Observations from the synthetic data.
Figure 1: Observations from real and synthetic data. The AV (orange) observes a set of 2D points (blue) Torming polygon around the road users. The true bounding boxes of each road user (green) were manually labelled in the real data, and pre-determined while constructing the synthetic data.
standard result from differential geometry [30, 31J, the transition density (9) satisfies
1/2 T(x MZ,.1)- (10)
(1 no (at/2re-1) det J (a)
It follows directly that Vo log fo( = V log no (0gl0:-1)-
Indeed for the marginals po tot Mo te
we can store the actions corresponding to tran-
sitions 2 - 2y during filtering, and it follows that for the class of SSMs described above, the score estimate reduces to:
T
= Volr(0)
Σ
Va log go (Velar) (:ts::))mnmttttt ( )dr..
t=0
Σ
V log To(a, p0( (T+ )r
(11)
where we use Lemma 4.1 to replace the gradient of the transition log density with the gradient of the policy log density in (8), and where a, the action sampled go from xz-1 to zt.
5 Experiments
Problem Setting, Our experiments focus on the problem of state estimation of observed road users (in particular other vehicles) from the viewpoint of AV, which involves the estimation of 2D poses from an observed sequence of 2D convex polygons in a "bird's eye view" (BEV) constructed from LiDAR point clouds at each time step. For these experiments, we assume that the size of the observed objects, the pose of the AV, and the association of observations with their corresponding objects are known priori. Some observations (and their corresponding states) are shown in Figure la. Here, the observation model must learn to describe the likelihood of 2D points around the pe- riphery of the observed road user (see [25] for a review on such models), while the transition model must learn to describe driving behaviour. We use feed-forward neural network to parameterise our observation model, where we provide it with range, bearing. and relative bearing from the viewpoint of the corresponding AV as features (Appendix A), and factor our transition model into determin- istic and differentiable motion model based on Ackermann dynamics [32] (Appendix B. and policy parameterised by another feed-forward neural network (Appendix B.2).
Baselines, Datasets, and Metrics. We compare the quality of the models learned using PF-SEFI (our method), DPF-SGR [17]. PFNET [5], and differentiating through a vanilla PF (ignoring the bias introduced by resampling). We evaluate our method (and the baselines) on real data collected from an AV in an urban environment, equipped with LiDARs, cameras, and radar sensors. All sensors were used associate LiDAR points to their corresponding objects, and the observations shown in Figure la were obtained via convex hull computation of the associated LiDAR points. In addition to using real data, we generate two synthetic datasets (with 25 and 50 step trajectories), using a hand-crafted policy, and an observation model trained using supervised learning on manually labelled trajectories (see Appendix and for more details). Example observations are shown in Figure lb. Unlike with real data, where the true models are unknown, synthetic datasets allow us

---

-3.0
-12
3380
-3.75
-2.0
7
-5.5
REMET
-222
-2.5
-3.5
-100
-425
200
P0 2700 1250 1909 1952
(a) MLL of test synthetic data with 25 step traeectories.
(b) MLL test synthetic data with (c) MLL test real data with 60 step trajectories.
50 step trajectories.
Figure 2: Marginal Log Likelihood (MLL) on synthetic and real test data for models trained using PF-SEFI (us), DPF-SGR, PFNET, and PF, plotted against the corresponding training steps. For synthetic data we also show the MLL. the true models.
to compare the learned models against a known ground truth. We measure the quality of learned models using the following metries:
Marginal Log Likelihood (MLL): The marginal log likelihood (r)) given by filtering obser- vations Yorr using the learned models.
Average Displacement Error (ADE) and Average Yaw Error (AYE): The average error in the positions and yaws respectively of the smoothed state estimates Eg (x (yo:r) against the true poses, Torr. For the synthetic data, the true poses are sampled while generating the data; for the real data, the true poses are obtained by humans manually labelling object trajectories from videos. These measure the quality of the learned models for the purposes of state estimation. Average Observation True Log Likelihood (AOTLL.): The average log likelihood of ob- servations conditioned on the true states under the learned observation model, i.c., go(yr 9o [ā,). This measures the quality of the learned observation model.
T+I oi Average Policy True Log Likelihood (APTLL): The average log likelihood of true actions, T1.r (only available for experiments with synthetic data since it is not possible to man- ually label latent actions) conditioned on the true states under the learned policy, i.e.,
+E log (ā [1) This measures the quality of the learned policy.
Results. Figure 2 shows the progress of the learned models by tracking MLL of held out test data for each of the three datasets (synthetic data with 25 steps, synthetic data with 50 steps, and real data with 60 steps), and for each of the four methods (PF-SEFI, DPF-SGR, PFNET, and PF). Table sum- marise the performance of the learned models at convergence. We pick the best hyper-parametent smoothing lag T. for PF-SEFI, and trade-off parameter for PFNET, in each of the experiments. Appendix C includes analysis of the training sensitivity of each of the hyper-parameter (see Fig- ures 5, 6, and 7). We find that PF-SEFI improves with increasing L up point, past which it is insensitive to the choice of L (see Figure 6, Appendix C).
In experiments with synthetic data with 25 steps (Figure 2a and Experiment A in Table 1), observe a clear gap in performance PF-SEFI and DPF-SGR relative to PFNET and PF. The im- provements over PF are likely due to the bias in PF's score estimates due to the non-differentiable resampling step, while the improvements over PENET are likely due to adverse effects of not re- sampling with the correct distribution similarly on this dataset, the difference is stark in the case of synthetic data with 50 steps (Figure 2b and Experiment B in Table 1). PF-SEFI is invariant to the length of the trajectories used, converging stably; however, all other methods, struggle to learn useful models. We postulate that since each of the baselines, in one way or another, differentiate through all time steps of the filter, the variance in their score estimates is too high for good learning through gradient ascent.'
each time step. While PF-SEFI and DPF-SGR perform
'The authors of DPF-SGR [17] recommend the of stop gradients not only for particle weights after resampling, i.e., tif 05' ao [17, Algorithm 1], but also, in the of bootstrap particle filters, while com- puting the likelihood ratio of = 11 ya (/ee (x](a=/11) before resampling, and while sampling from 2j go( (-Iay-_ ) [17, Section 4.1J. While these additional stop gradients significantly reduce variance, our experiments with them yielded extremely poor overall performance (even with synthetic data with 25 steps). The results report here thus make use of stop- hop-gradients only for particle weights after resampling.
7

---

Table 1: Metrics computed on held test data comparing PF-SEFI (us) against baselines. We ran experiments using different datasets - (A) synthetic data with 25 steps, (B) synthetic data with steps, and (C) real data with 60 steps. For experiments (A) and (B), we also compare against the performance of the true models. For experiment (C), we compare against the supervised observation model trained using manually labelled traeectories. For MLL, AOTLL, and APTLL, higher values imply better models, while for ADE and AYE, lower values imply better models.
Exp.
Method
MIL
AOTLL
APTLL
ADE (m)
AYE (rad)
TRUE
-3.161 = 0.003
-2.128
2.674
0,0M0 + 0.001
0.014 + 0.000
PE-SETI (us)
0.004
1: 0.028
2.661
0.014
0.021
0.016 1: 0.000
A
DPF-SGR
-3.159 + 0.004
-2.265 +: 0,010
2.594 + 0.027
0.165 + 0,008
0.014 + 0.000
PENET
-3.225 = 0.004
-2.487
+ 0.026
2.621 = 0.013
0,264 + 0.019
0.017 = 0.000
PF
-3.229 + 0.006
-2,484
1: 0.026
2.576 + 0.021
0.245 1 0.021
0.017 1 0.000
0.002
-2.165
2.693
: 0.001
0.012 + 0.000
PE-SEFT (us)
-3.14 + 0.005
-2.283 1: 0.015
2.505 1 0.042
0.165 1 0.013
0.014 + 0.000
DPF-SGR
-3.966
0.050
-2.636 +: 0.031
0.811 - 0.130
2.828
0.415
0.142 + 0.016
PENET
-4.169
0.046
-2.901
+ 0.039
0.539 = 0.077
2,809
= 0.176
0.148 = 0.008
PF
4.1118 + 0.038
-28841 1: 0.025
0.681 + 0.122
2.502
0.042
0.137 + 0.007
SUPERVISED
N/A
-2.224 + 0,006
N/A
N/A
N/A
PE-SEFI (us)
0.029
+ 0.029
N/A
: 0.011
0,034 : 0.006
DPF-SGR
-3.297 = 0.287
-2.236 + 0.218
N/A
0.643 = 0.177
0.081 = 0.477
PENET
-4,019 + 0.098
-2.752 + 0.079
N/A
0.746 + 0,091
1005 + 0.159
PF
3.848 0.045
-2,399 1 0.140
N/A
0,701 1 0.109
1,882 1 0.364
The results of our experiments with real data with 60 steps (Figure 2c and Experiment C in Table 1) are consistent with Experiment B (i.e., with experiments on synthetic data with 50 steps) and show that PF-SEFI is able to learn useful models. The learned observation model using PF-SEFI performs even better than the model that was trained offfine through supervision with manually labelled data (see AOTLL Table for Experiment C). We also find that sampling from the learned model produces observations that are qualitatively similar to the real data (Appendix D). While the supervised model is trained only on the subset of the observations that are labeled (labelling only a subset is common in practical applications due to the cost of labelling), PF-SEFI, by contrast, can leverage all observations in a sule-supe fashion. Moreover, we speculate that the labels contain noise and that the labelling distribution is biased towards observations that easy label. Both limitations hinder supervised learning.
6 Discussion, Limitations, and Future Work
In this work, proposed an efficient particle-based method for estimating the score function to learn a wide class of SSMs in a self-supervised way. Compared to previous particle methods that estimate the score, PF-SEFI is more computationally efficient, allowing to scale learning mod- els with many parameters. Unlike alternative methods, PF-SEFI is applicable to SSMs where the transition distribution is concentrated on a low-dimensional manifold, allowing us to apply real-world AV object tracking problem. We showed empirically that our method learns better models and is more stable in training than methods that use automatic differentiation estimate the score, and that we can learn an observation model that outperforms one trained using supervised learning.
While this solution is ideal for our problem. it does have number of limitations. Most notably, it is restricted to maximising the marginal log-likelihood of the data, while differentiating through the filter allows for arbitrary differentiable loss functions. Furthermore, our method is not suitable for estimating the parameters of a proposal distribution. Beyond these algorithmic limitations, in our application, the models that used were not very expressive. For the observation model, did not model important phenomena that affect partial observability such as occlusions and we restricted our states and observations 2D. For the policy, we used simplified policy with only basic features that are insufficient for controlling an agent in simulation. Furthermore, the policy and the motion model are both specific vehicles, and currently exclude other road users such pedestrians.
In future work, we aim to scale up our problem setting, by making both models more expressive, and to estimate more state dimensions, such as full 3D poses and sizes of objects. We also believe that learning policies as components of an SSM to explicitly account for observation noise is, in practice critical for learning good driving behaviour from demonstrations. Such policies could be used as models for predicting the behaviour of other road-users, or to control agents in simulation, and the method we proposed in this work offers an ideal starting point explore this.
8

---

Acknowledgments
We thank Drago Anguelov, Charles Qi and Congcong Li for their helpful feedback draft of the paper. We also thank the reviewers for taking the time give detailed and useful feedback on our initial submission. Finally, we thank the Waymo Research team and DeepMind for supporting this project.
References
[1] S. Thrun, W. Burgard, and D. Fox. Probabilistic Robotics. MIT Press, 2005.
[2] A. M. Igl, T. Rainforth, T. Jin, and F. Wood. Auto-encoding sequential Monte Carlo. In International Conference on Learning Representations, 2018.
[3] C. J. Maddison, J. Lawson, G. Tucker, N. Heess, M. Norouzi, A. Mnih, A. Doucet, and Y. Teh. Filtering variational objectives. Advances in Neural Information Processing Systems, 30, 2017.
[4] C. Naesseth, S. Linderman, R. Ranganath, and D. Blei. Variational sequential Monte Carlo. In International Conference on Artificial Intelligence and Statistics, pages 968-977 2018.
[5] Karkus, D. Hsu, and W. S. Lec. Particle filter networks with application to visual localization. In Conference on Robot Learning, pages 169-178, 2018.
[6] A. Corenflos, J. Thornton, G. Deligiannidis, and A. Doucet. Differentiable particle filtering reguopy- optimal transport. In International Conference on Machine Learning. pages 2100-2111, 2021. 2021.
[7] J. Lai, J. Domke, and D. Sheldon. Variational marginal particle filters. In International Con- ference on Artificial Intelligence and Statistics, pages 875-895, 2022.
[8] G. Poyiadjis, A. Doucet, and S. Singh. Particle approximations of the score and observed information matrix in state space models with application to parameter estimation. Biometrika, 98(1):65-80, 2011.
[9] N. Kantas, A. Doucet, S. Singh, J. Maciejowski, and Chopin. On particle methods for parameter estimation in state-space models. Statistical science, 30(3):328-335 2015.
[10] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, Dean, M. Devin, S. Ghemawat, Goodfellow, Harp, Irving, M. Isard, Jia, R. Joze- fowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, Sutskever, K. Talwar, Tucker, Van- houcke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. TensorFlow: Large-scale machine learning on heterogencous systems, 2015. URL ptttp://ttesooffooo org/. Software available from tensorflow.org.
[11] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, Te- jani, S. Chilamkurthy, B. Steiner, L. Fang. J. Bai. and S. Chintala. Pytorch: An imperative style. high- seeh-peeffrmancce deep learning library. Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: httpr//epooc.e...ee//ee/e wwwpeddiinnnnuuuiss..... cc
Paper .dff.
[12] A. Doucet and A. M. Johansen. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook Nonlinear Filtering, 12(656-704)3,, 2009.
[13] N. J. Gordon, D. J. Salmond, and A. F. Smith. Novel approach to nonlinear/non-Gaassia Bayesian state estimation. IEE Proceedings F (Radar and Signal Processing). 140(2):107- 113, 1993.
[14] M. K. Pitt and N. Shephard. Filtering via simulation: Auxiliary particle filters. Journal of the American Statistical Association, 94(446):590-599 1999.
[15] L. lonides, C. Bretó, and A. A. King. Inference for nonlinear dynamical systems. Proceed- ings of the National Academy of Sciences, 188449)1144881888 2006.

---

[16] J. Olsson and J. Westerborn. Efficient purticle-based online smoothing in general hidden markov models: the PaRIS algorithm. Bernoulli, 23(3):155-1196, 2017.
[17] A. Scibior and F. Wood. Differentiable particle filtering without modifying the forward pass. arXiv preprint arXiv: 106. 10314, 2021.
[18] G. Kitagawa and Sato. Monte Carlo smoothing and self-organising state-space model. In A. Doucet, N. De Freitas, and N. Gordon, editors, Sequential Monte Carlo Methods in Practice, pages 177-195. Springer, 2001.
[19] J. Olsson, O. Cappé, R. Douc, and E. Moulines. Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models. Bernoulli, 14(1):155- 179, 2008.
[20] X. Ma, P. Karkus, D. Hsu, and W. S. Lee. Particle filter recurrent neural networks. In AAAI Conference on Artificial Intelligence, 2020.
[21] X. Ma, Karkus, D. Hsu, Lee, and Ye. Discriminative particle filter reinforcement learning for complex partial observations. In International Conference Learning Represen- tations, 2020.
[22] R. Jonschkowski, D. Rastogi, and O. Brock. Differentiable particle filters: End-to-end learning with algorithmic priors. In Proceedings of Robotics: Science and Systems, 2018.
[23] M. Zhu, K. Murphy, and Jonschkowski. Towards differentiable resampling. arXiv preprint arXiv:2004. 10000.//990 /938, 2020.
[24] A. Kloss, G. Martius, and J. Bohg. How to train your differentiable filter. Autonomous Robots, 45(4):561-577 2021.
[25] K. Granstrom, M. Baum, and S. Reuter. Extended object tracking: Introduction, overview and applications. arXiv preprint arXiv:1604.00970 2016.
[26] R. S. S. Blackman, Popoli. Design and Analysis of Modern Tracking Systems. Artech House, Boston, 1999.
[27] K. Granström, S. Reuter, D. Meissner, and A. Scheel. A multiple model PHD approach to tracking of under assumed rectangular shape. In 17th International Conference on Information Fusion (FUSION). IEEE, 2014.
[28] P. Del Moral. Feynman-Kat Kac Formulae. Springer, 2004.
[29] F. Lindsten, P. Bunch, S. S. Singh, and T. B. Schön. Particle ancestor sampling for near- degenerate or intractable state transition models. arXiv preprint arXiv:1505.0633: 2015.
[30] S. G. Krantz and H. R. Parks. Geometric Integration Theory. Springer Science & Business Media, 2008.
[31] A. L. Caterini, G. Loaiza-Ganem, G. Pleiss, and J. P. Cunningham. Rectangular flows for manifold learning. Advances in Neural Information Processing Systems, 34, 2021.
[32] K. M. Lynch and F. C. Park. Modern Robotics. Cambridge University Press, 2017.
10


Reviews:
[{"review_id": "hbZZ8RmOM0t", "reviewer": "Reviewer_swzV", "summary": "This paper proposes a method for learning observation and transition models for birds-eye view multi-car autonomous driving scenarios.  Using a fixed-lag approximation of the score function along with a deterministic motion model, inference of high-dimensional models can be achieved.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe paper is well-written, and the method is generally well-justified.  The method leverages specific assumptions from the bird's eye view autonomous driving scenario to learn observation and transition models. \n\nWeaknesses/Questions:\n\nMore evaluations on the fixed-lag size would be beneficial. At what point is path degeneracy an issue? At what point is a fixed-lag uninformative? Since this most likely depends on the number of particles as well, it would be nice to see a comparison with the fixed-lag window size and number of particles being modified simultaneously.  \n\nWhy does the real data show less performance difference when modifying fixed-lag? Why did real data cause large gradients across all methods? If performance is independent of fixed-lag size, what is the method gaining? Plot shows minimum of L=5 for real-data, but what happens for L=1,…,5? If this small of window size gives similar results for real-data, then learning the observation and transition models may not require longer horizon inference.\n\nIf DPF-SGR performs better in terms of accuracy for 25 steps, what is preventing methods from also limiting the number of steps that is used for training?\n\nSince other methods may or may not differentiate through the marginal log-likelihood, does it make sense to even compare the baselines using this?\n\nRelative bearing is usually not available as a measurement, and estimates would be somewhat noisy.  Is this a realistic setup?\n\nOther comments:\n\nFigure 1 is somewhat wasteful in terms of white space, and also does not illustrate much.  Consider improving the figure as this is the only one in the paper.  One of the figures in the supplementary material may better illustrate the method and application than the current Figure 1.\n\nHow sensitive is the method to the agent's state noise? In reality, this will be imperfect.  \n\nIs assuming the motion model to be deterministic adequate? In real-world scenarios, this may not be the case.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, the method provides is well-written and provides interesting methodology.  While the evaluations demonstrate the effectiveness of the method, some additional experiments demonstrating the benefits of the proposed fixed-lag approximations is needed.  One of the baselines performs better on the 25 step simulate data, while the fixed-lag size does not show much difference in performance on the real data.\n\n\nUpdate:\nThe authors have addressed the main concern about the effect of the fixed-lag length, as well as other questions.  Therefore, I have changed my recommendation to weak accept.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "See above.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}, {"review_id": "d2YbYzFkeiK", "reviewer": "Reviewer_KijS", "summary": "This paper investigates particle-based state estimation under the presence of unknown observation and transition models. This is challenging for a number of reasons, in particular due to the non-differentiable resampling step in particle-based methods. Prior work has proposed leveraging Fisher’s identity to derive a maximum likelihood objective for the model parameters that bypasses the resampling step; however such existing approaches are computationally expensive for large models. This paper builds on this prior work and introduces a particle approximation that trades off bias for computations efficiency. The authors compare this approach with existing baselines and demonstrate superior performance on a real-world and synthetic AV dataset. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* This paper studies a well-motivated problem — in real world scenarios the observation / transition models may be unknown, necessitating methods that can estimate the parameters of these models.\n* The introduction and methods section are well-written. The paper does a good job detailing the shortcomings of prior work, providing background information for the method, and laying out the proposed approach in a step-by-step basis.\n* The experimental results are convincing, demonstrating superior performance compared to a number of baselines across a number of quantitative metrics. \n\nWeaknesses:\n* The paper highlights that this approach is useful in multi-agent settings, yet there is nothing about the method specifically tailored for multi-agent problems. While the experiments focus on a multi-agent setting, this method seems designed for any general state estimation problem. This makes the message of the paper somewhat confusing.\n* Even after reviewing the appendix, the paper provides sparse details about the datasets. Ideally the authors can help to answer the following questions: who collected the real world dataset? What is the nature of the agents in the dataset? In which specific ways are the real and synthetic datasets different?\n* The paper has a limited number of figures and visualizations. It would help to provide visualizations of the datasets and qualitative analysis of the results. Some of these are available in the appendix, perhaps they can be brought up to the main section of the paper.\n* The limitations that the authors describe in section 6, namely scaling this method to more complex empirical domains with occlusions and high-dimensional observations. While these are not strictly necessary components, they can help to strengthen the empirical findings.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "While this paper builds upon a body of prior work, it introduces important technical contributions that enable efficient particle-based estimation for large scale settings. The authors can improve the paper by clarifying their contribution specifically for multi-agent settings, providing more details about the datasets, providing more figures for greater interpretability, and demonstrating results on more complex domains.\n\n**Update**: I appreciate the improvements in clarity regarding the datasets, and the new experiments on (slightly) higher dimensional settings. These are relatively minor changes, so I am keeping my initial recommendation of weak accept.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "issues": "(prefer refer to the list of weaknesses)", "reviewer_expertise": "2: The reviewer is willing to defend the evaluation, but it is quite likely that the reviewer did not understand central parts of the paper"}, "extra_text": {}}, {"review_id": "PEA2WGwsXjQ", "reviewer": "Reviewer_ScN7", "summary": "The paper proposes a new approach for particle filtering approach for estimating the score function of state-space models (SSM). The authors do so using the Fisher Identity to circumvent the non-differentiable sampling step in particle filtering for estimating the score function. Moreover, they circumvent the potential issue of path degeneracy where the particles converge to a single one by using a fixed lag $L$ up to which the estimates are calculated, based on the assumption that observations after a time $t+L$ are not very useful for estimates at the current time step. They also derive the use of a motion model for policy-based approximations by showing that the gradient of the policy corresponds with that of the SSM, allowing it to be plugged into the gradient of the score function. Their results for approximating the states of external objects from Bird's eye views of a vehicle show good log-likelihoods and state estimates on both real and simulated datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n- The proposal for using the Fisher Identity for approximating SSMs enables low variance state estimates and also circumvents the non-differentiable nature of the sampling process.\n- The computational capacity is drastically reduced by considering a fixed time window up to which the estimates are calculated rather than going through the whole trajectory.\n- The paper is very well written. It is very easy to follow and the relevant works, preliminaries and approaches are explained in simple, understandable terms and the paper has a good flow from start to end.\n\n### Weaknesses\n- The paper would benefit from providing some more extensive information about the real dataset that is used, such as how the data was collected, sensors used, example data etc. It would have been better to showcase the results on some existing datasets, like KITTI for example, or even on a subset of it. This would allow the results to be more comprehensible. \n- The paper is currently missing an ablation study to show the extent to which the performance is affected by the different components. These could be different Lag lengths in Sec. 4.2 \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The proposed approach is well motivated and follows a simple, intuitive and useful extension to improve the score functions of SSMs along with the added advantage of reduced computational capacity and incorporating the policy gradient in the score function estimate. The experimental evaluation shows the benefits of the proposed method on both synthetic and real data (although the real dataset is much smaller compared to traditional driving datasets like KITTI for example). Moreover, the paper is very well written and the ideas are presented well, making it a nice read.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "- There are some issues that I found in Lines 140-141 and Eq. 4 as $\\alpha$ and $\\delta$ are not defined. This needs to be cleared to properly understand the approach and how it could potentially affect the score function.\n"}}, {"review_id": "0jadjn8XYYV", "reviewer": "Reviewer_8rdy", "summary": "The model proposes training non-linear, non-Gaussian SSMs using gradients approximated with Fisher's identity. In order to efficiently compute the required smoothing distributions, it proposes using fixed-lag smoothing. The resulting method is evaluated on a state estimation task in an autonomous vehicle setting where 2D poses of surrounding vehicles need to be estimated from observations.  ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well written and easy to follow, and the main idea, intuitions, and mathematical details are clear. \nUsing fixed-lag smoothing to approximate the required smoothing estimates efficiently is a simple idea.\nYet, as this seems to be the paper's main algorithmic contribution the effects and limitations of this approximation would need further investigation. While I agree, that the assumption seems reasonable for many systems, ablations for different smoothing lengths $L$ would be good to see. \nIn general, I do not believe the conducted experiment is sufficient to allow assessment of the method's full potential, as it relies on unrealistic assumptions (lines 218-220), largely pre-engineered transition and observation models, and only considers a narrow scope of applications. \n\n  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "While I like the writing and simplicity of the idea, I think the experiments are not sufficient to allow assessment of the method's potential impact beyond the narrow scope it was evaluated in. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "Besides the above-mentioned points, I have minor issues:\n- At first, I was a bit confused by Section 4.3, because I thought it to be a fundamental aspect of the method, not just a simplification of the dynamics model done for the specific conducted experiments (although I agree that the general method approach described in this section is applicable to a wider class of experiments). Maybe it could be considered to move this part to the experiment section.  \n- If I understand correctly, the MLL is computed based on the filtered estimates, while ADE and AYE use smoothed estimates. Why are different beliefs used here and are the smoothed ones \"fixed-lag\" using the full trajectory? "}}]

Author rebuttal:

Decision: Accept (poster)