Optimization or Architecture: How to Hack Kalman Filtering
Authors: Ido Greenberg
Netanel Yannay
Shie Mannor
Venue: NeurIPS 2023 poster (2023)
Keywords: non-linear filtering
Kalman filter
noise estimation
optimization
Cholesky parameterization
TL;DR: The linear Kalman filter may do better than non-linear neural networks - even in non-linear problems - if only it is optimized accordingly.

Abstract: In non-linear filtering, it is traditional to compare non-linear architectures such as neural networks to the standard linear Kalman Filter (KF). We observe that this mixes the evaluation of two separate components: the non-linear architecture, and the parameters optimization method. In particular, the non-linear model is often optimized, whereas the reference KF model is not. We argue that both should be optimized similarly, and to that end present the Optimized KF (OKF). We demonstrate that the KF may become competitive to neural models – if optimized using OKF. This implies that experimental conclusions of certain previous studies were derived from a flawed process. The advantage of OKF over the standard KF is further studied theoretically and empirically, in a variety of problems. Conveniently, OKF can replace the KF in real-world systems by merely updating the parameters.

Optimization or Architecture: How to Hack Kalman Filtering
Ido Greenberg Technion gido@campus technion.ac. ac. i1
Netanel Yannay ELTA Systems natiy40gmail.om com
Shie Mannor Technion, Nvidia Research shie@ee technion, ac.
Abstract
In non-linear filtering, it is traditional to compare non-linear architectures such as neural networks to the standard linear Kalman Filter (KF). We observe that this mixes the evaluation two separate components: the non-linear architecture, and the parameters optimization method. In particular, the non- lonelinear model is often optimized, whereas the reference KF model is not. We argue that both should be optimized similarly, and to that end present the Optimized KF (OKF). We demonstrate that the KF may become competitive to neural models - if optimized using OKF This implies that experimental conclusions of certain previous studies were derived from flawed process. The advantage of OKF over the standard KF is further studied theoretically and empirically, OKF can replace the KF in real-world systems by merely updating the parameters. Our experiments are published in Github and the OKF in PyPI
variety of problems. Conveniently,
Introduction
The Kalman Filter (KF) [almann 1960]
celebrated method for linear filtering and prediction, with applications in many fields including tracking, navigation, control and reinforcement learning [Zarchan and Musoff 2000 Kirubarajan, 2002 Kaufmann et al. 2023 The KF provides optimal predictions under certain assumptions (namely, linear models with i.i.d noise). In practical problems, these assumptions are often violated, rendering the KF sub-optimal and motivating the growing field of non-linear filtering. Many studies demonstrated the benefits of non-linear models over the KF Revach et al., 2022 Coskun al. 2017].
Originally. we sought to join this line of works. Motivated by a real-world Doppler radar problem, developed dedicated non-linear Neural KF (NKF) based on the LSTM sequential model. NKF achieved significantly better accuracy than the linear KF.
Then, during ablation tests, we noticed that the KF and NKF differ in both architecture and optimiza- tion. Specifically, the KF's noise parameters are traditionally determined by noise estimation [Odelson] al. [2006]; whereas NKF's parameters are optimized using supervised learning, To fairly evaluate the two architectures, we wished to apply the same optimization to both. To that end, we devised an Optimized KF (OKF, Section B KF and OKF have the same linear architecture: OKF only changes the noise parameters values. Yet, unlike KF, OKF outperformed NKF, which reversed the whole experimental conclusion, and made the neural network unnecessary for this problem (Section 4.
Our original error was comparing two different model architectures (KF and NKF) that were not optimized similarly. A review of the non-linear filtering literature reveals that this methodology is used in many studies. Specifically, for a baseline KF model, the parameters are often tuned by noise estimation faa Dai et al. 2020 Aydogmus and Aydogmus 2015 Revach et al.) [2022] by heuristics Jamil et al. 2020 Coskun et al. 201 Ullah et al. 2019 ; or are simply ignored [Gao et al. 2019
Bai et al. 2020 Zheng et al. 2019]. often without public code for examination. Hussein 2014
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Supervised: observations 2(x,
Which train data?
Assumptions hold?
No
Linear architecture?
No
Most non-linear filtering literature
Observations {2)
Yes
Yes
Most KH
Noise estimation
Optimize Q.R: OKF (ours)
lnterature
(KF Algorithm
Our scope
Figure I: Since the KF assumptions are often violated, noise estimation does not optimize the MSE. Instead, our method (OKF) optimizes the MSE directly. In particular, ncural network models should be tested against OKF rather than the non-optimized KF in contrast to the common practice in the literature.
even discusses the (Extended-) KF sensitivity to its parameters, and suggests a neural network with supervised learning- - yet never considers the same supervised learning for the KF itself. In all these 10 studies, the non-linear model's s added value cannot be inferred from the experimental evidence.
So far, OKF is presented as a methodological contribution: for comparison with non-linear methods, forms a more coherent baseline than KF. In addition, OKF provides a practical contribution:
achieves more accurate filtering than the KF, using identical architecture. This is demonstrated extensively in Section 5 and Appendix B in different domains, over different problem variations, using different KF baselines, with different data sizes, and even under distributional shifts.
Discrepancy of objectives: The advantage of OKF over KF may come as a surprise: KF's standard noise estimation is known to already obtain the MSE-optimal parameters! However, this optimality relies on unrealistic assumptions, often considered "fairy tales for undergraduates" [Thomson 1994.. When violated, a conflict emerges between noise estimation and MSE optimization. We study this in detail: (a) Section 5 analyzes the conflict theoretically under certain assumption violations; (b) Appendix B.I shows that even oracle noise estimation cannot optimize the MSE; (c) Appendix [B.2 shows that when using noise estimation, the MSE may degrade with more data, In this light, our findings can be summarized as follows (also see Fig. D::
Contribution: (a) We observe that in most scenarios, since the KF assumptions do not hold, noise estimation is not a proxy to MSE optimization. (b) We thus present the Optimized KF (OKF), also available PPPI package We analyze (theoretically and empirically) the consequences of neglecting to optimize the KF: (C) the standard KF tuning method leads to sub-optimal predictions; (d) the standard methodology in the literature compares an optimized model a non-optimized KF, hence may produce misleading conclusions. Note that we do not argue against the non-linear models; rather, we claim that their added value cannot be deduced from flawed experiments.
Scope: We focus on the supervised filtering setting, where training data includes both observations and the true system states (whose prediction is usually the objective). Such data is available in many practical applications. For example, the states may be provided by external accurate sensors such as GPS fa Dai et al. [2020]: by manual object labeling computer vision [Wojke et al. 2017 by controlled experiments of radar targets; or by simulations of known dynamic systems.
As demonstrated in the 10 studies cited above, this supervised setting is common in non-linear filtering. In linear Kalman filtering, this setting seems to be solved by trivial noise estimation; thus, the literature tends overlook and instead focuses on settings that do not permit trivial noise estimation, e.g., learning from observations alone. Nevertheless, we argue that even in the supervised setting, noise estimation is often not a proxy to MSE optimization, and thus should often be avoided.
2 Preliminaries
Consider the KF model for dynamic system with no control signal Kalman 1960
(1) X, is the system state at time I, and its estimation is usually the goal. Its dynamics are modeled by the linear operator Fr, with random noise W, whose covariance is Q. Z, is the observation, modeled by the operator H. with noise i4 whose covariance is R. The notation may be simplified F, H in the stationary case.
X = F.X, + We (we ~ (0, Q)).
Z, = H.X, + P. (M,, R)).

---

Predict
& == P.&e
P = F.P.F.t +
R PP
r+1 I+1
Pr
[=t+1
Update
K = Pe HT (H H. + R) R)
A
2, = of + K H&RE)
Po = (1-
Figure 2:
The KF algorithm. The prediction step is based on the motion model F3 with noise Q. whereas the update step is based on the observation model H. with noise ft.
The KF represents X, via estimation of the mean 2; and covariance P.. As shown in Fig. 2 the KF alternately predicts the next state (prediction step), and processes new information from incoming observations (update or filtering step). The KF relies on the matrices F,, H, Q, R. intended to represent F., H., Q, R of Eq. ① Whenever F,, H, are known and stationary, we may simplify the notation to F. F, H, H.
The KF estimator é; is optimal in terms of mean square errors (MSE) but only under a restrictive set of assumptions Kalmmnn 1960
H, are known and independent of X, (linear models); each sequence {we], }vr} is i.id; the covariances Q Q, R R are known; and &o. Po correspond to the mean and covariance of the initial Xo-
Assumption 1 (KF assumptions). F, = F,, fI,
Theorem (KF optimality; e.g., see Jazwinski (007). Humpherys et (2012). Under Assump- tion L the KF estimator 2, minimizes the MSE w.r.t. X.-
The KF accuracy strongly depends its parameters Q and R [Formentin and Bittanti 2014]. As motivated by Theorem [ these parameters are usually identified with the noise covariance Q, f2 and set accordingly: "the systematic and preferable approach determine the filter gain is to estimate the covariances from data" [Odelson et al., |006). In absence of system state data {x²} (the "ground truth"). many methods were suggested to estimate the covariances from observations {ze) alone [Mchra 1970, Zanni et al. 2017 Park et al. 2019 Feng et al., 2014). We focus on the supervised setting, where the states { are available in the training-data (but not inference).
Definition (Supervised data). Consider K trajectories of a dynamic system, with lengths {Ti AK We define their supervised data as the sequences true system states Tk E Rdd and observations {((K,,) 2 1/ K 1/k=1
1==1
Zk,t E Rd=:
If F,, H; are known, the supervised setting permits a direct calculation of the sample covariance matrices of the noise [Lacey] 1998]:
= Cov( (1 - F.,,, î Cou(( H.Hk,t)k.l
Since Theorem 1 guaraantees optimality when Q Q, R = R. and Eq. 2 provides a simple estimator for Q and R, Algorithm[]] has become the gold-standard tuning method for KF from supervised data. Algorithm 1 (KF noise cstimation). Given supervised data {(Z,,e, =k.e)) return Q and Å. of Eq. (2.
While Algorithm T indeed trivial to apply in the supervised setting, show below that when Assumption [ is violated, it no longer provides optimal predictions. Violation of Assumption can be partially handled by certain variations of the KF, such Extended KF (EKF) [Sorenson 1985 and Unscented KF (UKF) Waan and Van Der Merwe 2000).
Optimized Kalman Filter
Estimation of the KF noise parameters Q, R has been studied extensively various settings; yet, in supervised setting it is trivially solved by Algorithm I However, once Assumption lis violated,
3

---

such noise estimation is no longer a proxy to MSE optimization despite Theorem Instead, in this section we propose to determine Q and R via explicit MSE optimization. We rely on standard optimization methods for sequential supervised learning; as discussed below, the main challenge is to maintain the Symmetric and Positive Definite (SPD) structure of Q, R as covariance matrices.
Formally, we consider the KF (Fig- 2 as a prediction model 0g,e ( + Q, R), which estimates Tket given the observations {= 1r=1 and parameters Q, ft. We define the KF optimization problem:
K Tk
argmin Q'R
loss (#x,1 ((=,,))t=s Q', R') ZR.)),
s.t. Q' E S4 R' S (3)
where Sd C. Rddd is the space of Symmetric and Positive Definite matrices (SPD), and loss(-) is the objective function (e.g.. loss($, 2²) || z²12 for MSE). Prediction of future states can be expressed using the same Eq. (3 by changing the observed input from (2k,))== to {Zk,7 )
significant challenge in solving Eq. 33) is the SPD constraint, Standard numeric supervised opti- mization methods (e.g., Adam [Diederik P. Kingma 2014) may violate the constraint. While the SPD constraint is often bypassed using diagonal restriction Li et al. 2019 Formentin and Bittanti 2014].
this may significantly degrade the predictions, as demonstrated in the ablation tests in Appendix B.4 Instead, to maintain the complete expressiveness of Q and I, we use the Cholesky parameterization Pinheiro and Bates 1996].
The parameterization relies on Cholesky de- composition: any SPD matrix A E Rddd can be written as A = LL" where L is lower- triangular with positive entries along its di- agonal. Reversely, for any fower-trianger with positive diagonal, LL SPD. Thus, to represent an SPD A E Rdxd, we define A(L) == L.I. and parameterize [.) to be lower-triangular, have positive diagonal, and be differentiable the parameters
Algorithm 2: Optimized Kalman Filter (OKF)
Input: training data {( 2k,)) (Definition Dr batch size b; loss function (e.g..
MSE); optimization step function (e.g., Adam)
dæ Ien(2z),,, de e len(2,,) 3 Initialize 0g E (d)+++,, On E (d.++)
while training finished do
// Get Q, R. using Eq.
4
Q + ((O)) R L(ON)
camplee(11 (11,.., K}, size=b)
C 0
0
if <
8
for le in K. do
(I(0))
e Baca 1)/21
if -i=s j.
Initialize à E Rd. for in 1 Th do // KF stepa (Fig. 2
01--2)(-1)/2+3
if j.
10
where 0 E Rddd+11/2
11
$ + KF_predict (;; Q)
Both Cholesky parameterization and sequen- tial optimization methods are well known tools. Yet, for KF optimization from super- vised data, we are not aware of any previous attempts apply them together, noise es- timation (Algorithm)
12
i - KF update (t, 2n,1i R)
13
C C + los, zk.e)
02, RR optimization_steppe (fq. 0R))
15 Return Q, R
I is typically preferred.
We wrap the optimization process in the Optimized KF (OKF) in Algorithm 2 which outputs optimized parameters Q, R for Fig. 2 Note that Algorithm
2 optimizes the state estimation current time t. By switching Line 12 and Line 13 the optimization will instead be shifted to state prediction at the next time-step (as à becomes oblivious to the current observation Z4,z).
Lack of theoretical guarantees: Assumption cannot be trusted. neither noise estimation (Algorithm) 1 nor OKF (Algorithm [ can guarantee global optimality. Still, OKF pursues the MSE objective using standard optimization tools, which achieved successful results many non-convex problems (Zhong et al., 2020) over millions of parameters [Devlin et al., 2019). On the other hand, noise estimation pursues conflicting objective, and guarantees significant sub-optimality certain scenarios, analyzed in Section

---

trojectory abservation
15500
14000
KF
NKF
NKF
OKF
14500
OKF
NKF
MSE
MSE
12000
12500
10000
[12, 24] 124, 481 14B, 96) acceleration range
[04, 08) (OB, 161 116, 32) acceleration range
Figure 3: A sample trajectory and the corresponding predictions (pro- jected onto XY plane), in the Free- motion benchmark. The standard KF provides inaccurate predictions in certain turns.
(a) Free-motion benehmark
(b) No-tums benchmark
Figure 4: Test errons and 95% confidence intervals, over targets with different accelerations. The middle acceleration range coincides with the training accelerations (24-48 in (a) and 8- 1-16 in (b)), and the other ranges correspond to out-of-distribution generalization.
4 OKF vs. Neural KF: Is the Non-Linearity Helpful?
In this section, we demonstrate that comparing an optimized neural network non-optimized baseline may lead to incorrect conclusions: the network may seem superior even if the complicated architecture has no added value. The implied message is not against neural networks, but rather that evaluating them against non-optimized baseline carries crucial flaw.
The Doppler radar problem: We consider variant of the classic Doppler radar problem [Barton 1988 Roy and Mitra 2016]. where various targets trajectories are tracked in a homogeneous 3D space, given regular observations of Doppler radar. The state X (Igg Zy Tz Tuzs Fuy Tuu) R consists of 3D location and velocity. The goal is to minimize the MSE over the 3 location coordinates. While the true dynamics F are unknown to the KF, constant- constant-velocity model F can be used:
F
An observation Z TR consists of the location in spherical coordinates (range, azimuth, elevation) and the radial velocity (the Doppler signal), with an additive i.i.d Gaussian noise. After transformation to Cartesian coordinates, the observation model can be written
H = H = (
(6)
where r = 2/2 + 2 x3. Since = H(X) relies on the unknown location (Izz Ty. R2), we
instead substitute H = H (Z) in the KF update step in Fig.
Neural KF: The Neural Kalman Filter (NKF) incorporates an LSTM model into the KF framework.
as presented in Appendix and Fig. T6 We originally developed NKF to improve the prediction of
the non-linear highly-maneuuvering targets in the Doppler problem (e.g.. Fig. 3). and made honest efforts to engineer a well-motivated architecture. Regardless, a methodological flaw when comparing any optimized filtering method to the KF;; this methodological argument stands regardless of the technical quality of NKF. In addition, Appendix C presents similar results for other variants of NKF,
stress that this section demonstrates
Experiments: We train NKF and OKF on a dataset of simulated trajectories, representing realistic targets with free motion (as displayed in Fig. B). As second benchmark, we also train on a dataset of simplified trajectories, with speed changes but with no turns. The two benchmarks are specified detail in Appendix B. and correspond to Fig. LTd and Fig. The We tune the KF from the same datasets using Algorithm|] [ In addition to out-of-sample test trajectories, we also test generalization to ont-o--distriuution trajectories, generated using different ranges of target accelerations (affecting both speed changes and turns radiuses).
5

---

Fig. summarizes the test results. Compared to KF, NKF reduces the errors in both benchmarks, suggesting that the non-liness architecture pays off. However, optimization of the KF (using OKF) reduces the errors even further, and thus reverses the conclusion. That is, the advantage of NKF in this problem comes exclusively from optimization, and not at all from the expressive architecture.
OKF vs. KF:
Section 4 presents the methodological contribution of OKF for non-linear filtering, an optimized baseline for comparison, instead of the standard KF. this section, we study the advantage of OKF over the KF more generally. We show that OKF consistently outperforms the KF in a variety of scenarios from different domains. This carries considerable practical significance: unlike neural models, shifting from KF to OKF merely requires change of the parameters Q, R, hence can be deployed to real-world systems without additional overhead, complexity or latency inference.
Recall that by Theorem D the KF already optimal unless Assumption | is violated. Thus, the violations are discussed in depth, and the effects of certain violations are analyzed theoretically.
5.1 Doppler Radar Tracking
Theorem guarantees the optimality of Algorithm [ (KF). Yet, in Section 4 OKF outperforms the while the Doppler problem of Section
KF. This is made possible by the violation of Assumption T may not seem complex, the trajectories follow non-linear motion model (as displayed in Fig. 3).
Imagine that we simplified the problem by only simulating constant-velocity targets, making the true motion model F linear. Would this recover Assumption| and make OKF unnecessary? The answer is no; the adventurous reader may at- tempt to list all the remaining violations before reading on.
The simulated targets move mostly horizontally, with limited elevation changes. This is not ex- pressed by the KF's initial state distribution (fo, Po). To remedy this, may simulate mo- tion uniformly in all directions. A third viola- tion comes from the observation noise. While the radar noise is i.i.d in spherical coordinates (as mentioned in Section 4). it is not i.i.d in Cartesian coordinates (see discussion in Ap- pendix A.2). To overcome this, one may simu- late radar with (physically-impossible) Carte- sian i.i.d noise. This results the unrealistically- simplified problem visualized in Fig. []
Figure 5:
The original Doppler problem (left) is sim- plified to toy problem (right), with linear motion, isotropic flying directions and physically-impossilee radar. After all the simplifications, Assumption still does not hold, thus Algorithm iis still sub-optimal and outperformed by OKF
Despite the simplifications, it turns that Assumption still not met, as the observation model in Eq. 66 is still not linear (i.e., H = H(X) is not constant). As shown by Proposition 1 this single violation alone results in significant deviation of Algorithm from the optimal parameters.
We first define the simplified problem.
Problem 1 (The toy Doppler problem). The toy Doppler problem is the filtering problem modeled by Eq. TT) with constant-velocity dynamics F (Eq. (5)) Doppler observation H (Eq. ( and
Q = 0 6
R
where o., Og, O2, 0D > 0.
Recall that H = -XX) in Eq. 6 depends on the state X, which is unknown to the model. Thus, we
assume that H (X) is used in the KF update step (Fig. 2). with some estimator X ~ X (c.g..

---

H = (Z) in Section Hence, the effective noise is R = Cox(Z-HX) + Cov(Z- - R. Proposition analyzes the difference between R and R. To simplify the analysis, we further assume that the error X- - X within H ge.g.. Z - X) is independent of the target velocity.
Proposition 1. In the toy Doppler Problem 1 with the estimated observation model H, the effective observation noise R = Cov(Z HX) is:
R
R
where C = [[E[[[||l]] is the asymptotic lower bound ("big omega") of the expected square velocity u. particular, C > 0 and is unbounded as the typical velocity grows.
Proof sketch (see complete proof in Appendix A.I. We have Cou(Z H Cov(Z HX (H X) R + Cov( ))) where the last equality relies on the independence between the target velocity and the estimation error X - X. We then calculate Coo((H H)
Proposition T hhas an intuitive interpretation: when measuring the velocity, Algorithm onlly considers the inherent Doppler signal noise dp- However, the effective noise OD + also includes the trans- formation
Dop
X
Dop
3039
59
10062
96
21
29700
49
-6
from Doppler the Cartesian ordinates, caused by the uncertainty in ((X) itself. Notice that heuristic solutions such inflation of R would not recover the effective noise R, which only differs from R in one specific entry.
9050
D69
+8
97
(a) KF ft
(b) OKF/ R
Figure 6: The parameters A learned by KF and OKF the toy Doppler problem. The rows and columns' entries correspond to location (x, y. 2) and radial ve- locity (Doppler). The simulated noise variance is 100 for the positional dimensions and 52 for veloc- ity, and estimated accurately by the KF. However, OKF increases the noise associated with velocity, in accordance with Proposition] [] The decrease in the positional variance comes from scale-invariance in the toy problem, discussed in Appendix [..
Yet, as demonstrated below, OKF captures the ef- fective noise R successfully. Critically, does so from mere data: OKF does not require the user to specify the model correctly, or to even be aware of the violation of Assumption]
Experiments: We test KF and OKF on the toy Problem using the same methodology as in Sec- tion nn accordance with Proposition T OKF adapts the Doppler noise parameter: as shown in Fig. [6] increases ap in proportion to the location noise by factor of 13. Note that we refer to the proportion instead of absolute values due to scale-invariance in the toy problem, discussed in Appendix(A.] Following the optimization, OKF reduces the test MSE by 44% from 152 to 84.
Extended experiments:
This section and Section test OKF against KF in three specific variants of the Doppler problem. One may wonder if OKF's advantage generalizes other scenarios, such as:
Different subsets of violations of Assumption
Other baseline models than KF, e.g., Optimized Extended-KF;
Small training datasets;
MSE (/103)
16
Generalization to dittr--distiiuttion test data.
model
The extended experiments in Appendix address all of the con- cerns above by examining wide range of problem variations in the Doppler radar domain. addition, other domains are exper- imented below. In all of these experiments, OKF outperforms Algorithm T in terms of MSE.
KF
15
OKF
10a
10a
train samples
Finally, the goal-misallen maalinmeitg of Algorithm T is demonstrated di- rectly by two results: even oracle noise estimation fails to optimize the MSE (Appendix Bb and feeding more data to Algorithm may degrade the MSE (Appendix B.2).. Fig. presents sample of the results of Appendix B.2
Figure 7: Different train data sizes in the Doppler problem (Fig. B. Due to objective misalignment, Al- gorithm [ deteriorates with more train data.
7

---

grou
MOT20 -05: RAW
Figure 8: A sample of 2 trajectories in the first frame of MOT20 test video, along with the predictions of KF and OKF
Z-value(KF--O
125
2.0
z-value(KF-oo (KFF)) 17
square error
1.5
KF
square error
800
OKF
OKF
1.0
400
0.5
80
100
quantile [%]
quantile [%)]
(a) Video tracking
(b) Lidar-basee state estimation
Figure 9: Summary of the errors in the video and lidar problems. The dashed lines correspond MSE
Both z-values correspond to p-value 10-6, Each z-value is calculated over test trajectories as follows:
mean(( /N, where A, eFr,(KF) (rr,(OKF) is the square-error difference trajectory (A.)) 1 isis N.
5.2
Video Tracking
The MOT20 dataset (Dendorfer al. 2020) contains videos of real-world targets (mostly pedestrians, as shown in Fig. 8). along with their true location and size in every frame. For our experimental setup, since object detection is out of the scope, assume that the true locations known in real-time. The objective to predict the target location the next frame. The state space corresponds to the 2D location, size and velocity, and the observations include only the location and size. The underlying dynamics F the pedestrians are naturally unknown, and the standard constant-velocity model is used for F. This results in the following model:
00
Notice that the known observation model H H is linear (H is independent of X), hence poses substantial difference from Section 5.1 in terms of violations of Assumption []]
The first three videos with 1117 trajectories are used for training, and the last video with 1208 trajectories for testing. As shown in Fig. OKF reduces the test MSE by 18% with high statistical significance.
5.3 Lidar-based State Estimation in Self Driving
Consider the problem of state-estimation in self-driving, based on lidar measurements with respect to known landmarks [Moreira et al. 2020). The objective is to estimate the current vehicle location. We assume single landmark (since the landmark matching problem out of scope). We simulate driving trajectories consisting of multiple segments, with different accelerations and turn radiuses

---

(see Fig. 15a in the appendix). The state is the vehicle's 2D location and velocity. and F is modeled according to constant-velocity. The observation (both true H and modeled H) corresponds to the location, with an additive Gaussian i.i.d noise polar coordinates. This results in the following model:
0101 0010
= - (0100)
0110
0001
We train KF and OKF over 1400 trajectories and test them on 600 trajectories. As shown in Fig. 8b OKF reduces the test MSE by 10% with high statistical significance.
(.. Notice that the lidar problem differs from Section 1..1in in the linear observation model H, and from Section 5.2 in the additive noise. Both properties have major impact on the problem, as analyzed in Proposition 1 and below, respectively.
Theoretical analysis: As mentioned in Section 5.1 and discussed in Appendix AA.2| the i.i.d noise in polar coordinates is not i.idd in Cartesian ones. To isolate the i.i.d violation and study its effect, we define a simplified toy model with simplified states, no-motion model F, isotropic motion noise Q and only radial observation noise. Note that in contrast to Section 5.1 already linear.
the observation model is
Problem (The toy lidar problem). The toy lidar problem is the filtering problem modeled by Eq. ① with the following parameters:
1 1) 1 Q ) Rpotar
To 9)
F= H
R
for some unknown > O, with observation noise drawn i.idd from (00, Rpoter) in polar coordi- nates. The initial state X, follows radial distribution (i.e., with PDF of the form f((mm))))
Proposition 2. As the number N of train trajectories in Problem 2 grows, the noise parameter Ry (KF) estimated by Algorithm converges almost surely:
(KFFF) " Rest (TO)
(ro/2
ro/2
On the other hand, under regularity assumptions, the MSE is minimized by the parameter Ropt (T P), where r < ro/2.
Proof sketch (see complete proof Appendix A.2 For Rest. we calculate E[Rx and use the law of large numbers. For the calculation, we transform Rpolar to Cartesian coordinates using the random direction variable 0, and take the expectation over 0 U[, 2T))). The uniform distribution of 0 comes from the radial symmetry of the problem. For Ropt. we calculate and minimize the expected square error directly.
Intuitively, Proposition 2 shows that the i.dd violation reduces the effective noise. Note that the analysis only holds for the unrealistic toy Problem [ The empirical setting in this section less simplistic, and generalizing Proposition is not trivial. Fortumately, OKF optimizes directly from the data, and does not require such theoretical analysis. Fig. [h]ss that indeed, in accordance with the intuition of Proposition 2 OKF learns to reduce the values of it comparison to KE This results in reduced test errors specified above.
y
y
179
3
49
2
3
157
2
51
(a) KF Á
(b) OKF it
Figure 10: The parameters îž learned in the lidar problem. From data alone, OKF learns to decrease the noise parameters, consistently with Proposition 2

---

Related Work
Noise estimation of the KF parameters from observations alone has been studied for decades, as supervised data (Definition is often unavailable. Various methods were studied, based on autocorrelation [Mehra 1970 Carew and Belanger, 1973], EM [Shumway and Stoffer [005] and others [Odelson et al. 2006 F Feng et 2 Park et al. 2019). When supervised data is available, noise estimation reduces to Eq. (2) and considered a solved problem [Odelson et al. 2006]. We
show that while noise estimation is indeed easy from supervised data, it may be the wrong objective
to pursue.
Optimization: We apply gradient- based optimization to the KF with respect to its errors. In absence of supervised data, gradient-based optimization was suggested for other losses, such as smoothness [Barattt and Boyd, 2020]. In the supervised setting, noise estimation is typically preferred [Odelson al.) 2006], although optimization without gradients was suggested in Abbeel et al. [2005]. In
practice, "optimization" of KF is sometimes handled by trial and error [lamil et al. 2020] or grid
search [Formentin and Bittanti 20144] Coskun et 2017]. other cases, Q and
restricted to
be diagonal [Li et al. 2019 Formentin and Bittanti 2014]. However, such heuristics may not suffice
when the optimal parameters take non-trivial form.
Neural Networks (NNs) in filtering:
The NKF in Section relies on a recurrent NN. NNs are widely used in non-linear filtering, e.g., for online prediction [Gao et al.) 2019 Iter et al., 2016 Coskun et al.
2017 fa Dai et al. 2020 Belogolovsky et al., [022]. near-online prediction [Kim et al. [018]] and
ofiline prediction Liu et 2019b]. Learning visual features for tracking via NN was suggested by
Wojke et al.
2017]. NNs were also considered for related problems such as data association Liu
ct al.
2019a model switching [Deng et al. 2020]. and sensors fusion (Sengupta al. 2019).
In addition, all the 10 studies cited in Section ! [sed a NN model for non-linear filtering, with either KF or EKF as baseline for comparison. As discussed above, none has optimized the baseline model to a similar extent as the NN. As demonstrated in Section such methodology could lead to unjustified conclusions.
7 Summary
We observed that violation of the KF assumptions is common, and is potentially difficult to notice or model. Under such violation, we analyzed (theoretically and empirically) that the standard noise estimation of the KF parameters conflicts with MSE optimization. An immediate consequence is that the KF often used sub-optimally. A second consequence is that in many works in the literature, where a neural network is compared to the KF, the experiments become inconclusive: they cannot decide whether the network succeeded due to superior architecture, or merely because its parameters were optimized. We presented the Optimized KF (OKF), and demonstrated that it can solve both issues (Section 5 and Section respectively).
From a practical point of view, the OKF is available on PyPI and is easily applicable to new problems. Since its architecture is identical to the KF, and only the parameters are changed, the learned model causes neither inference-time delays nor deployment overhead. All these properties make the OKF powerful practical tool for both linear and non-linear filtering problems.
Acknowledgements
The authors thank Tal Malinovich, Ophir Nabati, Zahar Chikishev, Mark Kozdoba, Eli Meirom, Elad Sharony, Itai Shufaro and Shirli Di-Castro for their helpful advice. This work was partially funded by the European Union's s Horizon Europe Programme, under grant number 101070568.
10

---

References
Pieter Abbeel, Adam Coates, Michael Montemerlo, Andrew Ng, and Sebastian Thrun. Discriminative training of kalman filters. Robotics: Science and systems, pages 289-296, 06 2005.
Zafer Aydogmus and Omur Aydogmus. A comparison of artificial neural network and extended kalman filter based sensorless speed estimation. Measurement, 63:152-158, 2015. ISSN 0263- 2241.
Yu-ting Bai, Xiao-yi Wang, Xue-bo Jin, Zhi-yao Zhao, and Bai-hai hai Zhang. A neuron-based kalman filter with nonlinear autoregressive model. Sensors, 20(1):299, 2020.
S. T. Barratt and S. P. Boyd. Fitting kalman smoother to data. In 2020 American Control Conference
(ACC), pages 1526-1531, 2020.
David K Barton. Modern radar system analysis. Norwood, 1988.
Stav Belogolovsky, Ido Greenberg, Danny Fitan, and Shie Mannor. Continuous forecasting via neural eigen decomposition of stochastic dynamics. arXiv preprint arXiv:2202.00111 17, 2022.
B. Carew and Belanger. Identification of optimum filter steady-state gain for systems with unknown noise covariances. IEEE Transactions on Automatic Control, (8(6):582-887, 1973.
Huseyin Coskun, Felix Achilles, Robert DiPietro, Nassir Navab, and Federico Tombari. Long short-term memory kalman filters: Recurrent neural estimators for pose regularization. ICCV, 2017.
Patrick Dendorfer, Hamid Rezatofighi, Anton Milan, Javen Shi, Daniel Cremers, Ian Reid, Stefan Roth, Konrad Schindler, and Laura Leil- Mor20: A benchmark for multi object tracking in crowded scenes, 2020. URL https://motchaaleenne wwtpeeeeeeeeaatateeeeeeeee
Lichuan Deng. Da Li, and Ruifang Li. Improved IMM Algorithm based on RNNs. Journal of Physics Conference Series, 0118:012055 April 2020.
Jacob Devlin, Ming-Wos Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
Jimmy Ba Diederik P. Kingma. Adam: A method stochastic optimization, 2014. URL https
//arxiv. org/ /a4/s.//12.... 6980
Hai fa Dai, Hong wei Bian, Rong ying Wang, and Heng Ma. An ins/gnss integrated navigation in gnss denied environment using recurrent neural network. Defence Technology. 16(2):334-340) 2020. ISSN 2214-9147.
B. Feng, M. Fu, H. Ma, Y. Xia, and B. Wang. Kalman filter with recursive covariance es- timation- segiion--equuttiallll estimating process noise covariance. IEEE Transactions on Industrial Electronics, 61(11)):253-6666 2014.
Simone Formentin and Sergio Bittanti. An insight into noise covariance estimation for kalman filter design. IFAC Proceedings Volumes, 47(3):2358-22)) 2363, 2014. ISSN 1474-6670, 19th IFAC World Congress.
Chang Gao, Junkun Yan, Shenghua Zhou, Bo Chen, and Hongwei Liu. Long short-term memory- based recurrent neural networks for nonlinear target tracking. Signal Processing, 164, 05 2019.
Matthieu Geist and Olivier Pietquin. Kalman filtering colored noises: the (autoregressive) moving- average case. In MLASA 2011, pages 1-4, Honolulu, United States, December 2011.
Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. Neural Computation, 1997.
Jeffrey Humpherys, Preston Redd, and Jeremy West. A fresh look at the kalman filter. SIAM Review, 54(():801-822 823, 2012.
Ala A. Hussein. Kalman filters versus neural networks in battery state-of-charge estimation: comparative study. International Journal of Modern Nonlinear Theory and Application, 2014.
11

---

Dan Iter, Jonathan Kuck, and Philip Zhuang. Target tracking with kalman filterin,, knn and Istms, 2016.
Faisal Jamil al. Toward accurate position estimation using learning prediction algorithm in indoor navigation. Sensors, 2020.
Andrew H. Jazwinski. Stochastic processes and filtering theory. Courier Corporation, 2007.
R. E. Kalman. A New Approach to Linear Filtering and Prediction Problems. Journal of Basic Engineering, 82(1):35 45, 03 1960. ISSN 0021-9223.
Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. Champion-leve drone racing using deep reinforcement learning. Nature, 2023.
Chanho Kim, Fuxin Li, and James M. Rehg. Multi-object tracking with neural gating using bilinear 1stm. ECCV, September 2018.
Yaakov Bar-Shalom X.-Rong Li Thiagalingam Kirubarajan. Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software, John Wiley and Sons, Inc., 2002.
http://wwbbmmttt..ee///m Tony Lacey. Tutorial: The kalman filter, 1998, URL /t/pp//tt//bbbbttttt////rrrrrttt
ilter. .ed...df""
S. Li, C. De Wagter, and G. C. H. E. de Croon. Unsupervised tuning of filter parameters without ground-truth applied to acrial robots. IEEE Robotics and Automation Letters, 4(4):4102-4107, 2019.
Huajun Liu, Hui Zhang, and Christoph Mertz. Deepda: Lstm-based deep data association network for multi-targets tracking in clutter. CoRR, hat//197.09915.. 2019a.
Jingxian Liu, Zulin Wang, and Mai Xu. Deepmit: A deep learning maneuvering target-tracking algorithm based bidirectional Istm network. Information Fusion, 53, 06 2019b.
R. Mchra. On the identification of variances and adaptive kalman filtering. IEEE Transactions Automatic Control, 1552)115--144 1970.
A. Paulo Moreira, Paulo Costa, and José Lima. New approach for beacons based mobile robot localization using kalman filters. Procedia Manufacturing, 51:512-519, 2020. 30th International Conference on Flexible Automation and Intelligent Manufacturing (FAIM202I).
Dominic A. Neu, Johannes Lahann, and Peter Fettke. A systematic literature review on state-of-the-arr deep learning methods for process prediction. CoRR, abs/2101.092 .03200,0000 2021.
Brian Odelson, Alexander Lutz, and James Rawlings, The autocovariance least-squares method for estimating covariances: Application to model-based control of chemical reactors. Control Systems Technology, IEFE Transactions on, 14:532 540, 06 2006.
Sebin Park et al. Measurement noise recommendation for efficient kalman filtering over a large amount sensor data. Sensors, 2019.
José C. Pinheiro and Douglas M. Bates. Unconstrained parameterizations for variance-covaaiance matrices. Statistics and Computing, 6:289-296, 1996.
Guy Revach, Nir Shlezinger, Xiaoyong Ni, Adria Lopez Escoriza, Ruud JG Van Sloun, and Yonina C Eldar. Kalmannet: Neural network aided kalman filtering for partially known dynamics. IEFE Transactions on Signal Processing, 70:1532-152 2022.
Anirban Roy and Debjani Mitra. Multi-target trackers using cubature kalman filter for doppler radar tracking in clutter. IET Signal Processing, 10(8):888-905 901, 2016.
David E. Rumelhart et al. Learning representations by back-propagating errors. Nature, 1986.
A. Sengupta, F. Jin, and S. Cao. A dnn-Istm based target tracking approach using mmwave radar and camera sensor fusion. 2019 IEEE National Aerospace and Flectronics Conference (NAECON), pages 688-693, 2019.
12

---

Robert H. Shumway and David S. Stoffer. Time Series Analysis and Its Applications (Springer Texts in Statistics). Springer- Verlag, Berlin, Heidelberg, 2005, ISBN 0387989501.
Harold Wayne Sorenson. Kalman Filtering: Theory and Application. IEEE Press, 1985.
D. J. Thomson. Jackknifing multiple-window spectra. In Proceedings of ICASSP '94, IEEE International Conference Acousties, Speech and Signal Processing, volume vi, pages V1/73- V1/76 vol.6, 1994.
Israr Ullah, Muhammad Fayaz, and DoHyeun Kim. Improving accuracy of the kalman filter algorithm in dynamic conditions using ann-based learning module. Symmetry. II(1), 2019. ISSN 2073-8994.
E. A. Wan and R. Van Der Merwe. The unscented kalman filter for nonlinear estimation. Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, pages 153-158, 2000.
Nicolai Wojke, Alex Bewley, and Dietrich Paulus. Simple online and realtime tracking with deep association metric. 2017 IEEE International Conference on Image Processing (ICIP), pages 3645-3649, 2017.
L. Zanni, J. Le Boudec, R. Cherkaoui, and M. Paolone. A prediction-erro covariance estimator for adaptive kalman filtering in step-varying processes: Application to power-system state estimation. IEEE Transactions Control Systems Technology, 25(5):1683-166) 2017.
Paul Zarchan and Howard Musoff. Fundamentals of Kalman Filtering: A Practical Approach. Amer-
ican Institute of Aeronautics and Astronautics, 2000.
Tianyu Zheng, Yu Yao, Fenghua He, and Xinran Zhang. An rnn-based learnable extended kalman filter design and application. 2019 18th European Control Conference (ECC), pages 3304-3309, 2019.
Hui Zhong, Zaiyi Chen, Chuan Qin, Zai Huang, Vincent W. Zheng, Tong Xu, and Enhong Chen. Adam revisited: a weighted past gradients perspective. Frontiers of Computer Science, 2020.
13

---

Contents
Introduction
2 Preliminaries
Optimized Kalman Filter
4
OKF vs. Neural KF: Is the Non-Linearrty Helpful?
OKF vs. KF:
6
Related Work
10
Summary
10
Theoretical Analysis
15
OKF: Extended Experiments
17
Neural KF: Extended Discussion and Experiments
22
14

---

A Theoretical Analysis
A.1 Non-linear Observation
In this section, we discuss the relation between the theoretical analysis of Proposition and the empirical results shown in Fig. 6 Then, provide the proof of Proposition [
Fig. 6 vs. Proposition I Fig. 6 displays the noise parameters f learned by OKF in the toy problem.
In accordance with Proposition the noise aD associated with Doppler is increased compared to the true measurement noise R. In fact. not only a is increased, but also the positional variances are decreased, which is not explained by Proposition T This phenomenon origins in the absence of dynamics noise in this toy problem (Q = 0), which leads scale-invariance w.r.t. the absolute values of R. That is, if multiply the whole matrix R by a constant factor, the filtering errors are unaffected. Specifically, if we multiply R of Fig. 6b|by a factor of = 3, the positional variances become aligned with those of Fig. (6a) and op is increased by a factor of = 13 - in accordance with Proposition | We repeated the tests with this modified R, and indeed, the results were indistinguishable from the original OKF
Proof of Proposition 7 Recall that in this problem, the KF applies the update step using an estimated observation model H = H(X):
2z/T Zy/l 8z/F)
Denoting the normalized estimation error da' = $ -, we can rewrite as
do', del, da',
By shifting the observation model in Eq. (Vas Vy 22, MD) we receive
①1 from H to H, and denoting the noise by v
Ur
Z -I =x++==n
By
= =iix +
unn us duy, uy del dod
Vp- da'
where denotes the current target velocity. We see that the effective observation noise is D Z HX = (Uz, Py, Pz, PD - dx' =.
To show that all the off off-diagonal entries of R = Cou(i) vanish, recall that the estimation error dæ' assumed to be independent of the velocity u. According Eq. ( is also independent of u. Hence, Cov(da', "as Vz) = E(da', U2 Px)) = E(dx', Vn Ed) E(,,) which vanishes by symmetry (E(uz) = 0). The same result holds for coordinates y, Thus, R is diagonal. Finally, by denoting =aVar(da" u) > O we have Cov(i) R as required.
Relying again on symmetry E(u), E(dx') O, we can further calculate C Var(dz' ux) E([[d<'|] ( = ((E([u)]²)), where s ("big-omega") corresponds to an asymptotic lower bound.
A.2 Non-i.i.d Noise
The assumption of i.d.d noise in Assumption Tis violated in many practical scenarios. Certain models with non-i.i.d noise can be solved analytically, if modeled correctly. For example, if the noise is auto-regressive with a known order p. an adjusted KF model may consider the last p values of the noise itself part of the system state (Geist and Pietquin| 2011). However, the actual noise model is often unknown or infeasible to solve analytically.
Furthermore, the violation of the i.i.d assumption may even go unnoticed. We discuss potential example in Section 5 where the noise is i.i.d in spherical coordinates but is not so after the transfor- mation to Cartesian coordinates. To see that, consider a radar with noiseless angular estimation (i.e.,
15

---

only radial noise), and a low target (I: = 0). Clearly, most of the noise concentrates on the XY plane both in the current time-step and in the following ones (until the target moves away from the plane). Hence, the noise is statistically-deeendenn over time-steps.
We may formalize this intuition for the toy Problem
2 Denote the system state at time by By transforming Ryotar of Problem 2 to
X, = ((X,),, (X.))), and denote lan 0. = (X, )
[X+)]
Cartesian coordinates, the observation noise is drawn from the distribution i N(0 R(O.)), where
ru cos
To cos () sin(f)
R(0)
ro cos(O) sin(O)
ro (in
Since consecutive time steps are likely to have similar values of 6,, the noise V, is no longer independent across time steps.
The effect of this violation of the i.idd assumption is analyzed in Proposition whose proof is provided below.
Proof of Proposition 2
Noise estimation: First, notice that the whole setting of Problem 2 is invariant to the target direction 0: the initial state distribution is radial, and the motion noise Q is isotropic. Hence, for any target at any time-step, 8, [0, 2T)) is uniformly distributed. By direct calculation,
2n
Eo RN [KF))) Eg [ro cos² 0)
ro
cos
2 0d0
J
2T
2
2x
Eg (KF)2 Eg [To sin?
0] Eg [
ro
To sin
2 0de
Eg [x
Eg [ro cos sin 6] =.
Since the targets in the data are i.i.d, the noise estimation of Algorithm I converges almost surely according to the law of large numbers, as required:
(ro/2 0 (ro/²
(KFF n.S. Rest
ro/2)
Optimization:
We use again the radial symmetry and invariance to rotations in the problem: w.l.o.g, assume that the optimal noise covariance parameter is diagonal, i.c., Ropc(r) (6 for some r > Our goal is to find r, and in particular to compare it to ro/2.
At certain time t, where the system state is X,, denote E[X,] Fu (x1, x22) T and (X,
Po (8)) (where p 0). Denote the observation received time by (x) + dx 22 + dx2)
We are interested in the point-estimate å of the KF following the update step (Fig. D.. By substituting
Fo. Po. the observation and the noise parameter Rope (r) in the update step, we have
f zo + POl1T (HPPH + ())
-1 Hx0)) zo + PolPo + (T) 20)
dr1
21 p-r doi
zi0 +
n+ 0
dxy
22
p-r dxy
On the other hand, the true observation noise covariance at time t is R(0,) of Eq. (8) (for the random variable 0,). If we add the assumption that the state X, is normally distributed (Xe ~ N N( Po)), and use the true noise covariance R((O)), then the update step of Fig. 2 gives the true posterior expected state:
Io + Po(Po + R(O.)) -20)
Etruc
To aln? 6+p
no cas O sin 0
d.r drz
Fig
p+
ptro to cos 8+p
cos sin
p+rro
p+ro
(ro sin? 0+p)ds (ro coe sin 0)dx
I1 +
ptr (ro cos" 0+ p)dd (ro cos sin d)d
322 +
ptro
16

---

We can use the standard MSE decomposition for the point-estimate x, into the bias term of x and the variance term of the state distribution: MSE nar (Prruc) + M E (2, Notice
that MSF (rue is independent of our estimator, it corresponds to the inherent uncertainty Ptrue (defined by applying to Po the update step with the true covariance R(0:)). Thus, our objective is to minimize MS F (f, Ftrue) Ell Etruell [1])
For the calculation below, we denote a(r) p/ (p- + r) and use the identity sin 20 cos 0 sin 0. In addition, from radial symmetry of do 2 - we have Eldri] = E [[dzž] and Eldx;] = 0, thus
we can denote Var(dx;;) E E[dx]).
M S Ebias (f(a), Itrue) Ell Trrue 1
sossin 0 + dr
To (osii(20)
2
E
(a
dr2
p+ ro cos 0
p+ro
sinn(2)/2 dx1
+
(a
dzz
p + ro
p p+ ro
To sin 0 +
+ r sin (am((20)))) (20)
d
a
+ Ci
du Aidx dx
+ ro To cos 2 +p
(p+ro)²
1 sis sin))
+ dx3
a² 2a
022
+
dx + Agdx dx²
p To
(p + ro)²
+ 2p
1 sin sinn2(2))
=2va
2va
vC[ + C2) +
p ro
rp+
where C122 are independent of and 11 are multiplied by E(dz = 0 and vanish. To minimize we calculate
OMS Ebius (i(a), Itrre)
2p ro p + ro
40-a 2v
da
which gives us
p + ro/2
++77 ro
Notice that ESt clearly diverges as |a| + co, hence the only critical point necessarily cor- responds minimum of the MSE. Hence, the optimal MSE given when substituting the following in Ropt:
T == p/a - p= p² pro - (x² + pro/2)
pro
p ro/2
2p + ro
Finally, recall that (Rest) a = ro/2 22 and compare to directly:
1///2
(Rest)a (Ropt) = To/2 - 2p + ro
> 0.
B OKF: Extended Experiments
B.1
Additional Scenarios and Baselines: Case Study
In this section, we extend the experiments of Section 5.1 with detailed case study. The case study considers 5 types of tracking scenarios (benchmarks) and 4 variants of the KF (baselines) 20 experiments in total. In each experiment, KF. The experiments in Section [ and Section5..1 are 3 particular cases. For each benchmark, we simulate 1500 targets for training and 1000 targets for testing.
compare the test MSE of OKF against the standard
Benchmarks (scenarios): Section 5 discusses the sensitivity of Algorithm D to violations of As- sumption In this case study, consider 5 benchmarks with different subsets of violations of
17

---

Table Benchmarks and the properties that define them. "V" means the benchmark satisfies the property.
anisotropic
uncentered
acceleration
polar
turns
Benchmark
Toy
O
O
O
Close
V
V
O
ROOD
200
Const_v
V
V
V
Const_a
V
V
Free
V
V
(a) Toy
(b) Close
(c) Const_y
(d) Const
(e) Free motion
Figure 11: Samples of targets trajectories in the various benchmarks, projected the XY planc.
Assumption [ The Free Motion benchmark intended to represent a realistic Doppler radar problem, with targets and observations simulated as in Section 4 each target trajectory consists of multiple segments of different turns and accelerations. On the other extreme, the Toy benchmark (Problem ! introduces multiple simplifications (as visualized in Fig. 5 In the Toy benchmark, the only violation of Assumption 1 is the non-linear observation H, as discussed in Section [5. Note that Section 5.2 and Section 5.3 experiment with settings of linear observation model.
We design benchmarks within the spectrum of complexity between Toy and Free Motion. Each benchmark is defined as subset of the following properties, as specified in Table and visualized in Fig.
anisotropic: horizontal motion is more likely than vertical (otherwise direction is distributed uniformly).
polar: radar noise is generated i.i.d in spherical coordinates (otherwise noise is Cartesian i..d).
uncentered: targets are dispersed different locations far from the radar (otherwise they are concentrated in the center).
acceleration: speed change is allowed (through intervals of constant acceleration).
turns: non-straight motion is allowed.
Baselines (KF variants): All the experiments above compare OKF to the standard KF baseline.
In practice, other variants of the KF are often in use. Here define such variants different baselines to the experiments. In each experiment, we compare the baseline tuned by Algorithm to its Optimized version trained by Algorithm (denoted with the prefix "O" in its name). For Algorithm we use the Adam optimizer with single training epoch over the 1500 training trajectories, 10 trajectories per training batch, and learning rate of 0.01. The optimization was run for all baselines in parallel and required a few minutes per benchmark, on eight i9- 19-10900X CPU cores in a single Ubuntu machine.
The different baselines are designed as follows. EKF baselines use the non-linear Extended KF model Sooenson
1985 The EKF replaces the approximation H H22) of Section 4 with H V.h(t), where h())) ((x) x and is the current state estimate. Polar baselines (denoted with "p") represent the observation noise R with spherical coordinates, in which the polar radar noise is i.i.d.
Results: Table 2 summarizes the test errors (MSE) in all the experiments. In each cell, the left column corresponds to the baseline Algorithm I and the right to Algorithm 2 In the model names,
18

---

Table 2: Test MSE results of Algorithm and Algorithm
benchmarks (scenarios) and 4 baselines (variants KF). KFp we also consider an "oracle" baseline with perfect knowledge the noise.
Benchmark
KF
OKF
KFp
KFp (oracle)
OKFp
EKF
OEKF
EKFp
OEKFp
Toy
151.7
84.2
269.6
116.4
92.8
79,4
123.0
109.1
Close
25.0
24.8
22.6
22.5
22.5
26.4
26.1
24.5
24.1
Const_y v
90.2
90.0
102.3
102.3
89.2
102.5
99.7
112.7
102.1
Const_a
107.5
101.6
118.4
118.3
100.3
110.0
107.0
126.0
108.7
Free
125.9
118.8
145.6
139.3
117.9
135.8
121.9
149.3
120.0
25000
20000
EXFp
OEKF
Close
Const
Const a
Free
Scenario
(a)
RASE (OSe((esmmaee)jsssS wawmaatcojiSSlopttmmeeooo (oppotimeeed
2.2
1.8
1.6
1.4
1.0
Quantile over [%)
(b)
test. Algorithm and Algorithm in different benchmarks (scenarios) and baselines. This is a different presentation the results of Table (a) also includes 95% confidence intervals. (b) shows, for each of the 20 experiments (5 benchmarks x 4 baselines), the MSE ratio between Algorithm and Algorithm 2 We that Algorithm wins in all the experiments (ratio always larger than 1) in some cases large margins. The dashed represents average MSE ratio over over all the experiments, showing an average advantage of 20% OKF
Figure 12: Summary
"O" stands for optimized, "E" EKF and "p" for polar (or spherical). The same results are also shown with confidence intervals in Fig. 12 Below discuss the main findings.
Choosing the KF configuration is trivial: Consider the non-optimized KF baselines (left column in every cell in Table 2). In each benchmark, the results are sensitive to the baseline, i.c., to the choice of KF configuration R's coordinates and whether to use EKF, For example, the Toy benchmark, EKF is the best design, since the observation model H is non-linear. In other benchmarks, however, the winning baselines may come as a surprise:
1. Under non-isotropic motion direction (all benchmarks except Toy), EKF is worse than KF despite the non-linearity. It is possible that the horizontal prior reduces the stochasticity of Hi, making the derivative-based approximation unstable.
2. Even when the observation noise spherical i.i.d, spherical representation of R is not beneficial when targets are scattered far from the radar (last benchmarks). It is possible that with distant targets, Cartesian coordinates have a more important role in expressing the horizontal prior of the motion.
19

---

Since the best KF variant per benchmark seems hard to predict in advance, a practical system cannot rely on choosing the KF variant optimally and should rather be robust this choice.
OKF is more accurate and more baseline-robust: For every benchmark and every baseline (20 experiments in total), OKF (right column) outperformed noise estimation (left column). In addition, the variance between the baselines reduces under optimization, i.c., OKF makes the KF more robust the selected configuration.
OKF outperforms an oracle baseline: We designed an "oracle" KF baseline - with perfect knowl- edge of the observation noise covariance R. in spherical coordinates. We used it for all benchmarks except for Toy (in which the radar noise is not generated in spherical coordinates). Note that in the constant-speed benchmarks (Close and Const v), Q 0 and is estimated quite accurately; hence, in these benchmarks the oracle has a practically perfect knowledge of both noise covariances. Nevertheless, the oracle yields very similar results to Algorithm [
This indicates that the benefit of OKF not in a better estimation accuracy of Q and R but rather in optimizing the desired objective.
B.2 Sensitivity to Train Dataset Size
Each benchmark in the case-study of Appendix B.1 has 1500 targets in its train data. One may argue that numeric optimization may be more sensitive to smaller datasets than noise estimation; and even more so, when taking into account that the optimization procedure "wastes" a portion of the train data as a validation set.
In this section we test this concern empirically, by repeating some of the experiments of Appendix B. with smaller subsets of the train datasets - beginning from as few as 20 training trajectories. Fig. 13
shows that the advantage of OKF over KF holds consistently for all sizes of train datasets, although is indeed increases with the size. Interestingly, in the Free Motion benchmark, the test errors of KF and KFp increase with the amount of train data!
Const_a a
Free
22000
15000
model
model
KF
14000
20000
KFp
KFp
OKF
MSE
18000
OKFp
12000
16000
11000
14000
10a
10a
10a
10a
train samples
train samples
Figure 13: The advantage of OKF over KF holds consistently for all sizes of train datasets including as small datasets 20 trajectories. The shadowed areas correspond to 95% confidence intervals.
B.3 Generalization: Sensitivity to Distributional Shifts
In Appendix B. 1 demonstrate the robustness of OKF in different tracking scenarios: in every benchmark, OKF outperformed the standard KF over out-of-sample test data. This means that OKF did not overfit the noise in the training data. What about out-o--distribution test data? OKF learns patterns from the specific distribution of the train data - how well will it generalize to different distributions?
Section 4 Malready addresses this question to some extent, as OKF outperformes both KF and NKF over -fittr-diittibutiion target accelerations (affecting both speed changes and turns radius). In terms of Eq. (T). the modified acceleration corresponds to different magnitudes of motion noise Q: that is, change the noise after OKF optimized the noise parameters. Yet, OKF adapted the change without further optimization, with better accuracy than the standard KF. Thus, the results Section already provide a significant evidence for the robustness of OKF certain distributional shifts.
In this section, we present a yet stronger evidence for the robustness of OKF - not over a paramet- ric distributional shift, but over entirely different benchmarks. Specifically, we consider the 5
20

---

TVT1 HSAicad INGtaeed
To MALEO PILAOa
30
30
100
03
105
100
1:12
9111
1.29
CORPLY
8.85
LO1
386
1.00
631
0.06
056
2.00
0.94
0.55
1.00
1:44
0.55
0.08
1.01
0.15
1.0
200
1.0
010
Pret 3.00
039
3.00
(a) M SE ratio (SE(KF)/MMECCC for every KF-baseline (KF:KFP.EKFEKFP defined in Ap- pendix B.I. and for every pair of train-scenaris and test-scenario. The colormap scale is logarithmic (xx log MSF ratioo)) where red values represent advantage OKF (MSF_ratio 1).
Generalization between scenarios
logl MSE(est) MSE(opt))
10
Toy
Const, V
Const_
0.4
0.2
0.0
-0.2
KF variant
(b) For every train-scenario, MSE ratio is averaged over eceearceasiios and shown a logarithmic
scale. Positive values indicate advantage to OKF.
Figure 14: Generalization tests: OKF KF under distributional shifts between scenarios.
benchmarks (or scenarios) of Appendix B.I For every pair (train-scenario, test-scenario), we train both KF and OKF data of the train-scenario, then them on data of the test-scenario. For every such pair of scenarios, we measure the generalization advantage of OKF over KF through (where E_ratio
indicates advantage to OKF). To measure the total generalization advantage of model trained on a certain scenario, we calculate the geometric mean MSE_r E_ratio over all the test-scenarios (or equivalently, the standard mean over the logs of the ratios). The logarithmic scale guarantees symmetric view of this metric of ratio between two scores.
MSE_ratio MSF F)/SSE()MKSKK
This test is quite noisy, since model optimized for certain scenario may legitimately be inferior in other scenarios. Yet, considering all the results together in Fig. [4 it is evident that OKF provides more robust models: it generalizes better in most cases, sometimes by a large margin; and loses only few cases, always by small margin.
B.4 Ablation Test: Diagonal Optimization
The main challenge in Section 3 and Algorithm is to apply standard numeric optimization while keeping the symmetric and positive-defintt constraints (SPD) of the parameters Q, R. To that end, we apply the Cholesky parameterization to Q and R. In this section, we study the importance of this parameterization via an ablation test.
For the ablation, we define a naive version of OKF with a diagonal parameterization Q and R. Such parameterization is common in the literature: "since both the covariance matrices must be constrained to be positive semi-definite Q and R are often parameterized as diagonal matrices" [Formentin and Bittanti 2014). We denote the diagonal variant by DKF, and test on all 5 Doppler benchmarks of Appendix B.
Table displays the results. In the first 3 benchmarks, where the target motion is linear, DKF is indistinguishable from OKF, However, in the 2 non-lincar benchmarks (which are also the same benchmarks used in Section 4), DKF is inferior OKF, though still outperforms the standard KF.
21

---

Table 3: Ablation test: Diagonal optimized KF, tested the benchmarks of Appendix B.1
Benchmark
KF
DKF
OKF
Toy
151.7
84.2
84.2
Close
25.0
24.8
24.8
Const_v
90.2
90.1
90.0
Const_a
107.5
106.1
101.6
Free
125.9
121.1
118.8
B.5 Video Tracking: Dataset License
The MOT20 video dataset Dendorfer et al. 2020 is available under Creative Commons Attribution-
MonCoomercial-Shcnn ShareAAlkkee 3.0 License. In Section 5.2 we used the videos MOT20-01, MOT20-02 and MOT20-03 for training, and MOT20-05 for testing.
B.6 Lidar-based State Estimation: Visualization
Fig. 15 visualizes sample of the simulated trajectories and the model predictions in the lidar-basse state estimation of Section 5.3
trajectory
observation
OKF
(a)
(b)
Figure 15: (a) sample of simulated self- trajectories. (b) Segments of turns within sample trajectory, and the corresponding lidar-based estimations.
C Neural KF: Extended Discussion and Experiments
Preliminaries RNN and LSTM: Recurrent neural networks (RNN) [Rumelhart et al. 1986 are neural networks that are intended be iteratively fed with sequential data samples, and that pass information (the hidden state) over iterations. Every iteration, the hidden state is fed to the next copy of the network part of its input, along with the new data sample. Long Short Term Memory (LSTM) [Hochreiter and Schmidhuber, 1997] is an architecture of RNN that is particularly popular due to the linear flow of the hidden state over iterations, which allows to capture memory for relatively long term. The parameters of RNN are usually optimized in a supervised manner with respect to training dataset of input-output pairs.
Neural Kalman Filter: We introduce the Neural Kalman Filter (NKF), which incorporates an LSTM model into the KF framework. The framework provides probabilistic representation (rather than point estimate) and separation between the prediction and update steps. The LSTM is an architecture of recurrent neural networks, and is a key component in many SOTA algorithms for non-linear sequential prediction (Neu et al. 2021 We use it for the non-linear motion prediction.
As shown in Fig. 16 NKF uses separate LSTM networks for prediction and update steps. In the prediction step. the target acceleration is predicted on top of the linear motion model, instead of predicting the state directly. This regularized formulation is intended express our domain knowledge about the kinematic motion of physical targets.
Extended experiments: We extend the experiments of Section with additional versions of NKF:
Predicied-acceleration KF (aKF): F): variant of NKF that predicts the acceleration but not the covariances Q and R.
22

---

Predict
0g hift , Agf)
& = P2, + 7
Pe = PPPPT + Q6:c. hg)
stt1
Pt
1=1+1
Update
R2, h/ = R(& S.8P,zI zi, k))
Zt
Pt (1 R
R 2 k(2 HRE)
P3 = (1 KH)P!
Figure 16: The Neural Kalman Filter (NKF). Differences from Fig. are highlighted. At is constant; G, Q are the outputs of LSTM network with hidden state ha; and is the output of an LSTM with hidden state h,..
AMSE (wrt KF baseline)
1000
ANLL (wert KF baselina)
H
12, P2
12 oK
13 satt
16 1K
12
12
-2000
122
14 6K
-3000
13 BK
1.
16PH
NOPH
-4000
12<arc<<4
M4<accc88
ceacc<96
12<acc<24
24<acc<448
48<cacc<96
(a) Free-motion benchmark
AMSE (wrt KF baseline)
MNLL (wort KF baselino)
-1000
10 6k
10.8K
10
H
-2000
1x2
It
Has
-3000
10
-4000
af
aKT
-5000
MKFH
NK?
99
NCP
NIFH R
10
XQ
08<aoc<36 Scenario
04-c0ccx08
08<acc<416 Scenario
16<acc<332
(b) Const benchmark (no turns)
Figure 17: The relative MSE and NLL results various models comparison the standard model. The textual labels specify the absolute MSE and NLL. Note that certain bars of NLL are of entirely different scale and thus cropped in the figure (their values can be seen in the labels). In each benchmark, the models were trained with relation MSE loss, train data of the middle acceleration range: the two other acceleration ranges each benchmark correspond to generalization over distributional shifts.
Neural KF (NKF): the model used in Section 4 and illustrated in Fig. 16
Neural KF with H-prediction (NKFH): a variant of NKF that also predicts the observation model H in every step.
In addition, while still train with MSE loss, add the test metric of Negative-Loge Loaevee--o------eee Likelihodd (NLL)- of the true state w.r.t the estimated distribution. Note that the NLL has an important role in the multi-target matching problem (which is out of the scope of this work).
For each benchmark and each model, we train the model on train data with certain range of targets acceleration (note that acceleration affects both speed changes and turns sharpness), and tested it on targets with different acceleration ranges, some of them account for distributional shifts. For each model we train two variants one with Cartesian representation of the observation noise R, and one with spherical representation (as in the baselines of Appendix B. and we select the one with the higher validation MSE (where the validation data is portion of the data assigned for training).
23

---

Fig. 17a shows that in the free-motion benchmark, all the neural models improve the MSE in comparison the standard KF, yet are outperformed by OKF. Furthermore, while OKF has the best NLI,, the more complicated models NKF and NKFH increase the NLL in orders of magnitude. Note that the instability of NKFH expressed in poor generalization to lower accelerations addition to the extremely high NLL score.
Fig. shows that Const_a benchmark, all the 3 neural models improve the MSE in comparison to the standard KF, but only NKFH improves in comparison to OKF as well. On the other hand, NKFH still suffers from very high NLL.
In summary, all 3 variants of NKF outperform the standard KF in both benchmarks in terms MSE. However, when comparing OKF instead, akF and NKF become inferior, and the comparison between NKFH and OKF depends on the selected benchmark and metric.
24


Reviews:
[{"review_id": "wiskxWBfpJ", "reviewer": "Reviewer_JSXK", "summary": "This paper uses stochastic optimization to estimate the covariance of Kalman filters in supervised tracking problems. They find that online stochastic optimization in this manner tends to generalize better than either neural Kalman filters with LSTM base models or the older plug-in estimators for Kalman filter problems. Their argument for the outperformance of stochastic optimization in these problems relies heavily on the fact that the assumptions for the plug-in estimators ends up being violated in some manner, producing at least some amount of model mis-specification. The experiments cover a bunch of simulated data where the assumptions of the non-optimized Kalman filter can be tested rigorously, as well as less toy video tracking and lidar state estimation experiments.\n\nAfter the authors rebuttal I'm a bit happier about this paper. Thanks for showing the value of the Cholesky parameterization and answering the rest of my questions.", "questions": "When Assumption 1 is not violated, does the stochastic optimization approach converge to the same solution (just at a slower rate)?\n\nYou may wish to break Assumption 1 into four separate assumptions – stationarity, i.id-ness, known covariances, and good initial states. \nThis would probably cleanup some of the writing in the experiments by being able to which specific piece of assumption 1.\n\nCould the neural Kalman filter be combined with the optimized Kalman filter with both being optimized jointly?\n\nCould the authors run an ablation where the L Cholesky factor is parameterized diagonally? What is the impact of the full parameterization? \nI believe that this should become much less well-specified and harder to optimize.\n\nIs the formulated stochastic optimization problem still convex or at least reachable to a global optimum?\n\nWhy can we not map observations from cartesian to polar coordinates in the lidar and doppler experiments, thereby producing observations that are much less violating of assumption 1.\n\nSection 4: \nIn the doppler radar problem, could another benchmark (either a physics neural net or a neural ode) be used for the trajectory problem as well?\n\ntypos / writing:\n\nL357: studied -> studies\n\nSection 5: use a real name for section heading – e.g. “Comparing Optimized KF to KF”\nPersonally, I would probably combine sections 4 and 5 into a single section with the same sub-headings: experiments. Currently, the headings seem a bit informal.", "limitations": "I think motion tracking in crowds may not be a great application socially, but otherwise no other societal impacts.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality\n\n-\tI don’t have a great background in the Kalman filtering literature so I find it difficult to fairly judge originality. However, based on a cursory reading of the other literature, it seems to be a quite original work.\n\nQuality\n\n-\tOverall the paper seems to be of pretty high quality experiment-wise and technically. See typos at bottom.\n\nClarity\n\n-\tThe assumptions are spelled out pretty well, especially as each experiment tends to have a different set of assumptions.\n\nSignificance\n\n-\tThis paper ends up being yet another demonstration of “stochastic optimization tends to work better in practice than in theory” which seems certainly like a good addition to the ML community, even if it comes from a place that is a bit outside.\n\nIndeed, appendix b.2 shows that stochastic optimization is a bit more robust to model mis-specification in what you describe, as you show an example where adding more data definitely makes the error worse. Honestly, you could move this into the main text with some tigher writing.\n\n-\tOverall, between the authors’ software and the simplicity of the approach, it also seems like a fairly reasonable contribution to the system identification literature.", "weaknesses": "Originality\n\n-\tFrom my limited knowledge here, I do think that the Bayesians have been estimating parameters in a similar manner for GPs in state identification for a while. See for example [1], [2]\n\nQuality\n\n-\tA lack of comparison to non-Kalman filter benchmarks on some of these tasks limits the relevance to the ML community – perhaps physics NNs or even a direct LSTM would be a good baseline.\n\n\nClarity\n\n-\tI think that I am a little confused as to the broader application of the Kalman filter in this type of supervised learning setting however. The motivation for this specific subset (supervised like training estimation) could be explained a bit better.\n\n-\tThe implications of Assumption 1 should probably be explained a bit better right after the assumption and theorem is sketched out. Theorem 1 is also not really well written or formally stated – please either write it more formally or just drop the theorem heading.\n\nSignificance\n\n-\tTouched on a bit above, but I am a little puzzled by the when is the supervised estimation Kalman filtering problem actually used? From what I can tell all of the experiments are very distinguished from reality and simplified.\n\n-\tAt the end of the day, the approach is really just covariance matrix estimation for trajectories, which seems like it has been done before in things like [3], [4]\n\nReferences:\n\n[1] Reece and Roberts, 2010. An Introduction to Gaussian Processes for the Kalman Filter Expert\n\n[2] Deisenroth, Huber, & Hanebeck, ICML, 2009. Analytic moment-based Gaussian process filtering.\n\n[3] Xiao and Wu, Ann. Stat, 2012. Covariance matrix estimation for stationary time series.\n\n[4] Wang et al, Neurocomputing, 2017. An adaptive Kalman filter estimating noise covariance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed feedback. **Before responding in detail and presenting the new experiments as requested, we wish to focus on the comments about the significance of our contribution. This may clarify some of the reviewer’s questions, and we will appreciate it if the reviewer could re-evaluate our contribution given this discussion.**\n\n## Discussion on contribution\n\n**”When is the supervised estimation Kalman filtering problem actually used?”**\n\nAs mentioned in Line 20, our work in this setting originated from an actual real-world radar problem. The supervision of true states (or more precisely, states data with negligible errors) used to come from either monitored experiments or simulations. Following the review, we will discuss this more clearly in the scope paragraph. In addition, note that Lines 55-57 provide few other brief motivational examples. Finally, all the 10 works cited in Lines 33-37 follow this supervised filtering setting, indicating its relevance.\n\n**MSE optimization $\\ne$ covariance noise estimation**\n\nWe do not attempt to estimate the noise covariance matrix better than others; such estimation is trivial in our setting (by Eq. 2). The extensive literature on this topic corresponds to *other* settings (e.g., when the states {x} are unavailable in the train data). Instead, we observe that this common noise-estimation task is usually *not* a proxy to MSE optimization, hence we should often *give up* on accurate noise-estimation in favor of direct MSE optimization. As the reviewer mentioned, Appendix B2 shows that these goals are *contradictory*. Hence, **our key contribution is not *how* to optimize, but rather *what* to optimize** (although we also overcome how to optimize SPD parameters). We tried to convey this message in the intro. **Following the rebuttal, we will specify this distinction more clearly, and will add an explanatory diagram (which is attached in the global rebuttal post)**.\n\n_______\n\n## Detailed responses\n\nAs suggested, we will address the typos and the presentation of the preliminary assumption and theorem. Thanks for pointing them out.\n\n**Could the authors run an ablation where the L Cholesky factor is parameterized diagonally?**\n\n**We now ran it** on all the 5 Doppler benchmarks of Table 1 in Appendix B1. We added *DKF* - a variant of OKF with Diagonal parameterization of Q and R. On the first 3 benchmarks, where the target motion is linear, DKF was indistinguishable from OKF. However, on the 2 non-linear benchmarks (the same benchmarks used in Section 4), DKF was inferior to OKF (but superior to KF).\n\n| Benchmark   | KF    | DKF   | OKF   |\n|-------------|-------|-------|-------|\n| Toy  | 151.7 | 84.2  | 84.2  |\n| Close | 25.0  | 24.8  | 24.8  |\n| Const_v  | 90.2  | 90.1  | 90.0  |\n| Const_a (\"no turns\") | 107.5 | 106.1 | 101.6 |\n| Free motion | 125.9 | 121.1 | 118.8 |\n\n**When Assumption 1 is not violated, does the stochastic optimization converge to the same solution (just at a slower rate)?**\n\nWe typically use Algorithm 1 to initialize $Q,R$ for Algorithm 2. Under Assumption 1, this initialization is the globally-optimal solution. Thus, the GD in Algorithm 2 (with a sufficiently small learning rate) will just remain in this optimum. **We now added an empirical demonstration of this phenomenon**, on a toy version of the lidar problem, which satisfies Assumption 1. As expected, throughout the entire training process, OKF’s validation MSE remained identical to KF’s MSE.\n\n**Could NKF be combined with the optimized KF with both being optimized jointly?**\n\nThe NKF already includes optimization of Q and R, i.e., already combines OKF’s optimization of Q,R with the neural architecture.\n\n**Is the formulated stochastic optimization problem still convex?**\n\nSince our main premise is violation of the model assumptions, there are generally no theoretical guarantees of convexity. Instead, Algorithm 2 is backed by (a) empirical evidence in Sections 4,5; (b) external evidence of the strength of stochastic optimization in general; (c) the motivation to optimize the MSE rather than the contradicting objective of Algorithm 1.\n\n**Why can we not map observations from cartesian to polar coordinates, thereby producing observations that are much less violating assumption 1.**\n\nFollowing this motivation exactly, we produced the simplified toy environment where the noise is *Cartesian* i.i.d (Section 5.1). However, in the non-toy problem, the discrepancy is inherent and not just a matter of representation: the MSE objective is Cartesian, whereas the radar noise is polar.\n\n**A lack of comparison to non-Kalman filter benchmarks on some of these tasks limits the relevance to the ML community.**\n\nThe relevance to the ML community comes from two points:\n1. An *application of ML* to real-world filtering: thanks to the powerful ML optimization tools, we can easily optimize the MSE, instead of the wrong proxy objective of noise estimation.\n2. Experimental methodology when testing new models against classical ones: both models should be tested under the same optimization method.\n\nWe support both points in a variety of experiments in Sections 4, 5.1, 5.2, 5.3, B.1, B.2, B.3, C.\nHowever, we did not consider additional neural baselines (in addition to the 3 neural KFs in Appendix C) as supportive of our claim - since we do *not* claim that OKF is in general superior to neural networks (see Lines 36, 204).\n\n**”Bayesians have been estimating parameters in a similar manner for GPs in state identification.” / “The approach is really just covariance matrix estimation for trajectories, which seems like it has been done before in things like [3], [4].”**\n\nPlease see the discussion above about our contribution: covariance matrix estimation has indeed been studied extensively; however, we argue that covariance estimation is often not the desired objective, and instead propose to tune the KF parameters via the *contradicting* MSE objective.\n"}, {"review_id": "w61kt2J0iM", "reviewer": "Reviewer_mJXo", "summary": "The paper is considering a linear Gaussian filtering problem where the noise covariance are unknown. It assumes it has access to state and observation data. It aims to find the noise covariances by minimizing the estimation error of a Kalman Filter using those noise covariance  matrices. This is in contrast to the standard fitting approach of Algorithm 1. The empirical results show support for this approach compared to algorithm 1", "questions": "Please see the weakness above", "limitations": "no", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The paper has an honest tone that I appreciate\n- the idea of using the filtering error to optimize the filtering parameters is natural and reasonable.", "weaknesses": "- It is very difficult to follow the paper from the beginning and the title. It seems that this is continuation of the author's research, but the readers do not have the same context. \n- It is not exactly clear what is the main message of the paper. Is the paper arguing against using neural networks for nonlinear filtering problems? \n- There are also other, more filtering oriented, ways to estimate the covariance matrices, like optimizing Q and R to make the innovation error white. This is in the references in the paper, but [1] is also recent review of the topic. It will be good to do comparison with them\n- There are no guarantees for solving problem (3), but algorithm (1) has guarantees. \n\n[1] Zhang et. al. On the Identification of Noise Covariances and Adaptive Kalman Filtering: A New Look at a 50 Year-Old Problem", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for reading our work and providing honest feedback. We respond to the reviewer’s comments below. More importantly, the review specifies that the main message was not clear to the reviewer. **Since it is important for us to make the presentation as clear as possible, we recap our main message here in different words, and attach a new explanatory diagram (attached in the global rebuttal post). Does the diagram help? We will appreciate any suggestions regarding which points are not clear and why.** We will also appreciate it if the reviewer could re-evaluate our contribution accordingly.\n\n### Our recapped message\n1. The KF assumptions often do not hold.\n2. In such cases, instead of viewing Q,R as representatives of the noise, we should view them as parameters of the prediction model.\n3. Accordingly, we should optimize Q,R wrt the ultimate objective (MSE) rather than the proxy objective (noise estimation), since the proxy is no longer equivalent to the ultimate goal.\n4. In particular, when comparing KF to a neural network, notice that the neural network is indeed optimized for the MSE; for the comparison to be fair, the KF must be optimized similarly.\n5. We show that the literature is generally unaware of (4), and demonstrate that by not optimizing the KF, the whole experimental conclusion may be reversed.\n6. We suggest how to optimize the KF in a simple way despite the SPD structure of the parameters Q,R.\n\n______________________________\n\n## Responses to specific questions\n\n**Is the paper arguing against using neural networks for nonlinear filtering problems?**\n\nCertainly not. Neural networks may perform better or worse than the KF. Our claim is that to decide between these two possibilities, one must test the network and the KF under the same conditions, including similar optimization of their parameters. In Section 4 we demonstrate that this methodological issue is crucial: NKF outperforms the KF if the KF is not optimized, but not after the KF is optimized (by OKF).\n\n**There are also other, more filtering oriented, ways to estimate the covariance matrices… [1] is also recent review of the topic. It will be good to do comparison with them.**\n\nAs discussed in Lines 53-62, we address the supervised setting, where both states {x} and observations {z} are available for training, and thus the simple Algorithm 1 can be applied. In different settings, when the states {x} are unknown and Algorithm 1 is not applicable, many studies such as [1] propose alternative methods to estimate the noise.\n\nOur main claim is that the objective itself - noise estimation - should often be replaced. To stress this, we compare to an *oracle* baseline that knows the *exact* noise Q,R (Appendix B1). Contrary to the oracle, [1] operates in *harder* conditions than Algorithm 1, and thus using it as a baseline will not strengthen our message. However, we will stress the additional oracle baseline in the front manuscript. Thanks for pointing to this issue.\n\n**There are no guarantees for solving problem (3), but algorithm (1) has guarantees.**\n\nOur main premise is that the KF assumptions often do not hold, and their violation may be tricky to even notice (Lines 217-243). Hence, as discussed in Line 155, neither Algorithm 1 nor Algorithm 2 actually have optimality guarantees. However, Algorithm 2 is at least aligned with the desired MSE objective, whereas Algorithm 1 is not. For example, as demonstrated in Appendix B2, instead of converging to optimal predictions, Algorithm 1 may in fact *deteriorate* in MSE when fed with more data.\n"}, {"review_id": "R8ajt6tkpJ", "reviewer": "Reviewer_GLsN", "summary": "In the domain of nonlinear filtering, it's a usual practice to draw comparisons between nonlinear architectures like neural networks and traditional linear Kalman Filters (KF). This approach blends two distinct aspects: the nonlinear structure and the numerical optimization technique, thereby introducing potential issues in the comparison process. Specifically, nonlinear models are often subjected to optimization, unlike the benchmark KF models.\n\nThis paper proposes a solution to these challenges by introducing the Optimized KF (OKF), a method designed to customize numerical optimization in alignment with positive-definite KF parameters. The authors demonstrate that the apparent advantage of neural networks over KF can be completely neutralized when KF is optimized using OKF. Furthermore, our tests reveal that OKF surpasses the standard KF in various problem situations, suggesting that KF might present a sub-optimal choice.\n\nThe key contributions of this paper include:\n\n1. Highlighting a prevalent methodological error of comparing an optimized filtering model against an unoptimized KF.\n2. Introducing the Optimized KF (OKF) as a solution, showcasing the experimental results in comparison with nonlinear structures like existing neural networks and conventional linear KF, after switching the benchmark to OKF and utilizing the modified overall experimental outcomes.\n3. Establishing that OKF consistently achieves superior accuracies compared to KF across a spectrum of problems.\n4. Offering a theoretical analysis that highlights the sub-optimality of KF.", "questions": "1. Is the optimization method only using gradeient descent?\n2.  Is there any comparison with nonlinear architecture (like Neural network)?", "limitations": "The idea in this paper is very novel, but it feels like it just added gradient based optimization at the end. It would be nice to suggest various optimization methods or optimization methods suitable for Kalman Filter.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Motivation is clear and it seems like a meaningful study.\n2. It is well-proven formally.\n3. The proposed model seems novel.", "weaknesses": "The idea in this paper is very novel, but it feels like it just added gradient based optimization at the end.\nIs there any comparison with nonlinear architecture (like Neural network)?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful feedback. Please see our responses below.\n\n**It would be nice to suggest various optimization methods.**\n\nOur key insight is that noise estimation (Algorithm 1) is not a reliable proxy to MSE optimization, and instead the MSE should be optimized directly. The focus is on choosing this “right” objective to optimize; choosing how to optimize it, between SGD, Adam, etc., is beyond our scope.\n\n**Is the optimization method only using gradient descent?**\n\nMostly, but not only: as discussed in Section 3, we also overcame the challenge of optimizing the parameters without breaking their symmetric and positive-definite structure. For this, we used the Cholesky parameterization - a trick that can be applied on top of existing optimization methods.\n\nStill, how to numerically-optimize the MSE is not our main novelty. As discussed above, the main novelty is choosing to optimize the MSE at all, instead of the proxy task of noise estimation, after we observe that the two are usually not equivalent.\n\n**Is there any comparison with nonlinear architecture (like Neural network)?**\n\nSection 4 presents a comparison between the optimized linear KF (OKF) and the nonlinear neural network (NKF). Appendix C also experiments with other neural network variants.\n"}, {"review_id": "m7b6NB4esr", "reviewer": "Reviewer_hyJH", "summary": "The paper presents a new Kalman Filter (KF) algorithm named Optimized Kalman Filter, which applies an optimization to the supervised learning loss as usually used for training non-linear models (e.g. NKF). For linear KF, a more straightforward way is to estimate the noise covariance, which is directly plugged into the estimator with optimal MSE (if the covariances are known). The authors argue that the significant improvement of NKF is exclusively a result of the optimization approach, as they observe that OKF outperforms NKF in Doppler radar problem, a simple problem with true KF. They also demonstrate the benefits of OKF when the true model is non-linear in Doppler Radar Tracking and Video Tracking problem.", "questions": "See the weaknesses discussions. In general, it is not clear whether OKF will still outperform NKF on actually non-linear systems (not very possible when the system is highly nonlinear).", "limitations": "The paper did not discuss limitations of their work. I have pointed out a number of directions to improve the theoretical results. The paper conducted experiments on video tracking, which involves predicting human behavior. The authors should discuss the the potential social impact if their algorithm is applied to real products.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The paper demonstrate a simple yet profound neglect in the literature. That is all the previous literature did not fairly compare linear KF and NKF as the two settings differs in both architecture and the way of learning. I am not familiar with the literature of Kalman filtering. However, it is intriguing to see that linear models can performs as well as non-linear ones as long as they are learned properly. \n\nThe flow of writing is clear and easy to follow.\n\nThe experiments are complete and they cover what it takes to have the main story hold.", "weaknesses": "1. For the completeness, it would be nice to have a formal definition of the Neural KF and their optimization goal. How are the models normally set up? Do they also optimize the covariance matrices?\n2. The theoretical contribution is limited. All the results are either in the literature or follows immediately from simple algebra. Some interesting theory can be studied. It would be interesting to analyze the error introduced by estimated covariance matrices (Algorithm 1), which reveals their suboptimality. Could we write the closed form of the minimum-variance unbiased estimator when the covariance matrices are not known?\n3. In section 4, it would be interesting to compare with KF with known covariance matrices.\n4. In section 5, it would be interesting to compare with NKF as well, since the problem is apparently nonlinear", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful feedback. We will modify our manuscript accordingly. Please see our detailed responses below.\n\n**In section 4, it would be interesting to compare with KF with known covariance matrices.**\n\nWe conducted this experiment in Appendix B1. The oracle performed very similarly to Algorithm 1 (see Table 2); this indicates that the issue is to choose the right objective, rather than how to optimize/estimate it. Note that an oracle experiment in Section 4 is problematic, since the dynamics are nonlinear and the covariance matrix Q is not well-defined wrt the model F.\n\n**In section 5, it would be interesting to compare with NKF as well, since the problem is apparently nonlinear.**\n\nWe indeed tested NKF on our two most nonlinear benchmarks. All our Doppler-radar benchmarks are specified in Table 1 and Figure 9 in the appendix. Section 4 (NKF) corresponds to Fig 9d,9e, which are the most nonlinear and farthest from the KF assumptions. We will make the connection between the benchmarks clear. Thanks for pointing to it.\n\n**It is not clear whether OKF will still outperform NKF on actually non-linear systems (not very possible when the system is highly nonlinear).**\n\nAs specified above, we tested NKF vs. OKF on highly nonlinear systems. In fact, as mentioned in Line 20, NKF was the original goal of our work, and we made sure to design a benchmark where the neural network should have an added value.\n\nThe failure of NKF might have been caused by many factors: difficulty to learn the nonlinear models, too many degrees of freedom, architecture or optimization choices, etc. Note that the take-away is *not* that neural networks are not good, but rather that their comparison to KF should be done fairly using the *same* optimization. Indeed, in our case, the fair comparison revealed that NKF did not really learn the nonlinear modeling better, a crucial conclusion.\n\n**How are the NKF models normally set up? Do they also optimize the covariance matrices?**\n\nThe NKF architecture is reported for completeness in Appendix C and  Figure 14. Its optimization goal is the predictions MSE and optimized parameters are all neural network weights + all covariance parameters. In that sense, you may view OKF as an ablation test of NKF (as mentioned in Line 23).\n\n**Could we write the closed form of the minimum-variance unbiased estimator when the covariance matrices are not known?**\n\nSince our inherent premise is violation of the model assumptions, closed-form solutions are generally not available. Instead, in Proposition 1 and 2 we analyzed the deviation of the optimal parameters from Algorithm 1, for specific violations of Assumption 1. Regarding the effect on the MSE-gap, Table 2 in Appendix B1 indicates that the effect varies greatly between scenarios (OKF is always better, but by different margins). The reviewer’s suggestion of studying these effects theoretically under various assumptions, is indeed a very interesting direction for (probably multiple) future works.\n\n**The paper did not discuss limitations of their work.**\n\nWe consider the two main limitations to be: (a) we focus on the supervised setting, as discussed in Lines 53-62; (b) numerical optimization has little theoretical guarantees, as discussed in Lines 155-159. We will clarify the connection of these discussions to the limitations of our work. Thanks for pointing to it."}]

Author rebuttal:
We thank all the reviewers for their helpful comments. We are pleased that the reviewers found our work “**original**” (JSXK), **”honest”** (mJXo), **”meaningful”** and “**very novel**” (GLsN), and acknowledged that we point to a **“prevalent methodological error”** (GLsN) and a **“profound neglect in the literature”** (hyJH). The reviews considered our work **“high quality technically”** (JSXK) and **”well-proven formally”** (GLsN). They found our experiments “**complete**” (hyJH), covering a “**spectrum of problems**” (GLsN), each with a **“different set of assumptions”** (JSXK).
Finally, they considered our motivation **“clear”** (GLsN) and writing **”clear and easy to follow”** (hyJH).

Below we respond to the reviews and discuss the consequent modifications in our manuscript. We also include two additional experiments following JSXK’s suggestions, and attach a new diagram that summarizes our main message.

Meta-review: This paper's primary contribution is to point out that online adaptation of the linear Kalman filter drastically improves its performance, even in nonlinear dynamical systems, outperforming neural baselines. Reviewers generally agreed that the experiments were high-quality, and the message was well-communicated; on the other hand, they agreed that the novelty of the other contributions (new algorithms, and theoretical understanding) was limited. There was also some divergence of opinions on whether this message was sufficiently novel.

Thus, the paper decision hinges upon the subjective question of whether pointing out this (quoting a reviewer) "neglect in the literature" is a standalone contribution. After discussion and evaluating the other reviews, although consensus was not 100% reached, the reviewer with the strongest objection adjusted their score slightly. Overall, there seems to be enough precedent for the prevalence of non-adapted Kalman filters in experiments, to bring this paper's value above the acceptance threshold.

The reviewers have pointed out a couple of things that would immediately make the paper more convincing: a comparison with the ensemble KF, and (any) attempt at theoretical understanding. The authors are encouraged to incorporate these comments.
Decision: Accept (poster)