SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We can use the standard MSE decomposition for the point-estimate  $x$ , into the bias term of  $x$  and the variance term of the state distribution:  $MSE = MSE_{var}(P_{true}) + MSE_{bias}(x, x_{true})$ . Notice that  $MSE_{var}(P_{true})$  is independent of our estimator, as it corresponds to the inherent uncertainty  $P_{true}$  (defined by applying to  $P_0$  the update step with the true covariance  $R(\theta_t)$ ). Thus, our objective is to minimize  $MSE_{bias}(\hat{x}, x_{true}) = E[||\hat{x} - x_{true}||^2]$ .
For the calculation below, we denote  $a(r) \coloneqq p / (p + r)$  and use the identity  $\sin 2\theta = 2\cos \theta \sin \theta$ . In addition, from radial symmetry of  $dx = z - x_0$  we have  $E[dx_1^2] = E[dx_2^2]$  and  $E[dx_i] = 0$ , thus we can denote  $v \coloneqq Var(dx_i) = E[dx_i^2]$ .
$$
\begin{array}{l} M S E _ {b i a s} (\hat {x} (a), x _ {t r u e}) = E | | \hat {x} (a) - x _ {t r u e} | | ^ {2} \\ = E \left[ \left((a - \frac {r _ {0} \sin^ {2} \theta + p}{p + r _ {0}}) d x _ {1} + \frac {r _ {0} \sin (2 \theta) / 2}{p + r _ {0}} d x _ {2}\right) ^ {2} \right. \\ \left. + \left((a - \frac {r _ {0} \cos^ {2} \theta + p}{p + r _ {0}}) d x _ {2} + \frac {r _ {0} \sin (2 \theta) / 2}{p + r _ {0}} d x _ {1}\right) ^ {2} \right] \\ = E \left[ d x _ {1} ^ {2} \left(a ^ {2} - 2 a \frac {r _ {0} \sin^ {2} \theta + p}{p + r _ {0}} + C _ {1}\right) + \frac {r _ {0} ^ {2} \sin^ {2} (2 \theta) / 4}{(p + r _ {0}) ^ {2}} d x _ {2} ^ {2} + A _ {1} d x _ {1} d x _ {2} \right. \\ + d x _ {2} ^ {2} \left(a ^ {2} - 2 a \frac {r _ {0} \cos^ {2} \theta + p}{p + r _ {0}} + C _ {2}\right) + \frac {r _ {0} ^ {2} \sin^ {2} (2 \theta) / 4}{(p + r _ {0}) ^ {2}} d x _ {1} ^ {2} + A _ {2} d x _ {1} d x _ {2} \Bigg ] \\ = 2 v a ^ {2} - 2 v a \frac {r _ {0} + 2 p}{p + r _ {0}} + v (C _ {1} + C _ {2}) + v \frac {r _ {0} ^ {2} \sin^ {2} (2 \theta) / 2}{(p + r _ {0}) ^ {2}}, \\ \end{array}
$$
where  $C_{1,2}$  are independent of  $a$ , and  $A_{1,2}$  are multiplied by  $E[dx_1dx_2] = 0$  and vanish. To minimize we calculate
$$
0 = \frac {\partial M S E _ {b i a s} (\hat {x} (a) , x _ {t r u e})}{\partial a} = 4 v \cdot a - 2 v \frac {2 p + r _ {0}}{p + r _ {0}},
$$
$$
a = \frac {p + r _ {0} / 2}{p + r _ {0}}.
$$
Notice that  $MSE_{bias}$  clearly diverges as  $|a| \to \infty$ , hence the only critical point necessarily corresponds to a minimum of the  $MSE$ . Hence, the optimal  $MSE$  is given when substituting the following  $r$  in  $\hat{R}_{opt}$ :
$$
r = p / a - p = \frac {p ^ {2} + p r _ {0} - (p ^ {2} + p r _ {0} / 2)}{p + r _ {0} / 2} = \frac {p r _ {0}}{2 p + r _ {0}}.
$$
Finally, recall that  $(\hat{R}_{est})_{ii} = r_0 / 2$  and compare to  $r$  directly:
$$
(\hat {R} _ {e s t}) _ {i i} - (\hat {R} _ {o p t}) _ {i i} = r _ {0} / 2 - r = \frac {r _ {0} ^ {2} / 2}{2 p + r _ {0}} > 0.
$$
B OKF: Extended Experiments
B.1 Additional Scenarios and Baselines: A Case Study
In this section, we extend the experiments of Section 5.1 with a detailed case study. The case study considers 5 types of tracking scenarios (benchmarks) and 4 variants of the KF (basiines) - 20 experiments in total. In each experiment, we compare the test MSE of OKF against the standard KF. The experiments in Section 4 and Section 5.1 are 3 particular cases. For each benchmark, we simulate 1500 targets for training and 1000 targets for testing.
**Benchmarks (scenarios):** Section 5 discusses the sensitivity of Algorithm 1 to violations of Assumption 1. In this case study, we consider 5 benchmarks with different subsets of violations of
Table 1: Benchmarks and the properties that define them. "V" means that the benchmark satisfies the property.
Benchmark	anisotropic	polar	uncentered	acceleration	turns
Toy	O	O	O	O	O
Close	V	V	O	O	O
Const_v	V	V	V	O	O
Const_a	V	V	V	V	O
Free	V	V	V	V	V
(a) Toy
(b) Close
(c) Const_v
(d) Const_a
Figure 11: Samples of targets trajectories in the various benchmarks, projected onto the XY plane.
(e) Free motion
Assumption 1. The Free Motion benchmark is intended to represent a realistic Doppler radar problem, with targets and observations simulated as in Section 4: each target trajectory consists of multiple segments of different turns and accelerations. On the other extreme, the Toy benchmark (Problem 1) introduces multiple simplifications (as visualized in Fig. 5). In the Toy benchmark, the only violation of Assumption 1 is the non-linear observation  $H$ , as discussed in Section 5.1. Note that Section 5.2 and Section 5.3 experiment with settings of a linear observation model.
We design 5 benchmarks within the spectrum of complexity between Toy and Free Motion. Each benchmark is defined as a subset of the following properties, as specified in Table 1 and visualized in Fig. 11:
- anisotropic: horizontal motion is more likely than vertical (otherwise direction is distributed uniformly).
- polar: radar noise is generated i.i.d in spherical coordinates (otherwise noise is Cartesian i.i.d).
- uncentered: targets are dispersed in different locations far from the radar (otherwise they are concentrated in the center).
- acceleration: speed change is allowed (through intervals of constant acceleration).
- turns: non-straight motion is allowed.
Baselines (KF variants): All the experiments above compare OKF to the standard KF baseline. In practice, other variants of the KF are often in use. Here we define 4 such variants as different baselines to the experiments. In each experiment, we compare the baseline tuned by Algorithm 1 to its Optimized version trained by Algorithm 2 (denoted with the prefix "O" in its name). For Algorithm 2, we use the Adam optimizer with a single training epoch over the 1500 training trajectories, 10 trajectories per training batch, and learning rate of 0.01. The optimization was run for all baselines in parallel and required a few minutes per benchmark, on eight i9-10900X CPU cores in a single Ubuntu machine.
The different baselines are designed as follows. EKF baselines use the non-linear Extended KF model [Sorenson, 1985]. The EKF replaces the approximation  $H \approx H(z)$  of Section 4 with  $H \approx \nabla_{x} h(\hat{x})$ , where  $h(x) = H(x) \cdot x$  and  $\tilde{x}$  is the current state estimate. Polar baselines (denoted with "p") represent the observation noise  $R$  with spherical coordinates, in which the polar radar noise is i.i.d.
Results: Table 2 summarizes the test errors (MSE) in all the experiments. In each cell, the left column corresponds to the baseline Algorithm 1, and the right to Algorithm 2. In the model names,
Table 2: Test MSE results of Algorithm 1 and Algorithm 2 over 5 benchmarks (scenarios) and 4 baselines (variants of KF). For KFp we also consider an "oracle" baseline with perfect knowledge of the noise.
Benchmark	KF	OKF	KFp	KFp (oracle)	OKFp	EKF	OEKF	EKFp	OEKFp
Toy	151.7	84.2	269.6	-	116.4	92.8	79.4	123.0	109.1
Close	25.0	24.8	22.6	22.5	22.5	26.4	26.1	24.5	24.1
Const_v	90.2	90.0	102.3	102.3	89.2	102.5	99.7	112.7	102.1
Const_a	107.5	101.6	118.4	118.3	100.3	110.0	107.0	126.0	108.7
Free	125.9	118.8	145.6	139.3	117.9	135.8	121.9	149.3	120.0
(b)
Figure 12: Summary of the test MSE of Algorithm 1 and Algorithm 2 in different benchmarks (scenarios) and baselines. This is a different presentation of the results of Table 2. (a) also includes  $95\%$  confidence intervals. (b) shows, for each of the 20 experiments (5 benchmarks  $\times 4$  baselines), the MSE ratio between Algorithm 1 and Algorithm 2. We see that Algorithm 2 wins in all the experiments (ratio is always larger than 1) - in some cases by large margins. The dashed line represents the average MSE ratio over all the experiments, showing an average advantage of  $20\%$  to OKF.
"O" stands for optimized, "E" for EKF and "p" for polar (or spherical). The same results are also shown with confidence intervals in Fig. 12. Below we discuss the main findings.
Choosing the KF configuration is not trivial: Consider the non-optimized KF baselines (left column in every cell in Table 2). In each benchmark, the results are sensitive to the baseline, i.e., to the choice of KF configuration -  $R$ 's coordinates and whether to use EKF. For example, in the Toy benchmark, EKF is the best design, since the observation model  $H$  is non-linear. In other benchmarks, however, the winning baselines may come as a surprise:
1. Under non-isotropic motion direction (all benchmarks except Toy), EKF is worse than KF despite the non-linearity. It is possible that the horizontal prior reduces the stochasticity of  $H$ , making the derivative-based approximation unstable.
2. Even when the observation noise is spherical i.i.d, spherical representation of  $R$  is not beneficial when targets are scattered far from the radar (last 3 benchmarks). It is possible that with distant targets, Cartesian coordinates have a more important role in expressing the horizontal prior of the motion.
Since the best KF variant per benchmark seems hard to predict in advance, a practical system cannot rely on choosing the KF variant optimally – and should rather be robust to this choice.
OKF is more accurate and more baseline-robust: For every benchmark and every baseline (20 experiments in total), OKF (right column) outperformed noise estimation (left column). In addition, the variance between the baselines reduces under optimization, i.e., OKF makes the KF more robust to the selected configuration.
OKF outperforms an oracle baseline: We designed an "oracle" KF baseline - with perfect knowledge of the observation noise covariance  $R$  in spherical coordinates. We used it for all benchmarks except for Toy (in which the radar noise is not generated in spherical coordinates). Note that in the constant-speed benchmarks (Close and Const_v),  $Q = 0$  and is estimated quite accurately; hence, in these benchmarks the oracle has a practically perfect knowledge of both noise covariances. Nevertheless, the oracle yields very similar results to Algorithm 1. This indicates that the benefit of OKF is not in a better estimation accuracy of  $Q$  and  $R$ , but rather in optimizing the desired objective.
B.2 Sensitivity to Train Dataset Size
Each benchmark in the case-study of Appendix B.1 has 1500 targets in its train data. One may argue that numeric optimization may be more sensitive to smaller datasets than noise estimation; and even more so, when taking into account that the optimization procedure "wastes" a portion of the train data as a validation set.
In this section we test this concern empirically, by repeating some of the experiments of Appendix B.1 with smaller subsets of the train datasets – beginning from as few as 20 training trajectories. Fig. 13 shows that the advantage of OKF over KF holds consistently for all sizes of train datasets, although it is indeed increases with the size. Interestingly, in the Free Motion benchmark, the test errors of KF and KFp increase with the amount of train data!
Figure 13: The advantage of OKF over KF holds consistently for all sizes of train datasets – including as small datasets as 20 trajectories. The shadowed areas correspond to  $95\%$  confidence intervals.
B.3 Generalization: Sensitivity to Distributional Shifts
In Appendix B.1, we demonstrate the robustness of OKF in different tracking scenarios: in every benchmark, OKF outperformed the standard KF over out-of-sample test data. This means that OKF did not overfit the noise in the training data. What about out-of-distribution test data? OKF learns patterns from the specific distribution of the train data – how well will it generalize to different distributions?
Section 4 already addresses this question to some extent, as OKF outperforms both KF and NKF over out-of-distribution target accelerations (affecting both speed changes and turns radius). In terms of Eq. (1), the modified acceleration corresponds to different magnitudes of motion noise  $Q$ ; that is, we change the noise after OKF optimized the noise parameters. Yet, OKF adapted to the change without further optimization, with better accuracy than the standard KF. Thus, the results of Section 4 already provide a significant evidence for the robustness of OKF to certain distributional shifts.
In this section, we present a yet stronger evidence for the robustness of OKF – not over a parametric distributional shift, but over entirely different benchmarks. Specifically, we consider the 5
Figure 14: Generalization tests: OKF vs. KF under distributional shifts between scenarios.
benchmarks (or scenarios) of Appendix B.1. For every pair (train-scenario, test-scenario), we train both KF and OKF on data of the train-scenario, then test them on data of the test-scenario. For every such pair of scenarios, we measure the generalization advantage of OKF over KF through  $MSE\_ratio = MSE(KF) / MSE(OKF)$  (where  $MSE\_ratio > 1$  indicates advantage to OKF). To measure the total generalization advantage of a model trained on a certain scenario, we calculate the geometric mean of  $MSE\_ratio$  over all the test-scenarios (or equivalently, the standard mean over the logs of the ratios). The logarithmic scale guarantees a symmetric view of this metric of ratio between two scores.
This test is quite noisy, since a model optimized for a certain scenario may legitimately be inferior in other scenarios. Yet, considering all the results together in Fig. 14, it is evident that OKF provides more robust models: it generalizes better in most cases, sometimes by a large margin; and loses only in a few cases, always by a small margin.
B.4 Ablation Test: Diagonal Optimization
The main challenge in Section 3 and Algorithm 2 is to apply standard numeric optimization while keeping the symmetric and positive-definite constraints (SPD) of the parameters  $Q, R$ . To that end, we apply the Cholesky parameterization to  $Q$  and  $R$ . In this section, we study the importance of this parameterization via an ablation test.
For the ablation, we define a naive version of OKF with a diagonal parameterization of  $Q$  and  $R$ . Such parameterization is common in the literature: "since both the covariance matrices must be constrained to be positive semi-definite,  $Q$  and  $R$  are often parameterized as diagonal matrices" [Formentin and Bittanti, 2014]. We denote the diagonal variant by  $DKF$ , and test it on all 5 Doppler benchmarks of Appendix B.1.
Table 3 displays the results. In the first 3 benchmarks, where the target motion is linear, DKF is indistinguishable from OKF. However, in the 2 non-linear benchmarks (which are also the same benchmarks used in Section 4), DKF is inferior to OKF, though still outperforms the standard KF.
Table 3: Ablation test: Diagonal optimized KF, tested on the 5 benchmarks of Appendix B.1.
Benchmark	KF	DKF	OKF
Toy	151.7	84.2	84.2
Close	25.0	24.8	24.8
Const_v	90.2	90.1	90.0
Const_a	107.5	106.1	101.6
Free	125.9	121.1	118.8
B.5 Video Tracking: Dataset License
The MOT20 video dataset [Dendorfer et al., 2020] is available under Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License. In Section 5.2, we used the videos MOT20-01, MOT20-02 and MOT20-03 for training, and MOT20-05 for testing.
B.6 Lidar-based State Estimation: Visualization
Fig. 15 visualizes a sample of the simulated trajectories and the model predictions in the lidar-based state estimation of Section 5.3.
(a)
(b)
Figure 15: (a) A sample of simulated self-driving trajectories. (b) Segments of turns within a sample trajectory, and the corresponding lidar-based estimations.
C Neural KF: Extended Discussion and Experiments
Preliminaries - RNN and LSTM: Recurrent neural networks (RNN) [Rumelhart et al., 1986] are neural networks that are intended to be iteratively fed with sequential data samples, and that pass information (the hidden state) over iterations. Every iteration, the hidden state is fed to the next copy of the network as part of its input, along with the new data sample. Long Short Term Memory (LSTM) [Hochreiter and Schmidhuber, 1997] is an architecture of RNN that is particularly popular due to the linear flow of the hidden state over iterations, which allows to capture memory for relatively long term. The parameters of a RNN are usually optimized in a supervised manner with respect to a training dataset of input-output pairs.
Neural Kalman Filter: We introduce the Neural Kalman Filter (NKF), which incorporates an LSTM model into the KF framework. The framework provides a probabilistic representation (rather than point estimate) and a separation between the prediction and update steps. The LSTM is an architecture of recurrent neural networks, and is a key component in many SOTA algorithms for non-linear sequential prediction [Neu et al., 2021]. We use it for the non-linear motion prediction.
As shown in Fig. 16, NKF uses separate LSTM networks for prediction and update steps. In the prediction step, the target acceleration is predicted on top of the linear motion model, instead of predicting the state directly. This regularized formulation is intended to express our domain knowledge about the kinematic motion of physical targets.
Extended experiments: We extend the experiments of Section 4 with additional versions of NKF:
- Predicted-acceleration KF (aKF): a variant of NKF that predicts the acceleration but not the covariances  $Q$  and  $R$ .
Figure 16: The Neural Kalman Filter (NKF). Differences from Fig. 2 are highlighted.  $\Delta t$  is constant;  $G, Q$  are the outputs of an LSTM network with hidden state  $h_a$ ; and  $R$  is the output of an LSTM with hidden state  $h_r$ .
(b) Const_a benchmark (no turns)
Figure 17: The relative MSE and NLL results of various models in comparison to the standard KF model. The textual labels specify the absolute MSE and NLL. Note that certain bars of NLL are of entirely different scale and thus are cropped in the figure (their values can be seen in the labels). In each benchmark, the models were trained with relation to MSE loss, on train data of the middle acceleration-range: the two other acceleration ranges in each benchmark correspond to generalization over distributional shifts.
- Neural KF (NKF): the model used in Section 4 and illustrated in Fig. 16.
- Neural KF with H-prediction (NKFH): a variant of NKF that also predicts the observation model  $H$  in every step.
In addition, while we still train with MSE loss, we add the test metric of Negative-Log-Likelihood (NLL) – of the true state w.r.t the estimated distribution. Note that the NLL has an important role in the multi-target matching problem (which is out of the scope of this work).
For each benchmark and each model, we train the model on train data with a certain range of targets acceleration (note that acceleration affects both speed changes and turns sharpness), and tested it on targets with different acceleration ranges, some of them account for distributional shifts. For each model we train two variants – one with Cartesian representation of the observation noise  $R$ , and one with spherical representation (as in the baselines of Appendix B.1) – and we select the one with the higher validation MSE (where the validation data is a portion of the data assigned for training).
Fig. 17a shows that in the free-motion benchmark, all the 3 neural models improve the MSE in comparison to the standard KF, yet are outperformed by OKF. Furthermore, while OKF has the best NLL, the more complicated models NKF and NKFH increase the NLL in orders of magnitude. Note that the instability of NKFH is expressed in poor generalization to lower accelerations in addition to the extremely high NLL score.
Fig. 17b shows that in Const_a benchmark, all the 3 neural models improve the MSE in comparison to the standard KF, but only NKFH improves in comparison to OKF as well. On the other hand, NKFH still suffers from very high NLL.
In summary, all 3 variants of NKF outperform the standard KF in both benchmarks in terms of MSE. However, when comparing to OKF instead, aKF and NKF become inferior, and the comparison between NKFH and OKF depends on the selected benchmark and metric.