Kalman Filter Is All You Need: Optimization Works When Noise Estimation Fails

1 INTRODUCTION . The Kalman Filter ( KF ) ( Kalman , 1960 ) is a celebrated method for linear filtering and prediction , with applications in many fields including tracking , guidance , navigation and control ( Zarchan and Musoff , 2000 ; Kirubarajan , 2002 ) . Due to its simplicity and robustness , it remains highly popular – with over 10,000 citations in the last 5 years alone ( Google Scholar , 2021 ) – despite the rise of many non-linear sequential prediction models ( e.g. , recurrent neural networks ) . The KF relies on the following model for a dynamic system : Xt+1 = FtXt + ωt ( ωt ∼ N ( 0 , Q ) ) Zt = HtXt + νt ( νt ∼ N ( 0 , R ) ) ( 1 ) where Xt is the state of the system at time t ( whose estimation is usually the goal ) , and its dynamics are modeled by the linear operator Ft up to the random noise ωt with covariance Q ; and Zt is the observation , which is modeled by the linear operator Ht up to the noise νt with covariance R. When the operators Ft , Ht are not assumed to depend on time , the notation may be simplified to F , H . To use the KF , one must determine the noise parameters Q , R . The filtering errors ( i.e. , estimation errors of the states { Xt } ) are minimized when Q and R correspond to the true covariance matrices of the noise ( Humpherys et al. , 2012 ) . Thus , these parameters are usually determined by noise estimation . In absence of system state data { xt } ( the ” ground truth ” ) , many methods have been suggested to determine Q and R from observations data { zt } alone ( Abbeel et al. , 2005 ; Odelson et al. , 2006 ; Zanni et al. , 2017 ; Park et al. , 2019 ) . When ground-truth data is available , however , noise estimation trivially reduces to calculation of the sample covariance matrices ( Lacey , 1998 ) : R̂ : = Cov ( { zt −Htxt } t ) , Q̂ : = Cov ( { xt+1 − Ftxt } t ) . ( 2 ) Indeed , as stated by Odelson et al . ( 2006 ) , “ the more systematic and preferable approach to determine the filter gain is to estimate the covariances from data ” . Our work focuses on such problems with ground-truth available for learning ( but not for inference after the learning , of course ) , which was motivated by a real-world Doppler radar estimation problem . Noise estimation is often not optimal : The equivalence between noise estimation and errors minimization can be proved under the standard KF assumptions – that is , known and linear dynamics and observation models ( Ft , Ht ) , with i.i.d and normally-distributed noises ( { ωt } , { νt } ) ( Humpherys et al. , 2012 ) . However , as put by Thomson ( 1994 ) , “ experience with real-world data soon convinces one that stationarity and Gaussianity are fairy tales invented for the amusement of undergraduates ” – and linearity and independence can be safely added to this list . Therefore , under realistic assumptions , the covariance of the noise does not necessarily correspond to optimal filtering . We introduce a case study in the context of radar tracking , where we demonstrate that even using the true covariance of the noise ( ” oracle ” noise-estimation ) is sub-optimal in a variety of scenarios – including very simple scenarios with relatively minor violation of the KF assumptions . In Appendices E and F , we also analyze this phenomenon analytically for two private cases ( non-linearity in Doppler radar and non-i.i.d noise in lidar ) , where the violation of a single KF assumption is shown to modify the effective noise . By providing this extensive evidence that noise-estimation is a sub-optimal way to tune a KF even in presence of system-states ground-truth data , we re-open a problem that was considered solved for decades ( Kalman , 1960 ) . We also show that seemingly small changes in the properties of the scenario may lead to major changes in the desired design of the KF , e.g. , whether to use a KF or an Extended KF ( Sorenson , 1985 ) . In certain cases , the design choices are easy to overlook ( e.g. , Cartesian vs. spherical coordinates ) , and are not trivial to make even if noticed . As a result , it is impractical to manually choose or develop a variant of the KF for every problem . Rather , we should assume that our model is sub-optimal , and leverage data to deal with the sub-optimality as robustly as possible . Optimization is optimal : We consider Q and R as model parameters that should be optimized with respect to the filtering errors ( i.e. , system-state estimation errors ) – rather than estimating the noise . While both noise estimation and errors optimization rely on exploitation of data , only the latter explicitly addresses the actual goal of solving the filtering problem . Gradient-based optimization methods are usually effective in the field of machine learning , but applying them naively to the entries of Q and R may violate the symmetry and positive-definiteness ( SPD ) constraints of the covariance matrices . Indeed , even works that come as far as optimizing Q and R ( instead of estimating the noise ) usually apply limited optimization methods , e.g. , gridsearch ( Coskun et al. , 2017 ) or diagonal restriction of the covariance matrices ( Formentin and Bittanti , 2014 ; Li et al. , 2019 ) . To address this issue , we use a parameterization based on Cholesky decomposition ( Horn and Johnson , 1985 ) , which allows us to apply gradient-based optimization to SPD matrices . This method is computationally efficient compared to other general gradient-based methods for SPD optimization ( Tsuda et al. , 2005 ; Tibshirani , 2015 ) . We demonstrate that the optimization reduces the errors of the KF consistently : over different variants of the KF , over different violations of KF assumptions , over different domains ( tracking from radar , video or lidar ) , over small and large training datasets , and even under distributional shifts between train and test datasets . Furthermore , we show that optimization improves the robustness to design decisions , by shrinking the performance gaps between different variants of the KF . As explained above , we extensively justify the need to optimize the KF in many practical problems , and suggest a simple solution which is effective , robust , computationally efficient , and relies on standard tools in supervised machine learning . As a result , we believe that in the scope of filtering problems with available ground-truth , whenever the KF assumptions are not strictly-guaranteed , the suggested optimization method should become the new standard procedure for the KF tuning . Unfair comparison : Many learning algorithms have been suggested to address non-linearity in filtering problems , e.g. , based on Recurrent Neural Networks ( RNN ) . Such works often use a linear tool such as the KF as a baseline for comparison – with tuning parameters being sometimes ignored ( Gao et al. , 2019 ) , sometimes based on noise estimation ( fa Dai et al. , 2020 ) , and sometimes optimized in a limited manner using trial-and-error ( Jamil et al. , 2020 ) or grid-search ( Coskun et al. , 2017 ) . Our findings imply that such a methodology yields over-optimistic conclusions , since the baseline is not optimized to the same level as the learning model . This may result in adoption of over-complicated algorithms with no actual added value . Instead , any learning algorithm should be compared to a baseline that is optimized using a similar method ( e.g. , gradient-descent with respect to the errors ) . Indeed , we consider an extension of the KF based on LSTM , which is the key component in many SOTA algorithms for non-linear sequential prediction in recent years ( Neu et al. , 2021 ) . For radar tracking with non-linear motion , we demonstrate how the LSTM seems to provide a significant improvement over the KF . Then , we show that the whole improvement comes from optimization of parameters , and not from the expressive non-linear architecture . In particular , this result demonstrates the competitiveness of our suggested method versus SOTA sequential prediction models . Recent works in the area of machine learning have already shown that advanced algorithms often obtain most of their improvement from implementation nuances ( Engstrom et al. , 2019 ; Andrychowicz et al. , 2020 ; Henderson et al. , 2017 ) . Our work continues this line of thinking and raises awareness to this issue in the domain of filtering problems . Contribution : We show ( empirically and analytically ) that the gold-standard noise estimation method to tune the KF given ground-truth data is often sub-optimal ; demonstrate how this leads to underevaluation of the KF compared to optimized filtering models such as neural networks ; suggest a simple method to optimize the KF , using gradient-based optimization with Cholesky parameterization ; and extensively demonstrate its improved accuracy and its robustness to model misspecification . Limitations : This work relies on the availability of ground-truth system-states in the training data to allow supervised learning . Ground-truth data is often available from simulations , controlled experiments or manual labeling . Within this scope , we show that although noise estimation is straightforward ( Equation 2 ) , it is often not the right task to address . Another limitation is in the theoretical guarantees of gradient-based optimization ( see Appendix L ) , despite its wide success in many fields . The paper is organized as follows : Section 2 reviews the KF . Section 3 introduces our method for efficient KF optimization . Section 4 justifies the necessity of KF optimization through a detailed case study . Section 5 presents a neural version of the KF which reduces the errors compared to a standard KF – but not when compared to an optimized KF . Section 6 discusses related works . 2 PRELIMINARIES : THE KALMAN FILTER ALGORITHM . The KF algorithm ( Kalman , 1960 ; Humpherys et al. , 2012 ) relies on Equation 1 for a dynamic system model . It keeps an estimate of the state Xt , represented as the mean xt and covariance Pt of a normal distribution . As shown in Figure 1 , it alternately predicts the next state using the dynamics model ( prediction step ) , and processes new information from incoming observations ( update or filtering step ) . The KF yields optimal state estimations – but only under a restrictive set of assumptions ( Kalman , 1960 ) , as specified in Def- inition 2.1 . Note that normality of the noise is excluded since it is not necessary for optimality ( Humpherys et al. , 2012 ) , although it is also often assumed . Definition 2.1 ( KF assumptions ) . Ft , Ht of Equation 1 are known matrices that do not depend on the state ( i.e. , correspond to linear models ) ; ωt , νt are i.i.d random variables with zero-mean and constant , known covariance matrices Q , R , respectively ; and the initial state distribution is known . Certain assumptions violating in Definition 2.1 can be partially handled by variations of the KF , such as the Extended KF ( EKF ) ( Sorenson , 1985 ) which replaces the linear models Ft , Ht with local linear approximations , and the Unscented KF ( UKF ) ( Wan and Van Der Merwe , 2000 ) which applies the filtering through sigma-points sampled from the estimated distribution . The use of multiple tracking models alternately is also possible using switching mechanisms ( Mazor et al. , 1998 ) . While Ft and Ht are usually determined based on domain knowledge , Q and R are often estimated from data as the covariance of the noise . As mentioned in Section 1 , this can be done using Equation 2 if ground-truth data is available , or using more sophisticated methods otherwise ( Odelson et al. , 2006 ; Feng et al. , 2014 ; Park et al. , 2019 ) . See Appendix A for a detailed introduction of the KF and recurrent neural networks ( RNN , LSTM ) .

Summary: The paper studies the problem of using Kalman filter for estimating the state of a dynamical system when the noise covariance matrices, for both state dynamics and observation, are unknown. The paper assumes access to trajectories of both state and observation. In this setting, a natural approach to solve the problem is to use the data to form an estimate for the covariance matrices. However, the paper argues that an optimization procedure to find noise covariance matrices to minimize the MSE is favorable and should become the "new standard procedure for KF tuning". With several numerical experiments, the paper illustrates that the optimization based KF tuning provides much better and robust result compared to standard KF based on estimation, and the comparisons made in the advanced neural network based estimation literature is not fair.