R ECURRENT K ALMAN N ETWORKS : F ACTORIZED I NFERENCE IN H IGH -D IMENSIONAL
D EEP F EATURE S PACES
Anonymous authors
Paper under double-blind review
A BSTRACT
In order to integrate uncertainty estimates into deep time-series modelling,
Kalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep learning models. Yet, such approaches typically rely on approximate inference techniques such as variational inference which makes learning more complex and often less scalable due to approximation errors.
We propose a new deep approach to Kalman ﬁltering which can be learned directly in an end-to-end manner using backpropagation without additional approximations. Our approach uses a highdimensional factorized latent state representation for which the Kalman updates simplify to scalar operations and thus avoids hard to backpropagate, computationally heavy and potentially unstable matrix inversions.
Moreover, we use locally linear dynamic models to efﬁciently propagate the latent state to the next time step.
While our locally linear modelling and factorization assumptions are in general not true for the original low-dimensional state space of the system, the network ﬁnds a high-dimensional latent space where these assumptions hold to perform efﬁcient inference.
This state representation is learned jointly with the transition and noise models.
The resulting network architecture, which we call Recurrent Kalman Network (RKN), can be used for any time-series data, similar to a LSTM (Hochreiter and Schmidhuber, 1997) but uses an explicit representation of uncertainty.
As shown by our experiments, the RKN obtains much more accurate uncertainty estimates than an LSTM or Gated Recurrent Units (GRUs)
(Cho et al., 2014) while also showing a slightly improved prediction performance and outperforms various recent generative models on an image imputation task.
I NTRODUCTION
State-estimation in unstructured environments is a very challenging task as observations or measurements of the environment are often high-dimensional and only provide partial information about the state.
Images are a good example: Even for low resolution, the number of pixels can quickly exceed tens or hundreds of thousands and it is impossible to obtain any information about the dynamics, such as velocities, from a single image.
Additionally, the observations may be noisy or may not contain useful information for the task at hand.
Such noise can for example be introduced by poor illumination or motion blur and occlusions can prevent us from observing some or all relevant aspects of the scene.
In addition to state estimation, it is also often desirable to predict future states or observations, for example, in order to assess the consequences of future actions.
To this end, an initial estimate of the current state is necessary which again has to be inferred from observations.
In such environments we typically also have to deal with a high uncertainties in the state estimates. Being able to model this uncertainty is crucial in many decision making scenarios, e.g., if we need to decide to perform an action now or wait until more information about the scene is available.
Deep learning models have been very successful for time-series modelling in unstructured environments.
Classical models such as LSTMs (Hochreiter and Schmidhuber, 1997) or GRUs (Cho et al., 2014) perform well but fail to capture the uncertainty of the state estimate. Recent probabilistic deep learning approaches have used the Kalman ﬁlter (KF) as a tool to integrate uncertainty estimates into deep time-series modelling (Haarnoja et al., 2016;
Watter et al., 2015; Archer et al., 2015;
Fraccaro et al., 2017; Krishnan et al., 2017). These approaches use the KF to perform inference in a low-dimensional (latent) state space that is typically deﬁned by a deep encoder.
However, using
KF in such a state space comes with two main limitations.
In order to be usable for non-linear dynamics, we have to introduce approximations such as the extended KF (Haarnoja et al., 2016) and variational inference methods (Krishnan et al., 2017; Fraccaro et al., 2017). Moreover, the KF equations require computationally expensive matrix inversions that are hard to scale to high dimensional latent spaces for more complex systems and computationally demanding to fully backpropagate in an end-to-end manner. Most of these methods are implemented as (variational) auto-encoders and are therefore also limited to predicting future observations or imputing missing observations and can not be directly be applied to state estimation.
We introduce the Recurrent Kalman Network , an end-to-end learning approach for Kalman ﬁltering and prediction.
While Kalman ﬁltering in the original state space requires approximations due to the non-linear models as well as matrix inversions that are hard to back-propagate and computationally expensive, the RKN uses a learned high-dimensional latent state representation that allows for efﬁcient inference using locally linear transition models and a factorized belief state representation which avoids expensive and numerically problematic matrix inversions.
Conceptually, this idea is related to kernel methods which use high-dimensional feature spaces to approximate nonlinear functions with linear models (Gebhardt et al., 2017).
However, in difference to kernel feature spaces, our feature space is given by a deep encoder and learned in an end-to-end manner.
The RKN can be used for any time-series data set for which LSTMs and GRUs are currently the state of the art.
In contrast to those, the RKN uses an explicit representation of uncertainty which governs the gating between keeping the current information in memory or updating it with the current observation.
While the RKN shows a slightly improved performance in terms of state estimation errors, both LSTMs and GRUs struggle with estimating the uncertainty of the prediction while the RKN can provide accurate uncertainty estimates.
In relation to existing KF-based approaches, our approach can be used for state estimation as well as for generative tasks such as image imputation.
We also show that we outperform state of the art methods on a complex image imputation task.
1. R ELATED W ORK
Using encoders for time-series modelling of high-dimensional data such as images is a common approach.
Such encoders can also be easily integrated in well known deep time-series models such as LSTMs (Hochreiter and Schmidhuber, 1997) or GRUs (Cho et al., 2014). These models are very effective but do not provide good uncertainty estimates as shown in our experiments. Pixel to
Torques (P2T) (Wahlstr ¨ om et al., 2015) employs an autoencoder to obtain low dimensional latent representations from images together with a transition model. They subsequently use the models to perform control in the latent space. Embed to Control (E2C) (Watter et al., 2015) can be seen as an extension of the previous approach with the difference that a variational autoencoder (Kingma and
Welling, 2013) is used.
However, both of these approaches are not recurrent and rely on observations which allow inferring the whole state from a single observation.
They can therefore not deal with noisy or missing data.
Another family of approaches interprets encoder-decoder models as latent variable models that can be optimized efﬁciently by variational inference.
They derive a corresponding lower bound and optimize it using the stochastic gradient variational Bayes approach (Kingma and Welling, 2013). Black Box Variational Inference (BB-VI) (Archer et al., 2015) proposes a structured Gaussian variational approximation of the posterior, which simpliﬁes the inference step at the cost of maintaining a tri-diagonal covariance matrix of the full state.
To circumvent this issue, Structured Inference Networks (SIN) (Krishnan et al., 2017) employ a ﬂexible recurrent neural network to approximate the dynamic state update. Deep Variational Bayes Filters (DVBF) (Karl et al., 2016) integrate general Bayes ﬁlters into deep feature spaces while the Kalman Variational Autoencoder (KVAE) (Fraccaro et al., 2017) employs the classical Kalman Filter and allows not only ﬁltering but also smoothing.
Variational Sequential Monte Carlo (VSMC) (Naesseth et al., 2017) uses particle ﬁlters instead, however they are only learning the proposal function and are not working in learned latent spaces. Yet, these models can not be directly used for state estimation as they are formulated as generative models of the observations without the notion of the real state of the system. Moreover, the use of variational inference introduces an additional approximation that might affect the performance of the algorithms.
The approaches given in (Archer et al., 2015; Fraccaro et al., 2017; Haarnoja et al., 2016) directly use the Kalman update equations in the latent state, which limits these approaches to rather low dimensional latent states due to the expensive matrix inversions. scalestate uncernoise direct able est. tainty opt.
[Table] [Table] [Table]
The BackpropKF (Haarnoja et al., 2016) applies a CNN to estimate the observable parts of the true state given the observation.
Similar to our approach, this CNN additionally outputs a covariance matrix indicating the models certainty about the estimate and allows the subsequent use of an (extended) Kalman ﬁlter with known transition model.
In contrast, we let our model chose the feature space that is used for the inference such that locally linear models can be learned and the KF computations can be simpliﬁed due to our factorization assumptions.
A summary of all the approaches, listing also their basic properties, can be seen in Table 1.
We compare the approaches whether they are scaleable to high dimensional latent states, whether they can be used for state estimation, whether they can provide uncertainty estimates, whether they can handle noise or missing data and whether the objective can be optimized directly or via a lower bound. All probabilistic generative models rely on variational inference which optimizes a lower bound, which potentially affects the performance of the algorithms.
The P2T and E2C approaches rely on the Markov assumption and therefore can not deal with noise (or need very large window sizes).
Traditional recurrent models (LSTMs, GRUs) can be trained directly by backpropagation through time and therefore typically yield very good performance but are lacking uncertainty estimates (which, however, can be artiﬁcially added as in our experiments). Our RKN approach combines the advantages of all methods above as it can be learned by direct optimization without the use of a lower bound and it provides a principled way of representing uncertainty inside the neural network.
F ACTORIZED I NFERENCE IN D EEP F EATURE S PACES
Lifting the original input space to a high-dimensional feature space where linear operations are feasible is a common approach in machine learning, e.g., in kernel regression and SVMs.
The Recurrent
Kalman Network (RKN) transfers this idea to state estimation and ﬁltering, i.e., we learn a high dimensional deep feature space that allows for efﬁcient inference using the Kalman update equations even for complex systems with high dimensional observations.
To achieve this, we assume that the belief state representation can be factorized into independent Gaussian distributions as described in the following sections. 2. L ATENT O BSERVATION AND S TATE S PACES
The RKN encoder learns a mapping to a high-dimensional latent observation space W = R m . The encoder also outputs a vector of uncertainty estimates σ obs t , one for each entry of the latent observation w t .
Hence, the encoder can be represented by ( w t , σ obs t )
=
enc ( o t ) .
The latent state space Z
= R n of the RKN is related to the observation space W by the linear latent observation model H = (cid:2) I m 0 m × ( n − m ) (cid:3) , i.e., w =
Hz with w ∈ W and z ∈ Z . I m denotes the m × m identity matrix and 0 m × ( n − m ) denotes a m × ( n − m ) matrix ﬁlled with zeros.
The idea behind this choice is to split the latent state vector z t into two parts, a vector p t for holding information that can directly be extracted from the observations and a vector m t to store information inferred over time, e.g., velocities.
We refer to the former as the observation or upper part and the latter as the memory or lower part of the latent state.
For an ordinary dynamical system and images as observations the former may correspond to positions while the latter corresponds to velocities.
[Figure] Clearly, this choice only makes sense for m ≤ n and in this work we assume n = 2 m , i.e., for each observation unit p i , we also represent a memory unit m i that stores its velocity information.
For each sequence, we initialize z +0 with an all zeros vector and Σ +0 with 10 · I .
In practice, it is beneﬁcial to normalize w t since the statistics of noisy and noise free images differ considerably.
2. T HE T RANSITION M ODEL
To obtain a locally linear transition model we learn K constant transition matrices
A ( k ) and combine them using state dependent coefﬁcients α ( k ) ( z t ) , i.e., A t = (cid:80)
{+Kk =
 α ( k ) ( z t ) A ( k ) .+}
A small neural network with softmax output is used to learn α ( k ) .
Similar approaches are used in (Fraccaro et al., 2017; Karl et al., 2016).
Using a dense transition matrix in high-dimensional latent spaces is not feasible as it contains too many parameters and causes numerical instabilities and overﬁtting, as preliminary experiments showed.
Therefore, we design each A ( k ) to consist of four band matrices A ( k ) =
[ B ( k ) 11 , B ( k ) 12 ; B ( k ) 21 , B ( k ) 22 ] with bandwidth b .
This choice reduces the number of parameters while not affecting performance since the network is free to choose the state representation.
We assume the covariance of the transition noise to be diagonal and denote the vector containing the diagonal values by σ trans .
The noise is learned and independent of the state.
Moreover, it is crucial to correctly initialize the transition matrix.
Initially, the transition model should focus on copying the encoder output so that the encoder can learn how to extract good features if observations are available and useful.
On the other hand it is crucial that A does not yield an instable system.
We choose B ( k ) 11 =
B ( k )
=
I and B ( k ) 12 = − B ( k ) 21 = 0 .
2 · I .
2. F ACTORIZED C OVARIANCE R EPRESENTATION
Since the RKN learns high-dimensional representations, we can not work with the full state covariance matrices Σ + t and Σ − t .
We can also not fully factorize the state covariances by diagonal matrices as this neglects the correlation between the memory and the observation parts.
As the memory part is excluded from the observation model H , the Kalman update step would not change the memory nodes nor their uncertainty if we would only use a diagonal covariance matrix Σ + t .
Hence, for each observation node p i , we compute the covariance with its corresponding memory node m i .
All the other covariances are neglected.
This might be a crude approximation for many systems, however, as our network is free to choose its own state representation it can ﬁnd a representation where such a factorization works well in practice.
Thus, we use matrices of the form Σ t =
[ Σ u t , Σ s t ; Σ s t , Σ l t ] , where each of Σ u t , Σ s t , Σ l t ∈ R m × m is a diagonal matrix.
Again, we denote the vectors containing the diagonal values by σ u t , σ l t and σ s t .
2. F ACTORIZED I NFERENCE IN THE L ATENT S PACE
Inference in the latent state space can now be implemented, similar to a standard KF, using a prediction and an observation update. Prediction Update.
Equivalently to the classical Kalman Filter, the current prior (cid:0) z − t , Σ − t (cid:1) is obtained from the last posterior (cid:0) z + t − 1 , Σ + t − 1 (cid:1) by [Equation]
and Σ
[Equation]
t
[Equation]
However, the special structure of the covariances enables us to signiﬁcantly simplify the equation for the covariance.
While being straight forward, the full derivations are rather lengthy, thus, we refer to the supplementary material where the equations are given in Eqs. 7,8 and 9. Observation Update.
Next, the prior is updated using the latent observation ( w t , σ obs t ) .
Similar to the state, we split the Kalman gain matrix Q t into an upper Q u t and a lower part Q l t .
Both Q u t and Q l t are squared matrices.
Due to the simple latent observation model H = (cid:2) I m
[Equation]
(cid:3) and the factorized covariances all off-diagonal entries of Q u t and Q l t are zero and we can work with vectors representing the diagonals, i.e., q u t and q l t .
Those are obtained by
[Equation] where (cid:11) denotes an elementwise vector division. With this the update equation for the mean simpliﬁes to [Equation]
where (cid:12) denotes the elementwise vector product. The update equations for the individual covariance parts are given by
[Equation] where 1 m denotes the m dimensional vector consisting of ones. Again, we refer to the supplementary material for a more detailed derivations.
Besides avoiding the matrix inversion in the Kalman gain computation, the factorization of the covariance matrices reduces the total amount of numbers to store per matrix from n 2 to 3 m .
Additionally, working with σ s makes it trivial to ensure that the symmetry and positive deﬁniteness of the state covariance are not affected by numerical issue. 2. L OSS AND T RAINING
We consider two different potential output distributions, Gaussian distributions for estimating low dimensional observations and Bernoulli distributions for predicting high dimensional observations/images.
Both distributions are computed by decoders that use the current latent state estimate
z t as well its uncertainty estimates σ [Equation] + t [Equation]
Inferring states.
Let s 1: T be the ground truth sequence with dimension D s , the Gaussian loglikelihood for a single sequence is then computed as
[Equation]
where dec µ ( · ) and dec Σ ( · ) denote the parts of the decoder that are responsible for decoding the latent mean and latent variance respectively.
Inferring images.
Let o 1: T be images with D o pixels.
The Bernoulli log-likelihood for a single sequence is then given by
[Equation]
where o ( i ) t is the i th pixel of the t th image.
The pixels are in this case represented by grayscale values in the range of [0; 1] .
The i th dimension of the decoder is denoted by dec o,i (cid:0) z
+ t (cid:1) , where we use a sigmoid transfer function as output units for the decoder.
Gradients are computed using (truncated) backpropagation through time (BPTT)
(Werbos, 1990) and clipped.
We optimize the objective using the Adam (Kingma and Ba, 2014) stochastic gradient descent optimizer with default parameters.
2. T HE R ECURRENT K ALMAN N ETWORK
The prediction and observation updates results in a new type of recurrent neural network, that we call
Recurrent Kalman Network , which allows working in high dimensional state spaces while keeping numerical stability, computational efﬁciency and (relatively) low memory consumption.
Similar to the input gate in LSTMs (Hochreiter and Schmidhuber, 1997) and GRUs (Cho et al., 2014)
the Kalman gain can be seen as a gate controlling how much the current observation inﬂuences the state.
However, this gating explicitly depends on the uncertainty estimates of the latent state and observation and is computed in a principled manner.
Using sparse transition models allows working in higher dimensional spaces with considerably less parameters than LSTMs or GRUs.
For a ﬁxed bandwidth b and number of basis matrices k , the number of parameters of the RKN scales linear in the state size while it scales quadratically for LSTMs and GRUs.
Moreover, the RKN provides a principled method to deal with absent inputs by just omitting the update step and setting the posterior to the prior.
E VALUATION AND E XPERIMENTS
A full listing of hyperparameters and data set speciﬁcations can be found in the supplementary material 1 .
We compare to LSTM and GRU baselines for which we replaced the RKN transition layer with generic LSTM and GRU layers.
Those were given the encoder output as inputs and have an internal state size of 2 n .
The internal state was split into two equally large parts, the ﬁrst part was used to compute the mean and the second to compute the variance.
We additionally executed most of the following experiments using the root mean squared error to illustrate that our approach is also competitive in prediction accuracy.
The RMSE results can be found in the appendix.
3. P ENDULUM
We evaluated the RKN on a simple simulated pendulum with images of size 24 × 24 pixels as observations.
Gaussian transition noise with standard deviation of σ = 0 .
1 was added to the angular velocity after each step.
In the ﬁrst experiment, we evaluated ﬁltering in the presence of high observation noise.
We compare against LSTMs and GRUs as these are the only methods that can also perform state estimation.
The amount of noise varies between no noise at all and the whole image consisting of pure noise.
Furthermore, the noise is correlated over time, i.e., the model may observe pure noise for several consecutive time steps.
Details about the noise sampling process can be found in the appendix.
We represent the joint angle θ t as a two dimensional vector s t = θ t
= (sin( θ t ) , cos( θ t ))
T to avoid discontinuities.
We compared different instances of our model to evaluate the effects of assuming sparse transition models and factorized state covariances.
The results are given in Table 2.
The results show that our assumptions do not affect the performance, while we need to learn less parameters and can use much more efﬁcient computations using the factorized representation.
Note that for the more complex experiments with a higher dimensional latent state space, we were not able to experiment with full covariance matrices due to lack of memory and massive computation times.
Moreover, the RKN outperforms LSTM and GRU in all settings.
In a second experiment we evaluated the image prediction performance and compare against existing variational inference approaches. We randomly removed half of the images from the sequences and tasked the models with imputing those missing frames, i.e., we train the models to predict images instead of the position.
We compared our approach to the Kalman Variational Autoencoder ﬁnal
[Table]
[Table]
, we evaluated both.
Our approach outperforms all models. Only the informed KVAE yields comparable, but still slightly worse results while E2C and SIN fail to capture the dynamics.
The uninformed KVAE fails at identifying the non valid images. (KVAE) (Fraccaro et al., 2017), Embed to Control (E2C) (Watter et al., 2015) and Structured Inference Networks (SIN) (Krishnan et al., 2017). The results can be found in Table 3. Again, our RKN outperforms all other models.
This is surprising as the variational inference models use much more complex inference methods and in some cases even more information such as in the KVAE smoothing case. Sample sequences can be found in the supplementary material.
All hyperparameters are the same as for the previous experiment.
3. M ULTIPLE P ENDULUMS
Dealing with uncertainty in a principled manner becomes even more important if the observation noise affects different parts of the observation in different ways.
In such a scenario the model has to infer which parts are currently observable and which parts are not.
To obtain a simple experiment with that property, we repeated the pendulum experiments with three colored pendulums in one image.
The image was split into four equally sized square parts and the noise was generated individually for each part such that some pendulums may be occluded while others are visible.
Exemplary images are shown in the supplementary material.
A comparison to LSTM and GRU baselines can be found in Figure 2 and an exemplary trajectory in Figure 3.
The RKN again clearly outperforms the competing methods.
We also computed the quality of the uncertainty prediction by showing the histograms of the normalized prediction errors.
While this histogram has clearly a Gaussian shape for the RKN, it looks like a less regular distribution for the LSTMs.
3. Q UAD L INK
We repeated the ﬁltering experiment on a system with much more complicated dynamics, a quad link pendulum on images of size 48 × 48 pixels.
Since the individual links of the quad link may occlude each other different amounts of noise are induced for each link.
Two versions of this experiment were evaluated.
One without additional noise and one were we added noise generated by the same process used in the pendulum experiments.
You can ﬁnd the results in Table 4.
Furthermore, we repeated the imputation experiment with the quad link. We compared only to the informed KVAE, since it was the only model producing competitive results for the pendulum. Our approach achieved − 44 . 470 ± 0 .
208 (informed) and − 44 .
584 ± 0 . 236 (uninformed). The KVAE Model
Log Likelihood
RKN m =11 .
51 ± 1 .b
= 3 , k =
LSTM m =7 . 5224 ± 1 . LSTM m =7 . 429 ± 1 . GRU m =7 . 541 ± 1 .
GRU m =5 . 602 ± 1 . [Figure] variance also increases.
achieved − 52 .
608 ± 0 .
602 for smoothing and − 59 .
0218 ± 0 .
580 for ﬁltering .
Sample images can be found in the appendix.
3. KITTI D ATASET FOR V ISUAL O DOMETRY
We evaluated the RKN on the KITTI Data set for visual odometry (Geiger et al., 2012).
Following (Zhao et al., 2018) we constructed an encoder consisting of FlowNet2 (Ilg et al., 2017), four convolutional and two branches of fully connected layers.
Following the standard KITTI evaluation procedure achieve an transnational error of 4 .
24% and a rotational error of 0 .
0404 rad /100m .
Those result are competitive to other recent results on the
KITTI dataset using only monocular images (Zhao et al., 2018; Wang et al., 2018) and demonstrate the applicability of our approach to real world applications. [Table]
C ONCLUSION
In this paper, we introduced the Recurrent Kalman Network that jointly learns high-dimensional representations of the system in a latent space with locally linear transition models and factorized covariances.
The update equations in the high-dimensional space are based on the update equations of the classical Kalman ﬁlter, however due to the factorization assumptions they simplify to scalar operations that can be performed much faster and with greater numerical stability.
Our model outperforms generic LSTMs and GRUs on various state estimation tasks while providing reasonable uncertainty estimates. Additionally, it outperformed several generative models on an image imputation task.
Training is straight forward and can be done in an end-to-end fashion.
In future work we want to exploit the principled notion of a variance provided by our approach in scenarios where such a notion is beneﬁcial, e g. reinforcement learning.
Similar to (Fraccaro et al., 2017) we could expand our approach to not just ﬁlter but smooth over trajectories in ofﬂine postprocessing scenarios which could potentially increase the estimation performance signiﬁcantly A PPENDIX
The Kalman ﬁlter (Kalman et al., 1960) works by iteratively applying two steps, predict and update. It assumes additive Gaussian noise with zero mean and covariances Σ trans and Σ obs on both transitions and observations, which need to be given to the ﬁlter.
During the prediction step the transition model A is used to infer the next prior state estimate (cid:0) x − t , Σ − t (cid:1) , i.e., a-priori to the observation, from the last posterior estimate (cid:0) x + t − 1 , Σ + t − 1 (cid:1) , by [Equation] The prior estimate is then updated using the current observation w t and the observation model H to obtain the posterior estimate (cid:0) x + t , Σ + t (cid:1) , i.e., x + t = x − t + Q t (cid:0) w t − Hx − t (cid:1) and Σ + t =
( I − Q t H ) Σ − t , with Q t = Σ − t H T (cid:16) HΣ − t H T + Σ obs (cid:17) − 1 , where I denotes the identity matrix. The matrix Q t is referred to as the Kalman gain. The whole update step can be interpreted as a weighted average between state and observation estimate, where the weighting, i.e., Q t , depends on the uncertainty about those estimates. If currently no observation is present or future states should be predicted, the update step is omitted.
A S IMPLIFIED K ALMAN F ILTER F ORMULAS
As stated above the simple latent observation model H = (cid:2)
I m [Equation] [Equation] , as well as the assumed factorization of the covariance matrices allow us to simplify the Kalman Filter equations.
A. N OTATION
In the following derivations we neglect the time indices t and t + 1 for brevity.
For any matrix M , ˆM denotes a diagonal matrix with the same diagonal as M , m denotes a vector containing those diagonal elements and M ( ij ) denotes the entry at row i and column j .
Similarly, v ( i ) denotes the i -th entry of a vector v .
The point wise product between two vectors of same length (Hadamat Product) will be denoted by (cid:12) and the point wise division by (cid:11) .
A. P REDICTION S TEP
Mean:
[Equation] Covariance:
[Equation]
The computation of the mean can not be further simpliﬁed, however, depending on the state size and bandwidth, sparse matrix multiplications may be exploited.
For the covariance, let T = AΣ + .
Then,
T =
[Equation]
+ =
[Equation]
B B 12 B B(cid:21) (cid:34) ˆΣ
[Equation]
+ ˆΣ s
[Equation]
+ ˆΣ
[Equation]
+ ˆΣ
[Equation]
+ (cid:35) = (cid:34) B 11 ˆΣ u , + + B 12 ˆΣ s , + B 11 ˆΣ s , + + B 12 ˆΣ l , + B 21 ˆΣ u , + + B 22 ˆΣ s , +
B 21 ˆΣ s , + + B 22 ˆΣ l , + (cid:35) and Σ − = TA T + ˆΣ trans = (cid:34) B 11 ˆΣ u , + + B 12 ˆΣ s , + B 11 ˆΣ s , + + B 12 ˆΣ l , + B 21 ˆΣ u , + + B 22 ˆΣ s , +
B 21 ˆΣ s , + + B 22 ˆΣ l , + (cid:35) (cid:20) B T B T B T B T(cid:21)
+ ˆΣ trans = (cid:34) ˆΣ u , − ˆΣ s , − ˆ Σ s , − ˆ Σ l , − (cid:35) , with
[Equation]
Since we are only interested in the diagonal parts of Σ u , − , Σ l , − and Σ s , − i.e. ˆΣ u , − , ˆΣ l , − and ˆΣ s , − , we can further simplify these equations by realizing two properties of the terms above.
First, for any matrix M , N and a diagonal matrix ˆΣ it holds that
[Equation]
Hence, we can simplify the equations for the upper and lower part to
[Equation]
Second, since we are only interested in the diagonal of the result it is sufﬁcient to compute only the diagonals of the individual parts of the sums which are almost all of the same structure i.e.
S = MˆΣ + N T .
Let T = MˆΣ + , then each element of T can be computed as
[Equation]
Consequently, the elements of S =
TA T can be computed as
[Equation]
Ultimately, we are not interested in S but only in ˆS
[Equation]
Using this we obtain can obtain the entries of σ u, − , σ l, − and σ s, − by
[Equation]
which can be implemented efﬁciently using elementwise matrix multiplication and sum reduction.
A. U PDATE S TEP Kalman Gain
[Equation]
Mean
[Equation]
Covariance
[Equation]
First, note that
[Equation]
and
[Equation]
and thus the computation of the Kalman Gain only involves diagonal matrices.
Hence the Kalman
Gain matrix also consists of two diagonal matrices, i [Equation]
e., Q = (cid:34) [Equation] whose diagonals can be computed by
[Equation]
Using this result [Equation] the mean update can be simpliﬁed to
[Equation]
For the covariance we get:
[Equation]
Hence the diagonals of the individual parts can be computed as
[Equation]
B R OOT M EAN S QUARE E RROR R ESULTS
To evaluate the actual prediction performance of our approach we repeated some experiments using the RMSE as loss function.
Other than that and removing the variance output of the decoder no changes were made to the model, hyperparameters and learning procedure.
The results can be found in Table 5.
C V ISUALIZATION OF I MPUTATION R ESULTS
Exemplary results of the data imputation experiments conducted for the Pendulum and Quad Link experiment can be found in Figure
D N ETWORK A RCHITECTURES AND H YPER P ARAMETERS
For all experiments Adam (Kingma and Ba, 2014) with default parameters ( α = 10 − 3 , β 1 = 0 .
9 , β 2 = 0 .
999 and ε = 10 − 8 ) was used as an optimizer.
The gradients were computed using (truncated)
Backpropagation Trough Time (BpTT) (Werbos, 1990).
Further in all (transposed) convolutional layers layer normalization (LN) (Ba et al., 2016) was employed to normalize the ﬁlter responses.
”Same” padding was used.
The elu activation function (Clevert et al., 2015) plus a constant 1 is denoted by ( elu + 1) was used to ensure that the variance outputs are positive. [Table]
D. P ENDULUM AND M ULTIPLE P ENDULUM E XPERIMENTS
Observations Pendulum : Grayscale images of size 24 × 24 pixels.
Multiple Pendulum : RGB images of size 24 × 24 pixels.
See Figure 5 for examples.
Dataset: 1000 Train and 500 Test sequences of length 150 .
For the ﬁltering experiments noise according to section E was added , for imputation 50% of the images were removed randomly.
Encoder: 2 convolution + 1 fully connected + linear output & ( elu + 1) output:
• Convolution 1: 12, 5 × 5 ﬁlter, ReLU, 2 × 2 max pool with 2 × 2 stride • Convolution 2: 12, 3 × 3 ﬁlter with 2 × 2 stride, ReLU, 2 × 2 max pool with 2 × 2 stride • Pendulum : Fully Connected 1: 30, ReLU • Multiple Pendulum : Fully Connected 1: 90, ReLU
Transition Model Pendulum : 15 dimensional latent observation, 30 dimensional latent state.
Multiple Pendulum : 45 dimensional latent observation, 90 dimensonal latent state.
Both: bandwidth: 3, number of basis:
• α ( z t ) : No hidden layers - softmax output Decoder (for s + t ): 1 fully connected + linear output: • Fully Connected 1: 10, ReLU Decoder (for o + t ): 1 fully connected + 2 transposed convolution + transposed convolution output: • Fully Connected 1: 144 ReLU • Transposed Convolution 1: 16, 5 × 5 ﬁlter with 4 × 4 stride, ReLU • Transposed Convolution 2: 12, 3 × 3 ﬁlter with 2 × 2 stride, ReLU • Transposed Convolution
Out: Pendulum : 1 Multiple Pendulum : 3, 3 × 3 ﬁlter with 1 ×stride, Sigmoid [Figure] Decoder (for σ + t or σ + t ): 1 fully connected + ( elu + 1) : • Fully Connected
: 10, ReLU
D.
[Table] INK
Observations : Grayscale images of size 48 x 48 pixels.
Dataset : 4000 Train and 1000 Test sequences of length 150.
For the ﬁltering with additional noise experiments noise according to section D was added, for imputation 50% of the images were removed randomly.
Encoder: 2 convolution + 1 fully connected + linear output & ( elu + 1) output: • Convolution 1: 12, 5 × 5 ﬁlter with 2 × 2 stride, ReLU, 2 × 2 max pool with 2 × 2 stride • Convolution 2: 12, 3 × 3 ﬁlter with 2 × 2 stride, ReLU, 2 × 2 max pool with 2 × 2 stride • Fully Connected 1: 200 ReLU
Transition Model 100 dimensional latent observation, 200 dimensional latent state, bandwidth: 3, number of basis:
• α ( z t ) : No hidden layers - softmax output Decoder (for s + t ): 1 fully connected + linear output: • Fully Connected 1: 10, ReLU Decoder (for o + t ): 1 fully connected + 2 transposed convolution + transposed convolution output: • Fully Connected 1: 144 ReLU • Transposed Convolution 1: 16, 5 × 5 ﬁlter with 4 × 4 stride, ReLU • Transposed Convolution 2: 12, 3 × 3 ﬁlter with 4 × 4 stride, ReLU • Transposed Convolution
Out: 1, 1 × 1 stride, Sigmoid Decoder (for σ + t or σ + t ): 1 fully connected + ( elu + 1) : • Fully Connected
: 10, ReLU
D.
K ITTI Observation and Data Set: Images from the KITTI (Geiger et al., 2012) dataset for visual odometry. Sequences 00, 01, 02, 08, 09 used for training. Sequences 03, 04, 05, 06, 07, 10 for testing.
Encooder : FlowNet2 + 4 Convolutional + 2 Branches of dense Layers.
See (Zhao et al., 2018)
Transition Model: 100 dimensional latent observations, 200 dimensional latent state, bandwidth: 5, number of basis:• α ( z t ) :
No hidden layers - softmax output Decoder (for s + t ): 2 fully connected + linear output: • Fully Connected 1: 128, ReLU • Fully Connected 1: {+
 ,+} ReLU Decoder (for σ + t or σ + t ): 2 fully connected + ( elu + 1) :[-• Fully Connected 
-]
• Fully Connected 1: 128, ReLU • Fully Connected [-
 :-]1: 128, ReLU
E O BSERVATION N OISE GENERATION PROCESS
Let U ( x, y ) denote the uniform distribution from x to y .To
generate the noise for the pendulum task for each sequence a sequence of factors f t of same length was generated.
To correlate the factors they were sampled as f 0 ∼ U (0 , 1) and f t +1 = min ( max (0 , f t + r t ) , 1) with r t ∼ U ( − 0 . 2 , 0 . 2) .
Afterwards, for each sequence two thresholds t 1 ∼ U (0 . 0 , 0 . 25) and t 2 ∼ U (0 .
75 , 1) were sampled.
All f t < t 1 were set to 0 , all f t > t 2 to 1 and the rest was linearly mapped to the interval [0 , 1] .
Finally, for each image i t an image consisting of pure uniformly distributed noise i noiset was sampled and the observation computed as o t = f t · i t + (1 − f t ) · i noiset .