SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Vaisakh Shaj $^{1*}$  Saleh Gholam Zadeh $^{1,2}$  Ozan Demir $^{3}$  Luiz Ricardo Douat $^{3}$  Gerhard Neumann $^{1}$
$^{1}$ Karlsruhe Institute Of Technology (KIT), Germany
$^{2}$ SAP SE, Germany
$^{3}$ Robert Bosch GmbH, Germany
Intelligent agents use internal world models to reason and make predictions about different courses of their actions at many scales [21]. Devising learning paradigms and architectures that allow machines to learn world models that operate at multiple levels of temporal abstractions while dealing with complex uncertainty predictions is a major technical hurdle [17]. In this work, we propose a probabilistic formalism to learn multi-time scale world models which we call the Multi Time Scale State Space (MTS3) model. Our model uses a computationally efficient inference scheme on multiple time scales for highly accurate long-horizon predictions and uncertainty estimates over several seconds into the future. Our experiments, which focus on action conditional long horizon future predictions, show that MTS3 outperforms recent methods on several system identification benchmarks including complex simulated and real-world dynamical systems. Code is available at this repository: https://github.com/ALRhub/MTS3.
World models attempt to learn a compact and expressive representation of the environment dynamics from observed data. These models can predict possible future world states as a function of an imagined action sequence and are a key ingredient of model-predictive control [3] and model-based reinforcement learning (RL). One important dimension of world models is the level of temporal granularity or the time scale at which the model operates. Existing literature on world models operates at a single level of temporal abstraction, typically at a fine-grained level such as milliseconds. One drawback of single-time scale world models is that they may not capture longer-term trends and patterns in the data [17].
For efficient long-horizon prediction and planning, the model needs to predict at multiple levels of temporal abstractions [27, 22]. Intuitively, low-level temporal abstractions should contain precise details about the input so as to predict accurately in the short term, while high-level, abstract representations should simplify accurate long-term predictions. Both abstractions must also interrelate with each other at least in the sense that the higher-level predictions/plans can be turned into low-level moment-by-moment predictions. For example, in robotic manipulation, the robot must be able to perform precise and coordinated movements to grasp and manipulate the object at a fast time scale while at a slower time scale, the robot must also be able to recognize and utilize higher-level patterns and structures in the task, such as the shape, size and location of objects, and the overall goal of the manipulation task.
Furthermore, temporal abstractions can capture relevant task structures across dynamical systems under non-stationary which can be used to identify the similarities and differences between tasks, allowing the robot to transfer knowledge learned from one task to another [26, 17].
*Corresponding author. Email to <vaisakshs.shaj@gmail.com>.
In this work, we attempt to come up with a principled probabilistic formalism for learning such multi-time scale world models as a hierarchical sequential latent variable model. We show that such models can better capture the complex, non-linear dynamics of a system more efficiently and robustly than models that learn on a single timescale. This is exemplified in several challenging simulated and real-world prediction tasks such as the D4RL dataset, a simulated mobile robot and real manipulators including data from heavy machinery excavators.
State space models (SSMs) are Bayesian probabilistic graphical models [15, 12] that are popular for learning patterns and predicting behaviour in sequential data and dynamical systems. Formally, we define a state space model as a tuple  $(\mathcal{Z},\mathcal{A},\mathcal{O},f,h,\Delta t)$ , where  $\mathcal{Z}$  is the state space,  $\mathcal{A}$  the action space and  $\mathcal{O}$  the observation space of the SSM. The parameter  $\Delta t$  denotes the discretization time-step and  $f$  and  $h$  the dynamics and observation models respectively. We will consider the Gaussian state space model that is represented using the following equations
$$
\boldsymbol {z} _ {t} = f (\boldsymbol {z} _ {t - 1}, \boldsymbol {a} _ {t - 1}) + \epsilon_ {t}, \quad \boldsymbol {\epsilon} _ {t} \sim \mathcal {N} (\boldsymbol {0}, \boldsymbol {\Sigma} _ {\mathbf {z}}), \quad \text {a n d} \quad \boldsymbol {o} _ {t} = h (\boldsymbol {z} _ {t}) + \boldsymbol {v} _ {t}, \quad \boldsymbol {v} _ {t} \sim \mathcal {N} (\boldsymbol {0}, \boldsymbol {\Sigma} _ {\mathbf {o}}).
$$
Here  $\mathbf{z}_t \in \mathcal{Z}$ ,  $\mathbf{a}_t \in \mathcal{A}$  and  $\mathbf{o}_t \in \mathcal{O}$  are the latent states, actions and observations at time  $t$ . The vectors  $\boldsymbol{\epsilon}_t$  and  $\boldsymbol{v}_t$  denote zero-mean Gaussian noise. When  $f$  and  $h$  are linear/locally linear, inference can be performed efficiently via exact inference. There have been several works recently [9, 1, 25] where these closed-form solutions are coded as layers of a neural network in deep-state space model literature, i.e., the architecture of the network is informed by the structure of the probabilistic state estimator. Following this line of work, we propose a multi-time scale linear Gaussian state space model, whose inference can be performed via closed-form solutions.
(Locally-)Linear Gaussian SSMs. To perform inference in SSMs, we follow [1]. They use a (locally-)linear dynamics model. Moreover, they replace the observations  $\pmb{o}$  with a latent observation  $\pmb{w}$ . This latent observation is obtained by an encoder  $\pmb{w}_{o_t} = \mathrm{enc}_w(\pmb{o}_t)$  along with the uncertainty of this observation, i.e.,  $\pmb{\sigma}_{o_t} = \mathrm{enc}_{\sigma}(\pmb{o}_t)$ . Due to the non-linear observation encoder, a simplified linear observation model can now be used. Hence, the dynamics and observation models can be described as
$$
p \left(\boldsymbol {z} _ {t + 1} \mid \boldsymbol {z} _ {t}, \boldsymbol {a} _ {t}\right) = \mathcal {N} \left(\boldsymbol {A} _ {t} \boldsymbol {z} _ {t} + \boldsymbol {c} _ {t} + \boldsymbol {B} _ {t} \boldsymbol {a} _ {t}, \operatorname {d i a g} \left(\boldsymbol {\sigma} _ {z}\right)\right), \text {a n d} p \left(\boldsymbol {w} _ {o _ {t}} \mid \boldsymbol {z} _ {t}\right) = \mathcal {N} \left(\boldsymbol {H} \boldsymbol {z} _ {t}, \operatorname {d i a g} \left(\boldsymbol {\sigma} _ {o _ {t}}\right)\right),
$$
where a simple observation matrix of  $\mathbf{H} = [\mathbf{I},\mathbf{0}]$  is used. The underlying assumption behind this observation model is that the latent state  $\boldsymbol{z}_t = [p_t^T, d_t^T]^T$  has twice the dimensionality of the latent observation  $\boldsymbol{w}_t$  and only the first half of the latent state, i.e.,  $\boldsymbol{p}_t$ , can be observed. The second half of the latent state, i.e.,  $\boldsymbol{d}_t$ , serves as derivative or velocity units that can be used by the model to estimate the change of the observable part of the latent state.
Factorized Inference in Linear Gaussian SSMs. Inference in the introduced linear Gaussian SSM is straightforward and can be performed using Kalman prediction and observation updates. However, these updates involve high dimensional matrix inversions that are expensive to evaluate and hard to backpropagate for end-to-end learning. Hence, [1] introduce a factorization of the belief  $p(\boldsymbol{z}_t | \boldsymbol{o}_{1:t}, \boldsymbol{a}_{1:t-1}) = \mathcal{N}(\boldsymbol{\mu}_t, \boldsymbol{\Sigma}_t)$  such that only the diagonal and one off-diagonal vector of the covariance need to be computed, i.e.
$$
\boldsymbol {\Sigma} _ {t} = \left[ \begin{array}{c c} \boldsymbol {\Sigma} _ {t} ^ {u} & \boldsymbol {\Sigma} _ {t} ^ {s} \\ \boldsymbol {\Sigma} _ {t} ^ {s} & \boldsymbol {\Sigma} _ {t} ^ {l} \end{array} \right], \text {w i t h} \boldsymbol {\Sigma} _ {u} = \mathrm {d i a g} (\boldsymbol {\sigma} _ {t} ^ {s}),   \boldsymbol {\Sigma} _ {l} = \mathrm {d i a g} (\boldsymbol {\sigma} _ {t} ^ {l}) \text {a n d} \boldsymbol {\Sigma} _ {s} = \mathrm {d i a g} (\boldsymbol {\sigma} _ {t} ^ {s}).
$$
Using this factorization assumption, closed-form Gaussian inference can be performed using only scalar divisions which are fast and easy to back-propagate. These factorization assumptions form the basis for the inference update in our MTS3 model.
Bayesian Aggregation. To aggregate information from several observations into a consistent representation, [29] introduce Bayesian aggregation in the context of Meta-Learning. They again use an encoder to obtain a latent observation vector  $\mathbf{r}_{o_t}$  and its uncertainty vector  $\sigma_{o_t}$ . Given the observation model  $p(\mathbf{r}_{o_t} | \mathbf{z}) = \mathcal{N}(\mathbf{H} \mathbf{z}, \mathrm{diag}(\sigma_{o_t}))$  with  $\mathbf{H} = \mathbf{I}$  and a prior  $p(\mathbf{z}) = \mathcal{N}(\mu_0, \mathrm{diag}(\sigma_0))$ , the posterior  $p(\mathbf{z} | \mathbf{r}_{o_1: o_t})$  can again be effectively computed by Gaussian inference that involves only
scalar inversions. Note that computing this posterior is a simplified case of the Kalman update rule used in Gaussian SSMs [1], with no memory units,  $\pmb{H} = \pmb{I}$  and no dynamics. To increase efficiency, the update rule can be formulated in a batch manner for parallel processing instead of an incremental update [29].
3 Multi Time Scale State Space Models
Our goal is to learn a principled sequential latent variable model that can model the dynamics of partially observable robotic systems under multiple levels of temporal abstractions. To do so, we introduce a new formalism, called Multi Time Scale State Space (MTS3) Model, with the following desiderata: i) It is capable of modelling dynamics at multiple time scales. ii) It allows for a single global model to be learned that can be shared across changing configurations of the environments. iii) It can give accurate long-term predictions and uncertainty estimates. iv) It is probabilistically principled yet scalable during learning and inference.
Figure 1: MTS3 captures slow-moving long-term trends as the latent task dynamics and the fast-moving short-term trends as the latent state dynamics.
3.1 General Definition
An MTS3 model with 2 timescales is defined by two SSMs on a fast and a slow time scale respectively. Both SSMs are coupled via the latent state of the slow time scale SSM, which parametrizes/reconfigures" the system dynamics of the fast time scale SSM. While the fast time scale SSM runs at the original time step  $\Delta t$  of the dynamical system, the slow time scale SSM is only updated every  $H$  step, i.e., the slow time scale time step is given by  $H\Delta t$ . We will derive closed-form Gaussian inference for obtaining the beliefs for both time scales, resulting in variations of the Kalman update rule which are also fully differentiable and used to back-propagate the error signal [1, 9]. The definition with a 2-level MTS3 along with the inference and learning schemes that we propose is directly extendable to an arbitrary number of temporal abstractions by introducing additional feudal [5] hierarchies with longer discretization steps and is further detailed in Section 3.4.
3.1.1 Fast time-scale SSM
The fast time-scale (fts) SSM is given by  $S_{\mathrm{fast}} = (\mathcal{Z}, \mathcal{A}, \mathcal{O}, f_l^{\mathrm{fts}}, h^{\mathrm{fts}}, \Delta t, \mathcal{L})$ . Here,  $l \in \mathcal{L}$  is a task descriptor that parametrizes the dynamics model of the SSM and is held constant for H steps. We will denote the task descriptor for the  $k$ th time window of  $H$  steps as  $l_k$ . The probabilistic dynamics and observation model of the fast time scale for the  $t$ th time step in the  $k$ th window can then be described as
$$
p \left(\boldsymbol {z} _ {k, t} \mid \boldsymbol {z} _ {k, t - 1}, \boldsymbol {a} _ {k, t - 1}, \boldsymbol {l} _ {k}\right) = \mathcal {N} \left(f _ {\boldsymbol {l}} ^ {\mathrm {f t s}} \left(\boldsymbol {z} _ {k, t - 1}, \boldsymbol {a} _ {k, t - 1}, \boldsymbol {l} _ {k}\right), \mathbf {Q}\right), \text {a n d}
$$
$$
p \left(\boldsymbol {o} _ {k, t} \mid \boldsymbol {z} _ {k, t}\right) = \mathcal {N} \left(h ^ {\text {f t s}} \left(\boldsymbol {z} _ {k, t}\right), \mathbf {R}\right). \tag {1}
$$
Task-conditioned marginal transition model. Moreover, we have to consider the uncertainty in the task descriptor (which will, in the end, be estimated by the slow time scale model), i.e., instead of considering a single task descriptor  $l_{k}$ , we have to consider a distribution over task-descriptors  $p(l_{k})$  for inference in the fts-SSM. This distribution will be provided by the slow-time scale SSM for every time window  $k$ . We can further define the marginal task-conditioned transition model for time window  $k$  which is given by
$$
p _ {\boldsymbol {l} _ {k}} \left(\boldsymbol {z} _ {k, t} | \boldsymbol {z} _ {k, t - 1}, \boldsymbol {a} _ {k, t - 1}\right) = \int p \left(\boldsymbol {z} _ {k, t} | \boldsymbol {z} _ {k, t - 1}, \boldsymbol {a} _ {k, t - 1}, \boldsymbol {l} _ {k}\right) p \left(\boldsymbol {l} _ {k}\right) d \boldsymbol {l} _ {k} \tag {2}
$$
Latent observations. Following [1], we replace the observations by latent observations and their uncertainty, i.e., we use latent observation encoders to obtain  $\pmb{w}_{k,t} = \mathrm{enc}_w(\pmb{o}_{k,t})$  and an uncertainty encoder  $\sigma_{k,t} = \mathrm{enc}_{\sigma}(\pmb{o}_{k,t})$ . The observation model is hence given by  $p(\pmb{w}_{k,t}|\pmb{z}_{k,t}) = \mathcal{N}(h^{\mathrm{fts}}(\pmb{z}_{k,t}),\mathrm{diag}(\sigma_{k,t}))$ .
3.1.2 Slow time-scale SSM
The slow time-scale (sts) SSM only updates every H time step and uses the task parameter  $l$  as latent state representation. Formally, the SSM is defined as  $S_{\mathrm{slow}} = (\mathcal{L},\mathcal{E},\mathcal{T},f^{\mathrm{sts}},h^{\mathrm{sts}},H\Delta t)$ . It uses an abstract observation  $\beta \in \mathcal{B}$  and abstract action  $\alpha \in \mathcal{A}$  that summarizes the observations and actions respectively throughout the current time window. The general dynamics model is hence given by
$$
p \left(\boldsymbol {l} _ {k} \mid \boldsymbol {l} _ {k - 1}, \boldsymbol {\alpha} _ {k}\right) = \mathcal {N} \left(f ^ {\text {s t s}} \left(\boldsymbol {l} _ {k - 1}, \boldsymbol {\alpha} _ {k}\right), \boldsymbol {S}\right). \tag {3}
$$
While there exist many ways to implement the abstraction of observations and actions of the time windows, we choose to use a consistent formulation by fusing the information from all  $H$  time steps of time window  $k$  using Gaussian conditioning.
Figure 2: The graphical model corresponding to an MTS3 with 2 timescales. The latent task variable  $l_{k}$  captures the slow-changing dynamics using abstract observation inferred from  $\{\beta_{k,t}\}_{t=1}^{H}$  and abstract action  $\alpha_{k}$  as described in section 3.1.2. The inference in the fast time scale uses primitive observations  $\boldsymbol{w}_{k,t}$ , primitive actions  $\boldsymbol{a}_{k,t}$  and the latent task descriptor  $l_{k}$  which parameterizes the fast-changing dynamics of  $\boldsymbol{z}_{k,t}$  for a time window  $k$  as discussed in the section 3.2.
Observation abstraction. In terms of the abstract observation model, we choose to model  $H$  observations  $\beta_{k,t}$ ,  $t\in [1,H]$  for a single slow-scale time step  $k$ . All these observations can then be straightforwardly integrated into the belief state representation using incremental observation updates. The abstract observation and its uncertainty for time step  $t$  is again obtained by an encoder architecture, i.e,
$$
\boldsymbol {\beta} _ {k, t} = \operatorname {e n c} _ {\beta} (\boldsymbol {o} _ {k, t}, t), \quad \boldsymbol {\nu} _ {k, t} = \operatorname {e n c} _ {\nu} (\boldsymbol {o} _ {k, t}, t),
$$
and  $p(\beta_{k,t}|\boldsymbol{l}_k) = \mathcal{N}(h^{\mathrm{sts}}(\boldsymbol{l}_k),\mathrm{diag}(\nu_{k,t}))$ . Hence, the abstract observation  $\beta_{k,t}$  contains the actual observation  $\boldsymbol{o}_{k,t}$  at time step  $t$  as well as a temporal encoding for the time-step. While multiple Bayesian observation updates are permutation invariant, the temporal encoding preserves the relative time information between the observations, similar to current transformer architectures.
Action abstraction. The abstract action  $\alpha_{k}$  causes the transitions to the latent task  $l_{k}$  from  $l_{k-1}$ . It should contain the relevant information of all primitive actions  $a_{k,t}$ ,  $t \in [1, H]$  executed in the time window  $k$ . To do so, we again use Bayesian conditioning and latent action encoding. Each control action  $a_{k,t}$  and the encoding of time-step  $t$  is encoded into its latent representation and its uncertainty estimate, i.e.,
$$
\boldsymbol {\alpha} _ {k, t} = \operatorname {e n c} _ {\alpha} (\boldsymbol {a} _ {k, t}, t), \quad \boldsymbol {\rho} _ {k, t} = \operatorname {e n c} _ {\rho} (\boldsymbol {a} _ {k, t}, t).
$$
The single latent actions  $\alpha_{k,t}$  can be aggregated into a consistent representation  $\alpha_{k}$  using Bayesian aggregation [29]. To do so, we use the likelihood  $p(\alpha_{k,t}|\alpha_k) = \mathcal{N}(\alpha_k,\mathrm{diag}(\rho_{k,t}))$  and obtain the posterior  $p(\alpha_k|\alpha_{k,1:H}) = \mathcal{N}(\boldsymbol{\mu}_{\alpha_k},\boldsymbol{\Sigma}_{\alpha_k})$ , which is obtained by following the standard Bayesian aggregation equations, see Appendix A. Note that our abstract action representation also contains an uncertainty estimate which can be used to express different effects of the actions on the uncertainty of the prediction. Due to the Gaussian representations, we can compute the marginal transition model
$$
p _ {\boldsymbol {\alpha} _ {k}} \left(\boldsymbol {l} _ {k} \mid \boldsymbol {l} _ {k - 1}, \boldsymbol {\alpha} _ {k, 1: H}\right) = \int p _ {\boldsymbol {\alpha} _ {k}} \left(\boldsymbol {l} _ {k} \mid \boldsymbol {l} _ {k - 1}, \boldsymbol {\alpha} _ {k}\right) p \left(\boldsymbol {\alpha} _ {k} \mid \boldsymbol {\alpha} _ {k, 1: H}\right) d \boldsymbol {\alpha} _ {k}. \tag {4}
$$
This transition model is used for inference and its parameters are learned.
3.1.3 Connecting both SSMs via inference
In the upcoming sections, we will devise Bayesian update rules to obtain the prior  $p(\pmb{l}_k|\pmb{\beta}_{1:k-1}, \pmb{\alpha}_{1:k})$  and posterior  $p(\pmb{l}_k|\pmb{\beta}_{1:k}, \pmb{\alpha}_{1:k})$  belief state for the sts-SSM as well as the belief states for the fts-SSM. The prior belief  $p(\pmb{l}_k|\pmb{\beta}_{1:k-1}, \pmb{\alpha}_{1:k})$  contains all information up to time window  $k-1$  and serves as a distribution over the task-descriptor of the fts-SSM, which connects both SSMs. This connection allows us to learn both SSMs jointly in an end-to-end manner.
The probabilistic graphical model of our MTS3 model is depicted in Figure 2. In the next section, we will present the detailed realization of each SSM to perform closed-form Gaussian inference and end-to-end learning on both time scales.
3.2 Inference in the Fast Time-Scale SSM
The fts-SSM performs inference for a given time window  $k$  of horizon length  $H$ . To keep the notation uncluttered, we will also omit the time-window index  $k$  whenever the context is clear. We use a linear Gaussian task conditional transition model, i.e,
$$
p \left(\boldsymbol {z} _ {t} \mid \boldsymbol {z} _ {t - 1}, \boldsymbol {a} _ {t - 1}, \boldsymbol {l} _ {k}\right) = \mathcal {N} \left(\mathbf {A} \boldsymbol {z} _ {t - 1} + \mathbf {B} \boldsymbol {a} _ {t - 1} + \mathbf {C l} _ {k}, \mathbf {Q}\right), \tag {5}
$$
where  $A, B, C$  and  $Q$  are state-independent but learnable parameters. In our formulation, the task descriptor can only linearly modify the dynamics which was sufficient to obtain state-of-the-art performance in our experiments, but more complex parametrizations, such as locally linear models, would also be feasible. Following [1], we split the latent state  $\boldsymbol{z}_t = [\boldsymbol{p}_t, \boldsymbol{m}_t]^T$  into its observable part  $\boldsymbol{p}_t$  and a part  $\boldsymbol{m}_t$  that needs to be observed over time. We also use a linear observation model  $p(\boldsymbol{w}_t | \boldsymbol{z}_t) = \mathcal{N}(\boldsymbol{H} \boldsymbol{z}_t, \mathrm{diag}(\sigma_t))$  with  $\boldsymbol{H} = [\boldsymbol{I}, \boldsymbol{0}]$ .
We will assume that the distribution over the task descriptor is also given by a Gaussian distribution, i.e.,  $p(\boldsymbol{l}_k) = \mathcal{N}(\boldsymbol{\mu}_{\boldsymbol{l}_k}, \boldsymbol{\Sigma}_{\boldsymbol{l}_k})$ , which will be provided by the slow-time scale (sts) SSM, see Section 3.3. Given these modelling assumptions, the task variable can now be integrated out in closed form, resulting in the following task-conditioned marginal transition model
$$
p _ {\boldsymbol {l} _ {k}} \left(\boldsymbol {z} _ {t} \mid \boldsymbol {z} _ {t - 1}, \boldsymbol {a} _ {t - 1}\right) = \mathcal {N} \left(\mathbf {A} \boldsymbol {z} _ {t - 1} + \mathbf {B} \boldsymbol {a} _ {t - 1} + \mathbf {C} \boldsymbol {\mu} _ {\boldsymbol {l} _ {k}}, \mathbf {Q} + C \boldsymbol {\Sigma} _ {\boldsymbol {l} _ {k}} \boldsymbol {C} ^ {T}\right), \tag {6}
$$
which will be used instead of the standard dynamics equations. We follow the same factorization assumptions as in [1] and only estimate the diagonal elements of the block matrices of the covariance matrix of the belief, see Appendix B. The update equations for the Kalman prediction and observation updates are therefore equivalent to the RKN [1].
3.3 Inference in the Slow-Time Scale SSM
Prediction Update. We follow the same Gaussian inference scheme as for the fts-SSM, i.e., we again employ a linear dynamics model  $p(\boldsymbol{l}_k|\boldsymbol{l}_{k - 1},\boldsymbol{\alpha}_k) = \mathcal{N}(\boldsymbol{X}\boldsymbol{l}_{k - 1} + \boldsymbol{Y}\boldsymbol{\alpha}_k,\boldsymbol{S})$ , where  $\boldsymbol{X},\boldsymbol{Y}$  and  $\boldsymbol{S}$  are learnable parameters. The marginalized transition model for the abstract actions is then given by
$$
p _ {\boldsymbol {\alpha} _ {k}} \left(\boldsymbol {l} _ {k} \mid \boldsymbol {l} _ {k - 1}\right) = \int p \left(\boldsymbol {l} _ {k} \mid \boldsymbol {l} _ {k - 1}, \boldsymbol {\alpha} _ {k}\right) p \left(\boldsymbol {\alpha} _ {k}\right) d \boldsymbol {\alpha} _ {k} = \mathcal {N} \left(\mathbf {X} \boldsymbol {l} _ {k - 1} + \mathbf {Y} \boldsymbol {\mu} _ {\alpha_ {k}}, \mathbf {S} + \mathbf {Y} \boldsymbol {\Sigma} _ {\boldsymbol {\alpha} _ {k}} \mathbf {Y} ^ {T}\right). \tag {7}
$$
We can directly use this transition model to obtain the Kalman prediction update which computes the prior belief  $p_{\alpha_{1:k}}(l_k|\boldsymbol{\beta}_{1:k-1}) = \mathcal{N}(\boldsymbol{\mu}_{l_k}^{-}, \boldsymbol{\Sigma}_{l_k}^{-})$  from the posterior belief  $p_{\alpha_{1:k-1}}(l_{k-1}|\boldsymbol{\beta}_{1:k-1}) = \mathcal{N}(\boldsymbol{\mu}_{l_{k-1}}^{+}, \boldsymbol{\Sigma}_{l_{k-1}}^{+})$  of the previous time window, see Appendix A.
Observation Update. Similarly, we will use a linear observation model for the abstract observations  $p(\beta_{k,t}|l_k) = \mathcal{N}(\mathbf{H}l_k,\mathrm{diag}(\nu_{k,t}))$  with  $\mathbf{H} = [\mathbf{I},\mathbf{0}]$ . As can be seen from the definition of the observation matrix  $\mathbf{H}$ , the latent space is also decomposed into its observable and unobservable part, i.e.,  $l_{k} = [u_{k},v_{k}]$ . In difference to the standard factorized Kalman observation update given in Appendix A, we have to infer with a set of observations  $\vec{\beta}_{k,t}$  with  $t = 1\dots H$  for a single time window  $k$ . While in principle, the Kalman observation update can be applied incrementally  $H$  times to obtain the posterior  $p_{\alpha_{1:k}}(l_k|\beta_{1:k}) = \mathcal{N}(\pmb {\mu}_{l_k}^+, \pmb {\Sigma}_{l_k}^+)$ , such an update would be very slow and also