  Spatiotemporal imaging is common in medical imaging, with applications in
e.g. cardiac diagnostics, surgical guidance and radiotherapy monitoring. In
this paper, we present an unsupervised model that identifies the underlying
dynamics of the system, only based on the sequential images. The model maps the
input to a low-dimensional latent space wherein a linear relationship holds
between a hidden state process and the observed latent process. Knowledge of
the system dynamics enables denoising, imputation of missing values and
extrapolation of future image frames. We use a Variational Auto-Encoder (VAE)
for the dimensionality reduction and a Linear Gaussian State Space Model
(LGSSM) for the latent dynamics. The model, known as a Kalman Variational
Auto-Encoder, is end-to-end trainable and the weights, both in the VAE and
LGSSM, are simultaneously updated by maximizing the evidence lower bound of the
marginal log likelihood. Our experiment, on cardiac ultrasound time series,
shows that the dynamical model provide better reconstructions than a similar
model without dynamics. And also possibility to impute and extrapolate for
missing samples.
