  Deep generative models have demonstrated their effectiveness in learning
latent representation and modeling complex dependencies of time series. In this
paper, we present a Smoothness-Inducing Sequential Variational Auto-Encoder
(SISVAE) model for robust estimation and anomaly detection of multi-dimensional
time series. Our model is based on Variational Auto-Encoder (VAE), and its
backbone is fulfilled by a Recurrent Neural Network to capture latent temporal
structures of time series for both generative model and inference model.
Specifically, our model parameterizes mean and variance for each time-stamp
with flexible neural networks, resulting in a non-stationary model that can
work without the assumption of constant noise as commonly made by existing
Markov models. However, such a flexibility may cause the model fragile to
anomalies. To achieve robust density estimation which can also benefit
detection tasks, we propose a smoothness-inducing prior over possible
estimations. The proposed prior works as a regularizer that places penalty at
non-smooth reconstructions. Our model is learned efficiently with a novel
stochastic gradient variational Bayes estimator. In particular, we study two
decision criteria for anomaly detection: reconstruction probability and
reconstruction error. We show the effectiveness of our model on both synthetic
datasets and public real-world benchmarks.
