## 4 Proposed Method  
### Proposed Architecture  
The overall architecture of the MST-VAE is illustrated in Figure 4. The proposed architecture comprises an encoder as a variational net or \(qnet\) and a decoder as a generative net or \(pnet\). In this paper, we assume that the multivariate time series are generated from Gaussian processes. Therefore, the prior distribution \(p_{\theta}(z)\) is chosen as a normal standard \(\mathcal{N}(0,\mathbf{I})\). The posterior distributions of the latent \(q_{\phi}(z|x)\) and the observed \(p_{\theta}(x|z)\) are multivariate Gaussians \(\mathcal{N}(\mu_{z},\sigma_{z}^{2}\mathbf{I})\) and \(\mathcal{N}(\mu_{x},\sigma_{z}^{2}\mathbf{I})\), respectively. The \(\mu_{z}\), \(\mu_{x}\) and \(\sigma_{z}^{2}\), \(\sigma_{x}^{2}\) are means and variances of independent Gaussian components.  
Since the model inputs are window samples mentioned in Section 3.1, notation \(x\) in the architecture is implicitly referred to as a window sample in the MTS. To obtain the complex temporal patterns, the multi-scale module, which is the main component of the MST-VAE, is first applied to the input. The main idea of the multi-scale module is that different kernel  
Figure 4: Overall architecture of MST-VAE. Circles are stochastic variables, black rectangles are layers, and red dashed rectangles are multi-scale modules.  
sizes are employed to extract various temporal patterns. The kernel size in the short-scale module is relatively small compared to it in the long-scale module. In the hidden layers, we stack multiple Conv1D layers to reduce the temporal dimension of the input, and a stride of 2 is used in each Conv1D for downsizing. Furthermore, the number of Conv1D layers in the hidden layers is chosen depending on the window size. For anomaly detection in time series, compressing the input to a lower-dimensional representation has two main reasons. First, the low dimensional latent variable can capture the main patterns with a small amount of information from the input. Second, as the training data is commonly supposed to be normal, using a low dimensional latent variable helps the model learn the normal patterns from the data. Then, in the inference phrase, the dimension reduction can throw away information of some abnormal points since the latent variable tries to capture normal patterns from the input, leading the latent \(z\) for an abnormal input \(x\), which might be indistinguishable from normal ones [1]. However, the latent variable may fail to capture normal patterns if there are too many anomalies in the input \(x\). After extracting multi-scale features, the outputs of the short and long-scale modules are concatenated along the channel axis. To ensure the number of channels of \(z\) is the same as in the input, a convolutional map reduction layer (kernel size = 1) [34] is used to reduce the channel dimensionality. Similar to [10], we choose to derive means \(\mu_{z}\) and log standard deviations \(\log\sigma_{z}\) from Conv1D layers. The log standard deviation is used instead of the standard deviation to ensure the numerical stability in computation. Moreover, we also clip the log standard deviations to avoid unbounded values when standard deviations are extremely close to zero.  
In order to reconstruct the input from the latent \(z\), the same process as in the _qnet_ is employed to the _pnet_ except for replacing Conv1D layers in the hidden and feature reduction layers by upsampling layers (Deconv1D).