SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Drift doesn’t Matter: Dynamic Decomposition with Diffusion Reconstruction for Unstable Multivariate Time Series Anomaly Detection
Chengsen Wang* Zirui Zhuang* Qi Qi† Jingyu Wang†  
Xingyu Wang Haifeng Sun Jianxin Liao
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China {cswang, zhuangzirui, qiqi8266, wangjingyu}@bupt.edu.cn {wangxingyu, hfsun, liaoqx}@bupt.edu.cn
Many unsupervised methods have recently been proposed for multivariate time series anomaly detection. However, existing works mainly focus on stable data yet often omit the drift generated from non-stationary environments, which may lead to numerous false alarms. We propose Dynamic Decomposition with Diffusion Reconstruction  $(\mathrm{D}^3\mathrm{R})$ , a novel anomaly detection network for real-world unstable data to fill the gap.  $\mathrm{D}^3\mathrm{R}$  tackles the drift via decomposition and reconstruction. In the decomposition procedure, we utilize data-time mix-attention to dynamically decompose long-period multivariate time series, overcoming the limitation of the local sliding window. The information bottleneck is critical yet difficult to determine in the reconstruction procedure. To avoid retraining once the bottleneck changes, we control it externally by noise diffusion and directly reconstruct the polluted data. The whole model can be trained end-to-end. Extensive experiments on various real-world datasets demonstrate that  $\mathrm{D}^3\mathrm{R}$  significantly outperforms existing methods, with a  $11\%$  average relative improvement over the previous SOTA models. Code is available at https://github.com/FroststKing/D3R.
Due to the rarity of anomalies, unsupervised anomaly detection of multivariate time series is an essential area in data mining and industrial applications. Reconstruction-based models are commonly used in unsupervised anomaly detection. The reconstruction error is small for normal series while large for abnormal series. Based on this principle, most anomalies can be detected without the label. In the real world, the temporal patterns typically change over time as they are generated from non-stationary environments. For example, the growth in the popularity of a service would cause customer metrics (e.g., request count) to drift upwards over time. Ignoring these factors would cause a deterioration in the performance of the anomaly detector. Despite the enormous advancements of previous research, most focus on stable data. Figure 1 illustrates how the anomaly scores provided by existing approaches tend to increase in the red area when the data is unstable, leading to false alarms.
To handle the unstable data, we attempt to decompose it into a stable component and a trend component, focusing more on the stable part in the reconstruction procedure. However, the trend part cannot be ignored completely, as the drift may be anomalous. There are still two challenges in this
Figure 1: Anomaly scores of existing methods on SMD and SWaT datasets. All moments are normal. Vertical drift occurs in the red area. The moment with a higher anomaly score is more likely to be identified as an anomaly.
context: Challenge 1: Limitation of the decomposition for long-period time series. Most classical decomposition algorithms [2, 17, 27] are static, making it hard to be applied in the real world, where the data is updated in real time. With deep learning, some dynamic algorithms have been proposed; however, they are mainly based on averaging [29] or Fourier Transform [28, 26] within a local sliding window. Fundamentally, they can not apply to the data whose period is larger than the size of the sliding window. Challenge 2: High training cost of adjusting the information bottleneck in the reconstruction procedure. The information bottleneck is critical for reconstruction-based models, and adjusting it is difficult. If it is too small, normal data will be poorly reconstructed; if it is too large, abnormal data will be successfully reconstructed. Previous methods most rely on an internal information bottleneck, such as the latent space size. As the information bottleneck is an attribute of the model itself, any change to the bottleneck necessitates retraining the model.
After analyzing numerous data series, we systematically organize the distribution shift into two categories. Vertical drift refers to statistical characteristics such as mean changing over time, while horizontal drift refers to values shifting left or right at similar times in various periods. In response to the above challenges, we propose Dynamic Decomposition with Diffusion Reconstruction  $(\mathrm{D}^3\mathrm{R})$  for long-period unstable multivariate time series anomaly detection. To solve Challenge 1, we utilize timestamps as external information to overcome the limitations of the local sliding window. Specifically, the data-time mix-attention and offset subtraction are utilized to solve vertical and horizontal drift, respectively. Moreover, we introduced a disturbance strategy in training to increase the robustness of the model. To solve Challenge 2, we propose a new method named noise diffusion to control the information bottleneck externally. The diffusion [6] provides a new view of the information bottleneck, treating the noise as a bottleneck and the unpolluted information as a condition. Because the bottleneck is no longer an attribute of the model itself, the different sizes can be set during revision without retraining the model.
The contribution of our paper is summarised as follows:
- A novel dynamic decomposition method for long-period multivariate time series is proposed. It effectively utilizes external information to overcome the limitations of the local sliding window.
- A new approach to control information bottleneck externally by noise diffusion is also proposed. It avoids the high training cost of adjusting the information bottleneck.
- Based on the findings for the unstable data when taking unsupervised anomaly detection, we propose a novel anomaly detection network called  $\mathrm{D}^3\mathrm{R}$ . The  $\mathrm{D}^3\mathrm{R}$  achieves the new SOTA results on various real-world datasets and significantly outperforms baselines on unstable datasets.
As a significant real-world problem, unsupervised anomaly detection of multivariate time series has received much attention. According to the criterion for anomaly detection, the paradigms roughly fall
Figure 2: The architecture of  $\mathrm{D}^3\mathrm{R}$  mainly consists of two modules: dynamic decomposition and diffusion reconstruction. The detailed architecture of the decomposition blocks is shown on the right panel. The detailed architecture of the reconstruction backbone network is shown on the left panel.
into the categories of probability-based, linear transformation-based, proximity-based, outlier-based, and neural network-based approaches.
As for the probability-based approach, [12, 13] are modeled by a statistical probability function on a multivariate cumulative distribution, providing anomaly scores based on the probability of occurrence of the sample. In the linear transformation-based method, [20, 22] begin by mapping the multivariate data before determining the boundary from the mapping space. Proximity-based algorithms, such as [18, 5], seek to cluster data based on similarity and then calculate intra-cluster and inter-cluster distances. In the outlier-based approach, [14, 16] compare the outliers degree of the testing samples with the training samples to identify the anomaly.
Most of the approaches mentioned above do not consider the temporal continuity of series. In recent years, neural network-based methods have become increasingly significant. Current neural network-based approaches can be divided into prediction-based and reconstruction-based. Although prediction-based methods [21, 8] are effective in modeling for the next timestamp, they are susceptible to interference from historical information. Reconstruction-based approaches [32, 11] do a great job capturing the distribution across the whole series. However, they are sensitive to the size of the information bottleneck.
Furthermore, existing works mainly focus on stable data, and their performance may suffer significantly in the real world, where the drift is frequent.
An input of multivariate time series anomaly detection is denoted by  $\mathbf{X} \in \mathbb{R}^{n \times k}$ , where  $n$  is the length of timestamps, and  $k$  is the number of variables. The task is to produce an output vector  $\mathbf{y} \in \mathbb{R}^n$ , where  $y_i \in \{0,1\}$  denotes whether the  $i^{th}$  timestamp is an anomaly.
The overall architecture of  $\mathrm{D}^3\mathrm{R}$  is shown in Figure 2. The dynamic decomposition module first models the data and timestamp features by the data encoder and time encoder. Next, it uses stacked decomposition blocks to extract the stable component. Finally, we get the trend component by offset subtraction. The diffusion reconstruction module utilizes noise diffusion to construct information bottleneck externally and then directly reconstructs the polluted data by the backbone network. Reconstruction error is the anomaly score. In order to model both temporal dependency and dimension dependency, the data encoder and the reconstruction backbone network both consist of stacked spatial-
temporal transformer blocks. To increase the robustness of the model, we also propose a disturbance strategy during training.
3.2 Data preprocessing
We perform timestamp hard embedding, labeled stable component construction, and disturbance strategy for the input. Timestamp hard embedding is applied to the training and testing sets, while labeled stable component construction and disturbance strategy are applied to the training set only.
Timestamp hard embedding To make better use of timestamps in anomaly detection, we hardcode the timestamps of  $\mathbb{R}^{n\times 1}$  into an embedding  $\mathbf{X}_{\mathrm{time}}\in \mathbb{R}^{n\times 5}$  like [34, 29], with each dimension representing minute of the hour, hour of the day, day of the week, day of the month, and month of the year.
Labeled stable component construction We extract the trend  $\mathbf{T} \in \mathbb{R}^{n \times k}$  by moving average, then the labeled stable component  $\mathbf{S} = \mathbf{X} - \mathbf{T}, \mathbf{S} \in \mathbb{R}^{n \times k}$  is obtained. The construction of the labeled stable components prevents our model from being disturbed when the training data is unstable.
Disturbance strategy To increase the robustness of the model, we add a vertical drift  $\mathbf{d} \in \mathbb{R}^k$  sampled from a  $[-p, p]$  uniform distribution to each variable of the training data. The final input of dynamic decomposition module is  $\mathbf{X}_{\mathrm{d}} = \mathbf{X} + \mathbf{d}$ ,  $\mathbf{X}_{\mathrm{d}} \in \mathbb{R}^{n \times k}$ .
3.3 Dynamic decomposition
The dynamic decomposition module consists of a data encoder, a time encoder, stacked decomposition blocks, and an offset subtraction. The data encoder is implemented based on the spatial-temporal transformer block, which captures the temporal and dimension dependency. The output of data encoder is  $\mathbf{H}_{\mathrm{data}} \in \mathbb{R}^{n \times d_{\mathrm{model}}}$ , where  $d_{\mathrm{model}}$  is hidden state dimension in the model. The time encoder consists only of the temporal transformer block, which models the temporal correlation of timestamps to obtain  $\mathbf{H}_{\mathrm{time}} \in \mathbb{R}^{n \times d_{\mathrm{model}}}$ . Data-time mix-attention constitutes the subject of stacked decomposition blocks, which are used to extract the stable component  $\hat{\mathbf{S}} \in \mathbb{R}^{n \times k}$ . Finally, to solve the challenges of horizontal drift, we obtain the trend component  $\hat{\mathbf{T}}_{\mathrm{d}} \in \mathbb{R}^{n \times k}$  through offset subtraction.
Spatial-temporal transformer block The architecture of the spatial-temporal transformer block is shown in the solid line box on the left panel of Figure 2. Assuming the input of  $l^{th}$  layer is  $\mathbf{H}^l \in \mathbb{R}^{n \times d_{\mathrm{model}}}$  with  $n$  timestamps and  $d_{\mathrm{model}}$  dimensions. In the temporal transformer, we obtain temporal relationship  $\mathbf{H}_{\mathrm{temporal}}^l \in \mathbb{R}^{n \times d_{\mathrm{model}}}$  by directly applying multi-head self-attention (MSA) to  $n$  vectors of size  $d_{\mathrm{model}}$ :
$$
\hat {\mathbf {H}} _ {\mathrm {t e m p o r a l}} ^ {l} = \mathrm {L a y e r N o r m} \left(\mathbf {H} ^ {l} + \mathrm {M S A} \left(\mathbf {H} ^ {l}, \mathbf {H} ^ {l}, \mathbf {H} ^ {l}\right)\right)
$$
$$
\mathbf {H} _ {\text {t e m p o r a l}} ^ {l} = \text {L a y e r N o r m} \left(\hat {\mathbf {H}} _ {\text {t e m p o r a l}} ^ {l} + \text {F e e d F o r w a r d} \left(\hat {\mathbf {H}} _ {\text {t e m p o r a l}} ^ {l}\right)\right) \tag {1}
$$
where  $\hat{\mathbf{H}}_{\mathrm{temporal}}^{l}$  is intermediate variable. LayerNorm( $\cdot$ ) denotes layer normalization as widely adopted in [25, 34, 29], FeedForward( $\cdot$ ) denotes a multi-layer feedforward network, MSA(Q, K, V) denotes the multi-head self-attention [25] layer where Q, K, V serve as queries, keys and values. In the spatial transformer, we obtain dimension relationship  $\mathbf{H}_{\mathrm{spatial}}^{l} \in \mathbb{R}^{n \times d_{\mathrm{model}}}$  by applying MSA to  $d_{\mathrm{model}}$  vectors of size  $n$ :
$$
\hat {\mathbf {H}} _ {\text {s p a t i a l}} ^ {l} = \operatorname {L a y e r N o r m} \left(\left(\mathbf {H} ^ {l}\right) ^ {\mathrm {T}} + \operatorname {M S A} \left(\left(\mathbf {H} ^ {l}\right) ^ {\mathrm {T}}, \left(\mathbf {H} ^ {l}\right) ^ {\mathrm {T}}, \left(\mathbf {H} ^ {l}\right) ^ {\mathrm {T}}\right)\right)
$$
$$
\mathbf {H} _ {\text {s p a t i a l}} ^ {l} = \operatorname {L a y e r N o r m} \left(\left(\hat {\mathbf {H}} _ {\text {s p a t i a l}} ^ {l}\right) ^ {\mathrm {T}} + \operatorname {F e e d F o r w a r d} \left(\left(\hat {\mathbf {H}} _ {\text {s p a t i a l}} ^ {l}\right) ^ {\mathrm {T}}\right)\right) \tag {2}
$$
where  $\hat{\mathbf{H}}_{\mathrm{spatial}}^{l}$  is intermediate variable.  $(\cdot)^{\mathrm{T}}$  denotes the transposition of matrices. Finally, we obtain the input of  $l + 1^{th}$  layer  $\mathbf{H}^{l + 1}\in \mathbb{R}^{n\times d_{\mathrm{model}}}$  by:
$$
\hat {\mathbf {H}} ^ {l + 1} = \text {L a y e r N o r m} \left(\mathbf {H} _ {\text {t e m p o r a l}} ^ {l} \oplus \mathbf {H} _ {\text {s p a t i a l}} ^ {l}\right)
$$
$$
\mathbf {H} ^ {l + 1} = \text {L a y e r N o r m} \left(\operatorname {F e e d F o r w a r d} \left(\hat {\mathbf {H}} ^ {l + 1}\right)\right) \tag {3}
$$
where  $\hat{\mathbf{H}}^{l + 1}\in \mathbb{R}^{n\times 2d_{\mathrm{model}}}$  is intermediate variable, and  $\oplus$  represents concatenation.
Data-time mix-attention The original self-attention  $\mathcal{A}_{\mathrm{o}}(\mathbf{Q},\mathbf{K},\mathbf{V})$  [25] only models the data information and ignores the role of timestamps. We define the data-time mix-attention as:
$$
\mathcal {A} _ {\mathrm {m}} \left(\mathbf {Q} _ {\mathrm {d}}, \mathbf {K} _ {\mathrm {d}}, \mathbf {Q} _ {\mathrm {t}}, \mathbf {K} _ {\mathrm {t}}, \mathbf {V} _ {\mathrm {t}}\right) = \operatorname {S o f t m a x} \left(\frac {\mathbf {Q} _ {\mathrm {d}} \mathbf {K} _ {\mathrm {d}} ^ {\mathrm {T}} + \mathbf {Q} _ {\mathrm {t}} \mathbf {K} _ {\mathrm {t}} ^ {\mathrm {T}}}{\sqrt {d _ {\mathrm {k}}}}\right) \mathbf {V} _ {\mathrm {t}} \tag {4}
$$
where  $\mathrm{Softmax}(\cdot)$  is conducted row by row like [25, 34, 29],  $\mathbf{Q}_{\mathrm{d}}, \mathbf{K}_{\mathrm{d}} \in \mathbb{R}^{n \times d_{\mathrm{k}}}$  are length-n data queries, data keys of  $d_{\mathrm{k}}$  dimension mapped from  $\mathbf{H}_{\mathrm{data}}$ , respectively.  $\mathbf{Q}_{\mathrm{t}}, \mathbf{K}_{\mathrm{t}}, \mathbf{V}_{\mathrm{t}} \in \mathbb{R}^{n \times d_{\mathrm{k}}}$  are time queries, time keys and time values of  $d_{\mathrm{k}}$  dimension mapped from  $\mathbf{H}_{\mathrm{time}}$ , respectively.  $\mathrm{D}^{3}\mathrm{R}$  learns to autonomously merge data and time information by mapping them into attention space.
Decomposition block The structure of stacked decomposition blocks is shown on the right side of Figure 2. Assuming the input of  $l^{th}$  layer is  $\mathbf{H}_{\mathrm{data}}^l$  and  $\mathbf{H}_{\mathrm{time}}^l$ , we can obtain the sub-stable components  $\hat{\mathbf{S}}^l \in \mathbb{R}^{n \times k}$  by:
$$
\widetilde {\mathbf {H}} ^ {l} = \mathcal {A} _ {\mathrm {m}} \left(\mathbf {H} _ {\text {d a t a}} ^ {l}, \mathbf {H} _ {\text {d a t a}} ^ {l}, \mathbf {H} _ {\text {t i m e}} ^ {l}, \mathbf {H} _ {\text {t i m e}} ^ {l}, \mathbf {H} _ {\text {t i m e}} ^ {l}\right)
$$
$$
\begin{array}{l} \hat {\mathbf {H}} ^ {l} = \text {L a y e r N o r m} \left(\widetilde {\mathbf {H}} ^ {l} + \mathcal {A} _ {0} \left(\widetilde {\mathbf {H}} ^ {l}, \widetilde {\mathbf {H}} ^ {l}, \widetilde {\mathbf {H}} ^ {l}\right)\right) \\ \widetilde {\mathbf {S}} ^ {l} = \text {L a y e r N o r m} \left(\hat {\mathbf {H}} ^ {l} + \text {F e e d F o r w a r d} \left(\hat {\mathbf {H}} ^ {l}\right)\right) \end{array} \tag {5}
$$
$$
\hat {\mathbf {S}} ^ {l} = \operatorname {O u t} \left(\widetilde {\mathbf {S}} ^ {l}\right)
$$
where  $\widetilde{\mathbf{H}}^l, \hat{\mathbf{H}}^l, \widetilde{\mathbf{S}}^l \in \mathbb{R}^{n \times d_{\mathrm{model}}}$  are intermediate variables.  $\operatorname{Out}(\cdot)$  is a single linear layer. Then we can obtain the input of  $l + 1^{th}$  layer  $\mathbf{H}_{\mathrm{data}}^{l + 1} = \mathbf{H}_{\mathrm{data}}^l - \widetilde{\mathbf{S}}^l$  and  $\mathbf{H}_{\mathrm{time}}^{l + 1} = \mathbf{H}_{\mathrm{time}}^l$ . Finally, the sub-stable components  $\hat{\mathbf{S}}^l$  of all stacked decomposition blocks are summed to obtain the stable component  $\hat{\mathbf{S}}$  of the series.
Offset subtraction Although the same time slots in different periods usually have similar fluctuations, it is not strictly one-to-one correspondence. Here, we employ a simple method to handle the horizontal drift, considering the vector  $\mathbf{a}$  minus the vector  $\mathbf{b}$ , with a maximum horizontal offset  $d$ :
$$
\mathcal {M} \left(\mathbf {a} _ {i}, \mathbf {b}\right) = \operatorname {M i n} \left(\mathbf {a} _ {i} - \mathbf {b} _ {i - d}, \dots , \mathbf {a} _ {i} - \mathbf {b} _ {i + d}\right) \tag {6}
$$
where  $\mathbf{a}_i$  and  $\mathbf{b}_i$  are the  $i^{th}$  element in  $\mathbf{a}$  and  $\mathbf{b}$ , respectively.  $\mathrm{Min}(\cdot)$  means taking the minimum value. Based on the offset subtraction, we obtain the predicted trend component as  $\hat{\mathbf{T}}_{\mathrm{d}} = \mathcal{M}(\mathbf{X}_{\mathrm{d}},\hat{\mathbf{S}})$ .
3.4 Diffusion reconstruction
The diffusion reconstruction module mainly consists of a noise diffusion and a reconstruction backbone network. The noise diffusion is used to construct an information bottleneck externally by polluting the input data with noise, and the backbone network is used to reconstruct the polluted data directly.
Noise diffusion Given the original data  $x_0$ , the polluted data  $x_1, x_2, \dots, x_T$  are obtained by adding  $T$  step Gaussian noise. Each moment  $t$  in the noise addition process is only relevant to the  $t - 1$  moment. The hyperparameter  $\beta$  controls the ratio of added noise. We add trend retaining to the DDPM so that the model can concentrate on the more crucial stable components. The data  $x_t$  after  $t$  steps of noise pollution is:
$$
x _ {t} = \sqrt {\bar {\alpha} _ {t}} x _ {0} + \sqrt {1 - \bar {\alpha} _ {t}} \bar {z} _ {t} + \left(1 - \sqrt {\bar {\alpha} _ {t}}\right) b \tag {7}
$$
where  $\bar{\alpha}_t = \prod_{i=1}^t \alpha_i, \alpha_i = 1 - \beta_i, \bar{z}_t \sim \mathcal{N}(0,1)$  is the noise, and  $b$  is the retained information. The proof is given in Appendix A. So we can obtain the noisy data:
$$
\mathbf {X} _ {\mathrm {d}} ^ {\prime} = \sqrt {\bar {\alpha} _ {t}} \mathbf {X} _ {\mathrm {d}} + \sqrt {1 - \bar {\alpha} _ {t}} \overline {{\mathbf {Z}}} + (1 - \sqrt {\bar {\alpha} _ {t}}) \hat {\mathbf {T}} _ {\mathrm {d}} \tag {8}
$$
where  $\mathbf{X}_{\mathrm{d}}^{\prime}\in \mathbb{R}^{n\times k},\bar{\mathbf{Z}}\sim \mathcal{N}(0,\mathbf{I})$  , and  $\bar{\mathbf{Z}}\in \mathbb{R}^{n\times k}$
Table 1: Statistics of the datasets. A smaller ADF test statistic indicates a more stationary dataset.
Training Size	Testing Size	Series Number	Attacks Number	Anomaly Durations	Anomaly Rate	Frequency	ADF Test Statistic
PSM	132481	87841	25	73	1~8861	0.2776	1 minute	-9.2314
SMD	23688	23689	33	30	3~3161	0.1565	1 minute	-4.0947
SWaT	6840	7500	25	33	3~599	0.1263	1 minute	-2.9442
Backbone network As shown in the left panel of Figure 2, the backbone network consists of stacked spatial-temporal transformer blocks. The extracted trend  $\hat{\mathbf{T}}_{\mathrm{d}}$ , the noisy data  $\mathbf{X}_{\mathrm{d}}^{\prime}$ , and the timestamp  $\mathbf{X}_{\mathrm{time}}$  are encoded by the embedding layer first. To make the network more focused on the stable part without completely ignoring the trend part, we make the hidden vector:
$$
\mathbf {H} = \operatorname {e m b e d} \left(\mathbf {X} _ {\mathrm {d}} ^ {\prime}\right) - \operatorname {e m b e d} \left(\hat {\mathbf {T}} _ {\mathrm {d}}\right) + \operatorname {e m b e d} \left(\mathbf {X} _ {\text {t i m e}}\right) \tag {9}
$$
where  $\mathbf{H} \in \mathbb{R}^{n \times d_{\mathrm{model}}}$ ,  $\operatorname{embed}(\cdot)$  is the embedding network. Then, we model the temporal and dimension relationship by feeding  $\mathbf{H}$  into the stacked spatial-temporal Transformer blocks. After that, we sum the outputs of the stacked blocks with  $\hat{\mathbf{T}}_{\mathrm{d}}$  and send the result into the output layer. Finally, we directly obtain the reconstructed result  $\hat{\mathbf{X}}_{\mathrm{d}}$  of  $\mathbf{X}_{\mathrm{d}}$  instead of the predicted noise.
3.5 Joint optimization
As described in the previous section, the dynamic decomposition and diffusion reconstruction are interconnected. The dynamic decomposition module learns the inherent features of long-period unstable multivariate time series, i.e., the stable component. We use the Mean Square Error (MSE) between  $\mathbf{S}$  and  $\hat{\mathbf{S}}$  as the loss of this module. The diffusion reconstruction module directly reconstructs the data polluted by noise diffusion. It is worth noting that the output of this module is the reconstruction of  $\mathbf{X}_{\mathrm{d}}$ . Because the drift  $\mathbf{d}$  is added inside  $\mathrm{D}^3\mathrm{R}$ , and the external who calculates the loss is only aware of  $\mathbf{X}$ , we obtain the reconstruction of  $\mathbf{X}$  by  $\hat{\mathbf{X}} = \hat{\mathbf{X}}_{\mathrm{d}} - \mathbf{d}$ ,  $\hat{\mathbf{X}} \in \mathbb{R}^{n \times k}$ . Similar to the dynamic decomposition module, the MSE of  $\mathbf{X}$  and  $\hat{\mathbf{X}}$  is used directly as the loss of this module.
During the training process,  $\mathrm{D}^3\mathrm{R}$  needs to be trained end-to-end, so the loss function is defined as the sum of the two optimization objectives:
$$
L o s s = \frac {1}{n k} \sum_ {i = 1} ^ {n} \sum_ {j = 1} ^ {k} \left(\left(\mathbf {S} _ {i, j} - \hat {\mathbf {S}} _ {i, j}\right) ^ {2} + \left(\mathbf {X} _ {i, j} - \hat {\mathbf {X}} _ {i, j}\right) ^ {2}\right) \tag {10}
$$
where  $\mathbf{S}_{i,j},\hat{\mathbf{S}}_{i,j},\mathbf{X}_{i,j},\hat{\mathbf{X}}_{i,j}$  represent the labeled stable component, the predicted stable component, the original input and the reconstructed output of the  $j^{th}$  variable at the  $i^{th}$  timestamp, respectively.
3.6 Model inference
The online detection (inference) does not need labeled stable component construction and disturbance strategy in the data preprocessing stage. The anomaly score for the current timestamp consists only of the MSE of the original input  $\mathbf{X}$  and the reconstructed output. Then, we run the SPOT [23], a streaming algorithm based on extreme value theory, to label each timestamp.
4.1 Experimental settings