SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Datasets We evaluate  $\mathrm{D}^3\mathrm{R}$  extensively on three real-world datasets: PSM (Pooled Server Metrics) [1], SMD (Server Machine Dataset) [24], and SWaT (Secure Water Treatment) [15]. For each dataset, we only preserve continuous variables. That is why we do not employ MSL (Mars Science Laboratory rover) and SMAP (Soil Moisture Active Passive satellite) [8], which are discrete except for the first dimension. The normal data is divided into training data  $(80\%)$  and validation data  $(20\%)$ . Anomaly is only present in the testing data. More descriptions of the datasets are shown in Table 1. Further details about the datasets are available in Appendix B.1.
Table 2: Results in the three real-world datasets. The higher values for all metrics represent the better performance, and the best F1 scores are highlighted in bold.
Method	PSM	SMD	SWaT	Average F1
Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
Sampling	0.8439	0.5165	0.6408	0.7453	0.3144	0.4422	0.6062	0.8466	0.7065	0.5965
COPOD	0.7602	0.3175	0.4479	0.6676	0.1366	0.2268	0.9876	0.1180	0.2108	0.2951
ECOD	0.7460	0.3384	0.4656	0.7398	0.1615	0.2651	0.9761	0.1151	0.2059	0.3122
OCSVM	0.8761	0.4744	0.6155	0.0000	0.0000	0.0000	0.6196	0.7558	0.6810	0.4321
PCA	0.9220	0.3771	0.5353	0.8388	0.4019	0.5434	0.6358	0.7218	0.6761	0.5849
kNN	0.5317	1.0000	0.6943	0.6988	0.3368	0.4546	0.0000	0.0000	0.0000	0.3830
CBLOF	0.5990	0.9845	0.7449	0.8667	0.3352	0.4834	0.6308	0.7091	0.6677	0.6320
HBOS	1.0000	0.0654	0.1228	0.5628	0.8007	0.6610	0.5771	0.8049	0.6722	0.4853
IForest	1.0000	0.0335	0.0648	1.0000	0.0937	0.1713	0.6127	0.6280	0.6203	0.2855
LODA	0.9266	0.4017	0.5605	0.5902	0.6618	0.6240	0.6117	0.7014	0.6535	0.6126
VAE	0.6221	0.8772	0.7280	0.8209	0.4349	0.5686	0.6355	0.7218	0.6759	0.6575
DeepSVDD	0.7405	0.5064	0.6015	0.6498	0.6477	0.6488	0.5911	0.9353	0.7244	0.6582
LSTM-AE	0.7511	0.7586	0.7548	0.8496	0.4349	0.5753	0.6018	0.7219	0.6564	0.6622
MTAD-GAT	0.7990	0.6014	0.6863	0.8590	0.6769	0.7571	0.6590	0.7751	0.7123	0.7186
TFAD	0.7914	0.7163	0.7520	0.5632	0.9783	0.7149	0.6038	0.8196	0.6953	0.7207
Anomaly Transformer	0.5201	0.8504	0.6455	1.0000	0.0319	0.0619	0.5541	0.5994	0.5759	0.4278
Adversary	0.5351	0.8971	0.6703	0.5135	0.9663	0.6706	0.5410	0.7531	0.6297	0.6569
Our	0.6294	0.9619	0.7609	0.7715	0.9926	0.8682	0.7206	0.8529	0.7812	0.8034
Baselines We extensively compare our model with 15 baselines, including the probability-based methods: Sampling, COPOD [12], ECOD [13]; the linear transformation-based methods: OCSVM [20], PCA [22]; the proximity-based methods: kNN [18], CBLOF [5], HBOS [4]; the outlier-based methods: IForest [14], LODA [16]; the neural network-based methods: VAE [11], DeepSVDD [19], LSTM-AE [9], MTAD-GAT [33], TFAD [31], Anomaly Transformer [30]. Additionally, we set up an adversary algorithm, which is implemented simply. A timestamp will be marked as abnormal if its sequential id is divisible by  $n$ , else it will be marked as normal. In our experiment,  $n$  is set as 40. This adversary algorithm works as a timed detector and offers no information about the location of the anomaly. All baselines are based on our runs, using the identical hardware. We employ official or open-source implementations published in GitHub and follow the configurations recommended in their papers. Further details concerning the Baselines are available in Appendix B.2.
Metrics For performance evaluation, we use precision, recall, and F1 score. The classical metrics are reasonable for tasks with sample granularity but are not applicable for continuous time series, where anomalies are frequently continuous.
Most previous work [30, 33, 3, 21] employ the point adjustment method: If a point in a contiguous anomalous segment is detected correctly, all anomalies in the same segment are also considered to have been correctly detected. However, point adjustment is unreasonable, as [10] pointed out, and creates the illusion of progress. Assuming that ground truth and the predicted event are shown in Figure 3, although the real predicted event is just a timed detection, it still achieves an F1 score of 0.91 after point adjustment. The point adjustment may increase TP and decrease FN dramatically [10]. We test all baselines on all datasets with point adjustment
metrics. The detailed experiment results can be found in Appendix C. Experimental results show that the average F1 score of the adversary algorithm surprisingly outperforms all baselines, even though it can provide no anomaly location information. This phenomenon arises because point adjustment unfairly assigns higher weights to long anomaly events. Furthermore, this algorithm does not consider the adjacency of time series [7]. To address the aforementioned challenges, we employ an F1 score based on affiliation [7]. This score takes into account the average directed distance between predicted anomaly events and ground truth events to calculate precision, as well as the average directed distance between ground truth events and predicted anomaly events to determine recall.
Figure 3: Example of point adjustment strategy. The red area represents the ground truth of the anomaly event. The precision/recall/f1 before adjustment (Real) is  $0.20 / 0.05 / 0.08$ , while after adjustment (Adjusted) is  $0.84 / 1.00 / 0.91$ .
Figure 4: Results of effectiveness analyses. (a) is the visualization in the dynamic decomposition. (b) is the statistic results of sensitivity analyses in the diffusion reconstruction module.
Detailed implementation of  $\mathrm{D}^3\mathrm{R}$  can be found in Appendix B.3. All experiments are repeated 5 times, and we report the average for each metric.
4.2 Detection Results
On all three real-world datasets, as shown in Table 2,  $\mathrm{D}^3\mathrm{R}$  outperforms the adversary algorithm and achieves the best F1 performance, confirming its effectiveness and superiority. Specifically, we achieve  $0.61\%$  ( $0.7548 \rightarrow 0.7609$ ),  $11.11\%$  ( $0.7571 \rightarrow 0.8682$ ), and  $5.68\%$  ( $0.7244 \rightarrow 0.7812$ ) absolute improvement over the previous SOTA methods on PSM, SMD, and SWaT datasets, respectively.
The statistical and machine learning methods frequently perform poorly in generalization because they do not account for time series continuity. They may perform well on a partial dataset while performing poorly or even failing on another (e.g., OCSVM and CBLOF). In opposition, neural network-based methods usually perform more balanced over various datasets. Furthermore, much of the previous works were evaluated by point adjustment, leading to a false boom. The metrics based on affiliation provide a more objective and reasonable evaluation of various methods, although their scores will drop.
Noticeably,  $\mathrm{D}^{3}\mathrm{R}$  significantly outperforms the other methods on the SMD and SWaT datasets, which are typically characterized by high nonstationarity, suggesting the limitation of previous work on unstable real-world data. In our model, the dynamic decomposition and diffusion reconstruction modules complement each other. The former dynamically eliminates the unstable interference from the original data. The latter models the series from more crucial components. These designs tackle the shortcomings of previous work and maintain excellent robustness in complex real-world data.
In addition to the affiliation evaluation metric, we compare  $\mathrm{D}^3\mathrm{R}$  with other baselines in all datasets using the AUC score. The detailed experiment results can be found in Appendix D. For a more intuitive comparison, we visualize the anomaly scores of the  $\mathrm{D}^3\mathrm{R}$  and partial baselines, as shown in Appendix E. The quantitative and qualitative experimental results all show that  $\mathrm{D}^3\mathrm{R}$  outperforms other baselines. Moreover, to validate the practicality of  $\mathrm{D}^3\mathrm{R}$  in the actual production environment, we compare run time across different neural network-based algorithms in the SMD and present a summary in Appendix F. Both the training time and inference time of our model are acceptable.
4.3 Effectiveness verifications
Dynamic decomposition module We propose a dynamic decomposition module to tackle Challenge 1. To more intuitively verify the performance of the dynamic decomposition module, we provide visualization of the decomposition results on highly unstable datasets (SMD and SWaT). Figure 4(a) shows that our model robustly extracts the fundamental stable components, although real-world multivariate data is long-period and complex.
Diffusion reconstruction module We propose a diffusion reconstruction module to tackle Challenge 2. To verify the superiority of controlling information bottleneck externally, we replace the entire diffusion reconstruction module with the VAE module and perform sensitivity analyses for the VAE module and diffusion reconstruction module. The hyperparameter that significantly af
Table 3: Results of ablation studies. F1 scores are reported, with higher values meaning better performance. The best scores are highlighted in bold.
dataset	D3R	w/o temporal	w/o spatial	w/o time	w/o data	w/o offset	w/o disturbance	w/o diffusion	w/o trend
PSM	0.7609	0.7280	0.7571	0.7074	0.7374	0.7420	0.7508	0.7347	0.7592
SMD	0.8682	0.8169	0.8173	0.7403	0.8114	0.8446	0.6947	0.8392	0.8006
SWaT	0.7812	0.7031	0.7274	0.7563	0.6720	0.7205	0.7293	0.7293	0.7109
Average	0.8034	0.7493	0.7673	0.7347	0.7403	0.7690	0.7249	0.7677	0.7569
fcts the performance of the VAE module is the latent space size  $z$ . The hyperparameters that may affect the performance of the diffusion reconstruction module are the hidden state dimension  $d_{\mathrm{model}}$ , the noise ratio  $\beta$ , and the pollution step  $T$ . We scale each hyperparameter by a factor of  $\{0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0\}$ , the statistic results for the SMD are shown in Figure 4(b). As well as allowing adjusting the information bottleneck without retraining, the diffusion reconstruction module is significantly more effective (i.e., the F1 score is higher) and more robust (i.e., the F1 score is more concentrated) than the VAE module. During the hyperparameter changing, the F1 score span of the VAE module is  $8.96\%$  ( $0.7496 \rightarrow 0.8392$ ), while the maximum span of the diffusion reconstruction module  $(\beta)$  is  $3.30\%$  ( $0.8381 \rightarrow 0.8711$ ).
In order to verify the effectiveness and necessity of our designs, we perform ablation studies on both dynamic decomposition and diffusion reconstruction modules of  $\mathrm{D}^3\mathrm{R}$ , respectively.
Dynamic decomposition module In this module, our main designs are spatial-temporal transformer, data-time mix-attention, offset subtraction, and disturbance strategy. Their ablation results are shown in Table 3. The w/o temporal, w/o spatial, w/o time, w/o data, w/o offset, and w/o disturbance represent the variants of  $\mathrm{D}^3\mathrm{R}$  removing temporal transformer, spatial transformer, time attention, data attention, offset subtraction, and disturbance strategy, respectively. The temporal transformer and spatial transformer bring an average absolute improvement of  $5.41\%$  (0.7493→0.8034) and  $3.61\%$  (0.7673→0.8034), respectively. Compared with the spatial transformer, the temporal transformer is more critical. The spatial transformer performs much worse when the datasets have smaller series number (PSM and SWaT). When the series number is large (SMD), the dimension dependency is more prosperous, and the spatial transformer performs better. Moreover, time attention and data attention in data-time mix-attention bring a great absolute promotion of  $6.87\%$  (0.7347→0.8034) and  $6.31\%$  (0.7403→0.8034). While data attention is better suited to short-period data (SWaT), time attention is better suited to long-period data (PSM and SMD). Offset subtraction can further improve our model with  $3.27\%$  (0.7690→0.8034). Finally, the disturbance strategy brings a significant improvement of  $7.85\%$  (0.7249→0.8034), especially for the SMD and SWaT datasets with high nonstationary characteristics.
Diffusion reconstruction module This module uses noise diffusion to control the information bottleneck externally. To verify the superiority of noise diffusion, we replace the entire diffusion reconstruction module with the VAE module, and the ablation results are shown in Table 3 (w/o diffusion). As well as allowing adjustment of the information bottleneck without retraining, noise diffusion also provides a  $3.57\%$ $(0.7677\rightarrow 0.8034)$  absolute improvement. The following section will discuss more advantages of the diffusion reconstruction module. An additional innovation of this module is trend retaining, and the ablation results are shown in Table 3 (w/o retaining). After removing trend retaining, the performance of  $\mathrm{D}^3\mathrm{R}$  decreases significantly, especially in the highly nonstationary SMD dataset  $(0.8682\rightarrow 0.8006)$  and SWaT dataset  $(0.7812\rightarrow 0.7109)$ . Trend retaining enables the model to focus on the critical stable components, avoiding being distracted by irrelevant information.
4.5 Hyperparameter analyses
The hyperparameter that significantly affects the performance of the VAE module is the latent space size  $z$ . The hyperparameters that may affect the performance of  $\mathrm{D}^3\mathrm{R}$  are the added drift
Figure 5: Results of the sensitivity analysis. The higher F1 Score represents the better performance. The dark line represents the mean of 5 experiments, and the light area represents the range.
boundary  $p$ , the maximum horizontal offset  $d$ , the hidden state dimension  $d_{\mathrm{model}}$ , the noise ratio  $\beta$ , and the pollution step  $T$ . To analyze their influence on anomaly detection, we perform hyperparameter sensitivity analysis in the SMD. We scale each hyperparameter by a factor of  $\{0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0\}$ , and the results are shown in Figure 5.
Anomaly detection methods based on reconstruction require constructing the information bottleneck to reconstruct normal data and inhibit anomaly effectively. While traditional autoencoders control the information bottleneck from inside through the latent space size, our approach controls the information bottleneck from outside through noise diffusion. As the experimental results show, with the bottleneck going from strict to loose (latent space size  $z$ , noise ratio  $\beta$ , pollution step  $T$  going from small to large), the performance of the model goes from up to down. If the information bottleneck is too tight, the normal data cannot be reconstructed, and the precision is low. If the information bottleneck is too loose, the anomaly can be reconstructed well, and the recall is low. On the one hand, our approach to control the information bottleneck externally is more robust (smaller range of variation). On the other hand, as the information bottleneck is not an attribute of the model itself, we can try different bottleneck sizes when inference without retraining the model again.
Since it is no longer an information bottleneck,  $\mathrm{D}^3\mathrm{R}$  is not sensitive to the hidden state dimension  $d_{\mathrm{model}}$ . A larger hidden space tends to bring better results but also a greater computational cost. As for the added drift boundary  $p$  and maximum horizontal offset  $d$ , the impact on our model becomes small once they exceed a certain threshold. Overall,  $\mathrm{D}^3\mathrm{R}$  is robust to all hyperparameters we test.
This paper proposes Dynamic Decomposition with Diffusion Reconstruction  $(\mathrm{D}^3\mathrm{R})$  for long-period unstable multivariate time series anomaly detection to cover the overlook in previous work. We first decompose the long-period unstable multivariate time series dynamically and then directly reconstruct the data polluted by noise diffusion. Extensive experiments prove that  $\mathrm{D}^3\mathrm{R}$  significantly outperforms existing works. The method we proposed to break the limitation of the local sliding window is also meaningful for other long-period multivariate time series analysis tasks, such as prediction and imputation. Meanwhile, the approach to controlling information bottleneck externally can also be used for anomaly detection of other modal data, such as picture, video, and log data.
This work was supported in part by the National Natural Science Foundation of China under Grants (62101064, 62171057, 62201072, 62071067), in part by the Ministry of Education and China Mobile Joint Fund (MCM20200202, MCM20180101), in part by the Beijing University of Posts and Telecommunications-China Mobile Research Institute Joint Innovation Center.
A Noise diffusion proofs
In this section, we explain the proof of Equation 7 in our paper.
Suppose  $z \sim \mathcal{N}(0,1)$  is the Gaussian noise to be added,  $\beta$  controls the amount of noise added each step,  $\alpha = 1 - \beta$ , and  $b$  is the trend information to be retained. The  $x_{t}$  after  $t$  steps of noise pollution is:
$$
\begin{array}{l} x _ {t} = \sqrt {\alpha_ {t}} (x _ {t - 1} - b) + \sqrt {1 - \alpha_ {t}} z _ {t} + b \tag {11} \\ = \sqrt {\alpha_ {t}} x _ {t - 1} + \sqrt {1 - \alpha_ {t}} z _ {t} + (1 - \sqrt {\alpha_ {t}}) b \\ \end{array}
$$
The Gaussian distribution, as we know, has the following properties:
- Suppose  $x \sim N(\mu, \sigma^2)$ , then  $ax + b \sim N(a\mu + b, a^2\sigma^2)$ .
- Suppose  $x \sim N(\mu_x, \sigma_x^2), y \sim N(\mu_y, \sigma_y^2), x, y$  are independent random variables, then  $x + y \sim N(\mu_x + \mu_y, \sigma_x^2 + \sigma_y^2)$ .
Based on these properties, the noise diffusion process can be deduced further:
$$
\begin{array}{l} x _ {t} = \sqrt {\alpha_ {t}} (x _ {t - 1} - b) + \sqrt {1 - \alpha_ {t}} z _ {t} + b \\ = \sqrt {\alpha_ {t}} \left[ \sqrt {\alpha_ {t - 1}} \left(x _ {t - 2} - b\right) + \sqrt {1 - \alpha_ {t - 1}} z _ {t - 1} + b - b \right] + \sqrt {1 - \alpha_ {t}} z _ {t} + b \tag {12} \\ = \sqrt {\alpha_ {t} \alpha_ {t - 1}} x _ {t - 2} + \sqrt {1 - \alpha_ {t} \alpha_ {t - 1}} z _ {t} + \left(1 - \sqrt {\alpha_ {t} \alpha_ {t - 1}}\right) b \\ \end{array}
$$
Repeating the above process, we can directly infer  $x_{t}$  from  $x_{0}$  as follows:
$$
x _ {t} = \sqrt {\bar {\alpha} _ {t}} x _ {0} + \sqrt {1 - \bar {\alpha} _ {t}} \bar {z} _ {t} + \left(1 - \sqrt {\bar {\alpha} _ {t}}\right) b \tag {13}
$$
where  $\bar{\alpha}_t = \prod_{i=1}^t \alpha_i$  and  $\bar{z}_t \sim \mathcal{N}(0,1)$ .
B Detailed experimental settings
The datasets can be downloaded from the following:
- PSM: https://github.com/eBay/RANSynCoders/tree/main/data.
- SMD: https://github.com/NetManAIOps/OmniAnomaly/tree/master/ServerMachineDataset.
- SWaT: https://itrust.sutd.edu.sg/testbeds/secure-water-treatment-swat/.
All baselines are based on our runs, using the identical hardware. We employ official or open-source implementations published in GitHub and follow the configurations recommended in their papers. The baselines can be downloaded from the following:
- Sampling: https://github.com/yzhao062/pyod.
- COPOD: https://github.com/yzhao062/pyod.
- ECOD: https://github.com/yzhao062/pyod.
- OCSVM: https://github.com/yzhao062/pyod.
- PCA: https://github.com/yzhao062/pyod.
- kNN: https://github.com/yzhao062/pyod.
- CBLOF: https://github.com/yzhao062/pyod.
HBOS: https://github.com/yzhao062/pyod.
- IForest: https://github.com/yzhao062/pyod.
- LODA: https://github.com/yzhao062/pyod.
- VAE: https://github.com/yzhao062/pyod.
Table 4: Quantitative results after point adjustment in the three real-world datasets. The higher values for all metrics represent the better performance, and the best F1 scores are highlighted in bold.
Method	PSM	SMD	SWaT	Average F1
Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
Sampling	0.9986	0.8571	0.9224	0.9958	0.9690	0.9822	0.1583	0.9271	0.2705	0.7250
COPOD	0.9795	0.8572	0.9143	0.9975	0.8538	0.9201	0.9953	0.6695	0.8005	0.8783
ECOD	0.9565	0.8632	0.9075	0.9645	0.8573	0.9078	0.9954	0.6832	0.8103	0.8752
OCSVM	0.9990	0.8115	0.8955	0.0000	0.0000	0.0000	0.1596	0.9113	0.2716	0.3890
PCA	0.9996	0.7777	0.8748	0.9964	0.9720	0.9840	0.1645	0.9029	0.2783	0.7124
kNN	0.2776	1.0000	0.4345	0.9950	0.9676	0.9811	0.0000	0.0000	0.0000	0.4719
CBLOF	0.4301	0.9996	0.6014	0.9970	0.9698	0.9832	0.1635	0.9029	0.2769	0.6205
HBOS	1.0000	0.5957	0.7466	0.3745	0.9922	0.5437	0.3932	0.8965	0.5467	0.6123
IForest	1.0000	0.2204	0.3612	1.0000	0.8560	0.9224	0.4204	0.8141	0.5545	0.6127
LODA	0.9975	0.8465	0.9158	0.8452	0.9803	0.9077	0.2502	0.8912	0.3907	0.7381
VAE	0.6020	0.9486	0.7365	0.9953	0.9720	0.9835	0.1646	0.9029	0.2784	0.6661
DeepSVDD	0.6919	0.8779	0.7739	0.9575	0.9714	0.9644	0.1465	0.9842	0.2551	0.6645
LSTM-AE	0.7280	0.9288	0.8162	0.9961	0.9720	0.9839	0.1614	0.9029	0.2739	0.6913
MTAD-GAT	0.9589	0.8987	0.9279	0.9940	0.9857	0.9898	0.1745	0.9197	0.2934	0.7370
TFAD	0.8819	0.9851	0.9306	0.9991	0.9492	0.9735	0.2559	0.9387	0.4022	0.7688
Anomaly Transformer	0.9022	0.9879	0.9431	1.0000	0.8525	0.9204	0.9076	0.7159	0.8005	0.8880
Adversary	0.9384	0.9921	0.9645	0.8774	0.9649	0.9191	0.8309	0.8300	0.8304	0.9047
- DeepSVDD: https://github.com/yzhao062/pyod.
- LSTM-AE: https://github.com/matanle51/LSTM_AutoEncoder.
- MTAD-GAT: https://github.com/ML4ITS/mtad-gat-pytorch.
- TFAD: https://github.com/DAMO-DI-ML/CIKM22-TFAD.
- Anomaly Transformer: https://github.com/thuml/Anomaly-Transformer.
B.3 Implementation
In our experiment, the sliding window has a fixed size 64 for all datasets. We set the dimension of hidden states as 512, the dimension of attention as 64, the number of attention heads as 8, the number of network layers as 2, and the dropout as 0.6. We use grid search to obtain the best SPOT parameters for each dataset and record the results with the highest F1 scores. The  $\beta$  of the diffusion model changes from 0.0001 to 0.02 in 1000 steps like DDPM without adjustment. We add noise with 500 steps (selected from  $\{100,300,500,700,900\}$ ) as an external information bottleneck when the noise diffusion. The boundary of added drift in the disturbance strategy is 10 (selected from  $\{1,2.5,5,10,20\}$ ). The max offset of offset subtraction is 30 (selected from  $\{5,10,20,30,40\}$ ). We use the Adam optimizer with an initial learning rate of  $10^{-4}$ . The training process is early stopped within 8 epochs with a batch size of 8. The implementation of  $\mathrm{D}^3\mathrm{R}$  is carried out using Python 3.9.13 and PyTorch 1.11.0. All experiments are performed on a Ubuntu Server with a 12th Gen Intel(R) Core(TM) i9-12900K @ 3.60GHz processor and an NVIDIA GeForce RTX 3090 Graphics Card.
C Detection results with point adjusted
We test all baselines on all datasets with point-adjusted evaluation metrics, and the detailed quantitative results are shown in Table 4. Since the data processing and hardware are not the same as those in the original paper, the results of the baselines in our experiments are slightly different from those in the original paper. Nevertheless, the errors are still within a reasonable range.
The point adjustment method is unreasonable. Despite the no helpful information provided by the adversary algorithm, its average performance still exceeds other SOTA methods by  $2\%$  of the relative F1 score. Specifically, the adversary algorithm achieves an absolute improvement of  $2.14\%$ $(0.9431\rightarrow 0.9645)$  and  $2.01\%$ $(0.8103\rightarrow 0.8304)$  on the PSM and SWaT datasets, respectively. Only on the SMD dataset the adversary algorithm is slightly weaker than the best baseline  $(0.9898\rightarrow 0.9191)$ . If we further adjust the  $n$  in the adversary algorithm, it can also outperform all the baselines in the SMD dataset.