Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders

Authors: Qichao Shentu, Beibu Li, Kai Zhao, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo
Primary area: learning on time series and dynamical systems
Keywords: Time series, Anomaly detection

Abstract:
Time series anomaly detection plays a vital role in a wide range of applications. Existing methods require training one specific model for each dataset, which exhibits limited generalization capability across different target datasets, hindering anomaly detection performance in various scenarios with scarce training data. Aiming at this problem, we propose constructing a general time series anomaly detection model, which is pre-trained on extensive multi-domain datasets and can subsequently apply to a multitude of downstream scenarios. The significant divergence of time series data across different domains presents two primary challenges in building such a general model: (1) meeting the diverse requirements of appropriate information bottlenecks tailored to different datasets in one unified model, and (2) enabling distinguishment between multiple normal and abnormal patterns, both are crucial for effective anomaly detection in various target scenarios. To tackle these two challenges, we propose a general time series anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders (DADA), which enables flexible selection of bottlenecks based on different data and explicitly enhances clear differentiation between normal and abnormal series. We conduct extensive experiments on nine target datasets from different domains. After pre-training on multi-domain data, DADA, serving as a zero-shot anomaly detector for these datasets, still achieves competitive or even superior results compared to those models tailored to each specific dataset.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1729048428562

thread_type:
official_review

<thread_object>
creator: reviewer_ebsp

<content>
summary:
The author propose a solution for general time series anomaly detection, which is call time series anomaly Detector with Adaptive bottlenecks and Dual Adversarial decoders (DADA). This model is  pre-trained on multi-domain time series data. Two components called Adaptive Bottlenecks and Dual Adversarial Decoders is leveraged to enhance the model performance. The model achieves competitive performance on various datasets even compared with SOTA models trained specifically for each dataset.

strengths:
+ The task on zero-shot time series anomaly detection is quite challenging, the author build a unified end-to-end solution and demonstrate great results. 

+ The paper is well-organized, and it introduces the background and motivation of the proposed methods clearly. The figures also explain their method pretty well.

+ The paper demonstrate its experiment results on enough datasets.  The ablation studies on Sec 4.3 also makes the paper convincible.

weaknesses:
- The difference between the pretrain dataset and the evaluation dataset is not listed. How you select those dataset?

- It will be great to specify some more experiment details. Are there any data preprocessing on the input data?

- What's the computation cost(overhead) of importing Adaptive Bottlenecks and Dual Adversarial Decoders? Please specify.

questions:
- Can you list the data difference between the pretrain dataset and the evaluation dataset? How you select the pretrain dataset?

- Are there any data preprocessing on the input data?

- What's the computation cost(overhead) of importing Adaptive Bottlenecks and Dual Adversarial Decoders? Please specify.
</content>
</thread_object>

</thread>
<thread>
creation date:
1729761458807

thread_type:
official_review

<thread_object>
creator: reviewer_e5ze

<content>
summary:
The paper proposes DADA method for general time series anomaly detection, which is pre-trained on extensive multi-domain datasets and perform anomaly detection on various target scenarios without domain-specific training. DADA proposes Adaptive Bottlenecks to addresses the flexible reconstruction requirements of multi-domain data. DADA also employs Dual Adversarial Decoders amplify the decision boundaries between normal time series and common anomalies in an adversarial training way, which improves the general anomaly detection capability across different scenarios. The paper conducts extensive experiments on nine target datasets from different domains, and demonstrate that DADA achieves superior results compared with state-of-the-art models trained specifically for each dataset.

strengths:
The paper addresses the issue that each dataset in the field of time series anomaly detection requires a specifically trained model, which holds practical significance. The paper proposes a novel method DADA for zero-shot inference on the target domain after pre-training on multiple datasets from different domains. DADA proposes the Adaptive Bottlenecks to addresses the flexible reconstruction requirements of multi-domain data, along with dual adversarial decoders to enhance the clear differentiation between normal and abnormal patterns, showcasing a degree of originality. The paper has a clear structure and expression. In the experimental section of the paper, DADA demonstrates superiority over multiple state-of-the-art methods across nine datasets and provides extensive analysis to prove the necessity of each module, showcasing high quality.

weaknesses:
1. The paper lacks an analysis of the complementary mask rate's effect on anomaly detection performance. For example, the author could add an experiment to demonstrate the changes in anomaly detection F1-score under various complementary mask rates and explain why this mask performs the best.

2. In the experimental section, Figure 4 shows that after fine-tuning, the model's performance improved from 78.48% to 79.90% on MSL and from 84.57% to 85.50% on SMD, an increase of about 1%. The data shows that the improvement of the model after fine-tuning is small.

3. In the ablation study, Table 3 shows that the model performs better when removing the AdaBN, Adversarial, and Dual Decoders modules compared to removing only the Adversarial module. The paper explains that directly maximizing the anomalous time series will increases confusion. Further explanation is needed to clarify why maximizing the anomalous time series leads to increased confusion.

4. There are labeling mistakes in the experimental results. For example, in Table 8, the R_AUC_PR value of DADA on the PSM dataset is 47.40, which is less than ModernTCN's 47.48, but 47.40 is bolded. In Table 2, the F1 metric column for the CICIDS dataset is missing a underline marking.

questions:
1. The paper does not compare the inference time of DADA with the baselines. Compared to the baselines, does DADA have a shorter inference time?  In particular, when compared to other pre-trained models, does DADA have an advantage?

2. In the DADA model, the original data first undergoes complementary masking operations, and then features are extracted by the encoder. In subsection 3.2, it states that the masked portions of the original data input to the encoder have a value of 0, but there may also be 0 values in the non-masked portions of the original data. How does the encoder differentiate between the masked and non-masked portions?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730264715467

thread_type:
official_review

<thread_object>
creator: reviewer_e12v

<content>
summary:
This paper develops a novel general time series anomaly detection by pre-training on multi-domain time series.  The proposed adaptive bottleneck enables the general model to be applied to various target scenarios without requiring domain-specific training. To improve the general anomaly detection capability, the dual adversarial decoder is further employed to amplify the decision boundary.

strengths:
1. The general time series anomaly detection plays a key role in the real application.
2. DADA employs the adaptive bottlenecks for multi-domain time series data by dynamically selecting suitable bottleneck sizes. 
3. The effectiveness of both AdaBN and Dual Decoders has been demonstrated in the experiments.

weaknesses:
1. If anomaly injection is used to establish a supervised setting, this implies that all original samples(some anomalies are included) are treated as normal samples. I suggest the author add an experiment to examine performance under varying levels of anomaly contamination in the training samples.
2. During model training, two optimization losses are utilized, along with the optimization approach and any challenges encountered in the optimization process. The visualizations of loss curves for the two losses are essential for further demonstration.

questions:
1. Does "anomaly injection" refer to incorporating multiple types of anomaly patterns into each dataset? More details about the implementation of anomaly injection should be included in the paper since it is crucial for conducting the supervised setting.
2. How do you adapt various baseline methods pre-trained on multi-domain datasets for zero-shot evaluation if they are typically utilized in the standard form?
3. How can DADA be applied during the inference step to further fine-tune the target datasets for downstream tasks in the experiments?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730555338866

thread_type:
official_review

<thread_object>
creator: reviewer_hvuu

<content>
summary:
Authors propose a General time series anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders (DADA), which enables flexible selection of bottlenecks based on different data and explicitly enhances clear differentiation between normal and abnormal series. The zero-shot capability is indeed impressive.

strengths:
The writing is clear and easy to understand.
The pictures and tables in the paper are beautiful and clear.
The proposed method, DADA, can achieve zero-shot.

weaknesses:
1. AdaBN has almost no difference from the previous moe, yet the author exaggerates their design. I think there is a question of contribution. Moreover, the author does not explain the difference from Moe in the main text nor add relevant references [a]. I think this is somewhat deceptive. 
2. The role of AdaBN is not as great as the author claims. See the analysis in question 1. There is excessive exaggeration of contribution.
[a] Shazeer, Noam, et al. "Outrageously large neural networks: The sparsely-gated mixture-of-experts layer." arXiv preprint arXiv:1701.06538 (2017).

questions:
1. In the ablation experiment, the result of the fourth row is the second highest. In my opinion, AdaBN should be basically equivalent to moe, except for the different dimensions. However, at this time, the model effect is basically close to the final model, which means that the main source of performance is not the specially designed optimization method. This makes me doubt the component innovation of the paper. At the same time, Table 10 in the appendix also supports this conclusion. The result of moe 256 is completely better than the result of the fourth row. This seriously indicates that AdaBN is likely to have a much smaller actual effect than Moe 256. Further, the innovation of AdaBN should be greatly discounted.
2. The experimental analysis lacks the analysis of COMPLEMENTARY MASK MODELING. The author takes this item as the key point of the method but lacks further research.
3. The visualization analysis does not state what dataset is used, nor is there visualization of the original data. Furthermore, there is a lack of visualization comparison with other algorithms, lacking the credibility of the experiment.
4. I hope the author can open source the generation method and generation script of Monash+ and provide a directly usable data set. At present, the explanations in the paper are completely insufficient.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730721362598

thread_type:
official_review

<thread_object>
creator: reviewer_wtbt

<content>
summary:
This paper proposes a new general time series anomaly detector with adaptive bottleneck and dual adversarial decoder. By pre-training on multi-domain time series data, anomaly detection is performed efficiently in different target scenarios without domain-specific training.
The overall writing of this article is good and the logic is clear。
This paper first proposes an adaptive bottleneck module to compress features into the latent space, reflecting the different information densities of multi-domain data. It solves the challenge of different requirements for appropriate information bottlenecks for different datasets.
This paper uses dual adversarial decoders to expand the robustness distinction between normal and abnormal modes, improving the general anomaly detection ability in different scenarios.

strengths:
The overall writing of this article is good and the logic is clear。
This paper first proposes an adaptive bottleneck module to compress features into the latent space, reflecting the different information densities of multi-domain data. It solves the challenge of different requirements for appropriate information bottlenecks for different datasets.
This paper uses dual adversarial decoders to expand the robustness distinction between normal and abnormal modes, improving the general anomaly detection ability in different scenarios.

weaknesses:
The adaptive router formula of the adaptive bottleneck module proposed in this paper lacks certain innovation, and its formula is similar to the routing function proposed in “PATHFORMER: MULTI-SCALE TRANSFORMERS WITH ADAPTIVE PATHWAYS FOR TIME SERIES FORECASTING”.
Some formulas in this paper lack clear explanations, which may cause some confusion when reading. For example, the reconstruction process proposed in the article is denoted as Recon(·), but there is a lack of detailed description of this process.

questions:
I hope the author can explain the difference between the routing function proposed in "PATHFORMER: MULTI-SCALE TRANSFORMERS WITH ADAPTIVE PATHWAYS FOR TIME SERIES FORECASTING". At the same time, give a detailed explanation of the reconstruction process Recon(·).
</content>
</thread_object>

</thread>


Metareview:
metareview:
The work aims to learn a generalist model for zero-shot time series anomaly detection. The model is built with Adaptive Bottlenecks and Dual Adversarial Decoders. It is trained on multi-domain datasets and then tested directly on different datasets. 

**Strengths**
- Paper is well written [WTbT, hvUu, e5Ze, EBSp]
- The studied setting is practically sound [e12v, e5Ze]
- The model combining adaptive information bottleneck and dual adversarial decoders is novel [WTbT, e5Ze, EBSp]
- The model shows effective zero-shot TSAD performance across various downstream datasets [WTbT, hvUu, e12v, e5Ze, EBSp]


**Weaknesses**
- The adaptive bottleneck module itself lacks novelty [WTbT, hvUu]
- Lack of analysis on part of equations, modules or the empirical results [WTbT, hvUu, e5Ze]
- Lack of investigation into the performance under varying level of anomaly contamination [e12v]
- Some key implementation details of the proposed method, the evaluation protocol, the competing methods are missing [e12v, EBSp]
- The model has limited improvement if it is fine-tuned on the target dataset [e5Ze]
- Lack of computational time analysis [e5Ze, EBSp]