Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders
Authors: Qichao Shentu
Beibu Li
Kai Zhao
Yang Shu
Zhongwen Rao
Lujia Pan
Bin Yang
Chenjuan Guo
Venue: ICLR 2025 Poster (2025)
Keywords: Time series
Anomaly detection
Primary area: learning on time series and dynamical systems
TL;DR: We propose a general time series anomaly detection model that is pre-trained on multi-domain datasets and can subsequently apply to many downstream scenarios.

Abstract: Time series anomaly detection plays a vital role in a wide range of applications. Existing methods require training one specific model for each dataset, which exhibits limited generalization capability across different target datasets, hindering anomaly detection performance in various scenarios with scarce training data. Aiming at this problem, we propose constructing a general time series anomaly detection model, which is pre-trained on extensive multi-domain datasets and can subsequently apply to a multitude of downstream scenarios. The significant divergence of time series data across different domains presents two primary challenges in building such a general model: (1) meeting the diverse requirements of appropriate information bottlenecks tailored to different datasets in one unified model, and (2) enabling distinguishment between multiple normal and abnormal patterns, both are crucial for effective anomaly detection in various target scenarios. To tackle these two challenges, we propose a general time series anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders (DADA), which enables flexible selection of bottlenecks based on different data and explicitly enhances clear differentiation between normal and abnormal series. We conduct extensive experiments on nine target datasets from different domains. After pre-training on multi-domain data, DADA, serving as a zero-shot anomaly detector for these datasets, still achieves competitive or even superior results compared to those models tailored to each specific dataset.

Published as a conference paper at ICLR 2025
TOWARDS A GENERAL TIME SERIES ANOMALY DETECTOR WITH ADAPTIVE BOTTLENECKS AND DUAL ADVERSARIAL DECODERS
Qichao Shentu'; Beibu Li Kai Zhao', Yang Shu Zhongwen Rao", Lujia Pan?, Bin Yang', Chenjuan Guo!
East China Normal University, AAlloorg University, HHuawei Noah's Ark Lab fgeshentu, beibul) euuu. ecnu edu. en, {yshu, ejguo, byang) ecnu, acnu.edu.cc.. (kaiz)@cs aau. dk, (raozhongwen, panlujiaj@huawei.
ABSTRACT
Time series anomaly detection plays vital role in wide range of applications. Existing methods require training one specific model for each dataset, which ex- hibits limited generalization capability across different target datasets, hindering anomaly detection performance in various scenarios with scarce training data. Aiming at this problem, we propose constructing general time series anomaly detection model, which is pre-trained on extensive multi multi-domain datasets and can subsequently apply to multitude of downstream scenarios. The significant diver- gence of time series data across different domains presents two primary challenges in building such general model: (1) meeting the diverse requirements of appro- priate information bottlenecks tailored to different datasets in one unified model, and (2) enabling distinguishment between multiple normal and abnormal patterns, both are crucial for effective anomaly detection in various target scenarios. To tackle these two challenges, we propose a general time series anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders (DADA), which en- ables flexible selection of bottlenecks based on different data and explicitly en- hances clear differentiation between normal and abnormal series. We conduct extensive experiments on nine target datasets from different domains. After pre- training on multi-domain data, DADA, serving as zero-shot anomaly detector for these datasets, still achieves competitive or even superior results compared those models tailored to each specific dataset. The code is made available /ttpss//giteen com wwtp:///eccsssooeeeeeeeeeeeeeeee
INTRODUCTION
With the continuous advancement of technology and the widespread application of various sensors, time series data ubiquitous in many real-world scenarios (Anandakrishnan al., 2017 Cook et al. 2020 Kieu et al. 2022 Tian et al. 2024| Pan et al. 2023 Effectively detecting anomalies in time series data helps to identity potential issues time and takes necessary measures to ensure the normal operation of systems, thereby avoiding possible economic losses and security threats (Yang) ctt al.) 2021). For example, in the cyber-security field, timely detection of abnormal network traffic prevent service interruptions (Dong et al. 2023a Wang & Zhu 2022): in the healthcare sector, detecting anomalies is crucial for preventing diseases (Wang et al.
2023b Salem et al. 2021).
Anomaly detection methods based on deep learning (Xu et al. 2022: Su et al. 2019 dong-
hao wang xue 2024) have recently achieved significant success due to their powerful repre- sentation capabilities for data. However, existing methods for detecting anomalies in time ries data typically require constructing and training specific models for different datasets (Cook) al.) 2020 WWang et al.) (203a]] [Campos et al., [2022] YYang et al. 2023). Although these methods show good performance on each specific dataset, their generalization ability across different target scenarios is limited (Zhang al. 2023). In some scenarios, there are
"Equal contribution.

---

Published as a conference paper at ICLR 2025
enough data or resources for specific model training, such those where data collection is im- peded by data scarcity, time and labor costs, or user privacy concern.. Therefore, the fea- sibility of existing methods in practice may be greatly restricted. Aiming at this problem, we propose constructing a general time series anomaly detection (GTSAD) model. As shown in Figure [ by pre-training the model large time series data from multiple sources and do- mains, it is encouraged to learn anomaly detec- tion capabilitiies from richer temporal informa- tion, which gives the potential to mitigate the overfitting of domain-specific patterns and learn patterns and models that are more generalizable. Then, the model can be efficiently applied to a wide range of downstream scenarios. However, constructing general anomaly detection model still faces the following two major challenges.
GTSAD
1i Multi-Domain Time series Data
Inferease
Scenario 1
Pre train
DADA
Scenario
Scenarion
Standard
Scnvario 1 Train Duta
Detector 1
Inference
Scenario 1
First, the large-scale time series data are from multiple sources and domains with different in- formation densities, which raises the challenge of meeting the diverse requirements of appropri- ate information bottlenecks tailored different datasets in one unified model. Existing anomaly detection methods emphasize accurately captur-
Train
Inference
Scenario
Figure I: General time series anomaly detection (GTSAD) and standard methods training spe- cific detectors for each scenario.
ing the characteristics of normal data with autoencoder, due to the inherently scarce nature of anomalies (Chalapathy Chawla 2019). Information bottleneck considered as trade-off be- tween compactly compressing the intrinsic information of the original data and high-fidelity recon- struction (Kawaguchi al., 2023). large bottleneck can reconstruct the original data accurately but may cause the model to fit unnecessaty noise unrelated to normal patterns. On the contrary, a small bottleneck can compactly compress the intrinsic information but may result in the loss of diverse normal patterns and poor reconstruction (Wang al. 2023a). Existing anomaly detection methods typically tune a fixed internal information bottleneck for each scenario, which is limited in inflexible representation capability and insufficient generalization ability when facing multi-domain time series data with significantly different data distributions, periodicity, and noise levels (Liu et al. 2024a), making difficult adapt well downstream scenarios.
Second, the diverse manifestations of anomalies across multiple domains pose another significant challenge robust distinguishmmnt between normal patterns and diverse anomaly patterns. A model capable of general anomaly detection not only needs to have a clear understanding of spe- cific normal patterns in time series data from different domains but also requires clear delineation of decision boundaries between diverse normal and abnormal patterns. Existing anomaly detection methods are often based on the one-class classification assumption and only learn normal patterns in time series data for each specific domain, lacking explicit differentiation of the anomalies. Mean- while, simply learning to discriminate normal and abnormal patterns specific to one domain not generalizable enough, as the distributions of normal and abnormal patterns change across domains. As a result, they may struggle to distinguish between normal and abnormal patterns for general time series, where multi-domain data exist with diverse anomaly manifestations and the decision boundaries between normal and abnormal patterns are more complex.
In this paper, propose novel general time series anomaly Detector with Adaptive bottlenecks and Dual Adversarial decoders (DADA). For the first challenge, we consider the model's general- ization ability from the perspective of dynamic bottleneck and introduce the Adaptive Bottlenecks module to enhance the learning of normal time series patterns from multi-domain data. We employ bottleneck to compress features into the latent space, the size of which can manifest different infor- mation densities for multi- multi-domain data (Wang et al. 2023a). To meet the requirements of different information bottlenecks for divergent data, we establish a bottleneck pool containing various bot- tlenecks with different latent space sizes. Then, data-adaptive mechanism is further proposed to enable the flexible selection of proper internal sizes based on the unique reconstruction requirements of the input data. To address the second challenge, we propose the Dual Adversarial Decoders module, which works with the encoder to enlarge the robust distinguishment between normal and anomaly patterns, where the normal decoder learns normal patterns for accurate reconstruction of
2

---

Published as a conference paper at ICLR 2025
normal sequences, and the anomaly decoder learns different anomaly patterss from anomaly time series. By injecting noise that represents more common anomaly patterns, we prevent the model from overfitting nomain-specific anomaly patterns that may vary across domains. We design an adversarial training mechanism for the encoder and the anomaly decoder on the reconstruction of abnormal series, which learns explicit decision boundaries between normal time series and common anomalies during the multi-domain training, thereby improving anomaly detection capability across different scenarios. Our main contributions are summarized follows:
We propose DADA, a novel General Time Series Anomaly Detector with Adaptive Bot- tlenecks and Dual Adversarial Decoders. By pre-training on multi-domain time series data, we achieve the goal of "one-model-for-manee anomaly detection on various target scenarios efficiently without domain-specifie training.
meaning single model can perform
The proposed Adaptive Bottlenecks is the first to consider model's generalization ability from the perspective of the dynamic bottleneck in time series anomaly detection, which adaptively addresses the flexible reconstruction requirements of multi-domain data.
The proposed Dual Adversarial Decoders explicitly amplify the decision boundaries be- tween normal time series and common anomalies in an adversarial training way, which improves the general anomaly detection capability across different scenarios.
Our model acts zero-shot time series anomaly detector that achieves competitive su- perior performance on various downstream datasets, compared with state-of-the-ar models trained specifically for each dataset.
2 RELATED WORK
Time series anomaly detection. Time series anomaly detection methods mainly be catego- rized into none-learning, classical learning, and deep learning ones Zhao et al. (2022). Non-learning methods include density-based methods (Breunig al. 2000) that determine which data points are abnormal by analyzing the density of the distribution of data points in the cluster, and similarity- based methods (Ych al. 2016) that series significantly dissimilar from most series are marked as abnormal. Classical learning methods (Liu et al., 2008) use training dataset consisting of only normal data to classify whether the test data is similar the normal data or not (Schölkopf et al,) 1999). Deep learning methods mainly contain reconstruction-ben
and prediction- The reconstruction--asse methods compress the raw input data and then attempt to reconstruct this representation, the abnormal data is often difficult to reconstruct correctly (Zhao et al. 2022] Wu
methods.
et al. 2025). The classic approaches include the use of AEs in (Campos et al., [022]] Krizhevsky et al. 2012). VAEs in (Park et al. 2018 Li Li al. (0221b GANs in (Li et al. 2019 Schlegl et al. 2019) and more recently, the highly successful utilization of transformer architectures in (Chen al. 2022 Hu et al.] 2024). The prediction-based methods (Pang al., 2022) use past observa- tions to predict the current value and use the difference between the predicted result and the real value an anomaly criterion. Unlike these methods that require building a model for each dataset, our method mainly focuses on constructing general time series anomaly detection model by pre- training on multi-domain time series data, which can be efficiently applied various target datasets.
Time series pre-training models. Time series pre-training has received extensive attention. Some works develop general training strategies for time series. PatchTST (Nie et al., 2023) and SimMTM (Dong et al. 2023b) employ the BERT--tyle masked pre-training. InfoTS (Luo et al. 2023] and TS2Vec Yuee al. 2022) utilize contrastive learning between augmented views of data. Additionally. Some works repurpose large language models to time series tasks (Gruver et al.) 2023),
such as GPT4TS (Zhou et al. 2023). Recently, novel time series pre-training models with zero- shot
forecasting capabilities have emerged (Gazza & Canseco) 2023 Woo et al., 2024 Liu et al. 2024a Dooley et al.) 2023 Liu et al. 2024b; Gruver et al. 2023), which demonstrate remarkable perfor- mance even without training on the target datasets. Nonetheless, there remains a dearth of time series anomaly detection models that support zero-shot. Existing pre-training methods for anomaly detection necessitate training on the target dataset. Different from previous methods, DADA pre- trained on multi- datasets is uniquely designed for time series anomaly detection, excelling in zhoo anomaly detection across various target scenarios, thus filling this gap in the field.
3

---

Published as a conference paper at ICLR 2025
(a) Pretrain
(b) Adaptive Battlenecks
[c) Inference
Nermal Decoder
Anormaly Decoder
Fase
Pretroined Normal Decoder
Brodent Reveras
00000
Bottleneck Pool
2.29
00000
Pretroined Adoptive Boftlenecks
Adoptive Bottlenecks
0.999.
Pretroined Encoder
Encoder
Topnk
Router
Patch Complementory Mask
Patch & Multi-Comllsmentoyy Mask
- Manu
Luyee 4 -on A.
Represethotion
Complementary pair
Time Series
Time Series
Figure 2: (a) The workflow during the pre-training stage. DADA mainly consists of Patch and Complementary Mask, Encoder, Adaptive Bottlenecks, and Dual Adversarial Decoders, (b) The structure of Adaptive Bottlenecks. (c) The workflow during the inference stage.
3 METHODOLOGGY
RT containing T successive observations, where is the data dimensions. Time series anomaly detection outputs Diem = (y1, /2. ... ,YT), where Y1 {0, denoting whether the observation x; E RC at certain time f is anoma- lous or not. In this paper, we focus on building a general time series anomaly detection model that pre-trained on M multi-domain time series datasets D (D( JM where each dataset D( x X2 ....,...'. x((1)) E ITT00 xc(() has (+) vvriates and T time points. Then, the pre-
Given time series Diest (X1. X...........00 .T)
trained model can be applied for anomaly detection target downstream dataset Diest D without any fine-tuning. Time series from different domains have varying numbers of variates. Thus, we use channel independence
extend
model across different domains (Nie al. [023] Yang al.,
2023
Goswami et al. 2024; Liu et al., [024bb Wang et al., 2024). Specifically, C-variate time series processed into C separate univariate time series.
3. OVERALL ARCHITECTURE
We propose DADA, novel general time series anomaly Detector with Adaptive bottlenecks and Dual Adversarial decoders. As shown in Figure 2 DADA employs a mask-based reconstruction architecture model time series. During the pre-training stage, we utilize normal series, where no time points are labeled as anomalies, and abnormal series, which contains noise perturbation. Both normal and abnormal time series are concurrently input DADA. The patch and complementary mask module maps each series to masked patch embeddings, which are fed into the encoder to extract temporal features. DADA employs adaptive bottlenecks for multi-domain time series data by dynamically selecting suitable bottleneck sizes. Finally, the dual adversarial decoders are employed to reconstruct the normal and abnormal series. During the inference stage, the anomaly decoder removed, and DADA employs multiple complementary masks to produce multiple pairs of masked patch embeddings. The variance the reconstructed values serves as the anomaly score, where the normal data be stably reconstructed.
3.2 COMPLEMENTARY MASK MODELING
Time series mask modeling reconstructs masked parts using unmasked parts, to capture normal temporal dependencies from unmasked parts to masked parts. High reconstruction errors indicate anomalies that deviate from normal behavior. In this paper, employ complementary mask strategy by generating pair of masked series with mutually complementary mask positions, which reconstruct each other to further capture comprehensive bi-directional temporal dependencies.

---

Published as a conference paper at ICLR 2025
The complementary mask modeling consists of three steps. First, We use patch embedding to divide the univariate imput time series into P patches, each of dimension d. denoted as X E R2xd, The division of time series data with patches has proven to be very helpful for capturing local information within each patch and learning global information among different patches, enabling the capture of complex temporal patterns (Nie al. 2023] Yang et al. (023). Then, we generate masked series by randomly masking a portion of patches along the temporal dimension, formalized by Xm = M X, where M E {0, 1 Px1 is the mask and @ is the element-wiss multiplication. At the same time, we generate another masked time series that complements X according to the mask M, formalized by Xm (1- M) C X. Then, we reconstruct the masked parts, Xm. based on Xm. and vice versa. This provides the model fully utilizing all data points to capture comprehensive bi-directional temporal dependencies. Finally, we combine the complementary reconstructed results. Denoting the process of reconstruction as Recon(.) = Decoder( AdaBN((ncoder())) ) it passes its input through the Encoder, AdaBN, and Decoder to obtain the reconstruction results. The AdaBN and Decoder is detailed in Section 3.3 and Section 3.4 The final reconstructed results X is calculated by:
X = (1 - M) @ Recon (Xm) + M @ Recon (m))
3.3 ADAPTIVE BOTTLENECKS
As aforementioned. DADA is pre-trained on multi-domain time series datasets and applied to wide range of downstream scenarios. This requires DADA to have the ability to learn generaliz- able representations from multi-domain time series datasets that show significant differences in data distribution, noise levels, etc. (Woo et al., (024).. and exhibit distinct preferences for the bottle- neck. The size of the latent space can be considered as the internal information bottleneck of the model (Wang et al. 2023a). A large bottleneck may cause the model to fit unnecessary noise, while a small bottleneck may result in the loss of diverse normal patterns. The fixed bottleneck approaches used by existing methods (Campos et al.) 2022 Su et al., 2019 Sakurada & Yairi, 2014) are poor in generalization ability and fail to detect anomalies across domains. Thus, we innovatively con- sider the model's 's generalization ability from the perspective of dynamic bottleneck and propose an Adaptive Bottlenecks module (AdaBN) integrated with an adaptive router and a bottleneck pool, which dynamically allocates appropriate bottlenecks for multi-domain time series, enhancing the generalization ability of the model and enabling it to directly apply in various target scenarios.
We compress the features into different latent spaces achieve different infor- mation bottlenecks with different information densities. To accommodate the varying requirements of the multi-domain time series data, we have configured a pool of B different sizes of bottlenecks BN,()), i = 1,2, ..., B, and each of them can compress the features into different-sized latent space with dimension d,. The masked time series input encoder can generate the corresponding representation E Rd, where d, denotes the dimension of representation. The process of each bottleneck can be formalized as:
Bottleneck pool.
(z) UpNet, (DownNet, (z)).
(-w,Net,(-) represents the network which compresses the representation z into a latent space, Rd. + Rd, where di is the latent space dimension for the i-th bottleneck (-) and de < dr- UpNet,()) restores the representation from the latent space, Rd - R2,
Employing all bottlenecks within the bottleneck pool indiscriminately will di- minish model performance due to the mismatching between the information capacity of the latent space and the information densities of the data. An ideal model would dynamically allocate appro- priate bottlenecks based on the intrinsic properties of the time series data. Therefore, we introduce the Adaptive Router, a dynamic allocation strategy that flexibly select bottleneck size for each time series to address the flexible reconstruction requirements. As shown in Figure 1b), the adap- tive router employs a routing function generate the selecting weights of each bottleneck from the bottleneck pool based on the representation. To avoid repeatedly selecting certain bottlenecks, causing the corresponding bottlenecks be repeatedly updated while neglecting other potentially suitable bottlenecks, we add noise terms to increase randomness. We obtain the overall formula for the routing function R(z) as:
Adaptive router.
R (2) z W nutter + E Softplus (2W misee) : N (0, 1),
(3)
5

---

Published as a conference paper at ICLR 2025
where W router and noise E R are learnable matrices for weights generation, B is the number of bottlenecks and Softplus is the activation function. R(z) maps representation z to selection weights for B bottlenecks. To encourage the model to update key bottlenecks, the highest weights, denoting the set their indexes as K. Last, we assign higher attention weights the output of more important bottlenecks and fuse their final output as:
select & bottlenecks with
exp (R (z),)
AdaBN(z)
BN, (z).
(4)
SE Σ İjEK JEX exp (R()))
3.4 DUAL ADVERSARIAL DECODERS
Reconstruction of normal time series. As shown in Eq. 1 each input series X is reconstructed into result X. To achieve the goal of anomaly detection, we learn normal patterns from the recon- struction of normal time series. As shown Figure we use a feature extractor G(-; 0g), and a normal decoder (- structing normal series. The G includes the patch and complementary mask module, the encoder, and the adaptive bottlenecks. The decoder D, uses the output from the adaptive bottlenecks construct the series. We learn normal time series patterns by minimizing the reconstruction error of normal series:
(-; 0x)) with parameter 9g and 8n to denote the parts of the model used for recon-
N. Σ
No
Co 0m)
X
Σ
O
(5)
where N₂ the number of normal training series. X is the i-th normal series and X (0) is the corresponding reconstruction result.
Reconstruction of abnormal time series. However, merely learning to capture normal patterns is insufficient detect anomalies in new scenarios. As a general time series anomaly detection model, the model will confront multi-domain data with diverse anomaly manifestations, and some patterns may only be specific to certain do- main and fail to generalize across domains, making the decision boundaries more complex. Thus, need to plicitly enhance the model's ability to discriminate be- tween normal patterns and some common abnormal pat- terns. We incorporate abnormal series with noise per- turbations representing common anomaly patterns into the pre-training stage and propose an adversarial train- ing stage that minimizes reconstruction errors for normal series while maximizing them for abnormal ones. Sim- ply amplifying the reconstruction error for anomalies can confuse the feature extractor G' and fail to learn normal patterns. To address this, we introduce an anomaly decoder Da(::0a) with parameter 6g for adversarial training to constrain the model and encourage the feature extractor G(:;8, fg) to learn normal pattern features. We seek the parameters fg of the feature extractor that maximizes the reconstruction loss of the abnormal data to make the representations contain abnormal information little possible, while simultaneously seeking the parameters 0, of the anomaly decoder that minimizes the reconstruction loss of the abnormal data learn anomaly patterns. The reconstruction loss for abnormal series can be formalized as:
a44 06g
Lhg 00,
I.
Dx(::
G(
Data
dhg 2eg
-
Da 0,)
Irspiption Pesiiation
Gradient Bevees
a
La
a0
Figure 3: The forward and backward propagation of normal data and abnor- mal data.
N2 Σ
C,(0g, 0n)
x
Σ
1 (6)
where N₂ is the number of abnormal training series, x (6) and (1) denote the i-th abnormal series and its anomaly label. Our method does not strictly require labeled data. To reduce dependence on data and to avoid overfitting to specific anomaly patterns, generate anomalous data with more common anomaly patterns through anomaly injection. More details about anomaly injection shown in Appendix A.3

---

Published as a conference paper at ICLR 2025
Model training. As aforementioned, the overall optimization goal is seek the parameters 8g of feature extractor that maximize Eq. (6). while simultaneously seeking the parameters fg of anomaly decoder that minimize Eq. 16. In addition,
seek to minimize Eq. (5). As shown in Figure [3]
we employ a Gradient Reversal Layer (GRLL) (Ganin & Lempitsky 2015) between G and Da GRT. alters the gradient from Da multiplies it by -X, and passes it G. That is say, the partial derivatives of the loss for the encoder dca 880, is replaced with - ac= guiding parameters of different 86,"
optimization objectives toward the desired direction for gradient descent, thereby avoiding the need for two separate optimizations. Finally, our objective can be formalized by:
(0g (%) arg min Ln(Og, 0n) Ca 0a),
(7)
0% arg min La (0 0.)
8a
Anomaly criterion. During the inference stage, the model generates multiple pairs of complemen- tary masked series for single series and outputs multiple reconstructed series. Normal data points can be stably reconstructed. with the higher proximity of the reconstruction values. Conversely, reconstructions of abnormal data points are difficult and tend to be more unstable. Therefore, we utilize the variance of reconstructed values at the same time point as the anomaly score. Following existing works (Wang et al. [223a; (Su et al., 2019)), we run the SPOT (Siffer et al., (211)) to get the threshold 8, and time point is marked as an anomaly if its anomaly score is larger than 8.
EXPERIMENTS
4.1
EXPERIMENTAL SETTINGS
Datasets.
We employ a subset of the Monash (Godahewa et al. (022 data hub and 12 datasets (Li
ct al. 2021a Jacob et al. 2021 Ren et al. 2019 Roggen et al., 2010 Cui et al. 2016 Thill et al.
2020
Moody & Mark 2001 Greenwald et al. 1990; Laptev et al., 2015) from anomaly detection
task for pre-training. The pre-training datasets consist of approximately 400 million time points and intricately represent wide range of domains. To demonstrate the effectiveness of our method, we evaluate five widely used benchmark datasets: SMD (Su et al., 2019). MSL (Hundman et al. 2018), SMAP (Hundman et al. 2018). SWaT (Mathur & 'Tippenhauer (016)), PSM (Abdulaat 2021]. and NewrIPS-TS (including CICIDS, Creditcard, GECCO and SWAN) (Lai et al.) 2021). We also evaluate UCR (Wu & Keogh 2023) in the Appendix C.2 None of the downstream datasets for evaluation are included in the pre-training datasets. More details about the pre-training and evaluation datasets are shown in Appendix A
al.
Baselines. We compare our model with 19 baselines for comprehensive evaluations, including the linear transformation- based models: OCSVM (Schölkopf et al.) 1999), PCA (Shyu et al., 2003); the density estimation-ban based methods: HBOS (Goldstein & Dengel, 2012). LOF (Breunig) ct al. 2000); the outleer-based methods: IForest (Liu et al. 20088). LODA (Pevn)) (016); the neu- ral network-based models: AutoEncoder (AE) (Sakurada & Yair 2014). DAGMM (Zong et al.) LSTM (Hundman et al.] (018)), AAE-Ensemble (Campos et al. (002).. BeatGAN (Zhou
2018).
al.
2019), OmniAnomaly (Omni) Suu et al. 2019), Anomaly Transtormer (A.T.) (Xu et al. MEMTO (Song et al., 2024). DCCetector (Yang et al., 2023). D3R (Wang et al. 2003a
2022
GPT4TS
2023), MCdemTTN (donghao & wang xue 2024). SensitiveHUE (Feng al.) To ensure fair comparison, we adopt the same settings across all baselines, including the
2024).
metrics and thresholding protocol, following common practices in anomaly detection.
Metries. Many existing methods use Point Adjustments (PA) to adjust the detection result. How- ever, recent works have demonstrated that PA can lead to faulty performance evaluations (Huet et al. 2022). Even if only one point in an anomaly segment is correctly detected, PA will assume that the model has detected the entire segment correctly, which is unreasonable (Wang et al. 2023a). To overcome this problem, we use the affiliiaton-based Fl-score (FI) (Huet et al. 2022). which has been widely used recently (Wang et al. 2023a Yang et al. 2023). This score takes into account the average directed distance between predicted anomalies and ground truth anomaly events to calculate affiliated precision (P) and recall (R). Since the precision and recall are significantly influenced by thresholds, focusing solely on one of them fails to provide a comprehensive evaluation of the model,
7

---

Published as a conference paper at ICLR 2025
Table I: Results for five real-world datasets.
Dataset
SMD
MSL
SMAP
SWaT
PSM
Metrie
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
OCSVM
66.98
82.05
73:75
50.26
99.86
66.
41.05
69.337
51.38
56. 56.80
98.72
72.11
57.51
58. 11
57.81
PCA
64.92
86.06
74.01
52.69
98.33
68.
50.62
98.48
66.87
62.32
82.96
71.18
77.44
68
69
HBOS
60.34
64.11
62.17
59.25
83.32
69.25
41.54
66.17
51.04
54.49
91.35
68, 26
78.45
29.82
43.21
LOF
57.69
99. 10
72.92
49.89
12.18
59.00
47.92
82.86
60.72
53.20
96.73
68.65
53.90
99.91
70.02
Fores
71.94
94.27
81.61
53.87
94.58
68.65
41.12
68.91
$1.51
53.03
99.95
69.30
69.66
88,
78.07
LODA
66.09
84.37
74.12
57.79
95.65
72.05
51.51
100.00
68.00
56.30
70.34
62.54
62.22
87.38
72.69
69.22
98
81
70.
75.01
DAGMM
63.57
70.83
67.00
54,07
92.11
68.1
50.75
96.38
66.49
59,42
92.36
72.32
68.22
70.50
69.34
LSTM
60.12
81.77
70.35
58.82
14.68
23.49
55.25
27.70
36.90
49.99
82.11
62.15
57.06
95.92
71.55
BeMGAN
74.11
81.64
77.69
55.74
98.94
71.30
54,04
98.30
69.74
61.89
$3.46
71.08
58.81
99.08
73.81
79.09
75.77
77.40
67.61
98.51
68.70
62.76
80.79
74.55
CAE Ensemble
73.05
83:61
77.97
51,99
93.93
69.37
62.32
64.72
63.50
62.10
$2.90
71.01
73.17
73.66
73.42
MEMTO
49.69
98.05
$5.996
52.73
97.34
68.40
50.12
99.10
66.57
56.47
98.02
71.66
52.69
83.94
64.74
A.T.
54.08
97.07
69:46
51.04
95.36
66.49
56.91
96.69
71.65
53.63
98.27
69.39
54.26
82. 18
65.37
DCdetector
50.93
95.57
66.45
55,94
95.53
70.56
53.12
98,
68.99
53.25
98.12
69:03
54.72
86.36
66.99
SensidiveUUE
60.34
90.13
72.29
55,92
98.95
71.46
53.63
98.37
69.42
58,91
91.71
71.74
56.15
98.75
71.59
D3R
64.87
97.93
78.04
66.85
90.83
77.02
61.76
92.55
74,09
60.14
97.57
74.41
73.32
88.71
80.29
ModemTCNN
74.07
94.79
83.16
65.94
93.00
77.17
69.50
65.45
67.41
59.14
89.22
71.13
73.47
86.83
79.59
GPT4TS
73.33
95.97
83.14
64.86
95.43
77.23
63.52
90.56
74.67
56.84
91.46
70.11
73.61
91.13
81.44
(scro shot)
76.50
94.54
68,70
91.51
78.48
88.25
75.42
61.59
94.59
74.31
92. 11
82.26
Table Overall results for the NeurlPS-TS datasets.
Dataset
CICIDS
Creditcard
GECCO
SWAN
Metrie
Fl
AUC
Fl
AUC
F1
AUC
F1
AUC
A.T.
34.71
49.00
65.14
52.55
64.27
51.60
33.67
44.74
DCdetector
40.02
53.95
58.28
42.36
66.18
45.38
14.42
43.48
D3R
67.79
41.99
72.03
93.59
91.83
80.72
43,19
37.31
ModemTCN
51.74
65.33
73.80
95.55
90.18
95.95
46.45
52.63.
GPT4TS
54,00
67.91
72.88
95.58
88.11
90.21
47.27
$1.93
DADA (zero shot)
73.49
69.33
75.12
95.73
90.20
93.44
71.93
53.29
and thus anomaly detection requires balancing these two metrics. Therefore, we pay more attention to the FI score like widely used in anomaly detection Yang et al. (2023); Xu et al. (022). We also employed the AUC ROC (AUC) metric (Wang al.) 2023a Campos et al. 2022). More implement details are shown in the Appendix B ones are underlined.
All results are in %. The best ones are in bold and the second
4.2
MAIN RESULTS
Each evaluation dataset has a training set and a test set. The standard baseline methods are trained on the training set and tested on the corresponding test set. Differently, our model DADA follows the zero-shot protocol. After the model is pre-trained on multi-domain datasets, it does not train on the downstream training set and is directly tested on the downstream test set. Note that all downstream evaluation datasets are not included our pre-training datasets. As shown in Table I the baselines directly trained with full data in each specific dataset, DADA, as a zero-shot detector, achieves ofate-of-the-ant art results in all five datasets, which demonstrates that DADA learned general detection ability from wide range of pre-training data, with clear distinguishment between multiple normal and anomalous patterns. We also evaluate our DADA on the NeurlPS- TS benchmark in Table 2 compared with the five recent methods that perform well as shown in Table T It can observed that these baselines fail to achieve consistent good results across different datasets. However, benefiting from pre-training on large-scale data, DADA achieves the best most competitive results all datasets, including the NeurIPS-TS datasets which are rich in anomaly types (Xu et al. 2022 Yang al. 2023 It demonstrates the superior adaptability of DADA for different anomaly detection scenarios. More other metrics evaluation details can be found in Appendix CC.
compared to
Ablation study. We further delve into the impact of individual components on the performance of DADA in zero-shot setting, shown in Table 3 Firstly, explore the impact of the AdaBN module. The first line results use a single fixed bottleneck for multi-domain datasets, eliminating the adaptive selection for various bottlenecks. When remove the AdaBN module, the outcomes exhib-
8

---

Published as a conference paper at ICLR 2025
Table 3: Ablation studies. We study the influence of adaptive bottlenecks, adversarial training, and dual decoders. All results are in %, and the best in bold.
Dataset
SMD
MSL.
SMAP
Avg
AdaBN
Adversarial
Dual Decoders
P
F1
71.13
93.61
80.84
62.79
91.74
74.55
58.91
84.10
69.29
74.45
69.46
55.32
61.59
49.89
88.57
63.83
52.62
82.59
64.28
63.23
x
75.77
87.82
81.35
64.82
84.32
73.29
61.50
80.27
69.64
74.76
X
89.95
67.54
76.63
65.65
X 60
74.96
76.74
72.69
68.74
55.22
68.03
71.17
76.50
94,54
84.57
68.7
91.51
78.48
65.85
88.25
75.42
79.49
ited a decline of 5.04%. This emphasizes the criticality of dynamic bottlenecks for multi-domains pre-training. We also examine the influence of the dual adversarial decoders and its adversarial mechanism. The second line results directly maximize the reconstruction error of abnormal data, removing the adversarial mechanism. The third line results use single decoder to reconstruct nor- mal time series and abnormal time series, and the fourth line results simultaneously remove the adversarial mechanism and use a single decoder. We observe degradation in performance of w/o Adversarial by 16.26% compared with the fifth line results, suggesting the needs for confrontation with the feature extractor, and when directly maximizing abnormal time series, the abnormal decoder can achieve this goal by generating arbitrary reconstruction results and undermine the encoder's ability to model normal pat- terns. Besides, employing single decoder handle both normal and abnormal time series causes zero-shot performance to degrade, instructing the necessity of dual decoders.
compared with DADA, and 7.89% (71.17%-63.
4.3 MODEL ANALYSIS
We analyze the effectiveness of adaptive bottlenecks and multi- pre-training and visualize the anomaly scores. We conducted additional analytical experiments, and the results are presented in Appendix D and Appendix
Finetune with downstream data. As de- picted Figure 4 we further demonstrate the results of fine-tuning DADA downstream datasets under different data scarcities. After pre-training, the model develops strong gen- cralization capabilities and exhibits promising zero-shot anomaly detection performance. By applying the model to a downstream dataset for further learning of domain--pecifee patterns, the model's detection ability can be further en- hanced. After fine-tuning, the model's perfor- mance increases from 78. 18.48% to 79.90% on MSL, and from 84.57% to 85,50% on SMD.
MSL
SMD
85.5
78.4
84.0
10%
50%
100%
Data Percentage
Data Percentage
Figure 4: Results of fine-tuning DADA under dif- ferent data percentage.
Analysis on adaptive bottlenecks. To validate the effectiveness of AdaBN, we remove the Ad- aBN module and instead employ fixed bottleneck size. As shown in Figure 5eefft) we observe that different bottleneck sizes exhibit differrnt preferences for various downstream scenarios, while also introducing notable drawbacks in certain scenarios, e.g. the model with bottleneck size of 32 achieves 74.81% on Creditcard, whereas only 67.56% on MSL. However, DADA, with adaptive bottlenecks dynamically allocating appropriate bottlenecks, consistently outperforms models utiliz- ing a single bottleneck, which further substantiates the efficacy of our method. We also visualize the proportion of different adaptive bottleneck sizes that each dataset selects in Appendix E.5
Analysis on multi-domain pre-training. We further evaluate the baseline's zero-shot perfor- mance after multi-domain pre-training. Adopting the same setting as DADA, these models are first pre-trained extensively multi-domain datasets, followed by zero-shot evaluation the tar- get datasets. From the presented Figure Fiightt) we can observe that these baselines are incapable of effectively extracting generalization capability from multi-domain datasets, thus failing deliver

---

Published as a conference paper at ICLR 2025
GIGATS
6B000
SMAP
DADA
30
512
Creditcard
MSL
SMD
SMAP
SWAT
Figure 5: Model Analysis. (left) Comparison between DADA using AdaBN and using single bottleneck. (right) Pre-training baseline model on multi-domain datasets and then evaluate them as zero-shot detector on the target dataset.
Negelive
Fabse Pasttive
04
64
03
m
03
200
20
100
DADA (Ours)
DADA (W/O AdaBN)
DADA (W/ Dual Decoders)
Figure 6: Visualization of the anomaly scores on PSM dataset. The pink intervals denote the ground truth anomalies, while the blue lines represent the anomaly scores produced by corresponding mod- els. A red horizontal dashed line signifies the threshold. Any timestamp with an anomaly score exceeding this threshold identified as an anomaly.
satisfactory results on the target dataset. This indicates that although multi-domain pre-training is important, simply using multi-domain datasets without proper methods still cannot obtain desired generalization ability. In contrast, DADA, equipped with unique adaptive bottlenecks and dual adversarial decoders module tailored for the task of general anomaly detection, demonstrates great multi-domain pre-training and general detection ability.
Visual analysis. To further explain the efficacy of each module in DADA, we randomly sample time segments of length 100 for visualization. Gilustaess the anomaly scores at every times- tamp within given period and ground truth labels. After DADA removes AdaBN (W/O AdaBN), the model cannot adapt well to the normal pattern of multi-domain data and gets a higher false positive. After removing the dual decoders (W/O Dual Decoders), the model's decision boundary between normal and abnormal data becomes blurred and the false negative becomes higher. DADA (ours) shows that DADA not only achieves superior detection accuracy but also maintains lower false alarm rate, testament its refined performance in time series anomaly detection.
CONCLUSION
This paper proposes novel general time series anomaly detection model DADA. By pre-training on extensive multi-domain datasets, DADA can subsequently apply to a multitude downstream scenarios without fine-tuning. We propose adaptive bottlenecks meet the diverse requirements of information bottlenecks tailored to different datasets in one unified model. We also propose dual adversarial decoders explicitly enhance clear differentiation between normal and abnormal patterns. Extensive experiments prove that as a zero-shot detector, DADA still achieves competitive or even superior results compared to those models tailored each specific dataset.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by National Natural Science Foundation of China (62372179, 62406112).
REFERENCES
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In KDD, 2021.
Archana Anandakrishnan, Senthil Kumar, Alexander R. Statnikov, Tanveer A. Faruquie, and Di Xu. Anomaly detection in finance: Editors' introduction. In ADF KDD, 2017.
Markus M. Breunig. Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. LOF: identifying density-based local outliers. In SIGMOD, 2000.
David Campos, Tung Kieu, Chenjuan Guo, Feiteng Huang, Kai Zheng, Bin Yang, and Christian S. Jensen. Unsupervised time series outlier detection with diversity driven convolutional ensembles. PVLDB, ((33):611-623) 2022.
Raghavendra Chalapathy and Sanjay Chawla. Deep learning for anomaly detection: A survey. CoRR, 2019.
Zekai Chen, Dingshuo Chen, Xiao Zhang, Zixuan Yuan, and Xiuzhen Cheng. Learning graph structures with transformer for multivariate time-series anomaly detection in iot. IEEE Internet Things J., 2022.
Andrew A. Cook, Goksel Misirli, and Zhong Fan. Anomaly detection for iot time-series data; A survey, IEEE Internet Things J., 2020.
Yuwei Cui, Chetan Surpur, Subutai Ahmad, and Jeff Hawkins. A comparative study of HTM and other neural network models for online sequence learning with streaming data. In IJCN, 2016.
Feng Dong, Liu Wang, Xu Nie, Fei Shao, Haoyu Wang, Ding Li, Xiapu Luo, and Xusheng Xiao. DISTDET: A cost-effective distributed cyber threat detection system. In USENIX Security Sym- posium, 2023a.
Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang. Jianmin Wang, and Mingsheng Long. Simmtm: A simple pre-training framework for masked time-series modeling. NewrlPS, 2023b.
Luo donghao and wang xue. Modem1TCN: A modern pure convolution structure for general time series analysis. ICIR, 2024.
Samuel Dooley, Gurnoor Singh Khurana, Chirag Mohapatra, Siddartha Naidu, and Colin White. Forecastpfn: Synthetically-traine zero-shot forecasting. In NeurlPS, 2023.
Yuye Feng, Wei Zhang, Yao Fu, Weihao Jiang, Jiang Zhu, and Wenqi Ren. Sensitivehue: Multivari- ate time series anomaly detection by enhancing the sensitivity to normal patterns. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 782-793, 2024.
Yaroslav Ganin and Vietor S. Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, 2015.
Azul Garza and Max Mergenthaler Canseco. Timegpt-1. CoRR, 2023.
Rakshitha Godahewa, Christoph Bergmeir, Geoffrey I. Webb, Rob J. Hyndman, and Pablo Montero- Manso. Monash time series forecasting archive. In NeurlPS Datasets and Benchmarks, 2021.
Markus Goldstein and Andreas Dengel. Histogram-based outlier score (hbos): A fast unsupervised anomaly detection algorithm. K1-20/2: poster and demo track, 2012.
Mononito
Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. MOMENT: A family of open time-series foundation models. In ICML, 2024.

---

Published as a conference paper at ICLR 2025
S. D. Greenwald, Ramesh S. Patil, and Roger G. Mark. Improved detection and classification of arrhythmias corse-corrupte electrocardiograms using contextual information. [1990] Pro- ceedings Computers in Cardiology, 1990.
Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Gordon Wilson. Large language models zero-shot time series forecasters. In NeurlPS, 2023.
Shiyan Hu, Kai Zhao, Xiangfei Qiu, Yang Shu, Jilin Hu, Bin Yang, and Chenjuan Guo. Multire: Joint learning for time series anomaly prediction and detection with multi-scale reconstructive contrast. arXiv preprint arXiv:2410. 15997, 2024.
Alexis Huet, José Manuel Navarro, and Dario Rossi. Local evaluation of time series anomaly detec- tion algorithms. In KDD, 2022.
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Söderström. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In KDD, 2018.
Vincent Jacob, Fei Song, Arnaud Stiegler, Bijan Rad, Yanlei Diao, and Nesime Tatbul. Exathlon: A benchmark for explainable anomaly detection over time series. VLDB, 2021.
Robert A. Jacobs, Michael Jordan, Steven J. Nowlan, and Geoffrey Hinton. Adaptive mixtures of local experts. Neural Comput., 1991.
Kenji Kawaguchi, Zhun Deng. Xu Ji, and Jiaoyang Huang. How does information bottleneck help deep learning? In ICML, 2023.
Tung Kieu, Bin Yang, Chenjuan Guo, Razvan-Gabrin Cirstea, Yan Zhao, Yale Song, and Chris-
tian S. Jensen. Anomaly detection in time series with robust variational quasi-recurrent autoen- coders. In ICDE, 2022.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convo- lutional neural networks. In NeurlPS, 2012.
Kwei- Lai, Daochen Zha, Junjie Xu, Yue Zhao, Guanchu Wang. and Xia Ben Hu. Revisiting time series outlier detection: Definitions and benchmarks. In NeurlPS Datasets and Benchmarks, 2021.
N Laptev, S Amizadeh, and Y Billawala. S5-a labeled anomaly detection dataset, version 1.0 (16m), 2015.
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. MAD-GAN: mul- tivariate anomaly detection for time series data with generative adversarial networks. In ICANN, 2019.
Zhihan Li, Youjian Zhao, Jiaqi Han, Ya Su, Rui Jiao, Xidao Wen, and Dan Pei. Multivariate time se- ries anomaly detection and interpretation using hierarchical metric and temporal embedding. In KDD, 2021a.
Zhihan Li, Youjian Zhao, Jiaqi Han, Ya Su, Rui Jiao, Xidao Wen, and Dan Pei. Multivariate time se- ries anomaly detection and interpretation using hierarchical inter-metric and temporal embedding. In KDD, 2021b.
Fei Tony Liu, Kai Ming Ting, and Zhi-Ho Hua Zhou. Isolation forest. In ICDM, 2008
Xu Liu, Junfeng Hu, Yuan Li, Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmermann. Unitime: A language-empowere unified model for cross-domain time series forecasting. In www, 2024a.
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Transformers for time series analysis at scale. CORR, 2024b.
Dongsheng Luo, Wei Cheng, Yingheng Wang. Dongkuan Xu, Jingchao Ni, Wenchao Yu, Xuchao Zhang, Yanchi Liu, Yuncong Chen, Haifeng Chen, and Xiang Zhang. Time series contrastive learning with information-awarce augmentations. In AAAI, 2023.
12

---

Published as a conference paper at ICLR 2025
Aditya P. Mathur and Nils Ole Tippenhauer. Swat: water treatment testbed for research and training on ICS security. In CySWATER CPSWeek, 2016.
George B. Moody and Roger G. Mark. The impact of the mit-bih arrhythmia database. 1EEE Engineering in Medicine and Biology Magazine, 2001.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In ICLR, 2023.
Zhicheng Pan, Yihang Wang, Yingying Zhang, Sean Bin Yang, Yunyao Cheng, Peng Chen, Chen- juan Guo, Qingsong Wen, Xiduo Tian, Yunliang Dou, et al. Magicscaler: Uncertainty-aware, predictive autoscaling. VIDB,, 16 16(12)3388-3881, 2023.
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton van den Hengel. Deep learning for anomaly detection: review. ACM Comput. Surv. 2022.
John Paparrizos, Paul Boniol, Themis Palpanas, Ruey S Tsay. Aaron Elmore, and Michael J Franklin. Volume under the surface: new accuracy evaluation measure for time-series anomaly detection. VLDB, 15(1 1):2774-2787. 2022.
Daehyung Park, Yuuna Hoshi, and Charles C. Kemp. A multimodal anomaly detector for robot- assisted feeding using an Istm-based variational autoencoder. IEEE Robotics Autom. Lett., 2018 8.
Tomás Pevný. Loda: Lightweight on-line detector of anomalies. Mach. Learn., 2016
Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoy- ing Zhou, Christian Jensen, Zhenli Sheng, and Bin Yang. Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods. VLDB, 17(9):2333-2377, 2024.
Xiangfei Qiu, Xiuwen Li, Ruiyang Pang. Zhicheng Pan. Xingjaan Wu, Liu Yang. Jilin Hu, Yang Shu, Xuesong Lu, Chengeheng Yang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, and Bin Yang. Easytime: Time series forecasting made casy. ICDE, 2025a.
Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting. In SIGKDD, 2025b.
Hansheng Ren, Bixiong Xu, Yujing Wang, Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing, Mao Yang, Jie Tong, and Qi Zhang. Time-series anomaly detection service at microsoft. In KDD, 2019.
Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Kilian Förster, Gerhard Tröster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, Jakob Doppler, Clemens Holzmann, Mare Kurz, Gerald Holl, Ricardo Chavarriaga, Hesam Sagha, Hamidreza Bayati, Marco Creatura, and José del R. Millán. Collecting complex activity datasets in highly rich networked sensor environments. In INSS, 2010.
Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with nonlinear dimen- sionality reduction. In MLSDA @ PRICAI, 2014.
Osman Salem, Khalid Alsubhi, Ahmed Mehaoua, and Raouf Boutaba. Markov models for anomaly detection in wireless body area networks for secure health monitoring. IEEE J. Sel. Areas Com- mun., 2021.
Thomas Schlegl, Philipp Seeböck, Sebastian M. Waldstein, Georg Langs, and Ursula Schmidt- Erfurth. f-anogan: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Anal., 2019.
Sebastian Schmidl, Felix Naumann, and Thorsten Papenbrock. Autotsad: Unsupervised holistic anomaly detection for time series data. VLDBB 17(1))2987-3002, 2024.
Bernhard Schölkopf, Robert C. Williamson, Alexander J. Smola, John Shawe- TTalee, and John C. Platt. Support vector method for novelty detection. In NewrIPS, 1999.
13

---

Published as a conference paper at ICLR 2025
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gate mixture-of-ee ef-eeertss layer. arXiv preprint arXhv:1701.06338 2017.
Lifeng Shen, Zhuocong Li, and James T. Kwok. Timeseries anomaly detection using temporal hierarchical one-class network. NeurIPS, 2020.
Mei-Ling Shyu, Shu-Ching Chen, Kanoksri Sarinnapakom, and Liwu Chang. A novel anomaly detection scheme based principal component classifier. In Proceedings of International Con- ference on Data Mining, 2003.
Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, and Christine Largouët. Anomaly detection in streams with extreme value theory. In KDD, 2017.
Junho Song, Kconwoo Kim, Jeonglyul Oh, and Sungzoon Cho. Memto: Memory-guided trans- former for multivariate time series anomaly detection. NeurlPS, 36, 2024.
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In KDD, 2019.
Markus Thill, Wolfgang Konen, and Thomas Bäck. Markusthilllmgaab: The mackey-glass anomaly benchmark (version v1.0.1). https:: :ttps://doi... http://60...001055552888//// 02t.://55228//2mdd033733363333 April 2020.
Jindong Tian, Yuxuan Liang, Ronghui Xu, Peng Chen, Chenjuan Guo, Aoying Zhou, Lujia Pan, Zhongwen Rao, and Bin Yang. Air quality prediction with physics-informed dual neural odes in open systems. ICLR, 2024.
Cheng Wang and Hangyu Zhu, Wrongdoing monitor: A graph-based behavioral anomaly detection in cyber security. IEEE Trans. Inf. Forensics Secur., 2022.
Chengsen Wang, Zirui Zhuang, Qi Qi, Jingyu Wang, Xingyu Wang, Haifeng Sun, and Jianxin Liao. Drift doesn't 't matter: Dynamic decomposition with diffusion reconstruction for unstable multi- variate time series anomaly detection. In NeurIPS, 2023a.
Huazhang Wang, Zhaojing Luo, James Wei Luen Yip, Chuyang Ye, and Meihui Zhang. ECG- GAN; A framework for effective and interpretable electrocardiogram anomaly detection. In KDD, 2023b.
Yihang Wang, Yuying Qiu, Peng Chen, Kai Zhao, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, and Chenjuan Guo. Rose: Register assisted general time series forecasting with decomposed frequency learning. arXiv preprint 2rXiv:2405.17478, 2024.
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. CoRR, 2024.
Renjie Wu and Eamonn J. Keogh. Current time series anomaly detection benchmarks are flawed and are creating the illusion of progress. TKDE, 2023.
Xingjian Wu, Xiangfei Qiu, Zhengyu Li, Yihang Wang. Jilin Hu, Chenjuan Guo, Hui Xiong, and Bin Yang. Catch: Channel-aware multivariate time series anomaly detection via frequency patching. In ICLR, 2025.
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In ICLR, 2022.
Yiyuan Yang. Haifeng Zhang, and Yi Li. Pipeline safety early warning by collifeature-fusiinn CNN and lightgbm analysis of signals from distributed optical fiber sensors. IEEE Trans. Instrum. Meas., 2021.
Yiyuan Yang. Chaoli Zhang, Tian Zhou, Qingsong Wen, and Liang Sun. Dedetector: Dual attention contrastive representation learning for time series anomaly detection. In KDD, 2023.
14

---

Published as a conference paper at ICLR 2025
Chin-Chia Michael Yeh, Yan Zhu, Liudmila Ulanova, Nurjahan Begum, Yifei Ding, Hoang Anh Dau, Diego Furtado Silva, Abdullah Mueen, and Eamonn J. Keogh. Matrix profile I: all pairs similarity joins for time series: A unifying view that includes motifs, discords and shapelets. In ICDM, 2016.
Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. Ts2vec: Towards universal representation of time series. In AAAI, 2022.
Kexin Zhang, Qingsong Wen, Chaoli Zhang, Rongyao Cai, Ming Jin, Yong Liu, James Zhang, Yuxuan Liang, Guansong Pang, Dongjin Song, and Shirui Pan. Self-supervised learning for time series analysis: Taxonomy, progress, and prospects. CoRR, 2023.
Yan Zhao, Liwei Deng, Xuanhao Chen, Chenjuan Guo, Bin Yang, Tung Kieu, Feiteng Huang, Torben Bach Pedersen, Kai Zheng, and Christian S. Jensen. A comparative study on unsupervised anomaly detection for time series: Experiments and analysis. CoRR, 2022.
Bin Zhou, Shenghua Liu, Bryan Hooi, Xueqi Cheng, and Jing Ye. Beatgan: Anomalous rhythm detection using adversarially generated time series. In IJCAI, 2019.
Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In NeurlPS, 2023.
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dae-ki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. ICLR, 2018.
15

---

Published as a conference paper at ICLR 2025
A DATASETS
A.1 DATASETS FOR EVALUATION
We evaluate DADA and baselines on various datasets come from different domains, which can be broadly divided into spacecraft, servers, water treatment, finance, networking, and space weather: (I) SMD (Server Machine Dataset) collects resource utilization of computer clusters from an Inter- net company (Su et al. 2019). (2) MSL (Mars Science Laboratory dataset), collected by NASA, encompasses telemetry data reflecting the operational status of both sensors and actuators aboard the Martian rover (Hundman al.) 2018). (3) SMAP (Soil Moisture Active Passive dataset) is also collected by NASA and reflects soil moisture data from spacecraft monitoring systems (Hundman ct 2018). (4) SWaT (Secure Water Treatment) collects sensor data from a continuously operating infrastructure (Mathur & Tippenhauer 2016). (5) PSM (Pooled Server Metrics dataset) is collected from cBay Server Machines (Abdulaal et al. 2021). (6) NeurIPS-TS (NeurIPS 2021 Time Series Benchmark) is dataset proposed by (Lai et al. 2021 and use four sub-datasets: CICIDS, Creditcard, GECCO, SWAN with a variety of anomaly scenarios. (7) UCR Dataset collected by Wu & Keogh| (2023), which contains 250 sub-datasets. Each sub-datasets contains one dimen- sion and one anomaly segment. For MSL and SMAP datasets, we only retain the first continuous dimension (Wang et 2023a We summarize the datasets in Table
Table 4: Details of benchmark datasets for evaluation. AR (anomaly ratio) represents the abnormal proportion of the whole dataset.
Dataset
Domain
Dimension
Training
Validation
Test(abeled)
AR(%)
MSL.
Spacecraft
46,653
11,664
73,729
10.5
PSM
Server Machine
25
105,984
26,497
87,841
27.8
SMAP
Spacecraft
108, 146
27.037
427,617
12.8
SMD
Server Machine
38
566,724
141,681
708,420
4.2
SWaT
Water treatment
31
396,000
99,000
449,919
12.1
Creditcard
Finance
29
113,922
28,481
142,404
0.17
GECCO
Water treatment
55,408
13,852
69,261
1.25
CICIDS
Web
72
68,092
17,023
85,116
1.28
SWAN
Space Weather
48,000
12,000
60,000
23.8
UCR
Natural
1,790,680
447,670
6,143,541
0.6
A.2 DATASETS FOR PRE-TRAINING
We collect two datasets for pre-training, namely Anomaly Detection Data and Monash+, each of them contains normal time series and abnormal time series. None the downstream datasets for evaluation included in the pre- pre-training datasets. Firstly, we collect the Anomaly Detection Data from time series anomaly detection task. lists subsets in Anomaly Detection Data, including ASD (Li al. 2021a). Exathlon (Jacob et al. 2021). ECG (Moody & Mark 2001). MITDB (Moody & Mark 2001 ). OPP (Roggen et al. 2010 SVDB (Greenwald et al. 1990). GALA" IOPS (Ren al. 2019), MGAB (Thill et al. 2020). NYC (Cu et al. 2016), SKAP YAHOO Laptev al.,
2015). In the field of time series anomaly detection, datasets usually contain default normal training data and labeled data with anomalies. We take the training data of this part as normal data, and the data with anomaly labels in test data as abnormal data.
To further enlarge our pre-training data, we build the Monash" + dataset, The normal time series in Monash+ comes from the Monash Prediction Library (Godahewa et al. 2021 shown in Table and the abnormal time series in Monash+ are generated from Monash by anomaly injection. The anomaly injection is shown in Figure(7) These time series with different variate numbers and series lengths from multi-domains reflect the diverse application of the real world and the overall pre- training datasets are summarized in Table
16

---

Published as a conference paper at ICLR 2025
Table 5: Anomaly Detection Data selected from multi-domain time series anomaly detection task. AR (anomaly ratio) represents the abnormal proportion of the whole dataset.
Dataset
Domain
Dimension
Length
AR %)
ASD
Application Server
384,493
1.55
Exathlon
Application Server
multi
131,610
8.71
ECG
Health
1,60
4.7
MITDB
Health
13,650,000
3,44
OPP
Health
14,567,752
4.11
SVDB
Health
7,,09,660
4.68
GAIA
A1Ops
1,868,506
1.21
IOPS
Web
5,840,488
2.15
MGAB
Mackey-Glass
1,000,000
0.2
NYC
Transport
17,520
0.57
SKAB
Machinery
46,707
3.65
YAHOO
Multiple
565,827
0.62
Table 6: Multi-domain time series datasets from the Monash Prediction Library.
Dataset
Domain
Frequency
Length
Aus. Electricity Demand
Energy
Half Hourly
1,155.264
Wind
Energy
4 Seconds
7,397,147
Wind Farms
Energy
Minutely
172,178,060
Solar Power
Energy
4 Seconds
7,397,223
Solar
Energy
10 Minutes
7,200,857
London Smart Meters
Energy
Half Hourly
166,527,216
Saugeen River Flow
Nature
Daily
23,741
Sunspot
Nature
Daily
73,924
Weather
Nature
Daily
43,032,000
KDD Cup 2018
Nature
Daily
2,942,364
US Births
Nature
Daily
7,305
FRED JD
Economic
Monthly
77,896
Bitcoin
Economic
Daily
75,364
NNS
Economic
Daily
87,801
A.3 ANOMALY INJECTION
We use abnormal time series in the pre-training to explicitly enable robust distinguishment between normal and abnormal patterns with our dual adversarial decoders. Due to the scarcity of labeled data in the field of time series anomaly detection, enrich abnormal samples the Monash dataset. These injected anomalies can be considered more common anomaly patterns. As shown Figure 7 we generate an expanded collection of novel time series exhibiting diverse anomaly types and patterns, thereby enhancing our model's 's ability to distinguish between normal and abnormal time series. The injected anomalies cover point-wise anomalies and pattern anoma- lies. hmirror and vmirror alter the sequence trend by reversing the timestamps of sub-sequence or inverting the value magnitudes, respectively. outlier and scale generate outlier sequences or points by expanding (or reducing) the values of sub-sequences simulate pattern anomalies by adding Gaussian noise to sub-sequences or replacing them with an- other sub-sequence. Compress and stretch modify the period length of the sequence through interval sampling or interpolation. We use the default values recommended Schmiddl et al. (2024), and empirical results show that these defaults produce reliable and robust anomaly injection outcomes.
anomaly injection
specific time points. Noise and patlern
For anomaly injection, we randomly select a subsequence within the original time series and an anomaly type. If the subsequence has not previously been injected with an anomaly, we inject this anomaly into it. We repeat this process until the anomaly ratio in the entire sequence reaches the pre-defined number.
17

---

Published as a conference paper at ICLR 2025
na!
tnrimer
veiiingr
My
m
c
130
120
133
1ire
acale
outter
15
36c
325
4twwtchh
pxttrem
30 200 25
Figure 7: Anomaly injection. The blue lines represent the time series, while the pink areas intervals the anomalies we generated. We visualize the original time series and eight types of anomaly, including hmirror, vmirror, scale, outlier, noise, compress, stretch, and pattern.
Table 7: Description of pre-training datasets with normal data and abnormal data.
Dataset
Normal Time Series
Abnormal Time Series
Anomaly Detection Data Monash
Training set
Test set
All of Moansh data
Generated from Monash by anomaly injection
B IMPLEMENTATION DETAILS
We summarize the default hyper-parameter as follows. We use a dilated CNN as an encoder with ten layers Yue et al. (2022). In the implementation phase, we run a sliding window with a window size of 100 and conduct anomaly detection using non-overlapping windows the same existing works (Xu et al, 2022; Shen et al. 2020). Our patch size is 5. The bottleneck pool contains the sizes [16, 32, 64, 128, 192, 256] and the adaptive router selects = 3 bottlenecks with the highest weights. The AdamW optimizer with initial learning rate of 10-4 is used in pre-training. We set the batch size to 2048 with 5 epochs in pre-training. We do not use the "drop last" trick (Qiu al. 2025b during the testing phase ensure fair comparison, as advocated by TFB QQu et al. 2024). In the inference stage, we use 5 pairs of complementary masked series. After obtaining the anomaly score, we adopt commonly used SPOT (Siffer al., (217)) following existing works (Wang et al. [203al (Su et al. (2019) to get the threshold ò to determine whether each point is an outlier (Wang et al. (023aa). We conduct all experiments using Pytorch with NVIDIA Tesla-A800-8000 GPU.
C ADDITIONAL EXPERIMENTAL RESULTS
C.1 MORE EVALUATION METRICS
In the realm of time series anomaly detection, there a persistent debate regarding the most appro- priate evaluation metrics to employ. This is due to the fact that different metrics can offer varying insights into the performance of a model, and the choice metric can significantly influence the perceived effectiveness of anomaly detection model. In main text, we include the more rea- sonable Affiliated Fl and AUC ROC scores. To further enhance the evaluation of our model, we also include the Volume Under the Surface (VUS) metric (Paparrizos al. 2022), which takes
18

---

Published as a conference paper at ICLR 2025
anomaly events into consideration based the receiver operator characteristic curve. As shown in Table g DADA achieves good results on most metrics. GPT4TS, as a good baseline, demonstrates outstanding performance the SMD dataset. However, its performance across different datasets is not consistent. For instance, there remains a noticeable gap compared to DADA on the PSM dataset. In contrast, DADA exhibits robust detection capabílities across all three datasets.
Table 8: Multi-metrics results.
Dataset
Method
AUC_ROC
R.AUC_ROC
R.AUC.PR
VUS_ROC
VUS_PR
A.T.
66.42
50.02
51.22
7.09
51.17
7.96
D3R
78.02
53.34
62.89
11.20
61.69
11.01
SMD
GPT4TS
83.13
71.15
77.27
17.68
76.79
17.45
TCN
83.16
70.21
77.54
16.28
77.07
15.96
DADA
84.57
71.98
77.50
17.13
A.T.
66.49
34.05
38.65
10.25
38.90
10.41
D3R
77.02
45.77
56.35
20.05
55.69
19.76
MSL
GPT4TS
77.23
72.63
77.65
28.05
76.97
27.69
MCd
77.17
74.96
77.88
30.91
77.47
30.10
DADA
78.48
75.15
77.99
31,05
77.48
30.50
A.T.
65.37
50.11
52.70
33.12
51.86
33.09
D3R
80,29
50.22
52.93
39.62
52.18
39.31
PSM
GPT4TS
81.44
58.94
65.16
46.54
64.66
45.99
TCN
79.59
58.46
65.50
47.48
64.80
46.68
DADA
82.26
61.08
67.08
47.40
66.52
46,89
C.2 EVALUATION THE UCR BENCHMARK
We evaluate DADA UCR Anomaly Archive (Wu & Keogh 2023). where the task is to find the position of an anomaly within the test set. To achieve this, we compute the anomaly scores for each time step in the test set and subsequently rank them in descending order. Following the approach of Timer (Liu et al. 2024b), the time step with the a quantile hits the labeled anomaly interval in the test set, the anomaly detection is accomplished. Figure & presents the results of our evaluation. The left figure displays the number of datasets which the model successfully detect anomalies at quantile levels of 3% and 10%; higher count indicates stronger detection capability of the model.
The right figure shows the quantile distribution and the average quantile across all UCR datasets. DADA has lower average quantile compared Timer, and which indicates better performance of anomaly detection.
250
100%
Dibastts (A
200
DADA
Timer
201
Quintilc Distribution
90%
.... DA Avp: 7.6%
196
150
60%
Timer Avg: 12.5%
100
40%
0%
10%
50
150
200
250
Quantile
Datasets Count
Figure 8: Downstream anomaly detection results for UCR benchmark
D PARAMETER SENSITIVITY
We study the parameter sensitivity of the DADA. Figure 9 shows the model performance under different window sizes, & bottlenecks with the highest weights that the adaptive router can select, and different patch sizes. Window size is a very important parameter for time series analysis. In time series, single point does not contain any information, length of the sample context. Our experiments show that DADA insensitive to window size and performs well across different window sizes. We then explore the influence of parameter & in the adaptive bottlenecks. From the performance point of view, our model shows a relatively stable effect under different k, where achieve good performance when selecting bottleneck for each
the window size determines the
19

---

Published as a conference paper at ICLR 2025
sample. Finally, study the performance under different patch sizes. Patch splitting has proven to be very helpful for capturing local information within each patch and learning global information among different patches. The results in Figure show that DADA is robust under different patch sizes.
F1
%
F1 Scoree
70
70
60-
1M/
60-
1845
60
1941
1669P
26647
5356
9006
100
150
10
(2) Window Size
(b1
[c] Patch Size
Figure 9: (a) Parameter sensitivity for sliding window size, (b) bottlenecks with the highest weights that the adaptive router can select, and (c) patch sizes (c).
E
ADDITIONAL MODEL ANALYSIS
E.1 SHOW CASE
To provide more intuitive illustration of how our model works, we employ anomaly injection to generate univariate time series featuring various anomalies, including outliers, horizontal mirroring, pattern shifts, and scale changes. As shown in Figure [O] our findings indicate that DADA can robustly and timely distinguish these anomalies from normal data points, which is crucial for real- world applications it allows for the prompt prevention of potential losses. Moreover, leveraging the Dual Adversarial Decoders, if the model exhibits limitations in detecting some certain anomalies, its detection capabilities can be enhanced by incorporating the corresponding types of anomalies into the pre-training dataset.
hemirror
Input
Time
Time
Time
Time
Anomaly
Soores
Time
Time
Time
Figure 10: Visualization of learned criterion. The blue line in the first represents the time series, and the blue line in the second row represents the anomaly scores obtained by the model, while the pink areas intervals the anomalies.
In Figure II we visualize the anomaly scores of GPT4TS, ModemTTNN, and DADA on the PSM dataset. Additionally, we visualize the original sequences of four channels in the PSM dataset that contribute significantly to the anomalies. From the visualization results, is clear that DADA effectively distinguishes the normal and abnormal samples, accurately identifying anomaly points. In contrast, GPT4TS and ModemnTN can not effectively increase the gap between the anomaly scores of normal points and anomalies, leading to many false positives.
20

---

Published as a conference paper at ICLR 2025
Noath Bare
ccoe
wwwhhpaaal M
mnl
Teee GPT4TS
Origin Data
DADA
ModemTCN
Figure II: Visualization of anomaly score on PSM dataset. The green line in the origin data repre- sents the original time series. The blue line represents the anomaly scores obtained by the model, while the pink areas intervals the anomalies.
E.2 WITHOUT HUMAN-LABELEN DATA
Our model does not strictly rely on human-labeled data. As mentioned in Appendix A we can generate a variety of anomalies through anomaly injections. conduct an experiment to validate this. We do not use any labeled data and only consider the anomaly injection method to pre-train a model (Without Label), and the zero-shot results under are as shown in Table / It verifies that training model without human-labeled supervised labels does not significantly impact the performance.
Table 9: Detection results without human-labeled data.
Dataset
SMD
MSL
SMAP
SWAT
PSM
Metric
F1
F1
F1
F1
F1
Without Label
84.69
78.05
75.35
74.07
82.39
DADA
84.57
78.48
75.42
74.60
82.26
E.3 COMPARISON WITH MoE
Mixture-of of-Experts (MoE) (Jacobs et al. 1991 Shazeer et al.] 2017) includes the gating network and multiple experts, each with the structure. The gating network assigns different experts cording to the characteristics of the data. In our AdaBN, each bottleneck in the bottleneck pool has different hidden dimension. Each bottleneck compresses time series representation to different hidden space to accommodate the varying requirements of the multi-domain time series data. As shown in table I0] compare DADA with MoE, replace all bottlenecks in the bottleneck pool with consistent size of 256, which performs better compared to other hidden dimensions as shown Fi From Figure 55left), we can see that MSL, SMAP, and SMD datasets are well-suited for bottleneck size of 256, making it easier for MoE 256 to achieve good performance on these datasets. However, on datasets where the bottleneck size 256 is less effective, such as SWaT, GECCO, and Creditcard, MoE 256's performance is not good as AdaBN. It shows that MoE can- not solve the problem that different data require different bottlenecks in general time series anomaly detection. In contrast, DADA with AdaBN performs consistently across all datasets and shows sig- nificant improvements on SWaT, GECCO, and Creditcard compared to MoF 256, aligning with the motivation behind AdaBN's design that the model's bottleneck needs be adaptively selected.
Table 10: Comparison of DADA and MoE. Single 256 replaces the AdaBN with a single bottleneck with size 256. The MoE 256 changes all bottlenecks in the bottleneck pool to the same size 256. The result reported with Affiliated F1, and the best ones are in bold.
Dataset
SMD
SMAP
MSL
SWaT
PSM
GECCO
Creditcard
Metric
F1
F1
F1
F1
F1
F1
F1
single 256
81.69
74.37
77.01
69.93
80.15
85.12
70.96
MoE 256
84.25
74.87
78.41
71.68
81.69
87.81
71.36
DADA
84.57
75.42
78.48
74.60
82.26
90.20
75.12
21

---

Published as a conference paper at ICLR 2025
E.4 INFERENCE TIME
In Table 11 we compare DADA with other models on the SMD dataset in terms of training time, inference time, and Fl score. Training time refers the time required train the model for one epoch the target dataset, while inference time refers to the time required for a single forward pass on the testing set. We compare the baseline models A.T. and ModemTCN, as well the pre- pre-trained model GPT4TS. Additionally, we pre-train GPT4TS on multi-domain time series and include it in the comparison. can be found that ModernTCN but it still requires the training data to train the model on the target dataset. After pre-training multi-domain datasets, GPT4TS (multi-domain) and DADA can perform zero-shot on target dataset. However, GPT4TS (multi-domain) requires longer inference time and, due to not being specifically designed for anomaly detection, its detection performance is not ideal. In comparison, DADA has certain advantage in inference time and can effectively perform zero-shot learning without target training data.
advantageous terms inference time,
Table 11: Analysis on inference time.
Methods
Training Time(s)
Inference Time(s)
F1 Score
A.T. ModernTCN
782.5
0.093
66.42
97.4
0.016
83.16
GPT4TS
934.9
0.113
83.13
GPT4TS (multi-domain)
0.113
78.00
DADA
0.081
84.57
E.5 VISUALIZATION OF ADAPTIVE BOTTLENECKS
As shown in Figure 12 left), our visualization illustrates the proportion of different adaptive bottle- neck sizes that each dataset selects and prefers. The figure highlights distinct preferences among datasets, indicating variability in their selection criteria. Notably, the SMAP and MSL datasets, sourced from NASA spacecraft monitoring systems, display a similarity their data characteristics. This observation is further supported by experimental results in Figure 12 left), showcasing consistent preferences for bottleneck sizes within these datasets.
SWaT
Monash"
5MD
Monash and Anomaly Datasets
16
SMAP
Soore
128
192
256
0%
70%
60%
60%
80%
100%
MSL
SMAP
Figure 12: (left) Visualization the proportion of different bottleneck sizes that each dataset prefers.
(right) Pre-training with multi-domain datasets.
E.6 PRE-TRAINING WITH MULTI-DOMAIN DATASETS
model's performance. As depicted in Figure 12 right), we employ the Monash" + in Table and Anomaly Detection Data in Table individually for pre-training, followed by zero-shot evaluation the SMD, MSL, SMAP, SWaT, and PSM datasets. Our model consistently exhibits great performance with different pre-training datasets.
We investigated the impact of the pre-training datasets
22

---

Published as a conference paper at ICLR 2025
Furthermore, using both datasets together to expand the domain scope of pre-training datasets hances the breadth of information assimilated by our adaptive bottlenecks and dual adversarial de- coders. Consequently, this enrichment contributes to further improvements in model performance.
E.7 COMPLEMENTARY MASK ANALYSIS
The complementary mask is a way to enhance utilization during mask modeling. We mention this design for better reproducibility of our method. further analyze we compare the ordinary masks that reconstruct the masked part from the unmasked part once with the complementary masks Table T2 In terms results, the complementary mask slightly outperforms the ordinary mask.
Table 12: Comparison between ordinary mask and complementary mask.
Dataset
SMD
MSL
SMAP
SWaT
PSM
Metric
F1
F1
FI
F1
F1
Ordinary mask
84.25
78.51
75.03
74.32
81.87
Complementary mask
84.57
78.48
75.42
74.60
82.26
We further analyze the complementary mask rate's effect on anomaly detection performance. Since we use complementary masks, when data masked 30%, its complementary part is masked 70%. Therefore, analyze the effects of mask rates ranging from 10% to 50%. As shown in Table we find that maintaining a complementary mask ratio closer to 50% yields better performance. When the mask ratio is too high, the model's reconstruction becomes very difficult, leading poorer model performance.
Table 13: Analysis on complementary mask ratio.
Dataset Metric
SMD
SMAP
MSL
SWaT
PSM
Avg
F1
F1
F1
F1
F1
F1
0.1 / 0.9
82.41
77.82
70.71
63.57
75.55
74.00
0.2 / 0.8
83.62
78.21
70.69
64.12
79.94
75.32
0.3 / 0.7 83.98
77.98
74.17
70.22
81.86
77.64
0.4/ 0.6
84.34
78.12
75.51
74.57
82.37
78.98
0.5 0.5
84.57
78.48
75.42
74.60
82.26
79.07
E.8 ANOMALY CONTAMINATION TRAINING SAMPLE
Existing methods for time series anomaly detection typically assume that the training data mal. Exploring the impact potential anomalies in training data on the model is indeed an in- teresting research direction, but major challenge is the lack of labels in training data to identify potential anomalies. To explore this impact, we additionally inject anomalies into our pre-training normal data at rates of 0.1%, 5%, and 10%, and treat these data if they were normal. As shown in Table 14, the model's performance affected to some extent as the anomaly contamination ratio increases. Since 10% is already much higher than the potential contamination in the actual training set, this result is acceptable and our method is generally robust to some anomaly contamination.
E.9 VISUALIZATION OF PRE-TRAINING LOSS CURVES
As shown Figure 13 we present the pre-training loss curves for the abnormal and normal parts, denoted as Loss_nore and Loss_abnorm. As training progresses, Loss _oss_norm steadily declines, in- dicating improved reconstruction of normal data. contrast, Loss_abnorm initially decreases but then rises slowly, stabilizing later, and is always clearly higher than Loss_nome This is the expected effect of the adversarial training mechanism, as shown Eq. 7 where the Encoder maximizes the abnormal loss, and the anomaly decoder minimizes the abnormal loss. Eventually, Loss abnorm
23

---

Published as a conference paper at ICLR 2025
Table 14: Analysis anomaly contamination in pre-training normal data.
Dataset Metric
SMD
MSL
SWaT
SMAP
Avg
F1
F1
F1
F1
F1
0%
84.57
78.48
74.60
75.42
78.26
0.1 %
84.50
78.09
74.04
75.56
78.05
5 %
83.27
77.57
73.59
74.55
77.25
10 %
82.95
76.53
73.42
73.69
76.64
stabilizes steady value, indicating that the balance between the Encoder and the abnormal de- coder has been reached and normal and abnormal data can be clearly distinguished. Therefore, the entire training process is relatively stable.
0.9
Loss, Loss_norm norm
0.8
Loss, abnerm
0.6
0.5
0.3
30000
20000
30000
40000
50000
Figure 13: Pre-training loss curves for abnormal and normal parts, denoted as Loss norm and Lbnorm.
24


Reviews:
[{"review_id": "EnhYZvihZl", "reviewer": "Reviewer_WTbT", "summary": "This paper proposes a new general time series anomaly detector with adaptive bottleneck and dual adversarial decoder. By pre-training on multi-domain time series data, anomaly detection is performed efficiently in different target scenarios without domain-specific training.\nThe overall writing of this article is good and the logic is clear。\nThis paper first proposes an adaptive bottleneck module to compress features into the latent space, reflecting the different information densities of multi-domain data. It solves the challenge of different requirements for appropriate information bottlenecks for different datasets.\nThis paper uses dual adversarial decoders to expand the robustness distinction between normal and abnormal modes, improving the general anomaly detection ability in different scenarios.", "questions": "I hope the author can explain the difference between the routing function proposed in \"PATHFORMER: MULTI-SCALE TRANSFORMERS WITH ADAPTIVE PATHWAYS FOR TIME SERIES FORECASTING\". At the same time, give a detailed explanation of the reconstruction process Recon(·).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The overall writing of this article is good and the logic is clear。\nThis paper first proposes an adaptive bottleneck module to compress features into the latent space, reflecting the different information densities of multi-domain data. It solves the challenge of different requirements for appropriate information bottlenecks for different datasets.\nThis paper uses dual adversarial decoders to expand the robustness distinction between normal and abnormal modes, improving the general anomaly detection ability in different scenarios.", "weaknesses": "The adaptive router formula of the adaptive bottleneck module proposed in this paper lacks certain innovation, and its formula is similar to the routing function proposed in “PATHFORMER: MULTI-SCALE TRANSFORMERS WITH ADAPTIVE PATHWAYS FOR TIME SERIES FORECASTING”.\nSome formulas in this paper lack clear explanations, which may cause some confusion when reading. For example, the reconstruction process proposed in the article is denoted as Recon(·), but there is a lack of detailed description of this process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cMYIWhkeFv", "reviewer": "Reviewer_hvUu", "summary": "Authors propose a General time series anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders (DADA), which enables flexible selection of bottlenecks based on different data and explicitly enhances clear differentiation between normal and abnormal series. The zero-shot capability is indeed impressive.", "questions": "1. In the ablation experiment, the result of the fourth row is the second highest. In my opinion, AdaBN should be basically equivalent to moe, except for the different dimensions. However, at this time, the model effect is basically close to the final model, which means that the main source of performance is not the specially designed optimization method. This makes me doubt the component innovation of the paper. At the same time, Table 10 in the appendix also supports this conclusion. The result of moe 256 is completely better than the result of the fourth row. This seriously indicates that AdaBN is likely to have a much smaller actual effect than Moe 256. Further, the innovation of AdaBN should be greatly discounted.\n2. The experimental analysis lacks the analysis of COMPLEMENTARY MASK MODELING. The author takes this item as the key point of the method but lacks further research.\n3. The visualization analysis does not state what dataset is used, nor is there visualization of the original data. Furthermore, there is a lack of visualization comparison with other algorithms, lacking the credibility of the experiment.\n4. I hope the author can open source the generation method and generation script of Monash+ and provide a directly usable data set. At present, the explanations in the paper are completely insufficient.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The writing is clear and easy to understand.\nThe pictures and tables in the paper are beautiful and clear.\nThe proposed method, DADA, can achieve zero-shot.", "weaknesses": "1. AdaBN has almost no difference from the previous moe, yet the author exaggerates their design. I think there is a question of contribution. Moreover, the author does not explain the difference from Moe in the main text nor add relevant references [a]. I think this is somewhat deceptive. \n2. The role of AdaBN is not as great as the author claims. See the analysis in question 1. There is excessive exaggeration of contribution.\n[a] Shazeer, Noam, et al. \"Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.\" arXiv preprint arXiv:1701.06538 (2017).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MF7BOT87bD", "reviewer": "Reviewer_e12v", "summary": "This paper develops a novel general time series anomaly detection by pre-training on multi-domain time series.  The proposed adaptive bottleneck enables the general model to be applied to various target scenarios without requiring domain-specific training. To improve the general anomaly detection capability, the dual adversarial decoder is further employed to amplify the decision boundary.", "questions": "1. Does \"anomaly injection\" refer to incorporating multiple types of anomaly patterns into each dataset? More details about the implementation of anomaly injection should be included in the paper since it is crucial for conducting the supervised setting.\n2. How do you adapt various baseline methods pre-trained on multi-domain datasets for zero-shot evaluation if they are typically utilized in the standard form?\n3. How can DADA be applied during the inference step to further fine-tune the target datasets for downstream tasks in the experiments?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The general time series anomaly detection plays a key role in the real application.\n2. DADA employs the adaptive bottlenecks for multi-domain time series data by dynamically selecting suitable bottleneck sizes. \n3. The effectiveness of both AdaBN and Dual Decoders has been demonstrated in the experiments.", "weaknesses": "1. If anomaly injection is used to establish a supervised setting, this implies that all original samples(some anomalies are included) are treated as normal samples. I suggest the author add an experiment to examine performance under varying levels of anomaly contamination in the training samples.\n2. During model training, two optimization losses are utilized, along with the optimization approach and any challenges encountered in the optimization process. The visualizations of loss curves for the two losses are essential for further demonstration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ffrVFxzxA9", "reviewer": "Reviewer_e5Ze", "summary": "The paper proposes DADA method for general time series anomaly detection, which is pre-trained on extensive multi-domain datasets and perform anomaly detection on various target scenarios without domain-specific training. DADA proposes Adaptive Bottlenecks to addresses the flexible reconstruction requirements of multi-domain data. DADA also employs Dual Adversarial Decoders amplify the decision boundaries between normal time series and common anomalies in an adversarial training way, which improves the general anomaly detection capability across different scenarios. The paper conducts extensive experiments on nine target datasets from different domains, and demonstrate that DADA achieves superior results compared with state-of-the-art models trained specifically for each dataset.", "questions": "1. The paper does not compare the inference time of DADA with the baselines. Compared to the baselines, does DADA have a shorter inference time?  In particular, when compared to other pre-trained models, does DADA have an advantage?\n\n2. In the DADA model, the original data first undergoes complementary masking operations, and then features are extracted by the encoder. In subsection 3.2, it states that the masked portions of the original data input to the encoder have a value of 0, but there may also be 0 values in the non-masked portions of the original data. How does the encoder differentiate between the masked and non-masked portions?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper addresses the issue that each dataset in the field of time series anomaly detection requires a specifically trained model, which holds practical significance. The paper proposes a novel method DADA for zero-shot inference on the target domain after pre-training on multiple datasets from different domains. DADA proposes the Adaptive Bottlenecks to addresses the flexible reconstruction requirements of multi-domain data, along with dual adversarial decoders to enhance the clear differentiation between normal and abnormal patterns, showcasing a degree of originality. The paper has a clear structure and expression. In the experimental section of the paper, DADA demonstrates superiority over multiple state-of-the-art methods across nine datasets and provides extensive analysis to prove the necessity of each module, showcasing high quality.", "weaknesses": "1. The paper lacks an analysis of the complementary mask rate's effect on anomaly detection performance. For example, the author could add an experiment to demonstrate the changes in anomaly detection F1-score under various complementary mask rates and explain why this mask performs the best.\n\n2. In the experimental section, Figure 4 shows that after fine-tuning, the model's performance improved from 78.48% to 79.90% on MSL and from 84.57% to 85.50% on SMD, an increase of about 1%. The data shows that the improvement of the model after fine-tuning is small.\n\n3. In the ablation study, Table 3 shows that the model performs better when removing the AdaBN, Adversarial, and Dual Decoders modules compared to removing only the Adversarial module. The paper explains that directly maximizing the anomalous time series will increases confusion. Further explanation is needed to clarify why maximizing the anomalous time series leads to increased confusion.\n\n4. There are labeling mistakes in the experimental results. For example, in Table 8, the R_AUC_PR value of DADA on the PSM dataset is 47.40, which is less than ModernTCN's 47.48, but 47.40 is bolded. In Table 2, the F1 metric column for the CICIDS dataset is missing a underline marking.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RejJFM9G8v", "reviewer": "Reviewer_EBSp", "summary": "The author propose a solution for general time series anomaly detection, which is call time series anomaly Detector with Adaptive bottlenecks and Dual Adversarial decoders (DADA). This model is  pre-trained on multi-domain time series data. Two components called Adaptive Bottlenecks and Dual Adversarial Decoders is leveraged to enhance the model performance. The model achieves competitive performance on various datasets even compared with SOTA models trained specifically for each dataset.", "questions": "- Can you list the data difference between the pretrain dataset and the evaluation dataset? How you select the pretrain dataset?\n\n- Are there any data preprocessing on the input data?\n\n- What's the computation cost(overhead) of importing Adaptive Bottlenecks and Dual Adversarial Decoders? Please specify.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "+ The task on zero-shot time series anomaly detection is quite challenging, the author build a unified end-to-end solution and demonstrate great results. \n\n+ The paper is well-organized, and it introduces the background and motivation of the proposed methods clearly. The figures also explain their method pretty well.\n\n+ The paper demonstrate its experiment results on enough datasets.  The ablation studies on Sec 4.3 also makes the paper convincible.", "weaknesses": "- The difference between the pretrain dataset and the evaluation dataset is not listed. How you select those dataset?\n\n- It will be great to specify some more experiment details. Are there any data preprocessing on the input data?\n\n- What's the computation cost(overhead) of importing Adaptive Bottlenecks and Dual Adversarial Decoders? Please specify.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)