Multivariate Time Series Anomaly Detection with Idempotent Reconstruction
Authors: Xin Sun
Heng Zhou
Chao Li
Venue: NeurIPS 2025 poster ()
Keywords: multivariate time series
anomaly detection
idempotent generative network
Primary area: deep_learning
TL;DR: This paper introduces a novel module, IGAD, which can be integrated with reconstruction-based methods to address over generation and performance balance issues from a manifold perspective.

Abstract: Reconstruction-based methods are competitive choices for multivariate time series anomaly detection (MTS AD). However, one challenge these methods may suffer is over generalization, where abnormal inputs are also well reconstructed. In addition, balancing robustness and sensitivity is also important for final performance, as robustness ensures accurate detection in potentially noisy data, while sensitivity enables early detection of subtle anomalies. To address these problems, inspired by idempotent generative network, we take the view from the manifold and propose a novel module named **I**dempotent **G**eneration for **A**nomaly **D**etection (IGAD) which can be flexibly combined with a reconstruction-based method without introducing additional trainable parameters. We modify the manifold to make sure that normal time points can be mapped onto it while tightening it to drop out abnormal time points simultaneously. Regarding the latest findings of AD metrics, we evaluated IGAD on various methods with four real-world datasets, and they achieve visible improvements in VUS-PR than their predecessors, demonstrating the effective potential of IGAD for further improvements in MTS AD tasks. Our instructions on integrating IGAD into customized models and example codes are available at https://github.com/ProEcho1/Idempotent-Generation-for-Anomaly-Detection-IGAD.

Multivariate Time Series Anomaly Detection with Idempotent Reconstruction
Xin Sun Zhejiang University xinsun1@zju. edu. en
Heng Zhou Zhejiang University hengzhouOzju. edu. en
Chao Li" Zhejiang University chaoli@zju. edu
Abstract
Reconstruction-biet based methods competitive choices for multivariate time ries anomaly detection (MTS AD). However, one challenge these methods may suffer is over generalization, where abnormal inputs also well reconstructed. In addition, balancing robustness and sensitivity is also important for final perfor- mance, robustness ensures accurate detection in potentially noisy data, while sensitivity enables early detection of subtle anomalies. To address these prob- lems, inspired by idempotent generative network, take the view from the manifold and propose a novel module named Idempotent Generation for Anomaly Detection (IGAD) which can be flexibly combined with reconstructionn-ent based method without introducing additional trainable parameters. We modify the man- ifold make sure that normal time points can be mapped onto while tight- ening it to drop out abnormal time points simultancously. Regarding the latest findings of AD metrics, we evaluated IGAD on various methods with four real- world datasets, and they achieve visible improvements in VUS-PR than their predecessors, demonstrating the effective potential of IGAD for further improve- ments in MTS AD tasks, Our instructions models and example codes are available at Idempotent Gewpeettteeeeeeeinnneeeeeeee -oreerrraaion----oooaaaaaaaall Anomal A
integrating IGAD into customized
com/ProEcho1/
Introduction
Multivariate time series (MTS) are continuously collected from numerous sensors 3 [6]
which is
widely present in many critical scenarios, such as production data from multiple devices in modern
factories and monitoring data from various sensors in smart grids [12 [6] 25 The task of anomaly detection (AD) in multivariate time series lies in determining whether each time point within the series is normal or abnormal, helping to identify possible malfunctions and minimize losses where each time point in the series can be viewed as a time instance.
69 28 g)
Unsupervised anomaly detection methods often develop reconstruction-basee models, such as 62 [0]. All time points are label-absent and viewed as normal in the training data. These models tend to show an encoder-decoder architecture, producing small reconstruction errors for normal time points and larger reconstruction errors for abnormal ones after training the ability reconstruct only on these normal data, as in the second case in Fig [ However, one challenge these methods may suffer over generalization [88] where abnormal time points can also be reconstructed so well that it becomes harder to distinguish them (6) [7), as shown in the first case of Fig [ We conclude the reasons for this issue as that this may happen when: (1) the built model incorrectly captures the intrinsic patterns of abnormal series in contaminated dataset for training; (2) the model has an excessive decoding power, even for abnormal series. We provide math analysis for these in Appendix[A] The phenomenon of contaminated datasets exists across multiple domains,
54
The Corresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Moreover, is difficult and impractical to guarantee clean time series for training during data collection, or manually and accurately control the feature extraction capabilities of the encoders and the reconstruction capabilities of the decoders.
such as data mining [0 68 and computer vision A0 49
Meanwhile, an additional issue is the balance of robustness and sensitivity of these models. On the one hand, robustness to noisy data is frequently disregarded factor in time series anomaly detection applications, which refers to the capacity to accurately identify normal and abnormal time points even in the presence of noise. This is essential in practical applications, as real-world data unavoidably contain noise that can arise from sensor errors, loss of data transmission, or other external factors If model performs poorly on noisy data, may fail to detect true anoma- lies or falsely identify normal data as anomalous. Such situations can have severe consequences in critical appli- cations, including industrial monitor- ing and medical diagnostics. How- ever, sensitivity is crucial for identify- ing subtle deviations from normal pal- terns, which can indicate early signs of potential anomalies. High sensitiv- ity enables the model to detect these minor aberrations, ensuring timely in- terventions. A most excessive case can be illustrated in the third example of Fig [ where abnormal time points and normal time points affected by noise are close and mixed to distinguish. Hence, it is important to balance robustness and sensitivity to avoid false positives due to noise while maintaining the reliability of anomaly detection.
Case1
Case
Time
Croad Trrh Label
Rrcoeattutted
Asoeslly with Polie
Will www.ceecnrneeeeo Anematy
Figure 1: Three existing phenomeena in the methods based on reconstruction. The issues of over generalization, correct detection, and the conflict of robustness and sensitivity are shown from left to right, marked from Case Case 3.
To overcome these limitations, drawing inspiration from the idempotent generative network [47), we have adapted this method to the task of detecting anomalies in multivariate time series data. Our modification allows for the integration of this flexible module with existing reconstruction-bee based methods. First, we introduce the concept of idempotent generation. In general, we present f()), 2 as an encoder-decoder mapping network and a vector, respectively. An idempotent operator is one that, when applied multiple times in sequence, does not alter the result beyond the effect of the initial application, which can be denoted as f(())) = f (z). Second, we can regard the concept of a manifold the relationships between these multidimensional time points and the geometric structures they form, which reflect the underlying patterns and dynamics of the time series data. Each time point can be viewed as an instance in the manifold. Here, we assume that a source distribution P= and a target distribution Pz are in the same latent space. However, unlike Shocher et al. [47), we define Pa as a distribution consisting only of normal time instances. In this setting, when performing f(z) for a given normal time instance x ~ P.. the ideal outcome is that x remains unchanged. Further, we represent our target manifold as the subset of all normal time instances that can be assigned to themselves after applying f()) to minimize reconstruction errors.
Given these, AD tasks can benefit from two aspects: (1) we use a modified z under frequency domain features for a more controllable generation to address the challenge of balancing robustness and sensitivity, instead of directly sampling normal time instances as input, it is practical to perform a Fast Fourier Transform (FFT) to extract information about frequency components and spectral features, and an Inverse Fast Fourier Transform (IFFT) with resampling is selected to generate noise that contains intrinsic patterns of normal series. Then f(((z) = f() is performed to map these noise-affected instances the target manifold. Through artificially introduced random factors and the idempotent constraint, our goal is to balance robustness and sensitivity to capture inherent data patterns in normal time series; (2) we further introduce f()(z) & f(2 to tighten the target manifold to address the problem of over generalization. This adversarial strategy can be employed to prevent the manifold from incorrect expansion, excluding potential abnormal instances in the manifold. We will give more detailed explanations in Sect
(O I) from Pa. For time window consisting of
The major contributions of our work can be foree-folded: (1) We explore over generalization and the balance between robustness and sensitivity from a manifold perspective, establishing the links of the manifold with these limitations; (2) To overcome these mentioned issues, we propose a novel module
2

---

named Idempotent Generation for Anomaly Detection (IGAD), which can be flexibly integrated with reconstruction-aasee methods without introducing parameters that need be trained: (3) Based on our experimental results in VUS-PR, noise-affected verification, and the distributions of anomaly scores, we demonstrate the effectiveness of IGAD further improving the performance of different models in multivariate time series anomaly detection tasks.
Related Work
2.1 Multivariate Time Series Anomaly Detection
Traditional machine learning methods, such as LOF (7]. OC-SVM [4]. SVDD (SI, and Isolation Forest BT are widely used in anomaly detection. More advances include MPPCACD 655 and DAGMM [75], which integrate density estimation with deep representation learning. Clustering- based approaches, such as Deep SVDD [43, optimize a hypersphere to enclose normal samples, while extensions like Fuzzy C-Means (29 offer alternatives. In addition, deep models for sequential data 46 have been proposed. Contrastive learning-based methods, such as CARLA [I]
DCdetector 666] TS-TCC [15] and CoST [57] are also well-designed AD models or powerful representation learning models that can be used for downstream tasks like AD.
Reconstruction- based methods trained in a self-supervised manner to regenerate inputs with high accuracy have made notable progress. Early methods include LSTM-based encoder-decoder mod- els 335 36], with Onniln [50] a further development. DAGMM (75), MSCRED [70],
MTAD GAT 772,, USAD 4, CAE-M 771,, FGANomaly [13, Anomaly Transformer 622 M2N2
26 and XGBoost- based for in-core neutronare detectors [63 are also proposed as competitive coordinates. Most recently, spatial association- aware SARAD IO has been proposed with joint time-spatial features. CATCH (60] performs MTS AD with frequency patching. Meanwhile, ad- vanced time series foundation models such as OFA 77), TimesNet [88] FITS
64, Peri-midFormer [59] and ModernTCN [34] have shown their powerful abilities in various time series tasks, including reconstruction-ben anomaly detection. However, one challenge with which these methods may struggle is over generalization 48), where abnormal inputs are too well reconstructed 6 39
2.2 Idempotent Generative Network
The first work to raise the concept of idempotent generative network can be found in Shocher et al. 47], where they use this architecture to generate novel image samples belonging to a specific domain. Reconstruction objective, idempotent objective and tightness objective are included ensure potential instances lying on the target manifold, mapping a random vector to the manifold and tightening the manifold to avoid unnecessary expansion. Recently, TrajCLIP [67] has employed this concept to predict the trajectory of pedestrians.
addition, it is also used to perform conditional generation [42] and test-time training [14]. However, the applicability of this theory to address limitations in the time series domain has not been fully explored.
3 Method
3.1 Overview
In this paper, we propose IGAD, a novel and flexible module based on idempotent reconstruction, to further improve existing reconstruction-based anomaly detection methods. IGAD introduces two key objectives: the idempotent objective, ensuring that both æ and f()) lie the target manifold for improved robustness and sensitivity, and the tightness objective, which tightens the target manifold to mitigate the issues of over generalization. The overall architecture shown in Fig 2
3.2 Problem Setting
We define an MTS dataset as {x1 { 23; In} and each time point & E R* where n represents the total number of time instances and l represents the number of variables in MTS. In practice, we define a window size w, selecting every w time points as time interval, that is, r² E I g for i to m, with m the number of time intervals. Furthermore, we denote each
3

---

(x.
Reconstraction-bart MTS Anomuly
Detection Models
Afteation-based
RNN-based
CNN-based
LM-hased
GiN-bbased
GAN-based
Input Time Series
Output Time Series
Fiee crlton Noreaa
Fost Fouser Tosnkinns
Nesti Turiing
JUGA
lantace
Real
Time
Mat [(0)]
Fagss (P3n)(
Fi
Jinai
The overall architecture of reconstruction- based methods integrated with our proposed module IGAD. The upper part is the conventional reconstruction branch, which can be used to calculate Crecon [). The lower part shows the workflow of IGAD, where the arrow - and the blue arrow - in the right manifold stand for the training and frozen model, i.e., f(.) and f'(), respectively. After acquiring an augmented z² which consists of normal instances, it is passed through (=") and (=(()) to calculate Lisem and Light shown Eq. and Eq.(8)
Figure 2:
existing reconstruction-basee method as an idempotent generation function ʃ(+) that takes a time interval or vector z as input and then produces the corresponding output ((x*) or f(=). Here, we should mention that reconstructing each normal time instance well is equivalent to reconstructing each normal time interval well, since each time interval contains w time instances.
3.3
Optimization Objective
3.3.1
Reconstruction Objective
As crucial standard for reconstruction-bent based methods, they should well rebuild normal time intervals to produce minimal reconstruction loss. This objective can be achieved perfectly if f(x') can be as close to itself possible for each æ'. It is widely assumed that each x' in the training dataset is in normal state. To evaluate the distance between each x² and its reconstructed sample xtecon by f fxx), define this objective as:
1 Σ
Drecon (x, ((x))
|
1.ooooll with Crecom (ar, ((x)) Ey (Dreooon (2, [())),
(1)
where m is the total number of windows defined above. Based on this, we can present our target manifold, which contains only normal instances, more formally as:
Marget= (a* f(x') r'} {zy for each x, E x²).
(2)
This means that these models should reconstruct the samples well enough to make sure that each normal instance lies on the target manifold. However, they may exactly reconstruct abnormal instances, which affects the final detection accuracy. To further explore this, we introduce the following optimization objectives.
3.3.2 Idempotent Objective
The previous work performs a mapping from source distribution Pz (standard normal distribu- tion) to Pa to generate novel image samples. However, in the field of MTS AD, should inject more

---

crucial information into P. for a more controllable and reliable generation. Moreover, in practical situations, the MTS data collection process may be influenced by disturbances and fluctuations from physical environments, which can be natural noise. If a model is able to extract valid features from noisy normal time series data, indicates that the model has higher sensitivity to the intrinsic normal patterns. Given these, we reform the role of this objective to extract more natural features from normal instances, further balancing robustness and sensitivity rather than generating novel samples. Concretely, we make an FFT for each original time interval x' in order to convert the time-domain signal to a frequency-done process can be presented as:
signal, and the result usually takes a complex form. This
- x Re (R((**((F)) + i ImmIImX
where F is the Fourier transform operator, and x/' stands for the j-th time variable in the i-th time interval. X X+ is the transformed data with F representing the frequency variable in the frequency domain. Re(-) and Im(-) are selected obtain the real and imaginary part of (F).
After acquiring these, we calculate the mean and standard deviation of Re(X and Im(X for the time interval z', denoted as Pleal Ping Teal and iimag respectively. Now we can define two separate distributions on the real part and the imaginary part as Pi N(ufeat ( and
Pimay N (H imgg). Then, Sreal and Fimag can be randomly sampled from Pieal and Pi Pimag to get the rebuilt frequency variable:
Fri
rooon
real
2
iimag
(4)
By performing inverse fast Fourier transform on F which is defined as:
Preonn we can get the final modified latent vector 2².
Re(FF (Ficoo)))
where 2' is the augmented vector for time interval x². Instead of randomly sampling 21, we inject the intrinsic features of normal instances into the final 2$ while introducing possible randomness of real applications to ensure that Murget can contain enough normal instances. Then we utilize f(((2))) = -f(=) to strengthen the reliability of Murget. reconstructing time intervals with normal features as well as possible. To avoid the instability during training caused by MSE Loss being overly sensitive to potential noise, the optimization objective can be formulated as:
m
Datem (f()). m
-1((e)-(((EE)
3.3.3 Tightness Objective
By introducing the reconstruction objective and the idempotent objective, we pay more attention to the abilities of reconstruction and recognition of normal data patterns. However, we must take one extreme case into consideration: If one model learns a mapping that can be formulated as f(=") = 2t, that is, their outputs are just their inputs, they will satisfy all the mentioned objectives perfectly while losing their meaning in MTS anomaly detection.
Here, we further address the issue of over generalization from the perspective of the defined Marget- Specifically, as shown in 3ig there are two different flow paths of gra- dients in models when minimizing the idempotent objective defined as
Narnal Instane
Medificd listanee
(6). The first path is marked with green line, performing f(((=)) to ensure that z' is better mapped to Margett However, the second path marked with red line imports the po- tential risk of expanding the target manifold to include all visible sam- ples. This phenomenon can explain the reason for over generalization: ac-
Absocuee lastunce
Target ManiSod
Exepaniin
Expanded Mantisid
The expansion of manifold results in potential ab- normal instances are included in the target manifold Murget cident expansion of manifold in a model. It becomes necessary to encourage Murges be decorated adequately and reasonably to contain more normal instances, as well as avoid unnecessary expansion so that abnormal samples are not included in Murges- To address these, we divide our idempotent objective into two parts shown in Fig First, we exclusively optimize with respect to the first
Figure 3:
5

---

instantiation of f(-), while freezing the second instantiation as a fixed copy of the current state of f()), denoted as (.). Then the first part of the idempotent objective can formulated as:
= R. Dute (ftm( (())), = 77 ((=)) - S(((2")) Cidem (f(2),
1 (Datem (Fa)
(7)
Here, f'()) can be viewed as a static mapping function, so Lidem is designed to prevent the potential expansion of the marked red path in Fig by bringing f(=') and (=")) closer. Meanwhile, to
tighten Maryet and exclude potential abnormal instances, we present the second part of the idempotent objective as tightness objective. As we mentioned earlier, the red path in Figßteends tends to expand Marget- This inspires us that the Inverse Effect of this path can be selected to further tighten Murget- Concretely, we only optimize the second instantiation of f() and treat the first instantiation as frozen, using the opposite of the distance between (=") and f()) to simulate this inverse effect:
Digu ("s) 1
> m
((s)) s('(=)) Ligu ( (=)) (=)) = Dogg (1 (())
(8)
Due the relationship between idempotent objective and tightness objective which can be shown as: Lidem (f))) = Ligm (f'n(((()))),)))))))(())) ff)),
an adversarial training strategy is introduced to make sure that Murget strongly focus on normal time instances, while tighten Murget that potential abnormal instances are dropped out. However, there possible adverse effect in Light is that it may perform excessive modification to minimize its values, which increases the instability of the gradient during training. Hence, we smooth Light by:
Lught
Light tahn((
)aCCssoo
(10)
a-Crocon
where a is the control parameter. The meaning of this operation can be concluded that if certain x² has larger reconstruction loss, it is mapped far from Murga and it is unnecessary push the model so strong based on zf to effect the whole training process.
3.3.4 Final Objective
The final objective of a model can be divided into four parts: reconstruction loss (Crecun), idempotent loss (Cidem), tightness loss (Cight) and other auxiliary loss (Laux) each original model. We provide
the proofs for the coverage to normal instances in Appendix B and the final objective is:
L Avco Lacon + A Litem + Augh + Aun Caux = Ez, Dree Descon Vidam Dutem Aught Aun Danx). (11)
4 Experiment
4.1 Experiment Setting
4.1.1 Dataset and Baseline
In our experiments, we selected four redesigned public datasets commonly used for MTS AD in 32
including SMD) from (50), MSL. from 24 PSM from [1] and SMAP from 24
We have selected 15 reconstruction-bast multivariate time series anomaly detection methods in
experiments, including: (1) Specific models designed for MTS AD: SARAD [00, Anomaly
Transformer (62), FGANomaly [3],, CAE-M 771. MTAD-GAT 72), MSCRED 70, OnimAnomaly
(50 and DAGMM [75): (2) Time Series Foundation Models: FITS (64) Peri-midFormer [59],
TCN 34 OFA 774 and TimesNet (58). These basic models contain different intrinsic
architectures, including GNN [61, Attention Mechanism [53 27), GAN [18], GPT2 [41], and others
shown in Fig 2 Some models unavailable are re- re-implemented from Tuli et al. [52]. More detailed introductions and descriptions of these models can be found in Appendix C
4.1.2 Implemention Detail
The latest study on datasets and benchmarks 32 has designed well-organized datasets and searched for optimal hyperparameters in optimizer, learning rate, and weights of existing loss functions for

---

Table 1: VUS-PR reconstruction- models under different real-world datasets. All experi- mental results are repeated with five random seeds and reported as Mean + Standard Deviation. The model-level improvement ratios Amodel and the dataset-level improvement ratios Adsta calculated. Meanwhile, Wilcoxon signed rank tests are performed for anomaly scores, where we use *** **, and * to denote statistical significance in 1 1%, 5 and in cases where p-values are greater than 0.05.
Model
Venue
MSL
w/o IGAD
w/IGAD
Arros (%)
p-vauue
w/o
w w/IGAD
Ameei (%)
p-vauue
CATCH
ICLR, 2025
0.1904 0.3044
0.1970
+3.47
0.0331 0:0000
0.0234
0.006
+0.91
M2N2
AAAL 2024
0.0211
0.0362
-71.56
000
0.2021 2:182
0.8250
+307.61
ICLR.
0.0409
0,0551 +-002
-31.72
444
20000
0.0107
-22.94
ModernTNN
ICLR. 2024
0.1521 2:00019
0,1554 0:000
+78.47
0.0486
0,0620 2:001
-27.57
Peri-midFameer
NawrIPS, 2024
0.1545 1003073
0.1573
1 2007
+1.81
0.0330
0.0856
+7.88
SARAD
NeurPS, 2024
0.2266
1.03977
0,2256 1:00501
-0.44
0.0277 1001
0,0604
1
-81.95
TimesNet
ICLR, 2023
0.0741
0.0759
+2.43
0.0072
0.0073
+1.39
OEA
NeurIFS, 2023
0.0559
0.2077
0.1064
-90.34
0.2000
1-0.271
-26.51
A.T.
ICLR, 2022
0.0259 000005
0,0121 10002
-62.55
0.0063 2:0000
0,0061
2:0030
+1.59
PGANomaly
TKDE, 2021
0.3611 100991
0,3615 1:00111
+0.11
0.0514 1-12
0,0578
-12.45
CAP-M
TKDE, 2021
0.0888 1:0.40
0,1863
1.00
-2.82
0.0043 300
0,0045
1.099
+4.65
XX4
MTAD-GAT
2020
0.3761
0.1169
+000200
-10.76
0.1731 20
OmmiAnomally
KDD, 2019
0.2096
2 0092
0,2139 1:0221
+2.05
0.0052 200
0.0087
-67.31
MSCRED
AAAI 2019
0.3220 100
0.3245
1 10000
+0.78
0.0111 103000
0.0102
1.00
-8.11
DAGMM
ICLR, 2018
0.0322 0303
0,1338
1.0m
+315.53
0.002 0.0000
0,0068
1.000
-61.90
Ama
Mean: 0,1554
Menn: 0,1752
+12.71
Mean: 0,0416
Mean: 0.0920
+120.89
Dataset
Model
Venue
PSM
SMAP
w/o IGAD
/IGAD
Deses (%)
p-value
IGAD
//GAD
p-value
CATCH
ICLR, 2025
0.1281 02055
0.1326 :
+3.27
0,2882 00002
0.2931
+-0000
+1.70
M2N2
AAAI, 2024
0,2989 0.005
0,3010
0.921
+0.70
0.1934 2:33
0,1973
2-0845
+2.02
FITS
ICLR. 2024
0.1163 2.0.90
0.1173 1.0000
+41.6
0.2704 103116
0,2851
1.001%
+5.44
ModernICN
ICLR, 2024
0.1383
0.1337
-3.33
vee
0.4561
0:4144
Peri-midFomner
NeurIPS, 2034
0.1310 0.395
0.1311 1.00004
+0.08
0.5064 1
0,5067
+0.06
SARAD
NeurIPS, 2024
0.1499
: a.305
0.1550
: 0.000
+3.40
0.8469 0.309
0.8477 0.11766
+0.09
TemesNet
ICLR, 2023
0.1174 0.201
0.1297 0.0022
-10.18
0,2678 1.835
0,2731 1-0999
+2.09
OFA
NewrIPS 2023
0.1261 2:0000
0,1405 300,00
-11.42
0,2429
0,2969
+1.37
ICLR,
0.1158
0,1517
X8C
1:1100
RGANomaly
TKDE, 2021
0.1970 0.099
0.183 1.00511
-6.70
0,9835 111912
47.00
CAE-M
2021
0.1503 1 0.306
0.1504
1 00020
.0.07
0.0736 0
0.0736 0.001
0.00
MTAD-GAT
ICDM. 2020
0.1433 0.30s
0.1785
0.30
+24.56
0,2320 4:277
0,5131 1:14
+121.16
OmniAnomaly
KDD, 2019
0,11 0.3090
0,1131
000000
+0.25
0.0780
0,9060
+1061.51
MSCRED
2019
0,1902 0.11
0.1727
+000
-9.20
-32.00
DAGMM
ICLR, 2018
0.1672 03005
0.1675
#0.18
0.0748 2015
0.1083 0:140
+44.79
Aesaa (%)
Mean: 0,1542
Mean: 0,1592
+3.28
Meon: 0.3223
Meon: 0.4015
+25.83
the majority models included in this study, For some of our selected base models, which are not temporarily imported, we set their hyperparameters in their original papers or repositories as optimal ones. Then these models are also integrated into this proposed pipeline to run in a universal data flow. Under these settings. Are and Amux are fixed, then intervals 10.1, 1.0] with step size of 0.1 for Xitem and Aiiht: as well as [1.1, 1.5] with a step size of 0.1 for a are used for a detailed grid search. Optuna 2 is selected for the search process. We program our codes with Python 3.8. 13, PyTorch 1.13.0, CUDA 11.7 and Ubuntu 18.04 on a single NVIDIA RTX 3090 24GB GPU. All experiments are conducted under the same environments. Meanwhile, in our selected models, OFA 774 is an anomaly detection model based on GPT-2 [4], so it is trained after loading the pre-trained weights from huggingface. For the other models, they are trained from initialization according to the random seeds, following the standard training principle. More information on these can be found in Appendix
4.1.3 Evaluation Metric
It has been highlighted that traditionally used MTS AD metrics such as F1, AUC-PR, AUC-ROC and Affiliation-F1 could show potential evaluation issues, while in comparison, VUS-PR emerges as the most robust, accurate and fair evaluation measure [32]- Given these, VUS-PR is selected as our key metric. For further detailed information, we also record the rest of these metrics, and more results such as hyperparameter analysis and visualization can be found in Appendix E
Following the work pipeline in Liu and Paparrizos
we use reconstruction errors for MTS AD. Concretely, given the original time series X = {ro, 21. Z2 .I., and the reconstructed recon {a recon
2
recon recon recon) ). the reconstruction errors e can be calculated as:
x22
El
I x | 1|2 t = 0, 1,
1,
(12)
7

---

0.73
VUS-PA PR wit IGAD
VUS-PM without IGAD
VUS PR with IGAD
VUS-PR without IGAD
0.035
0.0307
0125
0,0256
0.0243
B,9225
0.0214
VIS
0.120
0.000
0,0455
0,0420
0. 0.0396
0.0374
0.003
176
5%
10%
15%
20%
0.000
176
5%
10%
15%
20%
Ncise Weight
Noise Weight
VUS-PR under noise for SARAD MSL.
(b) VUS-PR under noise for A.T. on SMD.
Figure 4: Performance under different weighted noise. The weights of noise vary from 1% to 20%.
8; = Et + de, t 0, O,I, .2
(13)
where a, denote other auxiliary detection scores defined in the original models. Then, the anomaly scores for each time point can be represented as spam with normalized 8, using Sealer. Finally, anomalies are detected using the threshold 5 = + 3a, where and a are the mean and standard deviation of morm which can be shown as:
f1,
if spam > 8 (Abnormal)
Label(s? norm
(14)
10
if shorn S 8 (Normal)
4.2 Result
4.2.1
Anomaly Detection w/ or w/o IGAD
To demonstrate the effectiveness of IGAD, we compare the performance of different methods on four real-world datasets. We calculate the improvement ratios from the perspectives of models (Amode) and datasets (Aama). The results in Tab[ssow that applying IGAD leads to noticeable improvements in performance. From the perspective of models, nearly 87% of the experiments (52 out of 60) show improvements, while more advanced performance observed across the four real-world datasets at the dataset level. Surprisingly, it is found that CAE-M [71] achieves average VUS-PR that is more than three times higher on dataset SMD after applying IGAD, and OmniAnomaly [50) even shows ten-fold improvement dataset SMAP. In addition, since varying degrees of enhancement are observed, Wilcoxon signed rank tests are conducted to verify statistical significance. More than 94% of the improved experiments (49 out of 52) show statistical significance. Meanwhile, also observe limited drops in performance for some models when experiments are conducted on specific datasets, which leaves room for further investigation into the reasons behind these phenomena.
4.2.2 Balance of Robustness and Sensitivity
In this part, to verify the balance of robustness and sensitivity, artificially introduce different weighted Gaussian noise into the original time series selected for testing. Specifically, we first sample Gaussian noise from the standard normal distribution (00 I) with the dimension as the testing data. Then the sampled noise is multiplied by noise weight variation [0.01, 0.05, 0.1, 0.15, 0.2] before being added to the time series for testing. In Fig 4 more cases shown to further support these. In Fig ((a) it can be observed that the model incorporated with IGAD not only shows higher VUS-PR than the different weighted noise. In comparison, the model without IGAD shows a downward trend with higher noise weights. In Fig 4bb) the model with IGAD also maintains stability and outperforms when faced with more challenging tasks for weights of 15% and 20%.
without IGAD, but also maintains stability and efficiency when meeting
4.2.3 Difference in Distributions of Abnormal Scores
According to the detection strategy shown as 14), an abnormal time instance time should have a relatively higher anomaly score sporm to be detected abnormal. Therefore, we further explore the

---

Dourca Jnd
Lrdrtpp 4033
1v (egraapieer Aanoie
READ
(a) Anomaly scores for CAE-M on SMD,
(b) Anomaly scores for OmniAnomaly on SMAP.
Anomaly score distributions for different models and datasets. In each sub-figure, the left one shows the distribution without IGAD, and the right one with IGAD.
Figure 5:
Table 2:
Results on different types of loss func- tions. i denotes the average value of all the mean values reported in Tab
Table 3: Results on different operation subset. denotes the average value of all the mean values reported in Tab
Type
Lidem
Light
VUS-PR
Å (%)
Component
VUS-PR
A (%)
Lidem
feghtt
Light
only Li
L1
Li
0.2080
N.A.
L + L2
Li
L2
0.1619
-22.16
0.2080
N.A.
C2 f
L2
L1
0.1857
-10.72
0.1914
-7.98
only L2
L2
L2
0.1684
-19.04
X
Invalid
N.A.
C, is the MAE Loss and C2 the MSE Loss. We calculate VUS-PR as the average values of all experiments shown in [15 basic models and datasets) of Litem and Light This experimental strategy is also performed in Tab 3]
V
0.1822
-2240
V
X
X
0.1704
-18.08
0.1803
-13.32
each combination
X
V
Invalid
N.A.
X
0.1684t -19.04
distributions of the anomaly scores and calculate the areas of overlap in their corresponding density maps. 5 we show the results of two different models on two different datasets. The results demonstrate that after using IGAD, more distinguishable distributions of the anomaly scores can be observed for normal and abnormal time instances, as well as smaller overlap areas.
4.3
Ablation Study
4.3.1
Different Loss Functions
Existing reconstruction-basse MTS AD methods commonly use MSE Loss (C2) to measure the differences between the original and reconstructed time series due its sensitivity to large errors, well-defined mathematical properties, and stability during optimization. However, we propose that MAE Loss (C1) is more suitable choice for idempotent objective (Lide)) and tightness objectives (Lright) because of its robustness to natural noise, which helps prevent over-tuning. We select different combinations of loss functions. The experiments summarized in Tab reveal consistent drop in performance when using MSE Loss, theoretically because its sensitivity to noise and randomness introduced by sampling artificially destabilize the training process.
4.3.2 The Effectiveness of Each Objective
Here, we conduct an ablation study to assess the contribution of each component in our proposed module, specifically the effectiveness of the idempotent objective (Cidem). tightness objective (Crigh), and the smoothness operation of the tightness objective (Cigh). The idempotent objective is designed to modify Mtarger to adequately capture normal instances. The tightness objective further refines much as possible, excluding potential abnormal instances. Furthermore,
Murget by tightening ensuring the smoothness of the tightness objective helps stabilize the training process. As shown in Tab the models that incorporate all three proposed components achieve the highest performance.

---

Table 4: VUS-PR on foundational models AE and VAE with different datasets.
Dataset
Model
SMD
MSL
IAA
w/ IGAD
Awodd (%)
p-value
w w/o IGAD
w IGAD
Awodd (9)
p-value
AE
0.3113=0.0029
0.3823=0.0053
+22.81
0.0083=0.000 00003
0.0370=0.0208
+345.78
VAB
0.355620.0041
0.355610.0037
0.00
0.0086-0.0004
0.008710.0002
+1.16
Dataset
Model
PSM
SMAP
w/oIGAD
w w/IGAD
Amoda (%)
p-value
w w/o IGAD
w W/GAA
Amode (%)
p-value
0.1437=0.0001
0.1640=0.0098
+14.13
0.0969=0.0007
0.914920.0784
+844.17
0.1464=0.0004
0.1465#0.0003
+0.07
se.
0.0947=0.0014 0.0930=0.0014
Original Signal
Augmented Signal
Augmented Signal
Augmented Signal
Augmented Signal
Amplitude
Time Step
Figure 6: A comparison of different signal augmentation strategies. Left: Frequency Resampling
Strategy in IGAD; Middle: Time-Domain PCA with k = 5; Right: Time-Domain PCA with k: = 10.
4.4 The Effect of IGAD on Foundational Models
As indicated in 32 models with simpler architectures tend to yield better performance. Given this point, we perform further experiments to explore the effect of IGAD on foundational neural architectures, including AutoEncoders (AE) and Variational AutoEncoders (VAE), and the results shown in Tab[4] The key findings here can be concluded as: (1) Before applying IGAD, VAE often shows higher levels than AE on the four selected datasets; (2) After applying IGAD, AE with IGAD shows higher improvements than VAE with IGAD, and even has better performance than VAE with IGAD; (3) AE and VAE achieve comparable results compared to certain complex models, which is consistent with the mentioned findings in [32]
4.5 Comparison between Frequency Resampling and Time-Domain PCA
FFT-based augmentation excels creating diverse, globally consistent variations by perturbing the entire spectral profile, making it highly effective for dynamic time series. Conversely, time-domain PCA is confined perturbing few learned, high-variance linear patterns, which limits the diversity and fails to capture complex dynamics. Moreover, the fixed, data-independent basis of FFT makes lightweight, parameter-free module, avoiding the hyperparameter tuning for the number of principal components k inherent to PCA. The results of comparison can be found in Fig 6
Conclusion
This paper introduces IGAD, novel module which can be easily integrated into reconstruction-bee based methods to enhance their effectiveness in detecting anomalies in MTS. Meanwhile, we conduct further experiments and explorations, addressing the issues of over generalization and overall performance balance of these models from the perspective of manifold. With defined optimization objectives, we aim to not only modify the target manifold, balancing the robustness and sensitivity inherent normal patterns of time series, but also tighten the manifold to exclude potential abnormal time instances. The expe experimental results demonstrate the effectiveness IGAD and show its great potential for further applications in MTS anomaly detection.
10

---

Acknowledgements
This work was supported part by Key R&D Program of Zhejiang Program No.2024C01065, in part by NSFC under grant No.U23A20326, and in part by State Key Laboratory of ICT Project No.ICT2025A09.
References
[1] Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Proceedings of the 27th ACM SIGKDD conference knowledge discovery & data mining, pages
2021.
[2] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623-2631, 2019.
[3] Ana Almeida, Susana Brás, Susana Sargento, and Filipe Cabral Pinto. Time series big data: a survey on data stream frameworks, analysis and algorithms. Journal of Big Data, 10(1):83,
2023.
[4] Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A Zuluaga. Usad: Unsupervised anomaly detection on multivariate time series. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pages 3395 -404, 2020.
[5] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE trunsactions on paltern analysis and machine intelligence, 35(8): 1798-1828, 2013.
[6] Ane Blazquez-Garefa, Angel Conde, Usue Mori, and Jose Lozano. review on out- lier/anomaly detection in time series data. ACM Computing Surveys, 54(3):1-33, 2021.
[7] Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identify- ing density-based local outliers. In Proceedings of the 2000 ACM SIGMOD International Conference Management of Data, pages 93 104, 2000.
[8] Ayan Chatterjec and Bestoun S Ahmed. lot anomaly detection methods and applications: A survey. Internet of Things, 19: 100568,,, 2022.
[9] Yongliang Cheng, Yan Xu, Hong Zhong, and Liu. Hs-ten: A semi-supervised hierarchical stacking temporal convolutional network for anomaly detection in iot. In 2019 IEEE 38th International Performance Computing and Communications Conference, pages 1-7. IEFE, 2019.
[10] Zhihao Dai, Ligang He, Shuang-hua Yang. and Matthew Leeke. Sarad: Spatial association- aware anomaly detection and diagnosis for multivariate time series. In 38th Conference on Neural Information Processing Systems (NeurIPS 2024), 2024.
[11] Zahra Zamanzadeh Darban, Geoffrey I Webb, Shirui Pan, Charu C Aggarwal, and Mahsa Salehi. Carla: Self-supervised contrastive representation learning for time series anomaly detection. Pattern Recognition, 157:110874, 2025.
[12] Flavio Di Martino and Franca Delmastro. Explainable for clinical and remote health applica- tions: survey on tabular and time series data. Artificial Intelligence Review, 56(6):5261-53) 2023.
[13] Bowen Du, Xuanxuan Sun, Junchen Ye, Ke Cheng, Jingyuan Wang, and Leilei Sun. Gan-based anomaly detection for multivariate time series using polluted training set. IEEE Transactions on Knowledge and Data Engineering. 35(12):12208 1220-112199 2021.
[14] Nikita Durasov, Assaf Shocher, Doruk Oner, Gal Chechik, Alexei A Efros, and Pascal Fua. It:: Idempotent test-time training, arXiv preprint ar10v:2410.04201, 2024.

---

[15] Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, Xiaoli Li, and Cuntai Guan. Self-supervised contrastive representation learning for semi-supervised time-series classification. IEEE Transactions on Patlern Analysis and Machine Intelligence, 45 120:15604-1160 2023.
[16] Mojtaba A Farahani, MR McCormick, Robert Gianinny, Frank Hudacheck, Ramy Harik, Zhichao Liu, and Thorsten Wuest. Time-series pattern recognition in smart manufacturing systems: A literature review and ontology. Journal of Manufacturing Systems, 69:208-241, 2023.
[17] Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality to detect anomaly: Memory- augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1705-1714, 2019.
(18) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commumications of the ACM, 63(1 1):139-144, 2020.
[19] Soufiane Hayou, Nikhil Ghosh, and Bin Yu. Lorat: Efficient low rank adaptation of large models. Proceedings of the 41st International Conference on Machine Learning, pages 17783- 17806, 2024.
[20] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In International conference on learning representations, 2017.
[21] Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504-5000 2006.
[22] Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural networks, 4(2):251-257, 1991.
[23] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, (2):3, 2022.
[24] Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. Detecting spacecraft anomalies using Istms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 387-395, 2018.
[25] Simon Kamm, Sushma Sri Veekati, Timo Müller, Nasser Jazdi, and Michael Weyrich. A survey on machine learning based analysis of heterogeneous data in industrial automation. Computers in Industry, 149: 10393,, 2023.
[26] Dongmin Kim, Sunghyun Park, and Jaegul Choo. When model meets new normals: test- time adaptation for unsupervised time-series anomaly detection. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 13113-13121, 2024.
[27] Diederik Kingma and Max Welling. Auto-encoding variational bayes. International Confer- ence on Learning Representations, 2014.
[28] Gen Li and Jason Jung. Deep learning for anomaly detection multivariate time series: Approaches, applications, and challenges. Information Fusion, 91:93-102, 2023.
[29] Jinbo Li, Hesam Izakian, Witold Pedryez, and Iqbal Jamal. Clustering based anomaly detection in multivariate time series data. Applied Soft Computing, 100: 2021.
[30] Xiaohui Lin, Zuoyong Li, Haoyi Fan, Yanggeng Fu, and Xinwei Chen. Exploiting negative correlation for unsupervised anomaly detection contaminated time series. Expert Systems with Applications, 143:123535, 2024.
12

---

[31] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In Proceedings of the IEEE International Conference on Data Mining, pages 413 413-422. IEEE, 2008
[32] Qinghua Liu and John Paparrizos. The elephant in the room: Towards reliable time-series anomaly detection benchmark. In The Thirty-eight Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track, 2024.
[33] Shih- Yang Liu, Chien-Yi Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang- -ing Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. In International Conference on Machine Learning, pages 22100-32121. PMLR, 2024.
Shin-Yang
[34] Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024.
[35] Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, Punect Agarwal, et al. Long short term memory networks for anomaly detection in time series. In Proceedings of the European Symposium Artificial Neural Networks, Computational Intelligence and Machine Learning, volume 2015, page 89, 2015.
[36] Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, and Gautam Shroff. Lstm-based encoder-decoder for multi-sensor anomaly detection. arXiv preprint 107.00160,, 2016.
[37] Harry Nyquist. Abridgment of certain topics in telegraph transmission theory. Journal of the AIEÉ, 214 1928.
[38] Bruno A Olshausen and David Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 38 (6588):6007-6666 1996.
[39] Hyunjong Park, Jongyoun Noh, and Bumsub Ham. Learning memory-guided normality for anomaly detection. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14372-14381, 2020.
[40] Chen Qiu, Aodong Li, Marius Kloft, Maja Rudolph, and Stephan Mandt. Latent outlier exposure for anomaly detection with contaminated data. In International conference on machine learning, pages 18153-18167. PMLR, 2022.
[41] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[42] Niccolò Ronchetti. Conditional idempotent generative networks. arXiv e-prints, pages arXiv- 2406, 2024.
[43] Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International Conference on Machine Learning, pages 4393 4402. PMLR, 2018.
[44] Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the support of a high dimenssinn distribution. Neural Computation, 13(7):1443- 1471, 2001.
[45] Claude Shannon. Communication in the presence of noise. Proceedings of the IRE, 37(1):
10-21, 1949.
[46] Lifeng Shen, Zhuocong Li, and James Kwok. Timeseries anomaly detection using temporal hierarchical one-class network. Advances in Neural Information Processing Systems, 33: 13016-13026, 2020.
[47] Assaf Shocher, Amil Dravid, Yossi Gandelsman, Inbar Mosseri, Michael Rubinstein, and Alexei Efros. Idempotent generative network. In Proceedings of the International Conference on Learning Representations, 2023.
13

---

[48] Junho Song, Keonwoo Kim, Jeonglyul Oh, and Sungzoon Cho. Memto: Memory-guide transformer for multivariate time series anomaly detection. Advances in Neural Information Processing Systems, 36, 2024.
[49] Qinliang Su, Bowen Tian, Hai Wan, and Jian Yin. Anomaly detection under contaminated data with contamination-immunn bidirectional gans. IEEE Trunsactions on Knowledge and Data Engineering, 36(1 1:560--5620, 2024.
[50] Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pages 2828 2837, 2019.
[51] David MJ Tax and Robert PW Duin. Support vector data description. Machine Learning, 54: 45-66, 2004.
[52] Shreshth Tuli, Giuliano Casale, and Nicholas R Jennings. Tranad: Deep transformer networks for anomaly detection multivariate time series data. arXiv preprint 2XXi:22211.07284 2022.
[53] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
[54] Chengsen Wang. Zirui Zhuang. Qi Qi, Jingyu Wang. Xingyu Wang. Haifeng Sun, and Jianxin Liao. Drift doesn't 't matter: dynamic decomposition with diffusion reconstruction for unstable multivariate time series anomaly detection. Advances Neural Information Processing Systems, 36, 2024.
[55] Shaowen Wang, Linxi Yu, and Jian Li. Lora-ga: Low-rank adaptation with gradient approxima- tion. Advances in Neural Information Processing Systems, 74459905-54931, 2024.
[56] Qingsong Wen, Linxiao Yang. Tian Zhou, and Liang Sun. Robust time series analysis and applications: An industrial perspective. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 4836-4837, 2022.
[57] Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Cost: Contrastive learning of disentangled seasonal-trent trend representations for time series forecasting. arXiv preprint arXiv:2202.01575. 2022.
[58] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis, 2023. URL https://arxiv org/abs/2i 02186
[59] Qiang Wu, Gechang Yao, Zhixi Feng, and Shuyuan Yang. Peri-midformer: Periodic pyramid transformer for time series analysis. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[60] Xingjian Wu, Xiangfei Qiu, Zhengyu Li, Yihang Wang, Jilin Hu, Chenjuan Guo, Hui Xiong, and Bin Yang. Catch: Channel-aware multivariate time series anomaly detection via frequency patching. The Thirleenth International Conference on Learning Representations, 2025.
[61] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4-24, 2020.
[62] Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In International Conference on Learning Representations, 2022. ( htt.tttttt/ooeeeeiiieeee httt::/s/pppprreiieeeeee/rr//rre
[63] Yong Xu, Yun-Ze Cai, and Lin Song. Anomaly detection for in-core neutron detectors based on a virtual redundancy model. IEEE Transactions on Instrumentation and Measurement, 72:1-11, 2023.
14

---

[64] Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. In The Twelfth Interaational Conference on Learning Representations, 2024.
[65] Takehisa Yairi, Naoya Takeishi, Tetsuo Oda, Yuta Nakajima, Naoki Nishimura, and Noboru Takata. A data-driven health monitoring method for satellite housekeeping data based on probabilistic clustering and dimensionality reduction. IEEE Transactions on Aerospace and Electronic Systems, 13(3):1384-144) 1401, 2017.
[66] Yiyuan Yang. Chaoli Zhang, Tian Zhou, Qingsong Wen, and Liang Sun. Dedetector: Dual attention contrastive representation learning for time series anomaly detection. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 2023.
[67] Pengfei Yao, Yinglong Zhu, Huikun Bi, Tianlu Mao, and Zhaoqi Wang. Trajelip: Pedestrian trajectory prediction method using contrastive learning and idempotent networks. Advances in Neural Information Processing Systems, 37:77023-77037, 2024.
[68] Jiahao Yu, Xin Gao, Baofeng Li, Feng Zhai, Jiansheng Lu, Bing Xue, Shiyuan Fu, and Chun Xiao. A filter-augmented auto-encoder with learnable normalization for robust multivariate time series anomaly detection. Neural networks, 170:478-493, 2024.
[69] Zahra Zamanzadeh Darban, Geoffrey I Webb, Shirui Pan, Charu Aggarwal, and Mahsa Salehi. Deep learning for time series anomaly detection: survey. ACM Computing Surveys, 57(1):
1 1-42, 2024.
[70] Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong. Haifeng Chen, and Nitesh V Chawla. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI conference on Artificial Intelligence, volume 33, pages 1409-144 1416, 2019.
[71] Yuxin Zhang, Yiqiang Chen, Jindong Wang, and Zhiwen Pan. Unsupervised deep anomaly detection for multi-sensor time-series signals. IEEE Transactions on Knowledge and Data Engineering, 35(2):2118-2133 2021.
[72] Hang Zhao, Yujing Wang. Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. Multivariate time-series anomaly detection via graph attention network. In Proceedings of the IEEE International Conference on Data Mining, pages 841-850. IEEE, 2020.
[73] Shuang Zhou, Daochen Zha, Xiao Shen, Xiao Huang, Rui Zhang, and Fu-Lai Chung, Denoising- aware contrastive learning for noisy time series. Proceedings of the Thirty-Thing International Joint Conference on Artificial Intelligence, 2024.
[74] Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 36:43322 2023.
[75] Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In Proceedings of the International Conference Learning Representations, 2018.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have explicitly described our claims, motivations, contributions, and experimental improvements in Abstract and Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of proposed method in Appendix F
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification: We provide complete proofs relative to our proposed methods in Appendix A and Appendix B
with defined assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed information on experimental settings in Sect 4.1 Ap- pendix and Appendix E.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have released the codes and datasets in the public repository https:// github. cww httpm//rEchoo////eee Ctmm/PProocool/eepttttttit with included instructions.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
gublic/guides PoodeSummissiooooliie
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips wttp:/cccccccccgccccceeococceeee www.//oooommiiioooooo..oo iwtobbmmsssoooooolomooomo
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We provide detailed information on experimental settings, hardware and software environments, hyperparameter search, and additional instructions in Sect Appendix([land Appendix E.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The results in T are under five random seeds with reported mean and standard deviation. In addition, Wilcoxon signed rank tests are introduced to further demonstrate the effectiveness of our proposed method.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We discuss the needed computer resources in Appendix E.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://nuuipppp http://///////ppp..cccc///////aa
Answer: [Yes]
Justification: We are sure that we have reviewed the NeurIPS Code of Ethics and followed the rules.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the impacts of proposed method in Appendix G
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The datasets used in experiments classic public data, and we have cited the original paper that produced the code packages or datasets.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our work does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, non-standard components. LLM is used only for writing and grammar check.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

Multivariate Time Series Anomaly Detection with Idempotent Reconstruction
Appendix
Contents
1
Introduction
2
Related Work
2.1
Multivariate Time Series Anomaly Detection
2.2
Idempotent Generative Network
3
Method
3.1
Overview
3.2
Problem Setting
3.3
Optimization Objective
3.3.1
Reconstruction Objective
3.3.2
Idempotent Objective
3.3.3
Tightness Objective
3.3.4
Final Objective
4
Experiment
4.1
Experiment Setting
4.1.1
Dataset and Baseline
4.1.2
Implemention Detail
4.1.3
Evaluation Metric
4.2
Result
4.2.1
Anomaly Detection w/ w/o IGAD
8
4.2.2
Balance Robustness and Sensitivity
4.2.3
Difference Distributions of Abnormal Scores
4.3
Ablation Study
9
4.3.1
Different Loss Functions
4.3.2
The Effectiveness of Each Objective
4.4
The Effect of IGAD on Foundational Models
10
4.5
Comparison between Frequency Resampling and Mome-Domiin PCA
10
5
Conclusion
10
A Theoretical Analysis of Over Generalization
25
A.1 Contaminated Data for Training and Over Expressive Models
................................
25
A.2 Optimization Dynamics with Regularization Constraint
25
A.3 Mechanistic Interpretation of Over Generalization
26
23

---

A.3.1 Censity-Driver Error Scaling
26
A.3.2 Latent Manifold Attraction
27
Proof of the convergence for IGAD
28
C Explanations for Basic Models
29
C.I Models
2
for MTS AD
29
C.2 Time Series Foundation Model
31
Hyperparameter Setting
33
E
More Detailed Experimental Results
34
E.1
Distinguishable Distributions of Anomaly Scores and Auxiliary Metrics
34
E,2
Efficiency Evaluation
E.3
Hyperparameter Analysis
45
E,4
Visualization of Latent Space
48
E.5
Maintain Data Patterns under Noise
49
E.6
Nyquist Criteria .eneriaa.......
53
E.7 Comparison with Contrastive baased Models
58
E,8 Codes for IGAD
59
Limitation and Future Work
59
G Impact Statements
59
24

---

A Theoretical Analysis of Over Generalization
As have discussed in Sect conclude the reasons for over generalization that this problem may happen for two factors:
A model incorrectly captures the intrinsic patterns of abnormal series in a contaminated dataset for training. This is unavoidable real-world scenarios because large amounts of time series data are collected under the assumption that all time instances collected are normal points, since a system typically operates correctly under most conditions. However, it is inevitable that the system will have offsets or abnormal states over the long period of the data collection process. Meanwhile, since manually checking each time point for anomalies consumes a lot of human and time resources, these abnormal instances remain in the dataset for training without label scrutiny.
A model has an excessive decoding power, even for abnormal series. This means that model not only learns how to reconstruct normal time instances but also gains the ability to learn how to reconstruct abnormal instances.
In this part, we provide mathematical proofs and analysis for over generalization to explore how these two factors have effect the training process. Concretely, in Sect[A.1] show the explanations in ideal and pure models, which have enough model capacity to learn everything. In addition, in Sect A.2 provide proofs and analysis in more general cases, and the effect of regularization is introduced, which aligns with the traditional principles of model design.
A.1 Contaminated Data for Training and Over Expressive Models
Let Pa and Pa denote the distributions of normal and abnormal instances, respectively. When training data contains contaminated anomalies with rate 1j E [0. 1). the empirical distribution will be transformed to Pirain (1 P, + r/P.. A reconstruction-basee model f X - X trained on the contaminated data aims to minimize the composite objective:
(15) Here, we consider compact support c where Supp(Pz) Supp(Pa) S K and Pz, Pa are absolutely continuous on K. By the universal approximation theorem [22]. in a given assumption space H. for any compact support K c X' containing both normal and anomalous instances, there exists neural network f E H, which can satisfy the following objective:
Crecon (f(-) (1 (1-1)) =p |||= - ||| f())
sup | #EK
f())l12 S Ve > 0.
(16)
This implies that sufficiently expressive models achieve arbitrarily small reconstruction errors on both distributions simultaneously. This theoretical capacity relies on the complexity of the unbounded model. Practical architectures with inductive biases, for example, the commonly selected bottleneck constraint, and other implicit regularization introduced during the design of the model alter the solution landscape.
A.2 Optimization Dynamics with Regularization Constraint
For more general cases in practice, when 1) >> we consider an encoder-decoder-likk mapping function fa( = g (he with a bottleneck dimension sufficiently smaller than the data dimension, that is, drouteneck < daam: which enforces information compression. This is demonstrated strategy in this type of structures because it enforces the learning of compressed representations that retain the intrinsic structure of the data while discarding redundant information [21 38). thus preventing trivial identity mappings [5] and promoting robust feature disentanglement 20 In terms of (15) the optimization dynamics is governed by:
(1 77) E, || fo (z) + E [| 1oln-fo(=-') fo('') + (,)(() 0,
(17)
where R(0) captures architectural constraints through implicit regularization and y quantifies the effective regularization strength from the bottleneck. To elucidate this equilibrium, more generally, we expanding the loss in the parameter space with R(0):
Crecom (fo) = f || fo(x)) (11 (x) + (P RR((). ((())
(18)
25

---

For reconstruction- based methods, R(0) can be introduced naturally to R(0) = (lf Here, R[x=(0] _ Be-p(z)0) [=)(2) with p(=10) is the empirical distribution induced by the encoder with - hy (z). This setting is justified by its alignment with the conservative estimation principle of the traditional design approach in the training process, that is, the model output is constrained to the typical pattern of the data itself, making it possible to reconstruct every time instance well. Crucially, since the training data are contaminated for y > 0, the empirical distribution p(z)0) and E(x](d) are influenced by both normal and abnormal instances. To form the corresponding Euler-Lagraage equation, we take the functional derivative 6Lmmm [a) = 0 to get the expression:
6f9
[l - P)(() + ())a()) fo(z)) ((f(()) E[z[0]),
(19)
We solve this elliptic equation under the strong bottleneck condition (r > (/aa(x)) and ? <
[11 - (x) + (x)))
(x) = [11 (z) +
(20)
(1 (x) P
(1 (1- (. (x)
[
(1 - (=((a) + ()( + (z (2- B[=(0])
(21)
= B[x[0]
(x (c-E(x(8))
(22)
+ [I=)))((c))++PPu (1)a)))
7²
= [|[[
(x BBa]
(1 P)) + Pa(a)
(1- + pp (x))
Tayke Exgunsion
(23)
[8[=00] (1
(x F(x(0))
(1
(a()))))))))
72
x E(2
+
(24)
[1 (x) + (x)}
| - -
A.3
Mechanistic Interpretation of Over Generalization
The analytical decomposition from Eq. 20 to Eq. 24 reveals the fundamental mechanisms that govern the reconstruction behavior.
Theorem When 0 and 7 the optimal funncinn fă (z) satisfies the following expression, which can be acquired from Eq. (24):
(1f( - x112 S
7
Bia
a
I1l
+I[0
1l2
(25)
A(x)
A(x)²
Unit Vector
Anomaly Suppression
x B[r O
(26)
A(a
A(z)2
where A(x) & (1-
P)= (/(a)z)) represents the local data density mixture point and is the term form direction correction. 2- 1
< 2
A.3.1 Density-Driven Driven Error Scaling
The decomposition of the optimal reconstruction function (x) Eq. 24 and 2q. reveals
critical mechanism that governs over generalization. With the defined mixed local density A(x) and
regularization strength % the primary error term A(²) exhibits an inverse proportionality to A(x), leading to the following regimes:
High-Density Regions (7 AA(x)))
72
7
0 = (z) x1|2 ~ O
fz( x2.
(27)
A(x)
A(z)²
Accurate reconstructions dominate as the density of normal instances suppresses abnormal residuals.
26

---

Low-Density Regions (7 ~ A(z)):
72 A(x))2
A(x) R (x)
x||2 ||[
x1|2
0
(x) ~ B[a
(28)
The issue of over generalization may emerge regularization forces reconstructions toward the latent manifold expectation.
A.3.2 Latent Manifold Attraction
The defined E [ encapsulates dual mathematical role within framework. Statisticallyy it is rigorously defined through the encoder-decoder architecture as E[r (0) - 1e (1 where p(=(0) represents the empirical latent distribution generated by the encoder (w))). Geometrically, it also serves as the L2 optimal projection anchor on the learned manifold Murget, which satisfies the following objective:
E [0]] = arg min Ez-P, 1|@ ==122 &a.Musss
(29)
where MMrrget = (=)} - (x)) denotes the manifold induced by the decoder. This dual role establishes E[zr[0] attractor, since both the statistical expectation of the decoder output and the geometric centroid minimize projection errors. These properties explain its ability to govern reconstruction behaviors while remaining sensitive the underlying data density (z).. thus providing a unified perspective to analyze the expansion of the manifold under regularization constraints.
The learning process establishes dynamic equilibrium between reconstruction fidelity and regular- ization forces, governed by the data density landscape. For normal samples Pa. the model aims to preserve the accurate mapping:
(0)
[[(()) - x112 <,
(30)
which preserves the geometric fidelity of normal instances on the manifold Margett Conversely, for anomalies Ta ~ Pa. the regularization term enforces alignment with the latent manifold expectation:
(2a) =e E| (]
([fe(=.) - || < E.
(31)
This competition induces critical contamination threshold 7rii defined by:
Merit sup {1 10) |... Il=. fo E fo(x)113)
(32)
When Terit, the expanded manifold Müngeet Manreet U (f.. P.} exhibits dimensional inflation (dim(M") margea))ss)), causing the anomaly normall separability to collapse.
From this point, inspired by the manifold theorem, we propose IGAD, which can benefit both the balance of robustness and sensitivity and the tightness of the target manifold Manges to climinate potential abnormal instances during training.
27

---

B Proof of the convergence for IGAD
Theorem Under ideal conditions, IGAD can converge to the target distribution, which consists only of all normal time instances for given dataset. For simplification, we select and 2 for x² and 2¹, respectively, We define the generated distribution, represented by Poly), as the PDF of y when fo(z) and 2 ~ P.. Here, we only pay attention to the loss items relative, iee. Lrecou Lidem and Ligght The final loss function can be divided into two parts:
L(0:0') >rreCrocom (0) + Aright (0: 0') + Cidem (f: (0:0')
(33)
Ex
We assume a large enough model capacity such that both terms can obtain a global minimum:
0* arg min L.: 0*) arg min Lidem (0; (0:0*)
(34)
Then, 30" Po. = Pa and for Asdem 1, this is the only one possible Po..
We first demonstrate the global minimum L.- After that, we further verify the global minimum for Lidem- For a given parameter in the parameter space and an input X, (ece((t) is used to calculate the differences between fo(X) and X.
Step Global minimum of Lt given the current parameters l*.
(0; 0*) B2 [D(Jo(x) Arigtt B.g (D((o(fe-())),
(3:5)
= / (o(x) Aright /
4 (for (0)6((())) Pa (z)dz
(36)
Change variables: let = for the left integral and == (a-(2) (a well-learned model should ensure that x and fa- (z) lie on the same Margaa for the right integral. Then (3,( can be transformed into the following formular:
L,(0; -)-00))=1 - = Do Do(u) - Aughu / (Pol))oo(u)ddg
(37)
ff Do(v) (P) Aright Po- (v)
dy
(38)
Regularization for Tightening the Manifold
Let M sup D(y1,y2) where the supremum is taken over all possible pairs yı, U2. Since 4g is non-negative, the global minimum is achieved when:
T a- (y)
M [ (o (n))) Vy
(39)
Step 2: Global minimum of Cidem
Citem (0, 0²) ) = E, D (fo()))))) E2 pg (f)()))
(40)
(41)
Substituting pg- from (9. and exchange the position of 0 and 0* because check the minimum of the inner f for Lidem, instead of the outer f in Lright
Li (0; 0*) = M E. (1( (P
(42)
Taking arg ming of Eq.442): 0" = M arg min E. (1P2(0)
(43)
Given these operations, if Pa- Pa and Aidem < 1, the loss value is O. Specifically, for Aidem 1, 0" Po- P. is the only minimizer because the total sum of the probability must be 1. In addition, any deviation where Po(y) < (v) implies By with Po(v) (2), increasing the loss.
28

---

C Explanations for Basic Models
In our experiments, we select 15 basic reconstruction based models with different structures to evaluate IGAD more comprehensively. We consider the selective strategy of these basic models from two perspectives. First, we select well-designed models specifically tailored for multivariate time series anomaly detection. In addition, we also take time series foundation models into consideration, which can be competitive candidates in various time series tasks, including reconstruction based MTS AD. Here, we provide more detailed descriptions of the selected models for better understanding of their structures.
C.1 Models Designed for MTS AD
We include the following models designed for MTS AD in our experiments: CATCH 60 M2N2 [26]. SARAD [0], Anomaly Transformer (62). FGANomaly [13), CAE-M [7I). MTAD-GAT [22).
MSCRED 77), OnimAnomaly [50] and DAGMM (75].
CATCH [60]: CATCH framework introduces two key innovations for multivariate time series anomaly detection: (1) A frequency patching mechanism that partitions the frequency domain into fine-grained bands to better capture diverse subsequence anomalies, addressing the limitations of coarse-grand grarse-gaiied frequency analysis in existing methods; (2) A novel Channel Fusion Module with dynamic correlation discovery mechanism that employs a bi-level optimization strategy to adaptively learn context-aware channel interactions, clustering relevant channels while mitigating noise from irrelevant ones through masked attention, effectively bridging the gap between thannel-independent and channel- -ddeendent approaches. The framework further enhances detection robustness through dual-domain reconstruction objective based on time and frequency, and a novel point-aligned scoring mechanism that synergizes temporal and spectral anomalies, enabling superior performance in detecting both point and heterogencous subsequence anomalies across varied real-worde and synthetic scenarios.
M2N2 26]: : M2N2 is novel test-time adaptation framework for unsupervised time series anomaly detection to address the new normal problem caused by distribution shifts between training and test data. First, a trend estimation module using exponential moving averages to dynamically detrend input sequences, enabling adaptation to evolving data patterns while preserving underlying dynamics. Then, a self-supervised model update strategy that selectively updates parameters during inference using predicted normal instances, effectively learning new normal patterns while mitigating contamination from anomalies. The approach bridges test-time adaptation with time series anomaly detection through its dual mechanism of trend-awer -wndre normalization and confidence-based parameter adjustment, requiring neither access to training data nor additional supervision. By combining real-time trend adaptation with model fine-tuning on detrended sequences, the method demonstrates superior robustness to distribution shifts across diverse weal-world benchmarks while maintaining computational efficiency suitable for streaming applications.
SARAD [10] SARAD is a novel approach for time series anomaly detection that integrates Spatial Association Reduction with data reconstruction via Transformer-baat based models. Its innovation lies in capturing both temporal and spatial dependencies within multivariate time series data, challenge that previous methods addressed largely only from temporal perspective. The key feature of SARAD is its dual focus on data reconstruction errors and progression reconstruction errors, where the latter focuses on spatial changes in anomaly propagation. SARAD leverages Multi-Head Self-Attention from Transformer layers to cap- ture spatial relationships between features over time, and uses this information in conjunction with progression-basee that may struggle with short-range anomalies or overlook spatially distributed anomalies, SARAD effectively identifies anomalies by observing how spatial associations evolve, even when the underlying data distribution is shifted.
metrics to robustly detect anomalies. Unlike traditional models
Anomaly Transformer [62]: Anomaly Transformer introduces a novel approach to time- series anomaly detection by leveraging Transformer-based model and focusing on the concept of association discrepancy. This model incorporates a dual-branch mechanism within the Anomaly-Attentin module, which enhances its ability to distinguish between nor-
29

---

mal and abnormal data points. The key innovation lies in the use of Association Discrepancy. measured through symmetrized Beilbackk-eebbbee divergence, between the learned series association and prior association. By employing a minimax strategy during optimization, the model minimizes the prior-association in the early phase while maximizing the asso- ciation discrepancy in the later phase, ensuring a more robust distinction between normal and abnormal time points. This method improves the detection performance by forcing the model to focus more on non-adjacent time series data, thus enhancing its sensitivity to anomalies. The final anomaly score a combination of reconstruction loss and association discrepancy, ensuring that both components contribute to detection, offering more accurate and interpretable framework for time-series anomaly detection.
The proposed model introduces novel approach to anomaly detection by leveraging Generative Adversarial Networks in the context of multivariate time series data, with a particular focus on handling polluted or noisy training sets. The core innovation lies in the use of GAN framework, where the generator learns to reconstruct normal time series, while the discriminator distinguishes between real and reconstructed data. Unlike traditional methods that may struggle with noisy or incomplete training data, this model introduces a specific mechanism to adapt the GAN training process to be robust to polluted data. It utilizes a data preprocessing strategy that filters out or reduces the impact of noisy segments, ensuring the model learns meaningful patterns from the time series. This unique approach enables the model to efficiently detect anomalies by leveraging the powerful generative capabilities of GANs while simultaneously addressing the challenges posed by noisy real-world time series data.
FGANomaly [3):
CAE-M [7]: CAE-M addresses several challenges in multivariate time-series anomaly detection, particularly in the presence of noisy data. This proposed approach integrates a convolutional autoencoder for feature extraction. which captures spatial dependencies in multi-sensor time-series signals, with memory network that combines both non-linear and linear prediction methods to capture temporal dependencies. The key innovation of CAE-M lies the joint optimization of these components using a compound objective function, which simultaneously minimizes reconstruction error, prediction error, and a regularization term based on Maximum Mean Discrepancy (MMD). The MMD penalty is particularly crucial as it mitigates the influence of noisy data by encouraging the learned feature distribution to approximate that of a Gaussian distribution, thus reducing over-fitting. This architecture allows the model to effectively differentiate between normal and anomalous data even when the training set is polluted with noise.
MTAD-GAT 722: MTAD-GAT introduces novel framework for anomaly detection in multivariate time series data by explicitly capturing the correlations between different features and timestamps. The unique structure of this model leverages two parallel Graph Attention Network (GAT) layers: one feature-orientee and one time-oriented. The feature- oriented GAT layer models the causal relationships between different time-series features, while the time-oriented GAT layer captures temporal dependencies within each time-series. This dual attention mechanism allows the model to dynamically learn both feature-wise and temporal dependencies. Furthermore, MTAD-GAT integrates both forecasting- based and reconstruction- based models, optimizing them through a joint objective function to enhance the representation of time-series data. The forecasting model focuses single- timestamp predictions, while the reconstruction model learns a latent representation of the entire time-series, making the model robust against various anomaly types.
MSCRED 770: MSCRED introduces an effective approach for unsupervised anomaly detection and diagnosis in multivariate time series. The core innovation lies in its ability to jointly tackle three key tasks; anomaly detection, root cause identification, and anomaly severity interpretation. MSCRED achieves this by constructing multi-scale system signature matrices that represent the cotrecoorrlltions between time series at different temporal resolutions. These signature matrices are then processed through fully convolutional encoder to capture spatial dependencies, while an attention based Convolutional Long Short-Term Memory Network models the temporal dependencies across time steps. The decoder reconstructs these matrices, and the residuals are used to identify anomalies. This architecture is enhanced by attention mechanism, which adaptively focuses on the most relevant historical time steps to improve anomaly detection.
30

---

OnimAnomaly [50]:
The proposed model introduces an innovative approach to anomaly detection by incorporating Stochastic Recurrent Neural Network (SRNN) to model the tem- poral dependencies and capture the inherent uncertainty within multivariate time-series data. The key innovation of this model the introduction of stochasticity in the recurrent network, where the model learns distribution over the hidden states instead of a deterministic hidden representation. This probabilistic approach allows the SRNN to better handle incomplete data by explicitly modeling the uncertainty in the data generation process. The network is structured to combine both temporal and spatial dependencies by employing a combination of recurrent layers with stochastic units and a mixture of Gaussian distributions to represent uncertainty. Furthermore, the model includes a robust loss function that incorporates both reconstruction error and regularization term based on the variance of the learned hidden states.
DAGMM 75 DAGMM introduces an architecture for unsupervised anomaly detection that combines the strengths of dimensionality reduction via deep autoencoder with density estimation through a Gaussian Mixture Model (GMM). The key point of this approach is the joint optimization, where both the dimensionality reduction and the density estimation components are optimized simultaneously in an end-to-end manner, eliminating the need main components:
for pre-training and decoupled training. This architecture includes compression network that reduces the dimensionality of input data and encodes it alongside the reconstruction error, and an estimation network that evaluates the likelihood of each data point within the GMM framework. This joint training, facilitated by the estimation network's regularization, allows the autoencoder to avoid suboptimal local minima and better capture the essential features of the data for anomaly detection.
C.2 Time Series Foundation Model
Time series foundation models have shown their powerful potential for downstream time series tasks, including forecasting, imputation, classification, and also reconstruction-basee based anomaly detection. We select FITS (64] Peri-midFormer [59], ModernTCN (34. OFA 74]. and TimesNet (58) in our
experiments.
FITS 644 FITS introduces an innovative approach time series analysis by operating within the complex frequency domain. It utilizes complex-valued linear interpolation to cap- ture both amplitude and phase information, enabling the model to effectively learn amplitude scaling and phase shifting. This ability allows FITS to achieve sfate-of-the-art performance in tasks such as forecasting and anomaly detection. Despite its advanced capabilities, FITS maintains remarkably compact architecture consisting of approximately 10,000 parameters, making it highly efficient. This compactness ensures that FITS is particularly well-suited for deployment on edge devices with limited computational resources, offering excellent balance of performance and efficiency. By leveraging these innovative techniques, FITS demonstrates that high accuracy can be achieved in time series analysis without the need for large, resource- resouree-intensvve models.
Peri-midFormer [59] Peri-midFormer presents an innovative transformer-based architecture that decomposes time series data into periodic pyramid structure. This decomposition captures multi-periodie variations by representing the time series at multiple levels, each cor- responding to diffferent periodic components. The model employs self-attention mechanisms to effectively capture complex temporal relationships across these levels, enhancing its performance in tasks such forecasting, imputation, classification, and anomaly detection.
ModemnTCN 44 Modern TCN revitalizes convolutional approaches in time series analysis by introducing a pure convolutional structure that efficiently captures both cross-time and cross-variable dependencies. By incorporating large convolutional kernels and multiple convolutional layers, TCN achieves substantial effective receptive fields, enabling to model complex temporal patterns effectively. This design results in ttateoof-the-ar art per- formance across various time series tasks, including forecasting, imputation, classification, and anomaly detection.
OFA 74]
One Fits All leverages pre-trained language models (LMs) to enhance time series analysis across multiple tasks. By fine-tuning these LMs on time series data, the model adapts the rich, generalized representations learned from large-scale textual data to the
31

---

specific characteristics of time series data. This approach demonstrates that pre-trained models from natural language or image domains can achieve comparable or even superior performance in time series tasks such as classification, forecasting, and anomaly detection, highlighting the versatility and power of pre-trained LMs in this context.
TimesNet (58 TimesNet introduces task-general backhone for time series analysis by transforming ID time series data into 2D tensors based on multiple periods. This transformation allows the model to capture both intraperiod and interperiod variations effectively. Utilizing parameter--fficienl inception block, TimesNet discovers multi- periodicity adaptively and extracts complex temporal variations from the transformed 2D tensors. This design enables TimesNet to achieve consistent state of the-hhtrr performance across five common time series analysis tasks, including short- and long-tern oon-ttemm forecasting, imputation, classification, and anomaly detection.
32

---

D Hyperparameter Setting
There five crucial hyperparameters during our experiments, including Are Andems Aight Aaux and a. In the latest study conducted by Liu and Paparrizos [32), the pipeline has made sufficient explorations for the optimal Aree and auux and these hyperparameters are fixed when performing experiments. Meanwhile, we also select certain reconstruction- based models that have not been temporarily imported into this pipeline. For these models, we use the hyperparameters suggested in their original papers repositories for Årec and Xaux- For the hyperparameters introduced by IGAD, we perform a detailed grid search for Aidem Aights and a. The search intervals for each hyperparameter can be shown
A [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0].
Aight [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]. a: [1.1, 1.2, 1.3, 1.4, 1.5].
Then, we summarize the optimal hyperparameters for each model and each dataset in Tab
Table 5: Optimal hyperparameters after grid search in our experiments.
Dataset
Model
Venue
SMD
PSM
SMAP
Adeem
Anghtt
Anihtt
Atdem
Anghht
Addem
Auight
CATCH
ICLR, 2025
0.5
0.9
1.1
0.2
1.5
0.3
0.5
1.4
0.5
1.5
M2N2
AAAI, 2024
0.5
0.4
1.2
0.1
0.5
1.4
0.1
0.3
1.5
0.1
1.0
1.3
FITS
ICLR, 2024
0.5
0.1
1.1
1.0
1.0
1.5
0.3
1.0
1.2
1.5
TCN
ICLR, 2024
0.1
133
0.1
0.1
Peri-mite Peri-midFormer
NeurIPS, 2024
0.1
1.0
1.5
0.5
0.4
1.4
0.8
0.9
1.4
0.1
0.1
1.3
SARAD
NeurISS,0222
0.1
0.1
1.1
0.1
0.9
1.5
0.4
0.2
1.5
0.1
0.1
1.1
TimesNet
ICLR, 2023
0.1
0.1
1.0
OFA
NeurIPS, 2023
0.1
1.3
0.5
A.T.
ICLR, 2022
0.9
1.0
1.4
1.0
0.2
1.1
0.8
0.7
1.4
0.7
0.8
1.3
FGANomaly
TKDE, 2021
0.2
0.1
1.4
0.1
0.1
L.
0.1
0.4
1.3
0.1
1.0
1.5
CAE-M
TKDF, 2021
MTAD-GAT
ICDM, 2021
0.8
0.8
1.1
0.2
0.3
1.3
0.1
0.6
1.4
0,9
1.0
1.4
OmniAnomaly
KDD, 2019
0.2
0.3
1.5
0.1
0.8
1.3
0.9
0.7
1.5
0.2
0.5
1.5
MSCRED
AAAI, 2019
0.1
0.5
1.5
0.1
0.1
1.1
1.5
DAGMM
ICLR, 2018
0.2
1.4
0.2
1.5
33

---

E More Detailed Experimental Results
E.1
Distinguishable Distributions of Anomaly Scores and Auxiliary Metries
In this section, we will provide more experimental results for other detailed information. First, we show more cases where the application of IGAD effectively generates more distinguishable distributions of anomaly scores for normal and abnormal instances, shown as Fig 7(a and Fig 7(b) Second, we list other evaluation metries for classification tasks such as AUC-PR, AUC-ROC, VUS- ROC and different types of FI from Tab to Tab[T3] which can still serve auxiliary metrics although they exist potential evaluation shortcomings the field of multivariate time series anomaly detection
32 Improvements in these metrics can also be observed.
Dverlap Area: 0.3255
Normal
Overlap Area: 0.2326
Normal
Abnormal
Abnormal
Without IGAD
With IGAD
Density
Density
3.3
1.6
2.6
11
0.0
0.0
0.0
0.6
1.0
0.6
1.0
Score
Score
(a) Anomaly scores for MSCRED on dataset SMD.
40.8
32.8
Overlap Area: 0.1588
Normal Abnormal
Overlap Area: 0.0419
Narmal
Abnornal
Without IGAD
With IGAD
30.6
24.6
Density
Density
20.4
16.4
10.2
8.2
0.0
0.0
0.2
0.8
0.2
Score
Score
(b) Anomaly scores for FGANomaly on dataset SMAP.
Figure 7: Anomaly score distributions for different models and datasets.
34

---

Table 6: More results on dataset SMD with IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-F FI
CATCH
0.2341 0.0038
0,8952 + 0.0019
0,8343
0.0031
0.1831
0.041
M2N2
0.0407 + 0.0080
0.6401 = 0.0772
0.5537 = 0.0630
0.0335
0,0308
FITS
0.0564 0.0023
0.7490 + 0.0122
0.7300
0.0175
0.0470
0.0200
ModernTCN
0,2486 1: 0.0011
0.8966 + 0.0002
0.8383 = 0.0004
0,3063
0.0022
Peri-midFormer
0.2026 0.0071
0.8797 1 0.0037
0.8209
0.0026
0.1694 0.0055
SARAD
0,2748 = 0.0096
0,9092 = 0.0054
0.8259 = 0.0089
0,3155
0.0120
TimesNet
0.0861 0.0347
0.7553 0.0272
0.7156
0.0298
0.1303 0.0628
OFA
0.1433 = 0.0254
0.6935 = 0.0121
0.6639 = 0.0101
0,2220 0.0353
A.T.
0,0679
+ 0.5137
0.4960 0.0536
0.0661
0.1288
FGANomaly
0,4929 = 0.0116
0.9324 + 0.0023
0.8705
0.0065
0.4943
0.0308
CAE-M
0.1687
0.6635
0.5314
0.1537
0.1326
0.1118
MTAD-GAT
0.4562
+ 0.0379
0.8911 + 0.0239
0.8886
0.0227
0.5304 + 0.0193
OmniAnomaly
0.2621
1 0.9052
0.8274
0.0024
0.2830
0.0099
MSCRED
0.4377
1 0.0217
0.9216
1 0.0331
0.8567
0.0329
0.3939
0.0093
DAGMM
0.2285
0.6875
0.0275
0.4962
0.0401
0. 1444
0.1425
Model
PA-FI
R- F1
Affiliation Fl
CATCH
0,4055 0.1009
0.2638 = 0.0186
0.0056 0.0000
0.6418 0.0039
M2N2
0,2570 0.2359
0,0666 = 0.0626
0.0027 = 0.0024
0,4552
0.1265
FITS
0.5331 0.0347
= 0,0938
0.0046
0.0000
0.7750
0.0177
Modern
0.4886 0.0004
0.3846 1 0.0015
0.0056 1. 0.0000
0.6524
0.0001
Peri-midFormer
0,4670 1 0.1321
0.2529 1 0.0294
0.0056 1: 0.0D0D
0.6198
0.0720
SARAD
0.4912 0.0114
0.3920 1 0.0141
0.0056 1: 0.0000
0.7231
0.0034
TimesNet
0.6006 = 0.0614
0.2222 = 0.0866
0.0047 = 0.0000
0.8155
0.0287
OFA
0.7442 0.0211
0.3747 = 0.0296
0.0050 0.0002
0.8561
0.0285
A.T.
0.3166 = 0.4273
0.2140 + 0.3299
0.0033 + 0.0031
0.5968
0.3153
FGANomaly
0,6531 0.0362
0,6095 = 0.0258
0.0056 0.0000
0.7839
0.0392
CAE-M
0.3069 = 0.1643
0,2520 = 0.1600
0.0313 = 0.0146
0,2563
0.1742
MTAD-GAT
0,8913 0.0249
0.7379 0.0282
0.0047
0.0003
0.9257
0.0210
Anomaly
0.5340 = 0.0336
0,3775 = 0.0115
0.0056 = 0.0000
0.7751
0.0282
MSCRED
0.6960 0.0402
1 0.6109
0.0056
0.0001
0.7587
0.0329
DAGMM
0.3319 1 0.2084
0.3286
1 0.1672
0.0106
1: 0.0067
0.3648
0.1950
35

---

Table 7: More results on dataset SMD without IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-F FI
CATCH
0.2272 0.0053
0.8891 + 0.0014
0.8222
0.0019
0.1811 I 0.031
M2N2
0.0234 +: 0.0003
0.4342 = 0.0034
0.3623 = 0.0n68
0.0487
0.0000
FITS
0.0419 0.0025
0.6946 + 0.0192
0.6769
0.0220
0.0356
0.0076
ModernTCN
0.1930 1: 0.0022
0.8792 + 0.0012
0.8188 = 0.0020
0.1676
0.0020
Peri-midFormer
0.2004 0.0063
0.8782 1 0.0036
0.8198
0.0028
0.1680
0.0042
SARAD
0.2766 = 0.0128
0.9103 = 0.0069
0.8264 = 0.0113
0,3134 0.0104
TimesNet
0.0824 0.0188
0.7583 0.0170
0.7147 0.02166
0.1294 0.0373
OFA
0,0630 = 0.0030
0.6887 = 0.0068
0.6778 = 0.0072
0. = 0.0112
A.T.
0,0266 0.0053
0.5213 + 0.0293
0.5175
0.0240
0.0037 0.0882
FGANomaly
0.4943 + 0.0063
0,9320 + 0.0071
0.8790
0.0143
0.4663 0.0094
CAE-M
0.1746 0.1266
0.6676
0.1213
0.5360
0.1568
O. 0.1268 0.1049
MTAD-GAT
0.3949 0.0031
0.8710 + 0.0043
0,8700
0.0043
0.5150
0.0048
OmniAnomaly
0.2578
+ 0.9003
0.8231
0.0020
0.2798
0.0017
MSCRED
0.4373 1 0.0293
0.9228
1 0.0326
0.8517
0.0271
0.3878 1: 0.0340
DAGMM
0.0983
0.0067
0.5554
0.0136
0.4055
0.0109
0.1041
0.0000
Model
PA-FI
Event-based-Nl
R-based- F1
Affiliation- -1
CATCH
0.3289 0.0441
0.2380 = 0.0248
0.0056 0.0000
0.6416
0.0018
M2N2
0,3832 0.0000
0,0851 = 0.0000
0.0044 = 0.0000
0.5238
0.0038
FITS
0,4901 0.0580
0,0768 = 0.0153
0.0046
0.0000
0.7382
0.0113
Modern
0.3416 0,0231
0.2309 1 0.0115
0.0056 1. 0.0000
0.6316
0.0002
Peri-midFormer
0.5292 1 0.0042
0.2522 1 0.0027
0.0056 1: 0.0D0D
0.6411
0.0023
SARAD
0.4905 1: 0.0130
0.3904 1 0.0142
0.0056 1: 0.0000
0.7315
0.0249
TimesNet
0.6230 = 0.0434
0.2284 = 0.0559
0.0047 = 0.0000
0.8128
0.0259
OFA
0.6813 0.0316
0.2243 = 0.0241
0.0045 0.0000
0.8178
0.0129
A.T.
0.1257 + 0.2811
0.0182 = 0.0407
0.0009 + 0.0020
0.3300
0.2620
FGANomaly
0.5990
0.5588 = 0.0201
0.0056 0.0000
0.7236
0.0351
CAE-M
0.3072 = 0.1654
0,2522 = 0.1607
0.031 = 0.0147
0,2562
0.1743
MTAD-GAT
0,8556 0.0044
0.7075 0.0070
0.0046
0.0000
0.9264
0.0007
Anomaly
1,0000 = 0.0000
1,0000 = 0.0000
0.3893 = 0.0526
0.9961
0.0011
MSCRED
0.6846 0.0686
1 0.5927
0.0056
0.0000
0.7394
DAGMM
0.2317 1 0.0012
0.1778
1 0.0014
0.0406
1 0.0008
0. 0.1789
0.0003
36

---

Table 8: More results on dataset MSL with IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-F F1
CATCH
0,0286 0.0007
0.7808 + 0.0045
0.7808
0.0045
0.0000
0.000
M2N2
0.8250 + 0.2050
0.9984 + 0.0030
0.9984 = 0.0m30
0.0000
0.0000
FITS
0.0103 0.0038
0.5104 +: 0.0508
0.5105
0.0508
0.0000
0,0000
ModernTCN
0.0634 1: 0.0424
0.8082 +: 0.0182
0.8085 = 0.0181
0.0718
0.0306
Peri-midFormer
0.0346 0.0015
0.8160 1: 0.0252
0.8160
0.0252
0.0168 0.0233
SARAD
0,0552 = 0.0635
0.6906 = 0.0499
0.6908 = 0.0497
0.0785 0.1079
TimesNet
0.0074 0.0030
0,4316 = 0.0824
0,4316
0.0626
0.0000 0.0000
OFA
0,0114 = 0.01
0.5077 = 0.0696
0.5077 = 0.0696
0.0200 0.0447
A.T.
0,0063
0.5069 = 0.8315 + 0.0532
0.5068
0.0152
0.0000
0.0000
FGANomaly
0,0525 + 0.0103
0.8314 = 0.0531
0.0986
0.0577
CAE-M
0,0041
0.2066
0.2067
0.0072
0.0000
0.0000
MTAD-GAT
0.3401 0.1288
0.7066 + 0.0655
0.7063
0.0656
0.3037
0.0928
OmniAnomaly
0.0088
0.5024
+ 0.5025
0.0862
0.0000
MSCRED
0.0099 0.0011
0.6883
1 0.0361
0.6883
0.0363
0.0000
0.0000
DAGMM
0.0072
0.0029
0.2631
0.0669
0.2630
0.0666
0.0000
0.0000
Model
PA-FI
-1
R-based-EE F1
Affiliation-
CATCH
0,0000 0.0000
0,0000 = 0.0000
0.0000 0.0000
0.9332 0.0038
M2N2
0,0000 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
Nan = NaN
FITS
0,0000 0.0000
0,0000 = 0.0000
0.0000 0.0000
0.7619 0.1013
Modern
0.2940 0.0347
0.0822 1: 0.0336
0.0210 1. 0.0001
0.9703 0.0013
Peri-midFormer
0.1529 1 0.2106
0.0214 1. 0.0298
0.0083 1. 0.0114
0.9682
0.0017
SARAD
0.2991 1: 0.4104
0.1415 1: 0.1965
0.0083 1 0.0114
0.9389
0.0360
TimesNet
0.0000 = 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
0.8442
0.0523
OFA
0.1467 0.3280
0.0400 + 0.0894
0.0032 + 0.0072
0.803 1 0.0757
A.T.
0.0000 + 0.0000
0.0000 = 0.0000
0.0000 + 0.0000
0.9499 NaN
FGANomaly
0.2739 0.1532
0.1135 = 0.0660
0.0177 0.0099
0.9842 0.0018
CAE-M
0.0000 = 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
NaN + NaN
MTAD-GAT
1.0000 0.0000
1,0000 = 0.0000
0.0119
0.0019
0.9990 0.0003
Anomaly
0.0000 = 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
0.6452 0.1220
MSCRED
0.0000
0,0000
1: 0.0000
0.0000
0.3804
DAGMM
0.0000 1: 0.0000
0,0000
1: 0.0000
0.0000
1: 0.0000
0.4275
0.0554
37

---

Table 9: More results on dataset MSL without IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-F1
CATCH
0,0286 0.0008
0.7807 + 0.0052
0.7806
0.0052
0.0000
0.000
M2N2
0,2024 +: 0. 1512
0.9637 = 0.0233
0.9637 = 0.0233
0.0000
0.0000
FITS
0.0419 0.0025
0.6946 + 0.0192
0.6769
0.0220
0.0356
0.0076
ModernTCN
0.1930 1: 0.0022
0.8792 + 0.0012
0.8188 = 0.0020
0.1676
0.0020
Peri-midFormer
0.2004 0.0063
0,8782 0.0036
0.8198
0.0028
0.1680 0.0042
SARAD
0.2766 = 0.0128
0,9103 = 0.0069
0.8264 = 0.0113
0,3134 0.0104
TimesNet
0.0824 0.0188
0.7583 0.0170
0.7147 0.02166
0.1294 0.0373
OFA
0.0630 + 0.0030
0.6887 + 0.0068
0.6778 = 0.0072
0.1178 0.0112
A.T.
0,0266 0.0053
0.5213 +
0.5175
0.0240
0.0037
FGANomaly
0.4943 + 0.0063
0,9320 + 0.0071
0.8790 = 0.0143
0.4663
0.0094
CAE-M
0.1746 0.1266
0.6676
0.5360
0.1568
O. 0.1268 0.1049
MTAD-GAT
0.3949 0.0031
0.8710
+ 0.0043
0,8700
0.0043
0.5150
0.0048
OmniAnomaly
0.2578
+ 0.9003
0.8231
0.0020
0.2798
0.0017
MSCRED
0.4373 1 0.0293
0.9228
1 0.0326
0.8517
0.0271
0.3878 1: 0.0340
DAGMM
0.0983 0.0067
0.5554
0.0136
0.4055
0.0109
0. 0014
0.0080
Model
PA-FI
R-based-E F1
Affiliation
CATCH
0,0000
0,0000 = 0.0000
0.0000 0.0000
0.9332 0.0040
M2N2
0,0000 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
Nan = NaN
FITS
0.4901 1: 0.0580
0.0768 1: 0.0153
0.0046 1: 0.0000
0.7382 0.0113
Modern
0.3416 0.0231
0.2309 1 0.0115
0.0056 1: 0.0000
0.6316 0.0002
Peri-midFormer
0.5292 1: 0.0042
0.2522 1 0.0027
0.0056 1: 0.0D0D
0.6411
0.0023
SARAD
0,4905 0.0130
0.3904 = 0.0142
0.0056 0.0000
0.7315
0.0249
TimesNet
0.6230 = 0.0434
0.2284 = 0.0559
0.0047 = 0.0000
0.8128
0.0259
OFA
0.6813 0.0316
0.2243 = 0.0241
0.0045
0.0000
0.8178
0.0129
A.T.
0.1257 + 0.2811
0.0182 = 0.0407
0.0009 + 0.0020
0.3300
0.2620
FGANomaly
0.5990
0.5588 = 0.0201
0.0056 0.0000
0.7236
0.0351
CAE-M
0.3072 = 0.1654
0,2522 = 0.1607
0.031 = 0.0147
0,2562
0.1743
MTAD-GAT
0,8556 0.0044
0.7075
0.0070
0.0046
0.0000
0.9264
0.0007
OmniAnomaly
1.0000 1 0.0000
1,0000
1 0.0000
0.3893 1 0.0526
0.9961
0.0011
MSCRED
0.6846
0.0686
1 0.5927
0.0056
0.0000
0.7394
0.0799
DAGMM
0.2317 1 0.0012
0.1778
1 0.0014
0.0406
1 0.0008
0. 0.1789
0.0003
38

---

Table 10: More results on dataset PSM with IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-F F1
CATCH
0.1380 0.0030
0.5680 + 0.0102
0,4974
0.0099
0.0257 0.027
M2N2
0.4124 + 0.0252
0.8527 = 0.0024
0.7793 = 0.0063
0.0548
0,0272
FITS
0.1203 0.000s
0.5137 + 0.0020
0,4559
0.0104
0.0144
0.0006
ModernTCN
0.1408 1: 0.0002
0.5659 + 0.0004
0.4938 = 0.0005
0.0291
0.0001
Peri-midFormer
0.1378 0.0003
0.5520 1 0.0009
0.4909
0.0035
0.0338
0.0005
SARAD
0.1787 = 0.0095
0.6627 = 0.0186
0.4476 = 0.0155
0.0310
0.0021
TimesNet
0.1355 0.0076
0,5516 0.0151
0,4839 0.0141
0.0259
0.0070
OFA
0.1490 = 0.0108
0,5700 = 0.0180
0.5147 = 0.0208
0.0145 0. 0.0057
A.T.
0.1809 0.0T39
+ 0.6199
0.5028 0.0635
0.1100 0.1421
FGANomaly
0,2401 = 0.0255
0.7214 + 0.0110
0.5769 = 0.0106
0.0067 0.0055
CAE-M
0.1773 0.0376
= 0.6427
0.4493
0.0245
0.0275 0.0117
MTAD-GAT
0.1904 + 0.0809
0.6783 + 0.0665
0.6099
0.0623
0.0284 1 0.0017
OmniAnomaly
0.1659
0.6233 + 0.7115
0.4324
0.0073
0.0257
MSCRED
0.1928 1 0.0246
1 0.0335
0.5034
0.0550
0.0220
1: 0.0124
DAGMM
0.2116
0.0047
0.6913
0.0045
0.4736
0.0070
0.0375 0.0050
Model
PA-FI
Wrent-based-Fi
R-based-FF F1
Affiliation- Fl
CATCH
0.3629 0.0010
0,0990 = 0.0084
0.0032 0.0000
0.1300
0.0100
M2N2
0.3320 0.0000
0,0274 = 0.0000
0.0069 = 0.0010
0.0215
0.0011
FITS
0.7490 0.0155
= 0.1
0.0030 0.0000
0.3014
0.0270
Modern
0.3624 0.0001
0.1077 1: 0.0002
0.0032 1. 0.0000
0.1360
0.0000
Peri-midFormer
0.3640 1 0.0001
0.1300 1. 0.0068
0.0032 1: 0.0DOD
0.1686
1.0116
SARAD
0.3313 0.0039
0,0710 1 0.0018
0.0032 1 0.0000
0.1130
0.0015
TimesNet
0.8278 = 0.0434
0.1563 = 0.0212
0.0031 + 0.0001
0.3117
0.0466
OFA
0.8248 0.0810
0.1879 = 0.0471
0.0031 + 0.0000
0.3150
0.0950
A.T.
0.3841 + 0.2627
0.12855 = 1321
0.0023 + 0.0013
0.3753
0.3316
FGANomaly
0,2710 0.1519
0,0360 0.0244
0.0031 0.0018
0.0634
0.0258
CAE-M
0,2783 = 0.1384
0,0730 = 0.0308
0.0032 + 0.0001
0.1079
0.0242
MTAD-GAT
0.7170 0.0350
0.1424 = 0.0388
0.0032 0.0000
0.2223
0.0190
Anomaly
0.0355 + 0.0001
0,0409 = 0.0002
0.0031 - 0.0000
0.0750
0.0000
MSCRED
0.1744 0.1600
0.0523 1
0.0032
0.0002
0.0945
0.0226
DAGMM
0.3400 1 0.0016
0.0794
1 0.0026
0.0032 1 0.0000
0.1168
0.0027
39

---

Table 11: More results on dataset PSM without IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-P FI
CATCH
0.1323 + 0.0036
0,5514 + 0.0096
0.5044
0.0091
0.0194 + 0.0166
M2N2
0,3915 + 0.0135
0.8464 + 0.0033
0.7856 = 0.0m33
0.0648
0.0000
FITS
0.1180 0.0044
0.5083 +: 0.0009
0,4732
0.0014
0.0104
0.0033
ModernTCN
0.1463 1: 0.0003
0.5742 + 0.000S
0.5101 = 0.0019
0.0316
0.0003
Peri-midFormer
0.1378 0.0004
0.5511 1 0.0014
0.4910
0.0042
0.0349
0.0005
SARAD
0.1568 = 0.0135
0.6524 = 0.0268
0.4493 = 0.0097
0.0290
0.0012
TimesNet
0,1211 0.0022
0.5136 = 0.0038
0,4610 0.0048
0.0173 0 0.0028
OFA
0.1310 = 0.0004
0.5390 = 0.000S
0.4772 = 0.0015
0.0110 0. 0.0006
A.T.
0.1162
0.5161
0.4875 0,0281
0.0000 0.0000
FGANomaly
0,2620 = 0.0119
0.7480 + 0.0082
0.5993
D.0080
0.0031
0.0012
CAE-M
0.1608
0.6548
0.4682
0.0U25
0.0248
0.0004
MTAD-GAT
0.1495 0.0023
0.6126 + 0.0056
0.5373
0.0069
0.0255
0.0005
OmniAnomaly
0.1655
0.6218 +
0.4314
0.0083
0.0257
0.0001
MSCRED
0.2165 1 0.0100
0.7431
1 0.0197
0.5425
0.0456
0.0220
0.0095
DAGMM
0.2126 0.0025
0.6912
0.0044
0.4765
0.0040
0.0350
0.0006
Model
PA-FI
Srent-basedFFi
R-based- F1
Affiliation F1
CATCH
0.7125 0.2074
0.1237 = 0.0276
0.0031 0.0001
0.2804
0.1142
M2N2
0.3320 0.0000
0,0274 = 0.0000
0.0073 = 0.0000
0.0219 0.0000
FITS
0.8835 0.0004
0.2184 = 0.0080
0.0031 0.0000
0.5232
0.0069
Modern
0.3639 0.0007
0.1138 1. 0.0073
0.0032 1. 0.0000
0.1757
0.0161
Peri-midFormer
0.3642 1 0.0002
0.1397 1. 0.0162
0.0032 1 0.0DOD
0.1911
0.0249
SARAD
0.1074 0.1207
0.0569 1 0.0049
0.0032 1 0.0000
0.1015
0.0059
TimesNet
0.8550 = 0.0021
0. 061648 = 0.0074
0.0031 = 0.0000
0.4767
0.0394
OFA
0.8683 0.0048
0.1960 = 0.0082
0.0031 + 0.0000
0.5743
0.0147
A.T.
0.0664 + 0.1484
0.0050 = 0.01
0.0004 + 0.0009
0.0792 NaN
FGANomaly
0.3214 0.0014
0.0200 0.0029
0.0028 0.0000
0.0386
0.0008
CAE-M
0.3148 = 0.0006
0.0547 = 0.0001
0.0035 = 0.0000
0.0944
0.0000
MTAD-GAT
0.3319 0.0016
0.0874 = 0.0019
0.0032
0.0000
0.1823
0.0002
Anomaly
0.0355 + 0.0001
0,0409 = 0.0002
0.0031 - 0.0000
0.0750
0.0000
MSCRED
0.3461
0.0633 1 0.0032 0.0001
0.1150
0.0204
DAGMM
0.3397 1 0.0005
0.0780 1 0.0002
0.0032 1 0.0000
0.1140
0.0002
40

---

Table 12: More results on dataset SMAP with IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-P Fl
CATCH
0.2942 0.0008
0,6088 + 0.0017
0,6088
0.0017
0.1329
0.0200
M2N2
0.1971 = 0.0427
0.6754 + 0.0868
0.6753 = 0.0869
0.0000
0.0000
FITS
0.2858 0.0126
0.7845 +: 0.0196
0.7845
0.0197
0.0194
0.0045
ModernTCN
0,4165 +: 0.0079
0.8004 + 0.0053
0.8004 = 0.0n54
0.2067
0.0025
Peri-midFormer
0.5096 0.0273
0.8312 1 0.0129
0.8298
0.01
0.1971
0.0030
SARAD
0,8437 = 0.0213
0.9320 = 0.0060
0.9337 = 0.0060
0,2425 0.0064
TimesNet
0.2731 0.0672
0.7194 = 0.0970
0.7194
0.0970
0.0330 0.0197
OFA
0.2977 = 0.0254
0.7897 = 0.0326
0.7897 = 0.0326
0.0425
0.0138
A.T.
0,2557 0.1120
0.6327 + 0.9957 + 0.0006
0.6326
0.0961
0.0439
0.0884
FGANomaly
0,9840 = 0.0012
0.9957 = 0.0006
0.3473
0.0099
CAE-M
0,0718
0.0203 = 0.8976 + 0.0501
0.0203
0.0141
0.0000
0.0000
MTAD-GAT
0.5121 0.2364
0.8976
0.0501
0.0718
0.0689
OmniAnomaly
0.9064
0.9111 + 0.3914 1 0515
0.9111
0.0284
0.4186
0.1677
MSCRED
0.1248 1: 0.0207
0.3914
0.1556
0.0000
0.0000
DAGMM
0.1106 0.0124
0.0597
0.0144
0.0597
0.0144
0.0225 0.0311
Model
PA-FI
Event-basede
R-based- F1
Affiliation- Fl
CATCH
1.0000 0.0000
1,0000 = 0.0000
0.3063 0.0027
0.9485 0.0002
M2N2
0,0000 0.0000
0,0000 = 0.0000
0,0000 = 0.0000
NaN = NaN
FITS
0.9020 0.0015
0,0988 = 0.0209
0.1662 = 0.0007
0.5542
0.0163
Modern
0.9986 0.000S
0.9879 1 0.0040
0.3322 1. 0.0008
0.9484
0.0005
Peri-midFormer
0.9998 1 0.0002
0.9984 1 0.0022
0.3288 1 0.0011
0,9498
0.0002
SARAD
0.9989 0.0009
0,9922 1 0.0063
0.3435 1 0.0021
0.9492
0.0009
TimesNet
0.9183 = 0.0165
1839 = O. 1334
0.1682 + 0.0027
0.5879
0.0432
OFA
0.9151 0.01
0.2188 = 0.0699
0. 0.0019
0.6027
0.0358
A.T.
0.7826 + 0.4380
0.2615 = 0.3943
0.1362 + 0.0768
0.6389 0. 1989
FGANomaly
1.0000 0.00000
1,0000 = 0.0000
0.3724 0.0032
0.9955 0.0001
CAE-M
0.0000 = 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
NaN + NaN
MTAD-GAT
0.9480 0.0293
0.4098 0.3328
0.1997
0.0676
0.6742 0.1815
Anomaly
1.0000 = 0.0000
1,0000
= 0.0000
0.3893 = 0.0526
0,9961
0.0011
MSCRED
0,0000
0.0000
1 0.0000
0.0000
0.3334
0.0001
DAGMM
0.4000 1 0.5477
0,4000
1: 0.5477
0.1096 1 0.1500
0.9490
0.0000
41

---

Table 13: More results on dataset SMAP without IGAD.
Model
AUC-PR
AUC-ROC
VUS-ROC
Standard-P F1
CATCH
0,2898 0.0013
0.5913 + 0.0052
0.5912
0.0052
0.1410
0.0511
M2N2
0.1954 + 0.0056
0.6345 = 0.0123
0.6344 = 0.0123
0.0909
0.0081
FITS
0.2713 0.01
0.7675 + 0.0154
0.7675
0.0154
0.0203
0.0048
ModernTCN
0,4594 1: 0.0098
0.8220 +: 0.0051
0.8220 = 0.0n50
0.1746
0.0053
Peri-midFormer
0.5105 0.0288
0.8304 1 0.0132
0.8304
0.0133
0.1977
0.0044
SARAD
0,8490 = 0.0184
0,9335 = 0.0053
0.9335 = 0.0053
0,2462
0.0076
TimesNet
0.2677 0.0742
0.7256 0.0993
0.7256
0.0993
0.0297 0.0168
OFA
0,2959 = 0.0256
0.7949 = 0.0254
0.7949 = 0.0254
0.0442 0.0148
A.T.
0.2397 0.0828
0.5897 + 0.0911
0.5895
0.0913
0.0810
0.1053
FGANomaly
0,9208 = 0.0270
0,9560 + 0.0060
0.9560 = 0.0060
0.0606
0.0012
CAE-M
0,0719
0.0218
0.0218
0.0157
0.0000
0.0000
MTAD-GAT
0.2324 + 0.0271
0.6605 + 0.0518
0.6604
0.0518
0.0340
0.0008
OmniAnomaly
0.0764
0.1051
+
0.1057
0.0000
MSCRED
0.0936 1: 0.0011
0. 0.12 1 0.0001
0.1213
0.0001
0.0000
0.0000
DAGMM
0.0746 0.0032
0.0242 0.0042
0.0242
0.0043
0.0018 0.0025
Model
PA-FI
Event-baseds
R-based-FE F1
Affiliation- Fl
CATCH
1,0000 0.0000
1,0000 = 0.0000
0.3093 0.0042
0.9489
0.0002
M2N2
0,8854 0.0087
0.3166 = 0.0229
0.1884 = 0.0014
0.7227
0.0062
FITS
0.9022 0.0017
0.1029 = 0.0215
0.1664 = 0.0008
0.5547
0.0177
Modern
1.0000 0.0000
1.0000 1 0.0000
0.3197 1 0.0041
0.9500
0.0003
Peri-midFormer
0.9998 1: 0.0002
0.9984 1. 0.0022
0.3291 1 0.0016
0,9498
0.0002
SARAD
0.9992 0.0006
0.9943 1 0.0042
0.3447 1 0.0025
0.9495
0.0006
TimesNet
0.9142 = 0.0143
0. 051997 = 0.0884
0.1677 = 0.0024
0.5771
0.03-49
OFA
0.9179 0.0129
0.2311 = 0.0778
0.1692 0.0021
0.6041 1 0.0382
A.T.
0.9865 + 0.0131
0.6238 + 0.3904
0.1939 + 0.0414
0.7903 O. 1618
FGANomaly
1.0000 0.0000
1,0000 0.0000
0.2745
0.0004
0.9487
0.0000
CAE-M
0.0000 = 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
NaN + NaN
MTAD-GAT
0.9059 0.0014
0.1674 = 0.0051
0.1685
0.0001
0.5657 0.0018
Anomaly
0.0000 = 0.0000
0,0000 = 0.0000
0.0000 = 0.0000
NaN + NaN
MSCRED
0.0000
0.0000
1: 0.0000
0.0000
NaN
DAGMM
0.4000 1: 0.5477
0,4000 1: 0.5477
0.0991
1: 0.1358
0.9553
0.0001
42

---

E.2 Efficiency Evaluation
In this section, we compare the Training Time per Epoch (Twfos Tw), GPU usage (g Gw). and CPU usage (Cunjon Cw) before and after applying IGAD for each dataset and each model from Tab[[4 to Tab[[]] From the defined formulas shown as 7 and D model will perform four additional mappings after the application of IGAD. More concretely, before integration with IGAD, a model performs a mapping f())), while [x),, f(=), (=),, (2)), and (z)())) are carried out sequentially under the effect of IGAD. While more operations are involved, the application of IGAD does not introduce additional parameters that need be trained, as IGAD leverages frozen copy of the training model. IGAD introduces additional time and resource consumption due to the mapping associated with the manifold constraints, but these costs occur only in the training phases. During inference, each model can perform only one mapping for reconstruction, calculate anomaly scores, and detect abnormal time points. This means that a model with IGAD can achieve the same efficiency as the same model without IGAD during inference. For dataset with larger scale, the time is possible to last longer.
In the evaluation of Training Time per Epoch, we find that in most cases, the models with IGAD show an increase in time of less than 10 seconds and in many cases, even less than 3 seconds per training epoch. The increase generally has limited impact on the overall training process, especially when higher performance is desired. Meanwhile, is also indicated that OFA [74] tends to show a higher increase during one epoch. This phenomenon can be theoretically concluded as that the utilize of pre-trained language model is more source-sensitive to fine-tune it for time series tasks. An additional fact shows that the PSM dataset needs more training time than SMD, MSL, and SMAP. The reason is that the data scale of the PSM dataset is larger than others, and the cumulative effect of multiple iterations in the same epoch and multiple mappings results in a longer training time. In the evaluation of Max GPU Allocation for Training and Max CPU Allocation for Training, different degrees of MB (about 8.34 GB), and the maximum CPU occupancy is around 537.44 MB (about 0.52GB). All experiments performed with single NVIDIA RTX 3090 24GB GPU, and 0.52 GB is also acceptable for most hardware conditions for deep learning currently. These additional GPU and CPU memories are selected to save frozen parameters, gradient graphs, and data segments to update the training model.
increase are listed. During these, the maximum GPU occupancy is around 8535.88 80
Meanwhile, we have also envisioned some potential strategies to reduce these additional computa- tional costs in our future implementations: (1) Accelerate calculation by data and model parallelism; (2) For large models such as OFA (7].. we can perform parameter-efficee efffcicnn fine-tuning with advanced methods, including [33 19 55 based on LoRA 23 to reduce the number of trainable parameters; (3) A memory mechanism can be included to reduce the number of true reconstructions. Concretely, when a piece of reconstructed time series is needed, we can index the memory module to generate this in terms of association.
Table 14; Efficiency evaluation on dataset SMD, comparing training time per epoch (Tw//n Tw), GPU usage (Gm Gw), and CPU usage (Cwnjox oor Cww) before and after applying IGAD.
Triiiin Time per Epach (s)
Max GPU Allocation for Training (MB)
Max CPU Allocation for Training (MB)
Model
Tajo
Tu
Tu Twfo
Suzo
Gx
Go Gayj
Cafa
Co
C, Cuyo
CATCH
6.0878
17.2277
11.1399
3038. 8540
5317.9945
2279.1405
2252.5469
2362.888 8828
110.3359
M2N2
0.8424
1,2809
0.4385
6.5044
10.9773
4,4729
2190.1797
2603. 9336
413.7539
FITS
1.1934
1.9876
0,7942
19.0562
39.9126
20.8564
4719.1914
4756.3789
37.1875
ModemTTN
3.4464
9.9653
6,5188
1679.2700
6333.5298
4654.2598
4776.7734
4793. 4648
16.6914
Peri-midForer
1.1767
5.8734
4.6967
289. 3027
8825, 1006
8535.7979
4773.2188
4826.9961
53,7773
SARAD
1.4122
2,5430
1.1308
778.5454
2392.6172
1614.0718
4751.1641
4766.2148
15,0508
TimesNet
1.3733
2.6364
1,2632
299. 299.8774
411.8228
111,9453
4790.5352
4832.9365
42.4013
OFA
3.6833
21.4194
17.7361
1918.6016
6223.6816
4305.0801
4863.4492
4910.8242
47,3750
A.T.
2,0369
3,9399
1,9030
1834,0044
4781.5825
2947.5761
4760.8867
4789.8633
28.9766
FGANomaly
1.6585
2.1942
0.5357
58.0522
$23.9907
465.9385
4739.8789
4746.5241
6.6452
CAE-M
0.9261
1,2022
0,2761
115.1748
301,7012
12645264
4752.7109
4749.4766
16,7656
MTAD-GAT
1.2675
0.9356
599.8101
2052.
4775.2891
4783.0039
7.7148
Omni/nnomaly
1,0875
1.8211
0.7336
45.4243
169.6577
124.2334
4748
4760.1894
11,8503
MSCRED
2.8281
10.2687
7.4406
3314.2231
8588 2192
5273.9961
4742.325
4743.5591
1.2309
DAGMM
0.9995
1.1030
0,1034
10.8193
21.5449
10.7256
4698.9609
4701.4833
2.5273
43

---

Table 15: Efficiency evaluation on dataset MSL, comparing training time per epoch (Tw/o: Tw), GPU usage (owfor Gw), and CPU usage (Cw Cw) before and after applying IGAD.
Training Time per Epuch (s)
Max GPU Allocation for Training (MB)
Max CPU Allocation for Training (MB)
Model
Tujo
T
Tu
Suy-
Su/e
Cuje
Cu Cuyo
CATCH
1,5938
3.3659
1.7721
4230.5103
8037.9696
3807.4593
2312.8828
2560.7383
247.8555
M2N2
0.5443
0.9091
0.3647
12.1436
20.9563
8.8127
2193.8672
2581 .3945
387.5273
FITS
0.8837
1,0449
0.1612
28.5674
58.7539
30.1865
4731.3164
4768.368 7638.3633
37,0469
ModerTTNN
1.2722
2.3140
1.0418
2438.
9111.5781
6673.2915
4776.7539
4792.0039
15,2500
Peri-mid Peri-midFormer
0.8942
1,4531
0,5589
725.4805
4876.8081
4151.3276
4819.9000
4861 1347
41 41.2284
SARAD
0.9593
1.1728
0.2136
1113. 1733
3489:0684
2375,8950
4747 0664
4777 9102
30,8438
TimesNet
0.8971
1,0556
0.1588
303.2749
419,8989
116.6240
4789 4727
4818.499 4922
29.0195
1.2976
3.4958
4310,5752
A.T.
0.98992
1.2458
0.2566
1253.3545
2974.2109
1720,8564
4768.0742
3750
29.30088
PGANomaly
0.9215
1.0265
165,9316
CAE-M
0.84799
0,8915
0.0436
165.8027
435. 1094
269.3066
4754.1888
4772.01566
17,8320
MTAD-GAT
0.8763
1.0014
0.1251
1006.0601
3351.1172
2345.0571
4759.
4762.5593
2.7859
OmniAnooaly
0.8426
0,9404
0.0978
49,0093
183.8770
134,8677
4740.9492
4768.8750
27.9258
MSCRED
1.2203
2.3347
1.1144
5435
12323 8037
7626.2603
4767.2188
4767 6666
0,4180
DAGMM
0.8396
0.8602
0.0206
15.6182
32,5830
16.9648
4727.1992
4728 4013
1.2031
Table 16: Efficiency evaluation on dataset PSM, comparing training time per epoch (Tw/os Tw), GPU usage (mm/or /or Gw), and CPU usage (Cwrjox jo Cw)) before and after applying IGAD.
Training Time per Epoch (s)
Max GPU Allocation for Training (MB)
Max CPU Allocation for Training (MB)
Model
Tujo
Tazj
Sa/o
9u.
G.. Gu/o
Cu/o
C.,
Cu/l
CATCH
44.4446
131.0026
86.5580
2228. 2228.2168
$258.5916
30.30,1749
2285. 8203
2389 8164
103.9961
2.5754
3.7534
4.7399
2229.9570
7725.6836
495.7266
FITS
4.0280
8.0651
4.0370
12,5127
26.6787
14.1660
4739.9600
476007330
20.7695
ModemTCN
15,9907
54.3645
38,3737
1105 6284
4172.5288
3066.9004
4769. 9844
4785.
15.1992
Peri midForerr
4.2939
22.7822
18.4883
146.5879
583.9233
437.3354
4780.700 7969
4785 6680
4.8711
SARAD
4,9100
14,7800
9,8700
551.7329
1610.5713
1058.8384
4745,7266 7266
4748. 1641
2,4375
5,4077
OFA
24,8965
225.4139
200.5114
1917.998S
6221. 9976
4303.9990
4872.3086
5395.2734
522.9648
13.7400
4878.5923
43.6623
FGANomaly
9.6031
15,4628
5,8597
56,7183
5440.7212
5384.002
4759 8750
4766 3008
6.4258
CAE-M
1,7830
3.0684
1.2854
76.6055
201.3789
124.7734
4756.0391
4762.
6.4688
MTAD-GAT
3,6808
10.4775
6.7966
374.0786
1268.0796
894.0010
4972.4648
5000.8047
28.3398
OmmiAnomaly
4.0156
10.3294
6.3138
43.8564
159 6650
115 8086
4752.7734
4767 4421
14 14.6687
MSCRED
2259.
5729.
3470.0078
0.0313
DAGMM
1.8711
3,0836
1,2125
7.1514
14.2622
7.1108
4740.0156
4756.8516
18.8359
Table 17: Efficiency evaluation on dataset SMAP, comparing training time per epoch (Twros Tw). GPU usage /o/ Gw), and CPU usage (Cw//ox Cw.) before and after applying IGAD.
Training Time per Epoch (s)
Max GPU Allocation for Training (MB)
Max CPU Allocation for Training (MB)
Model
Tajo
Tu
Tajo
Sujo
Go
Gayo
Cafo
C., Cazo
CATCH
2,5922
6.4548
3,8626
2228.2233 2236
4565.1578
2336. 2336.9341
2193.1484
2210.7266
17.5781
M2N2
0.6537
0,9807
0.3269
3.7534
7,4113
3.6579
2158.4492
2596.3633
437.9141
FITS
0.9403
1.2141
0,2737
13.1851
26.6787
13.4937
4717.2734
4723.304
6,0313
ModemTCN
1.6361
3,5830
1,9468
1105 6284
4172.5288
1066.9000
4753.0820
4772.8477
19.7656
Peri-miFForeer
1.0926
2.1401
1,0474
1263. 2222
1407.9321
144.7100
4773.390
4784.6780
11.2795
SARAD
1.0455
1.2557
0.3802
551
1610.5713
638.6339
4736.4648
4750.
13,6002
TimesNet
1.0640
1 1.6658
0,6017
306.8296
408.3433
101.5143
4765.6133
4771.3477
5.7344
2.2287
8.1973
6221.9977
A.T.
1,3751
2.2921
0,9170
1936.3228
5066.8916
3130.5688
4766.0039
4792.0977
26.0938
FGANemaly
0.2354
CAE-M
0.8584
0.9544
0.0960
76.6055
201.3789
124.7734
4736.6563
4746.7227
10.0664
MTAD-GAT
0.9592
1.3120
0.3528
374.0786
1268.0796
894.0010
4734.4883
4738.7656
4,2773
Omni Anomaly
0.9489
1 1,2228
0.2739
43. 8564
159.6650
115. 8086
4728.8945
4739.2734
10.3789
MSCRED
1.60551
3.7906
2.1856
2259.1724
5729.1802
3470.0078
4743.4219
4744.6059
1.1841
DAGMM
0.86155
0.9203
0.0588
7.1514
14,2622
7,1108
4711.6016
4737.3438
25,7422
44

---

E.3 Hyperparameter Analysis
Here, we list the instructions to choose the optimal Aadem Aught and a for given dataset and select the model DAGMM [75] with dataset SMAP to show the analysis of hyperparameters. The results are displayed Fig Fig and Fig TO
First, in Fig we calculate the frequency of Aidems Aiiht- and a for all the experiments conducted. The results indicate that, for most cases, the values of Xidem are located in the interval [0.1, 0.5], which means that a relatively smaller λıdem may be better for new datasets. For the values of >right they focus mainly on the two endpoint values and maintain relatively uniform distribution throughout the other central parts. Finally, larger a may be considered as a priority.
In the following part, the parameter sensitivity analysis conducted in the following also supports the instructions listed above. We show the results with mean and standard deviation in Fig and 95% confidence intervals in I0 Concretely,
fix two of Aiemm Aught and as the optimal values shown in Tab Then, we vary the remaining one from 0.1 to 2.0 with a step of 0.1. For Aidems the model achieves the best performance when Ademm is 0.1 (a smaller one). For Aight with the tightness effect changing from loose to strict (Aught changing from small to large), the performance of the model changes from up to down. We attribute it to tightness, which even drops out normal instances. For o, it is clearly shown that the performance improves with larger a and levels off when is greater than 1.5.
20
26
Forqueicy
Frequency
14
5
5-
5
2
2
0.1
0-2
0.3
0.4
0.5
0.5
0.7
0.0
0.
1.0
0.2
Apain
(a) Aidem
(b) Aught
22-
Frequancy
13
3:3
1.3
1.3
3.5
(c) a
Figure 8: Parameter frequency records for Xidems Aiiht and a.
45

---

0.16
Mean
Standard Deviation
0.14
VUS-PR
0.08
0.06
0.1
0.
1.0 1.1 111.1006778000011112222111111113
Aldem
(a)
Xidam
0.16
Mean
Standard Deviation
0.14
0.12
VUS-PR
0.10
0.08
0.06
0.1
0.2
0.3
0.4
0.7
0.8
0.9
1.0
1.1 1.2 1.3 1.5 1.6 1.9
Atight
(b) Auihtt
0.16
Mean
Standard Deviation
0.14
0.12
VUS-PR PR
0.10
0.08
0.06
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.7
a
Figure 9: Parameter sensitivity analysis for Aidem Aiihtt and with mean and standard deviation.
46

---

VUS-PR with 95%
0.14
0.12
WUS.FR
0.06
1.4
15
3.5
3.7
Pcam
(a) Aidem
VUS-PR
0.14
VUS-PR
0.08
0.06
1.5
1.7
(b) Aught
VUS-PR with 95% CI
0.14
0.12
VUS PR
0.06
1.2
1.3
20
2.2
20
79
20
(c) a
Figure 10: Parameter sensitivity analysis for Aidems Aight and a with 95% confidence intervals.
47

---

E,4 Visualization of Latent Space
To further verify the effectiveness of IGAD, we visualize the latent space of different models before and after applying IGAD in I1 It can be observed that, under the effect of IGAD, the model gains a clearer boundary to distinguish normal instances from abnormal instances. This aligns with our design principles to modify and tighten the target manifold Marget with the aim of containing enough normal instances and drop out potential abnormal instances.
0.8
0.8
K-SNE2
k-SNE
0.4
0.4
0.0
Normal (n=500)
Abnormal
0.0
Normal
Abnormal (n=500)
0.2
0.6
0.8
0.2
0.4
0.6
0.8
+-SNE
t-SNE
(a) DAGMM on SMD without IGAD.
(b) DAGMM on SMD with IGAD.
1.0
1.0
0.6
0.6
t.NNE
t.SNE
0.0
Normal (n=500)
Abnormal (n=500)
0.0
Normal (n- 500)
Abnormal (n=500)
0.0
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
+-SNE
t:SNE
(c) OFA SMD without IGAD.
(d) OFA SMD with IGAD.
1.0
1.0
0.6
0.6
0.2
0.2
Normal (n=500)
Abnormal (m=500)
Normal (n=5500)
Abnormal (n=500)
(e) OmniAnomaly SMAP without IGAD.
(1 OmniAnomaly on SMAP with IGAD.
Figure 11: The visualization of latent space using t-SNE before and after applying IGAD.
48

---

E,5 Maintain Data Patterns under Noise
In our experiments to demonstrate that IGAD can help balance robustness and sensitivity Sect (ect 4.2.3
we have incorporated a noise strategy into the testing data. In this part, we employ heatmaps to show these data with weighted noise from Fig 12 to Fig We have found that noise-effected testing data display similar change patterns with the original data, which means that our noise strategy can verify their abilities to balance the robustness and sensitivity of different models while maintaining the necessary information.
Nutoon
Huroom
Feaum
2 Prldoe 14
teout
INbde C4
(a) Original time series data.
(b) Time series data with noise weight 1%.
Tedee
Teaton
Twtur
Troun
Festou
fediow
teaves
teon
fedion
The
(c) Time series data with noise weight 5%.
(d) Time series data with noise weight 10%.
Tearon
tesice
Tretes
Tesioo
Teeum i0
Tesion
hare
hupor
hanus
hubee
ratum
Iutore
Then
(e) Time series data with noise weight 15%.
(f) Time series data with noise weight 20%.
Figure 12: Visualization for original data and noise-effect data SMD.
49

---

2
tesion
harun
feetes
feaace M
Iedoce
(a) Original series data.
Time series data with noise weight 1%.
Teron
1a 24
fotton
Tweure
Iesion M
lubice
fesios
Tixee
The
Time series data with noise weight 5%.
(d) Time series data with noise weight 10%.
Vedioe
Taton
butoon
Vessow
Totm
36
Tesun
haturt
Iubort
Thmw
(c) Time series data with noise weight 15%.
(f) Time series data with noise weight 20%.
Figure 13: Visualization for original data and noise-effect data on MSL.
50

---

tebere
Yeatom
hapine
varrow
d reaum jo
toro, be
harie
E
faatow
waldo 14
froum
feazie Mt
toun
ludion
vesce 22
ensioe
(a) Original series data.
Time series data with noise weight 1%.
repore
teacon
Yeanon
tedes
Tevisn
twore
hutore
Yarum
I feaum yo
vessoe Va
2
huture
mussee
harie
wamee
tesion St
tubre
teape
raun
outore
i1
reppon 22
(c) Time series data with noise weight 5%.
(d) Time series data with noise weight 10%.
tware
Pestom
batire
Texton
baber
tumm, 1o
Treur
hatee
buttee
14
reom
venoe
twur
(nsice
tuun 21
Imcore Ca
hatum in
(e) Time series data with noise weight 15%.
(f) Time series data with noise weight 20%.
Figure 14: Visualization for original data and noise-effect data on PSM.
51

---

rescs
tebere
Yeutom
bapine
redee
Teaion
d teaum t0
tepoon
hanie
freom i6
hann
lubhor
12
veace 22
(a) Original time series data.
Time series data with noise weight 1%.
repone
Yeacon
Yearon
tedets
Tevlin
twore
hutore
Yarun
I feaun io
reston va
2
Nuture
taris
teeus
tesion S4
tubee
teape
raun
outore
i1
respon 23
Time
(c) Time series data with noise weight 5%.
(d) Time series data with noise weight 10%.
tware
Prstom
Teoton
bobert
Tretn
hatre
bubtce
14
teom
tenox
twur
(nsice
tuon 2t
Imcore C3
Time
(e) Time series data with noise weight 15%.
(f) Time series data with noise weight 20%
Figure 15: Visualization for original data and noise-effect data on SMAP.
52

---

E.6 Nyquist Criteria
We perturbed the time series data in the frequency domain balance robustness and sensitivity in our experiments. This indicates that we should perform more analysis in the frequency domain to support this strategy. The Nyquist Criterion establishes fundamental requirement for faithful reconstruction of continuous-time signal from its discrete samples. This principle underpins modern digital signal processing systems, analog-to-digital conversion, and telecommunications The theorem originated from Harry Nyquist's work in [37] and was later rigorously formalized by Claude Shannon in information theory [55] It states that a band-limited signal with no frequeney components exceeding fmax (Hz) must be sampled at a rate fa 2 max to avoid aliasing and ensure complete signal recovery. conclude the key concepts in this theory as follows:
The minimum sampling rate m is termed the Nyquist rate.
Half of the sampling frequency is referred as the Nyquist frequency. flyq:
Spectral overlap, i.e., aliasing, may occur if < max fNyg
fmax causing irreversible
distortion.
To verify compliance with the Nyquist- Nyquist-Shannon Shannon sampling theorem, the proposed procedure shown in [ first determines the sampling rate fs and the corresponding Nyquist frequency fNyq from user-provide eser-provided sampling frequency descriptor for the imput multivariate time series dataset D. For each time series feature within D, its frequency spectrum is obtained by the Fast Fourier Transform (FFT). The significant frequency components are then identified by comparing their normalized magnitudes against relative threshold based on the peak magnitude in the spectrum. If the highest significant frequency detected any feature exceeds fNyq- the dataset is flagged non-compliant; otherwise, compliance is affirmed.
Algorithm Nyquist Criterion Compliance Verification
1: Input:
Multivariate time series dataset D E RRnk (n instances, variables)
Sampling frequency descriptor faese (e.g.. min")
2: Output: Boolean compliance status flagNyy
3: procedure CHECKNYQUIST(D, fuese)
4: Compute sampling rate: f. = 1/At according to foes
> Unit: Hz
5: Calculate Nyquist frequency: fNyyq = fa/2
6:
for each feature column d; E D (where i is the feature index) do
7:
Let N₂ = |d. be number of samples the current feature column. Compute FFT: Y, F(d,)
8:
9:
Generate frequency axis (positive frequencies): f, -fitfreg(NN....... At) N./2]
10:
Compute normalized magnitude: A; |Y [0: N
11:
Detect significant frequencies:
Fug Ef A(F) > 0.1 max(A,))
12:
if Fr sig Continue
then
13:
D No significant frequencies above threshold for this feature
14:
if max (F > fNyg then
return False Violation: Max significant frequency exceeds Nyquist frequency b All features comply with Nyquist criterion
16:
return True
From the study [50), we can get the information that selected datasets our experiments, SMD, MSL, PSM and SMAP, are sampled with sampling frequency of min, so we set fs 60 according to the unit seconds. After verification, all datasets in our experiments satisfy the Nyquist criterion. Further, due to the space constraints each page, we visualized the validation results for the first eight variables of each dataset from 16 to Fig 19 The full validation codes can be found in our anonymized repository and run directly to carry out and check the results of full verification.
53

---

0.22
6.3
02-
0.00
6.0
0.00
5009
10089
12089
23000
0.00
6,04
0.004
0.00
Time
Frecuency
Fastum
Peature spectiore States: Paas
3.0
6.00
(C.00030003
MacCip free 3030022223 Ind
18883
2084
0.50
1.322
Luke
Fastam
Feiture Spactrure Statas: Pars
Neperuue Specture
Threshed
Machay May 0802521
0.00
0.00
6.00
Time (rrtutee)
Trecuenc ltt
2.6
0.30
Raperase Specllure are
bi you itreched
2.00
0.00
5049
10099
15093
29000
0.000
5
0:004
0.008
Fastum de Timm Serie
Fecture S: Spectrure Stata:: Pars
6--
Manhip king 5.0000
662
0.300
3.30
Feeture Spectrure Stata:: Posx
Nepmore Spedirue
10000000200 HoC
Mac lrus 300000000000000
Tine (emaees)
Freaeny (he)
Nigtrucc Specture
Si Freo, Ttrasted
6.00
6.00
1000
10005
2008
0.50
1.22
0.00
Festum de Time Selee
34.
Feeture & Spectrur Stetvs Poes (Mle via frey.)
54
-0.00
0.04
0.00
Time
Prcuencyitii
Figure 16: Nyguist criteria verification on SMD.
54

---

2.--
6.4
0.6
-0.0
150
1004
1226
1890
1753
0.00
3.00
0004
0.004
0.00
Frecency
Fastum 2: lime
Feature a spectiun Statess Pass 1Mo freg.)
0.00
0.04
2.97
19..0
1250
1592
1192
0.50
1332
Fadtum
Festure 2 Spactrum Stallar lNle trey.)
0.04
Paptiuce Specium
R.20633330
-0.04
Time (eituee))
Piccenyylllr
Popttucs Specium (is00s5se
62
6.00
0.00
0.4
153
1006
1258
1993
1753
6.
0.000
5.02
0:04
0.508
Festum de Tims Serie
Feature S Spectrum States: Pass (No freg.)
24-
-0.00
-0.04
10C8
1353
0.300
Fastum Ser
Spectrure Statas: Pasy
0.22
feedituee
Maxsip lra 208033373
0330
Tine 210r
Payel dee Spaciius Sua R020632033
0.02
2.0
155
1000
1724
1503
1192
0.30
122
0:00
Festuee de Time Seres
3.0
Feeuree & Spectrie Stetvs Poes (Mle via frey.)
-0.00
0.04
1753
0.00
Figure 17: Nyquist criteria verification on MSL.
55

---

Papalad Seciuu
Pree
50569
330000
150090
208003
9.000
1002
6.04
Time
Hataraal Serve
Peoture Spectrus Stanas Pass
3.4
(8330333
(0,0003004
3.300
04633
200223
2.02
6002
0.54
Rastredd Tim Serie
Festure Specirum Statan:
Payslude free Thea
(1.0000000
0.0
Tim Oe PURE
2.6
dade kencoun
so Fire trem
4.4
1050S
2900e0
150993
209903
0:00
Nastreds Tim Serie
Festrre St Spectrun Statae: Pass
Nia Sig (1.0603cee
330008
L50030
240003
3.603
6648
Ne Time
Paystede Spectrum Strtue:
Sig (5,0600000
12000
Tiu 2 marnn
Pagil Sascoun
Sp. free
0550333
6.2
155091
200903
2.00
0:00
5104
Testared Tie Serte
Feeture &i Spectrum Steten Pess
08300333 lol
Nop Sg. "ie (6,0009000e for
330000
150830
8.00
Figure 18: Nyquist criteria verification on PSM.
56

---

Maydaai Seediie:
Pin Tecarel
-1.0
2003
4003
5000
1900
0.001
0.002
C004
0.04
Tiu
Tequano
Natuo tea Seees
Feature Spectrun Stana Pass
0.008
J893
42.00
0.30
Restrm Time Sedar
Festure Spectrum-
0.004
6,00
Tiu Denuee
Prequency End
Maur dadio npa
1003
6000
8900
0.00e
0.902
C004
0.004
Nastum d: Serie
Festure St Spectrum Sesbar Pass
0 0.8002
330005
3
2803
6300
0006
Time
Spectrum- Stabar
R:37
Mayr fade Spectium
6.44
5:2
516
Prepuees 201
bagi Spedisu (135111353
Sgg frea Timered
6.64
6.03
2003
4263
62.0
8200
0.06
0.30
E104
D:24
Tertim ToTe Sedte
Festure Bi Spectrum Staher Pars
0.304
0.80003230 Hay
Pat Fws C0a00C2
0,004
0,002
5,0001
2000
Tim menuuaa
Figure 19: Nyquist criteria verification on SMAP.
57

---

E.7 Comparison with Contrastive-based Models
As shown in Tab noticeable improvements can be observed for different models. In this section, our aim is to explan that reconstruction-basee methods are also competitive candidates, so it is necessary further optimize for better performance when conducting MTS AD. Given these, we compare selected reconstruction-base methods with contrastive-based models, including DCdetector [66]. TS-TCC 15] and CoST [57], which are also powerful tools in recent years. The results are listed in Tab 18 The results indicate that many reconstruction-baser methods also outperform contrastive-based methods and become more competitive under the effect of IGAD.
Table 18: VUS-PR under five random seeds for contrastive learning model. f denotes the average value of all the mean values reported in Tab
Dataset
Model
SMD
MSL.
PSM
SMAP
DCdetector
0.0210
0.0096
0.1117
0.1508
TS-TCC
0.0221
0.0088
0.1064
0. 0.1491
CoST
0.0217
0.0228
0.1118
0.1459
Dataset
Model
SMD
MSL
PSM
SMAP
Mean of Contrastive-basee Methods
0.0216
0.0137
0.1010
0.1486
Mean of Reconstruuttin--used Methods w/o IGAD
0.1554
0.0416
0.1542
0.3223
Mean of Reconstruction-bast Methods w/ IGAD
0.1752
0.0920
0.1592
0.4056
58

---

E,8 Codes for IGAD
For better understanding IGAD, we also provide pseudo-code block to help better describe the IGAD working flow during the training process, as shown in Code
1 Pirst, we define:
2 f: The training model initialized by f Model (parameters...) ....tr (device)
c_copy (the defined f)): The frozen model initialized by c_copy Model (parameters... rew.perrs.....rrriireeeeee.e.
requirees_grraddeeeeeeeeeeeeeeee (Faise) to (device)
5 def train in_ single
E_copy, data):
copyy is the frozen of current training model
c_copy state_dite (f. atate_dict
10
secon_data (data)
get_augumented date (data) f(z) f(211)
Get ani for x
f_a fz.detach () $ (2"5)
15
it = 2(E_z) ((E"
fz = copo (fz) (E(22)))
Calculate losses loss_rec = rreco__data data) pow (2) # Reconstruction
19
loss_idem (f_f2 f2) pow (2) $ Idempotent
loss_tight - (ff_z z_ pow (2) $ Tightness
loss_ aass_auxiliary if exists
24
Optimize for lcsses
Loss = Lambda_ree _oss_ree rec + lambda idem
+
Lambda_tight loss tight loss auxiliary if exists opt. perozgrrg
27
oss backward()
ste.
Listing 1: Python implementation for IGAD.
F Limitation and Future Work
Although significant improvements have been observed, there remain unexplained performance drops in limited number of experiments. This warrants further investigation to identify the underlying causes. Meanwhile, the slightly larger standard deviation observed in certain cases suggests the need to optimize the training process to achieve more stable convergence our future work. The workflow of IGAD also inspires us to explore potential strategies to reduce computational complexity for large models, such as OFA [74].
G Impact Statements
This paper presents work focused advancing the field of multivariate time series anomaly detection, with applications in healthcare, finance, and industrial monitoring. Although the ethical implications of anomaly detection generally well-established, the misuse of such methods in sensitive areas could lead to privacy concerns and unintended biases in decision-making. We believe that this research contributes improving anomaly detection techniques, improving system reliability, and early warning capabilities. Specific ethical issues are not identified beyond these general considerations.
59


Reviews:
[{"review_id": "7fYjNdmp5M", "reviewer": "Reviewer_3SEA", "summary": "This paper proposes a plug-and-play module named Idempotent Generation for Anomaly Detection (IGAD), which can be flexibly combined with a reconstruction-based method without introducing additional trainable parameters.  This method helps to release over-generalization and balance between robustness and sensitivity from a manifold perspective. Experiments on four datasets demonstrate its effectiveness.", "questions": "1. Are all the experimental results listed in Table 1reimplemented or directly from the original paper? Clear marks should be added.", "limitations": "Not mentioned.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Figure 1 clearly illustrates the motivation of this paper.\n2. In the experiments, 15 reconstruction-based multivariate time series anomaly detection methods are utilized as backbone models. Experimental results are sufficient.\n3. Representation is clear and fluent.\n\nWeakness: \n1. The lower part of Figure 2 should be improved with clear information flow, as well as its caption.\n2. There are lots of hyperparameters of losses in Eq.11. There is not sufficient analysis about how these hyperparameters influence the model performance. Do those hyperparameters need to be carefully fine-tuned?\n3. Is the method implemented on the pretrained backbone models or trained from initialization?\n4. The proposed model performs as an additional module based on the backbone model. Thus, it is important to analyze the increased complexity and inference time, compared to the backbones.", "was_revised": true, "final_justification": "My concerns have been addressed during the rebuttal, thus I have decided to raise my score to 5.", "rebuttal": "Thank you for your constructive comments. We provide our feedbacks as follows.\n# Main Comments:\n> **W1: The lower part of Figure 2 should be improved with clear information flow, as well as its caption.**\n\n**A-W1:** Thank you for pointing this out. We first further explain the information flow in Fig.2, then we will update and integrate these information into this figure and add more descriptions to improve it.\n\n1. **The first part of Fig.2 lies in the top half,** which means the conventional reconstruction workflow. A time window is fed into the backbone architecture for MTS AD, and the optimization objective is to minimize the deviation between the original time window $x^i$ and the reconstructed time window $x^i_\\text{recon}$. These operations are employed to calculate $\\mathcal{L}_\\text{recon}$:\n\\begin{equation}\n\\mathcal{D}\\_{\\mathrm{recon}}(x,f(x))=\\frac{1}{m}\\sum\\_{i=1}^{m}\\|\\|x^{i}-x\\_{\\mathrm{recon}}^{i}\\|\\|\\_{2}^{2}, \\mathcal{L}\\_{\\mathrm{recon}}(x,f(x))=\\mathbb{E}\\_{x}[\\mathcal{D}\\_{\\mathrm{recon}}(x,f(x))].\n\\end{equation}\n\n2. **The second part of Fig.2 lies in the bottom half,** which contains the workflow of a given model integrated with our proposed IGAD plugin. Concretely, **in the left side of this part**, when we have a time window $x^i$, we first conduct FFT on $x^i$ to get the frequency-domain signal. This signal can be expressed with a complex form. Here, according to Eq.3, we use $\\mathbf{Re}(\\cdot)$ and $\\mathbf{Im}(\\cdot)$ to extract the real part and image part. After we get the real part and image part, we can calculate the mean and standard deviation of them, which can be denoted as $\\mu_{\\text{real}}^i$, $\\sigma_{\\text{real}}^i$, $\\mu_{\\text{image}}^i$ and $\\sigma_{\\text{image}}^i$, respectively. Then, we randomly sample vectors from $\\mathcal{N}(\\mu_{\\text{real}}^i, \\sigma_{\\text{real}}^i)$ and $\\mathcal{N}(\\mu_{\\text{real}}^i, \\sigma_{\\text{real}}^i)$ to get the sampled-effected vector $z^i$ through IFFT for the following calculation. **As for the right side of this part**, it is the information flow of IGAD. The red arrow and the blue arrow in the right manifold stand for the training and frozen model, i.e., $f(\\cdot)$ and $f'(\\cdot)$, then $z^i$ is passed through multiple mappings $f(f'(z^i))$ and $f'(f(z^i))$ for the calculation of $\\mathcal{L}\\_\\text{idem}$ and $\\mathcal{L}_\\text{tight}$, respectively:\n\\begin{equation}\n\\mathcal{D}\\_{i\\mathrm{dem}}(f(z),f^{\\prime}(f(z)))=\\frac{1}{m}\\sum\\_{i=1}^m\\left|f(z^i)-f^{\\prime}(f(z^i))\\right|,\\mathcal{L}\\_{i\\mathrm{dem}}(f(z),f^{\\prime}(z))=\\mathbb{E}\\_z[\\mathcal{D}\\_{i\\mathrm{dem}}(f(z),f^{\\prime}(f(z)))],\n\\end{equation}\n\\begin{equation}\n\\mathcal{D}\\_{\\mathrm{dight}}(f^{\\prime}(z),f(f^{\\prime}(z)))=\\frac{1}{m}\\sum\\_{i=1}^{m}\\left|f^{\\prime}(z^{i})-f(f^{\\prime}(z^{i}))\\right|,\\mathcal{L}\\_{\\mathrm{dight}}(f(z),f^{\\prime}(z))=\\mathbb{E}\\_{z}[-\\mathcal{D}\\_{\\mathrm{dight}}(f^{\\prime}(z),f(f^{\\prime}(z)))].\n\\end{equation}\n\nWe hope these descriptions can help readers better unserstand the information flow of IGAD, and we will further improve this figure and caption in our manuscript.\n\n> **W2: There is not sufficient analysis about how these hyperparameters influence the model performance. Do those hyperparameters need to be carefully fine-tuned?**\n\n**A-W2:** Thank you for pointing out the importance of analyzing the hyperparameters introduced by IGAD. There are five relative hyperpatameters in Eq.11, including the weights of different loss items ($\\lambda_{\\text{recon}}$, $\\lambda_{\\text{idem}}$, $\\lambda_{\\text{tight}}$ and $\\lambda_{\\text{aux}}$) and the control parameter $\\alpha$. As we mentioned in Sect.4.1.2, the latest work on time series anomaly detection benchmark [1] has integrated the most of the models we have selected in our experiments and searched for the optimal hyperparameters in optimizer, learning rate, and weights of existing loss functions for them. For some of our selected base models, which are not temporarily imported, we set their hyperparameters in their original papers or repositories as the optimal ones. **These mean that $\\lambda_{\\text{recon}}$ and $\\lambda_{\\text{aux}}$ have been fixed as the optimal ones, and we need to explore the effects of $\\lambda_{\\text{idem}}$, $\\lambda_{\\text{tight}}$ and $\\alpha$ in IGAD.** The search interval for $\\lambda_{\\text{idem}}$ and $\\lambda_{\\text{tight}}$ is set as [0.1, 1.0] with a step of 0.1, and [1.1, 1.5] with a step of 0.1 for $\\alpha$. During this process, we select OPTUNA [2] to help search. **We summarize the results of hyperparameter influence in Appendix D (Page 23) and Appendix E.3 (Page 35 to Page 37), including Tab.4, Fig.7, Fig.8 and Fig.9.** We also list potential selection strategies for a new dataset. It can be concluded as that $\\lambda_{\\text{idem}}$ in [0.1, 0.5], not too strict $\\lambda_{\\text{tight}}$ and a larger $\\alpha$ tend to achieve better performance.\n\n[1] The Elephant in the Room: Towards A Reliable Time-Series Anomaly Detection Benchmark, NeurIPS DB Track, 2024.\n\n[2] OPTUNA: A Next-generation Hyperparameter Optimization Framework, KDD 2019.\n\n> **W3: Is the method implemented on the pretrained backbone models or trained from initialization?**\n\n**A-W3:** In our selected models, OFA is an anomaly detection model based on GPT-2, so it is trained after loaded the pre-trained weights from huggingface. For the other models, they are trained from initialization according to the random seeds, following the standard training principle.\n\n> **W4:It is important to analyze the increased complexity and inference time, compared to the backbones.**\n\n**A-W4:** Thank you for your suggestions on this key point. **We conduct effectiveness analysis and report the results in Appendix E.2 (Page 33).** In this part, we calculate the training time, max GPU allocation and max CPU allocation. We admit that additional mappings from IGAD will increase the time and resource allocation during training, **but these costs occur only in the training stage. During inference, each model can perform only one traditional mapping for reconstruction, calculate anomaly scores, and detect abnormal time points.** We find that in the most cases, the models with IGAD show an increase in time of less than 10 seconds. The increase generally has relatively limited impact on the overall training process, especially when higher performance is desired. Meanwhile, model OFA tends to show a higher increase. This is because that OFA is a pre-trained language model, which means that it has more dense parameters than traditional models for AD. To reduce these potential cost, we list three possible future strategies in this section (Page 33): \n1. Data and model parallelism;\n\n2. Fint-tune large models with parameter-efficient methods;\n\n3. Build memory mechanism to save necessary information, reducing the number of full reconstruction.\n\n> **Q1: Are all the experimental results listed in Table 1 reimplemented or directly from the original paper? Clear marks should be added.**\n\n**A-Q1:** All results listed in Tab.1 are **reimplemented.** We follow the experimental setting in Sect.4.1.2 and rerun models to report their results with five random seeds. We have added the corresponding information in Sect.4.1.2 for better clarification."}, {"review_id": "vRv4oAjFjI", "reviewer": "Reviewer_vmjz", "summary": "This paper proposes a novel module named IGAD to address the risk of over generalization in reconstruction-based methods for multivariate time series anomaly detection. IGAD draws on the ideas of idempotent generative networks and makes reasonable improvements for the task. And this paper presents sufficient theoretical analyses and experimental results.", "questions": "Q1: In some datasets and for certain models, the use of IGAD has instead led to performance degradation. Considering that these are results after hyperparameter search, is this related to the compatibility between IGAD and some methods? For example, CAE-M shows limited improvement on the PSM and SMAP datasets and even performance degradation on the SMD dataset; ModernTCN exhibits significant performance drops on both the PSM and SMAP datasets.  \nQ2: Why does the method with IGAD perform worse when the noise weight is smaller in Figure 4(b)?  \nQ3: The models marked with * are concentrated on the MSL dataset. Is this related to the characteristics of the dataset?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:  \nS1: The paper has reasonable motivation.   \nS2: The paper is easy to follow and well-written.    \nS3: The paper has sufficient and detailed theoretical analysis.   \n\nWeaknesses:   \nW1: The number of datasets is slightly small.    \nW2: The increased training time caused by IGAD limits its practicality, especially the more complex the base methods and datasets used with IGAD, the greater the additional time overhead introduced by IGAD.  \nW3: The method in the paper is based on idempotent generative networks, which limits its innovativeness.", "was_revised": true, "final_justification": "I recommend Borderline accept. My concerns are addressed after author rebuttal.", "rebuttal": "Thank you for your insightful and valuable suggestions. We provide our feedbacks as follows.\n# Main Comments:\n\n> **W1: The number of datasets is slightly small.**\n\n**A-W1:** Thank you for your suggestions that more datasets should be taken into consideration to further demonstrate the effectiveness of IGAD. We have added two larger datasets, SWaT and UCR, and shown the results in the following table.\n\n| **Model** | **Dataset** |  |  |  |  |  |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  | UCR |  |  | SWaT |  |  |\n|  | w / o IGAD | w / IGAD | $p$-value | w / o IGAD | w / IGAD | $p$-value |\n| CATCH | 0.0017  | 0.0019 ↑ | *** | 0.1472  | 0.1475 ↑ | *** |\n| M2N2 | 0.0037  | 0.0038 ↑ | *** | 0.4267  | 0.4442 ↑ | *** |\n| FITS | 0.0071  | 0.0203 ↑ | *** | 0.1448  | 0.1501 ↑ | * |\n| ModernTCN | 0.0028  | 0.0022 ↓ | *** | 0.1518  | 0.1523 ↑ | *** |\n| Peri-midFormer | 0.0018  | 0.0019 ↑ | *** | 0.1449  | 0.1445 ↓ | *** |\n| SARAD | 0.0026  | 0.0070 ↑ | *** | 0.1769  | 0.1780 ↑ | *** |\n| TimesNet | 0.0122  | 0.0124 ↑ | ** | 0.1302  | 0.1303 ↑ | *** |\n| OFA | 0.0157  | 0.0161 ↑ | *** | 0.1221  | 0.1222 ↑ | *** |\n| A.T. | 0.0046  | 0.0056 ↑ | *** | 0.1413  | 0.1622 ↑ | *** |\n| FGANomaly | 0.0020  | 0.3818 ↑ | *** | 0.1808  | 0.1809 ↑ | *** |\n| CAE-M | 0.6174  | 0.6311 ↑ | *** | 0.1804  | 0.2011 ↑ | *** |\n| MTAD-GAT | 0.0015  | 0.0021 ↑ | *** | 0.1540  | 0.1543 ↑ | *** |\n| OmniAnomaly | 0.0016  | 0.0309 ↑ | *** | 0.1768  | 0.1831 ↑ | *** |\n| MSCRED | 0.0021  | 0.0027 ↑ | *** | 0.1804  | 0.1802 ↓ | ** |\n| DAGMM | 0.5715  | 0.6481 ↑ | *** | 0.1804  | 0.2001 ↑ | *** |\n\nIt can be found that 27 out of 30 additional experiments show improvements, which helps further demonstrate the effectiveness of IGAD.\n\n> **W2: The increased training time limits its practicality**\n\n**A-W2:** Thank you for your concerns about the efficiency. First, we agree with you that it will cost more time for complex methods or datasets during training. Second, we wish to clarify that:\n\n1. **The additional computational overhead from IGAD only occurs during the training phase.** During inference, IGAD introduces no extra computational steps, meaning the inference efficiency is identical to that of the original base model. Given that anomaly detection systems often follow a \"train once, deploy long-term\" paradigm, in common cases, a moderate increase in training time is a reasonable trade-off for improved detection performance.\n\n2. **An advantage of IGAD is that it introduces no new trainable parameters.** It operates by leveraging a frozen copy of the training model. This ensures that model storage and deployment costs do not increase.\n\n3. **Our experiments show that the time overhead is controllable.** For most models, the training time increases by less than 10 seconds per epoch, and often by less than 3 seconds. We acknowledge that for larger models like OFA, the overhead is more significant, and we discuss potential optimizations for this in the future.\n\n> **W3: The method in the paper is based on idempotent generative networks, which limits its innovativeness.**\n\n**A-W3:** We admit that the idempotent generative network inspires us to conduct deeper exploration in the field of AD. However, we wish to respectfully clarify that transitioning this concept from novel sample generation to multivariate time series anomaly detection involved not only a simple application. To address the challenges in AD, such as over generalization and the balance between robustness and sensitivity, we have incorporated several key, task-specific modifications that distinguish our framework.\n1. **First, the fundamental goal and the definition of the manifold are different.** The original work aims to learn a manifold to generate novel samples belonging to a specific domain. In contrast, our objective is to improve existing reconstruction-based models for more precise anomaly identification. Consequently, we strictly redefine the learned target manifold $\\mathcal{M}_\\text{target}$ as a distribution consisting only of normal time instances, which is a conceptual departure from the original work, as our framework is optimized to learn the boundary of this \"normal-only\" manifold.\n\n2. **Second, the meaning for generating the latent vector $z$ is different.** The original idempotent generative network samples $z$ mainly for generating new images. However, we apply a FFT to a window of normal time series $x$, introduce randomness through resampling in the frequency domain, and then use an IFFT to produce the final augmented vector $z$. This design injects the intrinsic patterns and spectral features of real normal samples into $z$ while simulating the natural noise and fluctuations of real-world data acquisition.\n\n3. **Third, we have repurposed the optimization objectives.** In our framework, the idempotent objective $f(f(z))=f(z)$ is not for generating new samples. Instead, its role is shifted to ensure the model learns to recognize and reconstruct the augmented vectors $z$ that carry the intrinsic features of normal data, thus making the target manifold $\\mathcal{M}\\_\\text{target}$ more robust in covering diverse normal instances. In addition, the tightness objective $f(f(z)) \\neq f(z)$ is also repurposed for dropping out potential abnormal instances from $\\mathcal{M}\\_\\text{target}$.\n\n4. **Finally, our IGAD is designed as a plug-and-play module that introduces no additional trainable parameters.** This makes our approach flexible and compatible with different reconstruction-based AD models.\n\nWe hope these can help further clarify the innovation of IGAD and the differences from the traditional idempotent generative network.\n\n> **Q1: In some datasets and for certain models, the use of IGAD has instead led to performance degradation. Considering that these are results after hyperparameter search, is this related to the compatibility between IGAD and some methods?**\n\n**A-Q1:** Thank you for your insightful question. We agree that this is related to the compatibility between IGAD and the internal mechanisms of certain models. ModernTCN is a specialized tool, relying on a fine-tuned convolutional structure to extract features. While the manipulation of IGAD is generally beneficial in most cases, it might inadvertently alter the data representations in a way that is incompatible with the rigid learning structure of ModernTCN. The features that IGAD helps to shape may not be optimally suited for processing by the specific convolutional pathway, thus causing a drop in performance. In addition, CAE-M incorporates a Maximum Mean Discrepancy regularization term to push its feature distribution toward a Gaussian one. The objectives of IGAD, which also modify the data distribution, could conflict with the native regularization strategy of CAE-M, leading to a suboptimal result for complex multi-objective optimization problems. Finally, these cases are relatively rare, and IGAD leads to performance improvements in the majority of our experiments (52 out of 60), with statistically significant gains in most of those cases (49 out of 52), demonstrating its effectiveness and applicability.\n\n> **Q2: Why does the method with IGAD perform worse when the noise weight is smaller in Figure 4(b)?**\n\n**A-Q2:** In Fig.4(a), we show the ideal case that the model enhanced with IGAD get better performance under different noise levels. In Fig.4(b), we want to show another effect results of IGAD. The performance of IGAD is below the baseline at low noise levels (1% and 5%) but surpasses it at higher noise levels (15% and 20%). We think that this behavior validates one primary goal: to balance robustness and sensitivity. In a low-noise setting, the original A.T. model is well-tuned for the clean data patterns. IGAD, which involves generating a perturbed $z$ and optimizing for robustness, effectively acts as a form of regularization. This regularization, which is critical for noisy data, can slightly hinder the model from perfectly fitting the clean patterns, resulting in a marginal performance decrease. However, the true value of IGAD is revealed as noise increases. While the performance of baseline model degrades from 0.0307 to 0.0214, the model with IGAD maintains a highly stable performance with the trend of increase. This means that IGAD prevents the model from overfitting to clean data and equips it with a more robust understanding of the underlying data patterns in more challenging and realistic noisy scenarios.\n\n> **Q3: The models marked with * are concentrated on the MSL dataset. Is this related to the characteristics of the dataset?**\n\n**A-Q3:** Thank you for your further consideration on this. First, we agree that this is related to the dataset characteristics. The data in MSL dataset originated from spacecraft telemetry, and the normal operational patterns here are complex, variable, and noisy. In such cases, reconstruction-based models can generate a high baseline of reconstruction errors even for normal data points. These make MSL a challenging task and the error separability is already limited to some extent. Consequently, in certain cases, even though IGAD improves the VUS-PR score by the effect of regularization, the magnitude of its improvement on the score distribution is not substantial enough to be deemed statistically significant. Second, this phenomenon can also be amplified by factors such as the inherent sparsity of anomalies in the MSL dataset (MSL dataset has smaller average number and length of abnormal points than others), which affects the statistical power of the test. Third, VUS-PR can be highly sensitive to improvements at critical decision thresholds. It is valid that IGAD makes small but crucial adjustments to the anomaly scores right around the critical decision boundaries. This could significantly boost the VUS-PR value without causing a statistically significant shift across the entire distribution rank ordering, thus leading to a VUS-PR improvement with a p-value > 0.05."}, {"review_id": "lz6eVO8JiC", "reviewer": "Reviewer_KChG", "summary": "This paper introduces the use of a idempotent generative network framework for multivariate time series anomaly detection in order to address the over generalization problem of reconstruction based time series anomaly detection methods, i.e., reconstruction models that rely on the assumption that abnormal time series would be poorly reconstructed by a generative model that has only seen normal data, may not in practice be the case. Two hypotheses are provided for this behaviour: 1) there is contaminated data in the normal training samples that are impractical to remove, and 2) reconstruction models that have good regularization/generalization properties could generalize well to anomalies. The proposed solution, IGAD, adds an idempotent loss and a tightness loss to ensure a tight mapping of normal samples to the target manifold. Improvements shown over many base reconstruction models and ablation studies demonstrate the effectiveness of this approach.", "questions": "- Since the original IGN was tested on MNIST, can the proposed framework be applied to image data?\n- What is the justification for doing reconstruction on x and IGAD on z, why not use z for reconstruction, or use both x and z for IGAD?\n- How do we know that the tightness objective implicitly excludes anomalies from the manifold, when we don't have knowledge of the nature of the anomalies?\n- Are f and f' two separate networks of different weights? By the definition of idempotent, shouldn't f and f' be the same function? Is it the same network but in different points of time? Please clarify. It would also clear up how Eq (9) wouldn't cancel out.\n- The MAE over MSE experiment is interesting. Since 0.1684 VUS-PR is the performance of no IGAD with MSE, is it possible that the 0.2080 VUS-PR is attributed to the MAE loss instead of IGAD - this is potentially a weakness.", "limitations": "Some limitations have been discussed.", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n- Novel solution to the problem of over-generalization problem in reconstruction-based MTS AD methods\n- Theoretical analysis of how regularization leads to over-generalization and good reconstruction of anomalies (undesired)\n- Framework is compatible with almost every existing reconstruction-based MTS AD models, with no additional trainable parameters required\n- Consistent positive improvements of the framework over the original backbones.\n- A larger-than-average number of baselines is compared\n- Figure 5, Table 2, and Table 3 are impressive results\n- Robust protocol that includes optimizing the hyperparameters of baselines\n\nWeaknesses\n- Framework is almost the same as [45], although the application in MTS AD is novel\n- Proof of convergence is the same as [45]\n- Does not mention other work in the contaminated data in MTS AD, e.g., [Ref 1], and there are more in the general context of Anomaly detection. Note that the original IGN was tested on MNIST, so it is expected to cover contaminated AD in the non-TS field as well.\n- Does not achieve state-of-the-art results compared to [63]\n- Limited number of datasets (only 4, and they are small datasets). It would be more convincing to follow [63] and include SWaT, NIPS-TS, and UCR. More datasets would be favoured over a large number of baselines.\n- The delta_model percentage should be w/IGAD minus w/oIGAD as opposed to w/IGAD divided by w/oIGAD\n- Justify why VUS-PR is favoured over the other evaluation metrics. Can also report reconstruction error on abnormal data to be meaningful. \n- Figure 4b) is not very impressive looking compared to Figure 4a)\n\n[Ref 1] Contaminated Multivariate Time-Series Anomaly Detection with Spatio-Temporal Graph Conditional Diffusion Models\n\n\nMinor weaknesses:\n- Inconsistent use of the terms \"over generalization\" and \"over generation\"\n- The distinction between x and z is not clear in the introduction, but becomes clearer later.\n- Please clarify if a sliding window is used", "was_revised": true, "final_justification": "I maintain my score as was discussed in the rebuttal.", "rebuttal": "Thank you for your insightful suggestions. We provide our detailed responses below.\n# Main Comments:\n> **W1 & W2: The framework and proof of convergence are same as [45].**\n\n**A-W1 & W2:** Thank you for this concern. IGAD differs from [45] in its design concept, objectives, and its function as a task-specific plugin. Please refer to our response to **Reviewer vmjz on W3** for details due to page limits. Meanwhile, we redefine math objects in our convergence proof to validate that our target manifold learns the distribution of normal data only.\n\n> **W3: More works should be mentioned.**\n\n**A-W3:** Thank you for your suggestions that more comprehensive references should be included to further improve the integrity. After searching from 2023, we have found the following works as well as you kindly mentioned:\n1. Image, Tabular or Video: BiGAN, IAD, SAE-CRIT, LOE.\n2. TS: NegCo, TSAD-C, COUNTA, NormFAAE.\n\nThese outstanding works primarily process data contamination through methods such as sample weighting, designing robust loss functions, or explicitly separating clean and contaminated data for modeling. However, IGAD, from a manifold geometry perspective, uses adversarial objectives to tighten the normal data manifold, implicitly excluding contaminated and anomalous data.\n\n> **W4: Does not achieve state-of-the-art results compared to [63].**\n\n**A-W4:** The primary contribution of our paper is not a new, end-to-end SOTA model, but a novel, plug-and-play module, IGAD, without introducing additional trainable parameters. To demonstrate this , we included a comparison with contrastive-based models like [63] in **Tab.17 (Page 48)**. The results show that the average VUS-PR of reconstruction-based models with IGAD is higher than the average of contrastive-based methods.\n\n> **W5: Limited datasets.**\n\n**A-W5:** Thank you for this suggestion. SWaT and UCR datasets are added to further verify IGAD. We would greately appreciate it if you could refer to **Reviewer vmjz on W1.**\n\n> **W6: The calculation method for $\\Delta_\\text{model}$.**\n\n**A-W6:** Thank you for this suggestion. In the current version, $\\Delta_\\text{model}$ is reported as the improvement ratio, calculated by:\n\n$$\n\\frac{\\text{VUS-PR}\\_\\text{w / IGAD}-\\text{VUS-PR}\\_\\text{w / o IGAD}}{\\text{VUS-PR}\\_\\text{w / o IGAD}} * 100.\n$$\n\nWe chose a relative measure to better reflect the impact of improvements, especially for weak baselines. However, we agree that absolute results provide a direct reflection of gains and will add them to our manuscript for a more complete evaluation.\n\n> **W7: Why VUS-PR is favoured and report reconstruction errors on abnormal points.**\n\n**A-W7:** This finding is highlighted in [31]. Due to the constrains on uploading images and space limit, we would greatly appreciate it if you could refer to the **Fig.5 in [31].** The other metrics mainly suffer three issues: bias, indiscrimination and lack of adaptability. VUS-PR can effectively handle these issues, emerging as the most robust metric. We guarantee that we will add a section to further explain this. We also report the differences on average errors for abnormal points in the following table. Most cases show that model with IGAD tend to generate larger errors for anomalies. For readability, these values have been multied by $10^3$.\n\n| Model | Errors (w/ - w/o)|  |  |  |\n|---|---|---|---|---|\n|  | SMD | MSL | PSM | SMAP |\n| 1 | 2.0  | 0.3  | 123531.2  | 4.0  |\n| 2 | 11.5  | 12.2  | 34.0  | 103.2  |\n| 3 | 2.2  | 6.4  | 0.3  | -0.2  |\n| 4 | 2.6  | 1.2  | 0.1  | -1.2  |\n| 5 | 0.9  | -0.2  | 0.1  | 1.1  |\n| 6 | 4.7  | 98.8  | 15.3  | 7.3  |\n| 7 | 3.5  | 4.0  | 1.1  | 6.1  |\n| 8 | 438.4  | 14.4  | 5.6  | 3.2  |\n| 9 | 7.1  | 0.1  | 0.1  | 14.4  |\n| 10 | 485.2  | 395.4  | 358.6  | 95046.0  |\n| 11 | -64.9  | 68.7  | 12.4  | -78.2  |\n| 12 | 2463.1  | 231.8  | 59.4  | 2008.7  |\n| 13 | -0.1  | 4.4  | 0.1  | 332.2  |\n| 14 | -25.6  | 0.1  | -2.7  | 147.8  |\n| 15 | 110.6  | 134.4  | 10.8  | 103.6 |\n\n> **W8: Fig.4(b) is not very impressive looking compared to Fig.4(a).**\n\n**A-W8:** Thank you for your careful observation. We think that this is another reflect of the effect of IGAD and conclude this as the trade-off between over-fit to clean data and the ability for high-level noise. We would greatly appreciate it if you could refer to our response to **Reviewer vmjz on Q2** for more details.\n\n> **W9 & W10: Inconsistent term and the distinction on $x$ and $z$.**\n\n**A-W9 & W10:** Thank you pointing these out. We have located the terms and fixed them. Meanwhile, to improve readability, we will add a clarifying sentence in the introduction for final $x$ and $z$ to provide readers with a clearer initial understanding.\n\n> **W11: If a sliding window is used.**\n\n**A-W11:** Yes, as we stated in Sect.3.2, the whole dataset is split into different windows with the size of $w$, and the stride is 1 according to the global and unified settings in [31].\n\n> **Q1: Can the proposed framework be applied to image data?**\n\n**A-Q1:** Thank you for this constructive question. We think that IGAD can be applied into image AD with appropriate adaption:\n1. Image data also suffer the problem of contaminated data due to anomaly leakage or ambiguous boundaries, shown as works in W3, and over generalization from the constrains on reconstruction. These are consistent with the issues IGAD can address.\n2. While our FFT-based augmentation is technically applicable to images via 2D-FFT, it is designed for the global and periodic feature of time series, and have more physical meaning. A direct application may be suboptimal, as its global disturbance might influence the model to detect highly localized anomalies often found in images.\n\nIn conclusion, IGAD framework is transferable and its strategy for generating $z$ would need to be re-designed for images.\n\n> **Q2: What is the justification for doing reconstruction on x and IGAD on z?**\n\n**A-Q2:** We use reconstruction on $x$ and IGAD on $z$ because they are responsible for different goals. Concretely:\n1. Reconstruction on $x$: This is the fundamental objective. It forces the network to learn to accurately reconstruct the original normal time series data and define the core of the normal data manifold.\n2. IGAD on $z$: The vector $z$ is a perturbed version of $x$, representing a point nearly the ***boundary*** of the normal manifold. The idempotent objective then teaches the model that these slightly perturbed, yet still normal, points should be mapped stably onto the manifold. This ensures the learned manifold is robust to noise. Simultaneously, the tightness objective acts as a crucial adversarial regularizer. It prevents the manifold from expanding to achieve stability for addressing over generalization.\n3. If we were to use $z$ for reconstruction, i.e., $f(z)=z$, we would be directly teaching the model to reproduce noise, which is contradictory. Conversely, applying IGAD to $x$ would be less effective, as $x$ is already assumed to be an ideal point on the manifold, and the objective could be directly satisfied.\n\n> **Q3: How do we know tightness objective excludes anomalies from the manifold?**\n\n**A-Q3:** We think, in fact, we do not need prior knowledge about anomalies:\n1. Our principle is that the space of normal data is typically compact and has a specific, learnable structure, while the space of abnormal data is vast, diverse, and essentially unbounded. Therefore, we do not learn to identify every possible anomaly, but learn an tight and precise model of normality. Points that fail to conform to this strict model are flagged as anomalies.\n2. IGAD is designed to enforce through the manifold geometry. The tightness objective works adversarially to prevent the manifold from loosening its boundaries to accommodate points that are not central to the normal data distribution, penalizing the manifold expansion. Consequently, the diverse and unseen anomalies, which by definition do not share this compact structure, will fall far from this tightened manifold.\n3.  Meanwhile, more visualization results that support this can be found in **Fig.10 (Page 38)**. After applying IGAD, normal points (blue) are closer and abnormal points (red) are pushed further from the center of normal area. This is due to the effect of tightness objective that the manifold are tighten so that more anomalies are dropped out from this area.\n\n> **Q4: Relation between $f$ and $f'?$**\n\n**A-Q4:** $f'$ is the frozen and non-trainable copy of $f$, so they are two separate models with the same weights at all the time and share the same network architecture. $f'$ facilitates more efficient training rather than frequently opening or closing the gradient of $f$. Meanwhile, at the beginning of each training iteration, their weights are identical as $f'$ will copy the weights of $f$. **As indicated in the code example in Page 49,** it can also be verified $f'$ is the frozen copy of $f$, their weights are the same at all the time. This is also why the objectives in Eq.9 do not cancel out: Since $f'$ is fixed and frozen during gradient calculation, $\\mathcal{L}\\_\\text{idem}$ and $\\mathcal{L}\\_\\text{tight}$ create adversarial computational graphs to update $f$.\n\n> **Q5: Loss function or IGAD?**\n\n**A-Q5:** Thank you for this insightful question. We appreciate the opportunity to clarify this. We agree with you that IGAD with MSE loss shows the same results as the baseline (0.1684, Tab.2 vs. Tab.3). We interpret this as a key finding: IGAD is destabilized by the noise-sensitive MSE loss but is enabled by the robust MAE loss. The evidence for IGAD's contribution is in Tab.3, which show its effect within the optimal MAE setting. When our core IGAD objectives are removed, the average performance drops noticeably. This performance decrease, even while using a robust loss, demonstrates that the IGAD module is the primary driver of the improvement, with the MAE loss acting as the necessary stabilizing factor for it to function effectively."}, {"review_id": "54PSWqpu3b", "reviewer": "Reviewer_TXdD", "summary": "The authors propose a new framework to enhance general reconstruction-based time series anomaly detection methods. The main idea is to complement the training objective with an idempotent loss term and a tightness loss term. The former aims to avoid the model of being too affected by noisy normal data in the training phase, while the latter focus on tightening the learned manifold to mitigate the model capability of reconstructing abnormal instances. The proposal is evaluated with 15 models from the literature on 4 datasets. Most results indicate gains (sometimes by a large margin) when using the new strategy when compared to not using it.", "questions": "- Section 3.3.2: When explaining the idempotent objective, there is no obvious reason to pursue the frequency domain reconstruction path. Could, for instance, you elaborate on that? Is just a way to avoid the inclusion of trainable parameters? Could time-domain PCA be considered? This discussion could be a valuable inclusion to the text.\n\n- Section 3.3.3: The \"Inverse Effect\" is the main idea behind the tightness objective. However, it is not straightforward to link the accident expansion of the learned manifold and this operation. This section can be enhanced with a more thorough discussion on this.\n\n- Fig. 1: Can you make the font size slightly bigger?\n\n- Some references indicate the arXiv version, instead of the published version: [19, 27, 57, 69].", "limitations": "The authors state the limitations of their work.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "The paper is well organized and written, with each component of the IGAD framework detailed both visually and textually. The experiments are comprehensive, covering several recent models from the literature and presenting strong results when the proposed approach is included. Moreover, the fact that IGAD can used to enhance very distinct reconstruction-based methods is very interesting. The robustness and sensitivity analysis further highlight the advantages of IGAD. The ablation section is very detailed and justifies the methodological choices. I also praise the authors for the provided code and supplementary material, with several additional results.\n\nDespite not presenting serious weaknesses, it would be of interest to further explore the intuition behind the importance of the idempotent loss term. In practice, does it prevent the cases illustrated in Fig. 1? Perhaps some toy examples could be used to emphasize this point. Moreover, in the light of the work by Liu and Paparrizos, 2024 [31], which praises simpler models, it would be useful to verify how IDAG behaves with models such as standard AEs and VAEs.\n\nThe need of tuning the five hyperparameters in the IGAD objective could hinder its use in practice. Tab. 4 indicates that it would be difficult to fix default values across distinct datasets. Nevertheless, the authors argue that this choice may be reduced to three hyperparameters (the ones introduced by IGAD) and indicate good search intervals, which can be informed to standard optimization tools, such as Optuna.", "was_revised": true, "final_justification": "The authors provided detailed responses and clarifications to my questions. I maintain my favorable score.", "rebuttal": "Thank you for your insightful and constructive suggestions, which inspires us to further improve our work. We have carefully considered your points and provide our detailed responses below.\n# Main Comments:\n> **W1: Further explore the intuition behind the importance of the idempotent loss term. In practice, does it prevent the cases illustrated in Fig.1?**\n\n**A-W1:** Thank you for this insightful suggestion. First, we wish to clarify the role of the idempotent loss term ($\\mathcal{L}\\_\\text{idem}$) in addressing the challenges illustrated in Fig.1. In practice, the primary intuition behind the idempotent loss term is to enhance the ability to balance robustness against natural data noise with sensitivity to true anomalies, which directly corresponds to the scenario depicted in **Case 3 of Fig.1,** where normal data affected by noise becomes difficult to distinguish from subtle anomalies. Our method generates the augmented vector $z$ by applying a Fast Fourier Transform to a normal time window, sampling from the distributions of its frequency-domain features (real and imaginary parts), and then applying an Inverse FFT. This process ensures that $z$ retains the intrinsic patterns of the normal series while incorporating structured, naturalistic noise. By enforcing the idempotent constraint $f(f(z))=f(z)$, we expect the model to learn a stable mapping for these ***noisy-but-fundamentally-normal*** samples. In essence, it teaches the model to see through the noise, recognize the underlying normal structure within the perturbed signal, and map it to a consistent point on the target manifold. This enhances the model's ability to recognize core normal dynamics in the presence of real-world disturbances, thereby improving robustness without being overly sensitive to such fluctuations.\nThe issue of over generalization, shown in **Case 1**, is handled by the tightness objective $\\mathcal{L}\\_\\text{tight}$, which is divided from the original idempotent loss term. We further explore the effect and intuitive of this **in our response to Q2. We have also added more visualization examples in our work to show this.**\n\n> **W2: Verify how IDAG behaves with models such as standard AEs and VAEs.**\n\n**A-W2:** Thank you for your valuable suggestions and highlighting this important findings in [31]. We also agree that demonstrating the utility of IGAD for simpler base models is crucial. Here, we implement standard AE and VAE, recording their behaviours before and after applying IGAD to conduct further exploration. The experimental results are shown in the following table. We can observe that IGAD also shows positive effects on models with simpler structure, including AE and VAE here. Meanwhile, it can be noticed that:\n1. Before applying IGAD, VAE shows a higher level than AE on four datasets;\n2. After applying IGAD, AE with IGAD shows higher improvements than VAE with IGAD, and even has better performance than VAE with IGAD;\n3. AE and VAE achieve comparable results compared to certain complex models, which is consistent with the mentioned findings in [31].\n| Model | Dataset |  |  |  |  |  |  |  |  |  |  |  |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n|  | SMD |  |  | MSL |  |  | PSM |  |  | SMAP |  |  |\n|  | w / o IGAD | w / IGAD | $p$-value | w / o IGAD | w / IGAD | $p$-value | w / o IGAD | w / IGAD | $p$-value | w / o IGAD | w / IGAD | $p$-value |\n| AE | 0.3113±0.0029 | 0.3823±0.0053 | *** | 0.0083±0.0003 | 0.0370±0.0208 | *** | 0.1437±0.0001 | 0.1640±0.0098 | *** | 0.0969±0.0007 | 0.9149±0.0784 | *** |\n| VAE | 0.3556±0.0041 | 0.3556±0.0037 | *** | 0.0086±0.0004 | 0.0087±0.0002 | *** | 0.1464±0.0004 | 0.1465±0.0003 | *** | 0.0930±0.0014 | 0.0947±0.0014 | *** |\n\nWe conclude this as that VAE itself has a regularization by KL divergence on the latent space, which requires the prior distribution of latent vector $z$, denoted as $p_\\theta(z)$, is close to the standard normal distribution, which leaves non-regularization AE with more room for improvements.\n\n> **W3: Tab. 4 indicates that it would be difficult to fix default values across distinct datasets.**\n\n**A-W3:** Thank you for your concerns on choosing the optimal parameters for different datasets. Yes, we have provided a hyperparameter analysis in Appendix D and Appendix E.3 to help address this. For example, we have found that $\\lambda_\\text{idem}$ in the range [0.1, 0.5], not too strict $\\lambda_\\text{tight}$ and larger values for $\\alpha$ can be given priority consideration in the most cases. We hope these analysis can be helpful for reducing practical difficulty and further applying IGAD.\n\n> **Q1: The reason for frequency reconstruction path and the effect of time-domain PCA?**\n\n**A-Q1:** Thank you for this constructive suggestion which inspires us to further explore our design in IGAD. In fact, we choose frequency domain reconstruction path considering several advantages of this, not limited to avoid additional trainable parameters. First, concretely, these advantages can be stated as follows:\n1. Many multivariate time series are defined by their periodic dynamics. FFT decomposes a signal into a basis of functions, which is a powerful mathematically language for describing such patterns. Perturbing the signal in the frequency domain by sampling from the same distribution of real part and image part allows for direct manipulation of these crucial components, resulting in a complex, structured, and globally consistent form of noise in the time domain that is more realistic than simple additive noise, which helps us to generate the ***noisy-but-fundamentally-normal*** samples;\n2. The basis functions used in FFT are fixed and independent of specific data. This makes the method more flexible, eliminating the need to learn basis functions from data, offering higher computational efficiency, and being easy to implement. This aligns with our design principle for IGAD to be a lightweight, flexible module that does not require additional training parameters.\n\nSecond, we agree with you and believe that the time-domain PCA is a powerful candidates for this part. It also has its own advantages:\n1. Unlike the fixed basis functions of FFT, the principal components of PCA are learned entirely from the data itself. This means that PCA can capture the important linear variation patterns in a specific dataset;\n2. PCA can find the optimal linear low-dimensional representation of time series. By perturbing only a few of the most important principal components, it is possible to selectively modify the main structure of the signal.\n\nAfter comparison, we realized that the time-domain PCA may suffer the following issues:\n1. In contrast, although PCA can identify the main directions of data changes, these directions are linear and may not effectively capture the complex dynamics in the signal;\n2. The principal components of PCA represent the directions with the greatest variance in the data. Although perturbations to these components are effective, their impact may be more limited to the main linear patterns in the data and may not necessarily produce equally global and structural complex perturbations;\n3. PCA requires an additional, data-driven step to learn its principal component basis from time series data;\n4. The choice of the number of principle component $k$ can be viewed as an additional constrains. For example, if $k$ is too small, augmented $z$ may only reflect the most significant patterns of change in the data, resulting in simplified $z$ that might lack diversity; if $k$ is too large, it may introduce noise that already exists in the original data (reflected in principal components with small variance) into the perturbation process, and may not achieve the desired effect of retaining normal patterns and injecting randomness.\n\n> **Q2: More thorough discussion on the tightness objective and the Inverse Effect.**\n\n**A-Q2:** Thank you for pointing this out. The link between the \"Inverse Effect\" of our tightness objective and preventing the \"accidental expansion\" of the learned manifold is important for addressing the issue of over generalization. As described in Sect.3.3.3 and Fig.3 of the paper, a standard idempotent mapping causes the gradient to flow through both $f$ during optimization. **This produces a red path gradient that tends to “stretch” the manifold outward to include every $z$.** If the training data contains unlabeled outliers or the model has overly strong decoding capabilities, this expansion may erroneously include the outlier region within the normal manifold, leading to over generalization. To address this, our tightness objective, introduces an adversarial training dynamic by using a frozen copy of the model, $f'(\\cdot)$. This objective explicitly rewards the current model $f(\\cdot)$ for ***maximizing*** the distance between the reconstruction $f(f'(z))$ and the input $f'(z)$, where $f'(z)$ represents a point on the old manifold. **This push operation is the \"Inverse Effect.\" It directly restrict the tendency of expansion by preventing the model from learning a lazy or overly-inclusive mapping, i.e., $f(z)=z$, by *maximizing* the distance between the first mapping $f'(z)$ and $f(f'(z))$.** The model is constantly required to refine the representations produced by its past self, forcing it to learn a more precise and compact definition of normality. We will further integrate and update into our manuscript, ensuring the connection is clear to the reader for better understanding.\n\n> **Q3 & Q4: Font size and reference version?**\n\n**A-Q3 & Q4:** Thank you for pointing these out. We will increase the font size in Fig.1 for better readability in our next version. Meanwhile, After searching, these four papers have been published on ICML 2024, ICLR 2014, ICLR 2025 and IJCAI 2024. We will recheck the references in the text and update them to the published versions."}]

Author rebuttal:

Meta-review: This paper proposes IGAD, a plug-and-play module for reconstruction-based multivariate time series anomaly detection, inspired by Idempotent Generative Networks but adapted with redefined objectives, FFT/IFFT-based perturbations, and a tightness loss to mitigate over-generalization. IGAD requires no additional trainable parameters and can be integrated with diverse backbone models.

The method is evaluated on 15 models over 4 datasets, later expanded to 6 in rebuttal, showing consistent VUS-PR improvements in most cases, with detailed ablations and efficiency analysis. The inference cost is unchanged and training cost increases only moderately.

The reviewers are split: two recommend accept (strong empirical results, broad applicability), one gives borderline accept (concerns on dataset scope and overhead largely addressed), and one gives borderline reject (perceived high similarity to prior work [45] and no consistent SOTA performance over contrastive methods). The rebuttal added datasets, clarifications, and extra experiments, resolving most concerns except for novelty relative to IGN.

After looking at this borderline paper myself, I can recommend acceptence. While some novelty overlap is a valid concern, the adaptation to MTS AD, no-parameter-overhead design, and broad applicability make IGAD a practical contribution, which is well-validated by experiments.

For the camera ready version, I suggest the authors include all their promised changes.
Decision: Accept (poster)