Beyond Marginals: Learning Joint Spatio-Temporal Patterns for Multivariate Anomaly Detection
Authors: Padmaksha Roy
Almuatazbellah Boker
Lamine Mili
Venue: Accepted by TMLR ()

Abstract: In this paper, we aim to improve anomaly detection (AD) by incorporating the time-varying non-linear spatio-temporal correlations of the multi-variate time series data in the modeling process.  In multivariate AD, the simultaneous deviation of multiple nodes from their expected behavior can indicate an anomaly, even if no individual node shows a clearly abnormal pattern. In many existing approaches, time series variables are assumed to be (conditionally) independent, which oversimplifies real-world interactions. Our approach addresses this by modeling joint dependencies using a copula-based framework, which decouples the modeling of marginal distributions, temporal dynamics, and inter-variable dependencies. We use a transformer encoder to capture temporal patterns, and to model spatial (inter-variable) dependencies, we integrate a copula. Both components are trained jointly in a latent space using a self-supervised contrastive learning objective to learn meaningful feature representations to separate normal and anomaly samples.

Published in Transactions on Machine Learning Research (09/2025)
Beyond Marginals: Learning Joint Spatio-Temporte Patterns for Multivariate Anomaly Detection
Padmaksha Roy
padmaksha@@t.com edu
Department of Electrical and Computer Engineering Virginia Tech
Almuatazbellah Boker
boker@ut, edu
Department of Electrical and Computer Engineering Virginia Tech
Lamine Mili
Imili@vt.. edu
Department of Electrionl and Computer Engineering Virginia Tech
Reviewed on OpenReview: https: // opw.eeveee.........mm form?? id= sETTviokj
Abstract
In this paper, we aim to improve multivariate anomaly detection (AD) by modeling the time-varying non-linear spatio-temporal correlations found in multivariate time scries data In mullivariate time series data, an anomaly may be indicated by the simultaneous deviation of interrelated time series from their expected collective behavior, even when no individual time series exhibits a clearly abnormal pattern on its own. In many existing approaches, time series variables are assumed to be (conditionally) independent, which oversimplifies real-world interactions. Our approach addresses this by modeling joint dependencies the latent space and decoupling the modeling of marginal distributions, temporal dynamics, and inter-voriable dependencies. We use transformer encoder capture temporal patterns, and to model spatial (inter-variable) dependencies, we fit multi-variate likelihood and copula. The temporal and the spatial components are trained jointly in latent space using a self-supervised rastrvsiive learning objective to learn meaningful feature representations to separate normal and anomaly samples.
1 Introduction
Modern industrial systems rely on networks of interconnected sensors that produce vast streams of multivari- ate time series data during operation. Detecting anomalies in these data plays a critical role in identifying faults early, mitigating security threats, and maintaining system reliability and safety. industrial settings, time-series data are frequently used monitor the performance of machines, IT infrastructure, spacecraft, and engines. Anomaly detection has become vital component of time series analysis, enabling early detection of faults and preventing potential failures. Recent advances in deep learning have spurred the development of various methods to address this problem. For instance, recurrent neural networks (RNNs) Hundman et al. (2188;; Su et al. (2019); Canizo et al. (2019) have been widely used to capture temporal dependencics in multivariate soquences. Mcanwhile, other approaches employ graph-based models or Trans- former architectures Vaswani (017) to focus on variable relationships and sequential patterns Deng & Hooi (2021) Anomaly Transformer Xu (2021). These models effectively utilize temporal structures and adapt neural networks time series tasks. Despite these advancements, the detection of anomalies in multivariate time scries data remains challenge. The primary difficulty arises from the intricate temporal dependencies and correlations between multiple variables. Anomalies often manifest as subtle deviations that are hard to

---

Published in Transactions on Machine Learning Research (09/2025)
isolate from natural fluctuations without contextual awareness. In addition, real-world datasets frequently suffer from noise, missing values, and high dimensionality, further complicating the modeling process. A major limitation in this domain is the scarcity of labeled data. In many cases, it is unclear during training whether a given point represents an anomaly. This lack of ground-truth labels has driven the adoption of un- supervised learning approaches. Methods such as autoencoders and adversarial networks attempt to model data distributions without labels to identify deviations. However, unsupervised approaches often strug- gle with contextual anomalies and dependencies between variables, making detection unreliable in complex scenarios.
Multivariate time-scrics data, espocially high-order multivariate time series (HO-MTS), introduce additional layers of complexity that make anomaly detection particularly challenging. Unlike univariate time series, which involve a single variable observed over time, HO-MTS captures interdependencces between multiple variables, not just at a single time step but also across multiple time lags. This temporal and cross-variable dependency structure amplifies the difficulty of modeling and detecting anomalies. HO-MTS data require models to account for both spatial correlations (relationships between variables) and temporal dependencies (relationships across time steps). For instance, a sensor measuring pressure at time t might depend on the temperature reading at time (-1 or flow rate at time 2-2. These dependencies often span long time horizons, making it necessary to handle lagged interactions effectively. Traditional models, such as autoregressive methods, struggle to capture such intricate relationships, particularly when the data have nonlinear patterns or dependencies that are not explicitly obsorvable. This challenge is compounded when anomalies arisc from unexpected combinations of variable interactions rather than simple threshold violations, requiring models to analyze contextual anomalies instead of point anomalies. In HO-MTS, anomalies can occur as contextual deviations rather than isolated outliers. For example, sudden spike in temperature may not be anomalous if it follows an increase in pressure, but it could indicate a fault if the pressure remains constant. Anomalies are assumed to diverge from these correlated patterns of changes among multiple variables, considering their joint distributions.
Supervised learning methods, on the other hand, present an attractive alternative when labeled data is avail- able. These approaches can explicitly learn patterns associated with anomalies and differentiate them from normal behavior, especially when anomalies are subtle or involve relationships between multiple variables. Industrial datasets often contain very few labeled anomalies. Recent techniques, such as semi-supervised learning and contrastive learning, enable models to utilize a small set of labeled data while benefiting from larger unlaboled datasots. Supervised techniques also benefit from the ability to incorporate domain knowl- edge through labeled examples, improving interpretability and performance. Pre-trained models and data augmentation techniques can leverage small amounts of labeled data to improve performance significantly, Although unsupervised methods are useful when labeled data is unavailable, supervised learning provides better performance in scenarios where labeled data can be obtained (even in small quantities). It can model complex dependencies, improve interpretabilityy, and leverage domain-specific insights. Detecting anomalies in HO-MTS data requires capturing both temporal dependencies and spatial (variable) correlations effec- tively. While deep learning models such as Long Short-Term Memory networks (LSTMs) and Transformers have shown success in modeling temporal patterns, they often struggle to adequately capture latent depen- dencies across variables, especially in scenarios with complex interactions and nonlinear relationships among the variables Tian et al. (2023): Chen et al. (2023); Zheng et al. (2023).
Transformers Vaswani 2017 rely on self-attention mechanisms to capture long-range dependencies in se- quences. Recent studies such as Jeong et al. (2023). : Tuli et al. (2022). and Xu (2021) underscore the limitations of modern Transformer architectures in jointly capturing spatial and temporal dependencies for effective anomaly detection in multivariate time series data, AnomalyBERT Jeong et al. (2023) introduces a specialized Transformer architecture that incorporates 1D relative position bias to capture these multi- variate spatial interactions better. TranAD Tuli et al. (2022) highlight that plain Transformer architoctures struggle with detecting subtle anomalies in multivariate data, often failing when deviations are minor. To overcome this, the authors enhance Transformers using adversarial training and self-conditioning to amplify anomaly signals and improve generalization. By incorporating model-agnostic meta-learning, TranAD out-
performs basic Transformer models by a huge margin. The authors of Anomaly Transformer Xu] (2021) argue that "pointwise" reconstruction or prediction errors treat each timestamp in isolation, so they fail to
2

---

Published in Transactions on Machine Learning Research (09/2025)
capture longer-range dynamics and are casily overwhelmed by the vast majority of normal points. To address this, they introduce two complementary attention maps mars---rrr-ssssciaaiino from the raw self-attention and a Iearnable prior-association that biases toward local context- and define their divergence or association discrepancy as an anomaly score.
Our central contribution is a unified spatio-temporal modeling framework that combines Transformers (for temporal context) with modcling of multivariate joint spatial dependencies- integration we believe to be novel in the multivariate anomaly delection(AD) literature. We use the original Transformer model Vaswani (2017) solely as a temporal encoder. Rather than modifying the Transformer to capture spatial dependencics, our key innovation lics in integrating it with a theorctically grounded multivariate modcling Gaussian, Student-t likelihoods, and copula theory- to explicitly model nonlinear spatial dependencies. Following AnomalyBERT Jeong et al. (2023), , we assume a small pool of real anomaly representations and then generate additional anomaly sample representations via simple augmentations to ensure that our synthetic anomaly samples remain realistic. [Salinas et al. (2019) noted that modeling correlations among variables is critical in multivariate AD and methods that assune independence across time series fail when inter-variable correlations are critical. In anomaly detection, subtle but coordinated deviations across nodes can signal issues even if individual nodes appear normal. While prior works have acknowledged this need, to the best of our knowledge, ours is the first to bring together theoretical and architectural components in a cohesive framework with a simple self-supervised contrastive learning objective. Our contribution can be summarized as follows:
We introduce end-to-end training framework that jointly optimizes a Transformer encoder to extract temporal information in high-dimensiona time series and multivariate likelihood or Copula model to capture the latent space spatial dependency. By backpropagating through both components simulataneously,, the model discovers latent embeddings that preserve local and long-range time relations while also conforming to consistent seriable-oovvriibbe dependencies.
Instead of labeling individual samples, we treat each window or frame as a coherent sequence, then build contrastive loss that enforces normal frames to achieve a high log-likelihood density, while anomalous frames are pushed below margin in log- lig-likel space. This approach better captures short-range temporal structure and ensures that anomalies which exhibit subtle multivariate deviations are effectively separated in the latent embedding space.
Experiments conducted on multiple public benchmark multivariate and synthetic time series datasets demonstrate that both the variants our model consistently outperform state-of-the-as techniques, achieving higher precision, recall, and AUC-ROC scores, especially when the time series exhibits joint spatio-temporal dependencies.
1.1 Related Work
Time series anomaly detection has been approached using both supervised Jia et al. 2019 Cook et al. (2019) and unsupervised methods Audibert et al. (2020), Zhang et al. (2021), Thill et al. (2021), each catering to different challenges posed by the data. Unsupervised mothods dominate this field due to the lack of labeled anomalies in most real-world datasets. Techniques such as autoencoders, isolation forests, and Gaussian mixtures focus on modeling the distribution of normal data and identifying deviations as anomalies. Deep learning models, such as LSTMs and Autoencoders, leverage reconstruction errors or forecasting residuals to detect anomalies without requiring labels. However, these methods often struggle with contextual and correlated anomalies, especially in multivariate settings where relationships between variables evolve over time. Supervised methods, on the other hand, utilize labeled datasets to explicitly distinguish anomalies from normal patterns. Approaches such as RNN classifiers, attention-based networks Wang & Liu (2024): Zhao et al. (2020), and graph neural networks Zhao et al. (2020), Deng & Hool (2021) excel at learning complex dependencies and classifying anomalies when labeled data is available. Recent advances in semi- supervised learning AAkcy et al. (2019) and transfer learning have further extended supervised approaches to scenarios with limited labeled data, offering improved accuracy and interoperability. While unsupervised methods are widely used due to their flexibility, supervised approaches are gaining traction they can better
3

---

Published in Transactions on Machine Learning Research (09/2025)
capture latent dependencies and nonlinear correlations in high-order multivariate time series, particularly when integrated with copula-based models to enhance dependency modeling in latent spaces. Previously, encoder-decoder based architectures integrated with adversarial training framework Audibert et al. (2020)
that leveraged the strengths of both auloencoders and adversarial training while addressing the shortcomings of each approach were developed for multivariate time series anomaly detection.
TranAD Tuli et al. (2022) leverages attention-basel encoders, self-conditioning, adversarial training, and MAML for robust, elficient, and dala-efficient anomaly detection and diagnosis. STADN Tian et al. (2023)
integrates spatial-temporal information using graph attention and LSTM networks, predicts sensor behavior, and enhances anomaly detection by roconstructing prediction crrors for better discrimination. Anomaly- BERTJeong et al. (2023) addresses this issue by introducing a data degradation scheme for self-supervised model training. They specifically define four types of synthetic outliers and propose a degradation process in which parts of the input data are replaced with these outliers. In addition to leveraging the self-attention mechanism of the Transformer architecture, their approach transforms multivariate data points into temporal inpesentationss enriched with relative position bias and computes anomaly scores based on these represen- tations. TACTis Drouin et al. (2022) addressos the challenge of estimating the joint predictive distribution for high-dimensional multivariate time series by introducing a flexible approach built on the transformer architecture, leveraging an iirentioo-basedd decoder that is theoretically proven to replicate the behavior of non-parametric copulas. Anomaly-Transformeer Xu (2021) identifies anomalies by leveraging their tendency to form concentrated associations with adjacent points, unlike normal data which associates more broadly across the series. CARLA Darban et. al. 20225) leverages contrastive learning and self-supervised strategy to enhance anomaly detection by learning similar representations for adjacent windows and distinguishing anomalics based on proximity to the representation. Another recent work Tang et al.] (2024) addressed the problem of time series anomaly delection with self-supervised contrastive learning by designing a perturba- tion classifier to infer the pseudo-labels of data perturbations.
2 Proposed Framework
In this section, we elaborate the deep learning framework, contrastive loss function and the training strategy that we considered to develop our model.
2.1 Problem Statement
In our problem, we consider a multivariate time series X E RDT arising from a cyber-physical system, where D is the dimension of each time frame and T is the total length of the time series. We embed X into an embedding space Z E RdxT : capturing the most influential features. Our goal is to design a contrastive loss that leverages the joint distribution (or mutual information) among these latent variables, as measured via a copula log-density frames. Intuitively, normal data lies in a higher log-likelihood of the density function region in the latent space, rellecting consistent dependency patterns, whereas anomalies disrupt these dependencies and thus spread over a lower log- ligliikeldoood of the the density function. By constructing a contrastive objective that enforces a large gap in the log-density (or mutual information) between normal and anomalous frames, we exploit the fundamental distinction in their dependence structures to enhance anomaly delection within the latent feature space.
multivariate likelihood method, to effectively separate normal from anomalous
In this approach, each short subsequence of length L in the multivariate time series is labeled frame, which can be either normal (label 0) or anomalous (label 1). (A) Labeling Frames. If any limestamp in a length-T snippet is anomalous, then the entire frame is labeled as anomaly; otherwise, it is normal. This ensures that anomalies spanning multiple time steps are not overlooked. The anomalies are introduced at random positions, and their proportions vary from 25% to 50% of the normal window length.
(B) Generating Frame Pairs. Once frames are labeled, we form pairs (x,, that may be normal-norman normal anomaly, or anomaly anomaly. Such pairs can then be used in contrastive or nutual-information based learning, since each pair directly encodes local temporal dependencies and interactions among variables. If either sequence is anomalous, label the pair anooaly-involved.
4

---

Published in Transactions on Machine Learning Research (09/2025)
(C) Advantages for Time-Series. By treating small windows as frames rather than single samples, we can better capture short-range temporal correlations, especially when anomalies or key dependency patterns span multiple points.
(D) Estimating the multi-uariate log-density. Computing mutual information or fitting copula or mul- tivariate likelihood requires sufficient number of samples and joint view of multiple dimensions; having short subsoquenees provides richer data for these estimations. This approach results in more stable es- timates of dependency patterns, allowing the model to differentiate high-log-density regions (representing normal data) from co-llog-density regions (representing anomalies).
2.2 Transformer Encoder
In this sctting, we employ a Transformer encoder to project: fa P R X Z. Transformer processes the entire time series in parallel by applying self-attention across all time steps. This mechanism allows every position the sequence reach every other position, capturing both local and long-range dependencies. In particular, for longer time horizons, the model is less prone to forgetting distant signals compared to recurrent networks. Hence, each latent vector Z, (corresponding to snippet frame in time) encodes the key temporal patterns that matter to distinguish normal from anomaly sequences. Each dimension of z emerges from attention-based feature extraction, so normal data cluster around consistent pallerns, while anomalies, which break lypical variable interactions, are projected elsewhere in the latent space. This dimensionality reduction clarifies which features (and which timesteps) are most crucial to anomaly detection. We use the standard encoder basod on the paper [Vaswani (2017). We utilize the Transformer encoder layer from the Py'Torch library, which comprises self-attention mechanism and a feedforward network. Unlike Jeong al. (2023), we did not implement their custom relative positional encoding, our focus is solely on employing the Transformer model as encoding network.
2.3 Dependency Modeling in Latent Space
In our anomaly detection framework, the latent representation z E Rdd captured by Transformer encoder undergoes joint dependency modeling among the dimensions of z. In order to learn the correlation structure, we mainly focus on estimating the log-density
multivariate gaussian and student-t likelihood (claborated in Algo D and the copula density of two main copula families: the Gaussian copula and the Student-l copula (detailed in Algo 2). Each of these approaches primarily learns a correlation structure in the latent space, allowing us to evaluate how well given aligns with typical (normal) behavior.
2.3.1 Dependency Modeling with Copulas
A copula models how the embedding dimensions (Z1,..., za) co-vary, focusing their dependency structure irrespective of individual marginal distributions. If normal embeddings z maintain certain correlational pat- terns (e.g., 21 rises when 23 drops), the copula assigns them high log-density. Anomalies produce embeddings that violate these learned dependencies, thus yielding lower copula log-density
Sklar's Theorem states that if be any d-variate CDF with continuous marginals F Fa, then there exists unique copula. such that.
F(zI...., 2a) = ((F((z)). ..... Fa(za)).
Conversely, if C is a copula and Fi,..., , Fa are univariate CDFs, then the right-hand side defines a valid joint CDF.
The Gaussian copula with correlation matrix E is defined by:
Cy(un..... u2) ) = क5{ (ui). (u()),
where --1 is the inverse CDF of standard normal, and by is the d-variate standard normal CDF with correlation Σ.
5

---

Published in Transactions on Machine Learning Research (09/2025)
The Student-t copula with correlation matrix Σ and V degrees of freedom is:
(u... ua) (T, (u1),... T, (ua)),
where T is the inverse CDF of the univariate Student-t(v), and Th is the multivariate Student-t CDF.
Algorithm Dependency Modeling via Log-Density of Multivariate Likelihood
Input Latent vector E RR, Cholesky parameters o, dof (for Student 1)
Output: bikeliieldhodd log c(2; (,
1. Standardise the latents:
2 /
Zstdd
Transform unknown marginals to gaussian/student-t marginals(O.1):
+ (Zatd,)), Gaussian,
U4 f
i 1==1,.. I,...,,d d.
Student-t, To (2std,);
3. Quantile transform:
Φ Φ- (ws); Gaussian,
(trausf)
i= 1=....,,
T (u:), Student-t,
Correlation matrix: unpack @ + L (lower-triangular, diag softplus)) D + LL 5, Log likelihood:
(transt) T Y-1 (tranus) + log det Σ + dl log dogg2 Gaussian,
log c(2) log L|
untd 2
- log 1 E/2 log det (NT 2))
vdd log (tranaf)) D (transf)
Student-f.
2
return log c(z; Φ. v)
Algorithm Dependence Modeling via True Copula Log Density
Input Latent vector IRd, marginal CDPs (F)) Cholesky parameters o, degrees of freedom v (for
Student-()
Output: Log copula density log c(u; 2)
1. Standardise the latents zero mean and unit variance: Zaid + (z (z-H)/o.
2. Transform marginals using probability integral transform u; Fi(zatd,s) E [0, 1), i d.
3. Impose standard Normal/Studeni margins by quantile transform
Φ (w:),,
Gaussian,
U/
1= l=.....,d
T (u), Student-t, t,
Estimate dependence via the correlation matrix: unpack & L (lower-triangular diag- softplus);
Σ < LLT.
5. Compute copula log-density:
pal;; Σ) Ed= log (1(s),
Gaussian,
log c(u)
log fra E); Edn log fo (v)), Student-t.
where Pa and ip,, fea and fe.. are the d- and univariate Normal and student-t PDFs return log c(u)

---

Published in Transactions on Machine Learning Research (09/2025)
Since we concentrate on the correlation parameters & (which define L), our optimization updates only those elements to best fit. normal data. Anomalies are detected if they yield low log-likelihood under this Gaussian or student-t correlation structure. The Gaussian copula is simple and captures lincar correlation well. is suitable when tail dependence is not extreme; or when anomalies primarily break moderale cross-variable correlations. More details can be found in our appendix section.
The Transformer encoder ensures each embodding or latent vector z is an expressive representation of tem- poral and cross-feature relationships of the original time series. Meanwhile, the copula. provides precise measure of joint likelihood for those latent coordinates. Consequently, normal data reside in a dense, high- likelihood region of embedding space, whereas anomaly embeddings occupy a lower-likelhhod, lower-density region.
2.3.2 Student-t Copula and Student-t Likelihood for Heavy-tailed Dependencies
To address more pronounced tail behavior, we can replace the Gaussian assumption with Student-t copula Student-t iuultivariate likelihood, which has an extra degrees of freedom (v) parameter governing tail thickness.
This approach again optimizes primarily the correlation parameters & (related to L) and we may also learn or fix the v parameter. Hence, anomalies that yield unerpecled tail dependencies are flagged with low likelihood. Real-world signals often exhibit outliers or heavy-tailed distributions. Student-t copula and a Student-t multivariate likelihood accommodates such extremes more naturally than the Gaussian copula, assigning higher probability mass in the tails. Thus, anomalies deviating in heavy-tailed manner more cleanly separated. Tf data indeed show large spikes, l-copula student-t multivariate likelihood typically yields more robust anomaly detection than its Gaussian counterpart.
2.3.3 Learning the Correlation Parameter 6
In all the cases, our primary focus is on learning the correlation structure that defines how the latent dimensions co-vary for normal samples. Concretely, we store a vector of Cholesky parameters and the parameter in the Student-t case, and backprop through these when fitting normal data in training. At inference, any that fails to match this learned dependency pattern receives lower log-likelihood and is deemed more likely anomalous. Throughout, we primarily tune the correlation paramcters ø, cnabling an end-to-end training scheme where the latent encoder (e.g. a Transformer) supplies embedded features, and the copula measures how well they align with the normative correlation structure.
2.4 Contrastive Loss
We model the joint distribution of latent variables (Zi....,z)) za), represented by ca(z), which effectively captures their underlying dependencics irrespective of marginal distributions in case of a copula, and a mul- livariate log-densily log (z))) can be interpreted as reflecting the mutual information among these variables. If z aligns with the normal dependency structure, (z) high (i.e,, strong correlations), whereas anomalies that break these dependencies yield low copula log-density. Indeed, the copula log-likelihood is the sum of log co over samples; thus, maximizing copula likelihood is equivalent to maximizing the multi-dimensional dependency, or high MT, in the embedding space. To exploit this for anomaly detection, we construct contrustive loss that (1) maximizes logc, (z. for normal samples Zn, 0 are the encoder parameters. The goal is to ensure that normal data inhabit a high-MI region, and (2) forces anomalous samples into a lower-density region. Specifically, we jointly optimize the encoder paramelers 0 (mapping raw time series to embedding space z) and latent space dependency parameters & by minimizing the loss function
((0,0) -
Σ Nooorm
log co (2.(1)) +
a Σ aE Anom
max log co (za (0)) (Pnorm 5)},
(1)

---

Published in Transactions on Machine Learning Research (09/2025)
Ovczapping alding window Frunca
VV MMM
G((()))
Enboshiing contesstive
Na. Na
Na (0,000)
Local Cana pestunbation
((F:0)
Projection netwerk (Trusfemmrr)
Timefruue pairs
Atcealy
Libeliboods
(Fa.V.h. asomely (X)
Figure 1: Spatio- Temporal Dependency Aware Feature Learning Framework
where Norm and Anom are normal and anomaly data frames.
A common concern arises because the multi-variate log-density, log ca (x()),, is often negative for high- dimensional data, mazimizing this quantity corresponds to minimizing its negative. Hence, for normal samples we add the term - log Co zz, (6)) in the loss, ensuring that we push those log-densities toward zero (ie., less negative). To separate anomalies, we place a margin constraint that the anomalous log-density stays below Puorm - 6, where Mnorm < 0 is the average log-density over normal data and > 0 is a margin. We then penalize any anomaly whose log-density log ca(za (0)) exceeds (Wnorm - 6). Backpropagating with respect both encoder (0) and spatial dependency (ø) parameters end-to-end guides the model to learn the embeddings that preserve key dime-dependent correlations for normal data (thereby raising their mutual information), while anomaly embeddings deviate and incur higher penalty.
2.5 Synthetic Data Generation Scheme
Since our training data have only normal data, we generate degraded inputs by replacing parts window with outliers during the training phase. Similarly to Anomaly-BERT Jeong et al. (2023), we randomly select an interval [fo,th t within window Stpst, The selected sequence Stast then replaced by one of the synthetic outliers described in the following.
2.5.1 Local Perturbation
We extend the degradation approach of Jeong et al. (0233 by introducing local perturbation scheme that synthesizes anomalies through small, controlled modifications of real anomaly snippets. A randomly selected snippet resized to match the target window and perturbed by adding Gaussian noise, applying slight feature scaling (e.g., [0.95, 1.05)), or permuting a few rows to induce temporal variation. The final perturbed snippet is then overlaid onto the target sequence. This approach ensures that the generated anomalies remain close to the real anomaly distribution while introducing variability for robustness. For the local perturbation approach, utilize 10-15% of the labeled anomalies to create anomalies that closely mimic the characteristics of true anomalies.
3 Experiments and Results
In this section, begin by outlining the experimental setup. Following that, conduct series of experi- ments to evaluate the effectiveness of the model, classification results, and the results of ablation studies.
8

---

Published in Transactions on Machine Learning Research (09/2025)
3.1 Datasets and Baseline Methods
We present experimental results on five widely-used benchmark datasets: SWaT, WADI, SMAP, MSL, and SMD Goh et al. (2017), Ahmeed et al. (2017), Hundman et al. (2018), Su et al. (2019) These datasets are derived from various sources, including sensors in server machines, spacecraft, and water treatment or distribution systems. We use a small portion of the labeled anomalies (10-15 %) approximately to generate synthetic anomalies with our local perturbation scheme. Again, we use only the continuous features in all the datasets for modeling. Several advanced models have been proposed for anomaly detection in multivariate time scries. MERLIN Nakamura et al. (2020) is a sclf-supervised method that generates pscudo- labels by learning representations and applies contrastive learning to detect anomalies effectively. LSTM- NDT Hundman et al. (0118) leverages LSTM networks for neural density estimation, capturing temporal dependencies in multivariate time series. DAGMM Zong et al.] (2018) combines dimensionality reduction with Gaussian mixture models through deep autoencoding framework estimate density and identify anomalies. OmniA nomaiy (Su et al. (2019) employs a variational RNN-based architecture lo model temporal dependencies and reconstruct imputs using stochastic latent variables for anomaly detection. MSCRED Zhang et al. (2019) reconstructs multi-scale signature matrices via convolutional recurrent encoder-decoder to capture temporal correlations and detect anomalies. MAD-GAN Li et al. (2019) applies an adversarial framework using GANs to reconstruct normal patterns, flagging significant reconstruction deviations as anomalies. USAD Audibertt et al. (2020) integrates adversarial training with autoencoders in a unified framework to learn patterns and identify anomalies. MTAD-GAT Zhao et al. (2020) utilizes graph attention networks to effectively capture spatial and temporal dependencies in multivariate time series. CAF-M Zhang et al. (2021) applics convolutional autoencoder to roconstruct and prodict temporal dependencies, laciliaaiig anomaly detection. GDN Deng & Hooi (2021) leverages graph neural networks to model inter- variable dependencies, enhancing anomaly detection performance. Finally, TranAD Tuli et al. (2022) adopts a transformer-basee architecture with attention mechanisms to model long-term dependencies and effectively identify anomalies in multivariate time series. We also compared CARLA Darban et al. (0025 and Anomaly- Bert Jeong et al. (2023) as baseline models. The Plain Transformer is the standard architecture from Vaswani et al., with an additional variant incorporating the relative positional bias proposed in AnomalyBERT to capture spatial relationships.
Table Performance metrics (Precision (P), Recall (R), AUC, F1) for various methods across datasets- Mcans + std, NA (Not Available) indicates metrics that were not reported in the respective original papers.
Methol
R
AEC
F1
A3C
Ft.
ABC
F1
MERRLIN
ESTANND
0.016 100
033533
0.004
0.821 0.15
00221 0.00
9 (/6
0.3143 64
0. 60557-2 0.15
08028
1/000 005
81602 016
0.721
3.0L
DAGAM
01076 300
05998 16
01533 0.044
BLa12400 0.
Li0es 0/5
91878 L16
04843
01
0.522 0.00
62 02303-800
10000 055
810116- 016
3.6
MISCRED
3m
04
8.05
GAN
02233 240
015224
00
149225 1.224
00000 1050
034612000 2.
015035 11.0
1U20012100 1653
55RE2 2 500
0.506223 2310
USAD
01831 2/01
01320
04
1L5221 01.24
EL3092 1682
L 0055
0 2/6
0.440 0.14
0.0032
NTADGAA
CAE-M
0 2
017555
2.044
IU5225
11154
L300 0/55
EJB
0.464:301
1.104
017751 11.54
12300.2100
55500.2
0 3.14
GDN
025222 2/0
0.733 2404
IUS77T 20024
1082
CUROT 1055
S1207 EUG
(L5021 2.104
EU000 1433
25814 500
0.50002 2110
03823 3/0
033233
IL9335 00.54
0.801 6882
01067 1U%
1344.2000
(L5331 2:104
9.06 0.9900
25008
0.5434
KA
XA
KA
NA
039012 0182
NA
NA
RA
(.554)2-1144
SA
NA
NA
033001 3.C
Phen Truakeezr
0.5100 0.02
0.0000 2 02
9,5070 = B.0
07300 = 000
50000 0034
0.500
57000 EUB
0.4762 3.10
CARLA
011080:: 3/0
02333 20
SA
RA
017237
20104
0.5011 1192
0.8027
Morkd (Cpeelal
Our 2orkd 2 MMlllimzate
0,211
019922 0004
(1254 20144
0.1000 0082
0.999 0.05
0.5990:2 0.05
0.9979 0.00
0. (L55
0.999
0.9955-20000
Malool
SMLAP
ADC
Ft
ALC
F1
MERLIN
0/571 = 8.0
0.500
0003
03:3
0:3441 0.00
[1107 3100
0/200-1 6.16
07700 0005
0.3733-
-001
LSVIANDE
147708 2 0/0
8
03642
1
3.14
E1200- - 5.66
0.9912 100
0179214003
[U881 6/6
100
03550 11.55
05411 1.00
12112 5.3
13000 2.00
13500
2015
0.5724 1115
MSCUUSD
04277 2 8/6
03928 100
03322 21.55
12000110 2/5
200
0135511705
USAN
005
03833
4.100
210
0341544 1051
MTTD CAT
02710- 001
03915
005
01831 0.55
55853 -14
(700) L200--00
(-001-1 -10
09811 015
01588) 0051
CAE-M
10102 2 0/0
03571
100
05007-400 1105
1A10 334
1S00T112 1/6
(13927 41244
TERADD
(U22 2 E/D
103
0.014:00.0
05653 1.00
L 31N
13601 1/0
IL995 2 1010
(15314-2100) 1154
Ancency Thas
0.910--200 601
03555 005
11013-065
19:13 316
0990-116 116
NA
01899 +0055
Paly Thonebrree
0713)
8337) +5.0
11066
5800 0.00
3351 45/6
6 4003
6200 -081
6:415
CARLA
0.206- 6mm
0.3377 4 mm)
XA
05111-00
12H1 300
0.800--00 E.
NA
01330 003
Oue Soald (Copela)
(L33X0 tuss
019334
0.5002 0.00
0.9944
03332 0:003
0.5569 0.09
((iliindee))
9

---

Published in Transactions on Machine Learning Research (09/2025)
(a) epach =
(b) epoch = 5
(c) epoch = 20
(d) epoch = 25
We jointly train the transformer and copula (student-t) parameters in the latent space with the contrastive loss and plot the normal and anomaly likelihoods. We see the gradual separation in likelihoods as we train for more epochs using our method.
Figure 2:
3.2 Training Strategy
We train our network by jointly optimizing the Transformer encoder parameters, denoted 0, and the joint density parameters, denoted é, in a single end-to-end fashion. First, each multivariate time series snippet is passed through the Transformer encoder, which produces a latent embedding or feature space z(0) capturing relevant temporal and cross-feature dependencies. We then evaluate the copula log-density log cy (z(0) to quantify how well the latent feature space aligns with the joint dependency structure learned from normal data. During training, we formulate contrastive loss that marimizes log ca (z())) for normal frames while pushing anomalous frames below specified margin in log-density space. Specifically, for each normal snippet we minimize log cæ (z)), thus maximizing the copula likelihood, and for each anomalous snippet we include a term that penalizes its log-density if it is not sufficiently lower than the normal average. By backpropagating through both the Transformer and the copula parameters, the system iteratively updates 0 and Φ such that normal data cluster a high- lige-likelihood region, while anomalies are assigned lower density and thus become well separated in the latent feature space. Figure presents a schematic illustration of our details the training procedure.
model. Algo g
3.3 Ablation Study
In our ablation study, we systematically vary several hyperparameters to observe their impact on anomaly detection performance. First, we adjust the margin in our contrastive loss, finding that too small a margin may allow anomaly frames to encroach normal regions, while too large margin over-penalize borderline anomalies and lead to increased false positives. We also vary the percentage of anomaly data we inject or label during training, confirming that a higher anomaly fraction generally helps the model to better distinguish anomalies, although an unrealistically high percentage may degrade generalization to real, rarer anomalics.
Furthermore, we experiment with different multivariate likelihood and copula families in the latent feature space for instance, Gaussian copula, Student-t copula (better for heavy-tailed dependencies). We also vary window size and overlap in the construction of time-scries frames, observing that larger windows capture extended patterns but raise computational cost, while more overlap provides smoother coverage but increases data redundancy. Finally, we alter the Transformer encoder architecture by changing the number of layers and attention heads. Other results are provided in Table 1. In Figure 2d we observe that as training progresses, the model achieves increasingly better separation between the likelihoods the normal and anomalous instances in the latent space. More details on the hyperparameters can be found in the appendix section.
3.4 Anomaly Detection Threshold Selection
In this approach, we first derive a log-likelihood score for each time series frame by evaluating log co (z(()) where og is the joint density function and z is the latent embedding returned by our Transformer encoder. Intuitively, frame with higher (less negative) log-likelihood aligns better with normal behavior, while lower log-likelihood indicates a potential anomaly. We gather all logllikes from the validation set and
10

---

Published in Transactions on Machine Learning Research (09/2025)
systematically sean range of possible thresholds, from the minimum to the maximum observed score. For each candidate threshold т, we label a frame as anomalous if its log-likelihood is below т, and we compute the corresponding score on the validation data. We then select the threshold that mazimizes the F1, thus balancing precision and recall most effectively. Once this threshold determined, any future frame whose log-likelln likelihood drops below deemed anomalous, while frames exceeding are considered normal.
4 Gradient Computation and Backpropagation
We focus on computing gradients of the contrastive loss with respect to ø (the spatial parameters that capture joint dependency) and 0 (the Transformer encoder parameters that generate latent embeddings). Here, cach frame-a short subsequence of the time-scrics-is treated an individual item, potentially labeled 0 (normal) or (anomalous).
Algorithm Single batch update of Transformer encoder 0 and latent dependency parameters e Input Mini-batch f(**) , ) with y E {0,1}
Output: Updated encoder and dependency parameters respectively.
1. Initialise learning rate V/, margin weight y, and standardisation parameters (1,
Forward pass: for each N.
Encode: (0) < 5 (x("))
(i) Standardise: std - (z( p)/a
Apply probability integral transform to convert to uniform marginals: ( + Gx( k == ==l,...,d
Compute log- likeliidd density: (6) - log ((u()))))
Compute contrastive loss with soft margin:
((0) + max(0, 0, log (ogc((()), - (Mrorm - 5)}.
N N
3. Average total loss: ((0,)) t
g(
1=1
Backpropagation:
N
Latent space dependency gradient: VoL =
log el () .6)
N 1=1
N
dc (z
Encoder gradient: VoL = Σ 1=1
iz(() 00
5. Parameter updates: < 0 VoL, return 0.0
$ t 0 n VoL.
By treating entire frames rather than individual timesteps, we more naturally capture local temporal context, while the joint optimization of (0, Φ) encourages both suitable embedding space and accurate copula- based dependency model. Anomalics emerge frames that fail to fit this latent dependency pattern, receiving lower log-likelihood under c(u, Φ) and thus incurring higher contrastive penalty.
https://github. com/paadnakssaa1/DAAC.
11

---

Published in Transactions on Machine Learning Research (09/2025)
5 Hyperparameter Tuning and Results
In the below section, we experiment with the different values of the hyperparameters and validate our model performance. In Figure 3 our experiments reveal that the classification performance of our model is significantly influenced by the selected copula family used to capture the joint dependency of the latent variables. This finding suggests that the Student-t copula provides a more accurate representation of the mata-generating, process, characterized by heavy-tailed distributions, rather than the Gaussian distribution. We keep the other hyper-parameters like margin, batch-size, window-length fixed for all the cases.
(a) Latent Space with Gaus- sian copula (MSL)
(b) Latent Space with Gaus- sian copula (SWAT)
(c) Latent Space with Gans- sian copula (SMAP)
(d) Latent Space with Gaus- sian copula (SMD)
(e) Latent Space with Student-t copula (MSL)
(f) Latent Space with
(g) Latent
Space with
(h) Latent Space with
Student-t copula (SWAT)
Student-t copula (SMAP)
Student-t copula (SMD)
(1) Plain (MSL)
Transformer
(i) Plain (SWAT)
Transformer
(k) (SMAP)
Plain
Transformer
()) Plain
Transformer
(SMD)
Figure 3: Performance metrics (Precision, recall, AUC-ROC) over epochs the validation data when the latent space is modeled with the Gaussian Copula, Student-t Copula, and Plain Transformer for different datasets. We select the threshold based on the method described in section 3.4
Our model achieves, on average, 5 18% improvement in classification metrics over baseline methods on most datasets. With the WADI dataset, the performance degradation likely due to an exception our assump- tion that variables exhibit distinct joint dependency structures under normal and anomalous conditions(refer to the appendix section for the detail analysis). We also observe poor performance of other baselines the same dataset. Through ablation in Figure 3 we show that incorporating Student-t copula-capable of modeling heavy-tailed and non-linear dependencies -eadd to significant gains. Distributional analysis confirms that the fcature marginals are highly skowed and deviate from Gaussianity, necessitating more expressive copula models.
6 Conclusion:
In our paper, we have created a unified, end-to-end anomaly detection framework that captures both temporal and multivariate relationships in complex time scries. The joint modeling helps to capture the complex time- varying spatio-temppral non-linear correlations which are useful indicators of multi-variatt anomalies. This approach yields a latent representation guided by copula likelihoods, effectively separating normal frames (high likelihood) from anomalous ones (low likelihood). Future work may explore more efficient attention mechanisms, advanced mixture copulas for even richer tail behaviors to improve anomaly detection in the latent space.
12

---

Published in Transactions on Machine Learning Research (09/2025)
References
Chuadhry Mujecb Ahmed, Venkata Reddy Palleti, and Aditya P Mathur. Wadi: a water distribution testbed for research in the design of secure cyber physical systems. In Proceedings of the 3rd international workshop on cyber-physical systems for smart water networks, pp. 25-28, 28, 2017.
Samet Akcay, Amir Atapour-Abarghoned and Toby P Breckon. Ganomaly: Semi-supervised anomaly de- tection via adversarial training. In Computer Vision-ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2 6, 2018, Revised Selecled Papers, Part III 14, pp. 622 637. Springer, 2019.
Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A Zuluaga. Usad: Un- supervised anomaly detection on multivariate time series. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 0395-3404, 2020.
Mikel Canizo, Isaac Triguero, Angel Conde, and Eurique Onieva. Multi-head can rnn for multi-time series anomaly detection: An industrial case study. Neurocomputing, 363:246-200 260, 2019.
Katrina Chen, Mingbin Feng, and Tony S Wirjanto. Multivariate time series anomaly detection via dynamic graph forecasting. arXiv preprint 0tt.:::202200055.. 2023.
Andrew A Cook, Göksel Misrh,, and Zhong Fan. Anomaly detection for iot time-series data: A survey, IEEE Internet of Things Journal, ((7):6481-649) 2019.
Zahra Zamanzadeh Darban, Geoffrey I Webb, Shirui Pan, Charu C Aggarwal, and Mahsa Salehi. Carla: Self-supervised contrastive representation learning for time series anomaly detection. Pattern Recognition, 157:110874, 2025.
Ailin Deng and Bryan Hooi. Graph neural network-based anomaly detection in multivariate time series. In Proceedings of the AAAI conference artificial intelligence, volume 35, pp. 4027-4034 2021.
Alexandre Drouin, Étienne Marcotte, and Nicolas Chapados. Tactis: Transforner-ateenionall copulas for time series. In International Conference on Machine Learning, pp. 5447 5493. PMLR, 2022.
Jonathan Goh, Sridhar Adepu, Khurum Nazir Junejo, and Aditya Mathur. dataset to support research the design of secure water ment systems. In Critical Information Infrastructures Securily: 11th International Conference, CRITIS 2016, Paris, France, October 10 12, 2016, Revised Selected Papers 11, pp. 88-99. Springer, 2017.
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. Detecting spacecraft anomalies using Istms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 387 387-395, 2018.
Yungi Jeong, Eunseok Yang, Jung Hyun Ryu, Imseong Park, and Myungjoo Kang. Anomalybert: Self- supervised transformer for time series anomaly detection using data degradation scheme. arXiv preprinl arXiv:2305.0444 2023.
Watson Jía, Raj Mani Shukla, and Shamik Sengupta. Anomaly detection using supervised learning and mul- tiple statistical methods. In 2019 18th IEEE international conference machine learning and applications (ICMLA), pp. 1291-1297. IEEE, 2019.
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks. In International conference on artificial neural networks, pp. 703-716. Springer, 2019.
Takaaki Nakamura, Makoto Imamura, Ryan Mercer, and Eamonn Keogh. Merlin: Parameter-free discovery of arbitrary length anomalies in massive time series archives. In 2020 IEEE international conference data mining (ICDM), pp. 1190 1195. JEEE, 2020.
13

---

Published in Transactions on Machine Learning Research (09/2025)
David Salinas, Michael Bohlke-Schneider, Laurent Callot, Roberto Medico, and Jan Gasthaus. High- dimensional multivariate forecasting with low-rank gaussian copula processes. Advances in neural in- formation processing systems, 32, 2019.
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2828-28377 2019.
Liaoyuan Tang, Zheng Wang, Guanxiong He, Rong Wang, and Feiping Nic. Perturbation guiding contrastive representation learning for time scries anomaly detection. In Proc. 33rd Int. Joint Conf. Artif. Intell, pp. 4955-4963, 2024.
Markus Thill, Wolfgang Konen, Hao Wang, and Thomas Bäck. Temporal convolutional autoencoder for unsupervised anomaly detection in time series. Applied Soft Computing, 112:107751, 2021.
Zhiwen Tian, Ming Zhuo, Leyuan Liu, Junyi Chen, and Shijic Zhou. Anomaly dctection using spatial and temporal information in multivariate time series. Scientific Reports, 13(14:4400, 2023.
Shreshth Tuli, Giuliano Casale, and Nicholas R Jennings. Tranad: Deep transformer networks for anomaly detection in multivariate time series dala. Xrviv preprinl or Niw:2201.07288, 2022.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Chaoyang Wang and Guangyu Liu. From anomaly detection to classification with graph attention and transformer for multivariate time series. Advanced Engineering Informatics, 60:102357, 2024.
Jiehui Xu. Anomaly transformer: Time series anomaly detection with association discrepancy, arv preprint arXiv:21 (00::21000020 0.02642, 2021.
Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, and Nitesh V Chawla. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI conference on artificial 2019.
intelligence, volume 33, pp.
Yuxin Zhang, Yiqiang Chen, Jindong Wang, and Zhiwen Pan. Unsupervised deep anomaly detection for multi-sensor Lime-series signals. IEEE Transactions on Knowledge and Data Engineering, 3522)22118 2132, 2021.
Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang, Multivariate time-series anomaly detection via graph attention network. In 2020 IEEE international conference on data mining (ICDM), pp. 841-850. IEEE, 2020.
Yu Zheng, Huan Yee Koh, Ming Jin, Lianhua Chi, Khoa T Phan, Shirui Pan, Yi-Ping Phoebe Chen, and Wei Xiang. Correlation-aware spatial-tempon graph learning for multivariate timo-scries anomaly detection. IEEE transactions on neural networks and learning systems, 35(9):11802 11816, 2023.
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autocncoding gaussian mixture model for unsupervised anomaly detection. In International confer- ence on learning representations, 2018.
A Appendix
Copula Theory
1. Copula Function: Copulas allow us to represent the joint distribution random variables using their marginal distributions and a copula function that captures their dependency structure. The joint PDF can be writlen
p(z) = clu; Φ) II p(z.),
14

---

Published in Transactions on Machine Learning Research (09/2025)
where u -w1, Ug,...,, ud) with = (2)) (the marginal CDF of 2,), and c(u; Φ) is the copula density. 2. Dependency Structure: The copula density c(u;)) Φ) encapsulates the dependency structure among the variables, while the marginal distributions p(z,) account for their individual behaviors.
Rewriting Mutual Information Using Copulas
Substituting the copula representation p(z) into the mutual information formula:
c((;; Φ)) I p(zz)
I(Z) p(z) log
dz.
11 p(zi)
Simplifying the logarithmic term:
[(Z) p(z)) log dz.
Change of Variables From z to
To simplify the integral, we perform change of variables:
v, = (z)), 2; = Fz, (u:).
The Jacobian determinant of this transformation is given by:
J p(z.).
Thus, the volume element transforms as:
du
du
dz
[
I p(2))
Substituting this change of variables into the integral:
[(Z) f
f c(u () du.
Final Expression for Mutual Information
The mutual information among the variables Z is expressed in terms of the copula density as:
(ZZ) f c(u o) log logelwu c(u; o) du.
This formulation shows that mutual information is equivalent to the expected log-likelihood of the copula density. The copula density c(u; 6) captures the dependency structure among the variables, independent of their marginal distributions, providing a comprchensive mcasure of dependency.
B Why Latent Space -computation complexity and high-dimensional setting
We implement Gaussian and Student-t copulas in low-dimensional latent space by parameterizing the correlation matrix Σ LL" using its Cholesky factor L, with positive diagonals enforced via softplus function. Latent variables are slandardized,, mapped through the Student-l CDF, and transformed back using the inverse CDF (PPF). The log-determinant is computed as log | = 2 Σ log(Lu) in O(d) time, and the quadratic form xTE-Zz inversion has a worst-case cost of (d?3), the small latent dimension d makes these operations computationally efficient. In our case, with datasets having fewer than 100 features, the added overhead minimal. For high-dimenssoonal settings, SSalnaas et al. (2019) propose decomposing the covariance matrix into diagonal and low-rank component, reducing parameters from O(N2) to O(Nr), and overall complexity to O(Nr²). which is effectively O(N) for small fixed
' is ovaluated via cinsum operations with O(d)) complexity. Although Cholesky
15

---

Published in Transactions on Machine Learning Research (09/2025)
B.1 Latent Space Analysis
The latent space analysis reveals that projecting data into a narrow bottleneck (i.e., low-dimensional space) leads to degraded model performance. In contrast, the model performs significantly better in higher- dimensional latent spaces, such as 64 128 dimensions (Figure)
(a) Latent Space 128 dimen-
(b) Latent Space 128 dimen- sions on SWAT dataset, L 100
(c) Latent Space 64 dimen- sions SWAT datasct, L 200
(d) Latent Space 128 dimen- sions SWAT dataset, L 100
sions SWAT dataset, L 200
(e) Latent Space 32 dimen- sions on SWAT dataset, I. 100
(f) Latent Space 32 dimen-
(8) Latent Space 16 dimen- sions on SWAT dataset, L=
(h) Latent Space 16 dimen- sions on SWAT dataset, I, = 100
sions on SWAT dataset, I.
200
200
Figure Performance metrics (Precision, recall, AUC-ROC) over epochs on the validation data when the latent space dimension is changed. The top row is for higher dimensions 128 and 64, the bottom row is for lower dimensions and 16 for different context lengths T.
B.2 T-SNE projection of the latent space
Here, we visualize the test or validation data by projecting into the trained latent space using I-SNE, which reduces high-dimensional representations rearesstttnonnrpppreeaaaatttooso to two dimensions while preserving local neighborhood structure for better interpretability. However, we believe that unlike the likelihood score, t-SNE preserves neighborhood ranks, not absolute density, and may place normal and anomalous points close together despite vastly different likelihoods. Additionally, in high dimensions, pairwise distances become unreliable due to the concentration phenomenon, making t-SNE embeddings unstable. contrast, the copula likelihood leverages full joint density, providing a more reliable measure for anomaly detection (Figure 5
180
400
-3K
1009
100
(a) MSL dataset
(b) SMAP datasct
(c) SWAT dataset
Figure 5: Latent space projections using t-SNE various datasets.
16

---

Published in Transactions on Machine Learning Research (09/2025)
C Hyperparameters
The following are the key hyperparamcters on which our model performance depends.
Window Size (L): Each multivariate time-series snippet. of length L, meaning we process I. consecutive timestamps as a single input frame. This choice controls how much local context the model sees at once.
Overlap or Step Length: After extracting a window of length L, we often shift by smaller step (e.g.: 1./2 another fraction) for the next window, creating overlapping frames. Overlaps help capture transitions more smoothly and increase data availability, but can lead to redundancy the overlap too large.
Number (or Percentage) of Anomalies: We typically define what fraction (e.g. 10% or 30%) of frames label as anomalous for synthctically generated sets. We gradually vary this percentage and check the model performance with various percentages of synthetic anomaly samples.
Batch Size (B): The batch size indicates how many frames process in one forward/ backward pass.
Although not strictly a hyperparameter, we experiment with different numbers of layers, attention heads, and embedding dimensions.
Transformer Depth and Heads:
Copula Family and Parameters: In the latent feature space, we choose a specific copula family (Gaussian, Student-t) along with its parameters or cstimation method. This affects how well the model captures the joint dependency structure among latent feature dimensions.
Margin (6) in Contrastive Loss: We define margin 5 that ensures that anomalies remain sufficiently below the normal log- liglliiel region. Larger ó forces stricter separation but cause more false alarms if the model over-penalizes borderline samples. We employ a sample-level margin, ensuring that each anomaly sample lies below that margin.
Learning Rate and Optimizer: Finally, we choose an optimizer (c.g., Adam) and a learning rate for both the Transformer encoder parameters and the copula parameters. Tuning this crucial to ensure stable, efficient convergence.
Weightage (ax): We define the a parameter to balance the different components of our loss function. We experiment with different values of a.
D Average Detection Delay (ADD) Computation and Interpretation
Let {m:)²1 be the ground-truth labels (0=normal, 1=anomalous). We define each anomaly event as a contiguous run of ones; its true start t, is the first index where yt, = 1 and Ut,-1 = O. Using sliding windows of length L (stride=1), window covers samples [i,..., i L - 1] and thus ends at
fend,i = i + 1.
After thresholding log- ligllikel we obtain binary predictions pt 10. For cach event start t,, collect all flagged windows with p; =1 whose end tend, 2 ts, and set the detection time
la = minf lend,s PPs = 1, lead, 2 (,).
The per-event detection delay is then
A ta ts
The Average Detection Delay (ADD) is the mean of these per-event delays over N events:
N
ADD = Σ A
N
1=1
ADD measures the average number of time-steps between the true onset of an anomaly and the model's first alarm. A lower ADD indicates quicker detection. For example, ADD 20 means that, average, the detector flags event 20 time-steps after its actual start.
17

---

Published in Transactions on Machine Learning Research (09/2025)
(a) ADD (SWAT)
(b) ADD (SMAP)
(c) ADD (MSL)
(d) ADD (SMD)
Figure We plot the ADD vs epochs on various dalasets considering different context lengths(L) on the validation
data during training
In the ADD-vs-epoch plot, we observe that ADD decreases steadily during training and converges to zero for all window lengths for most datasets. By mid-training, the model consistently flags the anomaly in the very first eligible window (ending tz), yielding zero delay, This behavior confirms that the model learns to react immediately at the first sign of an anomaly training progresses (Figure)
E Marginal and Joint Dependency Perturbation Procedure on Synthetic Data
To synthesize anomalics that cxhibit both marginal shifts and joint-dependency changes, we apply the following sequence of perturbations to the latent features Z RTxd,
1. Affine Warp:
Z = 83 Zea + by. 8j Ul (8in:: maax), by (bminn maxx
where (Smin: Smax) and (bmin: bmax) control the strength of scale-and-shift.
Sequential Nonlinear Warps: Apply random curvature transforms in scries:
z" = (i
where each fi is one of
Power warp: + sign(x) |x|P, p ~ (Pminu Pmax). sign(z) In(1 (1 + a [xl), U(mmiu (maxx.
Log-soft warp:
Sinh warp: x + sinh(bx), U(bmin bmax).
3. Heteroskedastic Noise: Add time-varying Gaussian noise with amplitude modulated by simu- soid:
= = Zig Styi Enj ~ ~0,, = + A Asin(2ml/T)).
ensuring 2 0 by clipping.
By introducing affine and nonlinear warps plus hetcroskedastic noise and heavy-tails, we force a multivari- ale Gaussian or Student-I density which depends on correct marginal forms to react strongly to pure marginal distortions. In contrast, a copula-based detector relies only on rank-correlations and is invariont these marginal changes, it remains focused on true joint-dependency shifts from Buorm to Banom: yiclding robustness under minor to severe marginal perturbations.
E.1 Parameter Definitions and Evaluation Cases
dependency_strennne a Base coupling strength of the latent transition matrix Bnorm controls how strongly features co-evolve the normal regime.
mhomaly_dopendenn shift Magnitude of change from Buorm to Banomi larger 5 induces bigger joint- dependency shift for anomalies.
18

---

Published in Transactions on Machine Learning Research (09/2025)
Marginal change Degree of distortion applied to marginals via nooneenonlinaar warps, and heteroskedastic noise.
Case 1: Balanced (moderate) marginals and joint dependencies
moderate marginal shift.
moderate joint shift.
Apply moderate warp parameters and noise amplitudes to distort marginals.
Plain Transformer
Studeul-t Multivariate Model
Student-t Copula Model
Window Length (4)
Accuracy
Precision
Recall
AUC
Accuracy
Precision
Recall
AUC
Accuracy
Precision
Recall
AUC
20
0.511
0.321
0.866
0.727
0.931
0.950
0.917
0.983
0,981
0.981
0.981
0.995
50
0.517
0.324
0.856
0.725
0.919
0.933
0.904
0.974
0,978
0.980
0.975
0.995
100
0.538
0.334
0.849
0.735
0.920
0.941
0.895
0.972
0.974
0.988
0.950
0.993
2001
0.993
Table 2: Comparison of performance metrics between Plain Transformer, Student-t Multivariate Model, and Student- Copula Model across different window (context) lengths L.
Case High marginal drift, strong joint dependency shift
High marginal change.
large (strong joint-dependency shift).
Use extreme warp ranges and high heteroskedastic noise maximize marginal change.
Plain Transformer
Student-t Multivariate Model
Studeut-t Copuin Model
Window Length (4)
Accuracy
Precision
Recall
AUC
Accuracy
Precision
Recall
AUC
Accuracy
Precision
Recall
AUC
20
0.939
50
0.458
0.294
0.836
0.633
0.892
0.890
0,883
0.969
0.988
0.995
0.980
0.998
= 100
0.146
0.290
0.839
0.622
0.850
0.896
0.859
0.952
0.986
0.997
0.976
0.998
200
0.504
0.309
0.796
0.610
0.884
0.917
0.846
0.952
0.988
0.994
0.982
0.998
Table 3: Comparison of performance metrics between Plain Transformer, Student-t Multivariate Model, and Student- Copula Model across different window (context) lengths L (Case 2).
3. Case 3: High marginal drift, weak joint dependency shift
Marginal high change.
weak joint-dependeccy shift.
Use near-identily warp (e.g. small scales, zero noise) marginals remain essentially unchanged.
Plain Transformer
Student-t Multivariate Model
Student-t Copula
Window Length (L)
Accuracy
Precision
Recall
AUC
Accuracy
Precision
Recall
AUC
Accuracy
Precision
Recall
AUC
20
0.581
50
0.874
0.731
0.783
0.917
0.908
0.926
0.887
0.968
0.956
0.962
0.951
0.988
= 100
0,880
0.781
0.716
0.913
0.854
0.881
0.889
0.952
0.947
0.968
0.921
0.976
200
0.875
0.800
0.800
0.920
0.879
0.882
0.876
0.948
0.936
0.958
0.912
0.971
Table 4: Comparison of performance metrics between Plain Transformer, Student-t Multivariate Model, and Student-
Copula across different window (vontexi) lengths L.
Across all cases (Table 2, Table 3 and Table 4), the Student-t copula consistently outperforms the standard Student-t multivariate likelihood model, particularly when marginal distribution drifts are combined with weak strong shifts in joint dependencies. Interestingly, the plain transformer performs better when the joint-dependency shift is weaker which highlights the importance of modeling the joint dependency separately when its strong.
19

---

Published in Transactions on Machine Learning Research (09/2025)
F
Transformer Encoder Configurations
Although explored various Transformer encoder configurations, did not observe direct improvement in classification performance with increased model complexity, such adding more layers or attention heads. Interestingly, a lightweight model (e.g., Config 1, Config 2) achieved the best performance in our experiments.
Table 5: Detailed Configurations of Transformer Encoder Models
Parameter
Config 1
Config 2
Config 3
Config 4
Input Dimension (input_din dim)
64
32
128
Model Dimension (model_din
128
256
512
Number of Layers (num layers)
(tum_layers)
6
8
12
Number of Attention Heads (num heads)
2
4
8
8
Dropout Rate (dropout)
0.1
0.2
0.3
0.15
Pooling Mode (pooling mode)
Mean
Sun
Mean
Mean
Feedforward Dimension (dim [eedfoowwrd))
128
256
512
1024
G Baseline Comparison (LSTMs)
Here, we replace the Transformer with an alternative sequence model (LSTM) to compare their performance. We observe that the Transformer converges faster during training and achieves better performance on the SWAT datasot. However, across other datasets, the difference between the two models not significant. (See Figure 7 for SWAT results.)
(a) Performance
metries
(b)
Performance metrics
(c)
Performance metrics
(d) Performance metrics 200LSSTM-Copula)
L 20 (1STM+Copula)
L 20(LSTM-Copula
10O(LSTM+Copula
(e)
Performance
metrics
(f) Performance metrics L =
g
Performance metrics 100(Tfrurr Copula))
(h) Performance metrics
L = 20(Tfruur Copula)
20(Tfrmm Copula)
200 (oN(Trrrr-Copullll
Figure Performance metrics (Precision, recall, AUC-ROC) over epochs on the validation data when the latent space is modeled with LSTM and Student-t Copula(a-d) versus Transformer and Student-t Copula on the SWAT dataset for different context lengths(L..
H Dataset Details
The marginal histograms show that several SWaT features (e.g., FIT501, AIT201) exhibit clear distributional shifts between normal (blue) and anomaly (red) regimes, while others overlap more subtly. Under normal operation, the correlation heatmap reveals strong positive and negative dependencies among specific sensor pairs, forming distinct clusters. In the anomaly heatmap Figure [] many of these dependencies weaken, invert, disappear, indicating disrupted joint behavior. Together, these plots demonstrate that anomalies
20

---

Published in Transactions on Machine Learning Research (09/2025)
Table 6: Details of the benchmark datasets used in experiments.
Dataset
Train length
Test length
Anomaly % in test
Dimension
SWaT (2017)
495,000
449,919
12.13%
51
WADI (2017)
784,537
172,801
5.77%
123
MSL (2018)
58,317
T3,729
10.53%
SMAP (2018)
153,183
427,617
12.79%
25
SMD (2019)
25,300
25,301
4.16%
38
manifest both as marginal drifts and as changes in feature interdependence, motivating our copula-based modeling of joint distributions.
Comparison of SWaT and WADI: Marginal and Dependency Characteristics
1.1 Marginal Distributions
Most features cxhibit broad, continnous histograms under both normal and anomalous regimes. Anomalies manifest systematic shifts (new modes heavier shoulders) rather than isolated spikes.
SWaT:
Many sensors remain nearly constant during normal operation, producing spiky, low- variance marginals. Anomalies appear single-point outliers against Dirac-like baseline, resulting in extremely sparse, heavy-tailed histograms.
WADI:
1.2 Joint Dependency Patterns
The correlation heatmaps reveal clear block structures (clusters of mutually correlated variables) that deform smoothly under anomaly. These coherent dependency shifts are readily captured by rank-based copula estimators.
SWaT:
The normal-state heatmap is noisy and lacks large cohesive blocks. Under anomaly, only handful of near-constant sensors exhibit perfect correlations, while most variables remain uninfor- mative due to zero or near-zero variance.
WADI:
1.3 Implications for Spatio-tempora Detection
On SWaT, anomalics induce both marginal shifts and consistent joint-dependency changes across clusters. The copula component-berd invariant to marginals focuses on these dependency defor- mations, and the Transformer encodes their spatio-temporal patterns effectively.
On WADI, the scarcity of non-constant features and the predominance of one-off marginal spikes undermine rank-corrclation estimates. Copula inference becomes unstable or uninformative, and the model cannot learn a coherent dependency change to discriminate anomalies.
21

---

Published in Transactions on Machine Learning Research (09/2025)
Marginal Distribution: FIT101
Density
2
Normal Anomaly
-1.5
-1.0
-0.5
0.0
0.5
FIT101
Marginal Distribution: LIT101
Normal Anomaly
1.5
Density
1.0
0.5
0.0
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
UT101
Marginal Distribution:
AIT201
50
Normal Anomaly
Density
30
20
10
0.7
0.8
1.
1.1
AIT201
Marginal Distribution: AIT202
1.0
Normal Anomaly
0.8
Density
0.6
0.2
0.0
-25
-20
-15
-10
-5
0
AIT202
Marginal Distribution: AIT203
2.0
Normal Anomaly
1.5
Density
22
0.0
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
AIT203

---

Published in Transactions on Machine Learning Research (225)
Correlation Heatmap (Normal Data)
0.93
0.3
8443
2.54
0.49
0.58
0.16
0.15
2:000 0.15 0.045 0.44
0,250 9:15 DCCHLCO0224 431 0.0A D003
0.14 0L0220116 0.22-0.34
0.12
0.42
0.072
8.54
4.1
0.27
027
024 0.047
0.13
0.14 0.12
0.375
0.08E
0.15
-.27
-025 -5.77 023 018
025
0.07
0.5
4.018
0.75
0.5
0.82
0.25
0.45
9-44
0.54
0:299
0.25
0.12 0.0900 02230
0R0
0.21
0.17
0.45
0290.0009 N010000050.22 80550000000
-0.1
0.27
0.46
0.44
0.77
00170.009
4.11 0.085
0546 0000 00590 013
0.31
0.0000.01
0.52
0.013
0.22
026-0-122
0.01--.003
4.064
0.11
254
0.247.0 0.6-0.002
0::31
0.22
PVD0020 OCSOILDO
0L04)
083 1L0130.000
817 522
6.
1350 0ww0000000000000.000.00000 0011
0.076 0.20
0.76 0.97
0.72
0:11
017
4.18 1.11
A
0-42 0.44 0.21 0.32
0.6
022
O. 0.067 0.05
0.33
0.0
311
0.05
0.040
021
5.12
016 0.0
0.039
0.3 0.34
6.201
0.44
0.42
00055 0.009
2:44 0.073 3.25 0.11 0:13
0.11 0.22
022
0:15 0.011 0.06
0.12
0.17
0,030 0
022-0.088
0.055 0.057 D435
0.37 0.042 0.017
2048 0014 541
4.15 43 0.03 DcA
0.88 001
5:17
0.2
3116
4.33
0.77
6.14
0.72
0.36
0.22
40.9
20
0.56
0.13.6.05
0.5010.00223336
0,0550.00
0.44
0.50
0.29
DONG 0 28-0 www. 000.000000000000000000
0.0
0330
0:81
0.097
0010
R -000 60009.00720..1 00040.05700000 www.://00000000000000000
3.0400
0.48
0.8
0.039
0.005
-0.50
00TT0.044 2017 022 CAT1 0.07 017862001118 041
0047 0.003
0.775
9:16
0.6
0.00
0.
0.18
0.01
0.017
#
0.207
0.55
0.25
0017 4.023 0.5 0.13
2000
4.016
-2.15
0.225
-0.16
0.000
0.013
4
8013 018 307
0.07 0.190.0900000000110
002
0.02-0.091
0.37
0.3
0:50
0.20
5.000-0.030
0.36
0.25
0.012
0,0000
0063
2032 0.01 003 300
DET 0.04 24 422 3.01 0014 D.1
2012
0.042
0.51/0 000000110000000000
0.011 5011
00000000000000000000000000000000
4.02
010.00000000000000
0.00
Figure 9: Correlation heatmap of SWaT features during normal operation. Distinct clusters with strong intra-cluster correlations highlight the structured dependencies among sensors.
23

---

Published in Transactions on Machine Learning Research (025)
Correlation Heatmap (Anomaly Data)
-0.00
0.44
621
2.32
0.11
9.30
943
0.33
0.26
0.2
0.3
0.16
027
0.19
9.13
031
5.32
025
0.25
027
021
0.2
0.5
63
0.16
0.079 0:21
0.14
2.39
0.18
0.3
0.13
0.22
0.599
031
GET
0.11 0.0580.26
0.099
018
00--99000---00000000 064-0.058 000.0.0.....000..00000.
0.75
0.15
0.32
-0,899
039
0.18
022
0.22
015
0.6
9.21 6.21 022 022 022 0.0059
0,03
0.00
0.3
0.27
0.13
0.9
0:32
0.12
00:08-..2
6.744
6015 0.66 0.77
0.13 0.14 0.12 4.15 0.12
4,11
834 8.00E
4.11
0.56 0.19
06:
0.10 0.12 0002
0.009
C.76
0.033
0.44 027.0.07
0.25
0.13
0.7 0.14
DC74
013
0.44 0.1
S 8177
5047
0.651
61 6.19
4.6 607
0.19 40.5
K0:11
0.96
0.29
0.45
-3.44
4.87
83 0181
0655
A
0.28
0:18
0.21
0.51
C044
0.11 0.02 0.74 0.14
0.28
0,036
317
0.22
0,206
0.09
0.64
3.54
011
0::
0.12
0.11
0.34 61 0.35 0.12
0.19
2017
0.002
CO29
0.15
0.23 0.2
0.16
0.17
0300
9.17
0.003
0.50
0:00
0.24
04
8:
0.52 0.84 0.01
0.54 0.83 0.34 0619
0.74
4.11
3.50
0.18 45 4.41
631 4.1 A41
0.77
0.2
+0.60
4.:01
+0.00
0.34
0.29
0.53
4.37
-0.17
4.18
4.38 444 4.35 + 011100011
0.34
0.72
0.73 9.74 0.72
-00.5
00.00
0.764
0.11 0.2 037 0.14
0:50
2.44
0.51
0.99
03.55
0:35
00000
0.040.338
0.13 022 0.8
0.54
0.09
0.82
0.23
0:79
0.50
0:25
0.50
-0.50
0.570.150 0.11 0.2 015
234
0.2
0.90
0.04
0.33
0:20
00000
:1 6.14 0000000015 5.13 4.2 8.3
0.90
0.00
.0.1
4.12 -02 0.33 4012
0.2
0.92
4.88
-2.64
-044
+2.24
0.00
4.12
0.25 000-0003--006--- 4.13 0.22 0.30 4.36 4.15
014
0:86
391
01
0.59
0.03
+9
0.30
+3.72
0.59
0:95
0.59
0.59
0.59
0006
0.013 0.18 2013 0.00
0.5 00022 0.29 0217 0017
513 0m3
0823
0.12
GUL
0.20 0.03 OcCUM
www..000000000000.... 001300 C00 www.0000000000000 0011
0.00
Figure 10: Correlation heatmap of TWaT features during anomalous operation. Many previonsly strong correlations weaken or invert, revealing disrupted joint dependencies in the anomaly regime.
24

---

Published in Transactions on Machine Learning Research (09/2025)
Marginal Distribution: 3
800
Normal Anomaly
600
Density
400
0.76
0.77
0.78
0.79
0.80
0.81
3
Marginal Distribution: 4
Normal Anomaly
30
Density
20
10
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Marginal Distribution: 5
600
Normal Anomaly
Density
200
0.975
0.980
0.985
0.990
0.995
1.000
5
Marginal Distribution:
Normal Anomaly
400
Density
300
200
100
0.90
0.92
0.94
0.96
0.98
Marginal Distribution: 7
150
Normal Anomaly
Density
100
50
25
0.2
0.4
0.6
0.8
1.0
7


Reviews:
[{"review_id": "3qJZplf853", "reviewer": "Reviewer_LyxQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.   \nExplore the critical problem of time series detection.\n\nWeaknesses.\nHowever, the paper lacks formalization. First, both copula and abnormalities remain undefined despite being central concepts of the paper. Further, the paper assumes access to labeled abnormalities. This assumption reflects a classification setting for which multiple strategies have been proposed. Finally, some imprecisions weaken the paper (see requested changes)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work examines the application of copulas to identify anomalies in time series. Using a transformer architecture, the time series is embedded and then projected on a copula. Any point from a low-density region of the distribution is considered abnormal.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- In the abstract, the notion of a node is undefined and unclear without further context.\n- While mentioned in the Appendix, the notion of copula needs to be clearly defined, with a clear description of the fitting procedure.\n- While the setting appears to be the detection of abnormality in a given time series, it would be important to clearly state it as detecting abnormal time series among multiple time series could also be of interest. Furthermore, defining what constitutes an abnormality is crucial, as the assumed low-density region may not accurately reflect abnormalities depending on the chosen definition.\n- $z_n(\\theta)$ is undefined - is it equivalent to $f_\\theta(z_n)$?\n- In section 2.4, the comment on negative log-likelihood is unclear. The log-likelihood can be negative, but the aim should still be to maximize it. Currently, it reads as if the strategy ignores the likelihood associated with non-abnormalities.\n- Formalise the margin constraint and explain its enforcement (in algorithm 3, it is unclear why the margin is added to the likelihood itself)\n- Table 1 should contain confidence bounds to understand the significance of these results.\n- Algorithm 1 is unclear on the copula definition: the normal CDF is applied twice, while $u$ should be transformed so it follows a uniform.\n- In Section 3.7, the threshold selection is based on the validation set, which may misrepresent performance on a new dataset. Figure 3 should state how the threshold was chosen for precision and recall.\n- How is the gradient propagated jointly? The current formulation seems to compute them separately - similar to an alternate propagation rather than a joint one.", "broader_impact_concerns": "No critical concern", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6F4s5Mx6zO", "reviewer": "Reviewer_h3jr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1) The paper proposes an intuitive approach to anomaly detection in multivariate time-series, which is based on evaluating the log-likelihood of windows of the time-series and comparing it with a threshold\n2) The approach of creating latent representation using transformer and using copulas to create the joint likelihood is good and intuitive\n3) Good experimental results are shown for anomaly detection on multiple benchmark datasets against other existing methods\n\nWeaknesses:\n1) The paper is light on technical innovations in ML\n2) While experimental results are reported, there is less illustration on the nature of anomalies that are better detected by this approach than other approaches\n3) There is also no illustration on the latent space representation of the subsequences, on the basis of which anomalies may be separated from normal subsequences", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focuses on anomaly detection in multivariate time series. The aim is to identify such anomalies which may span across multiple variables, even though they may not be an anomaly with respect to any single variable. The proposed approach is to encode the time-series using a transformer, so that temporal patterns can be captured through attention mechanisms. Then a joint distribution is defined on the transformed time-series in latent space using Copulas. The idea is that a \"normal\" subsequence of the time-series will have higher log-likelihood than an \"anomalous\" subsequence, and this is enforced by contrastive learning at the time of training the parameters of the transformer and the copula. Good results are demonstrated for anomaly detection on several benchmark datasets compared to a number of state-of-the-art approaches.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It is mentioned in the abstract that the aim is to find such anomalies which may not be detected if we focus on any individual variate of the time-series, which is why the copula approach is required. It will be good to illustrate this point with synthetic data where these kinds of anomalies are created deliberately.\n\nWhile transformer seems to be a good and intuitive choice for encoding the subsequences into a latent space, it would be good to have comparisons with other models like LSTM. \n\nIt will be useful to have an illustration of the distribution of anomaly and normal subsequences in the latent space after transformation. Some t-SNE type of visualizations can make it compelling. The requirement for contrastive learning should also be highlighted this way. The dimensionality of the latent space too needs to be discussed.\n\nDetails of the benchmark datasets should be provided in a table. The performance of the proposed model is not great on the WADI dataset, but near-perfect on SWaT and other datasets. This needs to be explained with the nature of the anomalies in the different datasets. Also, for WADI dataset, recall is highest for MAD-GAN, but TranAD is highlighted.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LmI6fYiZA5", "reviewer": "Reviewer_zeTd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths. \n- Principled dependency module on top of a Transformer encoder, moving anomaly detection beyond purely marginal modelling.\n- Convenient end-to-end framework with a simple objective, avoiding complex multi-stage pipeline.\n\n## Weaknesses. \n### Unsubstantiated motivation. \nThe introduction asserts that LSTMs/Transformers “often struggle to capture latent dependencies” but cites no prior evidence and supplies no diagnostic experiment isolating that failure. Without either literature support or a synthetic test case where marginals are unchanged yet joint structure shifts, the central premise remains speculative.\n- Suggestion: add citations to empirical studies documenting this limitation and include a toy dataset where only cross-variable covariance changes; compare a marginal-only baseline with the proposed model to quantify the gap.  \n\n### Terminological mismatch: model is not a true copula\nThe encoder outputs unrestricted $\\mathbf z\\in\\mathbb R^{d_z}$. The “copula head” then evaluates an elliptical density directly on $\\mathbf z$; no probability–integral transform forces uniform marginals. Consequently the function lacks the mathematical properties of a copula.\n\n### Sliding-window labelling may delay alarms\nA window is flagged anomalous if any timestamp inside is anomalous. Near the onset of an event, the window is dominated by normal samples, potentially postponing detection by up to $L-1$ points. The paper evaluates only per-window F1/AUC, so latency cost is invisible.\n- Suggestion: add average detection delay (ADD) or time-to-detect metrics and vary $L$ to show the precision–delay trade-off. Compare with a point-wise baseline to quantify latency.\n\n### Stationarity assumption left implicit\nA single, global correlation matrix is learned and then fixed at test time, implying time-stationary dependence. Section 2 never acknowledges regime shifts, concept drift, or how the model would cope with them.\n- Suggestion: explicitly state the assumption; perform a drift experiment (train on regime A, test on regime B) or outline an adaptive extension (e.g. exponentially weighted covariance updates). \n\n### Missing formal definition of a copula\nThe manuscript jumps directly to “Gaussian/Student-t copula” without stating the standard definition (uniform marginals on $[0,1]^d$ and Sklar’s theorem). Reviewers unfamiliar with copula theory must look elsewhere.\n- Suggestion: open Section 2 with a concise formal definition with reference to Appendix A. Also place the Gaussian and Student-t specialisations there rather than in the experimental section.  \n\n### Performance claims over-state the empirical evidence\nTable 1 does not show uniform dominance. While providing a state-of-the-art model is not a necessary requirement, reproducibility concern still remains.\n- Suggestion: report mean ± std over multiple seeds. If possible run a statistical test to report p-value.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "An end-to-end Transformer with a Gaussian/Student-t dependency head learns joint, not just marginal, but using a contrastive objective, benchmarked with 15 baselines on five multivariate-time-series anomaly-detection benchmarks.\n\nPlease note that the following represents my initial impressions of the paper, and I am open to discussion. I welcome any corrections to potential misunderstandings.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I have incorporated requests in Strengths and Weakness section, please refer to it. My most critical concern is the first three.", "broader_impact_concerns": "I don't see critical concerns, but risk of false assurance in safety-critical monitoring could be worth discussing. The method’s current evaluation omits latency metrics and relies on a single, stationary dependence model; if deployed in time-critical application such as medical or autonomous-vehicle settings, delayed or missed detections could cause physical or financial harm. A Broader Impact Statement could discuss mitigation strategies (e.g., human-in-the-loop confirmation).", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)