Low Rank Transformer for Multivariate Time Series Anomaly Detection and Localization
Authors: Charalampos Shimillas
Kleanthis Malialis
Konstantinos Fokianos
Marios Polycarpou
Venue: ICLR 2026 Poster (2026)
Keywords: Anomaly detection
Anomaly localization
Multivariate time series
Space-time autoregression
Transformer
Primary area: learning on time series and dynamical systems

Abstract: Multivariate time series (MTS) anomaly diagnosis, which encompasses both anomaly detection and localization, is critical for the safety and reliability of complex, large-scale real-world systems. The vast majority of existing anomaly diagnosis methods offer limited theoretical insights, especially for anomaly localization, which is a vital but largely unexplored area. The aim of this contribution is to study the learning process of a Transformer when applied to MTS by revealing connections to statistical time series methods. Based on these theoretical insights, we propose the Attention Low-Rank Transformer (ALoRa-T) model, which applies low-rank regularization to self-attention, and we introduce the Attention Low-Rank score, effectively capturing the temporal characteristics of anomalies. Finally, to enable anomaly localization, we propose the ALoRa-Loc method, a novel approach that associates anomalies to specific variables by quantifying interrelationships among time series. Extensive experiments and real data analysis show that the proposed methodology significantly outperforms state-of-the-art methods in both detection and localization tasks. Code is available at: https://github.com/CharisShimillas/ALoRa.

Published as a conference paper at ICLR 2026
Low RANK TRANSFORMER FOR MULTIVARIATE TIME SERIES ANOMALY DETECTION AND LOCALIZATION
Charalampos Shimillas Kleanthis Malialis', Konstantinos Fokianos Marios M. Polycarpou 1.2
1 KIOS Research and Innovation Center of Excellence, University of Cyprus, Nicosia, Cyprus 2 Department of Electrical and Computer Engineering, University of Cyprus, Nicosia, Cyprus 3 DDepartment of Mathematics and Statistics, University of Cyprus, Nicosia, Cyprus
{shimilic 11as chezelampes, .klessthisss
ABSTRACT
Multivariate time series (MTS) anomaly diagnosis, which encompasses both anomaly detection and localization, is critical for the safety and reliability of complex, large-scale real-world systems. The vast majority of existing anomaly diagnosis methods offer limited theoretical insights, especially for anomaly lo- calization, which is vital but largely unexplored area. The aim of this contri- bution to study the learning process a Transformer when applied to MTS by revealing connections to statistical time series methods. Based these the- oretical insights, we propose the Attention Low-Rank Transformer (ALoRa-T) model, which applies low-rank regularization self-attention and we introduce the Attention Low-Rank score, effectively capturing the temporal characteristics of anomalies. Finally, to enable anomaly localization, we propose the ALoRa- Loe method, a novel approach that associates anomalies to specific variables by quantifying interrelationships among time series. Extensive experiments and real data analysis show that the proposed methodology significantly outperforms state- of of-the-an methods in both detection and localization tasks. Code is available at: http://gttubb.oo//CCCahhhhhii
INTRODUCTION
Driven by the rapid growth of the Internet of Things (IoT),, real-world systems have become more complex and vulnerable to faults. These anomalies frequently result in abnormal patterns for a stream of MTS. In this context, the diagnosis of anomalies in MTS is of great importance to ensure the reliability, safety, and efficiency of critical systems. Due to the scareity of labeled data, anomaly diagnosis is commonly formulated as unsupervised learning problem. It typically involves two key tasks: anomaly detection, which determines which timestamps are anomalous, and anomaly localization, which identifies the specific time series responsible for the detected anomalies.
MTS data exhibit complex dynamics, including temporal dependencies (relationships over time) and spatial dependencies (relationships across series). Effective anomaly diagnosis depends on reliably estimating these spatio-temporal dynamics. Deep learning models are widely applied to this task for their strong representation learning capabilities, with Transeermer--ased architectures shown to be especially effective in modeling the complex dynamics of MTS (Zerveas et al., 202T). However, the limited available theoretical insights into their decision process undermine relibbiityy and trust in safety-critical settings, while also complicating anomaly localization. Practitioners often raise important questions regarding the learning process, the interrelationships within the data learned by the model, and the need for a localization method, since detection alone provides limited practical value in complex large-scale systems. As illustrated in Fig. [ two of the main contributions this work are to address these open questions, which have remained largely unanswered.
In addition, several existing detection methods are built around the cojnt-adjustment evaluation strategy, whereby detecting any point within an anomalous segment suffices to consider the entire segment as an observed anomaly (Xu et al., [022] Song et al.] 20033) [Shen et al.) 2020). This approach often inflates performance metries, since even random scoring methods can appear com-

---

Published as a conference paper at ICLR 2026
(A]
Modeler
(B)
WAV
Contribution Weights
Recunstractioe space
Input space
Latent space
Cit
(1)
User
Bi 2(0)
Cu
A A reconsmactio
?
y
Ry 201
the dynumica
Eie
Cit
g10
line
(3) Can you find of anomaly?
Latcot space mothematiioal structure (Section 4)
Figure I: (A): Unresolved key questions commonly raised by practitioners. (B): The proposed local- ization method (Sec. 5.2). together with the theoretical analysis (Sec. 4 addresses these challenges. The figure illustrates the modeling process from the input space to the latent space and subsequently to the reconstructed outputs. An input series i E {1, ... d} can influence each latent space repre- sentation and subsequently each reconstructed output. These relationships learned by the model are quantified through contribution weights: Eaj E IR, capturing the influence of input series i on latent feature j E {I.... ....... ). and Cij E IR capturing its influence on the reconstructed output series j E {1,-.. d). These weights enable effective localization and enhance interpretability.
petitive under such evaluation (Kim et al. 2022: Huet et al. 2022). To ensure reliable evaluation, trustworthy detection metrics have to accurately reflect the temporal characteristics of anomalies.
To address the challenges of reliable detection scoring, interpretability, and effective localization, this study makes the following contributions:
We study how Transformer en- coders learn from MTS data, shedding light on how the learned representations relate to classical time series. These insights advance the design of anomaly diagnosis methods and deepen the understanding of Transformers sequential modeling. (Section
Theoretical insights of Transformer encoders on MTS:
Attention Low-Rank Transformer (ALoRa-T) for anomaly detection: We propose the ALoRa-T method, which enhances anomaly detection by applying low-rank regulariza- tion to self-attention, and introduce a novel detection metric, the Self-Attention Low-Rank score (ALoRa-T score). This method outperforms state-of-the-art methods across multi- ple benchmark datasets, and captures effectively the temporal characteristics of anomalies. (Section 5.1)
ALoRa-Loc method for anomaly localization: We introduce ALoRa-Loc, a localization method that first derives contribution weights that quantify the learned interrelationships between time series, offering key insights into the model's decisionnmaking process. More critically, these weights capture how anomalies propagate across time series during model- ing, enabling ALoRa-Loc to trace anomalies back to their origin and attribute them to the most relevant input time series. (Section 5.2
RELATED WORK
Due its importance for the safety and reliability of many critical infras- tructure systems, such as water distribution networks (Eliades et al.) (005),, power systems (Kyri- akides & Polycarpou 2015) and healthcare (Tang et al.) 2022). unsupervised anomaly detection in MTS has attracted significant attention. Traditional unsupervised methods (Liu et al. 2008 Bre- unig et al., 2000,, Schölkopf et al. (001) have been applied to this task, but may not perform well since they do not learn complex dependencies. To address these limitations, deep learning models that capture temporal and inter-variable relationships have shown improvements over existing meth- ods. A major class of deep learning approaches is reconstruction-basee methods, typically using an incoder-decoder architecture trained in a self-supervised manner. The main idea to learn mean- ingful latent representations that mimic the normal behavior. The models are expected to reconstruct normal sequences accurately. An unsuccessful reconstruction, in the sense of inflated errors, indi- cates potential anomalies in the data. Representative examples include RNN-based models such as
Anomaly detection:
2

---

Published as a conference paper at ICLR 2026
LSTM- VAE Park et al. 2018), OmniAnomaly (Su et al. 2019), , and InterFusion (Li et al. 2021). as
well as the CNN-based model MSCRED Zhang et al. 2019). More recently. Transformer models
Vaswani et al. 2017) have been proven highly effective for sequential modeling and well-suited to the complex dynamics of MTS (Nie et al. 2023). Anomaly-Transformn Transformer (Xu et al.) 2022) introduces the AssDis metrie, which compares each row of the self-artention matrix to a prior Gaussian distribu- tion. MEMTO (Song et al. (003) enhances a Transformer encoder with a gated memory module that learns memory items representing normal behavior. As new data arrives, it selectively updates these items. For anomaly detection, they introduce the LSD score, the distance between input queries and their closest memory items. Although the aforementioned models have shown strong performance under point-adjustment metrics, they may often not produce meaningful and reliable anomaly scores, as illustrated later in Figure 4 Lai et al. (2023) use the Performer model (Choromanski et al. 2021] to perform both point-wise and segment- wise reconstruction, defining the anomaly score as the ratio of their reconstruction errors. SARAD (Dai et al., 2024) uses a Transformer on transposed input windows, and thus the attention weights express feature- wise associations. To guide training and detect anomalies, SARAD relies on SAR metric, which tracks changes in attention weights, based on the intuition that anomalies disrupt feature associations.
Anomaly Localization: Anomaly localization has remained to a large extent an unexplored research area in deep learning. Recently has begun attract more attention, with some of the aforemen- tioned approaches also addressing this critical issue. Localization effectiveness depends on two key factors: first, the model's ability reconstruct normal patterns accurately. Second, the ability to interpret the reconstruction errors in way that isolates the true sources of anomalies without be- ing distorted by their effects. Some methods rely directly on the reconstruction error inform the localization. For example, SARAD and OmniAnomaly (Su et al. 2019) perform localization by ranking the reconstruction errors of individual time series. While this provides a baseline approach, anomaly propagation can obscure true anomaly sources. More advanced techniques refine the recon- struction signal to improve localization. InterFusion (Li et al.) 2021) applies Markov Chain Monte Carlo to adjust reconstruction errors. However, due to its computational complexity, it is limited to interpreting anomaly segments instead of individual time steps. DAEMON (Chen et al. (023)) leverages integrated gradients (Sundararajan et al.] 2017) to attribute anomalies back to individual dimensions. Finally, AERCA (Han et al.] (025) focuses exclusively on the localization task, without addressing detection and localization simultaneously while providing an interpretable approach and representing one of the most recent and effective methods in this area.
Research gaps: Although the aforementioned methods propose innovative architectures and scoring metrics, they often overlook key aspects: understanding what the model learns, how each time series represented, and how these representations contribute to the final decisions. Such understanding is crucial for making detection applicable and trustworthy, and for enabling effective localization.
3 MULTIVARIATE TIME SERIES (MTS) NOTATION
Multivariate Time Series (MTS) is sequence of observations recorded over time across multiple variables. It is denoted as Y = [/1, V/...., ynl where each row vector yı: E Rd. t = 1..... N, denotes the multivariate observation at time step f. Each column vector of Y. denoted as y e R. represents the i-th univariate time series of length N, and y/ E R is the value of the i-th series at time 1. Given the streaming nature of time-series data, modeling the MTS at an arbitrary time relies on its history Y (I-TT:). defined as the most recent T observations up to t: Y(r-T:)
[yt-T+1,... yr E RTxd Applying this construction to the full MTS Y E R xd produces a sequence of overlapping windows, each serving as the model input. Since the subsequent sections concern an arbitrary time t, for simplicity we denote the window Y Y((-TT)) by Y(p]-
4 THEORETICAL ANALYSIS OF TRANSFORMER ENCODER
This section presents a theoretical analysis of the learning dynamics of the Transformer encoder in the context of MTS. In particular, it relates the components a standard Transformer model to classical statistical models.
3

---

Published as a conference paper at ICLR 2026
Embedding: Given MTS window at time t, denoted as Yd RTxd Transformer encoder typically begins by embedding the input into a higher-dimensional space using a ID-convolutional layer, i.. it maps [[j] to (() E " where dmodel is the embedding (model) dimension.
(k) is computed Applying the ID-convolution, the k-th embedded time series value time step 1, Dt as:
(R) V/
Σ
(3)
Dits
Weighted average
This operation is mathematically equivalent to a learnable Vector Moving Average (VMA) filtering (Brockwell & Davis 2002), where the output at time is a weighted sum of multiple time series (k) determine the influence of time series i values in local window around t. The filter weights W.. on the output feature k, at lag j. The constant m is the kernel size.
Self-Attention Latent Space: The output of the embedding layer, Y E -2''ddaa is passed as input to the attention mechanism. At each layer the self-attention mechanism projects the previous layer's output ((-1) IT'dd into the Query (Q'). Key (K'), and Value (V') matrices using (v.)) learnable projection matrices ((Q.) {wij (2-0), }. ((k)) = {wlh (u(k))) and ( fwoy each of dimension R x dantell These projections are computed as:
Q' = (
K' ((k1), v 2(11)
The self-attention scores at layer are computed as the matrix 5t E RTXT:
(O'(K') M
((t) = softmax
(2)
Vimodal
where M E RTxT is an optional masking matrix, and the softmax is applied row-wise. The latent representation layer is updated through the residual attention mechanism, defined as:
z ( Z (1) (U-1), where
Z ()) ( (t w/(1),
(3)
skip connections are omitted, the update simplifies to ( z (1) By unrolling Eq. (3) we can derive formulations that relate the final latent representation, to useful statistical models. In the absence of skip connections, the latent representation at time step t can be written as:
Z, = 1g
(4)
where A, = S( ((T-)). ((1) E RIXT B W(V... W(VL) E R dhode X denae and S!!)
RIXT denotes the 1-th row of the final self-attention (SA) matrix S(L), When skip connections are included (the standard case), the final representation becomes
((L)
[0]
II (ITE( FEE+
Σ
II (V.)) YEZ
(5)
where I+ and It denote the indices in descending and ascending order, respectively. Based on Eqs 4 and (5). the transformation of the input data in the Transformer is expressed as left multi- plication of the self-attention matrices across layers and a right multiplication of the Value projection matrices. The former is input-dependent,, while the latter are learned parameters independent of the data. These two equations form the basis for the following proposition, which defines the struc- ture of the self-attention latent space. An analytical proof is provided in Appendix G along with additional details on the derived statements.
Proposition 1 (Space-Time Time Autoregressive (STAR) structure of the self-attention latent space).
Without skip connections: Each time series the Transformer's 's latent space follows a STAR-like _(1))
structure. In particular, it can be expressed as _(]) Edesi
dusia bui ES Oig 19 wwhich has the
exact same form as the classical STAR model (Cressie & Wilde 201T). The key distinction is that,
while traditional STAR models use fixed lag weights ong estimated by minimizing loss function (e.g.. mean squared error), the Transformer computes these weights dynamically from the inpur through the attention mechanism, with Q and K guiding their estimation in real time.
4

---

Published as a conference paper at ICLR 2026
2. With skip connections: Each time series in the final representation, Eq. ( can be interpreted as a linear combination of multiple STAR-like processes, where each component captures distinct temporal and feature-level dependencies of the original MTS.
3. With feed-forward layers: Adding feed-forward layers does not alter the STAR-like structure the latent space. The only difference is that the spatial weights take more complex form, as explained analytically in Proof[
Linear Projection for the Reconstruction: The final reconstructed MTS time step t is com- puted by applying a linear projection to the latent representation at the same time step:
dradd
Y, z W
(k) Dr
(L.)) jk 7
W
where W E Tid is the output projection matrix that maps the final latent representation to the original input space. This reveals that the model learns reconstruct each time series using linear combination of multiple STAR processes (Proposition !
5 PROPOSED ANOMALY DIAGNOSIS METHOD
This section presents the proposed anomaly diagnosis method. Section 5.1 introduces the ALoRa- model and its associated anomaly detection score, which together form the ALoRa-Det module, with overview provided in Fig. Section (.2tioon5522therr (h.2 presents the ALoRa-Loc method for anomaly localization, illustrated in Fig. | The corresponding pseudocodes are provided Appendix and computational analysis of the model is provided in Appendix
(3)
ALoRa layer
(1)
81 Rixd
(4)
ALoRa loss effect
Compute: Q'.
Normal Windoww
Abnormal
(
Σ
Weak Decoder
Window
(a) +1)
( R xdx.ee
8 7011 20-11,
5 20 D0g
Objective fujcctivo Minimization
Attention Low Rank layers
Crean IY 11-8116 11 Aom Σ
(8
Low rank, similar training
Incressed mank
(2)
Ligh Embed
ALoRa Score
Correlation structure
For (5.5) e Px
LightMTS- Embed
ALoRa-T(ww s(il)
Σ 1( hnd
Select
Top-K
Final Detection score
History: Yia in -e-
AS(a.) lb bill (ALoRo-T)
Pairs to ignoree
Figure 2: Overview of ALoRa-T and ALoRa-Det. (1) The architecture comprises the LightSMTS- Embed module, ALoRa layers, and a decoder.
(2) The embedding module exploits correlation structures retain only the most significant time series pairs, significantly reducing complexity by avoiding unnecessary information. (3) ALoRa layers impose a low-rank constraint on the self- attention matrix through novel loss and regularization term in the objective function, producing a signal for abnormality. (4) During inference, anomalous windows yield higher attention ranks, which are captured by the ALoRa score, providing clear indicator of anomalies.
5.1 ATTENTION LOW-RANK TRANSFORMER FOR ANOMALY DETECTION (ALORA-DET)
ALoRa-Det consists of the ALoRa-T architecture together with its detection scoring method. We begin by describing the ALoRa-T architecture. The model begins with lightweight MTS em- bedding module, followed by multi-head low-rank self-attention layers (ALoRa layers) with skip connections. As stated in Proposition (see the proof in Section G) feedforward layers do not alter the structure of the latent space and are therefore omitted to avoid unnecessary complexity. The reconstruction step is implemented through linear projection layer.
5

---

Published as a conference paper at ICLR 2026
LightMTS--Ebee As shown in Section embedding with ID convolutions is equivalent to ap- plying VMA filtering to MTS. However, using fully dense filters that mix all input series across embedding dimensions is both computationally expensive and less interpretable. In practice, not all time series are correlated with one another, and ignoring this fact can obscure the underlying dynam- ics in subsequent layers while further reducing interpretability. To address this, each convolutional kernel is restricted to aggregate information from exactly two input series. Specifically, for each
output time series k the (sparse) kernel contains only two con-zero weights (8) corresponding to a particular pair of input series. To avoid including pairs without strong dependence, only the top- K pairs are retained, ranked by Spearman correlation on the training data (commonly K 512). When the total number of possible pairs does not exceed K, all (5) pairs are included. This design improves efficiency, promotes sparsity, and enhances interpretability while preserving performance. An ablation study on the parameter efficiency and performance of the proposed embedding module, compared with the standard Transformer embedding, is provided in Appendix [.1
Low-Rank Regularization in Self-Attention: Based on Eq. since the SA-matrices are the only input-dependent learnable components, their spectral properties can serve as informative signals for anomaly detection. In particular, the rank of matrix, defined as the number of non-zero singular values, provides a useful indicator of abnormal behavior. Empirically, we observe that the rank of SA-matrices increases in the presence of anomalies, as illustrated in Fig. 3 (right). Motivated by this observation, we propose ALoRa layers. which extend the standard self-attention mechanism by explicitly promoting low-rank structure in the attention matrices. This is achieved through the ALoRa loss, a regularization term applied to each attention matrix 5 defined as the truncated Geman nuclear norm (Geman & Yang. 1995), which enforces singular values close to zero
CALORA (S())
Σ i=++1 (o, +1)
where 100 2..... ( 0 are the singular values of (t)), The parameter specifies the number of
leading singular values that are preserved without penalty. Since S row-stochastic (Eq. 2) with fixed largest singular value 0 = 1, we set r = 1, since penalizing it is unnecessary. The ALoRa layer extended to the multi-head attention (MHA) setting by defining ( 1/4 Σ s where
% denotes the self-attention matrix of head h 1...., H, and H is the number of heads.
Training: The objective function, LToual. used to train the model includes two key components:
LTtal
Y
Areg (ALoR
(8)
The first term penalizes the Frobenius norm of the reconstruction error, while the second term en- courages low-rank structure by summing the ALoRa losses of the SA-matrices 5()) across all lay- ers. The regularization strength is controlled by the parameter Areg. The effectiveness of the proposed low-rank regularization is illustrated in Fig. 3 which compares the rank of self-attention matrices with and without regularization. In both cases, rank differences between normal and anomalous inputs are evident but become substantially more pronounced with regularization. Consequently, anomalous patterns are more casily distinguished, providing
abnormality signal.
Model inference - Anomaly detection: As in training, the MTS is processed in overlapping windows to ensure fair reconstruction across all time steps, enabling unbiased anomaly detection Toscano & Recchioni] 2022). The detection anomaly score (AS) for the time steps t is defined as:
AS(y.) = 11y/ Dellz (ALLRa-))) E I,
T
ALoRa-T(ye: ((L)) = Σ 1
(a (1 >b1} E R,
(10)
where {a 1 (1)) decoding of the MTS at time step t. The indicator function 1 returns I if the condition is true and 0 otherwise. The threshold h1 is introduced because, while the low-rank regularization encour- ages many singular values to be close zero, they are often not exactly zero. Moreover, when the
are the singular values of the SA-matrix S(..) used during the encoding-

---

Published as a conference paper at ICLR 2026
SA matrix without anomalies SA matrix with ancesalliss
matrix without anomalies
mantx with anomalies
Kank
Layer (a) Without regularization
Layer (b) With regularization
Figure 3: Layer-wise rank of SA-matrices across transformer layers, without (left) and with (right) the low-rank regularization term, on the SMD dataset. The discrepancy in rank between normal and anomalous inputs becomes more pronounced with regularization, improving anomaly sensitivity.
anomaly score exceeds threshold, AS(()) h22 an anomaly alarm is triggered. Details about h and h2 selection are provided Appendix A The ALoRa-T score captures the temporal character- istics of anomalies, providing earlier and more reliable indications than other scoring functions, as shown in Figure (see also Figs. 12 and Appendix
5.2 ALORA-LOC FOR MTS ANOMALY LOCALIZATION AND INTERPRETATIONS
Effective anomaly localization in MTS requires a deep understanding of the model's learning pro- cess. Practitioners often ask: How does the model make decisions, and which time series influence each output? However, due to a limited understanding of the learning dynamics, such questions remain largely unanswered in the context of deep learning. In contrast, linear regression remains popular for its simplicity and interpretability. Each reconstructed value follows f; E, Gzzz + bi. where Cty quantifies the influence of input 2, output f
The proposed localization method (ALoRa-Loc) first addresses this gap by deriving weights that capture how each input time series contributes to both the learned latent representation and the construction each output time series. This is particularly important, since during model inference quantify how strongly the model relies tive contribution with magnitude Cij/ >, Ca. E {1,..., d]. For the anomaly localization task, such interpretability of the model's decision process is even more critical. As demonstrated in Ap- pendix E anomalies in one series can propagate to others during the modeling process, making it essential capture these propagation effects. The contribution weights Eij and Ctyy provide exactly this capability: Es traces the influence of input series on the latent features, while Cij extends this influence the reconstructed outputs. By leveraging these weights, ALoRa-Loc enables tracing anomalies backward from the reconstructions to the latent space and ultimately to the originating input series, thereby supporting accurate anomaly localization. The derivation the contribution weights is based on the theoretical analysis in Section 4 with analytical details in Appendix G.I
given time series i, measured by its rela-
Contribution of each input time series to the latent space: From Eq. and Eq. (5 the latent representation is governed by the product of two matrices. The left product captures the covariance contributions, whereas the right product corresponds to shared weights across all time series and therefore does not affect the variability of individual contributions (see Appendix G. for details).
Accounting for the embedding module, the final contribution weights from the input space to the latent space given by:
Th (W(V,) + 1), with skip connections (Eq. B
Eij
(k)
bujz
k=1
Th W(V.))
without skip connections (Eq.
(11)
Here, bij denotes the (i. (th entry of B Rddww Xdmats and wa.l are the embedding filter weights for input series lag I. In the absence of embedding, the weights reduce to E B.
(k)

---

Published as a conference paper at ICLR 2026
Contribution of each input time series to the reconstruction space: From Eq. (6) it follows that the overall contribution of the i-th input time series to the j-th reconstructed time series given by:
destd
C
Wkj
Exk
(12)
Based on these insights, we define the localization anomaly score (LAS) for each time series at time as:
d
LAS = Ciillvb 0. /
(13)
2=1
The intuition behind this score is that each term (Cis)l1) 13) represents the magnitude of the anomaly in the i-th time series that has propagated to the reconstruction of the j-th one. Summing over all j captures the total influence of the anomaly in time series i across the system. In practice, it is often more effective sum only over the top-k dimensions with the largest Ciy. focusing the most influential components. We refer to this variant as ALoRa-Loc (top-k).
6 EXPERIMENTS
6.1 DATASETS, BASELINES AND EVALUATION METRICS
Datasets & baselines: We evaluated AoRRa on six widely used datasets from diverse domains: SWaT (Mathur & Tippenhauer (016)) and HAI (Shin al. [022] from industrial control, SMD (Su ett al. 2019) and PSM (Abdulaal et al. 2021) from IT monitoring, and MSL (Hundman et al. 2018 from spacecraft telemetry. For localization evaluation, we use SMD, SWaT, and the MSDS dataset Nedelkoski et al. (020). We compare ALoRa against wide range of baselines, includ- ing classical methods (PCA (Shyu et al. 2003), KNN (Ramaswamy et al., 2000). IForest (Liu et al. 2008), LOF (Breunig et 20.0). OC-SVM (Schölkopf et al. (2001)), notable deep-learning models (Omni-Anomaly (Su al., 2019), Interfusion (L.i et al.) 2021). DAEMON (Chen et al. 2021). ). contrastive learning-basee methods (Yang et al., 2023). and recent SOTA approaches such as Anomaly Transformer (A.T) (Xu et al.) 2022). MEMTO (Song et al. 2023). NPSR (Lai et al. 2023 and SARAD (Dai et
2023 D&R R (Wang al.
also compared 2025 All results are from our own runs using official or public code with
2024). For localization
with AERCA (Han et al.
recommended settings. See Appendix B for details.
Recent studies on MTS anomaly detection evaluation have shown that range- based metrics are the most appropriate, they address the limitations of point-adjusted and point- wise evaluation methods (Liu & Paparrizos 2024). Moreover, since MTS anomaly detection in- volves highly imbalanced datasets, Fi-score-basse based metrics are considered the most reliable. Ac- cordingly, our evaluation focuses on the affiliation-based Fi-score, precision, and recall (Huet et al.) 2022]. For fair comparison, we report the best Fi-scores and the corresponding precision and re- call, avoiding method-specine thresholds. Additional results using the range- based Fi Fi-score (RF) (Hwang et al. 2019). VUS-AUC (VA), and VUS-PR (VPR) (Paparrizos et al.) 2022) are provided in Appendix Additional details on the selection of evaluation metrics are provided in Appendix B.3
Detection metrics:
Localization metrics: evaluaue localization performance using standard metrics such as Hit Rate (Su et al. 2019) and Normalized Discounted Cumulative Gain (NDCG) (Järvelin & Kekäläinen 2002). Additionally, we use the Interpretation Score (IPS), which measures how accurately anoma- lies are localized at the segment level. Further details are provided in Appendix B.3
6.2 RESULTS
All experiments were conducted over five runs, and report the average performance values. Due to space restrictions, the standard deviations are presented in Appendix
Detection Results: As shown in Table ALoRa-Det outperforms all baselines on four out five datasets and ranks second SWaT, where it still significantly outperforms most other methods.
8

---

Published as a conference paper at ICLR 2026
Compared to the second-best best performing models, ALoRa-Det achieves absolute improvements in affiliation-basee based Fi-score of 11.5% on SMD, 7.8% on PSM. 5.9% on MSL, and 8.9% on the HAI dataset. These results highlight the effectiveness and generalizability of detection approach. Notably, our method is not only highly accurate, but its A score also provides a highly infor- mative anomaly signal (see Figure and Appendix [F) that captures the characteristics of anomalies and enables much earlier detection than competing methods in most cases.
Table I: Detection performance. P. R and Fi denote Precision, Recall, and Fscoe respectively. The best Fi -corres are highlighted in bold, and the second-bes are underlined.
SMD
PSM
MSL
SWaT
HAI
Method
F1
P
R
P1
P
R
F1
R
F4
P
KNN
0.70
0.34
0.46
0.53
0.98
0.68
0.50
0.25
0.33
0.45
0.41
0.43
0.48
0.36
0.41
0.54
0.92
0.38
0.55
0.32
0.43
0.46
LOF
0.56 0.35
0.43
0.60
0.41
0.49
0.52
0.30
0.38
0.50
0.38
0.43
0.47
0.34
0.39
OC-SVM
0.44 0.28
0.34
0.88
0.47
0.62
0.53
0.29
0.38
0.46
0.36
0.40
0.42
0,30
0.35
Forest
1.00
0.09 0.17
1.00
0.03
0.06
0.62
0.08
0.14
0.50
0.11
0.18
0.68
0.13
Omni OmoiAnomaly
0.68
0.66 0.67
0.70
0.64
0.67
0.5
0.93
0.64
0.49
0.52
0.50
0.51
0.39 0.44
InterFusion
0.66 0.67
0.69
0.65
0.51
0.92
0.40
0.43
0.64
A.T:
0.58 0.88 0.70
0.55
0.83
0.66
0.51
0.96
0.67
0.57
0.37
0.45
0.61
0.52 0.56
DAEMON
0.70
0.72
0.68
0.52
0.59
DCdetector
0.54
0.94 0.69
0.53
0.82
0.67
0.51
0.9 0.65
0.58
0.76 0.66
0.72
0.86 0.78
MEMTO
0.78 0.86 0.79
0.63
0.75
0.68
0.53
0.95
0.67
0.60
0.61 0.60
0.60
0.66 0.64
NPSR
0.77 0.98 0.87
0.67
0.89
0.76
0.52
0.88
0.68
0.98
0.16
0.28
0.98
0.65 0.79
D R
0.77 0.99 0.87
0.63
0.96
0.76
0.65
0.63
0.64
0.65
0.77
0.71
0.74
0.87 0.79
SARAD
0.88 0.67 0.78
0.73
0.505
0.56
0.56
0.83
0.67
0.54
0.33
0.41
0.51
0.70
0.66
ALoRa-Det
[0.97 0.98 0.97
0.82
0.83
0.82
0.57
0.98
0.72
0.68
0.67
0.68
0.98
0.76 0.86
ALoRa (ALoRs-T)
[AssDic)
MEMTO (LSD)
SARAD (SAR)
5 F 5
Arorad xare
SMD
15.44
2334
25%
23.54
AMM
21.96
2334
Aroordy a3ru
0.5
PSM
06
2.2
47%
1.3
514
175
120
796
50c
511
470
402
53k
6ec
60c
Iene
Anomalles ground truth
Anomaly novel component each method]
Figure 4: Anomaly scores for the SMD and PSM datasets. The red segment indicates the ground truth of anomalies. MEMTO and Anomaly Transformer behave close to random guessing, making detection unreliable. While SARAD provides more informative scores, it does not fully capture anomaly temporal patterns, often resulting in increased waiting time (WT) until detection. In con- trast, ALoRa-T score captures these patterns effectively, enabling faster and more precise anomaly detection. Appendix F presents additional visualizations, including more datasets and methods.
To evaluate the practical efficiency of Aeeta-Det we compare its model size (number of learnable parameters), training time, and inference time per sample with other SOTA methods. Due to page limitations, the detailed results are provided in Appendix C As shown in Table o the proposed method is computationally efficient and suitable for real-world applications.
Computational Analysis:
Localization Results: We use ALoRa-Loc (top-2) for SMD and the standard version for the other two datasets. Table 2 shows that ALoRa-Loc consistently outperforms the compared methods. Moreover, ALoRa-Loc provides meaningful interpretations of the learning process, which are es- sential for practitioners, discussed in Section 5.2

---

Published as a conference paper at ICLR 2026
Table Localization performance. We report HR@P% NDCG@P% and range-based IPS @ P% for P (100, 150). The best scores are shown in bold, with the second-best underlined. The Interfusion method applies only segment- based localization, so only the IPS metric reported.
SMD
MSDS
SWaT
Method
100
150
150
100
150
100
150
100
150
100
100
150
100
150
100
150
MEMTO
0.32
0.48
0.26
0.36
0.19
0.28
0.14
0.33
0.10
0.22
0.02
0.04
0.01
0.01
0.01
0.02
0.05
0.05
OMNE
0.29
0.46
0.21
0.34
0.17
0.26
0.13
0.35
0.09
0.21
0.01
0.03
0.01
0.01
0.01
0.01
0.01
0.01
DAEMON
0.26
0.39
0.31
0.90
0.21
0.26
0:23
0.36
0.27
0.34
0.02
0.02
0.02
0.02
0.03
0.06
0.07
AERCA
0.21
0.25
0.36
0.45
0.13
0.17
0.33
0.56
0.31
0.40
0.08
0.12
0.013
0.014
0.03
0.04
0.028
0.029
ALoRa-Loc
0.56
0.76
0.60
0.70
0.60
0.11
0.30
0.57
0.32
0.45
0.03
0.05
0,042
0
0.041
0,056
0.16
01.20
7 ABLATION STUDIES
To verify the effectiveness of the individual components of the proposed method, we conduct a se- ries of ablation studies. First, we evaluate the contribution of the proposed ALoRA loss, Eq. (7).
and the corresponding ALoRA-T score, Eq. (TO). The results of this analysis, presented in Table
demonstrate the effectiveness of both the ALoRA loss and the ALoRA-T score. Additional abla-
tion studies exploring alternative implementations of the ALoRA loss are reported in Table 10 of Appendix D33
Table 3: Ablation Study of ALoRA-Loss and ALoRA-Score Effectiveness: We report the Fi score for each case.
Study Focus
ALoRA-Loss
ALoRA-Score
SMD
PSM
MSI.
SWaT
HAI
(Yes)
V (Yes)
0.97
0.82
0.72
0.68
0.86
Effectiveness of ALoRA-Loss
x (No)
V (Yes)
0.95
0.74
0.69
0.61
0.82
Effectiveness ALoRA-Score
(Yes)
X (No)
0.944
0.69
0.69
0.55
0.67
Next, for the Lightweight-MTTS embedding module, we evaluate how specific design choices affect both performance and computational efficiency. In particular, we compare model's performance when all time series pairs are used versus when only the Top-K pairs selected. The results presented in Table 4 illustrate that selecting only the top-K pairs is more effective in terms of both performance and computational efficiency. Additional ablation studies for the Lightweight-MTTS embedding module are provided in Appendix(Da. [DII] Finally, Appendix presentt an ablation study supporting decision to omit ferwadd layers. This choice is justified theoretically, since such layers do not alter the latent representation (Proposition) D. and is further confirmed experimentally, shown in Table 9
Table 4: Ablation Study LightMTS-Embed Design: Cost Computation Cost (Model Size in Millions (M) / Training Time (s) / Inference Time per Sample (ms)). Det Anomaly detection performance measured using F, score.
SMD
SWaT
HAI
Study Foeus
Top-K
Correlation
Det.
ALoRa-T
(Top-K)
Spearman
3.2M/ 0.13/5
0.97
3.2M/0.12/4
0.68
3.2M /0.13/36
0.86
(A) All Pairs Instead Top-K
x (All Pains)
Spearman
5.9M/ D. 90
0.96
19.57025762
0.68
108MX
0.85
Instead Spearman
(Top-Ki
3.2M/0.12/45
3.2M/0.13/36
CONCLUSION
We study Transformer encoders on MTS and contribute theoretical insights into their learning behav- ior. Based on this, we propose ALoRa-Ded for anomaly detection, and ALoRa-Loc for localization. Our methods achieve SOTA performance both tasks. Future work includes extending theoret- ical analysis from single-head to multi-hea head attention, and adapting our methods to settings where concept drift occurs alongside anomalous events.
10

---

Published as a conference paper at ICLR 2026
9 REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, the source code is available at: along with README file that outlines the steps required to reproduce the experiments and results presented in this work. Appendix A provides details on parameter selection, while Appendix [ describes the datasets used and the baseline methods. Datasets can be obtained from the sources listed in Appendix [. For each compared method, we provide source code links (Appendix B.2 and follow the recommended settings from the original studies.
ACKNOWLEDGEMENT
This work was supported by the European Union's Horizon Europe research and innovation pro- gramme under grant agreement No 101073307 (MSCA-DN LEMUR), the European Research Council (ERC) under grant agreement No 951424 (ERC-SyG Water-Futures), and the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy.
REFERENCES
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pp. 2485-2494. ACM, 2021.
URL
3467174
Markus M. Breunig, Hans-Peter Kriegel, Raymond Ng. and Jörg Sander. Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD) International Conference on Management of Data, SIGMOD '00, pp. 93-104, New York, NY, USA, 2000. Association for Computing Machinery. ISBN 1581132174. doi: 10.1145/342009.33533 335388
URL https://
doi. was..org/10.o
Peter J. Brockwell and Richard A. Davis. Time Seriex: Theory and Methods. Springer Series in Statistics. Springer, 2nd edition, 2002. ISBN 978-0387955411.
Xuanhao Chen, Liwei Deng, Feiteng Huang, Chengwei Zhang, Zongquan Zhang, Yan Zhao, and Kai Zheng. Daemon: Unsupervised anomaly detection and interpretation for multivariate time series. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pp. 2225-22222 22300,2,, 2021. 1.0028...
doi: 10. 1..110//CDE111130
Xuanhao Chen, Liwei Deng, Yan Zhao, and Kai Zheng. Adversarial autoencoder for unsupervised time series anomaly detection and interpretation. In Proceedings of the Sixteenth ACM Interna- tional Conference on Web Search and Data Mining, WSDM '23, pp. 267-275. Association for Computing Machinery, 2023. ISBN 9781450394079.
Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, David Benjamin Belanger, Lucy J Colwell, and Adrian Weller. Rethinking attention with per- formers. In International Conference on Learning Representations, 2021. URL https:
7/ opopeeeeview net http://www...e.ooooroooooaaaaaaa
N. Cressie and C.K. Wikle. Statistics for Spatio-Tempora Data. Wiley, 2011. 9780471692744.
ISBN
Zhihao Dai. Ligang He, Shuang-Hus Hua Yang. and Matthew Lecke. Sarad: Spatial association- aware anomaly detection and diagnosis for multivariate time series. In Advances in Neu- ral Information Processing Systems, volume 37, pp. 48371-48410. Curran Associates, Inc., new.iiseisss.... htpp://gg..nerrs.......prerrrr http://serr.llle/pppee///// Paper Senper--ooffeeenee..o pddf
2024. URL
File/
Demetrios G. Eliades. Kleanthis Malialis, Stelios Vrachimis, and Marios M. Polycarpou. Smart water networks as cyber- phy.sical-oecio--nnioomeee systems. IEEE Transactions on Industrial Systems, 3:55-105, 2025. doi: 10.1109/TICPSS.222..555

---

Published as a conference paper at ICLR 2026
D. Geman and Chengda Yang. Nonlinear image recovery with half-quadratic regularization. 1EEE Transactions on Image Processing, 4(7):932-99)) 1995. doi: 10.1109/83.392355.
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. The emer-
gence of clusters in self-attention dynamics. In Advances in Newral Information Pro- cessing Systems, volume 36. pp. 57026-57037. Curran Associates, Inc., 2023. URL https::/prrceeainggg.eeuee ctpppprrrpppppppppppppppp..ep..p pttppe/rrrriieeeppee///// file/
Xiao Han, Saima Absar, Lu Zhang, and Shuhan Yuan. Root cause analysis of anomalies in mul- tivariate time series through granger causal discovery. In The Thirleenth International Confer- ence on Learning Representations, 2025. URL
net/ forum?id
k38Th3x4d9
Alexis Huet, Jose Manuel Navarro, and Dario Rossi. Local evaluation of time series anomaly detec- tion algorithms. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 635-645, 2022.
doi: 10.1 (0.1145//53346788333333
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. Detecting spacecraft anomalies using Istms and nonparametric dynamic thresholding. In Pro- ceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 387 395. ACM, 2018. doi: 10.1144532118819.22199999 URL http:: acm.
org prg/doi/ 10. 1111111/321888 32 9845
Won-Seok Hwang, Jeong-Han Yun, Jonguk Kim, and Hyoung Chun Kim. Time-series aware pre- cision and recall for anomaly detection: Considering variety of detection result and addressing ambiguous labeling. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM '19, pp. -2444, New York, NY, USA, 2019. Associa- tion for Computing Machinery. ISBN 47814993 1145 335 784. 3358118
doi: 10.1145//357:844335881 URL
battps://doi.ozz
1145/3335511
Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst. 20(4):422 446, October 2002. ISSN
doi: 10.1145/5224155822188
URL https::/ttt..oooo
1145/ 528445558215.... 582418
Siwon Kim, Kukjin Choi, Hyun-Soo Choi, Byunghan Lee, and Sungroh Yoon. Towards a rigorous evaluation of time-series anomaly detection. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7):7194-720) 2022. doi: 10. 11.669//aai..36....2022 20680.
Elias Kyriakides and Marios Polycarpou. Intelligent monitoring, control, and security of critical infrastructure systems, volume 565. Springer, 2015.
Chih-Yu Yu (Andrew) Lai. Fan-Keng Sun, Zhenggi Gao, Jeffrey Lang, and Du- ane Boning. Nominality score conditioned time series anomaly detection by point/sequential reconstruction.
In Advances in Neural Information Process-
ing Systems, volume 36, pp- 76637-76655. Curran Associates, Inc., 2023. URL htt.://eeeeec.a...eeeoo....ooeaa fice00ee09711157/cc3330003311113
Kwei-Herng Lai, Daochen Zha, Junjie Xu, Yue Zhao, Guanchu Wang, and Xia Hu. Revisiting time series outlier detection: Definitions and benchmarks. In Proceedings of the Neural Information Processing Systems Track Datasets and Benchmarks, volume 1, 2021. URL https:
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. Mad-gan: Mul- tivariate anomaly detection for time series data with generative adversarial networks. In Arti- ficial Neural Networks and Machine Learning ICANN 2019: Text and Time Series, volume 11731 of Lecture Notes in Computer Science, pp. 716-716. Springer, 2019. URL https: 7/1ink borkkspring www.irringeee ctm/ cww/chapter////00// 00w.0000000000---10011 30490-
12

---

Published as a conference paper at ICLR 2026
Zhihan Li, Youjian Zhao, Jiaqi Han, Ya Su, Rui Jiao, Xidao Wen, and Dan Pei. Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Tem- poral Embedding. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Dis- covery & Data Mining, pp. 3220-3230. ACM, August 2021. ISBN 978-1-45033-832255. doi:
10. N0.1145/344544..46677 3467777
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In Proceedings of the 8th IEEE International Conference on Data Mining (ICDM), pp. 413-422, 2008. doi: 10. 109//DM. 2008.17.
Qinghua Liu and John Paparrizos. The elephant in the room: Towards a reliable time-series anomaly detection benchmark. In The Thirry-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL ltttss://openrewieww http:/iwwwwweeeeeeeeeeeeeeeeeeee wornee/oorumiie
R6kJtWsTGy
Aditya P. Mathur and Nils Ole Tippenhauer. SWaT: A water treatment testbed for research and training on ICS security. In Proceedings of the 2016 International Workshop on SyyeerPhhssie Systems for Smart Water Networks (CySWater), pp. 31-36. IEEE, 2016. doi: 10.1 109/CySWater.. 2016.7469060.
Sasho Nedelkoski, Jasmin Bogatinovski, Ajay Kumar Mandapati, Soeren Becker, Jorge Cardoso, and Odej Kao. Multi-source distributed system data for pi-powered analytics. pp. 161-176, Berlin, Heidelberg, 2020. Springer- Springer-Verlag, ISBN 978- 0-77----330-4778--- doi:
13. TRL https: http://tiiooo//////////ee 101.010..1077770 3- 030 4476--476---111 13
Yuqi Nie, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Confer- ence on Learning Representations, 2023. URL https::/ /tttD:://openreveew.. net/ formmidd JbdcOvIOcol.
John Paparrizos, Paul Boniol, Themis Palpanas, Ruey S. Tsay, Aaron Elmore, and Michael J. Franklin. Volume under the surface: a new accuracy evaluation measure for time-series anomaly detection. Proc. VLDB Endow. (((1)))777427877 July 2022. ISSN 2150- doi: 10.14778/551799.355138 Uttpttss:://ool org/10gg///o 14778 14771/335577773.33 3551830
Dachyung Park, Yuuna Hoshi, and Charles C. Kemp. A multimodal anomaly detector for robot- assisted feeding using an lstm-based variational autoencoder. IEEE Robotics and Automation Letters, 3(3):1544-1555 2018. doi: 10.1109/LRA.200 2..20.8888847..
Sridhar Ramaswamy, Rajeev Rastogi, and Kyuseok Shim. Efficient algorithms for mining outliers from large data sets. In Proceedings of the 2000 ACM SIGMOD International Conference on Manugement of Data, SIGMOD '00, pp. 427-438, New York, NY, USA, 2000. Association for Computing Machinery. ISBN 132174.74
Bernhard Schölkopf, John C. Platt, John C. Shawe-Taylor, Alex J. Smola, and Robert C. Williamson. Estimating the support of a high-dii sigh-dimensional distribution. Neural Comput., 13(7):1443-1471 July 2001. ISSN 0899-7667. doi: 10.1 10.
URL /tLhhtpss: http::::///o..orrg/
Lifeng Shen, Zhuocong Li, and James Kwok. Timeseries anomaly detection using temporal hierarchical one-class network. In Advances in Neural Information Processing Systems, vol-
ume 33, pp. 13016-13026. 2020. URL https: htsttp:::/ppoocednsss..eeee wttpppppcccpppp..peppp
Paper...d
Hyeok-Ki Shin, Woomyo Lee, Jen Yun, and Byung-Gil Min. Two ics security datasets and anomaly detection contest the hil-based augmented ies testbed. In Proceedings of the 14th Cyber Security Experimentation and Test Workshop, CSET '21, pp. 36-40, New York, NY, USA, doi: 10.1145/3474718.
2021. Association for Computing Machinery. ISBN 9781450390651. 3474719. URL
000 1el..1114344411 0tt.:/..000.....1111.1.11.1.1. 0.0.744718..77711 3474719
Mei-Ling Shyu, Shu- Shu-Ching Chen, Kanoksri Sarinnapakom,, and LiWu Chang. A novel anomaly detection scheme based principal component classifier. Technical report. Department of Elec- trical and Computer Engineering, University of Miami, Coral Gables, FL, USA, 2003. Technical Report.
13

---

Published as a conference paper at ICLR 2026
Junho Song, Keonwoo Kim, Jeonglyul Oh, and Sungzoon Cho. Memto: Kemory-guidder transformer for multivariate time series anomaly detection. In Advances in Neural In- formation Processing Systems, volume 36, pp. 57947-57963. Curran Associates, Inc.,
http:///pppppprrreeeee///////2/ 2023. URL. httpps://procett ngs. www.snnerripsscoom httpr///erra........o.o..oeeeree
fil 0al
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Pp- 1828-28311 ACM, July 2019. ISBN 978-1-4503-6201 doi: 10.1145/32995500333366
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, pp- 13299.328. JMr.. 2017.
Siyi Tang, Jared Dunnmon, Khaled Kamal Saab, Xuan Zhang. Qianying Huang, Florian Dubost, Daniel Rubin, and Christopher Lee-Messer. Self-supervised graph neural networks for improved electroencephaloggaphie seizure analysis. In International Conference on Learning Representa- tions, 2022. URL
nwt feet/Forum?id==xxxxx
Giacomo Toscano and Maria Cristina Recchioni. Bias-optimal vol-of-vol estimation: the role of window overlapping. Decisions in Economics and Finance, 45(1):137-185, 2022. doi: 10. 1007//110203-001-000449-944
com/ article/10.1000
S 610203-021-003455
Ruey S Tsay, Daniel Peña, and Alan E Pankratz. Outliers in multivariate time series. Biometrika, 87(4):789-80) 2000.
789.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems, volume 30. Curran Associates, Inc., 2017.
Chengsen Wang, Zirui Zhuang, Qi Qi, Jingyu Wang, Xingyu Wang, Haifeng Sun, and Jianxin Liao. Drift doesn't matter: Dynamic decomposition with diffusion reconstruc- tion for unstable multivariate time series anomaly detection. In Advances in Neural Information Processing Systems, volume 36, pp. 10758-10774. Curran Associates, Inc., 2023. URL
http:pppooooeddd........... neu http:///eeeeeeeeee....o///o////a File/ 3ll12225aeeB222aaaocaaaaaa222223
Cow.rer-oooooeeeeeecceoeae pdff
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series
anomaly detection with association discrepancy. In International Conference on Learning Repre- sentations (ICLR), 2022. URL https::tts:/:/eeereee
net dert/
Yiyuan Yang, Chaoli Zhang, Tian Zhou, Qingsong Wen, and Liang Sun. Dedetector: Dual attention
contrastive representation learning for time series anomaly detection. In Proceedings of the 29th
ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '23, pp. 3033-3045, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701030. doi:
10.1145/55000053559995.
hellhttts://ad
wrw/10...//.... 11...11155511 1111/1155033..... 3599295
George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-based framework for multivariate time series representation learning. Proceed- ings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD '21, pp. 2114-2124. Association for Computing Machinery, 2021. ISBN 9781430383325.
Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, and Nitesh V. Chawla A deep neural network for un- supervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applica- tions of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, AAAI' 19/AAA 19/EAAI 19. AAAI Press, 2019. ISBN 978-1-57735- 809-1.
14

---

Published as a conference paper at ICLR 2026
APPENDIX TO: Low RANK TRANSFORMER FOR MULTIVARIATE TIME SERIES ANOMALY DETECTION AND LOCALIZATION
A TRAINING DETAILS
ALoRa-T architecture design and hyperparameter selection: The ALoRa-T model consists of three self-attention layers with skip connections. Each layer uses H = attention heads for all datasets. Training is performed using the ADAM optimizer with learning rate of 10-1, and early stopping is applied to prevent overfitting. A fixed regularization parameter of Aveg 10 is used for all datasets.
Window Size T: The window size T is an important hyperparameter that determines how much past information the model can utilize at each time step. To assess its impact, we conduct an ablation study evaluating ALoRa-Det on the validation set with different values of T, as shown in Figure
-scorre across different window sizes (T)
0.95
SWaT
HAI
0.90
F1-score
0.80
0.75
0.65
0.60
20
40
Window size (T)
Figure 5: Validation Fl-scores across different window sizes for each dataset. The average over runs is reported along with the standard deviation.
The figure shows that ALoRa-Det's performance is not sensitive to the choice of window size. It is worth noting that smaller 7 values lead to faster training and inference, while larger values result in select the following window sizes for
longer processing times. Based on these considerations, each dataset: SMD (20), PSM (100), MSL (20), SWaT (20), and HAT (20). For the MSDS dataset used for localization, the window size is set 100.
Selection of hi, used in eq. 110 We present the process followed to select the threshold param- eter h1 As discussed Section 5, ALoRa-T incorporates a low-rank regularization term, which encourages the self-attention matrices S(L) to have a reduced-rank structure. The selection pro- cedure is automated and data-driven, as it must adapt to each dataset. With this procedure, hi is automatically determined for any new dataset based on the spectral properties of self-attention matrices observed under normal operating conditions. Specifically, the fourth and fifth largest eigenvalues of ((L) across the training sequence and track their trajecto- rics. The threshold h; then automatically chosen as the maximum value observed between these two eigenvalue trajectories. This ensures that only the dominant eigenvalues, typically correspond- ing to a rank of 3-4, are preserved. The choice to preserve the 3-4 largest eigenvalues informed by the findings of Geshkovski et al. (2023). Figure 6 iillustrates this selection process. Based this analysis, we set hy = 10 for SMD, 10 3 for PSM, 3 x 10 2 for SWaT and MSL, and 2 x 10-1 for HAI.
analyze the distribution of
15

---

Published as a conference paper at ICLR 2026
SMD
Largent Singular Value
Largent Singstar
2200
6003
4000
2003
22:2
385
Singular value
Singular value
PSM
Largest Seepller
Langest Singube
20300
0
1
13000
20088
50:00
5000
0
380
Singular
Singular value
MSL
4th Langest Singular vilue
Singular
3000
33:99
5000
10-1
Singular value
Singular value
SWAT
Largest Singular Value
Singurar Vakue
2003
3750
2000
3000
10'4
3015
32"5
10*
50-4
10-1
Singular
Singular
HAI
4th Langest Singular Voluee
5th Largest Singular Value
VV0
2330
3000
2300
3500
2000
3000
12"44
1924
1244
12140
127
12444
1245
127
Singular value
Singular value
Figure 6: Distribution of the fourth and fifth largest eigenvalues of S(L) computed over normal train- ing data. The red dashed line indicates the selected hı value, chosen to zero out smaller eigenvalues and ensure a typical rank no larger than 4.
16

---

Published as a conference paper at ICLR 2026
Selection of h2: The parameter h2 is the threshold used to convert anomaly scores into binary decision labels. Such a threshold is required for all anomaly detection methods to provide anomaly predictions. To avoid introducing bias from any particular threshold-sclectinnn strategy and to ensure a fair comparison across all methods, we follow a widely adopted evaluation protocol: for each method, including our own, we report the performance obtained using the best achievable thresh- old, i.c., the threshold that yields the highest performance. This approach is standard in the anomaly detection literature (Dai al., 2024 Lai al. 2023). In summary, selecting h2 in this manner guarantees fairness when comparing baselines: it evaluates the quality of the anomaly scores pro- duced by each method, rather than the particular preshhld-selectiee mechanism used convert those scores into labels.
17

---

Published as a conference paper at ICLR 2026
B EXPERIMENTAL SETTING
B. DATASETS
The datasets used in our experiments are publicly available and can be downloaded from the links provided below:
Secure Water Treatment (SWaT) Li al. (2019): This dataset captures data from real-world industrial water treatment system monitored by 51 sensors over 11 days. The first days contain normal operations, while the last 4 include 41 manually injected anomalies.
Download:
https://itrus sutd. edu wtw.:dded...ggiiis.eeeeeeeee 1abs home/
itrust Labst swat/
Server Machine Dataset (SMD) Su et al. 20119: A large-scale dataset with 38 features collected over five weeks from server machines in major Internet company, covering various system performance metrics.
Download:
hhaterr/erreeraachnneDttttt |ttp::///SSSluaa.oo/////raarllrr
Pooled Server Metrics (PSM) Abdulaal et al.
(021): Collected internally at eBay, this
dataset contains 25 dimensions from application server nodes, including 13 weeks of train-
ing data and 8 weeks of test data.
Download: https:
i/data
Mars Science Laboratory HunLmmn al. 2018) Provided by NASA, this dataset includes telemetry data with 55 features recorded from the MSL. rover during space mis- sions.
Download:
HII.-based Augmented ICS Security Dataset (HAT) Shin et al. (2021): Collected from realistic ICS testbed enhanced with a Hardware- hettwwaa----tee-llee Loothe--Lop simulator, this dataset contains 79 dimensions from sensors and actuators under normal conditions and during 38 simulated cyber-attacks.
Download:
Multi-Source Distributed System (MSDS) Nedelkoski et al. 2020 Collected
from cloud-based OpenStack testbed, this dataset contains multi multi-source monitoring Kata-including metrics, logs, and includes root cause labels, making it suitable for anomaly localization task.
normal and fault- snjtt-injected conditions. It
Download: https:: /ttp:://zeeodo.oo wwwp://oo.....eeoo.......... records /
Table summarizes key statistics of the datasets used in our experiments, including the number of features, training and test samples, and the anomaly proportion in the test set.
Table 5: Summary of the five benchmark datasets used in our experiments. 'Dim' denotes the number features. 'Train' and "Test" indicate the number of samples in the training and test sets. "Anomaly rate (%)' shows the percentage of anomalies in the test set.
Dim
Application
Train
Test
Anomaly rate %
SWaT
51
Water
495,000
449,919
0.121
SMD
38
Server
28479
28479
0.156
PSM
25
Server
132481
87841
0,278
MSL
55
Space
58317
73729
0.105
HAI
79
Power
921603
402005
0.223
MSDS
10
A1Ops
29268
29286
0.72
Dataset Preprocessing: For each dataset, first normalize each time series individually before passing the data to the model. This normalization is performed using the mean and standard devi- ation computed from the training set. The same normalization parameters then applied to the
18

---

Published as a conference paper at ICLR 2026
corresponding test set. For the SWaT and HAI datasets, which contain very large number of time steps due to high-fite frequency (per-second) sampling, we downsample the data to one-oinenn me-minute intervals by averaging the values over every 60-secona window.
B.2 BASELINES
We reproduced the results of all baseline methods using their official or publicly available imple- mentations on GitHub. For each baseline, we followed the configuration settings recommended in their respective papers to ensure optimal performance. The baselines and their implementations can be found at the following links:
KNN:
PCA: http://s/ggatubb..oo//////////// LOF: http://tsss//tth......oo////////
OCSVM: 115S5MMtttss///tttusccc//////2/0
IForest: httpp/////gttttoooooooaa////////
Transformer: Anomaly
http:://ithub..co
MEMTO: AUBEH///Eeree///hahaoooo/////eer
NPSR: httppp/s//attab..cco///aaclaaayp
SARAD: https///gtthub..cm////a1aaa///// D3R: http:://ggatub..cm////aagxyyyyyp
B.3
EVALUATION METRICS
Detection:
The evaluation multivariate time series anomaly detection has attracted growing re- search attention, largely due to the challenge of measuring model performance in a manner that aligns with real-world temporal decision-making. Anomalies usually span continuous ranges rather than isolated points. As a result, point-wise evaluation that treats each timestamp independently is inappropriate because it ignores temporal continuity and unfairly penalizes slight delays or near- misses in detection. Traditionally, the point-adjustment method was used to address this limitation, where detecting any point within an anomalous segment suffices to consider the entire segment as an observed anomaly (Xu et al.) 2022 Song et al. 2023 Shen et al. 2020). However, this ap- proach often inflates performance metrics, as even a random anomaly scoring method can achieve performance comparable that of a well-informed method, which effectively captures the temporal characteristics of anomalies (Kim et al. 2022 Huet et al. 2022). This limitation of point-adjusted metries is further illustrated in Figure
Recognizing these limitations, many studies (Liu & Paparrizos 2024 Hwang et al. 2019) have sug- gested range-based evaluation metrics the appropriate choice for multivariate time series anomaly detection. For instance, Hwang et al. 2019) proposed the range-based F1, precision, and recall (RF,, R-P. R-R). An advancement over these are the affiliation- FI, precision, and recall (Huet et al. 2022). which demonstrate superior performance and robustness compared to the afore- mentioned metrics. Unlike traditional metrics, this approach accounts for the temporal alignment between predicted and actual anomalies. Precision is computed as the average directed distance from predicted anomaly events to their nearest ground truth counterparts, while recall measures the average directed distance from ground truth events to their closest predictions. For a more detailed explanation how these metrics are computed, we refer the reader to the original paper (Huet et al.] 2022). Paparrizos et al. (2022) introduced VUS-ROC and VUS-PR as range-aware alternatives to traditional AUC. However, because MTS anomaly detection is highly imbalanced classification problem, Fiiki metrics the most appropriate and reliable for evaluation, while VUS metrics
19

---

Published as a conference paper at ICLR 2026
(a) Informative Anomaly score
Anomaly score
Threshold
(b) Anomaly detection decision
Label
Anomaly score
(c) Random Anomaly score
Threshold
Time
Figure 7: Line (a) shows the anomaly scores from informative anomaly scoring approach, while Line (c) represents a random scoring method. The middle section displays the binary decisions (where anomaly 1, otherwise which ultimately led to the same outcome. The red segments indicate the ground truth abnormal time steps. while the red dot in Line (c) marks the only time step that, by chance, exceeded the anomaly threshold.
made by both methods using the point adjustment technique,
provide additional insights but should used together with F1-like metrics. Given the superior ro- bustness and effectiveness of affilliationbbaee metrics, adopt the affiliation--aai based Fi-score as primary evaluation metric. Results for the range-based F; -core (RFi). along with the VUS-ROC (Vzoc) and VUS-PR (Vrn) metrics, reported in Table 11 Nevertheless, due to the highly im- balanced nature of MTS anomaly localization, Fl-like metrics remain the most appropriate choice, they explicitly capture the trade-off between precision and recall.
Localization: At each time step I, anomaly localization is evaluated by identifying which features (time series) contribute the detected anomaly. The mounnd-truth for localization is provided as binary label vector G, E {0, 1} where G = indicates that the i-th feature is anomalous at time and G, otherwise.
After computing the anomaly score AS(()) for each feature using eq. 13 features are ranked in descending order of their scores to identify those most likely responsible for the anomaly. The top-k features are selected as:
I Farr% Top-k features ranked by AS where [|G,] x P%]
For example, there are anomalous features (i.e., |G 3) and we evaluate P 150, then k = 5. The localization performance is then quantified using the Hit Rate (HR) at P%, which measures the fraction of truly anomalous features that appear in the top-k predictions:
GG 7.ap%
HR
(14)
[G,]
20

---

Published as a conference paper at ICLR 2026
Normalized Discounted Cumulative Gain (NDCG): Let ry {0, 1} indicate whether the j-th ranked feature in I Frap% is truly anomalous (i.e., belongs to G2). The Discounted Cumulative Gain (DCG) and its ideal counterpart (IDCG) are defined as:
Tj logg (j + 1)
DC
j=1
|G:1 2
IDCG, =
loga (j +
j=1
The resulting NDCG is:
DCG
NDCG 58%%
(15)
IDCG,
NDCG ranges from to 1, with higher values indicating better-ranked localization.
Interpretation Score (IPS): evaluate anomaly localization at segment level, we use Interpretation Score (IPS) Li et (202T). For each anomaly segment S, we first compute single score for each feature by taking the maximum anomaly score within the segment:
AS S:
max AS LES;
(16)
We then identify the top-ranked features as the predicted anomalous dimensions Ps., and compare them against the ground-truth anomalous features Gg,, The IPS measures the average proportion of correctly predicted features across all segments:
GGs, Ps.l |Gs.
IPS
(17)
where N is the total number of anomalous segments. Each segment is equally weighted in the final score.
C COMPUTATIONAL ANALYSIS
To evaluate the practical efficiency of ALoRa-Det, compare its model size (number of learnable parameters), total training time (in seconds), and inference time per sample (in milliseconds) against three state-of-the-art baselines: MEMTO, SARAD, and D R. The results across three benchmark datasets are reported in Table
Table 6: Computational efficiency comparison. The table reports the number of learnable parameters (in millions), total training time (s), and inference time per sample (ms).
SMD (d=38)
SWaT (d=51)
HAI (d=79)
Method
Params / Inf/ms / Ts)in(s))
Params Inf/ms / Train(s)
Params / Inf/ms / Train(s)
MEMTO
5.9M / 1.10/ 108
5.9M /
6M/ 6.8 96
SARAD
9.6M / 0.11/ / 147
9.6 0.16 652
9.6M / 0.48 / 126
D3R
52.3M 4.83 9994
52.3 / 6.27 283
52.5 M/ 21.6 /1077
ALoRa-Det
3.2 M/ 0.13 / 75
3.2M 0.12 / 45
3.2M 0.13 / 36
The results in Table 6 indicate that ALoRa-Det almost always outperforms the competing methods across all three metrics. Unlike other approaches, the total number of parameters remains constant
21

---

Published as a conference paper at ICLR 2026
imput dimensionality increases, while both inference time per sample and training time remain consistently low. These advantages become more evident as the dimensionality of the MTS grows, highlighting the scalability of the proposed method to nigh-dimensional settings. Taken together, these findings confirm that ALoRa-Det is not only more efficient than existing baselines but also highly suitable for real-time applications.
D ABLATION STUDIES SUPPLEMENT
D.1 LIGHTWEIGHT MTS EMBEDDING PARAMETER EFFICIENCY
Table 7shows the parameter efficiency of the lightweightemmeeddin embedding used our method, comparing it to standard fully dense filters.
Table Parameter comparison between standard fully dense filters with model dimension,dmotet = 512 and the proposed Lightweight MTS Embedding module.
Input Size d
Standard Weights
Custom Weights
10
15,360
90
20
30,720
380
40
61,440
1,024
50
76,800
1,024
100
153,600
1,024
200
307,200
1,024
In addition, compare the standard and Lightweight-MTTS embedding methods in terms of de- tection performance to demonstrate that the proposed lightweight approach does not compromise effectiveness. On the contrary, it maintains, and in some cases even improves performance. Figure B presents the Fj scores achieved by both embeddings, along with boxplots showing the distribution of reconstruction errors on normal test samples, providing further insight into the reconstruction behavior of each method.
Eniediinn
Bogii La::
Entaeddiy
Roppotor of
Eneededd sightrseebee
Boglst in stps
WighhWW Enbed
Deconitructin error
095 857
0.66
0,86
0.67
SMD
(b) HAT
(c) SWaT
Figure 8: Model performance under the standard and lightweight embeddings for the SMD, HAI, and SWaT datasets. each subplot, the figure on the left shows the F1 score comparison, and the figure on the right presents the boxplot of reconstruction errors on normal test samples.
Why the LightMTS embedding module use pairwise interactions: The proposed LightMTS embedding module was designed to reduce complexity by restricting interactions to pairwise lationships between time series. Increasing the number time series involved in each interaction, from pairs triplets higher-order groups, would lead to combinatorial explosion in the number of possible interactions. In such cases, the complexity would become comparable to that standard fully connected convolutional embeddings.
22

---

Published as a conference paper at ICLR 2026
Top-K Selection for the LightMTS--Ebber Module: The choice to select the 1op-512 pairs fol- lows standard design practice in Transformer architectures, where the embedding dimension is com- monly set to 512, as introduced in the original Transformer model (Vaswani et al. (017). This setting provides a strong balance between representational capacity and computational efficiency, enabling consistently low training and inference times across all datasets (Table 6). To assess the sensitivity of the model to this parameter, an ablation study for different values of K, is presented in Table The results indicate that the model is not highly sensitive this hyperparameter, as performance remains similar for K = 512 and K = 256, with the latter offering slightly lower accuracy but im- proved computational efficiency. However, as the model's computational efficiency is already very strong, as demonstrated in Table 6 K 512 remains a robust default choice. It generalizes well across datasets from diverse domains and provides sufficient representation-lenn -earning capacity even for multivariate time-series datasets with very high dimensionality.
Table 8: Ablation Study the Top-K pairs selection: The table reports the F1 score average over runs with the corresponding standard deviations for different values the parameter K.
K
SMD
PSM
MSL
SWaT
HAT
512
0.97 +: 0.012
0.82 + 0.014
0.72 0.011
0.68 + 0.010
0.86 + 0.015
256
0.965 + 0.014
0.82 = 0.014
0.72 = 0.011
0.67 0.011
0.854 + 0.015
128
0.95 + 0.012
0.805 + 0.015
0.713 4: 0.012
0.66 + 0.012
0.85 1 0.013
64
0.94 0.014
0.78 = 0.015
0.67 = 0.012
0.65 = 0.013
0.84 + 0.012
D.2
ABLATION STUDY FOR REMOVING FEED-FORWARD LAYERS
This ablation study examines how omitting feed-forward layers affects both computational efficiency and anomaly detection and localization performance. As stated in Proposition | feed-forward layers do not alter the latent representation, indicating that their contribution to performance is minimal. The experimental results in coblirc this theoretical insight. Specifically, when feed-forwar layers are included, performance either decreases or remains unchanged, while computational cost increases substantially.
Table 9: Ablation Study for Removing Feed-Forwel Layers: The table reports the number of learnable parameters in millions (Cost), the anomaly detection performance using the Fi- Fiocc (Det.), and the localization performance using the Hit-Rate @ 100 metric (Loc.), reported only for the datasets where localization labels are available.
SMD
PSM
MSL
SWaT
HAI
FFN
Cost
Det.
Loc.
Cost
Det.
Cost
Det.
Cost
Det.
Loc.
Cost
Det.
(Yes)
6.3
0.96
0.56
2.2
0.79
6.3
0.72
6.3
0.67
0.03
6.3
0.854
X (No)
3.2
0.97
0.50
1.1
0.82
3.2
0.72
3.2
0.68
0.042
3.2
0.86
D.3 ALORA-LOSS PLACEMENT
The proposed ALoRa loss (Eq, 7 is applied the average self-attention matrix across heads rather than to each head individually. To justify this choice, we provide an ablation study comparing the two approaches terms of both computational cost and performance.
As shown in Table IO both variants achieve nearly identical detection performance across all datasets. However, applying the ALoRa loss to the average self-attention matrix is slightly more computationally efficient, which is why it was preferred. Importantly, both approaches are capable of achieving our goal of reducing the rank while maintaining the effectiveness of the model. As a final note, even if the head-wise application of the ALoRa loss had been preferred, the proposed method would still remain more computationally efficient than other state-of-the- of- methods.
23

---

Published as a conference paper at ICLR 2026
Table 10: Ablation Study on ALoRA-Loss Placement: Head-Wise Averaged Self-Attention. Reported are the training time (in seconds) required for each case and the corresponding detection performance (FI-score).
SMD
SWaT
HAI
ALoRA-Loss Placement
Training time
F1
Training time
F1
Training time
Head-Wise ALoRA-Loss
155
0.97
0.67
78
0.86
OURS: Averaged Attention ALoRA-Loss
75
0.97
0.45
0.68
0.36
0.86
ANOMALOUS EFFECT ATTRIBUTION VIA EFFECT WEIGHTS
To examine how anomalies in one time series affect others through shared latent representations, we design a controlled simulation using a simple self-attention model. This setup also demonstrates how the effect weights introduced in Section 4 quantify influence in both latent and reconstruction spaces.
We generate bivariate i.i.d. data E Txx2 where:
[Wim + of), for h st< a1(2) ~ N 03) (((
otherwise
We set T 500, f = 200, and t2 = 300. The model consists of a 2-layer, 1-head self-attention mechanism. We fix the value matrices wi()) for layers 1, well the output projection matrix Woun, while the remaining parameters are kept learnable:
w = 12, W
0.2
0.7 0.3
0.11 0.9 Wosr = 0.9
0.8
0.1
This controlled setup allows us to observe how localized anomaly in one input series propagates through the latent space and affects the reconstructed time series. As shown in Fig. 9 the effect weights Ed and C1, reflect how the anomaly in the first input series propagates through the latent space and influences the reconstructed time series, respectively. Together, they provide meaningful and interpretable attributions of anomaly influence across the model's internal and output represen- tations.
24

---

Published as a conference paper at ICLR 2026
x11
x(2)
10-
10-
Time series value
8-
8-
2-
2-
100
200
300
400
500
100
200
300
400
500
(1)
z (2)
12-
E 0.2
E12 0.8
10
10-
Time series value
8-
8-
2-
2-
100
200
300
400
500
100
200
300
400
500
8 {1)
8 (2)
12
12
C11 0.74
C12 = 0.26
Time series value
8-
8-
6-
6-
4-
4-
2-
2-
0
0-
100
200
300
400
500
0
100
200
300
400
500
Time
Time
Figure 9: Averaged results over 100 simulation runs. Top: Input time series with an anomaly
introduced series between time steps 200 and 300. Middle: Latent representations: effect weights F, quantify the contribution of the anomaly in series to each latent dimension. Bottom: Reconstructed time series; effect weights Ci reflect how the anomaly influences the reconstruction of each output series.
F EXPERIMENTS: SUPPLEMENT
To assess the performance of the proposed ALORA-DET method, we provide additional results using three evaluation metries. The selection of these metrics is explained in Appendix [.3] Table [1] presents the performance of the proposed method and all compared methods using the range-based Fl-score (RF1), well as the VUS-ROC (VRoc) and VUS-PR (VeR).
Effectiveness of the ALoRA-T Score Across Anomaly Types: To assess the effectiveness of the ALoRA-T score, and to further provide empirical evidence that the attention rank increases in the presence of anomalies, examine how the rank of the metric-and thus the behavior of
25

---

Published as a conference paper at ICLR 2026
Range-aware evaluation metrics. RF denotes Range-based Fl-score; VROc and VpR denote the Volume Under the Surface ROC and precision, respectively. Best scores are highlighted in bold, and second-bes best are underlined.
Table 11:
SMD
PSM
MSL
SWaT
HAI
Method
R.F
Vioc
Vise
RFF
Vioc:
V/n
RF:
Vioc
Viu
RFFi
Vioc:
Vix
R.F.
Vinc
Vin
KNN
0.15
0.42
0.09
0.31
0.40
0.28
0.11
0.37
0.08
0.10
0.44
0.35
0.12
0.09
0.12
PCA
0.18
0.65
0.11
0.48
0.58
0.42
0.15
0.62
0.20
0.13
0.68
0.51
0.21
0.63
0.13
bolaaionForest
OC-SVM
0.11
0.58
0.08
0.19
0.53
0.37
0.13
0.59
0.18
0.03
0.62
0.52
0.23
0.64
0.14
Onomaly
0.17
InterFusion
0.18
0.36
0.08
0.19
0.34
0.12
0.14
0.53
0.12
0.09
0.40
0.14
0.28
0.75
0.23
A.T.
0.11
0.50
0.10
0.19
0.26
0.37
0.14
0.51
0.12
0.13
0.61
0.22
0.09
0.59
0.21
MEMTO
0.26
0.52
0.29
0.50
0.29
0.13
0.50
0.68
0.39
0.16
0.22
NPSR
0.43
0.89
0.48
0.49
0.63
0.47
0.20
0.53
0.13
0.33
0.83
0.64
0.20
0.75
0.44
D
0.1
0.91
0.57
0.65
0.46
0.18
0.61
0.15
0.09
0.84
0.38
0.1
0.89
0.61
SARAD
0.34
0.80
0.16
0.25
0.62
0.42
0.12
0.53
0.171
0.59
0.33
0.07
0.70
0.21
ALoRa-Det
0.28
the ALoRA-T score changes under different anomaly types. These anomaly types are the ones formalized and defined in the studies LLi et al. 2021) and (Tay et al. 2000).
Sessoral
Trend
Contextua--Supelet
Point outlier
Aromalous
WWW
000001
150
123
300
731
150
181
150
AtohaT
Anomalies ground truth
ALoRa-1 T Score
Figure 10: ALoRa-T scores for different anomaly types, following the definitions and categoriza- tions presented (Lai et al, 202T and (Tsay et al. 2000). The plotted ALoRa-T scores are averaged over 5 runs.
Based on Fig. 10 there is clear evidence that, across a variety of anomaly types, the rank of the self- attention matrices, and consequently the corresponding ALoRA-T score, increases in the presence of anomalies. This consistent behavior across different anomaly categories demonstrates the robustness and generalizability of the ALoRA-T score in detecting anomalous patterns.
26

---

Published as a conference paper at ICLR 2026
Comparison of ALoRa-T Scores with State-of-the-Art Methods: To support and extend the findings in the main paper, this appendix presents additional visualizations to demonstrate the gen- eralizability of the ALoRa-T anomaly scoring method. First, provide additional abnormal seg- ments from the SMD and PSM datasets (Fig. 11p to reinforce our earlier results. Second, we include anomaly scores from the SWaT and HAI datasets (Fig. [2] to illustrate ALoRa-T's effectiveness across multiple domains. Based on Fig. [2 and [I] the ALoRa-T score continues to stand out for its effectiveness capture the temporal characteristies of anomalies, providing timely and highly informative responses to anomalous patterns. The score rises quickly and remains aligned with the duration and structure of the anomalies. In some cases, the ALoRa-T score remains high for short period of time, even after the labeled anomaly segment ends. However, this behavior can self-attention matrices at post-anomaly timesteps still incorporate abnormal information from earlier points. The score gradually decreases after 7 timesteps-wher T' is the window size, once the attention mechanism no longer includes the carlier abnormal data. This demonstrates the effectiveness of the ALoRa-T score to accurately detect anomalies and to provide interpretable signals for both true and false alarms. In contrast, MEMTO and Anomaly Transformer still behave similarly their performance in the main results (see Fig. 4. where their outputs resemble random guessing. This makes their detection signals unreliable. SARAD and NPSR produce more informative patterns. However, they frequently fail to capture the temporal dynamics of anomalies accurately, often resulting in delayed or unsuccessful detection.
be meaningfully interpreted:
SMD
PSM
ALoRa (ALoRa-T)
Anomaly score
Anomely store
14.
15k
10k
3.5k
33K
33.2k
32.4k
32
Transformer
Anomaly score
Ancmaly score
Anomaly
(AssDic)
0.6
15k
36k
3.7K
3.5k
255
33.2k
32.4k
32.0k
32.8k
MEMTO (LSD)
Avomaly score
Anomaly
1.kk
1.kc
1.k
14k
35.2k
52.4k
32.8k
32.8k
SARAD (SAR)
Anomaly score
Anomallssore
0.6-
0.4
0.2
(Nominality)
Anemally score
NPSR
15k
15kk
35k
22.2k
Time
Time
Anomalies ground truth
Anomaly (The novel component each method)
Figure 11: Anomaly scores for the SMD and PSM, datasets. The red segment indicates the ground truth of anomalies
27

---

Published as a conference paper at ICLR 2026
TWaT
HAI
ALoRa (LLoR--T)
Anomaly score
Anomaly score
2.1k
0.4k
0.6k
0.94
3.0k
Anomaly
Transformer
(AssDic)
Anomalyy score
0.8
Anomaly soore
0.6
0.4
0.2
1.
1.96
2.18
2.3K
0.4K
0.6k
0.8k
1.0k
1.
MEMTO
Anomely score
Anomaly soore
(L.SD)
D.C-
0.4-
0.2
0.2
2.3k
0.4k
0.6k
0.8k
2.0K
Anomaly score
Anomaly score
SARAD
(SAR)
M
1.7k
1.9K
2:1k
2.3k
(Nominality)
Anomsly score
1.2
Anamaly score
2.5
NPSR
1.0
1.0
0.4
3.7k
3.00
2.18
2.3k
0.4k
0.6k
0.3k
1.0k
Anomalies ground truth
Anomaly score (The novel component of each method):
Figure 12: Anomaly scores for the SWat and HAL, datasets. The red segment indicates the ground truth of anomalies
28

---

Published as a conference paper at ICLR 2026
Standard Deviation of Anomaly Diagnosis Results: We report the standard deviations of the anomaly diagmosis results presented in Section 6 as well as for the additional evaluation metrics presented in Table IT] Specifically, Table [2 shows the standard deviations of the detection results reported in Table while Table [able[14[eppo reportt the standard deviations for the results in Table Finally, Table 13 presents the standard deviations of the localization results shown in Table
Table 12: Standard deviation of detection metries, for the results in Table 1 the standard deviation of Precision, Recall, and F1 Fi-re respectively.
P. R and Fi denote
SMD
PSM
MSL
SWaT
HAI
Method
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
Fi
KNN
0.006
0.004
0.005
0.003
0.002
0.002
0.009
0.006
0.008
0.003
0.002
0.002
0.006
0.004
0.005
PCA
0.009
0.007
0.008
0.006
0.004
0.005
0.003
0.002
0.002
0.009
0.007
0.008
0.003
0.002
0.002
LOF
0.002
0.007
0.006
0.004
0.005
0.006
0.005
OC-SVM
0.006
0.004
0.005
0.006
0.004
0.005
0.003
0.002
0.002
0.009
0.007
0.008
0.009
0.007
0.008
ForetionForest
0.009
0.007
0.008
0.003
0.002
0.002
0.006
0.004
0.005
0.006
0.004
0.005
0.007
0.008
OmniAnomaly
0.022
0.018
0.020
0.012
0.009
0.010
0.022
0.017
0.020
0.011
0.009
0.010
0.012
0.008
0.010
InterFosion
0.012
0.009
0.010
0.021
0.018
0.020
0.012
0.009
0.010
0.021
0.018
0.020
0.012
0.009
0.010
A.T.
0.021
0.023
0.020
0.020
0.03
0.030
0.020
0.016
0.018
0.017
0.014
0.016
0.02
0.016
0.017
DAEMON
0.014
0.009
0.012
0.010
0.009
0.011
0.011
0.010
0.011
0.018
0.017
0.018
0.013
0.008
0.011
DCdetector
0.021
0.018
MEMTO
0.065
0.055
0.060
0.009
0.006
0.007
0.009
0.007
0.008
0.025
0.019
0.022
0.053
0.045
0.050
NPSR
0.003
0.002
0.002
0.002
0.001
0.001
0.002
0.001
0.001
0.003
0.002
0.002
0.004
0.003
0.003
D
0.005
0.003
0.004
0.004
0.003
0.003
0.002
0.001
0.001
0.008
0.006
0.007
0.006
0.004
0.005
SARAD
0.10
0.028
0.022
0.13
0.08
0.021
0.021
0.044
0.018
0.025
0.022
0.013
0.017
0.10
0.110
ALoRa-Det
0.011
0.011
0.012
0.013
0.014
0.014
0.012
0.011
0.011
0.012
0.011
0.010
0.010
Table 13: Standard deviation of localization scores, for the results in Table 2 The Interfusion method applies only to segment- based localization, so the standard deviation is reported only for the IPS metric.
SMD
Mathod
HRRP
NDCGGP
JPS
HR.
NDCG&P
IPS@P
HRRP
NDCGOP
IPS&
100
1.50
100
150
150
MEMTO
0,000
0.019
1.024
0.027
0.022
0.041
0,081
0.048
0.621
0,099
0.005
0.118
0.0012
0,0035
0,0000
0,0012
0/0011
0.028
OMNT
0.041
0.041
0.045
0.045
0.028
0.029
0.04
0.03
1.03
0.04
3
0.002
0.02
0,025
D.G33
0022
0.01
0.0L
Ineertusion
0.029
0.331
0.001
0.001
0.01
0.01
SARAD
0.018
0.017
0.023
0.625
0.021
0.43
0.5
0.08
0.02
0.06
0.006
0.02
0.001
0.001
0.0020
0001
0001
0.01
DARMON
0.05
0.06
0.07
0.06
0.006
0.01
0.6
0.06
0.05
0.08
0.01
801
0.007
0.006
0,005
8006
0.01
0.06
AERCA
BU02
0.01
0.01
0.01
0.01
0.02
0.01
0.01
[UB]
0.01
0.01
0.01
0.61
0.0
0.02
0.02
0.02
0.02
ALeRa Loc
0.05
0.05
0.06
0.06
0.09
0.07
0.01
0.05
[U0]
0
0.001
0.003
0.006
0,203
00006
0.001
0.00
0.007
Table 14: Standard deviation for the results in Table I1 RF denotes Range- -soore VROC and Vp denote the Volume Under the Surface ROC and PR, respectively.
SMD
PSM
MSL
SWaT
HAI
Method
RF
Vioc
Vi8.
RF
Vioc
ViR
R.F
VRoc
VPR
RFF
VIOc
V/8
RFF
Vioc
ViR
KNN
0.006
0.008
0.004
0.005
0.006
0.005
0.007
0.009
0.006
0.004
0.006
0.005
0.006
0.007
0.005
PCA
0.009
0.012
0.006
0.008
0.010
0.007
0.006
0.009
0.005
0.010
0.013
0.007
0.007
0.009
0.006
IsolationFuresl
0.012
0.014
0.007
0.010
0.012
0.009
0.009
0.011
0.006
0.011
0,013
0.008
0.010
0.012
0.007
0C-SVM
0.010
0.011
0.006
0.011
0.012
0.008
0.007
0.010
0.006
0.012
0.014
0.009
0.011
0.013
0.007
OmniAnomaly
0.020
0.022
0.012
0.017
0.020
0.011
0.019
0.021
0.013
0.018
0.020
0.011
0.019
0.022
0.013
InterFusion
0.018
0.020
0.011
0.019
0.022
0.012
0.017
0.019
0.011
0.018
0.021
0.012
0.020
0.022
0.013
A.T.
0.017
0.011
0.019
0.012
MEMTO
0.025
0.028
0.015
0.021
0.025
0.013
0.023
0.027
0.015
0.026
0.030
0.016
0.022
0.026
0.014
NPSR
0.007
0.009
0.005
0.008
0.010
0.006
0.009
0.011
0.006
0.007
0.009
0.005
0.008
0.010
0.006
D
0.011
0.013
0.007
0.010
0.012
0.007
0.011
0.007
0.012
0.014
0.008
0.013
0.007
SARAD
0.013
0.016
0.009
0.015
0.018
0.010
0.014
0.017
0.010
0.015
0.018
0.011
0.014
0.017
0.010
ALoRa-Del
0.009
0.011
0.006
0.010
0.012
0.007
0.009
0.011
0.006
0.010
0.012
0.007
0.009
0.011
0.006
29

---

Published as a conference paper at ICLR 2026
PSEUDOCODES FOR ALORA-DET AND ALORA-LoC:
Algorithm ALoRa- Det: Attention Low-Rank Transformer for MTS Anomaly Detection
Input: Y E R?'xd Multivariate Time Series
H: Number of attention heads, L: Number of Transformer layers.
dmnntel: Embedding dimension, Areg AL.Ra regularization coefficient.
Output: Anomaly scores (g()) E R for a
Stage 1: LightMTS-Emmed
D see Section 5
Define kernel size m (default: 3) and time series pair set [(t,)]
2: Define sparse kernel tensor W e RR |P|xxxm
3: YI Conv ( 3 Ws)) Vk E [I]
Stage 2: ALoRa-T - Attention Low-Rank Transformer
4: Initialize total ALoRa loss: CALORa
5: for 1, ..... do
6:
Z + MHA(2 20-11),2 2())) Y 10
D MHA + Residual connection
7: 7 Z()
(Optionally apply activation function, e.g., GELU)
8: Compute LALORI (5(1)
Eq. ( 71 E (())
9: CALORA + CALORa - (ALORa(S
end for
Stage 3: Reconstruction and Training:
11: Reconstruction: Y, = (z) Wout E Txd
12: Training Objective:Croal Y - Y1 + Areg CALORa
Eq. (8)
Stage 4: Inference Anomaly Detection
13: Anomaly score for time / AS(y)) = |1 del ALoRa-T(ya)
P Eq 9
14: return AS(y)
Algorithm 2 ALoRa-Loc: Attention Low-Rank Transformer for MTS Anomaly Localization
Stage 1: ALoRa-T Parameter Tracking During Final Epoch of Training
1: During the last epoch of ALoRa-Det training:
2: Track LightMTSS-Enn Embed convolution kernels WA Rdxx for all k: = -..... dmodel
3: for each Transformer layer / L..... do
4: Track value projection matrix ((2.)) Bddusdd dduced
> If MHA is used: then (VV.)) [W(V.1.)..... W(V.H..). WV where each
(VVA.)) E Rdnne x Saaai and W V-pro... E IR Amute x xdaaas H is the number of heads
5: end for
6: Keep track of output projection matrix Wourt E R Releaaa xd
D (Eq. 66)
Stage 2: Compute Input-to-Output Contribution Matrices
D (see Section 4)
7: Compute contribution matrices B duuss Xdudde and E dxddaa
> see Section 4
8: Contribution of input time series to the reconstructed ones: C E Raxd
Eq. 12
Stage 3: Inference Anomaly Localization
9: ALoRa-Loc: LAS(() = -
j= Cjj (1)
(1)
yt
10: ALoRa-Loc (top-k): LAS (i..t.)pp)) E, Ejezt Ci 1ll1
I( indices of top-k values in row of C
12: ALoRa-Loc': LAS = (l 5/ l
return LAS(). (is(ittop)) and LAS,(1) for all i,t
30

---

Published as a conference paper at ICLR 2026
G ANALYTICAL PROOFS
Proof of Proposition To begin the proof, we first define the Space-Time Autoregressive (STAR) model. Let Yt R for f 1, ......,.. N, denote the multivariate observation at time step where (8) IR represents the value of the i-th series at time The STAR model assumes that each time yt
series is influenced by its own past values as well by the past values of spatially neighboring series.
STAR Model Definition. For STAR model with p temporal lags, and g spatial lags, the model defined as
9
y1
(1) tk -t-
B
Σ
(1) A X B
(18)
k=0 1=0
1=0
where B() E Rdxd is the spatial weight matrix of order 1, and B represents the spatial
dependency of time series on time series j in the I-th spatial lag. The coefficient ag denotes the temporal weight for lag & at time t, corresponding to the I-th spatial lag. The final equality follows by defining the matrix X = [yt-p.... yy E Rexd be represented as a single lag (i.e., the spatial lags are collapsed into one), the model simplifies to:
case where the spatial dependence can
STAR model with + temporal lags and one spatial lag:
Y/: ask B A.X
(19)
where A, E 1x is the vector of temporal weights used for modeling time step t. Finally, the j-th component of yr. denoted by Vi
(3) can be written as:
P - Σ
Ask Bij Di
Σ Bij
Σ k=
Ack
(5) Yi-k
(20)
k=
The STAR model can be fitted by minimizing the least square error between the observed data y/ and the fitted values. We note that all the learnable weights are not input-dependemt
Statement 1: When skip connections used in the self-attention mechanism. the update rule at layer is defined as:
Z (1h) (V where, Z Y
By unrolling the above equation, it follows that after L layers of self-attention, the latent represen- tation time step can be expressed as:
2 2 = S(L-)..... (VV))) (V(L))
(21)
where each ( RXT is the SA-matrix layer and s((L) RIxT is the l'th (last one) of the final SA-matrix. Each (V.3) R xdmas is the value projection matrix layer By defining
the product of attention matrices up to layer at time step as A, = s((4) ((. 5() E RIXT
and the product all value projection matrices as B = (VV1) (V.)) E Rdaaa x dunes the final latent representation is compactly expressed by:
Z, A, B 1xdeaa
(22)
Then, by expanding the above equation for each component, we can derive its analytical expression. The expression for each latent space time series at time step t is given by:
drunk
bus
Σ 9=1
Ciy
u(hh
(23)
By comparing the derived equation with Eq. 20). it is evident that the two models share the same structure. The key difference lies in how the weights azg obtained: traditional STAR models use
31

---

Published as a conference paper at ICLR 2026
fixed lag weights estimated by minimizing loss function (e.g., mean squared error), whereas the Transformer computes these weights dynamically through the attention mechanism, with the queries Q and keys K determining them in real time.
Statement
When skip connections applied in the self-attention mechanism, the update rule at layer is defined as:
z = 2 Z 11-1)
where 0 (=)0)())
(24)
To clarify the structure of the derived Eq. (5). assume without loss of generality that the total number of layers is
3. Then, the final representation can written as:
= Ut + S W(YD) + S w(v2) + (V.1) ((.3)
St (
W(333
+ S S. ly WOLD S
(25)
S S
W S st
Each component of Eq. 25 has the same structural form as Eq. Statement I, each such component induces structure analogous to that of STAR model. Thus, the overall latent representation Z, can interpreted as a linear combination of multiple STAR-like processes, where each component has its own temporal (atg) and spatial weights (bky), allowing for heterogeneous temporal and spatial lag structures across components. We can obtain a more specific interpretation of this representation by comparing Eq. (25) with the general definition of STAR model in Eq. (18). In particular, Eq. (25) corresponds to STAR model with T (the
23 As shown in the proof of
window size) temporal lags and g = 8 = (2) spatial lags.
Statement 3:
The feed-forward layer, linearly combines values across all series the same time step produce a new time series 1x,5) We now show that the transformed representation for each time series, after applying the feed-forward layer,
be written in the same structure as in Eq. (22): For ease of
presentation let T = 2 and the latent dimension dmodel = 3. The new j-th time series at time step t
defined as: (()) W1i (1) + 22,21
(2) + W3321
(3)
Expanding each 2;
6,47) wij (012 (at,)-1bh))) + amiil") + bu1 (ac,-1ihh)) aub!?)) + bsl (ar,-101)) + andi (
+w2j (i (ar--1ih11 + am.ih()) + 2az (at,-18(2) + a..i(e)) + baz (at,-11(()) + acrs((.))
+waj bis (arc-1811) + anti! + b23 (ar,--112) + a + b33 (ar,--1t + accvi (3)
By regrouping the terms,
((1) at,t- 196-1 + anih!") (wisbri w22b12 w31b13)
+ (0,1--1(-) + ani!l (wr + W22b22 + Way Waybb2))
byy
+ -(015-17(5) -(3) + aer (3)) (wisbar + w2yb32 + waybba)
bas
32

---

Big does not alter the interpretation of (k) To illustrate this, consider the case T 2 and Published as a conference paper at ICLR 2026
so for general latent space dimension dmodel and for general window size T:
tmodel
] where baj = = Wpybkr
(
bkj
Gegg Dgg
(26)
Therefore, from the first statement of the proposition, follows that the structure of the latent space time series remains unchanged, that is, it retains STAR-like form. The only difference is that the new weights bkj now determine the contribution of each input time series, replacing the original weights bkj.
G.1 DERIVATION OF CONTRIBUTION WEIGHTS FROM INPUT SPACE TO LATENT AND RECONSTRUCTION SPACES (RELATED TO SECTION 5.2
The latent representation at the final layer is given by Eq. In Eq. 25
we provide an ana-
lytical expression for the case of three-layer model. Each component Eq.
25
has the form
Each of these components contributes to the overall contribu- tion weights we aim to derive. As first step, we consider how each embedding -spcc time series contributes to the representation of the j-th time series (column) in the k-th component of Eq. (25)
A (R) Y [d] (A) for k ===0 0, 0...., 2'
[A] (M) Y [1] (k)). The j-th time series (column) in the k-th component, be expressed as:
destel
(1.)
(1))
Σ
(k) dig
2t
where 6.7 denotes the (rth) entry of (A) and measures how strongly the r-th embedding- space time series contributes to the j-th latent series the k-th component. The coefficient ozg is the
(1, g)-th entry of A and serves as temporal weighting factor. Importantly, the dependence on (kc
dmodel 3. In this setting, the three latent space time series are given by:
(1,A) 011 (a (a + a 2/ +
(1)
(R)
2t
+ o De + 31
1.1-1
1,1-1
+ ai,
24
(2.A) b12 (01 dit
(1) (k) (1)
1/
11,1-191
(k) (2)
dit
(k) (2)
+
D2
(k)
(k)
(3)
+ ait Vi
(k) (3)
(3,A) 1/ (k) (1)
(a 1 + o 0t 22
(a.t-10
10
(2)
az De
(k) (2)
+ b
(3)
+ ai, y1
(k) (3)
In all three equations above: The temporal attention weights a and a are shared across all parentheses in each equation and are also shared across all equations, as they remain the same for 2 (1.At) (2.k) (3.k) What differentiates how much each embedding space time series (z(0) =
contributes to each latent space series (z(iA)) only the contribution weight ((5))
Then, by summing the contribution weights over all components of Eq. (25), we obtain the matrix that characterizes how smbedding-space time series contribute to the latent space, defined as
B= II (V.) +T).
Finally, by accounting for the embedding module, the contribution weights from the raw input time series to the latent space time series are given by:
m-1
dinas
Eij
Will
buj
(27)
k=1
J
120
Then, the reconstruction of each time series is produced follows: (k) dmndet
(L.3)
E
witk 2t
33

---

Published as a conference paper at ICLR 2026
This explains why the final derived contribution weights of the input time series to the reconstructed time series are given by:
dmadel
Cif
Σ
wkj Eik
34


Reviews:
[{"review_id": "zEcXUVhCEB", "reviewer": "Reviewer_CXw2", "summary": "This paper advances multivariate time series (MTS) anomaly detection in three distinct aspects.  First, it provides theoretical insights into how the Transformer encoder represents and learns from the MTS data, revealing how its representations relate to classical time series. For instance, the authors equate the embedding process to Vector Moving Average (VMA) filtering, and the self-attention mechanism to the Space-Time Autoregressive (STAR) structure. Second, the authors propose Attention Low-Rank Transformer (ALoRa-T), which consists of the LightMTS-Embed module and Attention Low-Rank (ALoRa) layers, and a decoder. Lastly, given this new architecture, the authors propose a novel detection score and localization method: ALoRa-T score and ALoRa-Loc method.", "questions": "Please refer to the weaknesses above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The authors theoretically relate the Transformer architecture back to the techniques from classical time series modeling. Based on this insight, they propose technically sound and well-motivated modifications to the Transformer architecture, further specializing it for the task of MTS anomaly detection.\n\n- The authors propose novel detection and localization frameworks that are more reliable than previously used metrics.\n\n- Together, the proposed method and detection/localization methods successfully outperform other baselines. The experimental results are quite comprehensive, and the authors have included code and sufficient experimental details to reproduce the results.", "weaknesses": "- According to Table 1, it appears that AloRa-Det is more effective on some datasets (ex) HAI or SMD) than other (SwAT, MSL). What causes such a discrepancy in the results? Is ALoRa-Det more effective at detecting certain anomaly types than others?\n\n- The majority of the baselines are drawn from Transformer-backed anomaly detection methods (for a good reason). Yet, it would be helpful to add some baselines from other families of MTS anomaly detection methods, such as reconstruction or contrastive learning-based methods.\n\n- Do the authors expect their method to stay functional in application scenarios where anomalies and distributional shifts (concept drifts) appear mixed together? If so, how could ALoRa-T be extended or modified to such cases?\n\n- Although the authors present ablation studies in Section D, I believe a more thorough ablative study that investigates the effectiveness of each proposed technical component separately to assess its contribution is necessary.\n\n- Just a minor comment on paper formatting: I understand that the authors have chosen to move many of the experimental results due to the page constraint, but I personally think key results and analyses should still remain as a part of the main manuscript. I suggest that the authors truncate some of the materials in the introduction/related works to make room for the results section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LfyHsHjZik", "reviewer": "Reviewer_2urD", "summary": "The paper proposes ALoRa, a Transformer-based framework for multivariate time-series (MTS) anomaly detection and localization grounded in a theoretical analysis of Transformer encoders on MTS. The authors show that the encoder’s latent representations can be expressed as linear combinations of Space-Time Autoregressive (STAR) processes, which motivates (i) ALoRa-T—a Transformer with low-rank regularization on self-attention—and (ii) a detection score that counts significant singular values of the final attention matrix. They further derive contribution weights from inputs → latent → outputs to trace anomaly propagation and attribute anomalies to variables (ALoRa-Loc).", "questions": "(1) How sensitive is the final detection F1-score (which relies on the combined $AS(x_t)$) to this choice? For instance, what is the performance impact if $h_1$ is set 10x larger or 10x smaller than the value chosen via the eigenvalue distribution analysis?\n\n(2) Please provide explanation on how $h_2$ value was selected and why.\n\n(3) Do different anomaly types (point vs collective vs contextual) induce distinct singular-value patterns? Any class-wise analysis of detection latencies?\n\n(4) For segments where the anomaly propagates widely, how often does top-k ALoRa-Loc identify the true origin vs “most affected” variables? Could you report per-segment confusion analyses?\n\n(5) Some important ablations are missing: rank-only score vs error-only vs multiplicative combo; head-wise vs averaged penalty; all-pair vs top-K embeddings; FFN on/off at matched params. Could you please provide ablations on these?\n\n* A minor typo in line 228; \"throught\" $\\rightarrow$ \"thought\"", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The paper provides a coherent spectral perspective on attention that is simple to compute conceptually and ties to an interpretable diagnostic. \n\n(2) The authors diagnose that point-adjustment inflates results—sometimes making them indistinguishable from random scoring—and therefore pivot to range-aware/affiliation-based metrics, improving evaluation validity.\n\n(3) The localization section explicitly models propagation via contribution weights (E, C), which is more principled than per-dimension reconstruction heuristics. \n\n(4) The training objective is compact and implementable; the regularizer integrates cleanly with standard reconstruction losses.", "weaknesses": "(1) While the detection pipeline uses two thresholds ($h_1, h_2$), Appendix A provides data-driven approach of choosing threshold $h_1$, but this is still a per-dataset manual step, introducing hyperparameter sensitivity. Also, neither ablation on $h_2$ selection nor heuristics on choosing it was provided.\n\n(2) The paper’s central intuition—\\textit{anomalous windows yield higher attention rank}—is supported empirically (plots/observations) but lacks a formal guarantee. No theoretical background specifies conditions under which anomalies must raise rank (or non-anomalies must not).\n\n(3) ALoRa-Loc traces propagated influence, but ranking metrics like HR/NDCG/IPS do not distinguish origin variable from downstream affected variables; without per-segment confusion analyses, it’s unclear whether the method finds causes or merely effects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AQTxkAc9XZ", "reviewer": "Reviewer_NVbX", "summary": "This paper proposes a transformer-based framework for time-series anomaly detection that leverages attention rank analysis to interpret and localize anomalies. The key idea is that the rank of self-attention matrices increases when anomalies occur, providing a new signal for both detection and localization.", "questions": "See the weaknesses section", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The idea of detecting anomalies by analyzing the transformer’s learning behavior is original and insightful. It opens a new direction for understanding model-internal representations in time-series anomaly detection.\n\n2. The focus on anomaly localization is meaningful and practically valuable.", "weaknesses": "1. The paper uses Spearman correlation to estimate dependencies among sequence pairs but does not justify why this choice is preferred over Pearson correlation or Cosine Similarity. Furthermore, the paper states that only the top-K correlated pairs are retained, yet the criterion for determining K is not specified or experimentally analyzed.\n\n2. The central claim that “the rank of SA-matrices increases in the presence of anomalies” is only supported by empirical observation on a few datasets. The paper does not provide a theoretical explanation or evidence that this phenomenon holds consistently across diverse anomaly types and domains.\n\n3. The definitions of variables are inconsistent—sometimes the input sequence is denoted as x, other times as y, making the mathematical expressions difficult to follow.\n\n4. The inference process depends critically on the threshold h_2. Although the paper mentions that Appendix A describes its selection, the appendix does not include such details yet.\n\n5. Localization evaluation requires ground-truth information about the precise anomalous series. However, the datasets used in the experiments typically provide only record-level anomaly labels (anomalous or normal per timestamp) without explicit localization annotations. Could the authors clarify how the localization ground truth is obtained?\n\n6. The main text experiments are overly concise and lack detailed analysis. Although the appendix includes an ablation study, it only evaluates the embedding module. A more critical ablation, particularly on the ALoRa loss function, is missing and should be included to support the claimed effectiveness of the proposed loss.\n\n7. The paper does not provide the source code, and the methodological descriptions are not detailed enough to reproduce the reported results reliably.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9H28uiD9Kt", "reviewer": "Reviewer_Qfvm", "summary": "The paper tackles multivariate time series anomaly diagnosis, covering both detection and localization. It analyzes the learning behavior of Transformers from a theoretical perspective and connects it to classical statistical time-series analysis. Based on these insights, the authors propose the Attention Low-Rank Transformer (ALoRa-T) with low-rank regularization to better capture temporal anomaly patterns, and introduce ALoRa-Loc for variable-level anomaly localization. Experiments on real and synthetic datasets show that the proposed approach outperforms existing methods in both detection and localization tasks.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper offers valuable theoretical insights by linking the Transformer’s self-attention mechanism to established statistical time-series principles, providing a more interpretable foundation for deep anomaly detection models.\n2. Unlike many prior works focusing only on detection, the introduction of ALoRa-Loc enables variable-level anomaly attribution, advancing the underexplored area of multivariate anomaly localization.", "weaknesses": "1. The distinction between “time series” and “variable” is not consistently maintained throughout the paper. Since each variable corresponds to a univariate time series, the terminology should be clarified to avoid conceptual confusion.\n2. The paper states that each kernel learns representations from only two time series, but the motivation for selecting exactly two is not explained. \n3. The metrics used to evaluate anomaly localization ability — Hit Rate, Normalized Discounted Cumulative Gain (NDCG), and Interpretation Score — are not well-suited for this task. Hit Rate and NDCG are designed for ranking or recommendation settings, while Interpretation Score lacks a clear definition in the context of anomaly localization.\n4. The paper does not compare with established approaches for anomaly localization or root cause identification, such as “Root Cause Analysis of Anomalies in Multivariate Time Series through Granger Causal Discovery.”\n5. In Table 2, several numeric values use commas instead of decimal points.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)