Local Geometry Attention for Time Series Forecasting under Realistic Corruptions
Authors: Dongbin Kim
Youngjoo Park
Woojin Jeong
Jaewook Lee
Venue: ICLR 2026 Poster (2026)
Keywords: Local Geometry
Local Gaussian Process
Transformer Architecture
Time Series Analysis
Corruption Benchmark
Primary area: learning on time series and dynamical systems
TL;DR: We propose LGA, the local geometry-aware attention mechanism based on the local Gaussian Process, and introduce TSRBench, the first benchmark for evaluating time series forecasting model robustness under realistic corruptions.

Abstract: Transformers have demonstrated strong performance in time series forecasting, yet they often fail to capture the intrinsic structure of temporal data, making them susceptible to real-world noise and anomalies. Unlike in vision or language, the local geometry of temporal patterns is a critical feature in time series forecasting, but it is frequently disrupted by corruptions.
In this work, we address this gap with two key contributions. First, we propose Local Geometry Attention (LGA), a novel attention mechanism theoretically grounded in local Gaussian process theory. LGA adapts to the intrinsic data geometry by learning query-specific distance metrics, enabling it to model complex temporal dependencies and enhance resilience to noise. Second, we introduce TSRBench, the first comprehensive benchmark for evaluating forecasting robustness under realistic, statistically-grounded corruptions.
Experiments on TSRBench show that LGA significantly reduces performance degradation, consistently outperforming both Transformer and linear model. These results establish a foundation for developing robust time series models that can be deployed in real-world applications where data quality is not guaranteed. Our code is available at: https://github.com/dongbeank/LGA.

Published as a conference paper at ICLR 2026
LOCAL GEOMETRY ATTENTION FOR TIME SERIES FORECASTING UNDER REALISTIC CORRUPTIONS
Dongbin Kim', Youngjoo Park',
Woojin
Jaewook Lee!
'Seoul National University
(dongbi n413, youngjoo0913, jwj7955, jaewook @snu. ac.kr
ABSTRACT
Transformers have demonstrated strong performance in time series forecasting, yet they often fail to capture the intrinsic structure of temporal data, making them susceptible to real-world noise and anomalies. Unlike in vision or language, the local geometry of temporal patterns is critical feature in time series forecasting, but it is frequently disrupted by corruptions. In this work, we address this gap with two key contributions. First, propose Local Geometry Attention (LGA), novel attention mechanism theoretically grounded in local Gaussian process theory. LGA adapts the intrinsic data geometry by learning query-specific distance metrics, enabling it to model complex temporal dependencies and enhance resilience to noise. Second, we introduce TSRBench, the first comprehensive benchmark for evaluating forecasting robustness under realistic, patisticallyggrouneed corruptions. Experiments on TSRBench show that LGA significantly reduces performance degradation, consistently outperforming both Transformer and linear model. These results establish a foundation for developing robust time series models that can be deployed in real-world applications where data quality is not guaranteed. Our code is available at: https: //tpthub.. com/ wwnpb/ank////oooooooo/e/oa/..
INTRODUCTION
Transformer architectures have revolutionized deep learning across various domains since their introduction (Vaswani et al. 2017). Their success in natural language processing (Devlin et al. 2019) and computer vision (Dosovitskiy et al. 2020) has extended to time series analysis, where models like PatchTST Nie et al. 2023) have set new performance benchmarks.
However, time series data have unique characteristics that challenge standard Transformers. Unlike text or images, time series often exhibit complex temporal dependencies and non-unine non-uniform local data distributions, creating a structured "attention geometry" (Si et al., 2024, Lavin & Ahmad 2015). Standard attention mechanisms, which treat all inputs uniformly. may fail to adapt to these local statistical variations, seasonal patterns, and anomalies, leading to suboptimal performance and lack of robustness (Schmidl et al. 2022 Cheng et al. 2024).
Furthermore, while robustness to input corruptions a well-established evaluation standard in other fields, with benchmarks like ImageNet-C (Hendrycks & Dietterich 2019), comparable framework for time series forecasting is notably absent. This gap is critical, as real-world time series are frequently contaminated by issues like sensor failures and transmission noise, yet existing research has largely focused on synthetic adversarial attacks rather than realistic data degradation (Liu et al.) 2023 Cheng et al., 2024). These limitations highlight dual need: attention mechanisms that adapt to local temporal structure and principled benchmarks to assess their robustness.
To address these challenges, make two primary contributions:
We propose Local Geometry Attention (LGA), novel attention mechanism designed to adapt to the intrinsic data geometry of time series. Theoretically grounded in local Gaussian process theory, LGA learns query-specine distance metries to compute attention
CCorresponding author

---

Published as a conference paper at ICLR 2026
Data Points Query Point Query Point 2
100
Data Points Query Point
High Score on Anomaly (Standard Attention)
10
Anomally
088
0.6
0.0
0.0
2.0
1.5
-1.0
3.0
0.0
0.9
-3
25
30--0
(a) Attention score surfaces of LGA for two distinct query points
(b) Attention score surfaces of LGA and Standard Attention for query point, with anomaly
The core principles of Local Geometry Attention (LGA) illustrated on a 2D toy dataset. In (a). for two different data distributions, the attention score surface generated from each query point QQuery Point and Query Point 2) correctly captures the unique local geometry of its respective cluster. (b) demonstrates robustness of LGA: while standard attention (red surface) is distracted by the anomaly and assigns it high score, LGA (blue surface) correctly identifies it as an anomaly and maintains focus on the main data distribution.
Figure 1:
scores, enabling it to model complex local data structures without imposing strong global assumptions.
We introduce TSRBench. a comprehensive benchmark for evaluating time series forecasting robustness. TSRBench provides statistically grounded methods for injecting two canonical corruption types-spiii and level shifts-with controllable severity levels, addressing critical gap in principled robustness evaluation for time series.
We demonstrate practical effectiveness and robustness of LGA through extensive experi- ments. As illustrated in Figure [ LGA successfully identifies distinct data distributions and ignores anomalies, unlike standard attention. Our full empirical evaluation on the TSRBench benchmark further shows that LGA consistently and significantly mitigates performance degradation under various realistic corruptions.
We validate our approach through extensive experiments on standard forecasting datasets. The results demonstrate that LGA significantly mitigates performance degradation under corruption compared to existing baselines. highlighting the importance of designing both robust attention mechanisms and principled evaluation tools for time series forecasting.
2 RELATED WORK
Theoretical Approaches for Attention. Transformers (Vaswani et al. 2017) have become the
dominant architecture in natural language processing (Devlin al. 2019: Brown et al.) 2020 Raffel
et aal.2022)) and vision (Dosovitskiy et al. 2021: Liu et al. 2021 I Touvron et al. 2021 l) and have recently been extended to time series analysis (Wu et al.] 2021 Zhang & Yan 2023] Nie al. 2023). Attention is fundamental component of Transformer architecture and recent studies have offered several theoretical interpretations of its structure. Bui et al. (2024) interpret attention as cross- covariance between correlated Gaussian processes enable asymmetric uncertainty-aware attention. Caiiarllyyyyhh&& & Li (2023) proposes Sparse Gaussian process Attention (SGPA), which replaces the standard dot-product with symmetric kernel, allowing Bayesian inference via the GP posterior. Han et al. (2023) propose robust kernel density estimation (RKDE), which mitigates the influence of outlier keys in the computation of attention scores. They employ the Median-of-Ment (MoM)
Similarly, Chen
2

---

Published as a conference paper at ICLR 2026
principle (Jerrum et al.,
1986. Humbert et al. 2022) into RKDE to further improve computational
efficiency. Nielsen et al. 2024) constructs hyper- neighborhoods around queries to increase attention weights in contextually important directions. However, these approaches either rely global kernel assumptions or do not explicitly capture the local geometric structure of the data.
Robustness benchmarks under Realistic Corruptions. Robustness benchmarks are critical for evaluating model performance under contaminated inputs. In computer vision, ImageNet-C (Hendrycks & Dietterich 2019) set standard by testing models against various common corruptions multiple severity levels. Similar efforts exist in natural language processing (MeCoy al. 2020) Nie et al.] 2020). In contrast, the time series domain has largely focused on robustness against syn- thetic adversarial attacks (Liu et al. [023 Lin et al. (2024), which target model-specific vulncrabilitics but may not reflect naturally occurring data degradation.
Recent studies, however, emphasize the importance of evaluating models against realistic corruptions that mirror real-world phenomena (Cheng et al. (2024). Real-world time series often exhibit both minor point-wise disturbances and structured anomalies that signal meaningful, event-driven changes, such random spikes from sensor limitations sustained level shifts from hardware malfunctions (Si et al. 2024, (Schmidl et al. 2022). Despite the need, a standardized benchmark for such realistic corruptions in time series remains significant gap, underscoring the need for systematic framework to evaluate model robustness under diverse operational conditions.
LOCAL GEOMETRY ATTENTION
We propose Local Geometry Attention (LGA), novel approach that adapts to the intrinsic geometric structure of time series data, Unlike standard attention mechanisms that use dot product sim- ilarity in Euclidean space, LGA computes attention scores using query-specific distance metrics derived from Gaussian process theory. This approach captures the local geometric structure of the data manifold, enabling more effective attention mechanisms that reflect the inherent geometry of periodic temporal patterns, as illustrated in Figure 2 where similar periodic patterns naturally form clusters.
Keys (period=3)
Queries (perled-3)
Key-query embed- dings in PStchTST showing natu- ral clustering of periodic patterns. geometry in Section 3.1 we then connect this to local Gaussian processes to estimate data density in Section 3.2, and finally, we derive the geometry-awe aware attention scoring function in Section 3.3 With the theory established, Section 3.4 then presents the practical implementation, detailing how this theoretically-gronnded function is efficiently approximated for use in modern architectures.
In this section, we build LGA from the ground up. We start by developing its theoretical foundation in three parts: we first establish local kernel-covariance formulation to capture data
Figure 2:
3.1 LOCAL GEOMETRY ESTIMATION VIA HERNEL-COVARIANCE
To capture the local geometric structure of time series data, we employ local Gaussian processes instead of traditional global Gaussian processes. We first establish a local kemel-covariance formula- tion that naturally extends to local Gaussian process framework, providing a principled foundation for modeling the intrinsic geometry of time series and informing attention mechanism.
Local Kernel-covariance.
First, we define linear mappings that project an input R into key and query representations, k E Rdd and q E Rd, respectively. For a set of inputs (I1, XT) (r) and target point x., we obtain key vectors {k] ......... kr} and query vector q
We then define a feature mapping Φ IRd + W based on the difference between keys and the query: kk,,) k, q. The design matrix + is constructed as:
$ (x.)) = (á(k))," ((r)))
3

---

Published as a conference paper at ICLR 2026
The local kernel-covariance matrix for the target point is formed weighted sum of outer products:
T Σ wi(s))) (k, q.)(k, - q.)
E(r.)) = (xr.) W (x,, (
(I)
Here, W(x.) is T' diagonal matrix with weights wi(x.) on its diagonal. These weights are computed using a kernel function K, which measures the similarity between each key k; and the query q
K(Kg,9.)
wa. Σj=1 K(kj, q.)
In our implementation, we use a Gaussian kernel for K:
K K(k;, q.) = exp (-Ik;; 9.112/21²)
where h is the bandwidth parameter. More generally, K can be any kernel function, such as the compactly supported tri-cube or Epanechnikov kernels. Our local covaal-covariance matrix, as defined in Equation ( thus employs decaying kernel weights to approximate the inverse metric tensor on the data manifold. As demonstrated by Berrr & Sauer (2016), this formulation effectively captures the local geometry of the underlying manifold.
3.2 DENSITY ESTIMATION WITH LOCAL GAUSSIAN PROCESSES
Now consider a local Gaussian process regression model where all observed outputs are zero (t/ = for all i), based on the local kernel covariance centered a target point v.. The model takes the form:
f(x) +,, where f(x) ((x,,)(
Here, the noise term E is assumed to be i..d. Gaussian, E ~ NN(0, )?), and k(x,2') is the GP covariance kernel.
For target point v. (which maps to query q.). the local GP model is established. The predictive distribution for the latent function value at a new point (which maps to key k) has a zero mean. Reusing the feature map ()) = k qu, the predictive variance is given by:
0² (k) <kk) G(a.)o(k) (k- (qq)" "G(x)... q.)
where the matrix G(r.) is defined as:
G(x.)= of =2
Crucially, the predictive variance is smaller in regions dense with data points (keys) and larger in sparse regions (Williams & Rasmussen 2006 Kim & Lee (007)) This means the negative predictive variance, -02 2 (k), can serve as a surrogate for the data density around q- as experienced by k. g- This insight allows to use the variance function as a principled, data-driven way measure the similarity between points on the data manifold. We provide a formal theoretical justification for this approach in the Appendix A.2 Building on this connection, we next reformulate the attention mechanism itself.
3.3
CECMETRY-AWATER ATTENTION SCORING
Building upon this local Gaussian process framework, reformulate the attention score using the negative predictive variance. The similarity score between a query and key is defined as:
score k) -(k q) G(q)(k-o
9)
(5)
where G(q) the local geometry matrix estimated at the query's location. This score function computes the negative squared Mahalanobis distance, where the metric is adapted to the local data geometry. The softmax function then produces the final attention weights.
This geometry-aware scoring function forms the core of our Local Geometry Attention (LGA) mechanism. By leveraging the local geometric information encoded in the matrix G(q), our approach moves beyond simple Euclidean similarity better capture relationships between time series elements along the data manifold. This leads to more effective attention allocation for complex temporal patterns, demonstrated in Figure
4

---

Published as a conference paper at ICLR 2026
Attention
Avg. Weight
Sundert
Goceuerry
Alteriion Weights Slardiard Altn
Quary Patches
Attention Weights Local tieometry Altn
-1
-2
100
200
400
500
23
Key Patches
20
30
(a) An input time series with both natural (blue- shaded) and artificial (red-shaded) noisy regions
(b) Comparison of attention weights between standard attention and LGA
Figure 3: Effectiveness LGA on a real-world time series with corruptions. The input series in @ corrupted with noise. (b) shows that while standard attention is distracted by these noisy regions, LGA successfully diminishes their influence. This robust handling of noise enables LGA to achieve a superior MSE of 0.533, compared to 0.616 from standard attention.
Connection to Riemannian Geometry. The proposed geometry-awan attention scoring in Equa- tion (5) also has a strong theoretical interpretation within the framework of Riemannian geometry. On data manifold M equipped with Riemannian metric tensor G, the distance between two points q. k E .M is given by the geodesic distance, dist(q, k) inf So' (t)(t) G()(t)) dt, where the infimum is taken over all smooth paths y connecting q and k. While computationally intractable, for key a small neighborhood of a query q, the squared geodesic distance can be approximated by first-order Taylor expansion: dist(q, k)² (k - T (kq)(k-q)
This expression reveals that the local Riemannian geometry induces a Mahalanobis distance. Con- sequently, our proposed attention score, score(q, k) -kk-qp as the negative squared geodesic distance approximation. In this view, the matrix G(q) derived from our local Gaussian process framework in Equation 4 serves
G(q)( q) be interpreted - q)
empirical estimate of the local Riemannian metric tensor at the query point, allowing the attention mechanism to adapt to the intrinsic curvature of the data manifold.
This formulation provides a theoretically sound, geometry- attention mechanism. However, the direct computation the metric tensor G(q) for every query, which requires access the full set of keys as per Equation 4 is computationally prohibitive for large-scale models. Addressing this computational challenge requires an efficient implementation.
3.4 IMPLEMENTATION OF LOCAL GEOMETRY ATTENTION
To make Local Geometry Attention (LGA) computationally feasible, we train a small network, fo. to directly approximate the metric tensor G(q) from a given query vector q:
G(q) z fe(g)
(6)
This is motivated by the insight that a position of query on the data manifold implicitly defines its local geometric structure. As universal function approximators, neural networks are well-suited to learn this mapping. For each attention head, we employ a separate network fo, to predict its corresponding metric tensor.
To ensure the network fe generalizes well, we train it on two sets of query vectors: (1) subset S randomly sampled from the actual query vectors that appear during training, and (2) set Sgen of randomly generated vectors designed to explore a broader region of the representation space. For efficiency, approximate the target metric tensor Grue (computed via Equation (4) diagonal matrix. This is a practical offde-off that assumes independence between local feature dimensions to ensure computational tractability.

---

Published as a conference paper at ICLR 2026
22
21
21
14-
11
16
13
11-
200
203
402
200
300
600
200
270
450
200
(a) Original signal XT
(b) XT. w/ spike noise
(c) Xy w/ level shift (d) Xx w/ combined noise
Figure 4: Visualization of synthetic corruptions on the ETTml dataset. (a) Original time series
without any corruption. (b) Time series with spike corruptions. (c) Time series with level shift
corruptions. (d) Time series with a combination of corruptions, demonstrating how our TSRBench creates challenging test cases for time series models.
The networks are then trained to minimize the mean squared error between their prediction and the true metric tensor. The total loss is weighted sum over all layers L and heads H:
L
LG
Σ h
(q), (q)) +
Σ qE
((Gre Foy (q))
(7)
L.
Sgen
k=1
This pre-learning strategy decouples the expensive geometry estimation from training and inference of the model. Consequently, the local geometric structure can be rapidly approximated, making LGA practical for modern deep learning models. The complete algorithm is detailed in Appendix
4 CORRUPTED TIME SERIES BENCHMARK: TSRBENCH
To address the gap in standardized robustness evaluation for time series, introduce TSRBench, a new benchmark designed to assess model performance against realistic, statistically corruptions. Unlike adversarial attacks, TSRBench focuses on corruptions reflecting naturally occurring data degradation, such as sensor limitations or external shocks (Si et al. 2024 Schmid! al. 2022).
While evaluation on real-world datasets with genuine anomalies is ideal, it presents challenges for the specific task of forecasting robustness. fair evaluation requires the ground truth of the test be clean, which necessitates precise, time-aligned anomaly labels to exclude corrupted periods from the loss calculation. However, most real-world forecasting benchmarks lack such labels, making impossible reliably isolate the effects of input corruptions on future predictions. Our TSRBench addresses this critical gap by providing a controlled environment where corruptions are systematically injected into the imput, while the ground truth for the forecasting horizon remains clean. This enables a principled and reproducible assessment model robustness that difficult to achieve otherwise.
Our benchmark introduces two canonical corruption types: spikes and level shifts /Wang et al. 202
Lavin & Ahmad 2015). which model phenomena like transient bursts and sustained deviations. realistic test than common augmentation like jittering (Iglesias al.) 2023) To support a systematic evaluation, we adopt statistically grounded noise injection methods with controllable severity levels (Siffer et al. 2017 Wunderlich & Sklar 2023).
This provides
Local Corruption Functions. Given an original time series X RT. we generate two types of corruptions: spikes, espike and level shifts, e-hiltt The timing of these corruption events is governed by a Poisson process with rate reflecting the random arrival discrete perturbations.
Each corruption type is defined by amplitude parameter h and duration parameters (d,, d2. or d). For corruption event starting time the spike function e expike exponential spike, while the level shift function e (shi models a flat shift. These are defined as follows:
(s) simulates asymmetric

---

Published as a conference paper at ICLR 2026
ht
exp
ln(
di))
f if + d1 +
expuke
(s)
h1
sppik
exp
In(()
(s - d1) da
(8)
=shfft (s) = h
for T < T
Here, the duration parameters (d,, d2, d) are sampled from a geometric distribution with parameter p. and the sharpness is fixed at / = 10- Crucially, the corruption amplitudes (IrPike />ahet) are not arbitrary. They are calibrated based on time- maryiig statistical thresholds determined by a significance level This process, which uses the DSPOT algorithm (Siffer et al., (2017) to model extreme values, ensures that the injected corruptions represent statistically significant but realistic deviations from the normal behavior of signal. The complete generation algorithm is detailed in the Appendix F. (Algorithm 2
Each final corrupted signal is formed by summing the noise instances from all events: Spike (t),
spike (t)
and shill
TET
where is the set of starting times for corruption events. Figure visualizes examples of these corruptions.
Table 1: Parameter settings defining the five corruption severity levels. From level I to 5, the expected corruption frequency (A) and duration (p) increase, while the amplitude significance level (q) decreases. They collectively intensifying the corruption.
Params.
Description
2
3
4
Expected frequency of corruptions
0.002
0.004
0.004
0.008
0.008
Expected duration of each corruption
6
9
12
12
15
Significance level of amplitude
0.0016
0.0016
0.0004
0.0004
0.0001
Corruption Severity Levels. To comprehensively evaluate model robustness, we designed five severity levels that represent gradually increasing data degradation. The corrupted time series is generated by adding the synthesized corruptions to the original signal as described.
We control the severity of corruptions at each level using a parameter triplet (A,p,) q). which re- spectively regulate the frequency, expected duration, and amplitude significance of the anomalies. The specific values for each level, summarized in Table were established through extensive ex- perimentation create progressive scale of difficulty, from Level 1 (mild) to Level 5 (frequent, prolonged, and high-magnitude corruption). We conducted extensive experiments with various parameter configurations determine these settings, which are presented in Appendix F2
EXPERIMENTS
We empirically evaluate the robustness of LGA using our proposed TSRBench on six standard forecasting datasets: Weather"] Electricity and ETT (Zhou et 2021 (ETTh1, ETTh2, ETTml, ETTm2). We integrate LGA into the PatchTST architecture (Nic et al. 2023), referred to as PatchLGA, by replacing its standard Scaled-Dot Product (SDP) attention. We compare PatchL.GA against the original PatchTST and other strong baselines, including TimeMixer (Wang al.) 2024). CATS (Kim) ct al. 2024), and iTransformer (Liu et al. 2024), to assess its effectiveness across various model types.
5.1 LONG-TERM TIME SERIES FORECASTING RESULTS UNDER REALISTIC CORRUPTIONS
We evaluate forecasting models on TSRBench across six corruption levels (0-5) with an input length of 512. Table 10 summarizes the results for combined spike and level shift corruptions, averaged
https://wwer http::/www..aggeeeeepmmm...eeeer
cs. www.ws.....eeeooooooooo.oooooee
7

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison of PatchLGA, PatchTST, and TimeMixer across different datasets under combined corruptions. Results show average performance across forecasting horizons (96, 192, 336, 720). We report MSE and MAE varying severity levels 0-5 (0 = original data). Lower values are better, with the best results highlighted in bold.
Dataset
ETTml
ETTm2
Weather
Model
PatchLGGA
PatcTTTT
TineMixer
PatchLGA
PatcTTT
TuneMiser
PatchLGA
PatchTST
TineMixer
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.351
0.379 0.352 0.382
0.360 0.390
0.257
0.316
0.256
0.317
0.259
0.322
0.227
0.265
0.225
0.264
0.227 0.266
0.359
0.385 0,359 0.389
0.366 0.396
0.264
0.326
0.264
0.328
0.265 0.330
0.238
0.284
0.239
0.284
0.242 0.288
Severity
0,372 0.397 0,375 0,402
0.385
0,409
0.276
0,335
0.277
0.338
0.279
0.341
0.264
0.313
0,266
0.314
0.275
0,320
0,519
0.468 0.614 0.507
0.594
0.499
0.304
0.355
0.308
0.360
0.310
0.364
0.301
0.338
0.306
0.339
0.326 0,350
0.617
0.526
0.695
0.558
0.716
0.560
0.356
0.395
0.361
0.400
0.373
0.409
0.361
0.389
0.369
0.384
0.441
0.421
0.734
0.577 0.839 0.613
0.837 0.613
0.421
0.428
0.431
0.434
0.459
0.450
0.454
0.423
0.491
0.423 0.576 0.464
Dataset
ETTh1
ET
Electricity
Model
PatchLGGA
PatcTST
TimeMixer
PatchLGA
PatchTST
TineMixer
PatchLGA
PatchTST
TineMixer
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.415
0.428
0.415
0.429
0,436
0.445
0.337
0.385
0.343
0.387
0.351
0,395
0.161
0.255
0.160
0.254
0.165 0.259
0.416 0.430 0.416 0.430
0.437
0.447
0.338
0.387
0.345
0.389
0.352
0,394
0.168
0.264
0.169
0.265
0.172 0.268
Severity
0.435
0.436
0.441
0.451
0,345
0.349
0.397
0.406
0.276
0.184
0.279
0.184
0,471
0,465
0.474
0.468
0.498
0,483
0,358
0,407
0.362
0.407
0,384
0,420
0.193
0.288
0.201
0.293
0,201
0.293
0.547
0.510
0.550
0.513
0.572
0.526
0.382
0.425
0.391
0.427
0.416
0,443
0.226
0.317
0.232
0.321
0.237
0.323
0.712 0.580
0.724
0.588
0.733
0.596
0.404
0.436
0.427
0.445
0.459
0.465
0.284
0.352
0.290
0.358
0.312 0.361
over four forecasting horizons. First, the results at Severity O serve crucial baseline to ensure that our proposed robust attention mechanism does not degrade performance on the original, clean data. This confirms that integrating LGA does not compromise the inherent forecasting capability of model on clean data. As corruption severity increases, PatchLGA consistently shows the most robust performance. Its advantage becomes more pronounced at higher severity levels; for example, at level 5, PatchLGA achieves a 12.3% MSE reduction on ETTml and is 21 1.2% lower than Tixer on the large Weather dataset. These results confirm the effectiveness of modeling local geometry for enhancing robustness, especially when data quality cannot be guaranteed.
5.2 COMPARISON WITH ALTERNATIVE ROBUST ATTENTION METHODS
We evaluated robust attention mechanisms on time series forecasting with corruptions. Our proposed LGA consistently outperforms both standard attention and other robust attention variants including MoM (Han et al. 2023) and Elliptical attention (Nielsen et al. 2024).
Table 3: Averaged MSE comparison ETTml I for LGA, SDP, MoM, and Ellip- tical attention. Lower values indicate bet- ter performance, with the best result for each severity level highlighted in bold.
MOM
GiB
Elliptical 3.
0.60
SDP
LGA (ours)
Model
PSt
Severity
SDP
MoM
Ellip.
LGA
0.50
Speed (s/tter)
06
I
0.359
0.375
0.360
0.359
2
0.375
0.397
0.374
0.372
Comparison of MSE training speed, and memory usage. Attention closer the lower-left corner achieves better trade- offs between accuracy and efficiency.
Figure 5:
3
0.614
0.670
0.722
0.519
4 0.695
0.871
0.755
0.617
5 0.839
1.016
0.880
0.734
As shown in Table 3 while robust attention mechanisms successful in vision and language tasks underperform compared to standard attention when applied to corrupted time series data, our LGA specifically designed for temporal structure exhibits the smallest performance degradation under corruption. Unlike Elliptical attention, which trades speed for memory efficiency without robustness gains (Fig. 5). LGA effectively leverages local geometry in time series while seeing a moderate increase in memory usage, yet maintaining training speed comparable to standard attention.
8

---

Published as a conference paper at ICLR 2026
5.3 ROBUSTNESS ANALYSIS ACROSS ATTENTION MECHANISMS
We conducted experiments to evaluate the effectiveness of LGA across various types of attention strategies. In time series transformers, two primary attention approaches exist: temporal attention, which focuses on relationships between time steps within each variable, and channel attention, which emphasizes relationships between different variables at each time step.
Table 4: Comparison of attention mechanisms on ETTml under severity levels. Values show MSE averaged across forecasting horizons (96, 192, 336, 720). Bold values indicate the better performance for each experiment.
Model
PatchTST
CATS
iTransformer
Noise Type
Combined
Level Shift
Spike
Combined
Level Shift
Spike
Combined
Level Shin
Spike
Atnn Type
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
0.359
0.359
0.359
0.359
0.352
0.352
0.345
0.345
0.344
0.345
0.340
0.338
01425
0.429
0.423
0.427
0.411
0.413
Severity
0,372
0.375
0.370
0.574
0.353
0.355
0.362
0.364
0.359
0.361
0.342
0.340
0.459
0.465
0.455
0.462
0.413
0.114
0.519
0.506
0.362
0.364
0.662
0.779
0.765
0.355
0.756
0.434
0.617
0.695
0.602
0.691
0.367
0.369
0,972
0.986
0.923
0.965
0.357
0.366
1.000
1.027
0.949
0,970
0.453
0,465
5
0.734
0.839
0.718
0.848
0.372
0.374
1,037
1.102
1,003
1.084
0.358
0.366
1.266
1,309
1.202
1.229
0.469
0.485
As shown in Table LGA enhances robustness across diverse attention architectures. PatchTST, with its temporal self-attention, consistently achieves the most significant performance gains. While CATS shows notable peak improvement of up to 17.1 %, its gains are less consistent, likely because its cross-attention operates on linearly embedded noisy inputs. iTransformer displays modest but stable improvements because it applies linear embedding to the entire time series, which disrupts the local geometry created by temporal periodic patterns. These results validate that LGA broadly applicable technique for improving model robustness, demonstrating its benefit for temporal self- self-attention, cross- cross-attention, and channel-wiss attention.
5.4 IMPACT OF INPUT LENGTH ON ROBUSTNESS UNDER REALISTIC CORRUPTIONS
To investigate the relationship between temporal context and model robustness, evaluated PatchLGA, PatchTST, and TimeMixer by varying input sequence lengths under corrupted con- ditions. The results in Figure demonstrate superior capability of PatchLGA to leverage long historical contexts for accurate forecasting in the presence of noise.
8.00
0.75
0.80
Treatber
Avg. Mee
4.00
0.75
0:30
0.69
5.55
0.55
522 Langth
132
332
3334
512
720
326
512
3024
(a)
(b) 192
H 336
(d) H 720
Figure 6: Performance comparison of different models on ETTml across varying input lengths under corruption. The y-axis shows average MSE (lower is better) for forecasting different horizons H. while the x-axis represents input sequence length.
Across all forecasting horizons, PatchLGA (red line) shows consistent and significant reduction in MSE as the input sequence length increases, achieving the lowest MSE in nearly all experimental settings. This highlights its strong benefit in utilizing extended temporal data to mitigate the impact of corruptions. In contrast, while PStchTST (green line) also benefits from longer sequences, its performance remains inferior to PatchLGA. The linear model, TimeMixer (blue line), fails to capitalize on increased context, with its performance stagnating or degrading. In conclusion, these results provide compelling evidence that PatchLGA is effectively designed to enhance forecasting accuracy by effectively processing long, noisy input sequences, underscoring its suitability for real-world applications.

---

Published as a conference paper at ICLR 2026
5.5 QUALITATIVE ANALYSIS OF ATTENTION GEOMETRY
To intuitive understand how LGA handles corrupted data, visualize the attention score distribution the query-key space. We utilize Principal Component Analysis (PCA) to project the key and query vectors from representative attention head into 2D space. The experiment conducted on the ETTml dataset under Severity Level 5 corruption.
SDP Q/K Distribution (8D PCA)
LGA Q/K Distribution (8D PCA)
Query (Comupted)
Query (Corupted)
Key (Corrupted)
Key (Corrupted)
Query
Query
Key
Key
(a) Standard Dot-Product (SDP) Attention
(b) Local Geometry Attention (LGA)
Figure 7: PCA visualization of attention scores under corruption. Point size indicates attention magnitude, and dark points denote corrupted regions. Unlike (a) SDP, which assigns high weights to noise. (b) LGA effectively suppresses these offifoldfod anomalies.
As observed Figure 7 SDP attention (a) is prone to distraction by outliers, assigning them large attention weights. In contrast, LGA (b) successfully suppresses these outliers, assigning them minimal attention weights due to their distance from the high-density data manifold. Consequently, by filtering out anomalies, LGA ensures the model remains focused intrinsic temporal patterns, leading to significantly more robust and accurate forecasting under severe corruption.
5.6 COMPUTATIONAL EFFICIENCY OF LOCAL GEOMETRY ATTENTION
While LGA introduces operations for metric prediction and Mahalanobis scoring, its practical computational over- head remains marginal. Because the Feed-For Ferd-Forward Net- work (FFN) governs the primary computational budget of a Transformer block, the relative cost of the attention mod- ule is significantly offset. As shown in Table 7, replacing SDP attention with LGA increases the theoretical Floating Point Operations (FLOPs) of a full block by only 11.7% during inference and 21.6% during training. This confirms
Table 5: Relative overhead of LGA com- pared SDP (Full Transformer Block).
Method
Inference
Training
SDP
24.78 G
73.99 G
LGA (Ours)
27.67 G
89.96 G
Overhead
+11.7%
+21.6%
that I.A effectively captures local data geometry and provides robust modeling with minimal impact on computational efficiency. comprehensive hardware-agnostic analysis and further algorithmic optimizations are detailed in Appendix B.2
CONCLUSION
We introduced Local Geometry Attention (LGA), an attention mechanism grounded in local Gaussian process theory that adapts to the intrinsic geometry time series data. To validate its effectiveness, we also developed TSRBench, the first standardized benchmark for evaluating models against realistic corruptions. Our experiments confirm that geometry- aware approach of LGA provides a substantial strong baselines, especially under severe conditions. While our efficient
robustness advantage matrix approximation can be explored further, this work provides powerful new framework and a critical evaluation tool for developing forecasting models ready for real-world deployment.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(No. RS-2025-25420430) This research was also partly supported by the Institute Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2022-I1220988. Development of Artificial Intelligence Technology for Personalized Plug-and-Play Explanation and Verification of Explanation), and the National Research Foundation of Korea (NRF) grant funded by
the Korean government (MSIT) (No. RS RS-20024-0038885)
REFERENCES
Tyrus Berry and Timothy Sauer. Local kernels and the geometric structure of data. Applied and Computational Harmonic Analysis, 40(3):439 469, 2016.
Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems, 33:1877- 2020.
Long Minh Bui, Tho Tran Huu, Duy Dinh, Tan Minh Nguyen, and Trong Nghia Hoang. Revisiting kernel attention with correlated gaussian process representation. In The 40th Conference on Uncertainty in Artificial Intelligence, 2024. URL http:::://ppeneeeew... net forum?io
x1 x1IKOvu 3MW
Wenlong Chen and Yingzhen Li. Calibrating transformers via sparse gaussian processes. In
The Eleventh International Conference on Learning Representations, 2023. URL https
//operrevie net./ forum?ide PPVAFFXH11
Hao Cheng, Qingsong Wen, Yang Liu, and Liang Sun. RobustTSF: Towards theory and design of robust time series forecasting with anomalies. In The Twelfth International Conference on Learning Representations, 2024. httts:/t:
net formm? wid=1ttgianniee
Jacob Devlin, Ming-Woi Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers). pp. 4171-4186 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly. et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.11929 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x words: Transformers for image recognition at scale. In International Conference Learning Representations, 2021. URL htttps://pennreet net / Eorumooriio.o YicbF yicbFaNTTV.
Xing Han, Tongzheng Ren, Tan Nguyen, Khai Nguyen, Joydeep Ghosh, and Nhat Ho. Designing robust transformers using robust kernel density estimation. Advances in Neural Information Processing Systems, 36:53362-53388 2023.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corrup- tions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
Pierre Humbert, Batiste Le Bars, and Ludovic Minvielle. Robust kernel density estimation with mfdian-of-means principle. In International Conference on Machine Learning, pp. 9444-9465. PMLR, 2022.
Guillermo Iglesias, Edgar Talavera, Ángel KonzzálezPrreeto, Alberto Mozo, and Sandra Gómez- Canaval. Data augmentation techniques in time series domain: a survey and taxonomy. Neural Computing and Applications, 3511))0001111303111 2023.
11

---

Published as a conference paper at ICLR 2026
Romain Ilbert, Ambroise Odonnat, Vasilii Feofanov, Aladin Virmaux, Giuseppe Paolo, Themis Palpanas, and levgen Redko. Sanformer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention. In Proceedings of the 41 International Conference on Machine Learning, volume 235. PMLR, 2024. URL https
//proceedings ml www.srrssss.
html
Nick Janßen, Melanie Schaller, and Bodo Rosenhahn. Benchmarking m-Itsf: Frequeney and noise-based evaluation of multivariate long time series forecasting models. arXiv preprint arXiv:2510.04900 04900, 2025.
Mark R Jerrum, Leslie G Valiant, and Vijay V Vazirani. Random generation of combinatorial Structures from a uniform distribution. Theoretical computer science, 43:169-1888 1986.
Dongbin Kim, Jinseong Park, Jaewook Lee, and Hoki Kim. Are self-attentions effective for time series forecasting? In Advances in Neural Information Processing Systems, volume 37, 2024.
Hyun-Chul Kim and Jaewook Lee. Clustering based on gaussian processes. Neural computation, 19 (1 (11):388-3107, 2007.
Alexander Lavin and Subutai Ahmad. Evaluating real-time anomaly detection algorithms the numenta anomaly benchmark. In 2015 IEEE 14th international conference on machine learning and applicatioons (ICMLA), pp. 38-44. IEEE, 2015.
Xiao Lin. Zhining Liu, Dongqi Fu, Ruizhong Qiu, and Hanghang Tong. Backtime: Backdoor attacks on multivariate time series forecasting. Advances in Neural Information Processing Systems, 37: 131344-131368, 2024.
Linbo Liu, Youngsuk Park, Trong Nghia Hoang, Hilaf Hasson, and Luke Huan. Robust multivariate time-series forecasting: Adversarial attacks and defense mechanisms. In The Eleventh International Conference on Learning Representations, 2023. URL httph:t///// UtLhttts:://ppeeeeess http:p/////peeeeeeee.eeeeeeee/ee
Id=ctmLBsé11TD
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long, itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024. URL httt:htttt//peeeeeeee fttt/oormmidddJeeP..
7e Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
R Thomas McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. 57th Annual Meeting of the Association for Computa- tional Linguistics, ACL 2019, pp. 2428-444.. Association for Computational Linguistics (ACL), 2020.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial nli: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4885-4901, 2020.
Yuqi Nie, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-tern forecasting with transformers. In The Eleventh International Confer- ence on Learning Representations, 2023. URL http::::://ppeeeeee..... htt.:/iwwwwww.eeeeeee.eee..eeeee wwwpee//fooumiiie JbdcOvTOco1
Stefan Nielsen, Laziz Abdullaev, Rachel SY Teo, and Tan Nguyen. Elliptical attention. Advances in Neural Information Processing Systems, 37: (000099199999990 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-67 2020.
Sebastian Schmidl, Phillip Wenig, and Thorsten Papenbrock. Anomaly detection in time series: a comprehensive evaluation. Proceedings of the VLDB Endowment, 17(9):1779-1797 2022.
12

---

Published as a conference paper at ICLR 2026
Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the support of nigh-dimensional distribution, Neural computation, 13(7):1443-147)
2001.
Haotian Si, Jianhui Li, Changhua Pei, Hang Cui, Jingwen Yang. Yongqian Sun, Shenglin Zhang, Jingjing Li, Haiming Zhang. Jing Han, et al. Timeseriesbench: An industrial-grade benchmark for time series anomaly detection models. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE), pp. 61-72. IEEE, 2024.
Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, and Christine Largouet. Anomaly detection in streams with extreme value theory. In Pruceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1067-1075, 2017.
Alex J Smola and Bernhard Schölkopf. Learning with kernels, volume 4. Citeseer, 1998.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pp. 10347-10357. PMLR, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Chengyu Wang, Kui Wu, Tongqing Zhou, Guang Yu, and Zhiping Cai. Tsagen: synthetic time series generation for kpi anomaly detection. IEEE Transactions on Network and Service Management, 19(1):130-145, 2021.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma. James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint arXiv:2405. 14616, 2024.
Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning. volume 2. MIT press Cambridge, MA, 2006.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 11122419-224344 2021.
Adam Wunderlich and Jack Sklar. Data-driven modeling of noise time series with convolutional generative adversarial networks. Machine learning: science and technology, 4(3):035023, 2023.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference learning representations, 2023.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11 11106-11115, 2021.
13

---

Published as a conference paper at ICLR 2026
A MATHEMATICAL SUPPLEMENT TO SECTION 3.2
This supplement provides a detailed theoretical foundation for our proposed Local Geometry Attention (LGA). We begin by framing our approach within the context locally weighted regression with generalized basis functions, which naturally leads to the local kernel-covarial matrix. We then present a theoretical bound on the predictive variance of our Local Gaussian Process model, justifying its use reliable surrogate for data density.
A.1 LOCAL BASIS FUNCTION REGRESSION
Locally weighted regression, or loess, is non parametric method that fits simple models to localized subsets of data. This approach highly effective for ordered data, like time series, as it can capture complex patterns without assuming global model structure. A prime example is the Seasonal- Trend decomposition based on Loess (STL.) algorithm, widely used for robustness time series decomposition.
Our work extends this framework by using the difference between key and query embeddings as generalized, data-driven basis functions. For each target point (which maps query vector q.). we solve a separate weighted least squares problem:
min 3(q
Kik, q-) [v: - (k; p(a)))))((a.)'
(10)
where (K:),== are the key vectors corresponding to the input data. The term (k; q.) serves as our basis function, and the weight K (k;, q.) is determined by a Gaussian kernel:
K (k;; q.) exp (-lik; q.11//2h?)
(11)
Let B(q.) ) be the T x regression matrix whose i-th row (k; - q.) T. and let W(r.) be the T diagonal matrix of normalized weights (xx.)::
K K(k;, q.)
wi(x.)
(12)
Σラ q.)
The solution to the weighted least squares problem is 9(q.) (oo.)=(((q.)) T.q.))(((... W(x.)y, where (xr,) is the local kernel-covariance matrix identical to Equation (T:
E(x.) B(q.) (W(,)
Σ (r(x)) q=)(k; - T.)T.
The covariance of the estimated parameters 3(q.), assuming observation noise with covariance Enbe, is given by "ه = (B WB)- W' W BB W B) If we make a simplifying assumption that the weights are chosen as the inverse of the observation covariance, W z Ed this expression simplifies to " as (BWB) kemel-covariance matrix E(x,) within probabilistic framework, which we term "Local Gaussian Process" as it effectively constructs GP conditioned on localized data neighborhood.
_xx,)-1 This connection motivates using the local
A.2 THEORETICAL BOUND ON PREDICTIVE VARIANCE
Theorem 1. Let IP be fixed but unknown probability distribution over a space H. with no atomic components and support contained within the wnit ball. For the predictive variance function (3 of our Local Gaussian Process, let 8² = maxke (k;) o2 (k) denote the maximun predictive variance over a data sequence of size T. Then, for any 0, with probability at least 1 8. the following inequality holds:
2 (p p+ log
11
P (k: 02 (k) 8² 22) < T
28
for some p (log(T)).
14

---

Published as a conference paper at ICLR 2026
Proof. The derivation is based on the method in Kim & Lee (2007) adapted framework. The predictive variance is given by:
9. (k) = (k - q.)T G(. q-),
(13)
where G(x.)= = of We define a weighted feature mapping (k,) = (,,)),,((,
q.) and design matrix whose i-th row (k;k) The local kernel-covariance matrix is then
E(x.) T
Using the Sherman- Sherman-Woodburg Ghoodbury-Morrisee formula, the predictive variance can be expressed as:
og (k) lokk, k) kg (K + o
where the weighted kernel is k(k, k.) = (kk (k'), K is the Gram matrix with entries kkk,, ky). and kg is vector with entries k:(k,, k).
The function h(k) k (K + I²I) ky can be shown to be linear in transformed feature space. Let C K I and w v T C-1-p
1-1 L. Then,
(k)) (1)) wr/(k) tr (k) w)
By defining a new feature map (k) = (k) )) we can write h(k) = w T V2(k), making it linear. Thus, o² (k) = kik, k) - l(kk is also linear in this feature space (assuming ki(k, k) is constant or slowly varying).
Applying the theoretical results from Smola & Schölkopf 1998); Schölkopf et al. 2001). we obtain that, with probability at least 8, and letting o milkke (k.) h(k) = k o². for all E > 0, we have:
2 P kk 0g. (k) > 0² +2=] - T p log
T
26
where
Ci log(c222
+ De log
(2T - 1)
+ +1)) +2,
+
g2
D3
and the constants and terms are defined as follows:
In(2)
2
103,
De De
[w
This finding indicates that the variance function of Local Gaussian Process serves as a reliable surrogate for capturing the support (or high density regions) of high-dimensional data distribution. Notably, the estimated support set retains computational tractability even in high-dimensional regimes.
B IMPLEMENTATION DETAILS OF LGA
This section provides a detailed description our Local Geometry Attention (LGA) implementation. While the theoretical foundation in Section 3.2 involves computationally intensive operations, we introduce an efficient approximation that maintains the core benefits of geometry-aware attention while being tractable for large-scale time series forecasting. The complete algorithm is presented in Algorithm
B. IMPLEMENTATION OF fo
As described in Section 3.4 we employ a neural network fo approximate the metric tensor G(q)). The network architecture is a simple feedforward design. For each head h. the input query vector is projected from its head dimension D, to higher-dimensional space De. passed through non-linearity, and projected back to Dr, produce the diagonal components of the metric tensor:
fo, (q) Softplus (W (G)))(
(14)
15

---

Published as a conference paper at ICLR 2026
Algorithm Local Geometry Attention (LGA)
Require: Query Q E BBL Key K E RBXL XD Value V E Bx
Require: Model dimension D, number of heads H, head dimension D,, D/H, noise variance o² Output tensor E R
Ensure:
function LGA(QQ, K, V)
2: Step 1: Linear projection and reshape to multi-head format
3: q. f Wo QQ), reshape to (B, LQ, H, D2)
4: k, Wx (K.) reshape (B, LK, Ht Da)
5:
v. f Wv (V), reshape to B, LK: H, Dn)
6:
Step 2: Predict metric tensor
7:
Apply fa. to i. :] for each head h
8:
G x
Stack predictions across heads
9:
if training then
Step 3: Estimate true metric and update prediction networks
11:
Gune + WWMMANETTRRMETTICCQ...e.o..
k, detach())
12:
Update (fe, } using LG
# Σ
D h=1 [ 1, h Guve [:, h|]
13:
end
14:
Step 4: Compute attention scores and aggregate values
15:
S < MAHALANOBISS
k,, G.detac(()
b [, Lq: H, Lx)
A + softmax(S) along last dimension
(B, LQ. H, Lk)
17:
O + -Av;
b Weighted sum
(B, LQ, H, D
Step 5: Reshape and output projection
19:
Reshape O to (B, LQ. H D2) and apply Wo
return output I 1 LgxD
20:
end function
22:
function MAHALANOBISSCORFIG k, G)
Efficiently computes Sij -kk q:) Tdiag (G; - q:) for all pairs
Cross term: C + (q G)k
(B, LQ. H. Lx)
Quadratic term: Q T)
b (B, Lqq H, L,k)
26:
Q return R xx
S - 2C
27:
28:
end function
29:
function ESTIMATETRUEMETRICI/.
30:
Sample queries and keys:
31: Sample N₂ indices from batch dimension
32:
Graal [ssmmpled] RRN.xLLxHHx
33:
Qand ~ (d(5 (0(--,,).. X.Q Dx
q. concat( Greal Arand) E 22,,, XL k, + k [ssampled] E 18 NRN, XHxDD
35:
Compute Gaussian kernel weights:
37:
For each query-key pair, compute squared distance: df = ||q - k2112
38:
Wi softmax, (-d,, scale)
> Normalize over keys: (2N.. LQ LK. H)
39:
Compute weighted local covariance:
40:
For each dimension d: Σi, Lx Wij (qi. - kj
b (2N,, LQ, , H, Dw)
41:
Compute inverse covariance:
G + +2+22)
Element-wise inversion: (2N LQ, H, Da)
43:
return Guue
end function
16

---

Published as a conference paper at ICLR 2026
where WT E PR PG and W E Da are learnable weight matrices. We use separate networks for each head to capture distinct geometric patterns. The GELU activation enables the learning of complex mappings, and the final Softplus activation ensures the positive definiteness of the metric tensor by guaranteeing its diagonal elements are positive.
The hyperparameter De controls the expressiveness of the geometric approximation, As detailed in the main paper, we typically set Dg to 8 times the head dimension D For a model with L layers and H heads per layer, the total number of parameters introduced by our metric prediction networks is 2 L. H Dn. DG. This represents a modest increase in model size but significantly enhances its ability to adapt to local data geometry.
B.2 EFFICIENT SCORE COMPUTATION
The computation of the LGA score, -(k - q) G(q)(k q), can be optimized for efficiency, Since we approximate ( as a diagonal matrix, the quadratic form expands to:
(k q) G() q) = G(q)k (la)k-20- 2q
T G(q)k q G(q)q
(15)
When these scores are fed into a softmax function, any term that is constant for given query across all keys can be dropped, as the softmax is invariant to constant shifts. The term q G(q)q is one such constant. Therefore,
simplify the computation to:
score q, k) k G(q)k + 2q G
(16)
This simplification, combined with the diagonal approximation G(q), significantly reduces the computational overhead. Since the metric tensor is learned by so and not explicitly estimated during inference, these optimizations make I.A both theoretically sound and computationally efficient for practical applications.
B.3 COMPUTATIONAL COMPLEXITY AND EFFICIENCY ANALYSIS
To provide a hardware-agnostic comparison of computational cost, we analyzed the theoretical Floating Point Operations (FLOPs) for the proposed L.GA compared to the standard Scaled Dot- Product (SDP) attention. The analysis was conducted under the experimental setting used for the ETTml dataset (B- = 128, N 64, 16, 8).
Hardware-Agnostic FLOPs Comparison. We provide granular breakdown of operations to quantify the overhead of LGA.
SDP: The cost is dominated by linear projections (8BNC2) and the attention score compu- tation 22
LGA: The additional cost stems from the metric prediction network (4BNH and the Mahalanobis score calculation (6BHN² N2 D). During training, an auxiliary cost for metric learning incurred (6.NHH N D) where N, 256 is fixed sample size.
Table 6 summarizes the total GigaFLOPs (G). Training FLOPs approximated as 3x Forward FLOPs for linear layers to account for backward passes.
Table 6: Theoretical FLOPs comparison between SDP and LGA.
Module
Inference (G)
Training (G)
SDP
9.748
28.890
LGA (Ours)
12.640
44.862
Full Transformer Block Analysis. While LGA incurs an overhead in the attention module specif- ically, its impact is significantly diluted when considering the full Transformer block (Attention + Feed-Forward Network), as the FFN dominates the computational budget. Table 7 presents the relative overhead.
17

---

Published as a conference paper at ICLR 2026
Table 7: Relative overhead analysis of LGA compared SDP.
Setting
Module Scope
SDP
LGA (Ours)
Overhead
Attention Only
9.748 G
12.640 G
+29.7%
Inference
Full Block (Attn + FFN)
24.78 G
27.67 G
+11.7%
Attention Only
28.890
44.862 G
+55.3%
Training
Full Block (Attn + FFN)
73.99 G
89.96 G
+21.6%
As shown, the effective overhead on the full model inference is only 11.7%. This confirms that LGA offers practical efficiency, providing robust geometry-aware modeling with a manageable computational cost. cost.
C EXPERIMENTAL SETTINGS
C.I DATASETS
We assessed our approach using six widely recognized time series forecasting datasets: Weather Electricity and ETT (Zhou et al., 2021) (ETThI, ETTh2, ETTm1, ETTm2). These datasets were selected for their diverse periodic patterns and challenging real-world prediction characteristics. Their inherent variability and natural irregularities make them particularly suitable for evaluating robustness under spike and level shift corruptions, these datasets already contain patterns similar those found in noisy real-world scenarios. While the traffic dataset is commonly used in benchmarks, we excluded due its extremely high dimensionality (862 features), Since PatchTST employs channel independence, different dimensions share manifold representations, making the high-dimensionall nature of traffic inherently challenging for unified manifold learning approaches. Complete dataset specifications are provided in Table All datasets are sourced from Wu et al. (2021).
Table 8: Details of 6 real-world datasets.
Datasets
Features
Frequency
Samples
Domain
Weather
21
10-min
52,696
Weather
Electricity
321
Hourly
17,544
Electricity
ETTh1
7
Hourly
17.420
Temperature
ETTh2
7
Hourly
17,420
Temperature
ETTml
7
15-min
69,680
Temperature
ETTm2
7
15-min
69,680
Temperature
C.2 HYPERPARAMETER SETTINGS
All experiments are conducted with 2 Intel(R) Xeon(R) Gold 6226R CPUs 2.90GHz and 4 NVIDIA RTX 4090 24GB GPUs and NVIDIA H100 80GB GPU. We conducted all experiments by following the original settings of PatchTST (Nie et al. 2023), as detailed Table In addition, we fixed the random seed to 2021 (Nie al. 2023) to enhance experimental reproducibility. For most datasets, the input length was set to 512 to thoroughly evaluate how effectively our proposed LGA method embeds periodic patterns over extended temporal horizons. However, for the ETTh and ETTh2 datasets, we adopted an input length of 336, as this was reported to yield stable performance in the original PatchTST paper. For comprehensive evaluation, we examined multiple forecasting horizons (96, 192, 336, and 720 time steps). The feed-forward network architecture follows standard practice, expanding the dimension to 256 before reduction, with GELU activation functions applied between linear transformations.
es usi http:///ww....a.ooo////////////a
18

---

Published as a conference paper at ICLR 2026
The synthetically generated query Sgan is created based on empirical observation that the components the real query vectors (Seal) typically ranged between approximately -5 and Therefore, generated the vectors in See by sampling each component from a uniform distribution, 4(-5, 5), to ensure the network is exposed to a diverse yet relevant region the representation space.
Regarding baseline models, for TimeMixer (Wang et al. 2024) experiments, we applied the publicly available hyperparameter settings that were originally optimized for input length 96, these settings demonstrated sufficiently strong performance even when applied to our longer input length of 512. Similarly, for the experiments presented Fig. O which examines performance across various imput lengths, we maintained these consistent hyperparameter configurations throughout all comparisons. For CATS (Kim al. 2024), we strictly adhered the original hyperparameter settings as published in the original paper.
For the iTransformer (Liu al. 2024) implementation, we noted that publicly available hyper- parameters were optimized solely for 96-length input. As our primary goal was to conduct fair comparison between the SDP attention and our proposed LGA, we adhered to these original settings for all iTrrnsffrmer experiments. This configuration consists of a 2-layer architecture with an embedding dimension of 128.
Table 9: Hyperparameter settings of PatchTST with an input sequence length of 512.
Datasets
Layers
Emhedding Size
# of Heads
Batch Size
o²
Input Length
Weather
128
16
128
10
512
Electricity"
128
16
32
10
512
ETTh1
16
4
128
10
336
ETTh2
16
4
128
10
336
ETTml
128
16
128
10
512
ETTm2
128
16
128
10
512
For weather and electricity, o² when H=7200.
D FURTHER EXPERIMENTS ON LGA
To supplement the primary findings presented in the main text, conducted further experiments to validate the key design choices of our proposed methodology and to demonstrate its practical effectiveness real-world data.
D.1 STABILITY ANALYSIS AND DETAILED PERFORMANCE
In the main text, we demonstrated the superior robustness of LGA based on average performance metrics, Here, we provide comprehensive breakdown of these results to validate the training stability and reliability of our proposed method. Table 1O presents the MSE and MAE across six benchmark datasets, reporting the mean and standard deviation computed over three independent runs with different random seeds.
A critical observation from these detailed results is the stability of the learning process. As indicated by the standard deviations (denoted by the + values), PatchI.GA consistently exhibits lower variance across different initialization seeds compared to the baselines, particularly under high severity levels (Severity 4-5). While standard attention mechanisms (PatchTST) often show increased performance fluctuation when exposed to severe noise, LGA maintains tight confidence interval. This suggests that the local geometry prior acts as an effective regularizer, guiding the model towards robust convergence regardless of random initialization. Consequently, LGA not only improves forecasting accuracy but also ensures predictable and reliable deployment in real-world scenarios where data quality is inconsistent.
19

---

Published as a conference paper at ICLR 2026
Table 10: Performance comparison of PatchLGA and PatchTST across six datasets under combined, level shift, and spike corruptions. Results are averaged across forecasting horizons (96, 192, 336, 720) and reported as Mean + Std over 3 random seeds. This detailed breakdown highlights not only the superior accuracy (lower MSE/MAE) but also the training stability (lower Std) of LGA compared to the baseline.
BanhiN Suah
Cousllied
Lace Sun
Spdler
Coushlaed
Spdlee
MBE
MSE
MAR:
MAl:
MIE
MAA
Mile
MAE
MAR
83531
0.500 403001 0,3617
0,356 0030 0.386 0650
0.358
0.325
(L5050 DU10
( 0515 0.389 5.609
0:354) EUE00
0:3839 (0001
037377 40.002 03985
0.TTER 4 0621 0.3971 0019
0:547
EURC3
6.383
(URC3
EUSTE2 A 2013
QUEDS a baed
( a 5 2012
6:394 4 (U001
0.5141
41.0000 04683
8.327
00001
0.882 00009
EUSSED A020
FUSDOG 2077
(US09T
E.071
(U000
0.7755
6242
ETT12
PSidiST
Meki
Cherkiadd
Love San
Spái
Crerkind
Levi Sun
Spdicc
Siare
MAA
MAE
12010 03313
23235
03139
UUSIST XIEDR
(LESST 8977 OUTEST
02712 LAURR
02549 L00300
ILZN22 & 0001
0U5GGA 2 0935
0.229 (URRE
033555
LUR004
022177 20.000
631997 LUAAND
03004 20038
(3347 110000
1,290 3-5 0000 2. 3.5 8915
2216 (ARRE
0:314
(UXeS 2.0
11204
2010
(UNEO 5 8000 0.356 CACLS
6:571 2 NAA0
6:2.6 200013
12909
ILENID a: 00.06 0.PA0 a
0
R015 2
(12333
Weather
PURLINT
Hi:k
Leve StaT
Syat
Cherkond
Spec
Siecc
MSC
MAR
XSE
MAF
MISE
NAP
MSE
MAR
NS0
MAR
MSE
MAP
AAL TOSAT
1ESS IOIO TESS LIOS
0.741 2077 03137
0:335
+ 89%
6675 O -
00000
0635 3991 H 6671 8337 DIEES
6.77)--IEe0 DrEes
5019
3 6055 + C4045
0.3755 D0000
BES0 8 6071
30 1911 C117
Comblied
Law Swn
Spla
MBE
MAA
MAC
MAR
MAR
SISE
MAA
MAE
MAE
MAE
BAUSE 200HI UADSS
210000
FUU 2 WARIS
E4335
BAEEY SU TUEE BOST TLRESY & BEZT 1 CUCA)
EAITY EE22 91 EHOKEE
RAINS MANOY
ILATOR 2 DOLO DLEND- R0.1
0A712 493004
0JI67 40.011
DUMA2 4 0032 0.4422 0000
DUEOS
0.336
BATED A0.5 OUICE A025
0.490 C000 4621 a C8E37 04397 EUEOS
0-325 FUOCS
0.000 40.05
0.5119 402000
ELESC A 0054 5.1 A 2.233
6-428 EUIVI 64407
FLSEES A. 3H0 OLSTEZ 9
A C 8044 0.502 CACA) 64225 IUKOR
04412 a INRGE
67788 400112
0,925
40000
PANTT 0.14 9,56 0033
0/0033
6.429
00002
(L7239 2077
6.855 a C.050
Splic
MSE
MAR
XBR
MAR
SEDE
MAB TOROS
MIE
MAA
MAE
MSE
MAB
TUSES
TTRE- TOROTS
+I SOVE TOR SOW IS FINE BOO IF TEY CEEC
SUOG 2013 (N 117 6070
833 1815
(USGE
2 BEST (142002 BED)
0.3422
(UEN 03955 2 LUR04
DAST (LDD6 ME (UIS0 BS99 1 1.40EEER CICHI
63425 2UER)
03931 200022
04143 10000
04413 140354
(UBRO 3 ROOS BL4K0 3 1631
LUEMS
03597 2337/20AN SUAMN
HAIS5 20
ULALS 10000
(14142 O 8072 0.470 CACAS 2342/ 239R22
Spiu
Mal
[.1034 400000 0,2799 A.L.00
0.1084 A DOST 0.250 00000
3.154
DUEDOO
0:339
ELICOO AOO
(L224) A000
(LIONO 0833 0.551 EEC0
EUEEGO
0.541 4UD0E
0Jr00 ANDK0 0.245 A1.200
ELIGTT A 0036
2652 A (009
3823 DUERO
023% DUECE
(L2448 0.00
OLOORT A 8005 I 2.2544 a C4CO4
02350 a JUXKO
EL7766 A 2013 2722 A 1020
02325
ELEE29 2010010 ILET8R A.0000
ILIMO A 8.102
0.355 +0305 033666
07166
6016
DUE00 0-7657
(1AAE 1910 (11017 9991
(02022- -1 66755 1157- -
1643-6906
6/817-0000 6009
(1.66 - 0691 0 720 8 6611 1731 -iies 6.796--- DiRes
(1000 9995 11 + p0rt (1767T 6095 0 3 (63)) 0 6175--00e1
0.333 -0000
Table 11: Detailed performance comparison of Samformer Ilbert et al. (0044) and PatchSamformer variants on the ETTml dataset with input length 512 under combined corruptions. PatchSamformer denotes the application of patch embedding to the Samformer architecture. While patching induces significant vulnerability to noise in the linear Samformer model (SDP), LGA successfully stabilizes the architecture, achieving the lowest error rates across varying severity levels.
ETTul
H
96
192
336
720
Med.
Atten.
Metric
Sumformen
L.A
MSE
0.319
0.336
0.510
0.659
0.802
0.354
0,309
0.522
0.655
0.767
0.378
0.393
0.533
0.653
1.755
1.429
0,442
0.564
0.675 0.755
0.363
0.456
0.501
0.394
0.519
0.308
0.477
0.992
0,409
0.560
SDP
MSE
0.327
0.348
0.534
0.680
0.860
0.357
0.573
0.527
0.660
0.796
(1383
0.399 0,540
1.648
0.776
0.453
0.448
0.589
0.700 0.804
0.382
0.400 0,565
0.619
0,396
0.544
0.997
0.482
0,543
0.598
0.437
0.510 0.599
Pluch
Sunformmn
E.G
MAE
MSR
0.332
0.367
0.381
0.350
0.437
0.459
0.500
0.577
0.715
0.561
0.363
0,383
0.394
0.576
0.448
0.009
0.513 0.557
0.586
0.673
(1380
0.396
0.405
0.392 0.489
0:455
0,505
13576
0.656
0:547
0.424
0.453
0.431 0.482 0.529 0,566
0.443
0.533
0613 0.685
SDP
MSF
0.573
0.40
0.852
1.171
1,465
0.371
0.401
0.779
0.978
1.340
(1403
0951
0.458
0.445)
0.786
0.922 1,100
MAB
0.407
0.430
0.599
0.708
0,783
0.408
0.427
0.500
0.006
0.723
01423
0.441
0.628
0.711
1.776
01436
01450
01586
0.054
0.688
D.2 ADDITIONAL. EXPERIMENTS ON OTHER MODELS
To demonstrate the broad applicability of our method beyond standard Transformer architectures, we extended our evaluation to Samformer (Ilbert et al., 2024), a state-of-the-art model distinguished by its linear complexity and channel-wise attention mechanism. In this experiment, we analyzed four model variants on the ETTml dataset: the original Samformer, Samformer with LGA, and their respective "Patch" versions where the input time series is processed via patch embedding. The detailed results across all forecasting horizons and corruption severity levels are summarized in Table
20

---

Published as a conference paper at ICLR 2026
The experimental results highlight critical interaction between the model's structural bias and the input representation. A notable finding is the severe vulnerability introduced by patching in linear architectures. As observed in the "PatchSamformer SDP" results, applying patch embedding to Samformer leads to catastrophic performance degradation under realistic corruptions. For instance, at the shortest horizon (H 96) with Severity 5, the MSE soars to 1.465. We attribute this instability to the architectural design of Samformer; unlike conventional Transformers, it does not incorporate non-linear Feed-Forward Network (FFN) after the attention block. Consequently, the high-dimen ligh-dimensional linear projections inherent to patching appear to amplify input noise without the filtering or buffering effects typically provided by non- non-linear layers, making the model hypersensitive to corruptions.
However, replacing the standard attention with LGA in this vulnerable architecture completely reverses the degradation. The "PatchSamformer LGA" variant not only recovers from the failure observed in the SDP counterpart but also achieves the highest robustness among all compared models. For example, at the longest forecasting horizon (H 720) under maximum corruption (Severity 5), LGA reduces the MSE from 100 (SDP) to 0.685, significantly outperforming even the original Samformer baseline (0.804). This empirical evidence suggests that the local geometry prior of LGA acts as a potent regularizer, effectively stabilizing the attention mechanism and preventing overfitting noise, even in high-dimensional feature spaces that lack non-linear protection. These findings confirm that LGA is a versatile and critical component for enhancing robustness across diverse architectural paradigms.
D.3 EVALUATION ON SYNTHETIC BENCHMARKS
To rigorously validate the robustness of PatchLGA under theoretically controlled conditions, we conducted additional experiments utilizing the synthetic benchmark framework proposed by Janßen et al. JanBen et al. (2025). Unlike TSRBench, which injects corruptions into real-world data, this framework generates purely synthetic multivariate time series with parameterizable frequency and noise characteristics.
D.3.1 EXPERIMENTAL SETUP
Following the protocol in Janßen et al. 2025). we generated synthetic datasets across 7 distinct frequency bands, ranging from "Very Low' to Veryy High'. For each frequency band, we constructed multivariate dataset by combining three signal types (Sine, Smooth Square, and Smooth Sawtooth) with five distinct noise types (White, Brownian, Impulse, Trend-dependent, and Seasonal-dependen)) under four SNR levels.
Consistent with our main experiments, the models were trained with an input sequence length of 512. We evaluated the forecasting performance across four prediction horizons H E (96, 192,336, 720). This setup allows us to assess the model's capability to recover intrinsic signal geometry from noisy inputs across varying temporal scales.
D.3.2 RESULTS
Table 12 baseline.
presents the detailed performance comparison between PatchI.GA and the PStchTST
Table 12: MSE comparison of PatchI.GA and PatchTST on the synthetic benchmark by Janßen et al. (2025). Models were trained with an input length of 512 and evaluated across four prediction horizons H [96, 192, 336, 720). PatchLGA demonstrates superior accuracy (lower MSE) compared to the baseline across most frequency bands.
Freq.
Very High
High
Mid-High
Mid
Low-Mid
Low
Very Low
Attn Type
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
SDP
LGA
96
0.070
0.070
0.106
0.102
0.114 0.111 0.098
0.098
0.091
0.090 0.082
0.080
0.143
0.138
Horizzn
192
0.070
0.070
0.105
0.101
0.114
0.111 0.098
0.098
0.091
0.090
0.082
0.080
0.143
0.138
336
0.068
0.068
0.101
0.097
0.111
0.108
0.089
0.089
0.083
0.083
0.085
0.083
0.140
0.136
720
0.062
0.062
0.090
0.086
0.108
0.105
0.072
0.072
0.066
0.066
0.097
0.095
0.128 0.124
21

---

Published as a conference paper at ICLR 2026
The results indicate that PatchLGA consistently outperforms matches PatchTST across different frequency characteristics and forecasting horizons. Notably, (JanBen et ( highlight that models typically degrade in the 'Very Low' frequency band because the lookback window may not capture complete periodic cycles. this challenging regime, PatchLGG achieves a distinct improvement (e-g., reducing MSE from 0. 0.143 0.138 H 96), suggesting that LGA's manifold learning capability effectively captures intrinsic geometric structures even when temporal periodicity is locally ambiguous. Furthermore, LGA maintains superiority in 'High' frequency bands, demonstrating its versatility in handling rapid fluctuations contaminated by complex noise.
Combining these findings with our TSRBench results, we conclude that LGA offers robust perfor- mance in both real-world data with injected anomalies (TSRBench) and theoretically controlled
synthetic environments (Janßen et (025 ), validating its generalizability.
D.4 IMPACT OF QUERY SET SELECTION ON PERFORMANCE
We conduct ablation study validate choice of using combined query set (Spoth) for training the metric prediction network, as detailed in Section 3.4, As shown in Table [3] using the combined of real and synthetic queries consistently outperforms using either set alone, especially under severe corruption. This result confirms that a broader query coverage is crucial for robustly learning the local geometry of the data manifold.
Table 13: Ablation study on the query set selection for ETTm1, showing average MSE and MAE across all forecasting horizons. Lower values are better, with the best results highlighted in bold.
Severity
2
5
Query Set
Sooth
Sge
Syeal
Sooth
Sym
Syual
South
Sym
S
South
Sye
S
South
Syn
Syeal
MSE
0.359
0.358
0.360
0.372
0.371
0.374 0,519
0.524
0.525
0.617
0.630
0,626
0.734
0.752
0.744
0.385
0.387
0.386
0.397
0.398
0.398
0.468
0.473
0.471
0.526
0.534
0.531
0.577
0.586
0.582
D.5 VALIDATION ON REAL- WORLD ANOMALOUS DATA AND THE NECESSITY OF A PRINCIPLED BENCHMARK
To complement the results from our TSRBench, conducted additional experiment a dataset containing genuine, unscripted anomalies. This validation aims to demonstrate that the robustness of LGA extends beyond controlled, synthetic corruptions to the unpredictable nature of real-world operational data.
For this purpose, we used the AWSCloudWatch dataset from the widely-recognized Numenta Anomaly Benchmark (NAB) (Lavin & Ahmad 2015). This univariate time series represents AWS EC2 server CPU utilization and includes labeled periods of naturally occurring anomalous behavior. For the experiment, we used the same hyperparameter settings as the ETTml experiments, with an input sequence length of 1024 and forecasting horizon of 96.
Table 14: Performance the AWS CloudWatch dataset. The lower MAE indicates superior performance on real-workd anomalous data.
Model
MAE
PatchTST
7.97
PatchLGA (Ours)
6.22
The results in Table 14 show that PatchLGA significantly outperforms the baseline, achieving a 22% reduction in MAE. This provides strong empirical evidence that local geometry- approach of LGA is highly effective in mitigating the impact of genuine, real-world anomalies.
However, it is important highlight that conducting such validation for forecasting robustness is often infeasible with most publicly available datasets. A fair and rigorous evaluation requires several critical conditions to be met:
22

---

Published as a conference paper at ICLR 2026
Availability of precise anomaly labels: To verify that model is robustly forecasting the "normal" underlying pattern, the evaluation metric (e.g., MSE, MAE) must be calculated on ground truth that excludes the anomalous periods. Most forecasting benchmarks do not provide such granular labels.
No time lag anomaly labels: Even when labels are available, a time lag between the actual anomaly occurrence and its timestamp in the data makes it impossible to accurately identify which imput segments are corrupted and which future segments should be excluded from evaluation. This limitation renders most anomaly detection datasets unsuitable for this specific validation purpose.
These strict requirements underscore the challenge of using "in-the-wild" datasets for reproducible robustness research. In such scenarios, our TSRBench becomes particularly useful as it enables a controlled and systematic evaluation of robustness. By providing clean training data and systemati- cally corrupted test sets with known corruption boundaries, TSRBench allows for comprehensive robustness assessment that would be difficult, if not impossible, to achieve with purely real-world datasets where the nature, extent, and precise timing of corruptions are unknown and uncontrolled.
E ADDITIONAL EXPERIMENTAL RESULTS AND ANALYSIS
In Section 5 we provided averaged MSE results across forecasting horizons to demonstrate the robustness of PatchL compared to baseline models. This section presents the complete experimental results across all datasets, forecasting horizons, and corruption scenarios. These detailed results not only validate our main findings but also provide deeper insights into how different architectures respond to various levels and types of corruptions under specific forecasting conditions.
E.1 DETAILED PERFORMANCE ANALYSIS OF LONG-TERM FORECASTING UNDER REALISTIC CORRUPTIONS
We provide the full results of Section 5.1 for each dataset (ETTm1, ETTm2, Weather, ETTh1, ETTh2, and Electricity) across all forecasting horizons in Table 15 and T6 These detailed results expand on the average performance presented in Table 10
The complete dataset results demonstrate several important patterns. While PatchL generally outperforms baselines across most corruption types, the performance differences vary by corruption type. For level shifts, PatchLGA consistently shows superior robustness across most datasets and forecasting horizons. For spike corruptions, all models demonstrate relatively less performance degradation compared to other corruption types, with both PatchLGA and PatchTST maintaining reasonable robustness. However, the most significant finding appears in combined corruption scenar- ios (both spikes and level shifts occurring simultaneously). In these realistic cases, PST and TimeMixer often exhibit performance degradation considerably exceeding what would be expected from the individual corruption types alone, suggesting a compounding effect. PatchL-G, in contrast, maintains more consistent performance even under these challenging combined corruptions.
E.2 EXTENDED EVALUATION OF ATTENTION MECHANISM ROBUSTNESS
In this section, we provide more detailed analysis of the effectiveness of LGA across different attention mechanisms. Table 17 presents the complete performance results across all forecasting horizons and severity levels for three representative attention mechanisms: self-temporal attention (PatchTST), cross-temporal attention (CATS), and self-channel attention (iTransformer).
The detailed results in Table 17 confirm and extend our primary findings. A consistent trend across all architectures is that the performance gains from LGA become increasingly pronounced as corruption severity increases. For lower severity levels (1-2), the improvements are often modest, but as corruptions intensify (levels 3-5), the ability of LGA to adapt to the local data geometry provides clear and substantial advantage. The analysis reveals distinet patterns based on the attention type:
Self-Temporal Attention (PatchTST): This mechanism consistently demonstrates the greatest performance improvements when enhanced with LGA. The benefits are particularly significant for longer forecasting horizons under severe comruption. For example, severity
23

---

Published as a conference paper at ICLR 2026
Table 15: Full experimental result on ETTml, ETTm2, weather with 512 input lengths.
ETTml
Combined
Level Shift
Spike
Model
PatchT
PatchTST
VixeVixer
PatchT.GA
PStchTST
TimeMixer
PutchI.GA
PatchTST
TimeMixer
H Metric
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.297
0.353 0.304
0.357
0.307
0.363
0.297
0.352
0.304 0.356 0.306 0.363
0.289
0.345
0.293
0,346 0.300 0.357
0.314 0.368 0.326
0.377
0.335 0.383
0.312 0.366
0.325 0.576
0.332
0.380
0.288
0.346
0.294
0.348
0.303 0.359
6P
0.527 0.456
0.625 0.497
0.619 0,490
0.511
0.447
0.616
0.490
0.602
0.481
0.302 0.354
0.310
0.361
0.314 0.368
0.657
0.530 0.743 0.566
0.769
0,558
0.641
0.519
0.746
0.563
0.742
0.544
0.309 0.363
0.318
0.369
0.323 0.375
5
0.820 0.594 0.944 0.636
0,953
0.629
0.802
0.584
0.954
0.634
0.924
0.615
0.316 0.368
0.326 0.375
0.329 0.379
0.341
0.576 0.342 0.378
0.342 0.380
0.340
0.376
0.341
0.578
0.341
0.380
0.333
0.370
0.335 0.372
0.337 0.376
0.354
0.388 0.357 0.391
0.355 0.393
0.351
0.386
0.355
0.390
0.356
0.391
0.334
0.371
0.336 0.373
0.338 0.377
192
0.504
0.462 0,581 0.491
0.537 0.474
0.491
0.455
0.574
0.486
0.525
0.467
0,343
0.378
0.346
0.380
0,346 0.384
0.603 0.521 0.661
0.542
0.643
0.527
0.587
0.511
0.657
0.538
0.625
0.517
0.348
0.384
0.349
0.385
0.351 0.390
0.722 0.573
0.812
0.601
0.740
0.574
0.707
0.564
0.823
0.601
0.725
0.565
0.353
0.367
0.354
0,389
0.392
0.372 0.392
0.370 0.397
0.381
0.407
0.372
0.391
0.570
0.397
0.381
0.407
0,366
0.387
0.364
0.392
0.376 0.403
0.385 0.402 0.383
0.408
0.339
0.418
0.383
0.400
0.382
0.407
0.395
0.417
0.368
0.388
0.365
0.393
0.377 0.403
336
0.502 0.465
0.630
0.519
0.600
0.511
0.490
0,458
0.624
0.514 0.590
0.506
0.375
0.394
0.375
0.400
0.383 0.409
0.586 0.516 0.692
0.563
0.709
0.567 0.570
0.506
0.688
0.559
0.692
0.558
0.379
0.399
0.379
0,405
0.390 0.415
0.686
0.564
0,818
0.611
0.829
0.622
0.670
0.554
0.831
0.612
0.814
0.614
0,383
0,402
0.382
0.408
0.395 0.418
0.425
0.421
0.420 0.424
0.436
0.432
0.:455 0.420
0:420
0:424
0:435
0.431
0.420
0.417
0.416
0.421
0:430 0.427
0.436
0.429 0.435
0.432
0.449
0.441
0.435
0.428
0.432
0.431
0.447
0,439
0,421
0.417
0.416
0.421
0.431
0.428
720
0.543
0.622
0.519
0.620
0.533
0.482
0.605
0.515
0,423
0.425
0.426
0,442
0.436
0.623
0,536
0.684
0,562
0,745
0,587
0.609
0.526
0.673
0.554
0.722
0.575
0.431
0.427
0.430
0,430
0.447
0.442
5
0.709
0.578
0.604 0.782
0.828 0.626
0.694 0.568
0.785 0.601 0.806 0.614
0.435
0.430
0.435
0.433
0.452 0.444
ETTm2
Combined
Level Shift
Spike
Model
PatchLGA
PatchTST
TimeMixer
PatchLGA MAE
PatcTTST
TimeMixee
PatchLGA
PachTTST
TimeMixer
11 Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.177
0.269
0.178
0.270
0,181
0.273
0.176
0.267
0.177
0.269
0.180
0.272
0.168
0.258
0.167
0.257
0.173 0.263
0.194 0.283
0.193 0.283
0.196 0.286
0.191
0.280
0.191
0.281
0.194 0.283
0.169
0,260
0.169
0.260
0.174 0.265
96
0.232 0.308
0,236
0,311
0.234
0.313
0.228
0.304
0.233
0.307
0.229
0.307
0,171
0.263
0.171
0.263
0.176 0.269
0.302
0.358
0.304
0.359
0.305
0.362
0.299
0.353
0.302
0.356
0.296
0.355
0.175
0.267
0.175
0.267
0.180 0.273
0.396
0.399
0,400
0.401
0.409
0.410
0.388
0.343
0.394
0.400
0.270
0.180
0.270
0.277
0.231
0.305
0,232
0.308
0,234
0.309
0.230
0.304
0.231
0.307 0.233
0.309
0.224
0.296
0.224
0.297
0.228
0.301
0.244
0.316
0,246
0.319
0.248
0.321
0.242
0.313
0.244
0.316 0.247
0.318
0,226
0.298
0.226
0.300
0.229
0.302
192
0.273
0.337
0,280
0.343
0.281
0.346
0.269
0.332
0.276
0.339
0.278
0.342
0.228
0.301
0.222
0.303
0.231 0.305
0.327
0.379
0.336
0.386
0.344
0.321
0.374
0.332
0.382
0.339
0.389
0.230
0.304
0.230
0.306
0.233
0.307
0.395
0.414
0.411
0,422
0.437 0.438
0.387
0.407
0.406
0.417 0:427
0.432
0,234
0,308
0.234
0.310
0,236 0.311
0.283 0.338
0.280
0.339
0.287 0.347
0.282 0.338
0.279
0.338 0.287
0,346
0.279
0.331
0.275
0.331
0.284 0.339
0.294
0.347 0,292
0. 0.348
0,302 0.357
0.292
0,344
0.289
0.346
0.300
0.355
0.280
0.333
0.276
0.352
0.285 0.340
336
0.317
0.368
0.335
0.312
0.360
0.332
0.379
0.279 0.335
0.286 0.343
0.363
0.402
0,364
0,406
0.409
0.432
0.358
0.397
0.359
0.401 0,408
0.431
0,284
0.339
0.280 0.358
0.286 0.343
5
0.419
0.432
0.421
0.435
0.505
0.476
0.411 0.426
0.414 0.430
0.502
0.474
0.287
0.342
0.284 0.342
0.287 0.346
0.366
0.390 0.368
0.394
0.360
0.393
0.365
0.389
0.367
0.393
0.359
0.392
0.362
0.384
0.362
0.386
0.356 0.387
0.374
0.396 0.376
0.400
0.370
0,400
0.372
0.394
0.374
0.398
0.368
0.398
0,363
0.386
0,364
0.388
0.357 0.388
720
0.392 0.411 0.399
0.418
0.391
0.415
0.388
0.407
0.394
0.414
0.387
0.412
0.365
0.388
0.366
0.391
0.360 0.391
0.41 0.441
0.441
0.450
0.435
0.447
0.425
0.436
0.435
0.445 0.432
0.444
0.367
0.391
0.368
0.394
0.359 0.392
0.477 0.475 0.466 0.491
0.485
0.475
0.466
0.460
0.482
0.480
0.470
0.370
0.394
0.372
0.397
0.395
Weather
Combined
Lovel Shift
Spike PaccTTST
Model
PatchLGA
PatcTTST
TimeMixer
PatchLGA
PatchTST
TimnMiixrr
PatchLGGA
TimeMixer
H Meuic
MSE
MAE
MSE
MAE
MSE
MSE
MAE
MSE
MSE
MAE
MAE
MSE
MAE
MSE
MAE
0.165
0.224
0.167
0.224
0,161
0.317
0.164
0.221
0.165
0.222
0.160
0.215
0.150
0,204
0.150
0,202
0.149
0.201
0.197 0.256
0.202
0,259
0.189
0.244
0.195
0.251
0.199
0.254
0.187
0.240
0,153
0.208
0.153
0.206
0.150 0,203
96
0.244 0.285
0.256 0.288
0.240
0.272
0.241
0.278
0.252
0.281
0.235
0.267
0.155
0.211
0.155
0.210
0.152 0.206
0.309
0.343
0.323
0.338
0.349
0.306
0.333
0.310
0.324
0.333
0.323
0.161
0.224
0.160
0.221
0.155
0.214
0.407 0.578
0,453
0.378
0.509
0,381
0.401
0.366
0.429
0.359
0.483
0.365
0,166
0,230
0.164
0.227
0.158
0.219
0.209 0.264 0.209 0.263
0.206
0.262
0.2088
0.262
0.208
0.261
0.205
0.260
0.196
0.245
0.198
0.247
0.192 0.244
0.239 0.296 0,234
0.292
0.239
0.293
0.236
0.291
0.232
0.288
0.236
0.289
0.197
0,248
0,200
0.249
0.194 0.246
192
0.278 0.321 0.279
0.321
0.294
0.325
0.273
0.314
0.276
0.316
0.288
0.319
0.200
0.252
0.201
0.252
0. 096 0.249
0.340 0.374 0,358
0.370
0.400
0,392
0,330
0.360
0.346
0.359
0.383
0.378
0,204
0,263
0.203
0.261
0.199 0.258
0.477 0.403 0.512 0.418
0.544
0.437
0.402
0.385
0.487
0.403
0.516
0.419
0.209
0.269
0.206
0.266
0.202 0.263
0.257 0.301 0.225
0. 0.303
0.260
0.306
0.256
0.299
0.258
0.301
0.259
0.304
0.248
0.285
0.246
0.284
0.247 0.288
0.277 0.326
0.287
0.334
0.292
0.339
0.275
0.323
0.285
0.331
0.289
0.335
0.249
0.288
0.248
0.287
0.248 0.290
336
0.307 0.350
0.323 0.359
0.339
0.369
0.303
0.344
0.320
0.354
0.333
0.363
0,251
0.290
(1.251
0.291
0.250 0.292
0.358
0.396
0.375
0.398
0.443 0.437
0..447
0.383
0.367
0.388
0.421
0.420
0.253
0.299
(0.252
0.298
0.253 0.303
0.448
0.431
0.486
0.434
0.565
0.479
0..228
0.414
0.471
0.421
0.531
0.459
0.257
0.304
00.255
0.303
0.258 0.309
0.322
0:347
0.321
0.346
0.343
0.366
0.322
0.345
0:320
0.345
0.342
0.363
0.316
0.336
0.312
0.354
0.325
0.342
0.342
0.372
0.339
0.370
0.382
0.403
0.339
0.367
0.337
0.367
0.378
0.398
0,318
0.338
(1,314
0.336
0.326
0.345
720
0.376
0.397
0.365
0.390
0.431
0.434
0.372
0.390
0.361
0.385
0.424
0.426
0.320
0.341
0.316
0.338
0.328
0.347
0.435
0.442
0.419
0.430
0.572
0.518
0.421
0.426
0.408
0.417
0.551
0.501
0.315
0.346
0.360
0.545
0.479
0.512
0.462
0.685
0.55%
0.516
0.458
0.489
0.444
0.655
0.538
0.321
0.353
0.317
0.349
0.335
0.365
24

---

Published as a conference paper at ICLR 2026
Table 16: Full experimental result on ETTh1, ETTh2, electricity with 512 input lengths.
ETThi
Combined
Level
Spike
Medel
Patchi.GA
PST
TimeMixer
PatchIGGA
PatchTSV
TimeMixer
Patchi-GA
PST
TimeMixer
H
Metric
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAB
MSE MAE
MSE
MAB
MSE
MAB
MSE
MAB
MSE
MAE
0.373 0.399 0,370 0.397
0.376 0.402 0.372 0.399
0,370 0.396 0.376 0.402
0,371 0.397 0.368
0.394
0.374 0.400
0.380 0.406 0:377 0.404
0.383 0.410 0.379
0.405
0.376 0.402 0.383 0.408
0.371 0.398
0.368
0.395
0.374 0.401
0.440 0.441 0.444
0.441
0.443
0,445 0.434
0.436
0.438
0.436 0.438
0.440
0.376 0,402 0.375
0.400
0.378 0.405
0.499 0.478 0.505
0.480
0.512
0.486 0.484
0.468
0.492
0.472
0.496
0.476
0.383
0.408 0.385
0.407
0.387 0.411
0.669 0.547 0,700
0.557
0.687
0.559
0.636
0.530
0.676
0.544
0.652
0.540
0.399
0.417 0.407
0.418
0.401
0.421
0.417 0.425 0.414
0.425
0.440
0.417
0.424
0.424
0.434
0:440
0.415
0.423 0.413
0.423
0.432 0.438
0.422 0.430 0:421 0.431
0:438
0.444 0.422
0.429
0:420
0:430 0:438
0.444
0:415
0:423 0.414
0.424
0:432 0.439
192
0.467 0.459 0.474 0.465
0.480
0,470 0.463
0.455
0.469
0,460
0.476
0.466
0:418
0.426
0.419
0.428
0.434 0.441
0.533 0.500 0.536 0.504
0.538
0.506 0.521
0.491
0.526
0.498 0.527
0.500
0:425 0.432 0.427
0.434
0.441 0.445
0.693 0.568 0.720 0.586
0.678
0.570 0.668
0.554
0:707
0.577 0.653
0.556
0.436 0.440 0.440
0.444
0.450
0.452
0.427 0,452 0.434 0.437
0.464
0,463 0.427
0.432
0.434
0.437 0.464
0.463
0.427
0.432 0.433
0.437
0,463 0.462
0.431 0.436 0.437
0.442
0.469
0.467 0.430
0.435
0.437
0.440 0.467
0.465
0.427 0.432 0.434
0.438
0.464 0.463
.316
0.476 0.465 0.480
0.469
0.542
0.507 0.471
0.460
0.474 0.464
0.529
0.499
0.432 0.436 0438
0.441
0.474 0.469
0.554 0.513 0.558
0.516
0:619
0.553 0.540
0.503
0.543
0.507 0.591
0.537
0.440
0.442 0.447
0.448
0.489 0.479
0,717 0.584
0.712
0,586
0,796
0.631 0.689
0.568
0.685
0,570
0.745
0.605
0.450
0:451 0.458
0.456
0,508 0.492
0.446 0.463
0.445
0.463
0.475
0.480 0.446
0.463
0.446
0.463
0.475
0.480
0.447
0.463
0.446
0.463
0.475 0.480
0.467
0.449
0.467
0.483 0.448
0.466
0.448
0.466
0.476
0.482
0.463
0.446
0.464
0.481
720
0.500 0.496 0.499
0.497
0.527
0.510
0.495
0.493
0:494
0.493
0.521
0.506
0:450
0.466
0.450
0.467
0,480
0.483
0.601 0,549
0.600
0,551
0.620
0.558
0.579
0,537
0.578
0.538
0.597
0.546
0.466
0.476
0.467
0.477
0.497
0.493
0.622 0.768 0.621 0.765
0.770 0.623 0.730
0.601
0.727
0.603
0.728
0.603
0.479
0.486
0.481
0.488
0.512 0.503
ETTh2
Combined PaccTTST
Lovel Shift
Spike
Model
PatchL.GA
TimeMixer
PatchLGA
PaccTTST
TimeMixer
PatchLGA
PatcTTST
TimeMixer
11 Metric
MSE
MAE
MSE
MAG
MSE
MAE
MSE
MAG
MSE
MAD
MSC
MAG
MSE
MAD
MSC
MAG
MSE
MAE
0.284
0.344
0.277
0.358
0.292
0.348
0.283
0.343
0.276
0.337 0.291
0.348
0.282
0.341
0.275
0.336
0.289
0.347
0.291 0.353
0.287
0.350
0,304
0.361
0.291
0.352
0.286
0.348
0.302
0.360
0.282
0.342
0.275
0.337
0.290 0.348
96
0.305 0.363
0.306
0.362
0.326
0.376
0.304
0.361
0.303
0.360
0.322
0.373
0.282
0.543
0.277
0.339
0.293 0.350
0.385
0.343
0.389
0.401
0.329
0.380
0.384
0.352
0.287
0.347 0.282
0.294
0.353
0.367 0.401
0.386
0.411
0.402
0.423
0.362
0.396
0.378
0.404
0.394
0.417
0.287
0.348
0.282
0.344
0.295
0.354
0.347 0,388 0.358
0.394
0.351
0.387
0.347
0,388
0.358
0.393 0.351 0.387
0,344
0.387
0.559
0.393
0.351 0.386
0.354 0.397 0.362
0.401
0.361
0.398
0.354
0.397
0.364
0.401 0.360
0.397
0.346
0.387 0.357
0.392
0.352 0.387
192
0.364 0.406 0.373
0.410
0.381
0.365
0.405
0.315
0.410
0:377
0.409
0.347
0.388
0.358
0.394
0.354 0.389
0.387 0.423 0.405
0.430
0.412
0.433
0.384
0.421
0.402
0.427
0.408
0.430
0.350
0.391 0.362
0.397
0.354 0.390
0.406 0.433 0.437
0.446
0.451
0.453
0.403
0.430
0.432
0.442
0.445
0.449
0.350
0.392 0.362
0.398
0.355 0.391
0.334 0.389
0.359
0.400
0.359
0.406
0.334
0.388
0.359
0.400
0.359
0.405
0.334
0.387 0.360
0.400
0.405
0.341 0.399
0.358
0.404
0.369
0.418 0.343
0.399
0.363
0.406
0.370
0.417
0.332
0.387
0.355
0.398
0.358 0.405
336
0,350 0.408 0.365
0.412
0,390
0.433
0.352
0.407
0.372
0.414
0.388
0.431
0,333
0.389 0.355
0,400
0.359
0.408
0.373 0.423 0.387
0.426
0.428
0.456
0.369
0.420
0.387
0.424
0,420
0.450
0.339
0.393 0.364
0.405
0.365
0.413
0.387 0.430 0.414
0.440
0,470
0.477 0.384
0.427
0:415
0.439
0.458
0.469
0.340
0.394 0.363
0.368
0.416
0.367 0.429
0.379
0.423
0.405
0.436
0.386
0.428
0.379
0.422
0.405
0.436
0.388
0.429 0.380
0.423
0,406
0.437
0,396 0.439
0,388
0.432
0.416
0.447
0.397
0.438
0.389
0.432
0.417
0.446
0,387
0.429
0.378
0.423
0.405
0.437
720
0.412 0.452
0.405
0.444
0.438
0.461
0.412
0.449
0.406
0.443
0.437
0.459
0.388
0.432
0.379
0.424
0.406
0.440
0.433 0.468
0.428
0.462
0.466
0.480
0.424
0.460
0.422
0.457 0.458
0.475
0.395
0.436
0.385
0.429
0.412
0.443
0.457 0.481
0.470
0.485
0.514
0.505
0.447
0.473
0.462
0.479
0.503
0.498
0.396 0.438
0.386
0.430
0.414 0.445
Electricity
Combined
Level Shin
Spike
Model
PatchT.GA
PatchTST
TimeMixer
PatchT.GA
PatchTST
TimeMixer
PatchTGGA
PatchTST
TimeMixer
H Menric
MSE MAE
MSE
MAB
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAB
MSB
MAE
MSE
MAB
MSE
MAE
0.138 0.234
0.138
0.234
0.141
0.238
0.137
0.233
0.137
0.233
0.139
0.237
0.130
0.226
0.130
0.225
0.133
0.230
0.247
0.151 0.249
0.251 0.146
0.243
0.148
0.245
0.150
0.247
0.229
0.132
0.228
0.233
96
0.163 0.260
0.166
0.262
0.170
0.265
0.159
0.254
0.163
0.257
0.165
0.259
0.134
0.230
0.134
0.230
0.136 0.234
0.199 0.292
0.201
0.294
0.210
0.297
0.191
0.282
0.194
0.285
0.201
0.287
0.138
0.236
0.138
0.235
0.140 0.240
0.261 0.328 0.266
0.333
0.282
0.336
0:251
0.316
0.258
0.322 0.268
0.323
0.142
0.241 0.142
0.240
0.144 0.244
0.154 0.251 0.156
0.253
0.255
0.153
0.249
0.155
0.252 0.159
0.253
0.1448
0.244
0.149
0.244
0.154 0.247
165 0.263 0.168
0.266
0.1
0.267 0.162
0.259
0.166 0.263 0.169
0.264
0.0
0.246
0.151
0.247
0.156 0.250
192
0.178 0.274
0.181
0.278
0.188
0.281
0.174
0.269
0.177
0.273 0.184
0.276
0.151
0.248
0.152
0.249
0.157 0.251
0.303
0.209
0.306
0.227
0.312 0.201
0.294
0.202
0.297 0.219
0.303
0.155
0.253 0.156
0.254
0.256
0,267 0.337 0.256
0.338
0,301
0.352
0.255
0.325
0.246
0.326
0.289
0.341
0.159
0.257 0.161
0.259
0.164 0.260
0.171 0.269
0.172
0.269
0.179
0.274 0.171
0.267
0.171
0.268
0.178
0.273
0.165
0.262 061
0.261
0.173 0.267
0.182 0.281 0.185
0.283
0.192
0.287 0.179
0.277
0.183
0.280
0.189
0.284
0.167
0.264 0.166
0.263
0.175 0.270
335
0.19 0.292 0.202
0.297
0.210
0.301
0.191
0.287
1.1
0.293 0.206
0.296
0.168
0.266
0.265
0.176 0.272
0.229 0.321
0.231
0.324
0.246
0.331 0.220
0.311
0.225
0.317
0.238
0.323
0.172
0.271
0.172
0.270
0.180 0.276
0.288 0.356 0.287
0.361
0.326
0.372
0.276
0.344
0:278
0.351
0.313
0.361
0.176
0.275 0. 176
0.276
0.184 0.280
0.208 0.301 0.209
0.303
0:210
0.303
0.207
0.300
0.208
0.301
0.209
0.302
0.203
0.295 0.200
0.294
0.205 0.297
0.219 0.313 0.232
0.320
0.220
0.314
0.217
0.309
0.230
0.316
0.218
0.311
0.204 0.298 0.203 0.297
0.206 0.299
730
0.235 0.325 0.253
0.336
0.234
0.325
0.231
0.320
0.254
0.331
0.231
0.321
0.206 0.299 0.206
0.300
0.207 0.301
0.264 0.350
0.285
0.362
0.265
0.351
0.255
0.342
0.282
0.356
0.258
0.343
0.209
0.303
0.212
0.306
0.210 0.305
0.320 0.385
0.352
0.402
0.338
0.386
0.308
0.374
0.350
0.394
0.327
0.376
0.213
0.307
0.220
0.313
0.214 0.309
25

---

Published as a conference paper at ICLR 2026
level 5 with 336-step forecasting horizon, LGA reduces the MSE by 16.1% % compared to SDP attention (from 0.818 to 0.686).
Cross-Temporal Attention (CATS): LGA also enhances the robustness of cross-attention, although the improvements are less uniform than in the self-attention case. For instance, at a 720-step horizon with severity level 5, LGA improves the MSE by 21.5% (from 1.255 to 0.985). This aligns with our main analysis that operating on linearly embedded noisy inputs can sometimes limit the consistency of performance gains.
Self-Channel Attention (fTransformer): For this mechanism. LGA offers modest but stable improvements, particularly under high-severity corruption. The linear embedding applied to the entire time series, as noted in our main discussion, disrupts some temporal local geometry, making the gains less pronounced than with PatchT'T. For example, at the 720-step horizon under severity 5, LGA still provides a 4.0% reduction in MSE (from 1.439 to 1.382).
These comprehensive results further validate our conclusion that while LGA is a broadly applicable technique that enhances robustness across all tested attention mechanisms, its integration with self-temporal attention provides the most consistent and substantial improvements for time series forecasting under realistic corruptions.
Table 17: Detailed performance comparison of different attention mechanisms on the ETTml dataset with input length 512 under combined corruptions.
ETTml
96
176
Mod.
/Atton.
Metric
2
PachhSi
LGA
MSF
0.207
0.314
0.527
0.657
0.830
0.341
0.354
0.504
0.601
0.372
0.385
0.502
0.586
0.686
0.425
0.456
0.541
0,63
0.709
MAB
0.353
0.368
0.456
0.530
0,594
0.376
0.38%
0.462
0.521
0.513
(1392
0.402
0.465
0,516
0.564
0.421
0.429
0.488
0.536
0.578
0.304
0.326
0.357
0.661
0.370
SDP
MAR
0.357
0.377
0.497
0.566
0.636
0.378
0.391
0.491
0.542
0.001
(1397
0.408
0.519
0.563
0.611
0.424
0,432
0.519
0.562
0.604
0.313
0,423
1,100
CATS
MAR
0.341
0.358
0.454
0.563
0.616
0.365
0.577
0.488
0.582
0.624
(13900
0.400
0.492
0.577
0.624
0.424
0,432
0.560
0.665
0.647
SDP
MSE
0.289
0.313 0.687 0.862
1.036
0.327
0.347
0.778
0.949
1.146
0.355
0.368 0.711 0.889
0.979
0,410
0,426
0.946
1242 1.255
0.344
0.362 0.487
0.559
0.614
0.368
0.383
0.521
0.587
0
0.45
(1389)
0.400 0.526
0.591
0.630
0.472
0,433
0.630
0.724
0.722
iThm..
EGA
MSE
0.580
0.357
0.391 0.718 0,945
0.412 0.508 0.598
0.683
1,222
0,408
0.413
0.434
0,440
0,800
0.626
1.051
0.707
1.336
0.442 0.464 0.562
0.455
0.493 0.864
0.654
1,105
1.567
0.732
0.506
0.471
0,400
0.541
0.586
0.894
0.672 0.746
1.126 1,382
SDP
MSE
0.358
0.990
0.690
0.899
1.166
0,391
0.425
0.751
03965
1:229
0.453
0.488
03853
1.095
1,401
0.511
0,558
0.910
1,149
1.439
0.387
0.498
0.667
0.474
0.603
0.476
0.557
0.740
0,404
0.591
0.680
0.761
E.3 COMPREHENSIVE EVALUATION OF INPUT LENGTH IMPACT ON FORECASTING ROBUSTNESS
We examined how varying input sequence length affects forecasting performance under corrupted conditions. Here, we present a more detailed analysis with complete results across all input lengths, forecasting horizons, and severity levels for the ETTml dataset with combined corruptions. Table 18 provides comprehensive performance metries for PatchLGA, PatchTST, and TimeMixer with input lengths ranging from 192 to 1024 timesteps. These detailed results allow us to examine the complex relationship between input context, forecasting horizon, and model architecture under various corruption intensities.
At short input lengths (e.g., 192), TimeMixeris is competitive with or occasionally outperforms the trans- former models at low corruption severities (level 1), particularly for the shortest forecasting horizon (H-96). However, as input length increases, the transformer models, especially PatchLGA, demon- strate progressive and significant performance improvements, while the performance of TimeMixer tends to stagnate or even deteriorate longer input lengths (e.g., 720 and 1024). With an input length of 512, PatchI.GA consistently outperforms both alternatives across all forecasting horizons and severity levels, establishing as the most robust model at this context size. For the longest forecasting horizon (720), the impact of input length becomes even more critical. With a 192-timestep input at severity level 5, PatchLGA achieves an MSE of 1.140. When the input length is increased to 512 timesteps, the MSE improves to 0.709, substantial 37.8% reduction. This demonstrates that long-range dependencies, effectively captured by LGA, become increasingly important for distant forecasting, especially under severe corruptions.
These findings have important implications for deploying forecasting models in real-world scenarios. While linear models may be adequate for short-term forecasting with limited historical data and
26

---

Published as a conference paper at ICLR 2026
minimal corruption, transformer models with LGA provide substantial benefits when longer historical context is available, particularly under challenging corruption conditions.
Table 18: Detailed performance comparison across different input sequence lengths (192, 336, 512, 720, 1024) on the ETTml dataset with combined corruptions.
FTTmll
96
192
336
720
Model
Metri:
PatebLGGA
MSE
0.315
0.339
0,579
0.757
0.944
0.349
0,509
0.621
0.811
0.971
0.375
0.392
0.719
0,939
1,065
0.436
0,400
0.767
0.982
1.140
MAF
0.357
0.376
0.42
0.541
0.601
0.560
0.306
0.486
0.567
0.636
0.398
0413
0.516
0,606
0.688
0,499
0445
0.530
0.611
0.605
PoccTST
0.889
1020
1.275
0.852
1,061
1,266
0,440
1080
MAB
0.355
0.280
0,496
0,576
0.52
0.380
0,002
0.536
0.623
0.685
0.402
0.420
0.557
0,050
0.717
0.436
0.452
0.583
0.673
0.737
TimeMixer
MSC
0.312
0.392
0.567
0.857
1072
0.351
0.379
0.785
1057
1.25
0.16
0.401
0.735
0.851
1.00
0,039
0.463
0.765
0.88
1785
0.358
0.483
0.564
0.625
013366
0.407
0.526
0.619
0.692
9.417
0.521
0,591
0,002
0.439
0,453
0.545
0.009
0.678
PatchLGGA
MSC
0.299
0.218 0,589
0.778
0.939
0.336
0.352
0.595
0.775
0.563
0.371
0.287
0.659
0,846
0.912
0.424
0,439 0.656
0.500
0.885
0.375
0.42X
0.59%
316
PuchIIT
MSF
0.300
0.689
0.83
0.367
01925
1.165
0.402
0.73
0,850
0.430
03255
1092
MAE
0.358
0.382
01512
0,591
0.675
11.340]
0,399
0.536
0.613
0.685
0.4114
014422
0.555
0.624
0.710
1,434
0.452
0.621
0.689 0.748
TimeMixer
0.343
0.658
0.744
MAE
0.357
0.380
0,498
0.582
0.658
0.398
0.395
0.999
0.562
0.628
0.400
0.415
0.521
0.596
0.657
0.423
0:433
0,512
0.571 0.615
PatchIGGA
MSE
0.297
0.314 0.527
0.657
0.820
0.341 0.354 0.504
0.603
0.722
0.372
0.385
0.502
0,586
0.686
0.425
0:436 0.543
0.623 0.709
0.353
0.456
0.530
0.376
0,358
0.521
0.573
0.402
0:465
0.564
0.421
0.536 0,578
512
PachTTT
MSC
0.304
0.326 0,625
0.743 0.964
0.342
0.357 0.581
0.661
0.812
0.370
0.385
0.630
0,892
0.818
0,420
0,433 0.622
0.684 0.782
0.357
0.377 0.497 0,506
0.378
0,391
0.542
0.601
0.408
0.519
0.424
0,519
0.562
TimeMixer
0.317
0.619
0.700
11342
0.358
0.643
0.740
0.396
0:600
0,709
0.829
0.436
0.449
0.620
0.745
0.528
MAF
0.363
0.383
0,450
0,558
0.629
0.560
0,393 0.474
0.527
0.574
0.407
0.418
0.511
0,567
0.622
0.432
0,441
0.524
0.587 0.626
PatchLGG
0.305
0,475
0.336
0.388
0.426
0.403 0.681
MAE
0.35%
0.375
0.450
0.514 0.570
013377
0.38%
0.461
0.518
0.563
0.395
01400
0.464
0.516
0.560
0.422 0.429 0.485
0.533 0.573
720
PaccTTT
MSF
0.304
0.324
0.519 0,615 0.761
0.340
0,355 0.525
006
0.706
0.365
0.380
0.587
0.657
0.745
0,419 0.41
0.562
0/627 0.711
0.521
0.379
0.523
0.569
0.406
0.503
0.423
0.546
TimeMixer
MSE
0.332
0.358
0,705 0,940 1.079
0.344
0.364 0.590
0.707
0.831
0.371
0.384
0.540
0.629
0.741
0.456
0.468 0.636
0.745 0.353
MAE
0,380
0.399
0,549
0,650
0.697
0.384
0,398 0.490
0.546
0,603
0.396
0.405
0,483
0,536
0.586
0.449
0,456 0,536
0.590 0,611
PatchLGGA
0.309
0.471 0,509
[1:355
0.304
0.378
0.681
0.48%
0,571
0.052
0,415
0:423 0.513
0.592 0,666
MAB
0.360
0.275 0,448 0,510
0.560
01385
0,596
0.463
0.520
0.568
0.384
0.405
0.465
0,518
0.557
0.419
0,426 0.482
0.532 0.569
1024
PaccITST
0.303
0.604
113440
0.613
0.718
01.381
0.560
0.716
11,30
0,439
0.552
0.620 0.699
MAF
0,390
0.377
0.503
0.500
0.381
0.992 0.489
0.541 0.587
0.306
0.406
0:497
0.544
0.579
0.430
0:426 0.501 0.546 0.579
TimeMixer
MSE
0.322
0.347
0,594
0,703
0.853
11,359
0,375
0.573
016664
0.77%
0.399
0.417
0.633
0,753
0.884
1.454
03465
0.645
0.794 0.878
0,543
0,302
0.404
0.540
0.505
0.537
0.448
0.618 0.651
E. COMPREHENSIVE COMPARISON WITH ALTERNATIVE ROBUST ATTENTION METHODS
Table presents the detailed comparison between LGA and other robust attention mechanisms across all forecasting horizons and severity levels on the ETTml dataset with combined corruptions. The results reveal that LGA outperforms the alternative approaches in the vast majority of settings, with its advantage being particularly notable at higher corruption severities. For short-term forecasting (96 horizon) at severity level 5, LGA achieves an MSE of 0.820. This represents substantial 28.6% improvement over MoM and a 4.2% improvement over Elliptical attention. This advantage is pronounced for medium-term horizons. For instance, on the 336 horizon severity level 5, the MSE of LGA (0.686) is 37.1% % lower than MoM and 1 11.6% lower than Elliptical attention For long-term forecasting (720 horizon), while the competition closer, LGA still demonstrates clear benefits. At severity level 3, for example, MSE of LGA (0.543) is significantly better than SDP and Elliptical attention, although slightly higher than MoM in this specific case.
These comprehensive results confirm our findings that while robust attention mechanisms like MoM and Elliptical attention succeed in vision and language tasks, they do not transfer as effectively to time series forecasting. LGA, specifically tailored for capturing local geometry of temporal structures, yields superior robustness while maintaining computational efficiency comparable to standard attention.
Table 19: Detailed performance comparison of different robust attention methods on the ETTml dataset with input length 512 under combined corruptions.
ETTml
96
336
Mod.
Atienn
Metric
5
LGA
MSF
0.297
0.314
0.527
0.657
0.820
0.341
0.354
0.504
0.603
0.372
0.385
0.502
0.586
0.686
0.425
0.436
0.543
0,623
0.709
MAB
0.353
0.368
0.456
0.530
0.594
0.376
0,38%
0.462
0.521
0.573
0.32
0.402
0.465
0,516
1.564
11421
0.429
0.488
0.536
0.578
PutchTST
SDP
0.304
0.36
0.625
0.743
0.944
0.342
0.357
0.001
0312
0.700
0.383
0.630
0.692
0.118
0.420
0.433
0.622
0.844
0.782
MAB
0.357
0.377
0.497
0.566
0.636
0.378
0.391
0.191
0.542
0.601
0.397
0.108
0.519
0.563
0.611
0.424
0.432
0.519
0.562
0.604
Eliip.
0.301
0.319
0.856
0.354
0.636
0300
0.659
0.776
1,03
MAE
0.356
0.372
0.480
0.537
0.601
0.379
0.390
0.493
0.525
0.589
0.395
0.404
0.522
0.552
0.589
0.434
0.643
0.650
0.676
0.692
McM
MSE
0.325
0.359
0.700
0.941
1.149
0.354
0.378
0.735
0.961
1.130
0.387
0.407
0.722
0.961
1.091
0.435
0.443
0.535
0,620
0.696
MAR
0.575
0.399
0.530
0.640
0.714
0.392
0
0.542
0.645
0.706
0.409
0.473
0.548
0.652
0.202
0.424
0.430
0,485
0.538
0.571
27

---

Published as a conference paper at ICLR 2026
VAL of 740
0.41
(a) Severity Spectrum
(b) Evolution of Noise Parameters (A, P.
Figure 8: Selection and Analysis of Severity Levels. (a) The severity spectrum obtained by sorting the normalized MSE of all 37 parameter configurations. From this full spectrum, we selected five representative levels (L1-L5, marked in red) that provide comprehensive coverage of the difficulty range with distinct performance steps. (b) The evolution of noise parameters corresponding to the selected levels. The injection frequency (A) and segment length (p) strictly increase, while the transition probability (g. log-scale) decreases. This confirms that the increasing severity is driven by systematic intensification the noise mechanism rather than random permutations.
F
NOISE GENERATION AND PARAMETER SELECTION
This section provides detailed information our implementation of corruption functions and the experimental process to select appropriate noise parameters. While the main paper presented the theoretical foundation, here explain the practical implementation details and the empirical validation parameter settings.
F.1 IMPLEMENTATION OF CORRUPTION FUNCTIONS
implement realistic corruptions for time series data through a systematic process that combines Poisson event generation with extreme value theory for threshold estimation. Algorithm 2 presents the complete procedure for injecting both level shift and spike coruptiins into time series data. We fix the random seed to 2025 across all experiments to ensure reproducible results.
The GETTHRESHOLDS function the Algorithm requires detailed explanation, as implements the DSPOT (Drift SPOT) algorithm (Siffrr et al. 2017) to estimate extreme value thresholds. DSPOT extends the SPOT algorithm by explicitly accounting for distributional drift time series data. Given significance level g, the algorithm computes time-varying thresholds (t) inf(= P(X(I) >
z) < q} that represent the critical values for identifying extreme events at each time point.
The DSPOT algorithm operates by:
1. Initializing with burn-in period to establish baseline statistics
2. Iteratively updating Generalized Pareto Distribution (GPD) parameters as new data arrives
3. Computing upper and lower thresholds based on the estimated tail distributions
4. Adapting to potential distributional changes through drift detection mechanisms
For our implementation. we maintain consistent DSPOT hyperparameters across all datasets to ensure fair comparison and reproducibility. The algorithm returns both upper and lower thresholds that are used to determine realistic corruption magnitudes for level shifts and spikes, ensuring that the generated corruptions reflect statistically extreme but realistic deviations from normal behavior.
F.2 PARAMETER SELECTION PROCESS
Finding appropriate noise parameters for realistic corruptions requires both theoretical justification and empirical validation. Although the theoretical foundation of our noise model established in Section 4 experimental verification is necessary determine which parameter settings effectively induce progressively increasing severity across diverse data distributions.
To this end, conducted comprehensive grid search over 36 different parameter combinations (varying X. p. and q) alongside a clean bascline. We evaluated the original PatchTST on six benchmark datasets (ETTh1, ETTh2, ETTml, ETTm2, Electricity, and Weather) measure the performance
28

---

Published as a conference paper at ICLR 2026
Algorithm 2 TSRBench Corruption Injection
Require: Time series X IRT, corruption rate A, duration parameter p, amplitude parameter Ensure: Corrupted time series: X"shiff X Xeembinedd
1: Initialize: T <- length (X) gahift <xiike t- 0
Step 1. Generate corruption events and compute thresholds
2: T" - -
Extended time interval Number of events
3: N Poisson(X -T
4: T < (T.. ................................ TN where ~ 4(0, T
5: T < {TET: : :
Keep second half, shift to [0, T]
6: upper threshold, lower lower_threshee threshold <- GETTHRESHOLDS((X..
Step 2-1. Level shift corruption injection
7: for each T do
8: p'
Geometric parameter
9: d ~ Geometric(p') +
A> Shift duration Bidirectional SPOT variants
if IsBIDIRECTIONAL then Ashin min (upper_ supper_threshold_ ( 7 d) X[r : +r++0 +d)
if RRADOM() < 0.5
11:
X[T : d))
otherwise
12: else
A-hin + cun[uupper_thrshollle : d] X| :T + +d)
14: end if
eshift (r min(r + d, hehhitt
16: end for
Step 2-2. Spike corruption injection
17:
for each E T do
18:
d1 Geomeerric )), d2 ~ Geometric(
> Rising/decay durations
19:
if BiBiiIrraTTiNNA then
Bidirectional SPOT variants
upper_ tpper_threshold/ + d1] X|7 di]
if RANDOO()) < 0.5
20:
h.pikk
lower_threshold (r dil X(7 + +d]
otherwise
21:
else
22:
upper_threee thressotreehlld/ d1l - X(T + dil
23:
end if
24:
B
10-4
D Sharpness parameter
qike
exp
In(
di
(s T d1)
if < +
exp
In(9)
(s T di)
otherwise
spike [T min + d1 + d2, T)] +
27: end for
Step 3. Generate corrupted time series shift <- X + shift
28:
spike
X
spike
(x)i
if [shiitttt) le (t)
30: g combined
Vt [1,T]
spike (1)
otherwise
31: X Xcombined
X ccombined
32: return X spike commhined
29

---

Published as a conference paper at ICLR 2026
impact of each configuration. Figure illustrates the "Severity Spectrum,"" where sorted all configurations based on their average impact on model performance (Normalized MSE) across all datasets. This aggregated spectrum reveals the full range of difficulty levels our noise model can generate, independent of specific dataset characteristics.
From this continuous spectrum, we identified five representative severity levels (L1-L5) based the following criteria:
The selected levels are distributed across the spectrum cover distinct difficulty tiers, ensuring that the benchmark evaluates robustness under diverse conditions ranging from mild to extreme.
Comprehensive Coverage:
Physically Interpretable Evolution:
As shown Figure b)), we ensured that the chosen levels correspond to consistent physical intensification of the noise. Specifically, the injection frequency (A) strictly increases from 0.002 to 0.008, and the transition probability (g) decreases logarithmically. This monotonicity guarantees that the degradation in model performance is a direct result of the progressively challenging noise mechanics.
Table summarizes the final parameter sets derived from this process, which used for all subsequent robustness evaluations. Also, we provide the results of these experiments on the ETTml dataset Table 20 21 and 22
30

---

Published as a conference paper at ICLR 2026
Table 20: Performance comparison between PatchLGA and PatchTST on the ETTml dataset with both level shift and spike corruptions, using input length 512. PatchLGAA replaces standard self- attention with LGA, consistently showing improved robustness across all scenarios.
Combined
PatchLGA
PatchTST
0.002
0.004
0.008
0.002
0,004
0,008
H
9
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
1.6 10
0.297
0.353
0.305
0.361
0.326
0.379
0.304
0.357
0.316
0.370
0.336
0.387
4 10
0.330 0.372
0.403
0.408
0.466 0.451
0.370 0.396
0.474 0.446
0.534
0.488
1. 1.1
0.346
0.381
0.441
0.425
0.502
0.469
0.399
0.408
0.540
0.470
0.590
0.510
1.6-10 10
0.302
0.356
0.314
0.368
0.344
0.393
0.309
0.361
0.326
0.377
0.353
0.401
4.10-5
0.357
0.385
0.468
0.435
0.566
0.494
0,409
0.413
0.553
0.475
0.650
0.533
1.10-6 0.383
0.397
0.529
0.458
0.620
0,519
0.452
0.429
0.647
0.505
0.727
0,562
1.6 10-1
0.305
0.359
0.320
0.373
0.357 0.404
0.312 0.364
0.331
0.381
0.366 0.410
12
4.10 0.383
0.397
0.527
0,456
0.657
0,530
0,435
0,424
0.625
0,497
0.743 0.566
1.10 0.420
0.412
0.611
0.487
0.732 0.561
0.490
0.444
0.741
0.533
0.844 0.602
1.610 10-10- 0.307
0.361
0.324
0.377 0.366 0.413
0.314
0.365
0.333
0.384
0.374 0.418
15
4-10
0.407
0,406
0.577
0.473
0.730
0.558
0.463
0.434
0.674
0.514
0.829 0.596
1 1.10-5
0.455
0.425
0.681
0.510
0.820 0.594
0.530
0.457
0.809
0.554
0.944 0.636
1.6 10
0.341
0.376
0.347
0.383
0.360
0.395
0.342
0.378
0.350
0.387
0.363 0.398
4-10-
0.365
0.393
0.416
0,422
0.459
0.452
0.381
0.403
0.459
0,443
0.489 0.468
1.10
0.376
0.399
0.443
0.434
0.485
0.466
0.401
0.412
0,513
0.463
0.526 0.485
1.6-10 10
0.345
0.380
0.354
0.388
0.374
0.406
0.346
0.382
0.357
0.391
0.376 0.409
4.10-5
0.384
0.403
0.462
0.443
0.531
0.488
0.410
0.417
0.522
0.469
0.579 0.508
192
1.10
0.403
0.412
0,507
0.461 0.570
0.507
0.444
0.431
0.602
0.496
0.636 0.532
1.6 10
0.347
0.382
0.358
0.392
0.385
0.415
0.349
0.384
0.361
0.395
0.387 0.418
12
4. 10-5
0.403
0.412
0.504
0.462 0.603
0,521
0.434
0.427 0,581
0.491
0.661 0,542
1-10-
0.431
0.425
0.567
0.487 0.653 0.544
0.483
0.446 0.684 0.524
0.733 0.570
1.6100
0.350
0.383
0.361
0.395 0.391
0.421
0.352
0.385 0.364
0.398
0.394 0.424
15
4.10
0.420
0.420
0.544
0.479 0.663
0.547
0.456
0.437 0.627 0.508
0,730 0.570
1 1.1055
0.457
0.436
0.621
0.508
0.722
0.573
0.516
0.458
0.746
0.545
0.812 0.601
1.6 10-1
0.372
0.392
0.380
0.398
0.389
0.408
0.370
0.397
0.378
0.404
0.390 0.414
4- 1015
0.391
0.405
0.434
0.431
0.468
0.457
0.418
0.426
0.499
0.468
0.534 0.495
1 1.10-5
0.399
0.410
0.455
0.441
0.469 0.489
0.434
0.433
0.554
0.487
0.555 0.505
1.6.10
0.376
0.394
0.385
0,402
0.401
0.417
0.374
0.400
0.383
0.408
0.402 0.424
4. 10-1
0.405
0.413
0.468
0.448
0.526
0.487
0.453
0.442
0.569
0.496
0.620 0.534
336
1 10-0-
0.419
0.421
0.502
0.463
0.558
0.505
0.483
0.454
0.648
0.521
0.662 0.551
166 10
0.378
0.396
0.388
0.405
0.411
0.426
0.377
0.402 0.387
0.411
0.412 0.432
12
1.10-5 0.419
0.421
0.502
0.465
0.586
0.516
0.481
0.454
0.630
0.519
0.692 0.563
1.10-5 0.438
0.431
0.548
0.485
0.627
0.537
0.522
0.471 0.727
0.548
0.749 0.585
1.6 10-1 0.380
0.398
0.391
0.407 0.418
0.431
0,380
0.404
0,390
0.413
0.418 0.437
15
4.10
0.431
0.427
0.533
0.478
0.638
0.540
0.502
0.463
0.669
0.533
0.749 0.586
1. 10-5
0.456
0.439
0.590
0.503
0.686
0.564
0.553
0.483
0.777
0.565
0.818 0.611
16. 10-4
0.425
0.421
0.432
0.426 0.440
0.434
0.420
0.424 0.429 0.429
0.437 0.437
4-10-5 0.442
0.433
0.482
0.457
0.513
0.481
0.447
0.440
0.504
0.471
0.536 0.495
1 1-10-5
0.449
0.438
0.502
0.467 0.530
0.492
0.457
0.446
0.538
0.484
0.546 0.501
1.6 10-1
0.428
0.423
0.436
0.429
0.451
0.442
0.423
0.426
0.433
0.432
0.448 0.445
4. 10
0.454
0.440
0.513
0.474
0.568
0.509
0.469
0.451
0.559
0.495
0.612 0.530
720
1-10-5 0.466
0,447
0.543
0.487
0.593
0,523
0.491
0.461
0,615
0.514
0.630 0.540
1.6- 10
0.430
0.424
0.438
0.431
0.460
0.449
0.426
0.428
0.437
0.435
0.458 0.453
12
4 10-5
0.466
0.447
0.543
0.488
0.623
0.536
0.492
0.462
0.622
0.519
0.684 0.562
1-10 0.482
0.456
0.584
0.506
0.656
0.553
0.522
0.475 0.699
0.544
0.714 0.575
1.6 10-1
0.432
0.426
0.440
0.433
0.466
0.454
0.429
0.430
0.439
0.437
0.464 0.458
15
4.10
0.477
0.452
0.570
0.500
0.670
0.558
0.511
0.470
0.661
0.534
0.744 0.586
1. 1.10-5
0.497
0.463
0.620
0.522
0.709
0.578
0.549
0.485
0.753
0.564
0.782 0.604
31

---

Published as a conference paper at ICLR 2026
Table 21: Performance comparison between PatchLGA and PatchTST on the ETTml dataset with level shift corruptions, using input length 512. PatchLGA replaces standard self-attention with LGA, consistently showing improved robustness across all scenarios.
Level Shift
PatchLGA
PatchTST
0.002
0.004
0.008
0.002
0,004
0,008
H
9
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
1.6. 10
0.297
0.352
0.303
0.360
0.325
0.378
0.304
0.356
0.316
0.369
0.336
0.387
4 10
0.326 0.370
0.394
0.403
0.454
0.443
0.370 0.394
0.472
0.443
0.538
0.488
1. 1.10-
0.341
0.377
0.430
0.418
0.491
0.463
0.399
0.407
0.544
0.470
0.597
0.512
1.6- 16-10--4 0.300
0.355
0.312
0.366
0.342
0.391
0.308
0.360
0.325
0.376
0.353
0.400
4.10-5
0.353
0.382
0.455
0.427
0.552 0.484
0.407 0.411
0.547
0,470
0.653 0.531
96
1-10-6
0.377 0.393
0.512
0.450
0.607 0,511
0.449
0.426
0.644
0.502
0.737 0,563
1.6 10-1
0.303
0.357
0.317
0.370 0.354
0.401
0.311
0.363
0.329
0.379
0.365 0.409
12
4-10-5
0.378
0.393
0,511
0,447 0.641
0,519
0.432
0,421
0.616
0,490
0.746 0.563
1.10 0.413
0.407
0.592
0.477 0.716 0.552
0.485
0.440
0.737
0.528
0.853 0.601
1.6 10-10-4
0.305
0.359
0.320
0.373 0.363 0.409
0.313
0.364
0.331
0.381
0.374 0.416
15
4-10
0.401
0,402
0.560
0.463
0.707 0,545
0.459
0.430
0.664
0.504
0.830 0.592
1 1.10-5
0.446
0.419
0.661
0.499
0.802 0.584
0.523
0.451
0.807
0.547
0.954 0.634
1.6 10
0.340
0.376
0.345
0.382
0.358
0.393
0.341
0.378
0.349
0.386
0.363 0.398
4-10-
0.361
0.390
0.408
0,417
0.448
0.445
0.380
0.401
0.457
0,440
0.487 0.466
1.10
0.372
0.395
0.435
0.428
0.475
0.460
0.400
0.411
0,516
0.463
0.531 0.486
1.6-10 10
0.344
0.379
0.351
0.386
0.371
0.403
0.345
0.381
0.355
0.390
0.376 0.408
4.10-5
0.380
0.399
0.452
0.437
0.518
0.480
0.409
0.415
0.517
0.465
0.578 0.506
192
1.10
0.398
0.408
0.495
0.454
0.559
0.500
0.442
0.428
0.602
0.494
0.645 0.533
1.6 10
0.346
0.380
0.356
0.390
0.382
0.412
0.348
0.383
0.359
0.394
0.386 0.417
12
4. 10-55-
0.398
0.408
0.491
0.455
0.587
0,511
0.432
0.425 0.574 0.486
0.657 0.538
1 1-10-5 0.424
0.420
0.552
0.479 0.639 0.535
0.480
0.442 0.684
0.522
0.742 0.570
1.6. 16-10-4 0.349
0.382
0.358
0.393 0.389 0.418
0.351
0.384
0.362
0.396
0.394 0.423
15
4.10
0.415
0.415
0.530
0.470 0.643 0.536
0.454
0.434
0.618
0.501
0.725 0.565
1 1.10-5
0.449
0.430
0.605
0.499
0.707 0.564
0.512
0.454
0.746
0.541
0.823 0.601
1.6 10-1
0.372
0.391
0.378
0.397
0.387
0.406
0.370
0.397
0.377
0.404
0.390 0.414
4. 10-5
0.388
0.403
0.427
0.426
0.457
0.449
0,416
0.424
0.499
0.466
0.532 0.493
1.10-5 0.395
0.407
0.446
0.436
0.479
0.462
0.432
0.431 0.561
0.488
0.560 0.505
1.6. 16-10-4 0.375
0.394
0.383
0,400
0.399
0.415
0.374
0.400
0.382
0.407
0.402 0.424
4. 10-5
0.402
0.410
0.459
0.443
0.513 0.478
0.452
0.440
0.566
0.492
0.618 0.531
336
1 1010-
0.414
0.417
0.490
0.457
0.546 0.496
0.481
0.452
0,652
0.521
0.673 0.552
166 10
0.378
0.395
0.386
0.403
0.409 0.423
0.377
0.402 0.386
0.410
0.412 0.431
12
1.10-5
0.415
0.417
0.490
0.458
0.570 0.506
0.478
0.452
0.624
0.514
0.688 0.559
1.10-5 0.432
0.426
0.533
0.477
0.613 0.528
0.519
0.468
0.730
0.547
0.761 0.586
1.6 10-1 0.380
0.396
0.388
0.405
0.415 0.429
0,380
0.404
0,388
0.412
0.419 0.437
15
4.10
0.426
0.423
0.519
0.471
0.619
0.529
0.500
0.461
0.660
0.526
0.745 0.582
1. 1.10-5
0.448
0.434
0.573
0.494
0.670
0.554
0.550
0.480
0.778
0.563
0.831 0.612
16. 10-4
0.425
0.420
0.431
0.425
0.438
0.432
0.420
0.424
0.428 0.428
0.435 0.436
4-10-5 0.439
0.430
0.476
0.453
0.503
0.473
0.444
0.438
0.500
0.467
0.527 0.489
1 1-10-5
0.445
0.434
0.494
0.462
0.520
0.484
0.453
0.443
0.537
0.481
0.541 0.496
1.6 10-1
0.427
0.422
0.435
0.428
0.449
0.440
0.423
0.426
0.432
0.431
0.447 0.444
4. 1.10
0.451
0.437
0.506
0.468
0.556
0.501
0.466
0.449
0.555
0.491
0.603 0.523
720
1-10-5
0.461
0,443
0.533
0,481
0.582
0,515
0.487
0.458
0,614
0.511
0.630 0.536
1.6- 10
0.430
0.423
0.437
0.430
0.458 0.447
0.426
0.428
0.436
0.434
0.457 0.452
12
4 1.10-5
0.462
0.443
0.533
0.482
0.609
0.526 0.488
0.459
0.616
0.514
0.673 0.554
1-10
0.476
0.451
0.571
0.499
0.643 0.544
0.518
0.471
0.699
0.542
0.714 0.572
1.6 10-1
0.431
0.425
0.438
0.431
0.464
0.452 0.429
0.429
0.437
0.435
0.464 0.457
15
4.10
0.471
0.448
0.558
0.493
0.652
0.547
0.507
0.467
0,653
0.528
0.731
0.579
1. 1.10-
0.490
0.457
0.605
0.514
0.694
0.568
0.545
0.482
0.749
0.559
0.785 0.601
32

---

Published as a conference paper at ICLR 2026
Table 22: Performance comparison between PatchLGA and PatchTST on the ETTml dataset with spike corruptions, using input length 512. PatchLGA replaces standard self-attention with LGA, consistently showing improved robustness across all scenarios.
Spike
PatchLGA
PatchTST
0.002
0.004
0.008
0.002
0.004
0,008
H
9
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
1.6 10
0.289
0.345
0.289
0.345
0.290
0.347
0.293
0.346
0.293
0.347
0.294
0.348
4 10
0.291
0.347
0.296
0.350
0.301
0.356
0.296 0.349
0.301
0.354
0.306
0.358
1. 1.
0.293
0.348
0.297
0.351
0.302
0.356
0.297 0.350
0.301
0.354
0.306
0.359
1.6- 10-1
0.289
0.345
0.289
0.346
0.291
0.348
0.293
0.346
0.294
0.348
0.295
0.350
4.10
0.292
0.348
0.298
0.352
0.305 0.359
0.298 0.351
0.305
0.357
0.311 0.363
96
1.10-6
0.294
0.349
0.300
0.353
0.306 0.360
0.299 0.352
0.307
0.357
0.312 0.364
1.6 10-1 0.289
0.345
0.290
0.346
0.292 0.350
0.293 0.347 0.294
0.348
0.296 0.351
12
4-10-5 0.293
0.349
0.302
0.354
0.309 0.363
0.301 0.353 0.310
0.361
0.318 0.369
1.10 0.296
0.351
0.304
0.356
0.312 0.365
0.303 0.356 0.314
0.362
0.320 0.370
1.6-0- 10 0.289
0.345
0.291
0.347
0.293 0.350
0.293
0.347 0.295
0.350
0.297 0.352
15
4-10-5 0.294
0.349
0.304
0.357
0.314
0.366
0.303
0.355
0.316
0.365
0.324 0.374
1 1.10-5
0.297
0.352
0.308
0.359
0.316 0.368
0.307 0.358
0.320
0.368
0.326 0.375
1.6 10-1
0.333
0.370
0.334
0.371
0.372 0.335
0.335
0.372
0.335
0.372
0.336 0.373
4-10-
0.336
0.372
0.339
0.375
0.380 0.344
0.337
0.373
0.341
0.377
0.344 0.379
1.10
0.337
0.373
0.340
0.376
0.380 0.345
0.338 0.374
0.342
0.377
0.345 0.380
1.6-10 10
0.333
0.370
0.334
0.371
0.373 0.335
0.335
0.372
0.336
0.373
0.336 0.374
4.10-5
0.336
0.373
0.341
0.376
0.382 0.346
0.338 0.374
0.343
0.378
0.346 0.382
192
1.10
0.338
0.374
0.342
0.377
0.347
0.383
0.339
0.375
0.344
0.379
0.347 0.383
1.6 10
0.333
0.371
0.335
0.372
0.373 0.335
0.335
0.372
0.336
0.373
0.337 0.374
12
4. 10-5
0.337
0.374
0.343
0.378
0.348
0.384
0.339 0.375
0.346
0.380
0.349 0.385
1-10- 0.339
0.375
0.345
0.379
0.350
0.385
0.341 0.377
0.348
0.382
0.351 0.386
1.6. 16-10-- 0.333
0.371
0.335
0.372
0.335
0.373
0.335
0.372
0.337
0.374
0.337 0.375
15
4.10
0.338
0.375
0.345
0.380
0.351
0.386
0.341 0.376
0.349
0.383
0.353 0.387
1 1.10-5
0.341
0.376
0.347
0.382
0.387 0.353
0.344 0.379
0.352
0.385
0.354 0.389
1.6 10-1
0.366
0.387
0.367
0.388
0.389 0.368
0.364
0.392
0.364
0.392
0.365 0.393
4- 1-10
0.369
0.389
0.372
0.392
0.376 0.396
0.366 0.394
0.370
0.396
0.373 0.399
1 1.10-5 0.370
0.390
0.373
0.393
0.378
0.397
0.367 0.395 0.371 0.397
0.374 0.400
1.6. 16-10-4 0.367
0.387
0.368
0.388
0.369
0.390
0.364 0.393
0.365 0.393
0.366 0.394
4. 10-5 0.370
0.390
0.373
0.392
0.378
0.397
0.367 0.395
0.372 0.398
0.376 0.402
336
1 1.10-5 0.371
0.391
0.375
0.394
0.380
0.399
0.369
0.396 0.373
0.398
0.376 0.402
16. 10 0.367
0.387
0.368
0.388
0.369
0.390
0.365 0.393 0.365
0.393
0.366 0.394
1.10-5 0.371
0.390
0.375
0.394
0.379
0.399
0.368 0.396 0.375
0.400
0.379 0.405
1.10-5 0.372
0.392
0.377
0.395
0.381
0.401
0.371 0.398 0.377
0.401
0.379 0.405
1.6 10-1 0.367
0.388
0.369
0.389
0.368
0.390
0.365 0.393
0.366
0.394
0.365 0.394
15
4.10 0.372
0.391
0.377
0.395
0.381
0.400
0.370 0.396
0.378
0.403
0.382 0.408
1. 10-5
0.374
0.393
0.379
0.397
0.383
0.402
0.373
0.399
0.381
0.404
0.382 0.408
16. 10-4 0.420
0.417
0.421
0.417
0.422
0.418
0.416
0.421
0.416
0.421
0.417 0.421
4-10-5 0.423
0.419
0.425
0.421
0.429
0.425
0.419
0.422
0.422
0.424
0.427 0.427
1 101 0.424
0.420
0.426
0.422
0.431
0.426
0.420
0.423
0.424
0.425
0.429 0.429
1.6 10-1
0.420
0.417
0.421
0.417
0.422
0.419
0.416 0.421
0.416
0.421
0.418 0.422
4. 1.10 0.423
0.419
0.426
0.421
0.430
0.426
0.419
0.423
0.423
0.425
0.428 0.428
720
1-10-5
0.425
0,420
0.428
0.423
0.432
0.428
0,421 0.424
0.425
0.426
0.431 0.430
1.6- 10-4 0.420
0.417
0.421
0.417
0.422
0.419
0.416 0.421
0.417
0.421
0.418 0.422
12
4. 4.10-5
0.424
0.420
0.427
0.423
0.431
0.427
0.420
0.423
0.425
0.426
0.430 0.430
1-10 0.426
0.421
0.429
0.424
0.434
0.429
0.422
0.425
0.427
0.428
0.433 0.432
1.6 10-1 0.421
0.417
0.422
0.418
0.422
0.419
0.416
0.421
0.417
0.421
0.418 0.422
15
4.10
0.425
0.420
0.429
0.424
0.433
0.428
0.421
0.424
0.427
0.428
0.432
0.431
1.10-5
0.427
0.422
0.432
0.426
0.435
0.430
0.423
0.426
0.430
0.430
0.435 0.433
33

---

Published as a conference paper at ICLR 2026
G VISUALIZATION OF TSRBENCH
To provide a comprehensive visual understanding of TSRBench, we present a series of visualizations that illustrate the effects of different corruption types across various severity levels and datasets. These visualizations serve as a qualitative complement to the quantitative results presented in the main paper. Figures 9
12 13 and 14 display examples of time series data from our six benchmark datasets (ETThI, ETTh2, ETTml, ETTm2, Weather, and Electricity) under realistic corruptions at varying severity levels. Each figure shows level shift corruptions (left column) and spike corruptions (right column), demonstrating how these corruptions manifest differently across diverse time series data types.
As the severity level increases from 1 to 5, we can observe the progressive intensification of both corruption types. For spike corruptions, higher severity levels not only produce spikes with greater amplitudes but also increase their frequency throughout the time series. This creates challenging scenarios where models must distinguish between legitimate data points and anomalous spikes that occur more frequently and with larger magnitudes. Level shift corruptions, meanwhile, exhibit two key patterns as severity increases: first, the magnitude of the shifts becomes more pronounced, creating larger deviations from the original signal; second, the duration of these shifts becomes notably wider, meaning the corrupted signal remains in an altered state for longer periods. This temporal extension of corruption particularly challenging for forecasting models that rely on consistent patterns.
These visualizations highlight the statistically grounded nature of our corruption generation process. Rather than arbitrary or manual corruption placement, TSRBench simulates realistic corruptions that preserve the underlying data distribution while introducing controlled perturbations. This approach allows for systematic evaluation of model robustness under conditions that closely resemble real- world scenarios where data quality cannot be guaranteed. The progressive severity scale enables researchers to assess not only whether models are robust corruptions but also to quantify at which corruption intensity their performance begins to degrade significantly, providing valuable insights for deploying these models in practical applications.
H USE OF LARGE LANGUAGE MODELS
In the preparation of this paper, Large Language Model (LLM) was utilized as general-purpose writing-assien assistance tool. The role the LLM was limited to improving the quality of the prose, including enhancing clarity, correcting grammatical errors, and refining sentence structure to ensure the manuscript was articulate and readable.
34

---

Published as a conference paper at ICLR 2026
Level Shift
Exponential Spike
20
20
Severity Level
5
-5
Severity Level 2
20
M Mon
20
yan
10-
10-
Severity Level 3
20
20-
10-
10.
20
20
Severity Level
10
25
25
Severity Level
15-
15-
5-
5-
100 200 300
400
5:00
100
200
300
400
500
Figure 9: ETTh1 dataset with level shift corruptions (left) and spike corruptions (right) across severity levels 1-5. Each row represents a different severity level, demonstrating the progressive intensification of realistic corruptions in the time series data.
35

---

Published as a conference paper at ICLR 2026
Level Shift
Exponential Spike
50-
MmM m M Mwm.
/M m Mw M
Severity Level
45-
M
40
20-
30
24
50-
W M
Severity Level 2
45
NM W Im Mon
M
45.
Mm
/
40-
40-
30-
50
MM I m M. Mom MoW
50-
M
M
Severity Level
45-
45-
MM M
40-
ti
50
MM V m Man
Severity Level 4
M
MmMM
25
20
20
50-
M hayg
M
Severity Level 5
45
M
MMM
Me
30
8.
25
20
100
200
300
40.0
5:00
100
200
300
400
500
Figure 10: ETTh2 dataset with level shift corruptions (left) and spike corruptions (right) across severity levels 1-5. Each row represents different severity level, demonstrating the progressive intensification of realistic corruptions in the time series data.
36

---

Published as a conference paper at ICLR 2026
Level Shift
Exponential Spike
30-
Severity Level
15
10
90-
Severity Level 2
20-
15-
15
Sevenity Level 3
20
5-
25
Severity Level 4
20
5-
Severity Level 5
25
25
15-
10
100
200
300
4000
5:00
100
200
300
400
500
Figure 11: ETTml dataset with level shift corruptions (left) and spike corruptions (right) across severity levels 1-5. Each represents different severity level, demonstrating the progressive intensification of realistic corruptions in the time series data.
37

---

Published as a conference paper at ICLR 2026
Level Shift
Exponential Spike
30
35-
50
50
Severity Level
45
40
88
20
58
50
50
Severity Level
49
45
40-
35
50-
Severity Level
46
40-
38
Severity Level
40
25
35
30
30
15
20
55
50-
Severity Level
30
25
25
20
100
200
300
4000
5:00
100
200
300
400
500
Figure 12: ETTm2 dataset with level shift corruptions (left) and spike corruptions (right) across severity levels 1-5. Each represents different severity level, demonstrating the progressive intensification of realistic corruptions in the time series data.
38

---

Published as a conference paper at ICLR 2026
Level Shift
Exponential Spike
12. 8-
Severity Level 1
8-
6
w W M ma
ww Mhanrey
10
10-
Severity Level
W
2-
8-
Severity Level
W My
W
10
Severity 4
M
w/ 11
Unin
Severity Level
M
6
ww MWm
ww
Whin
6.
100
200
300
400
500
100
200
300
400
500
Figure 13: Weather dataset with level shift corruptions (left) and spike corruptions (right) across severity levels 1-5. Each row represents different severity level, demonstrating the progressive intensification of realistic corruptions in the time series data.
39

---

Published as a conference paper at ICLR 2026
Level Shift
Exponential Spike
4000
4000
37.50
3750
Severity Level
$5.00
10
5500
$2.50
3250
80.00
2500
2250
Severity Level 2.
3750
3750
25:00
3500
3250
3250
80:00
3000
2750
2250
Seventy Level 3
3750
38:00-
M
$2.50
3250
3000
3000
2750
2750
2500
2500
M
4000
MYM
4000
Severity Level 4
3750
2250
30:00
3000
2750
2750
25:0
2500
22.00
4000
4000
3750
3750
Severity Level
30:0
2750
2750
25:0
2500
2250
2250
100
200
300
400
500
100
200
300
400
500
Figure 14: Electricity dataset with level shift corruptions (left) and spike corruptions (right) across severity levels 1-5. Each row represents different severity level, demonstrating the progressive intensification of realistic corruptions in the time series data.
40


Reviews:
[{"review_id": "AYatRHaKjq", "reviewer": "Reviewer_Jv6F", "summary": "The manuscript introduces Local Geometry Attention, an attention mechanism for time-series forecasting that replaces dot-product similarity with a query-specific metric estimated from local Gaussian-process theory, aiming to make attention scores geometry-aware and less sensitive to noisy or anomalous keys. It also proposes TSRBench, a robustness benchmark that injects spike and level-shift corruptions using statistically grounded processes and calibrated severities.", "questions": "(a) Please expand the discussion of non-uniform, locally clustered time-series observations, since this is the core motivation for introducing LGA.\n\n(b) Explain in detail how queries and keys are embedded in the 2D panel in Figure 2 and how this visualization demonstrates a more effective attention mechanism.\n\n(c) Show how the proposed local kernel–covariance compares to vanilla dot-product attention under the same setup, and quantify robustness gains.\n\n(d) In Figure 5, replace training time with FLOPs to make comparisons hardware-agnostic.\n\n(e) Add more recent baselines in Table 2 to better position LGA and TSRBench.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "LGA is designed to improve robustness of attention to local anomalies and noise.\n\nTSRBench is a potentially useful benchmarking tool for researchers to evaluate robustness under controlled corruptions.", "weaknesses": "The description of LGA is not detailed enough. A step-by-step derivation of the attention score and kernel construction is needed to enhance the clarity. Please also provide the tensor shapes at each stage.\n\nThe GP motivation is under-explained: it is unclear how local GP assumptions translate into better attention weights or how the geometric structure is actually learned from data.\n\nThe set of baselines is limited, making it hard to judge whether robustness gains hold against recent strong time-series Transformers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rqaffk21DR", "reviewer": "Reviewer_WvGd", "summary": "This paper studies the robustness of time series forecasting transformer models under realistic corruptions. It first propose a novel attention mecanism derived from local Gaussian process theory called Local Geometry Attention (LGA). It takes into account the geometry of data to be more robust to corruptions in the time series data. It can be implemented by learning a neural network over the query vectors seen during training. Then, the paper introduces TSRBench, a benchmark with realistic corruptions (spikes and level shift) to estimate the robustness of models for forecasting. Experiments over 6 common forecasting benchmarks (with their corrupted versions) and 3 baselines show the performance benefits of the LGA attention incorporated into a PatchTST model.", "questions": "- The LGA is computed by training a neural network on query vectors seen during training. Does it mean that there are two interconnected training loops: one for the model, one for the LGA module? If it is the case, what is the additional computational cost and does it vary with the number of training steps? Could the authors clarify that please? \n- Were the experiments run over several seeds? If yes, could the authors make the standard deviation appear? If no, could the authors please conduct the experiments with 2 additional seeds to have an idea of the significance of the performance improvement?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and the proposed approach well motivated\n- The proposed benchmark is very interesting with sounded corruptions (inspired from ImageNet-C benchmark)\n- The experiments are convincing to show the performance improvement brought by LGA", "weaknesses": "- While the proposed method is sound, it requires training neural networks to compute the Local Geometry Attention. The current submission is missing a computational cost comparison when compared to self-attention\n- Since the main contribution is a novel attention mecanism, it would be interesting to see the comparison to other types of attention in addition to the traditional one (e.g., channel-wise attention [1, 2])\n- Connected to the previous weakness, it would be interesting to compare to more models such as iTransformer [1] or SAMformer [2] that reported robust performance with temporal/spatial and spatial attention respectively.\n- While the benchmarks is interesting, I believe the current proposal would be strengthen with additional models and datasets to clearly show the failure of other models under realistic corruptions\n- It would be interesting to see how LGA behaves when integrated into other models than PatchTST, for instance could it be applied on SAMformer with channel-wise attention or i-transformer with channel and temporal wise attention?\n\nOverall, I find the submission interesting with a well-motivated LGA and robustness benchmark. However, it would be strengthen with additional methods and computational cost comparison.\n\n*References*\n\n[1] Liu et al. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. ICLR 2024\n\n[2] Ilbert et al. SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel Wise Attention. ICML 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vFAY3Ize5M", "reviewer": "Reviewer_ESKs", "summary": "The contribution of this paper is twofold; it presents a method called local geometry attention designed (LGA) to model local structures of time series data in transformer neural networks, and it introduces TSRBench a corruption suite for evaluating robustness of time series models. In the empirical evaluation part of the submission, the LGA models are evaluated using TSRBench.\n\nMore specifically, LGA uses local Gaussian process regression to model the data/representation manifold near the query points of a an attention module. Since, computing the local geometry aware score of the model is computationally prohibitive, a second neural network is trained that predicts the metric tensor.  \nThe TSRBench consists of a method to insert shifts and spikes into time series at 5 different severity levels.\nIn the empirical evaluation, LGA is inserted into different transformer models (with PatchTST being the default choice) to assess robustness of long-term time series forecasts.", "questions": "- Is the robustness of LGA by design, or did it surprise you? If by design, what is the reason or intuition?\n- Would you explain in more detail how the network that predicts the metric tensor is trained. Is this done simultaneously to training the transformer, afterwards, or  alternately? Considering the former, could you explain again, why training such a network is more efficient than computing the metric, or if it only shift compute from inference to training?\n- Would you explain the connection to Riemannian geometry and what the point of the detour to the geodesic distance was. Is the connection only that the score is determined by a bilinear form that depends on the query point?\n- Would you compare TSRBench with this very recent robustness benchmark ( [Janßen et al., arxiv, Oct 2025](https://arxiv.org/pdf/2510.04900) )", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The LGA technique is universal in  the sense that it can easily be integrated into different transformer architectures and likely also in other networks that use attention modules.\n- On the six corrupted time series datasets that were used, LGA (+PatchTST) consistently achieves the best forecasting accuracy especially at large severity levels, indicating that LGA adds robustness wrt shift and spike corruptions.\n- The benchmarking tool comes with predefined and carefully calibrated severity levels which simplify its use.\n- The paper is very well written, and therefore easy to read and understand. (But certain parts need more explanation, see questions)\n- The paper is accompanied by an extensive supplementary material which provides the mathematical background, implementation details and further experiments.", "weaknesses": "- The LGA approach is only evaluated with respect to the (synthetic) TSRBench corruptions, i.e. shifts and spikes. It is not evaluated on other types of corruptions, for example on the anomaly types that are used in (Cheng et al, RobustTSF: ..., ICLR2024) or the synthesized outliers from (Lai et al., Revisiting Time Series Outlier Detection: Definitions and Benchmarks, NeurIPS 2021 Dataset and Benchmark track). One could therefore think that the benchmarks are designed to favor LGA.\n- The LGA approach is only evaluated on forecasting tasks. \n- It seems that the severity levels of TSRBench need to be calibrated for every dataset individually, and the guidelines are quite vague, i.e. \"we ensured that the performance differences between severity levels were neither too large nor too small, and that the degree of noise strictly increased with each level\".   \nMoreover, the resulting hyperparameter choices are model dependent and require training one (or multiple) neural networks per parameter choice. This hinders extending the benchmark to new datasets in a standardized manner, one of the key goals of this work (\"Despite the need, a standardized benchmark for such realistic corruptions in time series remains a significant gap\")\n- The paper makes several implementation choices whose effects are not ablated empirically. This includes learning the metric tensor with an additional neural network (instead of computing it for every query) and using only a diagonal metric. Moreover, while it is stated that both are done because of computational costs, no runtime or complexity analysis of them are presented. \n- For the datasets that are used, the manuscript refers to the Autoformer paper (Wu et al., NeurIPS 2021), but the actual originators of the datasets are not credited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lq4QsjZlep", "reviewer": "Reviewer_9J2N", "summary": "This paper proposes a new attention mechanism for time series forecasting, which is termed Local Geometry Attention. The designed LGA is inspired by local Gaussian process theory and aims to learning query-specific distance metrics and enable the model to learn complex temporal dependencies and enhance resilience to noise. Also, TSRBench, a comprehensive benchmark for evaluating time series forecasting robustness, is proposed.", "questions": "Please refer to the weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The motivation seems clear with intuitive figures (Fig.1 and 2)\n\n2. Clearly written and well-presented with source codes.", "weaknesses": "1. The technical details could be improved in a clearer way.\n\n2. According to the performance reported in Table 2, the improvements led by the proposed method seem to be marginal.\n\n3. the motivation of TSRBench is not clear. And more comprehensive datasets and baselines should be involved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)