PMDformer: Patch-Mean Decoupling Information Transformer for Long-term Forecasting
Authors: Ao Hu
Liangjian Wen
Jiang Duan
Yong Dai
YAN HE
Dongkai Wang
Jun Wang
Yukun Zhang
Ruoxi Jiang
Zenglin Xu
Venue: ICLR 2026 Poster (2026)
Keywords: Long-term Time Series Forecasting; Non-stationary; Patch-Mean Decoupling
Primary area: learning on time series and dynamical systems

Abstract: Long-term time series forecasting (LTSF) plays a crucial role in fields such as energy management, finance, and traffic prediction. Transformer-based models have adopted patch-based strategies to capture long-range dependencies, but accurately modeling shape similarities across patches and variables remains challenging due to scale differences. 
To address this, we introduce patch-mean decoupling (PMD), which separates the trend and residual shape information by subtracting the mean of each patch, preserving the original structure and ensuring that the attention mechanism captures true shape similarities. 
Futhermore, to more effectively model long-range dependencies and capture cross-variable relationships, we propose Trend Restoration Attention (TRA) and Proximal Variable Attention (PVA). The former module reintegrates the decoupled trend from PMD while calculating attention output. And the latter focuses cross-variable attention on the most relevant, recent time segments to avoid overfitting on outdated correlations. Combining these components, we propose PMDformer, a model designed to effectively capture shape similarity in long-term forecasting scenarios. Extensive experiments indicate that PMDformer outperforms existing state-of-the-art methods in stability and accuracy across multiple LTSF benchmarks. The code is available at https://github.com/aohu1105/PMDformer.

Published as a conference paper at ICLR 2026
PMDFORMER: PATCH-MEAN DECOUPLING INFOR- MATION TRANSFORMER FOR LONG-TERN FORECAST-
ING
Ao Hu 1.2
Liangjian Wen't Jiang Duan 1.+ Yong Daist
HE YANG Zenglin 2n2.3t
Dongkai Wang*
Jun Wang! Yukun Zhang4.2
Ruoxi Jiang
Southwestern University of Finance and Economics Shanghai Academy Al for Science
Fudan University *Harbin Institute of Technology, Shenzhen *X-Humanoid Research Institute
Chengdu Everimaging Science and Technology Co., Ltd. fhuao1105, w1 w1j6816, zenglin gmail aomail.coo com
duanj tdw.ufe..uuuuuuuu.u...eee
ABSTRACT
Long-term time series forecasting (LTSF) plays crucial role in fields such energy management, finance, and traffic prediction. Transformer-base models have adopted patch-based strategies to capture long-range dependencies, but ac- curately modeling shape similarities across patches and variables remains chal- lenging due to scale differences. To address this, we introduce patch-mean de- coupling (PMD). which separates the trend and residual shape information by subtracting the mean of each patch, preserving the original structure and ensur- ing that the attention mechanism captures true shape similarities. Futhermore, to more effectively model long-range dependencies and capture variable rela- tionships, we propose Trend Restoration Attention (TRA) and Proximal Variable Attention (PVA). The former module reintegrates the decoupled trend from PMD while calculating attention output. And the latter focuses varralvariable attention on the most relevant, recent time segments to avoid overfitting on outdated corre- lations. Combining these components, we propose PMDformer, a model designed to effectively capture shape similarity in long-term forecasting scenarios. Exten- sive experiments indicate that PMDformer outperforms existing state-of-the-art methods in stability and accuracy across multiple LTSF benchmarks. The code is PMD
available htttt:::: //tptt:////ttbb/b/ com aohul 1 PM
INTRODUCTION
Long-term time series forecasting (LTSF) is a key task in machine learning, with wide applications in areas like energy management (Box & Jenkins 1990), financial markets (Hu et al.) (025c). and
models have drawn inspiration from computer vision (Dosovitskiy et al. 2020). increasingly using patch-based strate- gies (Nie et al. 2023 Zhang & 2023| Chen et al., 20024] Wang et al. 2024c) to better capture long-range dependencies. Most of these approaches treat variables independently (VI) (Huang et al.
traffic prediction (Guo et al.] 2019 Yi et 2023b). Recent Transformer-basee
2025
Lin et al.) 2024), while pariable-dependee (VD) methods (Liu et al.) 2024a Luo & Wang
2024
that model interactions across variables have not yet shown clear gains over VI baselines.
Unlike 2D images with a fixed spatial structure, time series are one-dimensional curve Germain et al. (2024); Hamilton (2020), with the primary focus being capturing shape similarities between patches or variables (Grabocka et al. [0014] Kacprzyk et al. 2024) as well as modeling long-range trend et al. (023). For instance, two patches may share similar trends, such as gradual increases with comparable rates of change. Identifying such shape correspondence helps the model extract temporally consistent patterns and improves forecast accuracy. However, time series data is inher-
ently non-stationary (Fan et al. 2023 Liu et al. 2022b), where patch scales fluctuate wildly across
"Equal contribution.
Corresponding author.
I Projecct leader.

---

Published as a conference paper at ICLR 2026
Original Long Sequence (Before Nermaization)
Attention Weights (Before)
0.36
ETT
1.0
P2
0.29
0.8
P3
0.37
0.25
0.27
0.0
P1
Time
P3
P2
P2
P3
ECL
6:811
Solar
Normalized Patches (After mean subtroction)
Attention Weights (After)
Affestion
0.37
0.25
0.28
0.25
0.376
0.29 0.42
Weather
Traffic
Time
20113
Figure 1: Attention weights for three patches before and after patch-mean decoupling. Scale differences initially obscure true shape similarity, which are clearly revealed after decoupling as PMDformer. The results are the increased (red) or decreased (green) correlations, with analo- averages for all prediction lengths. gous similarity shown in blue for (P3, Pi) and (P3, P2).
Figure Comparison of the MSE of all baselines with our proposed
time. As illustrated in the top panels of Figure I The attention weight of (P1,P)) is higher than that of (P1, P2). despite the more similar shape between P1 and Pg. This occurs due to the different scales among PI, P2 and Ps. which influence the attention weights, thereby failing to reflect true shape similarity. Consequently, the model may learn incorrect similarity relationships, leading to performance degradation. Furthermore, this scale bias is even more pronounced when modeling dependencies between variables, further hindering the effectiveness of VD models.
To balance the scale differences of patches, recent methods have employed Patch Normalization (Liu iet al. 2023b),, which Z-score normalizes each patch by subtracting the mean and dividing by the standard deviation. However, the removal of the standard deviation inadvertently distorts the original shape of the patch. As a result, it hampers the model's ''s ability to identify shape similarities across patches or variables. In this paper, we propose a simple yet effective alternative method called patch-mean decoupling (PMD). We subtract the mean of each patch, which recenters each patch to zero mean and explicitly separates the long-range trend component which encoded in the means of patches from the residual shape information. Unlike Patch Normalization, the original amplitude variations and maintains the intrinsic shape structure, ensuring that the model better captures true shape similarities across patches. As shown in Figure 1 through method attention favors shape-aligned pairs (P1, P2) over shape-unaligni (P1,
method preserves
PMD thus enables more shape-focused attention across patches and variables, revealing true sim- ilarities obscured by scales. For cross-variable shape modeling, existing methods (Luo & Wang 2024 ZZhng& & Yan 2023) often compute interactions
the entire historical window. However, cross-variable relationships are often non-stationary and evolve over time, so recent interactions are the most predictive of future dynamics. For example, in financial markets asset correlations often spike sharply during crises. Relying on the entire historical dependencies introduces substantial noise and redundancy, degrading performance. To address this, we introduce proximal variable at- tention (PVA), which confines self-attention to the most recent patch- the segment most proximal to the prediction horizon. By capturing shape similarities among variables this temporally relevant window, PVA minimizes noise from historical drifts and risk overfitting.
Complementarily, recentering via PMD inherently attenuates the long-term trend signal, potentially overlooking global dependencies. To restore this without disrupting shape matching between tem- poral patches, we propose trend restoration attention (TRA), which explicitly injects the decoupled means (long-range trend information) into the value pathway of the attention mechanism. This seamless integration allows the model to jointly encode local shape patterns and global trend yield- ing more stable forecasts.
Building on above, we propose PMDformer, which combines patch- patch-mean mean decoupling (PMD) mod- ule, Proximal variable attention (PVA), trend reinsertion attention (TRA) and projection layer for
2

---

Published as a conference paper at ICLR 2026
final forecasting. The comparison of predictive aceuracy of our PMDformer and other state of the art models refer to Figure Our contributions are:
We introduce a novel mechanism to decouple trend and residual shape within the attention module via residual mean deduction, enabling more effectively capture shape similarity among temporal patches and varibles.
We introduce proximal variable attention, which focuses on the most recent patch to capture the most relevant shape similarities, mitigating overfitting.
We demonstrate the effectiveness of our approach through extensive experiments on a va- riety of LTSF benchmarks, showing that PMDformer provides more stable and accurate forecasts than current state-of-the-art methods.
RELATED WORK
Deep learning models have demonstrated remarkable performance in long-term time series forecast-
ing. These models can be broadly divided into Transformer-basee models Vaswani et al. (2017): Wu
al. (202); ); Liu et al. (2022a); Zhou et (2022), MLPP-basdd models Zeng et al. (2023);
Li et
(2023; Wang et al. (2024a); Hu et al. (2025b), GNN-based models Huang et al. (2023):
Yi al.
2023a and CNN-basee models Wang et al. 2203 : Eldele et al. (2024): Hu et al. (202aa).
Transformer- time series models. The success of Transformers (Vaswani et al.) 2017) in NLP has inspired their adaptation for LTSF to capture long-range dependencies. Early models treat series as token sequences with efficient attention: Informer (Zhou al. (02I) uses ProbSparse for complexity reduction; Pyraformer (Liu et al., 2022a) employs pyramidal attention; Autoformer (Wu ct al. 2021 adds decomposition; and FEDtormer (Zhou et al. 2022) incorporates frequency blocks. Yet, their efficacy is challenged by simple linear models (Zeng et al. 2023). underscoring needs for
better temporal modeling.
Patch-based time series models. Inspired by vision transformers (Dosovitskiy et al., 2020), re- cent works segment time series into overlapping non-overlapping patches to bolster local semantic capture. Transformer-basee
examples include PatchTST (Nie et al. 2023). which uses variable- independent shared encoders for temporal patch semantics (SOTA LTSF), and Pathformer (Chen et al. 2024) with multi-scale patches and adaptive path selection for dntrwiinerrdepeeddnniiee.. MLP variants like TSMixer (Ekambaram et al. (023) and PatchMixer (Gong al. (003) model patch relations via MLPs, while foundation models such as Moirai (Woo et al. 2024). Timer (Liu et al., 2024b), TimesFM (Das al. 2024), and LLM--bsed (Pan et al.) 2024 Jin et al., [223 leverage patches for pretraining and cross-modal alignment. Recent TimeBase (Huang et al.) 2025) em- ploys orthogonalized patches to reduce redundancy for SOTA efficiency, which further underscores patches' success in LTSF modeling.
Miich-Normalization. Due to the non-stationary nature of time series, some works (Fan et al. 2023 Kim et al. (022) apply normalization to mitigate scale discrepancies and stabilize distribu- tions. Among them, Patch-level normalization works include SAN (Liu et al., (023b). model- agnostic framework that adaptively normalizes slices by removing non-stationarity for flexible fore- casting, and SIN (Han et al.] 2024b), which selectively learns normalization parameters to maximize local invariance and global variability, enabling interpretable long-term predictions. However, these normalization methods distort intrinsic patch shapes by scaling with standard deviation, hindering true shape similarity capture. In contrast, our PMD overcomes through mean subtraction preserve amplitudes.
3 PROPOSED METHOD
We consider the task of long-term time series forecasting, where the goal is to predict the future evolution of multiple correlated variables given their historical observations. Formally, let X {z, E RC t 1, 2...... . L) denote an input sequence of length L, where C' the number of variables. Each x, (x) x7, ..... ...
2)) contains the values of all variables at time t. Given X, the
objective is to forecast the subsequent' T time steps Ý
{5x RC
I + 1 1,...
I. I+ T}.
3

---

Published as a conference paper at ICLR 2026
3.1 THE GENERAL STRUCTURE
Our proposed PMDformer architecture is a unified framework composed of four synergistic mod- ules designed to explicitly decouple the long-term trend from the shape structure, selectively focus on the most relevant inter-variable dependencies, and ensure the accurate restoration of global dy- namics for stable forecasting, as illustrated in Figure 3 (a) Patch-Mean Decoupling (PMD): This module partitions the input time series into non-overlapping patches and explicitly separates each patch into its long-term trend component and its residual shape component. (b) Proximal Vari- able Atfention (PVA): To capture the most relevant cross-variable dependencies, the PVA module focuses its self-attention mechanism only on the tokens of the last (proximal) patch, modeling interactions across all variables. (c) Trend Restoration A Attention (TRA): This module is designed to model the shape similarities across patches. Crucially, it then restores the long-range trend infor- mation into the value pathway, enabling to accurately capture and utilize the overall long- term trend. (d) Projection Layer: This final layer combines the learned temporal representations with the rein- corporated trend information through a fully connected projection produce the final predictions.
3.2 MODEL ARCHITECTURE
Patch-Mean Decoupling (PMD) & Embedding. We first divide the input sequence X {or E RCL, =t=1 into N non-overlapping patches of length S, where N LL/S]. For variable i E [C] and patch index j [N]. the raw patch vector is
Pj= (10- " "()-1)8+2:
2)s)) R.S.
We then compute its temporal mean and the corresponding mecn-decoupled residual:
S
Σ
2 d1z-1)S++s
P 1s:
where 1. is the S-dimensional all-ones vector. Each residual patch is then embedded into a d- dimensional representation through a shared linear projection. To encode location, we add learned positional embeddings to form the Transformer token:
Pj r WE + b 2pp
(3)
where Wg E RSxd bg E Rd, and Zp, E Rd denotes the positional embedding of patch j. By removing patch means before embedding, each patch is centered, which alleviates local inconsisten- cies across patches and variables so that attention mechanism can focus on shape similarities.
Proximal Variable Attention (PVA). Intuitively, accurate time series forecasting hinges on the immediate interactions between variables the most recent time steps, as these dependencies are most indicative of near-term changes. Therefore, the PVA module is designed to concentrate its attention mechanism on the most proximal (i.e., most recent) tokens to model these critical cross- variable relationships.
Let N be the index of the last (most recent) patch. We collect the most recent tokens of all C variables, denoted as P,y = ( J. where each token P E RRd derived from the Patch- Mean Decoupling (PMD) embedding. The PVA then applies Multi-Head Self-Attention (MHSA) exclusively within the set P to effectively capture the cross-variable shape dependencies that are most relevant for forecasting. Following the attention mechanism, Feed-Forward Network (FFN) is employed to enhance the non-linear feature representation:
PN = LayerNorm(MHSA((P)) + PN),
(4)
PN = LayerNorm(FEN(PP)) + PN).
(5)
Tokens from the earlier historical patches. specifically those indexed N maintain their original representation derived from the PMD module. Following the PVA operation, the refined token set Pv is concatenated with these remaining historical tokens along the patch dimension to form the full sequence of shape embeddings, denoted as P E RCxx This deliberate strategy of restricting cross- variable attention solely to the most proximal patch offers dual advantages: enhances model robustness by avoiding spurious long-range couplings from historical noise, and it improves computational efficiency by reducing the complexity from N) to O(C2).
4

---

Published as a conference paper at ICLR 2026
Output Time Series
(c) For Each Variable ::
Projection Layer
(d) For Each Variable i:
y E
R
Query
Trend Restoration Attention
p
Key
Attention
p
Restore
Value
Proximal Variable Attention
Pi
P
(b) Only Among the Last Token (N-th)
Embedding Loyer
(a) For Each Variable i:
P
Pa
P
P
P
p
X =x MAMA
Attention
I RM pt ERN mwwhHHM
Patching &
Patch Meann Decoupling
PC
RevIN
Input Time Series
Figure 3: Overview of the proposed PMDformer. The model comprises: (a) Patch-Mean Decou- pling module re-centers each patch and separates patches into trend and shape components; (b) Proximal Variable Attention operates only on the most recent token to capture variable interac- tions which are most relevant for forecasting; (c) Trend Restoration Attention restores long-range trends after value projections, restoring trend modeling; (d) Projection Linear adds the trend back model long-range trend information for stable and accurate predictions.
Trend Restoration Attention (TRA). Following the refinement of the most proximal tokens by the PVA module, the TRA module aims to capture temporal shape similarities across all historical patches while preserving long-range trend information. This is achieved by applying parameter- shared Transformer encoder (MHSA FFN) along the patch axis for each variable independently.
In this design, the Query(Q) and Key(K) projections operate solely the shape embeddings, ensur- ing that the resulting attention scores A emphasize precise inter inter-patch shape similarity. To counteract the potential loss of global dynamics inherent in shape-focused modeling, rate the per-patch mean (μ') into the Value (V) pathway. The additive reintegration is inspired by residual connections in ResNet (He al., 2016). Concretely, for the i-th variable's 's patch sequence P E R Nxd the computation is defined as:
explicitly incorpo-
Q' = P''e,
P'WK. K.
(6)
Q'(K))
A = Softmax
(7)
Vd
V = p'
(8)
where Wo: WK, Wy are the projection matrices, and p² is the per-patch mean (Eq. 2p. broadcast to match the dimensions of P'Wy. This architectural separation allows the Q/K pathway to model fine-grained local shape dependencies, while the V pathway ensures the preservation of the essential global trend dynamies. The resulting trend-integrated tokens then refined through Feed-Forward Network (FFN) to enhance the temporal representation learning.
Projection Layer. The temporal tokens produced by the TRA module are rich in shape dependen- cies but still require the final restoration of the global trend information for stable and accurate multi-step forecasting. This final step is essential to fully recover the original scale and long- dynamics that were decoupled earlier. To achieve this, before generating the multi-step forecasts, we re-incorporate the per-patch trend means (p²) into the refined shape embeddings:
Y (P'
W. b YERR.
(9)
5

---

Published as a conference paper at ICLR 2026
Here, W. (Nxd)xx and b, E R'' are the weight matrix and bias vector, respectively. The mean u² implicitly broadcast to align with the dimensions of P This final step ensures the model's predictions well-calibrated with the long-range trend observed in the input series.
3.3 THEORETICAL ANALYSIS
Scale Bias Without Patch-Mean Decoupling (PMD) Consider embedding raw patches x r+, where is the residual and y the patch mean. The attention logit between tokens (i,j) is given by:
= q = x MXj= = paps1 M1 14 Mr, + 15 M1 +
MMMr
(10)
cross
residual similarity
where M W W W We and is the all-ones vector. The first three terms depend on the means and can affect even dominate the residual similarity, inducing scale bias.
Proposition 1: Sufficient Condition for Level-Dominated Logits Let be a fixed query. A sufficient condition for the mean-dependent part of Ei to dominate the residual similarity uniformly over keys j is:
Ialleg/ M1| 2 || | + | [M]]]]]] ||rl
Iuy [M ||rlll
(11)
where || |/2 represents the spectral norm. This condition guarantces that the mean-dependent terms outweigh the residual term and cross terms, leading to scale-induced bias in attention. This confirms that attention is biased toward scale when the means are large, which motivates the need for patch- mean decoupling in our method.
4 EXPERIMENT
4.1 EXPERIMENT SETUP
Datasets We conduct experiments on widely-used and publicly available real-world datasets. These include: ECI[] Traffic] Weathe Solay ETTh1, ETTh2, ETTml, and ETTm Following the experimental protocol established in prior work (Wang et al. [2024b; Qiu et al. 2024 Liu al.) 2023a), we partition the datasets into training, validation, and test sets with the following ratios: 6:2:2 for the four ETT datasets and 7:1:2 for the remaining datasets. The detailed statistics of each dataset are summarized in Table
Table 1: Characteristics of Long-term Time Series Datasets. This table summarizes key attributes of each dataset, including the application domain; the number of variables; total time points; data split ratios for training, validation, and testing and sampling interval.
Domain
Electricity
Weather
Energy
Transportation
Dataset
ETThl
ETTh2
ETTml
ETTm2
ECL
Wcather
Solar-Energy
Traffic
Variables
7
7
321
21
137
862
Time Points
14,400
14,400
$7,600
57,600
26,304
52,696
52,560
17,544
Split Ratio
6:2:2
6:2:2
6:2:2
6:2:2
7:1:2
7:1:2
7:1:2
7:1:2
Sampling
hr
1 hr
15 min
15 min
hr
10 min
10 min
hr
Baselines
We compare PMDformer against baselines, including state-of-the-ar (SOTA) long-
term forecasting models: TQNet (Lin et al. 2025), TimeBase (Huang al. (2025). SOFTS Han
et al. 2024a), TSF (Lin et al.) 2024). ModernTCN (Luo & Wang 2024). iTransformer Liu
al. 2024a). TimeMixer (Wang et al. 2024a). and PatchT'ST (Nie al.
2023
hutps//pems.dor.cc.ggg
https//wwww.bgeeeeeannmgg.eeeeee

---

Published as a conference paper at ICLR 2026
Table 2: Comprehensive results for multivariable time series forecasting with a lookback window of 720 time steps. Performance metrics for TQNet (Lin et al. 2025) and SOFTS (Han et al. 2024a) were obtained through our experiments, while results for other methods were sourced from Time- Base (Huang et al. 2025). The best results are highlighted in bold, and the second-best are indicated with underlining.
Models
PMDformer
TQNet
TimeBuse
SOFTS
SparseTSF
ModemTCN
iTranfornee
TimeMixer
PatchTST
Year
OUTS
2025
2014
2024
2014
2025
Matric
MSE
MAE
MSE
MAF
MSE
MAE
MSE
MAF
MSE
MAF
MSE
MAR
MAF
MAE
MSE
MAF
0.122
0.143
0.130
0.133
0.220
0.139
0.239
0.151
0.135
0.233
0.347
0.240
0.144
0.231
0.151
0.153
0.160
0.155
0.250
0.155
0.253
0.256
0.256
0.166
0.261
0.169
0.182
0.171
0.162
0.267
0.270
0.267
0.377
0.272
0.194
0.291
0.307
0.224
0.310
0.208
0.300
0.192
0.289
0.204
0.301
0.339
0.313
0.235
0.259
Avg
00.448
0.241
0.164
0.261
0.167
0.258
0.175
0.268
0.180
0.364
0.158
0.255
0.166
0.264
0.170
0.272
0.109
0.266
00.388
0.212
0.398
0.297
0.355
0.389
0.368
0.382
00267
0.275
0.39%
0.204
0.363
0.250
Ttallic
192
0.367
0.227
0.397
0.277
0.403
0:271
0.365
0.258
0.399
0.272
0.393
0.271
0.393
0.283
0.404
0.295
0.382
0.258
336
01,779
0.235
0.4115
0.279
0.417
0.27M
0.300
0.276
0.417
0.279
11406
0.277
0.409
0.292
0.419
0.302
0.399
0.268
730
0.426
0.262
0.448
0. 104
0.456
0. 0.208
0.420
0.294
0.449
0.789
0.452
0.505
0.450
0.314
0.458
0.309
0.452
0.289
Avg
0,378
0.234
0:412
0.289
0.488
0.279
0.365
0.271
0.414
0.280
0.408
0,280
0:407
0.291
0.419
0,300
0.394
0.265
96
0.141
0.181
0.180
0.213
0.146
0. 198
0.165
0.219
0.174
0.231
0.155
0,210
0.159
0.212
0.163
0.223
0.149
0.199
Weather
192
0,1385
0.226
0.212
0.261
0.185
0.241
0.213
0.255
0.216
0.267
132005
0.254
0.203
0.252
0.201
0.254
0.193
0.243
336
0,236
0.274
0,260
0.299
0.236
0.261
0.272
0.305
0.260
0.299
13255
0.290
0.253
0.291
0.258
0.300
0.240
0.281
730
01305
0,323
0.328
0.343
0.309
0.331
0,380
0,371
0.325
0.345
0,517
0.336
0.317
0.337
0.339
0.548
0,312
0.354
Avg
0.237
0,251
0.240
0.279
0.219
0.263
0.258
0.268
0.344
0.286
0.233
0.273
0.235
0.275
0.238
0.281
0.224
0.264
96
0.160
0.193
0.181
0.342
0.179
0.348
0.192
0.250
0.205
0.341
0.196
0.258
0.217
0.255
0.233
0.271
0.205
0.250
Solar
0.178
0.211
0,203
0.261
0.213
0.259
0.197
0.259
0.215
0.265
0.221
0.290
0.208
0.257
0.238
0.293
0.227
0.260
0.190
0.218
0.219
0.222
0:212
0.213
0.276
13340
0.288
0.309
0.301
0.225
0.290
0.196
0.761
0.235
0.217
0.032
0.372
0.346
0.290
0.319
0.319
0.291
Aug
0.211
0.209
0,264
0.216
0.254
00200]
0.216
0.264
0,227
0.233
0.285
0.296
0.227
0.275
0.356
0,388
0.379
0,404
0.349
0.384
0.389
0.417
0,262
0.389
0,380
0.606
0.421
0.441
0.377
0.908
BT
192
316
0:320
0.429
0.184
0,441
0.45
0.387
0.608
0,418
0,410
Po277
0.866
0.458
0-443
0
9:12
0:128
0550
0:450
00428
0,456
0.244
0.466
0.469
0,87
0,448
0.465
0.400
0,413
0.4%6
0.651
0.646
020
0.45
0.4%
0.499
0
506
0.40
0.446
0.468
0.491
0.426
0.448
0.40
0.484
0.545
0.532
0.475
0.500
0.455
0.475
Avg
0.401
0.423
0.440
0.452
0.396
0.415
0.483
0.453
0,407
0.419
0.431
0:442
0.454
0.467
0.454
0.474
0.420
0.440
96
0.369
0,229
0.288
0.354
0.292
0.345
0.309
0.365
0.296
0.346
0,73
0:341
0,305
0.361
0.315
0.380
0.276
0.339
BTTh2
192
0.333
0,533
0.377
0.403
0.339
0.387
0.375
0,405
(1300
0.227
0.385
0,405
0.421
0.383
0.415
0.342
0,385
336
0.357
0.206
0.377
0.415
0.358
0.410
0:490
0-461
0160
0 198
1030
0:424
0,11
0.436
0.585
0.438
0.364
0.405
720
0.390
0.429
0.424
0.452
0.400
0.448
0.541
0.467
03383
0:425
0/408
0:448
0.448
0.470
0.432
0.471
0.395
0.434
Avg
0.337
0.382
0.367
0.406
0.347
0.398
0.397
0.425
0.348
0.287
0,347
0.391
0.392
0.422
0.379
0.426
0.344
0.391
95
0.379
0.328
0.296
0.309
0.311
0.381
0,503
0.361
0.314
0.359
03313
0.357
0.315
0.369
0.332
0.394
0.246
0.352
ETTml
192
01333
0.358
0.357
0.374
0.338
0.371
0.356
0.377
0.348
0.376
0354
0:317
0.349
0.388
0.355
0.308
0.335
0.373
336
0,361
0.383
0.389
0.393
0.161
0. 135
0,384
0.407
0.368
0.386
0.572
0.391
0.381
0.409
0.386
0.416
0.366
0.394
720
0,416
0.447
0.415
0.435
0.420
0.439
0.457
0.421
Avg
0.346
0.362
0.357
0.365
0.362
0.384
0.387
0.401
0.414
0.385
0.1155
0.169
0.167
0.259
0.185
0.274
0.167
0.259
0.259
0.179
0.274
0.285
0.260
ET1m2
0.213
0.252
0.251
0.219
0.256
0.219
0.297
0.244
0.312
0.314
0.329
0.319
0.298
0.267
0.282
0.354
0.366
0.271
0.330
0.362
0.353
720
0.347
0.373
0.371
0.353
0.392
0.405
0.253
0.280
0.362
0.393
0.387
0.407
0.412
0.352
0.386
Avg
0.346
0,304
0.263
0.325
0.253
0.312
0.293
0.341
0.253
0.312
0.204
0.326
0.279
0.338
0.347
0.251
0.319
1" Count
32
0
7
2
0
0
0
Setups Consistent with prior research (Huang et al. 2025). we use an input length L of 720 and evaluate prediction lengths T of {96. 192, 336, 720j. Results for TimeBase, SparseTTSF, iTrrnss- former, TimeMixer, and PatchTST are derived from the TimeBase study, while other outcomes are from our own experiments. All experiments are conducted using PyTorch (Paszke et al. 2019) on an NVIDIA A100 80GB GPU. The Adam optimizer (Kingma) 2014) is employed, with learning rates chosen from {2e-4, 5e-4, Ie-3, 1e-2). The number of patches N adjusted based on the requirements of each dataset.
4.2
MAIN RESULTS
Table summarizes the quantitative results for long-term time series forecasting across multiple prediction horizons and datasets. As shown, our proposed PMDformer achieves the lowest Mean Squared Error (MSE) and Mean Absolute Error (MAE) on 7 out of 8 real-world datasets, outper- forming all baselines in the majority of cases. This success is directly tied to PMDiormer's ability to overcome fundamental limitations in existing architectures.
Specifically, compared to the patch-based model TimeBase, PMDformer yields an average MSE reduction of 5.68% and MAE reduction of 6.61%. This improvement stems from our method's s capacity to indentify meaningful shape similarities across patches, a capability that TimeBase's s orthogonal patch selection inherently sacrifices reduce redundancy. Moreover, against TQNet,
7

---

Published as a conference paper at ICLR 2026
Table Ablation study on PMD module. We assess different modules for patch-wise normalization,
along with removing PMD module. Results are averaged across all prediction horizons.
ETTh2
ETTml
Weather
Traflic
Solar
Design
Norm
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
PMDformer
PMD
0.337
0.382
0.346
0.371
0.217
0.251
0.378
0.234
0.181
0.211
w/ sidev
0.354
0.392
0.347
0.370
0.218
0.252
0.396
0.259
0.205
0.221
Replace
SAN X
0.360
0.403
0.353
0.380
0.225
0.275
0.392
0.273
0.182
0.235
0.359
0.394
0.347
0.370
0.223
0.260
0.397
0.258
0.199
0.212
Table 4: Ablation studies on TRA and PVA modules in PMDfommer: Performance impacts of re- placements, removals, and order swaps across ETTh2, ETTml, Traffic, and Solar datasets. Results are averaged across all prediction horizons.
ETTh2
ETTml
Traffic
Solar
Design
TRA
PVA
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
PMDformer
Last Token
0.337
0,382
0.346
0.371
0.378
0.234
0.181
0.211
All Token
0.340
0.384
0.354
0.375
0.380
0.239
0.186
0.214
Replace
Self-attention
Last Token
0.345
0.386
0.352
0.372
0.388
0.251
0.196
0.217
Swap Order
0.342
0.385
0.350
0.372
0.379
0.235
0.188
0.216
Last Token
0.344
0.381
0.347
0.372
0.410
0.270
0.215
0.226
w/o
X 0.340
0.383
0.347
0.371
0.386
0.240
0.194
0.214
X
X 0.346
0.384
0.351
0.372
0.426
0.288
0.222
0.230
PMDformer achieves an average MSE reduction 8.62% and MAE reduction of 9.96%. TQNet's fixed periodic queries constrain its ability to handle diverse cycles, whereas PMDformer's adaptive proximal variable attention offers greater flexibility in modeling variables' shape similarities. Com- pared to the Transformer-basee iTrrnsffrmerr PMDformer delivers an average MSE reduction of 11.44% and MAE reduction of 12.38%. iTransformer captures dependencies among variable tokens embedded from the entire historical sequence, which can lead to overfitting early, weakly relevant variable relationships that degrade future predictions. In contrast, our PVA module succeed to avoid this by focusing the shape similarities of variables within the most nearest patch.
PMD Module Analysis. We assessed the effectiveness of the PMD module through extensive ablations conducted on five non- con-stationary benchmarks: ETTh2, ETTm1, Weather, Traffic, and So- lar (Wen al., 2023 Kim al. (025). Using fixed input length of 720, we tested the model's per- formance across various prediction horizons (96, 192, 336, and 720) against several patch-wise nor- malization variants: (i) mean-varian variance standardization ('w/ stdev'), (ii) utilizing the Scale-Adaptive Normalization (SAN) Liu et al. 2023b) method, and (iii) removing the PMD module entirely. As presented in Table 3 the PMDformer consistently achieves superior accuracy across all datasets. We attribute this advantage to the PMD module's 's per-patch centering mechanism, which effectively pre- serves crucial intra-patch shape information. This preservation allows the Transformer architecture specifically concentrate its attention modeling shape similarity. Furthermore, by explicitly injecting the patch mean as separated trend component into the Transformer pathway, PMDformer uniquely positioned to accurately capture and model long-term trends. In stark contrast, SAN explicitly decouples the scale and residual components for independent prediction. Since global scale estimation is inherently unstable in highly non-stationary series, this rigid decoupling under- mines the essential joint modeling of scale-se scale-shape interactions, consequently leading to overfitting and weaker generalization capabilities.
TRA& PVA Analysis. To assess the effectiveness of the TRA and PVA modules, we conducted ablation studies on the ETTh2, ETTml, Traffic, and Solar datasets. For the TRA module, we tested two alternatives: replacing it with standard self-attention or removing it entirely. For the PVA mod- ule, either modified it to compute variable shape similarity across all patches or removed the module completely. Additionally, we investigated structural variant that swaps the sequential order of the modules. The experimental outcomes summarized in Table 4
8

---

Published as a conference paper at ICLR 2026
ETTh1
Salar
ETTN2
ECL
0.44
0.21
0.42
0.10
0.18
0.33
0.15
0.37
0.300
6.86
0.3k
0.27
0.90
11494
16:24
Top_k Neorest Patches
Top_5 Necrest Patchas
Patch Size
Patch 5ix
Figure 4: Parameter Sensitivity Analysis. (a) Selection of the number of k nearest patches to the prediction sequence for capturing inter-variable dependencies on these patches. Superior and more stable performance is achieved when k I. (b) Different patch sizes are used to partition the input sequence, and a moderate patch size yields the optimal choice.
The results unequivocally show that PMDformer consistently outperforms all ablated variants across every dataset and configuration. When TRA is replaced with standard self-attention, performance degrades significantly because the crucial long-term trend information is neglected. Similarly, when PVA is forced to compute variable-wise shape similarity across all historical patches, performance decreases. This confirms our hypothesis that early variable relationships are often only weakly or spuriously correlated with the predictive sequences, justifying PVA's proximal focus. Furthermore, removing both TRA and PVA results the largest performance drop observed, emphatically high- lighting the dual importance of TRA in modeling temporal patch shapes and long-range trends, and PVA in capturing relevant variable-wise shape similarity. Finally, swapping the original or- der of TRA and PVA also causes notable performance degradation. When TRA is applied first, it compresses patch information too early, making it harder for the subsequent variable modeling to identify meaningful cross- variabl dependencies.
4.3 PARAMETER SENSITIVITY ANALYSIS
Patch Count for Cross-Vari Variable Modeling. We evaluate the impact of capturing variable patterss within different numbers of patches, where k E 21,2,355 2,5,5,7 7, 10). For each setting, the nearest patches future sequences are selected capture the shape similarity of variable, thereby further validating the effectiveness of PVA. Experiments are conducted on the ETTh1 and Solar datasets. As shown in Figure 4 (a), the mean squared error (MSE) exhibits an overall upward trend as gradually increases on the ETThl dataset. On the Solar dataset, this increase is more pronounced when predicting 192, 336, or 720 steps ahead, because future sequences are more weakly correlated with early variable relationships. Moreover, the MSE curves show some fluctuations, indicating that different values across all four prediction horizons, using k 1 yields more stable performance compared with larger k. This is because the nearest patch is typically more closely aligned with the target sequence to be predicted, making it more beneficial for accurate modeling.
may lead to more significant differences in prediction performance. contrast,
Patch Size. Different patch sizes lead to varying degrees of distinction among patches. To investi- gate this, we evaluate multiple patch sizes {8, 16, 24, 48, 72, 120) on the ETTh2 and ECL datasets. As shown in Figure E (b), both overly small and overly large patch sizes fail to deliver optimal performance. This is because excessively small patches provide insufficient shape information to distinguish similarity, making it difficult for the attention mechanism to capture underlying tem- poral dependencies or genuine variable correlations. Conversely, overly large patches reduce the number of tokens, thereby limiting the model's ability capture long-range dependencies. Based on these observations, we find that moderate patch sizes, particularly within (24, 48, 72), achieve a better trade-off and yield more robust performance.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this paper, we tackle challenges in long-term time series forecasting by emphasizing true shape similarities hidden by scale variations in non-stationary data. Our patch-mean decoupling (PMD) separates trends from residual shapes while preserving amplitudes, enabling shape-focused atten- tion across patches and variables. Integrated with proximal variable attention (PVA) for recent vnter-variable dependencies and trend restoration attention (TRA) for global trend reintegration. Ex- periments on LTSF benchmarks show PMDformer surpasses state-of-the-art baselines in accuracy and stability, underscoring the value of shape-centric Transformer designs. Future directions in- clude scaling to higher-dimensional multivariate data and multimodal integrations for applications in energy, finance, and traffic.
ACKNOWLEDGEMENT
This work supported by the Major Science and Technology Special Project of the Sichuan Provincial Department of Science and Technology (Grant No. 2024ZDZX0002), the Sichuan Provincial Innovation Group Project (Grant No. 2024NSFTD0054) Fundamental Research Funds for the Central Universities (JBK20251 1081). the Blockchain Research Center of China, the Natu- ral Science Foundation of China (Grant No. 62502397), the National Natural Science Foundation of China (Grant No. (2471197), and the Sichuan Provincial Philosophy and Social Science Fund (Grant No. SCJJ25ND091).
REFERENCES
George Edward Pelham Box and Gwilym Jenkins. Time Series Analysis, Forecasting and Control. Holden-Day, Inc., 1990.
Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting, arXiv preprint arXiv:2402.05556 2024.
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning, 2024.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 10.00//9999 1929, 2020.
Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mip-mixer model for multivariate time series forecasting. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023.
Emadeldeen Eldele. Mohamed Ragab, Zhenghua Chen, Min Wu, and Xiaoli Li. Tslanet: Rethinking transformers for time series representation learning. arXiv preprint 244.:2404.08472. 2024.
Wei Fan, Pengyang Wang, Dongkun Wang, Dongjie Wang, Yuanchun Zhou, and Yanjie Fu. Dish-ts: general paradigm for alleviating distribution shift in time series forecasting. In Proceedings of the AAAI conference on artificial intelligence, 2023.
Thibaut Germain, Samuel Gruffaz, Charles Truong, Alain Durmus, and Laurent Oudre. Shape analysis for time series. Advances neural information processing systems, 37.95607-95638.
2024.
Zeying Gong, Yujin Tang, and Junwei Liang. Patchmixer: A patch-mixing architecture for long- term time series forecasting, arXiv preprint (iXxx:231000065:: 2023.
Josif Grabocka, Nicolas Schilling, Martin Wistuba, and Lars shapelets. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 391-401, 2014.
Learning time-series
10

---

Published as a conference paper at ICLR 2026
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial-
temporal graph convolutional networks for traffic flow forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
James D Hamilton. Time series analysis. Princeton university press, 2020.
Lu Han, Xu- Yang Chen, Han-Jia Ye, and De- De-Chuan Zhan. Softs: Efficient multivariate time series forecasting with series-core fusion. arXiv preprint arXiv:2404. 14/97, 2024a.
Lu Han, Han-Jia Ye, and De-Chuan Zhan. Sin: Selective and interpretable normalization for long- term time series forecasting. In Forty-firsi International Conference on Machine Learning, 2024b.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.
Ao Hu, Liangjian Wen, Yong Dai, Shiyi Qi, Jun Wang, Zhi Chen, Xun Zhou, Dongkai Wang, Zenglin Xu, and Jiang Duan. Timecnn: Refining cross-variable interaction on time point for time series forecasting. Neural Networks, 2025a.
Ao Hu, Liangjian Wen, Jiang Duan, Yong Dai, Dongkai Wang, Shudong Huang, Jun Wang, and Zenglin Xu. Fdnet: High-frequency disentanglement network with information-theore guid- ance for multivariate time series forecasting. Pattern Recognition, 2025b.
Yifan Hu, Yuante Li, Peiyuan Liu, Yuxia Zhu, Naiqi Li, Tao Dai, Shu tao Xia, Dawei Cheng, and Changjun Jiang. Fintsh: A comprehensive and practical benchmark for financial time series forecasting, arXiv preprint arXiv:2502.18834 2025c.
Qihe Huang, Lei Shen, Ruixin Zhang. Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. NeurlPS, 2023.
Qihe Huang. Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang, and Yang Wang. Timebase: The power of minimalism in long-term time series forecasting. In Proceedings the Forty-Seconts International Conference on Machine Learning (ICML), 2025.
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin- Yu Chen, Yux- uan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728. 2023.
Krzysztof Kacprzyk, Tennison Liu, and Mihaela van der Schaar. Towards transparent time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
HyunGi Kim, Siwon Kim, Jisoo Mok, and Sungroh Yoon. Battling the non-stationarity in time series forecasting via test-time adaptation. AAAI, 2025.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Re- versible instance normalization for accurate time-series forecasting against distribution shift. In International conference on learning representations, 2021.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint 2014.
Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation linear mapping. arXiv preprint 2rXiv:2305.1072 2023.
Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with Ik parameters. In Forty-first International Conference Machine Learning. 2024.
Shengsheng Lin, Haojun Chen, Haijie Wu, Chunyun Qiu, and Weiwei Lin. Temporal query network for efficient multivariate time series forecasting. In Forty-second International Conference on Machine Learning, 2025.
11

---

Published as a conference paper at ICLR 2026
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and fore- casting. In ICLR, 2022a.
Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang. Lei Bai, Chao Huang. Zhenguang Liu, Bryan Hooi, and Roger Zimmermann. Largest: A benchmark dataset for large-scale traffic forecasting. In Advances in Neural Information Processing Systems, 2023a.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long, Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances neural information processing systems, 35:
9881-9893, 2022b.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. ICLR, 2024a.
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Generative pre-trained transformers are large time series models. arXiv preprint arXiv:2402.02336. 2024b.
Zhiding Liu, Mingyue Cheng, Zhi Li, Zhenya Huang, Qi Liu, Yanhu Xie, and Enhong Chen. Adap- tive normalization for non-stationary time series forecasting: A temporal slice perspective. In seirth-eeventh Conference on Neural Information Processing Systems, 2023b.
Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The twelfth international conference on learning representations, 2024.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023.
Zijie Pan, Yushan Jiang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka, and Dongjin Song. S'ip-Ilm: Semantic space informed prompt learning with 11m for time series forecasting, In Forty-first International Conference Machine Learning, 2024.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances neural information processing systems, 32, 2019.
Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoy- ing Zhou, Christian S Jensen, Zhenli Sheng, et al. Tfb: Towards comprehensive and fair bench- marking of time series forecasting methods. arXiv preprint arXiv:2403.2000 20150, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- fion processing systems, 2017.
Huiqiang Wang, Jian Peng, Feihu Huang. Jince Wang, Junhui Chen, and Yifei Xiao. Mien: Multi- scale local and global context modeling for long-term series forecasting. In ICLR, 2023.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and JUN ZHOU. Timemixer: Decomposable multiscale mixing for time series forecasting. In International Conference Learning Representations (ICIR), 2024a.
Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Yong Liu, Mingsheng Long, and Jianmin Wang. Deep time series models: A comprehensive survey and benchmark. arXiv preprint arXiv:2407.1320 2024b.
Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Yong Liu, Yunzhong Qiu, Haoran Zhang, Jianmin Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with ex- ogenous variables. Advances in Neural Information Processing Systems, 2024c.
Qingsong Wen, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan, et al. Onenet: Enhancing time series forecasting models under concept drift by online ensembling. Advances in Neural Information Processing Systems, (9869999-69988, 2023.
12

---

Published as a conference paper at ICLR 2026
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In JCML, 2024.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans-
formers with auto-correlation for long-term series forecasting. Advances neural information processing systems, 34:22419-22455 2021.
Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from pure graph perspective, Advances in neural information processing systems, (6669638-96666 2023a.
Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. In Thirry-seventh Conference on Neural Information Processing Systems, 2023b.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? AAAI, 2023.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. The eleventh international conference on learning representations, 2023.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pp. 27268-2277777 27286. PMLR, 2022.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 EFFICIENCY ANALYSIS
To evaluate the efficiency of our model in handling complex tasks, we conduct experiments under two settings: varying the number of variables and varying the input length. In the first setting, we fix the input length 720 and change the number of variables; the second setting, we fix the number of variables 100 and test PMDformer with different input lengths. The batch size is set to I in all experiments. The results are shown in Figure 5 Under both settings, compared with recent popular models such PatchTST (Nie et al. 2202), iTransformer (Liu et al. 2024a), and ModemICN (Luo & Wang 2024), PMDformer requires significantly less GPU memory, thereby reducing the overall computational cost.
former
PatchTST
ModernTON
iTransformer
10*
101
Memory (MB)
Memory (MB)
10a
10a
10a
10a
100
500
3000
146
480
960
1800
3400
5400
Number of Variables
Sequence Length
(a)
(b)
Figure 5: (a) Comparison of memory usage with varying number of variables C. (b) Comparison of memory usage with varying input sequence length L. PMDformer consistently requires the lowest memory.
Ground Truth PMD
w/ PMD
Value
0
100
200
300
400
Time Steps
Figure 6: Comparison on synthetic data. The ground truth alternates between pulse and sine shapes with varying scales. The "w/o PMD' yields smoothed outputs and struggles to recognize the shape similarity, while 'w/ PMD' effectively the shapes and trends.
A.2 COMPARISON ON SYNTHETIC DATA
To further validate the effectiveness of our PMD module, we conduct an experiment on a synthetic dataset. This dataset consists of patches alternating between different shapes: sharp pulse wave
14

---

Published as a conference paper at ICLR 2026
with large amplitude and smooth sine wave with small amplitude. To simulate non-stationary time series, the patches exhibit varying scales and are augmented with moderate noise. We compare a standard patch-based Transformer (w/o PMD) against coupling module (w/ PMD). As illustrated in Figure the "w/o PMD' model struggles to recognize true shape similarities due to scale differences between patches, leading predictions that resemble mostly smooth curves with inadequate trend fitting. In contrast, our 'w/ PMD model, by removing scale factors, enables attention focus more effectively on intrinsic shapes, resulting in predictions that better capture both the underlying patterns and long-rang trends.
model incorporating the patch-mean de-
B ON THE USE OF LARGE LANGUAGE MODELS
The authors used large language models (LLMs) exclusively for language polishing and minor rephrasing during the final writing stage. All scientific content, ideas, and initial drafts were created entirely by the authors without any text improved by LLMMs was carefully checked and edited by the authors. L.LMs played no role in developing research questions, designing experiments, analyzing results, or any other aspect of the research itself.
15


Reviews:
[{"review_id": "twXBXTyerF", "reviewer": "Reviewer_jQS8", "summary": "The paper proposes PMDformer, a Transformer-based model for long-term time series forecasting that preserves cross-patch and cross-variable shape similarities despite scale differences. It introduces three components: Patch-Mean Decoupling (PMD) to separate trend from residual shape by subtracting each patch’s mean; Proximal Variable Attention (PVA) to emphasize recent, relevant cross-variable relationships and mitigate overfitting to outdated correlations; and Trend Restoration Attention (TRA) to reintegrate global trend into the attention mechanism without distorting shape. The authors claim PMDformer achieves superior stability and accuracy over state-of-the-art baselines across multiple LTSF benchmarks.", "questions": "please see weakness", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is well written and easy to follow.\n\nThe proposed method demonstrates superior performance compared to several existing baselines.", "weaknesses": "The paper’s motivation is unclear: there is no well-defined objective or problem statement, and the connection between the proposed method and the problem it aims to solve is not clearly articulated.\n\nThere is little theoretical or empirical justification for the design of the proposed PMDFormer; the choice of each component appears arbitrary and based on empirical intuition, with no systematic evaluation.\n\nEfficiency is not discussed: the paper lacks a complexity analysis (parameter count, runtime, and memory usage) relative to the baselines.\n\nThe LLM usage statement is not in the original manuscript.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D5EkaY0JZP", "reviewer": "Reviewer_pzqK", "summary": "This work proposes a Transformer model named PMDformer for Long-Term Time Series Forecasting. The core idea is to decouple the trend and shape information of each time patch via Patch-Mean Decoupling (PMD), thereby preserving the original amplitude structure and avoiding the distortion of shape information caused by traditional normalization methods. Furthermore, the model introduces the Proximal Variable Attention (PVA) and Trend Restoration Attention (TRA) modules, which are designed to capture the most relevant short-term dependencies among variables and restore global trend information, respectively.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The ideas of this work are easy to understand. The description and presentation are clear.\n2. This work conducts ablation experiments for the three core modules—PMD, PVA, and TRA—on multiple datasets, validating the effectiveness of each module.", "weaknesses": "1. The foundational premise of the PMD module is not fully convincing. According to Figure 1, the original patches (P1, P2), which have more similar means, receive a lower attention score than (P1, P3). This observation appears to contradict the authors' claim that \"the scale differences initially obscure true shape similarity\", as the patches with similar scales (P1, P2) are not assigned higher attention. This inconsistency raises questions about the necessity and motivation of the proposed decoupling.\n2. In the TRA module, the operation of adding μ to V is not sufficiently motivated. Directly adding the raw, unprojected trend mean μ to the projected value V forcibly mixes vectors from disparate spaces—the original data space and the projected feature space.\n3. The \"double addition\" of the trend in the architecture appears redundant. The trend term μ is added in the TRA module and again at the final projection layer. This design is not well-principled and risks over-emphasizing the trend component, potentially distorting the learned representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zVtezARD0i", "reviewer": "Reviewer_rABD", "summary": "This paper proposes PMDformer, targeting the core issue that shape matching is often dominated by scale. It adopts Patch-Mean Decoupling (PMD), which removes only the mean of each patch, applies Proximal Variable Attention (PVA) to perform cross-variable attention on the nearest patch to the forecasting window, and uses Trend Restoration Attention (TRA) to inject trend information back into the Value branch without disturbing the Q/K shape alignment, thereby achieving a unified design that separates and then fuses “shape” and “trend.” Across eight datasets and multiple forecasting horizons, PMDformer shows stable improvements over strong baselines, and ablation studies validate the necessity of the three modules and their ordering.", "questions": "Q1. After injecting trend into the Value branch, could residual pathways indirectly affect the attention output and thus interfere with shape alignment?\n\nQ2. Beyond a fixed k=1, can a gated/learned scheduler adaptively determine the size of the proximal window or whether to include cross-variable attention over earlier patches?\n\nQ3. How do you rule out—or control for—the confounding effects introduced by the patch segmentation choice?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The problem is clearly defined, and each module is well-motivated by the design goals.\n\nS2. The paper includes a formal derivation of the “mean-dominance” condition and multi-module ablations; the experimental design is fairly complete, and PMDformer performs well.\n\nS3. The presentation is clear and the code structure is easy to follow.", "weaknesses": "W1. PMD is conceptually close to RevIN (reversible instance normalization that removes mean/variance per series) and to decomposition-based approaches such as DLinear and Autoformer (alleviating scale/distribution shift or decoupling trend and seasonality). The paper could strengthen the analysis to emphasize the differences.\n\nW2. Restricting cross-variable modeling to only the nearest patch helps suppress noise and overfitting, but may lose interpretable dependencies when long-lag cross-variable effects exist.\n\nW3. The similarity of patch-level trends depends heavily on the patch segmentation scheme, yet the paper does not analyze this sensitivity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N2DbGeeHha", "reviewer": "Reviewer_1ngW", "summary": "This paper proposes PMDFormer, a novel Transformer-based model for Long-term Time Series Forecasting (LTSF). The key insight is that the attention mechanism in existing patch-based models is often biased by the scale (amplitude) of patches, hindering its ability to capture true shape similarities. To address this, the authors introduce three core components: 1) Patch-Mean Decoupling (PMD), which centers patches by subtracting their mean to separate trend from shape; 2) Proximal Variable Attention (PVA), which focuses cross-variable modeling only on the most recent patch to avoid noisy historical correlations; and 3) Trend Restoration Attention (TRA), which reintegrates the global trend back into the attention mechanism. Extensive experiments on eight benchmarks show that PMDFormer achieves state-of-the-art performance, outperforming recent strong baselines. Ablation studies and theoretical analysis validate the design of each component.", "questions": "1. The PVA module operates on the embedded tokens after PMD. Given that PMD centers the patches, does this mean PVA is exclusively modeling the shape similarities of variables at the most recent time segment, completely independent of their absolute levels? Could there be scenarios where the absolute values (or scales) of variables in the proximal patch are also critical for prediction?\n\n2. The TRA module reintegrates the trend via a simple broadcast addition to the Value projection (Eq. 8). Was there an exploration of more complex fusion mechanisms (e.g., gating, concatenation followed by a linear layer)? Why was additive reintegration chosen as the most effective method?\n\n3. For large multivariate datasets (e.g., Traffic with 862 variables), can the authors provide detailed training/inference time and memory usage comparisons between PMDformer and baselines (e.g., iTransformer, TimeBase)? How does PMDformer’s efficiency scale with the number of variables (C) or input sequence length (L), and is there potential to further optimize the TRA module’s computation (e.g., via parameter sharing)?\n\n4. The paper finds moderate patch sizes (24–72) are optimal, but what guidelines would the authors recommend for selecting patch size based on dataset properties (e.g., sampling interval, length of input sequence, seasonality period)? For example, should a dataset with 10-minute sampling intervals (e.g., Weather) use a smaller patch size than one with 1-hour intervals (e.g., ECL)?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novel and well-motivated core idea. The identification of \"scale bias\" in patch-based attention is insightful. \n\n2. The model demonstrates compelling state-of-the-art performance, outperforming a wide range of recent and strong baselines across eight standard benchmarks.\n\n3. The paper provides thorough ablation studies and a theoretical analysis that convincingly justify the contribution of each proposed module and the underlying motivation.", "weaknesses": "1. The paper positions PMDFormer as effectively modeling cross-variable dependencies, a domain where many previous \"Variable-Dependent\" (VD) models have struggled. However, the PVA module is applied only to the most recent patch. While the results are excellent, this design choice essentially limits cross-variable modeling to a very short, recent context. The paper could more explicitly discuss the implications of this: is the success of PMDFormer evidence that long-range cross-variable dependencies are generally not useful for LTSF, or that they are too noisy to model effectively? A comparison of PVA's performance when applied to the last $K$ patches (beyond just the ablation on $k$ in the sensitivity analysis) could have deepened this analysis.\n\n2. While the efficiency of PVA is mentioned, a more formal and overall complexity analysis of the full PMDFormer architecture compared to other leading models (e.g., PatchTST, iTransformer, TimeBase) is missing. Given the use of a Transformer encoder in the TRA module and the separate PVA module, a discussion of the total parameter count and FLOPs would be beneficial for a complete picture.\n\n3. The figure references \"Figure X\" in the Patch Size sensitivity analysis (Page 9), which appears to be a placeholder. This should be corrected to the appropriate figure number (likely 4b).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)