PHAT: Modeling Period Heterogeneity for Multivariate Time Series Forecasting
Authors: Jiaming Ma
Qihe Huang
Haofeng Ma
Guanjun Wang
Sheng Huang
Zhengyang Zhou
Pengkun Wang
Xu Wang
Binwu Wang
Yang Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Time series forecasting
time series data
deep learning
Primary area: learning on time series and dynamical systems

Abstract: While existing multivariate time series forecasting models have advanced significantly in modeling periodicity, they largely neglect the periodic heterogeneity common in real-world data, where variables exhibit distinct and dynamically changing periods. To effectively capture this periodic heterogeneity, we propose PHAT (Period Heterogeneity-Aware Transformer). Specifically, PHAT arranges multivariate inputs into a three-dimensional "periodic bucket" tensor, where the dimensions correspond to variable group characteristics with similar periodicity, time steps aligned by phase, and offsets within the period. By restricting interactions within buckets and masking cross-bucket connections, PHAT effectively avoids interference from inconsistent periods. We also propose a positive-negative attention mechanism, which captures periodic dependencies from two perspectives: periodic alignment and periodic deviation. Additionally, the periodic alignment attention scores are decomposed into positive and negative components, with a modulation term encoding periodic priors. This modulation constrains the attention mechanism to more faithfully reflect the underlying periodic trends. A mathematical explanation is provided to support this property. We evaluate PHAT comprehensively on 14 real-world datasets against 18 baselines, and the results show that it significantly outperforms existing methods, achieving highly competitive forecasting performance. Our sources is available at GitHub.

Published as a conference paper at ICLR 2026
PHAT: MODELING PERIOD HETEROGENEITY FOR MULTIVARIATE TIME SERIES FORECASTING
Jiaming Ma', Qihe Huang', Haofeng Ma', Guanjun Wang', Sheng Huang',
Zhengyang Zhou!. 1.2 Pengkun Wang"?, 12, Xu Wang"?, Binwu Wang!.2, 12,,, Yang Wang!
University of Science and Technology of China (USTC), Hefei, Anhui, China
Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, China
The University of Nottingham Ningbo China, Ningbo, Zhejiang, China Ji ami ngMa @ma 1, ustc. edu.cr
ABSTRACT
While existing multivariate time series forecasting models have advanced signif- icantly in modeling periodicity, they largely neglect the periodic heterogeneity common in real-world data, where variates exhibit distinct and dynamically chang- ing periods. To effectively capture this periodic heterogeneity, we propose PHAT (Period Heterogeneity-Aie Aware Transformer). Specifically, PHAT arranges multivari- ate inputs into three-dimensiooal "periodic bucket" tensor, where the dimensions correspond to variate group characteristies with similar periodicity, time steps aligned by phase, and offsets within the period. By restricting interactions within buckets and masking cross- boos-buccket connections, PHAT effectively avoids inter- ference from inconsistent periods. We also propose positive-negative attention mechanism, which captures periodic dependencies from two perspectives: periodic alignment and periodic deviation. Additionally, the periodic alignment attention scores are decomposed into positive and negative components, with a modulation term encoding periodic priors. This modulation constrains the attention mechanism to more faithfully reflect the underlying periodic trends. A mathematical expla- nation is provided to support this property. We evaluate PHAT comprehensively on 14 real-world datasets against 18 baselines, and the results show that it signif- icantly outperforms existing methods, achieving highly competitive forecasting performance. Our sources is available at GitHub.
INTRODUCTION
Multivariate Time Series (MTS) forecasting serves as a core enabling technology for critical appli- cations such energy demand prediction, traffic management, financial modeling, and healtheare monitoring (Qiu et al., 2024b; Ma et al., 2025b; Huang et al., 2025a; Yue et al., 2025; Li et al., 2026).
Periodicity, crucial intrinsic characteristic of time series data, plays decisive role in enhancing forecasting performance through accurate modeling (Lin et al., 2024a; Zhou et al., 2022b; Lin et al., 2024b). To this end, researchers have developed a series of cutting-edge methods. One line of work modifies neural network architectures to adapt to periodicity (Luo & Wang, 2024). For instance, some studies leverage the strong capability of the Transformer architecture model long-range dependencies (Zhou et al., 2021; 2022b; Nie et al., 2022). In addition, seasonal-trem decomposition techniques have been widely employed (Hu et al., 2025b;b; Ma al., 2025d). These methods separate the original time series into seasonal and trend components, which are then modeled by parallel sub-networks. This strategy enables more efficient extraction and utilization of periodic information. Furthermore, recent studies have incorporated classical signal processing tools, particularly frequency domain analysis methods like the Fast Fourier Transform (FFT), to more precisely identify and model periodic patterns in time series data (Ye et al., 2024; Zhang et al., 2025).
"Corresponding authors: Yang Wang and Binwu Wang.

---

Published as a conference paper at ICLR 2026
(423
(b) No period
(c) Positive Period
(d) Pos. & Neg. Period
(a) Pooling among variates
The visualization of period heterogeneity phenomenon on ZafNoo dataset. The orange region in autocorrelation function represents the 95% confidence interval for the null hypothesis from Bartlett's Test (Arsham & Lovric, 201 I). Correlation coefficients that fall within this interval are not statistically significant and cannot be rejected as noise (Chandler, 1987).
Figure 1:
Despite the substantial progress, two major limitations remain: 1 Most existing models unify periodic modeling by treating variates as interchangeable channels for pooling and fusion, implicitly assuming shared, static periodic length. This overlooks the pronounced heterogeneity in periodic behavior across variates. As shown in Figure I, three variates from the ZafNoo dataset exhibit distinct period lengths. Forcing such diverse periodicities into the unified framework can lead models learn spurious temporal dynamics. @ Mainstream Transformer architectures amplify positive correlations but suppress negative ones during attention normalization, overlooking inverse or complementary dynamics inherent in periodic signals, as shown in Figure 1 (d). Yet such negative correlations offer critical insights into system dynamic. Therefore, crucial to develop multivariate time series forecasting models capable of accurately capturing periodic heterogeneity.
In this paper, we propose a Period Heterogeneity-Awaal Transformer (PHAT) for MTS forecasting. Specifically, PHAT restructures multivariate time series into a periodic bucket structure by grouping variates based on their periodic lengths. Within each bucket, each sequence further reshaped into a 2D tensor-rows align time steps by phase, while columns capture offsees within the period. Interactions are restricted to variates within the same bucket, enabling the model to capture diverse periodic patterns, while coss-buckket links masked to prevent interference between variates with differing periodicities. Second, PHAT introduces a Positive- Negative Self-Attention mechanism (PNA), which interprets periodic dependencies through two attention coefficients: phase alignment and periodic offset. The periodic offset coefficient is further decomposed into positive and negative components, with modulation term encoding periodic priors. This term reduces the positive weights and increases the negative weights for distant phase-aligned points (and vice versa), enabling the attention mechanism to more faithfully capture the periodic structure.
Contributions. Initial Exploration. We relax the single- period assumption to handle complex time series with heterogeneous periodicities and introduce PHAT, the first method expressly devel- oped to model such period heterogeneity. @ Periodic Bucket. PHAT introduces "periodic bucket" structure to manage time series data with heterogeneous periodicity, facilitating the learning of peri- odic patterns. 2 Novel Self-atiention. We further propose positive-negative attention mechanism that represents periodic information with distinct positive and negative components, combined with modulation term encoding periodic priors. Its favorable properties demonstrated mathemati- cally. Empirical Validation. On 14 real-world datasets with 18 baselines, PHAT achieves SOTA performance on approximately 73.95 % (71/96) of the metrics while maintaining low computational complexity. Additionally, PHAT demonstrates strong robustness in complex periodic scenarios.
RELATED WORK
Periodicity plays a pivotal role in the predictability of time series data (Lin et al., 2025; Wang et al., 2024a; Ma et al., 2025d). Recent advancements have introduced range of sophisticated techniques aimed at improving the ability to capture long-term dependencies, thereby enhancing the perception of periodic patterns. For example, classical seasonal-trend decomposition uses moving- average kernels for sliding aggregation to extract trend components (Zeng et al., 2023; Kingma & Ba, 2014), while CycleNet proposes learnable Cycle Decomposition to capture periodicity (Lin et al., 2024a). Architecturally, Transformer based models excel at modeling long-range dependencies: Autoformer connects sequences by periodically aggregating similar subsequences (Wu et al., 2021),
2

---

Published as a conference paper at ICLR 2026
Amplitude
Buckets
Receptive- Feeivee-ieeddd in PNA
Veriates
Weak
Positive Negative Attention
Pariod-aa uliiged
A Day
Multivariate Time Series
MAggumm Frequency
Beriod-offset of bucket
Figure 2: The overall architecture of PHAT. Each bucket contains the same periodic variates and uses PNA to capture the periodic attention mechanism.
while Modern leverages very large convolutional kernels substantially enlarge the receptive field and capture long-range temporal dependencies (Luo & Wang, 2024). Frequency- domann analysis and multi-scale modeling have also been used to strengthen periodic representations (Zhou et al., 2022b; Wang et al., 2024a).
However, existing methods typically handle multivariate series via pooling or adaptive fusion and model periodicity from unified perspective, implicitly assuming all variates share the same periodic patterns. This overlooks widespread periodic heterogeneity and often ignores negatively correlated periodic components, which can provide complementary perspectives and valuable information gains. We provide a detailed discussion of multivariate time series forecasting in Appendix A.
METHODOLOGY
Multivariate time-series data typically refer to either collection of multiple temporal objects or single object observed through multiple feature channels (Huang al., 2024b; Wu et al., 2020; Ma et al., 2025e). Given multivariate time series input X = [x1 X2, ,xy] E RCXT observed over the past T time steps, where x, E IR represents the observation time step across C variates. And we use X; E R to represent the input sequence corresponding variates i. The objective of the multivariate time-series forecasting to forecast the subsequent time steps
Y XT+1,XT42,
XT+L] E RCXL
As shown in Figure 2, we propose PHAT for modeling periodic heterogeneity, a common yet previously overlooked property of real-world multivariate time series, thus motivating our PHAT. PHAT includes a novel period bucket structure to manage time series. Subsequently. PHAT integrates positive-negative attention mechanism to precisely model the positive and negative correlations of periodicity. Finally, we perform weighted fusion of the generated periodic components based on their frequency saliency produce the prediction results.
3.1 PERIOD BUCKET FOR TIME SERIES
3.1.1 PERIOD DETECTION
Fast Fourier Transform (FFT) (Duhamel & Vetterli, 1990) is a commonly used tool for analyzing the characteristics of time series, especially periodicity (Liu, 2025; Wu et al., 2022b; Fang et al., 2023). Therefore, we first apply the FFT to each variate the input sequence X. Subsequently, we retain only the spectral magnitudes corresponding to the Top- K significant frequency components, which are then converted into discrete period lengths. This process can be expressed as follows:
T
P= Larg
0.5] E XX
arg Top [FFT(X)
where arg Top selects the indices the K most salient frequencies. represents the set of period lengths from C variates.
3

---

Published as a conference paper at ICLR 2026
3.1.2 TIME SERIES PERIOD BUCKET CONSTRUCTION AND REPRESENTATION
To improve periodicity modeling. we propose a novel period-bucket structure for reconstructing time series, consisting of two stages: bucketing and folding.
Bucketing This process groups variates by their dominant period lengths. Given C' periods. we first remove duplicate values, assuming there are V distinct elements. For any period length P.. we create a bucket matched to it, denoted as B,, and place the variates with a period length of P; into the bucket. Because variate can exhibit multiple periodicities, buckets are not necessarily disjoint. In addition, we assign Bucket-O to manage variates that do not exhibit significant periodic behavior.
Folding This process further reshapes the sequence of each variate. For the input sequence of variate in Bucket-b, denoted as Xj. we first use an linear layer to align it with the future window: X, + X, E Rxxd It is important to emphasize that the period length of variates in Bucket-b is Po- Next, segment X, into small fragments of length [L/Pb which may require zero-padding for Xj. This process can be expressed mathematically as:
X; Unflatten X,
Pb) E R..NN
X
Concat X X
0), if L mod Po > 0, E (R(NNN))
if mod P, =
where L/P] represents the number of periods contained in the sequence. So for Bucket-b that contains of [B,] time series, we can generate 3D bucket-structured input denoted as ( E 1 which adapts to the characteristics of periodic features: Periodic homogeneous variates are grouped in the first dimension; @ Time steps in the second dimension are periodic-offset within the same periodic; t Each time step in the last dimension is periodic-aligned.
Then, we perform interactions between variates learn their dependencies as follows,
z (b)
(b)
W b, Pxxxxx
(3)
where W R x.d and b TR learnable parameters with the dimension size d,- The processing of Bucket with periodicity B, is represented in Appendix B.
3.2 POSITIVE-NEGATIE X-SHAPE ATTENTION FOR PERIODICITY MODELING
Conventional self-attention allows unrestricted token interactions, which is suboptimal for capturing the periodic structures inherent in time series. To address this, we propose Positive-Negative Attention for Periodicity Modeling (PNA), featuring three key designs: 1 X-Shaped Receptive Field: PNA allocates attention along the rows and columns of the bucketed representation Z forming an X- shaped (cross-like) receptive field centered on each target element. This structure explicitly separates geriod-aligned (across- perros))period) and period-offse (within-periun (erhio)-priood) relationships, enhancing periodic pattern modeling. @ Strong Periodic Inductive Bias: Attention scores modulated by a periodic- distance-dependent term, enforcing inductive bias that aligns attention weights with the underlying periodic trends. 4 Decoupled Positive-Negative Correlation Modeling: PNA separately model positive and negative periodic dependencies, mitigating interference between opposing correlations and enabling more expressive representation of complex periodic interactions.
Specifically, given Bucket-b representation (b) R No xd, (except for Bucket-O, which lacks periodic characteristics), we first obtain multiple components using the following formula:
[Q:; Q2) Z (
K1 K2
Z W == (0)
A = oZ Wg),
where Wo. W K. Wy E Rd Wh Rdx1 are learnable projection parameters. Q: and IR x2d x2d are the query vectors, K₁ and K2 E XP,xNxx2 are the key vectors, and V E N,2d is the value vector, A is the weighted strength filter. a(-) is the sigmoid function. The calculation process of PNA can be written as:
Q2 E
PNA(Z ()) Attention QQ: Q2), K; V.A) = =Ax1 (Ax2V).
(5)
4

---

Published as a conference paper at ICLR 2026
where x, means the multiplication in the i-th dimension of the matrix. Periodic-aligmed attention A encodes dependencies between time-steps that share the same phase across periods, while periodic- offset attention A captures dependencies among time-steps within the same period.
Period-offset Attention A offset attention with separate query and key in each head as follows,
We compute the positive logits and negative logits n of the period-
u0Q x1 K E IRPPPPPeN,, 7 = QºQ X1 K2 E T.
where μ d the scale factor. The positive and negative logits are first adjusted by the periodic modulation terms and then separately normalized via softmax to produce attention coefficient matrices. These two matrices are subsequently fused form the final attention matrix as follows,
A Softmax A @ Softmax (f) E R B.XPoNN.
Ç/m, n] =imm n]
Softplus(CInn s)), n/m, n] = 1/ n|
Softplus(n [m,,)) s)),
#C
(6)
Positive Modulation Term
Negutive Modulation Term
where Ç[m, nl] denotes the attention coefficient between the m-th and n-th time steps within signal of period Po. corresponding to the (m, n) -th entry of the attention matrix Ç. We define the periodic relative distance between m-th and n-th time steps as 8m.n- For a fixed and n, let
AP A = {s h < 8h n U {m}} the set of time steps whose periodic distance to m is smaller than that between m and n. We aggregate the attention coefficients ([m, s] for all 8 E A After applying the Softplus(.) activation (Zheng et al., 2015), this term encourages the model to produce attention weights that decay monotonically with increasing periodic distance, thereby reinforcing the inductive bias toward local periodic structure. The periodic distance is computed as follows:
8 = min j) mod Br, (j - i) mod B,} E [0. [B,/2]],
8
Conversely, the farther apart the periodic positions of the time steps, the greater their negative correlation. Therefore, the set Vhs,a {s|ō%, > 8% _,,,, U {mm] includes time steps with larger periodic relative distances. Then, the negative modulation term is computed as the sum of the attention coefficients of these time steps and applied to the attention coefficients.
In this manner, PNA ensures that as the periodic relative distance between two time steps increases, their positive correlation coefficient decreases while their negative correlation coefficient increases. We provide mathematical explanation of this favorable property of the periodic-offset atten- tion, which can be found in Appendix C.2.
The row sums of the generated A are not strictly equal to 1, satisfying:
Vi = (1, Bo); n (1)...,No).
(9)
To address it, when generating the final attention, we inject a residual path with residual strength A to stabilize the information flow. Please refer to Equation 11.
Period-aligned Attention Á To capture dependencies among time steps that are phase-aligned with the periodicity, we employ simplified self-attention mechanism. This mechanism shares the same query, key, and value vectors as the positive component of the period-offset attention, as these phase-aligned time steps exhibit strong correlations. This process can be expressed as:
= Softmax(pQ)) (pQ1 X2 KT) E N
(10)
where μ is the learnalle coefficient.
Multi-head Attention Output We introduce a multi-head mechanism to enhance the model's expressiveness. The final computation is as follows:
(M = Concat(head', head, ..... head Wo.
(11)
head 7% Tanh (PNA(Z))) A @ Z +B,
(12)
5

---

Published as a conference paper at ICLR 2026
where H is the number of heads, Wo R is learnable parameters, and an E R, "/. 9n RR2dd. are learnable normalization parameters. @ means Hadamard Product. We use Dynamic Tanh (Zhu et al., 2025) to eliminate any remaining numerical instability.
special case involves the variates in the zero bucket, which lack periodicity. For these variates, the folding operation is skipped. When computing period-aligned attention and period-offset attention, absolute distance used instead of periodic distance. The details are represented in Appendix B.
3.3 BUCKET-WISE FORECASTING
Flatten&Alig Let us define the output of the b-th bucket after PNA learning as Z PRPx Then, we flatten its first two dimensions, and if zero-padding has been applied (i.e., (P& No) > L), we truncate the padded time steps: I P Nxxd + IR Lxd, Finally, we align the channels to the number
of variates originally in the bucket:
(b) (b)) w
(w) (8) bb(b) E IR B.
(13)
[Bb] denotes the number of where (()) E [8 and (bb) B are learnable parameters. variates in the b-th bucket.
Frequency-based Multi-period Prediction To predict the future value of the e-th variate among C' variates, we first determine which buckets contain this variate based on its Ke period lengths, and then extract the corresponding bucket representations for prediction, which can be expressed as:
(b) (
E ER
s.t. CE (B. (b (b=1,... 16......,NN)
(14)
b=1
(b) Avg Avg (Sootmaaaax((3 D). 8(0) Extract (FT((!()) (1))
(15)
where (( denotes the spectral magnitude extracted in the frequency domain for period length b After normalization, e( converted into the weight ah which is used to perform weighted
fusion of the corresponding bucket representation (.(0) yielding the final prediction. Finally, we can generate the prediction for C' variates Y E RCxL
4
EXPERIMENTS
4.1
EXPERIMENTAL SETUP
Protocol Settings All experiments are performed on an NVIDIA A100 GPU with 80 GB of memory using Py Torch.. For a comprehensive evaluation, we disable the "Drop Last" batch sampling procedure (Li et al., 2024; Qiu et al., 2024a). Optimization is performed with Adam (Kingma Ba, 2014). Mean squared error (MSE) and mean absolute error (MAE) are used for evaluation. Considering that different models vary in their sensitivity to input history, we treat the look back length T tunable hyperparameter and report the best performance, and also comparisons using fixed input length in Section D.3. The details of hyperparameters are summarized Table 4.
Table 1: Statistics of used datasets.
Exchanne
FRED--D
ET1h
ETTm
AUShamny
AUWan
ZafNoo
NASDAO
NYSE
Sumplee
791
7,588
728
14,640
57,800
35,064
35,061
985
19,931
19,225
1,244
1,243
Frequency
day
15mis
30min
30
Split Ration
7:1:2
7.1:2
7.1.2
6.2.2
6.222
6:2:2
6:2.2
7:122
7:1:2
7.122
7.1.2
7.1.2
Damain
Feonomic
Elactricty
Healhh
Sook
Datasets We evaluate our method on 15 datasets: 14 real-world benchmarks (NN5, Exchange, FRED-MD, ETThI, ETTh2, ETTml, ETTm2, AQShunyi, AQWan, ILI, CzeLan, ZatNoo, NASDAQ NYSE) and one synthetic dataset. Synthetic dataset is constructed by concatenating sequences from E (period 96) and ETTh1 (period 24) along the time axis to simulate varying periodicities (see Appendix D.4). To account for varying dataset sizes, we adopt two evaluation regimes: for datasets with fewer than 5,000 samples, the tunable range of look back length is T' (36, 104) and

---

Published as a conference paper at ICLR 2026
Table 2: Multivariate time series forecasting performance comparison. We report average MSE and
MAE. Best results are bold second-best are underlined
Methods
Ours
Timekan
xPatch
Amplitier
CycleNet
HimeNtixer
SparscISF
fTransfommer
Fathformer
2025
2025
2025
2025
2024
2024
2024
2024
2024
2024
Metries
MSE
MAE
MSE
MAE]
MSE
MAE
MSE
MAE]
MSE
MAE
MSE
MAE]
MSE
MAE]
MSE
MAE]
MSE
MAE]
MSE
MAB
96
0.316
0.356
0.324
0.37
0.327
0.364
0.334
0.377
0.327
0,369
0.327
0.376
0.322
0.364
0.347
0.336
10.364
0.318
0.366
192
0.369
0.394
0.364
0.399
0.384
0,403
0.381
0.407
0.374
0,800
0.381
0.408
0.371
0.395
0.398
0.422
0.377
0.398
0.366
0.398
336
0.384
0:412
0.394
0.421
0.400
0,427
0.413
0.435
0.401
0.422
0.402
0.432
0.387
0.408
0.419
0.439
0.408
0,421
0.396
0.421
720
0.412
0.44
0:425
0:453
0.456
0.460
0:431
0.456
0.48
0,458
0.444
0.46
0,402
0.431
0.460
0.466
0.444
0.459
0.427
0.448
96
0.3
0.298
0.220
0.301
0.241
0.50
0.296
192
0.273
0.323
0.285
0.334
0.276
0.325
0.277
0.333
0.274
0.327
0.280
0.335
0.284
0.334
0.292
0.346
0.278
00323
0:270
0.327
336
0
0.349
0.316
0.758
0.320
0.353
0.321
0.359
0.519
0.354
0.323
0.350
0.323
0.357
0.328
0.367
0.522
0.352
0:312
0.357
720
0.388
0.392
0.360
0.402
0.401
0.405
0
0.404
0.390
0
0.393
0.402
0
0,402
0.402
0.412
0.395
0.397
0.379
0.399
0.769
0,851
0,663
1,400
0.754
0.574
0.769
0.379
NQWi AQShuny) FRED MDD MDDExchange NNS
36
0.640
0.538
0.7
0.585
0,785
0.634
1,268
0,862
0.687
0.571
0.657
0.557
1,789
1.066
0.701
0.583
0.676
0.567
48
0.613
0.532
0.659
0.565
0,716
0.501
1,202
0,812
0.656
0.564
0.630
0.55
1,982
1.097
0.633
0.543
0.668
0.573
0.616
0.538
60
0.603
0.531
0.651
0.568
0,673
0.579
1.237
0.861
0.642
0,568
0.612
0.543
1.734
1.046
0.615
0.537
0.655
0.570
0.599
0.533
0.083
0.200
0,208
0.204
0.207
0.205
0.083
192
0.169
0.295
0,301
0,158
0.179
0.307
0.178
0,300
0.203
0.294
336
0.316
0.406
0,338
0:421
0.333
0.422
0.37
0.419
0.339
0,416
0.376
0.41
0.3600
0.437
0.331
0,417
0.354
0.429
0.323
0.41
720
0.773
0.669
0
0.719
0.809
0.718
0.929
0.719
0.854
0.698
0.884
0.707
0.882
0.714
0.846
0.603
0.909
0.716
0.70
0.682
24
31,495
17%
$5.290
1,450
1.192
$1.55%
01545
20.132
43,234
1.142
90,949
0.936
18.163
45:811
$2.339
70.560
96.051
54.221
56.870
1.276
48
$2.065
1,453
76. 144
1.542
73.426
1,505
102.150
124
077
2,043
90.283
1.2
139,333
7.5
80.574
1,607
96.877
1.728
58.
1615
60
76.299
1.679
96.
L.736
103.023
1.778
145.867
2.176
172.747
2.411
126.438
1.865
190,400
2.525
130.061
1.947
134.397
2,025
131.531
1.884
96
0.666
0.468
0.651
0.478
0.666
11:480
0.667
0.489
0.653
0.484 0.654
0.483
0.899
0.509
0.650
0.479
0.667
0,472
0.647
0.481
192
0:707
0.497
0.600
0.707
0.409
336
0.737
0.502
0.712
0,512
0,741
0.505
0.718
0.511
0.719
0.715
0,510
0.728
0.525
0,732
0.503
0.708
0.512
720
0.795
0..529
0.763
0.531
0.790
0.532
0.765
0.537
0.774
0.540
0.756
0.534
0.784
0.552
0.766
0.537
0.783
0.515
0.765
0.537
96
01746
0.452
0.749
0.467
0.752
00:255
0.762
0.47%
0.758
0.473
0.744
0466
0.706
01447
0.747
0.470
0.761
0,458
0.747
0.469
192
0,308
0.475
0.791
0
0,803
0.479
0.806
0.193
0.804
0.492
0,801
0
0,820
0,509
0.787
0.186
0.801
0.678
0.791
0.189
0.824
0.497
0.821
0.500
0.514
0.814
0.485
0.817
720
0.907
0.517
0.888
0.577
0,906
0.522
0.894
0.530
0,899
0.532
0.878
0.522
0,907
0,543
0,889
0.579
0,888
0.506
0.885
0.527
1.318
0.705
2.176
0.028
2,3.0
1.017
2,580
1.220
2.195
1,023
1.804
0.031
4.917
1.666)
1.783
0.346
2,086
0.922
1.801
0.874
ILI
1.437
1.523
0.775
2,166
0.978
0.993
2,221
2.255
1.011
18211
2.759
1.187
1,950
1,971
0,942
1.891
0.926
4.577
1.51K
1.746
0.860
1,985
1,912
0.882
1.743
0.867
0.926
1,461
0.777
2,010
0.967
2,269
1,013
2,857
0.920
4,697
1.521
0.63
0.929
0.925
NYSE (NASDNQ) ZalNoo OreLan
192
96
0
0.179
0.306
0.227
0.204
0.197
0.203
0.248
0.257
0.173
0.196
0.205
0.238
M177
03304
0.251
0.223
0.178
0.209
0.252
0.229
0.175
0.206
0.254
0230
0.2055
0.223
0.784
0.255
0.177
0.201
0.219
0.257
0.207
0.172
0.236
0.213
0.177
0.305
0232
0.252
336
0.236
0.254
0,239
0.291
0.118
0.36
0,33
0.274
0.242
0.280
0.230
0.277
0
0.308
0.232
0.282
0.240
0.262
0:232
0.277
720
0.285
0.293
0,246
0,308
0,258
0.26
0,251
0,325
0.308
0,350
0.298
0.310
96
0.439
0.383
0,,47
0.19
04433
0.306
0.443
0.446
0.409
0.441
0.396
0.510
0.452
0.439
0.408
0.435
0.39
0.438
0.401
192
0.503
0.429
0.465
0:433
0.498
0.437
0.045
0//12
0.503
0.545
0.498
0.441
0.542
0.475
0.505
0.443
0.50)
0.432
0.198
0.140
336
0.547
0.456
0,533
0.464
0.541
0.463
0315
0.169
0.543
0.467
0,543
0.466
0,586
0.493
0.555
0.473
0.55]
0.45
0.545
0.463
0.592
0.503]
0,595
0.554
0.588
0.500
0.486
24
0.416
0.473
0.541
0.517
0.500
01 1.500
0.753
0.630
0.631
0.570
0.720
0.612
1,363
0.876
0.576
0.540
0.572
0.568
0.11
0.541
36
0.619
0.600
1.013
0.732
0.865
0.665
1.179
0.784
0.886
0.675
0.951
0.699
1.223
0.854
0.691
0.600
0.836
0.683
0.862
0.683
48
0.949
0.731
1.2022
0.779
1.28
0.704
1.425
0.881
1.321
0.854
1,214
0.795
1,310
0.884
1.188
0.773
1,227
0.798
1.235
0.803
0.961
0.779
1.414
0.846
0.836
1.352
0,862
1.535
0.888
1.136
1.110
0.818
1.315
0.820
1.421
0.855
1.426
0.841
0.161
0.251
0.224
0.304
0.310
0.286
0,280
0,362
0.237
0.314
0.53
0.369
0.512
0,497
0.225
0.302
1,102
0814
0.26]
0.33
36
0.29%
0.352
0:322
0.760
0.366
0.387
0.640
0.599
0.395
0.411
0,678
0.61
0.669
0.566
0.392
0.409
1.596
1.002
0.385
0.40%
0.445
0.427
0.475
0.447
0.891
0.558
0,761
0.599
0.555
0.481
0,747
0.663
0.937
0.69]
0.529
0.150
1,910
1.16
0.689
0.562
60
0.619
0.525
0,660
0.534
0,886
0.616
0,950
0.685
0.759
0.584
1.027
0.807
1,219
0.814
0.687
0.557
2.123
1,188
0,810
0.616
Top1
Top2
81
23
24
4
13
3
12
14
forecasting horizons are L E {24, 48, 60]: for larger datasets, the tunable range of look back length is T' E (96, 336, 512} and forecasting horizons are L E {96, 192, 336, 720). Dataset statisties are summarized in Table 1.
Baselines We compare our model with 18 advanced multivariate time series forecasting baselines including TimeKAN (Huang et al., 2025b), xPatch (Stitsyuk & Choi, 2025), Amplifier (Fei et al., 2025), CycleNet (Lin et al., 2024a). TimeMixer (Wang et al., 2024b), SparseTSF (Lin et al., 2024b), iTransformer (Liu et al., 2023), Pathformer (Chen et al., 2024), PDF (Dai et al., 2024), FITS (Xu et al., 2023), PatchTST (Nie et al., 2022), Crossformer (Zhang & Yan, 2023), NLinear (Zeng et al., 2023), TimesNet (Wu et al., 2022a). FEDformer (Zhou et al., 2022b), Triformer (Cirstea et al., 2022), FiLM (Zhou et al., 2022a) and Non-stationary Transformer (Liu et al., 2022b).
4.2 FORECASTING PERFORMANCE COMPARISON
Due to space limitations, we averaged the results for ETThl and ETTh2 (denoted as ETTh) as well as ETTml and ETTm2 (denoted as ETTm). Complete results and additional baseline comparison experiments can be found in Appendix 7.
As shown in Table 2, among the advanced baselines, TimeKAN leverages the Kolmogorov-Arool network to approximate the complex nonlinear dynamics in time series, demonstrating effective ex- pressive power. xPatch introduces exponential seasonal-trend decomposition and amplifies low-energy
deasona-rrenddecoompsittttooo
7

---

Published as a conference paper at ICLR 2026
components in the spectrum, thereby enhancing sensitivity to subtle signals. Some architectures emphasize periodicity modeling. For example, PDF incorporates a multi-scale patch strategy to jointly capture periodic features, while TimeMixer introduces multi-scale modeling techniques, achieving remarkable performance. In the next section, we provide a detailed comparison of these periodicity-learnin models' performance on datasets with periodic heterogeneity. In conclusion, due to its strong adaptability to periodic heterogeneity, PHAT achieves SOTA performance. Notably, on the NYSE dataset, it improves MSE by up to 23.33% % It achieves the best results on 73.95 % of metrics and ranks in the top two for 84.38% % of metrics in Table 7.
4.3 ABLATION STUDIES
We design several ablation variants of PHAT to validate the contribution of its key components. Specifically, "w/o POA" removes the period-offset attention branch from PNA, while "w/o PAA" eliminates the period-aligned attention branch. "w/o Attn" cancels the self-attention mechanism entirely, reducing it to single ferwafo net. "w/o Bucket" disables the period- based variate grouping, treating each variate independently rather than within its detected period bucket.
As shown in Figure (a) (where taller bars indicate lower MSE for intuitive display), the "woo Bucket" variant exhibits significantly larger prediction errors, indicating that interactions between variates with different periodic characteristics increase the difficulty of capturing complex periodic patterns. The poor performance of the "w/o PAA" variant highlights the importance cross-period synchronization signals the same phase, especially on datasets with weaker periodicity, such as NN5 and CzeLan "w/o POA" variant achieves extremely poor predictive performance. We further conducted ablation experiments on the combination of periodic offset attention, and the results are shown in Table 10. We found that modeling dependencies between time steps from both positive and negative perspectives is beneficial. Additionally, the modulators configured for each perspective help generate attention coefficients that align more closely with periodic trends. Detailed comparison of combination ablation study on additional datasets is in Appendix D.8.
ETTh1 (L 96)
ILI 24)
NN5 (L = 24)
CAST
MPW VONNA Lagge DF
MAC
RAGBAO
4CT 40P NO
PILAT
EIT
TopR
MAZ
MAL
AGHENNI
EXT
(b) Hyperparameter sensitivity experiments of the number of PNA layers (upper) and the number of periods K (lower). The highlight star ">t" is corresponding optimal hyperparameters.
(a) blation study.
Figure 3: Ablation study and hyperparameters sensitivity experiments of phats
4.4 HYPERPARAMETER SENSITIVITY EXPERIMENTS
We further investigate the sensitivity of key hyperparameters: the number of PNA layers and the number of cycle lengths K selected for each variate, as shown in Figure 3. Experimental results (Figure (b-upper)) show that our model achieves strong performance with just one or two layers, thanks to the periodic bucket structure and the PNA mechanism, which effectively capture periodic patterns. contrast, adding more Transformer layers brings little gain and may even hurt performance due to overfitting. As for the hyperparameter K, Figure 3(b-lower) shows that on datasets with simple periodicity, a single dominant period ( (K=1) often sufficient. Using more periods tends introduce noise and degrade prediction accuracy.
4.5 COMPLEXITY ANALYSIS
We evaluate PHAT against advanced baselines on two datasets. As summarized in Table 3, PHAT achieves the best forecasting accuracy while substantially reducing model complexity. Compared with
8

---

Published as a conference paper at ICLR 2026
Transformer-basse methods (e.g., PatchTST, PDF) and recently advanced methods (e.g. TimeKAN, xPatch), PHAT cuts parameter counts by more than an order of magnitude and reduces MACs and FLOPs by over 98%. Even compared to lightweight architectures like TimeKAN, PHAT retains a Transformer-style backbone while reducing the parameter count by up to 91.27% with competitive inference latency. Because the computational cost of our attention scales with the square of the detected period length rather than with the full input sequence, so long sequences remain inexpensive process. practice, PHAT also typically requires only single layer capture temporal dependencies effectively.
Table Computational Complexity Comparison. # Para: All learnable parameters requiring gradient descent. MACs: acciuullc operations, # FLOPs: floating point operations. K: Kilo (10%).
M: Million (10%). Billion (10"). Inf: actual inference latency (s).
Models
MAE
Para
MACs
#FLOPs
Inf lat
Models
MAE
Pura
A MACs
FLOPS
Iaf lat
PEDiormer
0.463
3.4.M
1.7
1.3 B
20,044
#EDiormer
0.488
16.8 M
111.9 B
V5.7 B
0,480
BT7bll ( 06)
TimesNet
0,378
2.4
72.2
72.2 8
3,348
TimesNet
0.495
665.9 K
77.0 B
76.9 B
1.022
Crossformer
0.367
2.1
B
1,806
Crossformer
11.6
60.4 B
0.340
2.0
4.48
4.6 8
4.234
PDF
1.6
0.351
TimeMixce
0.345
397.3 K
3.2
3.1 B
3,967
TimeMixer
0.484
2.3 M
34.0
34.0
0.891
TimeKAN
0.346
52.8
205.4 M
455.0 M
6.542
TimeKAN
0.463
374.4 K
1.4
1.7
1.934
PHAT
0.330
3
2.9
7.0
1
PHAA
0.458
32.7 K
3.7.
16.5 M
0.462
4.6 VISUALIZATION OF PERIODIC POSITIVE-NEGATIVE COMPONENTS IN DATA, FEATURE, AND ATTENTION LEVELS
In this subsection, we further demonstrate that positive-negative periodic correlation indeed exist at all multiple levels. We provide comprehensive experimental analysis at three levels-sequence feature, and attention-to support our motivation on ETTh1 dataset of variates with daily period and NN5 with weekly period.
Sequence Level. We visualize the autocorrelation function (ACF) of the original time series data X, which demonstrates the presence of both positive and negative correlations between time steps.
Feature Level. We computed the autocorrelation function for each time step in the bucket representation (b)) As shown in Figure (a), across each hidden feature dimension, the resulting high-dimensional features continue to exhibit negative correlation, indicating that the positive and negative correlations present in the original signals persist after the complex-valued linear projection.
Attention Level. We aim to demonstrate two points: (1) attention coefficients do contain negative components; and (2) the softmax operation tends to smooth out or suppress these negative components. As shown in Figure (b) and (c), we visualized PHAT model's attention logits before and after the softmax, and compared them with control variant in which replaced by standard self-attention. The results show that before softmax, both models' logits exhibit clear positive and negative components; after softmax, the negative components of the standard self-attention nearly vanish, whereas our attention preserves negative correlations.
nositive-negative attention is
In contrast, our PNA mechanism preserves these negative correlations by separately modulating the positive and negative pathways, which is beneficial for capturing the complete periodic structure.
4.7 ATTENTION VISUALIZATION
As shown in Figure 5, we illustrate the autocorrelation coefficients between different time steps within a time series (left side) for various datasets, while the right side visualizes the corresponding cycle-shifted attention weights generated by our method. The attention matrix exhibits a clear periodic structure: bright bands represent positive attention, dark bands represent negative attention, and green areas indicate zero attention. We can observe that the trend between our generated attention matrix and the periodic correlation of time series data is consistent, demonstrating the ability capture

---

Published as a conference paper at ICLR 2026
underlying periodic structures. For time series with weaker negative periodicity (such ZafNoo), there are fewer negative dark regions in the attention. For non non-periodic time series (such as those from the ETTh2 and NASDAQ datasets), cycle-shifted attention mechanism naturally degrades to focus on each individual time step. Overall, our method effectively captures the periodic dynamics of time series.
Sequecee
Feature level
Allention lev.
Atteation level
ETThl (ayy
Negative
Before Softmax
After Softmox
Zero
Negative
AF input datu (left) and bucket representation ( (right).
(b) Vanilla self all--attenioon logits of QK" (left) and Softmax(uQK" 1) (right).
[c] Period-offsedt Ateention in PHAT.
Figure 4: Multi-level visualizations raw-seqeence level, feature level and attention level.
ZafNoo (half day)
ETThi (a
ETT (a (aday)
NN5 (a week)
II (a
NASDAQ (None)
Figure 5: Autocorrelation function (left) and periodic-offse attention weights (right). Highlighted
regions denote positive components; dark regions denote negative components.
5 CONCLUSION
this paper, we identify the challenge of periodic heterogeneity, where the periodic characteristics of variates within dataset differ. To address this, we propose Periodic Heterogeneity- Aware Transformer (PHAT). PHAT employs a periodic bucket structure manage multivariate time series with heterogencous periodicity. By introducing the periodic bucket mechanism and the novel Positive-Negative Attention (PNA) module, PHAT is able to seamlessly and accurately model heterogeneous periodic patterns while capturing both positive and negative phase dependencies. Extensive experiments conducted on real-world datasets demonstrate that PHAT achieves state-of- the-art performance across evaluation metrics.
ETHICS STATEMENT
Our work only focuses the time series forecasting problem, there no potential ethical risk.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
In the main text, we have strictly formalized the model architecture with equations. All the imple- mentation details included in the Appendix, including dataset descriptions, metrics, model, and experiment configurations. The code provided at an anonymous link.
ACKNOWLEDGMENT
This paper partially supported by the National Natural Science Foundation of China (No. 12227901) and the Natural Science Foundation of Jiangsu Province (BK20250482). The AI-driven experiments, simulations and model training were performed on the robotic Al-Scientist platform of Chinese Academy of Science.
REFERENCES
Hossein Arsham and Miodrag Lovric. Bartlett's 's test. International encyclopedia of statistical science, 2(2):20-23, 2011.
David Chandler. Introduction to modern statistical. Mechanics. Oxford University Press, Oxford, UK 5(449):11, 1987.
Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scalc transformers with adaptive pathways for time series forecasting, arXiv preprint arXIv:24022005950 2024.
Panayiotis Christou, Shichu Chen, Xupeng Chen, and Parijat Dube. Test time learning for time series forecasting. arXiv preprint vrXiv22409110122 2024.
Razvan-Gabriel Cirstea, Chenjuan Guo, Bin Yang, Tung Kieu, Xuanyi Dong, and Shirui Pan. Triformer: Triangular, variable-specifie attentions for long sequence multivariate time series forecasting full version. arXiv preprint arXiv:2204.137661 2022.
Tao Dai, Beiliang Wu, Peiyuan Liu, Naiqi Li, Jigang Bao, Yong Jiang, and Shu-Tao Xia. Periodicity decoupling framework for long-term series forecasting, In The Twelfth International Conference on Learning Representations, 2024.
Serge Dégerine and Sophie Lambert-Lacroix. Characterization of the partial autocorrelation function of nonstationary time series. Journal of Multivariate Analysis, 87(1):46-5 59, 2003.
Pierre Duhamel and Martin Vetterli. Fast fourier transforms: tutorial review and state of the art.
Signal processing, 19(4):259 299, 1990.
Yuchen Fang, Yanjun Qin, Haiyong Luo, Fang Zhao, and Kai Zheng. Stwave++: multi-scale efficient spectral graph attention network with long-term trends for disentangled traffic flow forecasting. IEEE Transactions on Knowledge and Data Engineering, 36(6):2671-2685 2023.
Jingru Fei, Kun Yi, Wei Fan, Qi Zhang, and Zhendong Niu, Amplifier: Bringing attention to neglected low-energy components in time series forecasting, In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 11645-111 1653, 2025.
Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Percira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu. Temporal convolutional neural (ten) network for effective weather forecasting using time-series data from the local weather station. Soft Computing, 24(21):16453- 16482, 2020.
Yifan Hu, Yuante Li, Peiyuan Liu, Yuxia Zhu, Naiqi Li, Tao Dai, Shu-tao Xia, Dawei Cheng, and Changjun Jiang. Fintsb: A comprehensive and practical benchmark for financial time series forecasting. arXiv preprint 0r.iv:2502.1883. 2025a.
Yifan Hu, Peiyuan Liu, Peng Zhu, Dawei Cheng, and Tao Dai. Adaptive multi-scale decomposition framework for time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 17359-17367, 2025b.

---

Published as a conference paper at ICLR 2026
Yifan Hu, Guibin Zhang, Peiyuan Liu, Disen Lan, Naiqi Li, Dawei Cheng, Tao Dai, Shu-Tao Xia, and graph filtration for time series forecasting.
Shirui Pan. Timefilte: Patch-specific In second International Conference Machine Learning. 2025c.
Yifan Hu, Jie Yang, Tian Zhou, Peiyuan Liu, Yujin Tang, Rong Jin, and Liang Sun. Bridging past and future: Distribution-awy -warre alignment for time series forecasting. In The Fourteenth International Conference on Learning Representations, 2026.
Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shouhong Ding, Zhengyang Zhou, and Yang Wang. Hdmixer: Hierarchical dependency with extendable patch for multivariate time series forecasting. In Proceedings of the AAAI conference artificial intelligence, volume 38, pp. 12608-12616, 2024a.
Qihe Huang,
Lei Shen, Ruixin Zhang. Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. Proc. of NeurlPS, 2024b.
Qihe Huang, Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang, and Yang Wang. Timebase: The power of minimalism in efficient long-term time series forecasting. In International Conference on Machine Learning, pp. 26227-26246. PMLR, 2025a.
Qihe Huang, Zhengyang Zhou, Yangze Li, Jiaming Ma, Kuo Yang, Binwu Wang, Xu Wang, and Yang Wang. Storm: Synergistic cross-scale spatio-temporal modeling for weather forecasting. In The Fourteenth International Conference on Learning Representations, 2026.
Songtao Huang, Zhen Zhao, Can Li, and Lei Bai. Timekan: Kan-based frequency decomposition learning architecture for long-term time series forecasting. arXiv preprint arX1v:2502.06910, 2025b.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6900 2014.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems, 32, 2019.
Zhe Li, Xiangfei Qiu, Peng Chen, Yihang Wang, Hanyin Cheng, Yang Shu, Jilin Hu, Chenjuan Guo, Aoying Zhou, Qingsong Wen, et al. Foundts: Comprehensive and unified benchmarking of foundation models for time series forecasting. arXiv preprint arXiv:2410.1 1802, 2024.
Zhengyu Li, Xiangfei Qiu, Yuhan Zhu, Xingjian Wu, Jilin Hu, Chenjuan Guo, and Bin Yang. Gegnet: Graph-consistent generative network for time series forecasting with exogenous variables. In ICLR, 2026.
Shengsheng Lin, Weiwei Lin, Xinyi Hu, Wentai Wu, Ruichao Mo, and Haocheng Zhong. Cyclenet: Enhancing time series forecasting through modeling periodic patters.. Advances in Neural Information Processing Systems,
2024a.
Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with 1k parameters. arXiv preprint arXiv:2405.0090 2024b.
Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and CL Philip Chen. Sparsetsf: Lightweight and robust time series forecasting via sparse modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dust- dar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In #
VALUEE, 2022a.
Xvyuan Liu, Xiangfei Qiu, Hanyin Cheng, Xingjian Wu, Chenjuan Guo, Bin Yang, and Jilin Hu. Astgi: Adaptive In ICLR, 2026a.
graph interactions for irregular multivariate time series forecasting.
12

---

Published as a conference paper at ICLR 2026
Xvyuan Liu, Xiangfei Qiu, Xingjian Wu, Zhengyu Li, Chenjuan Guo, Jilin Hu, and Bin Yang. Rethinking irregular time series forecasting: simple yet effective baseline. In AAAI, 2026b.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in neural information processing systems, 35: 9881-9893, 2022b.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023.
Ziqi Liu. Freqmoe: Enhancing time series forecasting through frequency decomposition mixture of experts. arXiv preprint 15 2025.
Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The twelfth international conference on learning representations, pp. 1-43, 2024.
Jiaming Ma, Zhiqing Cui, Binwu Wang, Pengkun Wang, Zhengyang Zhou, Zhe Zhao, and Yang Wang. Causal learning meet covariates: Empowering lightweight and effective nationwide air quality forecasting. In International Joint Conference on Artificial Intelligence, 2025a.
Jiaming Ma, Binwu Wang, Qihe Huang, Guanjun Wang. Pengkun Wang, Zhengyang Zhou, and Yang Wang. Mofo: Empowering long-term terg time series forecasting with periodic pattern modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025b.
Jiaming Ma Binwu Wang, Guanjun Wang, Kuo Yang, Zhengyang Zhou, Pengkun Wang, Xu Wang, and Yang Wang. Less but more: Linear adaptive graph learning empowering spatiotemporal forecasting, Advances in Neural Information Processing Systems, 2025c.
Jiaming Ma, Binwu Wang, Pengkun Wang, Zhengyang Zhou, Xu Wang, and Yang Wang. Bist: A lightweight and efficient bi-directional model for spatiotemporal prediction. Proceedings of the VLDB Endowment, 1 18(6):6663-1666 2025d.
Jiaming Ma, Binwu Wang, Pengkun Wang, Zhengyang Zhou, Xu Wang, and Yang Wang. Robust spatio-temporal centralized interaction for ood learning. In Forty-second International Conference on Machine Learning, 2025e.
Jiaming Ma, Binwu Wang, Pengkun Wang, Zhengyang Zhou, Yudong Zhang, Xu Wang, and Yang Wang. Mobimixer: A multi-scale spatiotemporal mixing model for mobile traffic prediction. IEEE Transactions Mobile Computing, 2025f.
Jamming Ma, Guanjun Wang, Sheng Huang, Kuo Yang, Binwu Wang, Pengkun Wang, and Yang Wang. Spatiotemporal causal decoupling model for air quality forecasting. arXiv preprint arXiv:2505.201 19, 2025g.
Jiaming Ma. Siyuan Mu, Ruilin Tang, Haofeng Ma. Qihe Huang, Zhengyang Zhou, Pengkun Wang, Binwu Wang, and Yang Wang. To see far, look close: Evolutionary forecasting for long-term time series. arXiv preprint arXiv:2601.220 231111.11,1 2026a.
Jiaming Ma, Binwu Wang, Pengkun Wang, Xu Wang, Zhengyang Zhou, and Yang Wang. A general relearner: Empowering spatiotemporal prediction by re-learning input-label residual. arXiv preprint 22w.:22022.0533. 2026b.
Yuqi Nie, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022.
Xihao Piao, Zheng Chen, Taichi Murayama, Yasuko Matsubara, and Yasushi Sakurai. Fredformer:
Frequency debiased transformer for time series forecasting. In Proceedings of the 30th ACM SIGKDD conference knowledge discovery and data mining, pp. 2400-2410, 2024.
Xiangfei Qiu, Jilin Hu. Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. TFB: Towards comprehensive and fair benchmarking of time series forecasting methods. In Proc. VLDB Endow., pp. 2363-2377, 2024a.
13

---

Published as a conference paper at ICLR 2026
Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting, arXiv preprint 9rXiv22412.10859 2024b.
Xiangfei Qiu, Zhe Li, Wanghui Qiu, Shiyan Hu, Lekui Zhou, Xingjian Wu, Zhengyu Li, Chenjuan Guo, Aoying Zhou, Zhenli Sheng, Jilin Hu, Christian S. Jensen, and Bin Yang. Tab: Unified benchmarking of time series anomaly detection methods. In Proc. VLDB Endow.. pp. 2775-2789, 2025a.
Xiangfei Qiu, Xingjian Wu, Hanyin Cheng, Xvyuan Liu, Chenjuan Guo, Jilin Hu, and Bin Yang. Dbloss: Decomposition-bast based loss function for time series forecasting, arXiv preprint arXiv:2510.23672, 2025b.
Xiangfei Qiu, Yuhan Zhu, Zhengyu Li, Xingjian Wu, Bin Yang, and Jilin Hu. Dag: A dual correlation network for time series forecasting with exogenous variables. arXiv preprint arXiv:2509.1453 2025c.
Lu Ren, Lan Du, David B Dunson, et al. Logistic stick-breaking process. Journal of Machine Learning Research, 12(1), 2011.
Artyom Stitsyuk and Jaesik Choi. xpatch: Dual-stream time series forecasting with exponential seasonal-trend decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 20601-20609, 2025.
Peiwang Tang and Weitai Zhang. Unlocking the power of patch: Patch-based mip for long-term time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 12640-12648, 2025.
Hao Wang, Haoxuan Li, Xu Chen, Mingming Gong, Zhichao Chen, et al. Optimal transport for time series imputation. The Thirteenth International Conference on Learning Representations, 2025a.
Hao Wang. Lichen Pan, Yuan Shen, Zhichao Chen, Degui Yang. Yifei Yang. Sen Zhang, Xinggao Liu, Haoxuan Li, and Dacheng Tao. Fredf: Learning to forecast in the frequency domain. In The Thirteenth International Conference on Learning Representations, 2025b.
Hao Wang, Licheng Pan, Zhichao Chen, Xu Chen, Qingyang Dai, Lei Wang, Haoxuan Li, and Zhouchen Lin. Time-ol: Time-series forecasting needs transformed label alignment. In Advances in Neural Information Processing Systems, 2025c.
Hao Wang. Zhichao Che, Licheng Pan, Xiaoyu Jiang, Yichen Song, Qunshan He, and Xinggao Li. Deepfilter: A transformer-stylee framework for accurate and efficient process monitoring. IEEE Transactions Artificial Intelligence, pp. 1-13, 2026a.
Hao Wang, Licheng Pan, Yuan Lu, Zhixuan Chu, Xiaoxi Li, Shuting He, Zhichao Chen, Haoxuan Li, Qingsong Wen, and Zhouchen Lin. Distdf: Time-series forecasting needs joint-distribution wasserstein alignment. In ICLR, pp. 1-9, 2026b.
Shiyu Wang. Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Shengtong Ju, Zhixuan Chu, and Ming Jin. Timemixer++: A general time series pattern machine for universal predictive analysis. arXiv preprint arXiv:2410. 16032, 2024a.
Shiyu Wang. Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint 2rXiv:2405.1400 2024b.
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125, 2022.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with cuto-correlation for long-tern term series forecasting. Advances in neural information processing systems, 34:22419-224300 2021.
14

---

Published as a conference paper at ICLR 2026
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d ad-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022a.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2022h.
Xingjian Wu, Xiangfei Qiu, Hanyin Cheng, Zhengyu Li, Jilin Hu, Chenjuan Guo, and Bin Yang. Enhancing time series forecasting through selective representation spaces: A patch perspective. In NeurlPS, 2025a.
Xingjian Wu, Xiangfei Qiu, Hongfan Gao, Jilin Hu, Bin Yang. and Chenjuan Guo. VAVAE: A ICML,, 2025b.
enhanced variational autoencoder for probabilistic time series forecasting, In
Xingjian Wu, Xiangfei Qiu, Zhengyu Li, Yihang Wang, Jilin Hu, Chenjuan Guo, Hui Xiong, and Bin Yang. CATCH: Channel-aware multivariate time series anomaly detection via frequency patching. In ICLR, 2025c.
Xingjian Wu, Jianxin Jin, Wanghui Qiu, Peng Chen, Yang Shu, Bin Yang. and Chenjuan Guo. Aurora: Towards universal generative multimodal time series forecasting. In ICLR, 2026.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang. Xiaojun Chang, and Chengqi Zhang, Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery data mining, pp. 753-763, 2020.
Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. arXiv preprint ar arXiv:2307.033: 037556, 2023.
Hengyu Ye, Jiadong Chen, Shijin Gong, Fuxin Jiang, Ticying Zhang, Jianjun Chen, and Xiaofeng Gao. Atfnet: Adaptive fieeurreyy ensembled network for long-term time series forecasting. arXiv preprint arXiv:2404.05/ 192, 2024.
Wenzhen Yue, Yong Liu, Hao Wang, Haoxuan Li, Xianghua Ying, Ruohao Guo, Bowei Xing, and Ji Shi. Olinear: A linear model for time series forecasting in orthogonally transformed domain. Advances in Neural Information Processing Systems, 2025.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference artificial intelligence, volume 37, pp. 11121-11128, 2023.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimession dependency for multivariate time series forecasting. In The eleventh international conference on learning representations, 2023.
Zhaodian Zhang, Guangpo Tian, Fenghua Guo, and Pengfei Wang. Tfp-mixer: A lightweight time and frequency combining model for multivariate long-temn term time series forecasting. Applied Intelligence, 55(7):1-18, 2025.
Hao Zheng, Zhanlei Yang, Wenju Liu, Jizhong Liang, and Yanpeng Li. Improving deep neural networks using softplus units. In 2015 International joint conference on neural networks (IJCNN), pp. 1-4. IEEE, 2015.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-11115, 2021.
Tian Zhou, Ziqing Ma. Qingsong Wen. Liang Sun, Tao Yao, Wotao Yin. Rong Jin, et al. Film: Frequency improved legendre memory model for long-term time series forecasting, Advances in neural information processing systems, 35: 15:77777 2022a.
15

---

Published as a conference paper at ICLR 2026
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequeney enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pp.
PMLR, 2022b.
Jiachen Zhu, Xinlei Chen, Kaiming He, Yann LeCun, and Zhuang Liu. Transformers without normalization. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 14901-14911, 2025.
16

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
2 Related Work
3
Methodology
3.1
Period Bucket for Time Series
3.1.1
Period Detection
3.1.2
Time Series Period Bucket Construction and Representation
3.2
Positive-Negative X-shape Attention for Periodicity Modeling
3.3
Bucket-wise Forecasting
4
Experiments
4.1
Experimental Setup
4.2
Forecasting Performance Comparison
4.3
Ablation Studies
4.4
Hyperparameter Sensitivity Experiments
8
4.5
Complexity Analysis
4.6
Visualization of Periodic Positive- Components in Data, Feature, and Atten- tion Levels
4.7
Attention Visualization
5
Conclusion
10
A Related Work
19
B The Bucket without Periodicity
19
C Mathematical Justification
20
C.1 Mathematics Definition of Period Heterogeneity
20
C.2 A statistical Interpretation of Period-offset Attention (Another View)
20
C,3 The Variance of Period-offset Attention under Period Heterogeneity
22
Experiments
22
D.I Experiments Details
22
D.2 Dataset Periodicity Test
22
D.3
Comparison with Fixed Input Length Settings
23
D.4 Synthetic Dataset
24
D.5
Detailed Forecasting Comparison
24
D.6 Comparison Periodic Heterogeneity Modeling
24
D.7 Comparison of Transformer Variants
24
17

---

Published as a conference paper at ICLR 2026
D.8 Detailed Ablation Study
24
Discussions
25
Use LLLM
27
18

---

Published as a conference paper at ICLR 2026
A
RELATED WORK
Multivariate Time Series Forecasting Deep learning has become the dominant paradigm for multivariate time series (MTS) forecasting thanks to its ability to model nonlinear dynamics and long-range dependencies (Wu al., 2025c;a; Ma et al., 2025f;e; Huang al., 2026; Hu et al., 2026). Researchers have developed numerous specialized and general-purpon purpose methods for time series modeling urchitectures (Qiu al., 2024b; Ma et al., 2026b; Wang et al., 2026a), channel interaction methodologies (Liu et al., 2026a; Hu et al., 2025c; Qiu et al., 2025c), optimization techniques (Qiu et al., 2025b; Ma et al., 2025e; Wang et al., b025c;b;a; 2026b;b), and inference strategies (Christou et al., 2024; Ma et al., 2026a). Early solutions relied on recurrent architectures (c.g.. LSTM, GRU) or convolutional models such as TCN (Hewage et al., 2020). More recently, the Transformer has emerged as popular backbone, offering flexible global dependency modeling for sequence-to- sequence tasks and coming to dominate the field (Wen et al., 2022). For example, ModernTNN (Luo & Wang, 2024) uses large convolutional kernels to substantially widen the receptive field and capture broader temporal dependencies, while Pyraformer (Liu et al., 2022a) combines TCN layers with a Transformer backbone to model both local and global patterns. These methods are applied in other complex scenarios within time series domain (Liu et al., 2026b; Qiu et al., 2025a; Wu et al., 2025b; Hu al., 2025a; Wu et al., 2026).
At the same time, lightweight MLP- models have gained traction by achieving strong forecasting performance with lower computational cost. DLinear (Zeng et al., 2023) decomposes series into trend and seasonal components via moving-averaag kernel and models each with separate linear modules; PatchMLP (Tang& & Zhang, 2025) introduces a patch- based approach and channel mixing to enhance inter-variate communication; and HDMixer (Huang et al., 2024a) employs flexible patch lengths to capture short-term dependencies within patches and long-term dependencies across patches while better modeling variata interactions. Overall, Transformer and lightweight MLP approaches present complementary trade-off's between representational capacity and computational efficiency.
Transformers for Time Series. Although the Transformer architecture has shown significant promise, it poses challenges such as high computational complexity (Zhang & Yan, 2023; Ma et al., 2025g). To mitigate this, various approaches have been proposed to modify the self-attention mechanism, incorporating techniques like distillation and temporal decomposition to reduce computational demands (Zhou et al., 2021; 2022b; Ma et al., 2025a). LogTTrass (Li et al., 2019) introducing a convolutional LogSperse attention that ensured long-distance interactions while reducing the number of interactions, is an early and influential attempt to apply Transformer. Informer (Zhou et al., 2)21) proposes ProbSparse attention while combining with a distillation mechanism to select the most representative query vectors to compute the attention scores. More recently, researchers have integrated signal processing methods with Transformers, leveraging techniques such as signal frequency analysis and decomposition (Piao et al., 2024; Zeng et al., 2023). These advancements have enabled models to more effectively capture and analyze the structural characteristics of time series data. However, the softmax operation in Transformers filters out negatively correlated cociull--preeerin. only positive correlations-wiii leads to loss of information.
B THE BUCKET WITHOUT PERIODICITY
Not all variates in a multivariate time series exhibit clear periodic behavior. To account for such cases, we introduce dedicated bucket Bo, which collects all variates whose Fourier spectra do not reveal statistically significant periodicity. Although B, does not correspond to any explicit cycle, it still processed through the proposed PNA to maintain architectural consistency and allow residual correlations to be captured. For notational convenience, we continue to use the terms period-offsee and period-aligned to denote the two orthogonal attention directions, even though no true period exists. Concretely, learning for bucket B, differs from periodic buckets in two key aspects:
No segmentation into periodic fragments. Since Bo has no detected period length, do not segment its sequences into smaller fragments. Instead, each variate j is withour any padding and folding operation preserving the full temporal structure without periodic folding.
19

---

Published as a conference paper at ICLR 2026
@ Offset attention with absolute distance. In the absence of periodic structure, the offset axis is defined by absolute temporal distance as follows,
8 - -0,
(16)
where the period-offset attention is also coupled with positive attention and negative attention with the same modulation as in periodic buckets. This allows PHAT to still capture local autocorrelations and longer-range and potential temporal recurrences.
An important consequence is that the period-aligned attention matrix degenerates into the identity since the period-aligned attention matrix Å E R after softmax normalization along the last axis, all entries are equal 1. Thus, the PNA attention in Bo equal to,
PNA(Z A X1
A (Ax2
X2 v) = A V R gxxixd
(17)
handled consistently within the PHAT framework: they bypass unnecessary folding while still benefiting from offset-based attention capture temporal dependencies. By unifying both periodic and non-periodic buckets under the same framework, PHAT achieves robustness across diverse real-world datasets where not all signals are strictly cyclical.
Overall, this design ensures that variates without periodicity
C MATHEMATICAL JUSTIFICATION
In this section, we provide mathematical justification to clarify the motivation of PHAT and to address concerns raised during the review process. Section C. definite the Period Heterogeneity by mathematical formulation. Section C.2 shows that period-offset attention exhibits periodic behavior aligned with the autocorrelation periodicity of time series. Section C.3 demonstrates that decomposition of positive and negative paths yields strictly lower variance than vanilla attention, offering a clearer learning-theee theoretic rationale for the effectiveness of our method.
C.I A MATHEMATICS DEFINITION OF PERIOD HETEROGENEITY
Here formally defines period heterogeneity.
Definition: Period Heterogeneity. Period heterogeneity refers to the phenomenon where different variates exhibit distinct periodic lengths and periodic correlation patterns.
Different periodic lengths. There exist two variates c, and whose dominant periods differ, i.c.,
Pa + Pay.
(18)
where Pa., and P., denote their corresponding fundamental period lengths.
Different periodic correlations. Although periodic patterns are commonly characterized by the presence of correlations across specific time lags, existing models typically overlook the sign of such correlations. In particular, real-world sequences may exhibit negative correlations between certain time steps within period. Let po(6) ACF Te,t-5) denote the autocorrelation function of variate at lag 8 within a single period. Then this kind of period heterogeneity exists if there exist a variate c* such that
sign(pe-(1P/2)) <.
(19)
This definition reminds us that modeling period heterogeneity not only need to differences in period lengths among variates, but also to the diversity of period correlation patterns-ranging from purely positive correlations to positive and negative correlation patterns.
C.2 A STATISTICAL INTERPRETATION OF PERIOD-OFFSET ATTENTION (ANOTHER VIEW)
In this subsection, we will show that period-offset attention aligns with the inductive bias of auto- correlation in real-world time series by separating positive and negative paths and assigning them different modulation types. In addition, through statistical stick- breaking perspective (Ren et al., 1), we provide an interpretable view of the mechanism and show that the upper bound of the attention logits is monotonic with respect to the period distance.
20

---

Published as a conference paper at ICLR 2026
Proposition. Period-offset Attention Stick-breaking Allocation Let (/m, n) and 17mm. n] denote the raw positive length Po. receptively. Define the modulated positive and negative logits of the period-offset attention as
and negative logit between time steps m and within a detected period of
Č(m, m] ((m,n) n|
SE Softplus(CIm, s)),
(20)
7mm, n| = =7[m, nl
Softplus(1) (m(m,
(21)
#E V (8)
where Ami. {s 6m.. <
8 mmn U [n) and Vin.. {s m. > 8 U {nz is the set of offsets
closer and farther m than under corresponding distance 6t m... Then, before the normalization step in softmax, the exponentiated logits satisfy the following stick-breakin breaking decomposition (Ren al., 2011):
exp(čIm, nl) = or (C(m, ml)
II
(1- 1-(o((mm, ())m,
(22)
{s/8m.. <bm,n.)
exp(fimm, nl) nl)
II
(1- (n-o,nlm,s) s)),
(23)
where (-) denotes the sigmoid function.
Proof of Proposition: The logits from positive branch of period-offset attention is,
č(m,n) m)
=(mm, m]
Softplus([Im, s])
A (0)
-[(m, n]]
Σ
(1+ exI+exp (Cp s)))
SE (6)
=[[m,n] - In (1 exp (Cim, nl))
Σ
In (1 + exp (C/m, s]))
SEA - [n)
exp (C|m, +
=In 1+ exp (c(m,
In ( exp (C(m, s()
(24)
SE {m}
exp (Cim, s))
=In 1 exp-1 (C|m,
Σ
In
+ exp (C (m,
SEA min (x)
[In(((m nl)) +
Σ
In
1
1 exp ())
(eA(h)
(n)
=In(a(( +
Σ
In [((mm s)),
A (b)
(n)
where (-)) is the sigmoid function and set (b) - [n] { < 8 means all of the closer
time step from m than n. Hence before the normalization operator in Softamax(-), the postie logits are activated by exponential function satisfying.
exp (Cim, nl) = cxp (In (a(Cm,, nl)) * exp (
Σ
In @Cim, sl)))
falbm, <6m,at
exp (In (a(((,, [l)
II
exp (In (1 (Smm, s))))
(25)
{#/5m.. <6m.n)
=o([[m, n))
II
(1- ((immm)
[s]8m,a <dm.n)
21

---

Published as a conference paper at ICLR 2026
The proof of the situation of negative logits is similar.
This identity reveals that the unnormalized attention coefficient assigned to time step n is exactly stick-breaking allocation. Attention cocfficient is allocated sequentially in the order of periodic distance: nearby offsets consume mass first, while farther offsets only receive the leftover. Unlike standard dot- dot-product attention, which distributes weights independently, our period-offset formulation embeds period-aware stick-breaking bias, yielding structured sparsity, interpretability. and inductive alignment with the autocorrelation structure of time series.
Corollary: Local dominance of periodic distance
From Proposition 1, it follows that for any reference time step m, the allocation weight to time step in period-offset attention is strictly upper-bounded by the leftover stick mass after all closer offsets have been considered:
exp(C(m, nl) <
II
(1 (1- (L[m,((m,s)))
(26)
{alom <dmmn)
exp(fimmn1]) <
II (1-o(m[[m,3)
(27)
{=10m,a220mma)
This corollary formalizes the intuition that local periodic dependencies are always prioritized: nearby (far away) time steps consume most of the positive (negative) attention budget, and more distant steps are suppressed unless strongly supported by their logits. This guarantees that period-offset attention respects the inductive bias of autocorrelation in real-worle time series, where closer (farther) periodic positions tend to carry stronger positive (negative) correlations.
C.3 THE VARIANCE OF PERIOD-OFFSET ATTENTION UNDER PERIOD HETEROGENEITY
As the request from reviewer (CVf, we add a brief learning-theowr theoretic argument to further strengthen the theoretical contribution. Let the vanilla attention between steps and be A.j = Softmax (p Q,K,) J )
with variance V[A,y] = o². Period-offset attention decomposes this interaction into positive and negative paths, leading to
v[A.] = Aij)
(28)
where Aij E (0, 1) when periodic heterogeneity induces non-zero negative path. Thus, period-offset attention yields strictly lower variance than vanilla attention. This provides clear learning- -heniingthhorettic intuition: the positive and negative decomposition reduces attention-logii variance in heterogeneous periodic settings, producing more stable and accurate estimates than vanilla attention.
D EXPERIMENTS
D.1 EXPERIMENTS DETAILS
We report the detailed hyperparameters of our model in Table 4. All experiments were conducted on single NVIDIA A100 GPU with 80 GB of memory using PyTorch. Forecasting performance was evaluated using mean squared error (MSE) and mean absolute error (MAE). For a comprehensive assessment, we disabled the "Drop Last" batch sampling procedure (Li et al., 2024; Qiu al., 2024a) and trained models with the Adam optimizer (Kingma & Ba, 2014). Because models vary in sensitivity input history, the look back length T was treated tunable hyperparameter and each model's 's best performance is reported. Hyperparameter settings are summarized in Table 4.
D.2 DATASET PERIODICITY TEST
We further visualize temporal correlations across multiple datasets. We use the Autocorrelation Function (Dégerine & Laabeix--Lacrix, 2003) to analyze the autoregressiveness between time steps. As shown Figure 6, most datasets exhibit complex periodic heterogeneity. For example, in the ETTh1 dataset, variate shows only positively correlated periodicity, while variate 4 demonstrates
22

---

Published as a conference paper at ICLR 2026
Table 4: Optimal hyperparameters on our experiments.
Hygenperancters
NNS
Exchange
FRED-MD
ETTbi
ETTh:
ETTml
ETTml
AQShuayi
AQWas
ILI
CecLan
ZatNoo
NASDAO
NYSE
Butch Size
16
32
32
16
32
Lesming
Dimenion
16
Layers
2
1
Head Number
Batch Size
16
52
16
64
Lesming
0.01
Model Dimensiz
96
12
24
Layers
Head Number
3.
513
336
512
Butch Size
16
32
16
64
Lesming
0,0001
0.0001
Midel Dimensor
53
16
10
16
Layera
Head Number
513
Batch Size
32
16
64
Lesming
0.001
0.0001
0.001
Model Dinensog
16
32
12
32
Layers
Head Number
512
ETThl
Average
Channel 03
Channel #1
Channel #5
ETTh2
Average
Channel #1
Channel #2
Channel #1
ETTml
Average
Channel #3
Chaninll #1
Channell #5
ETTm2
Average
Channell
Channel
Channel #4
Figure 6: The autocorreltionn visualization of ETT datasets.
bilateral periodic pattern with both positive and negative correlations, and variate 5 exhibits no significant periodicity. In contrast, the ETTh2 dataset lacks clear periodicity across the entire dataset.
D.3 COMPARISON WITH FIXED INPUT LENGTH SETTINGS
The handling of input windows time series forecasting tasks has always been flexible. In our experiments, treated the historical window length as hyperparameter, allowing models to flexibly select the length based on their own characteristics. We believe that this approach provides a more comprehensive evaluation. In contrast, maintaining fixed input window length avoids fairness
23

---

Published as a conference paper at ICLR 2026
concerns arising from unequal usage of historical information across models. Here, we do not assess the advantages or disadvantages of these two settings. Instead, we further supplement a comparison under the fixed look back window setting, with the results presented below. As shown in Table Our model still achieves the best performance.
Table 5: Comparison in the same input setting. ETTh1 is 96 96 and the others 24 36.
Dataset
Ours
TimeKAN
xPatch
Ampliter
CycleNet
TimeMixer
SpurseTSF
iTransfommrr
Pathformer
PDF
MSE
0.375
0.378
0,379
0.443
0.400
0.381
0,400
0.386
0.386
0.378
ETTH
MAE
0.389
0.402
0.101
0.428
0-415
0.397
0.403
0,405
0.392
0.396
2,062
2.857
2,704
3,455
2015
5.498
2,112
2,066
0.876
1.033
1.064
0.863
0,885
1.154
MSE
0.606
0,610
0.616
0,752
0,967
0.619
1,363
0.615
0.643
0.624
NASDAQ
MAE
0.534
0.542
0.539
0.629
0.729
0.546
0.876
0.546
0.578
0.550
D.4 SYNTHETIC DATASET
We systematically created a synthetic dataset by combining ETTh1 (with period of 24) and ETTml (with a period of 96), which record data from the same variates but at different time granularities. Each dataset was divided into four equal parts along the time axis. Then, we alternately sampled each segment to construct the Synthetic, represented as ETTh][:]], [ 21 ETTh 2 ETTml :3
[.31
As result, the synthetic dataset exhibits mixed periodic lengths of (24, 96, 24, 96).
D.5 DETAILED FORECASTING COMPARISON
We present additional benchmark comparison results Table 6 and provide fine fine-graine grained experimental results for ETTh in Table 7. Among the advanced baseline models, Crossformer achieves strong predictive performance due to its cross-dimension attention mechanism, which facilitates efficient modeling of long-term dependencies. PatchTST, the other hand, introduces sule-supe learning and adopts a channel channel-indeeenee independent strategy to capture complex temporal dynamics. Overall, PHAT achieves -tate-of-the-arr (SOTA) performance, demonstrating exceptional adaptability to periodic heterogeneity. It outperforms other models on the majority of metrics, ranking in the top two positions for 86% of the evaluation metrics, thereby showcasing significant performance advantage.
D.6 COMPARISON OF PERIODIC HETEROGENEITY MODELING
We further compared the performance of models specializing in periodic modeling under mixed periodicity scenarios, with results shown in Table 8. ETTh and ETTm exhibit clear periodicity, while ILI and CzeLan show significant differences in periodicity between variates. The NASDAQ dataset displays no periodicity, whereas the Synthetic dataset demonstrates mixed periodicity. Our analysis shows that these periodic learners struggle to handle periodic heterogeneity effectively. On datasets without clear periodic patterns, such as NASDAQ, PDF and SpareTSF exhibit significant performance error.. In contrast, our model remains robust to periodic heterogeneity.
D.7 COMPARISON OF TRANSFORMER VARIANTS
We further evaluated the effectiveness of the proposed positive-negative attention mechanism (PNA) by comparing it with other attention variants. As shown in Table 9, found that Crossformer improves representation capability by modeling the dependencies between variates across channels. However, the interaction of variates with different periodic characteristics may introduce confusion. In contrast, the proposed attention mechanism, PNA, achieved the lowest prediction error. This is because it comprehensively models both the positive and negative correlations of periodicity, demonstrating strong periodic representation capabilities.
D.8 DETAILED ABLATION STUDY
In this section, we conduct combined ablation studies across additional datasets and provide further analysis. Our results show that jointly modeling both positive and negative periodic correlation is
24

---

Published as a conference paper at ICLR 2026
Table 6: Multivariate forecasting results on 14 datasets. We report MSE and MAE. Best results are bold, second-best are underlined The corresponding results on ETThl and ETTh2 (denoted hT ETTml and ETTm2 (denoted ETTm) are averaged for better presentation.
Methods
Ours
PTTS
PatchTST
Crosiformer
Miinear
TimessNet
FEDlormer
Trilormer
RLLM
Stationary
2024
2023
2023
2023
2023
2022
2022
2022
2023
Metrics
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAB
96
0.316
0.327
0.371
0.326
0.367
0.331
0.371
0.362
0.391
0.358
0.400
0.668
0.543
0.326
0.169
0:156
192
0.360
0.394
0.365
0.399
0.379
0:405
0.566
0.523
0.384
0.404
0.42
0428
0418
0436
0.867
0.600
0.382
0.409
0.497
0.479
336
0.384
0.412
0.385
0.416
0.404
0.430
0.587
0.543
0.400
0.419
0.466
0.461
0.24
0.462
0.909
0.630
0.403
0.430
0.495
0.482
720
0.412
0.441
0.409
0:442
0.422
0.459
0.945
0.698
0.423
0.447
0.478
0.472
0,479
0.88%
1,025
0.600
0.444
0.4645
0.625
0.558
96
0.300
0.255
0.305
0.379
0.232
0.200
0.265
0.322
0.340
0.313
0.300
0.259
0.313
0.362
192
0.272
0.323
0.27%
D.328
0.275
0.331
0.372
0.413
0.287
0.335
0.323
0.357
0.436
044
0430
0.432
0.250
10.32
0.416
0:412
336
013111
0.349
0.320
0.255
0.319
0.359
0.501
0.516
0.323
0.356
0.368
0.386
0.492
0.472
0.559
0.458
0.326
0.357
0.505
0.453
720
0.388
0.392
0.390
0.397
0.389
0.402
0.752
0.613
0.396
0.400
0.444
0.428
0.536
0.501
1.209
0.666
0.393
0.400
0.595
0.506
24
0.681
0.351
0.870
0.063
0,740
0.590
0.741
0.39
0.758
0.592
0.224
0.579
0.785
0.618
1.382
0.229
0.316
0.651
1.274
0.900
NNS
36
0.640
0.538
0.814
0.655
0.694
0.595
0,703
0.589
0.693
0.577
0.717
0.585
0.727
0.606
1.352
0.920
0.583
0.702
1.318
0.93
48
0.613
0.532
0,780
0.644
0.667
0.585
0.669
0.575
0.688
0.587
0.647
0.556
0:623
0:555
1.348
0.918
0.989
0.741
1.277
0.905
0.6633
0.650
0.582
0,683
0.679
0.587
0.633
0.630
0.559
1.346
0.918
0.633
1.313
0.927
AQWin (QSSungg) FRED MD Exchange
96
0.083
0.205
0
0.199
0.067
0.204
0.231
0.356
0.085
0.204
0.112
0.242
0.138
0.268
0.201
03355
0.087
0.210
0.121
0.247
192
0.169
0.295
0.173
0.295
0.177
0.300
0.460
0.509
0.175
0.297
0.209
0.334
0.273
0.379
0.53
0.495
0.182
0.308
0.220
0.337
336
0.316
0.40%
0.297
0.399
1.034
0.825
0.330
0.409
0.358
0.435
0.457
0.485
0.705
0.630
0.318
0.419
0.352
0.437
720
0.773
0.609
0.825
0.684
0.843
0.692
1,576
1.021
0.838
0.690
0.944
0.736
1.158
0.828
1,395
0.915
0.815
0.681
0.725
0.686
24
21.4
0.903
56.779
1.374
$2.00K
0.963
585.599
3,559
$2.125
0.93
43,219
1,265
66.090
1.633
195.3667
5 (.P4))
401183
1,145
47,852
1,23X
36
38.363
1.161
97.396
1.374
1.345
398.
728
3.716
58.332
1.258
69.554
94,359
1.663
412.
165
5.210
90.434
1.670
68.
1.493
48
52,065
445.471
2.183
91.83
1.648
414,253
3.929
$2.184
1.M0
95,071
1.810
29.778
133
532
[31.08]
2.119
92,906
1.736
60
76.292
1.679
[96.613
2.525
[27.018
1.958
122.864
4.093
09.6625
1,88
116.341
1.976
773,615
2 2.435
132:464
5.299
[80
067
2.397
117.756
1.929
96
0.665
0.468
0.655
0.485
0.646
0.478
0.655
0.488
0.653
0,480
0.658
0,488
0.706
0.525
0.685
0.492
0.664
0.456
0.771
0.518
192
0.007
0.488
0
0,503
0.688
0.498
0.674
0.499
0.701
0.506
0.707
0.512
0.729
0.53]
0.681
0.50]
0.705
0.504
0.775
0.53
336
0.737
0.502
0.720
0.515
0.210
0.313
0.704
0.515
0.722
0.519
0.286
0.538
0.824
0.569
0.731
0.525
0.725
0.517
0.821
0.551
720
0.795
0.529
0.774
0
0.768
0.539
0.247
0,518
0.777
0,545
0.756
0.528
0.794
0.561
0.742
0.534
0.782
0,544
0.793
0.547
96 0.776
0.757
0.473
0.745
46.688
0.250
0.758
0.475
0.787
0.4%)
0.79
0.508
0.762
0.474
0.766
0,852
0.508
192
0,808
0.475
0.806
0.492
0.793
0.490
0.762
0.479
0.809
0.496
0.278
0.489
0.825
0.517
0.786
0.484
0.809
0,494
0.800
0.524
336
0.824
0.487
0.826
0.504
0.819
0.502
0.802
0.504
0.830
0.508
0.815
0.505
0.863
0.537
0.802
0.495
0.831
0.505
0.864
0.532
730
01907
0.517
0.900
0:532
0,800
0.533
0.829
2
0.906
0.538
0.869
0.519
0:907
0.552
0.852
0.519
0.90
0.536
0.807
0.526
24
1.318
0,705
2.182
1.002
1.922
0.872
2.581
1.096
1.958
0.919
2.131
0.958
2.39%
1002
6.052
1.730
2:256
0.956
2.354
1.066
36
1.523
0.775
2,330
1.051
1.809
0.866
3.549
1.196
1.920
0.916
2.612
0.974
2.410
1.005
6.111
L.743
2.133
0.992
2.226
1.031
1.437
0.775
2.761
1.184
1.99
0.883
3.85L
1,288
1.895
0.924
1.916
0.897
2,591
1.033
6.289
1.771
2.034
0.969
2.525
1.003
1.461
0.777
2.929
1.217
9914
0.896
4.692
1.450
1.964
0.9-7
1,995
0.905
2,539
1.070
7,000
1
1.974
0.929
2.410
1.010
96
0.170
0.284
0.187
0.243
0.176
0.232
0.581
0.443
0.178
0.228
0.179
0.239
0.231
0.311
0.818
0.519
0.180
0.232
0.210
0.296
NYSE NASDAQ ZaiNea Cxel.
336
192
0.236
0.206
0.254
0.227
0.247
0.214
0.292
0.263
0.236
0.205
0.263
0.286
0.971
0,705
0.596
0.503
0.244
0,210
0.280
0.252
0.265
0.216
0.313
0.28
0.298
0.283
0.349
0.563
0.962
1.161
0.639
0.589
0.245
0.212
0.281
0.255
0.311
0,250
0.315
0.341
0.285
0.320
0.316
0.763
0.290
0.526
0,263
0.338
0.426
0.449
0.741
0.782
0,312
0.328
0.379
96
0.439
0.383
0.449
0.412
0.429
0.405
0.630
0.418
0.447
0.410
0.478
0.419
0.476
0.450
0.441
0.419
0.451
0.411
0.534
0.449
192
0.503
0.429
0.511
0.447
0.4%4
0.449
0.479
0.449
0.503
0,447
(1491
0/420
0.544
0.470
0.493
0.451
0.508
0.448
0.562
0.475
336
0.547
0.456
0.544
0.469
0.538
0.475
0.505
0.545
0.470
0.551
0.480
0.628
0.523
0.525
0.467
0.549
0.614
0.502
720
0,592
0.477
0.585
0.491
0.573
0.486
0.560
0.494
0.589
0.497
0.626
0.511
0.653
0.562
0.564
0.491
0.598
0.504
0.092
0.542
24
0.416
0.473
0.709
0.645
0.649
0.567
1.149
0.745
0.557
0.522
0.587
0.533
0.557
0.45
2.737
1.334
0./07
0.615
0.655
0.607
36
0.610
0.600
1.058
0.778
0.821
0.682
1.414
0.885
0.869
0.668
0.792
0.664
0,808
0.628
3.387
1.534
1.379
0.835
0.991
0.695
48
0.731
0.834
0.793
2.108
1.136
1.152
0.770
1.216
0.783
1.137
0.746
3.425
1.555
1.179
1,260
0.814
60
0
0.779
1.153
0.818
1.247
0.843
2.276
1.201
1.284
0.809
1.220
0.768
1,251
0.783
3.313
1.537
1.303
0.853
L119
0.819
0.161
0.251
0.301
0:410
0.226
0.296
0.820
0.8-1
0,193
0.283
0.267
0.335
0.159
0.251
2.353
1.258
0.313
0.361
0.249
0.342
36
01.9%
0.497
0.376
(.289
0.344
3.358
0.371
0,403
0.,455
0.427
0.575
0.492
1.049
0.951
0.464
0.438
0.573
0.506
0.477
0.57
1,733
0.538
0.48
0.521
0.468
60
0.619
0.525
1.036
0.757
0.749
0.572
1.121
0.937
0.631
0.523
0.776
0.629
0.693
0.586
4.696
1.846
0.721
0.563
0.686
0.543
Topl
68
* TTop2
183
14
29
16
20
10
5
beneficial, as it yields a more complete representation of the underlying periodic characteristic. We also observe that using only positive periodicity performs slightly better than using only negative correlations, likely because positive correlations capture the most direct and dominant periodic dynamics. For datasets where negative periodic correlations are weak or largely absent (e.g., ETTh2, ETTm2, and AQShunyi), whether or not negative paths are included-ov whether their attenion logits are modulated has minimal impact on performance. This behavior arises naturally from the gating mechanism A, which adaptively determines the extent to which negative correlations should be modeled based dataset characteristics. As result, the model avoids unnecessary overfitting when negative periodicity is not present.
E DISCUSSIONS
Although both our model and patch- Transformers rely on operations such as unflattening or folding the sequence, their design philosophies diverge significantly.
Patch-based methods, such as PatchTST (Zeng al., 2023), Crossformer (Zhang & Yan, 2023), and PDF (Dai et al., 2024), divide the temporal sequence into contiguous segments, reinterpreting one dimension of each patch as a feature axis. This effectively performs temporal down-sampling by
25

---

Published as a conference paper at ICLR 2026
Table 7: Detailed forecasting results on ETThI, ETTh2, ETTml and ETTm2 datasets. We report
MSE and MAE, Best results are bold second-best are underlined
Datasels
ET1h1
E1112
kTTml
kTTm2
Top2
Methods
96
192
336
720
192
336
720
192
336
730
192
336
720
MSE
0.361
0.393
0.414
0.441
0.272
0.328
0.354
0.384
0.257
0.328
0.360
0.422
0.162
0.316
0.262
0.355
PHAT (Ours)
MAE
0.383
0.110
0.422
0.458
0.330
0.379
0.398
0.425
0.330
0.362
0.361
0.412
0.246
0.285
0.317
0.372
MSE
0.369
0.422
0.279
0.326
0.364
0.408
0,256
0.331
0.354
0.400
0,163
0.238
0.277
0.359
TimeKan
MAR
0.396
0,417
0.490
0.463
0.343
0. 100
0,411
0.443
0.346
0. 0.968
0.
0.417
0.254
0.259
0.330
0.387
MSE
0.37%
0.516
0.275
0.373
0.345
0.288
0.367
0.447
0.202
0.372
0.354
8
xPatch
MAE
0.395
0.26
0.51
0.493
0.312
0. 0.3800
0.403
0.427
0.360
0.428
0.247
0.289
0.320
0.381
MSE
0.573
0,144
0.442
0.455
0.245
0.348
0.363
0.467
0.292
0.337
0.365
0.427
0.174
0.226
0.276
0.356
Amplilier
MAE
0.467
0,474
0,444
0.386
0.419
0.351
0.388
MSE
0.371
0.706
0.431
0.450
0.279
0.312
0.371
0.426
0.299
0.335
0.365
0.117
0.159
0.214
0.269
0.365
CycleNet
MAE
0.396
0,15
0.430
0.464
0.341
0.385
0.413
0.451
0.348
0.367
0.386
0.414
0.247
0.286
0.322
0,382
MSE 0.372
0,413
0,438
0.486
1.221
0.349
0.366
0.401
0.2993
0.335
0.368
0.42%
0.105
0.225
0.277
0.36
TimeMixer
MAE
0.401
0.430
0.450
0.484
0.351
0.387
0.413
0.436
0.345
0.372
0.386
0.417
0.256
0.298
0.332
0.387
MSR
0.361
0.394
01:15
0.419
0.238
0.347
0.358
0.384
0.316
0.348
0.373
0.434
0.100
0.220
0.273
0.361
SparseTSP
MAE
0.386
0.406
0.419
0.440
0,341
0.381
0.396
0.422
0.355
0.376
0.387
0.422
0.256
0.292
0.327
0.381
MSE
0.386
0.495
0.297
0.372
0.388
0.424
0.300
0.341
0.374
0.129
0.175
0.242
0.282
0.375
iTranffrmmrr
MAE
0.405
0.440
0,460
0.487
0.348
0.103
0:117
0.411
0.353
0.380
0.396
0.130
0.266
0,312
0.337
0.391
MSE 0.372
0.408
0.438
0.450
0.279
0.345
0.378
0.437
0.290
0.337
0.374
0.428
0.164
0:219
0:267
0,361
5 Pathformer
MAE
0.392
0.115
0.434
0.463
00,360
0.108
0.155
0.335
0.363
0.381
0.116
0.288
0.319
0.377
MSE 0.360
0.392
0.418
0.456
0.276
0.339
0.374
0.398
0.256
0.321
0.354
0.40%
0.163
0.219
0.209
0.349
7 PDF
MAF
0.391
0.43
0,382
0..4%
0.433
0.164
0.415
0.290
0.330
0.382
MSE 0.376
0.400
0.419
0.435
0.277
0.331
0.360
0.382
0.303
0.337
0.368
0.430
0.165
0.219
0.272
0.359
FITS
MAE
0.396
04488
0.435
0.488
0.345
0.379
0.396
0.425
0.345
0.365
0. 0.384
08:41
0.254
0.291
0.336
0.381
MSE 0.377
0.409
0.431
0.457
0.274
0,348
0.377
0.106
0.289
0,329
0.362
0.416
0.165
0.22]
0.376
0,362
PatcbTST
MAE
0.397
0,455
0.444
0.477
0.337
0.384
0.416
0.441
0:343
0. 0.368
0,390
0.423
0:255
0.798
0.327
0.381
MSE 0.11
0.133
0.710
1,386
0.31
0.374
0.113
0.753
0.369
0.555
0.75
Crosdormer
MAE 0.435
0.438
0.457
0.514
0.603
0.607
0.628
0.882
0.367
0.410
0.432
0.613
0.391
0.416
0.600
0.612
MSE 0.385
01422
0.431
0.499
0.276
0.308
0.40%
0.301
0.355
0.312
0,430
0.218
0.273
0.361
NLinear
MAE 0,403
0,426
0.429
(0:452
0.382
0.408
0.441
0.379
0.385
0.418
0,252
0.290
0.336
0,382
MSE
0.589
(144g)
0.523
0.521
0.314
0,400
0.389
0.434
0.340
0. 3.792
03,423
0.475
0.189
0.254
0.313
0.415
TimesNet
MAE
0.495
0.435
0,448
0,404
0.426
0.453
0.310
0.345
0.402
MSE 0.379
0.420
0.458
0.475
0.337
0.415
0.389
0.483
0.463
0.575
0.618
0.612
0.216
0.297
0.366
0.459
FEDformer
MAE 0.419
0.444
0.466
0.488
0.380
0.428
0.457
0.488
0.163
0.516
0.511
0.551
0.309
0.360
0.400
0.450
MSE 0.390
0,444
0,692
0.549
0.936
1,290
1.325
1.500
0.349
0.387
0.426
0.482
0.276
0.473
0.692
1.936
Triformer
MAE 0.425
0.449
0:179
0.529
0.660
0.768
0.781
0.850
0.368
0.110
0.116
0.176
0.344
0:155
0.519
0.856
MSE 0.370
0.434
0,483
0.282
0.372
0.425
0.301
0.339
0.374
0.423
0.220
0.27
0,363
FiLM
MAR 0.394
0.416
0.435
0474
0.346
0,401
0,425
0.455
0:343
0.365
0.385
0.414
0.254
0.291
0.329
0.386
MSE 0.591
0.379
0.358
0.422
0,484
0.577
0.6%
0.452
0.554
Stationary
MAE
0.524 0,540
0.551
0.658
0.387
0.418
0.413
0.457
0.410
0.451
0.400
0.535
0.373
0.416
0.476
Table 8: Comparison with periodicity-modeling model. The results are average MAE among all
horizon. 1 is the relative percentage increasing regarding PHAT.
Datasets
ETTH
ETTM
ILI
CzeLan
NASDAQ
Synthetic
TimesNet
0.433.0..oms
0,373 10 3635
0.933
0.292 100
0.687 168770.344
0.376 (16.5
PDF
0.408
0,345
0.898
0.268
0,717
0.291 pio
CycleNet
0.412
0,344,1,00
1,77
0.971 FaR
0.269
0,747 6905
0.310
SpareTSF
0.400 g.4000r.000
0.350
1. 1,556
0.303
0.858 91%
0.339
PHAT (Ours)
0.400
0.338
0.758
0.245
0.646
0.243
Table 9: Comparison with various self-attention mechanisms The results are average MAE among all horizon.
Datasets
ETTh
ETTm
ILI
CrossFormer
0.441
0.380
0.742
+ Vanilla Transformer
0.424
0.366
0.768
PatchFormer
0.435
0.373
0.825
PHAT (Ours)
0.400
0.338
0.758
aggregating multiple time steps into a single token. While this approach improves computational efficiency, it comes the cost of mixing distinct temporal roles, leading to the loss of fine-grained alignment among the original time steps.
In contrast, our period folding approach preserves the full temporal resolution by avoiding any form of down-samp wwwn-ssmmlingg...... Instead, the sequence is reorganized into two orthogonal axes axes-peeee -pes-od--od--- offset and geriod-aligned--baseee on the detected periodic structure. Attention is applied along both
26

---

Published as a conference paper at ICLR 2026
Table 10: Details of combination Ablation Study of the period-offiet attention in PNA. Pos (Neg): Positive (Negative) components. Mod: Modulation.
Varianis
FTTml
ETTm2
NN3
PREDATD
AQNhumyi
11
Calam
NANDRO
Fox
MSG
MANE
MAL
MSG
MAA
MAL
315
MSL
MAL
MAL
0.367
0.553
0.599
0215
0.477
0.30
0.162
0.581
0,65
1,318
6.705
6.16
PMad KM6d
MSE
MAE
MSE
MAE
MSE
MAE
MS5
MAR
MSE
MAE
MSE
MAE
USE
MAE
MSE
MAE
MSE
MAE
MSE
MAR
0.368
0.39
0.255
0.54
0.309
0.345
0.160
0.63
0.554
0.42
0.09
0,465
1.60
0.74
0.17
0.84
0.42
0.79
0.365
0.386
0.292
0.300
0.305
0.330
0.167
0.44
0.685
0.533
24,740
0.943
0.666
0.468
1,535
0.764
0.174
0.252
0.19
0.478
0,362
0.385
0,272
0.330
0.45
0.334
0.160
0.43
0,683
0.554
23,876
0.907
0.605
0.468
1.466
0.750
0.177
0.334
0.416
0.479
0.361
1L383
0.272
0.300
0.27 0.30
0.162
0.246
0.681
0L551
21.45
0301
0/65
0.48
1318
0.705
0.17
1220
0.16
0,433
axes, enabling the model explicitly capture rich intra-period and inter-period dependencies while maintaining the integrity of the original temporal sequence.
related idea can be observed in TimesNet etal, 2022a), which also employs a folding operation. However, TimesNet extracts features from the folded representation using convolutional kernels. These kernels aggregate information across multiple positions, which can blur inherent periodic boundaries and disrupt latent cyclic correlations. This misalignment between convolutional receptive fields and true periodic structures limits TimesNet's 's ability to fully capture cycle-aware dependencies. In contrast, model applies X-shaped attention directly to the folded representation, preserving periodic fidelity and ensuring that the periodic structure is consistently leveraged throughout the learning process.
In summary, our fold-and-attend approach stands apart by preserving the full temporal sequence and explicitly structuring it along periodic axes. Unlike patch-based Transformers, which sacrifice temporal resolution for efficiency, or TimesNet, which disrupts periodic integrity with convolutional receptive fields, our design respects periodicity its core. This allows our model generalize effectively across diverse temporal patterns while offering enhanced interpretability in capturing heterogeneous periodic dependencies.
F USE OF LLM
We only use L.MSs as a language optimization tool to polish sentences, improving their readability and fluency.
27


Reviews:
[{"review_id": "YLYvCLQt9O", "reviewer": "Reviewer_aYqm", "summary": "The paper introduces PHAT (Period Heterogeneity-Aware Transformer), a novel architecture designed to handle temporal heterogeneity in video action recognition. Unlike existing temporal models that assume uniform periodicity, PHAT explicitly models multi-scale and non-uniform temporal patterns by decomposing video sequences into adaptive period components. Evaluations on benchmarks show consistent performance gains over recent baselines.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed PHAT design is principled and intuitive, integrating adaptive temporal decomposition into the Transformer framework without major architectural overhead. \n\n- The method is generalizable and can be incorporated into existing video backbones. \n\n- Experimental results are strong and consistent across multiple datasets, showing both improved accuracy and efficiency.", "weaknesses": "- The novelty is moderate, as the idea of handling multi-frequency or periodic dynamics has appeared in previous works on temporal Fourier attention and spectral modeling. \n\n- The mathematical formulation of heterogeneity modeling could be more rigorous; the “adaptive period tokens” are primarily empirical and not theoretically justified. \n\n- The comparisons focus mainly on uniform-period baselines but omit stronger contemporaneous temporal adaptation models. \n\n- The ablation studies are limited—particularly lacking analysis of how many period components are optimal or how PHAT behaves for short versus long actions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C0ORK9Fm2m", "reviewer": "Reviewer_rCFS", "summary": "This paper addresses the challenge that heterogeneous variables in multivariate time series may exhibit different periodic patterns. The authors introduce a **periodic bucket structure** that groups variables based on their periodic lengths, and then model each periodic group separately. To address the lack of “negative correlation” modeling in attention mechanisms, the paper proposes a **positive-negative period-aware attention mechanism**. Experiments across numerous datasets and baselines demonstrate strong performance.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Simple and efficient method, easy to understand.\n2. Clear motivation and well-organized structure.\n3. Extensive experiments with diverse datasets and baselines, offering strong empirical support.", "weaknesses": "1. The experimental validation regarding “attention ignoring negative correlations” is not convincing; raw data analysis alone is insufficient to justify modeling implications at the feature level.\n2. The paper should include an ablation that isolates the Frequency-based Multi-period Prediction component to clarify the exact gain contributed by the core modules, especially since prediction head size can directly affect performance in many settings.\n3. Table 4 only reports FLOPs, but **actual inference latency** is not provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hoo2KcQdJk", "reviewer": "Reviewer_tCVf", "summary": "This paper proposes PHAT (Period Heterogeneity-Aware Transformer) for multivariate time-series forecasting. The key idea is to (i) detect per-variable dominant periods via FFT, (ii) group variables into periodic buckets that share a period, fold sequences into a 3-D tensor (bucket × period-offset × period-aligned), and (iii) apply a Positive-Negative Attention (PNA) with X-shaped receptive field that models phase-aligned vs. within-period relations and explicitly decomposes positive/negative correlations with a period-distance modulation. A frequency-weighted fusion produces the final forecast. Across 14 real-world datasets and 18 baselines, PHAT reports SOTA/top-2 results on most metrics, with strong complexity reductions.", "questions": "As in Weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes PHAT (Period Heterogeneity-Aware Transformer) for multivariate time-series forecasting. The key idea is to (i) detect per-variable dominant periods via FFT, (ii) group variables into periodic buckets that share a period, fold sequences into a 3-D tensor (bucket × period-offset × period-aligned), and (iii) apply a Positive-Negative Attention (PNA) with X-shaped receptive field that models phase-aligned vs. within-period relations and explicitly decomposes positive/negative correlations with a period-distance modulation. A frequency-weighted fusion produces the final forecast. Across 14 real-world datasets and 18 baselines, PHAT reports SOTA/top-2 results on most metrics, with strong complexity reductions.", "weaknesses": "1. the main protocol tunes the look-back T per model and reports the best; a fixed-T comparison is deferred to the appendix. While both settings are shown, the primary table mixing tuned-T results across diverse baselines can blur fairness. Please foreground the fixed-T tables in the main paper (or add both side-by-side) and state the exact search ranges for T and other critical hparams per baseline. \n2. Sensitivity to period detection & K. Periods are extracted by FFT Top-K peaks and rounded to discrete lengths; buckets may overlap. The paper gives a small K-sweep but lacks robustness tests to mis-estimated periods, spectral noise, or drifting cycles. Please add stress tests varying K and perturbing detected periods ±{5–20%}, plus ablations on overlapping bucket policy and bucket cardinality.\n3. The math mainly supports monotonic distance behavior / stick-breaking view of the modulated logits. A brief learning-theoretic argument (e.g., why decomposing positive/negative paths with masking improves bias/variance vs. vanilla attention under heterogeneity) would strengthen the theory section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "chQJRzEgJE", "reviewer": "Reviewer_DkU6", "summary": "The paper provides an in-depth investigation into the frequency heterogeneity among variables in multivariate time series, highlighting that existing homogeneous modeling approaches fail to align with real-world data characteristics. To address this, the authors propose a bucket-based modeling framework that groups and models sequences according to their distinct periodic properties. Moreover, the paper explores the coexistence of positive and negative components within sequences, emphasizing that negative components are often suppressed by the softmax operation and thus require a redesigned attention mechanism to properly capture such “negative relations.” Experimental results demonstrate the effectiveness and robustness of the proposed method.", "questions": "see weeknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is clearly written and easy to follow, with a logical and coherent structure, though certain experimental details could be further refined.\n2. The experimental setup is fair and the comparisons are comprehensive.\n3. The motivation is well grounded, and the problem addressed carries strong research significance.", "weaknesses": "1. Positive-Negative Attention Mechanism: The proposed mechanism is conceptually sound; however, the paper would benefit from multi-level visualizations of the positive and negative components — at the sequence level, feature (patch) level, and attention level. Without such analyses, it remains unclear whether the originally negative correlations in the raw sequence might become positively correlated after complex linear projections. This visualization is crucial for validating the effectiveness of the proposed mechanism.\n2. It is recommended that Table 3 include results on more datasets to enhance the persuasiveness and comprehensiveness of the validation.\n3. The results in Figure 5 and Table 3 could be presented in greater detail, with extended analyses provided in the appendix.\n4. The paper could further discuss how to model sequences that share similar or overlapping periodicities. Since real-world time series often consist of multiple prominent periodic components, the strategy for bucket assignment and cross-bucket interaction among different periods remains an important open question deserving deeper exploration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)