Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift
Authors: Yanru Sun
Zongxia Xie
Emadeldeen Eldele
Dongyue Chen
Qinghua Hu
Min Wu
Venue: NeurIPS 2025 poster ()
Keywords: Time series forecasting
Pattern recognition
Distribution shift
Primary area: deep_learning

Abstract: Time series forecasting, which aims to predict future values based on historical data, has garnered significant attention due to its broad range of applications. However, real-world time series often exhibit heterogeneous pattern evolution across segments, such as seasonal variations, regime changes, or contextual shifts, making accurate forecasting challenging. Existing approaches, which typically train a single model to capture all these diverse patterns, often struggle with the pattern drifts between patches and may lead to poor generalization. To address these challenges, we propose TFPS, a novel architecture that leverages pattern-specific experts for more accurate and adaptable time series forecasting. TFPS employs a dual-domain encoder to capture both time-domain and frequency-domain features, enabling a more comprehensive understanding of temporal dynamics. It then performs subspace clustering to dynamically identify distinct patterns across data segments. Finally, these patterns are modeled by specialized experts, allowing the model to learn multiple predictive functions. Extensive experiments on real-world datasets demonstrate that TFPS outperforms state-of-the-art methods, particularly on datasets exhibiting significant distribution shifts. The data and code are available: https://github.com/syrGitHub/TFPS.

Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift
Yanru Sun, Zongxia Xie, Emadeldeen Eldele Dongyue Chen, Qinghua Hu, Min Wu
College of Intelligence and Computing, Tianjin University, China $12R, Agency for Science, Technology and Research, Singapore. Departmeett of Computer Science, Khalifa University, UAE.
(yanrusun, dyuechen, huginghua]@t edu, en, aaddiexie@hotmae com, emad0002@ntu. edu.sg, wumin@a-ster saaeeassstar..... ssw.sttttreeee .aaa.eedu.ggg..
Abstract
Time series forecasting, which aims to predict future values based on historical data, has gamerred significant attention due to its broad range of applications. However, real-world time series often exhibit heterogeneous pattern evolution across segments, such as seasonal variations, regime changes, or contextual shifts, making accurate forecasting challenging. Existing approaches, which typically train a single model to capture all these diverse patterns, often struggle with the pattern drifts between patches and may lead poor generalization. To address these challenges, we propose TFPS, a novel architecture that leverages pattern-specific experts for more accurate and adaptable time series forecasting. TFPS employs a dual-domain encoder to capture both time-domain and frequency-domain features, enabling more comprehensive understanding of temporal dynamics. It then performs subspace clustering to dynamically identify distinct patterns across data segments. Finally, these patterns are modeled by specialized experts, allowing the model to learn multiple predictive functions. Extensive experiments on real-world datasets demonstrate that TFPS outperforms ttateof-thhe-ar methods, particularly on datasets exhibiting significant distribution shifts. The data and code are available: httpp:://githun com www./oomm/aa.eeeeee http//yyyGtttu/TFSS
Introduction
Time series forecasting plays critical role in various domains, such as finance [1]] weather 3 3,63 [2]], traffic 335 22]. and others [59 68), by modeling the relationship between historical data and future outcomes. However, the inherent complexity of time series data, including temporal dependencies and non-stationarity, poses significant challenges in achieving reliable forecasts.
Recent Transformer- models have shown great promise in time series forecasting due to their ability to model long-range dependencies [29] 53). In particular, models like PatchTST [44] split continuous time series into discrete patches and process them with Transformer blocks. While these models are effective, a closer examination reveals that patches often exhibit distribution shifts, which are frequently associated with concept drift [36]. For example, patches from different regimes, seasons, or operating modes may not only differ in statistical properties (27), but also in the functional relationships between historical and future values [60] 56).. However, this variability contradicts the assumptions of most existing models [4 69 9 which adopt the Uniform Distribution Modeling (UDM) strategy by treating all patches samples from single underlying distribution. This oversimplified view ignores structural heterogeneity and temporal variation across segments, thereby limiting the model's ability generalize and degrading its forecasting performance 43 25
CCoresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Narddn Diin
Gooduat
Piot Poremer Traneferm
Fast Foz for Transform
(a) Sudden drift
(b) Gradual drift
Figure| 1: Illustration of distribution shifts between time series patches on the ETTh1 dataset, quantified by Wasserstein distance. The combined time- and frequency-domat views reveal richer and more complementary shift patterns arising from temporal non-stationarity.
To quantify these distributional shifts, we split the ETThl dataset into patches and analyze two representative cases: sudden drift and gradual drift, in both time and frequency domains. Specifically, we compute the Wasserstein distance between patches and visualize the results as heatmaps in Figure| [ which clearly illustrate the discrepancies across segments. Notably, sudden drift (Figure I (a)) leads to a sharp discrepancy between patches 9 and 10 and the remaining segments, while gradual drift (Figure (b)) reveals that patches 0 to 5 differ from patches 6 to 11, exhibiting progressive shift that makes forecasting more challenging. Furthermore, the frequency domain offers a complementary perspective on the shifts [42]. These observations highlight the complex and evolving nature of time series data, where different segments may follow distinct distributions and exhibit heterogeneous
temporal patterns 55
56 52
To address the challenges posed by distribution shifts in time series data, we propose a novel Time-Frequee Frequency Pattem-Speciil (TFPS) architecture to effectively model the complex temporal patterns. In particular, TFPS consists of the following three key components. The first is a Dual- Domain Encoder (DDE), which extracts features from both time and frequency domains provide comprehensive representation of the time series data, enabling the model to capture both short-term and long-term dependencies. Second, TFPS addresses the issue of concept drift by incorporating Pattern Identifier (PI), that utilizes a subspace clustering approach to dynamically identify the distinct patterns across patches. This enables the model to effectively handle nonlinear cluster boundaries and accurately assign patches to their corresponding clusters. Finally, TFPS constructs a Mixture of Pattern Experts (MoPE)-a set of specialized expert models, each tailored to a specific pattern identified by the PI. By dynamically assigning patches to the appropriate experts, TFPS learns pattern- pattern-specific predictive functions that effectively capture heterogeneous temporal dynamics and distributional variations. This specialized modeling strategy enhances the model's 's adaptability and yields significant forecasting improvements, particularly on datasets with severe distributional drift.
In summary, the key contributions of this work are:
We introduce a novel pattern-peccfii forecasting paradigm that enables segment-wise expert modeling based on latent pattern structure, overcoming the limitations of uniform modeling under distribution shift.
We propose TFPS, a dual-domain framework that integrates time- and sosquunncy-domalle representations with subspace clustering and dynamic expert routing, enabling the model to explicitly adapt to concept drift and capture evolving patterns in non-stationary time series.
We evaluate our approach nine real-world multivariate time series datasets, demonstrating its effectiveness. Our model achieves top-1 performance in 57 out of 72 settings, showcasing its competitive edge in improving forecasting accuracy.

---

2 Related Work
Time Series Forecasting Models. In recent years, deep models with claborately designed archi- tectures have achieved great progress in time series forecasting [51 67
Approaches like TimesNet [6]] and ModernTCN [4] utilize convolutional neural networks with time-series-spechie
28 45
modifications, making them better suited for forecasting tasks. Additionally, simpler architectures such Multi-Layer Perceptron (MLP)-based models [69 g have demonstrated competitive perfor-
mance. However, transformer-base models have gained particular prominence due to their ability model long-term dependencies in time series [70 62] 29 Notably, PatchTST 4 has become
a widely adopted Transformer variant, introducing a channel-independent patching mechanism to enhance temporal representations. This approach has been further extended by subsequent models [29 [].
While previous work has primarily focused capturing nonlinear dependencies in time series through enhanced model structures, our approach addresses the distribution shifts caused by evolving patterns within the data, which is key limitation of existing methods.
Non-stationary Time Series Forecasting. Non-stationariity time series data complicate predictive modeling, necessitating effective solutions to handle shifting distributions [6 11. To address varying distributions, normalization techniques have emerged as focal point in recent research, aiming mitigate non-stationary elements and align data with consistent distribution.
For instance, adaptive norm 46 applies z-score normalization using global statistics and DAIN 47 introduces a neural layer for adaptively normalizing each input instance. Reversible instance normal- ization (RevIN) [20 is proposed to alleviate series shift. Furthermore, Non-stationary transformer 31 points that directly stationariziing time series will damage the model's capability to capture spe- cific temporal dependencies and introduces an innovative de- stationary attention mechanism within self-attention frameworks. Recent advancement include Dish-TS [0] which identifies both intra- and inter-space distribution shifts time series data, and SAN (34 which applies normalization at the slice level, thus opening new avenues for handling non-stationary time series data. Lastly, SIN [7 introduces a novel method to selecting the statistics and learning normalization transformations to capture local invariance in time series data.
However, normalization methods can only address changes in statistical properties, and over-reliance on them may lead to over- staiioaaiizatinnn, where meaningful temporal variations are inadvertently smoothed out (34]. In contrast, our approach preserves the intrinsic non -taatittoonatty of the original series in the latent representation space, enabling the model to better adapt to evolving regimes by tailoring experts to diverse temporal patterns and distributional structures.
3 Method
3.1
Preliminaries
Time series forecasting aims uncover relationships between historical time series data and future data. Let X denote the time series, and x, represent the value at timestep t. Given the historical time series data X = [xt-L+1."" 2 E LxC. where L is the length of the look-back window and > 1 is the number of features in each timestep, the objective is to predict the future series Y [ 244.1 m1] "xx, where H is the forecast horizon.
3.2 Overall Architecture
Our model introduces three novel components: the Dual-Domain Encoder (DDE). the Pattern Identifier (PI), and the Mixture of Pattern Experts (MoPE), as illustrated in Figure The DDE goes beyond traditional time-domain encoding by incorporating frequency encoder that applies Fourier analysis, transforming time series data into the frequency domain. This enables the model to capture periodic patterns and frequency-sper sppecffcc features, providing a more comprehensive understanding of the data. The PI is clustering based module that distinguishes patches with distinct patterns, effectively addressing the variability in the data. MoPE then utilizes multiple MLP-based experts, each dedicated to modeling specific pattern, thereby enhancing the model's 's ability to adapt to the temporal dynamics of time series. Collectively, these components form a cohesive framework that effectively handles concept drift between patches, leading to more accurate time series forecasting.
3

---

Paek Domnila Eaceder
Exports
Lick Roal
Papeet
Tank
Linoa Driustiie Prides
Eacoler
A
Freqeceyy Paticaa
A
lecalller
Time Eneoder
Xye E
Misstree of Pattern Exports
Weighted
Crasat
(r) Time-Friqcccy Pattern-Spreific Architecture
(c) Frequency Enceder
Figure 2: The structure of our proposed TFPS. The input time series is divided into patches, and positional embeddings are added. These embeddings are processed through two branches: time- domain branch and frequency-domain branch. Each branch consists of three key components: (1) an encoder to capture patch-wise features, (2) a clustering mechanism to identify patches with similar patterns, and (3) mixture of pattern experts block to model the patterns of cach cluster. Finally, the outputs from both branches are combined for the final prediction.
3.3 Embedding Layer
Firstly, the input sequence X E R4XC is divided into patches of length P, resulting in N 5
(4-11 + 2] tokens, where S denotes the stride, defining the non-overlapping region between consecutive patches. Each patch is denoted P; E CCP. These patches are then projected into new dimension D, via linear transformation, such that, P: - P/ E RCD
Next, positional embeddings added to each patch preserve the temporal ordering disrupted dur- ing the segmentation process. The position embedding for the i-th patch, denoted as E, is vector of the same dimension the projected patch. The enhanced patch computed by summing the original patch and its positional embedding; XPE. p + E;, and XPE = {XPE) X Notably, the positional embeddings are learnable parameters, which enables the model capture the temporal dependencies in the time series more effectively. As result, the final enriched patch representations are Xpg CxNND
X ppxx }.
3.4 Dual-Domain Encoder
As shown in Figure I both time and frequency domains reveal distinct concept drifts that can significantly affect the performance of forecasting models. To effectively address these drifts, propose a moal-Domain Encoder (DDE) architecture that captures both temporal and frequency dependencies inherent in time series data.
utilize the patch-based Transformer 44 as an encoder to extract embeddings for each patch, capturing the global trend feature. The multi-head attention is employed to obtain the attention output O, E R NxD:
QKT
O, Attention K, V) Softmax
V.
Vd
(1)
Q XXPEWo K= X Wk, V= XPV
The encoder block also incorporates BatchNorm layers and feed-forward network with residual connections, as shown in Figure 2 (b). This process generates the temporal features E CxNDD
In parallel with the time encoder, we incorporate Frequency Encoder by replacing the self-attention sublayer of the Transformer with a Fourier sublayer (26]. This sublayer applies a 2D Fast Fourier Transform (the number of patches, hidden dimension) to the patch representation, expressed
O, = F Pattch (Fi (Xx)))

---

Pattern Identifier
Mixture of Pattern Experts
(Subsqace bases)
(
Nyumiii fired
Patch wise Exper
Patch wise Expert
X
X2
X
Futch-wise Expert
E LER (Cooo
It E ReAKO
Zy
subspacc affinity
Putch-wise Expert
4, 6
Subspace affinity vector
4: D(D
Linerr
ReLU
Linear
Figure 3: Illustration of the proposed Pattern Identifier and Mixture of Pattern Experts. The embedded representation z from DDE combines with subspace D to construct the subspace affinity vector, which yields the normalized subspace affinity S. Subsequently, the refined subspace affinity $ computed from S to provide self-supervised information. Then, we assign the corresponding patch-wise experts the embedded representation according S for modeling.
We only keep the real part of the result, and hence, we do not modify the Transformer. The structure of the Frequency Encoder is depicted Figure 2 (c), yielding frequency features 2y E Cx
layers in the
By modeling data in both the time and frequency domains, the DDE provides understanding of temporal patterns, enabling the model to effectively handle complexities such as concept drift and evolving dynamics. This dual-domain perspective enhances the model's robustness and predictive accuracy, offering a versatile foundation for real-world time series forecasting.
comprehensive
3.5 Pattern Identifier
To address the complex and evolving patterns in time series data, we introduce a novel Pattern Identifier (PI) module, an essential innovation within our framework. Unlike traditional approaches that treat the entire time series uniformly, our PI module dynamically classifies patches based on their distributional characteristics, enabling a more precise and adaptive modeling strategy.
The core of our approach lies in leveraging subspace clustering to detect concept shifts across multiple subspaces, as illustrated in Figure 3 The PI module plays central role by directly analyzing the intrinsic properties of each patch and clustering them into distinct groups based their latent patterns. In the time domain, PI enables TFPS to identify shifts in temporal characteristics such as seasonality and trends. In the frequency domain, captures shifts associated with frequency-specífic structures, like periodic behaviors and spectral changes, providing a comprehensive perspective evolving patterns throughout the series.
To provide clarity, Figure 3 showcases an application of the PI module exclusively within the time domain. However, the insights and methodology seamlessly extend the frequency domain, presenting a unified solution to the challenge of concept shifts.
The PI module iteratively refines subspace bases, which in turn improve representation learning and enable more accurate modeling of evolving patterns. It operates through the following three steps.
Construction of Subspace Bases. We define
variable D (1) (2)
(2)....
D (() to
represent the bases of K subspaces, where D consists of K blocks, each D E Rexd
D
1, u 1....ddj
,...KKK d.j = 1,..., .,, K. To control the column sizes of D, we impose the following constraint:
R1 = D
(3)
where 2 denotes the Hadamard product, and is identity matrix of size Kd Kd.
5

---

Subspaces Differentiation. To ensure the dissimilarity between different subspaces, we introduce the second constraint:
1 2
R₂ D
j
F²
(4)
1
OlI
where O is a matrix with off-diagonal d-size blocks set to and diagonal blocks set to Combining R₁ and R2 yields the regularization term for R:
R = a R1 + R2),
(5)
where is tuning parameters, fixed at 10-3 in this work.
Subspace Affinity and Refinement. We propose novel subspace affinity measure to assess the relationship between the embedded representation z from DDE and the subspace bases D. The affinity Stj, representing the probability that the embedded z; belongs to the j-th subspace, is defined
2 (()
+ nd
Sij E|( D nd)
(6)
where 1) parameter controlling the smoothness, fixed the same value d. To emphasize more confident assignments, we introduce a refined subspace affinity 8ty:
Σ 8ij
81g E, E3 sig)
This refinement sharpens the clustering by weighting high-confidence assignments more. The subspace clustering objective based on the Kullback-Leible divergence is:
$1
Leub (L(SS
$ soylog ΣΣ 3
(8)
The clustering loss defined as:
Lp = R + BLaub,
(9)
where 3 is hyperparameter balancing the regularization and subspace clustering terms. detailed sensitivity analysis of and is presented in Appendix[]
3.6 Mixture of Pattern Experts
Traditional time series forecasting methods often rely on uniform distribution modeling (UDM) approach, which struggles to adapt to the complexities of diverse and evolving patterns in real- world data. To address this limitation, we introduce the Mixture of Pattern Experts module (MoPE), which assigns specialized experts patches based their unique underlying patterns, enabling more precise and adaptive forecasting.
Given the cluster assignments obtained from the PI module, we apply the Patch-wise MoPE to the feature tensor z xxxNND The MoPE module consists of the following key components:
Gating Network. The gating network G calculates the gating weights for each expert based on the cluster assignment s and selects the top k experts. The gating weights are computed as:
G(s) Softmax(TopK((
(10)
Here, the top & logits selected and normalized using the Softmax function to produce the gating weights.
Expert Networks. The MoPE contains K expert networks, denoted as E Ex. Each expert network modeled as an MLP consisting of two linear layers and a ReLU activation. Given patch-wise feature z, each expert network Es processes the input to generate its respective output.

---

Output Aggregation. The final output of the MoPE module is a weighted sum of the outputs from all the selected experts, with the weights provided by the gating network:
K
Σ G())Ex(2).
(11)
After the frequency branch is processed by the inverse Fast Fourier transform, the time-frequency outputs he and hy, are concatenated to form h concat(he, hyy RCXNX22
Finally, linear transformation is applied the disentangled and patter-speccffe representations h generate the prediction: Y Li)ear(h) #x
This approach ensures that the MoPE dynamically assigns and aggregates contributions from various experts based on evolving patterns, improving the model's adaptability and accuracy.
3.7 Loss Function
Following
we use the Mean Squared Error (MSE) loss to quantify the discrepancy between predicted values Y and ground truth values Y: LMSE = (Y Y)². In addition to the MSE loss, we incorporate the clustering regularization loss from the PI module, yielding the final loss function:
44
= LMSE + Lp1, + LPI,
(12)
This combined loss ensures that the model not only minimizes forecasting emrors but also accurately identifies and maintains the integrity of pattern clusters across time. The algorithm is provided in the Appendix
4
Experiments
4.1 Experimental Setup
Datasets and Baselines. We conducted our experiments nine publicly available real-world multivariate time series datasets, i.., ETT (ETTh1, ETTh2, ETTmI, ETTm2), Exchange, Weather, Electricity, Traffic, and ILI. These datasets provided in [62 for time series forecasting, More details about these datasets are included in Appendix(A) A
We employed diverse set of wate-oofthee art forecasting models as baselines, categorized based on the type of information they utilize as follows. (1) Time-domain methods: PatchTST 44 DLinear
(69] TimesNet [61] and i'Transfrmmer [29]: (2) Frequency-domale methods: FEDformer
[7] and
FITS [64]; (3) Time-frequency methods: TFDNet-IK [42) and TSLANet
9
We rerun the
experiments with codes provided by their official implementation.
In addition, compare TFPS with recent foundation models, including AutoTimes (30, Moment [16], and Timer (32. We rerun all experiments for fair comparison: Tuto is reproduced using its official implementation, while Moment and Timer are evaluated based on the OpenLTM [32
We further include comparisons with normalization techniques, based architectures, and meth- ods designed to address distribution shifts. Comprehensive results are presented in Appendix G
Experiments Details. Following previous works 44 we used ADAM 21 as the default opti- mizer across all the experiments. We employed the MSE and mean absolute enror (MAE) as the evaluation metrics, where lower values indicate better performance. A detailed explanation pro- vided in A Appendix [E, TFPS was implemented by PyToorch [48) and trained on single NVIDIA RTX 3090 24GB GPU. We conducted grid search to optimize the following three parameters, i.c., learning rate {0.0001, 0. 0.0005, 0.001, 0.005, 0.01, 0.05). the number of experts in the time domain K, {1, 2, 4, 8), and the number of experts in the frequency domain K₁ = 1,224488.
4.2 Overall Performance Comparison
Table D highlights the consistent superiority of TFPS across multiple datasets and prediction horizons, securing the top performance in 57 out of 72 experimental configurations. In particular, TFPS demonstrates significant improvements over time-domain methods, with an overall improvement of
7

---

Table 1: Multivariate long-term forecasting results with prediction lengths H {24, 36, 48, 60) for ILI and H E {96, 192, 336, 720) for others. The input lengths are L = 104 for ILI and L = 96 for others. The best results are highlighted in bold and the second best are underlined.
TFRS
FITS
iTransformner (2024)
TRDNNe-IK
PaccTST
TimesNet (2023)
Dlinear (2023)
FEDioemer (2022)
Model
IMP.
(Our)
(2024)
(2004)
(2023)
(2023)
Meeni
MSE
MSE
MAB
MSE
MAE
MSB
MAB
MSE
MAB
MSB
MAE
MSE
MAB
MSE
MAE
MSB
MAE
MSE
MAE
ETT12 FTTM
192
96
-1.1%
4.8%
0,334
0.423
0.413
0.423
0.387
0.448
0.136
0.405
0.395
0.465
0.432
0,403
0.441
0.357
0.405
0.436
0.296
0.451
0.409
0.441
0.413
0.460
0.419
0.445
0.441
0.339
0.412
0.442
0.434
0.398
0.410
0.427
0.385
0.641
0.425
0.461
336
1.8%
0,484
0.461
0.491
0.187
0.489
0.663
0.491
0.46
0.485
0.462
0.497
0.463
0.491
0.467
0.499
0.477
0.69]
0.473
730
3.0%
0.488
0.476
0.505
0.495
0.509
0.486
0.508
0.503
0.501
-2.0%
0.313
0.355
0.290
0.501
0.315
0.374
0.342
0.353
192
-2.9%
0.405
0410
0.391
0.362
0.306
0.350
0.3
0.370
0.395
0.383
0.308
0.501
0.410
0.422
0:447
0.434
0.440
836
10.5%
0.392
0.415
0.401
0.49
0.416
0.425
0.424
0.422
0.416
0.423
0.424
0.431
11420
0.477
0.486
0.481
0.512
0.497
730
12.6%
0.410
0.433
0.419
0.439
0.418
07447
0,430
0.447
0.424
0.441
0.429
0,445
01433
0.448
0.732
0.614
0.467
0.476
ET1a2 ETTial
96
4.1%
0.327
0.367
0.329
0.30
0.368
0.354
0.575
0.342
0.577
0.331
0.369
0.351
0,570
0.537
0.377
0.346
0.374
0.350
0.406
192
2.0%
0.374
0.395
0.376
0.558
0.383
0.39
0.374
0.395
0.382
0.3M2
0,395
336
4.2%
0.401
0.408
0,403
0.414
0.425
0.415
0.418
0,418
0,405
0.410
0.412
0.412
0,433
0,422
0.414
0.414
0.448
0,458
720
.0.7%
(1477)
0.456
0.445
0.438
0.486
03.49
0.457
0.457
0.471
0.437
0.466
0.446
01444
0.488
0.478
0.455
0.491
0.479
96
6.9%
0.170
0.255
0.179
0.261
0.185
0.266
0.186
0.272
0.176
0.267
0.177
0.260
11182
0.262
0.184
0.226
0.193
0.285
192
7.1%
0.335
0.29%
1.243
0.315
0.247
0.305
1.225
1311
11345
0.302
0.228
0.506
13252
03007
0.282
0.357
0.256
0.524
336
4.0%
0.297
0.336
0,308
0.342
0.336
03003
0.340
0.313
0.541
0.364
0.321
0,364
720
3.6%
0.401
0.397
0.403
0:407
0.401
0.414
0.407
0,405
0.399
0.405
0.400
0.417
0.404
0:441
0:454
0.434
0.426
Exchange
96
12.7%
0,083
0.206
0,085
0.206
0.088
00210
0.086
0.206
0084
0.205
0.089
0.206
0,108
0223
0.089
0.219
0.136
0,255
192
11.2%
0.174
0:297
0,178
0.300
0,181
0.304
0.181
0,20
0.176
0.
0.999
0.178
0.302
0.219
0.342
0.180
0.319
0.279
0,381
336
10.4%
0.300
0:396
0.329
0.415
0.324
0:413
0.358
0.422
0.321
0:409
0.326
0:411
0.353
0.433
0.113
0.423
0.465
0.504
730
-1.39%
1.011
0.756
0.850
0 0693
0.846
0.696
0.853
0.606
0.335
0,689
0.840
0,600
0.912
0.734
0.537
0.690
1.189
0.876
Weathe
96
15.6%
0.154
0.302
0.176
0.216
0.167
0.214
0.176
0.316
0.
0 0.200
0.177
0.219
0.168
0.218
0 197
0.257
0.236
0.325
192
10.6%
0.305
0.349
258
0.225
0.7
0.225
336
9.1%
0.362
0.259
0.279
0.299
0,270
0.299
0:281
0.367
0.298
0.278
0.29%
0.283
0. 0.305
0.228
0.382
0,366
0,402
720
4.1%
0.344
0.342
0.355
0.355
0.347
0.545
0.358
0.350
0.147
0.346
0.351
0.346
0.355
0.353
0.347
0.382
0,407
0.422
Traflce Elecricity
96
14.6%
0.149
0.236
0.155
0.249
0,200
0.278
0.151
0.241
0.171
0.254
0.166
0.252
0.168
0.272
0.195
0.277
0.189
0.304
192
12.0%
0.162
0.255
0.170
0. 162
0.200
0.381
16
1558
0.189
0.760
0.174
0.261
0.186
0.289
0 0.104
0.281
0.198
0.312
356
0.2%
0.200
0.310
282
0.295
0.15
1.251
0.784
0.190
0.277
0.208
0.207
0.212
720
7.2%
0.220
0.320
0.224
0.318
0.256
005228
0.229
0.319
0.347
0.318
0.230
0.312
0.225
0.243
0.330
0.242
0.351
96
21.1%
0.427
0.296
0.475
0.367
0.651
0.588
0.42
0.295
0.519
0.314
0.446
0.254
0.650
0.397
0.575
0.357
192
37.7%
0,445
0.29%
0.478
0.306
0.603
0,364
0.448
11202
0.513
0.314
0.453
0.285
11618
0.325
0.600
0.372
0.613
0.381
336
17.0%
0.459
0.307
0.491
0.312
0.610
0,366
0.465
0.311
0.525
0.319
0.467
0.291
0.634
0.337
0.606
0.374
0.622
0.380
720
15.1%
0.496
0.313
0.528
0.331
0.648
0.387
0.501
0.333
0.561
0.336
0.501
0.492
0.659
0.349
0.616
0.395
0,630
0.383
24
40.9%
1.349
0.760
1,749
0.898
3.489
1.573
2.443
1.078
1.824
0.824
1.614
0.835
1.699
0.871
2:239
1.041
3.217
1.246
43.6%
1,239
0.752
3,530
1,370
2,455
1.475
0.859
2:238
1,089
2,688
48
40.4%
1,461
0
2.050
0.984
3,671
1.391
3.437
1.311
1.762
0.81
644
0.880
2,272
0.009
2:252
1.064
2.540
1.057
60
39.8%
1.458
0.836
2.240
1.039
4.050
1,482
2.734
1.155
1.758
0.863
16088
0.585
1.998
0.934
2:236
1,057
2.782
1.126
1" Count
37
3
0
Table 2: Ablation study of TFPS components. The model variants in our ablation study include the following configurations across both time and frequency branches: (a) inclusion of the encoder, PT and MoPE; (b) PI replaced with Linear; (c) only the encoder. The best results are in bold.
Time Branch
Frequency Branch
E
ETTh2
Encoder
Pl
MoPE
Encoder
Pl
MoPE
96
192
336
720
96
192
336
720
/
/
0.398
0.423
0.484
0.488
0,313
0.405
0.392
0.410
/
0.401
0.459
0.486
0.492
0.318
0.409
0.400
0.428
Lineur
0,401
0.451
0.494
0.509
0.325
0.411
0.400
0.434
0.414
0.460
0.501
0.500
0.339
0.411
0.426
0.431
0.455
0.507
0.539
0.576
0.324
0.407
0.417
0.436
Linear
0.503
0.535
0.558
0.583
0.398
0.446
0.457
0.444
0,552
0,583
0.591
0.594
0.371
0.426
0.418
0.463
9.5% in MSE and 6.4% in MAE. Compared to frequency-domain methods, TFPS shows even more pronounced enhancements, with MSE improved by 16.9% and MAE by 12.4%.
While the time-frrene frequeer methods like TSLANet and TFDNet perform competitively on several datasets, TFPS still outperforms them, showing improvement of 5.2% in MSE and 2.2% in MAE. These substantial improvements can be attributed to the integration of both time- and frequeney- domain information, combined with our innovative approach to modeling distinct patterns with specialized experts. By addressing the underlying concept shifts and capturing complex, evolving patterns in time series data, TFPS achieves more accurate predictions than other basclines.
8

---

Table 3: Compared with foundation models,
Table 4: Compared with normalization methods.
Model
IXP
TFPS
(Ouz)
Aus-Tinas (2024)
Monazi
(2024)
(2004)
Timas
Model
IMP.
TTPS
DLinear
MSE
MAE
MAE
MAE
SIN
SAN
Dish-T8
RevIN
ET
1.5%
0.448
0.454
0.456
0.461
0.451
0.305
0.386
03565
0.381
ET
2.4%
0.380
0.386
0.388
0.392
0.390
2fTm2
1.3%
0.343
0.345
0.374
0.305
0.364
0.273
0
0.337
0.318
12855
0.325
0.386
0.364
0.264
0.378
0.331
ETTml
2.3%
0.395
0.405
0.399
0.406
0.409
Trali:
3366
0.398
0.263
0.379
0.265
0.395
0.253
0.329
0,255
3.3%
Eoommry
3.74
0.389
0.249
0.108
0.261
115
0.263
0.165
0.258
Weather
5.3%
0.241
0.253
0.249
0.263
0.254
4.3 Ablation Study
Table 2 presents the MSE results of TFPS and its variants with different combinations of encoders, PI, and MoPE. 1) Best Result. The full TFPS model, i.., both the time and frequency branches, along with their respective encoders, PI, and MoPE are included, performs the best across all the forecast horizons for both datasets. 2) Linear vs. PI. We replace PI with a linear layer and find that generally results in higher MSE in most cases, indicating that accurately capturing specific patterss is crucial. 3) Impact of Pattern-aware Modeling. Additionally, when comparing the results with the encoder-only configuration, two variants with MoPE in each branch achieved improved MSE, further supporting the necessity of patter-aware modeling. 4) Importance of DDE. Furthermore, we find that both the time encoder and frequency encoder alone yield worse performance, with the time encoder playing a more significant role. In summary, incorporating both branches with PI and MoPE provides the best performance, while simpler configurations result in higher MSE. See Appendix K for an in-depth analysis of each component's contribution.
4.4 Comparsion with Foundation Models
To ensure fair comparison with foundation models, searched input lengths among 96, 192, 336, and 512. The average results across all forecasting lengths are included in Table with detailed results provided in Appendix F.2 As shown in Table 3 TFPS consistently outperforms recent foundation models. Notably, on challenging datasets such as ETTh2 and ETTm2, TFPS achieves substantial improvements MSE by 10.9% and 7.3%, respectively. Although TFPS performs slightly worse on the Traffic dataset, we attribute this to the relatively mild distribution shift observed in Traffic (see T)ble(5). which may reduce the benefit of our pattern-speciff modeling. These results suggest that TFPS not only matches but often surpasses harge-scale foundation models in forecasting accuracy, benefiting from its expert-based design that explicitly captures distributional heterogeneity.
4.5 Comparsion with Normalization Methods
Normalization methods can reduce fluctuations to enhance performance and are widely used for non-stationary time series forecasting [17 34 10
31 20). We compare our TFPS with these state-of- the-art normalization methods and Table presents the average MSE across all forecasting lengths for each dataset. While normalization improves stability by enforeing distributional consistency, TFPS retains the intrinsic non-stationarity and models diverse patterns through distribution-specific experts. achieving better adaptability and forecasting accuracy. Detailed results are provided in Appendix G.T
4.6 Visualization
We visualize the prediction curves for ETThl with H = 192. Given that DLinear exhibits competitive performance in Table I. we compare its results with those of TFPS Figure under two scenarios: (a) sudden drift caused by external factors random events, and (b) gradual drift where the trend is dominant. evident that DLinear struggles to achieve accurate predictions in both scenarios. In contrast, our TFPS consistently produces accurate forecasts despite these challenges, demonstrating its robustness in dealing with various concept dynamics.
4.7 Analysis of Experts
Qualitative Visualizations of Pattern Identifier. Through training, pattern experts in MoPE spontaneously specialize, and we present two examples in Figure We visualize the expert with the highest score the routed expert for each instance pair. In the provided examples, we observe that

---

Sudden Dritt
-1.0
-1.0
-12
Pretectio
Gradiul Dešft
Diiveer
ded Thunh
Ground
-1.7
300
Tustume
Future Time Stase
(a) Sudden Drift
(b) Gradual Drift
Figure 4: Visualizations of DLinear and TFPS on the ETThl dataset when H = 192.
4314
6,884
W
0.18
0,55
AW/A
Cica
(a) Expert
(b) Expert
(a) ETThl
(b) Weather
Figure 5: Interpretable patterns via PI. Expert-O specializes in downward trends, while Expert-4 focuses parabolic trends.
Figure 6: Experiments the number of experts when H = 96. Further analysis of expert be- havior is provided in Appendix H
expert-O specialize in downward- related concepts, while expert-4 focuses on parabolic trend. These examples also demonstrate the interpretability of MoPE.
Number of Experts. In Figure 6 we the learning rate 0.0001 and conducted four sets of experiments on the ETTh1 and Weather datasets, K2 = 1, K = (1,2,4,8, 8). to explore the effect of the number of frequency experts the results. For example, K, K,1K44 means that the TFPS contains I time experts and 4 frequency experts. We observed that K₂ 1K2 outperformed K₁ KK in both cases, suggesting that increasing the number of experts does not always lead better performance.
In addition, we conducted three experiments based on the optimal number of frequency experts to verify the impact of varying the number of time experts the results. As shown in Figure 6 the best results for ETTh1 were obtained with K,4K 22, while for Weather, the optimal results were achieved with K,4K, y8. Combined with the average Wasserstein distance in Table attribute this to the fact that, cases where concept drift is more severe, such as Weather, more experts are needed, whereas fewer experts sufficient when the drift is less severe.
Conclusion
In this paper, we propose novel pattern-aware time series forecasting framework, TFPS, which incorporates dual-domain mixture of pattern experts approach. Our TFPS framework aims to address the distribution shift across time series patches and effectively assigns pattern- pattern-specffic experts model them. Experimental results across eight diverse datasets demonstrate that TFPS surpasses state-of-the-art methods in both quantitative metrics and visualizations. Future work will focus on investigating evolving distribution shifts, particularly those introduced by the emergence of new patterns, such as unforeseen epidemics or outbreaks.
Acknowledgments and Disclosure of Funding
This work was supported in part by the National Natural Science Foundation of China under Grants U23B2049, 62376194, 61925602, 62406219, and 62436001; in part by the China Postdoctoral Science Foundation Tianjin Joint Support Program under Grant 2023T014TJ; and in part by the China Scholarship Council under Grant 202406250137.
10

---

References
[1] Khaled Alkilane, Yihang He, and Der-Horng Lee. Mixmamba: Time series modeling with adaptive expertisc. Information Fusion, 112:102589, 2024.
[2] Quentin Gregory Anthony, Yury Tokpanov, Paolo Glorioso, and Beren Millidge. Blackmamba: Mixture of experts for state-space models. ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024.
[3] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with neural networks. Nature, 619(7970):533-533 2023.
[4] Yushu Chen, Shengzhuo Liu, Jinzhe Yang, Hao Jing, Wenlai Zhao, and Guangwen Yang. A joint fime-frequency domain transformer for multivariate time series forecasting. Neural Networks, 176: 10634,, 2024.
[5] Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y Wu, et al. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint
2024.
[6] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, In Proceedings of the 2019 conference of the North American chapter of the association for computational linguisties: human language technologies, volume I (long and short papers), pages 4171-4188 2019.
[7] Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. Glam: Efficient scaling of language models with mixture-of mixture-of-experts, In International Conference on Machine Learning, pages 5547-5569, PMLR, 2022.
[8] Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 459 469, 2023.
[9] Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, and Xiaoli Li. TSLANet: Rethinking transformers for time series representation learning. In Forty-first International Conference Machine Learning. 2024.
[10] Wei Fan, Pengyang Wang, Dongkun Wang. Dongjie Wang, Yuanchun Zhou, and Yanjie Fu. Dish-ts: a general paradigm for alleviating distribution shift in time series forecasting. In Proceedings of the AAAI conference artificial intelligence, volume 37, pages 7522-7529, 2023.
[11] Wei Fan, Shun Zheng, Pengyang Wang, Rui Xie, Jiang Bian, and Yanjie Fu. Addressing distribution shift in time series forecasting with instance normalization flows. arXiv preprint arXiv:2401.1671 16777, 2024.
[12] William Fedus, Barret Zoph. and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1-39, 2022.
[13] Yunpeng Gong, Liqing Huang, and Lifei Chen. Eliminate deviation with deviation for data augmentation and a general multi-modal data learning method. arXiv preprint arXiv:2101.082 008533, 2021.
[14] Yunpeng Gong, Liqing Huang, and Lifei Chen. Person re-identification method based color attack and joint defence. Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 4313 4313-433 4322, 2022.
[15] Yunpeng Gong, Zhun Zhong, Yansong Qu, Zhiming Luo, Rongrong Ji, and Min Jiang. Cross- modality perturbation synergy attack for person re-identification. Advances in Neural Informa- tion Processing Systems, 37:23352-23377 2024.

---

[16] Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: A family of open time-series foundation models. In International Conference on Machine Learning, pages 16115-16152. PMLR, 2024.
[17] Lu Han, Han-Jit Ye, and De-Chuan Zhan. SIN: Selective and interpretable normalization for long-term time series forecasting. In Forty-first International Conference on Machine Learning. 2024.
[18] Hongbin Huang, Minghua Chen, and Xiao Qiao. Generative learning for financial time series with irregular and scale-invariant patterns. In The Twelfih International Conference on Learning Representations, 2024.
[19] Yushan Jiang, Kanghui Ning. Zijie Pan, Xuyang Shen, Jingchao Ni, Wenchao Yu, Anderson Schneider, Haifeng Chen, Yuriy Nevmyvaka, and Dongjin Song. Multi-modal time series analysis: tutorial and survey. In Proceedings of the 3/st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages
2025.
[20] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
[21] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69880 2014.
[22] Weiyang Kong, Ziyu Guo, and Yubao Liu. Spatio-temporal pivotal graph neural networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8627-8635, 2024.
[23] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 4/st international ACM SIGIR conference on research development in information retrieval, pages 95-104, 2018
[24] Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416-4422) 2023.
[25] Sanghyun Lee and Chanyoung Park. Continual traffic forecasting via mixture of experts. arXiv preprint arXiv:2406.033 0310000314,, 2024.
[26] James Lee- Thorp, Joshua Ainslie, Ilya Eckstein, and Santiago Ontanon. Fnet: Mixing tokens with fourier transforms. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pages 4313, 2022.
[27] Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint drXiv:2305.1072/ 2023.
[28] Xvyuan Liu, Xiangfei Qiu, Xingjian Wu, Zhengyu Li, Chenjuan Guo, Jilin Hu, and Bin Yang. Rethinking irregular time series forecasting: A simple yet effective baseline. arXiv preprint a5Xiv:2505. 1250, 2025.
[29] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
[30] Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Autotimes: Au- toregressive time series forecasters via large language models. Advances in Neural Information 2024.
Processing Systems, 37:122154-1222
[31] Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity time series forecasting. Advances Neural Information Processing Systems, 35:9881-9988 2022.
12

---

[32] Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer. generative pre- pre-trained transformers are large time series models. In Proceedings of the 4 Ist International Conference Machine Learning, pages 52369-323999 2024.
[33] Zhanyu Liu, Guanjie Zheng, and Yanwei Yu. Cross-city few-shot traffic forecasting via traffic pattern bank. In Pruceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 1451-1460, 2023.
[34] Zhiding Liu, Mingyue Cheng, Zhi Li, Zhenya Huang, Qi Liu, Yanhu Xie, and Enhong Chen. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. Advances in Neural Information Processing Systems, 36, 2023.
[35] Qingqing Long, Zheng Fang, Chen Fang, Chong Chen, Pengfei Wang. and Yuanchun Zhou. Un- veiling delay effects in traffic forecasting: A perspective from spatial-temporal delay differential equations. In Proceedings of the ACM on Web Conference 2024, pages 1035-1044, 2024.
[36] Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Joao Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE transactions on knowledge and data engineering, 31(12):2346- 2363, 2018.
[37] Shilin Lu, Zhuming Lian, Zihan Zhou, Shaocong Zhang, Chen Zhao, and Adams Wai-Kin Kin Kong. Does flux already know how to perform physically plausible image composition? arXiv preprint NXXi:25509.2278 2025.
[38] Shilin Lu, Yanzhu Liu, and Adams Wai-Kin Kong. Tf-icon: Diffusion- based training-free free cross-domain image composition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
[39] Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffusion models. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pages 6430-6440 2024.
[40] Shilin Lu, Zihan Zhou, Jiayou Lu, Yuanzhi Zhu, and Adams Wai-Kin Kong. Robust water- marking using generative priors against image editing: From benchmarking to advances. arXiv preprint vrXiv:2410.18777 2024.
[41] Donhaao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024.
[42] Yuxiao Luo, Ziyu Lyu, and Xingyu Huang. Tfdnet: Time-frqquency enhanced decomposed network for long-term time series forecasting, arXiv preprint arXiv:2308. 13386, 2023.
[43] Ronghao Ni, Zinan Lin, Shuaiqi Wang, and Giulia Fanti. Mixeure-of-iineaa--epett for long- term time series forecasting. In International Conference on Artificial Intelligence and Statistics, pages
PMLR, 2024.
[44] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023.
[45] Kanghui Ning, Zijie Pan. Yu Liu, Yushan Jiang, James Y Zhang, Kashif Rasul, Anderson Schneider, Lintao Ma, Yuriy Nevmyvaka, and Dongjin Song. Ts-rag: Retrieval-augmentt generation based time series foundation models are stronger zero-shot forecaster. arXiv preprint wwXiv:2503.07649 2025.
[46] Eduardo Ogasawara, Leonardo C Martinez, Daniel De Oliveira, Geraldo Zimbrão. Gisele Pappa, and Marta Mattoso. Adaptive normalization: A novel data normalization approach for non-stationary time series. In The 2010 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEF, 2010.
[47] Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, and Alexandros losifidis. Deep adaptive input normalization for time series forecasting. IEEE transactions on neural networks and learning systems, 3109):3700-375,, 2019.
13

---

[48] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga. et al. Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[49] Maciej Pióro, Kamil Ciebiera, Krystian Król, Jan Ludziejewski, Michal Krutul, Jakub Krajewski, Szymon Antoniak, Piotr Mitoš, Marek Cygan, and Sehastian Jaszezur. Moe-mamba: Efficient selective state space models with mixture of experts. In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024.
[50] Joan Puigcerver, Carlos Riquelme, Basil Mustafa, and Neil Houlsby. From sparse to soft mixtures of experts. arXiv preprint S.kiv::330..0995// 2023.
[51] Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow., 17(9))2363- 2377, 2024.
[52] Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1185-1196, 2025.
[53] Xiangfei Qiu, Yuhan Zhu, Zhengyu Li, Hanyin Cheng, Xingjian Wu, Chenjuan Guo, Bin Yang, and Jilin Hu. Dag: A dual causal network for time series forecasting with exogenous variables. arXiv preprint (tt.::2509.14933 2025.
[54] Aaditya Ramdas, Nicolás Garefa Trillos, and Marco Cuturi. On wasserstein two-sample testing and related families of nonparametric tests. Entropy, 19(2):47, 2017.
[55] Artsiom Sanakoyeu, Vadim Tschernezki, Uta Buchler, and Bjorn Ommer. Divide and conquer the embedding space for metric learning. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 471-480, 2019.
[56] Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao, et al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 2024.
[57] Noam Shazeer, * AAzalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely- gated mixture-of- experts layer. In International Conference on Learning Representations, 2017.
[58] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[59] Hao Wang, Zhiyu Wang, Yunlong Niu, Zhaoran Liu, Haozhe Li, Yilin Liao, Yuxin Huang, and Xinggao Liu. An accurate and interpretable framework for trustworthy process monitoring. IEEE Transactions on Artificial Intelligence, 2023.
[60] Qingsong Wen, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan, et al. Onenet: Enhancing time series forecasting models under concept drift by online ensembling. Advances in Neural Information Processing Systems, 1199999996990,,.0 2023.
[61] Haixu WU, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2023.
[62] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long, Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 44:22419-22433 2021.
14

---

[63] Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with unified deep model. Nature Machine Intelligence. 5(6):602-61) -110 2023.
[64] Zhijian Xu, Ailing Zeng, and Qiang Xu. FITS: Modeling time series with $10k$ parameters. In The Twelfth Interaational Conference on Learning Representations, 2024.
[65] Kaiwen Yan, Chen Long, Huisi Wu, and Zhenkun Wen. Multi-resolution expansion of analysis in time-frequeet freefrrruency domain for time series forecasting. IEEE Transactions Knowledge and Data Engineering, 2024.
[66] Zhangjing Yang, Weiwu Yan, Xiaolin Huang, and Lin Mei. Adaptive Cemporal-frequeeee network for time-series forecasting, IEEE Transactions on Knowledge and Data Engineering. 34(4):1576-1588) 1587, 2020.
[67] Chengqing Yu, Fei Wang, Zezhi Shao, Tangwen Qian, Zhao Zhang, Wei Wei, Zhulin An, Qi Wang, and Yongjun Xu. Ginar+: A robust end-to-end framework for multivariate time series forecasting with missing values. IEEE Transactions Knowledge and Data Engineering, 2025.
[68] Chengqing Yu, Fei Wang. Yilun Wang, Zezhi Shao, Tao Sun, Di Yao, and Yongjun Xu. Mgsf- former: A multi-granularinn spatiotemporal fusion transformer for air quality prediction. Infor- mation Fusion, 113:102607 2025.
[69] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[70] Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.
[71] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pages 27268-27286. PMLR, 2022.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims in the abstract and introduction accurately reflect the contribu- tions and scope of the paper. They are supported by the proposed methodology, theoretical motivation, and extensive empirical results across diverse datasets.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
The paper discusses the limitations of the proposed method in Section including its assumptions about fixed patch length and potential challenges under evolving distributions.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides sufficient details regarding the model architecture, training settings, data preprocessing, and evaluation metrics, enabling reproduction of the main experimental results and verification of the core claims.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
17

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
The source code is included in the supplementary materials facilitate reproducibility and further research.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips. htzjps.cc/ublic/guidde. htwpbl/icgggidse/ooeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Training and testing details described in Appendix A search ranges and sensitivity analysis are provided in Appendix !
The hyperparameter
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The paper provides sufficient details on the computational resources used, including the type of GPU, memory, and training time, in Section 4.1 This information allows readers to estimate the resources required for reproduction.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpp:://eeut cc http:///pppp.cttttccccccceeeeeee
Answer: [Yes]
Justification: We have read the NeurlPS Code of Ethics, and are certain that the paper conform to it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive and negative socictal impacts of the proposed method in Appendix M
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (c.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used in the paper, including datasets and code, are properly credited. Their licenses and terms of use are explicitly acknowledged and respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.papersstt
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The new assets introduced this paper, including the implementation code and model components, well documented. Documentation is provided alongside the code in the supplementary materials.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

A Dataset
We evaluate the performance of TFPS on eight widely used datasets, including four ETT datasets (ETThI, ETTh2, ETTml and ETTm2), Exchange, Weather, Electricity, and ILI. This subsection provides a summary of the datasets:
ETT 70 (Electricity Transformer Temperature) dataset contains two electric transformers, ETT and ETT2, collected from two separate counties. Each of them has two versions of sampling resolutions (15min & 1h). Thus, there are four ETT datasets: ETTm1, ETTm2, ETTh1, and ETTh2.
Exchange-Rate [33 the exchange-rate rate dataset contains the daily exchange rates of eight foreign countries including Australia, British, Canada, Switzerland, China, Japan, New Zealand, and Singapore ranging from 1990 to 2016.
Weather 62 dataset contains 21 meteorological indicators in Germany, such as humidity and air temperature.
Electricity 62 is dataset that describes 321 customers' hourly electricity consumption.
62 is dataset featuring hourly road occupancy rates from 862 sensors along the freeways the San Francisco Bay area.
Traffic
II.17 62 dataset collects the number of patients and influenza-like illness ratio in a weekly frequency.
For the data split, we follow [69] and split the data into training, validation, and testing by a ratio of 6:2:2 for the ETT datasets and 7:1:2 for the others. Details are shown in Table The best parameters are selected based on the lowest validation loss and then applied to the test set for performance evaluation. The data and codes are available: https://github. com/ htt/s//cucuuubFPSSSSS
Table 5: The statistics of the datasets.
Average Wasserstein (Time Domain)
Average Wasserstein' (Frequency Domain)
Datasets
Variates
Prediction Length
Timesteps
Granularity
ETThl
7
[96, 192, 336, 720)
17,420
hour
9.268
11.561
ETTH2
7
[96, 192, 336, 720)
17,420
1 hour
13.221
18.970
ETTm]
7
[96, 192, 336, 720)
69,680
15 min
9.336
10.660
ETTm2
[96. 192, 336, 720)
69.680
min
13,606
16.574
Exchange-Rad
(96, 336, 720)
7,588
day
0.132
0.144
Weather
21
(96, 192, 336, 720)
$2,696
10 min
39,742
77.422
Electricity
321
(96, 192, 336, 720)
26,304
1 hour
$20.162
1018.311
Traffic
862
[96, 192, 336, 720)
17,451
hour
0.011
0.028
[24, 36, 48, 60)
966
week
258881.714
381377.494
A large Wasserstein indicates a more severe drift.
B Related Work
Deep learning has achieved remarkable success across diverse domains such as computer vision 114
38 39 40 37). natural language processing [58 6), and multi-modality [13] 15), and has also
advanced the state of the art in time series modeling [56] 19
The Combination of Time and Frequency Domains. Time-domain models excel at capturing sequential trends, while frequency-domall models are essential for identifying periodic and oscillatory patterns. Recent research has increasingly emphasized integrating information from both domains to better interpret underlying patterns. For instance, ATFN [66] demonstrates the advantage of frequency domain methods for forecasting strongly periodic time series through frequfryyynncy adaptive
23

---

network. TFDNet 22 adopts branching structure to capture long- Long-term latent patterns and temporal periodicity from both domains. Similarly, JTFT 4 utilizes the frequency domain representation to extract multi-scale dependencies while enhancing local relationships modeling through time domain representation. TFMRN (65) expands data in both domains capture finer details that may not be evident in the original data. Recently, TSLANet [9] leverages Fourier analysis to enhance feature representation and capture both long-term and short-term interactions.
Building on these approaches, our proposed method, TFPS, introduces a novel Dual-Domate Encoder that effectively combines time and frequency domain information to capture both trend and periodic patterns. By integrating time-fre frequency features, TFPS significantly advances the field in addressing the complexities inherent in time series forecasting.
ceature-off-xperrr Mixture-of-Fine Experts (MoE) models have gained attention for their ability to scale efficiently by activating only subset of experts for each input, first introduced by (57). Despite their success, challenges such as training instability, expert redundancy, and limited expert specialization have been identified [50 55. These issues hinder the full potential of MoE models in real-world tasks.
Recent advances have integrated MoE with Transformers improve scalability and efficiency. For example, GLaM 77] and Switch Transformer [12] interleave MoE layers with Transformer blocks, reducing computational costs. Other models like state space models (SSMs) [49] 2], [1] combines
MoE with alternative architectures for enhanced scalability and inference speed.
In contrast, our approach introduces MoE into time series forecasting by assigning experts to specific time-frequee frequeccyy patterns, enabling more effective, patch-level adaptation. This approach represents a significant innovation in time series forecasting, offering a more targeted and effective way to handle varying patterns across both time and frequency domains.
C Wasserstein Distance
The Wasserstein distance, also called the Earth mover's distance the optimal transport distance, is similarity metric between two probability distributions. In the discrete case, the Wasserstein distance can be understood as the cost of an optimal transport plan to convert one distribution into the other. The cost is calculated as the product the amount of probability mass being moved and the distance it is being moved.
Given two miedimmnnio probability mass functions, and v, the first Wasserstein distance between them is defined as:
1, (u, v) = inf la yl (T( *EF(u.,) JRXR
(13)
where I'u, v) is the set of (probability) distributions IR whose marginals are u and on the first and second factors respectively. Here, u(z) represents the probability of u at position x, with the same interpretation for (Ir)..
In the special case of one-oineeet ine-dimensional distributions, the Wasserstein distance can be equivalently expressed using their cumulative distribution functions (CDFs), U and V, as:
1, [u, v)
U -V.
(14)
This equivalence is rigorously proved in 54
The input distributions can be empirical, therefore coming from samples whose values are effectively inputs of the function, or they can be seen as generalized functions, which case they are weighted sums of Dirac delta functions located at the specified values.
D Distribution Shifts in both Time and Frequency Domains
The time series X' is segmented into N patches, where each patch P., {Ix1 I.2..... ... Fr,P) consists of P consecutive timesteps for .... N.. For the frequency domain, we apply a Fourier transform FF to each patch Pn.. obtaining its frequency-domait representation as P., F(P.).
24

---

Each patch's probability distribution in the time domain is denoted as Pc(Pn). representing the statistical properties of Pn. while its frequency domain distribution, denoted p (Pr., captures its spectral characteristics.
The distribution shifts between two patches P: and Pj are characterized by the comparing their probability distributions both time and frequency domains. These shifts are defined as:
D. ( (P P,) (P))
0,
(15)
6,
(16)
where d is distance metric, such as Wasserstein distance Kullback-Leiin Leibler divergence, and 0 is threshold indicating significant distribution shift. If (P,,,, or (P P,)) exceeds 0, this implies a significant distribution shift between the two patches in either domain. is important to note that serves only conceptual threshold for defining distribution shifts in the analysis and does not participate in the modeling or training process of TFPS.
E
Metric Illustration
We use mean square error (MSE) and mean absolute error (MAE) as metrics for evaluation of all forecasting models. Then calculation of MSE and MAE can be described as:
1 L+H
MSE
Σ
(
(17)
H 1-11
L+
MAE 1
Σ i=L+1
Y, - Yz
(18)
H
where Ý predicted vector with H future values, while is the ground truth. In addition, we report the IMP (Improvement) metric, which is defined as:
IMP Avg MSE of baselines MSE of TFPS
X 100%.
(19)
Avg MSE of baselines
This metric quantifies the relative percentage improvement TFPS over the average MSE of all baseline methods. A higher IMP value indicates better overall performance of TFPS compared the baselines.
Hyperparameter-search Results
F.1 Comparsion with Specific Models
To ensure fair comparison between models, we conducted experiments using unified parameters and reported results in the main text.
In addition, considering that the reported results in different papers are mostly obtained through hyperparameter search, we provide the experiment results with the full version of the parameter search. We searched for input length among 96, 192. 336, and 512. The results are included in 6 All basclines are reproduced by their official code.
We find that the relative promotion of TFPS over TFDNet is smaller under comprehensive hyperparameter search than the unified hyperparameter setting. is worth noticing that TFPS runs much faster than TFDNet according the efficiency comparison in Table [2] Therefore, considering performance, byperparameter-searrd cost and efficiency, we believe TFPS is a practical model in real-world applications and valuable deep time series forecasting community.
F.2
Comparsion with Foundation Models
Table presents the detailed results of TFPS compared to recent foundation models (AutoTimes Moment [16), and Timer [32] across six datasets and four forecasting lengths. Consistent
30
25

---

Table Experiment results under hyperparameter searching for the long-term term forecasting task. The best results are highlighted in bold and the second best are underlined.
Model
IMP.
(Our)
TFPS
TSLANet (2024)
(2024)
FITS
iTamsfoomer (2024)
THENa1-IK (2023)
PatchTST (2023)
TimesNet (2023)
Dlincus
(2023)
PHDXommer
(2022)
Metric
MSB
MSH
MAE
MSE
MAE
MSE
MAE
MAE
MSB
MAE
MSB
MAE
96
1.5%%
0.372
0.404
0.374
0.395
0.387
0.405
0.360
0.387
0,412
0.364
0.405
0.385
0,425
192
5.718
0,401
0.410
0.417
0.412
0.441
0.436
04003
0.412
0.414
0.443
0.450
0.441
0.461
336
0.8%
0.409
0,402
0.412
0.429
0.478
0.491
0.63
0.420
0.467
0.447
0.448
0.91
0.473
720
11.2%
0.425
0.433
0.475
0.47
0.425
0.446
0.509
0.494
0.455
0.452
0.450
0.466
0.512
0.491
0.504
0.515
0.501
0.495
F7112
192
966
10.4%
0.3%
0.268
0.329
01325
01.76
0.285
0.331
0.37%
0.344
0.274
0.337
0.377
0.337
0.301
0.350
0.350
0.399
0.271
0,533
0.329
0.372
0.278
0.339
0.336
0.380
0.393
0.324
0.430
0.366
01388
0.200
0.353
01422
0.342
0.434
0.383
0.440
336
17.7%
0.329
0.401
0.319
0.357
0.360
0.398
0.424
0.42
0.351
0. 0.366
0.316
0 180
0.429
0.437
0.463
0.473
0.512
0.490
0.450
0.467
0.476
ETial
192
96
10.2%
8.5%
0.324
0.281
0.345
0.365
0.383
0.942
0.977
0.396
0.377
0.356
0.395
0.405
0.336
0.345
0.366
0.305
0.560
0.406
0.427
336
8.2%%
0.599
0,404
0.357
0.392
0.372
0.385
0:418
0.118
0.361
K375
0,367
0.391
0.433
0,432
0.372
0.389
0.488
0.458
720
8.25%
11409
0.408
0.425
0.475
0.428
0.416
0.487
0.457
0441
0.409
0.417
0.422
0.484
0.458
0:427
0.423
0:491
0.479
ETTu2
96
8.9%
0.158
0.243
0.167
0.256
0.165
0.355
0.186
0.272
0.158
0224
0.164
0.253
0.182
0.262
0.172
0.267
0.193
0.285
192
5.75%
0.222
0.502
0.221
0.204
0.220
0.201
0.254
0.314
0.219
0.261
0.231
0.792
0.252
0.307
0.337
0.314
0.256
0.334
336
8.5%
0.268
0.316
0.329
0.274
0.326
0.316
0.251
0.272
0.317
0.277
0.329
0.312
0.346
0.295
0.359
0.321
0.364
720
12.0%
0:414
0.407
0.366
0.374
0,365
0.427
0,639
0.434
0.426
Tratfe
17.8%
0.370
0.428
0.295
0.253
0.373
0.413
0.575
0.357
192
17.0%
0.390
0.258
0.395
0.272
0.408
0.388
0:448
0.302
0.391
0.260
0,395
0.273
0.618
0.323
0.424
0.290
0.613
0.381
336
17.24
0.A01
0.271
0.402
0.252
0.420
0.302
0.465
0.311
0.408
0.266
0.400
0.274
0.634
0.337
0.438
0.299
0.602
0.390
720
15.716
0.432
01
0.131
0.288
0.468
0.310
0.501
0.333
0:451
0.291
0.435
0.291
0.659
0.349
0.466
0,316
0.630
0.383
Eletnicity
96
10,315
0.128
0.220
0.137
0.229
0.135
0.231
0.148
0.239
0.130
0.222
0,130
0.223
0.168
0.272
0.140
0.237
0.188
0.303
192
11.9%
0.145
0.235
0.153
0.242
0.169
0.246
0.167
0.258
0.146
0.57
0.149
0.240
0.156
0.289
0.154
0.250
0.197
0.311
0.178
0.371
0.162
0.212
0.327
720
6.915
0.198
0.283
0.204
0.293
0.211
0.300
0.201
0.287
0.204
0.329
0.204
0.300
0.243
0.352
1" Count
0
Table 7: Detailed results of the comparison between TFPS and foundation models. The best results are highlighted in bold and the second best are underlined.
TFPS
AutoTimes (2024)
Moment (2024)
Timer (2024)
Model
IMP.
(Our)
Metric
MSE
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAB
96
-2.2%
0.372
0.404
0.365
0.405
0.369
0.406
0.389
0.392
192
-1.3%
0.401
0.410
0.392
0.423
0.405
0.431
0.391
0.413
ETTh1
336
0.5%
0.409
0.402
0.406
0.433
0.420
0.441
0.407
0.424
720
3.1%
0.423
0.433
0.423
0.450
0.466
0.479
0.421
0.441
6.3%
0.268
0.325
0.286
0.349
0.287
0.347
0.285
0.344
192
9.2%
0.329
0.376
0.351
0.393
0.371
0.401
0,365
0.400
ETTh2
336
17.2%
0.329
0.401
0.377
0.417
0.404
0.425
0,412
0.440
720
9.8%
0.412
0.441
0.439
0.464
0.463
0.476
0.467
0.487
1.3%
0.281
0.329
0.297
0.350
0,281
0.343
0.276
0.335
ETTml
142
1.2%
0.324
0.354
0.344
0.377
0.318
0.368
0.323
0.365
336
1.6%
0.359
0.404
0,380
0.398
0.356
0.391
0,358
0.388
720
4.8%
0,409
0.408
0.433
0.431
0.438
0.441
0,419
0.423
96
8.6%
0.158
0.243
0.181
0.266
0.170
0.258
0.167
0,254
5.5%
0.222
0.302
0.241
0.306
0,233
0.301
0.229
0.297
ETTm2
336
6.9%
0.268
0.316
0.295
0.341
0.287
0.340
0.282
0.335
720
8.1%
0.344
0.373
0.376
0.393
0,371
0.399
0,376
0.398
96
-5.5%
0.370
0.250
0.347
0.249
0,360
0.254
0.345
0.237
Traffie
192
-5.3%
0.390
0.258
0.366
0.258
0,381
0.265
0.365
0.247
336
-3.1%
0.401
0.271
0.383
0.267
0,404
0.277
0.381
0.256
720
-1.1%
0.432
0.294
0,420
0.286
0.438
0,297
0.424
0.280
96
3.4%
0.128
0.220
0.135
0.230
0.133
0.236
0.130
0.224
Electricity
192
4.3%
0.145
0.235
0.153
0.247
0.152
0.254
0.149
0.243
336
1.5%
0.166
0.258
0.172
0.266
0.166
0.265
0.168
0.263
720
5.2%
0.198
0.283
0.212
0.300
0.202
0.298
0.213
0.303
1" Count
30
2
2
14
with Table 6. for TFPS, we searched input lengths among 96, 192, 336, and 512, and report the best-performing configuration for each forecasting length.
TFPS achieves the best performance in 30 out of 48 settings, significantly outperforming AutoTimes and Moment, which only achieve 2 best scores each. Although Timer demonstrates competitive
26

---

Table Detailed results of the comparison between TFPS and normalization based methods using FEDformer. The best results are highlighted in bold and the second best are underlined.
FEDrommer
TFPS
Model
IMP.
SIN (2024)
SAN (2023)
DS (2023)
ReviN (2021)
(Our)
Metric
MSF
MSE
MAF
MSE
MAB
MSF
MAF
MSE
MAR
MSF
MAE
96
-0.9%
0.398
0.413
0.413
0.372
0.383
0.409
0.390
0.424
0.392
0.413
192
3.7%
0.423
0.423
0.443
0.417
0,431
0.438
0,441
0.458
0.443
0,444
336
-0.5%
0.484
0.461
0,465
0.448
0.471
0,456
0.495
0.486
0.495
0.467
720
4.8%
0.488
0.476
0.509
0.490
0,504
0.488
0.519
0.509
0.520
0.498
ETThh
96
34.0%
0.313
0.355
0.412
0.357
0.300
0.355
0.806
0.589
0.380
0.402
192
28.3%
0.405
0,410
0.472
0.453
0.392
0.413
0.936
0.659
0.457
0,443
336
38.2%
0.392
0.415
0.527
0.527
0.459
0.462
1.039
0.702
0.515
0.479
720
41.4%
0.410
0.433
0.593
0.639
0.462
0.472
1.237
0.759
0.507
0.487
ETTm2 ETTml
96
4.5%
0.327
0.367
0.373
0.320
0.311
0.355
0.348
0.397
0.340
0.385
192
2.9%
0.374
0.395
0.366
0.351
0.383
0.428
0.390
336
4.4%
0.401
0.408
0.418
0.405
0.390
0.407
0.438
0.450
0.432
0,436
720
-0.8%
0.479
0.456
0.451
0.475
0.456
0.444
0.497
0.481
0.497
0.466
96
37.5%
0,170
0.255
0.326
0.211
0.175
0.266
0.394
0.395
0.192
0.272
192
35.9%
0.235
0.296
0.402
0.316
0.246
0.315
0.552
0.472
0.270
0.320
336
38.66%
0.297
0.335
0.465
0.399
0.315
0.362
0.808
0.601
0.348
0.367
720
40.2%
0.401
0.397
0.555
0.547
0.412
0.422
1.282
0.771
0.430
0.415
Weather
96
30.7%
0.154
0.202
0.280
0.215
0.179
0.239
0.244
0.317
0.187
0.234
192
25.66
0.205
0.249
0.314
0.264
0.234
0.296
0.320
0.380
0.235
0.272
336
22.0%
0.262
0.289
0.329
0.293
0.304
0.348
0.424
0.452
0.287
0.307
720
21.3%
0.344
0.342
0,382
0.370
0,400
0.404
0.604
0.553
0.361
0.353
1 Count
24
Table 9: Detailed results of the comparison between TFPS and normalization- based methods using DLinear. The best results are highlighted in bold and the second best are underlined.
DLineur
Model
TMP.
TFPS
SAN
Dish- (2023)
ReviN
(Our)
(2024)
(2023)
(2021)
Metric
MSB
MSB
MAE
MSE
MAE|
| MSE
MAE
MSE
MAE
MSE
MAE
96
-1.6%
0.398
0.413
0.401
0.415
0.385
0.395
0.389
0.399
0.393
0.416
192
2.9%
0.423
0.423
0.438
0.456
0.432
0.423
0.443
0.433
0.431
0.428
ETTh1
336
-0.4%
0.484
0.462
0.446
0.463
0.487
0.456
0.483
720
4.6%
0,488
0.476
0,515
0.500
0,516
0,504
0.523
0,508
0.493
0.482
96
4.9%
0.313
0.355
0.359
0.359
0.319
0.364
0,317
0.365
0.322
0.361
ETTh2
192
1.1%
0.405
0.410
0.409
0.424
0.407
0.439
0.408
0.420
0,412
0.424
336
3.6%
0.392
0.415
0.398
0.429
0.411
0.425
0,416
0.426
0.403
0.427
720
2.7%
0.410
0.433
0,419
0.442
0.441
0.428
0.439
0.422
0.446
96
4.9%
0.327
0.367
0,350
0
0.333
0.374
0.343
0.375
0.352
0.369
ETTml
1.9%
0. 0.395
0,183
0.396
0.396
0.381
0.391
0.388
0.396
336
3.0%
0.401
0.408
0.413
0.416
0,406
0.418
0.416
0.417
0.419
0.414
720
0.0%%
0.479
0.456
0.473
0.452
0.483
0.451
0.482
0.458
0.478
0.463
96
5.5%
0.170
0.255
0.172
0.283
0.179
0.272
0.189
0.264
0.179
0.269
ETTm2
192
4.4%
0.235
0.296
0,249
0.301
0,239
0.316
0,249
0.302
0,248
0.302
336
1.2%
0.297
0.335
0.299
0.339
0.301
0.305
0.349
0.299
0.345
720
3.2%
0.401
0.397
0.412
0.421
0,404
0,408
0.429
0.402
0.411
0.402
96
4.5%
0.154
0.202
0.162
0.223
0.157
0.215
0.173
0.241
0.154
0.243
Weather
192
7.6%
0.205
0.249
0.216
0.259
0,214
0,258
0.225
0.263
0,233
0.265
336
6.8%
0.262
0.289
0,279
0.291
0.275
0.292
0.289
0.305
0.282
0.293
3.0%
0.355
0.341
0.366
0.369
0.362
1dt Crannt
33
3
0
performance (with 14 best scores), TFPS consistently achieves lower error in datasets with higher distribution shifts, such as ETTh2 and ETTm2, indicating its advantage in handling pattern hetero- geneity.
These results reinforce the effectiveness of TFPS's pattern-specifie modeling strategy, especially in scenarios where traditional large-scale foundation models struggle to generalize.
27

---

Table 10: Comparison between TFPS and MoE-based methods. The best results are highlighted in bold and the second best are underlined.
THPS
MoLE
MoU (2024)
KANATSF
Model
IMP.
(Our)
(2024)
(2024)
Metric
MSE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
-4.3%
0.398
0.413
0.383
0.392
0.381
0.382
0.40U
192
1.7%
0.423
0.423
0.434
0.426
0.429
0.430
0.430
0,426
ETThl
336
1.6%
0.484
0.461
0.489
0.478
0,488
0.463
0.498
0.467
720
8.2%
0.488
0.476
0.602
0.545
0,499
0.484
0.494
0.479
96
10.4%
0.313
0.355
0.413
0.360
0.317
0.358
0.318
0,358
192
10.3%
0.405
0.410
0.525
0.416
0,409
0,414
0.419
0.414
ETTh2
336
7.1%
0.392
0.415
0.423
0.434
0.397
0.420
0.447
0.452
720
8.4%
0.410
0.433
0.453
0.458
0.412
0.434
0.477
0.476
13.5%
0.327
0.367
0.338
0.380
0.465
0.442
0.333
0.371
ETTml
192
10.6%
0.374
0.395
0.388
0.403
0.483
0.455
0.384
0.399
336
11.8%
0.401
0.408
0.417
0.431
0,540
0.488
0:407
0.413
720
7.3%
0.479
0.456
0.486
0.472
0.583
0.509
0. 0.483
0.469
13.9%
0.170
0,255
0.258
0.271
0.179
0.263
0.175
0,260
192
3.8%
0.235
0.296
0.247
0.305
0,243
0.303
0.244
0,305
ETTm2
3.3%
0.297
0.335
0.308
0.306
0.343
0.308
0.347
720
13.7%
0.401
0.397
0.583
0.419
0.405
0.404
0.405
0.404
1 Count
30
G Compared with Other Methods
G.1 Compared with Normalization Methods
In this section, provide the detailed experimental results of the comparison between TFPS and five state-of-the-art normalization methods for non-stationary time series forecasting: SIN 17.. SAN 34] Dish-TS [10), and RevIN (20). We summarize the forecasting results of TFPS and baseline models in Table and Table Specifically, the results of FEDformer combined with SIN are taken from [7].
while those of FEDformer with other cemmalization-basee methods are reported by (34). For fair comparison, we additionally rerun all experiments for DLinear combined with each normalization method.
and Table presents the forecasting performance across all prediction lengths for each dataset, along with the relative improvements of TFPS over existing methods. As shown, TFPS consistently achieves the best performance in the majority of settings, demonstrating its strong adaptability to distributional and conceptual drifts in time series data.
We attribute this improvement to the accurate identification of pattern groups and the provision of specialized experts for each group, thereby avoiding the over-stationarization problem often associated with normalization methods.
G.2 Compared with MoE-based Methods
As shown in Table 10 unlike MoE-based methods that rely the Softmax function gating mechanism, our approach constructs a pattern recognizer to assign different experts to handle distinct patterns. This results in TFPS achieving relative improvements of 2.3%. 9.0%, 10.6%, and 9.1 % across the four datasets, respectively.
G.3 Compared with Distribution Shift Methods
As shown in Table II we compare with the methods for distribution shift. This results in TFPS achieving relative improvements of 6.7%, 6.6%, 4.8%, and 5.9% across the four datasets, respectively.
H Model Analysis
Detailed Results on the Number of Experts. We provide the full results on the number of experts for the ETThl and Weather datasct in Figure
28

---

Table 11: Comparison between TFPS and methods for Distribution Shift. highlighted in bold and the second best are underlined.
The best results are
THPS
Koopa (2024))
SOLID (2024)
OneNet
Model
IMP.
(Our)
(2024)
Metric
MSE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
7.9%
0.398
0.385
0.440
0,439
0.425
0.402
192
10.3%
0.423
0.423
0.445
0.434
0.492
0.466
0.452
0,443
ETThl
336
4.9%
0.484
0.461
0.489
0.460
0,525
0.481
0.492
0.482
720
4.4%
0.488
0.476
0.497
0.480
0.517
0.496
0.504
0.496
96
10.6%
0.313
0.355
0.318
0.360
0.318
0.359
0.382
0,362
192
4.7%
0.405
0.410
0.378
0.398
0,414
0.418
0.435
0.426
ETTh2
336
4.8%
0.392
0.415
0.415
0.430
0.398
0.421
0.426
0.419
720
6.8%
0.410
0.433
0.445
0.456
0.424
0.441
0.456
0.437
6.8%
0.327
0.367
0.329
0.359
0.329
0.370
0.374
0.392
192
2.0%
0.374
0.395
0,380
0.393
0.379
0,400
0.385
0.435
ETTml
336
8.7%
0.401
0.408
0.401
0.411
0.405
0.412
0.473
0.458
720
2.0%
0.479
0.456
0.475
0.448
0.482
0.464
0.496
0.483
5.3%
0.170
0,255
0.179
0.261
0.175
0.258
0.184
0.274
192
3.8%
0.235
0.296
0.246
0.305
0,241
0.302
0.248
0,384
ETTm2
0.297
0.335
0.310
0.348
0.303
0.342
0.313
0.374
720
9.0%
0.401
0.397
0.405
0.402
0.456
0.436
0.425
0.438
1 Count
25
6
0
T= 96
T 192
T 336
T= 720
K1K4
K1K4
K314
K14
<aka
Aara
KaKe
0.453
643
9.499
2001
6:20
K4K4
K4K4
K3K4
2350
A110
Kark
9.273
22232
0,29
0370
0.347/
0 0.196154
6.3%
6.20
JAZ25
0.34
<3KK4
<4K8
6
K414
XAC
Figure 7: Results of expert number experiments for ETThl and Weather.
In Figure 7 we set the learning rate to 0.0001 and conducted four sets of experiments the ETTh1 and Weather datasets, K, = 1, K2 = {1, 2, 8]. to explore the effect of the number of frequency experts the results. For example, K,1K,4 ,4 means that the TFPS contains I time experts and 4 frequency experts. We observed that K,1K22 outperformed K2 44 in most cases, suggesting that increasing the number of experts does not always lead to better performance.
In addition, we conducted three experiments based on the optimal number of frequency experts to verify the impact of varying the number of time experts on the results. As shown in Figure 7 the best results for ETThl were obtained with K,4K 2, K,8K 44 K,4K 44, K,4K 44, while for Weather, the optimal results were achieved with K,4K 88, K,4K 88, K,4K 4 and K,4K, 88 Combined with the average Wasserstein in T we attribute this to the fact that, in cases where concept drift is more severe, such as Weather, more experts are needed, whereas fewer experts are sufficient when the drift is less severe.
Comparing Inter- and Intra-Cluste Differences via Wasserstein Distance. To assess the ef- fectiveness of the module, we replace it with a linear layer and compare the resulting inter- and intra-cluster Wasserstein distance heatmaps in Figure 8 The diagonal elements represent the average Wasserstein distances of patches within the same clusters. these values are small, indicates that the difference of patches within the same cluster is relatively similar. The off-diagonal elements represent
29

---

(a) Linear layer
(b) Pattern Identifier
Figure 8: Heatmap showing the Wasserstein distances of inter- and intra-cluster patches on ETTh1.
18
05
-35 -2
(a) Linear
(b) Pattern Identifier
Figure 9: Visualization of the embedded representations with t-SNE ETThl for the time domain with H 96. (a) t-SNE visualization with a Linear Layer replacing the Patch Identifier for
comparison. (b) t-SNE visualization of TFPS.
(a) ETThl
(b) ETTml
(c) Weather
Figure 10: MoE Expert allocation distributions of TFPS: the x-axis corresponds to the experts, and the y-axis shows the proportion of tokens assigned each expert.
the average Wasserstein distances between patches from differrntt clusters, where larger values mean sigmificant differences between the clusters. We observe that when using PI, the intra-cluster drift is smaller, while the inter-cluster shift is more pronounced compared the linear layer. This indicates that our identifier effectively classifies and distinguishes between different patterns.
TFPS produces differentiated token embeddings by adapting to the characteristics of the data. Figure presentss the t-SNE visualization of the learned embedded representation on the ETTh for the time domain with H 96. In the Figure (a), where the pattern identifier is replaced with linear layer, the representation lacks clear clustering structures, resulting in scattered and indistinct groupings. In contrast, Figure (b) shows the visualization of the representation learned by the proposed method, which effectively captures discriminative features and reveals significantly clearer clustering patterns.
TFPS implements dataset-specific token embeddings assignment in data-driven way, effec- tively improving performance. Figure 10 visualizes the expert allocation distributions across various datasets. Notably, ETTh1 and ETTml exhibit high degree of consistency in their expert assignments, underscoring the model's ability to capture shared patterns. Conversely, the Weather dataset shows distinctly different allocation pattern, highlighting the method's sensitivity to unique dataset characteristics and its capability to tailor expert assignments accordingly.
30

---

Table 12: The GPU memory (MB) and speed (inference time) of each model.
TFPS
TSLANet
FITS
iTransfommer
TFDNe-IK
PatchTST
TimesNet
DLincar
FBIffrmer
MSE
0.423
0,488
0.445
0.441
0.458
0.460
0.441
0.434
0.441
GPU Memory (MB)
9.643
0481
0.019
3.304
0.246
0.205
2.345
0.142
62.191
Average Inforence Time (ms)
6.114
0.063
1.184
2,571
98.266
4,861
12,306
0.689
136.130
039
2.001
6231
D11 000
(b) 8
Figure 11: Parameter sensitivity of a and 3 of the proposed method on the ETTh1-96 dataset.
I Efficiency Analysis
To make this clearer, present the results of ETThl for a prediction length of 192 from Table T and include additional results on runtime and computational complexity in Table T2 Due to the sparsity of MoPE, TFPS achieves balance between performance and efficiency:
Performance Superiority. TFPS achieves an MSE of 0.423, outperforming TSLANet (0.448),
FITS (0.445), PatchTST (0.460), and FEDformer (.. This represents 5.6% improvement over TSLANet and 8.0% improvement over PatchTST, highlighting its significant accuracy gains. While DLincar achieves an MSE of 0.434, TFPS still demonstrates 2.5% relative improvement, making the most accurate model among all baselines.
Efficiency Gains. TEPS maintains competitive runtime and memory efficiency.
Runtime: TFPS runs in 6.457 ms. making 2.8x faster than PatchTST (17.851 ms) and 11.2 faster than TimesNet (72.196 ms).
Memory Usage: TFPS uses 9.643 MB GPU memory, significantly less than FEDformer (62.191 MB) and comparable to iTTranffommer (3.304 MB). This makes TFPS suitable for resource-constraineed applications while maintaining superior performance.
Balancing Trade-offs. While lightweight models like DLinear (0.434 MSE, 0.789 ms runtime) are slightly more efficient, TFPS delivers a performance improvement of 2.5%, providing a well-rounded solution that balances accuracy and efficiency effectively.
J Hyperparameter Sensitivity
In this section, we analysis the impact of the hyperparameters a and 9 on the performance.
Specifically, we performed grid search to optimize the byperparameters a₂ = {0.0001, 0.001, 0.01) and y (0.0001, 0.001, 0.01), shown in Figure II (a). After extensive testing, ultimately fixed at 0 10 in our experiments.
In addition, we conducted a grid search optimize the balance factors 3r (0.01, 0.05, 0.1, 0.5, 1} and By {0.01, 0.05, 0.1, 0.5, 1}. The performance under different parameter values is displayed in (b). from which we have the following observations:
Figure
Firstly, the performance is affected when the value of 3 is too low, indicating that the proposed clustering objective plays a crucial role in distinguishing patterns.
Second, excessive 33 also has negative on the performance. One plausible explanation is that the excessive value influences the learning of the inherent structure of original data, resulting in perturbation of the embedding space.
31

---

Table 13: In the table, w/ Imaginary indicates that we incorporate both the real and imaginary parts into the network.
ETTh1
ETTh2
96
192
336
720
96
192
336
720
TFPS
0.398
0.423
0.484
0.488
0.313
0.405
0.392
0.410
w/ Imaginary
0.397
0.424
0.487
0.486
0.312
0.406
0,391
0.399
Table 14: Ablation study of Pl components. The model variants in our ablation study include the following configurations across both time and frequency branches: (a) inclusion of the Time Pl; (b) inclusion of the Frequency PI; (c) exclusion of both. The best results are in bold.
ETTh1
ETTh2
Time Pl Frequency
96
192
336
720
96
192
336
720
0.398
0.423
0.484
0.488
0.313
0.405
0.392
0,410
X
0.404
0.454
0.490
0.503
0.322
0.413
0,410
0.425
0.405
0.456
0.493
0.509
0.324
0.415
0.412
0.430
0.407
0.458
0.497
0.513
0.328
0.418
0.419
0.435
Overall, we recommend setting 3 around 0.1 for optimal performance.
K
Full Ablation
K.1
Impacts of Real/Imaginary Parts
To further validate the robustness our approach, we adopted similar operations FreTTS to conduct experiments incorporating both the real and imaginary parts. The results in the Table show that the performance of TFPS with the real part only is very similar to that when both parts are included, while requiring fewer parameters. This further reinforces the conclusion that TFPS remains highly effective even when focusing solely on the real part of the Fourier transform.
K.2
Ablation on PI
The PI module plays crucial role identifying and characterizing distinct patterns within the time series data, while the gating network dynamically selects the relevant experts for each segment. This collaborative mechanism allows the model to specialize in handling different patterns and adapt effectively to distribution shifts, thus mitigating the overfitting risks that arise from treating all data equally.
To validate the importance of PI empirically, have conducted the ablation experiments comparing the model's performance by replacing the PI module with a linear layer in the Table main text. In addition, supplement some ablation experiments in Table 14 to further verify the effectiveness of PI.
K.3 Ablation on R and R2
We conducted ablation experiments to further verify the important roles of Rí and Ro, as shown in
Table 15
K.4 Replace MoPE with Alternative Designs
Here provide the complete results of alternative designs for TFPS.
As show in Table T6] we have conducted addition experiments where we replaced the MoPE module with weighted multi-output predictor and stacked self-attention layers, keeping all other components
32

---

Table 15: Ablation study of Loss Constraint. The model variants in our ablation study include the following configurations across both time and frequency branches: (a) inclusion of the R (b) inclusion of the R2: (c) exclusion of both. The best results are bold.
ETTh1
ETTh2
R1
R2
96
192
336
720
96
192
336
720
V
0.398
0.423
0.484
0.488
0.313
0.405
0.392
0.410
0.408
0.449
0.500
0:498
0.320
0.418
0.415
0.429
X
0.403
0.434
0.493
0.491
0,316
0,413
0.405
0.418
0.412
0.456
0.509
0.503
0.328
0.425
0.420
0.435
Table 16: Multi-output predictor and a stacked attention layer are used replace MoPE in ETTh1 and ETTh2 datasets.
ETTh1
ETTh2
96
192
336
720
96
192
336
720
TFPS
0,398
0.423
0.484
0.488
0.313
0.405
0.392
0.410
Multi-output Predictor
0.403
0.435
0.492
0.491
0.317
0.407
0.399
0.425
Attention Layers
0.399
0.452
0.492
0.508
0.334
0.407
0.409
0.451
and configurations identical. The results demonstrate that our proposed method significantly out- performs them, which validates the importance of the Top-K selection and sattern-awars design in enhancing the model's representation capacity. In contrast, multi-output predictor and self-attention typically treats all data points uniformly, which may limit its ability to capture subtle distribution shifts evolving patterns across patches.
L Algorithm of TFPS
We provide the pseudo-code of TFPS in Algorithm
1
M Broader Impact
Real-world Applications. TFPS addresses the crucial challenge of time series forecasting, which is valuable and urgent demand in extensive applications. Our method achieves consistent ttate-of-the-art performance four real-world applications: electricity, weather, exchange rate, illness. Researchers in these fields stand to benefit significantly from the enhanced forecasting capabilities of TFPS. We believe that improved time series forecasting holds the potential to empower decision- makinng and proactively manage risks in wide array of societal domains.
Academic Research. TFPS draws inspiration from classical time series analysis and stochastic process theory, contributing to the field by introducing a novel framework with the assistance pattern recognition. This innovative architecture and its associated methodologies represent significant advancements the field of time series forecasting, enhancing the model's ability to address distribution shifts and complex patterns effectively.
Model Robustness. Extensive experimentation with TFPS reveals robust performance without exceptional failure cases. Notably, TFPS exhibits impressive results and maintains robustness in datasets with distribution shifts. The pattern identifier structure within TFPS groups the time series into distinct patterns and adopts a mixture of pattern experts for further prediction, thereby alleviating prediction difficulties. However, it essential to note that, like any model, TFPS may face challenges when dealing with unpredictable patterns, where predictability is inherently limited. Understanding these nuances is crucial for appropriately applying and interpreting TFPS's outcomes.
Our work only focuses the scientific problem, there is potential ethical risk.
33

---

Algorithm Time-Frequency Pattern-Specife architecture Overall Architecture.
Input: Input lookback time series E Ix imput length predicted length H; variables number C: patch length P; feature dimension D; encoder layers number nı; random Gaussian distribution- initialized subspace D = [D D ... d g/K. Technically, set D 512, n as 2. Output: The prediction result Y.
,.....)))))) ,D()), each D Raxd where = C x D and
1: X X.transpose
X E E ROXL X RRCNND
2: Xpg Patch (X) + Position Embedding
3: b Time Encoder.
4: X² = -XrE
5: for in {1...... ................................ m]:
6: X LayerNorm (X,-) Self-Attn (X(-1)). 1))
x1-1 RCNNDD > X E RCXNXD
7: X = LayerNorr (X, FFed-Forraak (X2
8: End for
9:
X
D 2 ROXNDD
Pattern Identifier for Time Domain.
8t Subspace affinity (2,, D)
Eq. of the paper 8 RCNNDD Eq. of the paper & E RCNND
12: &, = Subspace refinement (st)
13: b Mixture of Temporal Pattern Experts.
14: G(s) Softmax (TopK (s)))
h4 Σ G(s)MLLPx(2)
Eq.
10 and Eq.
11 of the paper h2 RCNNDD
16: & Frequency Encoder.
17: X =XPE
> Eq.
of the paper X E RXNNP
18: for in nn}: (I....,n]:
X LayerNorm (x,-1 Fourier (X)-1)).
x RCNND X E RCXNDD
20: x LayerNorm (x/-1+ Feed-Forward (X!-1))
21: End for
2f = +
a 2/ RCXNDD
23: Þ Pattern Identifier for Frequency Domain.
24: 8 Subspace affinity (2,, D)
> Eq. 6 of the paper s E cxNND
25: 8 Subspace refinement (sy)
Eq. of the paper &y RCNND
Mixture of Frequency Pattern Experts.
27: G(s) = Softmax (TopK (sy))
28: h Σk= G(s)MLPP(2F)
Eq.
10 and Eq.
11 of the paper h, E RCXNDD
29: h = Concat(h,, hy)
> h E BCxX22D
30: for c in (1, (I...., C):
31:
Lineerr (Flatten (h)).
Project tokens back to predicted series ý E RCC
32: End for
33: -= Ytrrnsposs
v E x
34: Return Y
Output the final prediction
y #xc
34

---

N
Limitations
Though TFPS demonstrates promising performance on the benchmark dataset, there are still some limitations of this method. First, the patch length is primarily chosen heuristically, and the current design struggles with handling indivisible lengths or multi-perion characteristics in time series. While this approach works well in experiments, it lacks generalizability for real-world applications. Second, the real-world time series data undergo expansion, implying that the new patterns continually emerge over time, such as epidemic or outbreak that had not occurred before. Therefore, future work will focus on developing a more flexible and automatic patch length selection mechanism, as well as an extensible solution to address these evolving distribution shifts.
35


Reviews:
[{"review_id": "UfYoXx6awB", "reviewer": "Reviewer_LAjU", "summary": "This paper studies the time-series forecast problem under potential context shifts and distribution drifts and focus on achieving better generalization on datasets with heterogeneous evolution patterns across time frames. To fully utilize the predictive patterns in input time-series, the authors propose a novel TFPS architecture with a dual-domain encoder that extracts information from both time and frequency domain representation of multiple segments. Further, the latent embedding of different time-series segments are grouped via subspace clustering, which enables the adoption of mixture of expert (MoE) technique to enhance the expressiveness of TFPS. The proposed method is evaluated on nine real-world datasets and achieve better accuracy over multiple state-of-the-art time-series forecasting approaches. Extensive ablation studies have also been provided to elaborate the significance of the model design in TFPS.", "questions": "1. Consider the completeness of temporal patterns in each patch, I wonder if a sliding window (or allowing overlap between adjacent patches) would be more appropriate than the current hard segmentation of input time-series in TFPS.\n2. Would the linear projection and positional encoding in Fig. 1(a) introduce unnecessary biases to the Fourier transform result (and the frequency domain embedding $z_f$) in TFPS? Why not directly operate on the raw observations?\n3. Why is parameter $\\alpha$ hard-coded to be $10^{-3}$? Any good reason for this design choice? How to select the value of $\\alpha$ in practical applications?\n4. While assigning each temporal pattern to a dedicated expert network may enhance the capability of TFPS to model heterogeneous contexts shifts and temporal evolution, would this design prevent TFPS from effectively learning temporal dependencies across patches?\n5. I agree with the authors that similar frequency domain patterns may form a subspace since Fourier transform is linear under addition. However, addition of temporal patterns in time domain may not necessarily yields time-series with similar patterns, e.g., \"decrease\"+\"increase\"=\"constant\". There should be justification for the assumption that patches with similar temporal patterns have embedding $z_t$ in the same subspace. Specifically, how can a convex hull in the subspace (of $z_t$) be mapped to similar temporal patterns in time domain? I believe there would also be alignment issues considering the hard segmentation of patches in TFPS.\n6. In Eq. 8, what is the purpose to minimize the KL divergence between refined subspace affinity $\\hat{S}$ and the unnormalized subspace affinity score $S$? According to Eq. 7 and discussions around LN 182, $\\hat{S}$ is merely a \"sharpened\" version of $S$, which makes $\\mathcal{L}_{sub}$ in Eq. 8 sort of meaningless. Why bother to have so many hyperparameters?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The temporal patterns and frequency domain patterns are jointly utilized for time-series forecasting in TFPS, which enables a comprehensive consideration of the evolution patterns and distribution shifts in the underlying data generation processes. The segmentation of input time-series further allows TFPS to make predictions based on finer grained characteristics in the data stream.\n- The MoE structure in TFPS explicitly matches each patch of the input with its dedicated expert networks based on time- or frequency-domain patterns. This enables TFPS to effectively predict heterogeneous evolution trajectories according to evidence of context shifts in recent observations.\n- The proposed method is evaluated on nine publicly available time-series datasets against the baselines of eight state-of-the-art time-series forecasting approaches. Detailed experimental results and ablation studies are provided to demonstrate the effectiveness of TFPS.\n\nWeaknesses\n- The time-series segmentation rule described in Section 3.3 is kind of coarse. It could be possible that a complete temporal trend (hyperbolic curve or jumps) could be split into two patches, which may not be ideal and cause degradation in forecasting performance. It would be helpful if the authors can provide some discussion on this issue and why hard segmentation is preferred than a sliding window in TFPS.\n- There is no discussion on the requirement of time-series input. Could TFPS deal with irregularly sampled or very sparse time-series?\n- There are too many hyperparameters in TFPS: $\\alpha$, $\\beta$, $\\eta$, $d$, ... The complexity and intricate impact of these parameters may impede its adoption in practical applications. Parameter $\\alpha$ is even hard-coded to be $10^{-3}$ throughout the experiment. Are there any good reasons for this?\n- According to Table 1, the absolute performance gains of TFPS seem to be marginal compared to the second best baselines except for the ILI dataset. Also, instead of the average performance of baseline methods, the performance improvement (IMP) in Eq. 19 should be computed against the second best score on each row.", "was_revised": true, "final_justification": "After the rebuttal, the explanations on the large number of hyperparameters (not to mention the hard-coded ones) and KL loss term $\\mathcal{L}_{sub}$ by the authors are still not convincing enough. However, given that the authors have addressed many of my other concerns, I tend to maintain my initial positive rating for this paper.", "rebuttal": "We sincerely thank the reviewer for the thoughtful feedback and positive recognition of our design in modeling distribution shifts via dual-domain encoders and subspace-specific experts, as well as the thorough empirical validation. Below we give a point-by-point response to your concerns and suggestions.\n\n> **W1 \\& Q1. On sliding window vs. hard segmentation.**\n\nWe agree with the reviewer that hard segmentation may lead to degraded forecasting performance. We would like to clarify that TFPS operates under a **sliding window** setting rather than hard segments. As described in $\\underline{\\text{Section 3.3}}$, we follow the patching strategy of PatchTST, where the stride $S$ defines the overlap between adjacent patches.\n\nTo directly assess the reviewer's concern, we conducted additional experiments using **non-overlapping (hard) segmentation**, where each patch is strictly isolated. As expected, due to the loss of temporal continuity, performance deteriorated:\n\n|Dataset|ETTh1|ETTh2|ETTm1|ETTm2\n|-|-|-|-|-\n|TFPS (sliding)|**0.448**|**0.380**|**0.395**|**0.276**\n|hard segmentation|0.466|0.395|0.403|0.289\n\nThese results confirm that **preserving partial overlap** between patches improves the model's ability to capture longer-term dependencies and supports more stable expert assignment—both of which are critical for accurate forecasting in TFPS.\n\n> **W2. Discussion on the time-series input.**\n\nWe thank the reviewer for raising this important point. We agree that handling irregularly sampled or sparse time series is crucial for real-world applicability.\n\nWhile TFPS currently assumes regularly sampled sequences, its modular architecture allows for future extension. Specifically:\n- **Time-stamp encoding** and **attention masking** can be integrated into the encoders to directly handle missing or unevenly spaced data;\n- **Interpolation-based preprocessing** can reconstruct uniformly sampled inputs without altering the patching mechanism.\n\nThese strategies would enable consistent patch segmentation while modeling uncertainty introduced by irregular sampling, making TFPS applicable to a broader range of real-world scenarios. We will outline this in the revised Limitations section.\n\n> **W3 \\& Q3. On the choice of $\\alpha$.**\n\nIn practice, only $\\alpha$ and $\\beta$ require tuning, while other parameters are fixed across all experiments and do not need to be adjusted during deployment.\n\n- As shown in $\\underline{\\text{Appendix J}}$, we performed a - sensitivity analysis and selected $\\alpha, \\beta \\in [10^{-4}, 10^{-2}]$, a range chosen to **maintain balanced magnitudes across loss terms and ensure stable optimization**. Within this range, performance remains **robust** across datasets.\n- We set $\\alpha_t = \\alpha_f = 10^{-3}$ to achieve a **reliable trade-off between convergence and regularization**, which also improves reproducibility by eliminating the need for extensive tuning.\n\nWe observed similar robustness for $\\beta$, simplifying practical deployment.\n\n> **W4. On IMP calculation and the innovation.**\n\nWe thank the reviewer for this valuable comment. Based on comparisons with the second-best baseline, TFPS still achieves a relative MSE improvement of over **2.3% on the ETTh2 dataset**, along with competitive results on other benchmarks.\n\nBeyond the absolute MSE values, the key strength of TFPS lies in its architectural design. Unlike traditional models that adopt a Uniform Distribution Modeling approach, TFPS introduces a **divide-and-conquer paradigm** that adaptively decomposes the input distribution into latent sub-patterns and assigns them to specialized experts for targeted modeling. This enables the framework to learn **more flexible projection functions** that capture complex relationships between historical inputs and future predictions.\n\nThis design not only improves forecasting accuracy, but also **enhances robustness to distribution shifts**, **improves interpretability through expert specialization**, and **ensures greater stability across heterogeneous scenarios**. These benefits are often underappreciated or entirely overlooked by monolithic forecasting models.\n\n> **Q2. On the rationale behind linear projection and positional encoding.**\n\n- First, following PatchTST, the linear projection is a crucial step that maps raw patches into a Transformer latent space of dimension $D$. This produces **denoised and semantically enriched embeddings**, which are more robust for downstream modeling.\n- Second, positional encoding is essential, even in the frequency branch, as it introduces **temporal order awareness** prior to FFT. This allows the frequency encoder to distinguish between spectrally similar but temporally misaligned patterns. Empirically, we validate this design with an **ablation study** removing positional encoding:\n\n|Dataset|ETTh1|ETTh2|ETTm1|ETTm2\n|-|-|-|-|-\n|TFPS|**0.448**|**0.380**|**0.395**|**0.276**\n|w/oPosition|0.477|0.398|0.402|0.280\n\nUnlike traditional methods that directly operate on raw FFT values, TFPS **integrates frequency modeling into a learnable latent space**, enabling joint optimization of temporal and spectral semantics, which is an important innovation of our framework.\n\n> **Q4. On temporal dependencies across patches and expert isolation.**\n\nTFPS preserves the ability to model temporal dependencies across patches via two mechanisms:\n- **Time and frequency encoders** process the entire patch sequence using attention and feedforward layers, establishing early-stage temporal interactions before expert routing.\n- After expert inference, outputs from all patches are concatenated and passed through a **shared linear projection**, which acts as a global decoder to fuse inter-patch information.\n\n> **Q5. On dual-domain expert modeling and alignment.**\n\nWe thank the reviewer for the insightful question. To clarify, **TFPS does not assume or enforce alignment between time-domain and frequency-domain expert assignments**.\n\nEach domain operates independently:\n- The time experts captures local, position-sensitive variations (e.g., trends, shifts);\n- The frequency experts captures global, position-invariant structures (e.g., periodicity, harmonics).\n\nFigure 5 visualizes representative inputs assigned to specific **time-domain** experts for interpretability, it does not imply cross-domain alignment.\n\nThis dual-branch design allows the model to leverage **complementary inductive biases**, and both representations are fused only at the final prediction stage.\n\n> **Q6. On the purpose and necessity of KL divergence.**\n\nWe thank the reviewer for the thoughtful question regarding the role of KL divergence in our framework. The KL divergence loss in TFPS is not meant to reconstruct $S$, but to **refine and stabilize subspace assignment** through regularization.\n\n- $S$ is computed from raw latent features and may contain noisy, uncertain affinities—especially early in training.\n- $\\hat{S}$ is a sharpened version that emphasizes high-confidence entries via temperature-based sharpening.\n\nBy minimizing $\\text{KL}(\\hat{S} \\parallel S)$, we:\n\n- Encourage $S$ to concentrate on confident subspace relationships,\n- Suppress noisy or ambiguous affinities,\n- Stabilize expert routing and improve subspace consistency.\n\nThis loss acts as a structural regularizer rather than a reconstruction loss. Importantly, we empirically validate its necessity:\n\n|Dataset|ETTh1|ETTh2|ETTm1|ETTm2\n|-|-|-|-|-\n|TFPS|**0.448**|**0.380**|**0.395**|**0.276**\n|w/o$L_{KL}$|0.457|0.388|0.398|0.282\n\nThese results show consistent performance drops when removing the KL term, supporting its necessity."}, {"review_id": "NCAAkPOUpQ", "reviewer": "Reviewer_Td39", "summary": "The paper thoughtfully tackles the challenges of time series forecasting by proposing a novel architecture that integrates pattern-aware expert modeling. It introduces a distinctive mechanism for identifying pattern-specific subspaces, allowing the model to adaptively allocate specialized experts to different data patterns. The proposed approach is supported by thorough experiments, with the results consistently showcasing strong performance across multiple benchmarks. The authors provide well-organized and comprehensive ablation studies that effectively justify the design choices. The manuscript is clearly written, and the motivation is well-grounded in practical concerns about distribution shifts. While the approach is somewhat complex, the overall method demonstrates solid technical depth and meaningful innovation, contributing valuable insights to the field of time series analysis.", "questions": "1. How does TFPS handle time series with evolving or emerging patterns that were not seen during training?\n2. How sensitive is TFPS to the choice of the number of subspaces (K), and is there a principled strategy for selecting it across different datasets?", "limitations": "The proposed method introduces non-trivial architectural complexity due to its dual-branch structure and subspace-based expert routing, which may hinder practical adoption in resource-constrained settings.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n1. This paper presents an innovative divide-and-conquer strategy that adaptively identifies and decomposes complex patterns in time series data. Through a novel integration of dual-domain encoding and subspace clustering, it effectively addresses intra-series distribution shifts, with the motivation strongly supported by Wasserstein distance–based empirical analysis.\n2. A core contribution is the subspace-based Pattern Identifier, which introduces a refined subspace affinity mechanism for accurate patch-wise clustering and expert routing. This design enhances robustness to distribution shifts while eliminating the need for manual pattern annotation, marking a significant advancement in adaptive time series forecasting.\n\n## Weaknesses:\n1. While the expert modules demonstrate effectiveness, their design as relatively simple MLPs may constrain model expressiveness, particularly when handling complex or highly nonlinear patterns in real-world datasets.", "was_revised": true, "final_justification": "This paper presents a novel and technically sound architecture for time series forecasting that effectively addresses the challenge of intra-series distribution shifts. The core contribution—a pattern-aware expert model that uses subspace clustering for adaptive routing—is both innovative and well-motivated.\n\nMy initial concerns, particularly regarding the expressive capacity of the expert modules, have been fully addressed during the author discussion period. The authors provided sufficient clarification and evidence to convince me that their design choices are well-justified and do not significantly limit the model's effectiveness for its intended applications. Given that all my initial reservations have been resolved, I will keep my initial positive score and recommend acceptance.", "rebuttal": "We sincerely thank the reviewer for the encouraging feedback and for highlighting our contributions in pattern-specific expert modeling, dual-domain clustering, and adaptive subspace affinity design. Below we give a point-by-point response to your concerns and suggestions.\n\n>**W1. On the expressiveness of MLP-based experts.**\n\nWe appreciate the reviewer’s comment. While we explored **CNN-based** and **LSTM-based** alternatives, we found that the original MLP-based experts in TFPS actually yielded better forecasting performance across most datasets.\n\nThis observation highlights that **increased architectural complexity does not always lead to better results**. Specifically:\n\n- **Task alignment**: MLP-based experts better match the patch-level granularity and localized modeling requirements of TFPS.\n- **Training stability**: Their simplicity promotes more stable training and better generalization, especially when combined with the subspace-guided routing mechanism.\n- **Modular extensibility**: Although our current design uses MLPs, TFPS can flexibly accommodate more complex expert architectures when needed.\n\nTFPS’s primary innovation lies in **how it adaptively assigns patches to specialized experts via latent subspaces**, rather than overparameterizing the experts themselves.\n\n|Dataset|ETTh1|ETTh2|ETTm1|ETTm2\n|-|-|-|-|-\n|TFPS|**0.448**|**0.380**|**0.395**|**0.276**\n|CNN\\_Expert|0.464|0.394|0.406|0.280\n|LSTM\\_Expert|0.492|0.396|0.413|0.287\n\n> **Q1. Generalization to emerging or unseen patterns.**\n\nWe thank the reviewer for raising this important and exploratory question. TFPS addresses non-stationarity by routing input patches to specialized experts based on learned subspace affinities. This allows the model to **flexibly capture evolving dynamics and distributional shifts** in real-world time series.\n\nHowever, for entirely novel patterns outside the training distribution, additional adaptation mechanisms may be needed.\n\nWe consider this an important direction for future work. Approaches such as **test-time entropy minimization**, **meta-learning**, or **continual learning** could be integrated to enhance generalization to unseen patterns.\n\nWe will outline this in the revised Limitations section.\n\n> **Q2. On sensitivity to the number of subspaces K.**\n\nWe thank the reviewer for raising this important question. The number of subspaces $K$ determines **the granularity of expert specialization** in TFPS. In our experiments, we observe that **datasets with stronger distribution shifts benefit from larger $K$**, enabling better disentanglement of heterogeneous patterns. On **more stationary datasets, a smaller $K$ is typically sufficient**.\n\nWhile $K$ is currently selected via validation performance, TFPS demonstrates **robustness** across a range of values, and does not require precise tuning to perform well.\n\nAdditionally, we have explored the use of **Dirichlet Process Gaussian Mixture Models** to estimate the number of latent patterns in an **unsupervised** fashion. This ongoing work aims to further automate subspace discovery, and we plan to integrate it into future iterations of the framework."}, {"review_id": "LtIFDlGlic", "reviewer": "Reviewer_k7h4", "summary": "This paper proposes a novel architecture, TFPS, to address the generalization limitations of single-model approaches caused by pattern drift across time series segments. The method introduces pattern-specific expert networks for more accurate and adaptable forecasting. TFPS employs a dual-domain encoder to capture both time-domain and frequency-domain features for a comprehensive understanding of temporal dynamics. It further applies subspace clustering to dynamically identify distinct patterns within segments, which are then modeled by specialized experts. The model is capable of learning multiple predictive functions, and experimental results demonstrate strong performance.", "questions": "see weeknesses.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1、The paper is easy to understand and follow.\n\n2、The motivation is clearly presented, with intuitive and effective visual validations.\n\nWeaknesses：\n1、In Figure 1, the text on the heatmaps is not clear enough, and the boundary lines of the heatmaps appear inconsistent. In Figure 7, some elements overlap, affecting the readability of visual details.\n2、Although the experimental results show great performance, I suggest including more recent baselines. In particular, the results reported in Table 6 do not convincingly demonstrate a significant improvement, especially when compared to some recent methods in 2025 (e.g., Softs).\n3、In Table 9, I am curious about the comparison results on the electricity and traffic datasets. Based on my experience, it is often difficult to achieve similar great performance on these two datasets.\n4、An ablation study on the loss function is necessary to help readers better understand the actual contribution of each component.\n5、The paper incorporates position embedding, but according to recent studies, its effectiveness is often limited. It would be helpful to clarify whether any specific design or adaptation is used here, and ideally provide experimental evidence supporting its effectiveness.", "was_revised": true, "final_justification": "The authors have addressed several concerns through the rebuttal, including improved visual clarity, additional recent baselines, detailed analysis on challenging datasets, and ablations on both the loss function and positional embeddings, which strengthen methodological transparency. While these additions increase confidence in the soundness of the work, the performance gains over the strongest recent methods remain modest, and the overall novelty is incremental, with limited evidence of broader real-world generalization. Balancing these factors, the paper remains technically solid but without sufficient empirical or conceptual strength for acceptance, thus the borderline reject score is maintained.", "rebuttal": "We thank the reviewer for offering the valuable feedback and for recognizing the clarity of our presentation and the motivation design. Below we give a point-by-point response to your concerns and suggestions.\n\n> **W1. Clarity of Figure 1 and Figure 7.**\n\nWe thank the reviewer for this helpful suggestion, which we believe will significantly improve the presentation quality of the paper. In the revised version, we will **enlarge the font size** of heatmap annotations in Figure 1, **ensure consistent boundary line styles**, and reorganize Figure 7 to **eliminate overlapping elements** and improve overall visual clarity.\n\n> **W2. Comparison with recent baselines.**\n\nTo address the reviewer’s suggestion, we include **comparisons with several recent state-of-the-art methods**, including FilterTS, TimeMixer, SOFTS, and, FreTS, using their official implementations. As shown below, TFPS achieves competitive or superior MSE results on **6 out of 7 datasets** and performs particularly well on those with complex temporal dynamics, such as ECL and Weather.\n\nRegarding the results on the Traffic dataset, we believe that SOFTS performs better primarily due to its explicit modeling of **inter-variable dependencies**. As shown in Table 5, the dataset exhibits relatively stable temporal dynamics with limited distribution shift, and its strong channel-wise correlations may favor methods specifically designed to capture such relationships.\n\n||TFPS|FilterTS (2025)|TimeMixer (2024)|SOFTS (2024)|FreTS (2023)\n|-|-|-|-|-|-|\n|ETTh1|**0.448**|0.450|0.452|0.453|0.481\n|ETTh2|**0.380**|0.384|0.383|0.388|0.489\n|ETTm1|**0.395**|0.401|0.403|0.400|0.410\n|ETTm2|**0.276**|0.283|0.283|0.286|0.310\n|Traffic|0.457|0.470|0.483|**0.409**|0.519\n|ECL|**0.183**|0.192|0.190|0.189|0.209\n|Weather|**0.241**|0.247|0.247|0.253|0.256\n\nThese results demonstrate that TFPS maintains **strong performance** under competitive baselines, with consistent top-tier ranking across most datasets.\n\n[1] Wang Y, Liu Y, Duan X, et al. Filterts: Comprehensive frequency filtering for multivariate time series forecasting. AAAI 2025.\n\n[2] Wang S, Wu H, Shi X, et al. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. ICLR, 2024.\n\n[3] Han L, Chen X Y, Ye H J, et al. Softs: Efficient multivariate time series forecasting with series-core fusion. NeurIPS, 2024.\n\n[4] Yi K, Zhang Q, Fan W, et al. Frequency-domain MLPs are more effective learners in time series forecasting. NeurIPS, 2023.\n\n> **W3. Performance on ECL and Traffic datasets in Table 9.**\n\nWe thank the reviewer for this insightful comment and provide further analysis on the ECL and Traffic datasets. The TFPS results reported below are consistent with those in Table 1. As shown below, TFPS **significantly outperforms** several representative DLinear variants augmented with different normalization strategies, confirming its robustness and generalizability on these challenging benchmarks.\n\n|Dataset|TFPS|+SIN|+SAN|+Dis-TS|+NST|+RevIN\n|-|-|-|-|-|-|-\n|Traffic|**0.457**|0.583|0.619|1.220|0.662|0.678\n|ECL|**0.183**|0.204|0.207|0.208|0.209|0.215\n\n> **W4. Ablation study on the loss function.**\n\nWe conduct an ablation study to assess the contribution of each loss component in TFPS. Specifically, we evaluate the effect of removing the core forecasting loss $L_{\\text{MSE}}$ and the two subspace-guided pattern identification losses $L_{PI_t}$ and $L_{PI_f}$.\n\nAs shown below:\n- Removing $L_{\\text{MSE}}$ causes substantial degradation, confirming **its fundamental role in forecasting**,\n- Removing either $L_{PI_t}$ or  $L_{PI_f}$ also results in consistent performance drops, **validating their importance in guiding pattern-aware expert specialization**.\n\n|Dataset|ETTh1|ETTh2|ETTm1|ETTm2\n|-|-|-|-|-\n|TFPS|**0.448**|**0.380**|**0.395**|**0.276**\n|w/o$L_{\\text{MSE}}$|0.710|0.409|0.720|0.322\n|w/o$L_{PI_t}$|0.457|0.389|0.409|0.298\n|w/o$L_{PI_f}$|0.467|0.396|0.428|0.302\n\n> **W5. Effectiveness of positional embeddings.**\n\nWe thank the reviewer for raising this important point. While recent studies [1] have questioned the utility of positional embeddings in time-series models, such observations often refer to **step-wise** or **global fixed embeddings** used in token-level Transformers.\nIn contrast, TFPS adopts **learnable patch-wise** positional embeddings, specifically tailored to its patchified input structure, inspired by the design in PatchTST. This design helps encode relative segment positions and enhances inter-patch distinction.\n\nTo assess their effectiveness, we conduct an ablation study by removing the positional embeddings entirely. As shown below, this leads to consistent performance degradation across all datasets, with up to a 6% increase in MSE on ETTh1. These results confirm that **positional embeddings provide a meaningful inductive bias in TFPS’s patch-level representations**.\n\n|Dataset|ETTh1|ETTh2|ETTm1|ETTm2\n|-|-|-|-|-\n|TFPS|**0.448**|**0.380**|**0.395**|**0.276**\n|w/oPosition|0.477|0.398|0.402|0.280\n\n[1] Zeng A, Chen M, Zhang L, et al. Are transformers effective for time series forecasting? AAAI, 2023."}, {"review_id": "mxpxyOs7az", "reviewer": "Reviewer_n5rj", "summary": "The paper proposes a framework for time series forecasting under patch-level distribution shift by modeling heterogeneous temporal patterns with pattern-specific experts. It introduces three key components: a dual-domain encoder that extracts complementary features from both time and frequency domains, a pattern identifier that clusters patches via subspace affinity and KL-regularized assignment, and a mixture-of-pattern-experts module that routes each patch to specialized experts. Experiments on nine real-world datasets demonstrate the effectiveness of the proposed method over state-of-the-art baselines in long-term forecasting. Ablation studies validate the efficacy of the introduced components in the proposed framework.", "questions": "1) What is the major difference between the proposed clustering strategy and DEC?\n\n2) Why is there a inverse Fast Fourier Transform operation in the frequency branch of Figure 2?\n\n3) Since the authors claims the better performance over state-of-the-art baselines, could the author provide the more evaluations of recent methods (mentioned above)?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1) The idea of pattern specific modeling with the use of MoE allows the model to assign distinct forecasting functions to patches based on learned patterns, leading to improved adaptability under distribution shift.\n\n2) The visualizations provided in the experiments somewhat deliver certain interpretable components with analytical insights.\n\n3) The presentation quality is generally satisfactory.  \n\n\nWeaknesses:\n\n1) Equations (6)-(8) in Section 3.5 of TFPS follow a clustering strategy highly similar to deep embedded clustering [1] (DEC). Both frameworks compute soft cluster assignments, construct sharpened targets via squared normalization, and minimize KL divergence loss between the two distributions. However, the authors claim it is a novel component in line 176 without citing DEC, which overlooks an important prior work that directly inspired this formulation.\n\n2) In Figure 2, there is an inverse Fast Fourier Transform operation in the frequency branch, which is confusing and appears to be erroneous.\n\n3) There are some concerns regarding the contribution of  the proposed dual-domain encoder. The difference between the frequency-domain encoder and time-domain counterpart is simply the attention replaced by the real part of Fourier transform, which is initially designed for efficiency purposes according to the existing literature [2]. The analysis is still in the time-domain without any modeling of frequency components in the complex spectral space. \n\n4) This paper lacks some more recent important baseline comparison, including state-of-the-art time-domain forecasters (e.g., TimeMixer [3]) and frequency-domain forecasters (e.g., FreTS [4], FilterTS [5]). \n\n  \n[1] Unsupervised Deep Embedding for Clustering Analysis, ICML 2016\n\n[2] FNet: Mixing Tokens with Fourier Transforms, NAACL 2022\n\n[3] TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting, ICLR 2024\n\n[4] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting, NeurIPS 2023\n\n[5] FilterTS: Comprehensive Frequency Filtering for Multivariate Time Series Forecasting, AAAI 2025", "was_revised": true, "final_justification": "The lack of citation to DEC and the associated claims raised my concern regarding the novelty of the proposed component. The rebuttal experiments suggest that adding imaginary modeling is indeed necessary, but I am less concerned about this part.  My concern regarding the non-negligible performance gap between the rebuttal results and the original reported values remains. Long-term forecasting is a crowded area, which makes clear positioning and strong empirical support more important.\n\nOverall, I appreciate the authors' efforts and the rebuttal did resolve a few of my concerns, and I am less negative toward this paper.", "rebuttal": "We thank the reviewer for the constructive feedback and the recognition of our MoE-based pattern modeling, visualization insights, and presentation quality. Below we give a point-by-point response to your concerns and suggestions.\n\n> **W1 \\& Q1. Clarification regarding similarity to DEC.**\n\nWe thank the reviewer for pointing out the connection to Deep Embedded Clustering (DEC). We acknowledge its influence and will cite it appropriately in the revised manuscript. We also appreciate the opportunity to clarify the distinctions between our approach and DEC.\n\nWhile our formulation bears some structural similarity to DEC, the **underlying mathematical formulation** and **application objectives** differ substantially:\n\n- **Clustering mechanism**: DEC computes soft cluster assignments using pairwise similarities between embedded features and cluster centroids via a **t-distribution kernel**. In contrast, TFPS derives **subspace affinities** by projecting latent features onto learned subspace bases and measuring projection magnitudes, resulting in a **structure-aware similarity** that is better suited for patch-level modeling in time series.\n- **Downstream usage**: While DEC uses sharpened targets for **unsupervised representation learning**, TFPS leverages them to **guide distribution-specific expert routing** for time series forecasting. This introduces a distinct application context and serves a different modeling purpose.\n\nWe will make these distinctions clearer in Section 3.5 of the revised manuscript.\n\n> **W2 \\& Q2. The rationale for iFFT in Fig 2.**\n\nWe confirm that the iFFT operation in the frequency branch is intentional and serves a functional role.\n\nFollowing the extraction of spectral features via FFT and the identification of pattern-specific subspaces in the frequency domain, we apply iFFT to **reconstruct temporally aligned signals**. This design enables **seamless fusion** between frequency-aware and time-domain features in the final representation, as similarly adopted in recent works like FreTS and FITS.\n\nWe will revise the corresponding description in Section 3.6 to clarify this design and its role more explicitly.\n\n[1] Yi K, Zhang Q, Fan W, et al. Frequency-domain MLPs are more effective learners in time series forecasting. NeurIPS, 2023.\n\n[2] Xu Z, Zeng A, Xu Q. FITS: Modeling Time Series with $10 k $ Parameters. ICLR, 2024.\n\n> **W3. Function of the frequency-domain encoder.**\n\n1. Although FNet employs the Fourier transform as a lightweight replacement for attention in NLP, our frequency encoder serves a distinct purpose: it captures **global periodic patterns and frequency-aware representations** that complement the local dependencies modeled by the time-domain encoder.\n2. Rather than replacing attention, TFPS integrates diverse inductive biases through its dual-branch structure. We focus on the real-valued spectrum to **avoid training instability and unnecessary complexity**, as shown in Table 13.\n3. Empirically, the frequency branch is essential for **identifying distributional heterogeneity** (Table 2), validating its contribution beyond mere architectural variation.\n\n> **W4 \\& Q3. Comparison with recent baselines.**\n\nTo address the reviewer’s suggestion, we include **comparisons with several recent state-of-the-art methods**, including FilterTS, TimeMixer, SOFTS, and, FreTS, using their official implementations. As shown below, TFPS achieves competitive or superior MSE results on **6 out of 7 datasets** and performs particularly well on those with complex temporal dynamics, such as ECL and Weather.\n\nRegarding the results on the Traffic dataset, we believe that SOFTS performs better primarily due to its explicit modeling of **inter-variable dependencies**. As shown in Table 5, the dataset exhibits relatively stable temporal dynamics with limited distribution shift, and its strong channel-wise correlations may favor methods specifically designed to capture such relationships.\n\n||TFPS|FilterTS (2025)|TimeMixer (2024)|SOFTS (2024)|FreTS (2023)\n|-|-|-|-|-|-|\n|ETTh1|**0.448**|0.450|0.452|0.453|0.481\n|ETTh2|**0.380**|0.384|0.383|0.388|0.489\n|ETTm1|**0.395**|0.401|0.403|0.400|0.410\n|ETTm2|**0.276**|0.283|0.283|0.286|0.310\n|Traffic|0.457|0.470|0.483|**0.409**|0.519\n|ECL|**0.183**|0.192|0.190|0.189|0.209\n|Weather|**0.241**|0.247|0.247|0.253|0.256\n\nThese results demonstrate that TFPS maintains **strong performance** under competitive baselines, with consistent top-tier ranking across most datasets.\n\n[1] Wang Y, Liu Y, Duan X, et al. Filterts: Comprehensive frequency filtering for multivariate time series forecasting. AAAI 2025.\n\n[2] Wang S, Wu H, Shi X, et al. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. ICLR, 2024.\n\n[3] Han L, Chen X Y, Ye H J, et al. Softs: Efficient multivariate time series forecasting with series-core fusion. NeurIPS, 2024.\n\n[4] Yi K, Zhang Q, Fan W, et al. Frequency-domain MLPs are more effective learners in time series forecasting. NeurIPS, 2023."}]

Author rebuttal:

Meta-review: This paper proposes a dual-domain forecasting framework that integrates temporal and frequency representations with a subspace-based mixture-of-experts (MoE) design. The core idea is to assign patch-specific forecasting functions adaptively, aiming to better handle distribution shifts in multivariate time series forecasting task.

Strengths:

Well-motivated and clearly written, with intuitive visualizations and generally strong presentation quality.

The idea of pattern-specific modeling with MoE is appealing and has the potential to inspire follow-up work on heterogeneous time series forecasting.

Extensive experiments on nine public datasets and multiple baselines, with ablation studies and interpretability analysis, provide a relatively thorough empirical validation.

Weakness:

Practicality: the framework introduces a large number of hyperparameters, some hard-coded and heuristic, raising concerns about robustness and real-world usability. For example, the pre-defined patch size may impede its generalization capability. 

The reviewers raised concerns about the originality of some components, the clarity of the design choices (e.g., segmentation and clustering), missing baselines, and practical complexity due to many hyperparameters. While these are valid points, the rebuttal improved clarity, added experiments, and included additional citations, which alleviates the concerns.

Taking these factors into account, while questions on practicality, the paper provides a well-executed empirical study with reproducible code, a promising design idea for heterogeneous time series forecasting, and good presentation quality. I believe it will be of interest to the community, stimulate further research, and recommend a borderline accept.
Decision: Accept (poster)