GTM: A General Time-series Model for Enhanced Representation Learning of Time-Series data
Authors: Cheng HE
Xu Huang
Gangwei Jiang
Zhaoyi Li
Defu Lian
Hong Xie
Enhong Chen
xijie liang
Zhengzengrong
Patrick Lee
Venue: ICLR 2026 Poster (2026)
Keywords: Time series
Foundation Model
Representation learning
Pre-training strategy
Primary area: learning on time series and dynamical systems

Abstract: Despite recent progress in time-series foundation models, challenges persist in improving representation learning and adapting to diverse downstream tasks. We introduce a General Time-series Model (GTM), which advances representation learning via a novel frequency-domain attention mechanism that captures time-granularity-aware features, an aspect underexplored in prior research. We further propose a novel pre-training strategy that unifies reconstruction and autoregressive objectives through a hybrid masking mechanism. Our pre-training strategy, combined with 2D positional encoding and span shuffling, enhances the robustness and generalization of representations. GTM is established as the first generative-task-agnostic model for time-series analysis, enabling seamless adaptation to various generative tasks without any task-specific modifications. Extensive experiments demonstrate that GTM consistently outperforms SOTA models on various generative tasks and achieves strong classification results with minimal adaptation. Furthermore, GTM exhibits clear scaling behavior, with accuracy improving as model size and pre-training data increase.

Published as a conference paper at ICLR 2026
GTM: A GENERAL TIME-SERIES MODEL FOR EN- HANCED REPRESENTATION LEARNING OF TIME- SERIES DATA
Cheng Hel2, Xu Huang Gangwei Jiangh?
Zhaoyi Li Defu Lian
Hong Xie Enhong Chen Xijie Liang',
Zengrong Zheng', Patrick P. C. Lee
University of Science and Technology of China
2 State Key Laboratory of Cognitive Intelligence Shanghai Black Wing Asset Management 4 Di-Matrix Information Technology TTee Chinese University Hong Kong
(cheng he, xuhuanges, gwjiang, 1izhaoyi777] @mailmmll........ eeutcc.edu.com (liandefu, hongx87, cheneh @ustu.. edu.cr 1xjie@bl
arjje@blackee wwwwingawsaaeee ssseist........................s com, zengrong zhengêdi -at rix. peleeê cse. cuhk. edku.edu.h. hk
ABSTRACT
Despite recent progress in time-series foundation models, challenges persist in improving representation learning and adapting to diverse downstream tasks. We introduce General Time-series Model (GTM), which advances representation learning via novel frequency-dome -omain attention mechanism that captures time- granularity- aware features, aspect underexplored in prior research. We further propose novel pre-training strategy that unifies reconstruction and autoregressive objectives through hybrid masking mechanism. Our pre-training strategy, com- bined with 2D positional encoding and span shuffling, enhances the robustness and generalization of representations. GTM is established as the first generative-task agnostic model for time-series analysis, enabling seamless adaptation to various generative tasks without any task-specific modifications. Extensive experiments demonstrate that GTM consistently outperforms SOTA models various gen- erative tasks and achieves strong classification results with minimal adaptation. Furthermore, GTM exhibits clear scaling behavior, with accuracy improving model size and pre-training data increase.
INTRODUCTION
Foundation Models (FMs) have achieved remarkable success, owing to their ability to learn rich representations from large-scale data and transfer effectively to diverse downstream tasks (Bommasani al.) (021). However, extending these benefits to Time Series (TS) analysis remains challenging due to two major obstacles: (i) limited expressiveness of scalar, temporally indexed sequences, and (ii)
wide heterogeneity of downstream tasks.
Recent advances in Time-Series Foundation Models (TSFMs) fall into two main categories: (i) Forecasting-only FMs, which are tailored for forecasting tasks and leverage temporal features such as lag covariates and adaptive patches (Rasul et al. 2023 Fkambaram et al.) 2024 Shi et al. 2024); and (ii) Multi-task FMs, which employ autoregressive modeling, masked autoencoders. and contrastive learning to support multi-task adaptation (Liu et al., 2024b Zhang et al.) 20224) Dong al. 2024 Goswami al., 2024). While these models have improved feature extraction and generalization, they still require task-specific changes, especially for generative tasks, and rarely explore new perspectives beyond typical time-domain features.
In multi-task TS analysis, downstream tasks are generally categorized as either generative (c.g.. forecasting, imputation, anomaly detection), which requires modeling the underlying data distribution. or discriminative (e.g., classification), which maps TS inputs to categorical labels. Although recent
"Corresponding authors

---

Published as a conference paper at ICLR 2026
(a) 4-8 amp
(b) 5-min amp
(c) 1-hour amp
(d) 1-day amp
(c) 4-8 phase
(f) 5-min phase
(g) 1-hour phase
(h) 1-day phase
Figure 1: Amplitude- and phase-frequency joint distributions for TS data with varying granularities.
TSFMs can handle multiple generative tasks (Liu et al. 2024b Zhang et al. 2024 or adapt across both categories of tasks (Dong al. 20244 Gao al., (0224, they typically require modifications the token, pre-training, or projection-header levels to achieve such flexibility. To our knowledge, no TSFM can adapt to generative tasks truly task-agnostic manner without such changes.
In this work, we present comprehensive analysis of large-scale, multi-domain TS data using Fast Fourier Transform and 2D Kernel Density Estimation to estimate the joint probability distributions of umplitude-frequeee
and phase-frequene at various temporal granularities. As shown in Figure D these distributions differ significantly across time granularities, highlighting critical but unexplored dimension in TS representation learning. This empirical observation directly informs our model design, motivating the development frequency-domain network modules tailored to capture such multi-granularity representations.
Building on these insights, we propose a General Time Series Model (GTM), which explicitly incorporates time granularity adaptation to generative tasks, we introduce a novel pre-training framework that unifies reconstruction and autoregressive objectives via hybrid masking strategy. Our framework combines random and controlled consecutive tail masking. 2D positional encoding, and span shuffling. This design enables GTM learn robust and generalizable representations, allowing seamless adaptation to wide range of generative tasks without any task-specific modifications.
key factor for robust TS representation. To enable effective
Our main contributions
We design GTM, a TSFM built with novel Fourier attention mechanism to capture distributional differences across temporal granularities, substantially improving TS representation quality.
We propose a unified pre-training framework that integrates hybrid masking, 2D positional encod- ing, and span shuffling, jointly optimizing reconstruction and autoregressive objectives to enhance robustness and generalizability. This establishes GTM as the first seeerratvvettsk-aannnee
TSFM.
Extensive experiments demonstrate that GTM consistently outperforms state-of-the-ar baselines across a variety of benchmarks, offering scalable and cost-effective performance suitable for industrial applications.
The code and implementation details available at:
RELATED WORK
We focus TSEMs trained from scratch. Additional literature survey be found in Section B.1
Early Attempts. Early FMs adapted techniques for TS tasks, thereby laying the foundation for TSFMs. For example, TimesNet (Wu et al. 2023) transforms 1D time series into 2D feature maps
2

---

Published as a conference paper at ICLR 2026
using CNNs to capture multi- meniiippriodicctty patterns, while adding task-specific projection headers for diverse generative tasks. Similarly, PatchTST (Nie et al. [203) enhances pre-trained Transformers for forecasting by learning channel-independee inter-patch representations. Despite their progress, these models fall short of TSFM standards due to the lack of large-scale pre-training and effective adaptation across diverse tasks.
TSFMs for Forecasting.
Lag-Llama (Rasul et al. 2023) and GPHT (Liu et al. 2024c) both utilize decoder-only architectures to model temporal dependencies, with Lag-Llama incorporating lagged covariates and timestamp features, while GPHT employs hierarchical backbone for long-term forecasting across arbitrary time horizons. TimesFM (Das et al.)
(2024) pushes the boundaries by using stacked Transformer pretrained on (1100)) data points to learn domain-invariin representations. Other works, such as GPD Yang et al. 2024) and UTSD (Ma et al. 2024), explore the use of diffusion models to capture cross-domain correlations and improve robustness across diverse forecasting tasks. MOIRAI (Woo et al., 2024) and TTMs (Ekambaram et al., 2024) focus on multivariate time series forecasting, with MOIRAI tackling cross-frequency learning through a masked Transformer architecture and TTMs emphasizing the learning of cross-channel correlations. Finally, TIME-MOE (Shi et al. 2024) introduces a MOE design that offers flexibility and supports multi-resolution forecasting. Despite these advancements, most models primarily focus on modeling temporal dependencies and do not fully exploit richer, multi-domain information (e.g., frequency-domain features) that could enhance the ability to address more complex forecasting tasks.
Multi-task TSFMs. Recent work has greatly advanced the adaptability of TSFMs for diverse tasks. UP2ME (hang et al., 2024) combines Masked AutoEncoder pretraining with Graph Transformer fine-tuning for flexible adaptation. Timer (Liu et al. 2024b) adopts autoregressive, causal-attention framework, pretraining on unified sequences to improve generalization. For discriminative tasks,
TimeSiam (Dong et al.] (024)) applies Siamese contrastive learning. while LPTM (Kamarthi & Prakash 2023) fuses Transformer and GRU modules to extract robust tokenized representations from heterogeneous data. UniTS (Gao al. 2024) introduces task tokenization within a dual-tower Transformer, supporting both generative and classification tasks. Overall, masked reconstruction and contrastive learning are oriented towards representation learning by capturing intra-sequence patterns and inter-sequence similarities, respectively, with downstream adaptation typically achieved by replacing the projection head. Predictive pretraining, on the other hand, focuses on modeling long-term temporal dependencies forecast multi-step future outcomes, making it particularly suited to downstream predictive tasks. However, due to the absence of a unified pretraining objective, these models require task-specific modifications the tokenization (e.g.. UniTS), pre-training strategy (e.g.. Timer), or model level (e.g., UP2ME, TimeSiam, LPTM) to achieve strong downstream performance.
METHOD
3.1 DESIGN OVERVIEW
We denote TS by X [Xe. : e [C],t e [T]]. where C and T are the number of variables and timestamps, respectively. We pre-train our model, GTM, from scratch on the large-scale UTSD-12G dataset (Liu et al. 2024b), which covers diverse application domains. Figure shows the overall architecture:
Input Embedding: We apply Reversible Instance Normalization (imm et al. 2022). Channel Independence (CI), patching (Nie al. 2023), and masking (Du et al. (022) to transform raw TS data into univariate masked token sequences. Each token is further enriched with linear and positional embeddings before entering the backbone.
N-stack Decoder-only Backbone: GTM uses a decoder-only Transformer backbone to generate outputs autoregressively. To capture both temporal and frequency-doman information, we retain temporal self-attention module and design the Fourier attention module (details Section 3.2
Output Projection: A unified linear projection layer, followed by instance denormalization, produces outputs autoregressively for both pretraining and downstream tasks.
3

---

Published as a conference paper at ICLR 2026
Predicted Output
Freq Becala Natuondio
Firq Ioseee Panrartt
Tree Desuis Patera(i)
Frop Dunall Pettomph
Frey Beeeds PatkraM
Froy fleest
Xout M
Projection
Output
Linear Proj, +los. Denorm
Value vector Freq. domain
Softmax
@
vil
Query Yector
out 2 Riaaa
Learnable
N-stack
Pourler attention
Key vectors
M
Decoder-only Backbone
FFT
M45
Emaedding
Temporal Deedde))
2-kvel Ponitiona Encading
Madking Recalts
Xsn Rauuush
Lincar Projection
Imput Emhedding
Patching-Masking
Masked Infs.
General Time series Medel
15
P4
Xp e gl
Patches
24
822
Cl-Instaner Nerm
Level Pos.
R
Time Series Data
Masking Procrasing
Masked AutoRegresdon
Tempsral atteraaia mask full all. causall att.
with 2D positional encoddingg
Figure 2: GTM model architecture for pre-training. Left: input embedding, N-stack Transformer backbone, and output projection, to generate reconstruction results autoregressively. Lower right: Patching and masking using both full and causal attention mechanisms, adapted from the NLP field and optimized for TS pre-training. Upper right: A Fourier attention module designed to learn representation of TS data with varying granularities. Pseudo-code of GTM architecture and pre-training strategy is provided in Algorithm 1 in Appendix B.2.3
TS data pass through three key components:
3.2 N-STACK DECODER-ONLX BACKBONE
We design an N-stack decoder-only backbone that jointly models temporal and frequency patterns TS data. Each decoder block consists of standard temporal self-attention layer followed by a Fourier attention module, which incorporates frequency-domai information via FFT. To enable granularity
frequency modeling, we represent time granularity as a quintuple: (day, hour, minute, second, millisecond). For example, the ETTm dataset (Wu et al.) 2021) is encoded as [0, O, 15, 0, 0]. We also introduce five learnable key embeddings, each for typical granularity. Attention weights computed by taking the dot product of the query with each key, followed by softmax normalization, and used to combine five corresponding frequency learning matrices. In addition, a global frequency learning module operates in parallel capture patterns tied to any specific time granularity. This module is always active and complements the granularity-specife modules.
Temporal Fourier Attention. Given the embedded input H E R.Nooratt wheee Niotal the total number of masked and reconstructed patches and D is the embedding dimension-tha temporal self-attention module computes
H TemAAuOu = Self Attentionn(() KA, V.) E NeoaaID
where Q; Hin W K, H W and Vn H WV linear projections with learnable weight matrices. Next, column-wise FFT transforms each temporal patch into the frequency domain:
HEFT FFT ( TemAnOaa).
(2)
To capture frequency-speciffc patterns, we design six frequency-domain modules: five low-rank modules for five granularities, parameterized by (A;, B.J and one global module with full connection Wian- The time granularity is encoded as a quintuple and embedded into a query vector q qw?, Five learnable key vectors K2 represent the corresponding granularities. Fourier
q
attention weights are computed
SoftMax
and are subsequently used to aggregate Vdzx
the outputs of the five low-rank modules:
H HoorierAu
(A, + WAr i=1
(3)
4

---

Published as a conference paper at ICLR 2026
The final output is obtained by applying the inverse FFT:
Hour 1FFT FouricrAn) E IR X
(4)
This process is repeated for N stacked decoder-only layers, with each layer taking the output of the previous layer input:
(x) out = GTM_Deccoer
(H H (m) H (1-11)
(5)
out
where n E [N] and H Hnn
Output Projection: A unified linear projection maps the backbone output to patch-level predictions:
Xott LinProj H
out
(6)
This enables GTM support various generative tasks without further architectural changes.
3.3 PRE-TRAINING FRAMEWORK
We divide each time series into overlapping patches using CI and patching (Nie et al.) (023). For each variable, the series is split into overlapping windows of length and stride T. X; ix. then divided into No patches. Inspired by GLM (Du et al. 2022). we use
a hybrid masking strategy:
Randomly sample l patch spans (each consecutive group of patches).
Randomly permute the sampled spans, and pad learnable vectors [START] and [END] tokens to form input and target sequences.
Replace each span with single [MASK] token create corrupted input.
Apply controlled proportion of consecutive [MASK] tokens to at the tail.
Specifically, we introduce a hyperparameter pred_ratee ratio to flexibly control the probability of applying consecutive tail masking. As an example, for each training instance, a random variable ((,0 1) is sampled and a corrupted input can be constructed as follows:
X
X No h. [MASK] ..... [MASK]] if S ratio
XP
(andomMask(Xxi)
otherwise
where 1: |aNpl and representing the tail masking ratio. This approach unifies mask reconstruc- tion and autoregressive forecasting under single pre- wre-training objective, enabling the model to learn both general representations and future prediction capabilities. Based on this strategy, get:
X X Perpe [5]. S (1) [S]. S.
Y = [S. [.,..., Sa(e [4]]
(8)
where X Pap app denotes the masked input, a--) is a random permutation. The pre-training objective is autoregressively reconstruct all masked patches by minimizing MSE:
P o(uo = II P oouu X Sa()))
(9)
1
LOSS
Σ Xoout - y
(10)
Before feeding the backbone, apply trainable linear embedding and 2D positional encoding (Du et al. 2022), ensure that the backbone model is aware of the length of the masked span when
generating output patches:
H. W Wemb Xin WID W2D
(11)
We employ full attention for masked reconstruction and causal attention for autoregressive generation, effectively preventing information leakage.
5

---

Published as a conference paper at ICLR 2026
Table Avg. MSE & MAE forecasting results. Results are averaged over varying prediction lengths. Bold & underline indicate the best & 2nd-best results respectively. See full results in Table T8
Models
GPT4TS
UniTS-PMIT
TIMEE
PaccTTT
TimesNet
FEDiormer Autoformer Informer
dataxet
[SS
MSEE MAE
MSE]
MAE
MSE
MAT
MSE
MAE
MAE|MSE
MSE
MAE
MSE]
MAE]
MATE MSE]
[MA]] MSE
MAE
ETTh1
404
.29
.427
426
.461
.444
.402
413
.444
.458
.450
.422
.457
428
.43
.473
.476
1.040
.795
ETTml
339
.383
.350
.382
.400
446
.357
378
.382
515
.493
.91
.734
Weather
.25
266
.377
.270
.243
.273
.234
225
.263
.299
.287
.246
-300
.332
375
.35
.379
.334
.548
Traffic
.385
266
414
.294
.494
.313
.385
.90
.263
620
.336
.433
295
.603
.322
.616
.383
764
.466
Electricity
161
254
167
266
184
2282
158
159
.252
192
295
166
263
2007
321
2.14
3326
311
.99
3.4 FINE-TUNING FOR DOWNSTREAM TASKS
Due to its unified architecture and pre-training strategy, GTM yields robust representations and supports wide range of generative downstream tasks without task-specific modifications, aside from minor preprocessing (e.g.. removing masking and 2D positional encoding). This versatility enables GTM to deliver high-precision results across diverse time series applications (Section 44).
4 EXPERIMENTS
We extensively evaluate GTM primarily on generative tasks, while also including discriminative tasks, to demonstrate its advanced representation learning and seamless multi-task adaptability. Across tasks, GTM is compared with state-of-the-art baselines (see Appendix [.2.2p. We further analyze the benefits of large-scate pre- pre-training, generalization in zero-shot and few-shot settings, and perform ablation and scalability studies. Finally, we assess the computational overhead of GTM's key components and its overall model efficiency. Additional results on hyperparameter sensitivity analysis are provided Appendix [B..77 and B.3.8 confirming GT'''s cost-effectiveness and industrial applicability.
4.1 DATASETS DESCRIPTION
We use the large-scale public TS dataset, UTSD-12G, for pre-training, ensuring no downstream task-related data is included to prevent leakage. We conduct experiments on five widely used public datasets for forecasting and imputation (Wu et al. (02T), five popular labeled datasets for anomaly
detection (Su et al. 2019 Hundman et al. 20188) Mathur & Tippenhauer] 2016; Abdulaal et al., 2221). and ten standard datasets for classification (Bagnall et al. 2018). The detailed statistics of these public datasets are provided in Appendix B.2.
4.2 LONG-TERM FORECASTING
For long-term forecasting, select representative baselines and cite their results respectively. These SOTA models include the LLM-enhanced model GPT4TS (Zhou et al. 2023), the multi-task TSFM UniTS-PMT (Gao al. 2024), the task-specific TSFM TTME. TimesNet (Ekambaram al. 2024
Wu et al.
2023), the Transformer-based models PatchTST, FEDformer, Autoformer, Informer Nie al. 2023 Zhou et al.) 2022, Wu et al. 202T] Zhou et al., (202). and the MLP-based model Dlinear 2023). We focus on baselines that align closely with our experimental settings, excluding models that require pre-training and fine-tuning on the same datasets for downstream tasks. The long-term forecasting lengths include T E [96, 192, 336, 720) time points. We use MSE and MAE as evaluation metrics. Notably, GTM directly utilizes a pre-trained model without any modifications. As shown in Table [ GTM outperforms all SOTA models, achieving the highest total number of best- and bedbbes--llace results across tests with varying forecasting lengths, while PatchTST ranks second. Full results, additional baseline comparisons with SOTA TSFMs Sundial and Time-MOR MOE, and error bar analysis with 95% confidence intervals and more experiments on extended challenging, real-world datasets provided in Appendix B.3.1
(Zeng et al.

---

Published as a conference paper at ICLR 2026
Table Avg. MSE & MAE results of Imputation. Results are averaged over varying data missing ratios at the time-point level. Bold and underline denote the best and the 2nd-best results, respectively. Full results are listed in Table 22
Models
GTM
GPT4TS
TimesNet
PatchTST
DLinear
Fedformer
Informer
Dataset
MSE
MAE|
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE|
MSE
MAE
MSE
MAE
ETThl
0.053
0.152 0.069
0.173
0.078
0.187
0.115
0.224
0.201 0.306
0.117 0.246 0,161
0.279
ETTml
0.021
0.096 0.028
0.105
0.027 0.107
0.047
0.140
0.093 0.206
0.062
0.177
0.071
0.188
weather
0.030
0.054
0.031
0.056
0.030
0.054
0.060
0.144
0.052
0.110
0.099
0.203
0.045
0.104
Electricity
0.086
0.202
0.099 0.207
0.092 0.210
0.072
0.183 0.132 0.260 0.130
0.259
0.222
0.328
Table 3: The Fl scores for the anomaly detection tasks.
Models
GTM
UP2ME GPT4TS TimesNet
PatchTST FEDformer DLincar Autoformer
Informer
Dataset
F1(%)
FI(%)
F1(%)
F1(%)
F1(%)
F1(%)
FI(%)
F1(%)
F1(%)
MSL
82,53
82.45
81.84
78.70
78.57
84.88
79.05
84.06
SMAP
77.57
72.88
69.39
68.82
70.76
69.26
71.12
69.92
SWaT
94.78
93.85
94.23
93.02
85.72
93.19
87.52
92.74
81.43
SMD
85.47
83.31
86.89
84.61
84.62
85.08
77.10
85.11
81.65
PSM
95.43
97.16
97.13
97.34
96.08
97.23
93.55
93.29
77.10
Average
87.01
86.72
85.24
82.79
84.97
82.46
84.26
78.83
4.3
IMPUTATION
We use the same publicly available datasets in forecasting tasks and follow the protocol proposed by (Zhou al. 2023) for imputation tasks. To align with benchmark settings, we apply point-wise missing ratios for interpolation, and directly use pre-trained model for fine-tuning, only omitting the patching process. The point-wise imputation baselines include GPT4TS, TimesNet, PStchTST, FEDformer, Informer, and Dlinear. We conduct the task with varying missing data ratios of (12.5% 25%, 37.8 .5,, 50%} at the time-point level. Table T demonstrates that, preprocessing, GTM achieves significant performance improvements. Compared to the second-best model, GTM gets 23.1 % reduction in MSE, 12. % in MAE for ETTh1 data, and 25.0% reduction in MSE, 8.6% in MAE for ETTml data. More details are in Appendix B.3.2
without patch
4.4 ANOMALY DETECTION
For anomaly detection, we fine-tune the pre-trained GTM model in a self- self-super manner via data reconstruction, without any task-specific modifications. Following standard approach (Xu et al. 2018), points with reconstruction errors above a threshold are labeled as anomalies. We compare GTM against baselines. including the multi-task TSFMs (UP2ME, TimesNet), the LLM-enhanced model (GPT4TS), transformer- models TSatchTST, FEDformer, Autoformer, Informer), and the MI.P-based model (DLinear). As shown in Table 3 GTM achieves the highest F1 score across all baselines, with improvements ranging from 0.33% (over GPT4TS) to 10.38% (over Informer). We also report results the TSB-AD datasets, using various widely used measures (Liu & Paparrizos)
2024). along with a broad coverage of TSFMs' test results. See Appendix B.3.3 for details.
4.5 CLASSIFICATION
Although GTM is designed as a generative-task-agnone foundation model, it can be extended to discriminative tasks like classification. As outlined in Section[] [ we adapt only the output projection layer to map inputs to categorical labels, leaving the rest of the model unchanged. Following this approach, we fine-tune preitrained GTM 10 widely-used classification datasets (Bagnall et al.) 2018), with accuracy as the evaluation metric. As shown Table GTM achieves the most best-(5) and second-best(4, results compared SOTA multi-task TSFMs.
7

---

Published as a conference paper at ICLR 2026
Table 4: The Accuracy results of Classification tasks.
Datasct/Model
GTM
UNITS-SUP
UNITS-PMT
GPT4TS
TimesNet
iTTrasffrmer
EahanolConcentratiin
34.2
1
34.2
35.7
18.1
FaceDetectiio
69.9
65.4
58
69.2
68.6
66.3
Handwriting
34.8
/
/
32.7
32.1
24.2
Heartbeat
77.5
63.9
65.4
77.2
75.6
Japanese Vowels
92.I
92.2
90.3
98.6
98.4
96.6
PEMS-SF
88.4
83.2
82.7
87.9
89.6
87.9
SellRegulationSCPP
92.3
/
93.2
91.8
90.2
SellRegulationSCP2
60
48.9
$7.2
59.4
57.2
54.4
SpokenArraiieigitss
99.2
96.8
95.5
99.2
U UWaveGesturel.irreg
89.3
82.2
85.3
88.1
85.3
85.9
Best Count
5
0
2
Table 5: Zero-shot capability (MSE) of GTM compared to SOTA TSFMs.
Dataset
GTM
TIMER-IB
MOIRAI-S
MOMENT
TimesFM
CHRONOS-SI
ETThl 0.407
0,438
0,441
0.674
0.414 0.354
0.571
ETTml
0.593
0.690
0.562
0.670
0.632
Weather
0.172
0.181
0.195
0.255
Electricity
0.187
0.192
0.212
0.744
Traffic
0.542
0.458
0.616
1.293
Average
0.380
0.392
0.405
0.727
4.6 EFFECTIVENESS OF PRE-TRAINING
By pre-training on large-scale TS data across multiple temporal granularities, GTM learns richer, more diverse patterns. We first demonstrate the effectiveness of pre-training by evaluating GTM's generalization in zero-shot and few-shot settings. Table shows that, compared to 5 SOTA TSFMs:
Timer, MOIRAI-S Woo et al. 2024). MOMENTTGGswamm et al. 2024). TimesFM(Das et al. 2024). and Chronos-SI S1 Annari et al. 2024). GTM ranks first on average MSE across 5 datasets with forecasting length of 96 in zero-shot. In few-shot testing, Figure 3 shows that GTT outperforms TimesFM across 4 forecasting lengths ETTml data, achieving better performance with only 10% of the data for fine-tuning, improving results with the largest MSE reduction of 7.53%.
0.5
0.5
GTM
TimesFM
.004-444419
GTM complete GTM. advvoced I GTM base
.397
0.4
339-348..49
.88.3887.39
MSR
.145
20.3
.225-231.333
.161.163.165
96
192
336
720
ETThl
ETTml
Weather
Traffic
Electricity
Forecast
Datasets
Figure GTM VS. TimesFM in few-shot on Figure 4: Average results of long-term forecasting ETTml dataset, 10% samples for fine-tuning.
in ablation test.
We also compare the fine-tuned GTM pre-trained on UTSD datasets with the baseline GTM, which trained directly on task-specific datasets with random initialization. This further highlights the benefits of pre-treee pre-training across various tasks. Tables [ models across all datasets, covering forecasting tasks with varying prediction lengths and imputation tasks with different missing data ratios. The results show that, for forecasting, the fine-tuned GTM consistently outperforms the baseline GTM in every comparison. of 0.5%-7.88 7.8% 7.8% and in MAE of 0.8%-8.09. Similarly, for imputation, the fine-tuned GTM also outperforms the baseline GTM, achieving MSE reductions of 1.2% 11.7% and MAE reductions of 0.5% 0.5%%14.24 14.2%. More details are provided in Appendix [B..4] For anomaly detection, Table [ shows that with pre-trate pre-training, the fine-tuned GTM model achieves performance gains across all test datasets, with average increase of 1. 1.2% in F1-score compared to the baseline GTM model.
and 7n present the average performance of both
achieves reductions in MSE
8

---

Published as a conference paper at ICLR 2026
Table Avg. results of forecasting results com- pared with GTM model w/o pre-train. Table 24 shows full results in Appendix B.3.4
Table 7: Avg. Imputation results compared with GTM model without pre-training. Table25 in Appendix shows the full results.
Models
GTM
GTM no pretrain
Models
GTM
|GTM no pretrain
dataset
MSE
MAE|
MSE
MAE
dataset
MSE
MAE
MSE
MAE
ETThl
0.404
0.429 0.435
0,447
ETThl
0.053 0.152 0.055
0.156
ETTml
0.339
0.376 0.351
0.389
ETTml
0.021
0.096 0.023
0.100
Weather
0.225
0.266 0.244
0.289
Weather
0.030
0.054
0.034
0.063
Traffic
0.385
0.266 0.387
0.268
Electricity
0.086
0.202 0.087
0.203
Electricity
0.161 0.254 0.163
0.256
0.44
Table 8: Anomaly detection results compared with GTM model without pre-training.
dim-256
dim--512 dim-7768
0.43
Models
GTM
GTM no pretrain
MSE
dataset
F1(%)
F1(%)
MSL
82.53
81.92
0.41
SMAP
77.57
76,48
SWaT
94.78
94,66
SMD
85.47
82.11
0.40
PSM
95.43
95.42
Model Layers
Average
87.15(+1.2%)
86.11
Figure 5: Model scalability analysis (ETTh1).
4.7 ABLATION TESTS
We conduct ablation experiments on long-term forecasting tasks across different prediction lengths to evaluate the key components in the GTM model. We a baseline version of the GTM model without the frequency-domain analysis module and compare it with an advanced version that lacks the time-graad taaee-rramlaarty--aa -wware modules. By also comparing both with the complete GTM model, we gain insights into the impact of these key design elements.
Figure shows the average forecasting results for each dataset. The complete GTM model outper- forms all other models in every test. The advanced GTM model ranks second. This demonstrates that combining temporal and frequency-domain analysis, especially time granularity enables the GTM model to effectively learn distribution representations from TS datasets with varying time granularities. More details of ablation tests are given in Appendix B33.5
modules,
4.8 SCALABILITY ANALYSIS
FMs generally adhere to scaling laws, where their accuracy and capabilities scale predictably with both model size and training data (Kaplan et al. 2020). This is crucial for FM design and deployment. To explore the scalability of GTM, we pre-train the model with increasing model size (layers and dimensions) and data size, and conduct forecasting tests on various downstream tasks. Figure 5 shows the average forecasting results on the ETThl dataset for various forecasting lengths, including T E (96, 192, 336, 720} time points, using pre-traine trained models with different numbers of layers and embedding dimensions. The results indicate that GTM follows scaling laws, achieving better MSE with deeper and wider models. However, when the model depth is insufficient, increasing the width (embedding dimension) may not improve performance.
We also pre-trained GTM on different scales of the UTSD dataset and evaluated its forecasting performance for various forecasting lengths the ETThi and Weather datasets with fine-tuning. Figure 6 in Appendix B.3.6) shows that GTM performs better with larger pre-training datasets, as evidenced by the average MSE results, consistent with the expected data scaling laws.
4.9 COMPUTATIONAL OVERHEAD AND EFFICIENCY ANALYSIS
We compare GTM with four reproduced baseline models, including three TSFMs: Time-MORE MOEess) GPT-2(6)-768, 7668,, and one deep learning model: FEDformer-766 in terms of model size

---

Published as a conference paper at ICLR 2026
Table Comparison of model parameters and efficiency. Training and inference speeds are reported in seconds per iteration. An iteration is a full warkdd pass over a batch for training and forward pass for inference. All timings exclude data loading and logging overheads.
Model
Parameter
Training Speed
Inference Speed
Training Mem
Inference Mem
GTM
35.73M
0.290s/iter
0.165w/iter
8324.00MB
1250.00MB
Time- )
50.00M
0.840s/iter
0.095s/iter
1812.48MB
226. 206.
GPT-2(6)-768
82.28M
0. 1.104s/tter
0.054s/iter
5230.00MB
2566.00MB
FEDformer-768
30.75M
0.467s/iter
0. 0.172s/iter
9535.00MB
1880. 19MB
TimesNet-768
42.21M
1.849//iter
0.547s/iter
35871.00MB
1904.18MB
Table 10: Analysis of model inference latency and computational overhead in critical modules. F.A. denotes Fourier Attention module.
Table 11: Analysis of model inference latency across different frequency modules.
Low-rank modules
Inference
F.A.
Channel
Inference (s/item)
FFT+iFFT F.A.
(s/item)
(s/item)
GPU
Channel
(s/item)
(s/item)
0.030
0.020
1
0.043
0.0007
0.033
7
0.030
0.020
A100
0.044
0.0007
0.034
1
0.060
0.049
862
0.142
0.0009
0.103
10
7
0.061
0.050
0.041
0.0007
0.031
1
0.092
0.080
RTX4090
0.041
0.0007
0.031
20
7
0.094
0.081
862
0.144
0.0009
0.107
and efficiency. As shown in Table GTM achieves suitable trade-offs for industrial deployment: ranks second in parameter size(35.7M) training speed(0.290ssitt for batchsize 128), and inference meemory(1.25BB) and retains competitive performance in inference speed(0. speed(0.165s/tter) and training memory(8.32GB) demonstrating both efficiency and applicability for real-time deployment.
We further break down the computational overhead of the Fourier Attention module and FFTAFFT operations. Table 10 presents latency measured both A100 and RTX4090 GPUs. For univariate data (1440 input points, 96 prediction length). GTM achieves a total inference latency of just 0.043s/item, with FFT/FFFT and Fourier Attention modules introducing only marginal overhead. Similar results are observed for the multivariate case(ETT and Traffic data), confirming GTM's low-latency and capable for sub-second real-time streaming applications. We provide more model efficiency scale analysis with significantly longer prediction lengths in Appendix B.3.8
Finally, we evaluate how the number of low-rank modules in Fourier Attention affects inference latency. Table [T] shows that increasing the number of modules provides finer- finer-grained distribution representation across temporal granularities, with only gradual, sub-linear rise processing time. Even with 20 modules, latency stays below Is/item, meeting the demands of real-time applications. This highlights GTM's ability to balance model expressiveness and computational efficiency.
CONCLUSION
Large-scale TS analysis poses distinct challenges compared to LLMs, particularly in learning effective universal knowledge and building models for multi-task settings. In this paper, we propose GTM, general framework for TS analysis that utilizes decoder-only architecture. GTM incorporates granularity- attention mechanisms in both the temporal and frequency domains to improve TS representations. Furthermore, we introduce blank infilling pre-training strategy specifically designed for multi-task TS analysis, unifying all generative downstream tasks. Experimental results show that GTM either matches or outperforms SOTA methods across all generative TS analysis tasks. Additionally, our findings demonstrate that GTM adheres to scaling laws, achieving better performance with larger model sizes and more extensive pre- training datasets. However, challenges and limitations remain in the design of TSFM, including the lack of large-scale datasets and the absence of consistent benchmark models and settings. A detailed discussion of future work and limitations is provided in Appendix B.5 for further enhancement.
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGEMENTS
The work was supported by grants from the National Natural Science Foundation of China (Nos. 92367110 and U23A20319).
REFERENCES
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Feida Zhu, Beng Chin Ooi, and Chunyan Miao (eds.), KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, pp. 10.1 (4.114/4444588.44665
ACM, 2021. doi: 34455444444884 3467174
[RRlhttps://ddo
Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Gift-eval: A benchmark for general time series forecasting model evaluation. CoRR, abs/2410.10399. 2024. doi: 1044850/ARXIV2410010053 10. 4..4555/ax3.v.2111 . 10393
URL http::tts://et /rg/
Abdul Fatir Ansari, Lorenzo Stella, Ali Caner Türkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda-Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Schnide- and Yuyang Wang. Chronos: Learning the language of time series. CoRR, abs/24033078 2024. doi: (0-8390ARXX1:24000778... URL https? 2403. 10..o..og/10.44555/aaaa0 2783 07815
Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn J. Keogh. The UEA multivariate time series classification archive, 2018. CoRR, abs/181 .00075, 2018. URL
811 00075
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ B. Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg. Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dorottya Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho. Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, and et al. On the opportunities and risks of foundation models. CoRR, abs/2108.0722 2021. URL ht.tss://arxiv. wwtp:/iwv.rrraaooomomo abs///////// 222s//100 2108 01211100755555 07258
Defu Cao, Furong Jia, Sercan Ö. Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. TEMPO: prompt-based generative pre-trained transformer for time series forecasting. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL httph:tt/////.... fltps:://operevieee net fformmidd YH5w 20JuU
Hugo Carreira Pedro, David I Larson, and Carlos Coimbra. comprehensive dataset for the accelerated development and benchmarking of solar forecasting methods, June 2019. URL http:::/t///l...
org/10 5281 2tt.:///encddd2222599
Ching Chang, Wei- Yao Wang, Wen-Chih Peng, Tien-Fu Chen, and Sagar Samtani. Align and fine- tune: Enhancing Iims for time-series forecasting. In NeurlPS Workshop Time Series in the Age of Large Models, 2024.
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for series forecasting. In Forty-first International Conference Machine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL htllatts///oppeeeiieee net met.foo fort//orum221 tttt//oorummiooon222uasss

---

Published as a conference paper at ICLR 2026
Jiaxiang Dong, Haixu Wu, Yuxuan Wang, Yunzhong Qiu, Li Zhang, Jianmin Wang, and Mingsheng Long. Timesiam: A pre-training framework for siamese time-series modeling. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21 2024. OpenReview.net, 2024. URL wwtpt:tpps/... /otp://ee//eeeeeveeeee
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang. and Jie Tang. GLM:: general language model pretraining with autoregressive blank infilling. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 320-335. Association for Computational Linguistics, 2022. doi: 10.18653/V1/ 2022.ACL-LONG.26. URL /ttp://oii... www.orgg/......
http:/2222.ccllommm www.acllnnn....... 26
Vijay Ekambaram, Arindam Jati, Nam H. Nguyen, Pankaj Dayama, Chandra Reddy, Wesley M. Gifford, and Jayant Kalagnanam. Tiny time mixers (ttms): Fast pre-trained models for enhanced zero/few-shot forecasting of multivariate time series. CoRR, abs/2401.03955 2024. doi: 10.48550/ ARXIV.2401.03955. BtLhhtts::/ /doi. /tt.pp///aall...g......///////// 20t.0008555/aa...2....... 03955
Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitnik. Units: A unified multi-task time series model. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. MOMENT: A family of open time-series foundation models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024.
URL https::p///. http//ppeevviieeeeeeeeeeoeeeeeee ew net www.eeeewweeeeeeeeeeeee DoorummiddFVVFf65a5..
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Söderström. Detecting spacecraft anomalies using Istms and nonparametric dynamic thresholding. In Yike Guo and Faisal Farooq (eds.), Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, pp. URL //ti:
387 395. ACM, 2018. doi: 10. 10.1145/3219819.521988 321 981 3219845
1145/
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi. Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-llm: Time series forecasting by reprogramming large language models. In The Twelfth International Conference on Learning Representationss ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL httpps://openreet eew net/ wtwrrrmm?dduubbccPP....
Harshavardhan Kamarthi and B. Aditya Prakash. Large pre-trained time series models for cross- domain time series analysis tasks. CoRR, abs/2311. 2023. doi: 10.48550/ARXIV.2311. 11413. URL (tLhhtts: http:::://doi..rg/100// wtt.oorg.....5///2222222
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, 0bbs22001008661 2020.
2001 08361
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Hang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. The Tenth In- ternational Conference on Learning Representations, ICLR 2022, Virtall Event. April 25-29, 2022. OpenReview.net, 2022. URL htt.hhtts////mmm http::::////ppeeeeieee.. net foott//oommmm/ wttrmmmidd@GDAAOOOOC.
Qinghua Liu and John Paparrizos. The elephant in the room: Towards reliable time-series anomaly detection benchmark. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.). Advances in Neural Information Processing Systems 38: Annual Conference Neural Informa- tion Processing Systems 2024, NeurIPS 2024, Vancouver. BC, Canada, December 10 15. 2024, 2024. URL httt:://paperte .nw.pprr...iiipi...............e cc/paper files/ hilles/paper/224/ Benchma rks Track
Aatee--bsstaatt -aaetttttatttstss and
12

---

Published as a conference paper at ICLR 2026
Xu Liu, Junfeng Hu, Yuan Li, Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmermann. Unitime A -egguag--mmowwred unified model for cross-domain time series forecasting. In Tat-Seng Chua, Chong Wah Ngo, Ravi Kumar, Hady W. Lauw, and Roy Ka-Wei Lee (eds.), Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 17,17, 2024, 031.:/555883434644444 URL httph::t/pss..o orr.a..oorrgi
pp. 4095 4106. ACM, 2024a, doi: 1145/ 3645434
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Generative pre-trained transformers are large time series models. Forty-first International Conference on Machine Learning, ICML. 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, net forumm bYRbb 7DMM
2024b.
Yong Liu, Guo Qin, Zhiyuan Shi, Zhi Chen, Caiyin Yang. Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Sundial: A family of highly capable time series foundation models. arXiv preprint arXiv:2502.00886 2025.
Zhiding Liu, Jiqian Yang, Mingyue Cheng, Yucong Luo, and Zhi Li. Generative pretrained hier- archical transformer for time series forecasting, In Ricardo Baeza- Yates and Francesco Bonchi (eds.). Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024, pp. 2003-2013, ACM, 2024c. doi: 3671665715
00,1145/663752836671835 URLpttps:: doi org..o.rgol.... 1tt.//00.1155/33375252
Xiangkai Ma, Xiaobin Hong, Wenzhong Li, and Sanglu Lu. UTSD: unified time series diffusion model. CoRR, abs/2412.033068, 2024. doi: 1048850/ARXXV24422236666 URL htttps://doi 03068
org/ 000/10.485550 485.08550//66
Aditya P. Mathur and Nils Ole Tippenhauer. Swat: water treatment testbed for research and training on ICS security. In 2016 International Workshop on Cyher-physical Systems for Smart Water Networks, CySWater@ CPSWeek 2016, Vienna, Austria, April 11, 2016, pp. 31-36. IEEE Computer Society, 2016. doi: 10. WW0//CYSWATER2211..6666656
URL https: / http://doooooggggg
110.1109/C0 /11109//ySWaterr... 2016 6.016.00 70..6.449999 7469060
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. In The Fleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. net forum?iro 2id JbdcOvTOcol
URL
Adam Paszke, Sam Gross, Francisco Massa, Adam Lever, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buce Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 8024-8035, 2019. URL https: https:://procceddingg wwwp://cceeddiinnnnn.nin.e nwwrinuripp.... www.ipsssccce.....eeo wwtpcc/ppppp..eee 12201///ha 9/hash/ bdbca288fee (rlfeee7F92222 (0r Abst .tt.ralttt.mmml....
Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Bilos, Hena Ghonia, Nadhir Vincent Hassen, Anderson Schneider, Sahil Garg, Alexandre Drouin, Nicolas Chapados, Yuriy Nevmyvaka, and Irina Rish. Lag-Ilama:
Towards foundation models for time series forecasting. CoRR, hbs/22311.0277 2023. doi: 10. 48550 aat.sS/aa/Xvvvv.. 2310.....31..00 08278
48550/ARXIV.2310.088 URL / 4855://0118888/////
Avirup Saha, Prerna Agarwal, Sambit Ghosh, Neelamadhav Gantayat, and Renuka Sindhgatta. Towards business process observability. In Proceedings of the 7th Joint International Conference on Data Science & Management of Data I1 ACM IKDD CODS and 29th COMAD), pp. 257-265, 2024.
Lifeng Shen, Zhuocong Li, and James T. Kwok. Timeseries anomaly detection using temporal hierarchical one-class network. In Hugo Larochelle, Marc' Aurelio Ranzato. Raia Hadsell, Maria- Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems
13

---

Published as a conference paper at ICLR 2026
33: Annual Conference Neural Information Processing Systems 2020, NewrlPS 2020, December
6-12, 2020, virtual, 2020. URL hash/ 0aaa://2000011000111444442300502
Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Tome-moe: Billion-scale time series foundation models with mixture of experts. CoRR, abs/2409 16040, 2024. doi: 10, 015500/RXXIV.449.11660
URL https: org. 005.0445550/22330 2409]
16040
Ya Su. Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis (eds.), Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pp. 2828 2837, ACM, 2019. doi: 10.1145/3292500. 3330672. URL https: / /ttp:/dooi... org 0 11111114/322500 3330672
Mingtian Tan, Mike A Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen. Are language models actually useful for time series forecasting? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. htRps::::: / http://pppeeeeeeeeeeeeeeeee ew net/ wtt./oorruroo/////////o/o/ooommm d
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Marc' Aurelio Ran- zato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neu- ral Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 22430, 2021. htt/:tttt/////cceiiiing www.diiinnnnnnu.s..no... neurips bcc0d40028 wew.8887773398ddcdde 31dda313333cccccc1cccccaacaaa2a 9ab.aa92bbabbbbbbbbbbbbbb
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long, Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenRe- view.net, 2023. URL httppts:s:: http:///ppeeevveeeeee ew net/ for.mroirnojoioo fww.rummmiiiuuuuueeee__ UQw38440 40q
Haowen Xu, Wenxiao Chen, Nengwen Zhao, Zeyan Li, Jiahao Bu, Zhihan Li, Ying Liu, Youjian Zhao, Dan Pei, Yang Feng, Jie Chen, Zhaogang Wang, and Honglin Qiao. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. In Pierre-Antoine Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis (eds.), Proceedings of the 2018 World Wide Web Conference on World Wide Weh, WWW 2018, Lyon, France, April 23-27, 2018, pp. 187- 196. ACM, 2018. doi: 10.1145/3178876.1155555 315996..... URL https:://o1 org/ 1o. 70.11451 14144/335 788 878888866388838 3185996
Jiarui Yang. Tao Dai, Naiqi Li, Junxi Wu, Peiyuan Liu, Jinmin Li, Jigang Bao, Haigang Zhang, and Shutao Xia. Generative pre-trained diffusion paradigm for zero-shot time series forecasting. CRRR, abs/2406.02212. 2024. doi: 1044550/ARXVV2400.0222 02211. URL https://doot org/ 10. 10.44550/az10 www.v//arvvvv.. 2etexivv..4000 2406.0221222..
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Brian Williams, Yiling Chen, and Jennifer Neville (eds.), Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Appli- cations of Artificial Intelligence, IAAI 2023, Thirleenth Symposium Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, pp. 11121-11128. AAAI Press, 2023.
Yunhao Zhang, Minghao Liu, Shengyang Zhou, and Junchi Yan. UP2ME: univariate pre-training to multivariate fine-tuning as a general general-purposs. purpose framework for multivariate time series analy- sis. In Forty-first International Conference Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://///.......t.. http:::::///peeeveew.. net forum?id RR3uxW Zhx
14

---

Published as a conference paper at ICLR 2026
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thine Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, /Rtt:s:::
11106-11115,. AAAI Press, 2021. doi: 10.1609/AAALV33II...... http1//.1609/aai..13222
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 27268 27286. PMLR, 2022. URL https: pttp://d/oceeeingg..m.m mlr pressspres///eeee v1 62/ zhou22g. .t.22mll.mmmmmlmmmmmmmmmmmmm.
Tian Zhou, Peisong Niu, Xue Wang. Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neu- ral Information Processing Systems 36: Anmual Conference on Neural Information Pro- cessing Systems 2023, NeurIPS 2023, New Orleans, LA, USA. December 10 16. /aash/
2023, 2023. URL http::/ppppppr..sssssspaapeeeeee 86c1 oac.e//eeeecaae00019eeeeoeooorrr
html
15

---

Published as a conference paper at ICLR 2026
A
THE USE OF LARGE LANGUAGE MODELS
LLMs were used only occasionally to help polish the writing (propose new words, grammar and spelling correction). All technical ideas, experimental designs, analyses, conclusions, writing were developed and carried out entirely by the authors. The authors have full responsibility for the final text.
B TECHNICAL APPENDICES AND SUPPLEMENTARY MATERIAL
B.1 ADDITIONAL RELATED WORK
LLM Empowered TSFMs:
This line of works follow the paradigm that freeze LLM encoder backbones while simultaneously tuiinn/aaddd/n the input and projection heads for forecasting, and notable ones include MLJin et al., 2024), LLM4TS et al.. 2024), GTP4TS et al. 2024) and Tempo Cao et al. 2024).
ctt al. 2023), (iiu et al. 2024a), Chronos(
This effectiveness of this paradigm is currently under debate in the sense that some works present promising results while the latest ablation studies show the counterpart TTan et al. 2024).
Table 12 highlights the key distinctions between approach and existing SOTA models. First, whereas prior TSFMs primarily rely on temporal information from discrete scalar values, our method uniquely integrates both temporal and frequency-dee -omaan features through Fourier attention mecha- nism that captures time granularity- downstream task-specific customization the token, pre-training strategy, or model level. In contrast, our approach introduces a hybrid masking-bass pre-training strategy that unifies reconstruction and autoregressive objectives, enabling generative-task--nnotii adaptation without additional modifica- tions.
representations. Second, previous models often require
B.2 DETAILS OF IMPLEMENTATION AND EXPERIMENTAL SETTINGS
B.2.1 DATASETS DESCRIPTION
We use the UTSD-12G dataset, released by (Liu et al., 2024b), for pre-training. The Unified Time Series Dataset (UTSD) includes seven domains: Energy, Environment, Health, loT, Nature, Transportation, and Web, with varying sampling frequencics. contains up to 1 billion time points and hierarchical structures, supporting large-scale time series model research. The overall statistics of UTG is shown in Table
For downstream tasks such long-term forecasting and imputation, we conduct experiments five widely used public datasets from (Wu et al.) (02T): ETTH, ETTm. Weather, Electricity, and Traffic. For anomaly detection, we utilize five popular datasets: SMD (Su et al., 2019), MSL, SMAP (Hundman et al., 2018), SWaT (Mathur & Tippenhauer (016)) and PSM (Abdulaal et al. 2021). For classification, we select ten standard datasets from Bagnall et al. (2018): EthanolConcen- tration, FaceDetection, Handwriting, Heartbeat, Japanese Vowels, PEMS-SE, SelfRegulationSCPP. SelfRegulationSCPP ArookenArabiDDggiin and UWaveGestureLibrayy Dataset statistics for these tasks are summarized in Tables 14 [5] T6d Among these datasets, the ETTm dataset represents the longest-range testing scenario, spanning over 725 days and containing up to 69,680 time points at a 15-minute sampling interval.
B.2.2 BASELINE MODEL SELECTION
We summarize the baseline models in Table We classify these models into four categories, includ- ing LLM-enhanced models for TS analysis, MLP-based models, Transformer-based models, and TSFMs. The TSFMs are further divided into two sub-categories: task-specific foundation models and multi-task foundation models. Since each model has its own design goals and experimental settings, it is challenging align them all for reproducing their best results presented in papers. Therefore, follow established protocols from previous works and select typical models as benchmarks for each downstream task, ensuring fair comparison of GTM with SOTA results.
16

---

Published as a conference paper at ICLR 2026
Table 12: Comparison between GTM and SOTA time series foundation models trained from scratch. The models are characterized by their approach to representation learning, ability to handle down- stream tasks, and adaptability to multi-task scenarios. The list of the abbreviation of the table is: Temporal Domain: T. D., Frequency Domain: F. D., Anomaly Detection: AD., Inference Adaption: Inf. Ad.
Time Series Features
Downstream Tasks
Adaptabilitty
[
E.
Time Gran. Forocasting AD, Imputation
CLF. WWo Inf. Ad.
PatchTST, Lag GPD GPHE, Timesd., MOIRAL UTSD. TTMs, TIME-MOF
V
x
x
x
x
x
x
TimeSiam, LPTM
x
x
x
x
/
x
TIMER, UP2ME
x
/
UniTS
x
5
1
1
x
GTM(quus)
1
V
/
1
/
/
Table 13: Statistics of UTSD-12G dataset.
Domain
Dataset Number
Time Points
File Size
Freq.
Energy
3
175.066 775.06M
4334M
[4 sec, 30 min, Hourly]
Environment
31.54M
286M
[Hourly]
Health
9
289. 72M
2685M
[Ims, 2ms, 4ms, 8ms]
ToT
165.4M
2067M
[20ms]
Nature
11
241.4M
2227M
[33ms, Hourly, 3h, Daily]
Transport
3.13M
72M
[Hourly] [Daily]
Web
116.49M
388M
Table 14: Statistics of datasets for forecasting & imputation.
Dataset
Length
Dimension
Frequency
ETTh 17420
7
hour
ETTm
69680
15 min
Weather
52696
21
10 min
Electricity 26304
321
hour
Traffic
17544
862
hour
B.2.3 EXPERIMENTAL SETTINGS AND IMPLEMENTATION DETAILS
Pre-training In the pre-training stage, we trained our GTM model on the UTSD-12G 12G dataset (Liu et al. 2024b). During data preprocessing, defined lookback window of 1440 timestamps and split the raw data into overlapping samples with a stride - = 192. We then generated 15 patches with patch size T.p 96. To enable the model to learn both reconstruction and forecasting objectives, for each training instance, we empirically set the hyperparameter pred_ratio to 0.3, and masked the last 30% of the sequence (tail masking) with probability pred_ratee ratio For other critical model hyperparameters, we set the batch size to 1024 and the learning rate to x 10-5, using Adam as the optimizer with a cosine annealing learning rate decay. We trained for 30 epochs with an early stopping mechanism, and the decay steps were proportional to the number of training epochs. In the model backbone, we set the number of layers (N-stack) to 12 and the feature dimension to 768. The Fourier Knowledge Attention layer consisted of 5 attention modules, each with a low-rank matrix parameterized by AB, where A E |335511 B E R 1x355 We provide pseudo-code of GTM architecture and pre- rre-trannig strategy in Algorithm[ [ Finally, implemented the GTM model in PyTorch (Paszke et al. 2019) and trained it on 6 NVIDIA A 100 40GB GPUs.
Fine-tune
We present experimental settings for three generative downstream tasks.
Long-term Forecasting For long-term forecasting, we directly reuse the pre-trained GTM model without any special adaptations, only removing the masking process. We dy- namically choose back window in range [96, 1440) and forecast future time points T e {96, 192, 336, 720}. The results are compared with the best-performing results SOTA models presented in papers source codes.
17

---

Published as a conference paper at ICLR 2026
Table 15:
Statisties of datasets for anomaly detection.
Datasct
Training size
Validation size
Test size
Dimension
Frequency
Anomaly rate
MSL
46655
11664
73729
55
min
10.5%
SMAP
108 46
27037
427617
25
1 minn
12.8%
SMD
566724
141681
708420
38
I min
4.2%
SWaT
396000
99000
449919
sec
12.1%
PSM
105984
26497
87841
25
1 min
27.8%
Table 16: Statistics of datasets for classification.
Datasct
Train Cases
Test Cases
Dimensions
Length
Classes
EthanolConcentration
261
263
3
1751
FaceDetection
5890
3524
144
62
2
Handwriting
150
850
3
152
26
Heartbeat
405
Japanese Vowels
270
370
12
29
PEMS-SF
267
173
963
144
7
SclfRegulationSCPP
268
293
6
896
2
SclfRegulationSCCP2
200
180
7
1152
SpokenArabicDigiit
6599
2199
13
UWaaveestureLLibrary
120
320
3
315
8
Table 17: Selected SOTA baseline models for downstream tasks comparison.
Task
Method Types
Method
LLM-Enhanced for TS
GPT4TS
MLP-based
DLinear
PatchTST, FEDformer, Autoformer, Informer
Forecasting
Transformer based
task-specific foundation model
TTMs UTSD
UnITS-SUP, UmiTS-PPi PMM,, TimesNet
multi-task foundation model
I.L.-Enhanced for TS MLP-based
GPT4TS
DLinear
PatchTST, FEDformer, Autoformer, Informer
Anomaly Detection
Transformer based
task-specific foundation model
multi-tass foundation model
TimesNet, UP2ME
LLM- Enhanced for TS
GPT4TS
MLP-based
DLincar
PatchTST, FEDformer,
Imputation
Transformer- based
Autoformer Informer
task lusk-specific foundation model
UTSD
multi-task foundation model
TimesNet UP2ME
LLM- L.M--Enacced for TS
GPT4TS
MLP-hased
Classification
Transformer based
iTransformer
task task-specific foundation model
multi-task foundation model
UniTS-SUP, UNMTS-PMT, TimesNet
Imputation To align with benchmark settings, we follow the protocol proposed by Zhou et al. 2023) for imputation tasks. We use point-wise missing ratios of
18

---

Published as a conference paper at ICLR 2026
Algorithm 1 GTM pre-train strategy
Require: Input look_back back time series æ, a RR look_back window length I,, number of channels variables C; number of patches Np: patch length Lp: no. of masked patch Nmpi no. of reconstruction patch label Nep no. of total patches Niotal = Nmp + Nrp: patch embedding dimension D.
1: CI and Patching:
2: Patch(CI(()))
PPERNXX
3: Masking:
4: Xin = -Peept. Sin]
D. Xin E Nep
5: Embedding:
6: H Embedding(X
> H E R N
7: for 1 to N do
through GTM blocks
8: Temporal attention:
9: H AtoOut Self Altention (H%-1)
H8-1 b. TemAttut E Mootu
Fourier attention:
11: H Fourier (areiioo
b H out E R N
end for
13: Output Projection:
14: Xon = -MLP(H mux)
Xout E R Motat
return Xout
{1 (12.5%, 25%, 37.5%, 50%) at the time-point level for interpolation, omitting the patch- ing process. For all other aspects, reuse the settings from the pre-training stage.
Anomaly Detection We use common adjustment strategy (Xu et al. 2018 Su et al. 2019
Shen al. 2020) for anomaly detection: if an anomaly is detected at any time point in an abnormal segment, all anomalies in that segment considered detected. This approach is based on the fact that detecting one abnormal point usually triggers an alert for the entire segment in real-world scenarios. We calculate Fl-scores for each datasets to evaluate the results. the As we do other generative tasks, we directly reuse the GTM model settings from the pre-trate training stage.
Classification For discriminative tasks such as classification, we replace the projection head to output class label probabilities instead of future time step predictions, while keeping the rest of the model architecture unchanged to ensure smoothly adaptation. We employ cross-entropy loss, aiming to minimize the divergence between the predicted and true class distributions, which is equivalent to maximizing the log-likelihood of the correct label. Model performance is evaluated using accuracy, enabling direct comparison between GTM and SOTA TSFMs.
B.3 FULL RESULTS
Due to space limitations in the main body of the paper, we provide the full experimental results in this section, to complement the discussion in section
B.3.1 FORECASTING
Table 18 demonstrates the full results of long-tem term forecasting. it shows that GTM outperforms all the SOTA models, achieving the best result in 21 and second best in 22 out of total 50 tests. The second best model PatchTST, achieves the best in 14 and second best in 15.
19

---

Published as a conference paper at ICLR 2026
Table 18: Full results of MSE and MAE for long-term forecasting. We conduct experiments for different length T E (96, 192, 336, 720). Bold and underline numbers denote the best and the 2nd-best results, respectively.
Models
GTM
GPT4TS
[UniTS-PMTT
TTMr:
PatchTST
TimesNet
DLinear
|FEDformer /Autoformer] Informer
Datasct
T
[MSE
MAE
MSE
MSE MAE
MSE MAE
MSE
MAE)
MSE
MAB/MSE MAE[MSE MAE
MSE
MAE
MSE
MAE
96
.360
.398
.376
.397
390
411
363
.370
_400
384
.402
.375
.199
.376
.415
435
.466
.65
.713
192
.397
.422
.466
.188
432
A38
.394
413
.429
.46
.499
.405
.466
.23
.446
456
.457
1.008
.792
ETTh1
336
.420
.337
.442
480
.600
.003
.42
.440
469
.439
.43
.444
.462
486
.487
1.107
720
.438
.457
477
.456
542
.508
.449
447
.66
521
.500
.472
490
469
.492
515
.517
1.181
8665
Ang
.404
.429
.27
.426
.661
A54
.402
.13
.434
.458
4400
.422
.477
428
.453
473
.476
1.040
.795
96
.282
.341
.92
.346
.233
.293
.346
.338
.375
.299
344
.326
.390
.100
.42
.672
.571
192
.325
.366
.352
.372
.335
.333
.370
.374
.387
.335
.365
.365
.415
514
.495
.795
.669
ETTml
336
.353
.385
.366
.364
.392
.11
.369
.392
.492
1,212
720
.396
.410
.417
.421
.408
416
.420
.478
450
.425
421
.446
.458
.27
.493
1,166
.823
Aug ,339
.376
.352
.383
.350
.352
.382
.400
.406
.357
.378
.382
.422
51.5
.493
.961
.734
96
.147
.197
.162
.212
.157
.206
.154
.149
.198
.172
.220
.176
.237
.2338
.314
2249
.329
.300
.384
192
.192
.241
.204
.248
.208
2251
200
194
244
.219
2261
.220
.282
.325
.370
325
.370
.598
.544
Weather
336
.250
.291
.254
.291
.282
.306
.351
.351
.578
720
.310
.334
.326
.337
.344
.344
.324
314
.334
.365
.359
.323
.362
.415
.426
.415
.426
1.059
.741
Ang
.225
.266
.237
.270
.243
.273
.234
225
.263
.259
.287
,246
.300
.332
.375
3335
.379
.634
.548
96
1351
.250
.388
.282
J465
.298
1.372
360
.249
.593
.322
.410
.282
.576
.359
5597
.371
.719
.391
192
.373
2260
.407
.200
484
.306
.365
339
.256
.617
J336
.423
.277
.610
.800
600
.32
.69
.379
Traffic
136
.267
.412
.494
.312
.379
.392
.264
.336
.26
.608
.387
.777
720
.428
288
.450
.312
534
.335
.425
432
.286
.640
.350
.466
3315
662
.375
639
.35
.864
.472
Arg
.385
.266
.444
.294
.494
.31.3
.385
.390
.263
.620
.336
.433
.295
16603
.372
6616
.383
.764
.416
96
|131
.225
.139
.238
.157
.2.58
.129
.129
.222
.168
.272
.140
.237
.186
.302
.96
.313
.274
.368
192
.149
.243
.153
.251
.173
.272
148
.147
.240
.184
.289
.153
2449
.197
.311
211
.324
.296
.386
Electricity
336
.166
.259
.169
.266
185
.284
161
.163
.259
.19
.300
.169
.67
.213
.328
214
.327
.300
.394
720
.201
.292
.206
.297
.219
314
.133
.97
.290
.220
.30
.203
301
233
.34
236
.32
.373
.439
Avg
.161
225
.66
266
184
.282
.158
.59
.252
.92
.295
166
263
.007
.321
214
.326
.311
.397
We further conduct error bar analysis by running 10 independent trials for long-term forecasting tasks. The 95% confidence interval for each metric is calculated as to f0.025, n-1 x add vT where fo.025, n-1 is the 97.5th percentile of the Student's !-distribution with 1 degrees of freedom. For n 10 runs, it is approximately 2.26. As shown in Table 19 the error bars for both MSE and MAE across all prediction lengths and datasets are consistently low, indicating high reliability and stability of the reported results.
To provide comprehensive baseline comparison for long-term forecasting tasks, we further include two recent TSFMs, Sundial-Sman mmal et al. (2025) and Time-MOE-Ban Shi et al. (2024), both of which are comparable to GTM in model size and are evaluated under similar experimental settings. Note that Time-MOE- MOE-BBase was not evaluated on the Electricity dataset; therefore, best count statistics reported both for all tasks and for the subset where Time-MOF-BBan shown in Table 20, GTM achieves the best performance on 13 out of 20 metrics overall, and on 8 out of 15 metries when directly compared with Time-MOE-Baae
results are available. As
(numbers in parentheses in the table), slightly outperforming the Time-MOE-Bass model. In contrast, Sundial-Smal achieves the best result in only one case. These results demonstrate GTM's strong competitiveness and robustness across diverse datasets and prediction horizons.
For more experiments on challenging, real-world datasets, we have identified two such kind of datasets that have not yet been over exploited: one is an open Vollaac) solar energy forecasting Catase(Carreira Pedro et al. (2019) and the other is the I.2C (lead-to-cash) datase et al. (024)) which combines observations of Business Key Performance Indicators (Biz-KPIs) and IT events. We also have reproduced two SOTA models, PatchTST and TimesNet, conducting experiments on forecasting with various prediction length. Table 21 shows that GTM consistently delivers SOTA performance across both the PV and L2C datasets, achieving the best results in most test cases for both MSE and MAE metrics. The most significant improvement of GTM over competing methods is observed on the L2C dataset with a prediction length of 720. For MSE, GTM achieves a score of 0.7170, outperforming the second-best method (TimesNet, 1.2984) which means 44.8% reduction.
20

---

Published as a conference paper at ICLR 2026
Table 19: 95% CI error bar analysis for forecasting tasks.
Dataset
pred len
MSE
MSE error-bar
MAE
MAE error-bar
96
0,3611
+0.00093
0.3991
#0.00072
192
0.3990
+0.00099
0,4241
+0.00099
ETTh1
336
0,4236
+0.00 +0.00099
0,4395
+0,00099
720
0.4428
#0.00344
0.4643
#0.00265
Avg.
0.4066
#0.00157
0.4318
#0.00136
96
0.2828
+0.0014
0.3430
#0.0010
192
0.3304
+0.0022
0.3698
#0.0005
ETTml
336
0,3580
#0.0021
0,3890
#0.0009
720
0,4040
+0.0020
0,4122
#0.0016
Avg.
0.3438
+0.0019
0.3785
#0.0010
96
0.1474
#0.00043
0.1983
#0.00050
192
0.1943
#0.00107
0.2427
10.00115
Weather
336
0.2445
#0.00014
0.2876
#0.00021
720
0.3100
+0.00229
0.3355
#0.00115
Avg.
0,2241
+0.00099
0.2660
+0.00079
For MAE, GTM attains value of 0.5218 compared to PatchTST's 0.8460, resulting in 38.3% reduction.
B.3.2 IMPUTATION
Table 22
provides the full results of Imputation for various data missing ratios of (12.5%. 25%, 37.5%, 50% at the time-point level. Except for the Electricity dataset (where achieved second-best performance), GTM outperforms all other methods in other experiments.
B.3.3 EXTENDED ANOMALY DETECTION
Recent work by (Liu & Paparrizos 2024 has highlighted several critical challenges in time series anomaly detection, including flawed datasets and biased evaluation metries. To provide a more comprehensive evaluation of our model, utilize the TSB-AD benchmark, which features extensive and carefully curated collection of datasets, widely used measures, along with broad coverage of TSFMs testing results. Table 23 presents the mean accuracy scores across the TSB-AD-U datasets using various evaluation metrics. Compared to SOTA TSFMs such as MOMENT, TimesFM, Lag-Llama, Chronos, and other deep learning models, GTM achieves the best performance out of 9 metrics. These results demonstrate that GTM is highly adaptable and robust across diverse datasets and evaluation criteria.
B.3.4 EFFECTIVENESS OF PRE-TRAINING
Forecasting Table 24 presents a detailed comparison between the pre-trained GTM model and the GTM model without pre-training. We also conduct experiments for different length T E {96, 192, 336, 720). The results demonstrate that pre-trained GTM model outperforms the non-pre- trained version, highlighting the benefit of the pre-training stage in leveraging general knowledge from large-scale datasets.
Imputation Table 25 provides detailed results of comparison in Imputation tasks between the pre-trained GTM model and the GTM model without pre- pre-training. As described in Sed4.3] we also conduct experiment for different data missing ratios of (12.5% 25%, 37.5%, 50%) the time-point
21

---

Published as a conference paper at ICLR 2026
Table 20: Full MSE and MAE results for long-term forecasting additional SOTA TSFMs:
Time-MOE- and Sundial-Small. Experiments are conducted for prediction lengths T E {96, 192, 336, 7203. Bold numbers denote the best results.
Models
GTM
Time- Time-MORE MOE-b
Sundial-s
dataset
Pred len
MSE
MAE
MSE
MAE
MSE MAE
96
0.360
0.398
0.345
0.373
0.341 0.381
192
0.397 0.422
0.372
0.396
0.381 0.408
ETTh1
336
0.420
0.437
0.389
0.412
0.405 0.424
720
0.438
0.457
0.410
0.443
0.433 0,458
Avg.
0.404..4499
0.429
0.379
0.406
0.390 0.418
96
0.282
0.341
0.286
0.334
0.292 0.342
192
0.325
0.366
0.307
0.358
0.337 0.376
ETTml
336
0.353
0.385
0.354
0.390
0.370 0.401
720
0.396
0.410
0.433
0.445
0.418 0.433
Avg.
0.339
0.376
0.345
0.381
0.354 0.388
96
0.147
0.197
0.151
0.203
0.158 0.206
192
0.192
0.241
0.195
0.246
0.205 0.253
weather
336
0.250
0.291
0.247
0.288
0.254 0.290
720
0.310
0.334
0.352
0.366
0.315 0.336
Avg.
0.225
0.266
0.236
0.275
0.233 0.271
96
0.131
0.225
0.134 0.231
192
0.149
0.243
0.154 0.251
Electricity
336
0.166
0.259
0.174 0.271
720
0.201
0.292
0.215 0.307
Avg.
0.161
0.254
0.169 0.265
Best Count
13(8)
12(7)
level. As expected, the pre-trained GTM model outperforms the non-pre-trained version in all tests, achieving significant improvements.
B.3.5 ABLATION TEST
Table 26 presents the full ablation results for forecasting tasks with varying prediction lengths, includes T E [96, 192, 336, 720) time points. The comparison involves the complete GTM model, an advanced version of GTM without the frequency knowledge attention module, and a baseline version that includes only the temporal analysis module. The results demonstrate that the complete design of the GTM model effectively supports the learning of universal representations for MTS datasets with varying time granularities.
B.3.6 SCALABILITY ANALYSIS
Figure iilustrates the data scalability analysis, while Table 27 provides the complete forecasting results using different pre-trat iraitra data sizes. The findings confirm that GTM follows scaling laws, where pre-training on larger datasets consistently enhances fine-tuning performance across diverse downstream tasks.
22

---

Published as a conference paper at ICLR 2026
Table 21: Full results of MSE and MAE for long-term forecasting Photo Voltaic(PV) and Lead-to-
cash(L22) datasets. Bold numbers denote the best results.
Dataset
Pred_len
GTM
PatchTST
TimesNet
MSE
MAE
MSE
MAE
MSE
MAE
0.4463
0.3508
0.3516
0.3143
0.3330
0.3575
240
0.7692
0.5359
0.7732
0.5670
0.7345
0.5598
L2C
720
0.7170
0.5218
1.3400
0.8460
1.2984
0.8374
Avg-
0.6442
0.4695
0.8216
0.5758
0.7886
0.5849
60
0.1763
0.2504
0.2001
0.2892
0.25778
0.3921
240
0.3030
0.3880
0.3650
0.4691
0.3655
0.4691
PV
720
0.5476
0.5616
0.5780
0.58:52
0.4928
0.5414
Avg.
0.3423
0.4000
0.38.66
0.4478
0.3720
0.4675
Best_coumt
5
6
Table 22: Full results of Imputation. We conduct experiment for different data missing ratios of (1..5%, 25%, 37.5%, 50%) at the time-point level.
Models
GTM
GPT4TS
TimesNet
PatchTST
DLinear
Fedformer
Informer
Dataset
Mask Ratio
MSE
MAE
MSE MAE
MSE MAE
MSE MAF
MSE
MAF
MSE MAF
MSE MAE
12.5%
.034
.125
.043
.140
.057
.159
.093
.201
.151
.267
.070
.190
.114
.234
25%
.046
.143
.044
.16
.069
.178
.107
.277
.180
.292
.106
.236
.140
.262
ETTh1
37.5%
.059
.163
.072
.80
.084
.196
.120
.230
.215
.318
.124
.258
.174
.293
50%
.073
.179
.107
.216
.102
.215
.141
.248
.257
.347
.165
.299
.215
.325
Avg.
.053
.152
.069
.173
.078
.187
.115
.224
.201
.306
.117
.246
.161
.279
12.5%
.015
.082
.017
.085
.023
.101
.041
.130
.080
.193
.052
.166
.063
.180
25%
.019
.090
.022
.096
.023
.01
.044
.135
.080
.193
.052
.166
.063
.180
ETTml
37.5%
.029
.111
.029
.111
.143
.103
.219
.069
.191
.079
.200
50%
.029
.112
.040
128
.036
.124
.055
.151
.132
.248
.089
.218
.093
.218
Avg.
.021
.096
.028
105
.027
.17
.047
.140
.093
.206
.062
.177
.071
.188
12.5%
.026
.046
.026
.049
.025
.045
.029
.049
.039
.084
.041
.107
.218
.326
25%
.030
.055
.028
.052
.052
.031
.053
.048
.103
.064
.163
.219
.326
Weather
37.5%
.031
.057
.033
.060
.031
.057
.035
.058
.057
.117
.107
.229
.222
.328
50%
.034
.061
.037
.065
.034
.062
.038
.063
.066
.134
.183
.312
.228
.331
Avg.
.030
.054
.031
.056
.030
.054
.060
.144
.052
.110
.099
.203
.222
.328
12.5%
.077
.191
,080
.194
.085
.202
.055
.160
_092
.214
.107
.237
.037
.093
25%
.084
.199
.087
.203
.089
.206
.065
.175
.118
.247
.120
.251
.042
.100
Electricity
37.5%
.090
.206
.094
.211
.094
.213
.076
.189
.14
.276
.136
.266
.049
.111
50%
.096
.215
.101
.220
.100
.221
.091
.208
.175
.305
.158
.284
Avg.
.066
.02
.900
.207
.092
.210
.072
.183
.132
.260
.130
.259
.045
.04
B.3.7
ANALYSIS
The look-back window length and patch length are two critical hyperparameters in the GTM model. We conducted experiments with varying values for these parameters to analyze the model's 's sensitivity. Table shows shows that performance steadily improves the patch length increases, while Table 29 demonstrates that both MAE and MSE results are consistently enhanced as the book-back window length is extended.
23

---

Published as a conference paper at ICLR 2026
Table 23: Summary accuracy comparison of mean value TSB-AD-U by various metrics. The best-performing method as per each metric marked in bold.
ModrisiMetrics
AUC-PR AUC-ROC
VUS-PR
VUS-ROC
Standard-FI
PA-FI
Event-based-Fi
R-based-FI
AfilliatiooFII
GTM
0.33
0.71
0.36
0.78
0.38
0.86
0.71
0.36
0.91
MOMENT (FT)
TimesFM
0.28
0.67
0.3
0.74
0.34
0.84
0.63
0.34
0.89
Lag. Llama
0.25
0.65
0.27
0.72
0.3
0.77
0.59
0.31
0.88
Chronos
0.66
0.27
0.32
0.61
0.33
0.88
TimesNet
0.18
0.61
0.26
0.72
0.24
0.67
0.47
0.21
0.86
FTTS
0.17
0.61
0.26
0.73
0.23
0.65
0.42
0.2
0.86
Anomaly AnomalyTransfone Transformer
0.5
0.12
0.12
0.34
0.14
0.77
Table 24: Full results forecasting comparison between GTM and GTM w/o pre-train. conduct experiments for different length T (96, 192, 192,336, 720).
Models
GTM
GTM w/o pretrain
Dataset
pred len
MSE
MAE
MSE
MAE
0.360
0.398
0.376
0.412
192
0.397
0.422
0.411
0.428
ETThl
336
0.420
0.437
0.454
0.453
720
0.438
0.457
0.500
0.497
Avg.
0.404(+7.1 %)
0.429(+4.0%%
0.435
0.447
96
0.282
0.341
0.291
0.352
192
0.325
0.366
0.335
0.378
ETTml
336
0,353
0.385
0.366
0.397
720
0.396
0.410
0.415
0.429
Avg.
0.339(+3.3%)
0.376(3.3 %
0.351
0.389
96
0.147
0.197
0.154
0.204
192
0.192
0.241
0,212
0.267
Weather
336
0,250
0.291
0.275
0.323
720
0.310
0.334
0.337
0.365
Avg.
0.225(+7.8%)
0.266(+8.0%)
0.244
0,289
96
0,351
0.250
0.353
0.252
192
0.373
0.260
0.373
0.259
Traffic
0.388
0.267
0.391
0.270
720
0.428
0.288
0.432
0.291
Avg.
0.385(+0.5%)
0.266(+0.85)
0.387
0.268
96
0.131
0.225
0.132
0.225
192
0.149
0,243
0.150
0.244
Electricity
336
0.166
0.259
0.170
0.262
720
0.201
0.292
0.203
0.294
Avg.
0.161(+1.2%)
0.254(+0.8%)
0.163
0.256
B.3.8 MODEL EFFICIENCY ANALYSIS
We further analyze model efficiency by conducting experiments with longer prediction lengths and larger input look-back windows. As shown in Table 30 the inference time remains nearly constant even as both the look-back window and prediction length increase by an order of magnitude. This illustrates that GTM does not fully saturate the computational resources of the A100 GPU, demonstrating high efficiency at the current scales and is well-suited for practical deployment in real-world sub-second streaming applications.
24

---

Published as a conference paper at ICLR 2026
Table 25: Full results of Imputation comparison between GTM and GTM w/o pre-training. We conduct experiments for varying data missing ratios of {12.5% 25%, 37.5%, 50%) at the time-point level.
Models
GTM
GTM w/o pretrain
Dataset
Mask Ratio
MAE
MSE
MAE
12.5%
0.034
0.125
0.037
0.131
25%
0,046
0.143
0.048
0.146
ETThi
37.5%
0.059
0.163
0.060
0.163
50%
0.073
0.179
0.077
0.184
Avg.
0.053(+3.6%
0.152(+2.55%) %
0.055
0.156
12.5%
0.015
0.082
0.020
0.096
25%
0.019
0.090
0.019
0.091
ETTml
37.5%
0,023
0.100
0.024
0.101
50%
0.029
0.112
0.030
0.113
Avg.
0.021(+8.6%)
0.096(+4..0% %
0.023
0.100
12.5%
0.026
0.046
0.028
0.051
25%
0.030
0.055
0.029
0.056
Weather
37.5%
0.031
0.057
0.032
0.060
50%
0.034
0.061
0.049
0.088
Avg.
0.030(411.7%% ))
0.054(+14.29%)
0.034
0.063
12.5%
0.077
0.191
0.078
0.192
25%
0.084
0.199
0.084
0.199
Electricity
37.5%
0.090
0.206
0.091
0.207
50%
0.096
0.215
0.097
0,215
Avg.
0.086(+1.2%
0.202(+0.55 %
0.087
0.203
0.240
0.412
0.410
0.235
MSE
MSE
0.408
0.230
0.496
0.404
16
2G
4G
12G
16
2G
4G
12G
Data Size
Data Size
(a) Downstream results on ETThl
(b) Downstream results on Weather
Figure Data scalability analysis. GTM achieves better results with larger pre-training datasets.
From an architectural perspective, there are three mainstream output projection designs in time series forecasting models. Below clarify these designs and discuss their implications for flexibility and inference efficiency:
Flatten layer with a linear projection (direct mapping)
In this design, the backbone outputs a tensor of size [B, D] (batch size B, number of patches N,. feature dimension D), which is flattened and projected via linear layer of shape [N, x D. L,, where L is the prediction length. This approach is adopted by models such as PST TimesNet, Crossformer, FreTS, etc..
Limitations: The output head must be reconfigured for each L. limiting flexibility for variable-length forecasting. It is clear disadvantage for TSFMs. Moreover, inference time increases with larger due to the growing size of the output head.
Autoregressive Approach
25

---

Published as a conference paper at ICLR 2026
Table 26: Full results of ablation test in forecasting tasks. Experiments conducted for varying prediction lengths, includes T E {96, 192, 336, 720} time points.
Models
GTM
GTM w/o time_gran. gran.
GGTM w/o Freq.
Dataset
Pred_len
MSE
MAE
MSE
MAE
MSE
MAE
96
0.360
0.398
0.372
0.406
0.384
0.416
192
0.397
0.422
0.405
0.427
0.408
0.429
ETThl
336
0.420
0.437
0.428
0.437
0.433
0.443
720
0.438
0.457
0,450
0.463
0.449
0.466
Avg.
03404 %, 2.42%
0.229 % 0..90.99% %
0.414(1.19%)
0.433(1.37%)
0.419
0.439
0.282
0.341
0.299
0.353
0.301
0.354
192
0.325
0.366
0.334
0.372
0.335
0.375
ETTml
336
0.353
0.385
0.360
0.391
0.363
0.393
0.396
0.410
0.398
0.411
0.398
0.412
Avg.
0.339
0.376(2.08 % 1.57 ())%)
0.348(0.29%% 0.382(0.52%)
0.349
0.384
96
0.147
0.197
0.153
0.217
0.158
0.212
192
0.192
0.241
0.206
0.254
0.208
0.258
Weather
0,250
0.291
0.252
0.293
0.256
0.297
720
0.310
0.334
0.311
0.335
0.313
0.337
Avg.
0.225(3.43
22.60%
0.266(3.62%
.3.27%
(.231(0.86%)
0.275(0.336%
0.233
0.276
96
0.351
0.250
0.355
0.253
0.359
0.256
192
0.373
0.260
0.374
0.262
0.379
0.264
Traffic
336
0.388
0.267
0.389
0.270
0.393
0.271
720
0.428
0.288
0.431
0.291
0.435
0.293
Avg.
0.385(1.79 % (.552%)
0.266(1.85 1.12%%
0.387(1.28%)
0.269(0.74%)
0.392
0.271
96
0.131
0.225
0.132
0.226
0.134
0.227
192
0.149
0.243
0.150
0.246
0.152
0.248
Electricity
336
0.166
0.259
0.168
0.262
0.169
0.264
720
0.201
0.292
0.202
0,295
0.205
0.296
Avg.
1161(2.42%,1.2399)
0.254(1.936,1.17%%
0.1 0.163 % 0.257 %
0.165
0.259
Table 27: Full results of scalability analysis pre-trained data size in forecasting tasks. Experiments
are conducted for varying prediction lengths, includes T E {96, 192, 336, 320) time points.
Data_size
1G
2G
4G
12G
Pred
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.372
0.409
0.369
0.405
0.363
0.400
0.360
0.398
192
0.404
0.425
0.405
0.426
0.399
0.423
0.397
0.422
ETTh1
336
0.427
0.439
0.423
0.438
0.422
0.438
0.420
0.437
720
0.448
0.462
0.445
0.459
0.441
0.458
0.438
0.457
Avg.
0.413
0.434
0.410
0.432
0.406
0.429
0.404
0.429
96
0.147
0.197
0.148
0.199
0.147
0.198
0.147
0.197
192
0.193
0.244
0.192
0.241
0.193
0.242
0.192
0.241
Weather
336
0.257
0.295
0.253
0.292
0.251
0.291
0.250
0.291
720
0.364
0.361
0.351
0.352
0.321
0.340
0.310
0.334
Avg.
0.240
0.274
0.236
0.271
0.228
0.267
0.225
0.266
In this approach, the model predicts one future value at time: at each step t. it uses previous prediction 인t-1 together with the input history predict ú.. This process is repeated until the desired prediction length 7. is reached.
Advantage: Enables high flexibility, the same output head can generate forecasts of varying lengths without retraining.
26

---

Published as a conference paper at ICLR 2026
Table 28: Performance of GTM for Different Patch Lengths.
Patch-len
8
16
32
64
96
Dataset
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.426
0.443
0.416
0,441
0.422
0.439
0.413
0.437
0.405
0.429
ETTml
0.363
0.402
0.349
0.381
0.355
0.388
0.351
0.379
0.342
0.377
Table 29: Performance of GTM for different look-back window lengths.
Seq-len
96
192
336
512
672
1440
Dataset
MSF
MAE
MSF
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.435
0.452
0.428
0.447
0.416
0.439
0.418
0.440
0.411
0.433
0.405
0.429
ETTml
0.371
0.401
0.363
0.395
0.355
0.389
0.354
0.387
0.342
0.379
0.342
0.377
Table 30: Model efficiency analysis for varying prediction and look-back window lengths.
Inference
FFT + iFFT
Fourier Attention
GPU
Channels
Lookback Len.
Pred. Len.
(s/item)
(s/item)
(s/item)
1
1440
96
0.043
0.0007
0.033
1
2880
1440
0.043
0.0007
0.033
A100
5120
2880
0.043
0.0007
0.033
14400
5120
0.043
0.0007
0.033
Limitations: Inference latency scales linearly with L (since prediction is done step by step), and error may accumulate as the prediction length increases. For these reasons, SOTA TSFMs rarely use this mechanism for output projection.
Sequence to Sequence(seq approach
Here, the model's projection layer is designed directly output the entire prediction sequence of arbitrary length. In our implementation, the backbone output [B, Np. D is processed generate prrddd = N, x patchsize time points, corresponding to the look- back window. At post-processing. outputs are truncated to the required prediction length L. Advantages: Offers flexible output lengths, since the output head does not require specific configuration for each L, making highly suitable for aariable-lenghe forecasting. Inference time is generally insensitive to L, the whole sequence is produced in parallel. This design explains why, in our tests (with a fixed look-back window), inference latency remains nearly constant for different prediction lengths up the input window length. SOTA TSFMs such as TIMER, UP2ME, UniTS etc., adopt this approach.
Note: the distinction between the seq2seq and autoregressive approaches can sometimes ambiguous: for example, while TIMER follows seq2seq implementation, its paper describes the output generation process as "autoregressive..
B.4 VISUALIZATION ANALYSIS
B.4.1 DISTRIBUTION DISCREPANCY TS DATASETS
We conduct measurement analysis on U2G datasets and 5 popular multi-domain datasets for downstream tasks as described in Table 13 and 14 To complement the limited information available in the temporal domain, we transform the datasets into the frequency domain using FFT. This allows to analyze data distribution patterns from various perspectives. including amplitude, phase, periodicity, frequency resolution, etc.. Due to the complexity the joint distribution, we apply non-parametric estimation method, specifically 2-D Kernel Density Estimation (KDE) (Equation I2). to estimate the joint probability density distribution (PDF) of implitude-frequeene and frasencyy for time series data with varying granularities. We 2-D Gaussian kernel
27

---

Published as a conference paper at ICLR 2026
function (Equation 13) and 2-D Scott's rule (Equation 14) bandwidth fuction. Where denotes number of data samples, h is the bandwidth, a and μ are standard deviation and mean of the samples. The results presented in Figure [ It reveals notable discrepancies in the joint distributions across TS datasets with different time granularities. This observation highlights the importance of learning these distribution discrepancies critical knowledge in the process of building universal representation of MTS, which has often been overlooked in previous studies.
f(x.v) nhaly
K
y/
Vi
(12)
h2
by
(a pz)
(y - Myy
K 2
exp
(13)
20
20
h2 = by = n =0=000
(14)
B.4.2 LONG-TERM FORECASTING
To clearly present the results, we select some representative samples for visualization analysis. Figure shows the long-term forecasting results from 4 different datasets. We select 3 typical forecasting results from different dimensions of each datasets.
B.4.3 IMPUTATION
Figure & iillstraats the imputation results from three dimensions across four different datasets. Clearly, GTM can effectively reconstruct the missing data, adapting to varying data patterns.
B.4.4 ANOMALY DETECTION
Figure 9 demonstrates four anomaly events detected by GTM in two datasets, along with their corresponding anomaly scores. The results align precisely with the labeled anomalies the data.
B.5 LIMITATIONS AND FUTURE WORK
Although GTM achieves promising results in multi-task time series analysis, several important limitations remain. First, the current architecture primarily effective for data exhibiting clear periodicity or trend, while its robustness to low signal-to-noise ratio (SNR) or highly irregular time series is not yet fully understood. Future work will focus developing a frequency-domain time granularity- learning module and expanding GTM into a comprehensive Mixture-of-Expert (MoE) framework with gate control mechanisms, aiming further enhance its representation learning capacity and adaptability to complex temporal patterns. In addition, we plan to leverage the GIFT (Aksu al. 2024). larger-scale time series dataset for pre-training and utilize GIFT-Eval for downstream task evaluation, which will provide more rigorous and diverse assessment of GTM's generalization ability. However, the absence unified evaluation protocols and benchmarks-where algorithms are compared under consistent pre-training datasets, hyperparameter settings and ex- perimental conditions remains a significant barrier to fair and reproducible research in the field. Addressing these challenges, including improving model robustness and establishing standardized benchmarking practices, will crucial for advancing time series analysis and realizing the full potential of GTM in both academic and real-world scenarios.
28

---

Published as a conference paper at ICLR 2026
-11
175
(a) ELC dim. 0
(b) ELC dim. 165
(c) ELC dim. 320
2.2
Teme Se
(d) ETThl dim.
(e) ETThl dim. 1
(I) ETThl dim.
g ETTml dim.
(h) ETTml dim.
(i) ETTml dim. 4
15
15
20
15
301
100
(i) Weather dim.
(k) Weather dim. 7
(I) Weather dim. 19
Figure Visualization of forecasting results.
(a) MSL dim6 data
MSL dim9 data
(c) SMAP dim4 data
(d) SMAP dim5 data
(e) MSL dimo score
(T) MSL dim9 score
SMAP dim4 score
(h) SMAP dimS score
Figure 9: Visualization of anomaly detection results.
29

---

Published as a conference paper at ICLR 2026
ELC dim.
ELC dim. 150
ELC dim. 320
Grae ofter
-10
-10
Time
(d) ETThl dim.
(c) ETTh1 dim. 4
ETTh1 dim.
2.3
Cheen
122
05
g ETTml dim.
(h) ETTml dim.
ETTml dim.
60
Weather dim.
(k) Weather dim. 4
Weather dim.
Figure 8: Visualization of imputation results.
30


Reviews:
[{"review_id": "mSOjLF9orq", "reviewer": "Reviewer_B3pb", "summary": "The paper proposes **GTM (General Time-series Model)**, a generative-task-agnostic time-series foundation model that enhances representation learning by integrating temporal and frequency-domain features. It introduces a Fourier attention mechanism to capture time-granularity-aware patterns and a unified pre-training framework (hybrid masking, 2D positional encoding, span shuffling) that unifies reconstruction and autoregressive objectives. GTM seamlessly adapts to diverse generative tasks (forecasting, imputation, anomaly detection) without task-specific modifications, outperforms SOTA models across benchmarks, and exhibits clear scaling behavior with model size and pre-training data.", "questions": "1. Comparing to the models listed in the baseline, how much more/less computational resource does GTM consume? Can GTM still remain high performance if the fourier transform calculation time is limited?\n2. In the Fourier attention mechanism, different frequency patterns are taken into attention calculation by forming a vectorized feature. Does this strategy outperform mixing (by MLPs for instance) these frequency patterns into a representation embedding?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper addresses a critical gap in existing time series foundation models by introducing a Fourier attention mechanism that explicitly captures time-granularity-aware patterns from the frequency domain.\n2. As far as I am concerned, GTM is the first TSFMs that supports seamless adaptation to all generative tasks (forecasting, imputation, anomaly detection) without task-specific modifications (e.g., tokenization adjustments, projection header changes). \n3. The paper conducts rigorous experiments across diverse benchmarks with fair and consistent enhancements, including 5 datasets for forecasting/imputation, 5 for anomaly detection, and 10 for classification.", "weaknesses": "1. The paper does not deeply explore the computational overhead of the Fourier attention mechanism—specifically, how FFT/iFFT operations (integrated into each decoder block) impact inference latency for real-time applications (e.g., streaming sensor monitoring). \n2. The model analysis of this paper is insufficient. For example, no trade-off analysis (e.g., simplifying frequency modules to reduce latency) is provided, which is critical for practical deployment.\n3. The TSFM baselines are outdated. Please include more additional baselines such as TabPFN, Sundial, TimeMOE.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ljwZvHlRQJ", "reviewer": "Reviewer_bvHo", "summary": "This work introduces GTM, a general model for generative time series learning. The authors claim that this model is one of the first to be task-agnostic for generative tasks, allowing the model to be applied in a variety of settings without task-specific modifications. The core contribution comes down to the featurization of the time series, where the authors combine many previous methods and use multiple decompositions of the time series within the model. There are also some interesting architectural choices that enable the model's performance. The model performs well on downstream benchmarks, with the authors undergoing significant downstream testing.", "questions": "- Have the authors tested the model on long-range datasets? What is the maximum length of the datasets used in this paper?\n- Are the results over baselines statistically significant?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "- The writing is clear and straightforward, laying out the reasoning for the model components and describing them in sufficient detail. The architecture seems to have a lot of backing from literature, and the authors build nicely on previous work.\n- It is encouraging to see that the combination of lots of disparate components of time series analysis results in a performative model. I think this is an important contribution to demonstrate, and the authors motivate the use of these features very well. \n- The experiments in the paper are extensive and thorough, demonstrating the model’s applicability on a variety of downstream tasks. In addition, the appendix has a detailed number of experiments, which is helpful in observing model performance.", "weaknesses": "- One of the biggest weaknesses of the model is the resources required to run this architecture on practical time series data. A number of components of the model are very computationally expensive, including the full temporal attention and the internal FFT calls in the model. Some analysis of computational efficiency would be helpful.\n- The performance, while better than baselines across most tasks, is only marginally better than other models. In the field of time series foundation models, where a plethora of models exist that all perform better or worse in a specific setting, this is not very impressive. \n- There are no error bars anywhere in the results, making it difficult to determine significance of the observed gains in performance.\n- There are a number of suspicious results from the ablations, including that the pretraining is not much better than training on specific datasets, indicating that the “generality” of the model might not be optimal. In the case where the pretraining does not help, the authors would need to test against task-specific models as the regime becomes different. In addition, the boost in model performance is marginal in the scaling law study, raising questions as to the generality of the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7xdVRH6TBe", "reviewer": "Reviewer_mmJe", "summary": "The paper introduces the General Time-series Model , a novel foundation model based on the empirical finding that time-series data exhibits distinct frequency-domain distributions  at different temporal granularities . To exploit this, GTM incorporates a novel Fourier attention mechanism that learns to capture these granularity-aware representations. A key contribution is its unified pre-training strategy, which uses a hybrid masking approach to jointly optimize reconstruction and autoregressive objectives. This makes GTM the first generative-task-agnostic model, enabling it to handle diverse tasks like forecasting, imputation, and anomaly detection without any task-specific modifications. Extensive experiments confirm that GTM consistently outperforms or matches state-of-the-art models on these generative tasks, achieves strong results on classification, and demonstrates effective scaling with increased model size and data.", "questions": "Listed in Weakness", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is exceptionally well-motivated. It begins with a clear, empirical analysis using FFT and 2D KDE to demonstrate that the joint probability distributions of amplitude-frequency and phase-frequency vary significantly across different temporal granularities . This finding provides a strong, intuitive justification for the core architectural novelty of the proposed model.\n\n2. The paper introduces a novel Fourier attention mechanism designed to explicitly capture the granularity-aware features identified in the initial analysis. Instead of just applying a standard FFT, the module learns to weigh different frequency-domain transformations based on the input's time granularity. This is a clever method to inject domain-specific knowledge about time-scale properties directly into the learning process.\n\n3. A key contribution is the novel pre-training strategy that unifies reconstruction and autoregressive objectives. By adopting a GLM-style hybrid masking approach—which combines random span masking  with consecutive tail masking —the model is trained to handle a variety of generative tasks from the outset.\n\n4. This unified pre-training strategy successfully establishes GTM as a generative-task-agnostic model. The authors demonstrate that the same pre-trained model can be applied to forecasting, imputation, and anomaly detection without any task-specific architectural modifications, which is a significant step toward a true \"foundation model\" for time series.", "weaknesses": "1. The hybrid masking strategy is a core contribution, but a critical detail is underspecified. The paper mentions applying a \"controlled proportion of consecutive [MASK] tokens to at the tail\". The value or control mechanism for this proportion, which presumably balances the reconstruction and autoregressive objectives, is not defined in the main paper or the appendix's implementation details .\n\n2. The proposed Fourier attention module adds a significant number of operations within each decoder block: a standard temporal attention, an FFT, a granularity-based attention, five low-rank matrix multiplications, one full-matrix multiplication, and an inverse FFT . While the authors commendably provide an efficiency analysis in Table 24, which shows GTM is competitive, this added complexity is a non-trivial tradeoff for the performance gains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xGwweeYKSO", "reviewer": "Reviewer_boeL", "summary": "The paper proposes a time-series model that incorporates frequency-domain information in self-attention, a pre-training strategy that combines reconstruction and autoregressive objectives, and it shows that this model (after being trained on large dataset) it can be considered as a foundation model. Experimental results on a large number of time series datasets for different tasks shows that the model is competitive to existing ones, and in many cases leads to higher performance.\n\nThe main argument for the improved performance of the proposed model is the use of frequency domain features, something that the existing models are not exploiting. This argument has been used in many cases in the past, and it does not seem to be well-justified: no theoretical explanation is provided to prove the advantage of using a specific spate-time domain transformation (the Fourier transform) for internal neural network representations, while the experimental comparisons do not show a clear boost in performance that would be attributed to the adoption of a clearly better type of information.\n\nOverall, the experimental analysis is comprehensive and it shows advances for the proposed model.", "questions": "- What is the computational and memory costs of the proposed model?\n - How does the model compare with existing ones in inference speed?\n - Comparisons with more recent related models.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The proposed model shows to achieve comparable or even better (on average) performance compared to existing models in a variety of tasks and datasets.\n - A comprehensive experimental evaluation is provided.", "weaknesses": "- No computational and memory cost analyses are provided for the proposed model.\n - No comparisons in terms of inference time are provided for the proposed model in relation to the competition.\n - Comparisons with recent related models, e.g., UniTS, Moirai, ST-LLM, are not provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)