TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting
Authors: Jaebin Lee
Hankook Lee
Venue: NeurIPS 2025 poster ()
Keywords: Time Series Forecasting
Transformer
Attention
Self-Supervised Learning
Primary area: deep_learning

Abstract: In machine learning, effective modeling requires a holistic consideration of how to encode inputs, make predictions (i.e., decoding), and train the model. However, in time-series forecasting, prior work has predominantly focused on encoder design, often treating prediction and training as separate or secondary concerns. In this paper, we propose TimePerceiver, a unified encoder-decoder forecasting framework that is tightly aligned with an effective training strategy. To be specific, we first generalize the forecasting task to include diverse temporal prediction objectives such as extrapolation, interpolation, and imputation. Since this generalization requires handling input and target segments that are arbitrarily positioned along the temporal axis, we design a novel encoder-decoder architecture that can flexibly perceive and adapt to these varying positions. For encoding, we introduce a set of latent bottleneck representations that can interact with all input segments to jointly capture temporal and cross-channel dependencies. For decoding, we leverage learnable queries corresponding to target timestamps to effectively retrieve relevant information. Extensive experiments demonstrate that our framework consistently and significantly outperforms prior state-of-the-art baselines across a wide range of benchmark datasets.

:IMEVERCEIVER: An Encoder-Decoder Framework for Generalized Time-Series Forecasting
Jaebin Lee Sungkyunkwan University jaebin. leedskku. edu
Hankook Lee Sungkyunkwan University hankook lee@skku edu
Abstract
In machine learning, effective modeling requires a holistic consideration of how to encode inputs, make predictions (i.e., decoding), and train the model. How- ever, in time-series forecasting, prior work has predominantly focused on encoder design, often treating prediction and training separate secondary concerns. In this paper, we propose TIMEPERCHIVER, a unified encoder-decoder forecast- ing framework that is tightly aligned with effective training strategy. To be specific, we first generalize the forecasting task to include diverse temporal pre- diction objectives such extrapolation, interpolation, and imputation. Since this generalization requires handling input and target segments that are arbitrarily po- sitioned along the temporal axis, we design novel encoder-decoder architecture that can flexibly perceive and adapt to these varying positions. For encoding, we introduce input segments to jointly capture temporal and caassel dependencies. For decoding, we leverage learnable queries corresponding to target timestamps effectively retrieve relevant information. Extensive experiments demonstrate that our framework consistently and significantly outperforms prior state-of-the-art baselines across a wide range of benchmark datasets. The code is available at https//ggiiuubcoom//fffiittttttt wwwpiiinglaa//tmeeeeeeeeee PerePerciieer..
of latent bottleneck representations that can interact with all
Introduction
Time-series forecasting is fundamental task machine learning, aiming to predict future events based on past observations. It is of practical importance, as it plays crucial role in many wollwworld applications, including weather forecasting [1], electricity consumption forecasting [2]. and traffic flow prediction [3]. Despite decades of rapid advances in machine learning, time-series forecasting remains a challenging problem due to complex temporal dependencies, non-linear patterns, domain variability, and other factors. In recent years, numerous deep learning approaches [4-18] have been proposed to improve forecasting accuracy, and it continues to be an active area of research.
One promising and popular research direction is to design a new neural network architecture for time-series data, such as Transformers [49]], convolutional neural networks (CNNs) [11, 12], multi- layer perceptrons (MLPs) [33-15]. and state space models (SSMs) [17, 18]. These architectures primarily focus on capturing temporal and channel (i.e., variate) dependencies within input signals, and how to encode the input into a meaningful representation. The encoder architectures are often categorized into two groups: -andeeenndeeenneent encoders, which treat each variate separately and apply the same encoder across all variates, and channel-dependent encoders, which explicitly model interactions among variates. The channel-indeppedent encoders are considered simple yet robust [19]; however, they fundamentally overlook cross-channel interactions, which can be critical for multivariate time-series forecasting. In contrast, the channel -deenneed encoders [5, 6, 8] can inherently capture such cross-channel dependencies, but they often suffer from high computational
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

X1: N2.
X
Ni. M2.
M1ol
AM Nuh
Inguu
Target
Input
I- 12 3:4
Target
- -1.6...0000 8.10
r Mmmmmnn
m
fo
Mm
Mm
go
r
Xpace 181,
Xeuture
810
Xx (x,, Xy, Xx,
X,
(a) Standard formulation
(h) Generalized formulation (ours)
Figure I: (a) The standard time-series forecasting task aims to predict only the future values from past observations. In contrast, (b) our generalized task formulation aims to predict not only the future, but also the past and missing values based on arbitrary contextual information.
cost and do not consistently yield significant improvements in forecasting accuracy over channel- independent baselines.
While the encoder architecture is undoubtedly core component of time-series forecasting models, it is equally important to consider (i) how to accurately predict (i.e., decode) future signals from the encoded representations of past signals, and (ii) how effectively train the entire forecasting model. However, they often been studied independently. and little attention has been paid to how to effectively integrate them. For decoding, most prior works rely on simple linear projection that directly predicts the future from the encoded representations. This design offers advantages in terms of simplicity and training efficiency, but may struggle to fully capture complex temporal structures. For training, inspired by BERT [20], masking-and-reoom recosstrcntttiinnnn tasks [4, 16] have been commonly adopted pretrain encoders before supervised learning for time-series forecasting. In the pretraining stage, subset of temporal segments from the time-series data is masked, and the encoder learns to reconstruct the masked portions. Despite its effectiveness, the self-supervised learning approach and the two-stage training (i.e., (inernnnnnnftnuunnnng strategy remain questionable whether it is truly aligned with architectural designs (i.e., encoders and decoders) of time-series forecasting models.
Contribution. In this paper, we propose PIREPERC,,VER, unified framework that tightly integrates an encoder-decoder architecture with an effective training strategy tailored for time-scries forecasting. Our key idea is to generalize the standard forecasting task formulltee as predicting future values from a sequence of past observations- intto a broader formulation that encompasses extrapolation, interpolation, and imputation tasks along the temporal axis (see Figure 1). In this setting, the model learns to predict not only future values, but also past and missing values based on arbitrary contextual information. This generalized formulation enables the model to jointly learn temporal structures and predictive behavior in a single end-to-end training process, thereby fostering deeper understanding of temporal dynamics and eliminating the need for separate pretraining finetuuing pipeline.
To support our formulation, as illustrated in Figure 2, we design a novel attention based encoder- decoder architecture that can flexibly handle arbitrary and potentially discontinuous temporal seg- ments unlike conventional models that operate on fixed-length lookback windows and predict fixed- length forecasting horizons. Specifically, our encoder utilizes the cross-attention mechanism (i) to encode an arbitrary set of temporal segments into fixed-size set of latent bottleneck representations, and (ii) to contextualize each segment by leveraging the bottleneck representations. This bottleneck process enables the encoder to efficiently capture both temporal and cross-channel dependencies. To enhance the quality of the bottleneck representations, we also apply the self-attention mechanism within the bottleneck set. After encoding input segments, our decoder generates predictions via cross- attention between the representations of the input segments and Iearnable queries that correspond to target timestamps. This allows the decoder to selectively retrieve relevant information of the input and to produce temporally-awwe outputs. This design is naturally aligned with our learning objective, which includes the forecasting task as part of a broader set of temporal prediction tasks.
Through extensive experiments, our framework achieves state-of of-the-art art performance compared to recent baselines [4, 6-9, 11, 13, 14, 17] on standard benchmarks [1-3, 21, 22] (see Table 1). Notably, our model achieves 55 best and 17 second-best scores out of 80 settings, which are averaged over three different input lengths, demonstrating its strong overall performance. As a result, our framework records the best average rank with 1.375 in MSE and 1.550, indicating its consistent superiority

---

over the baselines. We also conduct ablation studies to verify the effectiveness of each component (Section 4.2) and analyze attention maps to gain insights into how the model operates (Section 4.3).
Overall, our work emphasizes the importance of aligning architectural design with task formulation in time-series forecasting. We hope that this perspective encourages a shift from encoder-centric designs toward unified approaches more closely aligned with the core objectives of time-series forecasting.
Preliminaries
2.1 Problem Statement:
Multivariate Time-Series Forecasting
In this paper, we aim solve the task of multivariate sene-serieseee forecasting, which requires to predict future values of multiple variables based on past observations. To formally define the task, let E R denote a multivariate observation at time step t, where C' is the number of variables (or channels). A multivariate time series of length 7 can be written
sequence X [x1, ..,..., Xr) E C Given lookback window X pust = [x ++t-+,,,... X₂] length I,, the goal of the task is to predict the future values X
X ................................ X++H] over a forecasting horizon of length H.
Solving the task requires effectively capturing key characteristics of time-series data, such tem- poral dependencies and cross-variable interactions. A common approach is to design forecasting architecture Se that can model such properties, and to train it using simple objective such mean squared eπror between the predicted and ground-truth future values as follows (see Figure la):
H
[(;; X pasts X Future)
Σ
&
X
X funure [8 & = (Xpss)) (1)
HC
This standard formulation, commonly used in many forecasting models and benchmarks [4, 6, 8, 22), assumes that both the lookback window and the forecasting horizon are fixed length and continuous.
2.2 Attention Mechanism
In this section, we formally describe the attention mechanism [23], which plays an important role in our TIMEPERCEIVER framework. It designed to dynamically capture dependencies between input tokens by computing their contextual relevance through learned similarity scores. Specifically, given the attention output is computed as:
queries Q IR N xx keys K RMxx and values V IR Mxd
QKT
Attention Q K, V) Softmax
V.
(2)
Vd
This has recently become a standard architectural component across various domains, including including NLP [23]. vision [24], tabular data [23], and time-series forecasting [4-7]. A common design for modular blocks combines attention with skip connections and feedforward networks (FFNs):
H U Attention (U, Z, Z),
AttnBlock(UU (U H FFF(H...
(4)
where U E Nxd and Z E Mxd denote input and context tokens, respectively. For simplicity, we
omit learnable parameters, layer normalization [25], and multi-head attention [23]. When U Z. the block corresponds to self-attention; otherwise, it performs cross-attention, allowing input tokens attend an external context.
In the context of time-series forecasting, attention-based models adopt various tokenization strategies capture temporal and multivariate patterns. A common approach divide the input time series into contiguous fixed-length temporal segments, or patches, and treat each patch as token [4. 5]. Alternatively, DefOrmableTST [7] selectively samples important time steps and treats them as tokens to capture temporal patterns, while iTransformer [6] represents the entire time series of each channel as single token. In this work, we simply adopt the common approach.
3 Methodology
In this work, we propose TIMEPERCEIVER, unified encoder-decoder framework for generalized time-series forecasting. Our framework is based on generalized formulation of the forecasting task

---

TIMEPERCEIVER
Parametzfiedd Modules
X.
Encoding
($3.3)
Arbitrary Patch Embedding (53.2)
Block
X
X
Latent Tokens
m
Channett
Renuentan
Impor
1PE
Inout Tokens
Time
Decoding (53.4)
Pachiyy
Randon Spit
Prediction
Block
Tokeems
Figure 2: An overview of our TIMEPERCEIVER framework.
(Section 3.1 and 3.2) and consists of two main components tailored to the formulation: an encoder that jointly captures complex temporal and channel dependencies (Section 3.3), and decoder that selects queries corresponding to target segments and retrieves relevant context (Section 3.4), Our overall framework is illustrated in Figure 2.
3.1 A Generalized Formulation of Time-series Forecasting
While the formulation described in Section 2.1 explicitly follows the forecasting task definition, inherently assumes a unidirectional temporal flow (i.e., from past to future), which may limit the ability to fully capture the underlying temporal dynamics. To address this limitation, we propose generalized formulation that extends the forecasting task to allow flexible conditioning on arbitrary temporal segments, enabling the model to learn a more comprehensive understanding the temporal dynamics. To define our generalized formulation, we begin by introducing notation for time series of arbitrary temporal segments. Given a multivariate time series X [x1, X2, xx]] and set of time indices I = (i1, (21,92,-. ........ 1|z] (1..... T}, we define the corresponding series X, as follows:
Xx [x: ZJ [x xizzi E Cx
Now, the generalized forecasting task can be defined by specifying a subset time indices I as the input context and treating the remaining indices task, a generalized forecasting model ge expected to operate on any choice of I and J, and to predict the values of the target indices follows (see Figure lb);
{I....,T) I as prediction targets. In this
go (X,, I.J).
(6)
The model is trained to minimize the mean squared error over the target indices J, formulated as:
((0;X,I,J)
Σ | 1 JEJ
X = & J| go II,IJJ).
IJC
This generalized formulation allows both the input and target time indices be arbitrary and non- contiguous, thereby encompassing extrapolation, interpolation, and imputation tasks and capturing more complex temporal patterns. For example, the standard model fo can be seen its special case:
fo (Xpst) = 8e(Xaast {t L I..... t), ft 1...., H3).
(8)
3.2 Constructing Arbitrary Patch Embeddings
Leveraging patch-based representations has recently become important design choice in time-series forecasting models [4] inspired by the success of the Vision Transformer [24]. Motivated by this trend, we extend our formulation in Section 1.1 to its patch-based version. To this end, given multivariate time series X E RCXT we first divide the time index set {1, 2,..... ,T) into disjoint subsets Pi, P2.... ... PN where P; ((i 1)P 1, 1....,i2 iP) and P = T/N is the patch length. Here, we assume T is divisible by overlapping patches along the temporal axis, denoted as Xp,..... X where each patch is defined
notational simplicity. This results in N fixed-length and non-

---

as xp = R : P.I E RCx, For our task formulation, we consider an arbitrary set of patch indices Inatch C f1.2....... ... and define the remaining set as Jpach = {I...... ........... .N3 Ipuch- This corresponds to selecting input and target time indices as I = Urezun. P; and J = JEJm Pj. respectively. Given the lookback window length L in the forecasting task, we set the number of selected input patches as p = L/P. i.e., [ L.
To encode the structural information of patches Xp,...., Xpv. we introduce two learnable positional embeddings: (7) temporal positional embedding (TPE), denoted as Etemporal E NR which encodes the temporal location of each patch, and (ii) channel positional embedding (CPE), denoted as Eehannel E TR CxD, which represents the identity of each channel. Using these positional embeddings,
we construct the input patch embedding ())) and the query patch embedding Q()) as follows:
H Picc W
E
Echannel
E
Eeeporra
E IR
Vi E Ipatch: Ve E (I,...., C)2
(9)
imput
Echannel
temporal
IR
vj E Fpucch: Ve E (1)...,c)
(10)
where P.,e E IR denotes the raw values of the i-th patch for channel e, and W Winput PR is a learnable input projection matrix. Note that H and Q()) are treated as sequences of embedding vectors Section 3.3 and 3.4, i.e., H E [[CI)ual) and Q!!) [(C)Jpul))
3.3 Encoding with Latent Bottleneck
Given the input patch embeddings H constructed in Section 3.2, our encoder aims to efficiently capture both temporal and cross-channel dependencies within the embeddings for time-series fore- casting. To this end, we propose latent bottleneck, a two-stage mechanism in which input tokens are first compressed into a fixed number of latent tokens and then projected back to the input tokens. This design enables efficient and selective modeling of key dependencies while avoiding the computational overhead of full attention.
We now formally describe our attention -assed encoder based on the latent bottleneck mechanism. To this end, we introduce a set of learnable latent tokens ( E RD where M is the number of latent tokens. Since these tokens adaptively model key dependencies, M can be significantly smaller than the number of input tokens Cl Staring from O( the latent bottleneck operates as follows:
1. The latent tokens attend to the input to collect contextual information:
() AttnBlock(Z H H E MxD
2. The latent tokens are refined via K self-attention layers:
z (-1) AttnBlock(Z** (k) (*), ) Vk L,...,K.
3. The updated latent tokens are used to update the input tokens:
H AttnBlock(H (K+1), ((K+))) [CCTTTpal)
This design acts as an efficient attention bottleneck that significantly reduces computation from O(N²) in full attention to O(N while selectively preserving informative patterns across both temporal and channel dimensions. We provide further ablation studies on the encoding mechanism and the latent bottleneck in Section 4.2 and Appendix G.
3.4 Decoding via Querying Target Patches
Based on the query patch embeddings (()) described in Section 3.2 and the encoded input patch embeddings H (1) from Section 3.3, our decoder is designed to generate predictions Xp, for target patches Pj. where j E Jpucch- Note that Q R(CIJaaa)) consists of C' x |Jpuichl query tokens where each token Q (0) gechnnel + 1 Intemporal E _ corresponds the query vector for channel and
patch P,, as defined in Section 3.2.
To retrieve relevant information from the input for each query, we apply cross-attention using Q as queries and (1) as keys and values:
Q (1) AttnBlock Q (0) (1), H! E @(CIspaa)
5

---

Table 1: Multivariate long-term forecasting results compared with baseline models. Bold indicates the best result, and underlined denotes the second best. The results are averaged over input lengths L E {96, 384, ]68} and prediction lengths H E (96, 192, 336, 720). Rank indicates the average position of each model in MSE and MAF across all prediction lengths. Results for each individual
input length are provided in Appendix E.
Models
TimePrrevieer
DeformNieTT
CARD
CATS
PotchTST
l'ransfomeer
S-Mamia
Dl.inear
Timesher
TDDE
Metric
MSF
MAP
MSE
MAR
MAR]
MSE
MAR]
MSE
MAF
MSE
MAR
MSE
MAR
MAF
MSE
MAR
MSR
MAR
96
10.147
0.199
0.155
0.203
0.1166
0.208
0.150
0.201
0.15%
0.207
0.166
0.214
0.162 0.212
0.178
0.238
0.1169
0.221
00.183
0.238
Weather
192
0.193
0.242
0.200
0.244
0.214
0.258
0.146
0.244
0.204
0.248
0.210
0.253
0.209
0.254
0.221
0.278
0.257
0.278
0.225
0.274
336
0.244
0.252
0.284
0.263
0.295
0.249
0.284
0.255
0.286
0.263
0.24/
0.266
0.319
01.270
0.309
720
0.323
0.338
0.325
0.335
0.354
0.351
0.322
0.338
0.3228
0.336
0.335
0.343
0.334
0.342
0.328
0.366
0.385
0.377
0.333
0.357
AAg [0.227
0.265
0.233
0.266
[0.247
0.278
0.229
0,266
0.236
0.269
0.214
0.276
0.243
0.276
0,219
0.300
[0.27]
0.299
0.25910.251
0.295
96
[0.173
0
0.177
0.237
0.194
0.257
0.184
0.228
0.207
0.285
0.188
0.242
0.191
0.242
0.232
0.322
0.222
0.291
0.226
0.324
Solur
1.2
0.119
0.199
0.254
0.210
0.231
0.212
0.263
0.219
0,341
0.318
0:277
0.351
326
0.206
0.248
0.208
0.263
0.230
0.289
0.218
0.253
0.249
0.312
0.226
0.275
0.229
0.279
0.280
0.355
0.275
0.337
0.291
0.367
720
0.217
0.259
0.214
0.266
0.251
0.301
0.233
0.26
0.250
0.314
0.232
0.282
0.236
0.284
0.291
0,360
0.276
0.329
0.300
0.361
Avg
00.198
0.241
0.199
0.255
0.228
0.282
0.211
0.247
0.234
0.303
0.214
0,266
0.219
0.268
0,266
0,345
|0226]
0.319
10.277
0.351
Elearieity
192
96
00.228
0.146
0.240
00:44
0.155
0.240
0.254
00.140
0.161
0:236
0.257
0.134
0.153
0.245
0.228
0:14%
0.163
0.346
0.256
0.143
0.160
0.256
0.240
0.157
0.136
0.252
0.253
D. 0.164
0.173
0,258
0.268
10.188
0.197
0.294
0.303
(0.180
0.186
002282
0.287
336
0.165
0.239
0.177
0.271
0.181
0.277
1.121
0.180
0.276
0.117
0:274
0.177
0.270
0. 1888
0.285
0.229
0.327
0.202
0.301
?30
0.211
0.200
0.303
0.201
0.292
0.218
0.306
00,220
0.313
0.19%
0.223
0.239
0.337
01,23
0.328
Avg [0.16]
0.254
0.110
0.267
00.174
0:260
06166
0:257
[0:177
0.271
10.175
0.271
0..166
0.262
0.187
0.282
|0.215
0.315
[0.201
00300
96
[0.38]
0.263
0.378
0.265
0..399
0.279
0.379
0.255
0.407
0.280
0.389
0.286
0.357
0.254
0.493
0.329
[0.571
0.310
0.310
0.100
Traffic
192
0.394
0.265
0.401
0.272
0.415
0.283
0.400
0.366
0:417
0.285
0.410
0.298
0.371
0.262
0:482
0.523
0.584
0.316
0.569
0.397
336
0,400
01,433
0.287
0.793
0,413
0.277
0,430
0.289
0.4428
0.597
0.317
0.588
0,404
720
0.445
0.293
0.448
0.300
0.461
0.308
0.448
0.291
0.165
0.307
0.168
0.331
0.438
0.295
0.530
0.350
0.619
0.330
0.588
0.105
Avg
00407
0.273
0.410
0,280
004426
0.291
0.411
0.272
0.430
0.290
0.424
0.306
0.367
0.270
0.499
0.333
|0.593
0.318
0.318|0.579
00.579
0.101
96
0.366
0.392
0.370
0.399
0,334
0.400
0..375
0.390
0.410
0.400
0,420
0.392
01416
0,400
(0.4%
0.438
(1,49
0.459
122
0,402
0-115
0.25
0,-21
0.426
0.419
0.432
0.429
0.446
0.437
0,418
0,426
0.481
0.467
0.497
0.481
320
D:425
0,431
0:411
0.420
0,451
0.445
0.443
0.445
0.458
0.452
0.472
0.466
0.470
0.467
0.481
0.451
0.561
0.503
0.525
0.499
720
0.450
0.461
0.456
0.469
0.463
0.473
0.476
0.4770
(1:473
0.481
0.533
0.520
0.514
01508
0.495
0,508
0.615
0.533
0.544
0.529
Axg
10.410
0.413
0.490
00.450
0.438
00.478
0.438)
0144%
0.444
00.461
0,463
0.445
0.459
0,447
[0.524
0.485
00,505
0.492
96
0:278
01336
00224
0.334
10,283
0.339
012282
0.342
00.784
0.303
0.
0.258
0.354
0,321
0,380
303375
0.405
[0,378
0:433
192
0.347
0.588
0.334
0.374
0.355
0.384
0.348
0.385
0.354
0.384
0.396
0.415
0,371
0.400
0.436
0.451
0.416
0.438
0.468
0.185
336
0.354
0.407
0.318
0.577
0.382
0.409
0.372
0.405
0.378
0.408
0.427
0,438
0.403
0.425
0.533
0.508
0.469
0.469
0.524
0.522
730
0.395
0..0%
0.497
0.431
0.418
0.409
0.439
0.432
0.454
0.423
0.448
0.847
0.656
0.484
0.482
(1,24
0.579
Avg
00.344
0.392
0.336
0.381
00.355
0.391
0.335
0.395|
0.356
0.394
0.350
0.417
0.374
0.407
0.535
0.499/0.436
0.499
|0.436
0.450|
[0.19
0.505
96
002991
0.339
0,300
0.350
00.306
0.349
0.296
0.342
0.305
0.354
0.321
0.368
0.317
0.365
0.318
0.356
[0.351
0.383
10.331
0.367
192
0.329
0.365
0.338
0.374
0,350
0.374
0.333
1.399
0.345
0.377
0.359
0.389
0.357
0.387
0.351
0.375
0.384
0.396
0.362
0.383
336
0.357
0.368
0.396
0
0.394
0.365
0.393
0.377
0.400
0.399
0.412
0.388
0.407
0.414
0.415
0.393
0.402
720
0.412
0.415
0.426
0.426
0.435
0.421
0:421
0.12
0.432
0.435
0.463
0.450
0.448
0.443
0.436
0.428
0.501
0.463
0.447
0.134
Avg
0.347
0.375
0.358
0.386
00.368
0.386
0.354
0.382
0.3655
0.391
0.366
0.405
0.377
0.401
0,372
0.388
J0.412
0.414
|0.383
0.397
96
00.66
0.254
0.172
0.259
001668
0.257
0.173
0.2611
0.177
0.262
0.181
0.271
0.179
0.268
0.173
0.268
10.216
0.290
(1.194
0.20%
ETTm2
192
0.225
0.233
0.298
0.234
0.301
0.237
0.307
0.239
0.244
0.313
0.239
0.509
0.328
0.262
0.323
(.262
0.343
336
00.283
0.327
0.289
0.356
0.283
0.333
0.288
0.339
0.296
0.345
0,301
0.349
0.294
0.345
0.325
0.386
0.320
0.360
0.330
0.389
720
00,371
0.384
0.371
0.389
0.386
0.395
0.337
0.395
(1.382
0.396
0,397
0.407
0.380
0.396
0.453
0.457
0.446
0.431
0.454
D.463
|Agg 00.261
0.267
0.321
0.2268
0.921
0.2269
0.336
0.273
0.327
0.335
0.273
0.530
|0.311
0.351
|03300
0.373
Rank
1.375
1.550
2,525
2.800
4.975
4.375
2,850
2.700
5,450
5.225
6.475
6.650
4.950
5.175
7.575
7.750
99.300
8.825
9.175
9.225
Final predictions for each channel and patch are then computed via a learnable linear projection:
Ve E (I, Vj E Jpmnch: Xp, = Q W oomius E R P. where W output E R PxP. e.j
This query-driven decoding mechanism allows the model to selectively retrieve and reconstruct the missing target patches based on their positional embeddings and contextualized encoder outputs. By actively learning to infer meaningful patterns from diverse and dynamically sampled input contexts, this decoder design enhances the model's 's ability capture temporal structure and inter-channel dependencies without relying on explicit supervision. This decoding strategy is a key component of our generalized forecasting formulation.
4 Experiments
We design our experiments to investigate the following:
Does our framework consistently outperform baselines across diverse benchmarks? 1$4.1)
How does each component of our framework contribute to overall performance? ($4.2)
How is our model designed to organize information flow through attention? ($4.3)

---

Table Performance comparison across different formulations, encoder attention mechanisms, and positional encoding (PE) sharing strategies. Bold indicates the best result within each configuration (i.e,, each unique combination of formulation, encoder attention, and PE sharing). A lower MSE or MAF indicates better performance. The last row in each configuration block corresponds to our proposed model. Each column reports the average performance over four prediction lengths
H E {96, 192, 336, 720) with input length L 384.
ETTh1
ETTml
Solar
ECL
Formulation
Encoder Attention
PE Sharing
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Standard Generalized
Latent Bottleneck
Sharing Sharing
0.420
0.437
0.355
0.382
0.194
0.243
0,169
0.265
Latent Bottleneck
0.404 0.420
0,338 0.370
0.182
0.233
0.157
0.249
Generalized
Full Self- Attentttentonn
Sharing
0.425 0.434
0.353
0.375
0.192
0.237
0.166
0.254
Generalized
Decoupled Self-Attention
Sharing
0.423
0.433
0.356
0.379
0.189
0.233
0.158
0.250
Generalized
Latent Bottleneck
Sharing
0.404 0.420
0.338
0.370
0.182
0.233
0.157
0.249
Generalized Latent Bottleneck
Not Sharing
0,423
0,432
0.342
0.374
0.193
0.232
0.163
0.254
Generalized
Latent Bottleneck
Sharing
0.404 0,420
0.338
0.370
0.182
0.233
0.157 0.249
Baselines. To ensure comprehensive comparison, we include strong recent baselines such as De- formableTST [7], CARD [8], CATS [9], PatchTST [4], and i'Transfommer [6]. which are Transformer- based, well models including SSM-based S-Mamba [17], CNN-based TimesNet [11], and MLP-based DLinear [13], and TiDE [14]. Our focus on Transformer-base methods reflects the architecture of our proposed model. Among these, CARD [8]. i'Tranffrmmer [6], S-Mamba [17], and TimesNet [11] can be categorized as channel-dependent models due to their explicit modeling of inter-variable interactions, while the are considered channel-independent. Additionally, we
provide comparisons with other baselines in Appendix I.
Setup. We evaluate model 8 real-world multivariate time series datasets, including ETT (ETThl, ETTh2, ETTml ETTm2) [22], Weather [1], Solar [21]. Electricity [2], and Traffic [3] to ensure robustness across diverse temporal patterns and domains. Detailed dataset explanation is described in Appendix A. To ensure fair and robust evaluation with other models, we average results across multiple input lengths (96, 384, 768) for each prediction length (96, 192, 336, 720), egg., the main comparisons in Table 1. For results reported in Deformable [7]. we directly refer to their values, while missing entries are reproduced under the same settings. All other experiments, including ablations and model-specific analysis, are conducted with fixed input length of 384. Training details can be found in Appendix B.
4.1 Multivariate Long-Term Time-Series Forecasting Results
The forecasting performance summarized in Table We evaluate forecasting performance using MSE and MAE, where lower values indicate better results. Across 8 datasets and 4 forecasting horizons, with two metrics per setting (MSE and MAE), our method achieves 55 best and 17 second best scores out of 80, establishing a new state-of-the-art, Notably, our model significantly outperforms recent
methods such as CARD [8] and iTransformer [6]. When averaged across all datasets, our model achieves 5.6% improvement in MSE and a 4.4% improvement in MAE compared to CARD. Against iTransformer. the improvements are even more substantial, with 8.5% lower MSE and 7.3% lower MAE, highlighting our model's 's strong capability in capturing complex temporal-channel dependencies.
4.2 Component Ablation Studies
We present component-wi wise analysis of our forecasting framework, focusing on three key design decisions: the formulation of the forecasting objective, the encoder attention mechanism, and the decoder design. Each choice offers multiple alternatives, and understanding their impact critical to the final model performance. Table 2 summarizes the performance of various configurations. In the following, we analyze each component detail and justify the decisions behind our proposed architecture. Full results are provided in Appendix F.
7

---

Nully contiguous
fully disjonnt
Morizon=96
Horizon=336
15%
MSE Reductton (%6
10%
10%
3%--
3%-
ETTh2
Solar
ETTh2 Solar
Figure 3: Relative MSE reduction (%) compared to the standard formulation with varying sampling strategies for target patches under imput length L = 384 and two prediction lengths H (96, 336]. Sampling strategies are (a) fully contiguous: only one contiguous segment in Jpatch. (b) fully disjoint: selected target indices are fully arbitrary, and (c) mixed: two equal-sized segments in Jpanch. Higher values indicate greater improvements over the standard formulation.
Formulation. We compare the Standard Formulation, where the model is trained to predict future values given a continuous segment of past observations, against the Generalized Formulation, which unifies extrapolation, interpolation, and imputation within forecasting task described in Section 3.1. This broader formulation allows the model to learn from more diverse temporal prediction objectives by randomly sampling prediction targets along the temporal axis. The results show that our general- ized formulation consistently yields better performance across all datasets and metrics, achieving an average improvement of 5.0% in MSE and 3.4% in MAE. This demonstrates the benefit of exposing the model to a wider range of temporal reasoning tasks during training, improving its ability to generalize across different forecasting scenarios.
Our generalized formulation can be easily applied to other patch-based architectures by replacing their prediction lay- ers with our query-based decoder. PatchTST [4] is a clear example: replacing its linear decoder by our query-based decoder and adopting it with our generalized formulation yields a TIMEPERCEIVER encoder design. As shown in Table 3, we observe that train- ing with the generalized formulation consistently outper- forms the standard formulation. This result indicates that our generalized formulation is broadly applicable across different model designs.
Table 3:
Performance comparison of the generalized formulation with PatchTST [4] on the ETThl dataset.
Model
TimePercelver
PStchTST
variant that uses PatchTST's
Metric
MSE
MAE
MSE
MAE
Standard
0,420 0.437
0.423 0,436 0.420 0,415 0.431
Generalized
0,404
To further analyze our generalized formulation, we implement three different patch sampling strate- gies: (a) selecting target patch indices Joatch contiguously (i.e., fully contiguous), (b) selecting Jpach randomly (i.e., fully disjoint), and (c) selecting Jpatc including two contiguous, equal-sized segments (i.e., mixed). The forecasting results are reported in Figure 3. All the strategies contribute to performance improvement, but the optimal configuration varies with dataset characteristics and prediction lengths. For example, while ETTh1 favors contiguous sampling, others benefit more from disjoint or mixed strategies, even among similar datasets such as ETTh1 and ETTh2. This highlights the importance of selecting an appropriate sampling ratio based on the specific forecasting context. Without hyperparameter tuning. selecting two equal-sized segments consistently yields reliable performance and serves robust default.
Encoder Attention. We compare different encoder attention mechanisms understand how temporal and channel dependencies should be modeled. The Self-Attention approach directly applies attention across input tokens, either jointly across temporal and channel dimensions or by splitting them. While simple, this method incurs a large memory and computation cost, especially as the number of input channels or time steps grows. Moreover, attending over all input tokens jointly can lead to diluted attention scores and unstable optimization. To address these issues, we adopt the Latent Bottleneck approach, where a fixed set of latent tokens attends the input. This design enables the model to compress relevant information from the input space into a smaller latent space, reducing the attention complexity from (N2)) to O(NM), where M & N is the number of latent tokens, as further analyzed in Appendix D. Unlike axis- attention, this latent-based method captures temporal and inter-channel dependencies jointly, leading to more expressive and efficient representations. Our results show that Latent Bottleneck outperforms both self-attention variants across all datasets.
8

---

(a) Encoder and decoder attention ETThl
(b) Encoder and decoder attention on ETTml
Figure 4: Cross-attention maps from the encoder and decoder on ETThl and ETTml. For each dataset, the left image shows attention from latent tokens to input patches (encoder), and the right shows attention from query tokens to input patches (decoder). In encoder maps, the x-axis shows flattened input patch indices (C' [Zpeca) ), and the y-axis represents latent tokens (M). In decoder maps, the x-axis is temporal input positions ([Zanchl L=384), and the y-axis is query tokens aligned with prediction patches (ITpnch) H 9 Each token spans 12 hours (ETTh1) or 6 hours (ETTml).
Decoder Design. In query-based decoding, it is essential for the model to distinguish between time points used as input and those expected as output. To enable this, we apply both temporal and channel positional embeddings, TPE and CPE, not only into the encoder but also into the decoder queries. We compare two strategies: one where the encoder and decoder share the same positional embeddings, and another where they use separate embeddings. As shown in Table 2, using shared positional embeddings generally leads to better performance across most datasets, supporting the importance of temporal alignment between encoder inputs and decoder queries. An exception is observed in the Solar dataset, where non-shared emheddings slightly outperform the shared version in terms of MAE.
4.3 Information Flow Analysis
Use of Latents. We conduct experiments with a variant, de- noted DirectLT, in which the decoder directly consumes latent tokens without re-expansion to full resolution. As demonstrated in Table 4, our decoder design yields substantially better per- formance compared to directly employing the encoded latent tokens as decoder inputs. The superiority of our design can be attributed to the following reasoning. In our framework, the latent bottleneck is designed to capture global temporal-channel patterns, while the original input sequence retains fine fine-grained local signals. Relying solely on the latent makes difficutt to reconstruct these detailed signals, ultimately leading perfor- mance degradation. In contrast, our framework enriches input patch representations by leveraging the latent as an auxiliary memory, enabling each to incorporate both global contexts and local details.
Table 4: Performance comparison where the decoder directly uses la- tents (referred to as DirectLT) on ETTh1.
Model
Ours
DirectLT
Metric
MSE
MAE
MSE
MAE
0.366
0.393
0.402
0.418
192
0.394
0.411
0.426
0.437
336
0.413
0.422
0.429
0.438
720
0.445
0.453
0.473
0.472
Avg
0.404
0.420
0.433 0.441
Cross Attention. To better understand how our model processes information throughout encoding and decoding, we visualize the cross-attention maps in Figure 4, using examples from the ETTh1 and ETTml datasets. For each dataset, the left side of the figure shows encoder-side attention from latent tokens to input patches, while the right side depicts decoder-side attention from query tokens to input patches. The encoder attention maps reveal that latent tokens distribute their focus across diverse regions of the input, spanning both temporal and channel dimensions. Rather than converging single dominant region, each latent appears to capture distinct patterns or features, indicating that the model effectively compresses input information while maintaining representational diversity.
On the decoder side, the attention maps illustrate how query tokens attend to input patches in temporally structured manner. Queries often align with regularly spaced regions in the input, suggesting that the model learns to exploit periodic patterns the data. In ETThl, where each input patch covers 12 hours, decoder queries commonly attend to patches spaced about two positions apart, which roughly corresponds to daily cycle (24 hours). ETTml, where each patch covers fixed time span of 6 hours, the attention aligns with intervals of four positions, reflecting quarter-day

---

rhythm. These resolution-a aware patterns demonstrate the model's 's ability to adapt its decoding strategy the temporal resolution of the dataset and capture underlying periodic structure effectively.
Related Work
Recent time-series forecasting research has mainly focused encoder design, particularly modeling channel interactions, categorized as iineee-indeeendentt (CI) [4, 7, 9, 13, 14] or channel- dependent (CD) approaches [5, 6, 8, 11, 17]. While CD methods leverage richer inter-channel information, effectively utilizing it remains challenging, leading to recent attempts to capture both temporal and cross-channel dependencies separately [5, 6]. Meanwhile, most models still rely simple linear projections [4, 6, 7] from encoder outputs for prediction, using a uniform training formulation [6-11, 13, 14, 17]. Although query-based decoders [9] have recently developed for time-series forecasting, they do not study how to effectively encode the input time series, In contrast, our unified framework leverages generalized formulation with an optimized structure.
Conclusion
In this work, propose TIMEPERCEIVER, unified framework that integrates architectural design and training strategy for multivariate time-series forecasting. Specifically, we introduce a generalized forecasting formulation that encompasses extrapolation, interpolation, and imputation. To support this formulation, we design a novel encoder-decoder architecture equipped with latent bottleneck representations and query-based decoding, enabling flexible modeling of arbitrarily positioned input and target segments. Extensive experiments across diverse benchmarks demonstrate that our framework consistently outperforms prior state-of-the-an methods. We hope work encourages the community to explore holistic modeling approaches that align architectural choices more closely with the underlying forecasting objectives.
Limitations and Broader Impacts. Please refer to Appendix for further discussion.
Acknowledgment
This work was partly supported by the grants from Institute of Information & Communications Technology Planning & Evaluation (ITTP), funded by the Korean government (MSIT; Ministry of Science and ICT): No. RS-2019-1190421, No. IIP-2022-RS-2200-100011] No. 2ITP-2025-RS- 2024-00437633, and No. RS-2025-25442555
References
[1] Wetterstation. Weather. http:://www..ggg..nnnnn..ddeeett
[2] UCI. Electricity, http:///tthhiisuucuuaddaaaaaaeat
12014. 120044
[3] PeMS. Traffic. http://pemss.ott.a.govv
[4] Yugi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations (ICLR), 2023.
[5] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference Learning Representa- tions (ICLR), 2023.
[6] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransforme:: Inverted transformers effective for time series forecasting. In International Conference on Learning Representations (ICLR), 2024.
[7] Luo Dongha and Wang Xue. DeformableTSST: Transformer for time series forecasting without over-reliance on patching. In Advances Neural Information Processing Systems (NeurIPS), 2024.
10

---

[8] Wang Xue, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. Card: Channel aligned robust blend transformer for time series forecasting. In International Conference on Learning Representations (ICIR), 2024.
[9] Dongbin Kim, Jinseong Park, Jaewook Lee, and Hoki Kim. Are self-attentions effective for time series forecasting? In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[10] Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting. In International Conference on Learning Representations (ICIR), 2024.
[11] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. International Conference on Learning Representations (ICLR), 2023.
[12] Zeying Gong, Yujin Tang, and Junwei Liang. Patchmixer: patch- architecture for long-term time series forecasting, 2024.
[13] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In The Association for the Advancement of Artificial Intelligence (AAAT), 2022.
[14] Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan K Mathur, Rajat Sen, and Rose Yu. Long- term forecasting with tiDE: Time-series dense encoder. Transactions on Machine Learning Research, 2023.
[15] Si-An Chen, Chun-Liang Li, Sercan O Arik, Nathanael Christian Yoder, and Tomas Pfister. TSMixer: An all-MLP architecture for time series forecast-ing. Transactions on Machine Learning Research, 2023.
[16] Seunghan Lee, Taeyoung Park, and Kibok Lee. Learning to embed time series patches indepen- dently. In International Conference on Learning Representations (ICLR), 2024.
[17] Zihan Wang. Fanheng Kong, Shi Feng, Ming Wang, Xiaocui Yang. Han Zhao, Daling Wang, and Yifei Zhang. Is mamba effective for time series forecasting?, 2024.
[18] Seunghan Lee, Juri Hong, Kibok Lee, and Taeyoung Park. Sequential order-robust mamba for time series forecasting, 2024.
[19] Lu Han, Han-Jia Ye, and De-Chuan Zhan. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting, arXiv preprint arXiv:2304..m 052066, 2023.
[20] Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. 2019.
[21] National Renewable Energy Laboratory. Solar power data for integration studics.
[22] Jianxin Li, Xiong Hui, and Wancai Zhang. Informer: Beyond efficient transformer for long se- quence time-series forecasting. In The Association for the Advancement of Artificial Intelligence (AAAI), 2021.
[23] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2017.
[24] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
[25] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016.

---

[26] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR). 2019.
[27] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2022.
[28] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurlPS), 2019.
[29] Andrew Jaegle, Sebastian Borgeaud, Jean- -aattist Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, Olivier Hénaff, Matthew M. Botvinick, Andrew Zisserman, Oriol Vinyals, and João Carreira. Perceiver io: A general architecture for structured inputs & outputs. In International Conference on Learning Representations (ICI.R), 2022.
[30] Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. In International Conference Learning Representations (ICLR), 2015.
[31] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention, 2020.
[32] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention- permutation-invarte neural networks. In International Conference on Machine Learning. 2019.
[33] Mikhail S. Burtsev, Yuri Kuratov, Anton Peganov, and Grigory Sapunov. Memory transformer, 2021.
[34] Luo donghao and wang xue. ModemTCN: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024.
[35] Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and JUN ZHOU Timemixer: Decomposable multiscale mixing for time series forecasting, In International Conference on Learning Representations (ICLR), 2024.
[36] Sihan Xu, Ziqiao Ma, Yidong Huang, Honglak Lee, and Joyce Chai. Cyclenet: Rethinking cycle consistent in text-guided diffusion for image manipulation. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[37] Zhenwei Zhang, Linghang Meng, and Yuantao Gu. Sageformer: Series-aware framework for long-term multivariate time series forecasting. IEEE Internet of Things Journal, 2024.
[38] Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. CrossGNN: Confronting noisy multivariate time series via cross interaction refinement. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[39] Wanlin Cai, Yuxuan Liang, Xianggen Liu, Jianshuai Feng, and Yuankai Wu. Msgnet: Learning multi-scale inter-series correlations for multivariate time series forecasting. In The Association for the Advancement of Artificial Intelligence (AAAI), 2024.
[40] Shengsheng Lin, Weiwei Lin, Wentai Wu, Feiyu Zhao, Ruichao Mo, and Haotong Zhang. Segrnn: Segment recurrent neural network for long-term time series forecasting, arXiv preprint arXiv:2308. 12000, 2023.
[41] Maurice Kraus, Felix Divo, Devendra Singh Dhami, and Kristian Kersting, xlstm-mixer: Multivariate time series forecasting by mixing via scalar memories. arXiv preprint, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to the abstract and Section 1.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Appendix J.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer Section 3, Section 4, and Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification: The code is accessible via Appendix B, where the full training setup is also detailed. All datasets used in our experiments are properly cited and can be accessed through the corresponding references.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.
Seapii/guiides/ooeeubbeee swt.iiissssooyoooooooooooooooill for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips.co.
wwwpc//gg//////eeeeeeeecce Cotpe//Coeeummiii wwwssssssiiiooooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer Section 4 and Appendix B. Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: All reported results are averaged over five trials. We found that the error bars are negligible, and due to space constraints, the error bars are omitted from our main table. Full experimental results will be made publicly available with our codebase.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : /wwrriss.ppp..e/ oc httppppllltccistiiuiiit
Answer: [Yes]
Justification: Please refer Appendix J. Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to Appendix J.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
We only use publicly available time-series forecasting benchmarks and open- source codebases. All datasets and code are properly credited and cited in accordance with their licenses and terms of use.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi twwoee com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
17

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Please refer to Section 3, Section and Appendix B. The code is accessible via Appendix B, where the full training setup is also detailed.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
18

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: LLMs did not contribute the method development in this research in any essential or novel capacity.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to LLM policy (https: /ttp///een.ppeeeeeeeeeeee/e www./occeeennse///////ccoocce /LLM) for what should or should not be described.
19

---

Appendix
A Dataset
Table 5: Details of benchmark datasets used in our experiments.
Dataset
Variables
Frequency
Time Steps
Domain
ETThl
7
hour
17,420
Temperature
ETTh2
7
hour
17,420
Temperature
ETTml
7
15 minutes
69,680
Temperature
ETTm2
7
15 minutes 69,680
Temperature
Weather
21
10 minutes 52,696
Weather
Solar
137
10 minutes 52,560
Energy
Electricity
321
1 hour 26,304
Electricity
Traffic
862
hour 17,544
Transportation
We utilize 8 widely adopted benchmark datasets for time series forecasting: ETT (ETTh1, ETTh2, ETTm1, ETTm2) [22], Weather [1], Solar [21]. Electricity [2], and Traffic [3]. These datasets are commonly used recent time series forecasting literature and span a wide range temporal resolutions, domains, and sequence characteristics. Descriptions of each dataset are provided as follows, while detailed statistics such as data size and sampling frequency are summarized in Table 5,
ETT [22] contains variables from two electricity transformers between 2016 and 2018, with four subsets: ETThI/ETTh2 (hourly) and ETTm LTTTm/EETTm2 (every 15 minutes).
Weather [1] contains 21 meteorological variables collected from a weather station in Germany
during 2020.
Solar [21] contains 137 variables representing solar power generation from photovoltaic plants in
Alabama during 2006.
Electricity [2] contains hourly electricity consumption variables from 321 clients, collected
between 2012 and 2014.
Traffic [3] contains road occupancy rate variables measured hourly by 862 sensors on San Francisco Bay Area freeways, covering the period from 2015 to 2016.
Following the standard protocol [7, 22] widely adopted in time series forecasting research, we perform 6:2:2 ratio for training, validation,
chronological split of each dataset. For the ETT datasets, and testing. For the remaining Shtasets-Weather Solar, Electricity, and Traffic-we apply a 7:1:2 split.
20

---

B Implementation Details
Algorithm 1 TEMEPERCEIVER
Require: Concatenated input token sequence XIn IR. Cx (ppnnl) input token indices Ipanch C
Iprcch patch length P. embedding dimension D, number of latent tokens M, latent dimension Dr
(1, N. target token indices Jpatch {1, , N,
Apply instance normalization (RevIN) X1,ms. and store (Ju σ)
2: H Iyy, mmput= + Eehannel + Etemporal
b H E (C((
// Encode input into latent tokens (information compression)
3: Use learnable latent array (
D Z E RMXDI.
4: Z(1) AttnBlock([[ (Z)) H H()))
D Z MxDx
5: for = K do
6:
Z(*+1) = AttnBlock(Z (k)) ((a), ())
7: end for
// Update input tokens using refined latent representation
8: H = AttnBlock( ()), (1+ Z
(k)
> H E (C
Predict target tokens by attending to updated input representation
9: Use learnable query Q())
(0) E (C
10: Q(1) AttnBlock (()) ( ((,,H()) H())
Q(1) R (xD
// Project query to patch-level predictions
11: Jpun Q W ougput
b X Jaac E Cx
Apply instance denormalization (RevIN) restore original scale
Hyperparameter Settings. For the TIMEPERCEIVER parameter 0, we use the AdamW optimizer [26] with a weight decay 0.05. The learning rate is set to x 10-4 for the Weather dataset and x 10-4 for all other datasets, with warmup phase of 5 epochs. We train for 50 epochs on the ETT and Solar datasets, and 100 epochs on all others. The batch size is set to 32 for the Solar, Electricity, and Traffic datasets, and 128 for the remaining datasets. We choose the patch size P E { {12, 16, 24, 48) and the embedding dimension and the latent dimension to Dr. (64, 128, 256). For simplicity, as in section 3.3, although in practice we use different values for different components. For the attention modules, in each attention module is set to twice the query dimension. Lastly, the separate ratio is selected from {0, 0.5, 1}, corresponding to the fully disjoint, mixed, and fully contiguous sampling settings, respectively, as illustrated in Figure 3. All results are reported as the average over 5 runs with different random seeds.
E {256, 512). The number of latents is set to M E {8, 16, 32}.
denote all latent dimensions
choose the number of heads Theads E (4, 8) and feedforward dimension da
Instance Normalization. To address distribution shifts between training and test data, we apply Reversible Instance Normalization (RevIN) [27]. Each input is normalized before processing, and the original statistics are restored after prediction.
Reproducibility Statement. We provide our code with training and evaluation scripts to ensure reproducibility httpgggiihhhco/fffffeeieeeeeeeee conducted using NVIDIA RTX 4090 GPUs with the PoTorc [28].
wargg-lla/iiiiin Par/eimeerreiieer. All the experiments were
21

---

C Error Bar
Table 6: Standard deviation performance results with input length L H {96, 192, 336, 720) on five random seeds.
384 and prediction lengths
Dataset
Weather
Solar
ECL
Traffic
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.144:00.00 0.195==0.003
0.163++0.006
0.213+0.005
0.125=0.001
0.219=0.003 0.373=00.001 0.262=0.002
192
0.190+0.0003 0.243++0000
0.176+0.003
0.228+0.002
0.142++0.002
0.235+0.001 0.383+0.002
0.263=0.003
336
0.242:+0.000 0.209=+0001
0.185-+0.003
0.246++0.004
0.161+00.03
0.254=0.004
0.399++0002
0.270=0.003
0.315=+0.033
0.336=00.001
0.19740.005
0.244+00003
0.200=0.002
0.288=0.003
0.433+0.005
0.289=0.002
Dataset
ETTh1
ETTh2
ETTml
ETTm2
Metric
MSE
MAE
MSE
MAE
MSR
MAE
MSE
MAE
0.366.10.004
0.393:10.003 0.272:10.004
0.333:10.005
0.283:10.007 0.334:100004
0.161=00.05
0.257=00004
0.394:10.002 0.411:0.000
0.333:10.003
0.381=10.004
0.321=0.004 0.359=0.002 0.215:0.003
0.285=0.004
336
0.413+0.006
0.422++0.006
+..069
0.399+00001
0.352+0.004
0.378+0.002
0.273+0.003
0.325-0.004
720
0.445:10.008
0.453-10.004
0.386.10.010
0.431110.009
0.399 0.410-10.02
0.357=10.006 0.375=00005
We evaluate the stability of TIMEPERCEIVER by reporting mean and standard deviation over five runs with lookback window L = 384. As shown in Table 6, across all 64 experimental settings, 54 cases exhibit a standard deviation less than or equal to 0.0005, indicating that TEIVERR remains highly stable.
22

---

D Computation Efficiency
tolattedor Dacagled
Alterton Cecoulied
Phorsymar Plac
04
130
1iput Langt
riput Langet
Ferscasing 1103
(a) GPU memory usage
(b) Training time comparison
(c) Parameter comparison
Figure (a) and (b) compare computational costs resulting from different encoder structures, including GPU memory usage and epoch time per input length. (c) shows the difference in parameter count between our model and other commonly used models with decoder structures based linear projection, across various forecasting horizons.
Table 7: Computation cost comparison with other baselines.
Datasct
TimePerceiver DeformableTST
CARD
PatchTST
Crossformer
TimesNet
Input length
96
384
96
384
96
384
96
384
96
384
96
384
Memory(MB)
972
2998
538
724
1512
6852
2768
4110
1300
3258
Runtime(iter/s)
0.0435
0.0583
0.0919
0.1153
0.0737
0.0828
0.0547
0.1156
0.1299
0.1350
0.1376
0.1385
We empirically validate the efficiency of our proposed encoder design through comparisons across three variants: our model with a latent bottleneck, full self-attention, and decoupled self-attention. As shown in Figure and Figure 5b, our method consistently achieves lower GPU memory usage and faster training time, particularly as the input length increases. This is attributed to the use of a small set of latent tokens, which enables the model to abstract and interact with the input more efficiently than direct token-to-token computation. In contrast, full self-attention suffers from quadratic complexity with respect to input length. Decoupled self-attention, which applies attention separately along the temporal and channel dimensions, avoids this quadratic scaling but still incurs higher overhead due to its dual attention structure.
Furthermore, Figure 5c highlights the parameter efficiency of query-based decoder [9]. Since each query is projected through shared linear layer, our model maintains nearly constant number of parameters regardless of the forecasting horizon. In contrast, models employing direct linear projection from encoded representations [4, 6]. exhibit linear growth in parameter count as the prediction length increases. These results show that our design remains efficient when handling both long input sequences and extended forecasting horizons.
Beyond these aspects, Table 7 shows that TIMEPERCEIVER demonstrates efficiency in both training time and memory footprint compared to baselines [4, 5, 7, 8, 11], regardless of the input length. This efficiency stems from the latent bottleneck that enables compression of core information efficiently.
23

---

E Full Results of Multivariate Long-Term Forecasting
To complement the main results reported in Table we report the full forecasting performance for each input length L E {96, 384, 768} in this appendix. These detailed results provide a more comprehensive view of model behavior across varying lookback window size.
Table 8: Multivariate long-term forecasting results compared with baseline models. Bold indicates the best result, and underlined denotes the second best. The results are reported using input length
L = 96 and prediction lengths H E {96, 192, 336, 720].
Models
TimcPereeiver
DeformaieSST
CARD
CATS
PatcNTST
(Transfomerr
S-Mamha
DLinear
Timesher
TIDE
Metric
MSE
MAP
MSF
MAE
MSE
MAR|
MSE
MAR]
MSE
MAE
MSE
MAR
MSE
MAP
MSE
MAF]
MSE
MAE |
MSP
MAE
Weather
192
96
00115
0.202
0.202
0.216
0.169
0.255
0.213
0.160
0.208
0.161
0.208
D. 0.207
0.250
0.137
0.235
0.218
0.259
0.174
0.214
0.165
0.210
0.196
0.255
0.112
0.261
0.220
0.242
00.202
0.261
0.29%
336
0.255
0.258
0.271
0.294
0.265
0.2M2
0.264
0.200
0.278
0.297
0.278
0.296
0.274
0.297
0.283
0.335
0.280
0.306
(0.287
D.335
720
00.388
0.340
0.347
0.344
0.346
0.346
0.342
0.34
0.354
0.348
0.358
0.347
0.350
0.345
0.345
0.381
0.365
0.359
0.351
0.386
|Arg
[0,238
0.269
0.251
0.276
0.2215
0.274
00.211
0.272
0.259
0.281
0.255
0.278
0.251
0.276
0,265
0.317
(.229
[02259
0.287
00.271 0.320
0.193
0.281
0.216
0.247
0.234
0.286
0.237
[0.250
0.292
0.3312
0.399
Solar
192
0.228
0.259
0.225
0.261
0.267
0.308
0.247
0.267
0.267
0.310
0.233
0.261
0.237
0:270
0.320
0.398
0.296
0.318
0.339
0.416
336
0.247
0.273
0.239
0.267
0.289
0.319
0.267
0.277
0.290
0.315
0.248
0.273
0.258
0.288
0.353
0.415
0.319
0.330
0.368
0.430
720
0.257
0.281
0.238
0.265
0.291
0.327
0.296
0.293
0.289
0.317
0.219
0.275
0,260
0.288
0,356
0.413
0,338
0.337
0.570
0.125
Avg
0.233
0.224
0.256
00.270
0.309
0.225
0.271
00.270
0.307
0.262
0.240
0,101
|0.30]
0.319
0.341
0.117
Electneity
96
10.33
0.225
0.158
0.255
10,155
0.246
0.144
0.2377
0.181
0.270
0.148
0.240
0.139
0.235
0. 1.19
0.282
10.668
0.272
00,237
0.329
192
0.150
0.244
0.168
0.265
0.170
0.259
0.163
0.250
0.188
0.274
0.162
0.253
0.159
0.255
0. 196
0.285
0.184
0.289
0.236
0.330
336
0.168
0.183
0.288
0,285
0.180
0.76X
0.204
0.19 0.300
(1.249
0.344
720
0.200
0.298
0.223
0.313
0.229
0.316
0.219
0.302
0.246
0.225
0.317
0.298
0,333
0.220
0.320
(1.284
0.373
Avg 00.165
0.258
0.183
0.278
[0.187
0.277
[0.178
0.264
00.205
0.290
00.178
0.270
0.170
0.265
0.212
0.300
0.900/0.199
[0.192 0.295
|0.251
0.344
96
00,100
0 2623
0.118
0.271
0.448
0.300
0.121
0.270
0.162
0.295
0.395
0,268
0.382
0.261
0.650
0.396
[0.593
0.321
00,805
0.193
Traffic
192
0:416
0.437
0.276
0,436
0.375
0.466
0.296
0.276
0.336
0.756
0.474
326
0.430
0.276
0.449
0.290
0.477
0.306
0.453
0.284
0.482
0.504
0.433
0.283
0.417
0.276
0.605
0.373
0.629
0.336
0.762
0.477
720
0.167
0.297
0.177
0.303
0.509
0.323
0.481
0.303
0.514
0.322
0.167
0.302
0.460
0.500
0.615
0.391
0.640
0.350
0.719
0.119
Avg
00:128
0.275
0.115
0.265
00415
0.308
00,150
0.283
0.181
0.304
0.428
0,282
0,414
0.276
0.625
0.583
|06620
0.336
[0.760
0.173
96
0.272
0.394
0:173
0.396
10,398
0:405
0.331
00:414
0.419
0.386
0.405
08386
0,405
0,400
10.384
0.402
(0:479
0:464
192
0:422
0,425
0:427
0.427
0,448
0.433
0.426
0.422
0.460
0.445
0.441
0.436
0.443
0.437
0.437
0.432
0.436
0.429
0.525
0.492
336
0.444
0.434
0:437
0.426
0.494
0.457
0.437
0.432
0.501
0.466
0.487
0.458
0.489
0.468
0.481
0.459
0.491
0.469
0.565
0.515
730
0.451
0.464
0,462
0.483
0.472
0.474
0.500
0.488
0.503
0.491
0.502
0.489
0.516
0.552
0.500
0.594
0.558
Avg
00:42
0.427
0.425
0.428
[0.456
0:442
0:447
0.428
[0:469
0.454
00.454
0.447
0.455
0.450
0.456
0,452
0,452|0.455
[0.458
0.450
(0.541
0.507
96
D.288
01338
0.281
0.334
10,294
0.341
00.287
0. 0.341
00.302
0.34%
0.299
0.349
0.206
01348
0.333
0.387
|0.340
0.374
(0400
0.440
ETTR
192
0.365
0.388
0.353
0,382
0,375
0.391
0
0.388
0.388
0.400
0,380
0,400
0.376
0.396
0.477
0.476
0.402
0.414
0.528
0.509
336
0.341
0.359
0.426
0.374
0.405
0.426
0.435
01.428
0.422
0.452
(1644
0.571
720
0.404
0.440
0.410
0.431
0.420
0.438
0.412
0.43%
0.441
0.446
0..277
0.445
0.426
0.444
0.657
0.462
0.468
(1874
0.679
Avg
0.355
0.395
0.346
0.382
00.378
0.399|
0.359
0.391
00.387
0.407
0.383
0.407
0.381
0.405
0.559
0.515|0.414
0.515
10.114
0.427
00661
0,550
96
0,300
0.347
0.316
0.358
0.327
0.359
0.318
0.347
0.329
0.367
0.334
0.368
0.333
0.368
0.345
0.372
00.33
0.375
0.336
0.387
192
0.343
0.354
0.380
0.381
0.357
0.377
0.367
0.377
0.376
0.387
0.398
00.04
336
0.371
0
0.379
0.405
0,603
0.401
0.387
0.410
0.399
0,410
0.426
0.420
0.408
0.413
0.413
0.413
0,410
0.411
0.428
0.425
720
0.428
0.426
0.113
0.438
0.467
0.438
0.418
0.437
0.154
0:139
0.191
0.459
0:475
0.448
0.174
0.153
0.478
0.450
0.187
0.161
Avg
00.362
0.383
0.373
0.395
00.392
0.395
00.378
0.387
0.400
0.407
0,410
0.398
0.405
0.407
|0.400
0.406
01,41
0.419
96
0.170
0..17
0.262
10.176
0.259
0.178
0.2661
0:175
0.259
0.
0.264
0.179
0.263
0.292
10.187
0.267
[0.207
0.305
ETTm2
192
0.236
0.293
0.243
0.301
0.246
0.306
0.248
0.308
0.241
0.302
0.250
0.309
0.250
0.309
0.284
0.362
0.249
0.309
0.290
0.364
336
00.298
0.330
0,310
0.348
0.3012
0.343
0.304
0.34%
0.305
0 0:345
0.311
0.348
0.312
0.349
0. 369
0.427
0:321
0.351
0.377
0.432
720
0.399
0.400
0.398
0.400
0.404
0..402
0.402
0.402
0.400
0.412
0.407
0.411
0.406
0.554
0.408
0.403
01558
0.524
Avg 00.276
0.316
0,2883
0.327
0.2281
0.328
0.2283
0.329
0.2881
0.326
0..288
0.332
0.258
0.332
0.350
0.401
|0.291
0.333
[0,358
0.404
24

---

Table 9: Multivariate long-term forecasting results compared with baseline models, Bold indicates the best result, and underlined denotes the second best. The results are reported using input length
L =
384 and prediction lengths H E {96, 192, 336, 7203.
Models
TimePerceiver
DefirmatieST
CARD
CATS
PatGTTST
TTrandommer
S-Mamiba
DLincar
TimesNet
TDE
Metric
MSB
MAE
MSE
MAE
MSE
MAB|
MSE
MAE]
MSE
MAE
MSE
MAB
MSE
MAB
MSE
MAE
MSB
MAE
MSB
MAE
Wealher
192
96
00.44
0.190
0.195
0.149
0.192
198
0.155
0.209
0.191
9.147
0.199
0.212
0.194
0.150
0.201
0.160
0.210
0.157
0.208
0.172
0.272
0.232
0.161
0.278
0.216
0.176
0.219
0.228
0.263
336
0.242
0.279
0.244
0.278
0.263
0.294
0.244
0,280
0.243
0.279
0.252
0.287
0.256
0.290
0.264
0.317
0.286
0.316
0.266
0.298
720
0.315
0.336
0.317
0.331
0.355
0.352
0.314
0.331
0.319
03335
0.325
0.339
0.322
0.336
0.324
0.364
0.373
0.368
0.333
0.345
Avg
0.223
0.263
0.225
0.261
0.2245
0.277
0.224
0.263
0.226
0.264
0.235
0.272
0.235
0.222
0.244
0.296
|0.264
0.295
0.2249
0.284
96
0..16
0.213
0.173
0.241
0.115
0:243
0.172
00.316
00.197
0.291
0.187
0.246
0.194
0.243
0.316
0.316
|02217
0.311
00.228
0,300
Solar
192
0.377
0.228
0.187
0.248
0.209
0.262
0.189
0.222
0:214
0.319
0.219
0.269
0.221
0.271
0.244
0.334
0.275
0.338
(.259
0.345
336
0,185
01245
0.193
0.260
0.209
0.272
0.197
0.23%
0.235
0.328
0.222
0.281
0.217
0.273
0.259
0.346
0.266
0.368
0.273
0.353
720
0.397
0.204
0.272
0.281
0.203
0.345
0.225
0.329
0.251
0.336
01,294
0,346
Avg
0.182
0.233
08.899
0.255
00.205
0.265
00.19
0.232
[0,218
0.317
00.212
0.272
0.216
0.268
0.251
0.337
|0,252
0.338
(0,264
0,336
Electricity
96
10.125
0.219
0.132
0.231
0..13
0.233
0.128
0.224
0.133
0,232
0.144
0.240
0.134
0.230
0.151
0.250
00.192
0.300
0.161
0.266
192
0.142
0.235
0.150
0.250
0.157
0.256
0.147
0.237
0.148
0.246
0.158
0.258
0.154
0.250
0.165
0.263
0.202
0,306
0,169
0.273
336
0.161
0.167
0.267
0.162
0.255
0.169
0.265
0.275
0.350
0.118
0.284
720
0.200
0.288
0.203
0.299
0.200
0.292
0.198
0,286
0.202
0.295
0.215
0.310
0.197
0.292
0.215
0.313
0.257
0.349
0.210
0.308
Avg
00.157
0.249
0.163
0.262
16.165
0.262
0.159
0.253
0.166
0.260
0.112
0.271
0,165
0.261
0.178
0.277
|0.228
0.326|0.180
0.326
[0.880
0,283
96
0.3373
0.262
0.362
0.262
0,380
0.270
0.358
0.248
0.385
0:277
0.399
0.298
0.346
0.250
0.428
0.304
10.556
0.305
0,,49
0.364
Tralikc
122
D.283
0.263
0.385
0.268
0,398
0:277
0.306
0.366
(0.40
0.285
0.414
0.112
0.437
0,308
0.557
0.304
01493
0.330
326
0.399
0.270
0.397
0.275
0.409
0.289
0.293
0.263
0.410
0.287
0.432
0.322
0.363
0.265
0.448
0.314
0.500
0.312
0.512
0.383
720
0.433
0.289
0.434
0.298
0.437
0.301
0.435
0.295
0.443
0.304
0.472
0.347
0.423
0.289
0
0.333
0.617
0.323
0.539
0.404
Avg
0..397
0.395
0.276
[0.406
0,284
0.343
0,410
0.288
0.315
|0.580
0.311
|0.508
0,380
96
00336
0.393
0.369
0.396
0.377
0.402
00.881
0.405
00.776
0.401
0.413
0,427
0,398
0.419
0,373
0,399
10.414
0.428
[0451]
0.455
192
0.394
0.411
0.410
0.417
0.604
0.418
0.416
0.433
0.410
0.421
0.449
0.450
0.436
0.444
0.407
0.421
0.469
0.467
0.482
0.473
336
0.413
0.422
0.391
0.414
0.428
0.435
0.435
0.410
0.134
0.139
0.155
0.158
0.118
0.455
0.524
0.415
0.502
0.180
0.501
0.186
720
0:445
0.447
0.464
0.459
0.451
0.460
0.528
0.501
0.645
0.542
0.510
0.509
Avg
0.400
0.420
0.404
0.433
00:42
0,429
00:411
0:435
01:18
0.435
[0:46]
0.464
0.445
0.457
0.424
0,442
|0.508
0.479
00446
0.481
96
0.2272
0.333
0.272
0.334
0.2255
0.336
0.283
0.346
0.275
0.337
0.306
0.326
0.300
0.357
0.305
0.368
10.347
0.395
0.370
0.431
ETT2
192
0.333
0.381
0.325
0.369
0.339
0.376
0.345
0.384
0.338
0.375
0,366
0.402
0.367
0.399
0.399
0.428
0.403
0.430
0.441
0.474
336
D.346
0.319
0.373
0,360
0.397
0.372
0.406
0.352
0,400
0.427
0.392
0.476
0.441
0.457
0.469
0.499
720
0.386
0.431
0.395
0.433
0.390
0.425
0.408
0.443
0.398
0.435
0.435
0.454
0.416
0.445
0.764
0.621
0.471
0.483
0.504
0.530
Avg
0.334
0.386
0.328
0.377
0.3441
0.384
0.352
0.395
00.341
0.385
0..377
0.412
0.369
0.405
0.485
0.473
|0.416
0.441
|0.446
0.484
96
0,2283
0.292
0.344
10.245
0.284
0.339
0.292
10.338
0.377
00.311
0.353
192
0.321
0.335
0.371
0.339
0.371
0.323
0.340
0.373
0.353
0.
0.344
0.382
0.337
0,368
0.365
0.386
(1.344
0:372
336
0.352
0.378
0.365
0.392
0.373
0.392
0.356
0.354
0.367
0.394
0.383
0.405
0,378
0.103
0.368
0.381
0.389
0.405
0.378
0.391
720
0.399
0.410
0.417
0,418
0.420
0.414
0.411
0.421
0.422
0.427
0.443
0,435
0,439
0.441
0.422
0.418
0.514
0.465
0.433
0.422
Avg 00.338
0.352
0.381
[0.357
0.381
0..44
0.377
00.35
0.385
10.372
0.399
0.379
(0:401
0.408
|0.367
0,385
96
[0.161
0.257
0.170
0.257
[0,165
0.256
0..12
0.260
0.166
0.255
0.179
0.271
0.181
0.269
0.165
0.260
00.217
0.290
00.89
0.292
ETTm2
192
0.215
0.285
0.227
0.295
0.229
0.298
0.234
0.306
0.228
0.299
0.243
0.313
0.233
0.308
0.258
0.319
0.251
0.317
0.250
0.333
336
0.273
0.325
0.275
0.327
0.271
0.328
0.278
0.335
0.280
0.331
0.295
0.315
0.281
0.338
0.309
0.370
0.312
0.358
0.309
0.372
720
0.357
0.365
0.384
0.367
0.383
0.363
0.39)
0.373
0.389
0.377
0.397
0.423
0.481
0.448
0.412
0.435
Avg 00.252
0.311
0.260
0.316
002250
0.316
0.262
0.323
0.262
0.319
0.273
0.332
0.266
0.327
0.279
0.343
00.35
0.353
|,299 0.358
25

---

Table 10: Multivariate long-term forecasting results compared with baseline models, Bold indicates the best result, and underlined denotes the second best. The results are reported using input length
L =
768 and prediction lengths H E {96, 192, 336, 7203.
Models
TimePerceiver
DefirmatieSTy
CARD
CATS
PatcTTST
iTrandoomer
S-Mamiba
DLincar
TimesNet
TDE
Metric
MSB
MAE
MSE
MAE
MSE
MAB|
MSE
MAE]
MSE
MAE
MSE
MAB
MSE
MAB
MSE
MAE
MSB
MAE
MSB
MAE
Wealher
192
96
0.144
0 1.200
0.146
0.191
198
0.152
0.208
0.143
0.188
0.240
0.197
0.148
0.194
0.202
0.165
0.217
0.165
0.210
0.19
0.167
0.267
0.226
0.177
0.228
0.296
0.171
0.214
0.225
0.261
336
0.239
0.283
0.241
0.280
0.262
0.298
0.239
0.282
0.243
0.282
0.259
0.296
0.267
0.301
0.256
0.306
0.311
0.334
0.258
0.295
720
0.316
0.337
0.310
0.331
0.362
0.356
0.309
0.333
0.310
0.328
0.321
0.342
0.330
0.346
0.315
0.353
0.416
0.404
0.321
0.340
Avg
0:221
0 1.264
0.222
0.262
0.2251
0.283
0.220
0.263
0.224
0 0.263
0.238
0.277
0,243
0.281 0.237
0.288
|0.290
0.316
10.241
0.280
96
00.558
0.211
0.165
0.238
0.117
0.246
0.163
0.232
0.191
0.275
0.174
0.242
0.174
0.238
0.190
0.273
102219
0.279
(0.197
0.2722
Solar
192
0,3800
0.228
0.184
0.254
0.210
0.26%
0.193
0.234
0:211
0.282
0.194
0.259
0.
198
0.264
0.211
0.291
0.228
01297
0.216
0.293
336
0.186
0.225
0.191
0.263
0.219
0.277
0.189
0.247
0.221
0.293
0.207
0.272
0.213
0.275
0.227
0.303
0.299
0.312
0.233
0.317
720
0.397
0.199
0.262
0.345
0.235
0.297
0.217
0.217
0.316
0.238
0.313
0.237
0,320
Avg
0.180
0.229
0.1855
0.254
00.210
0:271
0.116
0.237
(0.215
0.286
00.198
0.263
0.201
0.264
0.236
0.296|0.231
0.296
|0.231
0.300
00.221 03301
Electricity
96
10.128
0.225
0.132
0.234
10.133
0.230
0.126
0.222
0.130
0.235
0.142
0.241
0.134
0.232
0.145
0.243
10.208
0.31010.143
0,250
192
0.147
0.241
0.148
0.248
0.157
0.256
0.149
0.245
0.153
0.249
0.160
0.258
0.157
0.253
0.159
0.257
0.205
0,313
0.154
0.258
336
0.165
0.165
0.266
0.181
0.171
0.365
0.168
0.269
0.277
0.169
0.331
0.176
0.275
720
0.202
0.203
0.197
0.296
0.207
0.303
0.196
0.187
0.205
0.298
0.221
0.312
0.194
0.289
0.208
0.307
0.241
0.342
0.213
0.303
Avg
00.000
0.255
0.161
0.261
00.170
0.268
0.161
0.254
0.116
0.263
0.116
0.272
0,164
0.261
0.171
0.270
[0.219
0.324
0.32410.171
0.117
0.272
96
0.337
0 0.265
0.355
0.261
0,369
0.267
0.357
0.248
0.373
0.267
0.380
0.291
0.342
0.250
0.400
0.287
10.565
0.303
0,451
0.344
Tralikc
122
D.382
0.268
0
0,380
0:221
0,381
0:270
0.379
0.25%
0:384
0.269
0,400
0.306
0.362
01262
0.411
0.291
0.579
0.308
0.459
0.346
326
0.397
0.272
0.393
0.281
0,402
0.283
0.393
0.265
0.399
0.275
0.420
0.317
0.366
0.268
0.425
0.298
0.575
0.304
0.470
0.353
720
0.436
0.292
0.434
0.300
0.437
0.299
0.426
0.285
0.439
0.295
0.466
0.344
0.430
0.295
0.465
0.322
0.601
0.318
0.491
0.362
Avg
00.396
0.391
0.278
|0.397
0.264
0.399
0.277
0.300|0.559
|0.579
0.308
01.468
0.351
96
10,355
0.390
0.367
0,404
0.337
0:410
0.373
0.407
0.379
0,430
0.400
0,428
0,393
0.425
0,372
0,401
105500
0.483
[0,446
03:45
192
0.390
0.415
0.109
0.430
0.410
0.428
0.416
0.44]
0.416
0.435
0.433
0.452
0.432
0.451
0.111
0.425
0.537
0.505
0.483
0.478
336
0.419
0.436
0,406
0.437
0.450
0.413
0.456
0.161
0.110
0:451
0.175
0.181
0.474
0.479
0.438
0/150
0.691
0.561
0.508
0.196
0.458
0.503
0.489
0.469
0.508
0.680
0.556
0.518
0.519
Avg
0.405
0.430
0:410
0.438
00,422
0:442
0.437
0:450
0:426
0,444
0.469
0.478
0.458
0.471
0.427
0,446
|0.605
0.526
00448
0.488
96
10.278
0.338
0.269
0.334
0..279
0.341
0.277
0.339
0.275
0.339
0.307
0.363
0.298
0.356
0.325
0.386
[0.431
0.44660.366
0.427
ETT2
192
0.344
0.395
0.325
0.372
0.350
0.384
0.337
0.383
0.336
0.378
0.441
0.442
0.370
0.406
0.432
0.450
0.444
0.471
0.436
0.471
336
0.350
0.324
0.380
0,367
0.403
0.370
0.407
0.357
0.401
0.452
0.456
0.394
0.508
0.515
0.499
0.459
0.498
720
0.395
0.441
0.412
0.446
0.392
0.430
0.435
0.468
0.397
0.436
0.434
0.462
0.428
0.454
0.947
0.689
0.520
0.494
0.494
0.529
Avg
0.342
0.395
0.333
0.383
0.347
0.390
0.355
0.399
0.341
0.389
0.409
0.431
0.373
0.410
0.560
0.508
J0.477
0.483
0.483|0.4333
|0.439
0.480
96
00,283
0.347
0.229
0.285
0.295
|0.378
0.347
00.31
0.360
192
0.324
0.366
00.325
0.372
0.339
0.371
0.318
0.330
0.375
0.347
0. 0.7
0.350
0.390
0.337
0,368
0.414
0.415
01.34
0:374
336
0.348
0.379
0.359
0.390
0.368
0.388
0.353
0.354
0,361
0.395
0,388
0.112
0.377
0.106
0.364
0.381
0.443
0.429
0.373
0,391
720
0.410
0.409
0.4188
0.423
0.417
0.420
0.404
0.415
0.440
0.432
0.460
0,456
0,430
0.440
0.413
0.414
0.510
0.474
0.422
0.418
Avg
[0:342
0.348
0.383
[0.355
0.381
0..340
0.377
00,352
0.388
0.379
(0.46
0.429
|0.364
0.386
96
106
0.257
0.169
0.258
10.164
0.255
0.168
0.262
0.182
0.272
0.185
0.278
0.178
0.271
0.161
0.253
0.2244
0.314
00.16
0.291
ETTm2
192
0.224
0.293
0.229
0.299
0.226
0.298
0.230
0.307
0.247
0.314
0.239
0.317
0.235
0.511
0.222
0.302
0.287
0.342
0.245
0.333
336
0.275
0.327
0.280
0.333
0.272
0.328
0.283
0.339
0.303
0.354
0.298
0.355
0.290
0.347
0.296
0.361
0.326
0.370
0.304
0.373
720
0.357
0.349
0.384
0.390
0.399
0.367
0.39]
0.371
0.398
0.401
0.416
0.362
0.390
0.425
0.450
0.442
0.392
0.430
Avg
(0.256
0.315
0.257
0.319
00.263
0.320
0.262
0.325
0.276
0.335
0.281
0.341
0.266
0.330
0.270
0.335
|0.327
0.367
|0228 0.357
26

---

F Full Results of Component Ablation Studies
Table 11: Full performance comparison across different formulations, encoder attention mecha- nisms, and positional encoding (PE) sharing strategies. Bold indicates the best result among all configurations.
lower MSE or MAE indicates a better performance. The first row corresponds to our proposed model. The results are reported using input length I. = 384 and prediction lengths
H E {96, 192, 336, 720}.
ETTh1
ETTml
Solar
ECL
Formulation
Encoder Attention
PE Sharing
MSR
MAE
MSE
MAR
MSE
MAB
MSE
MAE
96
0.366
0.393
0.283
0.334
0.163 0.213
0.125
0.219
192
0.394
0.411
0.321
0.359
0.176
0.228
0.142
0.235
Generalized
Latent Bottleneck
Sharing
336
0.413
0.422
0.352
0.378
0.185
0.246
0.161
0.254
720
0.445
0.453
0.399
0.410
0.197
0.244
0.200
0.288
Avg
0.404
0.420
0.338
0.370
0.182
0.233
0.157
0.249
96
0.378
0,403
00.992
0.346
0.178
0.228
0.142
0.239
192
0.403
0.418
0.343
0.372
0.192
0.239
0,155
0.252
Standard
Latent Bottleneck
Sharing
336
0.426
0.432
0.373
0.389
0.198
0.256
0.173
0.270
720
0.474
0.495
0,410
0.419
0.209
0.248
0.207
0.299
Avg
0.420
0.437
0.355
0.382
0.194
0.243
0.169
0.265
96
0.381
0.403
0.289
0.337
0.174 0.215
0.128 0.224
192
0.406
0.422
0.341
0.365
0.188 0.227
0.150
0.242
Generalized
Full Self-Attenion
Sharing
336
0.438
0.436
0.364
0.383
0.203
0.255
0.162 0.256
720
0.473
0.475
0.417
0.416
0.204
0.251
0.204
0.292
Avg
0.425
0.434
00.533
0.375
0.192 0.237
0.161 0.254
96
00.885
0.406
0..188
0.360
0.173
0.220
0.126
0.221
192
0.411
0.422
0.331
0.358
0.184 0.231
0,146
0.238
Generalized
Decoupled Self-Attention
Sharing
336
0.432
0.434
0.366
0.383
0.195 0.232
0.160
0.252
720
0.462
0.468
0.407
0.413
0.205
0.247
0.201
0.290
Avg
0.423
0.433
00.366
0.379
189
0.233
0.158
0.250
96
0.378
0.398
0.288
0.341
0.174
0.215
0.128
0.220
192
0.409
0.418
0.327
0.359
0.178
0.222
0.146
0.241
Generalized
Full Self-Artention
Not Sharing
336
0.439
0.441
0.350
0.384
0.205
0.244
0.1
0.259
720
0.465
0.471
0.406
0.411
0.215
0.248
0.208
0.297
Avg
0.423 0.432
0.342
0.374 0.193 0.232 0.163 0.254
As shown in Table 11, this section presents the full results of the component-wise ablation study discussed in Section 4.2. While keeping the overall architecture and generalized formulation, we further discuss the impact of encoder attention and query design in more detail.
In the encoder, to simultaneously capture both temporal and channel dependencies, we needed a method that is not only effective but also computationally efficient. We achieved this through latent compression technique, which conceptually resembles the use of auxiliary memory in prior neural architectures that decouple computation from the input structure [29-33]. Specifically, we apply three layers of self-attention over a compact latent representation that functions as an auxiliary memory containing distilled yet essential information. This approach enables efficient and effective information extraction, and empirically demonstrates strong performance in terms of both efficiency and accuracy.
In the decoder, we previously discussed how sharing posi- tional embeddings facilitates temporal alignment between encoder inputs and decoder queries. In our main design, decouple the positional information by using sepa- rate channel and temporal positional embeddings (CPE and TPE). As shown in Table 12, one variation of this design combines the two into a single unified positional embedding, representing an alternative approach to query construction.
Table 12: Query design performance comparison.
Query Design Decoupled PR
Unified PF
Metric
MSE
MAE
MSE
MAE
96
0.366
0.393
00.399
0.399
192
0.394
0.411
0.405
0.418
ETThl
336
0.413
0.422
0.427
0.433
720
0.445
0.453
0.460
0.465
Avg
0,404
0.420
00.418
0.429
27

---

G
Ablation Study on Latent
Effectiveness of Latent. In our framework, the latent serves as an auxiliary memory [29-33] designed to capture diverse temporal- channel patterns. As shown in Table 13, the presence of the latent itself plays a key role in forecasting performance. Moreover, the performance is not highly sensitive, as long as the number of la- tents is reasonably large. This indicates that our latent bottleneck structure effectively aggregates temporal and channel dependen- cies, even with minimal capacity. We think that the latent may tend to learn redundant information. This could explain why per- formance does not vary significantly, especially when the latent dimension is large (e.g., D2 = 128). To examine the hypoth- esis, we conduct additional configurations with different latent dimensions D2 E (16, 64) under the Traffic dataset. As shown in Table 14, we observe a clear performance drop as the dimension decreases.
Table 13: Performance compar- ison across different latent size M E {0, 8, 32).
Dataset
ECL.
Traftic
Metric
MSE
MAE
MSE
MAE
0
0.165
0.256
0.415
0.288
8
0.158
0.249
0:401
0.272
16
0.159
0,250
0.403
0.273
32
0.157
0.249
0.397
0.271
Table 14: Performance comparison across different latent dimension LD E {16, 64, 128) with latent size M on traffic dataset. Bold indicates the best result. A lower MSE or MAE indicates better performance. The results are reported using input length L 384 and prediction lengths
H (96, 192, 336, 720}
Latent Dimension
16
64
128
Metric
MSE
MAE
MSE
MAE
MSE
MAE
96
0.392 0.271 0.385
0.266
0.378
0.264
192
0.392 0.265
0.394
0.262
0.388 0.263
336
0,415
0,284
0.411
0.282
0.403
0.276
720
0,436
0.294
0.435
0.290
0.433
0.290
Avg
[0.408
0.279
0.406
0.275
0.401 0.273
H
Imputation
Table 15: Performance comparison the imputation task with sequence length 192. The masking applied in units of patch length 24, and experiments are conducted by masking either one or two patches.
TimePerceiver
TimesNet
iTransformer
PatchTST
Dataset
Masked Patch
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
1
0.218
0.305
0.301
0.356
0.257
0.328
0.241
0.313
ETTb1
2
0.248
0.326
0.309
0.362
0.267
0.341
0.266
0.331
1
0.066
0.093
0.072
0.104
0.066
0.094
0.061
0.081
Weather
2
0.078
0.109
0.090
0.126
0.073
0.109
0.069
0.102
1
0.097
0.200
0.120
0.233
0.093
0.196
0.145 0.259
ECL
2
0.106
0.205
0.123
0.234
0.102
0.206
0.156 0.271
We conduct imputation experiments with three datasets (ETThI, Weather, Electricity) and three baselines [11, 6, 4J. We here set a more challenging task of imputation via patch-wise masking with a patch length of 24, as opposed to the easier setup of masking at individual timesteps. As shown in Table 15, TEREVERREIVER consistently achieves strong performance across all datasets, even though we did not specifically tailor our framework to the imputation task.
28

---

Table 16: Performance comparison with more baselines which are MLP-based, CNN-based and SSM- based models. Bold indicates the best result. A lower MSE or MAE indicates a better performance. Results are the average performance over four prediction lengths H E (96, 192, 336, 720] with input length I. = 96.
Models|
TimePereciver
TimeMixer
CycleNet
ModemTCN Sor-Mamba
Metric
MSE
MAE
MSE
MAB|
MSE
MAE
MSE
MAE
MSE
MAE
Weather
0.238
0.269
0.224 0.271
0.243
0.271
|0.240 0.274| 0.256 0.277
ECL
0.165
0.258
0.178
0.272
0.168
0.259
0.204 0.286
0.166 0.264
Traffic
0..428
0.275
00.484
0.297
0.472
0.313
00625
0.376
0.442 0.273
ETTh1
0.422
0.427
00.447
0,440
0.432
0.427 0,436 0.429 [0.433 0.436
ETTh2
0.355
0.395
0.364
0.395
0.383
0.404
0.358
0.398
00.376 0.405
ETTml
0.362
0.383
00,881
0.395
0.379
0.395
00.89
0,403
|0.391 0.400
ETTm2
00.766 0.316
0.275
0.323
0.266
0.314 0.280 0.323 0.281 0.327
Table 17: Performance comparison with more baselines which are Graph-based and RNN-based models. Bold indicates the best result. A lower MSE or MAE indicates a better performance. Results the average performance over four prediction lengths H {96, 192, 336, 720} with input length = 96. 1_' denotes result is not reported in the official publication.
Models
TimePerceiver
SageFormer
CrossGNN
MSGNet
SegRNN
xISTTM Mixer
Metric
MSE MAE MSE MAE MSE MAE MSE MAE| MSE
MAE|
MSE
MAE
Weather
0.238
0.269 0.247 0.273 0.247 0.289 0.249 0.278
0.252
0.299
0.254 0.275
ECL
0.165
0.258
0.175
0.273 0.201 0.271 0.194 0.300
0.184
0.278
0.174 0.259
Traffic
0.428
0.275
0.436
0.285
0.583
0.323
|0.651
0.324|
0.430 0.256
ETTh1
0,442
0.427
0.431
0.433 0.437
0.434
0,452
0.452|
0.425
0.429
0.442 0,430
ETTh2
0.355
0.395
0.374
0.403
0.363
0.418 0.396
0.417
0.374
0.406
0.377 0.402
ETTml
0.362
0.383
0.388
0.400
0.393
0.404
0.398
0.411
0.388
0.405
0.386 0.389
ETTm2
0.276
0.316
0.277 0.322 0.282 0.330 0.288
0.330
0.278
0.324
0.277 0.314
I Comparison with Other Baselines
In Table 1, we cover diverse model families (Transformer [4, 6-91, SSM [17], CNN [11], and MLP [13, 14]), while placing greater emphasis on Transformer-based baselines because our method is menssforerr-baseed As shown in Table 16, complement this focus, we add comparisons with recent non- Transformer models, including SSM [18]. CNN [34], and MLP [35, 36] based models. In additional comparisons with recent SOTA methods, our model achieves the best performance in 10 out of 14 cases.
As shown in Table 17, we here include more comparisons with recently proposed graph-based and RNN-based models that have attracted attention due to their novel foundations. Specifically, we compare against graph-based models such as SageFormer [37], CrossGNN [38], and MSGNet [39] as well as RNN-based models including SegRNN [40] and XLSTM-Mixer [41]. Our model still demonstrates superior performance in 12 out of 14 cases.
29

---

Discussion
Limitations. While this work presents a flexible and unified framework for time-series forecasting and demonstrates superior performance over recent baselines, it primarily focuses on multivariate forecasting benchmarks with regular time intervals, following common practice in the literature. Given that our framework is capable of modeling arbitrary time segments via latent bottlenecks, we believe it can extended irregular or event-based time series, which an important direction for future work.
formulation for time series modeling that broadens its applicability beyond traditional forecasting tasks. Unlike conventional methods that predict the future based only on past observations, our approach allows for flexible prediction across different temporal segments, including inferring from future to past or estimating intermediate time points. This makes applicable Therefore, we strongly believe our framework can be applied to diverse domains, such as healthcare, finance, and environmental monitoring, where time series data plays critical role, and can inspire future research toward unified and generalizable forecasting systems that better reflect the diversity of real-world applications.
Broader Impacts. This work presents
variety of real-world scenarios, such as reconstructing unobserved data.
Ethics Considerations. As our work focuses on core methodological contributions rather than specific applications, we do not foresee immediate ethical concerns.
30


Reviews:
[{"review_id": "XhsjJEMYpJ", "reviewer": "Reviewer_DDKH", "summary": "This paper presents **TimePerceiver**, a unified encoder-decoder framework for multivariate time series forecasting. The core idea is to generalise the training task beyond standard extrapolation tasks to include **interpolation** and **imputation** for arbitrary missing input.  \n\nTo that end, the authors propose a Transformer-based model using patch-based tokenisation and cross-variant attention. First, the context patches are transformed into a token representation augmented with temporal (patch position) and channel (index of dimension) information. The encoder compresses context patches into latent tokens (with size and dimensions to be chosen) and reprojects the contextualised input tokens into the original token space. The decoder uses tokens only based on temporal and channel information and uses cross-attention to transform these tokens into output tokens that are projected back into signal space.\n\nThe model is evaluated across 8 datasets with multiple forecasting horizons and consistently demonstrates strong performance, often achieving state-of-the-art or near-best results, with favourable computational and parameter efficiency. Furthermore, the authors show through ablation studies the benefits of their choices and show that the seasonality is recovered through the cross attention patterns.", "questions": "- The decoder operates by projecting contextualised query tokens back into full patch outputs. It remains unclear whether bypassing re-expansion to full resolution (i.e., using latent tokens directly) might yield a more efficient architecture. Have the authors tried directly using the encoded latent token as input for the decoder? If so, how much was the performance degrading?\n- I am not sure I understand the difference between interpolation and imputation in the description of the project.\n- I am not fully convinced that this framework would be able to directly handle arbitrary time segments as is, as this would blur the captured seasonality. I think additional information on the size of the patch should be integrated into the representation, perhaps through the same system as temporal and covariate index information. Have the authors tried anything in that regard?", "limitations": "The limitations of the models are correctly addressed but the statistical significance could be improved by including the standard deviation of the 5 runs of the experiments.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "- **Originality**:  While many of the ideas exist in other works: related ideas for the pretraining (e.g., masking-based pretraining) ([1,2]) and of having per channel and per patch attention by explicitely tokenizing them individually [3],  to the best of my knowledge, this is the first model not doing it in a two steps setting while limiting the computation and number of parameters needed. Thus, the learnable latent bottleneck is an interesting addition. \n\n- **Quality**:  \n  - (+)The model is built on clear motivations, and the experimental results are convincing. \n  - (-) Since the results were averaged over 5 runs, it could have been interesting to show the standard deviation of the results to show the statistical significance of the results (though it is not standard in the field).\n  - (+) The ablation study is compelling to experimentally show the benefits of their methods.\n  - (+) The visual analysis of the cross-attention map recovering the seasonality of the TS is an interesting addition. \n  - (-) Multiple latent sizes are tested, I suppose the choice is based on performance, but the procedure is not detailed. Studying the sensitivity of the model to this parameter should be included in the study. Furthermore, the reprojection in the original size is not motivated either.\n  - (-) Since the method was trained on an extended task able to handle imputation and interpolation, it could have been interesting to evaluate the performance on such tasks. \n\n\n- **Clarity**:  \n    The paper is well-structured with intuitive visualisations (e.g., Figures 1 and 2) that effectively explain the generalised forecasting setup and encoder-decoder flow. \n    \n- **Significance**:  Though many ideas seem to have been around already, this architecture with a learnable latent bottleneck allowing faster computation seems of interest for the community. Furthermore, the simplicity of the architecture, the very compelling results on different datasets and the superiority of the method against many SOTA methods make this contribution significant.\n\n\n    [1]|Dong, J., Wu, H., Zhang, H., Zhang, L., Wang, J., & Long, M. (2023). Simmtm: A simple pre-training framework for masked time-series modeling. _Advances in Neural Information Processing Systems_, _36_, 29996-30025.\n\n    [2] Kollovieh, M., Ansari, A. F., Bohlke-Schneider, M., Zschiegner, J., Wang, H., & Wang, Y. B. (2023). Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting. _Advances in Neural Information Processing Systems_, _36_, 28341-28364.\n\n    [3] Zhang, Y., & Yan, J. (2023, May). Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In _The eleventh international conference on learning representations_.", "was_revised": true, "final_justification": "The original submission was already quite convincing, presenting a simple yet effective architecture that leverages a latent bottleneck to enable efficient computation. The additional experiments have thoroughly addressed all of my concerns:\n- The inclusion of standard deviation demonstrates the statistical robustness of the results.\n- The analysis of latent size (extended in the response to Reviewer QYcP) and the \"reprojection layer\" provides sound justification for the architectural choices.\n- The additional experiments on imputation show competitive results compared to other baselines.\n- The additional details on training time and model complexity given to Reviewer rg1y make a compelling point for the benefits of the method despite not reaching SOTA on every dataset.\n\nI have increased my score in light of the authors' additional experiments, provided they are included in the final version.", "rebuttal": "Dear Reviewer DDKH,\n\nWe sincerely thank you for your helpful feedback and insightful comments. In what follows, we address your concerns one by one. Note that we only report MSE metrics in this rebuttal due to its limited space.\n\n---\n\n**[W1]** Since the results were averaged over 5 runs, it could have been interesting to show the standard deviation of the results to show the statistical significance of the results (though it is not standard in the field).\n\n**[Response]** Following your suggestion, we evaluate the stability of TimePerceiver by reporting mean and standard deviation over five runs with lookback window L=384. As shown below, across all 32 experimental settings, 25 cases exhibit a standard deviation less than or equal to 0.0005, indicating that TimePerceiver remains highly stable.\n\n| Input Length | ETTh1 | ETTh2 | ETTm1 | ETTm2 |\n| :---- | :---- | :---- | :---- | :---- |\n| 96 | 0.365±0.004 | 0.275±0.004 | 0.284±0.007 | 0.160±0.005 |\n| 192 | 0.397±0.002 | 0.335±0.003 | 0.323±0.004 | 0.215±0.003 |\n| 336 | 0.412±0.006 | 0.344±0.009 | 0.353±0.004 | 0.276±0.003 |\n| 720 | 0.442±0.008 | 0.392±0.010 | 0.399±0.002 | 0.359±0.006 |\n| **Input Length** | **Weather** | **Solar-Energy** | **ECL** | **Traffic** |\n| 96 | 0.142±0.002 | 0.161±0.006 | 0.124±0.001 | 0.374±0.001 |\n| 192 | 0.190±0.003 | 0.178±0.003 | 0.141±0.002 | 0.381±0.002 |\n| 336 | 0.244±0.000 | 0.188±0.003 | 0.161±0.003 | 0.396±0.002 |\n| 720 | 0.313±0.003 | 0.195±0.005 | 0.202±0.002 | 0.431±0.003 |\n\n---\n\n**[W2]** Multiple latent sizes are tested, I suppose the choice is based on performance, but the procedure is not detailed. Studying the sensitivity of the model to this parameter should be included in the study. \n\n**[Response]** Thank you for mentioning the important point. In our experiments, we simply chose the latent size based on validation. Following your suggestion, we further explore its sensitivity as reported below. We observe that the presence of a latent bottleneck clearly improves performance over having none, but the specific number of latent tokens plays only a minor role in determining the final performance, as they are all relatively small compared to the number of input tokens. This indicates that our model is not overly sensitive to this parameter, and even with a small latent size, we observe substantial performance gains. \n\n| Latent Size | ECL | Traffic |\n| :--- | :--- | :--- |\n| 0 | 0.165 | 0.415 |\n| 8 | 0.158 | 0.401 |\n| 16 | 0.159 | 0.404 |\n| 32 | 0.158 | 0.405 |\n\n---\n\n**[W3]** Since the method was trained on an extended task able to handle imputation and interpolation, it could have been interesting to evaluate the performance on such tasks.\n\n**[Response]** Following your suggestion, we conduct imputation experiments with three datasets (ETTh1, Weather, Electricity) and three baselines (TimesNet [1], iTransformer [2], PatchTST [3]). We here set a more challenging task of imputation via patch-wise masking with a patch length of 24, as opposed to the easier setup of masking at individual timesteps. As shown below, our TimePerceiver consistently outperforms the baselines across all datasets. We will add these results with more baselines into the final manuscript. \n\n| Dataset | Masked Patch | TimePerceiver | TimesNet | iTransformer | PatchTST |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| ETTh1 | 1 | 0.218 | 0.301 | 0.257 | 0.241 |\n|  | 2 | 0.248 | 0.309 | 0.267 | 0.266 |\n| Weather | 1 | 0.066 | 0.072 | 0.066 | 0.061 |\n|  | 2 | 0.078 | 0.090 | 0.073 | 0.069 |\n| Electricity | 1 | 0.097 | 0.120 | 0.093 | 0.145 |\n|  | 2 | 0.106 | 0.123 | 0.102 | 0.156 |\n\n\n[1] Wu et al., TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis, ICLR 2023 \\\n[2] Liu et al., iTransformer: Inverted Transformers Are Effective for Time Series Forecasting, ICLR 2024 \\\n[3] Nie et al., A Time Series is Worth 64 Words: Long-term Forecasting with Transformers, ICLR 2023\n\n---\n\n**[W2]** Furthermore, the reprojection in the original size is not motivated either.\\\n**[Q1]** The decoder operates by projecting contextualised query tokens back into full patch outputs. It remains unclear whether bypassing re-expansion to full resolution (i.e., using latent tokens directly) might yield a more efficient architecture. Have the authors tried directly using the encoded latent token as input for the decoder? If so, how much was the performance degrading?\n\n**[Response]** Thank you for the thoughtful comment. We have conducted experiments with the variant (referred to as DirectLT) you suggested in which the decoder uses latent tokens directly without re-expansion to full resolution. As shown below, our decoder design is significantly better than directly using the encoded latent token as input for the decoder. \n\nWe believe the superiority of our design stems from the following insight. In our framework, the latent bottleneck is designed to capture global temporal-channel patterns, while the original input sequence retains fine-grained local signals. Relying solely on the latent makes it difficult to reconstruct these detailed signals, ultimately leading to performance degradation. In contrast, our framework enriches input patch representations by leveraging the latent, enabling each to incorporate both global contexts and local details.\n\n| Input Length | Ours | DirectLT |\n| :--- | :--- | :--- |\n| 96 | 0.366 | 0.423 |\n| 192 | 0.394 | 0.437 |\n| 336 | 0.413 | 0.429 |\n| 720 | 0.445 | 0.473 |\n| Avg | 0.404 | 0.440 |\n\n---\n\n**[Q2]** I am not sure I understand the difference between interpolation and imputation in the description of the project.\n\n**[Response]** Your understanding is generally correct. Interpolation can indeed be considered a subset of imputation. However, we made the distinction because interpolation refers to filling in values within continuous gaps, while imputation includes both such cases and those where individual points are missing. As you insightfully pointed out, our model does not explicitly separate these cases, but rather handles them within a generalized formulation.\n\n---\n\n**[Q3-1]** I am not fully convinced that this framework would be able to directly handle arbitrary time segments as is, as this would blur the captured seasonality.\n\n**[Response]** We gracefully disagree with the concern that our framework may hinder the model’s ability to capture seasonality. On the contrary, we believe that the ability to handle arbitrary time segments enables the model to learn a broader range of recurring seasonal patterns. \n\nOur generalized formulation allows the model to learn from a wider range of configurations such as predicting the present given both past and future, or inferring the past from the future alone. By exposing the model to such diverse temporal contexts, we enhance rather than blur its ability to capture seasonal patterns. Crucially, latent tokens aggregate relationships among patches across every temporal‑channel tokens, creating a compressed global summary that captures seasonal patterns. This perspective is supported by our empirical results, where the generalized formulation consistently outperforms standard formulation across multiple settings.\n\n---\n\n**[Q3-2]**  I think additional information on the size of the patch should be integrated into the representation, perhaps through the same system as temporal and covariate index information. Have the authors tried anything in that regard?\n\n**[Response]** Regarding your suggestion to provide additional information on the size of the patch, since the current TimePerceiver implementation is trained with a fixed patch length, we did not include patch size embeddings in our framework. As shown below with our experiments on ETTh1, we inject a patch length embedding into both inputs and queries that produced no measurable gains in our ablation study. Therefore such a patch-size embedding has not proved necessary for the current setting.\n\nInstead, based on your idea, we further validate a new multi-resolution design with patch-size embeddings: (i) we patchify the input time-series with various patch sizes 12, 24, 48, and then (ii) use all patches as input tokens in our framework. As shown below, it has not yet delivered a clear performance lift. \n\nHowever, your insightful suggestion will be strongly considered in the future when developing time-series foundation models, especially when combined with our generalized formulation, since we believe that enabling the model to process time-series at multiple resolutions is critical for scaling it to handle large-scale data.\n\n| Input Length | TimePerceiver (Ours) | TimePerceiver (Patch Size Embedding) | TimePerceiver (Multi-Resolution) |\n| :--- | :--- | :--- | :--- |\n| 96 | 0.366 | 0.366 | 0.364 |\n| 192 | 0.394 | 0.390 | 0.395 |\n| 336 | 0.413 | 0.417 | 0.414 |\n| 720 | 0.445 | 0.445 | 0.473 |\n| Avg | 0.404 | 0.405 | 0.412 |"}, {"review_id": "UZ0lF6iwdg", "reviewer": "Reviewer_rg1y", "summary": "The authors introduce a fresh approach to tackling a variety of time-series forecasting challenges—not just predicting the future, but also filling in gaps and making sense of missing data. Unlike older methods that mostly focus on just the encoder part of the model, TimePerceiver brings the encoder, decoder, and training process together into one unified system. Its design uses clever attention mechanisms to understand complex patterns across both time and different data channels, and the decoder is flexible enough to handle any target points you care about. Instead of relying on a two-step training process, TimePerceiver is trained end-to-end, which streamlines things and helps it perform better. The authors back up their claims with thorough experiments, showing that their model consistently outperforms other top methods. Overall, the paper makes a strong case for rethinking how we design and train models for time-series forecasting by taking a broader, more integrated approach.", "questions": "1. What are the computational costs (training time, memory footprint) of TIMEPERCEIVER compared to the baselines, especially given the dual attention mechanisms and bottleneck structure?\n\nSuggestions: Please include a runtime and memory usage comparison across different sequence lengths. This is important to assess the trade-off between performance and scalability.\n\n2. Under what conditions does TIMEPERCEIVER struggle? Are there failure cases or performance drop-offs in certain data regimes (e.g., small datasets, irregular sampling)?\n\nSuggestions: Discuss known limitations and include a sensitivity analysis or case examples showing where the model might be less effective.\n\n3. How does each architectural component (e.g., latent bottlenecks, learnable queries, self-attention in bottlenecks) individually contribute to performance?\n\nSuggestions:  Please clearly  discuss each innovation's role. For example, what is the performance delta when removing learnable queries alone?\n\n4. Can the authors elaborate on specific real-world scenarios where the simultaneous handling of extrapolation, interpolation, and imputation is essential?\n\nSuggestions:  Add at least one case study or detailed scenario (e.g., in health, finance, or sensor networks) where this generalized forecasting capability provides a clear advantage over traditional models.", "limitations": "The authors do not address the limitations of the proposed framework. While the technical contributions are clearly presented, the following points  should be addressed:\n\na) The paper should explicitly discuss scenarios where TIMEPERCEIVER might fail or underperform, such as:  sparse time-series data, extremely long sequences where attention-based architectures struggle with memory and speed.\n\nb) The authors should comment on the training and inference costs. High computational demands can have environmental and accessibility implications.\n\nc) If deployed in domains like health or finance, biases in training data could propagate through the model. The authors should acknowledge these risks and suggest monitoring or mitigation strategies.\n\nd) If used in sensitive settings (e.g., medical records, financial logs), how does the model handle privacy risks, particularly if used for imputation of missing data?", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "This authors present how it brings together different parts of time-series forecasting—encoding, decoding, and training—into one well-designed, unified approach. The authors use attention mechanisms and bottleneck representations to help the model capture complex patterns, and they back up their claims with thorough experiments and diagrams that make even the vague concepts easier to understand. The authors expand the scope of forecasting to include not just predicting the future, but also filling in gaps and handling missing data, making their method useful for a wide range of real-world problems. The way the authors combine existing ideas in a new way shows real innovation and could have an impact across fields like health, finance, and climate science.\n\nOn the other hand, the model’s complexity could be hard to use in situations where resources are limited or real-time results are needed, and the paper doesn’t discuss these practical challenges. While the framework’s generality is a good idea, it would be more convincing if the authors gave more examples of where all these forecasting tasks are needed in the real world. Lastly, many of the individual building blocks come from other areas, so the true originality is in the combination rather than the components themselves.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer rg1y,\n\nWe sincerely thank you for your helpful feedback and insightful comments. In what follows, we address your concerns one by one. Note that we only report MSE metrics in this rebuttal due to its limited space.\n\n---\n\n**[Q1]** What are the computational costs (training time, memory footprint) of TIMEPERCEIVER compared to the baselines, especially given the dual attention mechanisms and bottleneck structure? Please include a runtime and memory usage comparison across different sequence lengths.\n\n**[Response]** As shown below, TimePerceiver demonstrates efficiency in both training time and memory footprint compared to baselines, regardless of the sequence length. This efficiency stems from the latent bottleneck that enables compression of core information efficiently. Additional analyses, such as the impact of longer prediction lengths, are provided in Appendix B.\n\n| Models | Seq Length | Memory(MB) | Runtime(s/iter) | Models | Seq Length | Memory(MB) | Runtime(s/iter) |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| TimePerceiver | 96 | 660 | 0.0435 | PatchTST | 96 | 1512 | 0.0547 |\n|  | 384 | 918 | 0.0583 |  | 384 | 6852 | 0.1156 |\n| DeformableTST | 96 | 972 | 0.0919 | Crossformer | 96 | 2768 | 0.1299 |\n|  | 384 | 2998 | 0.1153 |  | 384 | 4110 | 0.1350 |\n| CARD | 96 | 538 | 0.0737 | Timesnet | 96 | 1300 | 0.1376 |\n|  | 384 | 724 | 0.0828 |  | 384 | 3258 | 0.1385 |\n\n---\n\n**[Q2]** Under what conditions does TIMEPERCEIVER struggle? Are there failure cases or performance drop-offs in certain data regimes (e.g., small datasets, irregular sampling)? Discuss known limitations and include a sensitivity analysis or case examples showing where the model might be less effective.\n\n**[Response]** TimePerceiver is designed to be effective even in small datasets, as our generalized formulation enables the model to learn from a richer context beyond standard future prediction, especially by leveraging extrapolation, interpolation and imputation. However, in cases where the available time-series data is extremely short, the benefits of the generalized formulation may become less pronounced compared to the standard formulation. However, even in such cases, we observe that performance remains comparable to that of standard formulation.\n\n---\n\n**[Q3]** How does each architectural component (e.g., latent bottlenecks, learnable queries, self-attention in bottlenecks) individually contribute to performance? Please clearly discuss each innovation's role. For example, what is the performance delta when removing learnable queries alone?\n\n**[Response]** We would like to note that the analysis of each component is discussed in detail in Section 4.2 component ablation studies and Appendix D. Specifically, in the analysis, we conducted experiments by replacing each architectural component of TimePerceiver with a different structure or modifying the model formulation to verify the effectiveness of each component. We observe that the latent bottleneck contributed to performance improvement by efficiently aggregating temporal-channel dependencies. Additionally, the combination of learnable queries and a generalized formulation led to performance improvements due to its ability to handle diverse temporal contexts. We here further provide additional experiments with varying the number of latent tokens. As shown below, the results show a clear performance drop without latent bottleneck, highlighting its role in effectively modeling temporal and cross-channel dependencies. \n\n| Latent Size | ECL | Traffic |\n| :--- | :--- | :--- |\n| 0 | 0.165 | 0.415 |\n| 8 | 0.158 | 0.401 |\n| 16 | 0.159 | 0.404 |\n| 32 | 0.158 | 0.405 |\n\nWe would like to clarify that our generalized formulation cannot be implemented without our query-based decoder, so we did not provide the ablation study for our query-based decoder.\n\n---\n\n**[Q4]** Can the authors elaborate on specific real-world scenarios where the simultaneous handling of extrapolation, interpolation, and imputation is essential? Add at least one case study or detailed scenario (e.g., in health, finance, or sensor networks) where this generalized forecasting capability provides a clear advantage over traditional models.\n\n**[Response]** A concrete example is climate and environmental monitoring using sensor networks that measure variables such as temperature, humidity, and air quality. In such systems, our generalized forecasting capability is essential. First, imputation or interpolation is needed when sensor failures or communication issues lead to missing data. Second, extrapolation enables the estimation of past values from periods before sensor deployment. Lastly, forecasting is required to predict future conditions for effective planning and decision-making.\n\nWe believe our unified framework, built upon our generalized formulation, can flexibly support all of these objectives, making it particularly well-suited for real-world applications like environmental monitoring. We will incorporate this discussion into the final manuscript.\n\n---\n**[Limitations]** Thank you for the insightful comments. We provide additional responses regarding the mentioned limitations below. We also would like to note that we discussed limitations in Appendix E of the supplementary material. \n\n**[a]** Please refer to [Q2]. \\\n**[b]** Please refer to [Q1]. \\\n**[c]** We believe our framework is well-suited to address these biases, thanks to the generalized formulation that enables diverse contextual understanding and the latent bottleneck that effectively captures various temporal-channel dependencies. \\\n**[d]** Thanks to the the efficiency of our framework as shown in [Q1], we believe that it is executable in resource-limited edge devices without network access (e.g., uploading/downloading private data). This data isolation could mitigate such privacy risks."}, {"review_id": "oe3cqT1Vjc", "reviewer": "Reviewer_pmxb", "summary": "The author propose TimePerceiver, a generalized Transformer structure for time series forecasting. The authors propose three key innovations: latent bottleneck representations-based encoder, learnable query-based decoder and a hybrid architecture supporting flexible sequence length forecasting. \n\nExperiments on benchmarks (e.g., ETT, Traffic, Weather) demonstrate state-of-the-art (SOTA) performance.The model exhibits strong generalization and deterministic forecasting capabilities.", "questions": "Please refer to weaknesses.", "limitations": "yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.Well-written and clearly presents its contributions. 2. Interesting approach, and experimental results are soundness. 3. The analysis and discussion are convincing. \n\nWeaknesses:\n1. The novelty is weak, since the latent bottleneck representations of encoder is introduced in time series forecasting, “Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting” (ICLR2023). Also, there is no theoretical proofs for the bottleneck representation, please add. Therefore, the novelty is the learnable query-based decoder.\n2. As a generalized forecasting model, some generalized baselines should compare, \"One Fits All: Power General Time Series Analysis by Pretrained LM\", \"TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis\" and \"UniTS: A Unified Multi-Task Time Series Model\"\n3. In the learnable query of decoder, did the author consider adding covariates corresponding to the prediction target to enhance the expressiveness of the query? \n4. Since the author states that TimePerceiver can encompasses extrapolation, interpolation, and imputation tasks along the temporal axis in lines 54-56, could the author do some comparisons within these tasks, \"Optimal Transport for Time Series Imputation\", \"Csdi: Conditional score-based diffusion models for probabilistic time series imputation\".", "was_revised": true, "final_justification": "From the authors response, there are still three points that make it difficult to be competitive.\n\n1. There is no theoretical proofs for the bottleneck representations.\n\n2. The strategy of \"aggregates temporal and channel dependencies simultaneously\" is proposed by \"Unitst: Effectively modeling inter-series and intra-series dependencies for multivariate time series forecasting\", which is not novel in this manuscript.\n\n3. The experimental results show lower competitiveness, 57% experimental results are lower than baselines from the W2.", "rebuttal": "Dear Reviewer pmxb,\n\nWe sincerely thank you for your helpful feedback and insightful comments. In what follows, we address your concerns one by one. Note that we only report MSE metrics in this rebuttal due to its limited space.\n\n---\n\n**[W1]** The novelty is weak, since the latent bottleneck representations of encoder is introduced in time series forecasting, “Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting” (ICLR2023). Also, there is no theoretical proofs for the bottleneck representation, please add. Therefore, the novelty is the learnable query-based decoder.\n\n**[Response]** The use of latent bottleneck in TimePerceiver is fundamentally different from Crossformer [1] in the perspectives about (i) which information is aggregated into the latent and (ii) how to utilize the latent. First, our latent aggregates temporal and channel dependencies simultaneously via our latent bottleneck structure. This design explicitly captures interactions across both dimensions in a single operation. In contrast, Crossformer implicitly aggregates temporal-channel dependencies as interactions between temporal and channel information are considered separately. Here, Crossformer's bottleneck structure is used only in the cross-channel stage. Second, our latent is updated through multiple self-attention blocks. Such approach enables the latent to progressively refined through multiple self-attention blocks. The enhanced latent is then projected back to the input tokens, enabling it to incorporate more comprehensive and informative context. However, Crossformer's bottleneck structure is designed only for reducing computational cost, which is the reason why our TimePerceiver can outperform Crossformer.\n\n[1] Zhang et al., Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. ICLR 2023\n\n---\n\n**[W2]** As a generalized forecasting model, some generalized baselines should compare, \"One Fits All: Power General Time Series Analysis by Pretrained LM\", \"TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis\" and \"UniTS: A Unified Multi-Task Time Series Model\".\n\n**[Response]** Thank you for the suggestion. We would like to clarify that our generalized formulation differs from the use of the term generalized in foundation models in that we redefine time-series forecasting itself to include extrapolation, interpolation, and imputation in a unified framework. In contrast, the three papers you mentioned (One Fits All, TOTEM, and UniTS) treat generalized as building a single foundation model that supports a wide variety of tasks, including classification, anomaly detection, and forecasting. \n\nNevertheless, following your suggestion, we compare long-term forecasting results with the mentioned baselines with lookback window L=96. Note that “One Fits All” did not provide the results of L=96, we here omit the baseline. As shown below, our TimePerceiver outperforms TOTEM and is comparable to UniTS.\n\n| Datasets | TimePerceiver | TOTEM | UniTS |\n| :--- | :--- | :--- | :--- |\n| ETTh1 | 0.422 | 0.461 | 0.403 |\n| ETTh2 | 0.355 | 0.425 | 0.366 |\n| ETTm1 | 0.362 | 0.394 | 0.377 |\n| ETTm2 | 0.276 | 0.292 | 0.275 |\n| Weather | 0.238 | 0.239 | 0.235 |\n| ECL | 0.165 | 0.200 | 0.163 |\n| Traffic | 0.428 | 0.550 | 0.452 |\n\n---\n\n**[W3]** In the learnable query of decoder, did the author consider adding covariates corresponding to the prediction target to enhance the expressiveness of the query?\n\n**[Response]** We constructed the query not merely as learnable parameters, but as a composition of covariates corresponding to the prediction target including temporal information (referred to TPE) and channel-specific information (referred to CPE). As shown below, experiments on the ECL dataset demonstrate that removing CPE, which encodes channel information, leads to a performance drop. This demonstrates that adding covariates corresponding to the prediction target enhances the expressiveness of the query, as you expected. We also experimented with adding an embedding for patch size (referred to PSE). However, since our trained model uses a fixed patch size, this addition had minimal impact.\n\n| Input Length | TPE \\+ CPE (ours) | TPE | TPE \\+ CPE \\+ PSE |\n| :--- | :--- | :--- | :--- |\n| 96 | 0.125 | 0.135 | 0.127 |\n| 192 | 0.142 | 0.148 | 0.143 |\n| 336 | 0.161 | 0.166 | 0.161 |\n| 720 | 0.200 | 0.204 | 0.202 |\n| avg | 0.157 | 0.163 | 0.158 |\n\n---\n\n**[W4]** Since the author states that TimePerceiver can encompasses extrapolation, interpolation, and imputation tasks along the temporal axis in lines 54-56, could the author do some comparisons within these tasks, \"Optimal Transport for Time Series Imputation\", \"Csdi: Conditional score-based diffusion models for probabilistic time series imputation\".\n\n**[Response]** Following your suggestion, we compare our framework with PSW-I [1] under the standard imputation setup [2,3] using 6:2:2 training/validation/test splits. As shown below, our TimePerceiver outperforms the baseline, PSW-I, under the ETTh1 dataset. We will add more comparisons (with other baselines, e.g., CSDI, and other datasets, e.g., Weather) into the final manuscript.\n\n| Dataset | TimePerceiver | PSW-I |\n| :---- | :---- | :---- |\n| ETTh1 | 0.218 | 0.253 |\n\n[1] Wang et al., Optimal Transport for Time Series Imputation, ICLR 2025 \\\n[2] Wu et al., TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis, ICLR 2023 \\\n[3] Goswami et al., MOMENT: A Family of Open Time-series Foundation Models, ICML 2024"}, {"review_id": "6RwqBXp362", "reviewer": "Reviewer_QYcP", "summary": "The authors propose a new attention based encoder-decoder architecture along with a new training regime to unify the development of encoder and decoder as a single task (Generalized Formulation). This helps the encoder to attend to multiple arbitrary sampled segments of the input. Their experiments show that they outperform various (transformer focused) baselines on the standard multivariate long-term forecasting benchmark datasets. Furthermore, the authors propose a novel latent bottleneck  that compresses input patches into a small set of learnable tokens to improve from O(n^2) to O(NM), with M << N.", "questions": "- Q1. How does your architecture differ from T5? Please discuss how your generalized formulation can be projected to other architectures (or is this impossible?).\n    \n- Q2. In Table 2, could the authors provide all combinations? It is unclear what impact positional embedding sharing has on the standard formulation.\n    \n- Q3. While the focus is on transformer-based methods, a global view should include other architectures. Please provide results for modern RNN methods (xLSTM-Mixer), MLPs (CycleNet, TimeMixer, TimeMixer++), and convolutional models (ModernTCN).\n    \n- Q4. In your ablation, you report that ETTh1 works best for contiguous sampling. Could the authors provide a rationale? Simply stating that this highlights the importance of sampling ratio is insufficient, especially since one of the main claims is that previous pretraining methods do not model this behavior correctly.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- Strengths:\n    \n    - The paper introduces a novel unified encoder-decoder framework and an interesting idea of sampling multiple temporal segments.\n        \n    - The figures are clear and aid understanding of the architecture\n    - The manuscript is easy to follow.\n    - The experiments are extensive, although focused on long-term forecasting.\n        \n    - The authors outline research questions, which improves the structure.\n        \n- Weaknesses:\n    \n    - The authors omit recent architectures (e.g., CNN- and MLP-based models such as CycleNet, TimeMixer, TimeMixer++, and ModernTCN), as well as entire model families (RNNs) and newly introduced (x)LSTMs (xLSTM-Mixer, Tirex, SegRNN), all of which achieve SOTA results in time series forecasting.\n        \n    - Masking and reconstruction tasks are common pretraining strategies but not the only ones; prior time series pretraining works include TS-TCC, TFC, TS2Vec, and others that utilize multiple self-supervised losses.\n        \n    - The significance and novelty of the architecture are unclear, as the work appears to build directly on previous pretraining methods while ignoring modern foundation models that already use an encoder-decoder architecture (for example, Lag-Llama and Chronos on top of the T5 architecture). In general, the related work section is not exhaustive. The authors should include foundation models such as TimesFM (decoder-only) and Chronos or Chronos-Bolt (patch-based), as well as some of the earlier time series pretraining works.\n        \n    - While the unified view is appealing, it is confusing how this relates to general time series model analysis; if a unified view is employed, methods from imputation should also be included in the evaluation or the related work section.\n        \n    - The experiments over different input lengths are appreciated, but results without multiple seeds are less meaningful.\n        \n    - The notation for the generalized framework is confusing; equations 5–7 require multiple readings to understand.\n        \n    - The number of latent tokens is not sufficiently explored; reporting that 8, 16, and 32 tokens work best is not enough. What happens when decreasing or increasing this number? From a practical perspective, it is unclear how to choose this hyperparameter.\n\n\n### Refs\n\nKraus, Maurice, et al. \"xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories.\" _arXiv preprint arXiv:2410.16928_ (2024).\n\nAuer, Andreas, et al. \"TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning.\" _arXiv preprint arXiv:2505.23719_ (2025).\n\nLin, Shengsheng, et al. \"Segrnn: Segment recurrent neural network for long-term time series forecasting.\" _arXiv preprint arXiv:2308.11200_ (2023).\n\nEldele, Emadeldeen, et al. \"Time-series representation learning via temporal and contextual contrasting.\" IJCAI (2021)\n\nZhang et al. \"Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency\" NeurIPS (2022)\n\nYue, Zhihan, et al. \"Ts2vec: Towards universal representation of time series.\" _Proceedings of the AAAI conference on artificial intelligence_. Vol. 36. No. 8. 2022.\n\nAnsari, Abdul Fatir, et al. \"Chronos: Learning the language of time series.\" _arXiv preprint arXiv:2403.07815_ (2024).\n\nDas, Abhimanyu et al. \"A decoder-only foundation model for time-series forecasting\" ICML (2024)\n\nRasul, Kashif, et al. \"Lag-llama: Towards foundation models for time series forecasting.\" _R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models_. 2023.\n\nLin, Shengsheng, et al. \"Cyclenet: Enhancing time series forecasting through modeling periodic patterns.\" _Advances in Neural Information Processing Systems_ 37 (2024): 106315-106345.\n\nWang, Shiyu, et al. \"Timemixer: Decomposable multiscale mixing for time series forecasting.\" ICLR (2024).\n\nWang, Shiyu, et al. \"Timemixer++: A general time series pattern machine for universal predictive analysis.\" ICLR (2025).\n\nLuo, Donghao, and Xue Wang. \"Moderntcn: A modern pure convolution structure for general time series analysis.\" ICLR (2024).", "was_revised": true, "final_justification": "I've raised my score to a borderline accept as the authors answered all my remaining concerns, added missing baselines, and showcased how the components interplay with each other. Furthermore, they provided clarifications and added missing ablations. However, due to the still missing \"final\" baseline results and the sheer amount of changes that will be added to the paper, I am not comfortable going higher than a borderline accept. Otherwise, this is strong work and could be published at some point at NeurIPS or a similar venue.", "rebuttal": "Dear Reviewer QYcP,\n\nWe sincerely thank you for your helpful feedback and insightful comments. In what follows, we address your concerns one by one. Note that we only report MSE metrics in this rebuttal due to its limited space.\n\n---\n\n**[Q1-1, W3]** How does your architecture differ from T5?\n\n**[Response]**  TimePerceiver is fundamentally different from Chronos-T5 in the perspectives about how the model operates based on the context of time. Firstly, TimePerceiver consists of a decoder with query tokens containing time and channel information about target segments. Therefore, the decoder flexibly extracts information based on the time point we want to predict from the representation captured by the encoder through the latent bottleneck, which captures temporal-channel dependencies.\nHowever, as stated in the Chronos paper, Chronos-T5 completely omits time and frequency information and treats time series as a plain sequence of tokens. The decoder works in an auto-regressive manner, looking only at the tokens generated so far and producing the probability distribution for the next single token. As a result, the model relies purely on sequential token patterns to predict the next value, without any position-specific information that is unique to time-series data.\n\nTo sum up, TimePerceiver can respond flexibly even if the input and output intervals vary each time. On the other hand, Chronos-T5 remains stuck in a token-next-token approach, making it difficult to structurally solve generalized formulation other than predicting specific future intervals.\n\n---\n\n**[Q1-2]** Please discuss how your generalized formulation can be projected to other architectures (or is this impossible?).\n\n**[Response]** We believe that our generalized formulation can be easily projected to other patch-based architectures by replacing their prediction layers with our query-based decoder. PatchTST [1] is a clear example: replacing its linear decoder by our query-based decoder and adopting it with our generalized formulation yields a TimePerceiver variant that uses PatchTST’s encoder design. Not only that, but the more naturally a structure can handle temporal positional information, the more easily our idea can be projected.\n\n[1] Nie et al., A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. ICLR 2023\n\n---\n\n**[Q2-1]** In Table 2, could the authors provide all combinations? \n\n**[Response]** Below are the results for the combinations that were not shown for the standard formulation in Experiment 4.2 and Appendix D.\n\n| Formulation | Encoder Attention | ETTh1 | ETTm1 | Solar | ECL |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| Standard | Full Self-Attention | 0.437 | 0.368 | 0.194 | 0.173 |\n| Standard | Decoupled Self-Attention | 0.445 | 0.377 | 0.196 | 0.166 |\n\n---\n\n**[Q2-2]** It is unclear what impact positional embedding sharing has on the standard formulation.\n\n**[Response]** In the standard formulation, PE sharing means that channel positional embedding (CPE) is shared across the encoder and the decoder in our architecture. In contrast, our generalized formulation, PE sharing means that both CPE and temporal positional embedding (TPE) are shared. Therefore, PE sharing has less impact in the standard formulation, as we observed a slight performance difference in its ablation study. We will clarify the ablation experiments in the final manuscript.\n\n---\n\n**[Q3, W1]** While the focus is on transformer-based methods, a global view should include other architectures. Please provide results for modern RNN methods (xLSTM-Mixer), MLPs (CycleNet, TimeMixer, TimeMixer++), and convolutional models (ModernTCN).\n\n**[Response]** We provide comparison results for the methods you mentioned below. All evaluations were conducted with a lookback window L=96 for consistency. Since public results and scripts for xLSTM-Mixer under this setting are not available, we instead include SegRNN for comparison. \n\nExperimental results show that TimePerceiver consistently outperforms the baselines. While TimeMixer++ is reported to perform slightly better than our method, it is worth noting that the reported standard deviation in the original paper is relatively high (around 0.2 across most datasets), and the code has not been publicly released. Moreover, when we run experiments using an unofficially released version of the code, we were unable to reproduce the reported results. For these reasons, we are cautious about making a direct comparison with TimeMixer++.\n\n| Dataset | TimePerceiver | TimeMixer++ | TimeMixer | CycleNet | SegRNN | ModernTCN |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| Weather | 0.238±0.002 | 0.226±0.008 | 0.240 | 0.243 | 0.252 | 0.240 |\n| ECL | 0.165±0.003 | 0.165±0.017 | 0.178 | 0.168 | 0.184 | 0.204 |\n| Traffic | 0.428±0.008 | 0.416±0.027 | 0.484 | 0.472 | 0.651 | 0.625 |\n| ETTh1 | 0.422±0.005 | 0.419±0.023 | 0.447 | 0.432 | 0.425 | 0.436 |\n| ETTh2 | 0.355±0.007 | 0.339±0.020 | 0.364 | 0.383 | 0.374 | 0.358 |\n| ETTm1 | 0.362±0.003 | 0.369±0.019 | 0.381 | 0.379 | 0.388 | 0.389 |\n| ETTm2 | 0.276±0.002 | 0.269±0.021 | 0.275 | 0.266 | 0.278 | 0.280 |\n\n---\n\n**[Q4]** In your ablation, you report that ETTh1 works best for contiguous sampling. Could the authors provide a rationale? Simply stating that this highlights the importance of sampling ratio is insufficient, especially since one of the main claims is that previous pretraining methods do not model this behavior correctly.\n\n**[Response]** Based on our visualization and analysis of the datasets, we observed that (i) fully contiguous sampling is effective when capturing global patterns is important for the data within a single instance, whereas (ii) fully disjoint sampling performs better when the data contains frequently repeated local patterns.\nFirst, fully contiguous sampling proved effective not only for ETTh1 but also for the Weather dataset. Both datasets exhibit strong global patterns rather than repetitive local behaviors. In such cases, removing long contiguous segments from the input forces the model to infer high-level temporal dependencies, which helps in learning global trends more effectively.\nSecond, datasets such as Solar, ECL, and Traffic showed more consistent global patterns but contained local variations, often with repetitive short-term patterns. For these datasets, fully disjoint sampling led to better performance, as it encourages the model to solve many small-scale reconstruction tasks and thus better capture local dynamics. Therefore, when prior knowledge about the characteristics of the data is available, we recommend selecting a sampling strategy accordingly to best exploit that knowledge. In the absence of such prior information, using the mixed version serves as a robust default choice. We will incorporate this clarification into the revision.\n\n---\n\n**[W2]** Masking and reconstruction tasks are common pretraining strategies but not the only ones; prior time series pretraining works include TS-TCC, TFC, TS2Vec, and others that utilize multiple self-supervised losses.\n\n**[Response]** Our contribution lies not only in the generalized forecasting formulation but also in the encoder-decoder architecture that is closely aligned with this formulation. Furthermore, we believe that our framework is compatible with various self-supervised learning objectives mentioned (e.g., contrastive learning) and can be extended to incorporate them effectively.\n\n---\n\n**[W4]** While the unified view is appealing, it is confusing how this relates to general time series model analysis; if a unified view is employed, methods from imputation should also be included in the evaluation or the related work section.\n\n**[Response]** Following your suggestion, we conduct imputation experiments with three datasets (ETTh1, Weather, Electricity) and three baselines (TimesNet, iTransformer, PatchTST). We here set a more challenging task of imputation via patch-wise masking with a patch length of 24, as opposed to the easier setup of masking at individual timesteps. Due to the space limit, please refer to [Reviewer DDKH, W3 Response] for the imputation results. As shown in the results, our TimePerceiver consistently outperforms the baselines across all datasets. We will add these results with more baselines into the final manuscript.\n\n---\n\n**[W5]** The experiments over different input lengths are appreciated, but results without multiple seeds are less meaningful.\n\n**[Response]** In Appendix B, we clarify that all reported results are the average of 5 runs using different random seeds. Also, across all 32 experimental settings with lookback window L=384, 25 cases exhibit a standard deviation less than or equal to 0.0005, indicating that TimePerceiver remains highly stable. For details, refer to [Reviewer DDKH, W1 Response].\n\n---\n\n**[W6]** The number of latent tokens is not sufficiently explored; reporting that 8, 16, and 32 tokens work best is not enough. What happens when decreasing or increasing this number? From a practical perspective, it is unclear how to choose this hyperparameter.\n\n**[Response]** Thank you for mentioning the important point. In our experiments, we simply chose the latent size based on validation. Following your suggestion, we further explore its sensitivity as reported below. We observe that the presence of a latent bottleneck clearly improves performance over having none, but the specific number of latent tokens plays only a minor role in determining the final performance, as they are all relatively small compared to the number of input tokens. This indicates that our model is not overly sensitive to this parameter, and even with a small latent size, we observe substantial performance gains. \n\n| Latent Size | ECL | Traffic |\n| :--- | :--- | :--- |\n| 0 | 0.165 | 0.415 |\n| 8 | 0.158 | 0.401 |\n| 16 | 0.159 | 0.404 |\n| 32 | 0.158 | 0.405 |\n\n---"}]

Author rebuttal:

Meta-review: **(a) Scientific claims and findings.**
The paper proposes TimePerceiver, a generalized encoder–decoder framework for time-series forecasting that unifies extrapolation, interpolation, and imputation within one architecture. It introduces a latent bottleneck representation and learnable decoder queries, demonstrating strong empirical results across multiple benchmarks.

**(b) Strengths.**
The work is well-motivated, clearly presented, and empirically validated with extensive experiments and ablations. Its unified formulation and bottleneck design show efficiency and adaptability, and the results are competitive or state-of-the-art across diverse datasets.

**(c) Weaknesses.** 
Concerns remain about limited novelty relative to prior work, lack of theoretical justification for the bottleneck, incomplete coverage of recent baselines, and insufficient discussion of limitations such as scalability, real-world deployment, and statistical robustness.

**(d) Decision rationale**
Despite reservations about originality and missing baselines, the unified framework, convincing empirical performance, and thorough responses during the rebuttal collectively justify acceptance. The contribution is likely to be impactful for the time-series forecasting community.  Acceptance is recommended contingent on including the additional comparisons, analyses, and clarifications promised in the discussion.

**(e) Discussion and rebuttal.**
Reviewers initially raised concerns about novelty, missing baselines, computational costs, and lack of clarity in definitions and significance.  The authors addressed these by adding efficiency analyses, imputation experiments, ablation details, comparisons with stronger baselines, and case studies illustrating real-world use.  While some novelty and theoretical questions persist, the responses and planned revisions were sufficient to alleviate most concerns.  On balance, the paper benefits from the discussion and should be acceptable provided improvements are incorporated.
Decision: Accept (poster)