Time-o1: Time-Series Forecasting Needs Transformed Label Alignment
Authors: Hao Wang
Licheng Pan
Zhichao Chen
Xu Chen
Qingyang Dai
Lei Wang
Haoxuan Li
Zhouchen Lin
Venue: NeurIPS 2025 poster ()
Keywords: Time-Series
Label Autocorrelation
Orthogonalization
Primary area: applications
TL;DR: Learning to forecast in the transformed domain improves forecasting performance.

Abstract: Training time-series forecasting models poses unique challenges in loss function design. Most existing approaches adopt temporal mean squared error, but this study reveals two critical limitations: (1) it ignores the presence of label autocorrelation, which biases it from the true label sequence likelihood;  (2) it involves excessive number of tasks, which complicates optimization, especially for long-term forecasting. To address these issues, we introduce Time-o1, a transform-enhanced loss function for time-series forecasting. The central idea is to transform the label sequence into decorrelated components with discriminated significance. Models are then trained to align the most significant components, thereby effectively mitigating label autocorrelation and reducing task amount. Experiments demonstrate that Time-o1 achieves state-of-the-art performance and is compatible with various forecast models. Code is available at https://github.com/Master-PLC/Time-o1.

Time-ol: Time-Series Forecasting Needs Transformed Label Alignment
Hao Wang I+
Licheng Pan
Zhichao Chen?
Xu Chen 3f
Qingyang Dai
Lei Wang Haoxuan Li
Zhouchen Lin 2,6.71 L
Xiaohongshu Inc.
2 State Key Lab of General AI, School Intelligence Science and Technology, Peking University
Gaoling School of Artificial Intelligence, Renmin University of China Department of Control Science and Engineering, Zhejiang University Center for Data Science, Peking University Institute for Artificial Intelligence, Peking University Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, China
Abstract
Training time-series forecasting models poses unique challenges loss func- tion design. Most existing approaches adopt temporal mean squared error, but this study reveals two critical limitations: autocorrelation, which biases it from the true label sequence likelihood; @ it involves excessive number of tasks, which complicates optimization, especially for long-term forecasting. To address these issues, we introduce Time-o1, loss function for time-series forecasting. The central idea is transform the label sequence into decorrelated components with discrimi- nated significance. Models are then trained to align the most significant com- ponents, thereby effectively mitigating label autocorrelation and reducing task amount. Experiments demonstrate that Time-ol achieves state-of-the-at per- formance and is compatible with various forecast models. Code is available at https:/ /gi thub. com Com/Master-PLCC. Ptt.--meeoTmm........
it ignores the presence of label
Introduction
Time-series forecasting involves predicting future data from historical observations [62, 24] and has been applied across diverse domains, such as air quality prediction in meteorology [29], user behavior analysis in e-commerce [3], and process monitoring in manufacturing [49, 52]. To build effective forecasting models, two questions warrant investigation: (1) How to design neural network architecture to encode historical observations, and (2) How to devise a loss function to train the neural network. Both are critical for forecast model performance.
Recent research has primarily focused on developing neural network architectures [57, 61]. The key challenge lies in exploiting the autocorrelation in history sequences. To this end, various architectures have been proposed [25, 41, 53, 31]. Current progress centers on a debate between Transformers and simple linear models. Transformers, equipped with self-attention mechanisms, offer superior scalability [28, 35, 30]. In contrast, linear models, which encapsulate temporal dynamics using linear layers, straightforward to implement and demonstrate strong performance [67, 44, 65, 63]. These advancements showcase the rapid evolution in neural architecture design for time-series forecasting.
"This work was done in the internship at Xiaohongshu Inc. Both authors have equal contribution. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In contrast, the design of loss functions has received less attention [51, 40, 23]. Most studies adopt the temporal mean squared error (TMSE) as the loss function, which calculates the step-wise difference between the forecast and label sequences [28, 351. While it is effective in various scenarios, it exhibits ignores the presence of label autocorrelation, treating each label step as independent, which renders it biased from the true likelihood of the label sequence [51]; involves excessive number of tasks, where the task amount corresponds to the forecast horizon, which complicates optimization especially in long-term forecast scenarios [69].
two limitations in time-series forecasting:
To handle these challenges, we propose a consform-enhaneed loss function for time-serie forecasting (Time-ol). The key idea is to transform the label sequence into decorrelated components ranked by significance. By aligning the most significant decorrelated components, Time-ol mitigates label autocorrelation and reduces the number of tasks. Our main contributions are summarized as follows:
We formulate two critical challenges in designing loss functions for time-series forecasting: the label autocorrelation that induces bias, and the excessive number tasks that impedes optimization.
We propose Time-ol, novel loss function for training time-series forecast models. It transforms label and forecast sequences into decorrelated components ranked by significance and subsequently align them, which effectively addresses challenges 0-0 with theoretical guarantees.
We validate the efficacy of Time-ol through extensive experiments, where Time-ol consistently outperforms existing loss functions and enhances the performance of various forecast models.
2 Preliminaries
This paper focuses on the time-series forecasting problem [39, 56]. In general, we adhere to standard notational conventions: uppercase bold letters (e.g., X) denote matrices, lowercase bold letters (e.g.. x) denote vectors, and lowercase normal letters (e.g., 2)) denote scalars. Since the autocorrelation property central to our analysis manifests independently within each variate, adopt the univariate setting for problem formulation [35], which generalizes naturally to the multivariate setting.
A time series, denoted by {s1,..., 8m} E IR-M consists of a sequence of chronologically ordered observations. At any time step n, the history sequence is represented by x = [5p-H+1,-- 8n] E R and the corresponding label sequence is y = [$x+1.-... Sn+T] E R where H denotes the history length and T is the forecast horizon. The goal of time-series forecasting is to learn a model g H + ET that produces a forecast sequence ý closely matching the ground truth label sequence.
There are two aspects to building forecast models: (1) neural network architectures that effectively encode history sequences, and (2) loss functions for training these neural networks. While this paper focuses on the loss function, we provide a brief review of both aspects for contextualization.
2.1 Model architectures for time-series forecasting
Neural networks are widely employed to encode history sequences [32, 60] due to their ability to automatically model feature interactions and capture complex nonlinear autocorrelation [31. 12, 11]. Notable examples include recurrent neural networks (e.g.. S4 [10], Mamba [9], P-sLSTM [16]), convolutional neural networks (e.g., SCINet [27], TimesNet [58], MICN [54]), and graph neural networks (e.g., MTGNN [34], StemGNN [2]), each tailored to encode the autocorrelations within input sequences. The current progress centers comparisons between Transformer-basee and linear architectures. Transformers (e.g.. PatchTST [35], iTrrnsformer [28], FreeFormer [66]) exhibit substantial scalability with increasing data size but entail high computational costs. In contrast, linear architectures (e.g., DLinear [67], RLinear [44], OLinear [65], TimeBase [13]) are generally more efficient but less scalable with larger datasets and struggle to handle varying input lengths.
2.2 Loss functions for time-series forecasting
Modern time-series models predominantly adopt the direct forecast paradigm, generating T-step forecasts simultaneously using multi-output head [22, 28, 67]. The standard loss function is the temporal mean squared error (TMSE) between the forecast and label sequences, given by:
N
MSSE =
Y
Y
(yatt - )o,))²
(1)
2
n=1

---

where N is the number of samples. LMSE is widespread in recent studies (e.g., FreTS [64], Fred- Former [37]. iTransformer [28], DUET [41]); however, it proves biased in the presence of label autocorrelation [. To address this bias, one line of works performs shape alignment between the forecast and label sequences (e.g., Soft-DTWW [4], Dilate [18], and STRIPE [ [19)). While these meth- ods heuristically exploit label autocorrelation, they lack theoretical guarantees regarding unbiasedness and empirical evidence of improved performance [18]. A more recent and notable approach aligns the label and forecast sequence in the frequency domain [51, 23]; this strategy offers theoretical guarantees for bias reduction [5] and empirical improvements across scenarios [55, 20, 70].
3 Methodology
3.1 Motivation
The design of loss function is pivotal in training forecast models. Most previous studies employ LMSE in (I) as the default loss function [28. 35, 65]. However, it suffers from two limitations stemming from the inherent properties of time series. It ignores the presence of label autocorrelation. Specifically, each observation in time-series dependent on its past values [67]; this leads to label autocorrelation, i.c., different steps in the label sequence are correlated. However, LMSE assumes each label step is independent, which disregards label autocorrelation and thus yields biased loss function, as formulated in Theorem 3.1. @ It involves excessive number of tasks. Specifically. the number of forecast tasks in LISE corresponds directly to the forecast horizon T. While large forecast horizons are crucial for applications such as manufacturing (requiring long-horizon planning [50, 48)) and transportation (facilitating proactive traffic control [68, 33), they introduce optimization challenges, e.g., gradient conflicts [69, 26], which impedes convergence and leads to suboptimal performance.
Theorem 3.1 (Autocorrelation bias). Given univariate label sequence y E IR' where E E Rxx denotes the step-wise correlation coefficient, the loss function LMSE in (1) is biased from the true negative log- le-iikeiihood of the label sequence, which is given by:
Bias lyy yly - lly - 311
log E
(2)
where | = 'v... 'v The bias vanishes if different steps in y are decorrelated."
Designing a loss function to handle the two limitations is challenging. Our previous work [51] proposes a frequency-domain loss, which transforms the label and forecast sequences into frequency components and then aligns them. It is motivated by Theorem 3.1: bias vanishes if different components are decorrelated. However, the decorrelation of frequency components holds only when the forecast horizon T oo (see Theorem 3.3 in [51]). In real-world settings with a finite horizon, frequency components remain correlated, rendering FreDF ineffective to fully eliminate bias. Moreover, optimization difficulty remains, since transforming to the frequency domain retains the label length. Consequently, FreDF does not fully address the limitations 1 @ discussed in this paper.
Given the significance of loss function in training forecast models and the limitations of existing methods, is compelling to develop an innovative loss function to address the limitations and advance forecast performance. Concretely, two questions warrant investigation. How to devise loss function to eliminate autocorrelation bias and reduce task amount? Does improve forecast performance?
3.2 Transforming label sequence with optimized projection matrix
In this section, we present a method for transforming label sequences into latent components to climi- nate autocorrelation and distinguish significant components. Suppose Y E RxT contains univariate label sequences of N samples, P Pt. P2.... Pk] is the projection matrix; the components are produced as Z YP. The target is for Z to be decorrelated and ranked by significance. For example, FreDF specifies as a Fourier matrix, which does not adapt to specific data properties and thus fails to decorrelate the components and distinguish the significant components*.
'The pioneering work [51] identifies the bias under first-order Markov assumption on the label sequence. This study generalizes this bias without the first-order Markov assumption. In the multivariate case, different variates be treated separately to produce decorrelated components.
3

---

1.00
0.75
0.75
10
0.00
3.50
025
0.25
-10
-000
15
40
32
64
90
(a) Autocorrelation in label sequence and components.
(b) Volume label sequences and latent components.
Figure I: The autocorrelations and volumes in the label sequence Y and latent components Z.
natural approach to obtaining P with the desired properties is constructing optimization problem with constraints. Specifically, to find the k-th component, the projection vector P can be defined as:
(Yps) T (Yps)
P
argmax Pt
|P 1 =1 P3 P O, A i
subject to
where Zμ: = Ypd is the k-th component, the normalization constraint ||px|l imposed to avoid trivial solution: Ps: - co. The optimization target is maximize the variance of Zk. which is equivalent to maximizing its significance, as components with larger variance contain richer information. For k 1, the projection axis is required be orthogonal the previous redundancy. By solving the opumizations above from k = to K < 'T sequentially, we obtain the projection matrix P pi..... pkl. The components are then produced as YP*
avoid
Lemma 3.2 (Decorrelated components). Suppose Y E NxT contains normalized label sequences for N samples, Z [Zı,..., ..., žg] are the obtained components; for any k & k', we have z z 0. Lemma 3.3. The projection matrix P can be obtained via singular value decomposition (SVD): Y= = UA(P*) where E NNN and P RxKK consist of singular vectors, and the diagonal of A NxK consists of singular values.
Theoretical Justification. By Theorem 3.1, the autocorrelation bias vanishes as the label corre- lations are eliminated. Since the obtained components Z are decorrelated (Lemma 3.2), applying step-wise difference to align them suffers from little autocorrelation bias. Moreover, component signif- icance decreases from 21 to ZK they are derived under sequentially added constraints. Furthermore, P can be computed via SVD (Lemma 3.3), offering an efficient strategy to obtain P
Case study. To showcase the implications of the obtained components, a case study was conducted on the E dataset. Implementation details are provided in Appendix A. The results are illustrated in Fig. 1, with key observations summarized as follows:
Decorrelation property: Fig. (a) compares the autocorrelation volume in the label sequence and the transformed components obtained by (3). In the left panel, the value row i and column represents the correlation between the i-th and j-th steps in the label sequence. A large number of non-diagonal elements exhibit substantial values, with approximately 50.5% exceeding 0.25, indicating significant label autocorrelation. In contrast, the right panel shows negligible values for the non-diagonal elements. This demonstrates that transforming the label sequence into components effectively eliminates correlation, which empirically validates Lemma 3.2.
Significance discrimination: Fig. (b) compares the variance the label sequence and the transformed components in (3). In the left panel, the variance of the label sequence almost uniform across different steps (ranging from -1.5 to 1.5), indicating that different steps contribute similarly the overall information and are almost equally significant. In the right panel, small number of components has large variance. This demonstrates that the transform yields components with ranked significance. As a result, one can balance minor information loss with substantial decreases in optimization complexity by concentrating learning on the most significant components.
The transformation highly inspired by principal component analysis (PCA) [36]. However, one key distinction warrants emphasis. Existing works dominantly employ principal component analysis

---

on input features for obtaining informative representations [8, 6], in contrast, we adapt it to label sequence. specifically aiming reduce autocorrelation bias and simplify optimization for time-series forecasting. To our knowledge, this remains technically innovative strategy.
3.3 Model implementation
In this section, present the implementation details of Time-ol. The approach centers on extracting the latent components from the label sequence, then optimizing the forecast model using the most significant components.
Algorithm The workflow of Time-ol.
Input: Y: forecast sequences, Y: label sequences.
Parameter: or relative weight transformed loss, y: the ratio of retained components. Output: Lo.y: the obtained loss function.
Given history sequences X NR and la- bel sequences Y 1R,, the forecast model
1: Y + standardize Y).
generates forecast sequences Y g(X). In line with prior practices [28, 67, 37], we standardize Y (step 1). which is prerequisite for ensuring the decorrelation of the resulting components
2: K t round(y T)
P SVD(Y;:)
4: + YP" 2 + YP
5: Lortho,y + 12 ZI|1
(see Lemma 3.2). Next, following Lemma 3.3, we compute the optimal projection matrix P by applying SVD Y. retaining only the K right singular vectors corresponding to the largest sin-
LMSE + Y Y 11
7: La, := a Lortho.y + + (1-a)- LMSE.
gular values (steps 2-3). Subsequently, both Y and Y are projected into the latent component space (step 4). In this space, significance is strictly ordered: the first column captures the greatest significance (variance), which successively diminishes across subsequent columns.
Afterwards, we compute the transformed loss to align Y and Y in the transformed space (step 5);
Lortho,y I - 1z
(4)
where y controls the ratio of components retained, such that K = round(y T); ||-11, computes the sum of clement-wist wise absolute differences. Notably, we use the & norm instead of the squared norm following [, since different latent components vary greatly in scale (see Fig. 1), which makes the squared norm unstable. The & norm provides a more stable and robust optimization landscape.
Finally, we fuse the two loss functions, with 0 S S controlling the relative contribution (step 7):
Lary forthar + (1 ax) LMSE
(5)
By transforming both forecasts and labels into decorrelated components, Time-ol effectively duces autocorrelation bias. By focusing exclusively on the most significant components, Time-ol reduces optimization difficulty with minimal information loss. Time-ol is model-agnostic, offering practitioners the flexibility to employ the most suitable forecast model for each specific scenario.
Experiments
To demonstrate the efficacy of Time-ol, there are six aspects empirically investigated:
Does Time-ol work? We compare Time-ol with state-of-the-art baselines us- ing public datasets on long-term forecasting in Section 4.2 and short-term forecasting tasks in Appendix E. Moreover, we compare Time-ol with other loss functions in Section 4.3.
Performance:
2. Gain:
How does it work? Section 4.4 offers ablative study dissect the contributions of the individual factors of Time-ol, elucidating their roles in enhancing forecast accuracy.
Generality: Does it support other forecast models? Section 4.5 verifies the adaptability of Time-ol across different forecast models, with additional results in Appendix E.4.
Flexibility: Does it support alternative transformations? Section 4.5 also investigates generating latent components with other transformations to showease flexibility of implementation.
5. Sensitivity: Does it require careful fine-tuning? Section 4.6 presents sensitivity analysis of the hyperparameter a, where Time-ol maintains efficacy across broad range of parameter values.
6. Efficiency: Is it efficient? Section D shows the running time of Time-ol ol in diverse settings.
5

---

Table I: Long-term forecasting performance.
Time-ll
Prodiormer
MTandoomer (2004)
FieTS (2003)
TimrsNin
MICN
TIDE
DI ininerr
PRiiirmer (2002)
Autofomer
Tranddemmr
Models
(2024)
(2023)
(2003)
(2023)
(2003)
(2011)
Mettle
MSE
MSE
MSE
MSE
MSE
MSE
MSE
MSE
MSE
MSE
MSE
6T1ml
0.380
0.393
0.398
0,41
0.414
0/414
0:421
0,438
0.000
0,196
0.421
0,413
0.007
0,403
0.00
0.442
0457
0.525
0.991
0.999
0/648
ETTm
0,372
0,317
0,180
0.134
0.295
0.336
6,316
0.363
0.302
0.334
0.308
0.364
0,786
0.328
0.342
0.990
0.308
0.354
0.315
0.358
1,600
0917
ETThl
0.431
0.429
0.447
0.434
0,422
0.448
0,499
0,474
0.472
0.463
0,533
0.519
0,488
0.435
0446
0.453
0.47
0.470
0.477
0.483
6.83
0.774
0.359
0.377
0,385
0.524
0.409
0,620
0,378
0.329
0,452
0,448
2,088
0.122
0.300
0,199
0.212
0.1M2
0.215
0.292
0:112
0.214
0.289
02266
0.419
0.486
0426
0.598
0.390
0.691
0.530
0.634
0.373
0,655
0.640
0.398
0.662
00:16
0.692
Washer 0.241 0.280 0,261
0.282
0,269
0.289
0.249
0.203
0.271
0.295
0.264
0.321
0,372
0.291
0.265
0.317
0.326
0.372
0.319
0.363
0.699 0.601
PEMSOS 0,097 0.208 0,146
0.260
0,122 0.253
0.140
0.261
0.126
0.330 0.105
0.223
0,316
0.370
0:216
0.322 0.152 0.275 0.11
0.475 0.122 0.226
PEMSOS 0.141 0.171 0.142
0.174
0.152 0.153 0,318
0.349
0.226
0.422
0.340
Note: We fix the input length 96 following [28]. Bold and underlined denote best and second best results, respectively. indicates
average results over forecast borizons: T=96, 192, 336 and 720. Time-ol employs the top-perforning baseline on cach dataset as its underlying model.
4.1 Setup
Datasets. In this work, we conduct experiments on ETT (4 subsets), ECL, Traffic, Weather, and PEMS [28] for long-term forecasting task, and M4 for short-term forecasting task [58]. All datasets are split chronologically into training, validation, and testing sets following their official settings.
Baselines. We compare Time-ol with range of established models, including Transformer [45], Autoformer [59], FEDformer [71], iTransformer [28], Fredformer [37], DLinear [67], TiDE [5], FreTS [64], TimesNet [58], and MICN [54]. As a loss function, Time-ol is model-agnostic and can be integrated with any model architecture. By default, Time-ol employs the hest-performing baseline model on each dataset as its underlying model architecture for fair comparison.
The baseline models are reproduced using the scripts provided by Fredformer [37]. All baseline models are trained using the Adam [14] optimizer to minimize LSE in (1). Following the prestigious benchmark [38], the dropping-lan trick is disabled during the test phase. When integrating Time-ol to enhance an established model, we adhere to the associated hyperparameter settings in the public benchmark [37, 28], only tuning a, y and learning rate conservatively. Experiments conducted on Intel(R) Xeon(R) Platinum 8383C CPUs and NVIDIA RTX H100 GPUs.
Implementation.
4.2
Overall performance
Table presents the long-term forecasting results. Time-ol consistently improves base model performance. For example, on ETTh1, it reduces Fredformer's MSE by 0.016. Similar gains across other datasets further validate its effectiveness. These results suggest that modifying the loss function can yicld improvements comparable to, or even exceeding, those from architectural advancements. We attribute this to two key aspects of Time-ol: its decorrelation property for eliminating autocorrelation bias and its discrimination on significant components for simplifying optimization.
Showcases. We visualize the forecast sequences and the generated components to showcase the improvements of Time-ol forecast quality. A snapshot on ETTm2 with historical window H 96 and forecast horizon T = 336 is depicted in Fig. 2. Although the model trained using canonical DF captures general trends, its forecast struggles with large variations (e-g., peaks within steps 100-400). This reflects its difficulty in modeling significant, high-variance components. In contrast, Time-ol, by explicitly discriminating and aligning these significant components, generates forecast that accurately captures these large variations, including the peaks within steps 100-400.

---

Groundfruth Predictio
GrosndETuuh Predictonn
-05
-6
-0.5
-5
-10
GroundTraul Prediction
GroundTruth
-20
-15
Predetion
(a) The generated forecast with DF.
(b) The generated forecast with Time-ol ol.
Figure 2: The visualization of label and forecast sequences generated by models trained with TMSE versus Time-ol. In both (a) and (b), the left panels display the time-domain sequences (Y and Y), while the right panels illustrate their corresponding latent components (Z and Z).
Table 2: Comparable results with other loss functions for time-series forecast.
Loss
Time-ol
FreDF
Koopıan
Dilate
Soft-DTW DTW
DPTA
DF
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTml
0.379
0.393
0.384
0.394
0.389
0.400
0.389
0.400
0.397
0.402
0.396
0.402
0.387
0.398
Peefommer
ETTh1
0.431
0.429
0.438
0.434
0.452
0.443
0.453
0.442
0.460
0.449
0.460
0.449
0.447
0.434
ECL
0.178
0.270
0.282
0.187
0
0.206
0.298
0.202
0.294
0.284
Weather
0.258
iTrandorne
ETiml
0.395
0.401
0,405
0.405
0.413
0.416
0.407
0.412
0.417
0.415
0.416
0.415
0,411
0,414
FTTh1 0.438
0.434
0.442
0.437
0.455
0.451
0.452
0.448
0,470
0.457
0.463
0.454
0.452
0.448
ECL 0.170
0.260
0.176
0.264
0.178
0.269
0.178
0.269
0.175
0.266
0.177
0.267
0.179
0.270
Weather
0.286
0.292
0.291
Note: Bold and underlined denote best and second hest resalts, respectively. The reported results are iveruged over forecast horizons: T=96, 192, 336 and 720.
4.3 Loss function comparison
Table 2 compares Time-ol against other time-series loss functions: FreDF [51], Koopman [17], Di- late [18), Soft-DTW [4], and DPTA [42]. We integrate their official implementations into Fredformer and lTransformer. Overall, shape alignment losses (Dilate, Soft-DTW, DPTA) offer little performance gain over canonical DF (using TMSE loss), consistent with the findings in [18]. This phenomenon is rationalized by the fact that they do not mitigate the label autocorrelation nor reduce task amounts for simplifying optimization. FreDF improves performance by partly addressing autocorrelation bias. However, as discussed in Section 3. FreDF does not fully eliminate this bias, nor does it distinguish significant components to simplify the optimization landscape. Time-ol directly addresses these two limitations of FreDF, leading to its superior overall performance.
4.4
Ablation studies
Table 3 presents ablation study dissecting the contributions of critical factors in Time-ol: the decorrelation property and the task reduction effect. The main findings are summarized as follows.
Time-olt improves DF by reducing the number of tasks to optimize. To this end, it employs randomized matrix as the projection matrix to generate components and aligns only a subset of the obtained components. The involution ratio is finetuned on the validation set. It consistently improves over DF (c.g.. -0.012 MAE on Weather). This demonstrates that reducing tasks with minimal loss of label information can reduce optimization difficulty and improve performance.
Time-ol improves DF by aligning decorrelated components. To this end, the loss function is calculated in (5) with = 1. It also outperforms DF, achieving the second-best results overall. This demonstrates aligning decorrelated label components to mitigate bias benefits forecast performance.
Time-ol integrates both factors above by aligning the most significant decorrelated components and achieves the best overall performance, demonstrating the synergistic effect of these two factors.
7

---

Table 3: Ablation study results.
Model
Decurreiation
Reduction
Data
T-336
Ang
MSR
MAF
MSE
MAE
MSE
MAE
ETTml
0.326
0.361
0.365
0.382
0.395
0.404
0.459
0.444
0.387
ETIaI
ECL
0.150
0.242
0.158
0.259
0.182
0.274
0.218
0.304
0.179
0.270
Weather
0,174
00228
0.213
0.266
0,270
0.316
0.337
0,302
0.249
0.253
BTTml
0.338
0.366
0.369
0.383
0.397
0.403
0.458
0.441
0.391
0.308
ol
X
ECL
ETTal
0.376
0.395
0447
0.430
0.478
0.450
0.469
0.296
0.467
0,460
0.436
0.264
Waather
0.170
0.216
0.259
0.362
0.300
0.312
0.351
0.244
0:151
0.438
0.382
Timesot
ETTAI
0.373
0.395
0.455
0.23
0.766
0.455
0.676
0.465
0,489
0431
BCL
0.147
0.2
1.z
0.25
0.174
0267
0.205
0.29
0.172
0.20
Waather
0.172
0.220
0.211
0.259
0.353
0.344
ETTm]
0.321
0.357
0.360
0.378
0.389
0.400
0.447
0.435
0.379
0.393
0.368
Time-ol
ECL
0.145
0.235
0.159
0.249
0.173
0.264
0.203
0.292
0.170
0.260
Weather
0.169
0.219
0.210
0.258
0.239
0.297
0.327
0.349
0.241
0.250
Note: Bold and undertined denote best and accond-best results, respectively.
results,
Table 4: Varying transformations results.
ECL.
Weather
Transformation
MSE
MAE
MSE
MAE
None
0.179
0.270
0.249
0.293
RPCA
0,171
4.31% L
0.261
3.16%
0.244
1.78%
0.286
2.38%
SVD
0.175
2.24% 1
0,93%
FA
0.175
2.35% 1
0.265
1.82%
0.245
1.35%
0.287
1.97%
Ours
0.170
4.86% L
0.260
3.57%
0.241
2.94%
0,280
4,28%
Note: relers the relative reduction compared to the buseline (None). Bold and underlined denote best and second best results.
4.5
Generalization studies
In this section, we investigate the utility of Time-ol with different transformation strategies and forecast models, to showcase the generality of Time-ol. In the bar- bar-plots, the forecast errors are averaged over forecast lengths (96, 192, 336, 720), with error bars 50% confidence intervals.
Varying transformations. We select alternative approaches to transform the label sequence into latent components and report the forecast performance in Table 4. The selected transformation
methods include robust principal component analysis (RPCA) [1], SVD [7], and factor analysis [15]. Noting that the output of SVD yields components here, not projection matrix in Section 3.2. Implementation details are in Appendix C. Overall, all these transformation methods outperform canonical DF without transformation. However, the components obtained by these methods, including RPCA, cannot be guaranteed to be decorrelated. Consequently, autocorrelation bias may persist. In contrast, our approach ensures full decorrelation of the derived components (see Lemma 3.2), effectively addressing autocorrelation bias and leading to the best overall performance.
Varying forecast models. We explore the versatility of Time-ol in augmenting representative forecast models: Fredformer [37], iTransformer [28], FreTS [64], and DLinear [67]. As illustrated in Fig. 3, Time-ol improves forecast performance in all cases. For instance, on the Weather dataset, iTransformer and FreTS with Time-ol achieve substantial reductions in MSE-un up to 6.9% and 2.9%, respectively. Further evidence of Time-ol's 's versatility can be found in Appendix E.4. These results confirm Time-ol 's potential plug-and song-and-plyy -lay strategy to enhance diverse forecast models.
4.6 Hyperparameter sensitivity
In this section, we examine the impact of critical hyperparameters on the performance of Time-ol The results are presented in Table 5 and Table 6. Additional trends across different datasets and forecast lengths are provided in Appendix E.5. The primary observations are summarized as follows:
The coefficient determines the relative importance of the proposed transform-enhancel loss in (5). When a is set to 1, Time-ol exclusively uses the transform-enhand loss. Overall, increasing from 0 to I leads to improved forecast accuracy, with the best results typically achieved when

---

Tine-ot
-1.5%
0:32
Time
223%%
0.33
Tireoot
0.34
4.99%
0.30
0:90
-2.9%
0:37
-2.1%
4.3%
0.20
0.25
6.8
0.18
(a) ECL with MSE
(b) ECL. with MAE
(c) Weather with MSF
(d) Weather with MAE
Figure 3: Improvement of Time-ol applied to different forecast models, shown with colored bars for
means over forecast lengths (96, 192, 336, 720) and error bars for 50% confidence intervals.
Table 5: Hyperparameter results on (t.
Table 6: Hyperparameter results on 7.
ETTml
ETTh2
Weather
ETTml
ETTh2
Weather
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.3867
0.3979
0.3766
0.4019
0.2486
0.2930
0.1
0.3915
0.4002
0.3816
0.4029
0.2437
0.2845
0,3871
0.3983
0,3742
0.3982
0.2439
0.2851
0.3
0,3849
0.3964
0.3694
0.3961
0.2424
0.2825
0.3864
0.3976
0.3964
0.2432
0.2833
0.3943
0.3651
0.3923
0.2466
0.2877
0.3831
0.3959
0.3674
0.3943
0.2433
0,2849
0.7
0.3798
0.3930
0,3603
0.3886
0.2443
0.2861
0.3850
0.3933
0.3606
0,3890
0,2753
0.3209
0.3814
0.3940
0.3624
0.3903
0.2491
0.2924
Note: Bold and underlined denote hest and second-best results.
Note: Bold and underlined denote hest and second-hest results.
a is close to 1. The performance improvement is significant, e.g., MSE reduction on ETTh2 by 0.016, showcasing the utility of the
loss to improve forecast performance.
The coefficient 7 determines the ratio of retained components for training. The results demonstrate that preserving all label information (y = 1) does not necessarily yield optimal performance. Instead, the best results are obtained at < 1, such as 0.7 on ETTs and 0.3 on Weather. This phe- nomenon can be attributed to the trade-off between information loss and optimization complexity: by focusing on the top components, optimization simplified due reduced task amount. Besides, since the top components contain most information, the information loss by dropping the other components negligible. These factors synergistically contribute improved performance.
Conclusion
In this study, we highlight the importance of designing loss functions for training time-series fore- casting models. Two critical challenges are formulated: label autocorrelation, which induces bias, and number of tasks, which determines optimization complexity. To address these challenges, we introduce a model-agnostic loss function called Time-ol. This method transforms the label sequence into decorrelated components with discernible significance. Then, it trains forecast models to align the most significant components, which simultaneously mitigates label autocorrelation and reduces task amount. Empirically, Time-ol improves forecast performance across diverse datasets.
Limitations & future works. In this work, we investigate the challenges of label autocorrelation and excessive number of tasks in time-series forecasting. Nevertheless, these issues also manifest in areas such speech generation, target recognition, and dense image prediction. Applying Time-ol in these contexts is a promising avenue for future research. Additionally, history sequence also exhibits autocorrelation and contains redundancy. Transforming inputs to derive decorrelated, compact representations could offer additional performance gains and also warrants investigation.
Acknowledgments
Z. Lin was supported by the NSF China (No. 62276004) and the State Key Laboratory of General Artificial Intelligence. H. Li was supported by National Natural Science Foundation of China (623B2002).

---

References
[1] HanQin Cai, Jian-Feng Cai, and Ke Wei. Accelerated alternating projections for robust principal component analysis. Journal of Machine Learning Research, 20(20):1-33, 2019.
[2] Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Tong, et Spectral temporal graph neural network for multivariate time-series forecasting. In Proc. Adv. Neural Inf. Process. Syst. volume 33, pages 17766-17778 2020.
[3] Zhichao Chen, Leilei Ding, Zhixuan Chu, Yucheng Qi, Jianmin Huang, and Hao Wang. Monotonic neural ordinary differential equation: Time-series forecasting for cumulative data. In Proc. ACM Int. Conf. Inf. Knowl. Manag,, pages 4523-4529, 2023.
[4] Marco Cuturi and Mathicu Blondel. Soft-dtw: a differentiable loss function for time-series. In Proc. Int. Conf. Mach. Learn., pages 894-903. PMIR, 2017.
[5] Abhimanyu Das, Weihao Kong, Andrew Leach, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time- dense encoder. Trans. Mach. Learn. Res., 2023.
[6] Urška Demšar, Paul Harris, Chris Brunsdon, A Stewart Fotheringham, and Sean Mcl.oone. Principal component analysis on spatial data: overview. Ann. Assoc. Am. Geogr. (03(1))100-12))) 2013.
[7] Andrew Gibiansky. Cool linear algebra: Singular value decomposition. Andrew Gibiansky Blog, 29, 2013.
[8] Michael Greenacre, Patrick JF Groenen, Trevor Hastie. Alfoaso lodice d' E'Eza. Angelos Markos, and Elena Tuzhilina, Principal component analysis. Nat. Rev. Methods Primers, 2(1):100, 2022.
[9] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In Proc. Conf. Lang. Model., 2023.
[10] Albert Gu, Karan Goel, and Christopher Re, Efficiently modeling long sequences with structured state spaces. Proc. Int. Conf. Learn Represent, 2021.
[11] Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shouhong Ding, Zhengyang Zhou, and Yang Wang. Hdmixer: Hierarchical dependency with extendable patch for multivariate time series forecasting. In Proc. AAAI Conf. Artif. Intell., volume 38, pages 16668-12666,,111 2024.
[12] Qihe Huang, Zhengyang Zhou,, Yangze Li, Kuo Yang, Binwu Wang, and Yang Wang. Many minds, one goal: Time series forecasting via sub-task specialization and inter-agent cooperation. In Proc. Adv. Neural Inf. Process. Syst. 2025.
[13] Qihe Huang, Zhengyang Zhou, Kuo Yang, Zhongchao Yi, Xu Wang, and Yang Wang. Timebase: The power of minimalism in efficient long-term time series forecasting. In Proc. Ins. Conf. Mach. Learn. 2025.
[14] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proc. Int. Conf. Learn. Represent. pages 1 9, 2015.
[15] Paul Kline. An guide to fuctor analysis. Routledge, 2014.
[16] Yaxuan Kong, Zepu Wang, Yuqi Nie, Tian Zhou, Stefan Zohren, Yuxuan Liang, Peng Sun, and Qingsong Wen. Unlocking the power of Istm for long term time series forecasting. In Proc. AAAI Conf. Artif. Intell., volume 39, pages 11 11968-11976. 2025.
[17] Henning Lange, Steven L Brunton, and J Nathan Kutz. From fourier koopman: Spectral methods for long-term time series prediction. Journal of Machine Learning Research, 22(41):1-335 2021.
[18] Vincent Le Guen and Nicolas Thome. Shape and time distortion loss for training deep time series forecasting models. In Proc. Adv. Neural Inf. Process. Syst., volume 32, 2019.
[19] Vincent Le Guen and Nicolas Thome. Probabilistic time series forecasting with shape and temporal diversity. Proc. Adv. Neural Inf. Process. Syst., 33:4427-4440, 2020.
[20] Chen Li, Hongyang Zhang, Saqlain Abbas, Chenyu Ma, Yinhao Liu, and Xiaotong Tu. Panda: Patch-aware graph network with dual alignment for time series forecasting. In Proc. IEEE Int. Conf. Acoust, Speech Signal Process., pages 1-5, 2025.
[21] Haoxuan Li, Chunyuan Zheng, Shuyi Wang. Kunhan Wu, Eric Wang. Peng Wu, Zhi Geng. Xu Chen, and Xiao Xia Zhou. Relaxing the accurate imputation assumption in doubly robust learning for debiased collaborative filtering. In Proc. Int. Conf. Mach. Learn., volume 235, pages 29448 29460, 2024.
10

---

[22] Jianxin Li, Xiong Hui, and Wancai Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proc. AAAI Conf. Artif. Intell., 2021.
[23] Xinyu Li, Yuchen Luo, Hao Wang, Haoxuan Li, Liuhua Peng. Feng Liu, Yandong Guo, Kun Zhang, and Mingming Gong. Towards accurate time series forecasting via implicit decoding. Proc. Adv. Neural Inf. Process. Syst. 2025.
[24] Zhe Li, Xiangfci Qiu, Peng Chen, Yihang Wang, Hanyin Cheng, Yang Shu, Jilin Hu, Chenjuan Guo, Aoying Zhou, Christian S Jensen, et al. Tsfmm-bench A comprehensive and unified benchmark of foundation models for time series forecasting. In Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, pages 5595-5606, 2025.
[25] Zhe Li, Xiangfei Qiu, Peng Chen, Yihang Wang. Hanyin Cheng, Yang Shu, Jilin Hu, Chenjuan Guo, Aoying Zhou, Qingsong Wen, et al. Foundts: Comprehensive and unified benchmarking of foundation models for time series forecasting. arXiv preprint arXiv:2410. 11800.0.1110...... 2024.
[26] Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Proc. Ady. Neural Inf. Process. Syst. 18:18888-18899 2021.
[27] Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: time series modeling and forecasting with sample convolution and interaction. In Proc. Adv. Neural Inf. Process. Syst. 2022.
[28] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itrans- former: Inverted transformers are effective for time series forecasting. In Pruc. Int. Conf. Learn. Represent.. 2024.
[29] Jiaming Ma, Zhiqing Cui, Binwu Wang, Pengkun Wang, Zhengyang Zhou, Zhe Zhao, and Yang Wang. Causal learning meet covariates: Empowering lightweight and effective nationwide quality forecasting. Proc. Int. Joint Conf. Artif. Intel., 2025.
[30] Jiaming Ma, Binwu Wang, Qihe Huang, Guanjun Wang, Pengkun Wang, Zhengyang Zhou, and Yang Wang. Mofo: Empowering long-term time series forecasting with periodic pattern modeling. In Proc. Adv. Neural Inf. Process. Syst., 2025.
[31] Jiaming Ma, Binwu Wang, Guanjun Wang. Kuo Yang, Zhengyang Zhou, Pengkun Wang. Xu Wang. and Yang Wang. Less but more: Linear adaptive graph learning empowering spatiotemporal forecasting. In Proc. Adv. Neural Inf. Process. Syst. 2025.
[32] Jiaming Ma, Binwu Wang, Pengkun Wang, Zhengyang Zhou, Xu Wang, and Yang Wang. Bist: A lightweight and efficient bi-directional model for spatiotemporal prediction. Proc. VLDB Endow. 1663-16666666 2025.
[33] Jiaming Ma, Binwu Wang. Pengkun Wang, Zhengyang Zhou, Yudong Zhang, Xu Wang. and Yang Wang- Mohimixer: A multi-scale spatiotemporal mixing model for mobile traffic prediction. IEFE Trans. Mob. Comput., 2025.
[34] Gonzalo Matcos, Santiago Segarra, Antonio G. Marques, and Alejandro Ribciro. Connecting the dots: Identifying network structure via graph signal processing. IEEE Signal Process. Mag., 36(3):16-43 2019.
[35] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In Proc. Int. Conf. Learn. Represent., 2023.
[36] Karl Pearson. Principal components analysis. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 6(2):559, 1901.
[37] Xihao Piao, Zheng Chen, Taichi Murayama, Yasuko Matsubara, and Yasushi Sakurai. Fredformer: Frequency debiased transformer for time series forecasting. In Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, pages 2400-2410, 2024.
[38] Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tib: Towards comprehensive and fair benchmarking of time series forecasting methods. In Proc. VLDB Endow. pages 2363-2377, 2024.
[39] Xiangfei Qiu, Xiuwen Li. Ruiyang Pang. Zhicheng Pan, Xingjian Wu, Liu Yang, Jilin Hu, Yang Shu, Xuesong Lu, Chengcheng Yang. Chenjuan Guo, Aoying Zhou, Christian S. Jensen, and Bin Yang. Easytime: Time series forecasting made easy. In Proc. IEEE Int. Conf. Data Eng. 2025.
11

---

[40] Xiangfei Qiu. Xingjian Wu, Hanyin Cheng, Xvyuan Liu, Chenjuan Guo, Jilin Hu, and Bin Yang. Dbloss: Decomposition-baser loss function for time series forecasting. Proc. Adv. Neural Inf. Process. Syst., 2025.
[41] Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting. In Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, pages 1185-1199 199, 2025.
[42] Hiroaki Sakoe and Seibi Chiba. Dynamic programming algorithm optimization for spoken word recognition. ICEE Trans. Signal Process. 26(1):43-49, 2003.
[43] Gilbert Strang. Introduction to linear algebra. SIAM, 2016.
[44] William Toner and Luke Nicholas Darlow. An analysis of lincar time series forecasting models, In Proc. Inf. Conf. Mach. Learn., pages 18404-48427, PMLR, 2024.
[45] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proc. Adv. Neural Inf. Process. Syst., 2017.
[46] Hao Wang, Zhichao Chen, Jiajun Fan, Haoxuan Li, Tianqiao Liu, Weiming Liu, Quanyu Dai, Yichao Wang. Zhenhua Dong, and Ruiming Tang. Optimal transport for treatment effect estimation. In Proc. Adv. Neural Inf. Process. Syst., volume 36, pages 5404 5418, 2023.
[47] Hao Wang, Zhichao Chen, Zhaoran Liu, Xu Chen, Haoxuan Li, and Zhouchen Lin. Proximity matters: Local proximity enhanced halancing for treatment effect estimation. Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2025.
[48] Hao Wang. Zhichao Chen, Zhaoran Liu, Licheng Pan, Hu Xu, Yilin Liao, Haozhe Li, and Xinggao Liu. Spot-i: Similarity preserved optimal transport for industrial iot data imputation. IEEE Trans, Ind. Informat.. 20(12):14421 14429, 2024.
[49] Hao Wang, Zhichao Chen, Yuan Shen, Hui Zheng, Degui Yang. Dangjun Zhao, and Buge Liang. Unbiased recommender learning from implicit feedback via weakly supervised learning. In IEEE Trans. Neural Netw. Learn. Syst., 2025.
[50] Hao Wang, Xinggao Liu, Zhaoran Liu, Haozhe Li, Yilin Liao, Yuxin Huang, and Zhichao Chen. Lspt-d: Local similarity preserved transport for direct industrial data imputation. IEEE Trans. Autom, Sci. Eng.. 22:9438-9444
2025.
[51] Hao Wang, Licheng Pan, Yuan Shen, Zhichao Chen, Degui Yang. Yifei Yang, Sen Zhang, Xinggao Liu, Haoxuan Li, and Dacheng Tao. Fredf: Learning to forecast in the frequency domain. In Proc. Int. Conf. Learn. Represent., pages 1-9, 2025.
[52] Hao Wang. Zhiyu Wang, Yunlong Niu, Zhaoran Liu, Haozhe Li. Yilin Liao, Yuxin Huang, and Xinggao Liu. An accurate and interpretable framework for trustworthy process monitoring. IEFF Trans. Artif. Intell, 5(5):2241 2023.
[53] Haotian Wang, Haoxuan Li, Hao Zou, Haoang Chi, Long Lan, Wanrong Huang, and Wenjing Yang. Effective and efficient time-varying counterfactual prediction with state-space models. In Proc. Int. Conf. Learn. Represent., 2025.
[54] Huiqiang Wang, Jian Peng, Feihu Huang. Jince Wang, Junhui Chen, and Yifei Xiao. Micn: Multi-scale local and global context modeling for long-term series forecasting, In Proc. Int, Conf. Learn. Represent., 2023.
[55] Pengeheng Wang, Xiang Ma, Lizhen Cui, and Xuemei Li. Tseg: Efficient grouped channel interaction for multivariate time series forecasting. Expert Syst. Appl, page 130299, 2025.
[56] Yuxuan Wang. Haixu Wu, Jiaxiang Dong, Yong Liu, Mingsheng Long. and Jianmin Wang. Deep time scries models: A comprehensive survey and benchmark. arXiv preprint arXiv: 13278, 2024.
[57] Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transform- ers in time series: a survey. In Proc. Int. Joint Conf. Artif. Intell., pages 6778-6786, 2023.
[58] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang. and Mingsheng Long. Timesnet: Temporal 2d- variatia modeling for general time series analysis. In Proc. Inf. Conf. Learn. Represent. 2023.
20233.
[59] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with Auto- Cotrelaottl for long- -erm series forecasting, In Proc. Adv, Neural Inf. Process. Syst., 2021.
12

---

[60] Xingjian Wu, Zhengyu Li, Hanyin Cheng, Xiangfei Qiu, Jilin Hu, Chenjuan Guo, and Bin Yang. Unlocking the power of mixture-of-experts for task-aware time series analytics. arXiv preprint rrXiv:250922279, 2025.
[61] Xingjian Wu, Xiangfei Qiu, Hongfan Gao, Jilin Hu, Bin Yang. and Chenjuan Guo. K2vae: A koopman- kalman enhanced variational autoencoder for probabilistic time series forecasting. Proc. Int. Conf. Mach. Learn., 2025.
[62] Xinle Wu, Xingjian Wu, Dalin Zhang, Miao Zhang, Chenjuan Guo, Bin Yang, and Christian Jensen. Fully automated correlated time series forecasting in minutes. Proc. VLDB Endow. 18(2): 144-154, 2024.
[63] Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. Proc. Int. Conf. Learn. Represent., 2024.
[64] Kun Yi, Qi Zhang, Wei Fan. Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domail mips are more effective leamerrs in time series forecasting. In Proc. Adv. Neural Inf. Process. Syst., 2023.
[65] Wenzhen Yue, Yong Liu, Haoxuan Li, Hao Wang. Xianghua Ying, Ruohao Guo, Bowei Xing, and Shi. Olinear: A linear model for time series forecasting in orthogonally transformed domain. Proc. Adv. Neural Inf. Process. Syst., 2025.
[66] Wenzhen Yue, Yong Liu, Xianghua Ying, Bowei Xing, Ruohao Guo, and Ji Shi. Freeforner: Frequency enhanced transformer for multivariate time series forccasting. In Proc. Int. Joint Conf. Artif. Intell., pages 1606-361,, 2025.
[67] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proc. AAAI Conf. Artif. Intell., 2023.
[68] Weijia Zhang, Le Zhang. Jindong Han, Hao Liu, Yanjie Fu, Jingbo Zhou, Yu Mci, and Hui Xiong. Irregular traffic time series forecasting based on asynchronous peapooltemporal graph convolutional networks. In Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, pages 4302 4313, 2024.
[69] Zhang and Qiang Yang. A survey on multi-lask learning, IEEE Trans. Knowl, Data Eng. 34(12):5586- 5609, 2021.
[70] Yungeng Zhang, Yuan Chang, Xiaohou Shi, Yaqi Song, Feng Wang, and Mingehuan Yang. Adapting large language models to forecast in frequency domain. In Proc. IEEE Int. Conf. Acoust. Speech Signal Process., pages 1-5, 2025.
[71] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-ten term series forecasting, In Proc. Int. Conf. Mach. Learn., 2022.
13

---

A On the Implementation Details of Label Correlation Estimation
In this section, introduce the motivation and implementation details of the label autocorrelation estimation techniques in Fig. 1. Measuring label autocorrelation y/2 Yer is indeed challenging due the presence of confounding effect [47, 46, 21J. Specifically, the fork structure Yr +- + ye introduces spurious correlations between and Yr', thereby distorting the true strength of the label autocorrelation y/ ye of interest. This structural confounding undermines the validity of traditional measures such as Pearson correlation for quantifying label autocorrelation.
The previous work [51] employs the double machine learning (DML) method to estimate the ground- truth correlation while mitigating the influence of the fork structure. We adopt this in our experiments. DML is statistical technique designed estimate the causal effect of treatment while controlling for fork variables. Specifically, suppose we have a treatment variable T, an outcome variable y, and set of fork variables X. The goal is estimate the causal effect of T y while controlling for the influence of X. To this end, DML first orthogonalizes both the treatment and outcome with respect the fork variables. Two parametric models are employed to predict the treatment and outcome based on the fork variables. These predictions capture the impact of X on y and T. Subsequently, such impact of X' is eliminated by calculating the residuals. Finally, the DML method regresses the outcome residuals on the treatment residuals, thereby measuring the causal effect of 7 on y while removing the influence the fork variables.
outcome
In our experiments, we measure label autocorrelation by treating the input sequence x as the fork variable and different steps of the label sequences Y/ and Ye the treatment and outcome variables, respectively. Then, we estimate the treatment effect of yt on measuring the correlation between different components, we use different components 23 and 2 the treatment and outcome variables. Linear regression model is employed as the parametric model for both the treatment and outcome variables for efficiency, which is consistent with [51].
controlling x. Similarly, when
1.00
1.00
100
1.00
&
0.75 8-
9.75
8
0.75
31
0.75
0.00
0.50
0.00
0.50
0.00
(a) Correlation between different steps in label sequence.
2-
35
8
9
0.00
0.00
0.2
0.25
8
-0.00
-000
0.00
0.000
(b) Correlation between different extracted components.
Figure 4: The label autocorrelation in the original label sequence and the extracted components. The datasets are ETTh1, ETTh2, ETTml, and Weather from left to right. The forecast length is set to 96.
To further complement the case study in Fig. 1, we analyzed the correlation matrices of the label sequences and extracted components across multiple datasets, with the results presented Fig. The main observations summarized as follows.
Panel (a) displays the correlation matrix of the label sequence, characterized by substantial non- diagonal elements, which highlight the strong autocorrelation among the labels. In contrast, panel (b) shows the correlation matrix of the extracted components, where the non-diagonal elements are nearly zero, indicating effective decorrelation.
Compared the results reported in [51] [51], where some obtained components remain correlated, the non-diagonal elements in panel (b) are fully eliminated. This difference arises because the Fourier transform in [51] achieves decorrelation only when the original label sequence is nearly infinitely long (T oc), condition that is not met in real-world applications with finite forecast horizons.

---

This limitation stems from the predefined nature of the projection matrix, which lacks adaptation the specific properties of the data. In contrast, our method ensures decorrelation by solving constrained optimization problem, without relying on infinitely long forecast horizon, thereby providing a more reliable approach for handling autocorrelation bias.
B Theoretical Justification
Theorem B.1 (Autocorrelation bias, Theorem 3.1 in the main text). Given univariate label sequence y RT where Σ E RTxx denotes the step-wise correlation coefficient, the loss function CMSE in (1) is biased from the true negative log-likelihood of the label sequence, which is given by:
Bias lly V1E llyly- -1 -1y
log E
where [|v|2 - The bias vanishes different steps in y decorrelated."
Proof. The proof follows our previous work [51] but relaxes the first-order Markov assumption.
Suppose the label sequence follows multivariate normal distribution with mean vector y and covariance matrix Σ, where off-diagonal entries are Eij = Pejo² for ن j. Here, Pty denotes the partial correlation between y/ and yj- The log-likelihood of the label sequence Y can be expressed as:
1 2
log ply)
(2T) + log Σ + (y y) 9))
Removing the constant terms unrelated to y, obtain the practical negative log-likeldhood (PNLL):
PNT.L (y - ÿ) Σ (y 8)- On the other hand, the TMSE loss can be expressed as:
TMSE = | - $12 = (y 8) T I (y 8).
where I is the identity matrix. The difference between TMSE and PNLL can be expressed as:
Bias PNLL TMSE (y y) >> 8) (y - I(y y)),
which immediately vanishes if the label sequence is decorrelated, i..., Σ I. The proof is completed.
Lemma B.2 (Lemma 3.3 in the main text). The projection matrix P can be obtained via singular value decomposition (SVD): Y = (PPP where U E NN and p E IgKx consist of singular vectors, and the diagonal of A IR Nxk consists of singular values.
Proof. The proof is available in Section 7.3 in [43].
Lemma B.3 (Decorrelated components, Lemma 3.2 in the main text). Suppose Y E R contains normalized label sequences for samples, Z = [Z,,..., zk] are the obtained components; for any k + R. have z Z O.
Proof. For any two latent components zy and Z with + W,, we have:
z Z (Ypz) (Ypw) P. Y Yp=
Noting that p& and Pk are eigenvectors of Y Y [43], we have
Y Y Ypp ARPE,
Y Ypw A Dr',
(8)
which immediately follows by zg Zz ANP. Pws Recalling that different projection bases constrained orthogonal, i.e., Pa P for k' have
2 z5 = 0 for all & + k,
which completes the proof.
'The pioneering work [51] identifies the bias under the first-order Markov assumption on the label sequence. This study generalizes this bias without the first-order Markov assumption.
15

---

C Generalized Orthogonalization and Decorrelation Methods
In this section, introduce alternative transforms for obtaining latent components, each with distinct characteristics such as dimensionality reduction and noise isolation. A comparatively empirical study provided in Section 4.5.
The robust principal component analysis decomposes the data into a low-rank informative component and a sparse noise component, effectively separating structured signals from noise. Specifically, given Y E RNx,, is achieved by solving:
RPCA.
min
|V||- + A||S||,, subject to V V
(10)
where II is the nuclear norm, || is the element-wise E norm, and À is a regularization parameter Afterwards, it performs the principal component analysis on the obtained informative component V to derive the projection matrix P. The latent components are generated by Z YP. While this approach enhances noise elimination, it does not guarantee decorrelation of the derived components, as the projection matrix P is derived from V instead of the original data matrix Y.
SVD. The singular value decomposition provides a method to decompose the matrix into different components. Given Y e RN we have:
=AVAV.
(11)
where E Nxr and V E RTxR are singular vectors, A E RIIX is diagonal with rank r. The right singular vector is used as the projection matrix, and the latent components are generated by Z = YV. One key distinction here needs to be highlighted. Unlike the workflow in the main text (Algorithm 1), the label sequence is not normalized after window generation before computing SVD here, resulting in non-decorrelated components.
FA. Factor analysis models the observed data as linear combinations of small number of latent factors plus noise, capturing the covariance structure through these unobserved factors. Specifically, given mean-centered Y E R RNT the model assumes:
(12) where V R is the factor loading matrix, K is the number of latent factors (K < FN. E R contains the latent factor scores for each sample, and e IR NxT is the noise matrix. The standard assumption is that each factor f; No0 I) and noise e, NO0 v where V/ is a diagonal covariance matrix. The loadings V and factor scores F are typically estimated via maximum likelihood. The latent components
Y VF E
given by the estimated factor scores, i.e., Z Y YV-'F + F V 1p-1p F) YP 6, This approach captures the covariance structure of Y via a small number of factors, but does not necessarily guarantee uncorrelated or noise-isolated components.
D Complexity Analysis
In this section, we analyze the running cost of Time-ol. The core computation of Time-ol involves (a) calculating the projection matrix P via SVD, and (b) performing transformation on both Y and Y followed by calculating their wisn MAE loss. Given the target matrix Y NxT, the SVD step decomposes Y with an established complexity of ONNT²) (assuming N Z T and K = T). For the sequence transformation, each sample (row) in Y is multiplied by the projection matrix P' E Txx resulting in a total complexity of ( The computation of point-wise MAE loss across all samples and forecast steps is O(TT), which is negligible compared to the complexity of previous steps. Thus, the overall complexity per batch is dominated by the SVD and projection operations, both scaling ONTT.). The main findings from the empirical evaluations are follows.
Fig. 5 (a) presents the computational cost for calculating the projection matrix. Overall, it increases linearly with the sample size and quadratically with the prediction length, which aligns with the theoretical complexity. Importantly, this operation is performed only once before training begins, rendering the associated overhead acceptable.
'Adapted from source code of skleamn https: // thub. www.c///wcc.ttt-oooonetttt-----e 98ed9dc73/s httr/3////kearr https//ae/eee/eoosssttao/ http://cccooooiiinn/cccc//
16

---

Fig. 5 (b) presents the computational cost for the sequence transformation. The cost increases quadratically with the prediction length, but remains below 2 ms. This cost comparable that of a linear projection. Furthermore, sequence transformation is not required during inference.
In conclusion, Time-ol does not add complexity to model inference, and the additional complexity during the training stage is negligible.
1.25
1.00
Time (1)))
Time
1.5
Time
1.5
1.0
030
1.3
0.5
737
5754
22131
2050
3986
192
390
356 Prediclion
720
&Samppes
Preciction Length
Prediction Length
(a) Running cost of SVD.
(b) Running cost of transformation.
Figure 5: Running for projection matrix calculation (left panel with varying number of samples, right panel with varying prediction length) and sequence transformation (left panel for forward pass, right panel for backward pass, with average and shaded areas for 95% % confidence intervals).
E
More Experimental Results
E.1
Long-term forecast performance
Additional results on long-term forecast performance are available in Table 7.
E.2
Short-term forecast performance
Additional results on short-term forecast performance are available in Table 8, where Fredformer [37] serves the forecast model.
E.3
Showeases
Additional results on showcases are available Fig. and Fig.
E.4
Generalization studies
Additional results varying forecast models and transformations are available in Fig. 8 and Table 9.
E.5
Hyperparameter sensitivity
Additional results on hyperparameter sensitivity are available in Fig. 9 for and Fig. 10 for 7.
E.6
Comparison with different loss functions
Additional results on comparing different loss functions are available in Table 10.
E.7
Varying input length results
Additional results on varying input lengths available in Table 11 complementing the fixed length of used in the main text.
E.8
Random seed sensitivity
Additional results on varying random seeds are available in Table 12.
17

---

Table 7: The comprehensive results on the long-term forecasting task.
Time-el
Frodiormer
TTaroforme
Fi
TineaNet (2023)
MICN
TIDE
DLineu
FEDiorma
Autafonner (2021)
Tranffrmer
Models
(Ours)
(2024)
(2024)
(2023)
(2023)
(2023)
(2023)
(2022)
(2017)
Metries
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.321
0.357
0.326
0.361
0.338
0.372
0.342
0,368
0.394
0,319
0.366
0.353
0.374
0.346
0.373
0.401
0.434
0.485
0.468
0.503
m
192
0.360
0.37%
0.365
0.362
0.382
0.396
0.385
0.400
0.406
0.429
0.764
0.395
0.391
0.393
0.380
0.390
0.415
0.446
0.594
0.482
0.907
0.664
196
0.889
0.400
0.396
0.404
0.427
0.436
0416
0.421
0.454
0.646
0.
0:425
0.433
0.414
0.413
0.414
0.412
0.450
0.500
0.489
0.847
0.678
720
0.435
0.406
0.463
0.513
0.527
0.505
0.448
0.450
0.500
0.523 0.594
1.037
0.771
Ang
0.379
0.99%
0.484
0:414
0.438
0.396
0.413
0.407
0:407
0.457
0.526
0.491
0,799
96
0.172
0.251
0.177
0.260
0.182
0.265
0.183
0.184
0.262
0.178
0.277
0.182
0.365
0.283
0.205
0.289
0.218
0,300
0.336
0.441
ET BTTm2
191
0.236
0.294
0:347
0.300
0.257
0.318
0:764
0.320
0.257
0.908
0.766
0.341
0.247
0.904
0.280
0.356
0.271
0.314
0:282
0.340
1.410
0.881
336
0.293
0.335
0.300
0.340
0.520
0.386
0.322
0.369
0.315
0.545
0.299
0.354
0.907
0.345
0.375
0.420
0.527
0.306
0.335
0.370
1.940
1.070
0.423
0.508
0.435
2.914
Ang
0.317
0.280
0.324
0.295
0,316
0.902
0.334
0,300
0.256
0.528
0,302
0.356
00315
0,358
1.662
0.917
96
0.368
0.391
0:323
0.995
0.585
0.405
0.399
0.408
0.999
0.418
0,381
0.416
0.387
0.985
0.980
0.404
0.901
0.433
[0449
0.465
1.078
0.778
BTTH
0.024
0.422
0.432
0.640
0.437
0.451
0.652
0.497
0.489
0.839
0/442
0.460
0.418
0.648
0.489
0.469
1.010
0.789
236
0.467
0.441
0.486
0.449
0.480
0.457
0.501
0.472
0.488
0.469
0.589
0.558
0.482
0.447
0.485
0.467
0.487
0.084
0,511
0.500
0.908
0.743
0.463
0.467
0.504
0.608
0.549
0.663
0.502
0.514
0.498
0.967
0.785
Ang
00.43
0.429
0.447
0.434
0.452
0.448
0.489
0,474
0.472
0.463
0,533
0.519
0.448
0.435
0:455
0.453
0.447
0.470
0,477
0,483
0.963
0.774
96
6.282
0.330
0.299
0.344
0.301
0.949
0.315
0.324
0.531
0.308
0,351
0.399
0.291
0.340
0.350
0.383
0.351
0.391
10355
0.397
1,685
0.959
E1Th2
192
0.359
0.381
0,391
0.385
0.397
0,466
0.418
0,492
0.489
0.376
0.992
0.450
0.456
0.456
0.475
0.471
4.218
1.585
336
0.394
0.414
0:420
0.433
0.425
0.43
0.522
0.502
0.464
0.454
0.656
0.582
0.417
0.427
0.589
0.538
0.477
0,492
0.459
0.469
2:775
1.361
730
0.792
0.626
0.905
2.274
Ang
0.359
0.388
0.372
0.402
0.386
0.407
0.524
0.496
0.409
0.420
0.620
0.546
0.378
0.401
0.499
0.432
0.461
004448
0.460
2.688
96
0.145
0.235
0,161
0.258
0.150
0.242
0.180
0.266
0.170
0.272
0,170
0,281
0.197
0.274
0.197
0.282
0.187
0.302
0.089
0.304
0.253
0.330
ECL
0.159
0.349
0.174
0.269
0.299
0.184
0.183
0.385
0.277
0.286
0,322
0.271
0.371
0.252
0.356
336
0.173
0.264
0.194
0.200
0.182
0.274
0.190
0.200
0.203
0.902
0.100
0.208
0.212
0.202
0.200
0.301
0.211
0.336
0.243
0.352
0.269
0.363
0.214
0.234
0.221
Ang
00.170
0.360
0.284
0.179
0.271
0.190
0.288
0,212
0.506
0.192
0.302
0.215
0.202
0.312
0.301
0.214
0.338
0.344
0.354
0.265
0.358
96
0.393
0.365
0.461
0.327
0.397
0.271
0.551
0.323
3,590
0.316
0,498
0.298
0.646
0.386
0.649
0.397
0.588
0.367
0.575
0.356
0.689
0.366
Traffic
0.275
0:470
0.326
0.519
0.624
0.962
0.371
0.377
0.647
0.394
0.700
196
6.421
0.250
0.490
0.358
0.479
0.246
0.529
0.327
0.641
0.545
0.520
0.314
0.006
0.963
0.005
0.373
0.640
0.988
016944
0.446
0.877
0.366
720
0.298
0,321
0.253
0.602
0.303
0.573
0.670
0.356
0.567
0.643
0.385
0.395
0.718
0.450
0.711
0,468
0.681
Ang
0.419
0.250
00446
0.396
0.476
0.285
0.598
0.390
0.631
0.318
0.520
0.312
0.634
0.91%
0.605
0.384
0.640
0.908
0.660
0.416
0.092
0.519
96
0.169
0.219
0.180
0.220
0.171
0.210
0.174
0.225
0.183
0.229
0.179
0.344
0.192
0.232
0.194
0.253
0.235
0.310
0.233
0.306
0.423
0.448
Weather
0.346
02113
0.395
0:786
0,347
336
0.259
0.297
0,283
0.301
0.296
0.515
0.200
0,316
0.293
0.312
0,273
0,330
0.292
0.307
0.282
0.352
0.364
0.997
0.345
0,385
01948
0.086
720
0.337
0,349
0.358
0.345
0.362
0.353
0.337
0,366
0.361
0,360
0.399
0.364
0.353
0.385
0.411
0.429
0,412
0,420
0.861
0.685
Ang
6.341
0.250
0.261
0.281
0.269
0.259
0.369
0.293
0.211
0.295
0.264
0.321
0.212
0.291
0:265
0.317
0.526
0.572
0.319
0.365
0.699
0.001
12
6,000
6.17%
0:061
0.191
0.072
0.179
00085
0.198
0.004
0.301
0.006
0.217
0.117
0.236
0.005
0.220
0. 10008
0.239
0:233
0.366
1.106
0.206
PEMS05
24
0.196
0.121
0.240
0.106
0.217
0.129
0.116
0,005
0.210
0.322
0.207
0.131
0.255
0,485
0.117
0.221
36
0.105
0.319
0,180
0.292
0.137
0.251
0.173
0.296
0.134
0.237
0.107
0.222
0.379
0.418
0.258
0.361
0.159
0.255
0.327
0.415
0.127
0.233
0.124
0.238
0.201
0.316
0.174
0.285
0.207
0.315
0.161
0.262
0.125
0.342
0.535
0.516
0.319
0.410
0.209
0
0.679
0.634
0.139
0.245
Asg
8.097
0.208
0.146
0.260
0.122
0.233
0.149
0.261
0.126
0.230
0.106
0.223
0.316
0.370
0.216
0.322
0.152
0.275
0,411
0.475
0.122
0.226
12
6.081
0.183
01091
0.199
0.886
0.187
0.095
0.205
0.111
0.208
0.161
0.214
0.121
0.235
0.113
0.225
0.165
0.238
00233
0.354
0.204
0.212
PEMSO8
24
0.318
0.138
0.245
0.123
0.217
0.151
0.139
0.212
0.127
0.237
0.232
0.325
0.302
0.197
0.258
0.545
0.550
0232
0.251
36
6.157
0.253
0.199
0.303
0.170
0.268
0.203
0.313
2.688
0.300
0.148
0,252
0.376
0.427
0.295
0.371
0.241
0.326
0.379
0.436
0.246
0.263
0.207
0.204
0.355
0.358
0.218
0.306
0:347
0.334
0.189
0.272
0.175
0.270
0.543
0.527
0.380
0.420
0.902
0.375
0.531
0.502
0.278
0.297
Asg
6.141
0.237
10.17L
0.271
0.142
0.247
0.174
0.375
0.152
0.343
0.153
0.258
0.318
0.378
0.249
0.352 0. 0.226
0.312
0.422
0.456
0.240
0.261
1" Count
43
42
0
0
0
0
0
Note: We lix the inpul length as 96 lollewing [28). Bold typeface highlights the lop performance for each metric, while undertined text denotes second best results, Arg indicates results averaged over forecasting lengths: T=96, 192, 720.
18

---

Table 8: The comprehensive results on the short-term forecasting task.
Time-ol
Fedformer
Models
(Ours)
(2024)
(3024)
(2025))
(2023)
(2023)
(2023)
Metri
SMAPE
MASE
OWA
SMAPE
MASE
OHAA
SMAPE:
MASE:
OWA
SMAPE
MASE:
OWA
SMAPE
MASE
OHAA
SMAPE:
MASE
OWA
SMAPE
MASE
OWA
Norty
13.48
An00
I87991
13.509
1,028
0.794
13,997
3.145
0.518
13,576
5008
0.00
14.594
3.592
0.871
14.907
3,006
0.827
13.688
5089
0880
Quanenty
10.102
11,417
Mondll
12.69
0330
0.875
12,096
0.931
0,575
13.227
1,013
0.935
13,088
0.990
0.919
13,834
1,000
0.987
13,362
1.007
0.537
14.151
1.105
1.011
5,101
1,076
6.137
1.114
Awerage
11.841
1,585
0L851
11,500
1.500
0.854
12.298
1.68
895
12. 160
0.833
13.044
1.841
0.962
12.48
1.674
0.808
12.734
100
0914
Note: Bold typeface highlights top performance for cach metric. while underlined text denotes the sccond-best results. Aug indicates results averaged over forecasting lengths: yearly, quarterly, and monthly.
GroundTruth Pred coor
GroundTruB
0.0
Phod Coon
-0.5
-10
-10
-16
-15
-1.5
100
202
300
400
100
202
300
400
10
22
28
(a) Fredformer with Em case
0.0
0.0
-5
-1.0
-1.0
-10
-15
GroundTrum canicciion
-15
GroundTruh
Phadiction
200
300
100
200
400
30
10
30
(b) i'Transfrmmer with ETTm2 case
0.5
0.6
GrouhdTrue
0.4
0.4
Prod
SmunsTrah Prodiction
0.8
Prodittion
(c) Fredformer with ETTm2 case
2
GroultTri Pind ESor
GrouBuTTiun Powd Eainn
8.
0.0
10.4
114
-10
-10
0
130
200
200
400
120
200
X00
400
10
20
20
10
30
(d) iTransformer with ETTm2 case
Figure 6: The forecast sequences generated with DF and Time-ol. The forecast length is 336 and the experiment is conducted on ETTm2.
19

---

24
MM
MMA
Ground Prodiction
GroundTruth Prodiction
-12
-12
Fredformer with BCL case
GroundTruhh
GroundTiuum Presd
MM MMM
pommmr
1.2
1.2
Groundrruh Prediction
GroundTruth Prediction
-10
(b) Trransformer with ECL case
NWW M
MM MM
Prod odion
6
Ww
GroundTruth
-1.2
-1.2
-12
100
200
300
10
200
300
25
40
30
(c)
Fredformer BCL case
2.4
2.4
1.2
GmundTrahh Prodiction
RmundTrth Prodiction
(d) iTransformer with ECL case
Figure 7: The forecast sequences generated with DF and Time-ol. The forecast length is set to 192 and the experiment is conducted on ECL.
0.55
0.41
MAF
0.40
0.32
0.40
0.38
Fred.
Trans.
FreT8
Dune.
Fred.
iTrena.
Fre
DLine.
Fred.
Mans.
Frott
DL
Fred.
iTrana
FreT9
DUne.
0.46
with Timeol
man with Timeot
D:54
NMm with Timeot
cwn with Timeo!
0.44
Wil Tme-oo
Tino-11
0:51
Time-o1
Time-ot
MSE
0.48
0.40
0.40
0.38
0.35
024
0.30
Tred:
Fred.
Freds
ETTml
(b) ETTm2
(c) ETThl
(d) ETTh2
Figure 8: Performance of different forecast models with and without Time-ol. The forecast errors are averaged over forecast lengths and the error bars represent 50% confidence intervals.
20

---

Table 9: Varying tramsformation results.
Trans
PCA
RPCA
SVD
FA
DF
Metries
MSE
MAF
MSR
MAR
MSF
MAF
MSE
MAE
MSR
MAR
96
0.1449
0.2348
0.1450
0.2349
0.1450
0.2350
0,1478
0.2385
0.1500
0.2415
ECL
192
0.1592
0.2487
0.1394
0.2487
0.1595
0.2490
0.1619
0.2517
0.1681
0.2591
336
0.1731
0.2645
0.1732
0.1730
0.2643
0.1789
0.2711
0.1823
0.2744
720
0.2033
0.2920
0.2066
0.2214
0.3066
0.2095
0.2975
0.2145
0.3035
Avg
0.1701
0.2600
0.1710
0.2611
0.1747
0.2637
0.1745
0.2647
0.1787
0,2696
96
0.1692
0.2185
0.1715
0.2199
0.1723
0,2223
0,1717
0.2247
0.1737
0.2277
Weather
192
0.2102
0.2575
0.21166
0.2590
0.2116
0,2597
0.2125
0.2636
0.2128
0.2661
0.2586
0.2971
0.2676
0.2997
720
0.3271
0.3487
0.3303
0.3581
0.3394
0,3681
0.3354
0,3610
0.3372
0.3623
Avg
0.2413
0.2805
0.2441
0.2452
0,2872
10
0.00
090
86
6750
MBE
08
(b)
045
240
200
(c)
2.00
00
32
3.6
08
00
04
04
00
82
0.6
3.6
06
12
0.6
34
68
00
85
86
08
0H6
MOE
3115
0100
2100
(e)
MED
2.6
Figure 9: Time-ol improves Fredformer performance given a wide range of transformed loss strength a. These experiments are conducted on ETTh1 (a), ETTh2 (b), ETTml (c), ETTm2 (d), ECL (e), Weather (f) datasets. Different columns correspond different forecast lengths (from left right: 96, 192, 336, 720, and their average with shaded areas being 15% confidence intervals).
21

---

090
0.7
0.99
0V00
9072
M65
MB5
010
083
0.00
2010
1447
0.1
0.5
0.7
0.1
0.5
07
0.4
0.7
0.5
0.1
0.4
07
0.54
0.85
0.00
0.00
6.00
(d)
1/17
0.94
2411
0100
019
5300
3296
20
0.1
04
07
0.1
04
3.
13
04
0.7
12
0.1
0.4
07
0.1
04
07
Figure 10: Time-ol improves Fredformer performance given a wide range of rank ratio 7. These experiments conducted on ETTh1 (a), ETTh2 (b), ETTml (c), ETTm2 (d), ECL (e), and Weather (f) datasets. Different columns correspond to different forecast lengths (from left to right: 96, 192, 336, 720, and their average with shaded areas being 15% confidence intervals).
22

---

Table 10: Comparable results with different loss functions.
Loss
Time-1
FreDF
Koopman
Dilate
Soft-DIW
DPTA
DF
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Forecast model: FredFormer
305
U.321
0.357
U.326
0.355
0.335
0.368
0.337
0.367
0.312
0.363
0.332
0.354
0.126
U.351
ET7ml
192
0.366
0.384
0.370
0.365
0.382
336
0,389
0.400
0.392
0,400
0.399
0.408
0.397
0.406
0.406
0.409
0.409
0.410
0.396
0.404
720
0.447
0.435
0.455
0.440
0.456
0.441
0.457
0.443
0.478
0.450
0.476
0.4488
0.459
0.444
Avg
0.379
0.393
0.384
0.394
0.389
0.400
0.389
0.400
0.397
0.402
0.396
0.402
0.387
0.398
0.370
0.375
0.397
0.399
0.378
0.399
0.396
ETTh
192
0,424
0.422
0.436
0,437
0,438
0.434
0.439
0.435
0.439
0.435
0,438
0.433
0,437
0.425
336
0.467
0.44]
0.473
0.443
0.473
0.455
0.481
0:453
0.484
0.455
0,486
0.455
0.486
0,449
720
0.465
0.463
0.474
0.466
0.523
0.487
0.516
0.482
0,542
0,510
0,538
0,510
0.488
0.467
Avg
0,431
0.429
0.438
0.434
0.452
0.443
0.453
0.442
0.460
0.449
0,460
0.449
0.447
0.434
0.245
0.152
0.166
0.263
0.253
0.158
0.253
0.258
192
0.166
0,256
0.166
0.257
0.174
0.267
0.170
0.263
0.218
0.313
0,216
0.307
0.174
0.269
BCL
336
0.181
0.274
0.183
0.278
0.188
0.280
0.190
0.286
0.197
0.291
0.199
0.295
0.194
0,290
720
0,213
0.304
0.216
0.304
0.232
0.318
0.229
0.316
0.240
0.322
0.235
0.322
0,235
0.319
Avg
0.178
0,270
0.179
0.272
0.190
0.282
0,187
0,280
0.206
0.298
0,202
0.294
0.191
0,284
96
0.171
0.208
0.174
0,213
0.174
0.214
0.173
0.214
0.173
0.213
0.179
0.219
0.180
0,220
Weather
192
0.219
0.255
0.219
0.254
0.220
0.256
0.225
0.260
0.220
0.255
0,223
0.257
0.222
0.25%
336
0.277
0.295
0.278
0.296
0.280
0.298
0.280
0.299
0.281
0.296
0.281
0.298
0.283
0.301
720
0.353
0.346
0.354
0.347
0.354
0.347
0.385
0.348
0.369
0.355
0.356
0.347
0.358
0.348
Avg
0,255
0,276
0.256
0,277
0.257
0.279
0.258
0,280
0.261
0,280
0,260
0,280
0,261
0,282
model TTransforer
996
0.323
0.358
0.334
0.365
0.350
0.382
0.342
0.376
0.339
0.373
0.341
0.335
0.
0.372
El1ml
192
0.371
0.388
0.381
0.390
0.389
0.400
0.381
0.396
0.383
0.395
0.383
03395
0.382
0.396
336
0.408
0.407
0.417
0.412
0.425
0.423
0.418
0:418
0.439
00433
0.429
0,433
10:27
0.424
720
0.477
0.450
0.489
0.453
0.489
0.458
0.487
0.457
0.516
0.469
0.512
0.467
0.496
0.463
Avg.
0,395
0.401
0.405
0.405
0.413
0.416
0.407
0.412
0.417
0.415
0,416
0.415
0,411
0.414
96
0.378
0.393
0.378
0.395
0.392
0.411
0.385
0.405
0.387
0.405
0.386
0.405
0.385
0.405
ET
192
0.428
0.423
0.428
0.423
0.446
0.442
0.440
0.437
0.443
0.439
0.441
0.439
0.440
0.437
336
0.473
0.450
0.470
0.447
0.483
0.461
0.480
0.457
0.494
0.464
0.489
0.462
0.480
0.457
0.490
0.501
0.491
0.492
0.557
0.520
0.538
0.509
0.504
0.492
Avg
0.438
0.434
0.442
0.437
0.455
0.451
0.452
0.448
0.470
0.457
0.463
0.454
0.452
0.448
96
0.145
0.235
0.149
0.238
0.151
0.243
0.150
0.241
0.149
0.241
0.149
0.240
0.150
0,242
ECL
192
0.1.99
0.264
0,249
0.179
0.163
0,251
0.167
0.182
0.257
0.275
0.168
0.259
0.274
0.180
0,164
0.255
0.180
0.166
0.257
0.272
0.182
0.168
0,259
0.274
0.292
0.212
0.212
0.300
0.300
0.207
0.296
0,212
0.300
0.214
0.304
Avg
0.170
0.260
0.176
0.264
0.178
0.269
0.178
0.269
0.175
0.266
0.177
0.267
0.179
0.270
96
0.163
0.202
0.170
0.208
0.206
0.257
0.208
0.259
0.207
0.252
0.209
0.258
0.171
0,210
Weather
192
0.214
0.248
0.219
0.252
0.264
0.300
0.252
0. 0.285
0.264
0.303
0.258
0.291
0.246
0.278
0.294
0.279
0.309
0.326
0.31
0.328
0.314
0.312
0.331
0.296
0.313
720
0.351
0.344
0.358
0.347
0.377
0.369
0.374
0.364
0.384
0.377
0.383
0.373
0.362
0.353
Avg
0.251
0.272
0.257
0.289
0.313
0.309
0.292
0.316
0.291
0.313
0.289
23

---

Table 11: Varying input sequence length results on the Weather dataset.
Models
Time-ol
iTaasformer
Time-l
PaccTTST
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.163
0.202
0.210
0.213
0.244
192
0.214
0.248
0,246
0.278
0.224
0.257
0.229
0,263
336
0.274
0.294
0.296
0,313
0.276
0.296
0.287
0.303
720
0.351
0.344
0,362
0.353
0.353
0.346
0.363
0.353
0.250
0.272
0.269
0.289
0.257
0.278
0.270
0.291
Input sequence length
0.163
0.205
0.199
0.208
192
0.210
0.248
0.213
0,253
0.204
0.242
0.225
0.269
0.259
0.287
0.294
0.286
0.287
0.308
720
0.334
0.338
0,341
0.345
0.332
0.337
0.341
0,345
0.241
0.270
0.247
0.277
0.238
0.266
0.254
0,283
0.157
0.203
0.213
0.196
0.156
0.206
192
0.199
0.246
0.211
0.256
0.196
0.241
0.222
0.277
0.251
0.287
0.260
0,295
0.246
0.282
0.251
0.285
720
0.324
0.338
0,332
0.341
0.320
0.333
0.327
0.338
0.233
0.268
0.241
0.276
0.228
0.263
0.239
0.277
0.161
0.213
0.225
0.201
0.207
192
0.205
0.250
0,220
0,268
0.198
0.248
0.205
0,254
0.292
0.288
720
0.318
0.339
0.337
0.351
0.313
0.335
0.317
0,339
0.235
0.274
0.253
0.289
0.228
0.267
0.231
0.272
Table 12: Experimental results (meannssa) with varying seeds (2021-2025).
Dataset
ECL
Weather
Models
Time-o1
DF
Time-ol
DF
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.145 000
0.235_0.000
0.150=20.00:
0.242.10 o01
0.164_p-oo
0.203-20.001
0.190
0.232 014
192
0.160
0.249
0.166
0.257
0.216
0,250
0,240
0.272
336
0.17440.002
0.266-0.oom
0.181 20.00:
0.273 ox1
0.274_0.001
0.294:20.001
0.293 003
0.310 003
720
0.205 10.001
0.293
0.21620.000
0.303 o0a
0.353go.002
0.344
0,361 100.000
0.352 001
0.171
0.261
0.000
0.269 001
0.252
0.273
0.271 2..1440.00
0.292
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect our paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Our limitations are discussed in conclusions.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
25

---

Justification:
All proofs of theoretical results are involved in Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We detail our training and evaluation protocols the experimental setting section.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
We use open access data, and the code is provided in an anonymous link.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.cm cc/
public/guicn Sewriidee/ooeSSumiiiii ceapanissssoooooooiioioooooooios wwwsssssoooiioooooooooooooooiill olicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
//niss.. wwt.ccccpppccceeeeeeeeeec.ccceee hotoli/c/gdddee/oeuuubiiii
Cete//CooSuuiiiiii ssisinn wwwsssnoolli onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We detail our training and evaluation protocols in the experimental setting section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In Figure which are crucial for evaluating the efficacy of TransDF, we report the error bars to make the results more rigorous and comprehensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
27

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the type of compute workers, memory in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http:/::////ep./eeeeeeeeee http:/p..c..pubiiciiiictttttitii
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Since it is an algorithm segorithm-orientedd research, there is no societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not involve these issues.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The creators or original owners assets (e.g., code, data, models), used in the paper, are properly credited. The license and terms of are explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi whrode..tteeeee.... com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
29

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
We do introduce new datasets and benchmarks in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: We use open-access datasets and do not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: We use open-access datasets and do involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
30

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: We do not involve LLMs as any important, original, or non-standard compo- nents in this work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: :otpps:/n //wprries..////// for what should or should be described.
31


Reviews:
[{"review_id": "FIEJ3gV40c", "reviewer": "Reviewer_G79D", "summary": "The paper tackles a fundamental but underexplored aspect of neural time‑series forecasting, i.e., the choice of learning objective. Building on the standard direct‑forecast paradigm (which minimizes pointwise temporal MSE), the authors identify two critical shortcomings of this objective in the forecasting setting, Label autocorrelation bias and Excessive multitask complexity. To address these, the authors propose TransDF, which can be dropped in to any existing architecture without structural changes. Extensive experiments across multiple benchmark datasets (ETT, ECL, Weather, M4) and forecasting models (Transformers, MLPs, GNNs) demonstrate that TransDF outperforms both the canonical TMSE loss and a variety of shape‑alignment or frequency‑domain objectives, achieving new state‑of‑the‑art results. An ablation study further isolates the benefits of decorrelation versus task reduction, confirming that their combination is key to TransDF’s success .", "questions": "See Weaknesses.", "limitations": "The Limitations section is concise but could better discuss potential failure modes, e.g., cases where label autocorrelation structure is nonstationary or when small-variance components carry critical signals (anomaly forecasting).", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths:\n1. The proposed transformation is grounded in sound linear‑algebraic principles, which directly targets the source of bias and task redundancy.\n2. The method is model‑agnostic and “drop‑in”—it requires no architectural changes and gracefully extends to any direct‑forecasting network, as evidenced by consistent improvements on FredFormer, iTransformer, FreTS, DLinear, etc.\n3. TransDF’s randomized, data‑adapted projections guarantee full decorrelation at finite horizons and discriminate component importance, which is novel to me.\n\n## Weaknesses:\n1. The paper could more deeply explore or theoretically characterize when and why certain projection strategies might outperform the randomized approach.\n2. The Limitations section is concise but could better discuss potential failure modes, e.g., cases where label autocorrelation structure is nonstationary or when small-variance components carry critical signals (anomaly forecasting).\n3. The core idea of transforming labels into another basis has precedents in frequency‑domain losses (FreDF) and shape‑alignment objectives; the manuscript could more explicitly delineate how TransDF’s theoretical guarantees and practical gains substantially surpass these earlier approaches beyond empirical metrics.", "was_revised": true, "final_justification": "The rebuttal has well addressed my concerns.", "rebuttal": "Thank you very much for your encouraging support and appreciation of our significance, implementation, and novelty. Below are our responses to the specific query raised.\n\n---\n\n#### *[W1] The paper could more **deeply explore or theoretically characterize** when and why certain projection strategies might outperform the randomized approach.*\n\n**Response.** Thank you for this constructive suggestion. We have now included both **theoretical characterization and empirical exploration** to clarify when and why certain projection strategies outperform randomized approaches.\nSpecifically, as shown in the table below, we compare five variants differing in projection matrices $\\mathbf{P}$ and output dimensions (full $\\mathrm{T}$ versus reduced $\\mathrm{K} < \\mathrm{T}$). \n\n|Index| Model  | Projection Type | Projection Dimension | ECL | | | | Weather | | | | ETTh1 | | | |\n|-|--|-|-|--|--|--|-|--|--|-|--|--|-|--|--|\n|  | | | | MSE  | $\\Delta$ | MAE | $\\Delta$ | MSE | $\\Delta$ | MAE | $\\Delta$ | MSE | $\\Delta$ | MAE | $\\Delta$ |\n|1| DF | Identity | T | 0.179 | -  | 0.270 | -  |0.249 | -  | 0.293 | -  | 0.447 | - | 0.434 | - |\n|2| TransDF$^{\\star}$ | Random | T | 0.177 | 1.1%$\\downarrow$ | 0.265 | 1.9%$\\downarrow$ | 0.248 | 0.4%$\\downarrow$ | 0.283 | 3.4%$\\downarrow$ | 0.440 | 1.5%$\\downarrow$ | 0.435 | 0.2%$\\uparrow$ |\n|3| TransDF$^\\dagger$  | Random | K | 0.175 | 2.2%$\\downarrow$ | 0.264 | 2.2%$\\downarrow$ | 0.244 | 2.0%$\\downarrow$ | 0.281 | 4.1%$\\downarrow$ | 0.440 | 1.5%$\\downarrow$ | 0.435 | 0.2%$\\uparrow$ |\n|4| TransDF$^\\ddagger$  | Orthogonal | T | 0.172 | 3.9%$\\downarrow$ | 0.263 | 2.5%$\\downarrow$ | 0.244 | 2.0%$\\downarrow$ | 0.283 | 3.4%$\\downarrow$ | 0.439 | 1.7%$\\downarrow$ | 0.431 | 0.7%$\\downarrow$ |\n|5| TransDF  | Orthogonal | K | **0.170** | 5.0%$\\downarrow$ | **0.260** | 3.8%$\\downarrow$ | **0.241** | 3.2%$\\downarrow$ | **0.280** | 4.4%$\\downarrow$ | **0.431** | 3.5%$\\downarrow$ | **0.429** | 1.1 %$\\downarrow$|\n\n- Firstly, we recall the implementation of the 5 projection strategies in the table above. We perform projection on the label sequence $\\mathbf{Y}\\in\\mathbb{R}^{m\\times\\mathrm{T}}$ and prediction sequence $\\hat{\\mathbf{Y}}\\in\\mathbb{R}^{m\\times\\mathrm{T}}$ to obtain components: $\\mathbf{Z}=\\mathbf{YP}$, \n$\\hat{\\mathbf{Z}}=\\hat{\\mathbf{Y}}\\mathbf{P}$, where $\\mathbf{P}\\in\\mathbb{R}^{\\mathrm{T}\\times\\mathrm{K}}$, m is the number of samples, $\\mathrm{T}$ is prediction horizon, $\\mathrm{K}$ is the number of extracted components. \n    - DF sets $\\mathbf{P}$ as identity matrix (i.e., using the temporal MSE loss)\n    - TransDF$^{\\star}$ sets $\\mathbf{P}$ as random matrix with the shape $\\mathbb{R}^{\\mathrm{T}\\times\\mathrm{T}}$. TransDF$^\\dagger$ further sets the shape as $\\mathbb{R}^{\\mathrm{T}\\times\\mathrm{K}}$, i.e., considering $\\mathrm{K}$ components. \n    - TransDF$^\\ddagger$ sets $\\mathbf{P}$ as the orthogonal matrix calculated via SVD in Lemma 3.3, with the shape $\\mathbb{R}^{\\mathrm{T}\\times\\mathrm{T}}$. TransDF further sets the shape as $\\mathbb{R}^{\\mathrm{T}\\times\\mathrm{K}}$, i.e., considering $\\mathrm{K}$ components. \n- Secondly, we discuss when and why certain projection strategies might outperform the randomized approach (TransDF$^{\\star}$).\n    - **Orthogonal projection outperforms random projection.** **Empirically**, comparing line 2 and 4, TransDF$^{\\ddagger}$ outperforms TransDF$^\\star$; comparing line 3 and 5, TransDF outperforms TransDF$^\\dagger$. **Theoretically**, the improvement is attributed to the bias elimination effect of the orthogonal projection matrix (see Lemma 3.2), which enhances learning objective estimation accuracy and therefore improving forecast performance.\n    - **Reduced-dimension projection outperforms full-dimension projection.** **Empirically**, comparing line 2 and 3, TransDF$^{\\dagger}$ outperforms TransDF$^\\star$; comparing line 4 and 5, TransDF outperforms TransDF$^\\ddagger$.  comparing line 2 and 3, 4 and 5. **Theoretically**, the improvement is attributed to the reduced task amount, which mitigates gradient conflicts across different tasks and therefore improving forecast performance.\n\n#### *[W2] The Limitations section is concise but could better discuss potential failure modes, e.g., cases where label autocorrelation structure is nonstationary or when small-variance components carry critical signals (anomaly forecasting).*\n\n**Response.** We agree that the limitations section should be more detailed. The two points raised are insightful. We discuss them below and present a revised limitations section accordingly.\n- **On the case where label autocorrelation structure is nonstationary.** We observe that TransDF retains effective to reduce bias given nonstationary label autocorrelation structure. **Theoretically**, the derivation of decorrelation does not rely on the stationary autocorrelation structure (see Lemma 3.2). \n**Empirically**, for general time-series data that are not always stationary, the label autocorrelation can be effectively eliminated by the proposed method, with the correlation coefficient matrix being a diagonal matrix (see Figure 4 in appendix). **Therefore, TransDF remains applicable when label autocorrelation structure is non-stationary**, since it can theoretically and empirically reduce the label correlation and therefore the bias.\n- **On the case where small-variance component matters.** Our method assumes that components with larger variance are more significant, and thus prioritizes these in the projection. However, we acknowledge that this heuristic may be suboptimal where critical information is embedded in low-variance components. For example, in anomaly forecasting, the small-variance components associated with anomalies may carry low-variance but critical patterns. **We truly appreciate this point and incorporate it to strengthen `limitation section`.**\n> Limitations & future works.  TransDF aims to address label correlation bias and excessive task amount —challenges that are relevant to multi-step forecasting. In single-step forecasting scenarios such as streaming prediction, where these challenges are less relevant, TransDF might be not applicable. **Additionally, this study assumes that latent components with higher variance are more significant. While generally effective, this assumption may not hold in scenarios where crucial information is contained in low-variance component.** For example, in anomaly forecasting, subtle patterns indicative of anomalies may exhibit small variance. A promising direction for future work is to develop an adaptive projection strategy, possibly through meta-learning, that directly targets downstream task performance. \n\n\n\n#### *[W3] The core idea of transforming labels into another basis has precedents in frequency‑domain losses (FreDF) and shape‑alignment objectives; the manuscript could more explicitly delineate how TransDF’s **theoretical guarantees and practical gains** substantially surpass these earlier approaches beyond empirical metrics.*\n\n**Response.** Thank you for your prompting reminder on related works. We agree that it is necessary to further clarify the advantage of TransDF over FreDF and shape-alignment methods. Please find our concise resposne to address this concern below.\n- Firstly, we summarize key advantages of TransDF along two axes in the review report's suggestion.\n  - **Theoretical guarantees.** TransDF eliminates autocorrelation bias in **finite-horizon** settings, while FreDF requires **infinite horizons** and shape-alignment methods provide **no bias elimination guarantees**.\n  - **Practical gains.** TransDF **both** eliminates bias (improving learning objective estimation) and reduces optimization tasks (mitigating optimization difficulties such as gradient conflict), while existing methods address **neither** (shape-alignment) or only **one aspect partially** (FreDF).\n\n\n- Secondly, to provide more comprehensive response, we introduce and compare the three methods considering three factors: (1) whether they formally define the autocorrelation bias, (2) whether they effectively eliminate it, and (3) whether they reduce the amount of optimization tasks. The comparison is summarized below:\n    - **Shape alignment methods** mainly use the DTW loss for intuitive shape matching but **lack autocorrelation bias analysis**. DTW may actually **increase task amount** by matching single predictions to multiple labels.\n    - **FreDF** uses the frequency MAE loss as training objective. This work **identifies the autocorrelation bias under first-order Markov assumptions** but **provides bias elimination guarantees only as $T\\rightarrow \\infty$** (see Theorem 3.3, FreDF). In real-world settings with finite horizon, FreDF fails to eliminate bias. Moreover, frequency domain transformation preserves the label sequence's length and does not reduce the number of optimization tasks.\n    - **TransDF** **generalizes autocorrelation bias beyond first-order Markov assumptions** and **ensures bias elimination without requiring $T\\rightarrow \\infty$**, making it practically effective. It also **reduces task amount** by focusing on significant components.\n\n|Method|Autocorrelation bias formalization|Bias elimination|Task reduction|\n|-|-|-|-|\n|Shape alignment|$\\times$|$\\times$|$\\times$|\n|FreDF|$\\checkmark$|$\\circ$|$\\times$|\n|TransDF (Ours)|$\\checkmark$|$\\checkmark$|$\\checkmark$|"}, {"review_id": "wLV7tXNba4", "reviewer": "Reviewer_2azR", "summary": "TransDF proposes a novel training objective that decorrelates the label space and aligns the most informative components to mitigate label autocorrelation and reduce the task amount. It can be plugged into existing time-series forecasting (TSF) models by reformulating the loss objective and applying an SVD-based label transformation.", "questions": "Why does even TransDF† (which uses a random projection and only partial component alignment) outperform the canonical DF objective, especially on the Weather dataset?", "limitations": "The SVD-based label transformation relies solely on label-side prior statistics, making it less suitable for fully online or streaming scenarios. This limitation may hinder deployment in real-time TSF systems, where adaptive behavior and incremental updates are often required.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The paper clearly formulates the bias issue of the TMSE objective in Theorem 3.1 and addresses it by aligning fully decorrelated label components through SVD-based transformation.\n- While FreDF relies on the assumption of decorrelation between frequency bases,which only holds under infinite forecast horizons, TransDF removes label autocorrelation bias under realistic finite horizons and simultaneously reduces the number of forecasting tasks.\n\nWeaknesses\n- The proposed method applies a global projection matrix derived from SVD uniformly across all forecast steps. However, real-world time-series data often contain local pattern shifts or nonlinear/dynamic structures. Thus, a static global projection may fail to capture such local dynamics, which might explain smaller improvements on certain datasets (e.g., PEMS or Traffic).", "was_revised": true, "final_justification": "Overall, I appreciate the author's detailed responses and additional experiments, which have clarified my main concerns on local projection strategies and partial component alignment. My raised issues are largely addressed, and I find my original assessment remains appropriate, so I maintain it as it is. Although I could not engage more actively in the discussion phase, I found the author's rebuttal convincing and believe my initial review and score remain appropriate.", "rebuttal": "Thank you very much for your positive comments and appreciation of our contributions. Below are our responses to the specific query raised.\n\n---\n\n#### *[W1] The proposed method applies a global projection matrix derived from SVD uniformly across all forecast steps. However, real-world time-series data often contain local pattern shifts or nonlinear/dynamic structures. Thus, a static global projection may fail to capture such local dynamics, which might explain smaller improvements on certain datasets (e.g., PEMS or Traffic).*\n\n**Response**. Thank you for this kind and informative comments! We fully agree that `a static global projection may fail to capture such local dynamics`, which is an important direction for future research. Nevertheless, **we slightly note that the static global projection suffices to address the two research problems in the current work.** Please look at the concise analysis below.\n- **In this work, we target two challenges of designing learning objectives for time-series forecasting models:** (1) label correlation, which leads to bias from the true label likelihood; (2) excessive amount of tasks, which increases with the forecast horizon and complicates optimization.\n- **The global projection approach in Eq. (4) suffices to address the two challenges theoretically (Lemma 3.2) and empirically (Table 3)—our primary focus in this work.**\n- We concur that developing adaptive or local projection strategies can enhance modeling of dynamic and nonlinear structures. Nevertheless, it remains non-trivial to construct such methods while preserving the rigor of bias elimination and effective task reduction, **suggesting a promising avenue** for future exploration. One promising approach, for instance, may consider learnable projections with appropriate regularization to ensure decorrelation for debiasing and significance discrimination for reducing task amount.\n\n#### *[Q1] Why does even TransDF† (which uses a random projection and only partial component alignment) outperform the canonical DF objective, especially on the Weather dataset?*\n**Response**. This scenario also raises our attention. We attribute the superior performance of TransDF† over the canonical DF objective primarily to the reduction in optimization tasks.\n\n- **Firstly, partial alignment (aligning only a subset of components) introduces two opposing effects:** (1) some information is lost by discarding components, which may hinder learning and **reduce performance**, and (2) the number of tasks to optimize is reduced, which can reduce optimization difficulty and **improve performance**. The latter reflects a common issue in multitask learning: an excessive number of tasks often causes **gradient conflict** and hampers training [1,2].\n- **Secondly, the opposing impacts above make partial component alignment plausible to improve performance.** On representative datasets like Weather, tuning the number of aligned components allows partial alignment to outperform full alignment, suggesting that the benefits of reduced task conflict outweigh the drawbacks of partial information loss in this setting.\n\n\n#### *[L1] The SVD-based label transformation relies solely on label-side prior statistics, making it less suitable for fully online or streaming scenarios. This limitation may hinder deployment in real-time TSF systems, where adaptive behavior and incremental updates are often required.*\n\n**Response.** Thank you very much for highlighting this important aspect.\n- In this study, we investigate **two central challenges in multi-step (multi-task) forecasting**: (1) label correlation bias and (2) excessive task amount. We develop TransDF to address the two challenges.\n- We acknowledge that **in the training stage**, TransDF requires access to the full T-step ahead label sequence for the SVD-based label transformation. **This requirement makes TransDF less suitable for training scenarios where only 1-step ahead labels are available, such as fully online or streaming forecasting.**\n  - By the way, if complete T-step ahead label sequences are available during training, TransDF remains applicable. Importantly, during inference, the SVD transformation is not needed, so TransDF can be deployed on streaming data; for 1-step-ahead online forecasting, the first prediction step can be directly utilized.\n- Nevertheless, in standard online or streaming tasks, **the two challenges addressed in this work do not arise**, as these challenges are specific to multi-step forecasting scenarios with label sequences longer than one. Therefore, employing TransDF in fully online or streaming settings is unnecessary. \n- We agree it is important to clarify that TransDF and its targeted challenges are specific to multi-task (multi-step) forecasting scenarios. The enhanced limitation section is presented as follows.\n> Limitations & future works. **TransDF aims to address label correlation bias and excessive task amount —challenges that are relevant to multi-step forecasting. In single-step forecasting scenarios such as streaming prediction, where these challenges are less relevant, TransDF might be not applicable.** Additionally, this study assumes that latent components with higher variance are more significant. While generally effective, this assumption may not hold in scenarios where crucial information is contained in low-variance component. For example, in anomaly forecasting, subtle patterns indicative of anomalies may exhibit small variance. A promising direction for future work is to develop an adaptive projection strategy, possibly through meta-learning, that directly targets downstream task performance."}, {"review_id": "ha2Aw5MfEX", "reviewer": "Reviewer_Vg9Y", "summary": "This paper proposes a new label transformation method that transforms time series forecasting labels to decorrelate labels and mitigate the unimportant forecasting labels, so as to reduce forecasting tasks. The proposed method can then enhance the performance of forecasting models and maintain high efficiency. Experiment results show that the proposed method can help improve the performance of existing forecasting models as a plug-and-play module. Comprehensive results further prove the effectiveness of the label decorrelation and reduction of the number of tasks.", "questions": "How to interpret the sharp error increase at $\\alpha=1$ for some datasets like weather, while other datasets do not?\n\nHow does the proposed model perform when applied to non-deep learning forecasting models?", "limitations": "The paper discusses future work, but does not really discuss limitations, even though it has a \"limitations and future works\" section.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is well written with comprehensive details and results.\n2. The idea is well motivated and explained. The motivation examples are easy to follow and make sense.\n3. The proposed method is efficient but effective. Experiments are designed carefully to show the superiority of the proposed method and to research various aspects of the proposed method.\n4. The provided code is comprehensive and structured clearly for researchers to follow.\n\nWeaknesses:\n1. Some conclusions are very precise. For example, the paper claims, \"We observe that increasing $\\alpha$ from 0 to 1 generally leads to improved forecasting accuracy, with the best results typically achieved when $\\alpha$ is close to 1.\" However, from the results, the best results occur at $\\alpha=1$ for ETTh2, ETTm1, and ETTm2. This should be presented precisely and explained clearly in the paper.\n2. The parameter sensitivity on different tasks shows different patterns, with some datasets showing a decrease with $\\alpha$ increases, and on other datasets showing a first decrease and sharp increase at $\\alpha=1$. These inconsistent patterns indicate potential failure of unexpected behaviours in some cases.\n3. To demonstrate the effectiveness of the method as a plug-and-play module, it would be better to see whether the proposed method improves forecasting performance on more base forecasting models, like non-deep learning ones, instead of four pure deep learning forecasting models.\n4. The paper does not really discuss limitations, although it has a section called \"limitations and future works\".", "was_revised": true, "final_justification": "This paper is well written with convincing motivations, clear structures, comprehensive details, as well as a well-structured code repository. The proposed method also presents novelty and effectiveness for the documented existing problems. My main concern about the sharp error increase has been well addressed with detailed explanations and experiments during the rebuttal. I thus advocate for the acceptance of this paper.", "rebuttal": "Thank you so much for the positive comments and appreciation of our writing, comprehensive experiments, meaningful motivation, effective methodology and structured code repository.\nThe four points raised are very insightful. \nPlease kindly check our responses to further address associated concerns.\n\nWe observed that some comments in the weaknesses and questions sections convey similar points. We will selectively merge them to make the response concise and easy to follow.\n\n---\n\n\n#### *[W1] Some conclusions are very precise. For example, the paper claims, \"We observe that increasing from 0 to 1 generally leads to improved forecasting accuracy, with the best results typically achieved when $\\alpha$ is close to 1.\" However, from the results, the best results occur at for ETTh2, ETTm1, and ETTm2. This should be presented precisely and explained clearly in the paper.*\n**Response.** Thank you very much for your meticulous feedback! We appreciate your careful review of our experimental results and analysis. In response, we have revised the claim as follows: \n>We observe that increasing α from 0 to 1 generally leads to improved forecasting accuracy. The best results are often achieved as α approaches 1, and in some cases, specifically when  α=1 (e.g., ETTh2, ETTm1, ETTm2).` \n\n\n#### *[W2, Q1] The parameter sensitivity on different tasks shows different patterns, with some datasets showing a decrease with increases, and on other datasets showing a first decrease and sharp increase at $\\alpha=1$. These inconsistent patterns indicate potential failure of unexpected behaviors in some cases. How to interpret the sharp error increase at $\\alpha=1$ for some datasets like weather, while other datasets do not?*\n\n**Response.** This phenomenon also raises our attention. We checked the phenomenon and identify the issues via bad case analysis. \n- We conducted a case study of the extracted components on Weather and other datasets with normal behavior. **The component magnitudes in the Weather dataset are significantly lower than in other datasets.** For example, fixing the forecast horizon T=720, mmost components on the Weather dataset fluctuate within [-2,2], with informative non-top components close to zero. In contrast, components on the ETTh1 dataset typically range from -40 to 10 and show larger values of non-top components. **This small amplitude is unique to datasets that exhibit a sharp error increase at $\\alpha=1$ (such as Weather).**\n- Therefore, we hypothesize that this sharp increase at $\\alpha=1$ is **caused by the small amplitude of components**, resulting in diminished losses and gradients, which hinders the effective alignment of important yet low-magnitude components.\n- One strategy to counteract the small amplitude of components is increasing learning rate. We increase the learning rate to 0.005 and investigate the sensitivity of $\\alpha$ on Weather dataset once again. The results are presented below, where the **sharp error increase at $\\alpha=1$ is effectively alleviated.**\n\n|              | T=96   | T=96   | T=192  | T=192  | T=336  | T=336  | T=720  | T=720  | Avg    | Avg    |\n| ------------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |\n| Metric       | MAE    | MSE    | MAE    | MSE    | MAE    | MSE    | MAE    | MSE    | MAE    | MSE    |\n| $\\alpha=0$   | 0.228  | 0.174  | 0.266  | 0.213  | 0.316  | 0.270  | 0.362  | 0.337  | 0.293  | 0.249  |\n| $\\alpha=0.2$ | 0.226  | 0.172  | 0.262  | 0.212  | 0.304  | 0.263  | 0.361  | 0.335  | 0.288  | 0.245  |\n| $\\alpha=0.4$ | 0.225  | 0.172  | 0.266  | 0.213  | 0.307  | 0.264  | 0.353  | 0.331  | 0.288  | 0.245  |\n| $\\alpha=0.6$ | 0.223  | 0.172  | 0.265  | 0.213  | 0.308  | 0.264  | 0.355  | 0.332  | 0.288  | 0.245  |\n| $\\alpha=0.8$ | 0.220  | 0.172  | 0.259  | 0.211  | 0.301  | 0.261  | 0.362  | 0.332  | 0.285  | 0.244  |\n| $\\alpha=1.0$ | 0.216  | 0.169  | 0.258  | 0.213  | 0.302  | 0.264  | 0.351  | 0.331  | 0.282  | 0.244  |\n\n#### *[W3, Q2] To demonstrate the effectiveness of the method as a plug-and-play module, it would be better to see whether the proposed method improves forecasting performance on more base forecasting models, like **non-deep learning ones**, instead of pure deep learning forecasting models. How does the proposed model perform when applied to non-deep learning forecasting models?*\n\n**Response.** Thank you for your insightful comment and suggestion. We agree that clarifying the applicability of TransDF to non-deep learning methods is important. However, the challenges that TransDF addresses—label correlation bias and excessive task amount—are generally absent in most non-deep learning approaches, as outlined below:\n- **Firstly, TransDF is designed to improve the multitask, multi-step forecasting paradigm**, where multiple future values are predicted simultaneously. **The two challenges inherently arise from this multitask setting** (label correlation bias and excessive task amount).\n- **In contrast, non-deep learning methods with defined learning objectives are often implemented for single-step forecasting**. Examples include linear regression (LR) and gradient boosted decision trees (GBDT)~[1]. Although these methods can be extended to multi-step forecasting (e.g., via iterative prediction or by training T separate models for T steps), the resulting formulations remain single-task, with each model trained independently for one step ahead.\n-  **In single-task learning, the aforementioned challenges do not emerge**. Since only one label is predicted at a time, neither label correlation nor the need to reduce the number of tasks is relevant. Consequently, **applying TransDF to such settings is unnecessary.**\n- We agree it is important to clarify that TransDF and its targeted challenges are specific to multitask forecasting methods that are often deep models. **We have incorporated it to make our limitation section more concrete.** Please see the response to [W4] for details.\n\n\n#### *[W4] The paper does not really discuss limitations, although it has a section called \"limitations and future works\".*\n\n**Response.** Once again, we sincerely appreciate the kind and actionable feedback provided. In response, **we have expanded the “Limitations and Future Work” section** by incorporating the points discussed in [W3, Q2], making our discussion more concrete. The revised text is as follows:\n> Limitations & future works. TransDF aims to address label correlation bias and excessive task amount —challenges that are relevant to multi-step forecasting. In single-step forecasting scenarios such as streaming prediction, where these challenges are less relevant, TransDF might be not applicable. Additionally, this study assumes that latent components with higher variance are more significant. While generally effective, this assumption may not hold in scenarios where crucial information is contained in low-variance component. For example, in anomaly forecasting, subtle patterns indicative of anomalies may exhibit small variance. A promising direction for future work is to develop an adaptive projection strategy, possibly through meta-learning, that directly targets downstream task performance."}, {"review_id": "YPpxmn5ZDd", "reviewer": "Reviewer_nczu", "summary": "Existing methods predominantly utilize the temporal mean squared error, which faces two critical challenges: (1) label autocorrelation, which leads to bias from the label sequence likelihood; (2) excessive amount of tasks, which increases with the forecast horizon and complicates optimization. In this work, the authors propose Transform-enhanced Direct Forecast (TransDF), which transforms the label sequence into decorrelated components with discriminated significance. Models are trained to align the most significant components, thereby effectively mitigating label autocorrelation and reducing task amount.", "questions": "Please see Weaknesses!", "limitations": "Please see Weaknesses!", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The core idea of applying PCA-like transformations to the label space to mitigate autocorrelation bias and reduce task complexity is novel and impactful. The use of SVD to generate decorrelated components ranked by significance provides a theoretically grounded solution. This approach effectively bridges a gap in objective design, which has been underexplored compared to architectural advancements.\n2. The paper validates TransDF rigorously across 8+ datasets, 11+ SOTA baselines, and multiple forecasting models (Transformers, MLPs, etc.). Ablation studies clearly dissect the contributions of decorrelation and task reduction, while sensitivity analyses justify design choices. The consistent improvements (e.g., Fredformer + TransDF reduces MSE by 0.016 on ETTh1) demonstrate broad applicability.\n\nWeaknesses:\n1. While Theorem 3.1 states the bias of TMSE under autocorrelation, its proof is relegated to Appendix without intuitive explanation in the main text. The connection between SVD-based decorrelation (Lemma 3.2) and bias elimination (§3.2) lacks formal derivation. A corollary linking Lemma 3.2 to Theorem 3.1’s bias term would strengthen theoretical rigor.\n2. TransDF introduces SVD on label sequences during training. While §E.1 (Appendix) mentions efficiency, the main text does not quantify its cost relative to forecast horizon T or compare it to FreDF/Fourier transforms. For long horizons, this could be non-trivial. A complexity analysis for SVD) and wall-clock time comparisons would clarify practical trade-offs.", "was_revised": true, "final_justification": "The author provided more detailed experimental results, which addressed my concerns.", "rebuttal": "We sincerely appreciate the reviewer for the positive evaluation and appreciation of our novelty, theoretical ground, rigorous experiments, and broad applicability. The raised two weaknesses are concrete and actionable. Please kindly check our responses and actions to address the two weaknesses. \n\n---\n\n\n\n#### [W1] *While Theorem 3.1 states the bias of TMSE under autocorrelation, its proof is relegated to Appendix without **intuitive explanation** in the main text. The connection between SVD-based decorrelation (Lemma 3.2) and bias elimination (§3.2) lacks formal derivation. **A corollary linking Lemma 3.2 to Theorem 3.1’s bias term** would strengthen theoretical rigor.*\n\n**Response.** We sincerely appreciate the rigorous and helpful suggestion. Below we add (1) an intuitive explanation to the proof of Theorem 3.1, and (2) a corollary demonstrating Lemma 3.2 eliminates bias in Theorem 3.1.\n\n- We provide an intuitive explanation of Theorem 3.1 in the main text (after line 93). The content is presented as the bold fonts below.\n> First, TMSE introduces bias due to autocorrelation. In time-series forecasting, observations exhibit strong dependencies on their past values, resulting in step-wise correlation in the label sequence. In contrast, TMSE treats the forecast of each step as an independent task, thereby neglecting these correlations. This misalignment causes TMSE to be biased relative to the true likelihood of the label sequence, as formalized in Theorem 3.1. **Intuitively, assuming the label sequence obeys a Gaussian distribution; the true joint likelihood should incorporate autocorrelation via off-diagonal elements in covariance matrix $\\Sigma$. In contrast, TMSE only considers diagonal elements in $\\Sigma$,  ignoring off-diagonal covariances, which makes it biased from the true likelihood except in cases where labels are uncorrelated ($\\Sigma$ is diagonal).**\n\n\n- We add a corollary linking Lemma 3.2 to Theorem 3.1’s bias term (after line 139).\n\n>**Corollary 3.4** Let $\\mathbf{Z} = \\mathbf{Y}\\mathbf{P}^*$ denote the decorrelated components in Lemma 3.2. Then, applying TMSE to $\\mathbf{Z}$ is equivalent to maximizing their joint likelihood without the bias in Theorem 3.1.\n\n>**Proof sketch.** By Lemma 3.2, the components $\\mathbf{Z}$ are decorrelated and have a diagonal covariance matrix $\\Sigma=I$. According to Theorem 3.1, the bias term of applying TMSE becomes:\n$\n\\mathrm{Bias} = \\|\\mathbf{Z} - \\hat{\\mathbf{Z}}\\|_{I}^{2} - \\|\\mathbf{Z} - \\hat{\\mathbf{Z}}\\|^{2} -\\frac{1}{2}\\log|I| = 0.\n$\nThus, applying TMSE to $\\mathbf{Z}$ provides an unbiased estimation of their likelihood. $\\square$\n\n\n\n#### [W2] *TransDF introduces SVD on label sequences during training. While §E.1 (Appendix) mentions efficiency, the main text does not quantify its cost relative to forecast horizon T or compare it to FreDF/Fourier transforms. For long horizons, this could be non-trivial. **A complexity analysis for SVD and wall-clock time comparisons would clarify practical trade-offs.***\n\n**Response.** Thank you very much for your actionable and informative suggestion. You raise a valid concern about the practical efficiency of our method, particularly for long forecast horizons. Below, we provide a comprehensive analysis on **both theoretical complexity and empirical wall-clock time** of TransDF. We also include **FreDF** for comparison.\n\n\nOur TransDF framework involves 3 stages, each with its own computational characteristics:\n\n- In the pre-processing stage, TransDF calculates the projection matrix $\\mathbf{P}$ via SVD. \n    - **Complexity analysis.** Suppose $\\mathbf{Y}\\in\\mathbb{R}^{\\mathrm{m\\times T}}$ is the normalized label sequence ($\\mathrm{m}$: sample size,  $\\mathrm{T}$: forecast horizon); we first calculate the covariance matrix as $\\mathbf{S}=\\mathbf{Y}^\\top \\mathbf{Y}\\in\\mathbb{R}^{\\mathrm{T\\times T}}$, where the complexity is $\\mathcal{O}(\\mathrm{T^2m})$. Then, we apply SVD to $\\mathbf{S}$ to obtain projection matrix $\\mathbf{P}$, where the complexity is the standard complexity of SVD: $\\mathcal{O}(\\mathrm{T^3})$ [1]. Therefore, the overall complexity is $\\mathcal{O}(\\mathrm{T^3}+\\mathrm{T^2m})$. In contrast, FreDF does not require pre-processing.\n    - **Wall-clock time analysis.** We investigate wall-clock time of the pre-processing stage on the Weather dataset given varying sample size (m) and prediction horizon (T). The results are presented below with two key notes. (1) the running time increases with m and T, as indicated in complexity analysis. (2) **The pre-processing step only executes once per training process, which makes the cost tolerate given largest m and T (less than 18 seconds)**.\n\n\n|  |   T=96         |   T=192         |   T=336         |   T=720          |\n|-----------|---------------|-----------------|-----------------|------------------|\n|  m=7200    | 0.399$_{\\pm0.002}$ | 0.700$_{\\pm0.002}$ | 1.361$_{\\pm0.002}$ | 3.182$_{\\pm0.002}$  |\n|  m=14400    | 0.740$_{\\pm0.002}$ | 1.697$_{\\pm0.003}$ | 3.157$_{\\pm0.006}$ | 6.605$_{\\pm0.008}$  |\n|  m=21600    | 1.198$_{\\pm0.004}$ | 2.735$_{\\pm0.003}$ | 4.771$_{\\pm0.007}$ | 10.211$_{\\pm0.010}$ |\n|  m=28800    | 1.785$_{\\pm0.002}$ | 3.693$_{\\pm0.004}$ | 6.450$_{\\pm0.003}$ | 13.796$_{\\pm0.011}$ |\n|  m=36000    | 2.351$_{\\pm0.003}$ | 4.632$_{\\pm0.008}$ | 8.159$_{\\pm0.005}$ | 17.392$_{\\pm0.004}$ |\n\n- In the training stage, during each iteration, TransDF calculates the transformed-domain loss using $\\mathbf{P}$. The complexity is $\\mathcal{O}(\\mathrm{BTK})$. In contrast, the complexity of FreDF is $\\mathcal{O}(\\mathrm{BT log T})$.\n    - **Complexity analysis.** Suppose $\\mathbf{P}\\in\\mathbb{R}^{\\mathrm{T \\times K}}$ is the projection matrix obtained in the pre-processing stage ($\\mathrm{B}$: the batch size, $\\mathrm{T}$: prediction horizon). We first generate latent components as $\\mathbf{Z}=\\mathbf{YP}$, $\\hat{\\mathbf{Z}}=\\hat{\\mathbf{Y}}\\mathbf{P}$, where the complexity is $\\mathcal{O}(\\mathrm{BTK})$. We then compute the L1-loss between $\\mathbf{Z}$ and $\\hat{\\mathbf{Z}}$, where the complexity is $\\mathcal{O}(\\mathrm{BK})$. Therefore, the overall complexity is $\\mathcal{O}(\\mathrm{BTK})$, and increases to  $\\mathcal{O}(\\mathrm{BT^2})$ under the full alignment case ($\\mathrm{T=K}$).\n    - In contrast, **FreDF** applies FFT to generate latent components, with complexity $\\mathcal{O}(\\mathrm{BT log T})$,  which is more efficient than TransDF's $\\mathcal{O}(\\mathrm{BTK})$ given $\\mathrm{K}>\\log\\mathrm{T}$.\n    - **Wall-clock time analysis.** We investigate wall-clock time of this stage on the Weather dataset. Without loss of generality, we set batch size (B) as 4096 and consider the full alignment case $\\mathrm{T=K}$. We compare the time for calculating loss of DF, FreDF and TransDF in the table below and summarize two findings. (1) **TransDF's per-batch overhead is marginally higher than FreDF**, consistent to the complexity analysis. (2) **The absolute magnitude of this overhead is negligible  (≤1.17ms per batch).**\n\n| Method | T=96              | T=192              | T=336              | T=720              |\n|----------|-------------------|--------------------|--------------------|--------------------|\n| DF       | 0.057$_{\\pm 0.004}$   | 0.052$_{\\pm 0.010}$    | 0.060$_{\\pm 0.004}$    | 0.078$_{\\pm 0.018}$    |\n| FreDF    | 0.172$_{\\pm 0.034}$   | 0.182$_{\\pm 0.025}$    | 0.199$_{\\pm 0.009}$    | 0.707$_{\\pm 0.421}$    |\n| TransDF  | 0.198$_{\\pm 0.034}$   | 0.204$_{\\pm 0.029}$    | 0.220$_{\\pm 0.003}$    | 1.170$_{\\pm 0.807}$    |\n\n- **In the inference stage, both FreDF and TransDF have the same complexity with DF.** They operate identically to DF, directly using the forecast model's output sequence with no extra computations.\n- In conclusion, the complexity of TransDF is ignorable in both training and inference stages. For the pre-processing stage that only executes once, the complexity is also limited. It ensures that TransDF remains scalable for long label sequence and large datasets.\n\n\n\n\n**Reference**\n\n\n[1] Tutorial: Complexity analysis of Singular Value Decomposition and its variants. arXiv preprint arXiv:1906.12085, 2019."}]

Author rebuttal:

Meta-review: This work proposes a novel training objective that focuses on the decorrelation of the label space and aligning the most informative components to mitigate label autocorrelation and reduce the task amount. The work tackles a very important problem and was well received by the 4 reviewers. The overall motivation, the model agnostic nature and the core idea of the solution was well appreciated in the initial reviews. There were some major points of contention which included:

* Lack of a theoretical justification of the framework.
* Limitations of the work not adequately addressed.
* SInce the model focuses on obtaining a static global projection this may fail to capture the inherent local dynamics that are unique (to an extent) within different data sets.

The rebuttal provided by the authors was pretty comprehensive and was well appreciated by the reviewers. I read the paper and feel that this is an important contribution and will benefit from being presented in the conference. I recommend acceptance and request the authors to incorporate the rebuttal in the final version of the manuscript.
Decision: Accept (poster)