FreDF: Learning to Forecast in the Frequency Domain
Authors: Hao Wang
Lichen Pan
Yuan Shen
Zhichao Chen
Degui Yang
Yifei Yang
Sen Zhang
Xinggao Liu
Haoxuan Li
Dacheng Tao
Venue: ICLR 2025 Poster (2025)
Keywords: Time series
Long-term Forecast
Primary area: learning on time series and dynamical systems
TL;DR: Learning to forecast in the frequency domain improves forecasting performance.

Abstract: Time series modeling presents unique challenges due to autocorrelation in both historical data and future sequences. While current research predominantly addresses autocorrelation within historical data, the correlations among future labels are often overlooked. Specifically, modern forecasting models primarily adhere to the Direct Forecast (DF) paradigm, generating multi-step forecasts independently and disregarding label correlations over time. In this work, we demonstrate that the learning objective of DF is biased in the presence of label correlation. To address this issue, we propose the Frequency-enhanced Direct Forecast (FreDF), which mitigates label correlation by learning to forecast in the frequency domain, thereby reducing estimation bias. Our experiments show that FreDF significantly outperforms existing state-of-the-art methods and is compatible with a variety of forecast models. Code is available at https://github.com/Master-PLC/FreDF.

Published as a conference paper at ICLR 2025
FREDF: LEARNING TO FORECAST IN THE FREQUENCY DOMAIN
Hao Wang'
Licheng Pan'
Yuan Shen'
Zhichao Chen
Degui Yang*
Sen Zhang 1 DDppartment of Control Science and Engineering, Zhejiang University
Yifei Yang
Xinggao Liu² Haoxuan Lis
Dacheng Tao 6*
Schooo of Automation, Central South University
3
DDpaarmeett of Computer Science and Engineering, Shanghai Jiao Tong University "Trust and Safety Team, Tik Sydney, ByteDance Inc.
CCenter for Data Science, Peking University
6 "Generative AI Lab, College of Computing and Data Science, Nanyang Technological University Ho w--wrd@ouutlok com hxli@stu. p.t..u.p.u........ edu. en dacheng. tacêntu. edu.sg sg
ABSTRACT
Time series modeling presents unique challenges due to autocorrelation in both his- torical data and future sequences. While current research predominantly addresses autocorrelation within historical data, the correlations among future labels are often overlooked. Specifically, modern forecasting models primarily adhere to the Direct Forecast (DF) paradigm, generating multi-step forecasts independently and disre- garding label autocorrelation over time. In this work, we demonstrate that the learn- ing objective of DF is biased the presence of label autocorrelation. To address this issue, we propose the Frequency-enhane enhanced Direct Forecast (FreDF), which mit- igates label autocorrelation by learning to forecast in the frequency domain, thereby reducing estimation bias. Our experiments show that FreDF significantly outper- forms existing state-of-the-ar methods and compatible with a variety of forecast models. Code is available at https: //g thub. com coomMMasee /Mt.tatasstccc//a/ FeDF.
INTRODUCTION
Time series modeling aims to utilize historical sequence to predict future data, which has been successfully applied various fields (Qiu et al., 2025), including long-term forecasting in weather prediction (Bi et al., 2023), short-term predictions in security (Yan et al., 2024), and data imputation in industrial maintenance (Wang et al., 2024b). A key challenge in time series modeling, distinguishing it from canonical regression tasks, is the presence of autocorrelation, which refers the dependence between time steps inherent in both the input and label sequences.
To accommodate autocorrelation in input sequences, diverse forecast models have been developed, exemplified by recurrent (Salinas et al., 2020), convolution (Wu et al., 2023) and graph neural networks (Yi et al., 2023a). Recently, Transformer-hased models, utilizing self-attention mechanisms to dynamically assess autocorrelation, have gained prominence (Liu et al., 2024; Nie et al., 2023). Concurrently, there is growing trend of incorporating frequency analysis into forecast models. By representing the input sequence in the frequency domain, input autocorrelation can be efficiently accommodated, which proves to improve the forecast performance of Transformers (Zhou al., 2022) and Multi-Layer Perceptrons (MLPs) (Yi et al., 2023b). These pioneering works highlight the importance of autocorrelation and frequency analysis in advanced time series modeling.
Another critical aspect is the autocorrelation within the label sequence, where each future step is autoregressively dependent on its predecessors. This phenomenon, termed as label autocorrelation, poses critical issue warranting investigation. Specifically, recent forecasting methods predominantly employ the Direct Forecast (DF) paradigm (Liu et al., 2024; Nie et al., 2023), which generates multi- step predictions simultaneously via multi-output head (Liu et al., 2022b), optimizing forecast emrors across all steps concurrently. However, this approach implicitly assumes step-wise independence in
"Corresponding author.

---

Published as a conference paper at ICLR 2025
the label sequence, overlooking the label autocorrelation inherent in the time series forecast task. We theoretically demonstrate that this oversight results in biased forecasts, revealing a significant defect with the existing DF paradigm.
To address this issue, we introduce the Frequency-enhaaced Direct Forecast (FreDF), a straightfor ward yet effective refinement of the DF paradigm. The central idea is to align the forecasts and label sequences in the frequency domain, where the label correlation is found to be effectively diminished. This method resolves the discrepancy between the scope of DF and the characteristics of actual time series, while retaining DF's advantages, such as sample efficiency and simplicity of implementation. Our main contributions are summarized as follows:
We uncover label autocorrelation critical yet underexplored challenge in modern time series modeling and theoretically justify how biases the learning objective of the prevalent DF paradigm.
We propose FreDF, a straightforward yet effective modification to the DF paradigm that learns to forecast in the frequency domain, thereby mitigating label autocorrelation and reducing bias. To our knowledge, this is the first effort to utilize frequency analysis for enhancing forecast paradigms.
We validate the efficacy FreDF through comprehensive experiments, demonstrating its ability to enhance the performance of state-of state-of-thh-arr forecasting models across diverse range of datasets.
2
PRELIMINARIES AND RELATED WORK
2.1 PROBLEM DEFINITION
In this study, uppercase letters (e.g., Y) denote random matrix, with subscripts (e.g., Yı,) indi- cating matrix entries. An uppercase letter followed by parentheses (e.g., Y(n)) represents an observation of the random matrix. A multi-variate time series can be represented as sequence (I), X (2), X()), where X(n) E 11IDD is the sample at the n-th timestamp with D covari- ates. Define input sequence L E IR' H x and label sequence Y E RTx where H and are sequence lengths. At arbitrary n-th step, these sequences are observed as L [X(n H + 1), .... (n)) and Y = (X X (n + T)]. The goal of time series forecast is identifying a model g RH - RTXD within model family G (e.g., decision trees, neural networks) that gener- ates the prediction sequence Y (L) approximating the label sequence Y.
There are two critical aspects to accommodate autocorrelation in time series modeling: (1) selecting model family & that encodes autocorrelation input sequences, which underscores the design of model architectures; (2) generating forecasts that respect label autocorrelation, which highlights the efficacy of forecast paradigms. Our survey concentrates on examining both aspects.
2.2 MODEL ARCHITECTURES
To exploit autocorrelation in the input sequences, variety of architectures have been developed (Qiu et al., 2024; Li et al., 2024c). Initial statistical methods include VAR (Watson, 1993) and ARIMA (As- teriou & Hall, 2011). Subsequently, neural networks gained prominence for their ability automate feature interaction and capture nonlinear correlations. Exemplars include RNNs (e.g.. DeepAR (Sali- nas et al., 2020), S4 (Gu al., 2021)), CNNs (e.g.. TimesNet (Wu et al., 2023)), and GNNs (c.g.. MTGNN (Mateos et al., 2019)), each designed to effectively encode autocorrelation. Current progress has reached debate between Transformer- vantages and limitations. Transformers (e.g.. PatchTST (Nie et al., 2023), i'Trnsformer (Liu et al., 2024)) offer significant scalability as data size increases but incur high computational costs; MLPs (e.g., DLinear (Zeng et al., 2023), TimeMixer (Wang et al., 2024c)) are generally more efficient but less effective in scaling with larger datasets and struggle to accommodate varying input lengths.
and MLP-based architectures, each with its ad-
An emerging approach is representing sequence in the frequency domain (Wu et al., 2021; 2025). This method, in comparison to modeling autocorrelation the temporal domain, manages autocorrelation effectively with limited cost. A prominent example is FedFormer (Zhou et al., 2022), which computes attention scores in the frequency domain, leading to improved efficiency, efficacy, and noise reduction capabilities. The success of this technique extends to various architectures like Transformers (Zhou et al., 2022; Wu et al., 2021). I). MLPs (Yi et al., 2023b) and GNNs (Yi et al., 2023a; Cao et al., 2020), which makes it versatile plugin in the design of neural networks for time series forecast.
2

---

Published as a conference paper at ICLR 2025
2.3 ITERATIVE FORECAST V.S. DIRECT FORECAST
There are two paradigms to generate multi-step forecast: iterative forecast (IF) and direct forecast (DF) (Liu al., 2022b). The IF paradigm follows the canonical requence-to-sequenc manner, which forecasts one step at a time and uses previous predictions as input for subsequent forecasts. This recursive approach respects label autocorrelation forecast generation, widely used by early-stage methods (Lai et al., 2018; Salinas ct al., 2020). However, IF suffers from high variance due to error propagation, which significantly impairs performance long-term forecasts (Taieb & Atiya, 2015). Therefore, modern works (Li et al., 2021) advocate the DF paradigm, which generates multi-step forecasts simultaneously using multi-output head, featured by fast inference, implementation ease and superior accuracy. Currently, DF has been a dominant paradigm, continuing to be employed in modern works (Wu et al., 2023; Liu et al., 2024).
Significance of this work. Our work refines the DF paradigm by performing forecasting in the frequency domain In contrast to recent advancements that incorporate frequency analysis within model architectures to manage input autocorrelation (Yi ct al., 2023a;b; Wang al., 2025), accelerate computation (Lange et al., 2021), and improve generation quality (Yuan & Qiao, 2024), our approach specifically focuses refining the loss function to mitigate the bias caused by label autocorrelation, which is an unexplored yet significant aspect in modern time series analytics.
PROPOSED METHOD
3.1
MOTIVATION
Autocorrelation is a fundamental characteristic of time series data, where each observation is highly dependent on previous ones (Zeng et al., 2023). This characteristic sets time series apart from other types of data and creates specific modeling challenges. To accommodate autocorrelation, various neural network architectures have been developed (Wu et al., 2021; Liu et al., 2024), which effectively model the autocorrelation in input sequence. However, label autocorrelation cannot be handled via the modification of neural architectures. To effectively manage label autocorrelation, it is necessary create learning objectives that specifically consider these dependencies.
Modern time series forecasting models are primarily trained under the multitask learning manner, known as the direct forecasting (DF) paradigm. Specifically, the DF paradigm employs multi-output model ge IR + RTxD to generate T T-step forecasts Y = (L)).. The model parameters 0 are optimized by minimizing the temporal loss:
T (tmmp) := ΣΣ
Y 2
Yi
(I)
In this learning objective, the temporal loss at each forecast step is computed independently, treating each future time step separate task. While this method has shown empirical effectiveness, overlooks the autocorrelation present within the label sequence Y. Specifically, the label sequence is autoregressively generated, with Y1+1 being highly dependent on Y,, illustrated by the blue arrows in Fig. ((a). In contrast, the learning objective in (1) assumes that each step in the label sequence can be independently modeled, as indicated by the black arrows in Fig. (a). This misalignment between the model's s assumptions and the data's s characteristics introduces bias into the learning objective of the DF paradigm, as demonstrated in Theorem 3.1.
Theorem 3.1 (Bias of DF). Given input sequence L and label sequence Y, the learning objective (1) of the DF paradigm is biased against the practical negative-log -egative-log-likelhhoodddd (NLL), expressed as:
i-1 PE -83) v3)) 2
T Bias- = Σ i=1
T
E1 - V.)2
Σ i=1
Vi P.
1 j=1
(2)
20
20 - Ø
where Y; indicates the prediction at the i-th step, Pij denotes the partial correlation between Y: and
Y, given L. p = => P..
'Given the inferior performance of the IF paradigm (Li et al., 2021), this paper advocates adapting the DF paradigm to handle label autocorrelation, rather than revisiting TF to directly model label autocorrelation.
3

---

Published as a conference paper at ICLR 2025
L
48
00
29
04
Cormettin modeld by DF.
ignored by DF:
sutoromiistion
144
(a)
(d)
Figure 1: Visualizing label autocorrelation in time series forecasting. (a) shows the generation process of time series with dependencies depicted as arrows. (b) shows the label correlation in the time domain, where each element Pt.j indicates the partial correlation between and Y, given L. (c-d) shows the label correlation in the frequency domain, where each element Pt,j indicates the partial correlation between F; and Fj given shown with the real (c) and imaginary part (d). Due the symmetry inherent in FFT, the forecast length in the frequency domain is halved.
According to Theorem 3.1, the presence of label autocorrelation Piy causes the loss to be biased against the NLL of the real data. Notably, this bias diminishes to zero when the labels are uncorrelated (pij = 0). Therefore, label autocorrelation is a crucial aspect for training time series forecast models.
3.2 REDUCE LABEL AUTOCORRELATION WITH FOURIER TRANSFORM
As established in Theorem 3, the bias in the learning objective decreases as label autocorrelation diminishes. To achieve this reduction, promising strategy is transforming the label sequence into representation where autocorrelation minimized. The Discrete Fourier Transform (DFT), defined in Definition 3.2, offers intuitive approach, which projects the sequence onto set of orthogonal exponential bases. In this transformed space, the label sequence is described as a linear combination of predefined temporal patterns that orthogonal, which effectively bypasses the autocorrelation in the time domain. The efficacy of this transformation in reducing label autocorrelation is formalized in Theorem 3.3, where different frequency components become decorrelated. Consequently, the reduced Pizy lowers the bias against the NLL, which benefits the training of time series forecast models.
Definition 3.2 (Discrete Fourier Transform, DFT), The normalized DFT of a sequence (Yo, ..., 1r is defined the projection onto cies. The projection for frequency l is computed as
orthogonal Fourier bases at different frequen-
T-1
(-sr 2nk T ()) /VT.
F = Σ exp 1=0
where is the imaginary unit, exp(-) is the Fourier basis for disferent i values. The DFT comprises the set of projections F [F,,.., FT-1), denoted as == FFY(), which can be computed the Fast Fourier Transform (FFT) algorithm with complexity OT T).
Theorem 3.3 (Decorrelation between frequency components). Let Y be a zero-mean, discrete-
wide-sense stationary random process of length As T + oc, the DFT coefficients become asymptotically uncorrelated at different frequencies:
(fkk.
if W 00 if + k.
lim B[F+FR] T-000
where ft f and (J) is the power spectral density Y
Case study. To validate our theoretical claims, we conducted a case study on the Weather dataset, illustrated in Fig. Immplementation details and additional evidence are provided in Appendix A. The main observations are summarized as follows:
Evidence of Label Autocorrelation: Fig. 1 (b) quantifies the partial correlations between different steps Y, and Y3 of the label sequence Y. conditioned the input T., A number of non-diagonal
4

---

Published as a conference paper at ICLR 2025
elements exhibit substantial values, with approximately 37.5% exceeding 0.3. This indicates that different time steps in Y are correlated conditioned on L, confirming the presence of label autocor- relation. Moreover, the autocorrelation displays regular variations, evidenced by alternating light and dark regions in Fig. 1 (b), suggesting a periodic nature the series. Such label autocorrelation makes the learning objective of the naive DF paradigm biased, established in Theorem 3.1.
Efffct of Domain Transformation: Fig. 1 (c-d) visualize the partial correlations between different frequency components of the transformed label sequence F. The majority of non-diagonal elements show negligible values, with only about 3.6% exceeding 0.1. This demonstrates that transforming the label sequence to the frequency domain significantly reduces the partial correlations between different components, corroborating Theorem 3.3. The reduction in label correlation Pizij leads to a decrease in the bias identified Theorem 3. underscoring the potential forecasting in the frequency domain for more accurate and unbiased predictions.
3.3 MODEL IMPLEMENTATION
This section introduces FreDF, an innovative ap- proach that enhances the vanilla Direct Forecast (DF) training paradigm. FreDF aligns forecast and label sequences within the frequency domain, ef- fectively mitigating the bias introduced by label autocorrelation.
Input 1 e 22100
g
PN
P RT
Pe CTDD
((mpp)
(
Y mm
F cTv
Label
F(-
F1)
gifr)
As illustrated in Fig. 2, the input sequence L is fed into the model generate T'-step forecasts, expressed as y g()). The temporal forecast error ((mmp) computed according (1). Subse- quently, both the forecast and label sequences are transformed into the frequency domain using FFT. The frequency forecast error is then calculated as:
gllop
Ortneogional (orta) time hossis
Imaginary part
Figure 2: The workflow of FreDF, Key oper- ations in the time and frequency domains are highlighted in red and blue, respectively.
C ((eg)
F(²) - F
(3)
where Y ERT_ TxD 1-1 denotes the element-wise f norm, summing the absolute values of all elements within the matrix. Since FFT is differentiable (Wu et al., 2021; Zhou et al., 2022), ((reg) can be optimized using standard stochastic gradient descent methods. We advocate the use of the & loss the frequency domain instead of the squared loss due to the numerical characteristics of the transformed label sequence. Specifically, different frequency components often exhibit vastly varying magnitudes; lower frequencies possess significantly higher amplitudes compared to higher frequencies, making the squared loss prone to instability. By using the & loss, we seek for a more balanced and stable optimization process.
Finally, the temporal and frequency forecast errors are fused, with the weighting parameter 0 a controlling the relative contribution of each error:
L = a ((fea) + (1-a) ((tm)).
By aligning the forecast and label sequences in the frequency domain, FreDF mitigates the bias caused by label autocorrelation while maintaining the advantages of the DFT, including efficient inference and multi-task learning capabilities. Additionally, FreDF is model-agnostic, compatible with various forecasting models expands the potential applications FreDF across diverse time series forecasting scenarios, where different forecasting models may demonstrate superior performance.
g (c.g.. Transformers and MLPs). This flexibility significantly
4 EXPERIMENTS
To demonstrate the efficacy of FreDF, there are six aspects empirically investigated:
1. Performance: Does FreDF work? Section 4.2 compares FreDF with state-of-the-ans baselines using public datasets. The long-term forecasting task is investigated in Section 4.2 and the short-term forecasting and imputation tasks are explored in Appendix 1.
5

---

Published as a conference paper at ICLR 2025
Table Long-term forecasting performance.
FreDF
Thussformrr (2024)
FETTS
TimesNet (2023)
MICN
DLinear (2023)
FEDfomer
Autofommr (2021)
Trassomner (2017)
(Ours)
(2025)
Metrics
MSE
MAE
MSE
MAE
MSB
MAD
MSE
MAE MSE
MAE
MSE
MAE]
MIE
MAB
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
ETTml
0.392
0,999
0.415
0,416
0.007
0.15
10413
0.418]
0.39
0.309 0.423
10,419
0,419]
00404
0.807 0.400
0,451
10.96
(0943
0,733
[0.99] 0.632
ETTm2
0.378
0.319|
0,294
0.335
(.335
0.379
(0.297 0.532
0,300 0.356
[0.358 0.404
(0344 0.396 0.302
0.345
(.535 0.366
1.322
0.314]34111
1.432
ETTal
0.437
0.435]
0.449
0.447
(0458
0.474
(0445
0.465]
0.525
0.515
[0,628
0.574
(0462
0:458
0,411 0.457
(0476
0.477
[0.93
0.788
J076
0.636
ETTh2
0.371
0.396]
0.250
0.410
0.550
0,515 [0:413
0.426
0.024
0.024
0.519
[0,811
0,550
(.558 0.536 0,400
0.447
J0,478 0.483
3.290
1,119
3,325
1,445
0.250|
0.126
0.267
(0:200
0,207 (0:314 0,377
[0.25] 0.544
(0225 0.319 0.229
0,339
(0.258 0.339
(0.24
0.367
10617 0,908
Traffic
0.421
0.279
0.438
0.286
(0552 0.348
(0.535
0.309
0.666
0.355
(0.700
0.473
(.673 0.419
0.111 0.379
0.379
(0633 0.399
[,680
0.378
1.001
0.652
Weather 0.254 0.274]
0.281
0.281
0.901
(.255 0.290
10:262
0.296]
0.261 0.319
10.21
0.500
9,205 0.377 0311 0.301
J0.59 0.991
(0032 0.52
J0.84 0.92
PEMSUD
0229/0100 0.11 0.226|
(4146 0257
1o:118 0.223 6am9
10:316 0.30
0.134
0,002 (0.51 a.533 10:36 0:23 p0,66 0664
PEMSOR 0.541 0,23600.19 (.159 0,258 (0174 0.272 10.15 0.217 00777 0.59 J0.319 0.388 0.377 0252 0.322 (6660 0.52 (0:40 0:365 J0.73 0.69
Note: We lix the input length as 96 following the established benchnark (Liu et al., 2024), Bold typeface highllights the top performance for each metric, while underlined text denotes the second-hest results, The results are averaged over Forecast lengths (96, 192, 336 and 720), with full results in Table 5.
2. Mechanism: How does it work? Section 4.3 offers an ablative study to dissect the contributions of FreDF's individual components, elucidating their roles enhancing forecasting accuracy.
3. Generality: Does it support other forecasting models? Section 4.4 verifies the adaptability of FreDF across different forecasting models, with additional results documented in Appendix E.2.
4. Flexibility: Does it support alternative transformations to FFT? Section 4.4 replaces FFT with other transformations to showcase its flexibility of implementation.
Sensitivity: Does it require careful fine-tuning? Section 4.5 presents sensitivity analysis of the hyperparameter a, where FreDF maintains efficacy across a broad range of parameter values.
6. Efficiency: Is FreDF effective given limited samples? Section 4.6 offers learning curve analysis, where FreDF achieves comparable performance with limited samples to that obtained using substantially more time-domain labels, indicating an advantageous sample efficiency.
4.1 SETUP
Datasets. The datasets for long-term forecast and imputation include ETT (4 subsets), ECL, Traffic, Weather and PEMS (Liu et al., 2024). The dataset for short-term forecast is M4 following Wu et al. (2023). Each dataset is divided chronologically for training, validation and test. Detailed dataset descriptions are provided in Appendix D.I.
Baselines. Our baselines include various established models, which can be grouped into three categories: (1) Transformer-basee methods: Transformer (Vaswani al., 2017), Autoformer (Wu et al., 2021), FEDformer (Zhou et al., 2022), iTransformer (Liu et al., 2024); (2) MLP-based methods: DLinear (Zeng al., 2023), TiDE (Das al., 2023), FreTS (Yi et al., 2023b); (3) other notable models: TimesNet (Wu et al., 2023), MICN (Wang et al., 2023b), TCN (Bai et al., 2018).
Implementation. The baseline models are reproduced using the scripts provided by Liu et al. (2024). They are trained using the Adam (Kingma & Ba, 2015) optimizer to minimize the MSE loss. Datasets are split chronologically into training, validation, and test sets. Following the protocol outlined in the comprehensive benchmark (Qiu ct al., 2024), the dropping-last trick is disabled during the test phase. When integrating FreDF to enhance an established model, adhere to the associated hyperparameter settings in the public benchmark (Liu et al., 2024), only tuning a and learning rate conservatively. Experiments are conducted on Intel(R) Xeon(R) Platinum 8383C CPUs and NVIDIA RTX 3090 GPUs. More implementation details are provided in Appendix D.2.
4.2 OVERALL PERFORMANCE
The performance the long-term forecast task is presented in Table 1, where select iTransformer as the forecast model and enhance it with FreDF. Overall, FreDF improves the performance of (Transformer substantially. For instance, on the ETTml dataset, FreDF decreases the MSE

---

Published as a conference paper at ICLR 2025
TTranfoormr
Irransformer FreDF
Trranfoormer
iTrnnformer & FreOF
0.50
0.50
50
GroundTrup
80
Groundrruh
0.25
0.25
00
Pred ctann
50
Prediction
0.00
U00
0822
1025
40
40
0.50
0050
Prediction
Prediction
40 75
-0.75
300
400
(a)
(b)
(c)
(d)
Figure 3: Visualization of forecast sequence generated with and without FreDF in the time (a-b) and frequency (c-d) domains, using the iTransformer as the backbone model.
Table 2: Ablation study results.
Model
e (tmp)
((fgg)
Data
T=720
Avg
MSB
MAE
MSE
MAE
MSE
MAE
MAE
MAE
ETTml
0.346
0.379
0.391
0.400
0.426
0.422
0.493
0.460
0.414
0.415
ETTh1
0.390
0.409
0.442
0.440
0.479
0.457
0.483
0.479
0.446
DF
x
ECT
0.147
0.233
0.166
0.258
0.178
0.271
0.209
0.298
0.175
0.266
Weather
0.201
0,246
0.250
0.282
0.302
0.317
0.361
0.302
ETTml
0.324
0.361
0.374
0.387
0.403
0.405
0.468
0.443
0.392
0.399
ETTh1
0.380
0.399
0.429
0.425
0.474
0,451
0.467
0.464
0.435
FreDFf
X
BCL
0.144
0.232
0.158
0.247
0.171
0.262
0.204
0.291
0.169
0.258
Weather
0.165
0.255
0.295
ETTml
0.324
0.362
0,372
0.385
0.402
0.404
0.468
0.443
0.391
0,398
ETTh1
0.381
0.400
0.430
0.426
0.474
0.451
0.463
0.461
0.437
0.435
FreDF
ECL
0.144
0.233
0.158
0.247
0.172
0.263
0.204
0.293
0.169
0.259
Weather
0.163
0.293
0.273
of iTransformer by 0.019. Similar gains are evident in other datasets, which can be attributed to reconciliation of label autocorrelation with the DF paradigm, validating efficacy of FreDF.
Moreover, FreDF enhances the performance of iTransformer to surpass even those models that originally outperformed ITransformer on some datasets. It indicates that the improvements by FreDF exceed those achievable through dedicated architectural design alone, emphasizing the importance of handling label autocorrelation and FreDF.
Showcases. We visualize the forecast sequences to highlight the improvements of FreDF in forecast quality. An ETTm2 snapshot with T=336 is depicted in Fig. 3. Although the model without FreDF can follow the general trends of the label sequence, it struggles capture the sequence's 's high-frequency components, resulting in a forecast with a visibly lower frequency. Additionally, the forecast sequence exhibits numerous burrs. These issues reflect the limitations of forecasting in the time domain, namely difficulty capturing high-frequency components and the neglect autocorrelation between sequential steps. FreDF addresses these limitations effectively. The forecasts generated under FreDF not only keep pace with the label sequence, accurately capturing also exhibit smoother appearance with fewer irregularities, due its awareness of autocorrelation.
components, but
Table 3: Varying FFT implementation results.
ETThl
ETTml
ECL
Model
MSB
MAE
MSE
A
MAB
A
MSE
A
MAE
A
TTransfommer
0.449
0.447
0.415
0.416
0.176
0.267
FreDF-T
D.477
2.633
0.435
2.66%
0.392
5.44%
0.399
1 4.01%
0.170
1 3.4.%
0.259
2.77%%
FreDF-D
0.45
0.92%
0.440
1.42%
0.395
4.77%
0.398
4.33%
0.171
2.51%
0.260
2.52%
FreDF-2
0.432
3.94%
0.431
3.57%
0.392
5.60%
0.399
4.05%
0.166
5.32%%
0.256
1 4.20%
Note: A denotes the relative enorr reduction compared to iTransformer with DF paradigm.
7

---

Published as a conference paper at ICLR 2025
29%
7.0%
12.95
MAE
MAE
0.5%%
AMto
Trans
Trans
(a) ECL with MSF
(b) ECL. with MAF
Weather with MSF
(d) Weather with MAE
Figure 4: Benefit of incorporating FreDF in varying models, shown with colored bars for means over forecast lengths (96, 192, 336, 720) and error bars for 99.99 99.9% confidence intervals.
4.3 ABLATION STUDIES
In this section, we dissect the contributions of the temporal and frequency loss for enhancing forecast performance. The results are detailed in Table 2, where iTransformer used as the forecast model. Overall, the frequency loss consistently improves performance compared the temporal loss. The rationale is that label autocorrelation can be effectively managed in the frequency domain, aligning better with the conditional independence assumption inherent in DF. Moreover, learning to forecast in both domains generally showcase improvement compared to relying solely on one domain. However, the improvement over ((fen) is marginal. Hence, exclusively focusing on frequency domain forecasting emerges viable strategy in most cases, offering promising performance without the complexity of balancing learning objectives.
4.4 GENERALIZATION STUDIES
In this section, we investigate the utility of FreDF with different forecast models and domain transformation strategies, showcase the generality of FreDF. In the bar-plots, the forecast errors averaged over forecast lengths (96, 192, 336, 720), with error bars as 95% confidence intervals.
Varying forecast models. We explore the versatility of FreDF in augmenting representative neural forecasting models: iTransformer, DLinear, Autoformer, and Transformer. FreDF demonstrates significant enhancements across these models compared the traditional DF paradigm, illustrated in Fig. 4. Notably, Transformer-based models such as the Autoformer and Transformer substantially benefit from the integration of FreDF. On the ECL dataset, for instance, the Autoformer (developed in 2021) enhanced by FreDF outperforms DLinear (developed in 2023). More evidence of FreDF's versatility is provided Appendix These results confirm FreDF's potential plugin-and -play strategy to enhance various time series forecasting models.
Varying FFT implementations. We note that label autocorrelation exists between not only different steps, but also variables in multivariate forecasting. Therefore, we implement FFT along the time (FreDF-T) and variable dimension (FreDF-D) handle the corresponding correlations, with the outcomes illustrated in Table 3. In general, conducting FFT along the time and variable axis brings similar performance gain, which showcases the existence of correlation between different steps and variables, respectively. In particular, FreDF-T slightly outperforms FreDF-D, which underscores the relative importance of auto-correlation in the label sequence. Finally, strategic approach image, performing 2-dimensional FFT both time and
viewing the multivariate sequence variable axes (FreDF-2), which accommodates the correlations between both time steps and variables simultaneously and further improves performance.
Varying transformations. Motivated by the fact that FFT can be viewed as projections onto exponential bases, we extend the implementation of FreDF by replacing FFT with projections onto other established polynomials. Each polynomial set is adept at capturing specific data patterns, such as trends and periodicity, which challenging to learn in the time domain. The results summarized in Fig. 5. Notably, projections onto Legendre and Fourier bases demonstrate superior performance. This superiority is attributed to the orthogonality of the polynomials, feature not guaranteed by
8

---

Published as a conference paper at ICLR 2025
Radler
Ligends
0.43
Layaske Ciabyaar
0.40
Ligesds
0.11
MAE
0.42
0.40
0.40
0.34
0.3
0.22 04
0.8 022
04
ETTh with MSE
(h) ETTh1 with MAE
(c) ETTml with MSE
(d) ETTml with MAE
Figure 5: Varying projection bases results, shown with colored bars for means forecast lengths
(96, 192, 336, 720) and error bars for 99.9% confidence intervals.
-B- 152
338
192
336
-
152
336
0.42
-B-
192
336
0.41
MSC
MAE
0.25
MIE
MAE
0.40
0.38
0.39
030.108.3044.000 000.2/200000007700000000
0.00 1100000000000000000
0.00 1001820.30 0001000300050005005555
ECL with MSE
(b) ECL with MAE
(c) ETTml with MSE
(d) ETTml with MAE
Figure 6: Varying strength of frequency loss (a) results, shown with colored lines for T=192, 336.
others analyzed in Appendix C. It underscores orthogonality when selecting polynomials for implementing FreDF, which is pivotal for eliminating autocorrelation.
4.5 HYPERPARAMETER SENSITIVITY
The key hyperparameter of FreDF is the frequency loss strength a. The performance given different is summarized in Fig. 6. Overall, increasing a from 0 to I results in reduction of forecast error, albeit with a slight increase towards the end of this range. For instance, on the ECL dataset with T=192, both MAE and MSE decrease from approximately 0.258 and 0.167 to 0.247 and 0. 158, respectively. Such trend of diminishing error seems consistent across different forecast lengths and datasets, supporting the benefit of learning to forecast in the frequency domain. Notably, the optimal reduction in forecast error typically occurs at a values near I, such as 0.8 for the ETThl dataset, rather than the absolute value of 1. Therefore, unifying supervision signals from both time and frequency domains brings performance improvement. Similar trends are presented across different datasets and foreacst models, as discussed in Appendix E.3.
4.6 LEARNING-CURVE ANALYSIS
In this section, we investigate the sample efficiency of learning in the time versus frequency domains, with the corresponding learning curves in Fig. 7. Overall, given limited training data, learning in the frequency domain demonstrates remarkable efficacy. With only 30% of the training data, it achieves per- formance comparable to learning the time domain using the full training dataset.
0.52
0.45
52
0.4
0.5
0.8
1.0
0.2
0.4
0.5
1.0
Data Percantage
Parcantage
(b)
The underlying reason for this enhanced sample ef- ficiency can be attributed to the consistent and more straightforward nature of the data representation. For instance, sliding window However, in the frequency domain, these sequences present a similar pattern: a prominent spike at a specific frequency and negligible values elsewhere. This uniformity simplifies the learning process by
Figure 7: Learning curve ETTml dataset.
sine signal yields set of distinct sequences in the time domain.

---

Published as a conference paper at ICLR 2025
making patterns more consistent and easier to decipher, thus reducing the need for extensive training datasets.
5 CONCLUSION
In this study, we underscore the challenge of label autocorrelation in time series modeling, which biases the learning objective of the widely adopted DF paradigm. To tackle this challenge, we introduce model-agnostic learning objective: FreDF, which mitigates label autocorrelation by transforming the label sequence into the frequency domain, thereby effectively reducing the bias caused by label autocorrelation. The experiments demonstrate that FreDF effectively enhances the performance of prevalent forecast models.
Limitation & future works. In this work, we primarily utilize the Fourier transform for domain transformation. Despite empirical efficacy, the predefined set of exponential bases lacks the ability to adapt specific data properties. Alternative transforms such independent component analysis can produce orthogonal bases considering data properties, representing a valuable avenue for future research. Additionally, the issue of label autocorrelation extends beyond time series, affecting diverse contexts involving structural labels, such as 3D point clouds, speech, and images, The potential of FreDF to enhance performance these contexts awaits further exploration.
ACKNOWLEDGEMENT
This work was supported by National Natural Science Foundation China (623B2002, 12075212 The first author extends heartfelt gratitude to Prof. Degui Yang of Central South University, for his exceptional signal processing lectures and generous research guidance during S.T.E.M. studies.
REFERENCES
Dimitros Asteriou and Stephen G Hall. Arima models and the box- box-jenkins methodology. Appl. Econ., 2(2):265-286, 201
Shaojie Bai, Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint D.Xiv:8003.01277 2018.
Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium- range global weather forecasting with 3d neural networks. Nature, 619(7970)5533-538 2023.
Michela Bia, Martin Huber, and Lukáš Lafférs, Double machine learning for sample selection models.
J. Bus. Econ. Stat., 42(3):958-966 2024.
Defu Cao, Yujing Wang. Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, et al. Spectral temporal graph neural network for multivariate time-series forecasting. In Proc. Ady. Neural Inf. Process. Syst., volume 33, pp. 17766-17778, 2020.
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whit- ney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters: Double/debiased machine learning. Econom. J., 21)1), 2018
Abhimanyu Das, Weihao Kong, Andrew Leach, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXhv:2304.08424, 2023.
Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In Proc. Int. Conf. Learn. Represent., 2021.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Proc. Int. Conf. Learn. Represent., 2015.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and and short-term temporal patterns with deep neural networks. In SIGIR, 2018
10

---

Published as a conference paper at ICLR 2025
Henning Lange, Steven L Brunton, and J Nathan Kutz. From fourier to koopman: Spectral methods for long-term time series prediction. J. Mach. Learn. Res., 22(41): 2021.
Vincent Le Guen and Nicolas Thome. Shape and time distortion loss for training deep time series forecasting models. Proc. Adv. Neural Inf. Process. Syst., volume 32, 2019.
Vincent Le Guen and Nicolas Thome. Probabilistic time series forecasting with shape and temporal diversity. In Proc. Adv. Neural Inf. Process. Syst., volume 33, pp. 4427-4440, 2020.
Haoxuan Li, Kunhan Wu, Chunyuan Zheng, Yanghao Xiao, Hao Wang. Zhi Geng, Fuli Feng, Xiangnan He, and Peng Wu. Removing hidden confounding in recommendation: unified multi-task learning approach. Proc. Adv. Neural Inf. Process. Syst., 36:54614-54626 2024a.
Haoxuan Li, Chunyuan Zheng, Shuyi Wang, Kunhan Wu, Eric Wang, Peng Wu, Zhi Geng, Xu Chen, and Xiao-Hua Zhou, Relaxing the accurate imputation assumption in doubly robust learning for debiased collaborative filtering. In Proc. Int. Conf. Mach. Learn., volume 235, pp. 29448-29460, 2024b.
Jianxin Li, Xiong Hui, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proc. AAAI Conf. Artif. Intell., 2021.
Zhe Li, Xiangfei Qiu, Peng Chen, Yihang Wang, Hanyin Cheng, Yang Shu, Jilin Hu, Chenjuan Guo, Aoying Zhou, Qingsong Wen, al. Foundts: Comprehensive and unified benchmarking of foundation models for time series forecasting. arXiv preprint arXiv:2410.1 1802, 2024c.
Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: time series modeling and forecasting with sample convolution and interaction. In Proc. Ady. Neural Inf. Process. Syst., 2022a.
Shiyu Liu, Rohan Ghosh, and Mehul Motani. Towards better long-range time series forecasting using generative forecasting. CoRR, abs/2212.06142. 2022b.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma. and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In Proc. Int. Conf. Learn. Represent., 2024.
Gonzalo Mateos, Santiago Segarra, Antonio G. Marques, and Alejandro Ribeiro. Connecting the dots: Identifying network structure via graph signal processing. IEEE Signal Process. Mag., 36(3): 16-43, 2019.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In Proc. Int. Conf. Learn. Represent., 2023.
Xiangfci Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian Jensen, Zhenli Sheng, and Bin Yang. Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow, (7(9):2333-2377) 2024.
Xiangfei Qiu, Xiuwen Li, Ruiyang Pang, Zhicheng Pan, Xingjian Wu, Liu Yang. Jilin Hu, Yang Shu, Xuesong Lu, Chengeheng Yang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, and Bin Yang. Easytime: Time series forecasting made easy. In Proc. IEEE Int. Conf. Data Eng., 2025.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. Int. J. Forecast, 36(3):1181-119)) 2020.
Amin Shabani, Amir Abdi, Lili Meng, and Tristan Sylvain. Sealeformer: Iterative multi-scale scale refining transformens for time series forecasting. In Proc. Int. Conf. Learn. Represent. 2022.
Souhaib Ben Taieb and Amir F Atiya. A bias and variance analysis for multistep-ahead time series forecasting. IEEE Trans. Neural. Netw. Learn. Syst., 27(1):2 2015.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proc. Adv. Neural Inf. Process. Syst., 2017.
11

---

Published as a conference paper at ICLR 2025
Hao Wang, Zhichao Chen, Jiajun Fan, Haoxuan Li, Tianqiao Liu, Weiming Liu, Quanyu Dai, Yichao Wang, Zhenhua Dong, and Ruiming Tang. Optimal transport for treatment effect estimation. In Proc. Adv. Neural Inf. Process. Syst., 2023a.
Hao Wang, Zhichao Chen, Zhaoran Liu, Haozhe Li, Degui Yang, Xinggao Liu, and Haoxuan Li. Entire space counterfactual learning for reliable content recommendations. IEEE Trans. Inf. Forensics Security, pp. 1-12, 2024a.
Hao Wang. Xinggao Liu, Zhaoran Liu, Haozhe Li, Yilin Liao, Yuxin Huang, and Zhichao Chen. Lspt-d: Local similarity preserved transport for direct industrial data imputation. IEEE Trans. Autom. Sci. Eng., 2024b.
Hao Wang, Zhengnan Li, Haoxuan Li, Xu Chen, Mingming Gong, Bin Chen, and Zhichao Chen. Optimal transport for time series imputation. In Proc. Int. Conf. Learn. Represent. pp. 1-9, 2025.
Huiqiang Wang, Jian Peng, Feihu Huang. Jince Wang, Junhui Chen, and Yifei Xiao. Mien: Multi- scale local and global context modeling for long-term series forecasting, In Proc. Int. Conf. Learn. Represent. 2023b.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. In Proc. Int. Conf. Learn. Represent., 2024c.
Mark W. Watson. Vector autoregressions and cointegration. Working Paper Series, Macroeconomic
Issues, 4, 1993.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with Cuto-Correlatton for long-term series forecasting. In Proc. Adv. Neural Inf. Process. Syst., 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long, Timesnet: Temporal 2d 2d-variation modeling for general time series analysis. In Proc. Int. Conf. Learn. Represent. 2023.
Xingjian Wu, Xiangfei Qiu, Zhengyu Li, Yihang Wang, Jilin Hu, Chenjuan Guo, Hui Xiong, and Bin Yang. Catch: Channel-aware multivariate time series anomaly detection via frequency patching. In Proc. Int. Conf. Learn. Represent. 2025.
Feng Yan, Chunjie Yang, Xinmin Zhang, Chong Yang, and Zhiyong Ruan. Btpnet: A probabilistic spatial-temporal aware network for burn-through point multistep prediction in sintering process. IEEE Trans. Neural Netw. Learn. Syst., 2024.
Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from pure graph perspective. In Proc. Adv. Neural Inf. Process. Syst., 2023a.
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domait mips are more effective learners in time series forecasting. In Proc. Adv. Neural Inf. Process. Syst., 2023b.
Xinyu Yuan and Yan Qiao. Diffusion-ts: Interpretable diffusion for general time series generation. In Proc. Int. Conf. Learn. Represent., 2024.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proc. AAAI Conf. Artif. Intell., 2023.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proc. Int. Conf. Mach. Learn., 2022.
12

---

Published as a conference paper at ICLR 2025
A OVERVIEW OF DML FOR PARTIAL CORRELATION ESTIMATION
A. MOTIVATION
In this section, we introduce the rationale for employing double machine learning (DML) to quantify the partial correlations. Our focus is on the autocorrelation represented by Y; + Ye where 0 < < T. However, the fork structure Y < [(n) + Yv creates pseudo correlation between Yr and Y, (Wang et al., 2024a). In this case, the autocorrelation Y, + Yg is influenced by the pseudo correlations from the fork structure, rendering traditional correlation measures, such as Pearson correlation, ineffective for quantifying the autocorrelation Y, + Y (Li et al., 2024a;b).
To effectively address this influence and quantify partial correlation, it is essential employ methods that excel in distinguishing direct relationships from spurious ones (Wang et al., 2023a). DML chosen for calculating partial correlation due to its case of implementation and independence from exhaustive hyperparameter tuning. DML offers robust and reliable quantification of the autocorrelation that we care about (Bia et al., 2024; Chernozhukov et al., 2018).
A.2 METHOD
In this section, we detail the implementation of DML, a two-step procedure designed for estimating partial correlation. We defin T E IR as the treatment variable, y E IR as the outcome variable, X' RD as the control variable that needs depicted in Fig. 8 (b) which consists of two steps below.
accounted for. The implementation of DML is
Orthogonalization, This step involves orthogonalizing both the outcome ()) and the treatment (T) with respect to the control variables (-Y). To this end, we first use two machine learning models, namely and d, to predict the outcome and the treatment based on X. These predictions aim to capture the components in V and T that are influenced by X. Subsequently, such impact of X can be eliminated by calculating the residuals:
5 = o(((),
(5)
+ = ((()).
Regression. This step involves regressing the orthogonalized outcome I on the orthogonalized treatment T. A linear regression model is utilized for this purpose:
5 3T +
(6)
where is the error term; 8 the model coefficient that be identified via ordinary least squares. The 3 can be identified in a supervised learning manner, with the objective of minimizing the MSE between the prediction and real values. The identified 3 quantifies the partial correlation between the treatment and the outcome, having accounted for the influence of X.
By regressing the orthogonalized outcome on the orthogonalized treatment, DML captures the direct effect of the treatment on the outcome without the interference from control variables, as depicted in Fig. 8 (c). That is, DML isolates the desired partial correlation T + y from the influencing correlation X + y
A.3 EXPERIMENTAL SETTINGS
In this section, outline the experimental settings implemented to employ DML for quantifying the correlations of interest.
General settings. For the base learners & and v. we opt for a linear regression model optimized using ordinary least squares for its efficiency Following Appendix A. we treat the input sequence as the control variable to adjust, and simplify the process by considering the last step in as representative. Moreover, we focus exclusively on the correlations within the last feature of each
2The linear regression model, chosen for its computational efficiency, is crucial in managing the experiment's scale, where the total number of DML estimators can be exceedingly high (e.g., 36,864 for T=192).
13

---

Published as a conference paper at ICLR 2025
X
T
10-)
8
6
D
Pseudo correlation from confounder X
y
(a)
(h)
(c)
Figure 8: Visualization of partial correlation and DML approach for partial correlation quantification.
(a) The correlation graph where the pseudo correlation is caused by the fork structure T + X + V.
(b) The implementation of DML. where 3 is the identified strengtth of the partial correlation T - y
(c) The partial correlation identified by DML.
'ataset'. This focus makes Y a scalar value within the real number space rather than a D-dimensional vector in this experiment.
Specifications for identifying time-domain partial correlation. To assess the partial correlation Y; > Yr, we treat Y, the treatment and Y/ the outcome. The DML model is trained using a set of N observations: ( (L(n))n=1:N, (Yr(n) (=)n))_=:N. and {Ye (N) I=INN. The coefficient & derived from the DML model is interpreted as the strength of the partial correlation Y, Yr.
Specifications for identifying frequency-domain partial correlation. To quantify the partial correlation F& > Fk', we treat F& as the treatment and Fg as the outcome. The DML model is trained using set of N observations: [[()))==::N, (n))),n=1,, and {Fer (n)))==:NN. The coefficient 3 derived from the DML model is interpreted as the strength of the partial correlation Fk + Fv. A notable complexity arises because F& is complex number. Since DML is typically designed for real numbers instead of complex numbers, it requires a separate consideration of the real and imaginary parts of Fk.
A.4 MORE EXPERIMENTAL RESULTS
In this section, provide comprehensive results of the identified partial correlation strengths, which quantifies the autocorrelation effect in the time and frequency domain. Fig. presents the results on three different datasets: Traffic, ETTh1, and ECL, with forecast length set to 192. Fig. 10 presents the results for varying forecast lengths: 48, 96, 192, 336, on the ECL dataset.
The results show similar patterns those in the main text. Specifically, the non-diagonal elements in Fig. 9 (a-c) and Fig. 10 (a-d) often exhibit huge values, which affirms the presence label autocorrelation in the time domain. In contrast, the non-diagonal elements in Fig. 9 (d-i) and Fig. 10 (e-l) show negligible values, which suggests that frequency components of F are almost independent given L. These findings collectively verify (1) the existence of label autocorrelation in the time domain; (2) the mitigation of label correlation in the frequency domain.
Thhis focus is aligned with the study's 's objective of analyzing autocorrelation instead of inter-feature correla- tions, which simplifies the interpretation of results.
14

---

Published as a conference paper at ICLR 2025
0
1.0
0.8
0.8
0.8
48
48
72
96
96
120
120
0,4
0,4
144
144
144
0.2
0.2
168
168
192
192
24
48
96
120
144
168
102
24
72
96
120
168
24
72
120
168
192
(a)
(c)
100
1.
1.0
12
0.8
24
24
24
48
48
60
60
72
0.2
0.2
84
84
84
0.0
0.0
0.0
24
84
48
72
84
96
26
72
84
96
12
12
12
0.8
36
06
36
0.6
36
06
48
46
0.4
0.4
0.4
72
72
72
0.2
0.2
96
96
0.0
24
36
80
84
12
48
72
96
12
36
48
80
72
96
(h)
Figure 9: More comprehensive visualizations of label autocorrelation different domains and datasets, with columns representing different datasets: Traffic, ETTh1, and ECL, from left to right. Panels (a-c) show the label correlation in the time domain, where each element Pi,j indicates the partial correlation between Y; and Y, given L. Panels (d-i) show the label correlation the frequency domain, where each element Pi,j indicates the partial correlation between F; and F, given L, shown with the real (d-f) and imaginary part (g-i). Due to the symmetry inherent in FFT, the forecast length in the frequency domain is halved.
15

---

Published as a conference paper at ICLR 2025
42
0.8
06
08
05
08
0.4
24
128
02
02
84
000
192
86
100
100
190.
0.0
330
310
206
356
00
(b)
(c)
1.0
12-
08
00
0.0
84
0.4
00
04
42
02
0.
0.0
108
-00
(e)
(h)
10
1.0
12
0.
06
06
69
04
100
62
02
35
455
(i)
(i)
(k)
(I
Figure 10: More comprehensive visualizations label autocorrelation in different domains and label lengths, with columns representing label lengths H=48, 96, 192, 336 from left to right. Panels (a-d) show the label correlation in the time domain, where each element Pt.j indicates the partial correlation between Y; and Y, given L. Panels (e-I) show the label correlation the frequency domain, where each element Ps.j indicates the partial correlation between F; and Fy given L, shown with the real (e-h) and imaginary part (i-l).
16

---

Published as a conference paper at ICLR 2025
B THEORETICAL JUSTIFICATION
Theorem B.1 (Bias of vanilla DF, simplified). Given an input sequence L and a univariate label sequence Y 1Y,, Yz] (the forecast length is 2 for simplicity), the learning objective (1) of the DF paradigm is biased against the practical NLL, expressed as:
1 Bias 20 2V2 -2)2
1 20 (1 pp) (Y2
(82 P(i -i)))²
(7)
where Y; indicates the prediction at the i-th step and p denotes the partial correlation between Yi and 12 given L.
Proof. Aligning with the maximum likelihood analysis, we assume the label sequence obeys
normal distribution with mean [Y1 Y2] and covariance Ç = [[o po [po?, [ol] The negative log-likelihood (NLL) of given the input sequence can be expressed as
log L) log p(P L)
log (())
(Yi - (81-11)²
log(
exp(
V 2TT
202
(Y2 - -P/
p 1 -( )).
log(
cxp(
22 p2)
20 (pp))
Removing coefficients unrelated to the practical NLL that contributes the gradients to update is
1 NLL == 20 (Yi -1)= 20 p2) (12 (8/ p(vi - R1))
If the independence assumption of diffferent time step holds (i.e., Y1 and Y/ are conditionally independent given L),, we have & = 0, followed by P(Y/LL, Y1) LY) In this case, the MSE loss canonical DF mirrors the practical NLL:
1 MSE
20 (x 21)²
20 (Y2 Y22
where is often set to 1 when implementing MSE. If the independence assumption does not hold, ie., considering autocorrelation the label sequence, we have p * O. In this case, the MSE loss in the time domain is biased to the practical NLL, expressed as:
1 Bias 20 (Y22 22) 20 12) (Y2 - (82 p -1))
This bias introduced by label autocorrelation makes the MSE loss in the time domain fail to reflect the practical NI.L and therefore misleads the update of forecast model g under DF paradigm. Il
Theorem B.2 (Bias of vanillia DF). Given an input sequence L and a univariate label sequence Y, the learning objective (1) of the DF paradigm is biased against the practical NLL, expressed as:
Polly - 8,) 83)) 2
T
Bias- Σ (Yr 2;)2-
20 20
1
Σ i=1
Yi
8
(8)
20 - 2
#=1
where Y; indicates the prediction at the i-th step, Pij denotes the partial correlation between Y: and Y, given L, o = =E
Proof. We assume that the label sequence Y conditioned on the input sequence L follows a multivari- ate normal distribution with mean vector μ = [Y, Yz.... ........ Yrl and covariance matrix Σ, where the diagonal entries Σ; = o² and the off-diagonal entries are Σij = Pıjo² for i + j. Here, Pij denotes the partial correlation between Y, and Y, given the input sequence L. On the basis, the NLL of the
17

---

Published as a conference paper at ICLR 2025
label sequence Y given L can be decomposed into sum of conditional NLLs due to the properties of the multivariate normal distribution:
lY 11)= = Σ log p(y, L,Y,, Y2 ..... Y2-1),
where each conditional probability p(Y;
L, ,......,,YY .i-1) is Gaussian with mean Y, Σj= Piz (Y; Y and variance o² p3),, p Ej=1PP Péj. Thus, the NLL can be expressed as
T Σ
-Y L) -
log (og(2 (1 -1-p3 (1- pt) + 20 - p.)
Y; -
Y, + Σ 5=1
Pis Vy - 1,)
2
For the purpose of gradient-based optimization, terms independent of the model predictions Y; can be omitted. Therefore, the practical NLL contributing to the gradients is given by
2 Y E Po 83) ))
NLL Σ 21 - 19))
Yi
On the other hand, the DF paradigm typically employs the MSE loss, expressed as
MSE Σ - 20t
(Y, - -.)).
which deviates from the practical NLL. The bias is expressed as:
Σ pij 83) 2
T Bias = MSE NLL Σ
20 Y 8,)2
Y, Y:
20 -p?)
When there exists label autocorrelation, i.e., Pij f the bias above exists. In the special case where the label autocorrelation is diminished, iee. Pig - O, the bias approaches zero almost surely.
Corollary B.3 (Bias of vanilla DF, multivariate). Given input sequence L and multivariate label sequence E TxD, suppose RTxD the flattened version of Y obtained by concatenating the rows, the learning objective (1) of the DF paradigm is biased against the practical NLL:
- 21)) -20) 22
TxD Bias Σ
TxD Σ
i-1
20 -2 1=1
2,
2,
2,
+
Σ 1=1
Pullz
(9)
20 -1ppp p.))
where Z, indicates the prediction of Zi Pij denotes the partial correlation between Z₃ and Z₁ given
L, p Σド=1 P3
Proof. This corollary immediately follows from Theorem B.2, by viewing the multivariate label sequence 7 augmented univariate sequence.
Theorem B.4 (Decorrelation between frequency components). Suppose Y is a zero-mean, discrete- time, wide-sense stationary random process of length T. As oc the normalized DFT coefficients become asymptotically uncorrelated at different frequencies:
(fw) (0,
if k i # W.
lim E[FFF T
where fr. & and Sy (f) is the power spectral density Y
18

---

Published as a conference paper at ICLR 2025
Proof. Recalling that the normalized DFT coefficients Fk. are defined as Fk. 1/vTE T- Yye- k 0,1,.... T 1. On this basis, the expected value of the product FiF can be expressed as:
B|F&FM E
Yee 12ak//T
Σ C TT
(10)
T-IT-I ΣΣ
[t j2kkt/T j2aktt
r=0
where we interchanged the order of summation and expectation, and utilize the autocorrelation function [T[r]] B[ Denote = l 1', which allows us to rewrite 1' I т. This substitution leads us to:
T- 1+11 Σ
BIAFF1
Σ
By|r]e- /2 - /
1=
- 1//
Σ
/T
(=max(0,r)
which immediately follows switching the order of summation. The expression within the parentheses a summation of complex exponentials. When k * k', the inner term approaches zero due the mutual cancellation of the oscillatory exponentials:
lim B[ = 0.
T T
When & = k', the exponential term becomes unity, and the inner sum simplifies to:
min(T-1TT--1
lim T-
Σ
1/T lim 1 - T-200
(=ma((0,r))
which immediately follows by E[F2 Ft. = Sy (fxx), where Sy is the power spectral density of Y that can be calculated the DFT of Ry. The proof is therefore completed.
C GENERALIZED TRANSFORMATION ONTO DIFFERENT BASES
Transforming time series data onto predefined spaces is fundamental aspect of signal processing and data analysis, with various strategies available based the selected bases, such Fourier and Chebyshev bases. The selection of bases is determined by the specific characteristics and requirements of the analysis. Below, we present formal definitions of common transform techniques and their associated bases, where we formulate signals as continuous functions for the ease of demonstration.
Fourier transform. It employs exponential polynominals bases which prove be mutually orthogonal. These polynomials are effective for analyzing periodic signals or signals with a strong frequency component. Let be the frequency, the associated basis function and projection onto can be formulated as follows:
fk (t) = exp(- ktt)HH)
(11)
F5 x(t)ff(t)dt
Legendre transform. It uses the Legendre polynomials bases which prove to be mutually orthogonal on the interval[11, These polynomials are particularly useful for representing functions defined on finite interval, which makes them suitable for certain types of data smoothing and approximation tasks. The k-th polynomial and the associated projection can be formulated as:
1 d
fa(t) 2 dth [(t 1)
(12)
Fy = [ (t))dtt
19

---

Published as a conference paper at ICLR 2025
0.5
W
0.5
35
35
M
-05
GroundTruth
-0.5
GroundTruth
34
GroundTruth Prediction
GroundTruth Prediction
Prediction
Prediction
300
150
(a) DF and FreDF (Fourier bases).
(b) DF and FreDF (Legendre bases).
Figure 11: The label sequences (black lines) and forecast sequences generated by DF (blue lines) and FreDF (red lines). The forecast model used iTransformer, with experiments conducted on selected snapshots characterized by periodicity (a) and trend (b).
Chebyshev transform. It uses the Chebyshev polynomials as bases. These bases are not origi- nally orthogonal but become mutually orthogonal on the interval [-1, with respect the weight 1/ v/11- t². These polynomials are particularly useful for approximating functions with rapid varia- tions. The k-th Chebyshev polynomial and the associated projection can be formulated as follows:
fx (t) cos arccos(t)),
F5 [ fx)) -1- dt
(13)
Laguerre transform. It uses the Laguerre polynomials bases. These bases are NOT originally orthogonal but become mutually orthogonal on the interval [0, oo] with respect to the exponential weight exp(t). These polynomials are particularly useful quantum mechanics and other fields involving exponential decay. The k-th Laguerre polynomial and the associated projection can be formulated as follows:
de
fa(t) exp(t) (exp(-t)e*), dtk (1(()((()) dt
(14)
exp(t)
These polynomial sets are effective for capturing specific data patterns, such trends and periodicity. which can be difficult to learn the time domain. By incorporating these polynomial sets, FreDF enhances its flexibility to handle time series data with varying characteristics. A case study is presented in Fig. 11. Specifically, the forecast sequences generated by the canonical DF struggle to capture increasing trends or high-frequency periods; whereas those produced by FreDF effectively capture the dominant characteristics, thereby significantly improving forecast quality.
In summary, FreDF does not rely solety on Fourier bases but can be adapted to various bases, each with unique properties suitable for different applications. The selection of bases for FreDF depends on the characteristics of the data and the specific objectives of the analysis.
D REPRODUCTION DETAILS
D.1 DATASET DESCRIPTIONS
The datasets utilized this study cover a wide range of time series data, detailed in Table 4, each exhibiting unique characteristics and temporal resolutions:
ETT (Li et al., 2021) comprises data on 7 factors related to electricity transformers, collected from July 2016 to July 2018. This dataset is divided into four subsets: ETTh1 and ETTh2, with hourly recordings, and ETTml and ETTm2, documented every 15 minutes.
Weather (Wu et al., 2021) includes meteorological variables gathered every 10 minutes through- out 2020 from the Weather Station of the Max Planck Biogeochemistry Institute.
ECL (Electricity Consumption Load) (Wu et al., 2021) presents hourly electricity consumption data for 321 clients.
20

---

Published as a conference paper at ICLR 2025
Table 4:
Dataset description.
Dataset
D
Forecast length
Train validation / test
Frequency
Domain
ETTh1
96, 192, 336, 720
8545/2881/2881
Hourly
Health
ETTh2
96, 192, 336, 720
8545/2881/2881
Hourly
Health
ETTml
7
96, 192, 336, 720
34465/11521/11521
15min
Health
ETTm2
7
96, 192, 336, 720
34465/11521/11511
15min
Health
Weather
21
96, 192, 336, 720
36792/5271/105400
10min
Weather
ECL
321
96. 192. 336, 720
18317/2633/5261
Hourly
Electricity
Traffic
862
96, 192, 336, 720
12185/1757/3509
Hourly
Transportation
PEMS03
358
12, 24, 36, 48
15617/5135/5135
5min
Transportation
PEMS08
170
12, 24, 36, 48
10690/3548/26
Smin
Transportation
Note: D denotes the number variates. Frequency denotes the sampling intersal of time points. Train, Volidation, Test denotes the number of samples employed each split. The taxonomy aligns with Wi al. (2023).
Traffic (Wu al., 2021) features hourly road occupancy rates from 862 sensors in the San Francisco Bay area freeways, spanning from January 2015 to December 2016.
PEMS (Liu et al., 2022a) contains the public traffic network data in California collected by 5-minute windows. Two public subsets (PEMS03, PEMS08) are adopted this work.
The datasets are chronologically divided into training, validation, and test sets following the protocols outlined in (Qiu et al., 2024; Liu et al., 2024). The dropping-last trick is disabled during the test phase. The length of the input sequence is standardized at 96 across the ETT, Weather, ECL, and Traffic datasets, with varying label sequence lengths of 96, 192, 336, and 720.
D.2 IMPLEMENTATION DETAILS
The baseline models in this study reproduced using training scripts obtained from the iTransformer repository (Liu et al., 2024) after reproducibility verification. Models are trained using the Adam optimizer (Kingma & Ba, 2015), with learning rates selected from the set 10-3 3,5 x 10 4, 100* to minimize the MSE loss. The training is limited to a maximum of 10 epochs, incorporating an early stopping mechanism activated upon lack of improvement in validation performance over 3 epochs.
In experiments integrating FreDF to enhance an existing forecast model, we adhere to the associated hyperparameter settings from the public benchmark (Liu et al., 2024), tuning only a within [0, 1] and learning rate conservatively. Finetuning the learning rate is essential to handle the different magnitude of temporal and frequency losses. Fine-tuning is conducted minimize the MSE averaged across all forecast lengths on the validation dataset.
E MORE EXPERIMENTAL RESULTS
E. OVERALL PERFORMANCE
Long-term forecast. We provide comprehensive performance comparison on the long-term forecast task in Table 5. The iTransformer model is used to operationalize the FreDF paradigm. Despite the iTransformer's existing performance gap compared to other baseline models, the incorporation of FreDF enhances its performance in the majority of cases, securing the lowest MSE 31 out of 45 cases and MAE in 40 out of 45 cases. The few instances where FreDF does not achieve the lowest MSE are attributed to the inherent superiority of other models over ITTranffrmme in specific datasets (for example, Fre'TS versus iTransformer the Weather dataset).
Short-term forecast. We investigate the short-term forecast task in Table 6, with FreTS Yi et al. (2023b) serving as the forecasting model in the FreDF implementation. Consistent with the long-tem forecasting results, FreDF enhances FreTS's performance in most instances. Notably, there three
21

---

Published as a conference paper at ICLR 2025
Table 5: The comprehensive results on the long-term forecasting task.
Medels
FreDFF (Ours)
(Trandoomer (2024)
FteTS (2023)
TimesNet (2023)
MICN (2023)
TIDE (2023)
DiLitear (2023)
PiDioomer (2022)
Autodoemer (2021)
Transformer (2017)
(2017)
TCN
Metrics
MSE
MAR:
MSE
MAB:
MSE
MAR
MSE:
MAB
MSS
MAR
MSI:
MAB
MSR
MAR
MSE
MAR
MSE:
MAB
MSS
MAB
MSI:
MAB
96
0.324
0.362
0.346
0.379
0.339
0.374
0.338
0.379
01.318
1.366
0.364
0.367
0.345
0.372
0.389
0.427
0.468
0.463
0.591
0.549
0.357
0.613
0.385
0.400
0.350
0.400
0.396
0.404
0.411
0.509
0.629
0.626
0.404
0-422
0:426
0-428
0:428
0-225
0.51
0.527
0.861
0.656
720
0.469
0.444
0.461
0.485
0.462
0.644
0.514
0.501
0.61
0.472
0:45
0.488
0.369
0.803
0.911
0.653
Avg
0..322
0.399
0.416
0.415
0.418
0.423
0.419
0.407
0.451
0.517
0.753
0.632
96
0..173
0.252
0.184
0.266
00.190
0.382
0.185
0.764
0.178
0.275
0:200
0.305
0.195
0.294
0.114
0.284
0.340
0.319
0.311
0.408
3.125
1.345
BTTm22
0.315
0.364
0.359
0.349
0.758
1.350
336
0.334
0:351
0.405
0.345
0.299
0:354
0422
0.427
0.319
0.359
0.373
0.369
1.375
720
0.395
0.393
0.419
0.409
0.517
0.489
0:434
0.413
0.482
0.479
0:555
0.524
0.516
0.502
0:430
0.424
0.423
0.421
2.576
1.223
4.203
1.658
Aorg 0..778
0.319
0.335
0.335
0.379
0.352
0.356
0.004
0.396]
0.310
0.348
0.366
0.814
3.411
1.452
96
0.3322
0.400
0.410
0.412
0.43
0.418
0.464
0.410
0.18
0:441
0.691
0.633
ETTb1
0.619
336
0.474
0.451
0.480
0:457
0.503
0.475
0.470
0.530
0.603
0.487
0.465
0.468
0472
0.4%6
0.776
0.613
720
0.463
0.462
0.484
0:472
0.596
0.565
0.532
0.502
0.671
0.620
0:893
0.756
0.516
0.513
0.500
0.493
0:472
0.467
1.182
0.385
0:828
0.678
Avg 00.437
0.435
0.447
0.488
0.474
0.466
0.515
0.628
0.574
0.462
0.458
0.441
0.457
0.476
0.477
0.993
0.788
0.763
0.656
0.289
0.387
0:340
0:401
0.364
0:404
0:440
0.996
0.991
0-434
1.140
B1Th2
192
0,363
0.385
0.382
0:402
0.422
0.475
0.409
0417
0.400
0.509
0.474
0.430
0.443
0.511
5.081
1.814
1.300
0.419
0.426
0.830
0.564
0.524
0.489
0.451
0.671
0.8%
0.643
0.771
0.003
0.366
0.009
0.425
0.410
0.481
3:554
1.425
5.300
1.481
720
0.415
0.439
0:443
0:455
0:815
0.654
0:471
0.474
0
0.212
0:874
0.679
0.812
0.600
0.473
0.480
0:501
0.515
2:459
1.247
5.590
1.565
Avp 0.377
0.396
0.390
0.410
0.550
0.515
0.443
0.426
0.624
0.549
0.611
0.590
0.558
0.516
0.450
0.447
0.478
0.483
333996
1.419
3.925
1.455
96
0.233
0.1.8
0.229
0.109
0.277
0.273
0.250
0.329
0.210
0.302
0.315
0.315
0.352
0.621
ECL
197
836
0.172
0.889
0.263
0.347
0.120
0.101
0.28%
0:222
0.193
0.207
0.206
0.083
0.188
0:218
0.304
0.389
0.177
0.185
0:289
0.206
0.256
0:340
0.344
0.330
0.210
0:221
0.308
0.319
0:200
0:226
0.340
0.322
0.215
0.233
0.322
0.343
0.266
0.200
0.383
0.364
0.590
0.587
0.588
0.583
720
0.204
0.294
0.209
0.206
0.245
0.352
0.280
0.363
0.218
0.323
0.264
0.373
0.258
0.350
0.262
0.379
0.268
0.371
0.267
0.371
0.602
0.601
Avg
0..17
0.259
0.176
0.267
0.200
0.297
0.214
0.307
0.187
0.297
0.251
0.344
0.225
0.319
0.220
0.339
0.228
0.339
0.274
0.367
0.617
0.598
96
0,391
0.265
0,397
0.222
0.528
0.341
0.504
0.298
0:317
0.493
0.697
0:429
0.577
0.362
0.609
0:385
0.686
0.385
1,45L
0.744
Traffic
192
0.410
0.273
0.418
0.299
0.331
0.338
0.526
0.308
0.211
0.328
0.756
0.424
0.647
0.400
0.003
0.372
0333
0.400
0.079
0.377
0.442
0.622
335
0.424
0.280
0.431
0.385
0.351
0.345
0.500
0.310
0.641
0.342
0.762
0.477
04533
0.410
0.615
0.378
0.657
0.398
0.663
0.361
0.364
0.600
7.0
0.460
0.208
0.467
0.905
0.167
0.324
0:354
0.719
0.449
0.428
0.403
0.415
0.381
0.644
Avc
00441
0.279
0.428
0.786
0.552
0.348
0..35
0.300
0.636
0.335
0.760
0.473
0.671
0.419
0.611
0.379
0.630
0.399
|
0.600
0.376
1
0.652
Weatber
96
0.864
0.202
0.201
0.247
0.184
0.239
0.177
0.226
0.182
0.250
0.201
0.261
0.197
0.259
0.221
0.304
0.264
0.355
0.332
0.383
0.610
0.368
1.9
0.220
0.283
0.250
0.283
0:221
0.325
0:227
0.305
07:34
0:301
0.242
0.208
0.236
0.294
0:375
0.365
0.313
0.311
0.644
0.599
0.541
0.582
336
0.275
0.294
0.302
0.317
0.272
0.316
0.263
0.300
0.268
0.325
0.257
0.335
0.282
0.332
0.358
0.379
0.259
0.393
0.656
0.579
0.565
0.569
0.362
0.363
0.355
0.395
0.386
0.384
0.418
0.466
0.700
0.601
Avg
0.254
0.274
0.281
0.302
0.255
0.299
0.262
0.288
0.261
0.319
0:271
0.320
0.266
0.317
0.311
0.361
01349
0.391
0.652
0.552
0.364
0.572
PEMSO3
12
0.065
0.095
0.172
0.208
0.009
0.099
0.210
0.175
0.083
0.127
0.194
0.241
0.110
0.082
0.188
0.216
0.087
0.086
0.203
0.198
0.117
0.233
0.320
0.225
0.122
0.202
0.245
0.320
0.123
0.350
0.248
0.257
0.492
0.239
0.506
0.365
0.107
0.121
0.227
0.209
0.632
0.655
0.636
0.606
0.240
0.243
0.281
0:236
0.239
0.380
0.422
0.382
0.321
0.459
0.243
0.644
0.269
0.275
0:311
0:351
0:111
0:429
0.350
0.723
0.253
Axp
0.113
0.219
0.116
0.336
0.144
0.257
0.118
0.223
0.899
0.214
0.316
0.370
0.251
0.344
0.114
0.302
0.500
0.513
0.126
0,253
0.666
0.634
PEMSU8
12
0,000
0.182
0.085
0.180
0.095
0.204
0.110
0.200
2.193
0.87
0.121
0.231
0.150
0.274
0.175
0.275
0.446
0.433
0.213
0.236
0.600
0.600
24
0.118
0.220
0.131
0.356
0.150
0.250
0.142
0.250
0.235
0.339
0.232
0.326
0.245
0.350
0.211
0.305
0.488
0.500
0.258
0.256
0.701
0.622
0.255
0.282
0.305
0.258
0.300
0.425
0.417
0.358
0.513
0.277
0.637
0.293
0.323
0.341
0.374
0.324
0.527
0.437
0.409
0.371
0.751
0.295
0.648
Avg
0.341
0.235
0.159
0.258
0.174
0.277
0.154
0.245
0,717
0.459
0.319
0.378
0.294
0.377
0.232
0.322
0.030
0.572
0.249
0.266
0,713
0.629
1" Count|
31
40
6
1
0
Note: We fix the inputt length as 96 following iL et al., 2024). Bold typeface highlights the top performance for each metric, while underfined text denotes socond-best results. Aug indicates the results averaged over forecasting longths: T-96, 192, 336 and 720.
Table 6: The comprehensive results on the short-term forecasting task.
Models
FreDF (Oura)
FieTS (2023)
iTrasfommer (2014)
MECN (2023)
DLinear (2113)
Fedfomner (2025)
Autoformer (2013)
Matric
|MAPE MASE OWA
[SMAPE
MASE
OWA
(SMAPE MASE: OWA
SMAPE
MASR OWA
SMAPE MASE: OWA (SMAPE MASH OWA SMAPE MASE: (WA
Yealy
13,856
3,446
0.796
13.576
3.058
0.50:
13,797
3.143
0.810
14.904
3,392
0.873
14:307
3.094
0.87
13,648
3,000
0.806
18:477
4.56
1.101
Quarterly
1,229
0.919
10.961
0.916
10,503
1.346
0.912
11.417
1.021
10,500
1.237
0.928
14:254
1.314
Montily
12,99
0,983
0.913
13.088
0.92
0.919
13,227
1,013
0.915
13.834
1.080
0.987
13.362
1,007
0.937
14.181
1.105
1.011
18421
1,616
1.308
Others
5.294
3,614
1.127
5.563
3.21
1.19
5.101
3.419
1.076
631
4.201
1.308
5.12
1,649
1.116
4813
3.243
1,019
6.722
4.903
1.495
1,645
0.877
12.169
1,6
0.383
12.298
13,046
1854
0.962
12.45
1,674
0.898
12.734
16.851
2,443
1.26
Note: Bold typeface highlights the top performance each metric, while undertined text denotes the second hest results, Aug indicates the results avernged over forecasting lengths: yearly, quarterly, and monhlyy
22

---

Published as a conference paper at ICLR 2025
Table 7: The comprehensive results on the missing data imputation task.
FreDF (Ours)
STranfomer
FieTS
TimesNet (2023)
MICN (3023)
TIDE (20123)
DLineur (2012)
FEDoomee (2002)
Autoformer
Models
(2004)
(2023)
(2021)
Prasee
MSE
MSE
MAE
MAE
MSE
MSE
MSE
MAE
MAB
MSE
MSE
MAE
0.125
0.001:3
0.02790
0,00213
0
1001002
0,07543
0.01552
0,07267
0,00236 0:03371
(.45005
0.45514 0,00148
0,02300 0.023000.6662
0.3811
0,376554
0.35375
0.0028T
0.00402
0.04434
(.01089
6,07753
0.01245
0,07946
0.00284
0.03691
0.4177
0.45884
0.00154
0.12351
0.68233
0.37059
0.35261
0.575
0.00155
0.03650
0.004S8
0.04663
0.01000
0,08673 0.00323
0,62015
(55500
0,00175
0,02389
0.48119)
0.58105
0,37877
0,36093
0.00882
0.0010000000000000
0.0063
0.1400
000061
0,09510 0.00952
0.0028 0420942
59320)
0,00192
8.02219
0.681
0.5008
38059
0.4666
0,00112
0.01250
0.00359
0,04191
(.0105e 0.07807
000307000:370 0,08374 0,00259
0,44776 0.40572 0,00367 0.02334
0.08220
0.38104
0.37660
0.35799
0.128
6.01640000000
0.0404/0033994 033119
0. (00111
6.09219
1.38386
0.80777
0.25
0,00437
0,04255 0.00651
0,04303
(U03591
0,13665
000799 0,08999
0,00331
0.6 100
0,00063
0.11547
3.00064
1.31545
141033
0.81363
0.555
0.00553
0.01815 0.00342
0.11499
0.00411
0.04900
11225
73633
0.01
0.08812
9. 1.102
1.90500
4,0012
0.81040
0.00137
0.02382
0.00100
0.0253
2003200
0.13627
(104055 0,14285
OLIOSOS 18 70060 0,01025
0.065400
1.31899
1.46177 0,81796
Avg
0,00322
0.03575
0000333
0.83667
(U03290
0,13342
(.02445 0,10355 0L00ST1 0.04200 0.9377 0L60893 0,02405 0.059E3
3.10402 1.31350
1-46655
0,81246
0.125
0,00175 6.00059
0,00019
0.04102
400
0,00181
(000641
0,044033
0L00KE2 I BIOKESS 0,36363
DASSSO 0,00279
0.38026
0.431
1841184
0.25
0,00215
0.00354
0,04205
0.08097
0,03850
OLDOR54 0.04709
0,00235
0.03324
0.68162
0.37973
0.3315
0,41594
0.575
0,80182
0.03166
0,00280
0.00852
0.01388
0.08017
0.00251
0,03540
0.00454 D.04730 0.043300.21088 0,34020
0.00210 0.0E1E1
0.68183
0.37975
0.44331
0.42055
0,00114
0.02414
0.00 24
0.01008
0.02266
0.0001
000004
0.09472 0.00417 0.0494
27200 0.00175 0.07844
0.681
0.59902
044112 0,42387
0.80173 0.02996
0.00277
0900 013333
0.00k0s3
(100317
0,00917
0.08199
0339992
048888
0,41915
0.125
l.80222
0.0312410
004733
0.0400
0.04995
CLIOARA (L04200 15859 73871
0.0238T
10B8S)
1.31746
0.84467
0.25
0,80407
0.04259
0.00571
0,05996
N.04647
0,1355)
0.00494
0.04447
OUDOAST 0.04050 O.7564 0,59747 0.02491
0.115E1
3.12891
1.30754
145386
0. 0.84385
0.755
0,00306
0.03W93
0.00422
0.04519
0.0430
0.13585
0.00512
0.04007
0.00STS
595770
0.01944
10227
8.12788
1.91728
145464
0,94190
0,00129
0.02365
0.000999
o 904
049000
1.1366
000000
0,05224
OLDOSIN DOSS47 38775 R0497 0.01465 0LDNPM6
3.178801
1.30738
o 034644
Avg
0.80266 6.03560
0000036
(104715
0,13613
(.00536
0,04723
0L004T7 0,2000 0.76 0150622 0,02005 OL 10085
3.12829
1,31740
1.55994
0,84423
0.125
0.00029
0.01257
1000000
0.01191
(101018
0.0enss
(100066
0.06997
0.01WW6 U. 40TS 1 SOWER D 42255 O. 1ONNS 0.20000
0.41000
o. 0.20000
0.00061
0/00216
0,00991
0,04106
0.2360-
0.43557
0,40007
0:20118 0,29771
0.575
0,00990
0.02242
0.00211
0,004731001222
IU01022
0,08255
0.00230
0,00296
0,04375 0.15234 0,25310 0.378266 0,10500 0,23415
0.4599
0,41006
0.20998 0,30337
0.5
0.00103
6.02399
0000175
0.03177
0.01025
0,003844
0.00055
0,04520 D. 0. O 0434520 O. 0.10162
0.591
0.41011 0.21322 0, 0000000
0.00071
0.00197
0,08266
0,04169
0.38609 0,10550 0,23501
0.49857
0.2777 0,29969
0L125
b,80650
6.01250
0.0164600
p00061
0,06125
(000000
0,00514 0,05275
42631
0.135538 0,17599
0.25
0,80967
0.01513
0.00013
0.01715
(U00557
0,06305
0.00244
0,01550
0,00925
0X03900
0,29296 0.35453 0.30476 0,05019
0,4055
0.42635 0.13688 0,18177
0.375
0.80854
0.01443
0,00067
0.01700
0.06113
0.0005
0.80924
0.00326 0.03997 0.156 0.25013 0.00454
0.04811
0.40550
0.13831
0.5
0.80331
0.01107
0.00047
0.01420
0.06001
(10002
0,00463
0.00309 0.09020 0. 12578 0 042499 0.00492 (.0496)
0.4055)
0.426122
n. 1880
19051
Avg
0.00951
0.01331
m000 0.0157 5
0,06100
(100151 0,01332 0,000200 D 0.0 o. 0.1400 323.20 0,00484 0.00010 40054
0.42835
0 1.17777 1833
1" Caum
23
6
Note: The input length set 96 for all buselines. Bold typelace highlights the top performance for each metric, while underlined text denotes the second results. Aug indicates the results averaged over missing ratios: 0.125, 0.25, 0.375,
23

---

Published as a conference paper at ICLR 2025
ITransformer
DLinear
Autoformer
V
0.5
0.0
Truth Prediction
Ground Truth
-0.5
Trulh
Prediction
Prediction
-1.0
200
300
400
ITranstormer & FraDF
DLinear FreDF
Autoformer FreDF
0.5
0.5
0.5
0.0
0.0
0.0
-0.5
Truth
-0.5
GroundTruth
-0.5
GroundTruth
Prediction
Prediction
Prediction
0
100
200
300
400
100
200
300
400
100
200
300
400
(a)
(b)
(c)
Figure 12: The forecast sequences generated with DF and FreDF. The forecast length is 336 and the experiment is conducted on snapshot of ETTm2.
cases where FreTS outperforms FreDF. This occurs because the loss weight ar is tuned to minimize the validation error averaged across all forecast lengths instead of focusing on specific lengths. While it is feasible to fine-tune for each forecast length, we did not use this approach, as the current results suffice to demonstrate FreDF's effectiveness.
Missing data imputation. We investigate the imputation task in Table with iTransformer serving the forecasting model in the FreDF implementation. All models are trained using an autoencoding approach: given input sequences with missing entries, they are tasked with recovering the non non-missing entries during training, while they employed impute the missing entries during inference. The results demonstrate FreDF's efficacy in this task, significantly improving the performance of ITrranffrmme and outperforming most competitive methods. A unique aspect of this task is the irregularity the label sequences caused by the missing entries, which disrupts the physical semantics related to the Fourier transform. This indicates that the effectiveness of FreDF does not stem from the semantic characteristics of the Fourier transform itself, but rather from its ability to align the properties of time series data with the implicit assumptions of the DF paradigm, specifically the conditional independence of labels.
Showcases. We provide additional showcases illustrating the change forecast sequences Fig. 12 and 14. Overall, FreDF effectively mitigates blurs and captures high frequency components. These successes can be attributed to FreDF's unique capability to operate in the frequency domain, where the challenges of autocorrelation are mitigated, and the expression of high-frequency components becomes straightforward.
E.2 GENERALIZATION STUDIES
In this section, we further explore the versatility of FreDF in improving various forecasting models: iTransformer, DLinear, Autoformer, and Transformer. The results, displayed in Fig. 16, encompass five distinct datasets and are averaged over forecast lengths (96, 192, 336, 720), with error bars re- flecting 95% confidence intervals. FreDF significantly improves the performance of these forecasting models, particularly benefiting Transformer- based architectures like Autoformer and Transformer. These results affirm FreDF's s utility in enhancing neural forecasting models, highlighting its potential versatile training methodology in time series forecasting.
24

---

Published as a conference paper at ICLR 2025
iTranfforme
DLinear
Autoformer
125
125
125
GroundTruth
Truth
GroundTruth
100
100
100
75
75
50
50
50
25
25
25
0
12
16
12
16
20
0
12
16
20
24
iTransformer FreDF
DLinear & FreDF
Autoformer & FreDF
125
125
Truth
Truth
GroundTruth
100
Prediction
100
Prediction
Prediction
20
(a)
(b)
(c)
Figure 13: The spectrum of forecast sequences generated with DF and FreDF. The forecast length is set to 336 and the experiment is conducted on snapshot of ETTm2. Only the first 24 frequencies of the spectrum are presented.
iTransformer
DLinear
Autoformer
0.50
0.50
0.50
0.25
0.25
0.25
0.00
0.00
0.00
-0.25
-0.25
-0.50
Truth Prediction
Truth
-0.50
Groung
Prediction
Prediction
0.75
-0.75
-0755
iTransformer
DLinear FreDF
Autoformer
0.50
0.50
0.50
0.25
0.00
0.00
0.25
-0.25
-0.25
0.50
GroundTruth Prediction
-0.50
GroundTruth
-0.50
GrounsTruth Prediction
Prediction
0.75
-0.55
-0.75
0
100
200
300
400
100
200
300
400
100
200
300
400
(a)
(b)
(c)
Figure 14: The forecast sequences generated with DF and FreDF. The forecast length is set to 336 and the experiment is conducted another snapshot of ETTm2.
25

---

Published as a conference paper at ICLR 2025
iTransformer
DLinear
Autoformer
Truth
Ground Truth
80
Ground
Prediction
Prediction
Prediction
60
20
0
0
12
18
20
8
18
20
0
12
20
24
iTransformer FreDF
DLinear FreDF
Autoformer FreDF
Truh
80
Truth
80
GroundTruth
Prediction
Prediction
Prediction
60
60
40
40
40
20
20
8
12
16
20
24
8
16
20
0
12
20
24
(a)
(b)
(c)
Figure 15: The spectrum of forecast sequences generated with DF and FreDF. The forecast length
336 and the experiment conducted on another snapshot of ETTm2. Only the first 24 frequencies of the spectrum are presented.
THOO
MEE
MIE
Fort Ted
MARE
ETThl
(b) ETTh2
(c) ETTml
(d) ETTm2
(e) Traffic
Figure 16: Performance of different forecast models with and without FreDF. The forecast errors are averaged over forecast lengths and the error bars represent 95% confidence intervals.
26

---

Published as a conference paper at ICLR 2025
Table Comparable results with DTW-based loss.
Dataset
ETTml
ETTh!
Models
FreDF
Dilate
DPP
FreDF
Dilate
DPP
Metries
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.324
0.362
0.498
0.443
0.631
0.495
0.382
0.400
0.790
0.567
0.815
0.577
192
0.373
0.385
0,993
0.625
0.975
0.617
0.430
0.427
0.950
0.643
0.916
0.633
720
0.469
0.444
0,999
0.652
1.079
0.678
0.463
0.462
0.922
0.654
0.898
0.649
Avg
0.392
0.399
0.859
0.587
0.907
0.604
0.437
0.435
0.910
0.632
0.904
0.630
E.3 HYPERPARAMETER SENSITIVITY
In this section, we examine how adjusting the frequency loss weight a impacts the performance of FreDF across three models: iTransformer, Autoformer, and DLinear, with the results in Fig. 17, 18, and 19. We find that increasing a from 0 to 1 generally reduces forecast error across various datasets and forecast lengths, highlighting the benefits of frequency domain learning approach. Notably, the minimum forecast eтror often occurs at values close to 1, rather than at itself; for instance, 0.8 optimal for the ETThl dataset. This suggests that integrating supervisory signals from both time and frequency domains enhances forecasting performance. However, the improvement may be incremental compared to simply setting a
E.4 COMPARISON WITH DTW-BASED LEARNING OBJECTIVES
In this section, we compare FreDF with works that employ DTW as learning objectives align the shape of the forecast sequence with the label sequence: Dilate (Le Guen & Thome, 2019) and DPP (Le Guen & Thome, 2020). Notably, these works do not handle the bias introduced by label autocorrelation, which makes them independent to the contribution of FreDF. To make fair comparison, we integrated the official implementations of the loss functions into the iTransformer model. As shown in Table 8, FreDF significantly outperforms DTW-based methods across both datasets. This improvement stems from FreDF's unique ability debias the learning objective, capability that Dilate and DPP do not possess.
27

---

Published as a conference paper at ICLR 2025
041
MEE
USE
VAKE
MSE
LANE
046
0440
0.30
0.38
10.00
0.00
Q41
0.36
(b)
0.30
0A4
0.34
0.26
0001220004455500700000
000 0000/8803000338870000000
0.0 0001122030433330000000
0.25
0.000
USE
MBE
MAZ
0.32
000 120.000000022002000000000
016103632
(d)
0.25
WAN
0.20
0.0 02230403287700000
00311
000 20 77700011
000 10 20.002000000000 8000009913
0031 20011222040351100
1.5
0.40
0.35
0.30
000 212 011330035207700005
0.00
000L 0040 527000011
000 100.1222304055000000000 500000001300
003 2220012020000222222222222 S1.0000111
MEE
0.26
10E
-
VIE
-
WA
-
MEE
MAE
MEE
0:00
-a-
WE
MEE
1:48
0.2
0.75
0.25
540
1031026304400 4000444035-00000000000000 107000911
000 DE300000 7050813
00315 10311363000000 T0R0011
Figure 17: FreDF improves iTransformer performance given a wide range of frequency loss weight a. These experiments are conducted on ETThl (a), ETTh2 (b), ETTml (c), ETTm2 (d), ECL (e), Traffic (f) and Weather (g) datasets. Different columns correspond to different forecast lengths (from left to right: 96, 192, 336, 720, and their average with shaded areas being 50% confidence intervals).
28

---

Published as a conference paper at ICLR 2025
0.00
USE
1AKE
MSE
0.54
MANE
0.6
0.44
945
VIE
0,502
10.00
0.50
0.023
(b)
MAE
9.00
0.44
0.55
042
0.00
0.30
MAE
MIE
0442
MAE
MSE
1:00
aams
1100
0.215
0.12
0.000
000 20 8287000013
0001 000-020000055F0000000
(d)
WAE
USE
1APP MEET
0.25
UGE
Vi860
000 111304052877000000
000 00212200044331000000000
000 WwW.S8222000000000000 863060013
0031 N022-22300A2S2ECC 3C7000011
0.7
MAC
MIE
VESE
WAC
MBE
06
VEE"
MAE
MEE
2001 22011220220000000000000
3301820002405 2000 130010
300.1 0000122011403007330000
0.00
WIE
-
MEE
-
MAA
-
VIE
1ANE
-
MBE
MAE
-
MEE
SEE
000 1- 30 2050913
0O0 304000 107000081
000 P0 3050813
003 203000 7000211
Figure 18: FreDF improves Autoformer performance given wide range of frequency loss weight a. These experiments are conducted on ETThl (a), ETTh2 (b), ETTml (c), ETTm2 (d), ECL (e), Traffic (f) and Weather (g) datasets. Different columns correspond to different forecast lengths (from left to right: 96, 192, 336, 720, and their average with shaded areas being 50% confidence intervals).
29

---

Published as a conference paper at ICLR 2025
0.1
NEE
NAE
JAKE
MSE
0.915
MAE
947
volle
0.00
0012
0.30
0,53
0..00
0.015
10
0.008
0.20
0.50
0.68
0.00
0.37
0,3000
Weit
MABE
0.3055
DA14
0412
0.20111 Naatoo
003 31000001133
003 200-1363000..0 70770011
(c)
MBE
SAAE
MIE
0.00
MAZ
0.30
000 222300002257000000
0.0 2002422222225 30
0001 22030485T
0.0 51
0031 003/8202043 9C
(d)
MAE 1- 14FF
0.30
MAA
10 MSE
14AF MEE
MAN
(MSE
Viluo
000 1221 0113300022127000000000
003122600443000 211.0000....1100000000
0001 011.000000000000000000 0007000000000000000030
000.1 N001112300405110000000
0031 11200000311000000000000000000000
MAE - VIE
- MAE
-
MEE
M/e
-6-
VIEE
ME2
2001320594022200
830182002400 200.00000000000010
2301920033465
1.0010
0.24
MAN
MBE
028
MAN
USF
0.22
SAAE
MAA
NEE
0.000
MAE
18E
00111223300555327000000
00211220304-3555770000
0001
000 000112223000052507000000
Figure 19; FreDF improves DLinear performance given wide range of frequency loss weight These experiments are conducted on ETTh1 (a). ETTh2 (b), ETTml (c), ETTm2 (d), ECL (e), Traffic (f) and Weather (g) datasets. Different columns correspond to different forecast lengths (from left to right: 96, 192, 336, 720, and their average with shaded areas being 50% confidence intervals).
30


Reviews:
[{"review_id": "eAySsUH273", "reviewer": "Reviewer_ErxY", "summary": "This work introduces a new loss term for multi-step time-series forecasting that penalizes errors within a decorrelated representation space of the ground truth labels. In its current formulation, this decorrelated space is defined as the frequency representation of both labels and forecasts. Experimental results demonstrate that the proposed approach significantly enhances forecasting accuracy across various datasets and base models.", "questions": "1. In the experiments, how to determine the hyperparameters of FreDF and baselines?\n2. According to the ablation study, the frequency term seems to be almost all that's needed. Is it correct?\n3. During my attempt to reproduce the experiments, I encountered an error (`AttributeError: 'Exp_Short_Term_Forecast' object has no attribute 'seasonal_patterns'`) when running the short-term forecasting experiments. Is this error expected, or does it indicate a potential issue in the codebase?\n\nOverall I think this work is interesting and promising. I will consider raising my score if my questions could receive positive responses, especially my major concerns (W1, Q1, Q3).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The authors identify and address the bias introduced by label correlation in time-series modeling, which is a novel issue for me and holds substantial potential generality across different scenarios.\n2. The method is sound, straightforward and shows very promising results. The theoretical results are relatively persuasive, demonstrating the bias caused by label correlation, and subsequently FreDF's elimination of label correlation and thereby bias.\n3. The experiment is comprehensive. Extensive set of experiments showed that: the approach contributes to the state-of-the-art, different components of the loss contribute to performance, the approach is robust to hyperparameter values.", "weaknesses": "1. It seems that this work aims to penalize errors in a space that decorrelates labels. Experiments mainly achieve de-correlation via FFT or FFT2. It will be beneficial to explore the efficacy of other transformations beyond the Fourier transform?\n2. While the introduction discusses various forecasting models, it could benefit from a stronger focus on established forecasting paradigms (e.g., direct forecasting, iterative forecasting). The inclusion of forecast models (iTransformer, Linear) may detract from highlighting the contribution and role of FreDF that seems to be orthogonal to forecast models.\n3. The source code should be refined. The current implementation comprises numerous scripts, and it is somewhat unclear how each script relates to the experiments discussed in the manuscript. Besides, the current environment setup appears to depend on unexpected repositories like `torchmetrics` and `patool`. Including a `Dockerfile` or `conda` environment file would help a lot.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZSdyuR83LV", "reviewer": "Reviewer_3nNT", "summary": "Considering that in current time series prediction, DF only reuses general multi task learning methods without considering the dependency relationships  between labels. Especially, even if there is no correlation between labels, the above loss function can be used to train a multitasking model. When training the model, DF attempts to minimize the error between the predicted label sequence and the true label sequence; This assumes that the label sequence is conditionally independent between different time steps, thereby ignoring the correlation between each time step within the label sequence. So the author uses fast Fourier transform in the frequency domain to transform the data from a temporal perspective to a frequency perspective, in order to suppress autocorrelation. And the theoretical derivation of using frequency domain conversion to suppress time step correlation is given. A loss function combining time domain and frequency domain is designed, and the effectiveness of the design method is demonstrated by experimental results.", "questions": "1. Computational complexity of frequency domain conversion: Will the frequency domain conversion mentioned in the paper significantly increase computational complexity? How efficient is the FreDF method when dealing with large-scale datasets?\n2. Generalization ability: How does the FreDF method generalize across different domains and datasets? Has it been tested and validated in more practical application scenarios?\n3. Applicability of frequency domain features: How effective is the FreDF method for data with unclear or difficult to extract frequency domain features? Are there any relevant experimental results or analysis?\n4. Model interpretability: Does the FreDF method have interpretability in the frequency domain? How do users understand and interpret these predicted results?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Solving the problem of autocorrelation in label sequences: The FreDF method proposed in this paper effectively bypasses the autocorrelation problem in label sequences by predicting in the frequency domain. This is a common but not fully addressed problem in existing direct prediction models.\n2. Compatible with multiple prediction models: The FreDF method is not only suitable for existing state-of-the-art methods such as iTransformer, but can also be compatible with multiple prediction models. This compatibility makes it widely applicable in different prediction tasks.\n3. Significant improvement in predictive performance: Experimental results show that the FreDF method is significantly superior to existing methods in multi-step prediction tasks. This indicates that the method has high accuracy and reliability in processing complex time series data.\n4. Innovative applications of frequency domain analysis: By introducing frequency domain analysis into time series prediction, the FreDF method provides a new perspective to address autocorrelation issues in time series. This innovative application not only improves predictive performance, but also provides new directions for future research.", "weaknesses": "1. Complexity of frequency domain conversion: Although frequency domain analysis can bypass the autocorrelation problem in label sequences, frequency domain conversion itself may increase computational complexity and time cost. This may become a bottleneck when dealing with large-scale datasets.\n2. Model generalization ability: Although FreDF performed well in experiments, its generalization ability in different fields and datasets still needs further validation. Especially in practical applications, the model may need to be adjusted and optimized for specific tasks.\n3. Dependence on frequency domain features: The FreDF method relies on the extraction and utilization of frequency domain features, which may limit its applicability in certain situations. For example, for data with unclear or difficult to extract frequency domain features, the performance of FreDF may not be as expected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Kw8K85zE7i", "reviewer": "Reviewer_WM7u", "summary": "The paper presents a new Frequency-enhanced Direct Forecast method for time series predictions that aims to improve the predictions generated by direct forecast (DF) models by training the models using a combination of standard MSE errors and errors defined in the FFT transformed space. The new approach is motivated by the fact that the MSE errors for direct forecast (DF) models are biased and do not properly account for correlations in predicted sequences, while errors on the FFT on the predicted sequences may be more robust to such dependencies.  The experiments on multiple datasets and multiple baseline DF models demonstrate the improved performance of the new method over baselines.", "questions": "Are there any settings where defining the errors by transferring the predictions to the frequency domain is detrimental to the prediction accuracy?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: The idea of augmenting an objective function with a loss in alternate bases that is more robust to value dependences in prediction sequences, in this case Fourier bases, is novel and interesting.\n\nSignificance: The extension applies to a subclass of direct forecasting models that predict components of future sequences independently, ignoring dependences that may exist among them.  This makes the method potentially applicable to a variety of SOTA time series models. \n\nQuality: The authors attempt to analyze the problem of using simple additive error functions for predicting sequences and its fixes via errors in the FFT transformed space both theoretically and experimentally. Extensive experimentation across multiple datasets and tasks show the merits of the proposed approach compared to SOTA baselines.\n\nOther strengths: Exploration of properties and extension of the proposed framework and its benefits, such as, different prediction length, ablation models,  possible new transformations and errors defined on these transformations. \n\nCode: Authors provide the code for reproducing the experiments.", "weaknesses": "Limited analysis of a bias in Theorem 1: Theorem 1 considers only univariate sequences, cross-correlation terms are not accounted for in the Bias formula. The sequence has dimensionality D.  \n\nNotational inconsistency and formula errors in the paper: \n- The use of 'L' is overloaded, it denotes both the length and inputs of the input sequence. \n- Equation 1 has a mistake. Y and Y_hat should be compared on the same indexes. \n- Definition 3.2, uses 'j' instead of “i”  \n\nThe main results in the paper do not report performance over distinct random seeds. Given this is an experimental study, reporting the results on multiple distinct random seed could help to understand its sensitivity.\n\nThe short-term forecasting task results in the appendix report only qualitative time spans. It would be good to specify the results in terms of forecasting lengths, similar to the Table 1 for long-term forecasting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a01Qee6ZFZ", "reviewer": "Reviewer_6T6T", "summary": "This paper proposes to measure the distance between time series prediction outputs and target signals in the frequency domain. \nAs claimed, this allows to capture label autocorrelation in time series prediction tasks. Speciafically, in practice, distances in both time and frequency domains are utilized for training in a combined manner.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The presentation of the paper is clear and the idea is straightforward.", "weaknesses": "The proposed approach lacks novelty. Measuring time series structure (e.g., autocorrelation) in the frequency domain has been well-explored. Several prior works have investigated capturing time series characteristics or enhancing robustness in loss functions, including:\n  1. Incorporating Fourier transformation in loss functions [1,2].\n  1. Employing DTW-based loss to keep shape information of time series [3,4].\n   1. Utilizing multiresolution trends during training [5,6].\n\nThese approaches are all relevant to this study. However, this paper does not adequately investigate, discuss, or compare these related methods.\n\n[1] Henning Lange, et al. From Fourier to Koopman: Spectral Methods for Long-term Time Series Prediction. JMLR 2021.\n\n[2] Xinyu Yuan and Yan Qiao. Diffusion-TS: Interpretable Diffusion for General Time Series Generation. In ICLR 2024\n\n[3] Vincent Le Guen and Nicolas Thome. Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models. In NeurIPS 2019.\n\n[4] Vincent Le Guen and Nicolas Thome. Probabilistic time series forecasting with shape and temporal diversity. In NeurIPS 2020.\n\n[5] Shiyu Wang, et al. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. In ICLR 2024.\n\n[6] Amin Shabani, et al. Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting. In ICLR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)