Not All Frequencies Are Created Equal: Towards a Dynamic Fusion of Frequencies in Time-Series Forecasting

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 08:32:17

Abstract: Long-term time series forecasting is a long-standing challenge in various applications. A central issue in time series forecasting is that methods should expressively capture long-term dependency. Furthermore, time series forecasting methods should be flexible when applied to different scenarios. Although Fourier analysis offers an alternative to effectively capture reusable and periodic patterns to achieve long-term forecasting in different scenarios, existing methods often assume high-frequency components represent noise and should be discarded in time series forecasting. However, we conduct a series of motivation experiments and discover that the role of certain frequencies varies depending on the scenarios. In some scenarios, removing high-frequency components from the original time series can improve the forecasting performance, while in others scenarios, removing them is harmful to forecasting performance. Therefore, it is necessary to treat the frequencies differently according to specific scenarios. To achieve this, we first reformulate the time series forecasting problem as learning a transfer function of each frequency in the Fourier domain. Further, we design Frequency Dynamic Fusion (FreDF), which individually predicts each Fourier component, and dynamically fuses the output of different frequencies. Moreover, we provide a novel insight into the generalization ability of time series forecasting and propose the generalization bound of time series forecasting. Then we prove FreDF has a lower bound, indicating that FreDF has better generalization ability. Experiment results and ablation studies demonstrate the effectiveness of FreDF.

Main paper: 
 related work: 
 With the advancement of deep learning, various methods, including CNN [2, 41] , RNN [12, 33] , and Transformer-based approaches [34] , have been developed for time series forecasting tasks. While most previous works focus on learning models in the time domain (e.g., Informer [19] , PeriodFormer [21] , GCformer [43] , Preformer [9] , and Infomaxformer [32] ), the core of these methods lies in utilizing correlations in the time domain to forecast future data.In the Fourier domain, FEDformer [45] applies Transformer using Frequency Enhanced Blocks and Attention modules, and CoST [37] explores learning seasonal representations. FEDformer and TimesNet [38] utilize frequency for analysis and period calculation, mapping one-dimensional time series to two-dimensional. FiLM [44] retains low-frequency Fourier components. However, these methods, involving Fourier analysis, do not explicitly model time series forecasting problems in the Fourier domain. In contrast, we reformulate the time series forecasting problem as learning a transfer function of each frequency in the Fourier domain.Classical time series decomposition techniques [5] have been utilized to decompose time series into seasonal and trend components for interpretability. For instance, Autoformer [39] decomposes the data into trend and seasonal components, then employs the Transformer architecture for independent forecasts. Similarly, CoST [37] decomposes sequences into trend and seasonal components, carrying out separate forecasts in both time and Fourier domains. Different from these methods, our approach introduces a novel framework for dynamic decomposition, prediction, and fusion of time series. 
empirical analysis: 
 Several studies suggest that high-frequency signals often represent noise and therefore should be discarded [44] . However, we argue that the role of certain frequencies is not universal and can be varied across different scenarios. In some cases, high-frequency signals may indeed be noise, while in others, they may hold valuable information. To confirm this idea, we conduct experiments on three datasets. The experimental settings and analysis are detailed below.Transfer Function Dynamaic Fusion FDBLock 1 FDBLock 𝑙 FDBLock L We conduct experiments on six datasets: ETT(ETTm1, ETTm2, ETTh1, ETTh2), Weather, ECL, and Exchange-rate. For each dataset, we conduct a set of four forecasting tasks with the lookback length and prediction length both fixed to 96. The first task is the regular forecasting task. For the other three tasks, we transform the input series from the training set to the Fourier domain using Fast Fourier Transform (FFT) [28] and divide the frequency spectra into three subsets: the first third of the spectrum as low-frequency, the second third as middle-frequency, and the final third as high-frequency. We randomly set the Fourier coefficients corresponding to different subsets of the frequency spectrum to zero respectively in different experiments, and convert it back into the time domain as the input series. This step is to eliminate the influence of a certain subset of frequencies when training the model. We train an individual vanilla Transformer [34] following the standard setting [23] for each task in all three datasets. We visualize the prediction and ground truth of future series for all tasks and all datasets in Figure 1 .Figure 1 (a) shows that in the ETTm1 dataset, after eliminating highfrequency signals, the prediction results are closer to the ground truth compared to using all frequencies. However, the prediction results are further from the ground truth after eliminating lowfrequency or mid-frequency signals. Figure 1(b) shows that in the ETTm2 dataset, we will get more accurate prediction results after eliminating low-frequency from historical time series. On the contrary, we will get worse prediction results after eliminating low-frequency form historical time series, shown in Figure 1 (c). In Figure 1 (d), no matter which subsets of the frequency we eliminate in the ETTh2 dataset, the prediction results are more accurate than those obtained using the original frequency for prediction, among which, eliminating high frequency has a better effect. As shown in Figure 1 (e), in the Exchange-rate dataset, the prediction results are more accurate in the long term after eliminating mid-frequency or high-frequency signals. Conversely, the results predicted by eliminating low-frequency signals or using all signals are closer to ground truth in the mid-term. In the weather dataset, which is shown in Figure 1 (f), the results predicted by eliminating lowfrequency signals are more accurate in the short-term, predicted by eliminating high-frequency are more accurate in the mid-term, and predicted by eliminating mid-frequency are more accurate in the long term. Yet, with the absence of prior knowledge, it remains challenging to distinguish noise from vital features. Therefore, we cannot merely mark high-frequency signals as noise. Considering this, it is necessary to utilize different frequencies for forecasting and subsequently adopt a more rational method to weight these forecasting results, thus attaining the final prediction. 
method: 
 In this section, we begin by reformulating the time series forecasting problem in the Fourier domain. Subsequently, we propose FreDF, a model designed to predict the output of each frequency component respectively, then combine each output of different components using a dynamic fusion strategy. We also present theoretical evidence supporting the idea that this dynamic fusion strategy enhances the generalization ability of FreDF.To achieve effective long-term forecasting, the model must go beyond merely memorizing past data points; it needs to grasp the underlying physical rules or inherent dynamics of the observed phenomena [26] . These dynamics governing the behavior of the time series, are presumed to be independent and unchanging over time [30] . In Fourier analysis, any time series can be represented by a set of orthogonal bases, i.e., the Fourier components; this orthogonal characteristic helps represent each rule with the dynamic of a single Fourier component [16] . In this section, we assume that the time series forecasting task is under a Linear Time-invariant (LTI) condition for the independent and time-invariant property of the inherent dynamics without loss of generality. Specifically, from [30] , let 𝑥 (𝑡) ∈ I be the input function and 𝑦 (𝑡) ∈ O be the output function, they are both functions of time 𝑡 defined in Banach space I and O. The output of the LTI system can be defined as:EQUATIONThe goal of time-series forecasting can be regarded as finding a suitable transfer function ℎ : I → O.In discrete case, the Equation 1 can be express as: EQUATIONEQUATIONwhere 𝑗 is the imaginary unit, X [𝑘] is the 𝑘-th Fourier components, 𝑘 ∈ [0, 1, . . . , 𝐾] and 𝐾 is the total number of Fourier components.Theorem 4.1. (The convolution theorem [14] ). The convolution theorem states that the Fourier transform of a convolution of two functions equals the point-wise product of their Fourier transform:EQUATIONApplying DFT to the output sequence 𝑌 according to Theorem 4.1 can convert the convolution in Equation 2 into a multiplication in the Fourier domain as:EQUATIONNote that ℎ is an unknown operator in the aforementioned analysis. Therefore we propose to estimate F (ℎ) directly with a learnable matrix 𝐻 𝜃 , where 𝜃 is the parameter. The transfer process is:EQUATIONwhere Ŷ is the estimated output in Fourier domain.Applying inverse Discrete Fourier transform F -1 (iDFT) can convert the estimated output back to the time domain with:EQUATIONThe learning objective for the learnable matrix is then to minimize the Mean Square Error (MSE) between the estimated output and the ground truth of the output:EQUATION)So far, the time series forecasting problem in the time domain has been reformulated as learning a transfer function 𝐻 𝜃 in the Fourier domain.Based on the findings in Section 3, there is no universal criteria to determine the importance of a specific frequency in different situations, for the role of certain frequency changes across various scenarios. For instance, a frequency may be crucial in one scenario but negatively impact performance in another. To address this variability, we propose FreDF (Frequency Dynamic Fusion), which dynamically calculates the weights for the estimated prediction of each frequency, taking their importance into account. Our proposed FreDF consists of the Embedding, the FDBlock, and the Projection layers. We provide the pseudo-code of FreDF in algorithm 1.To predict the future 𝑆 timestamps, we padding 𝑋 [𝑛] in time dimension with 𝑆 zeros as unknown data.In the Embedding module, we lift the input time series into an embedding space:EQUATIONhere, 𝑀 1 [𝑛] is the embedded representation of the input time series, 𝑓 : R 𝐶 → R 𝐷 is a multi-layer perceptron (MLP) used for the embedding, 𝐶 is the number of variables in the input time series, and 𝐷 is the dimension of the embedding space. It's crucial to note that we are embedding the feature dimensions, not the time dimensions. This means that the transformation does not affect the temporal characteristics of the data. Therefore, subsequent operations, such as Fourier transformations that target the time dimensions, remain unaffected by the embedding process.Within each FDBlock, we first apply Fast Fourier Transform (FFT) [28] to the input embedding 𝑀 𝑙 [𝑛], transforming it into the Fourier domain, which is an efficient algorithm to perform DFT:EQUATIONhere M 𝑙 [𝑘] is the Fourier components, 𝑙 = 1, 2, . . . , 𝐿 denotes the 𝑙-th FDBlock and 𝐿 is the total number of FDBlocks.To facilitate this independent processing, we propose a decoupling strategy. Instead of treating the Fourier components M 𝑙 [𝑘] ∈Input: Time series data 𝑋 ∈ R 𝑇 ×𝐶 , lookback length 𝑇 , predict length 𝑆, variables number 𝐶, FDBlock number 𝐿, token dimension 𝐷, 𝐾 is computed as 𝐾 = 𝑇 +𝑆 2 + 1, frequency spectrum length 𝐾.1: 𝑋 ′ = ZeroPadTimeSeries(𝑋 , 𝑆) ⊲ X ∈ R (𝑇 +𝑆 ) ×𝐶 2: 𝑀 1 = MLP(𝑋 ′ ) ⊲ 𝑀 1 ∈ R (𝑇 +𝑆 ) ×𝐷 3: for 𝑙 = 1 to 𝐿 do 4:for 𝑚 = 1 to 𝐾 do 5:M 𝑙 [𝑘] = F (𝑀 𝑙 ) [𝑘] ⊲ M 𝑙 ∈ R 𝐾 ×𝐷 6:for 𝑘 = 1 to 𝐾 do 7:if 𝑘 ≠ 𝑚 then end for 13:Learn transfer function 𝐻 𝑙,𝑚 ⊲ 𝐻 𝑙,𝑚 ∈ R 𝐷 ×𝐷 14:M 𝑙,𝑚 𝑜𝑢𝑡 [𝑘] = M 𝑙,𝑚 𝑖𝑛 [𝑘] • 𝐻 𝑙,𝑚 15:𝑍 𝑙,𝑚 = F -1 (M 𝑙,𝑚 𝑜𝑢𝑡 )16:end for 17: C 𝐾 ×𝐷 as a whole, we create 𝐾 copies of each frequency component and only retain the 𝑚-th frequency in each copy, denoted as M 𝑙,𝑚 𝑖𝑛 [𝑘]:𝑀 𝑙+1 = M𝑙 = 𝐾 𝑚=0 𝑍 𝑙,𝑚 • 𝑊 𝑚 , ⊲ M𝑙 ∈ R (𝑇 +𝑆 )EQUATIONThis strategy allows us to maintain the original dimensionality of the data while enabling independent processing of each frequency. Next, based on subsection 4.1, we aim to learn transfer functions 𝐻 𝑙,𝑚 ∈ C 𝐷 ×𝐷 , 𝑚 ∈ [1, . . . , 𝐾]for each independent component M 𝑙,𝑚 𝑖𝑛 = M 𝑙 [𝑚], 𝑚 ∈ [1, . . . , 𝐾], and obtain the estimated output M 𝑙,𝑚 𝑜𝑢𝑡 in the Fourier domain with:EQUATIONThe estimated output for frequency 𝑚 in the time domain 𝑍 𝑙,𝑚 [𝑛] can be obtained by applying inverse Fast Fourier Transform (iFFT) to M 𝑙,𝑚 𝑜𝑢𝑡 . The result of this operation is represented as:EQUATIONSo far we have decomposed the prediction process of each individual frequency 𝑚.Next, we apply a trainable weight vector 𝑊 ∈ R 𝐾 , where each component 𝑊 𝑚 represents the importance of the 𝑚-th frequency when predicting the output embedding. The estimated output M𝑙 [𝑛] is then represented as a weighted sum of all the individual frequency predictions 𝑍 𝑙,𝑚 [𝑛], with each prediction multiplied by its corresponding weight 𝑊 𝑚 . The estimated output M𝑙 [𝑛] is represented as a weighted sum of all the individual frequency predictions 𝑍 𝑙,𝑚 [𝑛], as given by the following equation:EQUATIONwhere each prediction 𝑍 𝑙,𝑚 [𝑛] is multiplied by its corresponding weight 𝑊 𝑚 and the 𝑊 𝑚 can be either static or dynamic, i.e. fixed or learnable.The FDBlock is formulated as an iterative architecture, where each output M𝑙 [𝑛] of the 𝑙-th layer serves as the input of the (𝑙 + 1)th layer.During the training process, we aim to learn the transfer functions 𝐻 𝑙,𝑚 ∈ C 𝐷 ×𝐷 and the weight vector𝑊 ∈ R 𝐾 by minimizing a loss function, which measures the difference between the estimated output Ŷ [𝑛] and the true output 𝑌 [𝑛].After 𝐿 FDBlocks, we apply another MLP 𝑔 : R 𝐷 → R 𝐶 to the final estimated output M𝐿 [𝑛], projecting it back to the variable space. The result of this operation is represented as:EQUATION)In this subsection, we provide a theoretical analysis to demonstrate the effectiveness of our dynamic fusion method. Without loss of generality, time series forecasting methods could be regarded as auto-regressive models [5] , from this perspective, we indicate that the generalization ability of time series prediction models could be reflected in the following two aspects: the capacity to capture the long-term dependency of time series, as well as the capacity to achieve good prediction results in different scenarios.For simplicity, consider the fusion strategy in a regression setting using a mean squared loss function. Firstly, we propose to characterize the generalization error bound using Rademacher complexity [3] and separate the bound into three components (Theorem 4.2). Meanwhile, we also give further proof based on the above separation to illustrate that the dynamic fusion method achieves a better ability to capture long-term dependency under certain conditions (Theorem 4.3). Secondly, we demonstrate that the quantity of parameters in our method is fewer than compared methods, which indicates that our method is more flexible to apply to more scenarios, experiment results in Section 5.3 also validate our illustration as well. See Appendix A for details.Specifically, we use X, Y, and Z to denote the input space (historical sequence), target space (prediction sequence), and latent space. Define 𝑢 : X -→ Z is a fusion mapping from the input space to the latent space, 𝑔 : Z -→ Y is a task mapping. Our goal is to learn the fusion operator 𝑓 = 𝑔 • 𝑢 (𝑥), which is essentially a regression model. Under an 𝐻 frequency components scenario, 𝑓 ℎ is the frequency-specific composite function of frequency component 𝑥 ℎ . The final prediction of the dynamic fusion method is calculated by: 𝑓 (𝑥) = 𝐻 ℎ=1 𝑤 ℎ • 𝑓 ℎ (𝑥 ℎ ), where 𝑓 (𝑥) denotes the final prediction. In contrast to static fusion, i.e., every frequency is given a predefined weight which is a constant, dynamic fusion calculates the weights of every frequency dynamically. To distinguish them, denotes 𝑤 ℎ 𝑠𝑡𝑎𝑡𝑖𝑐 the weight of frequency ℎ in static situation and 𝑤 ℎ 𝑑 𝑦𝑛𝑎𝑚𝑖𝑐 the weight of frequency ℎ in dynamic situation. The generalization error of regression model 𝑓 is defined as:EQUATIONwhere D is the unknown joint distribution, 𝑙 is mean squared loss function. For convenience, we simplify the regression loss 𝑙 (𝑓 ℎ (𝑥 ℎ ), 𝑦) as 𝑙 ℎ . Now we present the first main result of frequency fusion.Theorem 4.2. Given the historical sequence 𝑋 𝑇 ∈ R 𝑇 ×𝐶 and the ground truth of prediction sequence 𝑌 𝑇 ′ ∈ R 𝑇 ′ ×𝐶 , Ê (𝑓 ℎ ) is the empirical error of 𝑓 ℎ on frequency ℎ. Then for any hypothesis 𝑓 in the finite set 𝐹 and 1 > 𝛿 > 0, with probability at least 1 -𝛿, it holds thatEQUATIONwhere E(𝑤 ℎ ) represents the expectations of fusion weights on joint distribution D, ℜ ℎ (𝑓 ℎ ) represents Rademacher complexity, and 𝐶𝑜𝑣 (𝑤 ℎ , 𝑙 ℎ ) represents the covariance between fusion weight and loss.Theorem 4.2 demonstrates that the generalization error of the regression model is bounded by the weighted average performances of all regression operators for each frequency in terms of empirical loss, model complexity, and the covariance between fusion weight and regression loss of all frequencies. After the general error bound is established, the next goal is to verify if dynamic fusion indeed achieves a tighter bound than that of static fusion. Informally, in Equation 17, the covariance term measures the joint variability of 𝑤 ℎ and 𝑙 ℎ . However, in static fusion, 𝑤 ℎ 𝑠𝑡𝑎𝑡𝑖𝑐 is a constant, which means that the covariance is equal to zero for any static fusion method. Thus the generalization error bound of static fusion methods is reduced to:EQUATION+ 𝐻 ∑︁ ℎ=1 (𝑤 ℎ 𝑠𝑡𝑎𝑡𝑖𝑐 )ℜ ℎ (𝑓 ℎ ) + 𝑀 √︂ 𝑙𝑛(1/𝛿) 2𝐻.So when the summation of the average empirical loss, the average complexity is invariant or smaller in dynamic fusion and the covariance is no greater than zero, we can ensure that dynamic fusion provably outperforms static fusion. This theorem is formally presented as: Theorem 4.3. Let GError(𝑓 𝑑 𝑦𝑛𝑎𝑚𝑖𝑐 ), GError(𝑓 𝑠𝑡𝑎𝑡𝑖𝑐 ) be the upper bound of generalization regression error of dynamic and static fusion method respectively. Ê (𝑓 ℎ ) is the empirical error defined in Theorem 4.2. Then for any hypothesis 𝑓 𝑑 𝑦𝑛𝑎𝑚𝑖𝑐 , 𝑓 𝑠𝑡𝑎𝑡𝑖𝑐 in finite set 𝐹 and 1 > 𝛿 > 0, it holds that GError(𝑓 𝑑 𝑦𝑛𝑎𝑚𝑖𝑐 ) ≤ GError(𝑓 𝑠𝑡𝑎𝑡𝑖𝑐 ) (19) with probability at least 1 -𝛿, if we haveEQUATIONand 𝑟 (𝑤 ℎ 𝑑𝑦𝑛𝑎𝑚𝑖𝑐 , 𝑙 ℎ ) ≤ 0 (21) for all frequencies, where 𝑟 is the Pearson correlation coefficient which measures the correlation between fusion weights 𝑤 ℎ 𝑑 𝑦𝑛𝑎𝑚𝑖𝑐 and the loss of each frequency 𝑙 ℎ . Theorem 4.2 and Theorem 4.3 verify that the dynamic fusion method has a lower generalization bound, which indicates the capacity to capture the long-term dependency of our method. Furthermore, suppose for each frequency, the regression operator used in dynamic and static fusion are of the same architecture, then the intrinsic complexity ℜ ℎ (𝑓 ℎ ) can be invariant. Thus, in this case, it holds thatEQUATIONIn Equation 22, it is easy to derive the conclusion that our model has a lower average complexity, corresponding to a lower quantity of parameters during the training process. Experiment results in Section 5.3 also validate this conclusion. 
experiments: 
 In this section, we first provide the details of the implementation and datasets. Next, we present the comparison results on eight benchmark datasets. Lastly, we conduct ablation studies to evaluate the effectiveness of each module in our method.All the experiments are implemented in PyTorch [31] and trained on NVIDIA V100 32GB GPUs. We use ADAM [15] with an initial learning rate in {10 -3 , 10 -4 } and MSELoss for model optimization.An early stopping counter is employed to stop the training process after three epochs if no loss degradation on the valid set is observed. The mean square error (MSE) and mean absolute error (MAE) are used as metrics. All experiments are repeated 3 times and the mean of the metrics is used in the final results. The transfer function is implemented using the complex 64 data type in PyTorch. The batch size is set to 4 and the number of training epochs is set to 10. We set the number of FDBlocks in our proposed model 𝐿 ∈ {1, 2, 3}. The dimension of series representations 𝐷 ∈ {64, 128, 256, 512}, or it is not embedded at all. We set the dropout rates in {0, 0.2, 0.4}.We thoroughly evaluate the proposed FreDF on various long-term time series forecasting benchmarks. For better comparison, we follow the experiment settings of iTransformer in [23] the prediction lengths for both training and evaluation vary within the set 𝑆 ∈ {96, 192, 336, 720}, with a fixed lookback length of 𝑇 = 96.We carefully choose 10 well-acknowledged forecasting models as our benchmark, including (1) Transformer-based methods: iTransformer [23] , Autoformer [39] , FEDformer [45] , Stationary [25] , Crossformer [42] , PatchTST [27] ; (2) Linear-based In this section, we conduct ablation studies to examine the influence of transfer functions, dynamic fusion mechanisms, and the number of parameters in the proposed FreDF.We conduct an ablation study about the influence of the transfer function. We remove the transfer function in FreDF as the control group, follow the setup in Section 5.2, and carry out predictions on the ETTh1, ETTm1, and Exchangerate dataset. We present the results in Table 2 , which illustrates the crucial role of the transfer function and confirms the correctness of our analysis in Section 4.1.We conduct an ablation study to investigate the influence of dynamic fusion. We replace the learnable weight vector with a fixed weight vector and name this modified model FreSF. Predictions are carried out on the ETT(ETTh1, ETTh2, ETTm1, ETTm2), Weather, and Exchang-rate datasets using the setup outlined in Section 5.2. The results are presented in Table 3 . Additionally, we visualize the prediction results (with a prediction length 𝑆 = 96) for both FreSF and FreDF in Appendix C. The experimental results demonstrate the effectiveness of the dynamic fusion strategy.We use iTransformer [23] , patchTST [27] , FEDformer [45] and FiLM [44] for comparison, and calculate the number of model parameters when forecasting the same task, present the results in Table 4 . Our model despite using a relatively small number of parameters, can achieve good accuracy in prediction tasks. This also validates the superiority of our model, which is consistent with the theoretical analysis in Section 4.3. 
conclusion: 
 In this paper, we experimentally explore the different roles of frequency in various scenarios.  
: 
 alerts [10] , and traffic forecasting [20] . With the development of deep learning [17] , numerous methods have emerged for this forecasting tasks [2, 13, 35, 46] . A central issue in time series forecasting is that existing methods could not expressively capture long-term dependency, which is often characterized as periodicity and trends [5, 7, 11, 18, 36] . However, Fourier analysis has the strong potential to deal with long-term dependency, thereby makes related methods more flexible when adapted to different scenarios [37] .In the realm of time series forecasting, an effective approach to addressing long-term dependency is to utilize Fourier analysis [24, 37, 38, 44, 45] . Fourier analysis is a powerful method that represents complex time series as a series of cosine functions, each with its unique frequency [6] . This capability to represent infinitely longterm trends with a finite set of frequency components makes it efficiency when applied to long-term time series forecasting.Existing methods based on Fourier analysis often assume that high-frequency components represent noise and should be discarded during forecasting tasks [44] . However, we argue that the role of certain frequencies varies in different scenarios. To validate this assumption, we conduct experiments on three datasets, eliminating low, middle, and high-frequency components respectively from the input of the training set to train a vanilla Transformer [34] . The results, depicted in Figure 1 , suggest that eliminating certain frequencies may improve performance in specific datasets while decreasing in others. In Exchange-rate(Figure 1 (e)), we get more accurate prediction results after eliminating high frequencies. But it is less precise in Figure 1(b) . The same phenomenon occurs at other frequencies. More detailed experimental setup and analysis are provided in Section 3.These findings emphasize that simply marking high-frequency components as noise is undesirable. Without prior knowledge, determining which frequencies compose noise remains uncertain [11] . Consequently, it is necessary to utilize different frequencies for forecasting and assign more rational weights to these forecasting results to improve the final prediction.To separately assess the impact of different frequencies, it is necessary to predict each frequency individually. To begin with, we propose a mathematical reformulation of the time series forecasting task in the Fourier domain. Then we propose Frequency Dynamic Fusion (FreDF), a novel framework to process time series datasets in decomposition, forecasting, and dynamic fusion, which individually forecasts each Fourier component, and dynamically fuses the output of different frequencies. The advantage of dynamic fusion lies in its capacity to flexibly adjust the weights of each frequency component, leading to more precise predictions. Additionally, we propose the generalization bound of time series forecasting based on Rademacher complexity [3] , and we prove that dynamic fusion improves the model's generalization ability. Experimental results on long-term forecasting datasets also confirm the superiority of FreDF. Overall, our contributions can be summarized as the following four points:• We conduct a series of experiments to explore the role of different frequencies in prediction. Based on experimental phenomena we discover that the role of certain frequencies varies depending on the scenarios.  
 
 Appendix:  


Decision: The method tackles long-term time series forecasting challenges by emphasizing the capture of robust long-term dependencies and flexibility across scenarios. The authors introduce Frequency Dynamic Fusion (FreDF), predicting and integrating individual Fourier components dynamically. Additionally, the study explores generalization in time series forecasting and establishes a lower bound to demonstrate FreDF's effectiveness. Extensive experiments across diverse datasets validate the method's efficacy.

Initially, the reviewers are not supportive of the work (wr,br,wa,br), with main concerns on lacking comparision with SOTA, and lacking experiments with other settings. The rebuttal managed to address some concerns, where one reviewer raised the score from br to ba, two reviewers maintained the original score of br and wa, and one reviewer did not give the final review. The AC reviewed the paper and the rebuttal, and found the finding of the role of certain frequencies varies depending on the scenarios interesting, and the performance was good. It seems that, the request comparisions and experiments from reviewer e9Zd and oZzd are completed.

Reviews: review: paper_summary: paper_summary: This paper studies the frequency learning and analyze the useful frequencies in time series forecasting. Several experiments are conducted.
strength_weakness: strength_weakness: 1. the writing is understandable.
2. the paper is easy to follow.
limitations: limitations: 1. The experiments mostly compare with Transformer-based forecasting methods, but ignore the frequency-based time series forecasting methods, such as FreTS [1], FITS [2].
2. This paper didn't discuss many state-of-the-art frequency-based time series forecasting methods including [1-4], which makes the motivation and contribution lack novelty.

[1] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting. In NeurIPS 2023.
[2] FITS: Modeling Time Series with 10⁢k Parameters. In ICLR 2024.
[3] FourierGNN: Rethinking multivariate time series forecasting from a pure graph perspective. In NeurIPS 2023.
[4] TSLANet: Rethinking Transformers for Time Series Representation Learning. In ICML 2024.
paper_summary: paper_summary: This paper analyzes how high-frequency components may play different roles in time series forecasting across various scenarios. It introduces Frequency Dynamic Fusion (FreDF), a method that individually predicts each Fourier component and dynamically integrates the outputs of different frequencies. Besides, they offer insight into the generalization capabilities of time series forecasting.
strength_weakness: strength_weakness: (1)  The paper introduces how the role of frequencies in forecasting changes depending on the scenario.

(2) They design a method called Frequency Dynamic Fusion (FreDF), which individually forecasts each Fourier component and dynamically fuses the outputs from different frequencies.

(3)  By proposing and validating a lower generalization bound for the FreDF method, this approach demonstrates enhanced generalization capabilities, ensuring robustness across various datasets and conditions.
limitations: limitations: (1) The paper focuses on time series prediction, specifically on frequency domain processing, which does not align closely with the main theme of the conference.

(2) The experimental design in the article arbitrarily segments the frequency domain into three stages based on their order of appearance, which lacks scientific rigor as frequencies typically overlap.

(3) The description of the dynamic fusion module's strategy in the article is somewhat vague and could benefit from clearer exposition.
paper_summary: paper_summary: This paper proposes a method to learn a corresponding pattern for each frequency component. In my opinion, this is a new perspective.
strength_weakness: strength_weakness: 1. The paper is well written, and easy to follow.

2. The paper proposes an interesting perspective to  explore the different roles of different frequency components.

2. The evaluation has demonstrated that the method have achieved superior performance.
limitations: limitations: 1. Some frequency-based methods have not been discussed and compared, such as FITS[1] and FreTS[2].

2. The experiment part seems to be lack of the analysis of the transfer function (e.g., visualization) and efficiency analysis.

Questions:

1. Could you further provide more details about the definition, explanation of transfer function? 

[1]: FITS: Modeling Time Series with 10k Parameters. ICLR 2024

[2]: Frequency-domain MLPs are More Effective Learners in Time Series Forecasting. NeurIPS 2023
paper_summary: paper_summary: To realize Time-Series Forecasting, this paper designs Frequency Dynamic Fusion (FreDF), which individually predicts each Fourier component, and dynamically fuses the output of different frequencies.
strength_weakness: strength_weakness: 1. The idea of analyzing the effects of different frequency domains is interesting.

2. The performance of the model is proved by experiments.

3. Well written.
limitations: limitations: 1. My biggest concern is that the current SOTA time series models mostly challege the setting that the input length is much smaller than the output prediction's length. In this paper, e.g. input: 96 vs. output: 720. In the main experiment, the history length is much smaller than the prediction length, i.e. T << S. A more practical setting is that for fixed prediction length S, the input length T should be viewed as a hyper-parameter to tune, which is used in [1,2]. Can you evaluate FreDF under this setting?

2. How many frequency domain components are used in this paper? How can authors obtain different frequency domain components after Fourier transform? From my current understanding, it seems that different components are obtained by downsampling. Some technical details need to be clearly stated. 

3. I think the ablation experiments in different frequency domains in the supplementary material should be placed in the main text, because this is the core motivation of the author.

4. The main role of frequency domain analysis is to mine periodic and global patterns. Given that this is the core motivation of the authors, I recommend that they conduct experiments on datasets with stable periodicity [3], such as PEMS04 and PEMS08.

5. What are the advantages of the multi-frequency domain over the multi-patch or multi-granularity [4,5]?

[1]  Are Transformers Effective for Time Series Forecasting?

[2] A TIME SERIES IS WORTH 64 WORDS: LONG-TERM FORECASTING WITH TRANSFORMERS

[3] Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis

[4] Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting

[5] TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting