FilterNet: Harnessing Frequency Filters for Time Series Forecasting

Authors: kun  yi
jingru  fei
qi  zhang
hui  he
shufeng  hao
defu  lian
wei  fan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 12:09:20

Abstract: Given the ubiquitous presence of time series data across various domains, precise forecasting of time series holds significant importance and finds widespread real-world applications such as energy, weather, healthcare, etc. While numerous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-art performance in time series forecasting. However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series with thousands of points. In this paper, we explore a novel perspective of enlightening signal processing for deep time series forecasting. Inspired by the filtering process, we introduce one simple yet effective network, namely FilterNet, built upon our proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals. Concretely, we propose two kinds of learnable filters in the FilterNet: (i) Plain shaping filter, that adopts a universal frequency kernel for signal filtering and temporal modeling; (ii) Contextual shaping filter, that utilizes filtered frequencies examined in terms of its compatibility with input signals for
dependency learning. Equipped with the two filters, FilterNet can approximately surrogate the linear and attention mappings widely adopted in time series literature, while enjoying superb abilities in handling high-frequency noises and utilizing the whole frequency spectrum that is beneficial for forecasting. Finally, we conduct extensive experiments on eight time series forecasting benchmarks, and experimental results have demonstrated our superior performance in terms of both effectiveness and efficiency compared with state-of-the-art methods. Our code is available at$^1$.

Main paper: 
 introduction: 
 Time series forecasting has been playing a pivotal role across a multitude of contemporary applications, spanning diverse domains such as climate analysis [1] , energy production [2] , traffic flow patterns [3] , financial markets [4] , and various industrial systems [5] . The ubiquity and profound significance of time series data has recently garnered substantial research efforts, culminating in a plethora of deep learning forecasting models [6] that have significantly enhanced the domain of time series forecasting.Previously, leveraging different kinds of deep neural networks derives a series of time series forecasting methods, such as Recurrent Neural Network-based methods including DeepAR [7] , LSTNet [8] ,(c) FilterNet with MSE=2.7e-05 Figure 1 : Performance of Mean Squared Error (MSE) on a simple synthetic multi-frequency signal. More details about the experimental settings can be found in Appendix C. 4 .Convolution Neural Network-based methods including TCN [9] , SCINet [10] , etc. Recently, however, with the continuing advancement of deep learning, two branches of methods that received particularly more attention have been steering the development of time series forecasting, i.e., Multilayer Perceptron (MLP)-based methods, such as N-BEATS [11] , DLinear [12] , and FreTS [13] , and Transformerbased methods, such as Informer [14] , Autoformer [15] , PatchTST [16] , and iTransformer [17] . While MLP-based models are capable of providing accurate forecasts, Transformer-based models continue to achieve state-of-the-art time series forecasting performance.However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series composed of thousands of timesteps. In designing a very simple simulation experiment on the synthetic data only composed of a low-, middle-and high-frequency signal respectively (see Figure 1 (a)), we find the state-of-the-art iTransformer [17] model performs much worse in forecasting (Figure 1 (b) and Figure 1(c) ). This observation shows that state-of-the-art Transformer-based model cannot utilize the full spectrum information, even for a naive signal of three different frequency components. In contrast, in the field of signal processing, a frequency filter enjoys many good properties such as frequency selectivity, signal conditioning, and multi-rate processing. These could have great potential in advancing the model's ability to extract key informative frequency patterns in time series forecasting. Thus, inspired by the filtering process [18] in signal processing, in this paper, we introduce one simple yet effective framework, namely FilterNet, for effective time series forecasting. Specifically, we start by proposing two kinds of learnable filters as the key units in our framework: (i) Plain shaping filter, which makes the naive but universal frequency kernel learnable for signal filtering and temporal modeling, and (ii) Contextual shaping filter, which utilizes filtered frequencies examined in terms of its compatibility with input signals for dependency learning. The plain shaping filter is more likely to be adopted in predefined conditions and efficient in handling simple time series structures, while the contextual filter can adaptively weight the filtering process based on the changing conditions of input and thus have more flexibility in facing more complex situations. Besides, these two filters as the built-in functions of the FilterNet can also approximately surrogate the widely adopted linear mappings and attention mappings in time series literature [12, 14, 17] . This also illustrates the effectiveness of our FilterNet in forecasting by selectively passing or attenuating certain signal components while capturing the core time series structure with adequate learning expressiveness. Moreover, since filters are better fit in the stationary frequency domain, we let filters wrapped by two reversible transformations, i.e., instance normalization [19] and fast Fourier transform [20] to reduce the influence of non-stationarity and accomplish the domain transformation of time series respectively. In summary, our contributions can be listed as follows:• In studying state-of-the-art deep Transformer-based time series forecasting models, an interesting observation from a simple simulation experiment motivates us to explore a novel perspective of enlightening signal processing techniques for deep time series forecasting.• Inspired by the filtering process in signal processing, we introduce a simple yet effective network, FilterNet, built upon our proposed two learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals, thereby enhancing the forecasting performance.• We conduct extensive experiments on eight time series forecasting benchmarks, and the results have demonstrated that our model achieves superior performance compared with state-of-the-art forecasting algorithms in terms of effectiveness and efficiency. 
related work: 
 Deep Learning-based Time Series Forecasting In recent years, deep learning-based methods have gained prominence in time series forecasting due to their ability to capture nonlinear and complex correlations [21] . These methods have employed various network architectures to learn temporal dependencies, such as RNN [8, 7] , TCN [9, 10] , etc. Notably, MLP-and Transformerbased methods have achieved competitive performance, emerging as state-of-the-art approaches. N-HiTS [22] integrates multi-rate input sampling and hierarchical interpolation with MLPs to enhance univariate forecasting. DLinear [12] introduces a simple approach using a single-layer linear model to capture temporal relationships between input and output time series data. RLinear [23] utilizes linear mapping to model periodic features, demonstrating robustness across diverse periods with increasing input length. In contrast to the simple structure of MLPs, Transformer's advanced attention mechanisms [24] empower the models [14, 15, 25, 26] to capture intricate dependencies and longrange interactions. PatchTST [16] segments time series into patches as input tokens to the Transformer and maintaining channel independence. iTransformer [17] inverts the Transformer's structure by treating independent series as variate tokens to capture multivariate correlations through attention.In recent developments, frequency technology has been increasingly integrated into deep learning models, significantly improving state-of-theart accuracy and efficiency in time series analysis [27] . These models leverage the benefits of frequency technology, such as high efficiency [28, 29] and energy compaction [13] , to enhance forecasting capabilities. Concretely, Autoformer [15] introduces the auto-correlation mechanism, improving self-attention implemented with Fast Fourier Transforms (FFT). FEDformer [25] enhances attention with a FFT-based frequency approach, determining attentive weights from query and key spectrums and conducting weighted summation in the frequency domain. DEPTS [30] utilizes Discrete Fourier Transform (DFT) to extract periodic patterns and contribute them to forecasting. FiLM [31] employs Fourier analysis to retain historical information while filtering out noisy signals.FreTS [13] introduces a frequency-domain Multi-Layer Perceptrons (MLPs) to learn channel and temporal dependencies. FourierGNN [29] transfers the operations of graph neural networks (GNNs) from the time domain to the frequency domain. FITS [32] applies a low pass filter to the input data followed by complex-valued linear mapping in the frequency domain.Unlike these methods, in this paper we propose a simple yet effective model FilterNet developed from a signal processing perspective, and apply all-pass frequency filters to design the network directly, rather than incorporating them into other network architectures, such as Transformers, MLPs, or GNNs, or utilizing them as low-pass filters, as done in FITS [32] and FiLM [31] . 
preliminaries: 
 Frequency Filters Frequency filters [33] are mathematical operators designed to modify the spectral content of signals. Specifically, given an input time series signal x[n] with its corresponding Fourier transform X [k], a frequency filter H[k] is applied to the signal to produce an output signal y[n] with its corresponding Fourier transform Y[k] = X [k]H[k].The frequency filter H[k] alters the amplitude and phase of specific frequency components in the input time series signal x[n] according to its frequency response characteristics, thereby shaping the spectral content of the output signal.According to the Convolution Theorem [34] , the point-wise multiplication in the frequency domain corresponds to the circular convolution operation between two corresponding signals in the time domain. Consequently, the frequency filter process can be expressed in the time domain as:EQUATIONwhere h[n] is the inverse Fourier transform of H[k] and ⊛ denotes the circular convolution operation. This formulation underscores the intrinsic connections between the frequency filter process and the circular convolution in the time domain, and it indicates that the frequency filter process can efficiently perform circular convolution operations. In time series forecasting, Transformer-based methods have achieved state-of-the-art performance, largely due to the self-attention mechanism [14, 15, 25, 17] , which can be interpreted as a form of global circular convolution [35] . This perspective opens up the possibility of integrating frequency filter technologies, which are well-known for their ability to isolate and enhance specific signal components, to further improve time series forecasting models.  
methodology: 
 As aforementioned, frequency filters enjoy numerous advantageous properties for time series forecasting, functioning equivalently to circular convolution operations in the time domain. Therefore, we design the time series forecaster from the perspective of frequency filters. In this regard, we propose FilterNet, a forecasting architecture grounded in frequency filters. First, we introduce the overall architecture of FilterNet in Section 4.1, which primarily comprises the basic blocks and the frequency filter block. Second, we delve into the details of two types of frequency filter blocks: the plain shaping filter presented in Section 4.2 and the contextual shaping filter discussed in Section 4.3.The overall architecture of FilterNet is depicted in Figure 2 , which mainly consists of the instance normalization part, the frequency filter block, and the feed-forward network. Specifically, for a given time series input X = [X 1:L 1 , X 1:L 2 , ..., X 1:L N ] ∈ R N ×L with the number of variables N and the lookback window length L, where X 1:L N ∈ R L denotes the N -th variable, we employ FilterNet to predict the future τ time steps Y = [X L+1:L+τ 1 , X L+1:L+τ 2 , ..., X L+1:L+τ N ] ∈ R N ×τ . We provide further analysis about the architecture design of FilterNet in Appendix A.Instance Normalization Non-stationarity is widely existing in time series data and poses a crucial challenge for accurate forecasting [19, 36] . Considering that time series data are typically collected over a long duration, these non-stationary sequences inevitably expose forecasting models to distribution shifts over time. Such shifts can result in performance degradation during testing due to the covariate shift or the conditional shift [37] . To address this problem, we utilize an instance normalization method, denoted as Norm, on the time series input X, which can be formulated as:EQUATIONwhere Mean L denotes the operation that calculates the mean value along the time dimension, and Std L represents the operation that calculates the standard deviation along the time dimension.Correspondingly, the inverse instance normalization, denoted as InverseNorm, is formulated as:EQUATIONwhere P = [P L+1:L+τ Frequency Filter Block Previous representative works primarily leverage MLP architectures (e.g., DLinear [12] , RLinear [23] ) or Transformer architectures (e.g., PatchTST [16] , iTransformer [17] ) to model the temporal dependencies among time series data. As mentioned earlier, time series forecasters can be implemented through performing a frequency filter process in the frequency domain, and thus we propose to directly apply the frequency filter in the frequency domain, denoted as FilterBlock, to replace the aforementioned methods for modeling corresponding temporal dependencies, such as:EQUATION)where F is Fourier transform, F -1 is inverse Fourier transform and H f ilter is the frequency filter.Inspired by MLP that randomly initializes a learnable weight parameters and Transformer that learns the data-dependent attention scores from data (further explanations are provided in Appendix B), we introduce two types of frequency filters, i.e., plain shaping filter (PaiFilter) and contextual shaping filter (TexFilter). PaiFilter applies a random initialized learnable weight H ϕ to instantiate the frequency filter H f ilter , and then the frequency filter process is reformulated as:EQUATIONTexFilter learns a data-dependent frequency filter H φ (F(Z)) from the input data by using a neural network H φ (), and then the corresponding frequency filter process is reformulated as:EQUATIONFeed-forward Network The frequency filter block has captured temporal dependencies among time series data, and then we employ a feed-forward network (FFN) to project them back onto the time series data and make predictions for the future τ time steps. As the output P of FFN are instance-normalized values, we conduct an inverse instance normalization operation (InverseNorm) on them and obtain the final predictions Ŷ. The entire process can be formulated as follows:EQUATIONŶ = InverseNorm(P).PaiFilter instantiates the frequency filter by randomly initializing learnable parameters and then performing multiplication with the input time series. In general, for multivariate time series data, the channel-independence strategy in channel modeling has proven to be more effective compared to the channel-mixing strategy [12, 16] . Following this principle, we also adopt the channel-independence strategy for designing the frequency filter. Specifically, we propose two types of plain shaping filters: the universal type, where parameters are shared across different channels, and the individual type, where parameters are unique to each channel, as illustrated in Figure 3(a) .Given the time series input Z ∈ R N ×L and the plain shaping filter H ϕ , we apply PaiFilter by:EQUATIONS = F -1 (S),where F is Fourier transform, F -1 is inverse Fourier transform, ⊙ L denotes the element-wise product along L dimension, H(U ni) ϕ ∈ C 1×L is the universal plain shaping filter, H (Ind) ϕ∈ C N ×L is the individual plain shaping filter, and S ∈ R N ×L is the output of PaiFilter. We further compare and analyze the two types of PaiFilter in Section 5.3.In contrast to PaiFilter, which randomly initializes the parameters of frequency filters and fixes them after training, TexFilter learns the parameters generated from the input data, allowing for better adaptation to the data. Consequently, we devise a neural network H φ that flexibly adjusts the frequency filter in response to the input data, as depicted in Figure 3(b) .Given the time series input Z ∈ R N ×L and its corresponding Fourier transform denoted as Z = F(Z) ∈ C N ×L , the network H φ is utilized to derive the contextual shaping filter, expressed as H φ : C N ×L → C N ×D . First, it embeds the raw data by a linear dense operation κ : C L → C D to improve the capability of modeling complex data. Then, it applies a series of complex-value multiplication with K learnable parameters W 1:K ∈ C 1×D yielding σ(κ(Z) ⊙ W 1:K ) where σ is the activation function, and finally outputs H φ (Z). Then we apply TexFilter by:Z = F(Z), E = κ(Z), H φ (Z) = σ(E ⊙ D W 1:K ), W 1:K = K i=1 W i (9) S = E ⊙ D H φ (Z), S = F -1 (S),where ⊙ D denotes the element-wise product along D dimension and S ∈ R N ×D is the output. The contextual shaping filter can adaptively weight the filtering process based on the changing conditions of input and thus have more flexibility in facing more complex situations. 
experiments: 
 In this section, we extensively experiment with eight real-world time series benchmarks to assess the performance of our proposed FilterNet. Furthermore, we conduct thorough analytical experiments concerning the frequency filters to validate the effectiveness of our proposed framework.Datasets We conduct empirical analyses on diverse datasets spanning multiple domains, including traffic, energy, and weather, among others. Specifically, we utilize datasets such as ETT datasets [14] , Exchange [38] , Traffic [15] , Electricity [15] , and Weather [15] , consistent with prior studies on long time series forecasting [16, 17, 25] . We preprocess all datasets according to the methods outlined in [16, 17] , and normalize them with the standard normalization method. We split the datasets into training, validation, and test sets in a 7:2:1 ratio. More dataset details are in Appendix C.1.Baselines We compare our proposed FilterNet with the representative and state-of-the-art models to evaluate their effectiveness for time series forecasting. We choose the baseline methods from four categories: (1) Frequency-based models, including FreTS [13] and FITS [32] ; (2) TCN-based models, such as MICN [39] and TimesNet [40] ; (3) MLP-based models, namely DLinear [12] and RLinear [23] ; and (4) Transformer-based models, which include Informer [14] , Autoformer [15] , Pyraformer [26] , FEDformer [25] , PatchTST [16] , and the more recent iTransformer [17] for comparison. Further details about the baselines can be found in Appendix C.2.Implementation Details All experiments are implemented using Pytorch 1.8 [41] and conducted on a single NVIDIA RTX 3080 10GB GPU. We employ MSE (Mean Squared Error) as the loss function and present MAE (Mean Absolute Errors) and MSE (Mean Squared Errors) results as the evaluation metrics. For further implementation details, please refer to Appendix C.3. We present the forecasting results of our FilterNet compared to several representative baselines on eight benchmarks with various prediction lengths in Table 1 . Additional results with different lookback window lengths are reported in Appendix F and G.3. Table 1 demonstrates that our model consistently outperforms other baselines across different benchmarks. The average improvement of FilterNet over all baseline models is statistically significant at the confidence of 95%. Specifically, PaiFilter performs well on small datasets (e.g., ETTh1), while TexFilter excels on large datasets (e.g., Electricity) due to the ability to model the more complex and contextual correlations present in larger datasets. Also, the prediction performance of iTransformer [17] , which achieves the best results on the Traffic dataset (862 variables) but not on smaller datasets, suggests that simpler structures may be more suitable for smaller datasets, while larger datasets require more contextual structures due to their complex relationships. Compared with FITS [32] built on low-pass filters, our model outperforms it validating an all-pass filter is more effective. Since PaiFilter is simple yet effective, the following FilterNet in the experimental section refer to PaiFilter unless otherwise stated.In this part, we conduct experiments to delve into a thorough exploration of frequency filters, including their modeling capabilities, comparisons among different types of frequency filters, and the various factors impacting their performance. Detailed experimental settings are provided in Appendix C.5.Modeling Capability of Frequency Filters Despite the simplicity of frequency filter architecture, Table 1 demonstrates that this architecture can also achieve competitive performance. Hence, in this part, we perform experiments to explore the modeling capability of frequency filters. Particularly, given the significance of trend and seasonal signals in time series forecasting, we investigate the efficacy of simple filters in modeling these aspects. We generate a trend signal and a multi-period signal with noise, and then we leverage the frequency filters (i.e., PaiFilter) to perform training on the two signals respectively. Subsequently, we produce prediction values based on the trained frequency filters. Specifically, Figure 4 (a) and 4(b) show that the filter can effectively model trend and periodic signals respectively even compared with state-of-the-art iTransformer [17] when the data contains noise. These results illustrate that the filter has excellent and robust modeling capabilities for trend and periodic signals which are important components for time series. This can also explain the effectiveness of FilterNet since the filters can perform well in such settings. signifies filters unique to each channel. The evaluation results are presented in Table 2 . It demonstrates that filters shared among different channels consistently outperform across all prediction lengths. In addition, we visualize the prediction values predicted on the ETTh1 dataset by the two different types of filters, as illustrated in Figure 11 (see Appendix G.1). The visualization reveals that the prediction values generated by filters shared among different channels exhibit a better fit than the unique filters. Therefore, the strategy of channel sharing seems to be better suited for time series forecasting and filter designs, which is also validated in DLinear [12] and PatchTST [16] .We present a prediction showcase on ETTh1 dataset, as shown in Figure 5 . We select iTransformer [17] , PatchTST [16] as the representative compared methods. Comparing with these different state-of-the-art models, we can observe FilterNet delivers the most accurate predictions of future series variations, which has demonstrated superior performance. In addition, we include more visualization cases and please refer to Appendix G.3. Visualization of Frequency Filters To provide a comprehensive overview of the frequency response characteristics of frequency filters, we conduct visualization experiments on the Weather, ETTh, and Traffic datasets with the lookback window length of 96 and the prediction length of 96.The frequency response characteristics of learned filters are visualized in Figure 7 . From Figures 7(a ) and 7(b), we can observe that compared with Transformer-based approaches (e.g., iTransformer [17] ,PatchTST [16] ) tend to attenuate high-frequency components and preserve low-frequency information, FilterNet exhibits a more nuanced and adaptive filtering behavior that can be capable of attending to all frequency components. Figure 7 (c) demonstrates that the main patterns of the Traffic dataset primarily resides in the low-frequency range. This observation also explains why iTransformer performs well on the Traffic dataset, despite its low-frequency nature. Overall, Figure 7 demonstrates that FilterNet possesses comprehensive processing capabilities. Moreover, visualization experiments conducted on the ETTm1 dataset across various prediction lengths, as shown in Figure 8 , further illustrate the extensive processing abilities of FilterNet. Additional results conducted on different lookback window lengths and prediction lengths can be found in Appendix G.2.The complexity of FilterNet is O(Log L) where L is the input length. To comprehensively assess efficiency, we evaluate it based on two dimensions: memory usage and training time. Specifically, we choose two different sizes of datasets: the Exchange (8 variables, 7588 timestamps) and Electricity datasets (321 variables, 26304 timestamps). We compare the efficiency of our FilterNet with the representative Transformer-and MLP-based methods under the same settings (lookback window length of 96 and prediction length of 96), and the results are shown in Figure 6 . It highlights that FilterNet surpasses other Transformer models, regardless of dataset size. While our approach exhibits similar efficiency to DLinear, our effective results outperform its performance. In Appendix E, we further conduct ablation studies to validate the rationale of FilterNet designs.  
conclusion remarks: 
 In this paper, we explore an interesting direction from a signal processing perspective and make a new attempt to apply frequency filters directly for time series forecasting. We propose a simple yet effective architecture, FilterNet, built upon our proposed two kinds of frequency filters to accomplish the forecasting. Our comprehensive empirical experiments on eight benchmarks have validated the superiority of our proposed method in terms of effectiveness and efficiency. We also include many careful and in-depth model analyses of FilterNet and the internal filters, which demonstrate many good properties. We hope this work can facilitate more future research integrating signal processing techniques or filtering processes with deep learning on time series modeling and accurate forecasting.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: Please see implementation details.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in the appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: The average improvement of FilterNet over all baseline models is statistically significant at the confidence of 95%.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: This paper is with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We have discussed the broader impact of time series forecasting in both abstract and introduction.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper does not have this risk.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We included it in implementation details.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The Necessity of Instance Normalization Block From the frequency perspective, the mean value is equal to zero frequency component. Specifically, given a signal x[n] with a length of N , we can obtain its corresponding discrete Fourier transform X [k] by:EQUATION)where j is the imaginary unit. We set k as 0 and then,EQUATIONAccording the above equation, we can find that the mean value1 N N -1 n=0 x[n]in the time domain is equal to the zero frequency component X [0] in the frequency domain. Similarly, we can also view the standard deviation from the frequency perspective, and it is related to the power spectral density.According to these analysis, the instance normalization is analogous to a form of data preprocessing. Given that filters are primarily crafted to discern particular patterns within input data, while instance normalization aims to normalize each instance in a dataset, a function distinct from the conventional role of filters, we treat instance normalization as a distinct block within our FilterNet architecture.Frequency Filter Block Recently, Transformer-and MLP-based methods have emerged as the two main paradigms for time series forecasting, exhibiting competitive performance compared to other model architectures. Building on prior work that conceptualizes self-attention and MLP architectures as forms of global convolution [35, 13] , it becomes apparent that frequency filters hold promise for time series forecasting tasks. Just as self-attention mechanisms capture global dependencies and MLPs learn to convolve features across the entire input space, frequency filters offer a means to extract and emphasize specific temporal patterns and trends from time series data. By applying frequency filters to time series data, we can learn recurring patterns, trends, and periodic behaviors that are essential for forecasting future time series data and making accurate predictions.Feed-forward Network Incorporating a feed-forward network within the FilterNet architecture is essential for enhancing the model's capacity to capture complex relationships and non-linear patterns within the data. While frequency filters excel at extracting specific frequency components and temporal patterns from time series data, they may not fully capture the intricate dependencies and higher-order interactions present in real-world datasets [17] . By integrating a feed-forward network, the model gains the ability to learn hierarchical representations and abstract patterns from the input data, allowing it to capture more nuanced relationships and make more accurate predictions. This combination of frequency filters and a feed-forward network leverages the strengths of both approaches, enabling the model to effectively process and analyze time series data across different frequency bands while accommodating the diverse and often nonlinear nature of temporal dynamics. Overall, the inclusion of a feed-forward network enriches the expressive power of FilterNet, leading to improved forecasting performance and robustness across various domains.Self-attention mechanism is a highly data-dependent operation that both derives its parameters from data and subsequently applies these parameters back to the data. Concretely, given the input data X, the self-attention can be formulated as:EQUATIONwhere Q (queries), K (keys), and V (values) are linear transformations of the input data X, as:EQUATIONwhere W Q , W K , and W V are learned weight matrices. Since the values Q, K, and V are derived directly from the input data X, the attention scores SA are inherently dependent on the data.Unlike self-attention mechanism that dynamically adapts to the input data during inference, MLPs maintain a consistent architecture regardless of the dataset characteristics. Specifically, for the input data X, MLP can be formulated as:EQUATIONwhere W are the learned weights and b are the learned biases. Once trained, the weights W and biases b remain static, meaning that they do not dynamically change in response to new data inputs.MLPs are straightforward, less data-dependent models that apply fixed transformations to the input data, making them suitable for tasks with static relationships between the input data. In contrast, self-attention mechanisms are highly data-dependent, dynamically computing attention scores based on the input data to capture complex, context-specific dependencies, making them ideal for tasks requiring an understanding of sequential or structured data.Inspired by the two paradigms, FilterNet designs two corresponding types of filters: plain shaping filters and contextual shaping filters. Plain shaping filters offer stability and efficiency, making them suitable for tasks with static relationships. In contrast, contextual shaping filters provide the flexibility to capture dynamic dependencies, excelling in tasks that require context-sensitive analysis. This dual approach allows FilterNet to effectively handle a wide range of data types and forecasting scenarios, combining the best aspects of both paradigms to achieve superior performance.We evaluate the performance of our proposed FilterNet on eight popular datasets, including Exchange, Weather, Traffic, Electricity, and ETT datasets. In detail, the Exchange2 dataset collects daily exchange rates of eight different countries including Singapore, Australia, British, Canada, Switzerland, China, Japan, and New Zealand ranging from 1990 to 2016. The Weather 3 We choose twelve well-acknowledged and state-of-the-art models for comparison to evaluate the effectiveness of our proposed FilterNet for time series forecasting, including Frequency-based models, TCN-based models, MLP-based models, and Transformer-based models. We introduce these models as follows:FreTS [13] introduces a novel direction of applying MLPs in the frequency domain to effectively capture the underlying patterns of time series, benefiting from global view and energy compaction. The official implementation is available at https://github.com/aikunyi/FreTS. To ensure fair and objective comparison, the results showed in Table 4 are obtained using instance normalization instead of min-max normalization in the original code. FITS [32] performs time series analysis through interpolation in the complex frequency domain, enjoying low cost with 10k parameters. The official implementation is available at https://github. com/VEWOXIC/FITS. Because its original paper doesn't provide the forecasting results with the fixed lookback length L = 96, we report the performance of FITS with lookback length L = 96 under five runs in Table 1 .MICN [39] employs multi-scale branch structure to model different potential patterns separately with linear complexity. It combines local features and global correlations to capture the overall view of time series. The official implementation is available at https://github.com/wanghq21/MICN. The experimental results showed in Table 4 follow its original paper.TimesNet [40] transforms 1D time series into a set of 2D tensors based on multiple periods to analyse temporal variations. The above transformation allows the 2D-variations to be easily captured by 2D kernels with encoding the intraperiod-and interperiod-variations into the columns and rows of the 2D tensors respectively. The official implementation is available at https://github.com/thuml/ TimesNet. The results showed in Table 1 follow iTransformer [17] and the results showed in Table 5 follow RLinear [23] .DLinear [12] utilizes a simple yet effective one-layer linear model to capture temporal relationships between input and output sequences. The official implementation is available at https://github. com/cure-lab/LTSF-Linear. We report the performance of DLinear with lookback length L ∈ {96, 336} under five runs in Table 1 and 5 .RLinear [23] uses linear mapping to model periodic features in multivariate time series with robustness for diverse periods when increasing the input length. It applies RevIN (reversible normalization) and CI (Channel Independent) to improve overall forecasting performance by simplifying learning about periodicity. The official implementation is available at https://github.com/plumprc/ RTSF. The experimental results showed in Table 1 follow iTransformer [17] .Informer [14] enhances Transformer with KL-divergence based ProbSparse attention for O(L log L) complexity, efficiently encoding dependencies among variables and introducing a novel architecture with a DMS forecasting strategy. The official implementation is available at https://github.com/ zhouhaoyi/Informer2020 and the experimental results showed in Table 4 follow FEDformer [25] .Autoformer [15] employs a deep decomposition architecture with auto-correlation mechanism to extract seasonal and trend components from input series, embedding the series decomposition block as an inner operator. The official implementation is available at https://github.com/thuml/ Autoformer and the experimental results showed in Table 4 follow FEDformer [25] .Pyraformer [26] introduces pyramidal attention module (PAM) with an O(L) time and memory complexity where the inter-scale tree structure summarizes features at different resolutions and the intra-scale neighboring connections model the temporal dependencies of different ranges. The official implementation is available at https://github.com/ant-research/Pyraformer and the experimental results showed in Table 4 follow DLinear [12] .FEDformer [25] implements sparse attention with low-rank approximation in frequency domain, enjoying linear computational complexity and memory cost. And it proposes mixture of experts decomposition to control the distribution shifting. The official implementation is available at https: //github.com/MAZiqing/FEDformer and the experimental results showed in Table 1 follow iTransformer [17] .PatchTST [16] divides time series data into subseries-level patches to extract local semantic information and adopts channel-independence strategy where each channel shares the same embedding and Transformer weights across all the series. The official implementation is available at https://github.com/yuqinie98/PatchTST. The experimental results showed in Table 1 follow iTransformer [17] . And because iTransformer doesn't provide the prediction results with lookback length L = 336, we report the performance of PatchTST with lookback length L = 336 under five runs in Table 5 .iTransformer [17] inverts the structure of Transformer without modifying any existing modules by encoding individual series into variate tokens. These tokens are utilized by the attention mechanism to capture multivariate correlations and FFNs are adopted for each variate token to learn nonlinear representations. The official implementation is available at https://github.com/thuml/ iTransformer. The experimental results showed in Table 1 follow its original paper. And because iTransformer doesn't provide the prediction results with lookback length L = 336, we report the performance of iTransformer with lookback length L = 336 under five runs in Table 5 .The architecture of our FilterNet is very simple and has two main hyperparameters, i.e., the bandwidth of filters and the hidden size of FFN. As shown in Figure 9 , the bandwidth of the filters corresponds to the lookback window length, so we select the lookback window length as the bandwidth accordingly. For the hidden size of FFN, we carefully tune the size over {64, 128, 256, 512}. Following the previous methods [16, 32] , we use RevIN [19] as our instance normalization block. Besides, we carefully tune the hyperparameters including the batch size and learning rate on the validation set, and we choose the settings with the best performance. We tune the batch size over {4, 8, 16, 32} and tune the learning rate over {0.01, 0.05, 0.001, 0.005, 0.0001, 0.0005}.To evaluate the Transformer's modeling ability for different frequency spectrums, we generate a signal consisting of three different frequency components, i.e., low-frequency, middle-frequency, and high-frequency. We then apply both iTransformer and FilterNet to this signal, respectively, and compare the forecasting results with the ground truth. The results are presented in Figure 1 .Modeling capability of frequency filters We generate two signals: a trend signal with Gaussian noise and a multi-periodic signal with Gaussian noise. We then apply PaiFilter to these signals with a lookback window length of 96 and a prediction length of 96. The results are displayed in Figure 4 .Visualization of Frequency Filters Given a filter H ∈ R 1×L , where L is its bandwidth, we visualize the frequency response characteristic of the filter by plotting the values in R 1×L . First, we perform a Fourier transform on these values to obtain the spectrum, which includes the frequency and its corresponding amplitude. Finally, we visualize the spectrum, as shown in Figures 7, 8 , and 12.The bandwidth parameter (i.e., L in Equation ( 8) and D in Equation ( 9)) holds significant importance in the functionality of filters. In this part, we conduct experiments on the Weather dataset to delve into the impact of bandwidth on forecasting performance. We explore a range of bandwidth values within the set {96, 128, 192, 256, 320, 386, 448, 512} while keeping the lookback window length and prediction length constant. Specifically, we conduct experiments to evaluate the impact under three different combinations of lookback window length and prediction length, i.e., 96 -→ 96, 96 -→ 192, and 192 -→ 192, and the results are represented in Figure 9 . We observe clear trends in the relationship between bandwidth settings and lookback window length. Figure 9 (a) and Figure 9(b) show that increasing the bandwidth results in minimal changes in forecasting performance. Figure 9 (c) demonstrates that while forecasting performance fluctuates with increasing bandwidth, it is optimal when the bandwidth equals the lookback window length. These results indicate that using the lookback window length as the bandwidth is sufficient since the filters can effectively model the data at this setting, and it also results in lower complexity.To validate the rationale behind the architectural design of our FilterNet, we conduct ablation studies on the ETTm1, ETTh1, and Electricity datasets. We evaluate the impact on the model's performance From the figure, we can conclude that each block is indispensable, as the removal of any component results in a noticeable decrease in performance. This highlights the critical role each block plays in the overall architecture of FilterNet, contributing to its effectiveness in time series forecasting.Table 4 compares the performance of various methods with our FilterNet, demonstrating that our model consistently outperforms the others. To further assess the performance of FilterNet under different lookback window lengths, we conducted experiments on the ETTh1, ETTm1, Exchange, Weather, and Electricity datasets with the lookback window length of 336. The results, shown in Table 5 , indicate that our model achieves the best performance across these datasets. To further compare the channel-shared and channel-unique filters, we visualize the prediction values by the corresponding filters. The results are shown in Figure 11 . The figure demonstrates that the values predicted by channel-shared filters closely align with the ground truth compared to those predicted by channel-unique filters. This observation is consistent with the findings presented in Table 2 , indicating the superiority of channel-shared filters.We further conduct visualization experiments to explore the learnable filters under different lookback window lengths and prediction lengths. The experiments are performed on the Electricity dataset, and the results are illustrated in Figure 12 . These figures illustrate that FilterNet possesses full spectrum learning capability, as the learnable filters exhibit values across the entire spectrum. Besides, we observe that the frequency primarily concentrates in the low and middle ranges which explains that some works based on low-pass filters can also achieve good performance.To further offer an evident comparison of our model with the state-of-the-art models, we present supplementary prediction showcases on ETTm1 dataset, and the results are shown in 13. We choose the following representative models, including iTransformer [17] , PatchTST [16] , and DLinear [12] , as the baselines. Comparing with these different types of state-of-the-art models, FilterNet delivers the most accurate predictions of future series variations, demonstrating superior performance. Justification: We will include the source data and code.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.https://github.com/laiguokun/multivariate-time-series-datahttps://www.bgc-jena.mpg.de/wetter/https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014https://github.com/zhouhaoyi/ETDataset 


Decision: This paper proposes a novel architecture called FilterNet for time series forecasting, which aims to balance efficiency and effectiveness by introducing frequency filters.
The contributions were generally well-received, with many reviewers acknowledging its technical soundness. 
However, some reviewers also raised concerns that the technical contributions might be slightly below the standards expected at NeurIPS. 
The authors' rebuttal effectively addressed many of the reviewers' concerns. 
Nevertheless, when considering the technical standards and the relative evaluation of other submissions, the paper's priority is lower, which recommends the final decision to reject the submission.

Reviews: review: paper_summary: paper_summary: This paper investigates an interesting direction from a signal processing perspective, making a new attempt to apply frequency filters directly for time series forecasting. A simple yet effective architecture called FilterNet is proposed, utilizing two types of frequency filters to achieve forecasting. Comprehensive empirical experiments on eight benchmarks confirm the superiority of this paper's proposed method in terms of effectiveness and efficiency.
strength_weakness: strength_weakness: 1. The marginal improvement over existing approaches, as shown in Table 1, raises doubts about the compelling rationale for adopting the proposed method.
2. The discussion on related work appears lacking. It is common knowledge that frequency filters are widely used in traditional time series analysis methods and signal processing. Therefore, a detailed exploration of the connections and distinctions between them is warranted in the related work section.
3. The experiments have space to improve. Including more recent baselines for comparison, particularly deep learning methods that involve frequency processing, would add value. Additionally, a thorough investigation into the impact of hyperparameters is missing from the study.
questions: questions: Please address the questions in the weaknesses.
limitations: limitations: No
paper_summary: paper_summary: The paper proposes to use Fourier transforms with parametrized filters
as layers for time series forecasting models. Filters can either be
learnable parameters directly (called "plain shaping filter") or be
a function in the inputs, gates (called "contextual shaping filter").
In experiments the authors show that their method outperforms
many baselines from the literature.
strength_weakness: strength_weakness: w1. Empirical comparison with results from a closely related baseline, FiLM
  are not shown, and FiLM reports better results for all but one dataset.
- FiLM reports better results for ETTm2, Exchange, and Traffic. The proposed
  FilterNet wins for Weather. Why do you not report those results?
  If FiLM overall is the better performing model, why is FilterNet still
  interesting? How is it simpler than FiLM? Is it maybe faster to train?

w2. Conceptual differences to existing layers using Fourier transforms
  are not discussed in detail.
- So far, we just learn that other architectures also use "other network architectures,
  such as Transformers". But a more detailed delineation would be useful,
  esp. as FiLM is not using Transformers.

minor comments:
- reference [30], FiLM, is missing its venue, NeurIPS.
questions: questions: Q1. Why do you not report the better results of FiLM?
Q2. If FiLM overall is the better performing model, why is FilterNet still
  interesting? How is it simpler than FiLM?
Q3. Is it maybe faster to train?
limitations: limitations: yes
paper_summary: paper_summary: This paper explores a novel direction that leverages frequency filters directly for time series forecasting, and proposes a simple yet effective framework, namely FilterNet, from a signal processing perspective. It designs two types of learnable frequency filters corresponding to the linear and attention mapping widely used in time series modeling, i.e., PaiFilter and TexFilter. Abundant analysis experiments verify the competitive modeling capability of frequency filters, and extensive experiments on eight real-world datasets demonstrate FilterNet achieves good performances in both accuracy and efficiency over state-of-the-art methods.
strength_weakness: strength_weakness: 1.	The visualization setting for Figure 1 appears to be missing in Appendix C.4, and I recommend adding some references about the spectral bias of neural networks, such as [1].
2.	Some typo errors. In line 282, the complexity of FilterNet should be O(LogL), not O(L LogL).
questions: questions: 1.	In ICML 2024, some works also adopt simple architectures for time series forecasting, such as SparseTSF [2]. Could you further compare the performance of FilterNet with SparseTSF?
2.	The recent literature has seen papers based on foundation models for time series forecasting. Conversely, some works are designed with simple architectures, yet achieve better performance than foundation models. Why can simple models (e.g., FilterNet) also achieve competitive performance?
[1] On the Spectral Bias of Neural Networks, in ICML 2019
[2] SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters, in ICML 2024
limitations: limitations: N/A
paper_summary: paper_summary: This paper introduces FilterNet, a novel neural network architecture for time series forecasting that leverages learnable frequency filters inspired by signal processing techniques. This paper proposes two types of filters within FilterNet: a plain shaping filter that uses a universal frequency kernel for signal filtering and temporal modeling, and a contextual shaping filter that examines filtered frequencies for compatibility with input signals to learn dependencies. This paper demonstrates that FilterNet can effectively handle high-frequency noise and utilize the full frequency spectrum, addressing key limitations of existing Transformer-based forecasters. This paper shows through extensive experiments on eight benchmarks that FilterNet achieves superior performance in both effectiveness and efficiency compared to state-of-the-art methods in time series forecasting.
strength_weakness: strength_weakness: 1. The paper mainly introduces two filters. The connection and difference between the two filters is not highlighted, which makes me curious about which one is better.
2. As far as I know, there exist many frequency filters in signal processing. The motivation for these two filters is not very clear.
3. In experiment part, the results of TexFilter on ETTm1, ETTm2, ETTh1 and Exchange are neither the best nor the second best. Does it show the filter cannot win in many cases of forecasting? Are there any analysis or explanation about it?
questions: questions: 1. Can the authors elaborate on the connection and difference of the two filters?
2. Can the authors give more details about proposed filters?
3. Can the authors explain the lower performance of TexFilter in some cases?
limitations: limitations: See weakness.

Comments: title: Thank the authors for your rebuttal
Reviewer response
Reviewer Response
comment: Dear Reviewer UnVh, 

we greatly appreciate your continued valuable feedback. Thanks again for  your  constructive suggestions.

Authors
Dear Reviewer 2waa,

With the discussion period nearing its end and less than 18 hours remaining, 
we kindly urge you to respond to our rebuttal.
Thanks for your feedback. We provide a response to your main concerns below.

1. We will consider including the standard deviations in the final version, although none of our baselines—such as iTransformer, PatchTST, FEDformer, TimesNet, DLinear, RLinear, FITS, FreTS, MICN, Pyraformer, Autoformer, and Informer—report standard deviations in their papers.
   
2. In the literature, it is quite common to design different model variants to adapt to various datasets. 
   For instance, FEDformer introduced two model variants, FEDformer-f and FEDformer-w;
   PatchTST designed two model variants, PatchTST/64 and PatchTST/42;
   DLinear included three model variants, DLinear, NLinear, and Linear.
  Switching between these variants typically involves adjusting a configuration parameter, 
   such as the "version" parameter for FEDformer, "patch_len" for PatchTST, or "model" for DLinear.
Typically, a model name is used generically to refer to all variants, with the specific variant being distinguished by its corresponding configuration parameters.

   For our proposed FilterNet, as mentioned in the previous rebuttal, TexFilter has demonstrated competitive performance on larger datasets (e.g., Traffic and Electricity) due to the need for more contextual structures, while PaiFilter performs well on smaller datasets (e.g., ETTh1). 

    In practice, especially for the industry, the balance between efficiency and effectiveness is what matters most. And which model is more suitable requires actual comparison and testing.

3. As highlighted in our previous rebuttal, the primary contribution of this paper lies in offering a novel viewpoint from signal processing and devising a simple yet effective architecture centered around a Frequency Filter Block. 
  FilterNet shows remarkable efficiency while simultaneously achieving state-of-the-art performance. Furthermore, we extensively investigate the efficacy and insights of frequency filters via introducing two distinct types of frequency filters, each exhibiting diversified performances across datasets of different scales. This exploration reveal the impact of frequency filters on time series forecasting and indicates that designing tailored architectures adaptive to data scale may enlighten a new path for future research endeavors.
Thank the authors for your rebuttal. I have reviewed all comments and the authors’ responses, which effectively addressed my concerns through clear explanations and additional experiments. I recommend incorporating these details into the manuscript or supplementary materials. I have also read the comments from other reviewers but I don't think their reviews can be too convincing to me to re-evaluate this paper. So I keep my original score and overall support the acceptance of this paper.
Thank you for the rebuttal. Here is my feedback:

1. Sure, the answer is not generated by GPT. The reviewer has worked in the field of time series analysis for many years since statistical methods. From a statistical standpoint, it is crucial or even compulsory to conduct rigorous statistical tests to evaluate improvements, especially when the improvements are insignificant. The reason you mentioned (other papers not reported) is not convincing and the evaluation goes against scientific rigor.

2. For instance, DLinear offers three variants, so why did the researchers only compare one? In future comparisons with your model, which variant should be selected? Which variant do you recommend? This is a very important problem in practice, especially for the industry.

3. The reason for not including TimeXer and Timer is reasonable.

4. My evaluation is centered on the technical novelty/contributions and practical results achieved by the proposed method. While the reviewer isn't particularly impressed by the technical novelty/contributions, the reviewer prioritizes methods that demonstrate strong performance in real-world applications. 

I have adjusted my ratings and would like to leave the discussion to ACs and other reviewers.
Thanks for your feedback. We provide a point-by-point response to your new concerns below.

**1. ....however, the improvement in predictive performance is neither significant nor consistent across all datasets, particularly on the Traffic dataset.**

A1. We think your comment is not professional about improvements in the domain of time series forecasting. We don't understand what level of improvement is considered sufficient in your opinion. 
As you mentioned, TimeXer, what you refer to as a more recent and powerful method, 
its performance is worse than iTransformer on the ECL, Traffic, and Weather datasets, and worse than RLinear on ETTh2 dataset (please see its Table 8). Similarly, iTransformer shows weaker performance on the ETTh and ETTm datasets. Are you also questioning their performance improvements, too? It is **ridiculous** the failure to achieve optimal performance across all datasets could also be a reason of rejection.

**2. How many experimental runs were conducted for the reported results? From my understanding, the results of deep learning algorithms can vary across different runs. Can you guarantee that your improvements are statistically significant across all datasets? Additionally, what are the standard deviations across all runs?**

A2. Your question seems to be not related to our submission. 
We are wondering if you generate this review comments by Generative AI (e.g., ChatGPT). 
If you have run the state-of-the-art baselines in the domain of time series forecasting (e.g., iTransformer, PatchTST), 
you would know these forecasting methods are usually stable thus they did not report std. 
We conducted the experiments three times with three seeds, and reported the average performance with the very stable results and less std. 
Thus we followed previous settings to report the performance.
The average improvement of FilterNet over all baseline models is statistically significant at the confidence of 95% (**please see in line 218 of our paper**).

**3. The performance of TexFilter and PaiFilter varies significantly across different datasets. What is the underlying reason for this fluctuation? It’s difficult to determine when to use TexFilter versus PaiFilter. A more rigorous justification or empirical study is needed. Otherwise, I would prefer a model that demonstrates consistent performance across all scenarios, such as PatchTST and [1, 2].**

A3. In the literature, compared with other model architectures, MLP-based and Transformer-based methods have achieved competitive performance. Inspired by the two paradigms, we design two corresponding types of filters. i.e., PaiFilter and TexFilter. PaiFilter offer stability and efficiency, making them suitable for tasks with static relationships. In contrast, TexFilter provides the flexibility to capture dynamic dependencies, excelling in tasks that require context-sensitive analysis. As shown in Table 1, PaiFilter performs well on small datasets (e.g., ETTh1), while TexFilter excels on large datasets (e.g., Traffic) due to the ability to model the more complex and contextual correlations present in larger datasets. This dual approach allows FilterNet to effectively handle a wide range of data types and forecasting scenarios, combining the best aspects of both paradigms to achieve superior performance.

**We have analyzed the two filters in Appendix B and have also given explanations about the results of Table 1 in lines 219-224**.

You again mentioned a **ridiculous** point. We don't know why the proposed method should be one model. Like DLinear, it also has studied three different variants, i.e., NLinear, Linear, and DLinear. Can you say you do not want to see the Linear family models in one paper?

**4. ... comparing FITS and iTransformer, more recent and powerful baselines, such as TimeXer [1] and Timer [2], should also be considered.**

A4. 
We have also compared FilterNet with SparseTSF (ICML, 2024) and Koopa (NeurIPS, 2023), and we are very confused that why we should compare with TimeXer which as only one preprint paper, and Timer as a generative pre-trained large model.
**What is even more crazy** is that you recommend we include Timer in the efficiency comparison. Timer is a pre-trained model, which has a 28B pre-trained parameters (see its Table 9). It is brutal.

**5. Why didn't the paper compare the efficiency of frequency-based methods (e.g., FITS, FilM) with the proposed method? Compared to other types of models, it is more necessary to compare it with them.Moreover, it would be beneficial to include TimeXer and Timer in the efficiency comparison as well.**

A5. We have compared the efficiency with FiLM, please refer to A6 in Response to Reviewer 2waa.
For FITS, it is a lightweight method and has 10k parameters. Obviously, FITS is more efficient. However, our model is not designed for a lightweight model, but a model that balances efficiency and effectiveness.
As for the efficiency comparison with TimeXer and Timer, please refer A4.
Thank you for addressing my questions. I still have the following concerns:
1. The authors have restated the technical contributions and efficiency achieved by the proposed method; however, the improvement in predictive performance is **neither significant nor consistent across all datasets**, particularly on the Traffic dataset. How many experimental runs were conducted for the reported results? From my understanding, the results of deep learning algorithms can vary across different runs. Can you guarantee that your improvements are statistically significant across all datasets? Additionally, what are the standard deviations across all runs?
2. The performance of TexFilter and PaiFilter varies significantly across different datasets. What is the underlying reason for this fluctuation? It’s difficult to determine when to use TexFilter versus PaiFilter. A more rigorous justification or empirical study is needed. Otherwise, I would prefer a model that demonstrates consistent performance across all scenarios, such as PatchTST and [1, 2].
3. Although the authors provide several results comparing FITS and iTransformer, more recent and powerful baselines, such as TimeXer [1] and Timer [2], should also be considered.
4. Why didn't the paper compare the efficiency of frequency-based methods (e.g., FITS, FilM) with the proposed method? Compared to other types of models, it is more necessary to compare it with them. Moreover, it would be beneficial to include TimeXer and Timer in the efficiency comparison as well.

Given the above concerns, I don't believe this paper is ready to publish on a top-tier venue like NeurIPS.

Reference:

[1] TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables. arXiv 2024. 

[2] Timer: Generative Pre-trained Transformers Are Large Time Series Models. ICML 2024.
Dear Reviewer 2waa,

We appreciate your time and effort in reviewing our paper. As the discussion period is drawing to a close, we wanted to kindly  inquire if there are any further concerns. 
Your feedbacks are really important to us, and we are also looking forward to further discussions with you before the discussion phase ends.
We value your input and would appreciate the opportunity to respond.

Thank you again for your valuable time and effort.

Best,

Authors
Dear Reviewer ufqs,

Thank you very much again for your time and efforts in reviewing our paper. We kindly remind that our discussion period is closing soon. We just wonder whether there is any further concern and hope to have a chance to respond before the discussion phase ends.

Many thanks, 

Authors