FilterNet: Harnessing Frequency Filters for Time Series Forecasting
Authors: Kun Yi
Jingru Fei
Qi Zhang
Hui He
Shufeng Hao
Defu Lian
Wei Fan
Venue: NeurIPS 2024 poster (2024)
Keywords: time series forecasting
learning in the frequency domain
Primary area: deep_learning_architectures

Abstract: Given the ubiquitous presence of time series data across various domains, precise forecasting of time series holds significant importance and finds widespread real-world applications such as energy, weather, healthcare, etc. While numerous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-art performance in time series forecasting. However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series with thousands of points. In this paper, we explore a novel perspective of enlightening signal processing for deep time series forecasting. Inspired by the filtering process, we introduce one simple yet effective network, namely FilterNet, built upon our proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals. Concretely, we propose two kinds of learnable filters in the FilterNet: (i) Plain shaping filter, that adopts a universal frequency kernel for signal filtering and temporal modeling; (ii) Contextual shaping filter, that utilizes filtered frequencies examined in terms of its compatibility with input signals for
dependency learning. Equipped with the two filters, FilterNet can approximately surrogate the linear and attention mappings widely adopted in time series literature, while enjoying superb abilities in handling high-frequency noises and utilizing the whole frequency spectrum that is beneficial for forecasting. Finally, we conduct extensive experiments on eight time series forecasting benchmarks, and experimental results have demonstrated our superior performance in terms of both effectiveness and efficiency compared with state-of-the-art methods. Our code is available at$^1$.

FilterNet: Harnessing Frequency Filters for Time Series Forecasting
Kun Yi!.2, Jingru Fei Qi Zhang", Hui He Shufeng Hao', Defu Lian", Wei Fan North China Institute of Computing Technology, State Information Center of China 3 BBeijing Institute of Technology. *Tongji University, Taiyuan University of Technology University of Science and Technology of China, 'University Oxford kunyi.cn @ gmail.com, [jingrufei, hehui617) @ @bit.edu.en. zhangqi cs tongji.edu.cc haoshufeng @ tyut.edu.cn, liandefu @ ustc. edu.cn, weifan oxford @ gmail.com
Abstract
Given the ubiquitous presence of time series data across various domains, pre- cise forecasting of time series holds significant importance and finds widespread real-world applications such as energy, weather, healthcare, etc. While numer- ous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-ar performance in time series fore- casting. However, forecasters based on Transformers are still suffering from vulnerability high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series with thousands of points. In this paper, we explore a novel perspective of enlightening signal processing for deep time series forecasting. Inspired by the filtering process, we introduce one simple yet effective network, namely FilterNet, built upon our proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain compo- nents of time series signals. Concretely,
propose two kinds of learnable filters in the FilterNet: (i) Plain shaping filter, that adopts universal frequency kernel for signal filtering and temporal modeling: (ii) Contextual shaping filter, that utilizes filtered frequencies examined in terms its compatibility with input signals for dependency learning. Equipped with the two filters, FilterNet can approximately surrogate the linear and attention mappings widely adopted in time series literature, while enjoying superb abilities in handling high-frequency noises and utilizing the whole frequency spectrum that is beneficial for forecasting. Finally, we conduct ex- tensive experiments on eight time series forecasting benchmarks, and experimental results have demonstrated our superior performance in terms of both effectiveness and efficiency compared with state-of-the-ar methods. Our code is available at
Introduction
Time series forecasting has been playing pivotal role across multitude of contemporary applications, spanning diverse domains such as climate analysis [1, energy production (2], traffic flow patterns (3), financial markets [4) and various industrial systems 5 The ubiquity and profound significance of
time series data has recently garnered substantial research efforts, culminating in a plethora of deep
learning forecasting models (6) that have significantly enhanced the domain of time series forecasting.
Previously, leveraging different kinds of deep neural networks derives series of time series forecast- ing methods, such as Recurrent Neural Network-bass methods including DeepAR [7]. LSTNet (8]
"Corresponding author
https://github. http:///ihhuuoooommaauuugu///e/e
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

44-
Levirppent
Willh Inpoer
125
105
175
26
(a) The spectrum of input signal(b) iTransformer with MSE=1.1c-01 (c) FilterNet with 25-=2.7c-05
Figure I: Performance of Mean Squared Error (MSE) on simple synthetic multi-frequeng signal.
More details about the experimental settings can be found in Appendix C.4
Convolution Neural Network- -bsed methods including TCN [91 SCINet [10], etc. Recently, however, with the continuing advancement of deep learning, two branches of methods that received particularly more attention have been steering the development of time series forecasting, i.c., Multilayer Percep-
tron (MLP)-based methods, such as N-BEATS [I], DLinear [1), and FreTS [3), and Transformer- based methods, such as Informer [1], Autoformer [1], PatchTST [16], and i'Transformer [17]. While MLP-bbaed models are capable of providing accurate forecasts, Transformer-base models continue to achieve state-of satte-ofthe--eee -t-thhe-rt art time series forecasting performance.
However, forecasters based on Transformers signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the comerssonns for accurately predicting time series composed of thousands of timesteps. In designing a very simple simulation experiment on the synthetic data only composed of a low-, middle- and high-frequens signal respectively (see Figure [[a). we find the offte-of-the-arr iTransformer [I7] model performs much worse in forecasting (Figure (b) and Figure [(c).. This observation shows that of-te-of- Transformer-bast -based model cannot utilize the full spectrum information, even for naive signal of three different frequency components. In contrast, in the field of signal processing, frequency filter enjoys many good properties such as frequency selectivity, signal conditioning, and multi-rate processing. These could have great potential in advancing the model's ability to extract key informative frequency patterns time series forecasting.
still suffering from vulnerability to high-frequency
Thus, inspired by the filtering process [8] in signal processing, in this paper, we introduce one simple yet effective framework, namely FilterNet, for effective time series forecasting. Specifically, we start by proposing two kinds of learnable filters as the key units in our framework: (i) Plain shaping filter, which makes the naive but universal frequency kernel learnable for signal filtering and temporal modeling, and (ii) Contextual shaping filter, which utilizes filtered frequencies examined in terms of its compatibility with input signals for dependency learning. The plain shaping filter is more likely to be adopted in predefined conditions and efficient in handling simple time series structures, while the contextual filter can adaptively weight the filtering process based on the changing conditions of input and thus have more flexibility in facing more complex situations. Besides, these two filters as the built-in functions of the FilterNet can also approximately surrogate the widely adopted linear mappings and attention mappings in time series literature [12] 14 [7]. This also illustrates the effectiveness of our FilterNet in forecasting by selectively passing or attenuating certain signal components while capturing the core time series structure with adequate learning expressiveness. Moreover, since filters are better fit in the stationary frequency domain, we let filters wrapped by two reversible transformations, i.e., instance normalization [19] and fast Fourier transform [20] to reduce the influence of non-stations -tattiiiiona and accomplish the domain transformation of time series respectively. In summary, contributions can be listed follows:
In studying offtc-of-theaare deep Transformer- based time series forecasting models, interesting observation from a simple simulation experiment motivates us to explore novel perspective of enlightening signal processing techniques for deep time series forecasting.
Inspired by the filtering process in signal processing, we introduce a simple yet effective network, FilterNet, built upon our proposed two learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals, thereby enhancing the forecasting performance.
We conduct extensive experiments on eight time series forecasting benchmarks, and the results have demonstrated that our model achieves superior performance compared with oate-of-the-ant forecasting algorithms in terms effectiveness and efficiency.

---

2 Related Work
Deep Learning-based Time Series Forecasting In recent years, deep learning- baaning-bassed methods have gained prominence in time series forecasting due to their ability to capture nonlinear and complex correlations [21]. These methods have employed various network architectures to learn temporal dependencies, such as RNN 88 77. TCN p [0], etc. Notably, MLP- and Transformer- based methods have achieved competitive performance, emerging as state-of-the-a approaches. N-HiTS [22] integrates multi-rate input sampling and hierarchical interpolation with MLPs to enhance univariate forecasting. DLinear [12] introduces a simple approach using a single-layer linear model to capture temporal relationships between input and output time series data. RLinear [3] utilizes linear mapping to model periodic features, demonstrating robustness across diverse periods with increasing input length. In contrast to the simple structure of MLPs, Transformer's advanced attention mechanisms [24] empower the models [14] 15 25 26 to capture intricate dependencies and long- range interactions. PatchTST (16) segments time series into patches as input tokens the Transformer and maintaining channel independence. iTransformer [17] inverts the Transformer's structure by treating independent series as variate tokens to capture multivariate correlations through attention.
Time Series Modeling with Frequency Learning In recent developments, frequency technology has been increasingly integrated into deep learning models, significantly improving state-of-the art accuracy and efficiency in time series analysis [27]. These models leverage the benefits of
frequency technology, such as high efficiency [28, 29 and energy compaction [3, to enhance forecasting capabilities. Concretely, Autoformer [15] introduces the auto-correlation mechanism, improving self-attention implemented with Fast Fourier Transforms (FFT). FEDformer [25] enhances attention with a FFT-based frequency approach, determining attentive weights from query and key spectrums and conducting weighted summation in the frequency domain. DEPIS [30] utilizes Discrete Fourier Transform (DFT) to extract periodic patterns and contribute them to forecasting. FiLM 31 employs Fourier analysis to retain historical information while filtering out noisy signals. FreTS K introduces a frequency-domain Multi-Layer Perceptrons (MLPs) to learn channel and temporal dependencies. FourierGNN (29) transfers the operations of graph neural networks (GNNs) from the time domain the frequency domain. FITS [32] applies a low pass filter to the input data followed by complex-valued linear mapping in the frequency domain.
Unlike these methods, in this paper we propose simple yet effective model FilterNet developed from a signal processing perspective, and apply all-pass frequency filters to design the network directly, rather than incorporating them into other network architectures, such as Transformers, MLPs, or GNNs, or utilizing them as low-pass filters, as done in FITS [32] and FiLM [3]]
3 Preliminaries
Frequency Filters Frequency filters [33] are mathematical operators designed to modify the spectral content of signals. Specifically, given an input time series signal z[r]] with its corresponding Fourier transform x(k)), a frequency filter H[k] is applied to the signal to produce an output signal
y/n] with its corresponding Fourier transform )'[A:] [[]]][()) The frequency filter H[k] alters the
amplitude and phase of specific frequency components in the input time series signal x [r]] according to its frequency response characteristics, thereby shaping the spectral content of the output signal.
According to the Convolution Theorem the point-wise multiplication in the frequency domain corresponds to the circular convolution operation between two corresponding signals in the time domain. Consequently, the frequency filter process can be expressed in the time domain as:
y[k] = [[]))*(M) + y/m] h [m] @ r/n],
(1)
where h[n]] is the inverse Fourier transform of H[k] and @ denotes the circular convolution operation. This formulation underscores the intrinsic connections between the frequency filter process and the circular convolution in the time domain, and indicates that the frequency filter process can efficiently perform circular convolution operations. In time series forecasting, Transformer based methods have achieved state-of-the-art performance, largely due to the self-attention mechanism [14 15 25 17),
which can be interpreted as a form of global circular convolution [35] This perspective opens up the possibility of integrating frequency filter technologies, which are well-known for their ability to isolate and enhance specific signal components, to further improve time series forecasting models.
3

---

Output
1 Platn Shaping Filter
Inverse Instance
Platn
Normalization
Shaping
Z
F (2)
FFN
(2)
5
Frequency Filter Block
F (2)
Contectual
Shaping Filler
2C,CF (0)
Instance
+())
Normalization
(()))((()))
Input
Contextual Shaping Filler
Figure 2: The overall architecture of FilterNet. (i) Instance normalization is employed to address the non-stationarity among time series data; (ii) The frequency filter block applied to capture the temporal patterns, which has two different implementations, i.e., plain shaping filter and contextual shaping filter; (iii) Feed-forward network is adopted project the temporal patterns extracted by frequency filter block back onto the time series data and make predictions.
Methodology
As aforementioned, frequency filters enjoy numerous advantageous properties for time series fore- casting, functioning equivalently cireular convolution operations in the time domain. Therefore, we design the time series forecaster from the perspective of frequency filters. In this regard, we propose FilterNet, forecasting architecture grounded frequency filters. First, introduce the overall architecture of FilterNet in Section 4.1 which primarily comprises the basic blocks and the frequency filter block. Second, we delve into the details of two types frequency filter blocks: the plain shaping filter presented in Section 4.2 and the contextual shaping filter discussed in Section 4.3
4.1
Overview
The overall architecture of FilterNet is depicted in Figure 2 which mainly consists of the instance normalization part, the frequency filter block, and the feed-forward network. Specifically, for a given time series input X [X]=*. X1-1 .... X1 E NxL with the number of variables N and the lookback window length L. where X R denotes the N-th variable, employ FilterNet to predict the future time steps Y [X(+1LL+,, X X ,..., x e NxT, We provide further analysis about the architecture design FilterNet in Appendix
Instance Normalization Non-stationarity is widely existing in time series data and poses a crucial challenge for accurate forecasting [19 36. Considering that time series data are typically collected over a long duration, these non-stationary sequences inevitably expose forecasting models to dis- tribution shifts over time. Such shifts can result in performance degradation during testing due to the covariate shift or the conditional shift (37]. To address this problem, we utilize an instance normalization method, denoted as Norm, on the time series input X, which can be formulated as:
Norm(X) XIL Meang (X!:L) N R=
(td
where Meany denotes the operation that calculates the mean value along the time dimension, and Stdy represents the operation that calculates the standard deviation along the time dimension.
Correspondingly, the inverse instance normalization, denoted as InverseNorm, is formulated as:
InverseNorm(2 (P)
[p.+1:L Std, (X!: Mean (X.
(3)
where P = p + pl. p 1++1: ..... ph II++i g. are the predictive values.

---

Milter
Z
Hilter
(a) Plain Shaping Filter
(b) Contextual Shaping Filter
Figure 3: The structure of frequency filters. (a) Plain shaping filter: the plain shaping filter is initialized randomly with channel-shared (left) or channel-unique (right) parameters, and then performs circular convolution (i.e., the symbol @ with the input time series; (b) Contextual shaping filter: the contextual shaping filter firstly learns a data- deta-dependent filter and then conducts multiplication (i.e., the symbol @) with the frequency representation of the input time series.
Previous representative works primarily leverage MLP architectures (e.g.. DLinear [22 RLinear (23 or Transformer architectures (e.g.. PatchTST [[]] iTransformer (17]) to model the temporal dependencies among time series data. As mentioned earlier, time series forecasters can be implemented through performing a frequency filter process the frequency domain, and thus we propose to directly apply the frequency filter in the frequency domain, denoted as FilterBlock, to replace the aforementioned methods for modeling corresponding temporal dependencies, such as:
Frequency Filter Block
filter
FilterBlock(2) (Z) = =F (F falter);
where F is Fourier transform, F-1 is inverse Fourier transform and fatte is the frequency filter.
Inspired by MLP that randomly initializes learnable weight parameters and Transformer that learns the deteddee attention scores from data (further explanations are provided in Appendix [B), we introduce two types of frequency filters, i..., plain shaping filter (PaiFilter) and contextual shaping filter (TexFilter). PaiFilter applies random initialized learnable weight Hy to instantiate the frequency filter Hitttrr, and then the frequency filter process is reformulated as:
FilterBlookk(Z) J ( (F(Z)H)).
TexFilter learns a data-dependent frequency filter Hyp (FZ()) from the input data by using a neural network Hep (), and then the corresponding frequency filter process is reformulated as:
FilterBlook(Z)) F (F(ZH,
(6)
Feed-forward Network The frequency filter block has captured temporal dependencies among time series data, and then employ ferd-forward network (FFN) to project them back onto the time series data and make predictions for the future time steps. As the output P of FFN are instance-normalize values, conduct an inverse instance normalization operation (InverseNorm) on them and obtain the final predictions Y. The entire process can be formulated as follows:
= FFN(SS)
Y = InverseNorm(P).
4.2 Plain Shaping Filter
PaiFilter instantiates the frequency filter by randomly initializing learnable parameters and then performing multiplication with the imput time series. In general, for multivariate time series data, the frannel-independdnnc strategy in channel modeling has proven to be more effective compared to the channel-mixing strategy [1]
16). Following this principle, we also adopt the channel-independence strategy for designing the frequency filter. Specifically, we propose two types of plain shaping filters: the universal type, where parameters are shared across different channels, and the individual type, where parameters are unique each channel, as illustrated in Figure 3(a)
Given the time series input Z E N and the plain shaping filter Hgs we apply PaiFilter by:
Z F(Z).
S Øい Ho,
Ho E (Un)) H(/md)
(8)
$ F (S),
5

---

where Fis Fourier transform. F is inverse Fourier transform, Or denotes the element-wise product along L dimension, ( Clx is the universal plain shaping filter, H C is the individual plain shaping filter, and S E NRN is the output of PaiFilter. We further compare and analyze the two types of PaiFilter in Section 5.3
4.3 Contextual Shaping Filter
In contrast to PaiFilter, which randomly initializes the parameters of frequency filters and fixes them after training, TexFilter learns the parameters generated from the input data, allowing for better adaptation to the data. Consequently, we devise a neural network Hyp that flcxibly adjusts the frequency filter in response to the input data, as depicted in Figure 3(b)
Given the time series input E RN and its corresponding Fourier transform denoted as F(Z) E CN the network Hip is utilized to derive the contextual shaping filter, expressed as Hhg CN + CN First, it embeds the raw data by a linear dense operation & C + C to improve the capability of modeling complex data. Then, it applies a series of complex-value multiplication with K learnable parameters W E ClxD yielding a (x(2) 2) @ Wi.)) where is the activation function, and finally outputs (2). Then we apply TexFilter by:
Z =(Z),
E k(2),
K
(z) o8E on Wix),, WI.K II w
(9)
8 & oD Hz,
$ F=
where Op denotes the element-wise product along D dimension and S E R Nx is the output. The contextual shaping filter can adaptively weight the filtering process based on the changing conditions of input and thus have more flcxibility in facing more complex situations.
5 Experiments
In this section, we extensively experiment with eight real-world time series benchmarks to assess the performance of our proposed FilterNet. Furthermore, we conduct thorough analytical experiments concerning the frequency filters to validate the effectiveness of our proposed framework.
5.1 Experimental Setup
Datasets We conduct empirical analyses on diverse datasets spanning multiple domains, including traffic, energy, and weather, among others. Specifically, we utilize datasets such as ETT datasets [14]. Exchange [38) Traffic [15]. Electricity [15], and Weather [15], consistent with prior studies on long time series forecasting [6 25). We preprocess all datasets according the methods outlined in [16] 17), and normalize them with the standard normalization method. We split the datasets into training, validation, and test sets in 7:2:1 ratio. More dataset details are in Appendix C. C.
Baselines We compare our proposed FilterNet with the representative and state-of siiie--fftheeeee art models to evaluate their effectiveness for time series forecasting. We choose the baseline methods from four categories: (1) Frequency-basee models, including FreTS [3] and FITS [32]; (2) TCN-based
models, such as MICN [9 and TimesNet 44): (3) Ml.P-based models, namely DLinnarr [2 and
RLinear [23]: and (4) Transformer-hasee models, which include Informer [14] Autoformer [15].
Pyraformer (26). FEDformer 255, PatchT'T [16), and the more recent i'Transformer
17 for compar-
ison. Further details about the baselines can be found in Appendix C.2
Implementation Details All experiments are implemented using Pytorch 1.8 41 and conducted on single NVIDIA RTX 3080 10GB GPU. We employ MSE (Mean Squared Error) as the loss function and present MAE (Mean Absolute Errors) and MSE (Mean Squared Errors) results as the evaluation metries. For further implementation details, please refer Appendix C.3

---

Table 1: Forecasting results for prediction lengths {96, 192, 336, 720} with lookback window length L = 96. The best results are in red and the second best are blue. Due to space limit, additional results with other baselines and under different lookback length are provided in Tables 4 and S
Midels
TexFitter
PuiFitter
iTrnnformer
PaichTST
FEDfommrr
TimesNe:
DLinear
RLineer
UTT
Metrics
MSC
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAG
MSE
MAE
MSE
MAA
MSE
MAE
0.321
0.361
0.318
0.35%
0.534
0:530
0.370
0,40
0.338
0.375
0.344
0.520
0.355
0.576
0.355
0.375
0.367
0.357
0.364
0.377
0.367
0:426
0.374
0.387
0.379
0.393
0.387
0.392
0.393
0.401
0.409
0.346
0.401
0.426
0.420
0.399
0.410
0.455
0.459
0410
0.41
0,10
0.411
0.424
0.415
0.424
0.414
720
0.477
0.448
0.456
0.464
0.491
0.459
0.454
0.439
0.543
0.490
0.478
0.450
0.473
0.450
0.487
0.450
0.487
0.449
ETTa2
192
56
0.175
0.240
0.301
0..25
0.174
0.240
0.257
0.300
0.180
0.250
0.264
0.309
0.175
0.241
0.259
0.302
0.209
0.203
0.528
0.287
0.187
0.269
0.267
0.309
0.187
0.222
0.281
0.349
0.182
0.246
0.265
0.304
0.183
0.247
0.256
0.305
336
0.311
0.347
0.297
0.339
0.311
0. 348
0.305
0.549
0.325
0. 0.366
0.321
0.351
0.316
0.372
0.307
0.342
0.307
0.342
720
0.414
0,405
0.392
0.393
0.412
0.407
0,402
0.400
0.421
0.415
0.408
0.403
0.452
0.457
0.407
0
0.407
0.399
96
00,382
0,402
0.375
0.394
0.386
0.405
0.144
0.419
0.376
0.430
0.384
0.402
0.363
0.396
0.386
0.395
0.386
0.396
ET112 ETTM
192
0.450
0.429
0.436
0.422
0.441
0.436
0.400
0.445
0.430
0.448
0.436
0.429
0.435
0.426
01437
0.424
0.456
0.423
336
0.472
0,451
0,476
0.443
0.487
0.458
0.501
0.466
0.459
0:465
0.491
0:489
0.479
0.457
0.479
0.446
0.478
0.444
0,473
0.474
0.503
0.500
0,506
0.521
0.500
0.517
0.495
0.293
0.797
0.500
0.358
0.340
0,318
0.363
0.350
192
0.374
0.396
0.369
0,395
0.380
0,400
0.388
0,000
0:239
0.439
0,402
0.414
0,449
0.454
0,401
0.412
0,381
0.3%
0.417
0.430
0.429
0,432
0.128
0:126
0:596
0,452
0,467
0.469
0,426
0/442
0,426
0.438
0.449
0.400
0.430
0.446
0.427
0,445
1
1,446
0,263
0474
0.462
0.468
0.656
0.571
0,442
00454
0.451
0.446
0.147
0.245
0.176
0,254
0.108
02:0
18180
0.20
0.193
0.505
0.168
0,272
0.195
0.277
0.201
0.281
0,200
0.273
ECL
192
0.160
0.250
0.185
0220
0.102
0253
12228
D:201
0.3115
0.184
0.289
0.19
0.280
1100
0:223
0200
0.250
336
0.173
0.267
0.201
0,286
8.1788
0269
0.204
0.203
0:214
0. 6.320
0.195
0.300
0207
0.706
0210
0.208
0214
0.295
730
0.210
0.309
0.242
03119
0.225
0.177
0.246
0.304
0.246
0. 0.355
0.220
0.
0.320
0224
0.320
0.257
0.331
0.255
0.327
Trallee Exchange
96
0,091
0.211
0,083
0.202
0.086
0.006
0:088
0,305
0.148
0.27%
0.107
0.254
0.085
0.210
0.093
0.217
0.084
0.203
192
0.186
0.505
0.174
0.296
0177
0.000
0.176
0.509
0.271
0.315
0.226
0.344
0.178
0.209
0.184
0.307
0.177
0.298
306
0.380
0.649
0.326
0.413
0.331
0.117
0.301
0.397
0.080
0:427
0.367
0.148
0.296
0.409
0.351
0.432
0.321
0.410
720
0.895
0,712
0,840
0.670
0.847
0.681
0.901
0.714
1.195
0.695
0.964
0,746
0.861
0.671
0,856
0.714
0.828
0.685
0.294
0.506
0.395
0.422
0.587
0.595
0.650
0.589
0.651
0.391
0.451
0.307
0.508
0.417
0.466
0.604
0.617
0.600
0.601
0.366
0.363
0.479
0.316
0.518
0,335
0.433
0.482
0.621
0.629
0.336
0.686
0.374
0.809
0.369
0.609
0.366
0.495
0.323
0.553
0.354
0.467
0.302
0.514
0.432
0.636
0.582
0.640
0.350
0.646
0.395
0.647
0.387
0.647
0.385
Woather
96
0.162
0.207
0.164
0.210
0.174
0.214
0.177
0.218
0.217
0.296
0.172
0.220
0.194
0.248
0.192
0.222
0.166
0,213
192
0.210
0.250
0.214
0.253
0.221
0.258
0.225
0.299
0.216
0.336
0.219
0.26]
0.234
0.290
0.240
0.271
0.213
0.254
336
0.265
0.290
0.268
0.293
0.27%
0.286
0:278
10.99
0,339
0.300
0.280
0.306
0.285
0.335
0.292
0.507
0.269
0.254
720
0.342
0.340
0.344
0.142
0.358
0.347
0.354
0.348
0.400
0,428
0.365
0.389
0.348
0.385
0.364
0.353
0.346
0.343
5.2
Main Results
We present the forecasting results of our FilterNet compared to several representative baselines on eight benchmarks with various prediction lengths in Table [ Additional results with different lookback window lengths are reported in Appendix (F and G.3 Table demoonsraaes that our model consistently outperforms other baselines across different benchmarks. The average improvement of FilterNet over all baseline models is statistically significant at the confidence of 95%. Specifically, PaiFilter performs well small datasets (e.g., ETTh1), while TexFiltes excels on large datasets (e.g., Electricci) due to the ability to model the more complex and contextual correlations present in larger datasets. Also, the prediction performance of iTransformer [17], which achieves the best results on the Traffic dataset (862 variables) but not on smaller datasets, suggests that simpler structures may be more suitable for smaller datasets, while larger datasets require more contextual structures due to their complex relationships. Compared with FITS (32) built on low-pass filters, our model outperforms it validating an all -aas filter is more effective. Since PaiFilter is simple yet effective, the following FilterNet in the experimental section refer to PaiFilter unless otherwise stated.
5.3 Model Analysis
In this part, we conduct experiments to delve into a thorough exploration of frequency filters, including their modeling capabilities, comparisons among different types of frequency filters, and the various factors impacting their performance. Detailed experimental settings are provided in Appendix C.5
Modeling Capability of Frequency Filters Despite the simplicity of frequency filter architecture, [
that this architecture can also achieve competitive performance. Hence, in this part, we perform experiments to explore the modeling capability of frequency filters. Particularly, given the significance of trend and seasonal signals in time series forecasting, we investigate the efficacy of simple filters in modeling these aspects. We generate a trend signal and multi-period signal with noise, and then we leverage the frequency filters (i.e., PaiFilter) to perform training on the two signals respectively. Subsequently. we produce prediction values based on the trained frequency filters. Specifically, Figure 4[a) and (4b) show that the filter can effectively model trend and periodic signals respectively even compared with state-of-the-ar iTransformer 17 when the data contains noise. These results illustrate that the filter has excellent and robust modeling capabilities for trend and periodic signals which are important components for time series. This can also explain the effectiveness of FilterNet since the filters can perform well in such settings.
7

---

Grundfuthh
2
GroundTarh
2.21
Filneter
2.00
IrpuDoa
2.00
160 200
25
75
100 125
175
200
Time
Time
(a) Prediction on trend signals with noises
(b) Prediction multi- multi-periodic signals with noises
Figure 4: Predictions produced by FilterNet trend and multi- periodic signals with noises. When adding noises for interference, FilterNet can perform more robust forecasting than iTransformer [17].
Table 2: Performance evaluation of forecasting using two different kinds of frequency filters on the ETThl and Exchange datasets with a lookback window size of 96 and the prediction lengths T E {96, 336, 720). Results highlighted in red indicate the best performance.
Datasets
ETThl
Exchange
Lengths
720
336
720
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAE
0.375
0.394
0.46
0.422
0.476
0.443
0.474
0.469
0.083
0.202
0.174
0.296
0.326
0.413
0.840
0.670
0.382
0.402
0.430
0.429
0.472
0.411
0.481
0.473
0.091
0.211
0.186
0,305
0.380
0.449
0.896
0,712
Shared vs. Unique Filters Among Channels To analyze the different channel strategies of filters, we further conduct experiments on the ETTh and Exchange datasets. Specifically, we compare forecasting performance under different prediction lengths between two different types of frequency filters, i.e., H(Un)) and H(dnd) H filters are shared across different channels, whereas
H. signifies filters unique to each channel. The evaluation results are presented in Table 2 It demonstrates that filters shared among different channels consistently outperform across all prediction lengths. In addition, different types of filters, as illustrated in Figure IT (see Appendix G.J). The visualization reveals that the prediction values generated by filters shared among different channels exhibit a better than the unique filters. Therefore, the strategy of channel sharing seems to be better suited for time series forecasting and filter designs, which is also validated DLinear [12 and PatchTST [16]
visualize the prediction values predicted the ETTh] dataset by the two
Visualization of Prediction We present prediction showcase on ETTh1 dataset, as shown in Figure 5 We select iTransformer [17]. PatchTST [16] as the representative compared methods. Comparing with these different state-of-the-an models, we can observe FilterNet delivers the most accurate predictions of future series variations, which has demonstrated superior performance. In addition, we include more visualization cases and please refer to Appendix G.3
(a) FilterNet
(b) iTransformer
(c) PatchTST
Figure 5: Visualization of prediction the ETTh dataset with lookback and horizon length as 96.
Visualization of Frequency Filters To provide comprehensive overview of the frequency re- sponse characteristics of frequency filters, we conduct visualization experiments on the Weather, ETTh, and Traffic datasets with the lookback window length of 96 and the prediction length of 96.
The frequency response characteristics of learned filters are visualized in Figure From Figures ((a) and (b)) we can observe that compared with Transformer-bee based approaches (e.g., iTransformer I7
PatchTST [[6] tend to attenuate high-frequency components and preserve low- low-frequency infor- mation, FilterNet exhibits more nuanced and adaptive filtering behavior that can be capable of attending to all frequency components. Figure 7(c) demonstrates that the main patterns of the Traffic dataset primarily resides in the low-frequency range. This observation also explains why iTransformer performs well the Traffic dataset, despite its low-frequency nature. Overall, Figure demonstrates

---

3.5
0.
0.
Prequency
Prequency
(a) Weather
(b) ETThl
(c) Traflic
Figure 7: Spectrum visualizations of filters learned the Weather, ETTh1, and Traffic datasets.
Amelhbate
Anoittat
30
3.0
20
(a) 96
96
(h) 96 192
(c) 96
336
Figure 8: Spectrum visualizations of filters learned on ETTml with different prediction lengths.
that FilterNet possesses comprehensive processing capabilities. Moreover, visualization experiments conducted on the ETTml dataset across various prediction lengths, shown in Figure 8 further illustrate the extensive processing abilities of FilterNet. Additional results conducted on different lookback window lengths and prediction lengths can be found in Appendix G.2
Efficiency Analysis The complexity of FilterNet is O(Lo L) where the input length. To comprehensively assess efficiency, we evaluate it based on two dimensions: memory usage and training time. Specifically, timestamps) and Electricity datasets (321 variables, 26304 timestamps). We compare the efficiency of our FilterNet with the representative Transformer- and MLP-based methods under the same settings (lookback window length of 96 and prediction length of 96), and the results are shown in Figure & It highlights that FilterNet surpasses other Transformer models, regardless dataset size. While our approach exhibits similar efficiency to DLinear, our effective results outperform its performance. In Appendix B
choose two different sizes of datasets: the Exchange (8 variables, 7588
we further conduct ablation studies validate the rationale of FilterNet designs.
Exchange (8 Variables)
Electricit (321 Variables)
Memury Evuiprint
Meinuuy Fouiprini
0.27
0.36,
0.7GB,
Autofermer
LOGB, 35.2s
0.7GB, 4.84
0.8GB, 29.2s
0,5GB 0,B
1.3GB
0.5GB
0.7GB
1,3BB
80.18
Informer 6:7G,, 38.0s
8.322
DiLincar 0.5GB, 0.9s
Transformer 0.6GB, 2.64
PatchTNT
Transformer
0.6GR, 12.66
168.4s
0.7GB 5.2s
0.20
FilterNet 0.6BB, 14.2s
0.68
FiltorNet
FreTS PatekTTT 0.7GR,3.8t
0.15
Transformer Antuformer
0.8GB, 37.8s
0.04
laning Ialepoch)
Zaiing Iorepoohs
Figure 6: Model effectiveness and efficiency comparison on the Exchange and Electricity datasets.
Conclusion Remarks
In this paper, explore an interesting direction from signal processing perspective and make new attempt apply frequency filters directly for time series forecasting. We propose simple yet effective architecture, FilterNer, built upon our proposed two kinds of frequency filters to accomplish the forecasting. Our comprehensive empirical experiments on eight benchmarks have validated the superiority of our proposed method terms of effectiveness and efficiency. We also include many careful and in-depth model analyses of FilterNet and the internal filters, which demonstrate many good properties. hope this work can facilitate more future research integrating signal processing techniques or filtering processes with deep learning time series modeling and accurate forecasting.

---

Acknowledgments and Disclosure of Funding
The work was supported part by the National Natural Science Foundation of China under Grant 92367110, the Shanghai Baiyulan TalentPlan Pujiang Project under Grant 23PJ1413800, and Shanxi Scholarship Council of China (2024-61).
References
[1] Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with unified deep model. Nat. Mac. Intell., 5(6):602-611 61 2023.
[2] Kun Yi, Qi Zhang. Hui He, Kaize Shi, Liang Hu, Ning An, and Zhendong Niu. Deep coupling network for multivariate time series forecasting. ACM Trans. Inf. Syst., 42:5):177: 2024.
[3] Hui He, Qi Zhang, Simeng Bai, Kun Yi, and Zhendong Niu. CATN: cross attentive tree-aware network for multivariate time series forecasting. In AAAI, pages 4030-4038. AAAI Press, 2022.
[4] Adebiyi A Ariyo, Adewumi O Adewumi, and Charles K Ayo. Stock price prediction using the arima model. In 2014 UKSim-AMSS 16th international conference computer modelling and simulation, pages 106-112 12. IEEE, 2014.
[5] Nijat Mehdiyev, Johannes Lahann, Andreas Emrich, David Enke, Peter Fettke, and Peter Loos. Time series classification using deep learning for process planning: A case from the process industry. Procedia Computer Science, 1 11:2242-24,, 2017.
[6] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a survey. Philosoph- ical Transactions of the Royal Society A, 37902194):202000099 2021.
[7] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181-119, 2020.
[8] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In SIGIR, pages 95-104, 2018.
[9] Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolu- tional and recurrent networks for sequence modeling. CoRR, abs/1803.0127/. 2018.
[10] Minhao Liu, Ailing Zeng. Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma. and Qiang Xu. Scinet: time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816-5885 2022.
[11] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv: 10437, 2019.
[12] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI, pages 11121-11128. AAAI Press, 2023.
[13] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mlps are more effective learners in time series forecasting. In NeurlPS, 2023.
[14] Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, In AAAI, pages 1 11106-11115, 2021.
[15] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In NeurlPS, pages 22419- 22430. 2021.
10

---

[16] Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
[17] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In ICLR. OpenReview.net, 2024.
[18] Richard Asselin. Frequency filter for time integrations. Monthly Weather Review, 100(6):487- 490, 1972.
[19] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021.
[20] Pierre Duhamel and Martin Vetterli. Fast fourier transforms: a tutorial review and a state the art. Signal processing, 19(4):25--299 1990.
[21] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: survey. Philosoph- ical Trunsactions of the Royal Suciety A: Mathematical, Physical and Engineering Sciences, $7922094):22200209 feb 2021.
[22] Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza, Max Mergenthaler, and Artur Dubrawski. N-hits: Neural hierarchical interpolation for time series forecasting. CORR, Abs/2201.12886 2022.
[23] Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. CoRR, (ttx/2305.1072... 2023.
[24] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pages 5998-6000 2017.
[25] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML, 2022.
[26] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting, In ICLR, 2021.
[27] Kun Yi, Qi Zhang, Longbing Cao, Shoujin Wang, Guodong Long, Liang Hu, Hui He, Zhendong Niu, Wei Fan, and Hui Xiong. A survey on deep learning based time series analysis with frequency transformation. CoRR, abs/2302.02173, 2023.
[28] Wei Fan, Kun Yi, Hangting Ye, Zhiyuan Ning, Qi Zhang, and Ning An. Deep frequency derivative learning for non-stationary time series forecasting. IJCAI, 2024.
[29] Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective. In NewrIPS, 2023.
[30] Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tic- Yan Liu. DEPTS: deep expansion learning for periodic time series forecasting. In ICLR. OpenReview.net, 2022.
[31] Tian Zhou, Ziqing Ma, Xue Wang, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, and Rong Jin. Film: Frequency improved legendre memory model for long-term time series forecasting. In NeurIPS, 2022.
[32] Zhijian Xu, Ailing Zeng, and Qiang Xu. FITS: modeling time series with 10k parameters. CoRR, abs/2307.03756, 2023.
[33] Fausto Pedro García Márquez and Noor Zaman. Digital filters and signal processing. BoD- Books on Demand, 2013.
[34] RTMAC Lu. Algorithms for discrete Fourier transform and convolution. Springer, 1989.
11

---

[35] John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, and Bryan Catanzaro. Adaptive fourier neural operators: Efficient token mixers for transformers. CoRR. abs/2111.13555 2021.
[36] Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881-9892 2022.
[37] Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Deeptime: Deep time-index meta-learning for non-stationary time-series forecasting. arXiv preprint arXiv:2207.06046, 2022.
[38] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In SIGIR, pages 95-104, 2018.
[39] Huiqiang Wang, Jian Peng. Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. MICN: multi-scale local and global context modeling for long-term series forecasting. ICLR. OpenReview.net, 2023.
[40] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long, Timesnet: Temporal 2d-variation modeling for general time series analysis. In ICLR. OpenReview.net, 2023.
[41] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, gegh-performance deep learning library. In NeurIPS, pages 8024-8035, 2019.
12

---

A More Analysis about the Architecture of FilterNet
The Necessity of Instance Normalization Block is equal to zero frequency component. Specifically, given a signal xr(n] with a length of N, we can obtain its corresponding discrete Fourier transform [[k] by:
From the frequency perspective, the mean value
N
x rlmje 2mjnk/N
(10)
n=0
where j is the imaginary unit. We set k as 0 and then,
N
X
x
r/mle2mino/N
(11)
N Σ
s/m].
According the above equation, we can find that the mean value x E x[n] the time domain is equal to the zero frequency component X in the frequency domain. Similarly, we can also view the standard deviation from the frequency perspective, and is related to the power spectral density.
According to these analysis, the instance normalization is analogous to a form of data preprocessing. Given that filters are primarily crafted to discern particular patterns within input data, while instance normalization aims to normalize each instance in dataset, function distinct from the conventional role of filters, we treat instance normalization as a distinct block within our FilterNet architecture.
Frequency Filter Block Recently, Transformer- and MLP-based methods have emerged as the two main paradigms for time series forecasting, exhibiting competitive performance compared other model architectures. Building on prior work that conceptualizes self-attention and MLP architectures as forms of global convolution 35 [K] it becomes apparent that frequency filters hold promise for time series forecasting tasks. Just as self-attention mechanisms capture global dependencies
and MLPs learn convolve features across the entire input space, frequency filters offer means to extract and emphasize specific temporal patterns and trends from time series data. By applying frequency filters time series data, we can learn recurring patterns, trends, and periodic behaviors that are essential for forecasting future time series data and making accurate predictions.
Feed-forward Network Incorporating feed-forward network within the FilterNet architecture is essential for enhancing the model's capacity to capture complex relationships and non-linear patterns within the data. While frequency filters excel extracting specific frequency components and temporal patterns from time series data, they may not fully capture the intricate dependencies and higher-order interactions present in real-world datasets [17]. By integrating feed-forward network, the model gains the ability to learn hicrarchical representations and abstract patterns from the input data, allowing it to capture more nuanced relationships and make more accurate predictions. This combination of frequency filters and feed-forward network leverages the strengths of both approaches, enabling the model to effectively process and analyze time series data across different frequency bands while accommodating the diverse and often nonlinear nature of temporal dynamics. Overall, the inclusion of a feed-forward network enriches the expressive power of FilterNet, leading to improved forecasting performance and robustness across various domains.
B Explanations about the Design of Two Filters
Self-attention mechanism is a highly data- data-dependent operation that both derives its parameters from data and subsequently applies these parameters back to the data. Concretely, given the input data X, the self-attention can be formulated as:
QKT
SA(Q, K, V) softmax(
VV,
(12)
Vdg
where Q (queries), K (keys), and V (values) are linear transformations of the input data X, as:
Q WoX, K WkX, V WvX,
(13)
13

---

where Wo. WK. and Wy are learned weight matrices. Since the values Q, K, and V are derived directly from the ipput data X, the attention scores SA are inherently dependent on the data.
Unlike self-attention mechanism that dynamically adapts to the input data during inference, MI.s maintain consistent architecture regardless of the dataset characteristics. Specifically, for the input data X, MLP can be formulated as:
(LP(X) W +
(14)
where W are the learned weights and b are the learned biases. Once trained, the weights W and biases 6 remain static, meaning that they do not dynamically change in response new data inputs,
MLPs are straightforward, less data-dependent models that apply fixed transformations to the input data, making them suitable for tasks with static relationships between the input data. In contrast, self-attention mechanisms are highly data-dependent, dynamically computing attention scores based on the input data to capture complex, context- dependencies, making them ideal for tasks requiring an understanding of sequential or structured data.
Inspired by the two paradigms, FilterNet designs two corresponding types of filters: plain shaping filters and contextual shaping filters. Plain shaping filters offer stabilitty and efficiency, making them suitable for tasks with static relationships. In contrast, contextual shaping filters provide the flexibility capture dynamic dependencies, excelling in tasks that require context-sensitive analysis. This dual approach allows FilterNet to effectively handle a wide range of data types and forecasting scenarios, combining the best aspects of both paradigms achieve superior performance.
C Experimental Details
C.1 Datasets
We evaluate the performance of our proposed FilterNet on eight popular datasets, including Ex- change, Weather, Traffic, Electricity, and ETT datasets. In detail, the Exchange dataset collects daily exchange rates of eight different countries including Singapore, Australia. British, Canada, Switzerland, China, Japan, and New Zealand ranging from 1990 to 2016. The Weather dataset, including 21 meteorological indicators such as air temperature and humidity, is collected every 10 minutes from the Weather Station of the Max Planck Institute for Biogeachemistry in 2020. The Traffic [15] dataset contains hourly traffic data measured by 862 sensors on San Francisco Bay area freeways, which has been collected since January 1, 2015. The Electricity dataset collects the hourly electricity consumption of 321 clients from 2012 to 2014. The ETT (Electricity Transformer Temperature) datasets contain two visions of the sub-dataset: ETTh and ETTm, collected from electricity transformers every 15 minutes and hour between July 2016 and July 2018. Thus, in total we have 4 ETT datasets (ETTm1, ETTm2, ETTh1, and ETTh2) recording features such load and oil temperature. The details about these datasets are summarized in Table
Table 3: The details of datasets.
Datasets
ETThl
ET
FTTml
FTTm2
Flectricity
Weather
Traffic
Exchange
Variables
7
7
7
321
21
862
8
Timesteps
17420
Frequency
Hourly Electricity
Hourly Electricity
15min Electricity
15min Electricity
Hourly Electricity
10min
Hourly Traflic
Daily
Information
Weather
Economy
C.2 Baselines
We choose twelve well-acknowledged and state-of-the-ar models for comparison evaluate the effectiveness of our proposed FilterNet for time series forecasting, including Frequency-based models, TCN-based models, MLP-based models, and Transformer-basse models. We introduce these models as follows:
https://githubbco hatp:://gitttt.co///agguuuuu/// http://twww.....att/////aaatttaa
hhtpss//www.hhc-jona... www.a..p..d///eteee.rreeeeeee
"htpss//acchive.iss..u wwwp://wwwwwii.iiiiiiiiioom
12014
14

---

FreTS [33 introduces novel direction of applying MLPs in the frequency domain to effectively capture the underlying patterns of time series, benefiting from global view and energy compaction. The official implementation is available at https::/ //op:::///iie git..tttttb...... com/ aikunyi/FreT To ensure fair and objective comparison, the results showed in Table are obtained using instance normalization instead of min-max normalization in the original code.
performs time series analysis through interpolation in the complex frequency domain, enjoying low cost with 10k parameters. The official implementation is available at cattts::/giihin /////I Because its original paper doesn't 't provide the forecasting results with the fixed lookback length L = 96, we report the performance of FITS with lookback length L = 96 under five runs in [
FITS 32)
MICN 39 employs multi-scale branch structure to model different potential patterns separately with linear complexity. It combines local features and global correlations to capture the overall view of time series. The official implementation is available at fhttps: //tpt//gttttbbbt com www.vangggg1//MOO... The experimental results showed in Table follow its original paper.
TimesNet 40) transforms ID time series into set of 2D tensors based on multiple periods to analyse temporal variations. The above transformation allows the 2D-variations to be easily captured by 2D kernels with encoding the intraperiod- and crrperiodd-variaionn into the columns and rows of the 2D tensors respectively. The official implementation is available at https: http::://ghtttt - com/thume /tmmtumll/// TimesNet The results showed in Table I follow iTransformer and the results showed in Table 3 follow RLinear
23
DLinear 12 utilizes a simple yet effective one-layer linear model to capture temporal relationships
between input and output sequences. The official implementation is available at wittt:t:////
//g //sttss//g/ thub We report the performance of DLinear with lookback length L E
com/cure-ioe Laaa//TSF-Lineet
} under five runs in Table [
RLinear 23 uses linear mapping model periodic features in multivariate time series with robust- ness for diverse periods when increasing the input length. It applies RevIN (reversible normalization) and CI (Channel Independent) to improve overall forecasting performance by simplifying learning about periodicity. The official implementation is available at https: //githuu. com/ camm/lumprcc RTSF
The experimental results showed in Table follow i'Tranffrmmer
Informer 14 enhances Transformer with KL--dieereence based ProbSparse attention for ( log L)
complexity, efficiently encoding dependencies among variables and introducing a novel architecture com/
with DMS forecasting strategy. The official implementation is available at hhttt::///// //gp:::////te tug.th.b..........m thub.
chouhaoyi/Inforrme2000 and the experimental results showed in Table follow FEDiormer 255
Autoformer [5] employs a deep decomposition architecture with auto-correlation mechanism to extract seasonal and trend components from imput series, embedding the series decomposition block as an inner operator. The official implementation is available at (attps: h/tps:://githbl com/ com/tuum//
Autoformer and the experimental results showed in Table 4 follow FElformer (25
Pyraformer 26 introduces pyramidal attention module (PAM) with an OLL) time and memory
complexity where the inter-scale tree structure summarizes features at different resolutions and
the intra-scale neighboring connections model the temporal dependencies of different ranges. The official implementation is available at htttp:///githue com/ www.cott reserrtteeecc/eeaeeeee htwecccchyyraoorro ormer and the experimental results showed in Table DLinear 11].
implements sparse attention with low-rank approximation in frequency domain, enjoying linear computational complexity and memory cost, And proposes mixture of experts decomposition to control the distribution shifting. The official implementation is available at https 7/github. com/MAZiqing/ Faiinn///DDfoooae ormer and the experimental results showed in Table follow TTranfformrr [1].
FEDformer (25
PatchTST [16] divides time series data into subseries-level patches to extract local semantic in- formation and adopts channel-independenne strategy where each channel shares the same embed- ding and Transformer weights across all the series. The official implementation is available at https://gg thub. commyuqinie98/ PPTTcchTTT The experimental results showed in Table follow iTransformer [17]. And because iTransformer doesn't provide the prediction results with lookback length L = 336, we report the performance of PSt with lookback length L = 336 under five runs in Table
15

---

iTrransfommer [17] inverts the structure of Transformer without modifying any existing modules by encoding individual series into variate tokens. These tokens are utilized by the attention mechanism to capture multivariate correlations and FFNs are adopted for each variate token to learn nonlin- ear representations. The official implementation is available at /https://gil wtt.ttt......m.tote hoom/hhmm// iTransf )rmer) The experimental results showed in f its original paper. And because iTransformer doesn't provide the prediction results with lookback length = 336, we report the performance of iTransformer with lookback length L 336 under five runs in Table
C.3 Implementation Details
The architecture of our FilterNet is very simple and has two main hyperparameters, i.c., the bandwidth of filters and the hidden size of FFN. As shown Figure the bandwidth the filters corresponds to the lookback window length, so we select the lookback window length as the bandwidth accordingly.
For the hidden size of FFN, we carefully tune the size over [64, 128, 256, 512]. Following the previous methods [66 32), we use RevIN [19] as our instance normalization block. Besides, we carefully tune hyperparameters including the batch size and learning rate on the validation set, and we choose the settings with the best performance. We tune the batch size over [4, 8, 16, 32) and
tune the learning rate over (0.01, 0.05, 0.001, 0.005, 0.0001, 0.0005).
C.4 Experimental Settings for Simulation Experiments
To evaluate the Transformer's modeling ability for different frequency spectrums, we generate signal consisting of three different frequency components, i.e., low-frequency, middle-frequent and high-frequency. We then apply both iTransformer and FilterNet to this signal, respectively, and compare the forecasting results with the ground truth. The results are presented in Figure
C.5 Experimental Settings for Filters Analysis
Modeling capability of frequency filters We generate two signals: a trend signal with Gaussian noise and a multi-periodic signal with Gaussian noise. We then apply PaiFilter to these signals with a lookback window length of 96 and a prediction length of 96. The results are displayed in Figure 4
RIxA, where L is its bandwidth, we visualize the frequency response characteristic of the filter by plotting the values in 1x First, we perform a Fourier transform on these values to obtain the spectrum, which includes the frequency and its corresponding amplitude. Finally, we visualize the spectrum, as shown in Figures 7 and 12
Visualization of Frequency Filters Given a filter
D Study of the Bandwidth of Frequency Filters
The bandwidth parameter (i.e., L in Equation 8 and D in Equation ( holds significant importance in the functionality of filters. In this part, we conduct experiments on the Weather dataset to delve into the impact of bandwidth within the set {96, 128, 192, 256, 320, 386, 448, 512} while keeping the lookback window length and prediction length constant. Specifically, we conduct experiments evaluate the impact under three different combinations of lookback window length and prediction length, i.e., 96 - 96, 96 - 192, and 192 192, and the results represented in Figure 9 relationship between bandwidth settings and lookback window length. Figure 9(a) and Figure (b) show that increasing the bandwidth results in minimal changes in forecasting performance. Figure [c] demonstratsss that while forecasting performance fluctuates with increasing bandwidth, it is optimal when the bandwidth equals the lookback window length. These results indicate that using the lookback window length as the bandwidth is sufficient since the filters can effectively model the data at this setting, and it also results in lower complexity.
forecasting performance. We explore range of bandwidth values
We observe clear trends in the
E Ablation Study
To validate the rationale behind the architectural design of our FilterNet, we conduct ablation studies on the ETTml, ETThl, and Electricity datasets. We evaluate the impact the model's 's performance
16

---

0.20
MBE
0.26
0.22
0.28
0.24
0.20
0.24
0.22
0.18
0.22
0.21
0.14
0.20
0.20
0.10
128
192
256
384
512
0.18
128
192
256
512
0.18
192
256
320
354
446
512
Bandwidtt
Bandwidth
Bandwidth
(a) 96 +
96
(b) 96
+ 192
(c) 192
+ 192
Figure MSE and MAE of filters under different bandwidths on the Weather dataset.
238
0.20
0.20
3.38
1.00
0.38
0.40
(a) ETThl
(b) ETTml
(c) Electricity
Figure 10: Ablation Studies the ETThL, ETTml, and Electricity datasets.
by eliminating the particular component of the FilterNet architecture. The evaluation results are present in Figure 10 In the figure, W/O Norm indicates that instance normalization and inverse instance normalization have been removed from FilterNet. W/ Filter sigmifies the removal of the filter block, and W W/O W/OFFT, FFN denotes the exclusion of the feed-forward network. The experiments are conducted with lookback window length of 96 and output length of 96. From the figure, we can conclude that each block indispensable, as the removal of any component results noticeable decrease in performance. This highlights the critical role each block plays in the overall architecture of FilterNet, contributing to effectiveness time series forecasting.
Additional Results
Table compares the performance of various methods with our FilterNet, demonstrating that model consistently outperforms the others. To further assess the performance of FilterNet under different lookback window lengths, conducted experiments the ETTh1, ETTm1, Exchange, Weather, and Electricity datasets with the lookback window length of 336. The results, shown in Table indicate that our model achieves the best performance across these datasets.
17

---

Table 4: Multivariate long-term forecasting results with prediction lengths 7 E {96, 392, 720} and fixed lookback length L = 96. The best results are in red and the second best are blue.
Models
TexFilter
PaiFilter
Autoformer
Informer
Pyraformer
MICN
FreIS
Metries
MSB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.321
0.361
0.318
0.358
0.505
0.475
0.672
0.571
0.543
0.510
0.316
0.362
0.335
0.372
192
0.367
0.387
0.364
0.383
0.553
0.496
0.795
0.669
0.557
0.537
0.363
0.390
0.388
0.401
336
0.401
0.409
0.396
0.406
0.621
0.537
1.212
0.871
0.754
0.655
0.408
0.426
0.421
0.426
720
0.477
0.448
0.456
0.444
0.671
0.561
1.166
0.823
0.908
0.724
0.481
0.476
0.486
0.465
0.175
0.25%
0.174
0.255
0.339
0.365
0.435
0.507
0.179
0.275
0.189
0.277
ET7m2
192
0.240
0.301
0.240
0.300
0.281
0.340
0.533
0,563
0.730
0.673
0.307
0.376
0.258
0.326
336
0.311
0.347
0.297
0.339
0.339
0.372
1.363
0.887
1.201
0.845
0.325
0.388
0.343
0.390
720
0.414
0.405
0.392
0.393
0.422
0.419
3.379
1.338
3.625
1.451
0.502
0.490
0.495
0.480
96
0.382
0,402
0.375
0.394
0.449
0.459
0.865
0.713
0.664
0.612
0.421
0,431
0.395
0.407
192
0.430
0.429
0.436
0.422
0,500
0,482
1.008
0.792
0.790
0.681
0.474
0.487
0.448
0.440
336
0.472
0,451
0.476
0,443
0.521
0,496
1.107
0.809
0.891
0.738
0.569
0.551
0.499
0.472
720
0.481
0.473
0.474
0.469
0,514
0.512
1,181
0.865
0.963
0.782
0.770
0.672
0.558
0.532
96
0,293
0.343
0.292
0,343
0.358
0.397
3.755
1.525
0.645
0.597
0.299
0,364
0.309
0.364
192
0.374
0.396
0.369
0.395
0.456
0.452
5.602
1.931
0.788
0.683
0.441
0.454
0.395
0.425
336
0,417
0.430
0.420
0,432
0,482
0,486
4.721
1.835
0.907
0.747
0.654
0.567
0.462
0.467
720
0.449
0.460
0.430
0.446
0,51.
0,511
3.647
1.625
0.963
0.783
0.956
0,716
0.721
0.604
96
0.147
0.245
0.176
0.264
0.201
0.317
0.274
0.368
0.386
0.449
0.164
0.269
0.176
0.258
BCL
0.160
0.250
0.185
0.270
0.222
0.334
0.296
0.386
0.386
0.443
0.177
0.285
0.175
0.262
336
0.173
0.267
0,202
0.286
0,231
0.338
0,300
0.394
0.378
0.443
0.193
0.304
0.185
0.278
720
0.210
0.349
0.242
0.319
0.254
0.361
0.373
0.439
0.376
0..45
0.212
0.321
0.220
0.315
Exchunge
96
0.091
0.211
0.083
0.202
0.197
0.323
0.847
0.752
0.376
1.105
0.102
0.235
0.091
0.217
0.186
0.305
0.174
0.296
0,300
0.369
1.204
0.895
1.748
1.151
0.172
0.316
0.175
0.310
336
0.380
0.449
0.326
0.413
0.509
0,524
1.672
1.036
1.874
1.172
0.272
0.407
0.334
0.434
0.896
0.712
0.840
0.670
1.447
0.941
2.478
1.310
1.943
1,206
0.658
0.771
0.674
96
0.430
0.294
0.506
0.336
0.613
0,388
0.719
0.391
2,085
0,468
0.519
0.309
0.593
0.378
Traffic
192
0.452
0.307
0.508
0.333
0.616
0.382
0.696
0.379
0.867
0.467
0.537
0,315
0.595
0.377
336
0.470
0.316
0.518
0.335
0.622
0.337
0.777
0.420
0.869
0.469
0.534
0.313
0.609
0.385
720
0.498
0.323
0.553
0.354
0,660
0.408
0.864
0.472
0.881
0.473
0.577
0.325
0.673
0.418
Weather
96
0.162
0.207
0.164
0.210
0.266
0.336
0.300
0.384
0.896
0.556
0.161
0.229
0.174
0.208
192
0.210
0.250
0.214
0.252
0.307
0,367
0.598
0,544
0.622
0.624
0.220
0.281
0.219
0.250
336
0.265
0.290
0.268
0.293
0.359
0.395
0.578
0.523
0.739
0.753
0.278
0.331
0.273
0.290
720
0.342
0.340
0.344
0.342
0.419
0.428
1.059
0,741
1,004
0.934
0.311
0,356
0.334
0.332
Table 5: Time series forecasting comparison. We set the lookback window size I. as 336 and the prediction length as T E {96, 192, 72). The best results are in red and the second best are blue.
Models
Paiffilter
DLinear
iTranstormer
PatchTST
TimesNet
Metries
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.379
0.404
0.384
0.405
0.402
0.418
0.381
0.405
0.398
0.418
ETThl
192
0.417
0.428
0,430
0.442
0,450
0.449
0.442
0.446
0.447
0.449
336
0.437
0.443
0.447
0.448
0.479
0.470
0.445
0.454
0.493
0.468
720
0.458
0.472
0.504
0.515
0.584
0.548
0.490
0.493
0.518
0.504
96
0.289
0.344
0.300
0.345
0.303
0.357
0.294
0.345
0.335
0.380
ETTml
192
0.331
0.369
0.336
0.366
0.345
0.383
0.334
0.371
0.358
0.388
336
0.364
0.389
0.372
0.390
0.382
0.405
0.371
0.392
0.406
0.418
720
0.425
0.423
0.427
0.423
0.443
0.439
0.421
0.419
0.449
0.443
Weather Exchange
96
0.087
0.216
0.085
0.209
0.099
0.226
0.093
0.213
0.117
0.253
192
0.163
0.301
0.162
0.296
0.216
0.337
0.194
0.315
0.298
0.410
336
0.287
0.399
0.350
0.445
0.395
0.466
0.354
0.435
0.456
0.513
720
0.413
0.492
0.898
0.725
0.962
0.745
0.903
0.712
1.608
0.961
96
0.150
0.183
0.175
0.235
0.164
0.216
0.151
0.197
0.172
0.220
192
0.193
0.221
0.218
0.278
0.205
0.251
0.197
0.244
0.219
0.261
336
0.246
0.258
0.263
0.314
0.256
0.290
0.251
0.285
0.280
0.306
720
0.308
0.295
0.324
0.362
0.326
0.338
0.321
0.335
0.365
0.359
Electricity
96
0.132
0.224
0.140
0.237
0.133
0.229
0.130
0.222
0.168
0.272
192
0.143
0.237
0.154
0.250
0.156
0.251
0.1448
0.240
0.184
0.289
336
0.155
0.253
0..
0.268
0.1
0.267
0.167
0.261
1.8
0.300
720
0.195
0.292
0.204
0.300
0.209
0.304
0.202
0.291
0.220
0.320
18

---

Oreundfouth
-2.c-
-15
-2.
-0.
-0.
-1.0
-10
-1.
(a) Forecasting with channel-shared filters
(b) Forecasting with channel-umique filters
Figure 11: Visualizations on the ETTh1 dataset.
Amplttaa
Proquency
Proquency
Prequency
(a) - 96 Electricity
(b) 336
192 on Electricity
(c) 336
336 on Electricity
Figure 12: Spectrum visualizations the filters learned on the Electricity dataset with lookback window length of 336 and prediction lengths {96, 192,
G
Visualizations
G.1
Visualization of Channel-shared vs Channel-unique Filters
To further compare the channel-shared and channel-unique filters, we visualize the prediction values by the corresponding filters. The results shown in Figure [I] The figure demonstrates that the values predicted by channel-shared filters closely align with the ground truth compared to those predicted by channel-unique filters. This observation is consistent with the findings presented in Table 2 indicating the superiority of channel-shared filters.
G.2 Visualization of Frequency Filters
We further conduct visualization experiments to explore the learnable filters under different lookback window lengths and prediction lengths. The experiments are performed on the Electricity dataset, and the results are illustrated in Figure 12 These figures illustrate that FilterNet possesses full spectrum learning capability, as the learnable filters exhibit values across the entire spectrum. Besides, we observe that the frequency primarily concentrates in the low and middle ranges which explains that some works based on low-pass filters can also achieve good performance.
G.3 Visualizations of Predictions
To further offer an evident comparison of our model with the state-of-the-art models, we present supplementary prediction showcases on ETTml dataset, and the results are shown [3 We choose the following representative models, including iTransformer [17]. PatchTST (16), and DLinear (12), as the baselines. Comparing with these different types of state-of-the-art models, FilterNet delivers the most accurate predictions of future series variations, demonstrating superior performance.
19

---

Proditio re.
Predise houChei
Thee
Thme
(a) FilterNet
(b) iTransformer
-77
AW
Gourthue
4.3
-20
43
-25
-10
-.6
-15
Marn fulh
-12
Time
Time
(c) DLinear
(d) PatchTST
Figure 13: Visualization of forecasting results on the ETTml dataset with lookback window length
L = 96 and prediction length r 96.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims are clearly written in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitation of our method in the conclusion remarks.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: No theoretical proofs.
21

---

Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We included the implementation details in the main paper and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
22

---

Answer: [Yes]
Justification:
We will include the source data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com htte:/uulllllliiiee/e//eeeuaaass ssiniiiicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Please see implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, the appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The average improvement of FilterNet over all baseline models is statistically significant at the confidence of 95%.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
23

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:: /ttppp////aauoop..cc///////////a
Answer: [Yes]
Justification: This paper is with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the broader impact of time series forecasting in both abstract and introduction.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not have this risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We included it in implementation details.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
25

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: N/A.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "PertUvlFjE", "reviewer": "Reviewer_ufqs", "summary": "This paper investigates an interesting direction from a signal processing perspective, making a new attempt to apply frequency filters directly for time series forecasting. A simple yet effective architecture called FilterNet is proposed, utilizing two types of frequency filters to achieve forecasting. Comprehensive empirical experiments on eight benchmarks confirm the superiority of this paper's proposed method in terms of effectiveness and efficiency.", "questions": "Please address the questions in the weaknesses.", "limitations": "No", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is in good writing shape and easy to follow.\n2. The proposed method seems technically solid.\n3. Extensive experiments on many datasets have been conducted to verify the effectiveness of the approach.", "weaknesses": "1. The marginal improvement over existing approaches, as shown in Table 1, raises doubts about the compelling rationale for adopting the proposed method.\n2. The discussion on related work appears lacking. It is common knowledge that frequency filters are widely used in traditional time series analysis methods and signal processing. Therefore, a detailed exploration of the connections and distinctions between them is warranted in the related work section.\n3. The experiments have space to improve. Including more recent baselines for comparison, particularly deep learning methods that involve frequency processing, would add value. Additionally, a thorough investigation into the impact of hyperparameters is missing from the study.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your constructive comments and suggestions. We provide a point-by-point response to your comments below:\n\n**W1. The marginal improvement over existing approaches, as shown in Table 1, raises doubts about the compelling rationale for adopting the proposed method.**\n\nA1. The primary contribution of this paper lies in offering a novel viewpoint from signal processing and devising a simple yet effecitve architecture centered around a Frequency Filter Block.\nIn contrast to prior research, FilterNet show remarkable efficiency (**as shown in Figure 8**) while simultaneously achieving state-of-the-art performance.\nFurthermore, we extensively investigate the efficacy and insights of frequency filters via introducing two distinct types of frequency filters, each exhibiting diversified performances across datasets of different scales. This exploration reveal the impact of frequency filters on time series forecasting and indicates that designing tailored architectures adaptive to data scale may enlighten a new path for future research endeavors\n\n**W2. The discussion on related work appears lacking. It is common knowledge that frequency filters are widely used in traditional time series analysis methods and signal processing. Therefore, a detailed exploration of the connections and distinctions between them is warranted in the related work section.**\n\nA2. \nIn signal processing, filters are employed to selectively enhance or attenuate specific frequency components of a signal. \nThere are various types of filters, including low-pass filter, band-pass filter, Butterworth filter, and Chebyshev filter, etc.\nThe choice of filter type is highly dependent on the signal characteristics and the desired outcome.\nHowever, in time series forecasting scenarios, the signal characteristics and desired outcomes are unknown in advance. \nThis uncertainty complicates the selection and design of appropriate filters, necessitating learnable or data-driven approaches to effectively capture and forecast the underlying patterns in the time series data.\nIn FilterNet, unlike FITS and FiLM, which directly use low-pass filters, we leverage learnable frequency filters. \nAdditionally, we design two filters that are inspired by Transformer and MLP architectures. \nThis approach allows the model to learn the most relevant frequency components, improving forecasting performance without requiring prior knowledge of the signal characteristics.\n\nWe will add these discussions in the final version.\n\n**W3. The experiments have space to improve. Including more recent baselines for comparison, particularly deep learning methods that involve frequency processing, would add value. Additionally, a thorough investigation into the impact of hyperparameters is missing from the study.**\n\nA3. \n1. We choose four categories of baselines: Frequency-based, Transformer-based, MLP-based, and TCN-based methods,\n   including the latest SOTA methods, i.e., the Transformer-based iTransformer (ICLR, 2024) and the Frequency-based FITS (ICLR, 2024).\n   We further compare FilterNet with SparseTSF (ICML, 2024) (please refer to A3 in Response to Reviewer UnVh) and FiLM (NeurIPS, 2022) (please refer to A1 in Response to Reviewer 2waa).\n   Additionally, we include Koopa (NeurIPS, 2023), which is designed for addressing non-stationary issues in time series by disentangling time-invariant and time-variant dynamics through Fourier Filter, as shown below:\n   \n|  Model | | TexFilter| | PaiFilter| | Koopa| |\n|:--------:|:----|:----|:----|:----|:----|:----|:----|\n|  Metric | | MSE|MAE | MSE|MAE | MSE|MAE | \n|Exchange | 96\t| 0.091| \t0.211| \t0.083| \t0.202| \t0.090| \t0.210| \n| | 192\t| 0.186\t| 0.305| \t0.174| \t0.296| \t0.188| \t0.310| \n| | 336\t| 0.380 | 0.449| \t0.326| \t0.413| \t0.371| \t0.444| \n| | 720\t| 0.896 | 0.712| \t0.840| \t0.670| \t0.896| \t0.713| \n| Traffic |96|\t0.430|\t0.294|\t0.506|\t0.336|\t0.517|\t0.349|\n| |192|\t0.452|\t0.307|\t0.508|\t0.333|\t0.529|\t0.358|\n| |336|\t0.470|\t0.316|\t0.518|\t0.335|\t0.540|\t0.361|\n| |720|\t0.498|\t0.323|\t0.553|\t0.354|\t0.591|\t0.386|\n\nWe choose two datasets: one smaller dataset (Exchange) and one larger dataset (Traffic), and we compare them with the look-back window length of 96.\nFrom the table, we can find that FilterNet outperforms Koopa whether the dataset is smaller or larger.\n\n2. The bandwidth of frequency filters holds significant importance in the functionality of filters, \n   and we have analyzed the bandwidth parameter in the experimental part (see **lines 254-266**).\n   For PaiFilter, there are no hyperparameters other than a bandwidth parameter.\n   For TexFilter, there is one hyperparameter $K$, and we conduct parameter sensitivity experiments about K as below:\n   \n|   | TexFilter| | \n|:--------:|:----|:----|\n| Metric  |  MSE|MAE | \n|  K=1 | 0.147 | 0.245 | \n|  K=2 | 0.145 | 0.243 | \n|  K=3 | 0.142| 0.242| \n\nAlthough adding layers can improve performance, considering resource costs with additional layers, we use a one-layer architecture (i.e.,\nK=1), similar to DLinear, FreTS, and FiLM, which also use a one-layer architecture.\nWe will add the analysis in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ur0taejbPj", "reviewer": "Reviewer_2waa", "summary": "The paper proposes to use Fourier transforms with parametrized filters\nas layers for time series forecasting models. Filters can either be\nlearnable parameters directly (called \"plain shaping filter\") or be\na function in the inputs, gates (called \"contextual shaping filter\").\nIn experiments the authors show that their method outperforms\nmany baselines from the literature.", "questions": "Q1. Why do you not report the better results of FiLM?\nQ2. If FiLM overall is the better performing model, why is FilterNet still\n  interesting? How is it simpler than FiLM?\nQ3. Is it maybe faster to train?", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "s1. two simple, but promising ideas: i) use Fourier transforms to\n  represent time series and ii) represent the filters by gates.\ns2. well written paper\ns3. standard experiment with improvements over several baselines", "weaknesses": "w1. Empirical comparison with results from a closely related baseline, FiLM\n  are not shown, and FiLM reports better results for all but one dataset.\n- FiLM reports better results for ETTm2, Exchange, and Traffic. The proposed\n  FilterNet wins for Weather. Why do you not report those results?\n  If FiLM overall is the better performing model, why is FilterNet still\n  interesting? How is it simpler than FiLM? Is it maybe faster to train?\n\nw2. Conceptual differences to existing layers using Fourier transforms\n  are not discussed in detail.\n- So far, we just learn that other architectures also use \"other network architectures,\n  such as Transformers\". But a more detailed delineation would be useful,\n  esp. as FiLM is not using Transformers.\n\nminor comments:\n- reference [30], FiLM, is missing its venue, NeurIPS.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your review regarding our paper. We would like to respond to your comments and hope that we address all your concerns below:\n\n**W1. Empirical comparison with results from a closely related baseline, FiLM are not shown, and FiLM reports better results for all but one dataset.**\n\nA1. \n1. We have compared FilterNet with the newest representative SOTA methods, i.e., the Transformer-based method iTransformer (ICLR, 2024) and the frequency-based method FITS (ICLR, 2024).\n**Note that FITS has already compared with FiLM (NeurIPS, 2022). According to the reported results from FITS (Tables 8 and 9), it shows that FITS outperforms FiLM**. Therefore, we only compare with FITS and do not choose FiLM as a baseline.\n2. **Note that FiLM conducted experiments with various input length (as described in the caption of Table 1 in FiLM), and the\ninput length is tuned for best forecasting performance**. In contrast, in the FilterNet, we follow the settings of iTransformer (ICLR, 2024), PatchTST (ICLR, 2023), and FEDformer (ICML, 2022), and set the input length fixed, such as 96 in Table 1. As a result, direct comparison to the findings in the original FiLM paper is not feasible. \n   To further compare FilterNet with FiLM, we conduct additional experiments with a fixed input length of 96, and the results are shown in below:\n   \n|  Model | | TexFilter| | PaiFilter| | FiLM| |\n|:--------:|:----|:----|:----|:----|:----|:----|:----|\n|  Metric | | MSE|MAE | MSE|MAE | MSE|MAE | MSE|MAE  | \n|ETTm2|96\t|0.180|\t0.262|\t0.174|\t0.257|\t0.183|\t0.266|\n| |192|\t0.246|\t0.306|\t0.240|\t0.301|\t0.248|\t0.305|\n| |336|\t0.312|\t0.348|\t0.297|\t0.339|\t0.309|\t0.343|\n| |720|\t0.414|\t0.405|\t0.397|\t0.398|\t0.409|\t0.400 |\n|Electricity|96| \t0.147| \t0.245| \t0.176| \t0.264|\t0.198| \t0.274| \n| | 192| \t0.160| \t0.250| \t0.185| \t0.270| \t0.198| \t0.278| \n| | 336| \t0.173| \t0.267| \t0.202| \t0.286| \t0.217| \t0.300| \n| | 720| \t0.210| \t0.309| \t0.242| \t0.319| \t0.278| \t0.356| \n|Exchange|96|\t0.091|\t0.211|\t0.083|\t0.202|\t0.087|\t0.211|\n| |192|\t0.186|\t0.305|\t0.174|\t0.296|\t0.182|\t0.308|\n| |336|\t0.380|\t0.449|\t0.326|\t0.413|\t0.351|\t0.427|\n| |720|\t0.896|\t0.712|\t0.840|\t0.670|\t0.906|\t0.715|\n|Traffic|96|\t0.430|\t0.294|\t0.506|\t0.336|\t0.646|\t0.384|\n| |192|\t0.452|\t0.307|\t0.508|\t0.333|\t0.600|\t0.361|\n| |336|\t0.470|\t0.316|\t0.518|\t0.335|\t0.609|\t0.367|\n| |720|\t0.498|\t0.323|\t0.553|\t0.354|\t0.691|\t0.425|\n|Weather|96|\t0.162|\t0.207|\t0.164|\t0.210|\t0.196|\t0.237|\n| |192|\t0.210|\t0.250|\t0.216|\t0.252|\t0.239|\t0.271|\n| |336|\t0.265|\t0.290|\t0.273|\t0.294|\t0.289|\t0.306|\n| |720|\t0.342|\t0.340|\t0.344|\t0.342|\t0.360|\t0.351|\n\nAlso, these results of FiLM with the input length of 96 can be validated by other papers, such as [1] (see its Table 13). \nAccording to the above table, we can find that overall, FilterNet outperforms FiLM. We will add these results in the final version.\n\n[1]TIMEMIXER: DECOMPOSABLE MULTISCALE MIXING FOR TIME SERIES FORECASTING, ICLR, 2024\n\n**W2. Conceptual differences to existing layers using Fourier transforms are not discussed in detail.So far, we just learn that other architectures also use \"other network architectures, such as Transformers\". But a more detailed delineation would be useful, esp. as FiLM is not using Transformers.**\n\nA2. \nWe organize the time series modeling using Fourier transforms from two perspectives: the incorporated neural network and the filter types.\nAutoformer and FEDformer build on the Transformer architecture and enhance its effectiveness by leveraging Fourier technologies. \nFreTS builds on MLP and enhance its modeling capability by leveraging frequency technologies. \nAlthough FITS and FiLM do not incorporate network architectures, they function as low-pass filters.\nIn contrast, FilterNet is an all-pass filter to avoid information loss \nand also does not incorporate network architectures, ensuring high efficiency.\n\nWe will reorganize the discussion in the final version to make it clearer.\n\n**minor comments**\n\nA3. Thanks for your comments. We will correct it in the final version.\n\n**Q1. Why do you not report the better results of FiLM?**\n\nA4. Please refer to A1.\n\n**Q2. If FiLM overall is the better performing model, why is FilterNet still interesting? How is it simpler than FiLM?**\n\nA5. \n1. About the performance, please refer to A1.\n2. About the efficiency, please refer to A6.\n3. About the architecture, FiLM first leverages a Legendre Projection Layer to obtain a compact representation and then employs a Frequency Enhance Layer to learn the projection of representations.\n   In contrast, FilterNet uses only a Frequency Filter Block to model the temporal dependencies.\n\n**Q3. Is it maybe faster to train?**\n\nA6. Both FiLM and FilterNet conduct complex multiplication between the input and the learnable weights.\nHowever, the learnable weights in FiLM are 3-dimensional $W \\in \\mathbf{C}^{L\\times N\\times N}$ where N is the number of Legendre Polynomials,\nwhile in FilterNet they are 1-dimensional $W \\in \\mathbf{C}^{1\\times L}$.\nThis makes the calculation in FiLM more complicated than in FilterNet. \nTo further compare the efficiency of FiLM with FilterNet, we conduct efficiency experiments on the Exchange dataset, and the results are shown below:\n\n|Model | Memory Usage (G)|  Training Time (s/epoch)|\n|:--------:|:----|:----|\n|  FiLM | 0.9 |  48.2 |\n|  FilterNet | 0.5 |  1.5 |\n\nFrom the table, we can find that FilterNet has faster training speed and less memory usage than FiLM.\nThe efficiency of other models is shown in Figure 8.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OUGHLcE2G7", "reviewer": "Reviewer_UnVh", "summary": "This paper explores a novel direction that leverages frequency filters directly for time series forecasting, and proposes a simple yet effective framework, namely FilterNet, from a signal processing perspective. It designs two types of learnable frequency filters corresponding to the linear and attention mapping widely used in time series modeling, i.e., PaiFilter and TexFilter. Abundant analysis experiments verify the competitive modeling capability of frequency filters, and extensive experiments on eight real-world datasets demonstrate FilterNet achieves good performances in both accuracy and efficiency over state-of-the-art methods.", "questions": "1.\tIn ICML 2024, some works also adopt simple architectures for time series forecasting, such as SparseTSF [2]. Could you further compare the performance of FilterNet with SparseTSF?\n2.\tThe recent literature has seen papers based on foundation models for time series forecasting. Conversely, some works are designed with simple architectures, yet achieve better performance than foundation models. Why can simple models (e.g., FilterNet) also achieve competitive performance?\n[1] On the Spectral Bias of Neural Networks, in ICML 2019\n[2] SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters, in ICML 2024", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.\tThis paper is well-written overall, and the contribution and novelty of the proposed work are clearly presented.\n2.\tThis paper introduces a novel and straightforward framework based on the frequency filters for time series forecasting. It offers a new direction by incorporating more signal processing theory into time series analysis.\n3.\tThe frequency filter has a simple architecture but demonstrates strong temporal modeling capabilities for time series. Results in Tables 1, 4, and 5 show that the two types of proposed frequency filters are suitable for different datasets and settings, which provides valuable insights for future model design.\n4.\tThis paper seems to have solid and extensive experiments. The experiments are conducted on eight real-world different datasets, and abundant evaluation analysis and visualization experiments are presented.", "weaknesses": "1.\tThe visualization setting for Figure 1 appears to be missing in Appendix C.4, and I recommend adding some references about the spectral bias of neural networks, such as [1].\n2.\tSome typo errors. In line 282, the complexity of FilterNet should be O(LogL), not O(L LogL).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your positive comments and constructive suggestions. We provide a point-by-point response to your comments as follows, and we hope that we address all your concerns below:\n\n**W1. The visualization setting for Figure 1 appears to be missing in Appendix C.4, and I recommend adding some references about the spectral bias of neural networks, such as [1].**\n\nA1. Thanks for your very careful reading. For Figure 1, we generate a simple signal composed of three sine functions, each with a different frequency: low-frequency, middle-frequency, and high-frequency.\nThen, we conducted experiments using iTransformer and FilterNet on the same signal, reported the MSE results, and visualized the prediction values and ground-truth values.\nWe will add these in the final version, and add some references about the spectral bias of neural network.\n\n**W2. Some typo errors**\n\nA2. Thanks, we will correct it in the final version.\n\n**Q1. In ICML 2024, some works also adopt simple architectures for time series forecasting, such as SparseTSF [2]. Could you further compare the performance of FilterNet with SparseTSF?**\n\nA3. To further compare the performance of SparseTSF with FilterNet, we conduct experiments on Electricity and Traffic datasets with the look-back window length of 96. \nSince SparseTSF only reported the MSE results, we also report the results of MSE, and the results are shown as follows:\n\n|  Model | | TexFilter| PaiFilter| SparseTSF|\n|:--------:|:----|:----|:----|:----|\n|  Metric | | MSE|MSE|MSE|\n|  Electricity | 96| 0.147| 0.176| 0.209| \n|   | 192| 0.160 |0.185 |0.202 |\n|   | 336| 0.173 |0.202 |0.217 |\n|   | 720| 0.210 |0.242 |0.259 |\n|  Traffic | 96| 0.430| 0.506| 0.672|  \n|   | 192| 0.452| 0.508| 0.608 |\n|   | 336| 0.470| 0.518| 0.609|\n|   | 720| 0.498| 0.553| 0.650|\n\nOverall, FilterNet outperforms SparseTSF. The performance of SparseTSF can be validated from its paper (**see Table 12 in SparseTSF**).\nSparseTSF has a simple architecture, which limits its capability for modeling complex relationships.\nThis results in SparseTSF achieving worse performance on larger datasets. \nThis limitation is also validated in FilterNet's performance comparison. \nOn the larger datesets, TexFilter has better performance than PaiFilter, while on smaller datasets, PaiFilter has better than TexFilter.\n\n**Q2. The recent literature has seen papers based on foundation models for time series forecasting. Conversely, some works are designed with simple architectures, yet achieve better performance than foundation models. Why can simple models (e.g., FilterNet) also achieve competitive performance?**\n\nA4. It is a good question. As claimed in SparseTSF, the basis for accurate long-term time series forecasting\nlies in the inherent periodicity and trend of the data.\nSince the periodicity and trend information are very simple, and they can be modeled by a simple network, such as 1k parameters in SparseTSF.\nIn a recent paper [1], it discovers that LLM-based methods perform the same or worse than basic ablations, \nyet require orders of magnitude more compute. It shows that replacing language models with simple attention layers,\nbasic transformer blocks, randomly-initialized language models, and even removing the language\nmodel entirely, yields comparable or better performance. Also, in the paper [1], \nit finds that simple methods (patching with one-layer attention) can work so well.\nCombining the findings of FilterNet, we can conclude that simple models can also achieve competitive performance.\n\n[1] Are Language Models Actually Useful for Time Series Forecasting? https://arxiv.org/abs/2406.16964", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hn5nxeYrKa", "reviewer": "Reviewer_t3Wx", "summary": "This paper introduces FilterNet, a novel neural network architecture for time series forecasting that leverages learnable frequency filters inspired by signal processing techniques. This paper proposes two types of filters within FilterNet: a plain shaping filter that uses a universal frequency kernel for signal filtering and temporal modeling, and a contextual shaping filter that examines filtered frequencies for compatibility with input signals to learn dependencies. This paper demonstrates that FilterNet can effectively handle high-frequency noise and utilize the full frequency spectrum, addressing key limitations of existing Transformer-based forecasters. This paper shows through extensive experiments on eight benchmarks that FilterNet achieves superior performance in both effectiveness and efficiency compared to state-of-the-art methods in time series forecasting.", "questions": "1. Can the authors elaborate on the connection and difference of the two filters?\n2. Can the authors give more details about proposed filters?\n3. Can the authors explain the lower performance of TexFilter in some cases?", "limitations": "See weakness.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. This paper studies an interesting research direction, which is to adapt the advanced techniques in the signal processing domain into the time series forecasting, rather than following the existing transformer-based time series works. This is the original and novel contribution to the time series community.\n2. The main model part is overall descent. The simple but effective architectures built upon the frequency filters are proposed. I notice there is not many redundant components in the basic model, compared with existing transformer models. It can somehow demonstrates the effectiveness of their proposed two filters.\n3. The experiments are solid and convincing. Several experimental results are reported on existing benchmark datasets. Also, some experiments on synthetic periodic and trend signals with noises shown in experiments make sense. The efficiency analysis and visualization is intuitive.", "weaknesses": "1. The paper mainly introduces two filters. The connection and difference between the two filters is not highlighted, which makes me curious about which one is better.\n2. As far as I know, there exist many frequency filters in signal processing. The motivation for these two filters is not very clear.\n3. In experiment part, the results of TexFilter on ETTm1, ETTm2, ETTh1 and Exchange are neither the best nor the second best. Does it show the filter cannot win in many cases of forecasting? Are there any analysis or explanation about it?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your review and the positive comments regarding our paper. We would like to respond to your comments as follows, and we hope that we address all your concerns below:\n\n**W1. The paper mainly introduces two filters. The connection and difference between the two filters is not highlighted, which makes me curious about which one is better.**\n\nA1. In the literature, compared with other model architectures, MLP-based and Transformer-based methods have achieved competitive performance.\nInspired by the two paradigms, we design two corresponding types of filters. i.e., PaiFilter and TexFilter.\nPaiFilter offer stability and efficiency, making them\nsuitable for tasks with static relationships. In contrast, TexFilter provides the flexibility\nto capture dynamic dependencies, excelling in tasks that require context-sensitive analysis.\n**As shown in Table 1**, PaiFilter performs well on small datasets (e.g., ETTh1), while TexFilter excels on large datasets\n(e.g., Traffic) due to the ability to model the more complex and contextual correlations present in\nlarger datasets.\nThis dual approach allows FilterNet to effectively handle a wide range of data types and forecasting scenarios,\ncombining the best aspects of both paradigms to achieve superior performance.\n\n**In Appendix, we have analyzed the two filters, and introduced the connections (Frequency Filter Block in Appendix A) and differences between them (Explanations about the Design of Two Filters in Appendix B)**.\n\n\n**W2. As far as I know, there exist many frequency filters in signal processing. The motivation for these two filters is not very clear.**\n\nA2. Please refer to A2 in Response to Reviewer ufqs.\n\n**W3. In experiment part, the results of TexFilter on ETTm1, ETTm2, ETTh1 and Exchange are neither the best nor the second best. Does it show the filter cannot win in many cases of forecasting? Are there any analysis or explanation about it?**\n\nA3. Inspired by self-attention mechanisms that are highly data-dependent, dynamically computing attention scores based\non the input data, TexFilter is designed to learn the filter parameters generated from the input data.\nThis makes TexFilter particularly suitable for the scenarios that require context-sensitive analysis and an understanding of sequential or structured data.\nAccording to Table 1, TexFilter has shown competitive performance in large datasets (e.g., Traffic and Electricity)\nbecause larger datasets require more contextual structures\ndue to their complex relationships.\nHowever, for the smaller datasets (e.g., ETT and Exchange), since the relationships are relatively simpler, \nusing a complex design carries a risk of overfitting. \nFor example, in **Figure 10**, the performance with channel-shared filters achieved better results than channel-unique filters, \nwhich indicates that a simpler architecture is more suitable for smaller dataset.\nIn such cases, TexFilter might become overly specialized to the limited data, potentially reducing its generalizability.\n\nWe have analyzed TexFilter in **Appendix B** and have also given explanations about the results of Table 1 in **lines 219-224**.\n\n**Q1. Can the authors elaborate on the connection and difference of the two filters?**\n\nA4. Please refer to A1.\n\n**Q2. Can the authors give more details about proposed filters?**\n\nA5.Both PaiFilter and TexFilter belong to frequency filters, and the difference is the filter function $\\mathcal{H}\\_{filter}$.\nFor PaiFilter, $\\mathcal{H}\\_{filter}$ is equal to $\\mathcal{H}\\_{\\phi}$ where $\\mathcal{H}_{\\phi}$ is a random initialized learnable weight.\nFor TexFilter, $\\mathcal{H}\\_{filter}$ is equal to $\\mathcal{H}\\_{\\varphi}(\\mathcal{F}(\\mathbf{Z}))$ where $\\mathcal{H}\\_{\\varphi}()$ is a data-dependent frequency filter learned from the input data.\nWe have detailed these in **Equations (1), (4), (5), and (6)**.\n\n**Q3. Can the authors explain the lower performance of TexFilter in some cases?**\n\nA6. Please refer to A3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel architecture called FilterNet for time series forecasting, which aims to balance efficiency and effectiveness by introducing frequency filters.
The contributions were generally well-received, with many reviewers acknowledging its technical soundness. 
However, some reviewers also raised concerns that the technical contributions might be slightly below the standards expected at NeurIPS. 
The authors' rebuttal effectively addressed many of the reviewers' concerns. 
Nevertheless, when considering the technical standards and the relative evaluation of other submissions, the paper's priority is lower, which recommends the final decision to reject the submission.
Decision: Accept (poster)