Title: Frequency Adaptive Normalization For Non-stationary Time Series Forecasting
Venue: NeurIPS 2024
Accepted: yes
Keywords: Timeseries Normalization

Non-stationary Timeseries Forecasting

Timeseries Forecasting
Abstract: Time series forecasting typically needs to address non-stationary data with evolving trend and seasonal patterns. To address the non-stationarity, reversible instance normalization has been recently proposed to alleviate impacts from the trend with certain statistical measures, e.g., mean and variance. Although they  demonstrate improved predictive accuracy, they are limited to expressing basic trends and are incapable of handling seasonal patterns. To address this limitation, this paper proposes a new instance normalization solution, called frequency adaptive normalization (FAN), which extends instance normalization in handling both dynamic trend and seasonal patterns. Specifically, we employ the Fourier transform to identify instance-wise predominant frequent components that cover most non-stationary factors. 
Furthermore,  the discrepancy of those frequency components between inputs and outputs is explicitly modeled as a prediction task with a simple MLP model. FAN is a model-agnostic method that can be applied to arbitrary predictive backbones.  We instantiate FAN on four widely used forecasting models as the backbone and evaluate their prediction performance improvements on eight benchmark datasets. FAN demonstrates significant performance advancement, achieving 7.76\%$\sim$37.90\%  average  improvements in MSE. Our code is publicly available at http://github.com/icannotnamemyself/FAN.

Frequency Adaptive Normalization For Non-stationary Time Series Forecasting

Abstract

Time series forecasting typically needs to address non-stationary data with evolving trend and seasonal patterns. To address the non-stationarity, reversible instance normalization has been recently proposed to alleviate impacts from the trend with certain statistical measures, e.g., mean and variance. Although they demonstrate improved predictive accuracy, they are limited to expressing basic trends and are incapable of handling seasonal patterns. To address this limitation, this paper proposes a new instance normalization solution, called frequency adaptive normalization (FAN), which extends instance normalization in handling both dynamic trend and seasonal patterns. Specifically, we employ the Fourier transform to identify instance-wise predominant frequent components that cover most non-stationary factors. Furthermore, the discrepancy of those frequency components between inputs and outputs is explicitly modeled as a prediction task with a simple MLP model. FAN is a model-agnostic method that can be applied to arbitrary predictive backbones. We instantiate FAN on four widely used forecasting models as the backbone and evaluate their prediction performance improvements on eight benchmark datasets. FAN demonstrates significant performance advancement, achieving 7.76% ∼ 37.90% average improvements in MSE. Our code is publicly available 2 .

1 Introduction

Time series forecasting plays a key role in various fields such as traffic [8], finance [23] and infectious disease [1], etc. Recent research focuses on deep learning-based methods, as they demonstrate promising capabilities to capture complex dependencies between variables [47, 42, 46]. However, time series with trends and seasonality, also called non-stationary time series [15], create covariate pattern shifts across different time steps. These dynamics pose significant challenges in forecasting.

To mitigate non-stationarity issues, reversible normalization has been proposed [33, 17] which first removes non-stationary information from the input and returns the information back to rebuild the output. Current work focuses on removing non-stationary signals with statistical measures, e.g., mean and variance in the time domain [10, 28]. However, while these methods have improved prediction accuracy, these statistical measures are only capable of extracting the most prominent component, i.e., the trend, leaving substantial room for improvement. They, we argued, can hardly measure the characteristics of seasonal patterns, which are quite common in many time series. This significantly limits their capability to handle the non-stationarity, especially the seasonal patterns.

We illustrate an example featuring one of the simplest non-stationary signals in Fig. 1. This graph shows a time-variant signal with a gradually damping frequency, which is widely seen in many passive systems, e.g., spring-mass damper systems. As depicted in Fig. 1, the three input stages

̸

Consistent Mean And Variance

𝝁𝟏

𝝈𝟏

=

𝝁𝟐

𝝈𝟐

=

𝝁𝟑

𝝈𝟑

𝒇𝟏

≠ 𝒇𝟐

𝒇𝟏

𝒇𝟐

𝒇𝟑

Non-stationary Time series

Non-stationary

Information

Non-stationarity

(highlighted in different background colors) exhibit the same mean and variance but differ in Fourier frequencies. Previous methods that model non-stationary information using means and variances can hardly distinguish this type of change in the time domain. In comparison, changes in periodic signals can be easily identified with the instance-wise Fourier transform ( f 1 = f 2 = f 3 ). Thus, in this context, the principal Fourier components provide a more effective representation of non-stationarity com-

̸

pared to statistical values such as mean and variance. This simple example also shows that many existing frequency-based solutions, e.g., TimesNet [41], Koopa [27], which assume that the principal frequencies of the input signal are constant, can not identify the evolving principal frequencies.

With this inspiration, we introduce a novel instance-based normalization method, named F requency A daptive N ormalization (FAN). Rather than normalizing temporal statistical measures, FAN mitigates the impacts from the non-stationarity by filtering top K dominant components in the Fourier domain for each input instance, this approach can handle unified non-stationary fact composed of both trend and seasonal patterns. Furthermore, as those removed patterns might evolve from inputs to outputs, we employ a pattern adaptation module to forecast future non-stationary information rather than assuming these patterns remain unchanged.

In summary, our main contributions are: 1) We illustrate the limitations of reversible instance normalization methods in using temporal distribution statistics to remove impacts from non-stationarity. To address this limitation, we introduce a novel reversible normalization method, named FAN, which adeptly addresses both trend and seasonal non-stationary patterns within time series data. 2) We explicitly address pattern evolvement with a simple MLP that predicts the top K frequency signals of the horizon series and applies these predictions to reconstruct the output. 3) We apply FAN to four general backbones for time series forecasting across eight real-world popular benchmarks. The results demonstrate that FAN significantly improves their predictive effectiveness. Furthermore, a comparative analysis between FAN and state-of-the-art normalization techniques underscores the superiority of our proposed solution.

2 Related Work

Time series forecasting has been a hot topic of study for many years. This section provides discussions on related work from the following three perspectives.

2.1 Time Series Forecasting

Traditional statistical methods, such as ARIMA [2], assume the stationarity of the time series and dependencies between temporal steps. Although these methods provide theoretical guarantees, they typically require data with ideal properties, which is often inconsistent with real-world scenarios [42]. Besides, they can only handle a limited amount of data and features. In recent years, the field has witnessed a significant proliferation in the application of deep learning techniques for multivariate time series forecasting, a development ascribed to their ability in handling high-dimensional datasets. Consequently, various methods have been proposed to model time series data. Work based on recurrent neural networks [36, 4] preserves the current state and models the evolution of time series as a recurrent process. However, they generally suffer from a limited receptive field, which restricts their ability to capture long temporal dependencies [47]. Inspired by their successes in Computer Vision (CV) and Natural Language Processing (NLP), convolutional neural networks and the self-attention mechanism have been extensively utilized in time series forecasting [22, 19, 40, 25]. Nevertheless, those works still face difficulties in handling non-stationary data with covariate pattern shifts. Making an accurate prediction for non-stationary time series remains challenging.

2.2 Non-stationary Time Series Forecasting

To address non-stationarity, many methods directly model non-stationary phenomena with different modeling techniques. Li et al. [24] utilize a domain-adaptation paradigm to predict data distributions. Du et al. [6] propose an adaptive RNN to alleviate the impact of non-stationary factors through distribution matching. Liu et al. [26] introduce a non-stationary Transformer with de-stationary attention that incorporates non-stationary factors in self-attention mechanisms. To model non-linear dynamic systems, several models based on Koopman theory [21, 29, 37, 44] have been proposed with Fourier transform. To learn different patterns at different scales, Wang et al. [38] employs global and local Koopman operators. Liu et al. [27] model non-stationarity identified with Fourier transform and use Koopman operators to learn those components. However, these solutions typically select fixed frequency components based on the whole sequence rather than frequencies based on inputs. This time-invariant assumption can hardly be true in real-world scenarios.

2.3 Instance-wise Normalization against Non-stationarity

To mitigate the time-variant property of non-stationary time series, a set of instance-wise normalization methods have been proposed to remove the impacts from non-stationarity. To reflect instance-wise changes, Ogasawara et al. [31] propose the usage of normalization based on local properties rather than global statistics. Passalis et al. [33] introduce an adaptive and learnable approach to this instancewise normalization paradigm. However, although these methods effectively remove non-stationary components from inputs, they still need to predict the non-stationary time series in the output series, which remains challenging. In response, reversible instance normalization [17] is introduced by reintegrating the removed non-stationary components back to reconstruct the output. However, it still assumes unchanged trends between inputs and outputs. Kim et al. [17] developed RevIN, which mainly addresses evolving trends between input sequences. Recent works [10, 28] explore trends at a finer granularity, e.g., at the sliced level. However, these approaches still model non-stationarity with temporal statistical distribution parameters and fail to account for evolving seasonality, which is a crucial aspect of non-stationarity [35, 11, 45].

3 Proposed Method: FAN

Given a multivariate time series X ∈ R N × D , where N is the total time steps and D denotes the number of feature dimensions. We use inputs series with length L to predict outputs series within length H . The forecast task can be formulated as: X t -L : t →X t +1: t + H , where X t -L : t ∈ R L × D and X t +1: t + H ∈ R H × D . For a clearer notation, we denote the input and output series as X t ∈ R L × D and Y t ∈ R H × D respectively.

Our proposed method, FAN, consists of symmetrically structured instance-wise normalization and denormalization layers, illustrated in Fig. 2. The normalization process removes the impacts of non-stationary signals through frequency domain decomposition (upper left part of Fig. 2), while the denormalization process, supported by a prediction module, addresses potential shifts in frequency components between the input and output (lower part of Fig. 2).

3.1 Frequency-based Normalization

First, FAN removes the top K dominant components in the frequency domain for each input instance, so the forecasting backbone can concentrate on the stationary aspects of the input. We term this process as Frequency Residual Learning (FRL). The input at time t , X t , is multivariate with D dimension, and each dimension might have different frequency patterns; thus, we apply the FRL to each dimension in a channel independence setting [30]. Here, the FRL is realized by the 1-dim Discrete Fourier Transform (DFT) with DFT( · ) towards each input X t :

Equ. 1 shows that DFT( · ) transforms an input into Fourier components in complex values, denoted Z t ∈ C T × D . Then, TopK( · ) selects the frequency set with the top K largest amplitude, which are calculated with Amp( · ) function. Filter is the operation to filter out the K t frequency from Z t . To mitigate the impact of non-stationary signals, FRL restores the top K components into time domain components X non t with IDFT( · ) .

+

Frequency Residual Learning

Magnitude

Frequency

Trend & Seasonal Non-stationarity Patterns

Stationary forecast

(backbone)

Non-stationary forecast

Normalization

Denormalization

Filter

IDFT

Magnitude

Frequency

𝑔𝜃

DFT

DFT

𝐗𝐭

𝐗𝐭

𝒓𝒆𝒔

𝐗𝒕

𝒏𝒐𝒏

෡

𝐘𝒕

𝑛𝑜𝑛

෡

𝐘𝐭

𝒓𝒆𝒔

෡

𝐘𝐭

ℒ 𝜃,𝜙

𝑓𝑜𝑟𝑒𝑐𝑎𝑠𝑡

𝐘𝐭

෡

𝐘𝐭

ℒ 𝜙

𝑛𝑜𝑛𝑠𝑡𝑎𝑡

𝐘𝐭

𝑛𝑜𝑛 ෡

𝐘𝐭

𝑛𝑜𝑛

𝐘𝐭

Loss Functions

Filter

IDFT

With X non t , we can easily normalize the inputs and get the stationary components by removing X non t from X t , which is

Here, DFT( · ) and IDFT( · ) can be performed using Fast Fourier Transform (FFT) [3] with a computational complexity of O ( L log L ) . And the TopK and Filter operations both exhibit complexity of O ( L + K ) . It is important to note that all these operations are GPU-friendly and can be fully paralleled. Thus, the impact of applying these operations independently on each dimension can be largely mitigated. GPU-friendly PyTorch pseudocode is in Appendix A.2. After the normalization step, the normalized sequences X res t can be more stationary and have a more consistent covariate distribution, the theoretical proof is provided in Appendix C.

3.2 Forecast & Denormalization

As a result, the normalization layer allows the forecast backbone model g θ to focus more on the dynamics within the inputs. Here, following the reversible instance normalization paradigm, the forecast backbone g θ receives the transformed data X res t as input and forecasts only the stationary part Y res t of the outputs Y t . This design makes it easier for the model to forecast non-stationary time series. Then, we apply the removed non-stationary information back to the output. We define this process as:

where ˆ Y non t is the reconstruct signal of X non t . We illustrate ˆ Y non as follow:

Non-stationary shift forecasting. For reverse instance normalization, we need to estimate ˆ Y non t in the outputs. As an input and its corresponding output are rather close, RevIN [17] directly adds X non t back by assuming Y non t with exactly the same trend as X non t . However, this assumption can hardly be true as the non-stationary information between the input and output may evolve. Furthermore, although SAN [28] and Dish-TS [10] predict statistics to address the discrepancy between the input and output, these statistics can only represent the most salient trend patterns.

To this end, rather than predicting statistics [10, 28], we use a simple MLP model q ϕ to directly predict future values of the composite top K frequency components for D dimensions, defined as:

where W 1 , W 2 , W 3 are all learnable parameters. Here, since X non t only contains top K frequency information, it is difficult to capture variations in other frequencies solely relying on X non t . Therefore,

we concatenate the top K components with the original input X t to handle potential frequency variations.

Loss Functions. To help with the residual learning process, we incorporate a prior guidance loss for the prediction of principal frequency components, the final loss is defined in Eq. 5. The forecast with prior guidance can be considered a multi-task optimization problem [12], where L nonstat ϕ ensures q ϕ accurately predict the non-stationary principal frequency component and L forecast θ,ϕ guarantees that both model optimizes along the overall forecast accuracy.

Here, the mean square error loss is used for both loss functions.

4 Experiments

4.1 Experiment Setup

Datasets. We use eight popular datasets in multivariate time series forecasting as benchmarks, including: (1-4) ETT (Electricity Transformer Temperature) 3 [47] records the oil temperature and load of the electricity transformers from July 2016 to July 2018. Four subsets are included in this dataset, where ETThs are sampled per hour and ETTms per 15 minutes. (5) Electricity 4 contains the electricity consumption of 321 clients from July 2016 to July 2019 per 15 minutes. (6) ExchangeRate 5 contains the daily exchange rates of 8 countries from 1990 to 2016. (7) Traffic 6 includes the hourly traffic load of San Francisco freeways recorded by 862 sensors from 2015 to 2016. (8) Weather 7 is made up of 21 indicators of weather, including air temperature and humidity collected every 10 minutes in 2021.

For preprocessing, we apply z-score normalization [12] on all datasets to scale different variables to the same scale. Note that z-score normalization is unable to handle non-stationary time series since the statistics remain unchanged for different input instances [17]. The split ratio for training, validation, and test sets is set to 7:2:1 for all the datasets. We report datasets properties in Table 1, including (1) Trend Variation: Differences in the means across different sections of the dataset. (2) Seasonality Variation: We report the average variance over the Fourier spectrum to examine the presence of evolving seasonality. Other dataset details can be found at Appendix B.

Evaluation. We set the prediction length H ∈ { 96 , 168 , 336 , 720 } , covering both short- and longterm rediction [30]. A fixed input window length L = 96 is used for all datasets. We evaluate the performance of baselines using mean squared error (MSE) and mean absolute error (MAE). The MSE and MAE are computed on z-score normalized data to measure different variables on the same scale. We report the final results on the test set for the model that performed optimally on the validation set.

Backbone Models. FAN is model-agnostic and could be applied to any prediction backbones. To validate its effectiveness, four state-of-the-art time-series forecasting model are used: MLPbased DLinear [46], Transformer-based Informer [47] and FEDformer[48], and convolutional neural network-based SCINet [25]. Notably, FEDformer also employs the Fourier transform for analyzing

seasonality. Results later show that FAN continues to make considerable improvements over these frequency-based solutions like FEDformer.

Implementation and Settings. For the non-stationary prediction module q ϕ in FAN, the MLP model has hidden sizes [64, 128, 128]. All the experiments are implemented by PyTorch [34] and are conducted for five runs with fixed seeds { 1 , 2 , 3 , 4 , 5 } on NVIDIA RTX 4090 GPU (24GB). For the different baselines, we follow the implementation and settings provided in their official code repository. ADAM [18] as the default optimizer across all the experiments. More experiment details, including training details and hyperparameter, can be found in Appendix A.1.

Selections of Hyperparameter K . FAN allows for K to be any integer number less than L . Regarding the selection of K , we analyze these benchmarks and found that the main variation frequencies of these datasets are within 10% of the maximum amplitude. Therefore, we select the value of K based on the average maximum amplitude within 10% in the training set, the selected K is shown in Table 1. More evidence of this selection strategy is at Sec. 4.4, and we provide a detailed hyperparameter sensitivity analysis at Appendix D.1.

4.2 Main Results

We report MAE/MSE forecasting errors of the baselines and FAN in Table 2. Since the performance in ETT datasets shows similar results, only results of ETTm2 are reported. The full results of the ETT benchmarks and further discussion are in Appendix E.2.

As shown in Table 2, our proposed FAN effectively enhances the performance of all four backbone models, by a large margin, achieving state-of-the-art performance on five datasets. Specifically, on the ETTm2, Electricity, Exchange, Traffic, and Weather datasets, the average MSE performance improvements are rather significant: 10.81%, 21.49%, 51.27%, 21.97%, and 21.55% respectively. It clearly shows that frequency residual learning of FAN effectively mitigates the impacts of evolving seasonal and trend patterns and enhances the stationarity that simplifies the prediction for backbones.

FAN demonstrates increasing performance improvements as the prediction length extends in the Informer backbone, with MSE improvements of 9.87%, 18.87%, 36.91%, 16.26%, and 20.05%, from 96 steps to 720 steps. We believe this can be attributed to the fact that the periodicity contained in the prediction series increases with step length, and the FAN's pattern prediction module helps uncover periodicity in longer step lengths, thereby enhancing long-term prediction effectiveness. It is important to note that even in the models that utilize FFT to analyze seasonal patterns, like FEDformer, we still observe significant performance improvements (19.81%). This conclusion underscores our model's advantage in handling non-stationary aspects by directly extracting non-stationary seasonality patterns rather than learning these patterns.

4.3 Comparison With Reversible Instance Normalization Methods

In this section, we compare FAN with three state-of-the-art normalization methods for non-stationary time series forecasting: SAN [28], Dish-TS [10], and RevIN [17], with the same experimental setup as Sect. 4.2. We report the average MSE over all the forecasting lengths of all backbones for all datasets in Table 3. It is evident that FAN generally outperforms the baseline models, except for a few cases with a close margin. Here, SAN generally ranks second as it slices the whole sequence into sub-series which can make seasonal patterns into evolving trends that could be partially predicted with its statistics prediction module. In comparison, RevIN and Dish-TS have much worse performance. Detailed results of all cases and further discussions are provided in Appendix E.4.

Show Cases. Fig. 3 illustrates the forecasting results with DLinear backbone in Traffic to show why FAN has performance advantages. This data has very clear evolving seasonality with daily waveform patterns. FAN can extract trends and seasonal patterns especially the seasonal patterns during weekends while Dish-TS and RevIN only focus on trends statistics. Furthermore, FAN can adaptively adjust frequency pattern forecasting results based on the input main frequency signals, capturing the evolving patterns between the input and horizon. Fig. 3(a) clears shows FAN can identify the seasonal patterns with increasing amplitudes from hour 100 ∼ 150.

(a) FAN

(d) RevIN

(c) Dish-TS

(b) SAN

Stationarity Analysis. To verify our model's effectiveness against non-stationarity, we use the ADF test [39] to examine the stationarity of the data after normalization. The results are shown in Fig. 4(a), smaller value (further from the center) indicates higher stationarity. Compared to previous normalization methods, our model achieves greater stationarity across all datasets, particularly in cases with larger seasonal patterns (Traffic, ETTh1, ETTm1). In some datasets, e.g., Weather, Exchange, despite having less apparent seasonality, our model still enhances stationarity. We attribute this to our method's ability to adaptively capture the low-frequency trend signals, such as mixed-linear changes, while other methods assume consistent distribution over a period and remove estimated statistics, due to which they might fail to capture these intricate trend patterns.

Model Efficiency. Wecompare the performance, training time per iteration, and number of parameters with SAN on Traffic with D = 862 , H = 720 . DLinear is used for both as the backbone. The results are shown in Fig. 4(b). FAN and SAN have similar training iteration times, but FAN has 29 . 79% less parameters. Moreover, FAN achieves a 15 . 56% improvement in MSE and a 15 . 30% improvement in MAE. This highlights our model's effectiveness and efficiency.

Various Input Length. In time series learning, the non-stationarity of inputs varies with the choice and change of the time window [35], which in turn impacts the performance of deep learning models [27]. Therefore, we compare the performance changes under different input lengths on

-12.77

-11.78

-11.07

-7.40

-8.49

Electricity

ETTm2

ETTm1

ETTh2

ETTh1

RevIN

Dish-TS

SAN

FAN

Exchange

Traffic

Weather

-8.79

-7.67

0

-8.92

(a) Stationarity Analysis

(b) Model Efficiency

ETTm2 dataset and DLinear as the backbone. Fig. 4(c) shows FAN exhibits the best performance across all windowed data. As can be seen, compared to other models, as the input length increases, among these normalizations, the enhancement of increases the most. The MSE enhancement with previous SOTAs increases from 0.49% in short inputs L = 48 to 4.37% in long inputs with L = 336 . This demonstrates that the instance-wise DFT is capable of extracting more seasonal patterns from the longer input windows.

4.4 TopK vs. Frequency Distributions

As different datasets might have different non-stationary patterns, it is crucial to select appropriate K frequency components from inputs. We study the relations between the selected TopK and the frequency distribution on the ExchangeRate and Traffic datasets.

Fig. 5 plots the frequency amplitude distribution for frequency 0 ∼ 32 by performing DFT towards the different input instances with L = 96 of the whole training sequences. Here, we can see the clear relation between the selection of K and the frequency amplitude distributions. As we can see, the Traffic dataset contains rich seasonal signals ranging from 0 ∼ 32 while the ExchangeRate dataset only has mainly one principal frequency component with frequency 0 (trend) in the inputs. Thus, the prediction on the ExchangeRate dataset might not benefit from a bigger K while a bigger K indeed helps for the Traffic dataset. Results for more datasets are in Appendix B.

4.5 Ablation Studies

Main Components. This section aims to evaluate the effectiveness of various FAN's designs. Three variants are studied: 'w/o predict' denotes the removal of the non-stationary pattern prediction module and directly reconstructing ˆ Y non with X non . 'pure backbone' refers to the omission of the reconstruction in the output or 'w/o backbone' is the omission of the stationary part. We evaluate their performance on two non-stationary datasets, ETTh1 and Weather. The experimental settings are consistent with those described in Section 4.2. The evaluation results, along with the standard deviations, are presented in Table 4. The results show that FAN achieves best performance across all metrics in all variants. FAN w/o backbone ranks second as the learning model of FAN already learns the principle changes. In comparison, the results from pure backbone is the weakest, as it cannot handle nonstationary signals. FAN w/o predict also has poor performance. Those results clearly

show that trends and seasonal patterns do evolve and that our proposed residual frequency learning is crucial in dealing with these changes.

Instance-wise vs. Global Fourier Analysis. This section investigates the effect from instance-wise Fourier Analysis of FAN. Previous Fourier-based methods select predominant Fourier signals based on fixed frequencies [41, 27, 43, 9]. However, as shown in Fig. 6, on the Traffic and Electricity datasets, the predominant components from the input-wise view are not fixed but exhibit distinct distribution characteristics and vary across the inputs. The assumption of fixed spectrum and the reality of changing frequency limits their performance, supported with two additional experiments on Fourier-based backbones at Appendix E.3.

Here, we compare the performance of FAN with fixed frequencies computed using global sequences and original FAN with instance-specific frequencies.The results are shown in Table 5.

As shown in Table 5, by selecting instance-wise predominant frequencies, FAN achieves an average improvement of 18.50% and 10.29% on the Electricity and Traffic datasets respectively. This highlights instance-wise frequency selection rather than assuming fixed frequency patterns.

5 Conclusion

In this paper, we study the problem of non-stationary time series prediction. We identify the fact that traditional statistical measurement-based instance-wise normalization can not effectively recover the evolving seasonal patterns. We propose FAN to perform instance normalization for each input window. The Fourier transform is used to remove the main frequency components in the inputs and reconstruct the Fourier basis through denormalization. To address the evolving trend and seasonal patterns between inputs and outputs, we utilize a simple MLP model to predict the changes in the extracted non-stationary pattern. The effectiveness of FAN is verified with a set of experiments on eight widely used benchmark datasets. Compared to other state-of-the-art normalization baselines, FANsignificantly improves the prediction performance and outperforms state-of-the-art normalization methods. One potential avenue for improvement involves the autonomous determination of an optimal K for the selection of principal frequency components.

Acknowledgments and Disclosure of Funding

This work is supported in part by the National Natural Science Foundation of China (Grant No. 6247075381). We would also like to thank the anonymous reviewers for their constructive feedback and suggestions.

References

Andrea L Bertozzi, Elisa Franco, George Mohler, Martin B Short, and Daniel Sledge. The challenges of modeling and forecasting the spread of covid-19. Proceedings of the National Academy of Sciences , 117(29):16732-16738, 2020.

George EP Box and Gwilym M Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C (Applied Statistics) , 17(2):91-109, 1968.

E Oran Brigham. The fast Fourier transform and its applications . Prentice-Hall, Inc., 1988.

Rahul Dey and Fathi M Salem. Gate-variants of gated recurrent unit (gru) neural networks. In 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS) , pages 1597-1600. IEEE, 2017.

MMDodson and AM Silva. Fourier analysis and the sampling theorem. In Proceedings of the Royal Irish Academy. Section A: Mathematical and Physical Sciences , pages 81-108. JSTOR, 1985.

Yuntao Du, Jindong Wang, Wenjie Feng, Sinno Pan, Tao Qin, Renjun Xu, and Chongjun Wang. Adarnn: Adaptive learning and forecasting of time series. In Proceedings of the 30th ACM international conference on information & knowledge management , pages 402-411, 2021.

Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 459-469, 2023.

Alireza Ermagun and David Levinson. Spatiotemporal traffic forecasting: review and proposed directions. Transport Reviews , 38(6):786-814, 2018.

Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie-Yan Liu. Depts: Deep expansion learning for periodic time series forecasting. arXiv preprint arXiv:2203.07681 , 2022.

Wei Fan, Pengyang Wang, Dongkun Wang, Dongjie Wang, Yuanchun Zhou, and Yanjie Fu. Dish-ts: a general paradigm for alleviating distribution shift in time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 7522-7529, 2023.

Wei Fan, Kun Yi, Hangting Ye, Zhiyuan Ning, Qi Zhang, and Ning An. Deep frequency derivative learning for non-stationary time series forecasting. arXiv preprint arXiv:2407.00502 , 2024.

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning . MIT press, 2016.

Geoffrey Grimmett and David Stirzaker. Probability and random processes . Oxford university press, 2020.

Huan He, Owen Queen, Teddy Koker, Consuelo Cuevas, Theodoros Tsiligkaridis, and Marinka Zitnik. Domain adaptation for time series under feature and label shifts. In International Conference on Machine Learning , pages 12746-12774. PMLR, 2023.

Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice . OTexts, 2018.

Lars Kegel, Martin Hahmann, and Wolfgang Lehner. Feature-based comparison and generation of time series. In Proceedings of the 30th international conference on scientific and statistical database management , pages 1-12, 2018.

Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations , 2021.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451 , 2020.

Lambert H Koopmans. The spectral analysis of time series . Elsevier, 1995.

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval , pages 95-104, 2018.

Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolutional networks for action segmentation and detection. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 156-165, 2017.

Audeliano Wolian Li and Guilherme Sousa Bastos. Stock market forecasting using deep learning and technical analysis: a systematic review. IEEE access , 8:185232-185242, 2020.

Wendi Li, Xiao Yang, Weiqing Liu, Yingce Xia, and Jiang Bian. Ddg-da: Data distribution generation for predictable concept drift adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 4092-4100, 2022.

Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems , 35:5816-5828, 2022.

Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems , 35:9881-9893, 2022.

Yong Liu, Chenyu Li, Jianmin Wang, and Mingsheng Long. Koopa: Learning non-stationary time series dynamics with koopman predictors. Advances in Neural Information Processing Systems , 36, 2024.

Zhiding Liu, Mingyue Cheng, Zhi Li, Zhenya Huang, Qi Liu, Yanhu Xie, and Enhong Chen. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. Advances in Neural Information Processing Systems , 36, 2024.

Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications , 9(1):4950, 2018.

Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730 , 2022.

Eduardo Ogasawara, Leonardo C Martinez, Daniel De Oliveira, Geraldo Zimbr˜ ao, Gisele L Pappa, and Marta Mattoso. Adaptive normalization: A novel data normalization approach for non-stationary time series. In The 2010 International Joint Conference on Neural Networks (IJCNN) , pages 1-8. IEEE, 2010.

Alan V Oppenheim. Applications of digital signal processing. Englewood Cliffs , 1978.

Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, and Alexandros Iosifidis. Deep adaptive input normalization for time series forecasting. IEEE transactions on neural networks and learning systems , 31(9):3760-3765, 2019.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.

Maurice Bertram Priestley. Spectral analysis and time series , volume 890. Academic press London, 1981.

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems , 27, 2014.

Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. Learning koopman invariant subspaces for dynamic mode decomposition. Advances in neural information processing systems , 30, 2017.

Rui Wang, Yihe Dong, Sercan ¨ O Arik, and Rose Yu. Koopman neural forecaster for time series with temporal distribution shifts. arXiv preprint arXiv:2210.03675 , 2022.

Annette Witt, J¨ urgen Kurths, and A Pikovsky. Testing stationarity in time series. physical Review E , 58(2):1800, 1998.

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems , 34:22419-22430, 2021.

Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The eleventh international conference on learning representations , 2022.

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining , pages 753-763, 2020.

Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10 k parameters. arXiv preprint arXiv:2307.03756 , 2023.

Enoch Yeung, Soumya Kundu, and Nathan Hodas. Learning deep neural network representations for koopman operators of nonlinear dynamical systems. In 2019 American Control Conference (ACC) , pages 4832-4839. IEEE, 2019.

Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mlps are more effective learners in time series forecasting. Advances in Neural Information Processing Systems , 36, 2024.

Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence , volume 37, pages 11121-11128, 2023.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence , volume 35, pages 11106-11115, 2021.

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning , pages 27268-27286. PMLR, 2022.

A Reproducibility

A.1 Experiment Details

We make our code publicly available 2, including the backbones and baselines; the backbones and baselines code are based on their official public GitHub repositories and we use the default parameter settings or the optimal parameter settings in their paper. We used a batch size of 32, a learning rate of 0.0003, and trained each run for 100 epochs, with an early stopper set to patience as 5. For the experimental results, K is set as the number of frequencies greater than 10% of the maximum amplitude. For a fair comparison, other normalization methods were also tuned accordingly to ensure optimal results, only the normalization hyperparameters were tuned, and no other experiment parameters are tuned during the experiment phase.

A.2 Pseudocode of GPU-Friendly Normalization

B Dataset Details

B.1 Fourier Amplitude Distribution

Frequency amplitude variation and composition are closely related to the non-stationary pattern shift [20]. To analyse its effect, we use L = 96 to plot the frequency amplitude distribution of all input windows used in our eight benchmarks in Fig. 7.

In Fig. 7, it can be clearly observed that in many datasets such as ETTh1, ETTh2, ETTm1, Traffic, and Electricity datasets, besides the low-frequency trend patterns, the high-frequency parts also exhibit significant variations, especially in ETTh1, ETTm1, and Traffic datasets. This may also be the reason for our significant improvements on these datasets (with maximum improvements of 19.90%, 7.02%, and 18.65% respectively). However, in the ETTh2, ExchangeRate, and Weather datasets, which have relatively low seasonal variation, our model's improvement compared to state-of-the-art methods is relatively smaller. This is naturally because these datasets do not contain much seasonal non-stationary information for further improvement.

B.2 Main Frequency Density

FAN select top K amplitude signals, compared to previous methods based on Fourier transform, we do not use a fixed frequency set [41, 27] or randomly select [48] the frequencies. This is aligned with our observations in real data: the principal frequency signals may have a distinct distribution,

ETTh1

ETTh2

001

150

75

100

50

25

50

口

89 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Frequency

Frequency

ETTm1

ETTm2

150

75

00

25

89 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Frequency

Frequency

ExchangeRate

Traffic

150

001

50

20

旺

口

01 234 56 7 8910 11 12131415 16 17 18 1920 2122232425262728 29

Frequency

Frequency

Weather

Electricity

200

150

75

100

50

25

白

7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

89 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Frequency

Frequency

rather than being composed solely of fixed or pure random frequency signals. We plot the probability of input frequencies being selected into the top 10 signals in the input signal, as shown in Fig. 8. Although the low-frequency trend signals dominate in amplitude, many high-frequency signals still play a dominant role in some inputs, highlighting the importance of considering the entire spectrum, not just the low/high or random selected frequencies. Furthermore, this analysis shows that there may be significant differences of the main frequency components between different inputs.

However, previous methods based on the Fourier transform assumed that the main frequency signal is constant across inputs [41, 27]. In contrast, our model can dynamically extract Fourier-based signals from the inputs which enables better extraction of seasonal information, especially when the input patterns vary a lot.

ETTh1

ETTh2

ETTml

ETTm2

1.0

1.0

1.0

1.0

0.8

0.8

0.8

0.8

Top 10 Probability

d ot do

ed n do

rqed 0t do

0.6

0.6

0.6

0.6

0.4

0.4

0

0.4

0.2

0.2

0.2

0.2

0.0

0

10

20

30

40

0.0

0

10

20

30

40

0°0

0

10

20

30

40

0.0

0

10

20

30

40

Frequency

Frequeney

Frequency

Frequency

Electricity

ExchangeRate

Weather

Traffic

1.0

1.0

1.0

1.0

80

0.8

80

0.8

d o do

Aqed 0t do

sed 0 do

0.6

0.6

0.6

0.6

0.4

0.4

0

0.4

0.2

0.2

0.2

0.2

0.0

0

10

20

00

40

0.0

0

10

20

30

40

0°0

0

10

20

30

40

0.0

0

10

20

30

40

Frequency

Frequeney

Frequency

Frequency

B.3 Variation of Main/Residual Components

We examine the relative variations of the normalized main and residual components in Fig. 9. The quantitative results are obtained by calculating the relative amplitude variations of the Fourier components between the input and output in the frequency domain. In particular, across all benchmarks, the variations in the main frequency components are smaller than those in the residuals. We believe that this is the reason why a simple MLP is effective enough to capture the main frequency variations, as its shift is relatively small, as shown in the Appendix D.2.

B.4 Trend/Seasonal Variation

We explain more details of how the trend and seasonal variation in Table 1 are calculated.

Trend Variation To capture global trend shifts, we calculate the mean values over different regions of the dataset. Specifically, given a timeseries dataset X ∈ R N × D , we first chronologically split it into X train , X val , and X test , representing the training, validation, and testing datasets, respectively. The trend variations are then computed as follows:

where the subscripts indicate the dimension of mean, | · | denotes the absolute value operation, and X val , test represents the concatenation of the validation and test sets. Note that, to obtain relative results across different datasets, the trend variation is normalized by dividing by the mean of the training dataset. We obtain the first dimension to be the value inthe main content Table 1.

Seasonal variations. We evaluate seasonal changes by analyzing the variations in Fourier frequencies across all input instances. Given the inputs, X ∈ R N i × L × D where N i is the number of inputs. We first obtain the FFT results of all inputs, denoted as Z ∈ C N i × L × D . Then, we calculate the variance across different inputs and normalize this variance by dividing by the mean of each input as:

where the subscripts indicate the dimension of the operation. We sum the results across all channels for the value in Table 1.

C Theoritical Analysis

This section discusses the effect of FAN on stationarity and temporal distribution in a theoretical perspective. We conclude that FAN enhances the stationarity of the input and mitigates distribution in the time domain.

C.1 Preliminary

Discrete Fourier Transform. Given a multivariate time series input X , we independently apply the 1-dim Fourier transform to each dimension x ( i ) , hence, we illustrate in vector settings. For a discrete

time series vector x ∈ R L with L time steps, it is transformed into Fourier domain by applying the 1-dim DFT, and can be transformed back using 1-dim IDFT, which can be defined as:

where w is current frequency, t is current time step, and z represents the Fourier transformation results which is a complex vector with real and imaginary parts, the amplitude and phase can be calculated as:

where Im( z [ m ]) and Re( z [ m ]) indicate imaginary and real parts of a complex number, and atan2 is the two-argument form of arctan.

Distribution Of Fourier Components. The distributions of Fourier amplitude and phase can be modeled as Rayleigh distribution and uniform distribution respectively [14], thus the probabilistic density function can be represented as:

where a and p denotes a amplitude and phase scalar variable, σ is the scale parameter of the distribution. Thus, the frequency domain distribution ( T non-identically-distributed variables) can be modeled as a joint Rayleigh distribution with different scale parameters:

C.2 Variance Over Spectrum

Along with the time series spectral theory [35], a time series with smaller variance in the spectrum is more stationary, in this section, we try to prove the proposed FAN can reduce the variance over spectrum, thus enhance the stationarity of the input data. Hence, we prove that, given an univariate time series real value vector x ∈ R T , after removing main frequency components z [ k ] ∈ K , the variance on spectrum can be reduced Var( a res ) < Var( a ) .

Here, the marginal distribution of the amplitude vector (the spectrum) a is represented as a joint Rayleigh distribution with different scale parameters:

Note that although we assume that the frequency components are independent with each other, this assumption is actually widely used [16] since it is quite possible that a specific component changes independently, e.g., the daily weekly changes while the monthly periodicity stays the same. Following the principle of additivity of variance for independent variables [13], the variance of the amplitude vector a can be expressed as follows:

after removing frequencies k ∈ K , the joint distribution actually becomes:

thus, the variance of the whole distribution after removing top K -amplitude signals reduces to a smaller number, since the independent variance of of each dimension is positive, which is:

C.3 Influence On Temporal Distribution

Relation with Temporal Statistics. The zero frequency of the Fourier transform divided by L is actually the mean of the statistical measure, and the energy of the Fourier transform of frequency components above zero is equivalent to the variance of the input scaled by L , proved as follow:

According to Parseval theorem [5], for a discrete signal x , its energy is identical in both the time and frequency domain:

Thus the variance of the input signal can be defined as the energy of Fourier components with frequency above zero.

(19)

Influence On Mean. Since the mean is equal to the zero frequency component in time domain and for any other components, the expectation is zero since they are all number of periodic sin/cos signals, after removing the zero frequency component, the expectation is then equal to zero. Due to this property, this is also known as the 'detrending' in traditional signal processing [32], proved as:

Influence On Variance. Since the The normalization step select and remove top K amplitude Fourier components, the Fourier spectrum energy will be significantly diminished, defined as:

thus, the input variance then can be largely reduced, which is:

In summary, our method can effectively reduce the range of data distribution, which is crucial for enhancing the performance of the backbone model and minimizing the risk of overfitting [12].

C.4 Fourier Spectrum Empirical Analysis

The variance in the Fourier spectrum is an important indicator reflecting stationarity [20]. The closer the frequency components are to each other, the smaller the variance between the components, thus the stronger the stationarity [35]. Therefore, we compare the changes in frequency domain components for different methods and present the results in Fig. 10. In Fig. 10, after FAN's normalization step, the distribution exhibits alignment of the input and output, and the range of the distribution mean has decreased to 8, compared with previous methods which are round 80, 70, 70 respectively for SAN, Dish-TS and RevIN. However, other methods still show significant differences between the input and output distributions, with the range of the frequency domain amplitude distribution reaching up to 80, indicating the presence of strong non-stationary signals. This highlights the effectiveness of our method in handling non-stationarity, especially for seasonal periodic signals, which previous methods have not successfully considered.

(b) SAN

(a) FAN

(d) RevIN

(c) Dish-TS

D Ablation Study

D.1 Hyperparameter Analysis

Our model incorporates a hyperparameter K , which represents the maximum frequency count selection. In this section, we provide a sensitivity analysis for this parameter in Fig. 11. We observe that our proposed FAN achieves stable performance across various parameter settings.

ETTh1

ETTh2

ETTm1

ETTm2

H-96

0.16

0.500

0.14

=90

0.55

H=168

H=336

0.15

0.475

0.13

H=168

H=336

MSE

000

H=720

MSE

0.14

MSE

0.425

0.450

H=336

H=

MSE

0.11

0.12

H=720

0.45

0.13

H=96

0.400

H=720

0.10

0.40

0.12

H=168

H=336

0.375

0.350

0.09

4

59

162432

0.11

59

16

24

32

0.325

3481116242832

0.08

3

32

K

K

K

K

ExchangeRate

Electricity

Traffic

Weather

3S

0.20

0.30

0.25

H-96

H=168

H=336

H=720

S

0.22

0.21

H=96

H=168

MSE

0.50

0.48

0.46

H336

H=96

H=720

H=168

0.325

0.350

0.300

0.275

H=168

H=336

H=720

0.20

H=336

0.250

0.15

H=720

0.44

0.19

0.225

0.10

0.42

0.200

0.18

0.05

0.175

32

18

$2

K

Moreover, from Fig. 11, we note the following observations: (1) As the prediction length increases, the need for a larger K becomes more apparent, significantly enhancing performance. This is likely

due to the increased prediction steps that bring more frequency information into the model. For example, in the ExchangeRate dataset, when the prediction length is 720, K = 16 outperforms using only 3 and 6 frequencies. (2) In datasets with higher sampling rates, such as Traffic and ETTm2, larger K values enhance performance at all prediction steps. This could be attributed to the finer granularity of sampling in these datasets (minute-level) compared to yearly sampling in ExchangeRate and hourly in ETTh2, resulting in richer frequency signals in ETTm2 and Traffic, thereby enhancing FAN's performance on these datasets.

D.2 Pattern Prediction Module

To justify our rather simple MLP structure for predicting the future main frequency component, we extended the basic MLP with three additional layers to observe the results. These layers include

+MLP: adding an additional MLP layer on top of the basic MLP.

+GRU [4]: adding a GRU layer, which is a recurrent neural network, mitigates the problem of gradient vanishing through the gating mechanism.

+TSMixer [7]: adding a TSMixer layer, which is a state-of-the-art lightweight model that also considers inter-dimensional relationships.

We perform ablation on ETTh1, ExchangeRate, Weather datasets, under experiment settings of Section 4.2, we report the MAE/MSE evaluation metrics, and the result is shown at Table 6.

In Table 6, the three-layer MLP of FAN performed best overall on three datasets and more complex models tend to perform worse. We believe this is due to the following reasons: (1) The main frequency signal provides a baseline position for the backbone model, leading to more robust predictions and thus to greater model robustness. (2) The main frequency signal is subject to underlying physical characteristics, resulting in relatively slower changes. This has been verified by observations in Appendix B.3, showing that the main frequency signal changes at least 40.27% more slowly compared to the residual frequency signal. Therefore, a simple three-layer MLP is sufficient to provide effective and somewhat more robust predictions. However, a four-layer MLP and a GRU also achieved the best performance in some metrics, indicating that there is still room for improvement in future work.

E Full Results And Discussions

E.1 Experiment On Synthetic Data

To fully demonstrate the effectiveness of our method on signals with varying non-stationary frequencies, we generated a synthetic multidimensional time-series dataset using composite sinusoidal signals [32] to verify the effectivenss of FAN on evolving non-stationary time series. Each dimension is composed of i superimposed sinusoidal signals with linearly changing amplitude, generated as X ( i ) t = ∑ i j =1 a t sin 2 π T j t, i = 1 , . . . , D . where a t is the signal amplitude, T i are the periodicities, for example, daily periodicity in hour T i = 24 , and t is the current time step.

In the synthetic dataset, each synthetic signal is a combination of multiple sinusoidal signals with linear changes and fixed periods, and varies in the training, validation, and test sets. In Table 7, we list the settings of these signals, every synthetic signal is a composition of these signals, e.g., Syn-5 contains Sig1-5, Syn-9 contains Sig1-9, the generation code can also be found in our code repository2.

We conduct a 720-step multivariate forecasting experiment on synthetic data using DLinear as the backbone model and compared with other reversible normalization methods. Results are shown in Table 8. We observe averaged improvements ranging from 19.75% to 47.04%. As the number of composite frequencies increases (from Syn-5 to Syn-9), the prediction difficulty escalates. Previous normalization methods failed to make further improvements as probably because they can not address seasonality patterns shift. Conversely, our model's enhancements steadily increase. The MAE/MSE improvements ranges from 19.75% to 32.45% and from 23.76% to 41.75%. This underscores the effectiveness of FAN in handling intricate seasonality patterns.

We compare the performance of different normalization methods in Fig. 12. Beyond performance, FAN also surpasses other models in performance on synthetic datasets, requiring only three epochs to achieve a smaller test loss compared to ten epochs required by other normalizations. This demonstrates that varying periodic signals indeed affect the predictive performance of models and the non-stationarity that normalization methods must counteract, with our model proving effective in handling these challenges.

(a) Syn-5

(b) Syn-7

(c) Syn-9

0

2

4

6

8

10

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

Loss

Epochs

0

2

4

6

8

10

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Loss

Epochs

RevIN

Dish-TS

SAN

FAN

0

2

4

6

8

10

0.1

0.2

0.3

0.4

0.5

0.6

Loss

Epochs

E.2 Full results of ETT benchmarks

We present the full results of ETT benchmarks in Table 9. In the entire ETT benchmarks, FAN demonstrates improvements over the original models in 114 out of 128 metrics. Specifically, our model exhibits average enhancements of, 18.43%, 31.64%, and 12.04% for FEDformer, Informer, and SCINet respectively. It's worth noting that despite FEDformer's utilization of Fourier transform for seasonality analysis, it still struggles with handling changing seasonality patterns, thus leaving

room for our 31.64% improvement. However, FAN fails to enhance the DLinear backbone in the ETTm1 dataset. We attribute this to the inherently higher stationarity of the ETTm1 dataset and its fewer trend changes (Table 1), which may not align well with the DLinear model which is based on the moving average.

E.3 FAN for Fourier-based Bakcbones

To demonstrate the effectiveness of our method in extracting non-stationary seasonal patterns, we select two other models based on the Fourier transform and perform additional experiments, including: (1) TimesNet [41], which generates 2D variations of time series data using Fourier Transform; (2) Koopa [27], which employs Fourier Transform for dynamic time series modeling based on koopman theory. It is important to note that although both models extract the top k signals, their main frequency selection is based on the average dimensions of the training set rather than the input-specific, which may limit their models' ability to handle varying dimensions and inputs. Furthermore, they internally use RevIN [17] in their model implementation, for a fair comparison, we remove this part or replace it with FAN, we present the experiment results in Table 10.

As in Table 10, Even with state-of-the-art models based on Fourier-transform, our model still demonstrates performance improvements across almost all datasets. Specifically, for long inputs, our model consistently shows performance enhancements. In particular, on the Exchange dataset, our model achieves a maximum improvement of 84.85% in TimesNet and 26.10% in Koopa. We believe the significant improvement in TimesNet is due to its lack of handling seasonal non-stationarity compared to Koopa. However, Although Koopa explicitly handle non-stationarity, we still observe improvements in Koopa. For the ETTm2, Electricity, Traffic, and Weather datasets, our model shows stable MSE performance large improvements for short-term 96 steps and long-term 720 steps inputs by 4.71%/6.09%, 1.30%/8.57%, -1.96%/2.57%, and -1.78%/2.43%, respectively. This improvement is likely due to our model's approach of learning directly from the changes in primary frequency components and our instance-wise and dimension-specific frequency analysis.

E.4 Full Results of Baselines Comparison

In Table 11, we provide the detailed experimental results of the comparison between FAN and state-of-the-art normalization methods for non-stationary time series normalization: RevIN [17], Dish-TS [10], and SAN [28].

The table clearly shows that FAN outperforms existing approaches in most cases, particularly in the ExchangeRate dataset, our long-term 720-step prediction significantly outperforms other baselines, highlighting the importance of handling seasonal non-stationary information in long-term predictions. However, we fail to make further improvement on Weather and ETTh2 dataset. Considering that in these two datasets, the seasonal variation is very small (as in Appendix B.1), the limited seasonal non-stationary information might have led to the inability to further optimization.

F Limitations

Though FAN shows promising performance, there are still some limitations. First, we removed a significant amount of non-stationary trend and seasonal information. While this is effective in most baselines, it may lead to an over-stationary issue, causing a decline in the backbone's performance. Second, we largely select the frequency counts K in a search-based manner or based on dataset priors. This approach may lead to incorrect K value selection, resulting in an under-stationary issue or more severe over-stationary issue. Additionally, our non-stationary pattern extraction is based on the Fourier transform, and a finite number of Fourier signals cannot represent all periodic signals, which may hinder our ability to handle some waveforms, e.g. square waves. Therefore, future work can focus on more effective instance-specific K value selection, strict dynamic control of non-stationarity elimination, and other methods for extracting non-stationary waveforms.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While '[Yes] ' is generally preferable to '[No] ', it is perfectly acceptable to answer '[No] ' provided a proper justification is given (e.g., 'error bars are not reported because it would be too computationally expensive' or 'we were unable to find the license for the dataset we used'). In general, answering '[No] ' or '[NA] ' is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist' ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: This paper proposes a novel normalization method for non-stationary time series forecasting that could handle both evolving trend and seasonal patterns.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitation of proposed method, FAN, at Appendix F.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All assumptions and proofs are clearly stated in our manuscript and are detailed in Appendix C.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The experimental settings are detailed in Sec. 4.2 and Appendix A.1. All experiments can be easily reproduced with our code.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We make our code publicly available at 2, including data and detail documentation.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The experimental settings and hyperparameter selection details are in Sec. 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: See the results with standard deviation in Table 4, 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We discuss the experiments compute resources at Sec. 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes, this paper conform the the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper is a foundational research and has no direct negative social impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All data and methods are explicitly mentioned.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: All data, code, and documentation are given in the Supplementary Material and our public repository 2.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper introduces FAN, a novel instance normalization technique designed to address both dynamic trends and seasonal patterns. FAN is a model-agnostic method that can be integrated with various predictive models. It significantly enhances performance, achieving average MSE improvements of 7.76% to 37.90%.

STRENGTHS:
- Non-stationary time series forecasting has long been a highly challenging problem. Despite extensive research, there remains significant value in further exploration.
- This paper is well-written, precise in expression, and comprehensive in content. It offers a new perspective on mitigating non-stationarity issues in non-stationary time series forecasting.
- The experiments in this paper are relatively comprehensive.

WEAKNESSES:
- There are some shortcomings in the experimental descriptions. For example, the calculation methods for Trend Variation and Seasonality Variation are not clearly explained, and the ADF test values are provided only after normalization, lacking a direct comparison with the values before normalization.
- The selection of the hyperparameter K plays a crucial role in the effectiveness of the model.

QUESTIONS:
See the weakness.

LIMITATIONS:
Limitations are in the appendix.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Frequency Adaptive Normalization (FAN) to improve time series forecasting by addressing non-stationary data with evolving trends and seasonal patterns. Unlike reversible instance normalization, which handles trends but not seasonal patterns, FAN employs the Fourier transform to identify and model predominant frequency components. This model-agnostic approach, applied to four forecasting models, shows significant performance improvements across eight benchmark datasets.\n\nSTRENGTHS:\n1. The paper is well-written and structured, with a well-motivated idea and promising experimental results.\n\n\n2. The code is available for checking and reproducing the results.\n\nWEAKNESSES:\n1. I noticed a significant disparity between the results reported in Table 2 and those reported by SAN, despite using the same model configurations. This performance gap is unexpected and warrants further investigation.\n\n\n2. A key question is whether FAN's capabilities fully encompass those of SAN. Specifically, can FAN handle changes in statistical properties, such as mean and variation, that characterize seasonal and trend patterns? A more detailed comparison between FAN and previous relevant methods would be beneficial.\n\n\n3. I noticed that FAN consumes 0.24 million parameters, which is significantly more than FITS [1], an entirely frequency-based benchmark model. It would be helpful to clarify why such a large number of parameters is necessary to address non-stationarity.\n\n\n4. Previous studies [1][2] have shown that purely using frequency-domain representation can achieve accurate future variation estimation with simple, lightweight models. The proposed method, however, relies on a backbone model in addition to FAN to handle specific parts of the evolution. Does this combination offer an advantage over methods based solely on frequency components?\n\n\n5. Furthermore, can FAN enhance frequency-based methods in a similar manner to DLinear and FEDformer?\n\n[1] Xu, Zhijian, Ailing Zeng, and Qiang Xu. \"FITS: Modeling time series with $10 k $ parameters.\" arXiv preprint arXiv:2307.03756 (2023).\n\n[2] Yi, Kun, et al. \"Frequency-domain MLPs are more effective learners in time series forecasting.\" Advances in Neural Information Processing Systems 36 (2024).\n\nQUESTIONS:\nPlease address the weak points I raised above.\n\nLIMITATIONS:\nThere is no discussion on the limitations or social impacts of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-04"}

comments: SUMMARY:
This paper proposes a new instance normalization solution called frequency adaptive normalization (FAN) to address non-stationary data in time series forecasting. This paper extends instance normalization to handle both dynamic trend and seasonal patterns by employing Fourier transform to identify predominant frequency components. This paper introduces a simple MLP model to predict the discrepancy of frequency components between inputs and outputs. Several experiments are conducted to prove the effectiveness of the proposed method.

STRENGTHS:
1. this paper is easy to understand. The writing is good.

2. this paper is well-motivated. Since the current normalization techniques are not specifically focused on periodic patterns, the authors try to use frequency techniques to enhance them.

3. the experiments are well-sounded. Several experiments are conducted.

4. the overall framework is simple but effective, which is good for time series forecasting against distribution shifts.

WEAKNESSES:
1. Missing related work. Several frequency-focused or periodicity-focused works can be discussed [1-3].

2. Most adopted backbones seem to be earlier works of time series forecasting. Adding more backbones such as PatchTST can make the experiments better.

[1] DEPTS: Deep expansion learning for periodic time series forecasting. In ICLR.

[2] Frequency-domain MLPs are more effective learners in time series forecasting. In NeurIPS.

[3] Deep Frequency Derivative Learning for Non-stationary Time Series Forecasting. In IJCAI.

QUESTIONS:
Can more related works be discussed?
Can more results be provided?

LIMITATIONS:
See weakness.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents FAN - frequency adaptive normalization - as an alternative approach to de-trending seasonality in non-stationary data through Fourier transform decomposition. The method relies on dynamically identifying K instance-wise predominant frequency components; the evolution of these components is then modeled using a simple MLP approach rather than assuming they remain unchanged. The authors employ FAN for eight benchmark forecasting problems using four different ML backbones and assess the performance with and without FAN. The authors provide the code through a public GitHub repository.\n\nSTRENGTHS:\nThe paper addresses the problem of evolving seasonality in non-stationary data through utilizing instance-wise frequency components, rather than global frequency analysis, coupled with an MLP approach to capture non-linear relationships in the evolution of the predominant non-stationary components. The main contribution appears to be a dynamic selection of the K relevant frequencies, as opposed to previous methods that assume a fixed frequency set across inputs or select frequencies randomly, as well as modeling the frequency components’ evolution using an MLP approach rather than assuming they remain constant or through predicting the statistics.\n\nWEAKNESSES:\nWhile the contributions appear novel and the paper presents both theoretical considerations and applied results, the robustness of the methodology is not well established by the authors. The authors state that the selection of K is done based on inspection of the data as “the average maximum amplitude within 10% of the training set” and is shown to vary between 2 and 30 for the benchmark datasets. However, the ablation study (which is presented only in the Appendix and showcases only a subset of the datasets) does not properly support the selection method -  the selected K for the presented datasets were 3, 5, 2, and 30 while the sensitivity analysis only utilizes K = 4, 8, 12, and 24. It would have been more pertinent to do dataset-specific analysis as the hyperparameter is dataset-specific. Furthermore, there appears to be a prediction length dependency that is not properly investigated. In terms of establishing the performance of FAN, the authors present comparisons between the performance of different backbone models with and without FAN as the main result and relegate comparison to other normalization methods to averages over all prediction lengths in the main paper. While the premise of improving prediction performance through normalization is indeed established through the former comparison, that is not the scope of the paper. Furthermore, we are unable to reproduce the percentage MAE and MSE improvements stated for FAN in section 4.2, especially those used to claim improved performance with prediction length. \n\nThe paper showcases an overall logical organization and contains most necessary sections for properly presenting the work (relegating the limitations section to the Appendix is a questionable choice). However, there are several places where grammar and phrasing reduce clarity and make readability difficult. A few examples: the description of the selection of the hyperparameter K as “the average maximum amplitude within 10% of the training set” is not clear; FAN is presented as a normalization *method* that can be combined with any ML model, however, in Section E1, FAN is referred to as a *model*; the y-axis in Fig 4b is nonsensical and overestimates the difference between models; Fig 10 would be clearer if it showed difference between input and output frequencies rather than overlaying the two. Also, the figures throughout the paper and Appendix would benefit from increased font size (most are barely legible without significant zoom).  Further, there are a few statements throughout the paper that indicate overconfidence in the results, or confusion in interpreting them. For example, Fig 4c is used to support FAN’s improved performance with increasing input length compared to other models. While MSE is indeed lower for FAN for longer input lengths, all models show the same trend in decreased MSE with increased input length. Similarly, Fig 12 is stated to demonstrate the higher convergence speed of FAN compared to other normalization methods. Again, the metric (loss as opposed to MSE in Fig 4c) is indeed lower for FAN, but the compared-to methods seem to plateau at earlier epochs than FAN, indicating the opposite of the claim of which methods converge faster. \n\nWhile the proposed normalization method does present some novelty that can be built upon by the authors and others, the significance of the contribution is difficult to assess due to the aforementioned limitations in proven robustness of methodology. The results would also benefit from statistical analysis to properly elucidate the improved performance of the methods - simply indicating lower errors while not investigating if the error reduction is significant compared to other methods and using models without FAN does not properly support FAN’s performance.\n\nQUESTIONS:\n1. What are the assumptions for correlation between input dimensions?\n2. How come you select a data normalization method (z-score) that does not handle non-stationary series when your data is inherently non-stationary? How does this affect your results?\n3. How have you calculated the MAE and MSE improvements, specifically for Tab 2? We are not able to recreate the stated percentages, and cannot support the statement about improved performance with increased prediction length.\n4. What is the comparison between on line 266?\n5. In Section C4 it is stated that \"the range of the distribution mean has decreased to 8\" - decreased from what? How is this seen in Fig 10?\n\nLIMITATIONS:\nThe authors do not present limitations in the main paper but have opted to relegate this section to the Appendix. While we understand that the page restrictions for the paper puts restraints on the presented content, adjustments should be made to ensure important information is presented in the main paper. The content of the limitation section is also not satisfactory and, in fact, enforces the aforementioned concerns about the contributions of the paper. Namely, the authors discuss their proposed methodology for selection of the hyperparameter K, which is presented as one of the novel contributions of their works, and state that their approach “may lead to incorrect K value selection”. It would have been advisory to focus the current study on elucidating the robustness of the proposed methodology, rather than simply presenting results of one implementation.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}