FredNormer: Frequency Domain Normalization for Non-stationary Time Series Forecasting

## Abstract

Abstract Recent normalization-based methods have shown great success in tackling the distribution shift issue, facilitating non-stationary time series forecasting.
Since these methods operate in the time domain, they may fail to fully capture the dynamic patterns that are more apparent in the frequency domain, leading to suboptimal results.
This paper first theoretically analyzes how normalization methods affect frequency components.
We prove that the current normalization methods that operate in the time domain uniformly scale non-zero frequencies. Thus, they struggle to determine components that contribute to more robust forecasting.
Therefore, we propose \method , which observes datasets from a frequency perspective and adaptively up-weights the key frequency components.
To this end, \method consists of two components: a statistical metric that normalizes the input samples based on their frequency stability and a learnable weighting layer that adjusts stability and introduces sample-specific variations.
Notably, \method is a plug-and-play module, which does not compromise the efficiency compared to existing normalization methods.
Extensive experiments show that \method improves the averaged MSE of backbone forecasting models by 33.3% and 55.3% on the ETTm2 dataset.
Compared to the baseline normalization methods, \method achieves 18 top-1 results and 6 top-2 results out of 28 settings.

## 1 Introduction

Deep learning models have demonstrated significant success in time series forecasting (Moosavi et al., ; Zhou et al., ; Wu et al., ; M. et al., ; Nie et al., ; Zhang & Yan, ; Liu et al., ).
These models aim to extract diverse and informative patterns from historical observations to enhance the accuracy of future time series predictions.
To achieve accurate time series predictions, a key challenge is that time series data derived from numerous real-world systems exhibit dynamic and evolving patterns, i.e., a phenomenon known as non-stationarity (Stoica et al., ; Box et al., ; Xie et al., ; Rhif et al., ).
This characteristic typically results in discrepancies among training, testing, and future unseen data distributions.
Consequently, the non-stationary characteristics of time series data necessitate the development of forecasting models that are robust to such temporal shifts in data distribution, while failing to address this challenge often leads to representation degradation and compromised model generalization (Kim et al., ; Du et al., ; Lu et al., ).

A recent fashion to tackle the above-mentioned distribution shift issue is leveraging plug-and-play normalization methods (Kim et al., ; Fan et al., ; Liu et al., ; Han et al., ).
These methods typically normalize the input time series to a unified distribution, removing non-stationarity explicitly to reduce discrepancies in data distributions.
During the forecasting stage, a denormalization is applied, reintroducing the distribution statistics information to the data.
This step ensures the forecasting results are accurate while reflecting the inherent variability and fluctuation in the time series, enhancing generalization.
Since they focus on scaling the inputs and outputs, these methods function as “model-friendly modules” that could be easily integrated into various forecasting models without any transformation.
However, existing works still face several challenges.

(1) Existing methods model non-stationary in the time domain that may not fully capture dynamic and evolving patterns in time series.
Conceptually, a series of time observations is considered a complex set of waves that varies over time (Proakis & Manolakis, ).
These temporal variations, manifested as various frequency waves, are intermixed in the real world (He et al., ; Wu et al., ; Piao et al., ).
Modeling solely in the time domain struggles to distinguish between different frequency components within superimposed time series, resulting in entangled patterns and sub-optimal performance.
Recent works have shown that explicitly modeling frequency can enhance representation quality and forecasting accuracy (Yi et al., ; Zhang et al., ; Piao et al., ).
(2) Existing methods primarily rely on z-score normalization to rescale the input distribution.
However, as illustrated in Figure 1, this normalization applies uniform scaling across all frequency components, which leaves frequency-specific patterns unaltered. Such uniform scaling may reduce distributional differences across frequencies, potentially obscuring important time-invariant features crucial for generalization to unseen time series.

Figure: Figure 1: How does z-score normalization affect the frequency amplitudes? (a) Normalization in the time domain compresses the variability of the data, leading to a more compact distribution. (b) The amplitudes of non-zero frequencies are just uniformly scaled after normalization.
Refer to caption: x1.png

To tackle the challenges above, this paper proposes a novel solution for the distribution shift issue in time series forecasting by modeling the non-stationarity in the frequency domain.
We first investigate why time-domain normalization does not provide significant benefits for capturing dynamics in frequencies and theoretically prove our findings.
We then propose \methodbased on the insight we have gained. Specifically, \methodlearns time-invariant frequency components, termed stable frequencies, to suppress non-stationary for robust forecasting.
Finally, we propose a new normalization metric tailored for quantifying the importance or stability of frequencies.
Since \methodoperates only on the input time series, it is plug-and-play, making it easily adaptable to various forecasting models and complementary to existing normalization methods.

Contributions and Novelty.
\methodtackles the aforementioned two challenges as follows:
(1) We first transform the input time series data from the time domain to the spectral domain and extract the statistical significance of frequencies across training sets.
We then propose a linear projection to capture data-specific properties, which adjusts the statistical significance used to weight the spectrum.
The processed time series data, which carries more stable components while filtering out non-stationary elements, is finally used as input for the forecasting models.
(2) To quantify frequency significance, we propose a stability metric based on the well-known Coefficient of Variation (CV) (Aja-Fernández & Alberola-López, ; Jalilibal et al., ).
It computes the ratio between the mean and variance of frequency amplitude, providing a relative measure (scaling) of variability rather than the absolute scaling/rescaling (Reed et al., ; Abdi, ).
In summary, our contributions lie in:

- •
Theoretical Analysis.
We theoretically analyze how normalization-based methods function in the frequency domain and why they fail to suppress non-stationary frequencies.
- •
Novel Problem Formulation.
We are the first to investigate a frequency-based module to tackle the distributional issue in non-stationary time series.
Hence, we formulate a key research question in this paper: How can we effectively capture stable frequencies to support robust forecasting?
- •
Simple, Efficient, Model-Agnostic Method.
We propose \method, which explicitly learns the statistical significance of frequencies using a new stability metric.
Only simple linear projection layers with a few parameters need learning and tuning.

We apply \methodto different forecasting models to validate its effectiveness across various datasets.
Overall, in non-stationary datasets, such as Traffic, we improved PatchTST and iTransformer by 33.3% and 55.3%, respectively.
\methodachieved 18 top-1 results and 6 top-2 results out of 28 settings compared to the baselines and outperformed the SOTA normalization method, with speed improvements ranging from 60% to 70% in 16 out of 28 settings.

Notations.
Vectors and matrices are denoted by lowercase and uppercase boldface letters, respectively (e.g., ${\bm{x}}$, $\mathbf{X}$).
We consider a training dataset with $N$ labeled samples consisting of $L$ timestamps of past observations and $H$ timestamps of future data.
The Discrete Fourier Transform (DFT) of a time series $\mathbf{X}$ is represented by the coefficient matrix $\mathbf{F}\in\mathbb{R}^{L}$:
$\mathbf{F}(k)=\sum_{t=0}^{L-1}x(t)e^{-j\frac{2\pi}{L}kt},\quad\text{for }k=0,1
,\dots,L-1.$
The amplitude matrix $\mathbf{A}$ of a time series $\mathbf{X}$ is defined as $\mathbf{A}=\left[\left|\sum_{t=0}^{L-1}x(t)e^{-j\frac{2\pi}{L}kt}\right|\right
]^{L-1}_{k=0}$, where $|\cdot|$ represents the 2-norm of the DFT coefficients.
The indicator function $\mathbb{1}\{k=0\}$ equals $1$ if $k$ does not equal 0 0, and 0 0 otherwise.
$\mu(\cdot)$ and $\delta(\cdot)$ represent the mean and the standard deviation, respectively.

## 2 Problem Formulation

In this section, we first define frequency stability and then investigate how normalization in the time domain affects the frequency domain and its influences.
Finally, we formulate the research question.

###### Definition 1

(Frequency Stability).
Given a training set containing $N$ samples, we define the statistical stability of the $k$-th component as the reciprocal of the Coefficient of Variation $\gamma$:

$$ $S(k):=\frac{1}{\gamma(A(k))}=\frac{\mu(A(k))}{\sigma(A(k))}$ (1) $$

where $\mu(A(k))=\frac{1}{N}\sum_{i=1}^{N}A^{i}(k)$ and $\sigma(A(k))=\sqrt{\frac{1}{N}\sum_{i=1}^{N}\left(A^{i}(k)-\mu(A(k))\right)^{2}}$ are the mean and standard deviation of the amplitude across the training set.

$S(k)$ measures the statistical significance of each frequency across the dataset.
A frequency component with higher stability denotes lower relative variability, i.e., more stable; otherwise, it is considered unstable.
All stable components are included in the subset $\mathcal{O}$.

###### Definition 2

(Stable Frequency Subset).
Given $K-1$ non-zero frequencies, a subset $\mathcal{O}=\{1,\dots,M\}$, where $M\ll K$, contains $M$ components with higher stability $S(k)$.

###### Definition 3

(Linearity of Fourier Transform.)
For any functions $f_{1}$ and $f_{2}$, and constants $a$ and $b$, the Fourier Transform $\mathcal{F}$ satisfies:

$$ $\mathcal{F}(af_{1}+bf_{2})=a\mathcal{F}(f_{1})+b\mathcal{F}(f_{2})$ (2) $$

Thus, we investigate the variations of $\mathcal{O}$ (Def. 2) before and after z-score normalization in the time domain.
Meanwhile, the linearity property (Def. 3) allows us to map this normalization to its corresponding operations in the frequency domain.

###### Lemma 1

Normalization in the time domain uniformly scales non-zero frequency components.

Proof 1.
For a normalized time series $\mathbf{X}_{z}(t)=\frac{\mathbf{X}(t)-\mu(\mathbf{X})}{\sigma(\mathbf{X})}$, a Fourier transform $\mathcal{F}(\cdot)$ is applied by:

$$ $\mathcal{F}(\mathbf{X}_{z}(t))=\mathcal{F}(\frac{\mathbf{X}(t)-\mu(\mathbf{X}) }{\sigma(\mathbf{X})})=\frac{1}{\sigma(\mathbf{X})}\left(\mathcal{F}(\mathbf{X }(t))-\mu(\mathbf{X})\mathbb{1}\{k=0\}\right)\quad\text{(by linearity, Def. \ref{Def:FourierTransformLinearity})}$ (3) $$

The left item $\mathcal{F}(\mathbf{X}(t))$ is the Fourier Transform of $\mathbf{X}$.
Since $\mu(\mathbf{X})$ is a constant, the resulted Fourier transformation can be represented by $\mu(\mathbf{X})\mathbb{1}\{k=0\}$, where $\mathbb{1}\{k=0\}$ is an indicator function, that is, for $k\neq 0$, $\mathbb{1}\{k=0\}=0$.
Then for the non-zero frequencies ($k\neq 0$),

$$ $\mathcal{F}(\mathbf{X}_{z}(t))=\frac{1}{\sigma(\mathbf{X})}\left(\mathcal{F}( \mathbf{X}(t))-\mu(\mathbf{X})\cdot 0\right)=\frac{1}{\sigma(\mathbf{X})} \mathcal{F}(\mathbf{X}(t))\quad\text{for }k\neq 0$ (4) $$

Here, the amplitudes of the non-zero frequency components are scaled by $\frac{1}{\sigma(\mathbf{X})}$:

$$ $|\mathbf{A}_{z}(k)|=\frac{1}{\sigma(\mathbf{X})}|\mathbf{A}(k)|,\quad\text{for }k\neq 0$ (5) $$

Here, $\mathbf{A}$ is the amplitudes of frequencies. Its definition can be found in Notation in Section 1.
Thus, normalization uniformly scales all non-zero frequency components in the frequency domain.
A more detailed proof of Lemma 1 can be found in Appendix A.1.

###### Theorem 1

(
The proportion of $\mathcal{O}$ to the spectrum is unchanged after normalization
).

The energy proportion of $\mathcal{O}$ in the spectrum is defined by the sum of the amplitudes divided by the energy of the entire spectrum.
Then if Lemma 1 holds, we have:

$$ $\frac{\sum_{k\in\mathcal{O}}|\mathbf{A}_{z}(k)|}{\sum_{k=1}^{K-1}|\mathbf{A}_{ z}(k)|}=\frac{\sum_{k\in\mathcal{O}}|\mathbf{A}(k)|}{\sum_{k=1}^{K-1}|\mathbf{ A}(k)|}$ (6) $$

The left and right items represent the ratio after and before normalization, respectively.
As shown, this ratio remains the same.
A proof of Theorem 1 can be found in Appendix A.2.

Remark.
Since Theorem 1 holds, the normalization operation keeps the proportion unchanged.
The stable components in $\mathcal{O}$ are often intermixed with unstable components in the time series data.
This makes forecasting models struggle to distinguish between stable and unstable components, resulting in entangled patterns and sub-optimal performance (Elvander & Jakobsson, ; Piao et al., ).
This motivates us to increase the weights of stable components and suppress unstable ones.
In this paper, we aim to learn a frequency-based method for assigning higher weights to stable components and improving forecasting performance for future data.

###### Problem 1

(Enhancing stable components for better generalization).
Given a time series dataset with stable frequency components $\mathcal{O}$, our goal is to develop a module $f(\cdot)$ that dynamically adjusts $\mathcal{O}$, or the amplitudes, increasing the energy proportion to the spectrum:

$$ $f(\mathbf{A}(k)):=w(k)\cdot\mathbf{A}(k),\quad\text{where }w(k_{1})>w(k_{2}), \text{ for }k_{1}\in\mathcal{O}\text{ and }k_{2}\notin\mathcal{O}.$ $$

Here $w(\cdot)$ represents the weighting function.
In this paper, we assume $w(\cdot)$ should have two properties:
$\rm(\hskip 1.79993pti\hskip 1.79993pt)$ extracting the statistical significance of frequencies across samples in the training set;
$\rm(\hskip 1.79993ptii\hskip 1.79993pt)$ capturing data-specific properties to adjust the statistical significance.

Figure: Figure 2: (a) Frequency variations across a sequence of time series samples. The red bar denotes the unstable frequency components, while stable ones are in gray. (b) An overview of \method.
Refer to caption: x2.png

Figure: Algorithm 1 Frequency Stability Measure

###### Definition 1

###### Definition 2

###### Definition 3

###### Lemma 1

###### Theorem 1

###### Problem 1

## 3 Method: FredNormer

In this section, we present \method(as shown in Figure 2), which consists of two components: $\rm(\hskip 1.79993pti\hskip 1.79993pt)$ a frequency stability measure, and $\rm(\hskip 1.79993ptii\hskip 1.79993pt)$ a frequency stability weighting layer, followed by:

- •
First, we compute the frequency stability, defined in Definition 1, for a given time series dataset.
The output is a statistical measure that can be tuned for different data scenarios.
- •
Second, the input time series is transformed into the frequency spectrum using the DFT.
- •
Third, during the training phase, a learnable linear projection adjusts the frequency stability measure for the spectrum to introduce sample-specific variation, increasing distributional diversity.
- •
Finally, \methodtransforms the adjusted frequency spectrum back into the time domain using the Inverse-DFT (IDFT) that serves as input for subsequent various forecasting models.

The detailed workflows, including two key components, are presented in Algorithms 1 and 2.

Frequency Stability Measure.
Given a training set contains $N$ time series samples $\mathcal{X}=\{\mathbf{X}^{(i)}\}_{i=1}^{N}$,
we first apply the DFT to each sample $\mathbf{X}^{(i)}\in\mathbb{R}^{L\times C}$ to transform it into $\mathbf{A}^{(i)}\in\mathbb{R}^{K\times C}$.
Here, $K$ is the number of frequency components, and $C$ denotes the number of channels.
The frequency stability measure $S(k)\in\mathbb{R}^{K\times C}$ is then applied:

$$ $S(k)=\frac{\mu(\mathbf{A}(k))}{\sigma(\mathbf{A}(k))}=\frac{\frac{1}{N}\sum_{i =1}^{N}\mathbf{A}(k)^{(i)}}{\sqrt{\frac{1}{N}\sum_{i=1}^{N}\left(\mathbf{A}(k) ^{(i)}-\mu(\mathbf{A}(k))\right)^{2}}}$ (7) $$

where $\mu(\mathbf{A}(k))\in\mathbb{R}^{K\times C}$ and $\sigma(\mathbf{A}(k))\in\mathbb{R}^{K\times C}$.
A larger $\mu(\cdot)$ indicates a higher energy proportion in the spectrum, while a higher $\sigma(\cdot)$ denotes greater sample dispersion.
$S$ has two key properties:

- •
It captures the distribution of each frequency component across the entire training set.
This allows the forecasting model to learn the overall stability of each component across different samples.
- •
$S$ is a dimensionless measure that allows for a fair comparison between different frequency components, thus avoiding uniform frequency scaling, as defined in Theorem 1.

Frequency Stability Weighting Layer.
Given the input multivariate time series data $\mathbf{X}\in\mathbb{R}^{L\times C}$, we first apply a 1-D differencing operation to smooth the data and then transform $\mathbf{X}$ into the spectrum, decomposing it into the DFT coefficients:

$$ $\mathbf{F}(k,c)=\sum_{l=0}^{L-1}\Delta(\mathbf{X}(l,c))\cdot e^{-2\pi ikn/L}, \quad k=0,1,\ldots,K-1$ (8) $$

where $\Delta(\cdot)$ denotes the differencing operation, and two matrices are produced, representing the real and imaginary parts, $(\mathbf{F}_{r},\mathbf{F}_{i})$, formulated as:

$$ $\displaystyle\mathbf{F}_{r}(k,c)=\sum_{l=0}^{L-1}\mathbf{X}(l,c)\cdot\cos\left (\frac{2\pi kn}{L}\right)\quad\mathbf{F}_{i}(k,c)=-\sum_{l=0}^{L-1}\mathbf{X}( l,c)\cdot\sin\left(\frac{2\pi kn}{L}\right)$ (9) $$

Next, we apply two linear projections to $\mathbf{S}$ to the real and imaginary parts separately:

$$ $\mathbf{F}^{\prime}_{r}=\mathbf{F}_{r}\odot\left(\mathbf{S}\times\mathbf{W}_{r }+\mathbf{B}_{r}\right),\quad\mathbf{F}^{\prime}_{i}=\mathbf{F}_{i}\odot\left( \mathbf{S}\times\mathbf{W}_{i}+\mathbf{B}_{i}\right),$ (10) $$

where
$\mathbf{W}_{r}$ and $\mathbf{W}_{i}\in\mathbb{R}^{K\times 1}$ denote weight matrices for the $\mathbf{F}_{r}$ and $\mathbf{F}_{i}$, respectively.
$\mathbf{B}_{r},\mathbf{B}_{i}\in\mathbb{R}^{K}$ are bias vectors, and
$\odot$ denotes Hadamard product.
We handle the real and imaginary parts with separate networks because they correspond to different basis functions, allowing us to capture diverse temporal dynamics (Zhang et al., ; Piao et al., ).
Finally, we transform $(\mathbf{F}_{r},\mathbf{F}_{i})$, with enhanced stable frequency components, back into the time domain by:

$$ $\mathbf{X^{\prime}}(l,c)=\sum_{k=0}^{K-1}\left(\mathbf{F}^{\prime}_{r}(k,c) \cdot\cos\left(\frac{2\pi kl}{L}\right)-\mathbf{F}^{\prime}_{i}(k,c)\cdot\sin \left(\frac{2\pi kl}{L}\right)\right)$ (11) $$

where $\mathbf{X^{\prime}}$, with the same size as $\mathbf{X}\in\mathbb{R}^{L\times C}$, serves as the input for various forecasting models.

## 4 Experiments

**Table 1: Multivariate forecasting results (average) with forecasting lengths $H\in\{96,192,336,720\}$ for all datasets and fixed input sequence length $L=96$.**
| Models | PatchTST (Nie et al., ) | iTransformer (Liu et al., ) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | + Ours | Ori | + Ours | Ori |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| Electricity | 0.197 $\pm$ 0.027 | 0.296 $\pm$ 0.033 | 0.218 $\pm$ 0.31 | 0.307 $\pm$ 0.032 | 0.169 $\pm$ 0.035 | 0.262 $\pm$ 0.041 | 0.179 $\pm$ 0.28 | 0.279 $\pm$ 0.046 |
| ETTh1 | 0.438 $\pm$ 0.024 | 0.437 $\pm$ 0.035 | 0.480 $\pm$ 0.037 | 0.481 $\pm$ 0.031 | 0.445 $\pm$ 0.017 | 0.443 $\pm$ 0.026 | 0.511 $\pm$ 0.033 | 0.496 $\pm$ 0.036 |
| ETTh2 | 0.379 $\pm$ 0.032 | 0.380 $\pm$ 0.038 | 0.604 $\pm$ 0.130 | 0.524 $\pm$ 0.027 | 0.376 $\pm$ 0.041 | 0.400 $\pm$ 0.057 | 0.813 $\pm$ 0.134 | 0.666 $\pm$ 0.072 |
| ETTm1 | 0.390 $\pm$ 0.027 | 0.398 $\pm$ 0.025 | 0.419 $\pm$ 0.055 | 0.432 $\pm$ 0.047 | 0.396 $\pm$ 0.026 | 0.406 $\pm$ 0.056 | 0.447 $\pm$ 0.026 | 0.457 $\pm$ 0.061 |
| ETTm2 | 0.280 $\pm$ 0.032 | 0.325 $\pm$ 0.031 | 0.420 $\pm$ 0.035 | 0.424 $\pm$ 0.044 | 0.283 $\pm$ 0.020 | 0.327 $\pm$ 0.026 | 0.633 $\pm$ 0.055 | 0.489 $\pm$ 0.041 |
| Traffic | 0.427 $\pm$ 0.029 | 0.285 $\pm$ 0.025 | 0.619 $\pm$ 0.077 | 0.365 $\pm$ 0.029 | 0.424 $\pm$ 0.031 | 0.282 $\pm$ 0.027 | 0.576 $\pm$ 0.069 | 0.372 $\pm$ 0.035 |
| Weather | 0.251 $\pm$ 0.019 | 0.276 $\pm$ 0.017 | 0.255 $\pm$ 0.021 | 0.312 $\pm$ 0.031 | 0.246 $\pm$ 0.023 | 0.274 $\pm$ 0.017 | 0.274 $\pm$ 0.029 | 0.320 $\pm$ 0.041 |

### 4.1 Experimental Settings

Datasets.
We conducted experiments on seven public time series datasets, including Weather, four ETT repositories (ETTh1, ETTh2, ETTm1, ETTm2), Electricity (ECL), and Traffic dataset.
For example, the Electricity dataset includes the hourly electricity consumption record in 321 households.
All datasets are available in (Liu et al., ) .

Baselines.
We selected RevIN (Kim et al., ) and SAN (Liu et al., ) as our baselines.

- •
RevIN is widely used as a fundamental module in various forecasting models, including PatchTST (Nie et al., ), Crossformer (Zhang & Yan, ), iTransformer (Liu et al., ), Fredformer (Piao et al., ), among others (Wang et al., ).
- •
SAN (Liu et al., ) is the new state-of-the-art (SOTA) method, outperforming several non-stationary forecasting modules (Kim et al., ; Fan et al., ) and models (Liu et al., ).

Backbones and Setup.
For fair comparisons, we selected three forecasting models, including DLinear (Zeng et al., ), PatchTST (Nie et al., ), and iTransformer (Liu et al., ), as the backbone, and deployed all three modules (\method, RevIN, and SAN) for evaluation.
DLinear is a simple yet efficient forecasting model with an architecture solely involving MLPs.
PatchTST and iTransformer are two well-known Transformer methods that frequently serve as baselines in various forecasting research (Liu et al., ; ; Piao et al., ; Zhang et al., ).
We followed the implementation and setup provided in (Liu et al., )(^1^11https://github.com/icantnamemyself/SAN) and (Liu et al., )(^2^22https://github.com/thuml/Time-Series-Library).

Experiments Details.
We used mean squared error (MSE) and mean absolute error (MAE) as the evaluation metrics, where lower values indicate better performance.
All experiments were implemented on a single NVIDIA RTX A6000 48GB GPU with CUDA V12.4.
More details of the datasets are in Appendix B.1, the baselines are in B.2, the backbones and setup are in B.3, and other details of the experiments are in B.4.

### 4.2 Results

Main Results.
Table 1 presents the overall forecasting results using iTransformer and PatchTST as the backbone across seven datasets.
We set the forecasting lengths as $H\in\{96,192,336,720\}$, with the input sequence length $L=96$.
Here, we present the averaged MSE and MAE over four forecasting lengths.
We combine our module with a z-score normalization-denormalization operation in all experiments.
Obviously, applying \methodconsistently improved the performance of the backbone models across all datasets, as shown in all bold results.
More importantly, in datasets with complex frequency characteristics, such as ETTm2, \methodimproves PatchTST and iTransformer by 33.3% ($0.420\rightarrow 0.280$) and 55.3% ($0.633\rightarrow 0.283$), respectively.
This improvement is attributed to giving higher weights to stable frequency components, allowing them to dominate the adjusted input time series.

**Table 2: Multivariate forecasting results (average) with $H\in\{96,192,336,720\}$ for all datasets and fixed input sequence length $L=96$. The best and second best results are highlighted. Ours* represents the results where both \methodand SAN are used in the backbones.**
| Models | MLP-based (DLinear(Zeng et al., )) | Transformer-based (iTransformer(Liu et al., )) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | + Ours* | + Ours | +SAN | + RevIN | + Ours* | + Ours | +SAN | + RevIN |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| Electricity | 0.161 | 0.257 | 0.168 | 0.262 | 0.163 | 0.260 | 0.225 | 0.316 | 0.175 | 0.273 | 0.169 | 0.262 | 0.195 | 0.283 | 0.205 | 0.272 |
| ETTh1 | 0.413 | 0.424 | 0.407 | 0.419 | 0.421 | 0.427 | 0.460 | 0.456 | 0.455 | 0.449 | 0.445 | 0.443 | 0.466 | 0.455 | 0.463 | 0.452 |
| ETTh2 | 0.339 | 0.384 | 0.337 | 0.384 | 0.342 | 0.387 | 0.561 | 0.518 | 0.378 | 0.408 | 0.376 | 0.400 | 0.392 | 0.413 | 0.385 | 0.412 |
| ETTm1 | 0.341 | 0.372 | 0.357 | 0.375 | 0.344 | 0.376 | 0.413 | 0.407 | 0.389 | 0.398 | 0.396 | 0.406 | 0.401 | 0.406 | 0.406 | 0.410 |
| ETTm2 | 0.255 | 0.316 | 0.256 | 0.313 | 0.260 | 0.318 | 0.350 | 0.413 | 0.285 | 0.334 | 0.283 | 0.327 | 0.287 | 0.336 | 0.294 | 0.337 |
| Traffic | 0.432 | 0.297 | 0.430 | 0.291 | 0.440 | 0.302 | 0.624 | 0.383 | 0.459 | 0.313 | 0.424 | 0.282 | 0.520 | 0.341 | 0.430 | 0.312 |
| Weather | 0.224 | 0.271 | 0.237 | 0.272 | 0.227 | 0.276 | 0.265 | 0.317 | 0.244 | 0.282 | 0.246 | 0.274 | 0.247 | 0.291 | 0.263 | 0.288 |
| Count | 4 | 4 | 3 | 4 | 0 | 0 | 0 | 0 | 2 | 1 | 5 | 6 | 0 | 0 | 0 | 0 |

**Table 3: Detailed results of three selected datasets with $H\in\{96,192,336,720\}$ and input sequence length $L\in\{96,336\}$. The best and second best results are highlighted. Ours* represents the results where both \methodand SAN are used in the backbones.**
| Models | DLinear (Zeng et al., ) | iTransformer (Liu et al., ) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | + Ours* | + Ours | + SAN | + RevIN | + Ours* | + Ours | + SAN | + RevIN |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| Electricity | 96 | 0.135 | 0.230 | 0.140 | 0.237 | 0.137 | 0.234 | 0.210 | 0.278 | 0.145 | 0.244 | 0.143 | 0.237 | 0.171 | 0.262 | 0.152 | 0.251 |
| 192 | 0.149 | 0.245 | 0.155 | 0.249 | 0.151 | 0.247 | 0.210 | 0.304 | 0.169 | 0.266 | 0.159 | 0.252 | 0.180 | 0.270 | 0.264 | 0.255 |  |
| 336 | 0.165 | 0.262 | 0.171 | 0.267 | 0.166 | 0.264 | 0.223 | 0.309 | 0.178 | 0.271 | 0.172 | 0.266 | 0.194 | 0.284 | 0.180 | 0.272 |  |
| 720 | 0.198 | 0.291 | 0.208 | 0.298 | 0.201 | 0.295 | 0.257 | 0.349 | 0.210 | 0.311 | 0.205 | 0.295 | 0.237 | 0.319 | 0.227 | 0.312 |  |
| ETTh1 | 96 | 0.375 | 0.398 | 0.371 | 0.392 | 0.383 | 0.399 | 0.396 | 0.410 | 0.380 | 0.400 | 0.389 | 0.404 | 0.398 | 0.411 | 0.394 | 0.409 |
| 192 | 0.410 | 0.417 | 0.404 | 0.412 | 0.419 | 0.419 | 0.445 | 0.440 | 0.429 | 0.427 | 0.447 | 0.440 | 0.438 | 0.435 | 0.460 | 0.449 |  |
| 336 | 0.430 | 0.427 | 0.426 | 0.426 | 0.437 | 0.432 | 0.487 | 0.465 | 0.479 | 0.451 | 0.492 | 0.463 | 0.481 | 0.456 | 0.501 | 0.475 |  |
| 720 | 0.437 | 0.455 | 0.428 | 0.448 | 0.446 | 0.459 | 0.512 | 0.510 | 0.491 | 0.471 | 0.496 | 0.482 | 0.528 | 0.502 | 0.521 | 0.504 |  |
| Traffic | 96 | 0.410 | 0.286 | 0.408 | 0.277 | 0.412 | 0.288 | 0.648 | 0.396 | 0.400 | 0.271 | 0.394 | 0.268 | 0.502 | 0.329 | 0.401 | 0.277 |
| 192 | 0.427 | 0.288 | 0.422 | 0.283 | 0.429 | 0.297 | 0.598 | 0.370 | 0.470 | 0.319 | 0.413 | 0.277 | 0.490 | 0.331 | 0.421 | 0.282 |  |
| 336 | 0.439 | 0.305 | 0.436 | 0.295 | 0.445 | 0.306 | 0.605 | 0.373 | 0.489 | 0.333 | 0.428 | 0.283 | 0.512 | 0.341 | 0.434 | 0.389 |  |
| 720 | 0.454 | 0.311 | 0.455 | 0.311 | 0.474 | 0.319 | 0.645 | 0.395 | 0.478 | 0.330 | 0.463 | 0.301 | 0.576 | 0.364 | 0.465 | 0.302 |  |

Comparison with Baseline Normalization Methods.
Table 2 presents the average comparison results between \methodand the baseline normalization methods, i.e., RevIN and SAN.
We use the same parameters and forecasting length as in Table 1.
For iTransformer, the input sequence length is $L=96$, and $L=336$ for DLinear.
As shown, \method(denoted as ”Ours” in the table) achieves *18 top-1 results and 6 top-2 results out of 28 settings*.
For instance, n the ETTh1 dataset, \methodimproves the MSE values for DLinear and iTransformer to $0.407$ and $0.445$, outperforming RevIN ($0.460$ and $0.463$) and SAN ($0.421$ and $0.466$).
Similarly, in the Traffic dataset, \methodimproves the MSE value to $0.430$, compared to RevIN ($0.624$) and SAN ($0.440$).
Notably, as we highlighted in Section 1, one purpose of \methodis to complement existing normalization methods in the frequency domain.
Here, Ours* represents the incorporation of SAN into the backbones, which further improves the second-best results (underlined in the table) to the best.
For example, in the ETTh1 dataset with $H=96$ and $192$ on the iTransformer backbone, the results improved from $0.389$ and $0.447$ to $0.380$ and $0.429$.
Table 3 shows detailed results on three selected datasets, with all results provided in Appendix C.1.

Figure: Figure 3: Visualization of input sequences before and after applying \methodon the Traffic, ETTh1, and ETTh2 datasets. The green line shows the input data, the blue line represents the forecasting target, and the orange line illustrates the input data generated by \method. The red line represents the frequency stability measure of each dataset.
Refer to caption: x3.png

Frequency Stability Measure Analysis.
Figure 3 presents an empirical analysis visualizing the frequency stability measures across three datasets.
The green line and the blue line represent the amplitudes (FFT outputs) of the input series and forecasting target, respectively.
The orange line shows the adjusted input series using \method.
The red line represents the frequency stability measures, assigning higher weights to components with significant fluctuations appearing in both the input series and forecasting target, while down-weighting components with low amplitudes.
Meanwhile, we observe that these measures are adaptive to different datasets.
Interestingly, the fourth-shaped frequency component exhibits relatively consistent amplitudes, and although its amplitude value is lower, our metric assigns it a higher weight.
In the Electricity dataset, there is a degeneration trend in amplitude, but due to high consistency, the higher frequency components maintain a higher weight.

Running Time.
Figure 3 presents the running time results for \methodand the SOTA SAN across four datasets.
Full computation results for all seven datasets are available in Appendix C.2. We compare the computation time for both methods at the shortest forecasting length ($H=96$) and the longest ($H=720$). The results show the average computation time (in seconds per epoch) using DLinear and PatchTST as backbone models.
\methodconsistently outperforms SAN across all datasets.
Notably, we achieved improvements of *60% to 70% in 16 out of 28 settings* (see Appendix C.2).
These improvements are primarily due to the fact that \methodonly utilizes DFT and linear layers during the training phase, minimizing its impact on computation time.

Figure: Figure 4: Comparison of running times (s/epoch) between \methodand SAN on DLinear and PatchTST. Forecasting lengths $H\in\{96,720\}$ for all datasets and input sequence length $L=96$.
Refer to caption: x4.png

**Table 4: Results for each setting in the ablation study with forecasting lengths $H\in\{96,720\}$ for all datasets. The best results are highlighted.**
| Dataset | ETTh1 | ETTm1 | Weather |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Length | 96 | 720 | 96 | 720 | 96 | 720 |  |
| Ours | DLinear | 0.371 $\pm$ 0.032 | 0.428 $\pm$ 0.039 | 0.299 $\pm$ 0.021 | 0.425 $\pm$ 0.032 | 0.162 $\pm$ 0.045 | 0.325 $\pm$ 0.041 |
| iTransformer | 0.389 $\pm$ 0.023 | 0.496 $\pm$ 0.034 | 0.330 $\pm$ 0.035 | 0.475 $\pm$ 0.043 | 0.162 $\pm$ 0.044 | 0.340 $\pm$ 0.036 |  |
| Low-pass | DLinear | 0.379 $\pm$ 0.025 | 0.435 $\pm$ 0.036 | 0.305 $\pm$ 0.022 | 0.431 $\pm$ 0.031 | 0.170 $\pm$ 0.089 | 0.337 $\pm$ 0.024 |
| iTransformer | 0.401 $\pm$ 0.017 | 0.502 $\pm$ 0.029 | 0.335 $\pm$ 0.033 | 0.483 $\pm$ 0.047 | 0.171 $\pm$ 0.055 | 0.356 $\pm$ 0.039 |  |
| Random | DLinear | 0.393 $\pm$ 0.066 | 0.440 $\pm$ 0.087 | 0.308 $\pm$ 0.053 | 0.429 $\pm$ 0.079 | 0.171 $\pm$ 0.102 | 0.345 $\pm$ 0.065 |
| iTransformer | 0.407 $\pm$ 0.048 | 0.533 $\pm$ 0.055 | 0.339 $\pm$ 0.055 | 0.487 $\pm$ 0.061 | 0.172 $\pm$ 0.088 | 0.372 $\pm$ 0.059 |  |

Frequency Measure Ablation.
Table 4 shows the results of replacing the frequency stability measure with two alternative filters.
Our metric can be viewed as a filter that learns statistical measures across datasets and then applies filtering.
Since \methodmay focus on higher frequency components, we first compare it to a low-pass filter.
Additionally, we include frequency random selection, a selective method proposed by FEDformer (Zhou et al., ).
The results show that our frequency stability score consistently achieved the best accuracy, demonstrating that extracting stable features from the spectrum helps the model learn consistent patterns.

## 5 Related Works.

Time Series Forecasting.
Transformers have demonstrated significant success in time series forecasting (Nie et al., ; Zhang & Yan, ; Jiang et al., ), with early works focusing on improving computational efficiency (Li et al., ; Beltagy et al., ; Zhou et al., ; Liu et al., ) and recent works focusing on modeling temporal dependencies (Nie et al., ; Liu et al., ).
Some other works argue that understanding cross-channel correlations is critical for accurate forecasting.
Approaches utilizing Graph Neural Networks (GNNs) (Wu et al., ; Cao et al., ) and channel-wise Transformer-based frameworks like Crossformer (Zhang & Yan, ) and iTransformer (Liu et al., ) captures channel-wise dependencies for forecasting.
Normalization-based Methods.
RevIN (Kim et al., ) is an innovative normalization work for suppressing non-stationary.
It employs z-score normalization (i.e., mean of 0 and variance of 1) for input samples, then denormalizes the outputs using the same statistics.
Dish-TS (Fan et al., ) utilizes learned mean and variance for denormalization.
SAN (Liu et al., ) models non-stationary in a set of fine-grained sub-series and proposes an additional loss function to predict their statistics.
Instead of mean and variance, SIN (Han et al., ) proposes an independent neural network to learn features as the objectives of normalization and denormalization adaptively.
However, existing methods focus on modeling statistical variations in the time domain (Liu et al., ).
Frequency Analysis Methods.
Incorporating frequency information into models can improve forecasting (Wu et al., ; Zhou et al., ; Wang et al., ; Wu et al., ; Yi et al., ).
Recent study (Piao et al., ) recognizes a learning bias issue of frequency in the time domain modeling.
CoST (Woo et al., ) proposes a pre-training strategy to learn time-invariant representations in the frequency domain.
FiLM (Zhou et al., ) employs a low-rank approximation method to extract informative frequencies.
Koopa (Liu et al., ) introduces Koopman dynamics (Koopman, ) to learn time-invariant frequency features.
While promising, existing methods are costly and architecture-specific, limiting their generalization to other forecasting models.

## 6 Conclusion

This paper theoretically analyzed the effect of normalization methods on frequency components.
We proved that current time-domain normalization methods uniformly scale non-zero frequencies, making it difficult to identify components that contribute to robust forecasting.
To address this, we proposed \method, which analyzed datasets from a frequency perspective and adaptively up-weighted key frequency components.
\methodconsisted of two components: a statistical metric that normalized input samples based on frequency stability, and a learnable weighting layer that adjusted stability and introduced sample-specific variations.
Notably, \methodwas a plug-and-play module that maintained efficiency compared to existing normalization methods.
Extensive experiments showed that \methodreduced the average MSE of backbone forecasting models by 33.3% and 55.3% on the ETTm2 dataset.
Compared to baseline normalization methods, \methodachieved 18 top-1 and 6 top-2 results out of 28 settings.

## Appendix A Proofs

We now give the proof of Lemma and theorem in Sec.2.
We begin with the proof of Lemma 1.

### A.1 Proof of Lemma 1

Now, we provide proof of normalization in the time domain uniformly scales non-zero frequency components.

###### Lemma 1

(Time Domain Normalization Equates to Uniform Scaling in the Frequency Domain)

Normalization in the time domain uniformly scales non-zero frequency components.

Proof:

Let $\mathbf{X}(t)$ be a time series with mean $\mu(\mathbf{X})$ and standard deviation $\sigma(\mathbf{X})$. After applying z-score normalization, we obtain:

$$ $\mathbf{X}_{z}(t)=\frac{\mathbf{X}(t)-\mu(\mathbf{X})}{\sigma(\mathbf{X})}$ (12) $$

Applying the Fourier Transform $\mathcal{F}$ to $\mathbf{X}_{z}(t)$ and using the linearity property (Definition 3), we have:

$$ $\displaystyle\mathcal{F}[\mathbf{X}_{z}(t)]$ $\displaystyle=\mathcal{F}\left[\frac{\mathbf{X}(t)-\mu(\mathbf{X})}{\sigma( \mathbf{X})}\right]$ (13) $\displaystyle=\frac{1}{\sigma(\mathbf{X})}\mathcal{F}\left[\mathbf{X}(t)-\mu( \mathbf{X})\right]$ (14) $\displaystyle=\frac{1}{\sigma(\mathbf{X})}\left(\mathcal{F}[\mathbf{X}(t)]-\mu (\mathbf{X})\mathcal{F}[1]\right)$ (15) $$

Since $\mathcal{F}[1]$ is the Fourier Transform of the constant function $1$, which equals $\mathbb{1}\{k=0\}$ (the indicator function):

$$ $\mathcal{F}[1]=\mathbb{1}\{k=0\}$ (16) $$

The indicator function $\mathbb{1}\{k=0\}$ is nonzero only at $k=0$. Therefore, for non-zero frequency components ($k\neq 0$):

$$ $\displaystyle\mathcal{F}[\mathbf{X}_{z}(t)]$ $\displaystyle=\frac{1}{\sigma(\mathbf{X})}\left(\mathcal{F}[\mathbf{X}(t)]-\mu (\mathbf{X})\mathbb{1}\{k=0\}\right)$ (17) $\displaystyle=\frac{1}{\sigma(\mathbf{X})}\mathcal{F}[\mathbf{X}(t)]\quad\text {for }k\neq 0$ (18) $$

This shows that, for $k\neq 0$, the Fourier Transform of the normalized signal is the original Fourier Transform scaled by $\frac{1}{\sigma(\mathbf{X})}$.

Therefore, the amplitudes satisfy:

$$ $|\mathbf{A}_{z}(k)|=\frac{1}{\sigma(\mathbf{X})}|\mathbf{A}(k)|,\quad\text{for }k\neq 0$ (19) $$

Here, $|\mathbf{A}(k)|$ and $|\mathbf{A}_{z}(k)|$ are the amplitudes before and after normalization, respectively.

Thus, normalization in the time domain uniformly scales all non-zero frequency components by the factor $\frac{1}{\sigma(\mathbf{X})}$.

###### Lemma 1

### A.2 Proof of Theorem 1

Now, based on the proof of lemma 1, we provide proof of normalization Preserves the Proportion of $\mathcal{O}$ in the Spectrum.

###### Theorem 1

(Normalization Preserves the Proportion of $\mathcal{O}$ in the Spectrum)

The energy proportion of $\mathcal{O}$ in the spectrum is defined as the sum of its amplitudes divided by the sum of amplitudes of the entire spectrum. If Lemma 1 holds, then:

$$ $\frac{\sum_{k\in\mathcal{O}}|\mathbf{A}_{z}(k)|}{\sum_{k=1}^{K-1}|\mathbf{A}_{ z}(k)|}=\frac{\sum_{k\in\mathcal{O}}|\mathbf{A}(k)|}{\sum_{k=1}^{K-1}|\mathbf{ A}(k)|}$ (20) $$

The left side represents the ratio after normalization, and the right side represents the ratio before normalization. As shown, this ratio remains unchanged.

Proof:

From Lemma 1, for all non-zero frequencies $k\neq 0$, we have:

$$ $|\mathbf{A}_{z}(k)|=\frac{1}{\sigma(\mathbf{X})}|\mathbf{A}(k)|$ (21) $$

Therefore, the sum of amplitudes in $\mathcal{O}$ after normalization is:

$$ $\displaystyle\sum_{k\in\mathcal{O}}|\mathbf{A}_{z}(k)|$ $\displaystyle=\sum_{k\in\mathcal{O}}\frac{1}{\sigma(\mathbf{X})}|\mathbf{A}(k)|$ (22) $\displaystyle=\frac{1}{\sigma(\mathbf{X})}\sum_{k\in\mathcal{O}}|\mathbf{A}(k)|$ (23) $$

Similarly, the sum of amplitudes of the entire spectrum (excluding $k=0$) after normalization is:

$$ $\displaystyle\sum_{k=1}^{K-1}|\mathbf{A}_{z}(k)|$ $\displaystyle=\sum_{k=1}^{K-1}\frac{1}{\sigma(\mathbf{X})}|\mathbf{A}(k)|$ (24) $\displaystyle=\frac{1}{\sigma(\mathbf{X})}\sum_{k=1}^{K-1}|\mathbf{A}(k)|$ (25) $$

Calculating the energy proportion after normalization:

$$ $\displaystyle\frac{\sum_{k\in\mathcal{O}}|\mathbf{A}_{z}(k)|}{\sum_{k=1}^{K-1} |\mathbf{A}_{z}(k)|}$ $\displaystyle=\frac{\frac{1}{\sigma(\mathbf{X})}\sum_{k\in\mathcal{O}}|\mathbf {A}(k)|}{\frac{1}{\sigma(\mathbf{X})}\sum_{k=1}^{K-1}|\mathbf{A}(k)|}$ (26) $\displaystyle=\frac{\sum_{k\in\mathcal{O}}|\mathbf{A}(k)|}{\sum_{k=1}^{K-1}| \mathbf{A}(k)|}$ (27) $$

The scaling factor $\frac{1}{\sigma(\mathbf{X})}$ cancels out in the numerator and denominator. Therefore, the energy proportion of $\mathcal{O}$ remains the same after normalization.

###### Theorem 1

## Appendix B Details of the experiments

### B.1 Details of the datasets.

Weather contains 21 channels (e.g., temperature and humidity) and is recorded every 10 minutes in 2020.
ETT (Zhou et al., ) (Electricity Transformer Temperature) consists of two hourly-level datasets (ETTh1, ETTh2) and two 15-minute-level datasets (ETTm1, ETTm2).
Electricity (Lai et al., ), from the UCI Machine Learning Repository and preprocessed by, is composed of the hourly electricity consumption of 321 clients in kWh from 2012 to 2014.
Solar-Energy (Lai et al., ) records the solar power production of 137 PV plants in 2006, sampled every 10 minutes.
Trafﬁc contains hourly road occupancy rates measured by 862 sensors on San Francisco Bay area freeways from January 2015 to December 2016.
More details of these datasets can be found in Table.5.

**Table 5: Overview of Datasets**
| Dataset | Source | Resolution | Channels | Time Range |
| --- | --- | --- | --- | --- |
| Weather | Autoformer(Wu et al., ) | Every 10 minutes | 21 (e.g., temperature, humidity) | 2020 |
| ETTh1 | Informer(Zhou et al., ) | Hourly | 7 states of a electrical transformer | 2016-2017 |
| ETTh2 | Informer(Zhou et al., ) | Hourly | 7 states of a electrical transformer | 2017-2018 |
| ETTm1 | Informer(Zhou et al., ) | Every 15 minutes | 7 states of a electrical transformer | 2016-2017 |
| ETTm2 | Informer(Zhou et al., ) | Every 15 minutes | 7 states of a electrical transformer | 2017-2018 |
| Electricity | UCI ML Repository | Hourly | 321 clients’ consumption | 2012-2014 |
| Traffic | Informer(Zhou et al., ) | Hourly | 862 sensors’ occupancy | 2015-2016 |

### B.2 Details of the baselines

Reversible Instance Normalization.
Reversible Instance Normalization (Revin) normalizes each input sample using z-score normalization while preserving the original mean and variance.
Revin reverses the normalization to model outputs by using the saved statistics and applies learnable scaling and shifting parameters ($\gamma$ and $\beta$).

Figure: Algorithm 3 Reversible Instance Normalization (Revin)

Sequential Adaptive Normalization.
Sequential Adaptive Normalization (SAN) has two train phases.
In the first phase, SAN is trained to learn the relationships between patches of input and target data by mapping their means and variances.
In the second phase, SAN parameters are frozen, and only the forecasting model is trained. During inference, input data is normalized using SAN, and the model output is reverse-normalized with predicted statistics by SAN.

Figure: Algorithm 4 Sequential Adaptive Normalization (SAN)

### B.3 Details of the backbones and setup

In our study, we selected three distinct forecasting models to evaluate the effectiveness of our proposed normalization techniques.
DLinear is an MLP-based model renowned for its lightweight architecture, utilizing two separate multilayer perceptrons (MLPs) to learn the periodic and trend components of the data independently.

PatchTST and iTransformer are both Transformer-based models with unique approaches to handling time-series data.
PatchTST introduces a patching operation that segments each input time series into multiple patches, which are then used as input tokens for the transformer, effectively capturing local temporal patterns.
In contrast, iTransformer emphasizes channel-wise attention by treating the entire sequence of each channel as a transformer token and employing self-attention mechanisms to learn the relationships between different channels.

For all models, we first compute the frequency stability measure across the entire training dataset, a fixed computational process that typically takes less than one second.
Following this, we apply a simple, parameter-free normalization and denormalization method.
After normalization, the input data is processed through our custom weighting layer before being fed into the forecasting models.

### B.4 Other experiments details

Loss Function.
For our experiments, we adhere to a conventional approach by employing the Mean Squared Error (MSE) loss function, implemented as nn.MSELoss in our framework. The MSE loss quantifies the average squared difference between the predicted values and the actual target values, providing a straightforward measure of prediction accuracy. Mathematically, the MSE loss is expressed as
$\mathcal{L}_{\text{MSE}}=\frac{1}{N}\sum_{i=1}^{N}\left(\hat{y}_{i}-y_{i}
\right)^{2}$,
where $N$ is the number of samples, $\hat{y}_{i}$ represents the predicted value, and $y_{i}$ denotes the true target value for the $i$-th sample. This loss function effectively penalizes larger errors more heavily, encouraging the model to achieve higher precision in its predictions.

Computational Resources.
All experiments were conducted on an NVIDIA RTX A6000 GPU with 48GB of memory, utilizing CUDA version 12.4 for accelerated computation. This high-performance computational setup facilitated efficient training and evaluation of our forecasting models, ensuring timely execution of experiments even with large-scale time-series data.

## Appendix C The Full Results.

### C.1 Full Long-term Forecasting results.

Table C.1 presents the comprehensive results discussed in Section 4.2 of our paper.
This table includes the prediction accuracy outcomes on the [Electricity, ETTh1, ETTh2, ETTm1, ETTm2, Traffic, Weather] dataset, utilizing the [DLinear, PatchTST, iTransformer] as the backbone model.
We have compared our method against all baseline models across all forecasting horizons ($H\in\{96,192,336,720\}$).

**Table 6: Detailed results of comparing our proposal and other normalization methods. The best results are highlighted in bold.**
| Models | DLinear (Zeng et al., ) | PatchTST (Nie et al., ) | iTransformer (Liu et al., ) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | + Ours* | + Ours | + SAN | + RevIN | + Ours* | + Ours | + SAN | + RevIN | + Ours* | + Ours | + SAN | + RevIN |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| Electricity | 96 | 0.135 | 0.230 | 0.140 | 0.237 | 0.137 | 0.234 | 0.210 | 0.278 | 0.175 | 0.266 | 0.190 | 0.280 | 0.182 | 0.271 | 0.212 | 0.297 | 0.145 | 0.244 | 0.143 | 0.237 | 0.171 | 0.262 | 0.152 | 0.251 |
| 192 | 0.149 | 0.245 | 0.155 | 0.249 | 0.151 | 0.247 | 0.210 | 0.304 | 0.183 | 0.273 | 0.195 | 0.286 | 0.186 | 0.276 | 0.213 | 0.300 | 0.169 | 0.266 | 0.159 | 0.252 | 0.180 | 0.270 | 0.165 | 0.255 |  |
| 336 | 0.165 | 0.262 | 0.171 | 0.267 | 0.166 | 0.264 | 0.223 | 0.309 | 0.198 | 0.289 | 0.211 | 0.301 | 0.200 | 0.290 | 0.227 | 0.314 | 0.178 | 0.271 | 0.172 | 0.266 | 0.194 | 0.284 | 0.180 | 0.272 |  |
| 720 | 0.198 | 0.291 | 0.208 | 0.298 | 0.201 | 0.295 | 0.257 | 0.349 | 0.233 | 0.317 | 0.253 | 0.334 | 0.237 | 0.322 | 0.268 | 0.344 | 0.210 | 0.311 | 0.205 | 0.295 | 0.237 | 0.319 | 0.227 | 0.312 |  |
| ETTh1 | 96 | 0.375 | 0.398 | 0.371 | 0.392 | 0.383 | 0.399 | 0.396 | 0.410 | 0.380 | 0.401 | 0.374 | 0.395 | 0.387 | 0.405 | 0.392 | 0.413 | 0.380 | 0.400 | 0.389 | 0.404 | 0.398 | 0.411 | 0.394 | 0.409 |
| 192 | 0.410 | 0.417 | 0.404 | 0.412 | 0.419 | 0.419 | 0.445 | 0.440 | 0.442 | 0.439 | 0.424 | 0.428 | 0.445 | 0.440 | 0.448 | 0.436 | 0.429 | 0.427 | 0.447 | 0.440 | 0.438 | 0.435 | 0.460 | 0.449 |  |
| 336 | 0.430 | 0.427 | 0.426 | 0.426 | 0.437 | 0.432 | 0.487 | 0.465 | 0.480 | 0.456 | 0.471 | 0.452 | 0.505 | 0.471 | 0.489 | 0.456 | 0.479 | 0.451 | 0.492 | 0.463 | 0.481 | 0.456 | 0.501 | 0.475 |  |
| 720 | 0.437 | 0.455 | 0.428 | 0.448 | 0.446 | 0.459 | 0.512 | 0.510 | 0.519 | 0.501 | 0.514 | 0.500 | 0.527 | 0.507 | 0.525 | 0.503 | 0.491 | 0.471 | 0.496 | 0.482 | 0.528 | 0.502 | 0.521 | 0.504 |  |
| ETTh2 | 96 | 0.273 | 0.335 | 0.273 | 0.336 | 0.277 | 0.338 | 0.344 | 0.397 | 0.292 | 0.347 | 0.301 | 0.349 | 0.314 | 0.361 | 0.344 | 0.397 | 0.298 | 0.352 | 0.297 | 0.345 | 0.302 | 0.354 | 0.300 | 0.349 |
| 192 | 0.335 | 0.374 | 0.336 | 0.376 | 0.340 | 0.378 | 0.485 | 0.481 | 0.385 | 0.402 | 0.380 | 0.399 | 0.391 | 0.421 | 0.389 | 0.411 | 0.371 | 0.402 | 0.380 | 0.395 | 0.383 | 0.402 | 0.381 | 0.415 |  |
| 336 | 0.361 | 0.399 | 0.355 | 0.395 | 0.356 | 0.398 | 0.582 | 0.536 | 0.431 | 0.438 | 0.410 | 0.424 | 0.444 | 0.466 | 0.437 | 0.451 | 0.425 | 0.435 | 0.420 | 0.428 | 0.435 | 0.441 | 0.433 | 0.442 |  |
| 720 | 0.388 | 0.429 | 0.384 | 0.423 | 0.396 | 0.435 | 0.836 | 0.659 | 0.429 | 0.461 | 0.422 | 0.443 | 0.467 | 0.484 | 0.430 | 0.481 | 0.420 | 0.444 | 0.410 | 0.432 | 0.448 | 0.457 | 0.426 | 0.445 |  |
| ETTm1 | 96 | 0.285 | 0.339 | 0.299 | 0.341 | 0.288 | 0.342 | 0.353 | 0.374 | 0.322 | 0.359 | 0.321 | 0.362 | 0.325 | 0.361 | 0.353 | 0.374 | 0.326 | 0.361 | 0.330 | 0.370 | 0.331 | 0.373 | 0.341 | 0.376 |
| 192 | 0.321 | 0.359 | 0.336 | 0.364 | 0.323 | 0.363 | 0.391 | 0.392 | 0.350 | 0.379 | 0.365 | 0.388 | 0.355 | 0.381 | 0.391 | 0.401 | 0.365 | 0.384 | 0.374 | 0.391 | 0.376 | 0.381 | 0.380 | 0.394 |  |
| 336 | 0.355 | 0.380 | 0.370 | 0.383 | 0.357 | 0.384 | 0.423 | 0.413 | 0.381 | 0.401 | 0.407 | 0.408 | 0.385 | 0.402 | 0.423 | 0.413 | 0.395 | 0.403 | 0.408 | 0.414 | 0.412 | 0.418 | 0.419 | 0.418 |  |
| 720 | 0.405 | 0.411 | 0.425 | 0.414 | 0.409 | 0.415 | 0.486 | 0.449 | 0.446 | 0.436 | 0.464 | 0.442 | 0.450 | 0.437 | 0.486 | 0.459 | 0.471 | 0.447 | 0.475 | 0.449 | 0.485 | 0.453 | 0.486 | 0.455 |  |
| ETTm2 | 96 | 0.163 | 0.255 | 0.165 | 0.254 | 0.166 | 0.258 | 0.194 | 0.293 | 0.177 | 0.272 | 0.179 | 0.262 | 0.184 | 0.277 | 0.185 | 0.272 | 0.178 | 0.272 | 0.176 | 0.258 | 0.180 | 0.272 | 0.200 | 0.281 |
| 192 | 0.222 | 0.300 | 0.220 | 0.291 | 0.223 | 0.302 | 0.283 | 0.360 | 0.245 | 0.319 | 0.240 | 0.300 | 0.249 | 0.325 | 0.252 | 0.320 | 0.247 | 0.311 | 0.241 | 0.302 | 0.248 | 0.315 | 0.252 | 0.312 |  |
| 336 | 0.272 | 0.329 | 0.273 | 0.325 | 0.272 | 0.331 | 0.371 | 0.450 | 0.298 | 0.253 | 0.310 | 0.347 | 0.330 | 0.378 | 0.315 | 0.351 | 0.307 | 0.351 | 0.307 | 0.347 | 0.308 | 0.352 | 0.314 | 0.352 |  |
| 720 | 0.365 | 0.383 | 0.368 | 0.383 | 0.380 | 0.384 | 0.555 | 0.509 | 0.405 | 0.401 | 0.409 | 0.404 | 0.423 | 0.431 | 0.415 | 0.408 | 0.409 | 0.403 | 0.410 | 0.402 | 0.412 | 0.407 | 0.411 | 0.405 |  |
| Traffic | 96 | 0.410 | 0.286 | 0.408 | 0.277 | 0.412 | 0.288 | 0.648 | 0.396 | 0.497 | 0.342 | 0.527 | 0.339 | 0.530 | 0.340 | 0.650 | 0.396 | 0.400 | 0.271 | 0.394 | 0.268 | 0.502 | 0.329 | 0.401 | 0.277 |
| 192 | 0.427 | 0.288 | 0.422 | 0.283 | 0.429 | 0.297 | 0.598 | 0.370 | 0.499 | 0.339 | 0.502 | 0.331 | 0.516 | 0.338 | 0.597 | 0.359 | 0.470 | 0.319 | 0.413 | 0.277 | 0.490 | 0.331 | 0.421 | 0.282 |  |
| 336 | 0.439 | 0.305 | 0.436 | 0.295 | 0.445 | 0.306 | 0.605 | 0.373 | 0.520 | 0.349 | 0.510 | 0.327 | 0.533 | 0.343 | 0.605 | 0.362 | 0.489 | 0.333 | 0.428 | 0.283 | 0.512 | 0.341 | 0.434 | 0.389 |  |
| 720 | 0.454 | 0.311 | 0.455 | 0.311 | 0.474 | 0.319 | 0.645 | 0.395 | 0.550 | 0.349 | 0.545 | 0.345 | 0.575 | 0.367 | 0.642 | 0.381 | 0.478 | 0.330 | 0.463 | 0.301 | 0.576 | 0.364 | 0.465 | 0.302 |  |
| Weather | 96 | 0.150 | 0.208 | 0.162 | 0.212 | 0.152 | 0.210 | 0.196 | 0.256 | 0.167 | 0.225 | 0.166 | 0.207 | 0.170 | 0.229 | 0.195 | 0.235 | 0.165 | 0.221 | 0.162 | 0.204 | 0.170 | 0.227 | 0.175 | 0.225 |
| 192 | 0.194 | 0.251 | 0.207 | 0.251 | 0.196 | 0.254 | 0.238 | 0.299 | 0.208 | 0.263 | 0.216 | 0.253 | 0.211 | 0.270 | 0.240 | 0.270 | 0.212 | 0.261 | 0.213 | 0.252 | 0.214 | 0.270 | 0.225 | 0.257 |  |
| 336 | 0.243 | 0.289 | 0.256 | 0.288 | 0.246 | 0.294 | 0.281 | 0.330 | 0.255 | 0.301 | 0.273 | 0.295 | 0.261 | 0.310 | 0.291 | 0.306 | 0.261 | 0.304 | 0.271 | 0.295 | 0.265 | 0.309 | 0.280 | 0.307 |  |
| 720 | 0.311 | 0.339 | 0.325 | 0.337 | 0.315 | 0.346 | 0.346 | 0.384 | 0.326 | 0.349 | 0.351 | 0.346 | 0.332 | 0.359 | 0.364 | 0.353 | 0.338 | 0.345 | 0.340 | 0.347 | 0.342 | 0.358 | 0.373 | 0.366 |  |

### C.2 Full Results of Running Time

Table 7 presents the comprehensive results discussed in Section 4.2 of our paper.
This table includes the prediction accuracy outcomes on the [Electricity, ETTh1, ETTh2, ETTm1, ETTm2, Traffic, Weather] dataset, utilizing the [DLinear, PatchTST] as the backbone model.
We have compared our method against all baseline models across all forecasting horizons ($H\in\{96,720\}$).

**Table 7: Running time comparison with forecasting lengths $H\in\{96,720\}$ for all datasets and fixed input sequence length $L=96$. The best results are highlighted.**
| Model | Electricity | ETTh1 | ETTh2 | ETTm1 | ETTm2 | Traffic | Weather |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| H | 96 | 720 | 96 | 720 | 96 | 720 | 96 | 720 | 96 | 720 | 96 | 720 | 96 | 720 |
| DLinear (Zeng et al., ) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| + Ours | 16.718 | 27.545 | 3.004 | 3.082 | 2.820 | 3.222 | 13.456 | 13.124 | 11.839 | 12.949 | 23.217 | 43.413 | 13.970 | 16.536 |
| + SAN | 19.159 | 34.914 | 8.688 | 8.054 | 8.373 | 7.811 | 36.706 | 36.564 | 36.620 | 36.550 | 31.378 | 54.518 | 37.739 | 40.731 |
| IMP(%) | 12.8% | 21.1% | 65.5% | 60.5% | 66.6% | 58.3% | 63.1% | 64.2% | 67.8% | 64.3% | 26.0% | 20.3% | 63.0% | 59.9% |
| PatchTST (Nie et al., ) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| + Ours | 99.104 | 103.781 | 7.952 | 8.215 | 14.687 | 14.122 | 54.526 | 28.944 | 59.406 | 26.233 | 209.006 | 215.718 | 69.107 | 49.628 |
| + SAN | 313.697 | 322.083 | 16.226 | 15.309 | 16.078 | 14.998 | 63.686 | 65.839 | 65.978 | 66.798 | 550.026 | 557.730 | 81.973 | 81.462 |
| IMP(%) | 68.4% | 67.7% | 51.6% | 46.2% | 8.00% | 5.20% | 14.3% | 56.0% | 10.0% | 60.7% | 61.6% | 61.2% | 15.9% | 39.8% |

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: See weakness.
soundness: 2
strengths: 1. this paper studies an interesting problem in time series forecasting, i.e., the distribution shift problem.

2. the paper is understandable and easy to follow.
summary: This paper reveals that traditional time-domain normalization methods uniformly scale non-zero frequencies, which limits their ability to effectively handle distribution shifts in time series forecasting. To address this limitation, the authors propose FredNormer, a plug-and-play module that combines statistical frequency stability normalization with learnable sample-specific weighting, enabling better adaptation to key frequency components and more robust forecasting performance.
weaknesses: 1. this paper is not very well-motivated. The paper mentioned that "Modeling solely in the time domain struggles to distinguish between different frequency components within superimposed time series" as their first motivation. Does it have any difference with the distribution shift or non-stationary forecasting? Also, it mentioned that "the z-score normalization applies uniform scaling across all frequency components, which leaves frequency-specific patterns unaltered". I believe only RevIN uses "learnable" z-score normalization but other methods like SIN, SAN, FAN do not use z-score normalization. What they do cannot be equal to z-score normalization. So how can you use z-score to summarize these works?

2. this paper is kind of confusing for the theoretical analysis. Based on the Theorem 1, the authors mentioned that "the normalization operation keeps the proportion unchanged". The proportion is unchanged does not mean it cannot normalize the time series. So the theoretical analysis cannot show the existing normalization would fail. Moreover, how can you say your defined stability is correlated with the non-sationarity? Why the frequency components are entangled and thus influence normalization? These parts don't have theoretical analysis I believe.

3. the experiments are somehow lack of comparisons with state-of-the-art normalization techniques, such as SIN [1].

[1] SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting. In ICML.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Regarding the contribution, refer to W1.

2. For the experiment, please see W4 and W5.
soundness: 3
strengths: 1. The authors provide a theoretical proof showing that time-domain normalization is not effective for frequency components.
2. The proposed method is a simple and effective method for learning time-invariant frequency components to suppress non-stationary in the frequency domain.
3. FredNormer is demonstrated to be effective, achieving state-of-the-art (SOTA) performance in experiments.
summary: This paper focuses on the issue of distribution shift in the frequency domain to improve non-stationary time series forecasting. Its primary contribution lies in the theoretical analysis of how normalization methods impact frequency components and in adaptively up-weighting key frequency components. Specifically, the proposed method, FredNormer, includes a statistical metric to normalize input samples and a learnable weighting layer to adjust stability.
weaknesses: 1. FAN is also a method based on adaptive normalization in the frequency domain, capable of handling both trend and seasonal non-stationary patterns in time series data. Compared with it, although this paper provides a theoretical guarantee, its contribution appears insufficient. 

[1] Ye W, Deng S, Zou Q, et al. Frequency Adaptive Normalization For Non-stationary Time Series Forecasting[J]. Advances in Neural Information Processing Systems, 2024.

2. In Non-stationary Transformer, the authors argue that removing inherent non-stationarity from time series may reduce the model's ability to forecast real-world bursty events. Could suppressing non-stationary information in FredNormer similarly lead to over-stationarization, limiting its practical applicability? 

[1] Liu Y, Wu H, Wang J, et al. Non-stationary transformers: Exploring the stationarity in time series forecasting[J]. Advances in Neural Information Processing Systems, 2022, 35: 9881-9893.

3. In Definition 2, you mention "M components with higher stability S(k)." How do you define "higher" stability? What is the threshold value used, and how is it selected? 

4. The authors only compare against with Transformer-based and MLP-based methods. Why not include comparisons with more CNN-based baselines, such as TSLANet, ModernTCN, and TimesNet, to demonstrate the robustness and generalization of FredNormer?

[1] Eldele E, Ragab M, Chen Z, et al. TSLANet: Rethinking Transformers for Time Series Representation Learning[C]//Forty-first International Conference on Machine Learning, 2024.

[2] Luo D, Wang X. Moderntcn: A modern pure convolution structure for general time series analysis[C]//The Twelfth International Conference on Learning Representations. 2024.

[3] Wu H, Hu T, Liu Y, et al. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis[C]//The Eleventh International Conference on Learning Representations, 2023.

5. I am curious about FredNormer’s impact on frequency-domain methods like FiTS and FreTS. Have you tested its performance on these models?

Minor Error:
1. Section 4.2 mentions running time, but Figure 4 seems to be missing. 
2. In Section 4.2, you refer to a "fourth-shaped frequency component." Could you clarify what you mean by this?
3. In the notations section, the indicator function I{k=0} is said to equal 1 if k is not equal to 0. 
In Proof 1, you state that for k≠1,I{k=0}=0. Could you clarify which case is correct?

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: * In the remark on line 196, the authors note that intermixed stable and unstable components lead to entangled patterns and thus lead to sub-optimal forecasting performance of current models. Could the authors clarify how their proposed model differentiates between stable and unstable components?

* What is the purpose of smoothing the data before applying the discrete Fourier transform in Equation (8)? How might the performance of the proposed model be impacted if the transform were applied directly to the raw data?

Also, see the weaknesses.
soundness: 2
strengths: * The authors define Frequency Stability $\mathrm{S}$ across the entire dataset based on the discrete Fourier transform coefficients, and they theoretically demonstrate that time-domain normalization can only uniformly scale non-zero frequency components, leaving the proportion of the Stable Frequency Subset $\mathcal{O}$ within the spectrum unchanged after normalization.

* The authors propose to enhance stable frequency components for better generalization in TS forecasting. They introduce a learnable layer that assigns greater weights to stable frequency elements based on $\mathrm{S}$.
summary: The paper notes that normalization-based methods can only partially address the distribution shift problem in non-stationary time series forecasting, as these methods operate solely in the time domain and may overlook prominent dynamic patterns in the frequency domain. To tackle this, they propose FredNormer, a plug-and-play module that dynamically adjusts the weight of each frequency component based on a proposed measure termed Frequency Stability $\mathrm{S}$.
weaknesses: * The contribution of the paper seems limited. According to equation (10), the core approach mainly involves applying a linear transformation to frequency components based on the matrix $\mathrm{S}$. It is unclear how $\mathrm{S}$ functions as a constraint to realize the key motivation of enhancing the weight of stable frequency components.

* In Definition 2, the authors introduce the concept of a Stable Frequency Subset $\mathcal{O}$ but do not provide a clear criterion for determining what qualifies as a stable frequency. Some notations, such as Stable Frequency Subset $\mathcal{O}$, and Theorem 1 presented in Section 2 are not well linked to the design of the method proposed in Section 3.

* The experimental evaluation lacks thoroughness, as it includes only three baseline models (Dlinear, PatchTST, and iTransformer). Although the authors mention other models, such as CrossFormer, they do not include it in their comparisons. Additionally, the Nonstationary Transformer [1] would also be a relevant model for further comparison.

* The empirical analysis in Figure 3 is confusing to the reviewer. Could the authors provide additional clarification on how the adjustments to the amplitudes of the input series and forecasting target, based on the Frequency Stability score, affect model prediction accuracy? Additionally, could you explain why these adjustments are effective in enhancing the model's performance? Both Equations (9) and (10) have large spacing from the preceding text.

[1] Liu, Yong, et al. "Non-stationary transformers: Exploring the stationarity in time series forecasting." Advances in Neural Information Processing Systems 35 (2022): 9881-9893.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: please refer to the weakness
soundness: 3
strengths: 1. This paper is well-written and easy to follow.

2. Extensive experiments have demonstrated that FredNormer enhances the performance of backbone forecasting models, resulting in notable improvements.
summary: This paper presents FredNormer, a method designed to tackle the distribution shift issue in the frequency domain. It comprises two components: a statistical metric and a learnable weighting layer. Extensive experiments demonstrate that FredNormer enhances forecasting performance.
weaknesses: My main concern is regarding the contribution of this work. What are the key differences between FredNormer and the two referenced papers [1][2]?



[1] Frequency Adaptive Normalization For Non-stationary Time Series Forecasting, in NeurIPS 2024


[2] Deep Frequency Derivative Learning for Non-stationary Time Series Forecasting, in IJCAI 2024

---

confidence: 5
contribution: 3
initial_rating: 5
presentation: 3
questions: 1. Why is there a lack of discussion and explanation on the introduction of the first-order difference in the paper, especially when it does not align with the proofs provided for Algorithm 1 and Algorithm 2?

2. In the Frequency Stability Measure Analysis, why are there no comparisons with other methods in the ablation experiments, specifically when the proposed FredNormer method is being evaluated?

3. The experimental results for the reproduction of iTransformer and PatchTST in the current paper appear significantly inferior to those reported in their original papers. When comparing the FredFormer's experimental results to those in other papers, there seems to be no notable improvement.

Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., & Long, M. (2023). itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625.
Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2022). A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730.
soundness: 2
strengths: The paper is generally well-structured, with a clear progression from introduction to conclusion. The use of technical language is appropriate for the audience, and the figures and tables are mostly supportive of the text.

The authors propose FredNormer, a frequency domain normalization method that adaptively up-weights key frequency components based on their stability. The method is model-agnostic and can be integrated into various forecasting models without compromising efficiency. 

This paper tackles an important problem in time series forecasting—the non-stationarity of real-world data. The idea of approaching normalization from the frequency domain is novel and could potentially offer new insights into the field.
summary: This research tackles a fundamental challenge in time series forecasting—the non-stationarity of real-world data—by introducing a novel perspective on normalization within the frequency domain. The authors propose a versatile plug-and-play module that analyzes datasets from a frequency standpoint and dynamically enhances the significance of key frequency components. Comprehensive experiments demonstrate that the FredNormer module enhances the average Mean Squared Error (MSE) of various underlying forecasting models.
weaknesses: The importance of the questions posed by the paper is not in doubt, but the execution and resulting conclusions may be less reliable due to experimental concerns. The technical claims of the paper are where the most significant concerns lie. The experimental methodology appears to have several flaws, which may compromise the validity of the results. Without a solid experimental foundation, the technical claims, no matter how innovative, cannot be adequately supported. The paper would benefit from a thorough revision of the experimental design and possibly additional experiments to validate the findings.  

The initial discussion in the article pertains to the equivalence of linear combinations before and after applying the Fast Fourier Transform (FFT). However, this proof fails to address the first-order differences introduced in the practical algorithmic procedure. Specifically, Algorithm 1 computes a quantity $S$ based on the batch mean and variance of frequency domain points derived from the input sequence. Conversely, Algorithm 2 utilizes this $S$ to perform a linear transformation on the frequency domain representation of the first-order differences of the sequence. It is evident that the actual algorithmic workflow does not fully align with the theoretical proof provided.

When evaluating the experimental runtime, it is important to note that if the speed is not measured during the stable phase of training, factors such as initial data loading during training runs may obscure the speed reduction effect attributable to the FredNormer plugin.

The potential contribution of the paper to the field is significant if the method can be validated. However, as it stands, the experimental issues limit the trust that can be placed in the results.