Year: 2021

Abstract:
The statistical properties in a time-series often change, i.e., the data distributions differ over time. This temporal distribution change is one of the main challenges that prevent accurate time-series forecasting. To address this issue, we propose a simple but effective normalization method called reversible instance normalization (RevIN). To be specific, RevIN consists of two different steps, normalization and denormalization. The former normalizes the input to fix its distribution in terms of the mean and variance, while the latter returns the output to the original distribution. In addition, RevIN is model-agnostic, generally applicable to various time-series forecasting models with significant improvements in forecasting performance. As shown in Fig. 1, RevIN effectively enhances the performance of the baselines. Our extensive experimental results verify the general applicability and performance improvements on various real-world datasets.

heading: 1 INTRODUCTION
text: Time-series forecasting plays a significant role in various daily problems such as health care, economics, and traffic engineering (Che et al., 2018; Bauer et al., 2016; Zhang et al., 2017). Recently, time-series forecasting models have achieved comparative performance on these problems by overcoming several challenges, e.g., long-term forecasting (Zhou et al., 2021) and missing value im-
putation (Tang et al., 2020). However, these time-series forecasting models often severely suffer from statistical properties that change over time, which is also widely known as a distribution shift problem; for example, the mean and variance of the time-series data distribution vary over time. Accordingly, the input sequences to the forecasting models have different underlying distributions from each other. In the case of the train and test data, they are usually divided into data before and after a specific point in time. Thus, the distributions of the two data often hardly overlap, which is commonly known as the reason for model performance degradation. Thus, we assume that if we normalize every sequence in the data to have identical mean and variance and provide the transformed data to the model, the discrepancy in the distribution will be reduced, thereby improving the model performance. However, if the input to the model is shifted and scaled, the prediction of the model can be shifted and scaled as much as the input; since this will differ from the groundtruth distribution, the accuracy of the model will decrease. Also, we further hypothesize that if the model output, shifted and scaled, can be returned to its original distribution explicitly, we can alleviate the discrepancy in the distribution of the data during the forward pass of the model as well as allow its prediction to obey the original distribution. Here, reconstructing the original distribution can be achieved by applying denormalization on the output of the model as a reversed function of the normalization applied on the input data. Inspired by these, we propose a simple yet effective normalization method, reversible instance normalization (RevIN), which first normalizes the input sequences and then denormalizes the model output sequences to solve the time-series forecasting task against the distribution shift problem. RevIN is symmetrically structured to return the model output to the original distribution in the denormalization layer by scaling and shifting the model output as much as the input data is shifted and scaled in the normalization layer. Adding only the normalization and denormalization layers on the input and output of the model, respectively, RevIN can be applied to any model without modification to the model architecture. We empirically found that RevIN effectively alleviates the distribution discrepancy within the data, as well as significantly improves the model performance when applied to various state-of-the-art time-series forecasting methods. To verify the effectiveness of RevIN, we conduct extensive quantitative evaluations with various baselines: Informer (Zhou et al., 2021), N-BEATS (Oreshkin et al., 2020), and SCINet (Liu et al., 2021). We also present an in-depth analysis of the behavior of the model, including the verification of the assumptions on reversible instance normalization (See Section 3.2).
RevIN is a flexible, end-to-end trainable layer that can be applied to any arbitrarily chosen layers, effectively suppressing non-stationary information (mean and standard deviation of the instance) from one layer and restoring it on the other layer at a virtually symmetric position. Despite its remarkable performance, there has been no work on generalizing and expanding the instance-wise normalization-and-denormalization as a flexibly applicable, trainable layer in the time-series domain. Recently, deep learning-based time-series forecasting approaches, such as Informer (Zhou et al., 2021) and N-BEATS (Oreshkin et al., 2020), have shown outstanding performance in timeseries forecasting. However, they overlooked the importance of normalization, merely using simple global preprocessing to the model input without further exploration and expecting their endto-end deep learning model to replace the role. Despite the simplicity of our method, there have been no cases of using such techniques in modern deep-learning-based time-series forecasting approaches (Zhou et al., 2021; Liu et al., 2021; Oreshkin et al., 2020). In this sense, we enlight the importance of the appropriate normalization method in deep-learning-based time-series approaches; we propose a carefully designed, deep-learning-friendly module, for time-series forecasting, by combining the learnable affine transformation with the method, which has been widely accepted in recent deep-learning-based normalization work (Ulyanov et al., 2016).
In summary, our contributions are as follows:
• We propose a simple and effective normalization-and-denormalization method for timeseries, called RevIN, symmetrically structured to remove and restore the statistical information of a time-series instance; it is a generally-applicable layer to arbitrary deep neural networks with negligible cost and significantly improves the performance by reducing the distribution discrepancy within the data.
• By adding RevIN to the baseline, we achieve the state-of-the-art performance on four largescale real-world datasets with a significant margin.
• We conduct extensive evaluations on RevIN with quantitative analysis and qualitative visualizations to verify its effectiveness, addressing the distribution shift problem.

heading: 2 RELATED WORK
text: Time-series forecasting. Time-series forecasting methods are mainly categorized into three distinct approaches: (1) statistical methods, (2) hybrid methods, and (3) deep learning-based methods. As an example of the statistical models, exponential smoothing forecasting (Holt, 2004; Winters, 1960) is a well-established benchmark to predict future values. Statistical models have several advantages, e.g., interpretability and theoretically well guaranteed. In order to boost the performance, recent work proposed a hybrid model (Smyl, 2020) which incorporates a deep learning module with a statistical model. It achieved better performance over the statistical methods in the M4 timeseries forecasting competition. The deep learning-based method basically follows the sequenceto-sequence framework to model the time-series forecasting. Initially, deep learning-based models utilized variations of recurrent neural networks (RNNs). However, to overcome the limitation of the limited receptive field, several studies utilize advanced techniques, such as the dilatation and attention module. For instance, SCINet (Liu et al., 2021) and Informer (Zhou et al., 2021) modified the sequence-to-sequence-based model to improve the performance for long sequences. However, most previous deep learning-based models are hard to interpret compared to statistical models. Thus, inspired by statistical models, N-BEATS (Oreshkin et al., 2020) designed an interpretable layer for time-series forecasting by encouraging the model to explicitly learn trend, seasonality, and residual components. This model shows superior performance on M4 competition datasets.
Distribution shift. Although there are various models for time-series forecasting, they often suffer from non-stationary time-series, where the data distribution changes over time. Domain adaptation (DA) (Tzeng et al., 2017; Ganin et al., 2016; Wang et al., 2018) and domain generalization (DG) (Wang et al., 2021; Li et al., 2018; Muandet et al., 2013) are common solutions for alleviating the distribution shift. A domain adaptation algorithm attempts to reduce the distribution gap between source and target domains. A domain generalization algorithm only relies on the source domain and hopes to generalize on the target domain. Both DA and DG have a common objective that bridges the gap between source and target distributions. However, in non-stationary time-series, defining a domain is not as straightforward since the data distribution shifts over time. Recently, Du et al. (Du et al., 2021) proposed Adaptive RNNs (AdaRNNs) to handle the distribution shift problems for non-stationary time-series data. It first characterizes the distribution information by splitting the training data into periods. Then, it matches distributions of the discovered periods to generalize the model. However, unlike AdaRNNs that are costly expensive, RevIN is simple yet effective and model-agnostic since the method can be adopted easily to any deep-learning model.

heading: 3 PROPOSED METHOD
text: In this section, we propose reversible instance normalization (RevIN) for alleviating the distribution shift problem in time-series, which causes the discrepancy among the input sequences of the training data, as well as the discrepancy between train and test data distributions. Section 3.1 describes RevIN in detail and Section 3.2 discusses how RevIN alleviates the distribution discrepancy in time-series throughout qualitative visualization.

heading: 3.1 REVERSIBLE INSTANCE NORMALIZATION
text: We consider the multivariate time-series forecasting task in discrete time, with a sliding window size of Tx for input data. Here, N,K, Tx, and Ty indicate the number of the sequences, the number of variables, the input sequence length, and the prediction length. Let X = {x(i)}Ni=1 and Y = {y(i)}Ni=1 denote the input data and corresponding target, respectively. In RevIN, the input sequence length Tx and the prediction length Ty can be different since the observations are normalized and denormalized across the temporal dimension, as will be explained below. Given an input sequence x(i) ∈ RK×Tx , we aim to solve the time-series forecasting problem, which is to predict the subsequent values y(i) ∈ RK×Ty . Our proposed method, RevIN, symmetrically transforms the input data x(i) and the prediction output ỹ(i) of the network, as illustrated in Fig. 2. First, we normalize the input data x(i) with the instance-specific mean and standard deviation, which is widely accepted as instance normalization (Ulyanov et al., 2016). The mean and standard deviation are computed for
every instance x(i)k· of the input data (Fig. 2(a-3)) as
Et[x(i)kt ] = 1
Tx Tx∑ j=1 x (i) kj and Var[x (i) kt ] = 1 Tx Tx∑ j=1 ( x (i) kj − Et[x (i) kt ] )2 . (1)
Using these statistics, we normalize the input data x(i) as (Fig. 2(a-1))
x̂ (i) kt = γk
( x (i) kt − Et[x (i) kt ]√
Var[x (i) kt ] +
) + βk, (2)
where γ, β ∈ RK are learnable affine parameter vectors. The normalized sequences have the same mean and variance, where the non-stationary information is reduced. As a result, the denormalization layer allows the model to accurately predict the local dynamics within the sequence while receiving inputs of consistent distributions in terms of the mean and variance.
The model then receives the transformed data x̂(i) as input and forecasts their future values. However, the input data have different statistics from the original distribution, and only observing the normalized input x̂(i), it is difficult to capture the original distribution of the input x(i). Thus, to make this easier for the model, we explicitly restore the non-stationary properties removed from the input data to the model output by reversing the normalization step (Fig. 2(a-3)). Such denormalization step can return the model output to the original time-series value as well (Ogasawara et al., 2010). Accordingly, we denormalize the model output ỹ(i) by applying the reciprocal of the normalization in Eq. 2 (Fig. 2(a-2)) as
ŷ (i) kt =
√ Var[x
(i) kt ] + · ( ỹ (i) kt − β γ ) + Et[x(i)kt ], (3)
where the same statistics used in the normalization step in Eq. (2) are used for the scaling and shifting. Now, ŷ(i) is the final prediction of the model instead of ỹ(i).
Simply added to the input and output of a network, RevIN can be easily applied to any network with negligible cost while effectively improving time-series forecasting performance. Additionally, each step of RevIN, normalization and denormalization, can be applied to any activations of the intermediate layers in the network. However, our method is most effective when applied to the layers of symmetric encoder-decoder structure. In a typical time-series forecasting model, the boundary between the encoder and the decoder is often unclear. Therefore, we apply normalization and denormalization to the input and output of a forecasting network, which can be interpreted as an encoderdecoder structure, generating subsequent values given input data. Additional analysis on selecting layers to apply RevIN is provided in Section 4.2.3.

heading: 3.2 EFFECT OF REVERSIBLE INSTANCE NORMALIZATION ON DISTRIBUTION SHIFT
text: This section verifies that our method indeed alleviates the discrepancy in distributions during the forward pass of the network, and then reconstructs it in the network output layer through the denormalization step. We analyze the distributions of the train and test data before and after each step of our method, i.e., the normalization on input and denormalization on output, as shown in Fig. 3.
As a result, we observe that RevIN significantly reduces the discrepancy between the train and test data distributions in the normalization step (Fig. 3(b)) compared to the original data (Fig. 3(a)). In Fig. 3(a), the distributions of the train and test data hardly overlap, especially in the ETTh2 dataset, which is caused by the distribution shift problem. Also, the distributions have multiple peaks, which can suggest that the sequences in the data have different distributions from each other.
However, in our method, the normalization step transforms each data distribution into mean-centered distributions. Thus, it can be said that the multimodal distribution in the input data ((Fig. 3(a)) is caused by the different distributions in sequences. Also, the train and test data distributions become similar to each other. It demonstrates that the normalization step of RevIN can alleviate the distribution shift problem, reducing the distribution discrepancy between train and test datasets. Moreover, the baseline model, where RevIN is applied, still maintains the similarly-shaped distributions in the prediction output (Fig. 3(c)). As expected, these are then returned back to the original distribution by the denormalization step of RevIN (Fig. 3(d)). Without denormalization, the model needs to re-
construct the values that follow the original distributions (Fig. 3(d)) only using the input with the transformed distributions (Fig. 3(b)). Additionally, we hypothesize that the distribution discrepancy is also reduced in the intermediate layers of the model when RevIN is applied at the input and output layers; RevIN does not need to be applied on the intermediate layers, which will be discussed in Section 4.2.3. As a result, this procedure of RevIN can be considered as making problems easier first and then turning them back rather than directly solving the challenging problem where the distribution shift problem exists.

heading: 4 EXPERIMENTS
text: This section describes the experimental setup and presents extensive experiments on RevIN.

heading: 4.1 EXPERIMENTAL SETUP
text: Datasets. We evaluate our methods on two large-scale real-world time-series datasets. (i) Electricity transformer temperature (ETT)1 is electrical transformer temperature data collected from two different counties in China for two years. The data consists of seven features: oil temperature and six power load features. Following the same protocol of Informer (Zhou et al., 2021), we split the dataset into three datasets: ETTh1, ETTh2, and ETTm1. ETTh1 and ETTh2 data are measured every hour at different locations. ETTm1 data are measured every 15 minutes. For all three datasets, we use the first 12 months, the middle four months, and the last four months as training, validation, and test data, respectively. (ii) Electricity Counting Load (ECL)2 data contains the electricity counting load collected from 321 clients, and the data measured from each client is used as a feature value. We convert the dataset to hourly basis following the prior work (Zhou et al., 2021). For the ECL dataset, we use 15, 3, and 4 months as train, validation, and test data, respectively.
Experimental details. We set prediction window lengths as 1 day (1d), 2d, 7d, 14d, 30d, and 40d for the datasets measured on an hourly basis, ETTh1, ETTh2, and ECL. For the ETTm1 dataset, which is measured every 15 minutes, we chose 6 hours (6h), 12h, 3d, 7d, and 14d as prediction window lengths. We evaluate the time-series forecasting performance in terms of mean squared error (MSE) and mean absolute error (MAE). Following the same evaluation procedure of the previous study (Zhou et al., 2021), we compute the MSE and MAE on normalized data, since the values are not on the same scale. More details on experimental settings, including training details and hyperparameters, are provided in Appendix A.11.
Compared baselines. RevIN is a model-agnostic method that can be easily added to any network with further improvements. In this paper, we evaluate RevIN by adopting it on three time-series forecasting models: Informer (Zhou et al., 2021), N-BEATS (Oreshkin et al., 2020), and SCINet (Liu et al., 2021). All the models forecast the time-series sequence non-autoregressively. The exact training details of the models are reported in Appendix A.12. For all experiments, we compare RevIN and the baselines under the same hyperparameter settings, including the input and prediction lengths.

heading: 4.2 RESULTS AND ANALYSIS
text: This section presents quantitative analyses and qualitative visualizations of the experimental results of RevIN when it is adopted to the state-of-the-art time-series forecasting models.

heading: 4.2.1 EFFECTIVENESS OF REVERSIBLE INSTANCE NORMALIZATION ON VARIOUS TIME-SERIES FORECASTING MODELS
text: Table 1 demonstrates how much RevIN improves the forecasting accuracy of the baseline models. We compare the performance between RevIN and three baseline models, Informer, N-BEATS, and SCINet, on the four datasets with increasing prediction lengths. When adopted to various baselines, RevIN consistently outperforms their performance with a large margin, achieving the state-of-the-art performance on the four datasets. Moreover, the effectiveness of RevIN is more evident in the long sequence prediction, where it remarkably reduces the errors of the baselines. For example, when
1https://github.com/zhouhaoyi/ETDataset 2https://archive.ics.uci.edu/ml/ datasets/ElectricityLoadDiagrams20112014
the prediction length increases from 24 to 960 on the ETTh2 dataset, the forecasting error of NBEATS significantly degrades from 0.403 to 6.408. In contrast, RevIN shows only a slight increase in the error, i.e., from 0.192 to 0.471; it makes the baseline model become robust to the prediction length. A similar tendency appears on the other prediction lengths, datasets, and baseline models as well. These results demonstrate that RevIN is a simple-yet-effective, model-agnostic method that can easily be added to any existing model while effectively improving the forecasting performance.
We further analyze the effect of RevIN on the long prediction sequence quantitatively as shown in Table 2. We increase the model prediction length from 48 to 960 when the prediction length is fixed to 48. As a result, RevIN consistently outperforms the baselines, showing robust performance against
the prediction length. The difference in the forecasting error between SCINet and RevIN is relatively small when the prediction length is short (e.g., 48), but RevIN remarkably surpasses SCINet with a significant margin when the prediction length becomes long (e.g., 336, 720, and 960). These results show that our method makes the baseline models become robust to the prediction length.
To further study why RevIN shows stable performance even for the long prediction length, we analyze the forecasting error for each time step. The error at the t-th time step is computed as MSEt = 1 N ∑N i=1 1 K ∑K k=1(ŷ (i) kt −y (i) kt )
2. Fig. 4 provides the results on the ETTh2 dataset comparing the baselines and RevIN. In the cases of N-BEATS and SCINet, the error extremely increases as predicting the distant future. RevIN alleviates such a significant increase in error, outperforming the baselines with a large margin. Informer also shows unstable performance for the different time steps. The error is large in the early time step and becomes relatively constant for the future compared to the other baseline models. RevIN allows the models to have consistent small errors at every time step. The results demonstrate the effectiveness of RevIN on forecasting long sequences.

heading: 4.2.2 COMPARISON WITH OTHER NORMALIZATION METHODS
text: We compare RevIN with classical and state-of-the-art normalization methods, including minmax normalization, z-score normalization, layer normalization (Ba et al., 2016), batch normalization (Ioffe & Szegedy, 2015), instance normalization (Ulyanov et al., 2016), and deep adaptive input normalization (DAIN) (Passalis et al., 2019) as shown in Table 3. The normalization statistics for min-max and z-score normalization methods are computed for each input instance, not for the entire data. Additionally, we use batch normalization as the replacement of the input normalization method in RevIN, named reversible batch normalization (RevBN). Layer normalization cannot be used similarly since its standardization operation is not reversible when the input and prediction lengths are
different as our experimental settings. Here, we provide the experimental results on the four datasets, using N-BEATS as the baseline. Complete results including different prediction lengths are provided in Table 10 in Appendix A.7. RevIN shows outstanding performance compared to other normalization methods, especially on ETTh2 and ETTm1 datasets. Moreover, RevBN also improves the forecasting performance of batch normalization. Especially, its error is largely decreased when predicting the long sequences, e.g., 960 and 1344. It implies that the denormalization step of RevIN is essential for improving long sequence forecasting as a normalization layer. However, batch normalization applies an identical normalization to all the input sequences using the statistics obtained from the entire training data, so it can not reduce the discrepancy between the distributions of the train and test data. As a result, RevIN, which uses instance-wise normalization, surpasses RevBN with a significant margin, demonstrating that its successful reduction of the distribution discrepancy indeed effectively improves the performance. Moreover, RevIN not only improves the performance of the baselines but also has the advantage of being lightweight compared to the other method, say, DAIN that requires at least 3K2 additional parameters, when K is the number of variables; RevIN only requires 2K additional parameters, which is almost negligible costs for the baseline models.

heading: 4.2.3 ANALYSIS ON DISTRIBUTION SHIFT IN THE INTERMEDIATE LAYERS
text: We can assume that RevIN can reduce the distribution shift at an intermediate feature level as well. Thus, we analyze the feature divergence between the train and test data for Informer and RevIN on the four datasets. Informer consists of two layers for the encoder and one layer for the decoder. Thus, we use the output of the first (Layer-1) and the second (Layer-2) layers of the encoder for the analysis to obtain the intermediate layer features. Following previous work (Pan et al., 2018), we compute the average feature divergence using symmetric KL divergence. The details on the calculation of the feature divergence are provided in Appendix A.10. As shown in Fig. 5, the feature divergence between the train and test data is significantly reduced as RevIN is added to the baseline model. It implies that the normalization applied to the input can resolve the distribution shift in every layer of a model. Therefore, our proposed method successfully alleviates the distribution shift problem even only added to the input and output layers in a model. This allows our method to be adopted to any network without modification on the architectures, as a model-agnostic method, even effectively improving the model performance. Note that our method can be added to the intermediate layers as well, still greatly improving the model performance, as shown in Appendix A.4.

heading: 5 CONCLUSION
text: We propose a simple yet effective normalization method tailored to time-series forecasting. RevIN consists of symmetrically structured normalization and denormalization layers, where the shifting and scaling applied on the input data is exactly reversed back on the output prediction, effectively reducing the discrepancy between train and test data distributions. Easily applied to input and output layers with a negligible number of additional parameters, RevIN significantly improves the forecasting performance of a model without any modification in the architecture of the model. By adding RevIN, we achieve the state-of-the-art performance on four large-scale real-world datasets with a significant margin. The extensive experiments demonstrate the effectiveness of RevIN for accurate time-series forecasting against the distribution shift problem.

heading: 6 REPRODUCIBILITY STATEMENT
text: To ensure reproducibility, we provide the source code of our method as the supplementary material, including the pretrained model weights. In the main manuscript, Section 4.1 describes how we conduct data preprocessing on the dataset used in the experiments. In the appendix, Section A.11 explains the experimental details including the seed for the experiments. Also, we provide a detailed explanation of hyperparameter settings with the reproduction details of the baseline models in Section A.12 since our method is an add-on module with no additional hyperparameters.

heading: A APPENDIX
text: The supplementary material presents additional information, visualizations, and experimental results that support the main manuscript. Section A.1 shows experimental results on three real-world datasets along with qualitative analysis on the Nasdaq dataset to verify the effectiveness of our
method on obvious non-stationary time-series. Section A.2 addresses the potential of RevIN on a cross-domain time-series forecasting task. Section A.3 and Section A.4 conduct the hyperparameter sensitivity analysis and the ablation study on the proposed method, respectively. Section A.5 evaluates our method using the complementary metrics that measure the similarity between the two sequences. Section A.6 and Section A.7 show the comparison results of our method and existing normalization methods. Section A.8, Section A.9, and Section A.10 provide the algorithm of RevIN, the theoretical justification of RevIN, and the calculation details of the feature divergence used in Section 4.2.3, respectively. Section A.11 and Section A.12 describe additional implementation details and the reproduction details for baselines where RevIN is applied, respectively. Section A.13 illustrates additional quantitative results of RevIN and the baseline models. Lastly, Section A.14 provides extra quantitative results, including the standard deviation values for five experiments, which can not be located in the manuscript due to the lack of space.

heading: A.1 EXPERIMENTAL RESULTS ON ADDITIONAL REAL-WORLD TIME-SERIES DATASETS
text: We evaluate our proposed method on four large-scale real-world time-series datasets, the ETTh1, ETTh2, ETTm1, and ECL datasets. Additionally, this section provides experimental results on three real-world datasets: the two real-world datasets taken from the UCI repository, the air quality dataset and the Nasdaq dataset, and the M4 competition dataset (Makridakis et al., 2020). Air quality 3 data is hourly data with 13 variables of length 9537. We set the prediction length as 24, 48, 168, 336, 720 and the corresponding input length as 48, 96, 168, 360 so that their ratio becomes 2x, 2x, 1x, 0.5x, 0.5x. Nasdaq4 dataset consists of 82 features, including the closing measured since 2010. It is measured daily and has 1984 data samples for each variable in total. We fix the input sequence length to 60 and prolong the prediction length as 30, 60, 120. M45 competition dataset consists of six different hourly, daily, weekly, monthly, quarterly, and yearly sets, containing 100,000 test data. We follow the original experimental protocol of the M4 competition (Makridakis et al., 2020). For evaluation, we measure the micro-averaged mean absolute error and mean squared error on the M4 competition dataset: we first compute the metric independently for each set, i.e., hourly, daily, weekly, monthly, quarterly, and yearly sets, and then calculate the weighted average of the metrics using the contributions of each set as weights.
3https://archive.ics.uci.edu/ml/datasets/Air+Quality 4https://archive.ics.uci.edu/ml/datasets/CNNpred%3A+CNN-based+stock+market+prediction+using+
a+diverse+set+of+variables 5https://mofc.unic.ac.cy/m4/
As shown in Table 4, the model adopting RevIN significantly improves the forecasting performance of the baseline on all three datasets. Notably, RevIN shows outstanding performance on the Nasdaq dataset, reducing the prediction errors by more than half compared to the baseline.
To verify the effectiveness of our method on obvious non-stationary time-series, we conduct qualitative analysis on the Nasdaq dataset. As shown in Fig. 6, the Nasdaq index (the variable ’Close’) has steadily increased since 2002. Accordingly, when data are divided into the train and test data based on a specific point in time, the test data values tend to be higher than the train data values. In other words, the data severely suffers from the distribution shift problem, where the train and test data show a discrepancy in their distribution. As a result, even existing SOTA networks cannot predict the future values appropriately, as shown in Fig. 6. The baseline fails to keep up with the trend of data, whose mean value continues to increase, and thus the prediction results are shifted. However, RevIN mitigates such distribution discrepancy and remarkably increases the prediction performance of the baseline SOTA model. Similarly, RevIN shows superior performance on the decreasing data (‘DE1’ in Fig. 6), accurately predicting the changing mean of the data.

heading: A.2 CROSS-DOMAIN EVALUATION
text: Under review as a conference paper at ICLR 2022
informer - ours (input length 조절) scinet - ours (input length 조절) 5 run mean stdev 5 run mean stdev 5 run mean stdev input length MSE MAE MSE MAE MSE MAE N-BEATS+RevIN 48 0.654 0.552 0.048 0.022 1.107621 0.795153 0.051652 0.028718 0.557 0.51 0.01518 0.007796 168 0.701 0.58 0.059 0.022 1.054384 0.789533 0.060217 0.032052 0.629 0.55 0.029848 0.015965 336 0.652 0.567 0.013 0.005133 1.011444 0.7638 0.030177 0.018922 0.657 0.572 0.033888 0.01808 480 0.678 0.575 0.019 0.009 0.902 0.715 0.545 0.526 720 0.755 0.608 0.089 0.038 0.932926 0.721999 0.035302 0.015506 0.75 0.624 0.040144 0.019514 960 0.767 0.626 0.082 0.031 1.074106 0.782313 0.075402 0.030497 0.846 0.679 0.085333 0.036964
48 0.907 0.73 1.225 0.893 0.894 0.731 168 1.046 0.805 1.318 0.947 0.942 0.744 336 1.495 0.959 1.39 0.966 1.059 0.787 480 1.383 0.932 1.47 0.99 0.604 0.574 720 2.714 1.15 1.314 0.936 1.205 0.845
960 4.161 1.364 1.351 0.959 1.07 0.802
N-BEATS +RevIN Informer +RevIN SCINet +RevIN
MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE 48 2 48 0.907 0.73 0.654 0.552 48 1.225 0.893 1.107621 0.795153 48 0.894 0.731 0.557 0.51 168 7 168 1.046 0.805 0.701 0.58 168 1.318 0.947 1.054384 0.789533 168 0.942 0.744 0.629 0.55 336 14 336 1.495 0.959 0.652 0.567 336 1.39 0.966 1.011444 0.7638 336 1.059 0.787 0.657 0.572 480 20 480 1.383 0.932 0.678 0.575 480 1.47 0.99 0.902 0.715 480 0.604 0.574 0.545 0.526 720 30 720 2.714 1.15 0.755 0.608 720 1.314 0.936 0.932926 0.721999 720 1.205 0.845 0.75 0.624
960 4.161 1.364 0.767 0.626 960 1.351 0.959 1.074106 0.782313 960 1.07 0.802 0.846 0.679
stdev
0.026 0.014 0.048 0.022 0.054 0.021 0.052 0.029 0.038 0.02 0.015 0.008 0.072 0.032 0.059 0.022 0.043 0.021 0.06 0.032 0.05 0.025 0.03 0.016 0.333 0.113 0.013 0.005133 0.099 0.046 0.03 0.019 0.117 0.041 0.034 0.018 0.033 0.025 0.019 0.009 0.124 0.052 0.033 0.025 0.017 0.014 0.01 0.005 1.35 0.125 0.089 0.038 0.047 0.029 0.035 0.016 0.185 0.061 0.04 0.02
0.64 0.082 0.031 0.062 0.036 0.075 0.03 0.109 0.044 0.085 0.037
0.4 0.5 0.5 0.6 0.6 0.7 0.7 0.8 0.8 0.9 0.9

heading: 48 168 336 480 720 960
text: N-BEATS+RevIN MSE MAE
0.4
0.6
0.8
1.0
MSE MAE
0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2
48 168 336 480 720 960
N-BEATS+RevIN MSE MAE
0.4
0.5
0.6
0.7
0.8
0.9
1.0
48 168 336 480 720 960
N-BEATS+RevIN MSE MAE
0.4
0.9
1.4
1.9
2.4
2.9
3.4
3.9
4.4
4.9
48 168 336 480 720 960 N-BEATS (MSE) N-BEATS (MAE) + RevIN (MSE) + RevIN (MAE)
0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0
48 168 336 480 720 960 Input sequence length
N-BEATS MSE N-BEATS MAE +RevIN MSE +RevIN MAE 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9
48 168 336 480 720 960 Input sequence length
Informer MSE Informer MAE +RevIN MSE +RevIN MAE ±. 
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3
48 168 336 480 720 960 Input sequence length
SCINet MSE SCINet MAE +RevIN MSE +RevIN MAE
We analyze the cross-domain evaluation results to analyze further the capability of RevIN to alleviate cross-domain distribution discrepancy. As RevIN reduces the discrepancy in the train and test distributions, it can also reduce the discrepancy in the cross-domain distributions if the input sequences from each dataset have similar local dynamics. We use ETT datasets for the evaluation since they have the same features. However, their distributions can show a significant discrepancy because ETTh1 and ETTh2 are measured from different locations, and ETTh and ETTm1 datasets have different measurement intervals. We train the baseline and RevIN using SCINet as a baseline on ETT datasets and evaluate the trained model on the other datasets. We report the average MSE of five runs in Table 5 and variance in Table 11. Despite the difference between distributions of the datasets, RevIN shows superior performance in cross-domain evaluations. Especially, RevIN outperforms a SCINet with a large margin when the model needs to reduce the discrepancy in ETTh2 and ETTm1 datasets. The results demonstrate that RevIN successfully reduces the discrepancy in the input sequence distributions, leading to better generalization performance.

heading: A.3 HYPERPARAMETER SENSITIVITY ANALYSIS
text: We analyze the hyperparameter sensitivity of the proposed method RevIN in comparison with the baseline SOTA models. Input sequence length can be a crucial hyperparameter to RevIN since it computes the mean and variance across each input sequence and thus can determine the prediction accuracy and stability of the training process. In Fig. 7, we increase the model input sequence length from 48 (two days) to 960 (40 days) to see its impact on RevIN when the prediction length is fixed to 960 (40 days). First, the results demonstrate that the models adopting RevIN consistently outperform the baselines for various input sequence lengths in terms of the MAE and MAE. More importantly, RevIN makes the baseline models more robust to the input length. In other words, RevIN shows stable performance for various input lengths in contrast to the baseline models, which shows a high variance in their performance according to the input length. Notably, in N-BEATS, the average error as well as the standard deviation of the error significantly increase as prolonging the input length. This is because the trend of data is expressed as a linear function in N-BEATS. As the input length becomes prolonged, there is a chance that the variance (or non-stationarity) in the time-series values becomes higher, decreasing the accuracy of the linear trend to fit the data unless the data is monotonically increasing or decreasing. Also, the mispredicted trends linearly increase the error in the future values. However, when N-BEATS adopts the RevIN layer, it removes the non-stationary statistics from the input, and thus, the model shows robust performance against the input length. Removing the variability, i.e., normalizing its mean and standard deviation, before feeding it to the model and returning it to the output would make the model learning stable.

heading: A.4 ABLATION STUDY
text: We ablate the affine transformation from RevIN to see its impact on the performance. We conduct the experiments on the ETTh1, ETTh2, ETTm1, ECL, Nasdaq, and air quality datasets using N-BEATS
as the baseline. The results in Table 6 show that the affine transformation contributes to performance improvement consistently on a variety of datasets.
As mentioned earlier, a model can add RevIN in an intermediate layer, even to several layers. While existing approaches are a preprocessing-and-postprocessing method applied outside of the main prediction model, RevIN is an end-to-end trainable layer that can be added to any layer in the model as batch normalization and instance normalization, which are the recently proposed deep learningbased normalization layers. Thus, we verify that adopting RevIN to the intermediate layers instead of the input and output layers can improve the forecasting performance as well. We add RevIN at the first stack of N-BEATS and SCINet and evaluate their performance on the six datasets.
The results in Table 7 demonstrate that even when added to the intermediate layers, RevIN improves the performance of the baselines as a learnable normalization layer. We mainly focus on adding RevIN to the input and output of a model since it shows robust performance on average. Nevertheless, the model adopting RevIN in the intermediate layers also shows the best performance frequently, consistently outperforming the baseline without RevIN. This performance is even better than the dynamic normalization methods, LSTNet∗ and ES-RNN∗, when comparing them where NBEATS is used as the baseline (Table 9). For example, when the prediction length is 960, the mean squared errors of LSTNet∗, ES-RNN∗, RevIN (inter.), and RevIN (i/o) are 5.627, 1.338, 0.523, and 0.471, on average.

heading: A.5 EVALUATION ON SIMILARITY METRICS
text: We additionally evaluate our method using the metrics that measure similarity between the two sequences, dynamic time warping (DTW) and temporal distortion index (TDI) (Le Guen & Thome, 2019; Frı́as-Paredes et al., 2017). As shown in Table 8, the results show that our approach significantly improves the baseline models across all datasets in terms of the DTW and TDI. Notably,
RevIN shows a significant margin compared to the baseline for long prediction length. For example, when RevIN is added, the average DTW decreases from 38.348 to 15.240 for Informer, from 53.148 to 12.766 for N-BEATS, and from 20.498 to 11.080 for SCINet when the prediction length is 960. There are few cases where the proposed method predicts a less similar sequence than the base-
line, but the margin is minimal in terms of either DTW or TDI compared to the significant margin found when our method outperforms the baseline. These results demonstrate that the model adopting RevIN can generate a sequence more similar to the groundtruth than the baseline, especially showing a better prediction accuracy on the long sequence.

heading: A.6 COMPARISON WITH EXISTING DYNAMIC NORMALIZATION METHODS
text: We compare RevIN with the normalization methods proposed in LSTNet (Lai et al., 2017) and ES-RNN (Smyl, 2020), as shown in Table 9. Similarly to adding RevIN to the baseline model, we add the autoregressive linear bypass module of LSTNet and the modified Holt-Winters exponential smoothing of ES-RNN to the baseline model, respectively. RevIN consistently shows the best performance among the normalization methods adopted on N-BEATS showing a significant margin. The autoregressive linear bypass module of LSTNet (LSTNet∗) also consistently reduces the prediction error compared to the baseline, but the performance improvement is smaller than our method. For example, when the prediction length is 960 on the ETTh2 dataset, N-BEATS shows an average error of 6.408 and LSTNet∗ reduces the error to 5.627, but which is still much higher than RevIN that reduces the error to 0.471. Similarly, when the prediction length is 1344 on the ETTm1 dataset, the baseline shows an average error of 14.613 and LSTNet∗ largely decreases the error to 5.592, but RevIN more significantly decreases the error to 0.631. LSTNet utilizes a linear autoregressive
(a) model input (b) normalized input (c) model output (d) denormalized output
ES RN
N *
Re vI
N LS
TN et * De ns ity De ns
ity De
ns ity
Figure 8: Effect of RevIN on distribution discrepancy between train and test data compared to existing dynamic normalization methods. We compare RevIN with LSTNet∗ that adds the autoregressive linear bypass module of LSTNet to the baseline and ES-RNN∗ that adds the exponential smoothing of ES-RNN to the baseline. The analysis is conducted on the ETTh2 dataset with a prediction length of 960 using N-BEATS as the baseline model. From left to right columns, we compare the train and test data distributions on each step of the sequential process in each method.
model to make the non-linear neural network robust to scale change. However, it does not change the statistics of the input sequence. Thus, the discrepancy between the train and test datasets still exists for the non-linear neural network. In the case of ES-RNN∗, the training of the model is unstable and it often degrades the baseline performance, e.g., when the prediction length is 24 and 48 on the ETTh1 and ETTm1 datasets and the prediction length is 672 on the ETTm1 dataset. However, it significantly reduces the error for the long prediction length much better than LSTNet∗, but still worse than RevIN, e.g., when the prediction length is 960 on the ETTh2 dataset.
Additionally, we further analyze the data distributions of the methods on the ETTh2 dataset with the prediction length of 960 as shown in Fig. 8. From left to right columns, we compare the train and test data distributions on each step of the sequential process in each method.
ES-RNN∗ shows the distributions of (a) the original input, (b) the normalized input where the level and seasonality are removed by its proposed method, (c) the model prediction output, (d) the denormalized output where the level and seasonality is multiplied back to the original distribution.
LSTNet∗ shows the distributions of (a) the original input, (b) the same original input since the method does not transform the input data before feeding it to the prediction model, the model prediction output before (c) and after (d) adding the output of the autoregressive model.
RevIN shows the distributions of (a) the original input, (b) the input normalized by RevIN, (c) the model prediction output, and (d) the denormalized output.
In Fig. 8(a), the train and test data shows discrepancy in their distributions. Also, they have several peaks, not being centered to the mean. This implies that time-series in the data would have different means from each other. In Fig. 8(b), both RevIN and ES-RNN∗ transform data distributions into mean-centered distributions. Particularly, ES-RNN∗ makes the distribution extremely concentrated on the mean with a small variance. The two methods make sequences have similar statistics, thereby alleviating the distribution shift problem in the input data. This would lead to competitive performance on the long prediction sequence in contrast to LSTNet∗.
LSTNet∗ cannot resolve the distribution discrepancy because it has no module that can change the input statistics. Also, the distributions of the model output (Fig. 8(c)) completely differ from the input (Fig. 8(b)). The methods cannot make the input and output distribution to be consistent. In addition, its autoregressive model barely affects the model output distribution as shown in Fig. 8(c,d). There is almost no difference between the model output and the final output in LSTNet∗.
Also, the distributions of the final output (Fig. 8(d)) significantly differ from the original data (Fig. 8(a)) in LSTNet∗, which implies that they fail to learn the appropriate data distribution, and this would be the main reason for higher prediction error than RevIN. Similarly, although ES-RNN∗ alleviates the distribution discrepancy in the input data, it fails to return the model output (Fig. 8(d)) back to the original distribution (Fig. 8(a)), especially for the test data.
On the other hand, in RevIN, the distributions of the final output (Fig. 8(d)) return well to the original distributions (Fig. 8(a)). Also, RevIN makes the input and output of the model to maintain the consistent distributions while the train and test data are overlapped. Thus, RevIN can show robust performance against the input and output lengths.

heading: A.7 ADDITIONAL RESULTS ON COMPARISON WITH OTHER NORMALIZATION METHODS
text: This section provides the additional results on comparison with other normalization methods, which can not be located in the main manuscript due to the lack of space. The forecasting error of RevIN and existing normalization methods are evaluated on the ETTh1, ETTh2, ETTm1, and ECL datasets in Table 10. RevIN consistently outperforms the other normalization methods across all datasets. Interestingly, when the denormalization step is added to batch normalization (RevBN) as RevIN, the model better forecasts the long sequence, e.g., when the prediction length is 960, than batch normalization.
In the case of DAIN, it learns to transform the mean and variance values of the current input and use the new values to normalize the data. As a result, the input sequences can have different means and variances from each other, which indicates that the distribution shift can still exist after the normalization. As shown in our paper, the denormalization step shows a critical role in improving the model performance by recovering the prediction to the original value. However, a denormalization step cannot be added to DAIN since it has the Hadamard multiplication operation at the last step, which is not reversible when the input and prediction sequence lengths are different. These differences would be the reason for its inferior performance to RevIN, even requiring more computational costs and a larger amount of model parameters (at least threeD×D matrices, whereD is the number of variables).

heading: A.8 ALGORITHM OF REVERSIBLE INSTANCE NORMALIZATION
text: Algorithm 7 summarizes the RevIN procedure. Reversible instance normalization is the symmetric structure of the normalization (line 4-5) and denormalization operation (line 7-8). RevIN simply scales and shifts the input (line 2-5) and output (line 7-8) of a model using identical statistics. Indeed, gθ in Algorithm 7 (line 6) can be any network. RevIN is a model-agnostic normalization method easily adopted to any model with negligible cost but effectively improves forecasting performance.

heading: A.9 THEORETICAL JUSTIFICATION OF REVIN AGAINST DISTRIBUTION SHIFT PROBLEM
text: Let {x(i)}Ni=1 be our data, where x(i) ∈ RK×Tx denotes a time-series with K variables of length Tx. Consider a simple univariate case where K = 1 without the loss of generality. Then, x(i) ∈ RTx denotes the i-th time-series in the data. Consider two such data, the training and test data, whose distributions are denoted as Ptra and Ptst, respectively. The distribution shift appears when distribution of the data sequences are different (Du et al., 2021). That is,
Ptra(x) 6= Ptst(x). (4)
In our work, we consider the distribution shift problem in terms of the mean and the variance. Assume that the given data suffer from the distribution shift problem in terms of the mean and variance, which can be expressed as
E[xtra] 6= E[xtst] or Var[xtra] 6= Var[xtst], for xtra ∼ Ptra, xtst ∼ Ptst. (5)
In order to solve this problem, the normalization step of RevIN can be applied to the input training sample x(i) ∼ Ptra. Then, the input sample can be transformed as
x̂(i) = x(i) − E[x(i)]√
Var[x(i)] · γ + β. (6)
By the laws of expectation and variance,
E[x̂(i)] = β and Var[x̂(i)] = γ2, for all i (7)
By symmetry, this also holds for x(i) ∼ Ptst, for all i. Therefore, the mean and variance of the training and test data distributions become identical. Thus, by definition, the distribution shift problem is solved for the input training and test data by the normalization step. Given the normalized time-series data, the forecasting model f : RTx → RTy , parameterized by θ, predicts the corresponding future values as ỹ = fθ(x̂). Then, the denormalization step of RevIN returns the dynamics of the original data, E[x(i)] and Var[x(i)] to the prediction output so that model does not have to reconstruct them from the normalized input. The model prediction ỹ(i) can be denormalized as
ŷ(i) = ỹ(i) − β γ · √ Var[x(i)] + E[x(i)] (8)
By the laws of expectation and variance, the mean and variance of ỹ(i) can be expressed as
E[ŷ(i)] = ∆ + E[x(i)] and Var[ŷ(i)] = λ ·Var[x(i)]. (9)
The denormalization step allows the mean and variance of the final prediction values to be expressed as the difference from the input statistics. Here, since the input data x(i) and the groundtruth future values are consecutive sequences, one can assume that their difference in the mean and variance can be expressed as Eq. (9) as well. Under the assumption, when adopting the denormalization step of RevIN, the model only needs to capture the difference from the input statistics, ∆ and λ, to accurately predict the statistics of the future values. The denormalization step allows the model to focus on capturing the difference in the distribution between input and output by removing all common non-stationary statistics between the input and output to the model.

heading: A.10 CALCULATION DETAILS ON FEATURE DIVERGENCE
text: This section explains how the feature divergence in Section 4.2.3 is calculated. Following previous work (Pan et al., 2018), we calculate the average feature divergence of each feature between train and test datasets using symmetric KL divergence, assuming that the output features follow a Gaussian distribution with mean µ and variance σ2. Then, the equation for feature divergence of the k-th feature fk is
D(f traink ||f testk ) = KL(f traink ||f testk ) +KL(f testk ||f traink ), (10)
where KL(fAk ||fBk ) = log σBk σAk + σAk 2 + (µAk − µBk )2 2σBk 2 − 1 2 . (11)

heading: A.11 EXPERIMENTAL DETAILS
text: We exactly follow the experimental settings of the baseline models by using their official code to conduct experiments except for N-BEATS that have no officially released code. We reproduce the model and set hyperparameters as stated in the original N-BEATS paper (Oreshkin et al., 2020). Additionally, we conduct a grid search for the learning rate of N-BEATS with the range of [1e-5, 1e3] and train the model with the weight decay with the factor of 0.001 to stabilize training. We train and test the models with the following seeds: 12, 22, 32, 42, and 52. Experiments using N-BEATS and Informer as backbone are performed on NVIDIA TITAN RTX, and experiments using SCINet are conducted on NVIDIA TITAN Xp. Following the multivariate time-series forecasting settings of prior work (Zhou et al., 2021; Liu et al., 2021), we select the input window lengths from 2d, 4d, 7d, 14d, 15d, 20d for the hourly datasets, the ETTh1, ETTh2, and ECL datasets, and from 12h, 1d, 4d, 7d for the ETTm1 dataset. Particularly, we set the ratio of input length to prediction length to be from 2.0 to 0.35 as the prediction length becomes longer. In the case of SCINet, we follow the setting of the original paper for the input length since the method requires the input sequence length to meet a specific condition.

heading: A.12 REPRODUCTION DETAILS FOR BASELINE MODELS
text: This section describes the implementation details of baselines, Informer, N-BEATS, and SCINet. Note that we compare each baseline model and RevIN using the same hyperparameters except for the presence of RevIN.
Informer. We use the official open-source code of Informer 6. We follow the provided hyperparameters for the experiments they studied. In the case of the ECL dataset, we use the same hyperparameter settings for the ETTh1 dataset since it is not officially provided.
N-BEATS. N-BEATS is reproduced using the PyTorch framework (Paszke et al., 2019). We follow the hyperparameter settings of the N-BEATS-I model. We train N-BEATS to minimize the mean squared error between the predicted results and groundtruth values. To compare with other methods, we report the prediction results of a single model instead of the ensemble method that N-BEATS used in the original paper. Since N-BEATS is the model tailored to univariate time-series forecasting, we flatten the multivariate input sequence to have a single dimension before feeding it to the model.
SCINet. We follow the experimental settings provided in the official code 7 of SCINet. Different from other baseline models, when the prediction length is 720, the input sequence length is set as 736 since SCINet requires the input length to be the multiple of 32 due to its hierarchical architecture (Liu et al., 2021).

heading: A.13 ADDITIONAL QUALITATIVE RESULTS
text: Informer fails to capture the repeated pattern of increase and decrease on ETTh2 dataset. Moreover, the prediction results of Informer are inaccurately scaled and shifted. With RevIN, Informer predicts reasonable results. Without any modification to the network architecture, RevIN can improve the forecasting performance. Although the pattern is accurately captured for N-BEATS, the prediction results are inaccurately scaled, resulting in low performance. RevIN improves the performance of N-BEATS in that the model predicts accurately. For SCINet, RevIN improves the baseline to predict more precise results.

heading: A.14 EXTRA QUANTITATIVE RESULTS
text: Table 11 and Table 12 show the standard deviation for five independent experiments of cross-domain evaluation and for comparison on long sequence forecasting, respectively. Table 13 describes the full results of comparison with baselines and RevIN, including the standard deviation for five runs and comparison with the reported performance in the original papers. It can be seen as RevIN improves the performance as statistically significant than previous state-of-the-art models.
6https://github.com/zhouhaoyi/Informer2020 7https://github.com/cure-lab/SCINet
Under review as a conference paper at ICLR 2022 Ta bl e 13 :T im ese ri es fo re ca st in g re su lts .T he m ea n an d st an da rd de vi at io n of th e fiv e in de pe nd en te xp er im en ts ar e re co rd ed .† de no te s th at th e re su lts ar e ob ta in ed fr om th e or ig in al pa pe r. M et ho ds In fo rm er † In fo rm er In fo rm er +R ev IN N -B E A T S N -B E A T S+ R ev IN SC IN et † SC IN et SC IN et +R ev IN M et ri c M SE M A E M SE M A E M SE M A E M SE M A E M SE M A E M SE M A E M SE M A E M SE M A E ETTh1 24 0. 57 7 0. 54 9 0. 55 0 ± 0. 04 1 0. 53 6 ± 0. 02 5 0. 50 4 ± 0. 04 9 0. 47 2 ± 0. 02 5 0. 47 8 ± 0. 02 2 0. 50 5 ± 0. 01 2 0. 33 0 ± 0. 00 6 0. 37 3 ± 0. 00 4 0. 31 1 0. 34 8 0. 33 8 ± 0. 01 2 0. 37 3 ± 0. 00 9 0. 30 8 ± 0. 00 3 0. 34 7 ± 0. 00 2 48 0. 68 5 0. 62 5 0. 77 2 ± 0. 12 2 0. 66 8 ± 0. 05 5 0. 64 6 ± 0. 03 9 0. 54 7 ± 0. 01 5 0. 53 6 ± 0. 06 0 0. 54 2 ± 0. 04 1 0. 37 2 ± 0. 00 1 0. 40 0 ± 0. 00 2 0. 36 4 0. 38 8 0. 43 6 ± 0. 02 5 0. 45 9 ± 0. 02 1 0. 36 5 ± 0. 00 5 0. 38 9 ± 0. 00 3 16 8 0. 93 1 0. 75 2 1. 13 8 ± 0. 09 6 0. 85 3 ± 0. 04 5 0. 65 5 ± 0. 05 5 0. 56 1 ± 0. 02 4 1. 00 5 ± 0. 14 6 0. 78 2 ± 0. 06 4 0. 46 6 ± 0. 03 0 0. 45 2 ± 0. 01 4 0. 49 7 0. 49 1 0. 45 9 ± 0. 01 5 0. 46 1 ± 0. 01 3 0. 40 6 ± 0. 00 3 0. 41 6 ± 0. 00 3 33 6 1. 12 8 0. 87 3 1. 27 8 ± 0. 12 9 0. 90 9 ± 0. 05 8 1. 05 8 ± 0. 11 9 0. 75 8 ± 0. 05 9 0. 93 2 ± 0. 07 9 0. 74 3 ± 0. 04 2 0. 51 5 ± 0. 01 3 0. 48 3 ± 0. 00 8 0. 49 1 0. 49 4 0. 52 7 ± 0. 01 0 0. 51 3 ± 0. 00 6 0. 46 7 ± 0. 00 5 0. 47 1 ± 0. 00 3 72 0 1. 21 5 0. 89 6 1. 35 7 ± 0. 05 6 0. 94 5 ± 0. 00 9 0. 92 6 ± 0. 05 7 0. 71 7 ± 0. 03 6 1. 38 9 ± 0. 23 0 0. 92 6 ± 0. 06 6 0. 57 6 ± 0. 03 5 0. 53 4 ± 0. 01 8 0. 61 2 0. 58 2 0. 59 6 ± 0. 01 5 0. 57 1 ± 0. 01 3 0. 50 7 ± 0. 00 6 0. 50 5 ± 0. 00 4 96 0 · · 1. 47 0 ± 0. 12 4 0. 99 0 ± 0. 05 2 0. 90 2 ± 0. 03 3 0. 71 5 ± 0. 02 5 1. 38 3 ± 0. 38 0 0. 93 2 ± 0. 12 0 0. 67 8 ± 0. 01 9 0. 57 5 ± 0. 00 9 · · 0. 60 4 ± 0. 01 7 0. 57 4 ± 0. 01 4 0. 54 5 ± 0. 01 0 0. 52 6 ± 0. 00 5 ETTh2 24 0. 72 0 0. 66 5 0. 45 0 ± 0. 09 9 0. 52 0 ± 0. 07 1 0. 23 8 ± 0. 01 0 0. 32 5 ± 0. 00 6 0. 40 3 ± 0. 18 5 0. 47 2 ± 0. 10 1 0. 19 2 ± 0. 00 3 0. 27 6 ± 0. 00 2 0. 18 3 0. 27 1 0. 19 9 ± 0. 02 6 0. 29 5 ± 0. 02 7 0. 18 0 ± 0. 00 4 0. 26 3 ± 0. 00 2 48 1. 45 7 1. 00 1 2. 17 1 ± 0. 09 4 1. 20 0 ± 0. 04 8 0. 36 1 ± 0. 02 3 0. 40 4 ± 0. 01 4 1. 33 0 ± 0. 24 0 0. 91 8 ± 0. 07 3 0. 25 4 ± 0. 01 1 0. 32 0 ± 0. 00 8 0. 25 9 0. 34 1 0. 35 0 ± 0. 02 5 0. 42 2 ± 0. 02 7 0. 23 1 ± 0. 00 6 0. 30 2 ± 0. 00 6 16 8 3. 48 9 1. 51 5 8. 15 7 ± 0. 63 1 2. 55 8 ± 0. 11 3 0. 85 9 ± 0. 07 2 0. 64 9 ± 0. 02 6 7. 17 4 ± 0. 44 9 2. 32 9 ± 0. 04 9 0. 41 0 ± 0. 01 0 0. 41 8 ± 0. 00 5 0. 52 8 0. 50 9 0. 55 9 ± 0. 04 4 0. 51 8 ± 0. 02 5 0. 33 7 ± 0. 00 7 0. 37 8 ± 0. 00 3 33 6 2. 72 3 1. 34 0 4. 74 6 ± 0. 45 5 1. 84 4 ± 0. 10 2 0. 89 0 ± 0. 05 7 0. 67 3 ± 0. 02 3 4. 85 9 ± 0. 26 8 1. 86 3 ± 0. 04 3 0. 44 9 ± 0. 01 1 0. 44 7 ± 0. 00 6 0. 64 8 0. 60 8 0. 66 4 ± 0. 07 3 0. 58 3 ± 0. 03 0 0. 35 7 ± 0. 00 3 0. 40 3 ± 0. 00 2 72 0 3. 46 7 1. 47 3 3. 19 0 ± 0. 32 6 1. 52 9 ± 0. 08 5 0. 57 6 ± 0. 04 4 0. 54 6 ± 0. 02 5 5. 65 6 ± 1. 05 3 2. 01 2 ± 0. 18 6 0. 49 6 ± 0. 00 8 0. 48 2 ± 0. 00 2 1. 07 4 0. 76 1 1. 54 6 ± 0. 37 8 0. 94 4 ± 0. 14 1 0. 41 1 ± 0. 00 3 0. 44 5 ± 0. 00 2 96 0 · · 2. 97 2 ± 0. 18 3 1. 44 1 ± 0. 03 5 0. 60 0 ± 0. 03 3 0. 57 0 ± 0. 01 8 6. 40 8 ± 2. 03 9 2. 07 7 ± 0. 24 2 0. 47 1 ± 0. 01 5 0. 48 1 ± 0. 00 8 · · 1. 86 2 ± 0. 15 3 1. 06 6 ± 0. 05 5 0. 43 8 ± 0. 00 7 0. 46 2 ± 0. 00 4
ETTm1
24 0.
32 3
0. 36
9 0.
33 0
± 0.
02 1
0. 38 2 ± 0. 01
7 0.
30 9
± 0.
02 0
0. 35 2 ± 0. 01
0 0.
44 3
± 0.
04 3
0. 43 7 ± 0. 03
5 0.
40 3
± 0.
00 6
0. 39 2 ± 0. 00
5 0.
12 7
0. 22
6 0.
13 0
± 0.
00 3
0. 23 1 ± 0. 00
3 0.
10 6
± 0.
00 2
0. 19 6 ± 0. 00
1
48 0.
49 4
0. 50
3 0.
49 9
± 0.
02 4
0. 48 6 ± 0. 01
2 0.
39 0
± 0.
00 8
0. 39 1 ± 0. 00
6 0.
45 3
± 0.
03 4
0. 47 2 ± 0. 01
8 0.
32 8
± 0.
01 0
0. 37 1 ± 0. 00
7 0.
15 0
0. 26
1 0.
15 5
± 0.
00 4
0. 26 2 ± 0. 00
4 0.
13 5
± 0.
00 3
0. 22 2 ± 0. 00
2
96 0.
67 8
0. 61
4 0.
60 5
± 0.
03 3
0. 55 4 ± 0. 02
7 0.
40 5
± 0.
01 3
0. 41 1 ± 0. 00
6 0.
60 3
± 0.
05 1
0. 58 1 ± 0. 02
7 0.
37 9
± 0.
01 1
0. 40 6 ± 0. 00
7 0.
19 0
0. 29
1 0.
19 5
± 0.
01 2
0. 29 1 ± 0. 01
3 0.
16 2
± 0.
00 1
0. 24 7 ± 0. 00
1
28 8
1. 05
6 0.
78 6
0. 90 6 ± 0. 03
9 0.
73 8
± 0.
02 8
0. 56 3 ± 0. 02
4 0.
50 2
± 0.
01 5
0. 84 9 ± 0. 09
5 0.
70 2
± 0.
05 1
0. 45 1 ± 0. 01
6 0.
44 5
± 0.
00 8
0. 41
7 0.
46 2
0. 36 1 ± 0. 00
8 0.
41 9
± 0.
00 4
0. 26 5 ± 0. 00
3 0.
32 1
± 0.
00 2
67 2
1. 19
2 0.
92 6
0. 94 3 ± 0. 06
2 0.
76 0
± 0.
03 4
0. 66 3 ± 0. 08
2 0.
55 0
± 0.
03 1
0. 86 0 ± 0. 05
7 0.
72 6
± 0.
02 6
0. 55 5 ± 0. 01
1 0.
51 1
± 0.
00 8
0. 55
4 0.
52 7
1. 02 0 ± 0. 04
0 0.
75 6
± 0.
02 5
0. 35 7 ± 0. 00
4 0.
38 0
± 0.
00 2
13 44
· ·
1. 09 5 ± 0. 06
5 0.
82 3
± 0.
04 0
0. 82 4 ± 0. 03
9 0.
63 2
± 0.
01 9
14 .6 13 ± 26 .1
08 1.
94 8
± 1.
65 5
0. 63 1 ± 0. 06
1 0.
55 6
± 0.
02 0
· ·
1. 84 1 ± 0. 24
2 1.
04 4
± 0.
10 0
0. 41 2 ± 0. 00
8 0.
42 2
± 0.
00 3
ECL
24 ·
· 0.
25 0
± 0.
00 5
0. 35 8 ± 0. 00
5 0.
14 8
± 0.
00 1
0. 25 7 ± 0. 00
1 0.
27 9
± 0.
00 7
0. 37 2 ± 0. 00
3 0.
17 6
± 0.
00 2
0. 28 5 ± 0. 00
1 ·
· 0.
13 8
± 0.
00 4
0. 24 6 ± 0. 00
5 0.
11 2
± 0.
00 1
0. 20 7 ± 0. 00
1
48 0.
34 4
0. 39
3 0.
30 0
± 0.
01 0
0. 38 6 ± 0. 00
5 0.
17 1
± 0.
00 4
0. 27 9 ± 0. 00
3 0.
30 9
± 0.
00 7
0. 38 8 ± 0. 00
4 0.
19 4
± 0.
00 1
0. 30 1 ± 0. 00
1 ·
· 0.
16 3
± 0.
00 7
0. 26 5 ± 0. 00
7 0.
12 6
± 0.
00 1
0. 22 2 ± 0. 00
1
16 8
0. 36
8 0.
42 4
0. 34 5 ± 0. 01
2 0.
42 3
± 0.
01 0
0. 26 1 ± 0. 01
0 0.
35 4
± 0.
00 7
0. 33 3 ± 0. 01
6 0.
41 0
± 0.
01 2
0. 21 8 ± 0. 00
2 0.
32 0
± 0.
00 1
· ·
0. 17 7 ± 0. 00
3 0.
28 1
± 0.
00 5
0. 15 3 ± 0. 00
3 0.
24 9
± 0.
00 2
33 6
0. 38
1 0.
43 1
0. 42 9 ± 0. 04
1 0.
47 3
± 0.
02 3
0. 35 6 ± 0. 02
6 0.
41 4
± 0.
01 5
0. 32 6 ± 0. 00
4 0.
40 6
± 0.
00 1
0. 24 1 ± 0. 00
5 0.
33 7
± 0.
00 2
· ·
0. 20 2 ± 0. 00
4 0.
30 8
± 0.
00 4
0. 16 2 ± 0. 00
1 0.
26 2
± 0.
00 1
72 0
0. 40
6 0.
44 3
0. 85 1 ± 0. 08
8 0.
71 9
± 0.
07 2
0. 83 4 ± 0. 12
2 0.
70 0
± 0.
07 6
0. 42 0 ± 0. 09
4 0.
46 7
± 0.
05 8
0. 30 3 ± 0. 01
2 0.
38 3
± 0.
01 1
· ·
0. 23 4 ± 0. 00
6 0.
33 3
± 0.
00 4
0. 18 3 ± 0. 00
3 0.
28 1
± 0.
00 2
96 0
0. 46
0 0.
54 8
0. 93 0 ± 0. 07
5 0.
75 0
± 0.
04 2
0. 89 4 ± 0. 04
7 0.
74 1
± 0.
03 1
0. 39 9 ± 0. 02
2 0.
45 5
± 0.
01 7
0. 32 5 ± 0. 01
9 0.
39 8
± 0.
01 5
· ·
0. 23 5 ± 0. 01
1 0.
33 0
± 0.
00 8
0. 20 0 ± 0. 00
3 0.
29 2
± 0.
00 2

Review summaries:
This work proposes to use a normalization method to address temporal distribution shift in time-series forecasting. The proposed approach, *RevIN*, consists of two steps: instance normalization on input sequences and "de-normalization" of output sequences by re-using statistics (mean and variance) computed during the normalization step. Experiments are conducted on two time-series datasets (ETT and ECL) with varying splits and prediction windows lengths. Results show that RevIN used on top of deep-learning based methods for time-series forecasting (Informer, N-BEATS and SCINet) improves prediction performances, in particular for long sequence prediction. Finally, they conduct an empirical comparison with other normalization methods, such as batch normalization and min-max normalization, to evaluate the adequateness of instance normalization and of the denormalization step. 

The authors propose a "reversible instance normalization" as an input pre and post processing procedure to improve the forecasting of any given base model - targeted at addressing the distribution shift that is common in time series data - e.g., time series are typically non-stationary.  This works by normalizing each time window input to a (deep learning) forecast model, applying the model on the normalized data, then unnormalizing the predictions to get the final predictions.   The normalization is done by subtracting the mean and dividing by the std. dev. in a current (input instance) window, followed by scaling and shifting by learnable, shared cross-instance (global), scaling and shifting parameters per input feature / variable.    The authors perform extensive experiments to show the proposed approach significantly improves the base metric score results for 3 recent, state-of-the-art forecasting algorithms across several datasets, and further that it significantly improves over other normalization approaches, and helps align distributions between train and test windows.