SlowGuess/NeurIPS_2023_OCR huggingface dataset:
sampled every 15 minutes. (2) Electricity<sup>4</sup> contains the electricity consumption data of 321 clients from July 2016 to July 2019. (3) Exchange<sup>5</sup> [19] collects the daily exchange rates of 8 countries from 1990 to 2016. (4) Traffic<sup>6</sup> includes the hourly traffic load of San Francisco freeways recorded by 862 sensors from 2015 to 2016. (5) Weather<sup>7</sup> is made up of 21 indicators of weather, including air temperature and humidity collected every 10 minutes in 2021. (6) ILI<sup>8</sup> records the weekly ratio of influenza-like illness patients versus the total patients by the Centers for Disease Control and Prevention of the United States from 2002 to 2021. The detailed information about these datasets are listed in the Table. 2. We also report the ADF test (Augmented Dickey-Fuller Test) [9] results in the table, which evaluate the stationarity of a time series. Following the standard protocol, we split each dataset into training, validation and testing sets according to the chronological order. The split ratio is 6:2:2 for ETT dataset and 7:1:2 for the other datasets [38]. Also, we apply a z-score normalization on them based on the statistics of training data as preprocessing to measure different variables on the same scale. Note that z-score normalization is unable to handle non-stationary time series since the statistics are fixed during normalization [28].
Table 2: The Statistics of Each Dataset.
Dataset	Variables	Sampling Frequency	Length	Slicing Length	ADF*
Electricity	321	1 Hour	26,304	24	-8.44
Exchange	8	1 Day	7,588	6	-1.90
Traffic	862	1 Hour	17,544	24	-15.02
Weather	21	10 Minutes	52,696	12	-26.68
ILI	7	1 Week	966	6	-5.33
ETTh1&ETTh2	7	1 Hour	17,420	24	-5.91&-4.13
ETTm1&ETTm2	7	15 Minutes	69,680	12	-14.98&-5.66
* A smaller ADF test result indicates a more stationary time series data
Backbone models SAN is a model-agnostic framework that can be applied to arbitrary time series forecasting models. To evidence the effectiveness of the framework, we select some mainstream models based on different architectures and evaluate their performance under both multivariate and univariate settings: Linear model based DLinear [39], Transformer based Autoformer [38] and FEDformer [45], and dilated convolution based SCINet [24]. We follow the implementation and settings provided in the official code of DLinear $^{9}$  and SCINet $^{10}$  to implement these models.
Experiments details We use ADAM [18] as the default optimizer across all the experiments and report the mean squared error (MSE) and mean absolute error (MAE) as the evaluation metrics. A lower MSE/MAE indicates a better performance. For the statistics prediction module in SAN, we use a simple two-layer perceptron network with a hidden size the same as the embedding size of the backbone model for simplicity. The detailed implementation of the statistics prediction module can be referred to in Appendix C.1. All the experiments are implemented by PyTorch [29] and are conducted for three runs with a fixed random seed on a single NVIDIA RTX 3090 24GB GPU.
Slicing length Regarding the selection of slicing length for each dataset, we adopt a heuristic idea that real-world time series data exhibit similar changing patterns within artificially defined or actual periods (daily, weekly, etc.). Combining the frequencies of benchmark datasets, we establish a range of  $\{6,12,24,48\}$  as slicing lengths such that most settings cover a meaningful time span. For example, we selected a slicing length of 24 for datasets such as ETTh1, Electricity and Traffic with a frequency of 1 hour. This ensures that each time slice contained data within a day and guarantees optimal performance among candidates. Here we admit that one limitation of our method is that the current design cannot handle indivisible length such that we set the slicing length to 6 which approximately represents a weekly period instead of 7 in the Exchange dataset. We present the ablation study on the effect of slicing length in Appendix B.5.
4https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014
$^{5}$ https://github.com/laiguokun/multivariate-time-series-data
<sup>6</sup>http://pems.dot.ca.gov
<sup>7</sup>https://www.bgc-jena.mpg.de/wetter/
<sup>8</sup>https://gis.cdc.gov/grasp/fluview/fluportaldshield.html
<sup>9</sup>https://github.com/cure-lab/LTSF-Linear
10 https://github.com/cure-lab/SCINet
Table 3: Forecasting errors under the multivariate setting. The bold values indicate better performance.
Methods Metric	Dlinear	+ SAN	FEDformer	+ SAN	Autoformer	+ SAN	SCINet	+ SAN
MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Electricity	96	0.140	0.237	0.137	0.234	0.185	0.300	0.164	0.272	0.195	0.309	0.172	0.281	0.213	0.316	0.152	0.256
192	0.153	0.250	0.151	0.247	0.196	0.310	0.179	0.286	0.215	0.325	0.195	0.300	0.224	0.329	0.163	0.266
336	0.168	0.267	0.166	0.264	0.215	0.330	0.191	0.299	0.237	0.344	0.211	0.316	0.230	0.334	0.178	0.283
720	0.203	0.301	0.201	0.295	0.244	0.352	0.230	0.334	0.292	0.375	0.236	0.335	0.260	0.356	0.206	0.307
Exchange	96	0.086	0.213	0.085	0.214	0.152	0.281	0.079	0.205	0.152	0.283	0.082	0.208	0.126	0.269	0.082	0.200
192	0.161	0.297	0.177	0.317	0.273	0.380	0.156	0.295	0.369	0.437	0.157	0.296	0.266	0.392	0.169	0.293
336	0.338	0.437	0.294	0.407	0.452	0.498	0.260	0.384	0.534	0.544	0.262	0.385	0.574	0.541	0.320	0.409
720	0.999	0.755	0.726	0.649	1.151	0.830	0.697	0.633	1.222	0.848	0.689	0.629	1.136	0.818	0.892	0.712
Traffic	96	0.411	0.283	0.412	0.288	0.579	0.363	0.536	0.330	0.654	0.403	0.569	0.350	0.626	0.393	0.542	0.344
192	0.423	0.289	0.429	0.297	0.608	0.376	0.565	0.345	0.654	0.410	0.594	0.364	0.613	0.396	0.545	0.358
336	0.437	0.297	0.445	0.306	0.620	0.385	0.580	0.354	0.629	0.391	0.591	0.363	0.625	0.398	0.563	0.369
720	0.467	0.316	0.474	0.319	0.630	0.387	0.607	0.367	0.657	0.402	0.623	0.380	0.639	0.409	0.607	0.381
Weather	96	0.175	0.237	0.152	0.210	0.246	0.328	0.179	0.239	0.247	0.320	0.194	0.256	0.181	0.260	0.169	0.232
192	0.217	0.275	0.196	0.254	0.281	0.341	0.234	0.296	0.302	0.361	0.258	0.316	0.239	0.311	0.215	0.275
336	0.263	0.314	0.246	0.294	0.337	0.376	0.304	0.348	0.362	0.394	0.329	0.367	0.293	0.348	0.267	0.314
720	0.325	0.366	0.315	0.346	0.414	0.426	0.400	0.404	0.427	0.433	0.440	0.438	0.345	0.380	0.338	0.365
ILI	24	2.297	1.055	2.122	1.001	3.205	1.255	2.614	1.119	3.309	1.270	2.777	1.157	7.467	2.039	2.776	1.163
36	2.323	1.070	2.029	0.978	3.148	1.288	2.537	1.079	3.207	1.216	2.649	1.104	7.035	1.948	2.411	1.026
48	2.262	1.065	2.041	0.971	2.913	1.168	2.416	1.032	3.166	1.198	2.420	1.029	7.225	1.955	2.295	1.004
60	2.443	1.124	2.089	0.973	2.853	1.161	2.299	1.003	2.947	1.159	2.401	1.021	7.335	1.957	2.487	1.063
ETTh2	96	0.292	0.356	0.277	0.338	0.341	0.382	0.300	0.355	0.384	0.420	0.316	0.366	0.690	0.625	0.294	0.347
192	0.383	0.418	0.340	0.378	0.426	0.436	0.392	0.413	0.457	0.454	0.413	0.426	0.991	0.742	0.374	0.398
336	0.473	0.477	0.356	0.398	0.481	0.479	0.459	0.462	0.468	0.473	0.446	0.457	1.028	0.759	0.412	0.430
720	0.708	0.599	0.396	0.435	0.458	0.477	0.462	0.472	0.473	0.485	0.471	0.474	1.363	0.885	0.437	0.461
Table 4: Comparison between SAN and existing normalization approaches. The best results are highlighted in bold.
Methods	FEDformer	Autoformer
+SAN	+RevIN	+NST	+Dish-TS	IMP(%)	+SAN	+RevIN	+NST	+Dish-TS	IMP(%)
Electricity	0.191	0.200	0.198	0.203	3.54	0.204	0.219	0.213	0.231	4.23
Exchange	0.298	0.474	0.480	0.704	37.13	0.297	0.495	0.494	1.008	39.88
Traffic	0.572	0.647	0.649	0.652	11.59	0.594	0.666	0.664	0.677	10.54
Weather	0.279	0.268	0.267	0.398	-4.49	0.305	0.290	0.290	0.433	-5.17
ILI	2.467	2.962	3.084	2.846	13.32	2.562	3.151	3.235	3.180	18.69
ETTh1	0.447	0.463	0.456	0.461	1.97	0.518	0.519	0.521	0.521	0.19
ETTh2	0.404	0.465	0.481	1.004	13.12	0.411	0.489	0.465	1.175	11.61
ETTm1	0.377	0.415	0.411	0.422	8.27	0.406	0.562	0.535	0.567	24.11
ETTm2	0.287	0.310	0.315	0.759	7.42	0.311	0.325	0.331	0.894	4.31
4.2 Main Results
We report the multivariate forecasting results in Table 3. The ILI dataset has a forecasting horizon of  $L_{out} \in \{24, 36, 48, 60\}$  while the others have a forecasting horizon of  $L_{out} \in \{96, 192, 336, 720\}$ . As for the input sequence length, we follow the traditional protocol and fix  $L_{in} = 96$  for Autoformer, FEDformer and SCINet with respect to all datasets ( $L_{in} = 36$  for ILI dataset) and extend it to 336 (96 for ILI dataset) for DLinear. Full benchmarks of ETT datasets and univariate results are provided in the Appendix.
As shown in the table, we clearly find that our proposed SAN framework can boost these models by a large margin in most cases of the benchmark dataset. We attribute this improvement to two aspects. Firstly, SAN mitigates the impact of non-stationary factors, as demonstrated by the performance on three typical non-stationary datasets (Exchange, ILI and ETTh2, determined by ADF test results). Specifically, under all experimental forecasting lengths with DLinear, SAN achieves an average MSE reduction of  $7.67\%$  in the Exchange dataset,  $11.13\%$  in the ILI dataset and  $21.29\%$  in the ETTh2 dataset. This conclusion applies to other backbone models as well and the enhancement is even more pronounced. Secondly, even in long-term forecasting scenarios where the difficulty of forecasting increases significantly with the length of the forecast, SAN imposes constraints on
backbone models to produce more reliable results using a novel statistical prediction module. For instance, when predicting for a length of 720 time steps, SCINet accompanied by SAN achieves a  $70.37\%$  reduction in MSE on the ETTh2 dataset and a  $20.77\%$  reduction on the Electricity dataset. These improvements make SCINet comparable to other forecasting models and suggest that SAN can help stabilize outputs in long-term forecasting scenarios.
Figure 3: Visualization of long-term forecasting results of a sample of ETTm2 dataset given by FEDformer enhanced with different normalization methods.
4.3 Comparison With Normalization Methods
In this section, we compare SAN with three state-of-the-art normalization methods for non-stationary time series forecasting: RevIN [17], Non-Stationary Transformers (NST) [25] and Dish-TS [10]. Following the same experimental settings in Section 4.2, we report the average MSE evaluation of Autoformer and FEDformer over all the forecasting lengths for each dataset and the relative improvements in Table 4. Other models are not as involved backbone since NST can only suit Transformer-based models, while the rest methods are much more flexible to be applied for arbitrary forecasting models on the contrary.
It can be concluded that SAN achieves the best performance among existing normalization methods. The improvement is significant with an average MSE decrease of  $10.71\%$  by FEDformer. SAN consistently performs better than baseline models except for Weather and the improvement is more pronounced in typical non-stationary datasets like Exchange and ILI (determined by the ADF test). The comparison reveals that SAN may be more effective at removing non-stationary factors from a time-slicing perspective rather than considering the entire instance. Additionally, the proposed two-stage training schema is crucial as it enables SAN to outperform Dish-TS by a large margin, which ignores the bi-level optimization nature. However, this exceptional ability of SAN may lead to an over-stationarization problem [25], resulting in decreased performance on the Weather dataset. The detailed results of all cases and further discussions are provided in Appendix B.6.
4.4 Qualitative Evaluation
The quality of prediction results in time series forecasting is crucial, in addition to the accuracy of metrics. Figure 3 displays a sample forecast on the ETTm2 dataset using FEDformer as the backbone with SAN, RevIN, NST or Dish-TS enhancements. The input length is 96 and the forecasting length is set to 336. It's evident that SAN produces more realistic predictions while its counterparts even fail to capture the scale of future data. We guess the poor quality of RevIN and NST is caused by their coarse way of denormalizing. Although the mean value of an input sequence can be considered a maximum likelihood estimation for future data, it's likely that non-stationary datasets' distribution will change significantly in comparison to inputs. Therefore, simply denormalizing output from backbone models with input sequence statistics may lead to mismatches like those seen in RevIN and NST forecasts where both scales are similar. As for Dish-TS, though the method tries to learn future distribution, it ignores the bi-level optimization nature and its entangled learning schema limits the estimation accuracy of statistics and finally leads to poor performance. On the opposite, SAN models the dynamic nature of time series from a slicing perspective and introduces an independent statistics prediction module to learn to predict the future distribution for denormalizing by a two-stage training schema. In this way, we adaptively adjust the scale and bias of forecasting results based on the statistic predictions, capturing the tendency of future data. As a result, though the average value of the input is rather low, SAN still produces higher predictions that are consistent with ground truth.
In this study, we focused on alleviating the non-stationary property of time series data using a novel slice view. We proposed the SAN framework for time series forecasting, which is a model-agnostic approach that normalizes the input by removing non-stationary factors and restores them to the output through denormalization on a per-slice basis. Additionally, with the help of a novel statistics prediction module, SAN simplifies non-stationary forecasting by dividing it into two subtasks to improve forecasting model performance. To demonstrate the superiority of SAN, we conducted experiments on a widely used benchmark dataset and found that SAN significantly improves mainstream forecasting models and outperforms state-of-the-art normalization methods. We hope that SAN can serve as a foundation component for time series forecasting, and stimulate further research on modeling time series from a slice perspective.
6 Acknowledgement
This research was partially supported by grants from the National Natural Science Foundation of China (Grant No. U20A20229). This work also thanked to the support of funding MAI2022C007. We furthermore thanked the anonymous reviewers for their constructive comments.
[1] George EP Box and Gwilym M Jenkins. 1968. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C (Applied Statistics) 17, 2 (1968), 91-109.
[2] Hongjie Chen, Ryan A Rossi, Kanak Mahadik, Sungchul Kim, and Hoda Eldardiry. 2021. Graph deep factors for forecasting with applications to cloud resource allocation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 106-116.
[3] Mingyue Cheng, Qi Liu, Zhiding Liu, Zhi Li, Yucong Luo, and Enhong Chen. 2023. FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification. In Proceedings of the ACM Web Conference 2023. 1437-1445.
[4] Mingyue Cheng, Qi Liu, Zhiding Liu, Hao Zhang, Rujiao Zhang, and Enhong Chen. 2023. TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders. arXiv preprint arXiv:2303.00320 (2023).
[5] Benoit Colson, Patrice Marcotte, and Gilles Savard. 2007. An overview of bilevel optimization. Annals of operations research 153, 1 (2007), 235-256.
[6] Shohreh Deldari, Daniel V Smith, Hao Xue, and Flora D Salim. 2021. Time series change point detection with self-supervised contrastive predictive coding. In Proceedings of the Web Conference 2021. 3124-3135.
[7] Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W Tsang. 2021. St-norm: Spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 269-278.
[8] Yuntao Du, Jindong Wang, Wenjie Feng, Sinno Pan, Tao Qin, Renjun Xu, and Chongjun Wang. 2021. Adarnn: Adaptive learning and forecasting of time series. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 402-411.
[9] Graham Elliott, Thomas J Rothenberg, and James H Stock. 1992. Efficient tests for an autoregressive unit root.
[10] Wei Fan, Pengyang Wang, Dongkun Wang, Dongjie Wang, Yuanchun Zhou, and Yanjie Fu. 2023. DishTS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting. arXiv preprint arXiv:2302.14829 (2023).
[11] Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. 2018. Bilevel programming for hyperparameter optimization and meta-learning. In International Conference on Machine Learning. PMLR, 1568-1577.
[12] Stephen Gould, Basura Fernando, Anoop Cherian, Peter Anderson, Rodrigo Santa Cruz, and Edison Guo. 2016. On differentiating parameterized argmin and argmax problems with application to bi-level optimization. arXiv preprint arXiv:1607.05447 (2016).
[32] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting 36, 3 (2020), 1181-1191.
[33] Arunesh Kumar Singh, S Khatoon Ibraheem, Md Muazzam, and DK Chaturvedi. 2013. An overview of electricity demand forecasting techniques. Network and complex systems 3, 3 (2013), 38-48.
[34] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016).
[35] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[36] Qingsong Wen, Zhe Zhang, Yan Li, and Liang Sun. 2020. Fast RobustSTL: Efficient and robust seasonal-trend decomposition for time series with complex patterns. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2203-2213.
[37] Ruofeng Wen, Kari Torkkola, Balakrishnan Narayanaswamy, and Dhruv Madeka. 2017. A multi-horizon quantile recurrent forecaster. arXiv preprint arXiv:1711.11053 (2017).
[38] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems 34 (2021), 22419-22430.
[39] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2023. Are Transformers Effective for Time Series Forecasting? Proceedings of the AAAI Conference on Artificial Intelligence.
[40] G Peter Zhang. 2003. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50 (2003), 159-175.
[41] Yunhao Zhang and Junchi Yan. 2022. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations.
[42] Yi Zheng, Qi Liu, Enhong Chen, Yong Ge, and J Leon Zhao. 2014. Time series classification using multi-channels deep convolutional neural networks. In International conference on web-age information management. Springer, 298-310.
[43] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11106-11115.
[44] Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, et al. 2022. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in Neural Information Processing Systems 35 (2022), 12677-12690.
[45] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proc. 39th International Conference on Machine Learning (ICML 2022) (Baltimore, Maryland).
A Effects of SAN on Non-stationary Time Series Forecasting
A.1 Discussions
As illustrated in the main paper, our proposed SAN is a compact plug-and-play framework. We will first give a brief discussion in this section on how SAN can be effective.
It is of utmost importance that SAN can well alleviate the impact of the non-stationary nature of time series data. Forecasting models may encounter a non-i.i.d problem with non-stationary data, that is, the marginal distribution of each input instance can be different, which may lead to a huge difference between the distribution of the training set and the test set. Thus the models can not generalize well in future predictions. However, SAN will normalize all the input instances into a standard normal distribution and force the mean and variance of the training and test data distributions to be identical. In this way, all the data instances are from the same distribution, therefore the forecasting task is simplified as the models can get rid of the noises caused by non-stationary factors and only focus on mining the time-invariant patterns. Moreover, compared to existing normalization methods for forecasting, our modeling of the non-stationary property in a time slice view is more in-depth and realistic, so SAN can better remove the non-stationary factors in input sequences while keeping their instinct information in the normalization phase. Hence, SAN is theoretically expected to perform better in non-stationary time series forecasting.
Another part that contributes to the effectiveness of SAN is the statistics prediction module and the two-stage training schema. With the statistics prediction module independently modeling the evolving trends of statistical properties, SAN adopts more precise statistics for adaptive denormalization than existing solutions. Moreover, the proposed two-stage strategy actually simplifies the original forecasting task by divide and conquer: In the first stage we try to learn the general direction and dispersion of the future data, which is easy to fit and is conducted by the light statistics prediction module. Next, we utilize the powerful backbone model to discover the scale-free periodic-like features to estimate future values under the guidance of the well-trained statistics prediction module. Therefore, backbone models in SAN are actually responsible for an easier subtask. Considering that SAN can usually give reliable estimations on future distributions, SAN is expected to perform well on non-stationary time series forecasting by splitting the task into two simpler subtasks.
A.2 Theoretical Analysis
Using the same notation in the paper, we prove that all the inputs after SAN's normalization follow a standard normal distribution, validating SAN's capability to remove the non-stationary factors theoretically.
In detail, for arbitrary input sequence  $\pmb{x}^i$ , SAN first split it into  $M$  non-overlapping slices  $\{\pmb{x}_j^i\}_{j=1}^M$  and normalize them according to their statistics. Therefore we will get:
$$
\forall i, j \mathbb {E} \left[ \bar {\boldsymbol {x}} _ {j} ^ {i} \right] = 0, \operatorname {V a r} \left[ \bar {\boldsymbol {x}} _ {j} ^ {i} \right] = I \tag {6}
$$
And as for the statistics of normalized input  $\bar{x}^i$ , it satisfies the following equations:
$$
\begin{array}{l} \mathbb {E} \left[ \bar {\boldsymbol {x}} ^ {i} \right] = \mathbb {E} _ {j} \left[ \mathbb {E} \left[ \bar {\boldsymbol {x}} _ {j} ^ {i} \right] \right] \\ = \mathbb {E} _ {j} [ 0 ] \tag {7} \\ = 0 \\ \end{array}
$$
$$
\begin{array}{l} V a r [ \bar {\boldsymbol {x}} ^ {i} ] = \frac {\sum_ {t = 0} ^ {L _ {i n}} (\bar {x} _ {: , t} ^ {i} - \mathbb {E} [ \bar {\boldsymbol {x}} ^ {i} ]) ^ {2}}{L _ {i n}} \\ = \frac {\sum_ {t = 0} ^ {L _ {i n}} (\bar {x} _ {: , t} ^ {i}) ^ {2}}{M T} \tag {8} \\ = \frac {1}{M} * (\frac {\sum_ {t = 0} ^ {T} (\bar {x} _ {: , t} ^ {i}) ^ {2}}{T} + \frac {\sum_ {t = T} ^ {2 T} (\bar {x} _ {: , t} ^ {i}) ^ {2}}{T} +, \dots , \frac {\sum_ {t = (M - 1) T} ^ {M T} (\bar {x} _ {: , t} ^ {i}) ^ {2}}{T}) \\ = \mathbb {E} _ {j} [ V a r [ \bar {\boldsymbol {x}} _ {j} ^ {i} ] ] \\ = I \\ \end{array}
$$