SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The proof is shown in Appendix D.2. Therefore, the operations of FreMLP, i.e.,  $\mathcal{H}\mathcal{W} + \mathcal{B}$ , are equal to the operations  $(\mathbf{H} * W + B)$  in the time domain. This implies that the operations of frequency-domain MLPs can be viewed as global convolutions in the time domain.
To evaluate the performance of FreTS, we conduct extensive experiments on thirteen real-world time series benchmarks, covering short-term forecasting and long-term forecasting settings to compare with corresponding state-of-the-art methods.
Datasets Our empirical results are performed on various domains of datasets, including traffic, energy, web, traffic, electrocardiogram, and healthcare, etc. Specifically, for the task of short-term forecasting, we adopt Solar  $^{2}$ , Wiki [37], Traffic [37], Electricity  $^{3}$ , ECG [16], METR-LA [38], and COVID-19 [4] datasets, following previous forecasting literature [16]. For the task of long-term forecasting, we adopt Weather [14], Exchange [10], Traffic [14], Electricity [14], and ETT datasets [13], following previous long time series forecasting works [13, 14, 30, 39]. We preprocess all datasets following [16, 13, 14] and normalize them with the min-max normalization. We split the datasets into training, validation, and test sets by the ratio of 7:2:1 except for the COVID-19 datasets with 6:2:2. More dataset details are in Appendix B.1.
Baselines We compare our FreTS with the representative and state-of-the-art models for both short-term and long-term forecasting to evaluate their effectiveness. For short-term forecasting, we comprise FreTS against VAR [23], SFM [29], LSTM [10], TCN [11], GraphWaveNet [27], DeepGLO [37], StemGNN [16], MTGNN [15], and AGCRN [17] for comparison. We also include TAMP-S2GC nets [4], DCRNN [38] and STGCN [40], which require pre-defined graph structures, for comparison. For long-term forecasting, we include Informer [13], Autoformer [14], Reformer [18], FEDformer [30], LTSF-Linear [35], and the more recent PatchTST [39] for comparison. Additional details about the baselines can be found in Appendix B.2.
$^{2}$ https://www.nrel.gov/grid/solar-power-data.html
<sup>3</sup>https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014
Implementation Details Our model is implemented with Pytorch 1.8 [41], and all experiments are conducted on a single NVIDIA RTX 3080 10GB GPU. We take MSE (Mean Squared Error) as the loss function and report MAE (Mean Absolute Errors) and RMSE (Root Mean Squared Errors) results as the evaluation metrics. For additional implementation details, please refer to Appendix B.3.
Table 1: Short-term forecasting comparison. The best results are in bold, and the second best results are underlined. Full benchmarks of short-term forecasting are in Appendix F.1.
Models	Solar	Wiki	Traffic	ECG	Electricity	COVID-19
MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
VAR	0.184	0.234	0.052	0.094	0.535	1.133	0.120	0.170	0.101	0.163	0.226	0.326
SFM	0.161	0.283	0.081	0.156	0.029	0.044	0.095	0.135	0.086	0.129	0.205	0.308
LSTNet	0.148	0.200	0.054	0.090	0.026	0.057	0.079	0.115	0.075	0.138	0.248	0.305
TCN	0.176	0.222	0.094	0.142	0.052	0.067	0.078	0.107	0.057	0.083	0.317	0.354
DeepGLO	0.178	0.400	0.110	0.113	0.025	0.037	0.110	0.163	0.090	0.131	0.169	0.253
Reformer	0.234	0.292	0.047	0.083	0.029	0.042	0.062	0.090	0.078	0.129	0.152	0.209
Informer	0.151	0.199	0.051	0.086	0.020	0.033	0.056	0.085	0.074	0.123	0.200	0.259
Autoformer	0.150	0.193	0.069	0.103	0.029	0.043	0.055	0.081	0.056	0.083	0.159	0.211
FEDformer	0.139	0.182	0.068	0.098	0.025	0.038	0.055	0.080	0.055	0.081	0.160	0.219
GraphWaveNet	0.183	0.238	0.061	0.105	0.013	0.034	0.093	0.142	0.094	0.140	0.201	0.255
StemGNN	0.176	0.222	0.190	0.255	0.080	0.135	0.100	0.130	0.070	0.101	0.421	0.508
MTGNN	0.151	0.207	0.101	0.140	0.013	0.030	0.090	0.139	0.077	0.113	0.394	0.488
AGCRN	0.123	0.214	0.044	0.079	0.084	0.166	0.055	0.080	0.074	0.116	0.254	0.309
FreTS (Ours)	0.120	0.162	0.041	0.074	0.011	0.023	0.053	0.078	0.050	0.076	0.123	0.167
Table 2: Long-term forecasting comparison. We set the lookback window size  $L$  as 96 and the prediction length as  $\tau \in \{96,192,336,720\}$  except for traffic dataset whose prediction length is set as  $\tau \in \{48,96,192,336\}$ . The best results are in bold and the second best are underlined. Full results of long-term forecasting are included in Appendix F.2.
Models Metrics	FreTS	PatchTST	LTSF-Linear	FEDformer	Autoformer	Informer	Reformer
MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
Weather	96	0.032	0.071	0.034	0.074	0.040	0.081	0.050	0.088	0.064	0.104	0.101	0.139	0.108	0.152
192	0.040	0.081	0.042	0.084	0.048	0.089	0.051	0.092	0.061	0.103	0.097	0.134	0.147	0.201
336	0.046	0.090	0.049	0.094	0.056	0.098	0.057	0.100	0.059	0.101	0.115	0.155	0.154	0.203
720	0.055	0.099	0.056	0.102	0.065	0.106	0.064	0.109	0.065	0.110	0.132	0.175	0.173	0.228
Exchange	96	0.037	0.051	0.039	0.052	0.038	0.052	0.050	0.067	0.050	0.066	0.066	0.084	0.126	0.146
192	0.050	0.067	0.055	0.074	0.053	0.069	0.064	0.082	0.063	0.083	0.068	0.088	0.147	0.169
336	0.062	0.082	0.071	0.093	0.064	0.085	0.080	0.105	0.075	0.101	0.093	0.127	0.157	0.189
720	0.088	0.110	0.132	0.166	0.092	0.116	0.151	0.183	0.150	0.181	0.117	0.170	0.166	0.201
Traffic	48	0.018	0.036	0.016	0.032	0.020	0.039	0.022	0.036	0.026	0.042	0.023	0.039	0.035	0.053
96	0.020	0.038	0.018	0.035	0.022	0.042	0.023	0.044	0.033	0.050	0.030	0.047	0.035	0.054
192	0.019	0.038	0.020	0.039	0.020	0.040	0.022	0.042	0.035	0.053	0.034	0.053	0.035	0.054
336	0.020	0.039	0.021	0.040	0.021	0.041	0.021	0.040	0.032	0.050	0.035	0.054	0.035	0.055
Electricity	96	0.039	0.065	0.041	0.067	0.045	0.075	0.049	0.072	0.051	0.075	0.094	0.124	0.095	0.125
192	0.040	0.064	0.042	0.066	0.043	0.070	0.049	0.072	0.072	0.099	0.105	0.138	0.121	0.152
336	0.046	0.072	0.043	0.067	0.044	0.071	0.051	0.075	0.084	0.115	0.112	0.144	0.122	0.152
720	0.052	0.079	0.055	0.081	0.054	0.080	0.055	0.077	0.088	0.119	0.116	0.148	0.120	0.151
ETThl	96	0.061	0.087	0.065	0.091	0.063	0.089	0.072	0.096	0.079	0.105	0.093	0.121	0.113	0.143
192	0.065	0.091	0.069	0.094	0.067	0.094	0.076	0.100	0.086	0.114	0.103	0.137	0.120	0.148
336	0.070	0.096	0.073	0.099	0.070	0.097	0.080	0.105	0.088	0.119	0.112	0.145	0.124	0.155
720	0.082	0.108	0.087	0.113	0.082	0.108	0.090	0.116	0.102	0.136	0.125	0.157	0.126	0.155
ETTm1	96	0.052	0.077	0.055	0.082	0.055	0.080	0.063	0.087	0.081	0.109	0.070	0.096	0.065	0.089
192	0.057	0.083	0.059	0.085	0.060	0.087	0.068	0.093	0.083	0.112	0.082	0.107	0.081	0.108
336	0.062	0.089	0.064	0.091	0.065	0.093	0.075	0.102	0.091	0.125	0.090	0.119	0.100	0.128
720	0.069	0.096	0.070	0.097	0.072	0.099	0.081	0.108	0.093	0.126	0.115	0.149	0.132	0.163
Short-Term Time Series Forecasting Table 1 presents the forecasting accuracy of our FreTS compared to thirteen baselines on six datasets, with an input length of 12 and a prediction length of 12. The best results are highlighted in bold and the second-best results are underlined. From the table, we observe that FreTS outperforms all baselines on MAE and RMSE across all datasets, and on average it makes improvement of  $9.4\%$  on MAE and  $11.6\%$  on RMSE. We credit this to the fact that FreTS explicitly models both channel and temporal dependencies, and it flexibly unifies channel and temporal modeling in the frequency domain, which can effectively capture the key patterns with the global view and energy compaction. We further report the complete benchmarks of short-term forecasting under different steps on different datasets (including METR-LA dataset) in Appendix F.1.
Long-term Time Series Forecasting Table 2 showcases the long-term forecasting results of FreTS compared to six representative baselines on six benchmarks with various prediction lengths. For the traffic dataset, we select 48 as the lookback window size  $L$  with the prediction lengths  $\tau \in \{48,96,192,336\}$ . For the other datasets, the input lookback window length is set to 96 and the prediction length is set to  $\tau \in \{96,192,336,720\}$ . The results demonstrate that FreTS outperforms all baselines on all datasets. Quantitatively, compared with the best results of Transformer-based models, FreTS has an average decrease of more than  $20\%$  in MAE and RMSE. Compared with more recent LSTF-Linear [35] and the SOTA PathchTST [39], FreTS can still outperform them in general. In addition, we provide further comparison of FreTS and other baselines and report performance under different lookback window sizes in Appendix F.2. Combining Tables 1 and 2, we can conclude that FreTS achieves competitive performance in both short-term and long-term forecasting task.
Frequency Channel and Temporal Learners We analyze the effects of frequency channel and temporal learners in Table 3 in both short-term and long-term experimental settings. We consider two variants: FreCL: we remove the frequency temporal learner from FreTS, and FreTL: we remove the frequency channel learner from FreTS. From the comparison, we observe that the frequency chan
Table 3: Ablation studies of frequency channel and temporal learners in both short-term and long-term forecasting. 'I/O' indicates lookback window sizes/prediction lengths.
Tasks	Short-term	Long-term
Dataset I/O	Electricity 12/12	METR-LA 12/12	Exchange 96/336	Weather 96/336
Metrics	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
FreCL	0.054	0.080	0.086	0.168	0.067	0.086	0.051	0.094
FreTL	0.058	0.086	0.085	0.167	0.065	0.085	0.047	0.091
FreTS	0.050	0.076	0.080	0.166	0.062	0.082	0.046	0.090
nel learner plays a more important role in short-term forecasting. In long-term forecasting, we note that the frequency temporal learner is more effective than the frequency channel learner. In Appendix E.1, we also conduct the experiments and report performance on other datasets. Interestingly, we find out the channel learner would lead to the worse performance in some long-term forecasting cases. A potential explanation is that the channel independent strategy [39] brings more benefit to forecasting.
FreMLP vs. MLP We further study the effectiveness of FreMLP in time series forecasting. We use FreMLP to replace the original MLP component in the existing SOTA MLP-based models (i.e., DLinear and NLinear [35]), and compare their performances with the original DLinear and NLinear under the same experimental settings. The experimental results are presented in Table 4. From the table, we easily observe that for any prediction length, the performance of both DLinear and NLinear models has been improved after replacing the corresponding MLP component with our FreMLP. Quantitatively, incorporating FreMLP into the DLinear model brings an average improvement of  $6.4\%$  in MAE and  $11.4\%$  in RMSE on the Exchange dataset, and  $4.9\%$  in MAE and  $3.5\%$  in RMSE on the Weather dataset. A similar improvement has also been achieved on the two datasets with regard to NLinear, according to Table 4. These results confirm the effectiveness of FreMLP compared to MLP again and we include more implementation details and analysis in Appendix B.5.
Table 4: Ablation study on the Exchange and Weather datasets with a lookback window size of 96 and the prediction length  $\tau \in \{96,192,336,720\}$ . DLinear (FreMLP)/NLinear (FreMLP) means that we replace the MLPs in DLinear/NLinear with FreMLP. The best results are in bold.
Datasets	Exchange	Weather
Lengths	96	192	336	720	96	192	336	720
Metrics	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
DLinear	0.037	0.051	0.054	0.072	0.071	0.095	0.095	0.119	0.041	0.081	0.047	0.089	0.056	0.098	0.065	0.106
DLinear (FreMLP)	0.036	0.049	0.053	0.071	0.063	0.071	0.086	0.101	0.038	0.078	0.045	0.086	0.055	0.097	0.061	0.100
NLinear	0.037	0.051	0.051	0.069	0.069	0.093	0.115	0.146	0.037	0.081	0.045	0.089	0.052	0.098	0.058	0.106
NLinear (FreMLP)	0.036	0.050	0.049	0.067	0.067	0.091	0.109	0.139	0.035	0.076	0.043	0.084	0.050	0.094	0.057	0.103
The complexity of our proposed FreTS is  $\mathcal{O}(N\log N + L\log L)$ . We perform efficiency comparisons with some state-of-the-art GNN-based methods and Transformer-based models under different numbers of variables  $N$  and prediction lengths  $\tau$ , respectively. On the Wiki dataset, we conduct experiments over  $N\in \{1000,2000,3000,4000,5000\}$  under the same lookback window size of 12
(a) Parameters (left) and training time (right) under different variable numbers
(b) Parameters (left) and training time (right) under different prediction lengths
Figure 4: Efficiency analysis (model parameters and training time) on the Wiki and Exchange dataset. (a) The efficiency comparison under different number of variables: the number of variables is enlarged from 1000 to 5000 with the input window size as 12 and the prediction length as 12 on Wiki dataset. (b) The efficiency comparison under the prediction lengths: we conduct experiments with prediction lengths prolonged from 96 to 480 under the same window size of 96 on the Exchange dataset.
and prediction length of 12, as shown in Figure 4(a). From the figure, we can find that: (1) The amount of FreTS parameters is agnostic to  $N$ . (2) Compared with AGCRN, FreTS incurs an average  $30\%$  reduction of the number of parameters and  $20\%$  reduction of training time. On the Exchange dataset, we conduct experiments on different prediction lengths  $\tau \in \{96, 192, 336, 480\}$  with the same input length of 96. The results are shown in Figure 4(b). It demonstrates: (1) Compared with Transformer-based methods (FEDformer [30], Autoformer [14], and Informer [13]), FreTS reduces the number of parameters by at least 3 times. (2) The training time of FreTS is averagely 3 times faster than Informer, 5 times faster than Autoformer, and more than 10 times faster than FEDformer. These show our great potential in real-world deployment.
4.4 Visualization Analysis
In Figure 5, we visualize the learned weights  $\mathcal{W}$  in FreMLP on the Traffic dataset with a lookback window size of 48 and prediction length of 192. As the weights  $\mathcal{W}$  are complex numbers, we provide visualizations of the real part  $\mathcal{W}_r$  (presented in (a)) and the imaginary part  $\mathcal{W}_i$  (presented in (b)) separately. From the figure, we can observe that both the real and imaginary parts play a crucial role in learning process: the weight coefficients of the real or imaginary part exhibit energy aggregation characteristics (clear diagonal pat
terns) which can facilitate to learn the significant features. In Appendix E.2, we further conduct a detailed analysis on the effects of the real and imaginary parts in different contexts of forecasting, and the effects of the two parts in the FreMLP. We examine their individual contributions and investigate how they influence the final performance. Additional visualizations of the weights on different datasets with various settings, as well as visualizations of global periodic patterns, can be found in Appendix G.1 and Appendix G.2, respectively.
(a) The real part  $\mathcal{W}_r$
Figure 5: Visualizing learned weights of FreMLP on the Traffic dataset.  $\mathcal{W}_r$  represents the real part of  $\mathcal{W}$  and  $\mathcal{W}_i$  represents the imaginary part.
(b) The imaginary part  $\mathcal{W}_i$
5 Conclusion Remarks
In this paper, we explore a novel direction and make a new attempt to apply frequency-domain MLPs for time series forecasting. We have redesigned MLPs in the frequency domain that can effectively capture the underlying patterns of time series with global view and energy compaction. We then verify this design by a simple yet effective architecture, FreTS, built upon the frequency-domain MLPs for time series forecasting. Our comprehensive empirical experiments on seven benchmarks of short-term forecasting and six benchmarks of long-term forecasting have validated the superiority of our proposed methods. Simple MLPs have several advantages and lay the foundation of modern deep learning, which have great potential for satisfied performance with high efficiency. We hope this work can facilitate more future research of MLPs on time series modeling.
[1] Edward N Lorenz. Empirical orthogonal functions and statistical weather prediction, volume 1. Massachusetts Institute of Technology, Department of Meteorology Cambridge, 1956.
[2] Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine-grained air quality based on big data. In KDD, pages 2267-2276, 2015.
[3] Hui He, Qi Zhang, Simeng Bai, Kun Yi, and Zhendong Niu. CATN: cross attentive tree-aware network for multivariate time series forecasting. In AAAI, pages 4030-4038. AAAI Press, 2022.
[4] Yuzhou Chen, Ignacio Segovia-Dominguez, Baris Coskunuzer, and Yulia Gel. TAMP-s2GCNets: Coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In International Conference on Learning Representations, 2022.
[5] Benjamin F King. Market and industry factors in stock price behavior. the Journal of Business, 39(1):139-190, 1966.
[6] Adebiyi A Ariyo, Adewumi O Adewumi, and Charles K Ayo. Stock price prediction using the arima model. In 2014 UKSim-AMSS 16th international conference on computer modelling and simulation, pages 106–112. IEEE, 2014.
[7] Charles C Holt. Forecasting trends and seasonal by exponentially weighted moving averages. ONR Memorandum, 52(2), 1957.
[8] Peter Whittle. Prediction and regulation by linear least-square methods. English Universities Press, 1963.
[9] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deeper: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181-1191, 2020.
[10] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In SIGIR, pages 95–104, 2018.
[11] Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR, abs/1803.01271, 2018.
[12] Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816-5828, 2022.
[13] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, pages 11106-11115, 2021.
[14] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In NeurIPS, pages 22419-22430, 2021.
[15] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In KDD, pages 753-763, 2020.
[16] Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. Spectral temporal graph neural network for multivariate time-series forecasting. In NeurIPS, 2020.
[36] Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In ICLR (Workshop Poster), 2013.
[37] Rajat Sen, Hsiang-Fu Yu, and Inderjit S. Dhillon. Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. In NeurIPS, pages 4838-4847, 2019.
[38] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In ICLR (Poster), 2018.
[39] Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
[40] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In *IJCAI*, pages 3634–3640, 2018.
[41] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, pages 8024–8035, 2019.
Table 5: Notation.
Xt	multivariate time series with a lookback window of L
at timestamps t, Xt ∈ RN×L
Xt	the multivariate values of N distinct series at timestamp t, Xt ∈ RN
Yt	the prediction target with a horizon window of length τ
at timestamps t, Yt ∈ RN×τ
Ht	the hidden representation of Xt, Ht ∈ RN×L×d
Zt	the output of the frequency channel learner, Zt ∈ RN×L×d
St	the output of the frequency temporal learner, St ∈ RN×L×d
Hchan	the domain conversion of Ht on channel dimensions, Hchan ∈ CN×L×d
Zchan	the FreMLP output of Hchan, Zchan ∈ CN×L×d
Ztemp	the domain conversion of Zt on temporal dimensions, Ztemp ∈ CN×L×d
Stemp	the FreMLP output of Ztemp, Stemp ∈ CN×L×d
Wchan	the complex number weight matrix of FreMLP in the frequency
channel learner, Wchan ∈ CD×d
Bchan	the complex number bias of FreMLP in the frequency channel
learner, Bchan ∈ CD
Wtemp	the complex number weight matrix of FreMLP in the frequency
temporal learner, Wtemp ∈ CD×d
Btemp	the complex number bias of FreMLP in the frequency
temporal learner, Btemp ∈ CD
We adopt thirteen real-world benchmarks in the experiments to evaluate the accuracy of short-term and long-term forecasting. The details of the datasets are as follows:
Solar<sup>4</sup>: It is about the solar power collected by National Renewable Energy Laboratory. We choose the power plant data points in Florida as the data set which contains 593 points. The data is collected from 01/01/2006 to 31/12/2016 with the sampling interval of every 1 hour.
Wiki [37]: It contains a number of daily views of different Wikipedia articles and is collected from 1/7/2015 to 31/12/2016. It consists of approximately  $145k$  time series and we randomly choose  $5k$  from them as our experimental data set.
Traffic [37]: It contains hourly traffic data from 963 San Francisco freeway car lanes for short-term forecasting settings while it contains 862 car lanes for long-term forecasting. It is collected since 01/01/2015 with a sampling interval of every 1 hour.
ECG<sup>5</sup>: It is about Electrocardiogram(ECG) from the UCR time-series classification archive. It contains 140 nodes and each node has a length of 5000.
<sup>4</sup>https://www.nrel.gov/grid/solar-power-data.html
<sup>5</sup>http://www.timeseriesclassification.com/description.php?Dataset=ECG5000
Electricity<sup>6</sup>: It contains electricity consumption of 370 clients for short-term forecasting while it contains electricity consumption of 321 clients for long-term forecasting. It is collected since 01/01/2011. The data sampling interval is every 15 minutes.
COVID-19 [4]: It is about COVID-19 hospitalization in the U.S. state of California (CA) from 01/02/2020 to 31/12/2020 provided by the Johns Hopkins University with the sampling interval of every day.
METR-LA<sup>7</sup>: It contains traffic information collected from loop detectors in the highway of Los Angeles County. It contains 207 sensors which are from 01/03/2012 to 30/06/2012 and the data sampling interval is every 5 minutes.
Exchange<sup>8</sup>: It contains the collection of the daily exchange rates of eight foreign countries including Australia, British, Canada, Switzerland, China, Japan, New Zealand, and Singapore ranging from 1990 to 2016 and the data sampling interval is every 1 day.
Weather<sup>9</sup>: It collects 21 meteorological indicators, such as humidity and air temperature, from the Weather Station of the Max Planck Biogeochemistry Institute in Germany in 2020. The data sampling interval is every 10 minutes.
$\mathbf{E}\mathbf{T}\mathbf{T}^{10}$ : It is collected from two different electric transformers labeled with 1 and 2, and each of them contains 2 different resolutions (15 minutes and 1 hour) denoted with m and h. We use ETTh1 and ETTm1 as our long-term forecasting benchmarks.