SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 2: The network architecture of MTS forecasting with FourierGNN (blue characters denote complex values, such as  $\mathcal{X}_t^{\mathcal{G}}\mathcal{S}_i$ ). Given the hypervariate graph  $\mathcal{G} = (X_t^{\mathcal{G}},A_t^{\mathcal{G}})$ , we 1) embed nodes of  $X_{t}^{\mathcal{G}}\in \mathbb{R}^{NT\times 1}$  to obtain node embeddings  $\mathbf{X}_t^{\mathcal{G}}\in \mathbb{R}^{NT\times d}$ ; 2) feed embedded hypervariate graphs to FourierGNN: (i) transform  $\mathbf{X}_t^{\mathcal{G}}$  with DFT to  $\mathcal{X}_t^{\mathcal{G}}\in \mathbb{C}^{NT\times d}$ ; (ii) conduct recursive multiplications and make summation to output  $\mathcal{Y}_t^{\mathcal{G}}$ ; (iii) transform  $\mathcal{Y}_t^{\mathcal{G}}$  back to time domain by IDFT, resulting in  $\mathbf{Y}_t^{\mathcal{G}}\in \mathbb{R}^{NT\times d}$ ; 3) generate  $\tau$ -step predictions  $\hat{Y}_t\in \mathbb{R}^{N\times \tau}$  via feeding  $\mathbf{Y}_t^{\mathcal{G}}$  to fully-connected layers.
Subsequently, to capture the spatiotemporal dependencies simultaneously, we aim to feed multiple embedded hypervariate graphs with  $\mathbf{X}_t^{\mathcal{G}}$  to FourierGNN. First, we perform Discrete Fourier Transform (DFT)  $\mathcal{F}$  on each discrete spatio-temporal dimension of the embeddings  $\mathbf{X}_t^{\mathcal{G}}$  and obtain the frequency output  $\mathcal{X}_t^{\mathcal{G}}\coloneqq \mathcal{F}(\mathbf{X}_t^{\mathcal{G}})\in \mathbb{C}^{NT\times d}$ . Then, we perform a recursive multiplication between  $\mathcal{X}_t^{\mathcal{G}}$  and FGOs  $S_{0:k}$  in Fourier space and output the resulting representations  $\mathcal{Y}_t^{\mathcal{G}}$  as:
$$
\mathcal {Y} _ {t} ^ {\mathcal {G}} = \operatorname {F o u r i e r G N N} \left(\mathbf {X} _ {t} ^ {\mathcal {G}}, A _ {t} ^ {\mathcal {G}}\right) = \sum_ {k = 0} ^ {K} \sigma \left(\mathcal {F} \left(\mathbf {X} _ {t} ^ {\mathcal {G}}\right) \mathcal {S} _ {0: k} + b _ {k}\right), \quad \mathcal {S} _ {0: k} = \prod_ {i = 0} ^ {k} \mathcal {S} _ {i}. \tag {8}
$$
Then  $\mathcal{Y}_t^{\mathcal{G}}$  are transformed back to the time domain using Inverse Discrete Fourier Transform (IDFT)  $\mathcal{F}^{-1}$ , which yields  $\mathbf{Y}_t^{\mathcal{G}}\coloneqq \mathcal{F}^{-1}(\mathcal{Y}_t^{\mathcal{G}})\in \mathbb{R}^{NT\times d}$ .
Finally, according to the FourierGNN output  $\mathbf{Y}_t^{\mathcal{G}}$  which encodes spatiotemporal inter-dependencies, we use two layer feed-forward networks (FFN) (see Appendix E.4 for more details) to project it onto  $\tau$  future steps, resulting in  $\hat{Y}_{t} = \mathrm{FFN}(\mathbf{Y}_{t}^{\mathcal{G}})\in \mathbb{R}^{N\times \tau}$ .
To evaluate the performance of FourierGNN, we conduct extensive experiments on seven real-world time series benchmarks to compare with state-of-the-art graph neural network-based methods.
Datasets. We evaluate our proposed method on seven representative datasets from various application scenarios, including traffic, energy, web traffic, electrocardiogram, and COVID-19. All datasets are normalized using the min-max normalization. Except the COVID-19 dataset, we split the other datasets into training, validation, and test sets with the ratio of 7:2:1 in a chronological order. For the COVID-19 dataset, the ratio is 6:2:2. More detailed information about datasets are in Appendix E.1.
Baselines. We conduct a comprehensive comparison of the forecasting performance between our FourierGNN and several representative and state-of-the-art (SOTA) models on the seven datasets, including classic method VAR [32], deep learning-based models such as SFM [24], LSTM [33], TCN [8], DeepGLO [34], and CoST [36]. We also compare FourierGNN against GNN-based models like GraphWaveNet [17], StemGNN [10], MTGNN [13], and AGCRN [2], and two representative Transformer-based models like Reformer [35] and Informer [9], as well as two frequency enhanced Transformer-based models including Autoformer [4] and FEDformer [27]. In addition, we compare FourierGNN with SOTA models such as TAMP-S2GCnets [11], DCRNN [18], and STGCN [1], which require pre-defined graph structures. Please refer to Appendix E.2 for more implementation details of the adopted baselines.
Experimental Settings. All experiments are conducted in Python using Pytorch 1.8 [37] (except for SFM [24] which uses Keras) and performed on single NVIDIA RTX 3080 10G GPU. Our model is trained using RMSProp with a learning rate of  $10^{-5}$  and MSE (Mean Squared Error) as the loss function. The best parameters for all comparative models are chosen through careful parameter tuning on the validation set. We use Mean Absolute Errors (MAE), Root Mean Squared Errors (RMSE), and
Table 1: Overall performance of forecasting models on the six datasets.
Datasets
Models	Solar	Wiki	Traffic
MAE	RMSE	MAPE(%)	MAE	RMSE	MAPE(%)	MAE	RMSE	MAPE(%)
VAR [32]	0.184	0.234	577.10	0.057	0.094	96.58	0.535	1.133	550.12
SFM [24]	0.161	0.283	362.89	0.081	0.156	104.47	0.029	0.044	59.33
LSTNet [33]	0.148	0.200	132.95	0.054	0.090	118.24	0.026	0.057	25.77
TCN [8]	0.176	0.222	142.23	0.094	0.142	99.66	0.052	0.067	-
DeepGLO [34]	0.178	0.400	346.78	0.110	0.113	119.60	0.025	0.037	33.32
Reformer [35]	0.234	0.292	128.58	0.048	0.085	73.61	0.029	0.042	112.58
Informer [9]	0.151	0.199	128.45	0.051	0.086	80.50	0.020	0.033	59.34
Autoformer [4]	0.150	0.193	103.79	0.069	0.103	121.90	0.029	0.043	100.02
FEDformer [27]	0.139	0.182	100.92	0.068	0.098	123.10	0.025	0.038	85.12
GraphWaveNet [17]	0.183	0.238	603	0.061	0.105	136.12	0.013	0.034	33.78
StemGNN [10]	0.176	0.222	128.39	0.190	0.255	117.92	0.080	0.135	64.51
MTGNN [13]	0.151	0.207	507.91	0.101	0.140	122.96	0.013	0.030	29.53
AGCRN [2]	0.123	0.214	353.03	0.044	0.079	78.52	0.084	0.166	31.73
FourierGNN	0.120	0.162	116.48	0.041	0.076	64.50	0.011	0.023	28.71
Datasets
Models	ECG	Electricity	COVID-19
MAE	RMSE	MAPE(%)	MAE	RMSE	MAPE(%)	MAE	RMSE	MAPE(%)
VAR [32]	0.120	0.170	22.56	0.101	0.163	43.11	0.226	0.326	191.95
SFM [24]	0.095	0.135	24.20	0.086	0.129	33.71	0.205	0.308	76.08
LSTNet [33]	0.079	0.115	18.68	0.075	0.138	29.95	0.248	0.305	89.04
TCN [8]	0.078	0.107	17.59	0.057	0.083	26.64	0.317	0.354	151.78
DeepGLO [34]	0.110	0.163	43.90	0.090	0.131	29.40	0.169	0.253	75.19
Reformer [35]	0.062	0.090	13.58	0.078	0.129	33.37	0.152	0.209	132.78
Informer [9]	0.056	0.085	11.99	0.070	0.119	32.66	0.200	0.259	155.55
Autoformer [4]	0.055	0.081	11.37	0.056	0.083	25.94	0.159	0.211	136.24
FEDformer [27]	0.055	0.080	11.16	0.055	0.081	25.84	0.160	0.219	134.45
GraphWaveNet [17]	0.093	0.142	40.19	0.094	0.140	37.01	0.201	0.255	100.83
StemGNN [10]	0.100	0.130	29.62	0.070	0.101	-	0.421	0.508	141.01
MTGNN [13]	0.090	0.139	35.04	0.077	0.113	29.77	0.394	0.488	88.13
AGCRN [2]	0.055	0.080	11.75	0.074	0.116	26.08	0.254	0.309	83.37
FourierGNN	0.052	0.078	10.97	0.051	0.077	24.28	0.123	0.168	71.52
Table 2: Performance comparison under different prediction lengths on the COVID-19 dataset.
Length Metrics	3	6	9	12
MAE	RMSE	MAPE(%)	MAE	RMSE	MAPE(%)	MAE	RMSE	MAPE(%)	MAE	RMSE	MAPE(%)
GraphWaveNet [17]	0.092	0.129	53.00	0.133	0.179	65.11	0.171	0.225	80.91	0.201	0.255	100.83
StemGNN [10]	0.247	0.318	99.98	0.344	0.429	125.81	0.359	0.442	131.14	0.421	0.508	141.01
AGCRN [2]	0.130	0.172	76.73	0.171	0.218	79.07	0.224	0.277	82.90	0.254	0.309	83.37
MTGNN [13]	0.276	0.379	91.42	0.446	0.513	133.49	0.484	0.548	139.52	0.394	0.488	88.13
TAMP-S2GCnets [11]	0.140	0.190	50.01	0.150	0.200	55.72	0.170	0.230	71.78	0.180	0.230	65.76
CoST [36]	0.122	0.246	68.74	0.157	0.318	72.84	0.183	0.364	77.04	0.202	0.377	80.81
FourierGNN(ours)	0.071	0.103	61.02	0.093	0.131	65.72	0.109	0.148	69.59	0.123	0.168	71.52
Mean Absolute Percentage Error (MAPE) to measure the performance. The evaluation details are in Appendix E.3 and more experimental settings are in Appendix E.4.
We present the evaluation results with an input length of 12 and a prediction length of 12 in Table 1. Overall, FourierGNN achieves a new state-of-the-art on all datasets. On average, FourierGNN makes an improvement of  $9.4\%$  in MAE and  $10.9\%$  in RMSE compared to the best-performing across all datasets. Among these baselines, Reformer, Informer, Autoformer, and FEDformer are Transformer-based models that demonstrate competitive performance on Electricity and COVID-19 datasets, as they excel at capturing temporal dependencies. However, they have limitations in capturing the spatial dependencies explicitly. GraphWaveNet, MTGNN, StemGNN, and AGCRN are GNN-based models that show promising results on Wiki, Traffic, Solar, and ECG datasets, primarily due to their capability to handle spatial dependencies among variates. However, they are limited in their capacity to simultaneously capture spatiotemporal dependencies. FourierGNN outperforms the baseline models since it can learn comprehensive spatiotemporal dependencies simultaneously and attends to time-varying dependencies among variates.
Multi-Step Forecasting To further evaluate the performance in multi-step forecasting, we compare FourierGNN with other GNN-based MTS models (including StemGNN [10], AGCRN [2],
GraphWaveNet [17], MTGNN [13], and TAMP-S2GCNets [11]) and a representation learning model (CoST [36]) on COVID-19 dataset under different prediction lengths, and the results are shown in Table 2. It shows that FourierGNN achieves an average  $30.1\%$  and  $30.2\%$  improvement on MAE and RMSE respectively over the best baseline. In Appendix F, we include more experiments and analysis under different prediction lengths, and further compare FourierGNN with models that require pre-defined graph structures.
5.3 Model Analysis
Efficiency Analysis We investigate the parameter volumes and training time costs of FourierGNN, StemGNN [10], AGCRN [2], GraphWaveNet [17], and MTGNN [13] on two representative datasets, including the Wiki dataset and the Traffic dataset. The results are reported in Table 3, showing the comparison of parameter volumes and average time costs over five rounds of experiments. In terms of parameters, FourierGNN exhibits the lowest volume of parameters among the comparative models. Specifically, it achieves a reduction of  $32.2\%$  and  $9.5\%$  in parameters compared to GraphWaveNet on Traffic and Wiki datasets, respectively. This reduction is mainly attributed that FourierGNN has shared scale-free parameters for each node. Regarding training time, FourierGNN runs much faster than all baseline models, and it demonstrates efficiency improvements of  $5.8\%$  and  $23.3\%$  over the fast baseline GraphWaveNet on Traffic and Wiki datasets, respectively. Considering variate number of Wiki dataset is about twice larger than that of Traffic dataset, FourierGNN exhibits larger efficiency superiority with the baselines. These findings highlight the high efficiency of FourierGNN in computing graph operations and its scalability to large datasets with extensive graphs, which is important for the pure graph formulation due to the larger size of hypervariate graphs with  $NT$  nodes.
Table 3: Comparisons of parameter volumes and training time costs on datasets Traffic and Wiki.
Models	Traffic	Parameters	Wiki Training (s/epoch)
Parameters	Training (s/epoch)
StemGNN	1,606,140	185.86±2.22	4,102,406	92.95±1.39
MTGNN	707,516	169.34±1.56	1,533,436	28.69±0.83
AGCRN	749,940	113.46±1.91	755,740	22.48±1.01
GraphWaveNet	280,860	105.38±1.24	292,460	21.23±0.76
FourierGNN	190,564	99.25±1.07	264,804	16.28±0.48
Ablation Study We perform an ablation study on the METR-LA dataset to assess the individual contributions of different components in FourierGNN. The results, presented in Table 4, validate the effectiveness of each component. Specifically, w/o Embedding emphasizes the significance of performing node embedding to improve model generalization. w/o Dynamic FGO using the same FGO verifies the effectiveness of applying different FGOs in capturing time-varying dependencies. In addition, w/o Residual represents FourierGNN without the  $K = 0$  layer, while w/o Summation adopts the last order (layer) output, i.e.,  $\mathcal{X}S_{0:K}$ , as the output of FourierGNN. These results demonstrate the importance of high-order diffusion and the contribution of multi-order diffusion. More results and analysis of the ablation study are provided in Appendix G.3.
Table 4: Ablation study on METR-LA dataset.
metrics	w/o Embedding	w/o Dynamic FGO	w/o Residual	w/o Summation	FourierGNN
MAE	0.053	0.055	0.054	0.054	0.050
RMSE	0.116	0.114	0.115	0.114	0.113
MAPE(%)	86.73	86.69	86.75	86.62	86.30
5.4 Visualization
To gain a better understanding of the hypervariate graph and FourierGNN in spatiotemporal modeling for MTS forecasting, We conduct visualization experiments on the METR-LA and COVID-19 datasets. Please refer to Appendix E.5 for more information on the visualization techniques used.
Visualization of temporal representations learned by FourierGNN In order to showcase the temporal dependencies learning capability of FourierGNN, we visualize the temporal adjacency matrix of different variates. Specifically, we randomly select 8 counties from the COVID-19 dataset and calculate the relations of 12 consecutive time steps for each county. Then, we visualize the
(a)  $N = 7$
(b)  $\mathrm{N} = {18}$
(c)  $\mathrm{N} = {24}$
(d)  $\mathrm{N} = 29$
(e)  $\mathrm{N} = 38$
(f)  $\mathrm{N} = {45}$
(g)  $\mathrm{N} = 46$
Figure 3: The temporal adjacency matrix of eight variates on COVID-19 dataset.
(h)  $\mathrm{N} = 55$
adjacency matrix by a heatmap, and the results are illustrated in Figure 3, where  $N$  denotes the index of the country (variate). It shows that FourierGNN learns distinct temporal patterns for each county, indicating that the hypervariate graph can encode rich and discriminative temporal dependencies.
Visualization of spatial representations learned by FourierGNN To investigate the spatial correlations learning capability of FourierGNN, we visualize the generated adjacency matrix based on the representations learned by FourierGNN on the METR-LA dataset. Specifically, we randomly select 20 detectors and visualize their corresponding adjacency matrix via a heatmap, as depicted in Figure 4.
By examining the adjacency matrix in conjunction with the actual road map, we observe: 1) the detectors (7, 8, 9, 11, 13, 18) are very close w.r.t. the physical distance, corresponding to the high values of their correlations with each other in the heatmap; 2) the detectors 4, 14 and 16 have small overall correlation values since they are far from other detectors; 3) however, compared with detectors 14 and 16, the detector 4 has slightly higher correlation values to other detectors, e.g., 7, 8, 9, which is because although they are far
Figure 4: The adjacency matrix (right) learned by FourierGNN and the corresponding road map (left).
apart, the detectors 4, 7, 8, 9 are on the same road. The results verify that the hypervariate graph structure can represent highly interpretative correlations.
Moreover, to gain a understanding of how FGO works, we visualize the output of each layer of FourierGNN, and the visualization results demonstrate that FGO can adaptively and effectively capture important patterns while removing noises to a learn discriminative model. Further details can be found in Appendix H.1. Additionally, to investigate the ability of FourierGNN to capture time-varying dependencies among variates, we further visualize the spatial correlations at different timestamps. The results illustrate that FourierGNN can effectively attend to the temporal variability in the data. For more information, please refer to Appendix H.2.
6 Conclusion Remarks
In this paper, we explore an interesting direction of directly applying graph networks for MTS forecasting from a pure graph perspective. To overcome the previous separate spatial and temporal modeling problem, we build a hypervariate graph, regarding each series value as a graph node, which considers spatiotemporal dynamics unitedly. Then, we formulate time series forecasting on the hypervariate graph and propose FourierGNN by stacking Fourier Graph Operator (FGO) to perform matrix multiplications in Fourier space, which can accommodate adequate learning expressiveness with much lower complexity. Extensive experiments demonstrate that FourierGNN achieves state-of-the-art performances with higher efficiency and fewer parameters, and the hypervariate graph structure exhibits strong capabilities to encode spatiotemporal inter-dependencies.
The work was supported in part by the National Key Research and Development Program of China under Grant 2020AAA0104903 and 2019YFB1406300, and National Natural Science Foundation of China under Grant 62072039 and 62272048.
[1] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In *IJCAI*, pages 3634–3640, 2018.
[2] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. In NeurIPS, 2020.
[3] Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine-grained air quality based on big data. In KDD, pages 2267-2276, 2015.
[4] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In NeurIPS, pages 22419-22430, 2021.
[5] Hui He, Qi Zhang, Simeng Bai, Kun Yi, and Zhendong Niu. CATN: cross attentive tree-aware network for multivariate time series forecasting. In AAAI, pages 4030-4038. AAAI Press, 2022.
[6] Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie-Yan Liu. DEPTS: deep expansion learning for periodic time series forecasting. In ICLR. OpenReview.net, 2022.
[7] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deeper: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181-1191, 2020.
[8] Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR, abs/1803.01271, 2018.
[9] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, pages 11106-11115, 2021.
[10] Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. Spectral temporal graph neural network for multivariate time-series forecasting. In NeurIPS, 2020.
[11] Yuzhou Chen, Ignacio Segovia-Dominguez, Baris Coskunuzer, and Yulia Gel. TAMP-s2GCNets: Coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In International Conference on Learning Representations, 2022.
[12] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In *IJCAI*, pages 3634–3640, 2018.
[13] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In KDD, pages 753-763, 2020.
[14] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR (Poster). OpenReview.net, 2017.
[15] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. CoRR, abs/1710.10903, 2017.
[16] Kate Smith-Miles and Leo Lopes. Measuring instance difficulty for combinatorial optimization problems. Comput. Oper. Res., 39(5):875-889, 2012.
[37] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, pages 8024-8035, 2019.
[38] Hoang Anh Dau, Anthony J. Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn J. Keogh. The UCR time series archive. IEEE CAA J. Autom. Sinica, 6(6):1293-1305, 2019.
Table 5: Notations
Xt	multivariate time series input at timestamps t, X ∈ RN×T
xt	the multivariate values of N series at timestamp t, xt ∈ RN
Yt	the next τ timestamps of multivariate time series, Yt ∈ RN×τ
Ŷt	the prediction values of multivariate time series for next τ timestamps,Ŷt ∈ RN×τ
N	the number of series
T	the lookback window size
τ	the prediction length of multivariate time series forecasting
Gt	the hypervariate graph, Gt = {XtG, AtG} attributed to XtG
XtG	the nodes of the hypervariate graph, XtG ∈ RNT×1
AtG	the adjacency matrix of Gt, AtG ∈ RNT×NT
S	the Fourier Graph Operator
d	the embedding dimension
XtG	the embedding of XtG, XtG ∈ RNT×d
XtG	the spectrum of XtG, XtG ∈ CNT×d
YtG	the output of FourierGNN, YtG ∈ CNT×d
θg	the parameters of the graph network
θt	the parameters of the temporal network
θG	the network parameters for hypervariate graphs
Eφ	the embedding matrix, Eφ ∈ R1×d
κ	the kernel function
W	the weight matrix
b	the complex bias weights
F	Discrete Fourier Transform
F-1	Inverse Discrete Fourier Transform
F	the forecasting model
B Convolution Theorem
The convolution theorem [29] is one of the most important properties of the Fourier transform. It states the Fourier transform of a convolution of two signals equals the pointwise product of their Fourier transforms in the frequency domain. Given a signal  $x[n]$  and a filter  $h[n]$ , the convolution theorem can be defined as follows:
$$
\mathcal {F} ((x * h) [ n ]) = \mathcal {F} (x) \mathcal {F} (h) \tag {9}
$$
where  $(x*h)[n] = \sum_{m=0}^{N-1}h[m]x[(n-m)_N]$  denotes the convolution of  $x$  and  $h$ ,  $(n-m)_N$  denotes  $(n-m)$  modulo  $N$ , and  $\mathcal{F}(x)$  and  $\mathcal{F}(h)$  denote discrete Fourier transform of  $x[n]$  and  $h[n]$ , respectively.
C Explanations and Proofs
C.1 The Explanations of the Hypervariate Graph Structure
Note that the time lag effect between time-series variables is a common phenomenon in real-world multivariate time series scenarios, for example, the time lag influence between two financial assets (e.g. dollar and gold) of a portfolio. It is beneficial but challenging to consider dependencies between different variables under different timestamps.
The hypervariate graph connecting any two variables at any two timestamps aims to encode high-resolution spatiotemporal dependencies. It embodies not only the intra-series temporal dependencies (node connections of each individual variable), inter-series spatial dependencies (node connections under each single time step), and also the time-varying spatiotemporal dependencies (node connections between different variables at different time steps). By leveraging the hypervariate graph structure, we can effectively learn the spatial and temporal dependencies. This approach is distinct from previous methods that represent the spatial and temporal dependencies separately using two network structures.
C.2 The Interpretation of  $n$ -invariant FGO
Why  $\mathcal{F}(\kappa) \in \mathbb{C}^{n \times d \times d}$ ? From Definition 2, we know that the kernel  $\kappa$  is defined as a matrix-valued projection, i.e.,  $\kappa : [n] \times [n] \to \mathbb{R}^{d \times d}$ . Note that we assume  $\kappa$  is in the special case of the Green's kernel, i.e., a translation-invariant kernel  $\kappa[i,j] = \kappa[i - j]$ . Accordingly,  $\kappa$  can be reduced:  $\kappa : [n] \to \mathbb{R}^{d \times d}$  where we can parameterize  $\mathcal{F}(\kappa)$  with a complex-valued matrix  $\mathbb{C}^{n \times d \times d}$ .
What is  $n$ -invariant FGO? Turning to our case of the fully-connected hypervariate graph, we can consider a special case of  $\kappa$ , i.e., a space-invariant kernel  $\kappa[i,j] = \kappa[\varrho]$  with  $\varrho$  being a constant scalar. Accordingly, we can parameterize FGO  $S$  with a  $n$ -invariant complex-valued matrix  $\mathbb{C}^{d \times d}$ .