Title: FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective
Venue: NeurIPS 2023
Accepted: yes
Keywords: multivariate time series forecasting

fourier space
Abstract: Multivariate time series (MTS) forecasting has shown great importance in numerous industries. Current state-of-the-art graph neural network (GNN)-based forecasting methods usually require both graph networks (e.g., GCN) and temporal networks (e.g., LSTM) to capture inter-series (spatial) dynamics and intra-series (temporal) dependencies, respectively. However, the uncertain compatibility of the two networks puts an extra burden on handcrafted model designs. Moreover, the separate spatial and temporal modeling naturally violates the unified spatiotemporal inter-dependencies in real world, which largely hinders the forecasting performance. To overcome these problems, we explore an interesting direction of directly applying graph networks and rethink MTS forecasting from a pure graph perspective. We first define a novel data structure, hypervariate graph, which regards each series value (regardless of variates or timestamps) as a graph node, and represents sliding windows as space-time fully-connected graphs. This perspective considers spatiotemporal dynamics unitedly and reformulates classic MTS forecasting into the predictions on hypervariate graphs.
 Then, we propose a novel architecture Fourier Graph Neural Network (FourierGNN) by stacking our proposed Fourier Graph Operator (FGO) to perform matrix multiplications in Fourier space. FourierGNN accommodates adequate expressiveness and achieves much lower complexity, which can effectively and efficiently accomplish {the forecasting}. Besides, our theoretical analysis reveals FGO's equivalence to graph convolutions in the time domain, which further verifies the validity of FourierGNN. Extensive experiments on seven datasets have demonstrated our superior performance with higher efficiency and fewer parameters compared with state-of-the-art methods. Code is available at this repository: https://github.com/aikunyi/FourierGNN.

FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective

Abstract

Multivariate time series (MTS) forecasting has shown great importance in numerous industries. Current state-of-the-art graph neural network (GNN)-based forecasting methods usually require both graph networks (e.g., GCN) and temporal networks (e.g., LSTM) to capture inter-series (spatial) dynamics and intra-series (temporal) dependencies, respectively. However, the uncertain compatibility of the two networks puts an extra burden on handcrafted model designs. Moreover, the separate spatial and temporal modeling naturally violates the unified spatiotemporal inter-dependencies in real world, which largely hinders the forecasting performance. To overcome these problems, we explore an interesting direction of directly applying graph networks and rethink MTS forecasting from a pure graph perspective. We first define a novel data structure, hypervariate graph , which regards each series value (regardless of variates or timestamps) as a graph node, and represents sliding windows as space-time fully-connected graphs. This perspective considers spatiotemporal dynamics unitedly and reformulates classic MTS forecasting into the predictions on hypervariate graphs. Then, we propose a novel architecture Fourier Graph Neural Network (FourierGNN) by stacking our proposed Fourier Graph Operator (FGO) to perform matrix multiplications in Fourier space . FourierGNN accommodates adequate expressiveness and achieves much lower complexity, which can effectively and efficiently accomplish the forecasting. Besides, our theoretical analysis reveals FGO's equivalence to graph convolutions in the time domain, which further verifies the validity of FourierGNN. Extensive experiments on seven datasets have demonstrated our superior performance with higher efficiency and fewer parameters compared with state-of-the-art methods. Code is available at this repository: https://github.com/aikunyi/FourierGNN .

1 Introduction

Multivariate time series (MTS) forecasting plays an important role in numerous real-world scenarios, such as traffic flow prediction in transportation systems [1, 2], temperature estimation in weather forecasting [3, 4], and electricity consumption planning in the energy market [5, 6], etc. In MTS forecasting, the core challenge is to model intra-series (temporal) dependencies and simultaneously capture inter-series (spatial) correlations. Existing literature has primarily focused on the temporal modeling and proposed several forecasting architectures, including Recurrent Neural Network (RNN)-based methods (e.g., DeepAR [7]), Convolution Neural Network (CNN)-based methods (e.g., Temporal Convolution Network [8]) and more recent Transformer-based methods (e.g., Informer [9]

and Autoformer [4]). In addition, another branch of MTS forecasting methods has been developed to not only model temporal dependencies but also places emphasis on spatial correlations. The most representative methods are the emerging Graph Neural Network (GNN)-based approaches [10, 2, 11] that have achieved state-of-the-art performance in the MTS forecasting task.

Previous GNN-based forecasting methods (e.g., STGCN [12] and TAMP-S2GCNets [11]) heavily rely on a pre-defined graph structure to specify the spatial correlations, which as a matter of fact cannot capture the spatial dynamics , i.e., the time-evolving spatial correlation patterns. Later advanced approaches (e.g., StemGNN [10], MTGNN [13], AGCRN [2]) can automatically learn inter-series correlations and accordingly model spatial dynamics without pre-defined priors, but almost all of them are designed by stacking graph networks (e.g., GCN and GAT) to capture spatial dynamics and temporal networks (e.g., LSTM and GRU) to capture temporal dependencies . However, the uncertain compatibility of the graph networks and the temporal networks puts extra burden on handcrafted model designs, which hinders the forecasting performance. Moreover, the respective modeling for the two networks separately learn spatial/temporal correlations, which naturally violate the real-world unified spatiotemporal inter-dependencies. In this paper, we explore an opposite direction of directly applying graph networks for forecasting and investigate an interesting question: can pure graph networks capture spatial dynamics and temporal dependencies even without temporal networks?

To answer this question, we rethink the MTS forecasting task from a pure graph perspective. We start with building a new data structure, hypervariate graph , to represent time series with a united view of spatial/temporal dynamics. The core idea of the hypervariate graph is to construct a space-time fully-connected structure. Specifically, given a multivariate time series window (say input window) X t ∈ R N × T at timestamp t , where N is the number of series (variates) and T is the length of input window, we construct a corresponding hypervariate graph structure represented as G T t = ( X T t , A T t ) , which is initialized as a fully-connected graph of NT nodes with adjacency matrix A T t ∈ R NT × NT

and node features X T t ∈ R NT × 1 by regarding each value x ( n ) t ∈ R 1 (variate n at step t ) of input window as a distinct node of a hypervariate graph. Such a special structure design formulates both intra- and inter-series correlations of multivariate series as pure node-node dependencies in the hypervariate graph. Different from classic formulations that make spatial-correlated graphs and learn dynamics in a two-stage (spatial and temporal) process [13], our perspective views spatiotemporal correlations as a whole. It abandons the uncertain compatibility of spatial/temporal modeling, constructs adaptive space-time inter-dependencies, and brings up higher-resolution fusion across multiple variates and timestamps in MTS forecasting.

Then, with such a graph structure, the multivariate forecasting can be originally formulated into the predictions on the hypervariate graph. However, the node number of the hypervariate graph increase with the number of series ( N ) and the window length ( T ), leading to a graph of large order and size. This could make classic graph networks (e.g., GCN [14], GAT [15]) computationally expensive (usually with quadratic complexity) and suffer from optimization difficulty in obtaining accurate node representations [16]. To this end, we propose a novel architecture, Fourier Graph Neural Network (FourierGNN), for MTS forecasting from a pure graph perspective. Specifically, FourierGNN is built upon our proposed Fourier Graph Operator (FGO), which as a replacement of classic graph operation units (e.g., convolutions), performs matrix multiplications in Fourier space of graphs. By stacking FGO layers in Fourier space, FourierGNN can accommodate adequate learning expressiveness and in the mean time achieve much lower complexity (Log-linear complexity), which thus can effectively and efficiently accomplish MTS forecasting. Besides, we present theoretical analysis to demonstrate that the FGO is equivalent to graph convolutions in the time domain, which further explains the validity of FourierGNN.

Finally, we perform extensive experiments on seven real-world benchmarks. Experimental results demonstrate that FourierGNN achieves an average of more than 10% improvement in accuracy compared with state-of-the-art methods. In addition, FourierGNN achieves higher forecasting efficiency, which has about 14.6% less costs in training time and 20% less parameter volumes, compared with most lightweight GNN-based forecasting methods.

2 Related Work

Graph Neural Networks for Multivariate Time Series Forecasting Multivariate time series (MTS) have embraced GNN due to their best capability of modeling structural dependencies between variates [17, 2, 13, 12, 11, 18, 19]. Most of these models, such as STGCN [12], DCRNN [18],

and TAMP-S2GCNets [11], require a pre-defined graph structure which is usually unknown in most cases. For this limitation, some studies enable to automatically learn the graphs by the interseries correlations, e.g., by node similarity [20, 2, 17] or self-attention mechanism [10]. However, these methods always adopt a graph network for spatial correlations and a temporal network for temporal dependencies separately [20, 17, 10, 2]. For example, AGCRN [2] use a GCN [14] and a GRU [21], GraphWaveNet [17] use a GCN and a TCN [8], etc. In this paper, we propose an unified spatiotemporal formulation with pure graph networks for MTS forecasting.

Multivariate Time Series Forecasting with Fourier Transform Recently, many MTS forecasting models have integrated the Fourier theory into deep neural networks [22, 23]. For instance, SFM [24] decomposes the hidden state of LSTM into multiple frequencies by Discrete Fourier Transform (DFT). mWDN[25] decomposes the time series into multilevel sub-series by discrete wavelet decomposition (DWT) and feeds them to LSTM network. ATFN [26] proposes a Discrete Fourier Transformbased block to capture dynamic and complicated periodic patterns of time series data. FEDformer [27] proposes Discrete Fourier Transform-based attention mechanism with low-rank approximation in frequency. While these models only capture temporal dependencies with Fourier Transform, StemGNN [10] takes the advantages of both spatial correlations and temporal dependencies in the spectral domain by utilizing Graph Fourier Transform (GFT) to perform graph convolutions and Discrete Fourier Transform (DFT) to calculate the series relationships.

3 Problem Definition

Given the multivariate time series input, i.e., the lookback window X t = [ x t -T +1 , ..., x t ] ∈ R N × T at timestamps t with the number of series (variates) N and the lookback window size T , where x t ∈ R N denotes the multivariate values of N series at timestamp t . Then, the multivariate time series forecasting task is to predict the next τ timestamps Y t = [ x t +1 , ..., x t + τ ] ∈ R N × τ based on the historical T observations X t = [ x t -T +1 , ..., x t ] . The forecasting process can be given by:

where ˆ Y t are the predictions corresponding to the ground truth Y t . The forecasting function is denoted as F θ parameterized by θ . In practice, many MTS forecasting models usually leverage a graph network (assume parameterized by θ g ) to learn the spatial dynamics and a temporal network (assume parameterized by θ t ) to learn the temporal dependencies, respectively [17, 10, 2, 13, 11]. Thus, the original definition of Equation (1) can be rewritten to:

where original parameters θ are exposed to the parameters of the graph network θ g and the temporal network θ t to make prediction based on the learned spatial-temporal dependencies.

4 Methodology

In this section, we elaborate on our proposed framework: First, we start with our pure graph formulation with a novel hypervariate graph structure for MTS forecasting in Section 4.1. Then, we illustrate the proposed neural architecture, Fourier Graph Neural Network (FourierGNN), for this formulation in Section 4.2. Besides, we theoretically analyze FourierGNN to demonstrate its architecture validity, and also conduct complexity analysis to show its efficiency. Finally, we introduce certain inductive bias to instantiate FourierGNN for MTS forecasting in Section 4.3.

4.1 The Pure Graph Formulation

To overcome the uncertain compatibility of the graph network and the temporal network as aforementioned in Section 1, and learn the united spatiotemporal dynamics, we propose a pure graph formulation that refines Equation (2) by a novel data structure, hypervariate graph , for time series.

Definition 1 ( Hypervariate Graph ) . Given a multivariate time series window as input X t ∈ R N × T of N variates at timestamp t , we construct a hypervariate graph of NT nodes, G t = ( X G t , A G t ) , by regarding each element of X t as one node of G t such that X G t ∈ R NT × 1 stands for the node feature and A G t ∈ R NT × NT is the adjacency matrix initialized to make G t as a fully-connected graph.

Since the prior graph structure is usually unknown in most multivariate time series scenarios [10, 2, 13], and the elements of X t are spatially or temporally correlated with each other because of time lag effect [28], we assume all nodes in the hypervariate graph G t are fully-connected. The hypervariate graph G t contains NT nodes representing the values of each variate at each timestamp in X t , which can learn a high-resolution representation across timestamps and variates (more explanations of the hypervariate graph can be seen in Appendix C.1). We present an example hypervariate graph of three time series in Figure 1. Thus, with such a data structure, we can reformulate the multivariate time series forecasting Equation (2) into:

Hypervariate Graph

Input Window

...

...

...

Time

Time

Time

...

...

...

...

task into the predictions on the hypervariate graphs, and accordingly rewrite

where θ G stands for the network parameters for hypervariate graphs. With such a formulation, we can view the spatial dynamics and the temporal dependencies from an united perspective, which benefits modeling the real-world spatiotemporal inter-dependencies.

4.2 FourierGNN

Though the pure graph formulation can enhance spatiotemporal modeling, the order and size of hypervariate graphs increase with the number of variates N and the size of window T , which makes classic graph networks (e.g., GCN [14] and GAT [15]) computationally expensive (usually quadratic complexity) and suffer from optimization difficulty in obtaining accurate hidden node representations [16]. In this regard, we propose an efficient and effective method, FourierGNN, for the pure graph formulation. The main architecture of FourierGNN is built upon our proposed Fourier Graph Operator (FGO), a learnable network layer in Fourier space, which is detailed as follows.

Definition 2 ( Fourier Graph Operator ) . Given a graph G = ( X,A ) with node features X ∈ R n × d and the adjacency matrix A ∈ R n × n , where n is the number of nodes and d is the number of features, we introduce a weight matrix W ∈ R d × d to acquire a tailored Green's kernel κ : [ n ] × [ n ] - → R d × d with κ [ i, j ] := A ij ◦ W and κ [ i, j ] = κ [ i -j ] . We define S A,W := F ( κ ) ∈ C n × d × d as a Fourier Graph Operator (FGO), where F denotes Discrete Fourier Transform (DFT).

According to the convolution theorem [29] (see Appendix B), we can write the multiplication between F ( X ) and FGO S A,W in Fourier space as:

where ( X ∗ κ )[ i ] denotes the convolution of X and κ . As defined κ [ i, j ] = A ij ◦ W , it yields ∑ n j =1 X [ j ] κ [ i, j ] = ∑ n j =1 A ij X [ j ] W = AXW . Accordingly, we can get the convolution equation:

In particular, turning to our case of the fully-connected hypervariate graph, we can adopt a n -invariant FGO S ∈ C d × d that has a computationally low cost compared to previous C n × d × d . We provide more details and explanations in Appendix C.2.

From Equation (5), we can observe that performing the multiplication between F ( X ) and FGO S in Fourier space corresponds to a graph shift operation (i.e., a graph convolution) in the time domain [20]. Since the multiplications in Fourier space ( O ( n ) ) have much lower complexity than the above shift operations ( O ( n 2 ) ) in the time domain (See Complexity Analysis below), it motivates us to develop a highly efficient graph neural network in Fourier space.

To this end, we propose the Fourier Graph Neural Networks (FourierGNN) based on FGO. Specifically, by stacking multiple layers of FGOs, we can define the K -layer Fourier graph neural networks given a graph G = ( X,A ) with node features X ∈ R n × d and the adjacency matrix A ∈ R n × n as:

Herein, S k is the FGO in the k -th layer, satisfying F ( X ) S k = F ( A k XW k ) with W k ∈ R d × d being the weights and A k ∈ R n × n corresponding to the k -th adjacency matrix sharing the same sparsity pattern of A , and b k ∈ C d are the complex-valued biases parameters; F stands for Discrete Fourier Transform; σ is the activation function. In particular, S 0 , W 0 , A 0 are the identity matrix, and we adopt identical activation at k = 0 to obtain residual F ( X ) . All operations in FourierGNN are performed in Fourier space. Thus, all parameters, i.e., {S k , b k } K k =1 , are complex numbers.

The core operation of FourierGNN is the summation of recursive multiplications with nonlinear activation functions. Specifically, the recursive multiplications between F ( X ) and S , i.e., F ( X ) S 0: k , are equivalent to the multi-order convolutions on the graph structure (see Theoretical Analysis below). Nonlinear activation functions σ are introduced to address the capability limitations of modeling nonlinear information diffusion on graphs in the summation.

Theoretical Analysis We theoretically analyze the effectiveness and interpretability of FourierGNN and verify the validity of its architecture. For convenience, we exclude the non-linear activation function σ and learnable bias parameters b from Equation (6), and focus on F ( X ) S 0: k .

Proposition 1. Given a graph G = ( X,A ) with node features X ∈ R n × d and adjacency matrix A ∈ R n × n , the recursive multiplication of FGOs in Fourier space is equivalent to multi-order convolutions in the time domain:

where A 0 , S 0 , W 0 are the identity matrix, A k ∈ R n × n corresponds to the k -th diffusion step sharing the same sparsity pattern of A , W k ∈ R d × d is the k -th weight matrix, S k ∈ C d × d is the k -th FGO satisfying F ( A k XW k ) = F ( X ) S k , and F and F -1 denote DFT and its inverse, respectively.

In the time domain, operation A k :0 XW 0: k adopts different weights W k ∈ R d × d to weigh the information of different neighbors in different diffusion orders, beneficial to capture the extensive dependencies on graphs [20, 30, 31]. This indicates FourierGNN is expressive in modeling the complex correlations among graph nodes, i.e., spatiotemporal dependencies in the hypervariate graph. The proof of Proposition 1 and more explanations of FourierGNN are provided in Appendix C.3.

Complexity Analysis The time complexity of F ( X ) S is O ( nd log n + nd 2 ) , which includes the Discrete Fourier Transform (DFT), the Inverse Discrete Fourier Transform (IDFT), and the matrix multiplication in the Fourier space. Comparatively, the time complexity of the equivalent operations of F ( X ) S in the time domain, i.e., AXW , is O ( n 2 d + nd 2 ) . Then, as a K -order summation of a recursive multiplication of F ( X ) S , FourierGNN, achieves the time complexity of O ( nd log n + Knd 2 ) , including DFT and IDFT, and the recursive multiplication of FGOs. Overall, the Log-linear O ( n log n ) complexity makes FourierGNN much more efficient.

FourierGNN vs Other Graph Networks We analyze the connection and difference between our FourierGNN with GCN [14] and GAT [15]. From the complexity perspective, FourierGNN with log-linear complexity shows much higher efficiency than GCN and GAT. Regarding the network architecture, we analyze them from two main perspectives: (1) Domain. GAT implements operations in the time domain, while GCN and FourierGNN are in Fourier space. However, GCN achieves the transformation through the Graph Fourier Transform (GFT), whereas FourierGNN utilizes the Discrete Fourier Transform (DFT). (2) Information diffusion: GAT aggregates neighbor nodes with varying weights to via attention mechanisms. FourierGNN and GCN update node information via convoluting neighbor nodes. Different from GCN, FourierGNN assigns varying importance to neighbor nodes in different diffusion steps. We provide a detailed comparison in Appendix D.

4.3 Multivariate Time Series Forecasting with FourierGNN

In this section, we instantiate FourierGNN for MTS forecasting. The overall architecture of our model is illustrated in Figure 2. Given the MTS input data X t ∈ R N × T , first we construct a fully-connected hypervariate graph G t = ( X G t , A G t ) with X G t ∈ R NT × 1 and A G t ∈ { 1 } n × n . Then, we project X G t into node embeddings X G t ∈ R NT × d by assigning a d -dimension vector for each node using an embedding matrix E ϕ ∈ R 1 × d , i.e., X G t = X G t × E ϕ .

Node

Embedding

DFT

IDFT

FFN

FGO

+

…

FGO

Output

FGO

Time Domain

Time Domain

Hypervariate Graph

෡

Y

t

𝜎

𝜎

𝜎

𝜎

...

...

...

𝒮1

ෑ

𝑖=1

𝐾

𝒮𝑖𝒳

A

t

𝓖

𝘹

t

𝓖

)

,

𝓖 = (

t

𝓖

𝒳

t

𝓖

𝗬

t

𝒮2

𝒮𝐾

t

𝓖

𝒴

A

t

𝓖

t

𝓖

𝐗

Subsequently, to capture the spatiotemporal dependencies simultaneously, we aim to feed multiple embeded hypervariate graphs with X G t to FourierGNN. First, we perform Discrete Fourier Transform (DFT) F on each discrete spatio-temporal dimension of the embeddings X G t and obtain the frequency output X G t := F ( X G t ) ∈ C NT × d . Then, we perform a recursive multiplication between X G t and FGOs S 0: k in Fourier space and output the resulting representations Y G t as:

Then Y G t are transformed back to the time domain using Inverse Discrete Fourier Transform (IDFT) F -1 , which yields Y G t := F -1 ( Y G t ) ∈ R NT × d .

Finally, according to the FourierGNN output Y G t which encodes spatiotemporal inter-dependencies, we use two layer feed-forward networks (FFN) (see Appendix E.4 for more details) to project it onto τ future steps, resulting in ˆ Y t = FFN( Y G t ) ∈ R N × τ .

5 Experiments

To evaluate the performance of FourierGNN, we conduct extensive experiments on seven real-world time series benchmarks to compare with state-of-the-art graph neural network-based methods.

5.1 Experimental Setup

Datasets . We evaluate our proposed method on seven representative datasets from various application scenarios, including traffic, energy, web traffic, electrocardiogram, and COVID-19. All datasets are normalized using the min-max normalization. Except the COVID-19 dataset, we split the other datasets into training, validation, and test sets with the ratio of 7:2:1 in a chronological order. For the COVID-19 dataset, the ratio is 6:2:2. More detailed information about datasets are in Appendix E.1.

Baselines . We conduct a comprehensive comparison of the forecasting performance between our FourierGNN and several representative and state-of-the-art (SOTA) models on the seven datasets, including classic method VAR [32], deep learning-based models such as SFM [24], LSTNet [33], TCN [8], DeepGLO [34], and CoST [36]. We also compare FourierGNN against GNN-based models like GraphWaveNet [17], StemGNN [10], MTGNN [13], and AGCRN [2], and two representative Transformer-based models like Reformer [35] and Informer [9], as well as two frequency enhanced Transformer-based models including Autoformer [4] and FEDformer [27]. In addition, we compare FourierGNN with SOTA models such as TAMP-S2GCNets [11], DCRNN [18], and STGCN [1], which require pre-defined graph structures. Please refer to Appendix E.2 for more implementation details of the adopted baselines.

Experimental Settings . All experiments are conducted in Python using Pytorch 1.8 [37] (except for SFM [24] which uses Keras) and performed on single NVIDIA RTX 3080 10G GPU. Our model is trained using RMSProp with a learning rate of 10 -5 and MSE (Mean Squared Error) as the loss function. The best parameters for all comparative models are chosen through careful parameter tuning on the validation set. We use Mean Absolute Errors (MAE), Root Mean Squared Errors (RMSE), and

Mean Absolute Percentage Error (MAPE) to measure the performance. The evaluation details are in Appendix E.3 and more experimental settings are in Appendix E.4.

5.2 Main Results

We present the evaluation results with an input length of 12 and a prediction length of 12 in Table 1. Overall, FourierGNN achieves a new state-of-the-art on all datasets. On average, FourierGNN makes an improvement of 9.4% in MAE and 10.9% in RMSE compared to the best-performing across all datasets. Among these baselines, Reformer, Informer, Autoformer, and FEDformer are Transformer-based models that demonstrate competitive performance on Electricity and COVID19 datasets, as they excel at capturing temporal dependencies. However, they have limitations in capturing the spatial dependencies explicitly. GraphWaveNet, MTGNN, StemGNN, and AGCRN are GNN-based models that show promising results on Wiki, Traffic, Solar, and ECG datasets, primarily due to their capability to handle spatial dependencies among variates. However, they are limited in their capacity to simultaneously capture spatiotemporal dependencies. FourierGNN outperforms the baseline models since it can learn comprehensive spatiotemporal dependencies simultaneously and attends to time-varying dependencies among variates.

Multi-Step Forecasting To further evaluate the performance in multi-step forecasting, we compare FourierGNN with other GNN-based MTS models (including StemGNN [10], AGCRN [2],

GraphWaveNet [17], MTGNN [13], and TAMP-S2GCNets [11]) and a representation learning model (CoST [36]) on COVID-19 dataset under different prediction lengths, and the results are shown in Table 2. It shows that FourierGNN achieves an average 30.1% and 30.2% improvement on MAE and RMSE respectively over the best baseline. In Appendix F, we include more experiments and analysis under different prediction lengths, and further compare FourierGNN with models that require pre-defined graph structures.

5.3 Model Analysis

Efficiency Analysis We investigate the parameter volumes and training time costs of FourierGNN, StemGNN [10], AGCRN [2], GraphWaveNet [17], and MTGNN [13] on two representative datasets, including the Wiki dataset and the Traffic dataset. The results are reported in Table 3, showing the comparison of parameter volumes and average time costs over five rounds of experiments. In terms of parameters, FourierGNN exhibits the lowest volume of parameters among the comparative models. Specifically, it achieves a reduction of 32.2% and 9.5% in parameters compared to GraphWaveNet on Traffic and Wiki datasets, respectively. This reduction is mainly attributed that FourierGNN has shared scale-free parameters for each node. Regarding training time, FourierGNN runs much faster than all baseline models, and it demonstrates efficiency improvements of 5.8% and 23.3% over the fast baseline GraphWaveNet on Traffic and Wiki datasets, respectively. Considering variate number of Wiki dataset is about twice larger than that of Traffic dataset, FourierGNN exhibits larger efficiency superiority with the baselines. These findings highlight the high efficiency of FourierGNN in computing graph operations and its scalability to large datasets with extensive graphs, which is important for the pure graph formulation due to the larger size of hypervariate graphs with NT nodes.

Ablation Study We perform an ablation study on the METR-LA dataset to assess the individual contributions of different components in FourierGNN. The results, presented in Table 4, validate the effectiveness of each component. Specifically, w/o Embedding emphasizes the significance of performing node embedding to improve model generalization. w/o Dynamic FGO using the same FGO verifies the effectiveness of applying different FGOs in capturing time-varying dependencies. In addition, w/o Residual represents FourierGNN without the K = 0 layer, while w/o Summation adopts the last order (layer) output, i.e., X S 0: K , as the output of FourierGNN. These results demonstrate the importance of high-order diffusion and the contribution of multi-order diffusion. More results and analysis of the ablation study are provided in Appendix G.3.

5.4 Visualization

To gain a better understanding of the hypervariate graph and FourierGNN in spatiotemporal modeling for MTS forecasting, We conduct visualization experiments on the METR-LA and COVID-19 datasets. Please refer to Appendix E.5 for more information on the visualization techniques used.

Visualization of temporal representations learned by FourierGNN In order to showcase the temporal dependencies learning capability of FourierGNN, we visualize the temporal adjacency matrix of different variates. Specifically, we randomly select 8 counties from the COVID-19 dataset and calculate the relations of 12 consecutive time steps for each county. Then, we visualize the

(b) N=18

(c) N=24

(d) N=29

(e) N=38

(f) N=45

(g) N=46

3.6

6.0

3.4

5.5

3.2

5.0

3.0

20

4.5

2.8

4.0

2.6

3.5

2.4

O'E

2.2

2.5

4.5

8.0

2.2

4.0

7.5

2.0

1.8

3.5

6.5

1.6

3.0

6.0

1.4

2.5

5.5

2.0

adjacency matrix by a heatmap, and the results are illustrated in Figure 3, where N denotes the index of the country (variate). It shows that FourierGNN learns distinct temporal patterns for each county, indicating that the hypervariate graph can encode rich and discriminative temporal dependencies.

Visualization of spatial representations learned by FourierGNN To investigate the spatial correlations learning capability of FourierGNN, we visualize the generated adjacency matrix based on the representations learned by FourierGNN on the METR-LA dataset. Specifically, we randomly select 20 detectors and visualize their corresponding adjacency matrix via a heatmap, as depicted in Figure 4.

By examining the adjacency matrix in conjunction with the actual road map, we observe: 1) the detectors (7, 8, 9, 11, 13, 18) are very close w.r.t. the physical distance, corresponding to the high values of their correlations with each other in the heatmap; 2) the detectors 4, 14 and 16 have small overall correlation values since they are far from other detectors; 3) however, compared with detectors 14 and 16, the detector 4 has slightly higher correlation values to other detectors, e.g., 7, 8, 9, which is because although they are far

1.0

0.9

16

Glendale

9

913

11

181

7

- 0.7

0.8

c0

51413121110

-0.6

- 0.5

- 0.4

LosAngeles

- 0.3

13]1415 16 17 18 19

apart, the detectors 4, 7, 8, 9 are on the same road. The results verify that the hypervariate graph structure can represent highly interpretative correlations.

Moreover, to gain a understanding of how FGO works, we visualize the output of each layer of FourierGNN, and the visualization results demonstrate that FGO can adaptively and effectively capture important patterns while removing noises to a learn discriminative model. Further details can be found in Appendix H.1. Additionally, to investigate the ability of FourierGNN to capture time-varying dependencies among variates, we further visualize the spatial correlations at different timestamps. The results illustrate that FourierGNN can effectively attend to the temporal variability in the data. For more information, please refer to Appendix H.2.

6 Conclusion Remarks

In this paper, we explore an interesting direction of directly applying graph networks for MTS forecasting from a pure graph perspective. To overcome the previous separate spatial and temporal modeling problem, we build a hypervariate graph, regarding each series value as a graph node, which considers spatiotemporal dynamics unitedly. Then, we formulate time series forecasting on the hypervariate graph and propose FourierGNN by stacking Fourier Graph Operator (FGO) to perform matrix multiplications in Fourier space, which can accommodate adequate learning expressiveness with much lower complexity. Extensive experiments demonstrate that FourierGNN achieves stateof-the-art performances with higher efficiency and fewer parameters, and the hypervariate graph structure exhibits strong capabilities to encode spatiotemporal inter-dependencies.

Acknowledgments and Disclosure of Funding

The work was supported in part by the National Key Research and Development Program of China under Grant 2020AAA0104903 and 2019YFB1406300, and National Natural Science Foundation of China under Grant 62072039 and 62272048.

References

Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In IJCAI , pages 3634-3640, 2018.

Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. In NeurIPS , 2020.

Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine-grained air quality based on big data. In KDD , pages 2267-2276, 2015.

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In NeurIPS , pages 2241922430, 2021.

Hui He, Qi Zhang, Simeng Bai, Kun Yi, and Zhendong Niu. CATN: cross attentive tree-aware network for multivariate time series forecasting. In AAAI , pages 4030-4038. AAAI Press, 2022.

Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie-Yan Liu. DEPTS: deep expansion learning for periodic time series forecasting. In ICLR . OpenReview.net, 2022.

David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting , 36(3):1181-1191, 2020.

Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR , abs/1803.01271, 2018.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI , pages 11106-11115, 2021.

Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. Spectral temporal graph neural network for multivariate time-series forecasting. In NeurIPS , 2020.

Yuzhou Chen, Ignacio Segovia-Dominguez, Baris Coskunuzer, and Yulia Gel. TAMP-s2GCNets: Coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In International Conference on Learning Representations , 2022.

Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In IJCAI , pages 3634-3640, 2018.

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In KDD , pages 753-763, 2020.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR (Poster) . OpenReview.net, 2017.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. CoRR , abs/1710.10903, 2017.

Kate Smith-Miles and Leo Lopes. Measuring instance difficulty for combinatorial optimization problems. Comput. Oper. Res. , 39(5):875-889, 2012.

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In IJCAI , pages 1907-1913, 2019.

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In ICLR (Poster) , 2018.

Leyan Deng, Defu Lian, Zhenya Huang, and Enhong Chen. Graph convolutional adversarial networks for spatiotemporal anomaly detection. IEEE Transactions on Neural Networks and Learning Systems , 33(6):2416-2428, 2022.

Gonzalo Mateos, Santiago Segarra, Antonio G. Marques, and Alejandro Ribeiro. Connecting the dots: Identifying network structure via graph signal processing. IEEE Signal Process. Mag. , 36(3):16-43, 2019.

Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoderdecoder for statistical machine translation. In EMNLP , pages 1724-1734. ACL, 2014.

Kun Yi, Qi Zhang, Longbing Cao, Shoujin Wang, Guodong Long, Liang Hu, Hui He, Zhendong Niu, Wei Fan, and Hui Xiong. A survey on deep learning based time series analysis with frequency transformation. CoRR , abs/2302.02173, 2023.

Tian Zhou, Ziqing Ma, Xue Wang, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, and Rong Jin. Film: Frequency improved legendre memory model for long-term time series forecasting. 2022.

Liheng Zhang, Charu C. Aggarwal, and Guo-Jun Qi. Stock price prediction via discovering multi-frequency trading patterns. In KDD , pages 2141-2149, 2017.

Jingyuan Wang, Ze Wang, Jianfeng Li, and Junjie Wu. Multilevel wavelet decomposition network for interpretable time series analysis. In KDD , pages 2437-2446, 2018.

Zhangjing Yang, Weiwu Yan, Xiaolin Huang, and Lin Mei. Adaptive temporal-frequency network for time-series forecasting. IEEE Trans. Knowl. Data Eng. , 34(4):1576-1587, 2022.

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML , 2022.

William W. S. Wei. Time series analysis - univariate and multivariate methods . Addison-Wesley, 1989.

Y. Katznelson. An introduction to harmonic analysis. Cambridge University Press, , 1970.

Santiago Segarra, Gonzalo Mateos, Antonio G. Marques, and Alejandro Ribeiro. Blind identification of graph filters. IEEE Trans. Signal Process. , 65(5):1146-1159, 2017.

Elvin Isufi, Fernando Gama, and Alejandro Ribeiro. Edgenets: Edge varying graph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021.

Mark W. Watson. Vector autoregressions and cointegration. Working Paper Series, Macroeconomic Issues , 4, 1993.

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In SIGIR , pages 95-104, 2018.

Rajat Sen, Hsiang-Fu Yu, and Inderjit S. Dhillon. Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. In NeurIPS , pages 4838-4847, 2019.

Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In ICLR , 2020.

Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven C. H. Hoi. Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. In ICLR . OpenReview.net, 2022.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS , pages 8024-8035, 2019.

Hoang Anh Dau, Anthony J. Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn J. Keogh. The UCR time series archive. IEEE CAA J. Autom. Sinica , 6(6):1293-1305, 2019.

A Notation

B Convolution Theorem

The convolution theorem [29] is one of the most important properties of the Fourier transform. It states the Fourier transform of a convolution of two signals equals the pointwise product of their Fourier transforms in the frequency domain. Given a signal x [ n ] and a filter h [ n ] , the convolution theorem can be defined as follows:

where ( x ∗ h )[ n ] = ∑ N -1 m =0 h [ m ] x [( n -m ) N ] denotes the convolution of x and h , ( n -m ) N denotes ( n -m ) modulo N, and F ( x ) and F ( h ) denote discrete Fourier transform of x [ n ] and h [ n ] , respectively.

C Explanations and Proofs

C.1 The Explanations of the Hypervariate Graph Structure

Note that the time lag effect between time-series variables is a common phenomenon in real-world multivariate time series scenarios, for example, the time lag influence between two financial assets (e.g. dollar and gold) of a portfolio. It is beneficial but challenging to consider dependencies between different variables under different timestamps.

The hypervariate graph connecting any two variables at any two timestamps aims to encode highresolution spatiotemporal dependencies. It embodies not only the intra-series temporal dependencies (node connections of each individual variable), inter-series spatial dependencies (node connections under each single time step), and also the time-varying spatiotemporal dependencies (node connections between different variables at different time steps). By leveraging the hypervariate graph structure, we can effectively learn the spatial and temporal dependencies. This approach is distinct from previous methods that represent the spatial and temporal dependencies separately using two network structures.

C.2 The Interpretation of n -invariant FGO

Why F ( κ ) ∈ C n × d × d ? From Definition 2, we know that the kernel κ is defined as a matrixvalued projection, i.e., κ : [ n ] × [ n ] - → R d × d . Note that we assume κ is in the special case of the Green's kernel, i.e., a translation-invariant kernel κ [ i, j ] = κ [ i -j ] . Accordingly, κ can be reduced: κ : [ n ] - → R d × d where we can parameterize F ( κ ) with a complex-valued matrix C n × d × d .

What is n -invariant FGO? Turning to our case of the fully-connected hypervariate graph, we can consider a special case of κ , i.e., a space-invariant kernel κ [ i, j ] = κ [ ϱ ] with ϱ being a constant scalar. Accordingly, we can parameterize FGO S with a n -invariant complex-valued matrix C d × d .

The interpretation of n -invariant FGO. An n -invariant FGO is similar to a shared-weight convolution kernel or filter of CNNs that slide along ( [ n ] × [ n ] ) input features, which effectively reduces parameter volumes and saves computation costs. Note that although we adopt the same transformation (i.e., the n -invariant FGO) over NT frequency points, we embed the raw MTS inputs in the d -dimension distributive space beforehand and then perform FourierGNN over MTS embeddings, which can be analogized as d convolution kernels/filters in each convolutional layer in CNNs. This can ensure FourierGNN is able to learn informative features/patterns to improve its model capacity (See the following analysis of the effectiveness of n -invariant FGO).

The effectiveness of n -invariant FGO. In addition, the n -invariant parameterized FGO is empirically proven effective to improve model generalization and achieve superior forecasting performance (See the ablation study in Section 5.3 for more details). Although parameterizing F ( κ ) ∈ C n × d × d (i.e., an n -variant FGO) may be more powerful and flexible than the n -invariant FGO in terms of forecasting performance, it introduces much more parameters and training time costs, especially in case of multi-layer FourierGNN, and may obtain inferior performance due to inadequate training or overfitting. As indicated in Table 6, the FourierGNN with the n -invariant FGO achieves slightly better performance than that with the n -variant FGO on ECG and COVID-19, respectively. Notably, the FourierGNN with the n -variant FGO introduces a much larger parameter volume proportional to n and requires significantly more training time. In contrast, n -invariant FGO is n -agnostic and lightweight, which is a more wise and efficient alternative. These results confirm our design and verify the effectiveness and applicability of n -invariant FGO.

C.3 Proof of Proposition 1 and Interpretation of FourierGNN

Proposition 1. Given a graph G = ( X,A ) with node features X ∈ R n × d and adjacency matrix A ∈ R n × n , the recursive multiplication of FGOs in Fourier space is equivalent to multi-order convolutions in the time domain:

where A 0 , S 0 , W 0 are the identity matrix, A k ∈ R n × n corresponds to the k -th diffusion step sharing the same sparsity pattern of A , W k ∈ R d × d is the k -th weight matrix, S k ∈ C d × d is the k -th FGO satisfying F ( A k XW k ) = F ( X ) S k , and F and F -1 denote DFT and its inverse, respectively.

Proof. The proof aims to demonstrate the equivalence between the recursive multiplication of FGOs in Fourier space and multi-order convolutions in the time domain. According to F ( A k XW k ) = F ( X ) S k , we expand the multi-order convolutions A 0: K XW 0: K in the time domain using a set of FGOs in Fourier space:

Thus, the FourierGNN can be rewritten as (for convenience, we exclude the non-linear activation function σ and learnable bias parameters b ):

From the right part of the above equation, we can observe that it assigns different weights to weigh the information of different neighbors in each diffusion order. This property enable FourierGNN to capture the complex correlations (i.e., spatiotemporal dependencies) in the hypervariate graph, which is empirically verified in our visualization experiments.

D Compared with Other Graph Neural Networks

Graph Convolutional Networks . Graph convolutional networks (GCNs) depend on the Laplacian eigenbasis to perform the multi-order graph convolutions over a given graph structure. Compared with GCNs, FourierGNN as an efficient alternative to multi-order graph convolutions has three main differences: 1) No eigendecompositions or similar costly matrix operations are required. FourierGNN transforms the input into Fourier domain by discrete Fourier transform (DFT) instead of graph Fourier transform (GFT); 2) Explicitly assigning various importance to nodes of the same neighborhood with different diffusion steps. FourierGNN adopts different Fourier Graph Operators S in different diffusion steps corresponding to different dependencies among nodes; 3) FourierGNN is invariant to the discretization N , T . It parameterizes the graph convolution via Fourier Graph Operators which are invariant to the graph structure and graph scale.

Graph Attention Networks . Graph attention networks (GATs) are non-spectral attention-based graph neural networks. GATs use node representations to calculate the attention weights (i.e., edge weights) varying with different graph attention layers. Accordingly, both GATs and FourierGNN do

not depend on eigendecompositions and adopt varying edge weights with different diffusion steps (layers). However, FourierGNN can efficiently perform graph convolutions in Fourier space. For a complete graph, the time complexity of the attention calculation of K layers is proportional to Kn 2 where n is the number of nodes, while a K -layer FourierGNN infers the graph structure in Fourier space with the time complexity proportional to n log n . In addition, compared with GATs that implicitly achieve edge-varying weights with different layers, FourierGNN adopts different FGOs in different diffusion steps explicitly.

E Experiment Details

E.1 Datasets

We use seven public multivariate benchmarks for multivariate time series forecasting and these benchmark datasets are summarized in Table 7.

Solar 2 : This dataset is about solar power collected by National Renewable Energy Laboratory. We choose the power plant data points in Florida as the data set which contains 593 points. The data is collected from 2006/01/01 to 2016/12/31 with the sampling interval of every 1 hour.

Wiki [34] : This dataset contains a number of daily views of different Wikipedia articles and is collected from 2015/7/1 to 2016/12/31. It consists of approximately 145 k time series and we randomly choose 2 k from them as our experimental data set.

Traffic [34] : This dataset contains hourly traffic data from 963 San Francisco freeway car lanes. The traffic data are collected since 2015/01/01 with the sampling interval of every 1 hour.

ECG 3 : This dataset is about Electrocardiogram(ECG) from the UCR time-series classification archive [38]. It contains 140 nodes and each node has a length of 5000.

Electricity 4 : This dataset contains the electricity consumption of 370 clients and is collected since 2011/01/01. The data sampling interval is every 15 minutes.

COVID-19 5 : This dataset is about COVID-19 hospitalization in the U.S. states of California (CA) from 01/02/2020 to 31/12/2020 provided by the Johns Hopkins University with the sampling interval of every one day.

METR-LA 6 : This dataset contains traffic information collected from loop detectors in the highway of Los Angeles County from 01/03/2012 to 30/06/2012. It contains 207 sensors and the data sampling interval is every 5 minutes.

E.2 Baselines

In experiments, we conduct a comprehensive comparison of the forecasting performance between our FourierGNN and representative and state-of-the-art (SOTA) models as follows.

VAR [32]: VAR is a classic linear autoregressive model. We use the Statsmodels library ( https: //www.statsmodels.org ) which is a Python package that provides statistical computations to realize the V AR.

DeepGLO [34]: DeepGLO models the relationships among variables by matrix factorization and employs a temporal convolution neural network to introduce non-linear relationships. We download the source code from: https://github.com/rajatsen91/deepglo . Wefollow the recommended configuration as our experimental settings for wiki, electricity, and traffic datasets. For covid datasets, the vertical and horizontal batch size is set to 64, the rank of the global model is set to 64, the number of channels is set to [32, 32, 32, 1], and the period is set to 7.

LSTNet [33]: LSTNet uses a CNN to capture inter-variable relationships and an RNN to discover long-term patterns. We download the source code from: https://github.com/laiguokun/ LSTNet . In our experiment, we use the recommended configuration where the number of CNN hidden units is 100, the kernel size of the CNN layers is 4, the dropout is 0.2, the RNN hidden units is 100, the number of RNN hidden layers is 1, the learning rate is 0.001 and the optimizer is Adam.

TCN [8]: TCN is a causal convolution model for regression prediction. We download the source code from: https://github.com/locuslab/TCN . We utilize the same configuration as the polyphonic music task exampled in the open source code where the dropout is 0.25, the kernel size is 5, the number of hidden units is 150, the number of levels is 4 and the optimizer is Adam.

Reformer [35]: Reformer combines the modeling capacity of a Transformer with an architecture that can be executed efficiently on long sequences and with small memory use. We download the source code from: https://github.com/thuml/Autoformer . We follow the recommended configuration as the experimental settings.

Informer [9]: Informer leverages an efficient self-attention mechanism to encode the dependencies among variables. We download the source code from: https://github.com/zhouhaoyi/ Informer2020 . We follow the recommended configuration as our experimental settings where the dropout is 0.05, the number of encoder layers is 2, the number of decoder layers is 1, the learning rate is 0.0001, and the optimizer is Adam.

Autoformer [4]: Autoformer proposes a decomposition architecture by embedding the series decomposition block as an inner operator, which can progressively aggregate the long-term trend part from intermediate prediction. We download the source code from: https://github.com/thuml/ Autoformer . We follow the recommended configuration as our experimental settings with 2 encoder layers and 1 decoder layer.

FEDformer [27]: FEDformer proposes an attention mechanism with low-rank approximation in frequency and a mixture of expert decomposition to control the distribution shifting. We download the source code from: https://github.com/MAZiqing/FEDformer . We use FEB-f as the Frequency Enhanced Block and select the random mode with 64 as the experimental mode.

SFM [24]: On the basis of the LSTM model, SFM introduces a series of different frequency components in the cell states. We download the source code from: https://github.com/z331565360/ State-Frequency-Memory-stock-prediction . We follow the recommended settings where the learning rate is 0.01, the frequency dimension is 10, the hidden dimension is 10 and the optimizer is RMSProp.

StemGNN [10]: StemGNN leverages GFT and DFT to capture dependencies among variables in the frequency domain. We download the source code from: https://github.com/microsoft/ StemGNN . We use the recommended configuration of stemGNN as our experiment setting where the optimizer is RMSProp, the learning rate is 0.0001, the number of stacked layers is 5, and the dropout rate is 0.5.

MTGNN [13]: MTGNN proposes an effective method to exploit the inherent dependency relationships among multiple time series. We download the source code from: https://github.com/ nnzhan/MTGNN . Because the experimental datasets have no static features, we set the parameter load_static_feature to false. We construct the graph by the adaptive adjacency matrix and add the graph convolution layer. Regarding other parameters, we adopt the recommended settings.

GraphWaveNet [17]: GraphWaveNet introduces an adaptive dependency matrix learning to capture the hidden spatial dependency. We download the source code from: https://github.com/ nnzhan/Graph-WaveNet . Since our datasets have no prior defined graph structures, we use only adaptive adjacent matrix. We add a graph convolution layer and randomly initialize the adjacent matrix. We adopt the recommended configuration as our experimental settings where the learning rate is 0.001, the dropout is 0.3, the number of epochs is 50, and the optimizer is Adam.

AGCRN [2]: AGCRN proposes a data-adaptive graph generation module for discovering spatial correlations from data. We download the source code from: https://github.com/LeiBAI/ AGCRN . We follow the recommended configuration as our experimental settings where the embedding dimension is 10, the learning rate is 0.003, and the optimizer is Adam.

TAMP-S2GCNets [11]: TAMP-S2GCNets explores the utility of MP to enhance knowledge representation mechanisms within the time-aware DL paradigm. We download the source code from: https: //www.dropbox.com/sh/n0ajd5l0tdeyb80/AABGn-ejfV1YtRwjf_L0AOsNa?dl=0 . TAMPS2GCNets requires predefined graph topology and we use the California State topology provided by the source code as input. We adopt the recommended configuration as our experimental settings on COVID-19.

DCRNN [18]: DCRNN uses bidirectional graph random walk to model spatial dependency and recurrent neural network to capture the temporal dynamics. We download the source code from: https://github.com/liyaguang/DCRNN . We follow the recommended configuration as our experimental settings with the batch size is 64, the learning rate is 0 . 01 , the input dimension is 2 and the optimizer is Adam. DCRNN requires a pre-defined graph structure and we use the adjacency matrix as the pre-defined structure provided by the METR-LA dataset.

STGCN [1]: STGCN integrates graph convolution and gated temporal convolution through spatial-temporal convolutional blocks. We download the source code from: https://github.com/ VeritasYin/STGCN_IJCAI-18 . We use the recommended configuration as our experimental settings where the batch size is 50, the learning rate is 0 . 001 and the optimizer is Adam. STGCN requires a pre-defined graph structure and we leverage the adjacency matrix as the pre-defined structures provided by the METR-LA dataset.

CoST [36]: CoST separates the representation learning and downstream forecasting task and proposes a contrastive learning framework that learns disentangled season-trend representations for time series forecasting tasks. We download the source code from: https://github.com/salesforce/CoST . We set the representation dimension to 320, the learning rate to 0.001, and the batch size to 32. Inputs are min-max normalization, we perform a 70/20/10 train/validation/test split for the METR-LA dataset and 60/20/20 for the COVID-19 dataset.

E.3 Evaluation Metrics

We use MAE (Mean Absolute Error), RMSE (Root Mean Square Error), and MAPE (Mean Absolute Percentage Error) as the evaluation metrics in the experiments.

Specifically, given the groudtruth at timestamps t , Y t = [ x t +1 , ..., x t + τ ] ∈ R N × τ , and the predictions ˆ Y t = [ˆ x t +1 , ..., ˆ x t + τ ] ∈ R N × τ for future τ steps at timestamp t , the metrics are defined as follows:

with x ij ∈ Y t and ˆ x ij ∈ ˆ Y t .

E.4 Experimental Settings

We summarize the implementation details of the proposed FourierGNN as follows. Note that the details of the baselines are introduced in their corresponding descriptions (see Section E.2).

Network details. The fully connected feed-forward network (FFN) consists of three linear transformations with LeakyReLU activations in between. The FFN is formulated as follows:

where W 1 ∈ R ( Td ) × d ffn 1 , W 2 ∈ R d ffn 1 × d ffn 2 and W 3 ∈ R d ffn 2 × τ are the weights of the three layers respectively, and b 1 ∈ R d ffn 1 , b 2 ∈ R d ffn 2 and b 3 ∈ R τ are the biases of the three layers respectively. Here, d ffn 1 and d ffn 2 are the dimensions of the three layers. In addition, we adopt a ReLU activation function in Equation 6.

Training details. We carefully tune the hyperparameters, including the embedding size, batch size, d ffn 1 and d ffn 2 , on the validation set and choose the settings with the best performance for FourierGNN on different datasets. Specifically, the embedding size and batch size are tuned over { 32 , 64 , 128 , 256 , 512 } and { 2 , 4 , 8 , 16 , 32 , 64 , 128 } respectively. For the COVID-19 dataset, the embedding size is 256, and the batch size is set to 4. For the Traffic, Solar, and Wiki datasets, the embedding size is 128, and the batch size is set to 2. For the METR-LA, ECG, and Electricity datasets, the embedding size is 128, and the batch size is set to 32.

To reduce the number of parameters, we adopt a linear transform to reshape the original time domain representation Y G t ∈ R NT × d to Y t ∈ R N × T × d , and map Y t to a low-dimensional tensor Y t ∈ R N × l × d with l < T . We then reshape Y t ∈ R N × ( ld ) and feed it to FFN. We perform a grid search on the dimensions of FFN, i.e., d ffn 1 and d ffn 2 , over { 32 , 64 , 128 , 256 , 512 } and tune the intermediate dimension l over { 2 , 4 , 6 , 8 , 12 } . The settings of the three hyperparameters over all datasets are shown in Table 8. By default, we set the diffusion step (layers) K = 3 for all datasets.

E.5 Details for Visualization Experiments

To verify the effectiveness of FourierGNN in learning the spatiotemporal dependencies on the hypervariate graph, we obtain the output of FourierGNN as the node representation, denoted as Y G t = IDFT(FourierGNN( X G t )) ∈ R NT × d with Inverse Discrete Fourier Transform ( IDFT ). Then, we visualize the adjacency matrix A calculated based the flatten node representation Y G t ∈ R NT × d , formulated as A = Y G t ( Y G t ) T ∈ R NT × NT , to show the variable correlations. Note that A is normalized via A / max( A ) . Since it is not feasible to directly visualize the huge adjacency matrix A of the hypervariate graph, we visualize its different subgraphs in Figures 3, 4, 9, and 10 to better verify the learned spatiotemporal information on the hypervariate graph from different perspectives.

Figure 3. We select 8 counties and visualize the correlations between 12 consecutive time steps for each selected county respectively. Figure 3 reflects the temporal correlations within each variable.

Figure 4: On the METR-LA dataset, we average its adjacency matrix A over the temporal dimension (i.e., marginalizing T ) to A ′ ∈ R N × N . Then, we randomly select 20 detectors out of all N = 207 detectors and obtain their corresponding sub adjacency matrix ( R 20 × 20 ) from A ′ for visualization. We further compare the sub-adjacency with the real road map (generated by the Google map tool) to verify the learned dependencies between different detectors.

Figure 9. Since we adopt a 3 -layer FourierGNN, we can calculate four adjacency matrices based on the spectrum input X G t of FourierGNN and the outputs of each layer in FourierGNN. Following the

way of visualization in Figure 4, we select 10 counties and two timestamps on the four adjacency matrices for visualization. Figure 9 shows the effects of each layer of FourierGNN in filtering or enhancing variable correlations.

Figure 10. On the COVID-19 dataset, we randomly select 10 counties out of N = 55 counties and obtain their four sub-adjacency matrices of four consecutive days for visualization. Each of the four sub adjacency matrices R 10 × 10 embodies the dependencies between counties in one day. Figure 10 reflects the time-varying dependencies between counties (i.e., variables).

F Additional Results

To further evaluate the performance of our model FourierGNN in multi-step forecasting, we conduct more experiments on the Wiki, METR-LA, and ECG datasets, respectively. We compare our model FourierGNN with five models (including StemGNN [10], AGCRN [2], GraphWaveNet [17], MTGNN[13], and Informer [9]) on the Wiki dataset under different prediction lengths, and the results are shown in Table 9. From the table, we observe that FourierGNN outperforms other models on MAE, RMSE, and MAPE metrics for all the prediction lengths. On average, FourierGNN improves MAE, RMSE, and MAPE by 7.4%, 3.5%, and 22.3%, respectively. Among these models, AGCRN shows promising performances since it captures the spatial and temporal correlations adaptively. However, it fails to simultaneously capture spatiotemporal dependencies, limiting its forecasting performance. In contrast, our model captures comprehensive spatiotemporal dependencies simultaneously on a hypervariate graph for multivariate time series forecasting.

Furthermore, we compare our model FourierGNN with seven MTS models (including STGCN [1], DCRNN [18], StemGNN [10], AGCRN [2], GraphWaveNet [17], MTGNN [13], Informer [9], and CoST [36]) on the METR-LA dataset which has a predefined graph topology in the data, and the results are shown in Table 10. On average, we improve 5 . 7% on MAE and 1 . 5% on RMSE. Among these models, StemGNN achieves competitive performance because it combines GFT to capture the spatial dependencies and DFT to capture the temporal dependencies. However, it is also limited to simultaneously capturing spatiotemporal dependencies. CoST learns disentangled seasonal-trend representations for time series forecasting via contrastive learning and obtains competitive results. But, our model still outperforms CoST. Because, compared with CoST, our model not only can learn the dynamic temporal representations, but also capture the discriminative spatial representations. Besides, STGCN and DCRNN require pre-defined graph structures. But StemGNN and our model outperform them for all steps, and AGCRN outperforms them when the prediction lengths are 9 and 12. This also shows that a novel adaptive graph learning can precisely capture the hidden spatial dependency. In addition, we compare FourierGNN with the baseline models under the different prediction lengths on the ECG dataset, as shown in Figure 5. It reports that FourierGNN achieves the best performances (MAE, RMSE, and MAPE) for all prediction lengths.

(a) MAE

(b) RMSE

(c) MAPE

0.12

AGCRN

DeepGLO

GraphWaveNet

0.16

AGCRN

DeepGLO

GraphWaveNet

50

AGCRN

DeepGLO

GraphWaveNet

Informer

Informer

LSTNet

MAE

0.10

LSTNet

MTGNN

FourierGNN

StemGNN

RMSE

0.12

0.14

LSTNet

MTGNN

FourierGNN

StemGNN

MAPE(%)

40

OE

MTGNN

StemGNN

FourierGNN

0.08

0.10

20

0.06

0.08

10

123456

5789101112

123456

5789101112

123456789101112

prediction lengths

prediction lengths

prediction lengths

G Further Analyses

G.1 Scalability Analysis

We further conduct experiments on the Wiki dataset to investigate the performance of FourierGNN under different graph sizes ( N × T ). The results are shown in Figure 6, where Figure 6(a), Figure 6(b) and Figure 6(c) show MAE, RMSE, and MAPE at the different number of nodes, respectively. From these figures, we observe that FourierGNN keeps a leading edge over the other state-of-the-art MTS models as the number of nodes increases. The results demonstrate the superiority and scalability of FourierGNN on large-scale datasets.

(a) MAE

(b) RMSE

(c) MAPE

MTGNN

AGCRN

0.30

AGCRN

MTGNN

180

AGCRN

MTGNN

0.20

Informer

StemGNN

GraphWaveNet

0.25

Informer

GraphWaveNet

StemGNN

160

1

FourierGNN

GraphWaveNet

StemGNN

MAE

0.15

FourierGNN

RMSE

0.20

FourierGNN

MAPE(%)

140

120

0.10

0.15

100

0.05

0.10

80

300

350

400

450

500

300

350

400

450

500

300

350

400

450

500

Number of Nodes

Number of Nodes

Number of Nodes

G.2 Parameter Analysis

We evaluate the forecasting performance of our model FourierGNN under different diffusion steps (layers) on the COVID-19 dataset, as illustrated in Table 11. The table shows that FourierGNN achieves increasingly better performance from K = 1 to K = 4 and achieves the best results when K = 3 . With the further increase of K , FourierGNN obtains inferior performance. The results indicate that high-order diffusion information

is beneficial for improving forecasting accuracy, but the diffusion information may gradually weaken the effect or even bring noises to forecasting with the increase of the order.

In addition, we conduct additional experiments on the ECG dataset to analyze the effect of the input lookback window length T and the embedding dimension d , as shown in Figure 7 and Figure 8, respectively. Figure 7 shows that the performance (including RMSE and MAPE) of FourierGNN gets better as the input lookback window length increases, indicating that FourierGNN can learn a comprehensive hypervariate graph from long MTS inputs to capture the spatial and temporal dependencies. Moreover, Figure 8 shows that the performance (RMSE and MAPE) first increases and then decreases with the increase of the embedding size, which is attributed that a large embedding

0.0800

RMSE

MAPE

F 11.3

0.0795

11.2

0.0790

RMSE

11.1

MAPE(%)

0.0785

11.0

0.0780

10.9

0.0775

10.8

5

10

15

20

25

30

35

input length T

0.0789

RMSE

-11.125

MAPE

0.0788

11.100

11.075

RMSE

0.0787

11.050

MAPE(%

0.0786

11.025

11.000

0.0785

10.975

0.0784

10.950

40

60

80

100

120

140

160

embedding size

size improves the fitting ability of FourierGNN but it may easily lead to the overfitting issue especially when the embedding size is too large.

G.3 Ablation Study

We provide more details about each variant used in this section and Section 5.3.

w/o Embedding . A variant of FourierGNN feeds the raw MTS input instead of its embeddings into the graph convolution in Fourier space.

w/o Dynamic FGO . A variant of FourierGNN uses the same FGO for all diffusion steps instead of applying different FGOs in different diffusion steps. It corresponds to a vanilla graph filter.

w/o Residual . A variant of FourierGNN does not have the K = 0 layer output, i.e., X G t , in the summation.

w/o Summation . A variant of FourierGNN adopts the last order (layer) output as the final frequency output of the FourierGNN.

We conduct another ablation study on the COVID-19 dataset to further investigate the effects of the different components of our FourierGNN. The results are shown in Table 12, which confirms the results in Table 4 and further verifies the effectiveness of each component in FourierGNN. Both Table 12 and Table 4 report that the embedding and dynamic FGOs in FourierGNN contribute more than the design of residual and summation to the state-of-the-art performance of FourierGNN.

H Visualizations

H.1 Visualization of the Diffusion Process in FourierGNN

To gain insight into the operation of the FGO, we visualize the frequency output of each layer in our FourierGNN. We select 10 counties from the COVID-19 dataset and visualize their adjacency matrices at two different timestamps, as shown in Figure 9. From left to right, the results correspond to the original spectrum of the input, as well as the outputs of the first, second, and third layers of the FourierGNN. From the top, we can find that as the number of layers increases, some correlation values are reduced, indicating that some correlations are filtered out. In contrast, the bottom case illustrates some correlations are enhanced as the number of layers increases. These results show that FGO can adaptively and effectively capture important patterns while removing noises, enabling the learning of a discriminative model.

1.0

1.0

1.0

6°0

0.9

0.8

0.8

0.8

0.8

0.7

0.7

0.6

0.6

0.6

0.6

0.4

0.4

0.5

0.5

0.4

0.4

0.2

0.2

0.3

0.3

0.2

0.2

0.0

1.0

1.0

6'0

0.9

0.8

0.8

0.8

0.8

0.6

0.6

0.7

0.7

0.6

0.4

-0.4

0.5

0.4

0.2

0.2

0.3

0.3

- 0.0

0.0

0.2

0.2

H.2 Visualization of Time-Varying Dependencies Learned by FourierGNN

Furthermore, we explore the capability of FourierGNN in capturing time-varying dependencies among variables. To investigate this, we perform additional experiments to visualize the adjacency matrix of 10 randomly-selected counties over four consecutive days on the COVID-19 dataset. The visualization results, displayed as a heatmap in Figure 10, reveal clear spatial patterns that exhibit continuous evolution in the temporal dimension. This is because FourierGNN can attend to the time-varying variability of the spatiotemporal dependencies. These results verify that our model enjoys the feasibility of exploiting the time-varying dependencies among variables.

Based on the insights gained from these visualization results, we can conclude that the hypervariate graph structure exhibits strong capabilities to encode spatiotemporal dependencies. By incorporating FGOs, FourierGNN can effectively attend to and exploit the time-varying dependencies among variates. The synergy between the hypervariate graph structure and FGOs empowers FourierGNN to capture and model intricate spatiotemporal relationships with remarkable effectiveness.

1.0

1.0

1.0

1.0

0.9

0.9

0.9

- 0.9

- 0.8

- 0.8

N

- 0.8

N

- 0.8

- 0.7

M

0.7

M

0.7

- 0.7

0.6

0.6

- 0.6

- 0.6

0.5

0.5

- 0.5

0.5

'0 -

0.4

90

- 0.4

- 0.3

- 0.4

- 0.3

- 0.2

0.4

2

0.2

0

1

2

E

4

9

- 0.2

0

1

2

3

7

9

7

- 0.2

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents FourierGNN for multivariate time series forecasting from a pure graph perspective, performing matrix multiplications in Fourier space, which has not been investigated so far. They design a new hypervariate graph structure to consider spatiotemporal dynamics unitedly and reformulate the graph operations on the hypervariate graph in Fourier space. Extensive experiments have demonstrated superior performance with higher efficiency and fewer parameters over state-of-the-art methods.\n\nSTRENGTHS:\n1. The paper is overall well-written. I think the contribution and novelty of the proposed work are clearly presented.\n2. The idea of directly applying graph networks for multivariate time series forecasting seems novel and exciting. The hypervariate graph structure can encode spatiotemporal dependencies unitedly, and the visualizations in the experiments have also verified its advantages.\n3. The authors argue that performing the multiplication between the input and the proposed FGO in Fourier space is equivalent to a graph convolution in the time domain, while the multiplications in Fourier space have lower complexity. The argument is proved by the authors and seems very meaningful. I think this argument provides a new path for conducting graph operations in Fourier space.\n4. The effectiveness is validated through extensive experiments across seven real-world datasets. Sufficient analysis including efficiency analysis and ablation study is also provided.\n\nWEAKNESSES:\n1. In the efficiency analysis section, the authors assess the overall efficiency of the forecasting process. To specifically evaluate the efficiency of FourierGNN, it would be more appropriate to focus on comparing the number of parameters involved in the graph operations with those of the baselines.\n2. The MAPE (Mean Absolute Percentage Error) results on some datasets (e.g., Traffic, COVID-19) do not achieve state-of-the-art performance.\n\nQUESTIONS:\nFourierGNN demonstrates impressive performance across multiple datasets, with notable success observed on the COVID dataset. Could you explain the reasons that contribute to the superior performance of FourierGNN on the COVID dataset compared to other datasets?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper has studied a popular and important problem, i.e., modeling the intricate spatial and temporal dependencies among multivariate time series for accurate forecasting. To overcome the main limitation that existing works always separately model spatial and temporal, this work designs a new hypervariate graph structure to encode the spatiotemporal dynamics unitedly, and proposes a novel FourierGNN to learn the spatiotemporal dependencies. Extensive experiments on seven real-world datasets demonstrate FourierGNN achieves good performances in both accuracy and efficiency compared with state-of-the-art methods.\n\nSTRENGTHS:\nThis paper presents a novel formulation from a pure graph perspective to model spatiotemporal dependencies for MTS forecasting. The work is interesting and original, which is different from previous GNN-based methods that typically model spatial and temporal dependencies with distinct graph and temporal networks. The novel pure graph modeling is straightforward yet quite meaningful for MTS forecasting, which brings up good inspiring insights.\n\nThis paper proposes a graph neural network, namely FourierGNN, to cooperate with the pure graph formulation and provides a theoretical guarantee of effectiveness. \n\nThis paper seems to have solid, extensive and diverse experiments, which are conducted on seven real-world different datasets. The extensive experimental results have clearly demonstrated the obvious performance improvement of FourierGNN in terms of accuracy and efficiency over state-of-the-art MTS forecasting models.\n\nWEAKNESSES:\nI have two concerns. \nMy first concern is about the hypervariate graph structure. Although the authors stated that all variables at all timestamps can encode high-resolution relationships, it also may introduce some redundant information or unwanted correlations, such as some variables that are far apart in time. How can this be properly handled? Besides, I find that the core operations of FourierGNN are conducted in the Fourier space, so I am curious whether transferring the hypervariate graph structure into the Fourier space help to learn dependencies. \n\nIn addition, another concern is whether FourierGNN can also be applied to other domains apart from the multivariate forecasting. Maybe it can be extended to other graph tasks.\n\nMinor typos: In the caption of Figure 2, Given the hypervariate graph $\\mathcal{G}=(X^\\mathcal{G}_t, A^\\mathcal{G}_t)$  should be $\\mathcal{G}_t=(X^\\mathcal{G}_t, A^\\mathcal{G}_t)$\n\nQUESTIONS:\n1.Besides MTS forecasting, can FourierGNN be applied to other domains as well?\n2.How does the proposed model handle the redundant information in the hypervariate graph structure?\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the time series forecasting problem.\nThe authors propose a model that represents each scalar\nobservation as a node in a (fully connected) graph,\nencodes the nodes and finally regresses the future\nobservations on all previous encoded observations\n(via a fully connected layer). As encoding they propose\na discrete fourier transform followed by several node-wise\nfully connected layers and finally an inverse fourier transform.\nIn experiments on 6 datasets they show improvements over\nseveral baselines.\n\nSTRENGTHS:\ns1. interesting problem: time series forecasting\ns2. interesting approach: parametrizing transformations in fourier space\ns3. results showing improvements over many baselines\n\nWEAKNESSES:\nw1. a main aspect of the proposed model is not clear. \nw2. missed recent related work.\nw3. experimental protocol is unclear and deviates from the literature.  \nw4. for a key component of the proposed model, there is no ablation study\n  demonstrating its impact.\n\nQUESTIONS:\nw1. a main aspect of the proposed model is not clear. \n- the authors describe the fourier graph layer as applying the discrete\n  fourier transform on the graph features X\\in\\R^{nd\\times 1} of a complete graph,\n  explicitly not the graph fourier transform. Which dimension is the time\n  dimension for this fourier transform? The node dimension of X contains\n  nd many entries, one for each time point and channel. While one can\n  compute a fourier transform also over this vector, what is it semantics?\n  How does it capture time varying information? And would this not crucially\n  depend on how on is ordering these nd many entries?\n\nw2. missed recent related work.\n- esp. PatchTST [Nie et al. 2023] and dlinear [Zeng et al. 2023] are well\n  known model outperforming your baselines by a good margin.\n\nw3. experimental protocol is unclear and deviates from the literature.  \n- what is the forecasting horizon \\tau you are using?\n- the experiments report only a single number per dataset and error measure,\n  while in the literature usually  results for a portfolio of forecasting horizons\n  are reported (e.g., 96, 192, 336, 720, e.g., the Fedformer). Why do you deviate\n  from this standard? This way it also is not possible to compare your results\n  with the results published in those papers.\n- usually results are also reported for further datasets such as ETTm2, Exchange,\n  Weather and ILI. How does your model compare to those published results?\n\nw4. for a key component of the proposed model, there is no ablation study\n  demonstrating its impact.\n- while there are many papers that simply operate on the fourier spectrum\n  of a time series, one key component of the model of the authors seems\n  to be that after these operations they move back into the time domain\n  by the inverse fourier transform. What is the impact of this back transformation?\n   \nreferences:\n- Nie, Yuqi, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. “A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers.” arXiv, March 5, 2023. https://doi.org/10.48550/arXiv.2211.14730.\n- Zeng, Ailing, Muxi Chen, Lei Zhang, and Qiang Xu. “Are Transformers Effective for Time Series Forecasting?” In AAAI, 2023.\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors study a problem in GNN-based multivariate time series (MTS) forecasting, i.e. modeling spatial correlations and temporal dependencies in the same time. In particular, the authors do not follow previous works on regarding the input as T graphs and capturing temporal dependencies between graphs by temporal networks, but propose to formulate T graphs as a hypervariate graph(a pure fully-connected graph with NT nodes). Specifically, to deal with the huge node amount NT in a hypervariate graph, the authors propose the FourierGNN which utilize matrix multiplications in the Fourier space of graphs to decrease the quadratic computational complexity to quasi linear one. Empirical results on seven datasets show the effectiveness of this method.\n\nSTRENGTHS:\n1. The authors provide a novel view of graph-based MTS forecasting called hypervariate graph.\n2. The authors propose Fourier Graph Operator and FourierGNN for efficient convolution on hypervariate graph.\n3. In the authors’ experiments, FourierGNN obtains superior performance.\n\nWEAKNESSES:\n1.\tThe detail of the experiments is not clear. The authors run previous models on different datasets but do not show whether some hyperparameters, such as feature dim, change with the dataset.  \n2.\tThe authors claim that FourierGNN is efficient for hypervariate graph, but do not provide other networks’ performance such as GCN on hypervariate graph. \n3.\tThe models compared with Efficiency Analysis are too old.\n4.\tThe authors do not explain the reason for the Green kernel. The Green kernel is a strong assumpition. In line 161, k_{ij} = A_{ij} W and k_{ij} = k_{i-j} results A_{j+k}_{j} equals for any j, which strictly restricts the topology of the hypervariate graph. Besides, there is no visualization of A in each layer to check whether the green kernel assumption is satisfied during training.\n\nQUESTIONS:\n1.\tWhat’s the performance of FourierGNN on longer prediction lengths such as {24, 36, 48, 60} in FEDformer?\n2.\tWhat’s the meaning of “the same sparsity pattern” in line 196?\n3.\tHow to determine the value of A_i in equation (7), A_0 is identity and what about A_1, A_2?\n\nLIMITATIONS:\nPlease check the weakness", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}