Original: E DGE -V ARYING F OURIER G RAPH N ETWORKS FOR M ULTIVARIATE T IME S ERIES F ORECASTING
Revised: E DGE -V ARYING F OURIER G RAPH N ETWORKS FOR M ULTIVARIATE T IME S ERIES F ORECASTING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The key problem in multivariate time series (MTS) analysis and forecasting aims to disclose the underlying couplings between variables that drive the co-movements.
Revised: The key to multivariate time series (MTS) analysis and forecasting is to disclose the underlying couplings between variables that drive the co-movements.

Original: Considerable recent successful MTS methods are built with graph neural networks (GNNs) due to their essential capacity for relational modeling.
Revised: Considerable recent successful MTS methods are built with graph neural networks (GNNs) due to their essential capacity for relational modeling.

Original: However, previous work often used a static graph structure of time-series variables for modeling
Revised: However, previous work often used a static graph structure of time-series variables for modeling

Original: MTS failing to capture their ever-changing correlations over time.
Revised: MTS, but failed to capture their ever-changing correlations over time.

Original: To this end, a fully-connected supra-graph connecting any two variables at any two timestamps is adaptively learned to capturethe high-resolution variable dependencies via an efficient graph convolutional network. Specifically, we construct the Edge-Varying
Revised: In this paper, we build a fully-connected supra-graph, representing non-static correlations between any two variables at any two timestamps, to capture high-resolution spatial-temporal dependencies. Whereas, conducting graph convolutions on such a supra-graph heavily increases computational complexity.

Original:  Fourier Graph Networks (EV-FGN) equipped with Fourier Graph Shift Operator (FGSO) which efficiently performs graph convolution in the frequency domain. As a result, a high-efficiency scale-free parameter learning scheme is derived for MTS analysis and forecasting according to  the convolution theorem.
Revised: As a result, we propose the novel Edge-Varying Fourier Graph Networks (EV-FGN), which reformulates the graph convolutions in the frequency domain with high efficiency and scale-free parameters, and applies edge varying graph filters to capture the time-varying variable dependencies.

Original: Extensive experiments show that EV-FGN outperforms state-of-the-art methods on seven real-world MTS datasets.
Revised: Extensive experiments show that EV-FGN outperforms state-of-the-art methods on seven real-world MTS datasets.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Multivariate time series (MTS) forecasting is a key ingredient in many real-world scenarios, including weather forecasting Zheng et al.
Revised: Multivariate time series (MTS) forecasting is a key ingredient in many real-world scenarios, including weather forecasting Zheng et al.

Original: (2015), decision making Borovykh et al.
Revised: (2015), decision making Borovykh et al.

Original: traffic forecasting
Revised: traffic forecasting

Original: Yu et al.
Revised: Yu et al.

Original: (2018a)
Revised: (2018a)

Original: ; Bai et al.
Revised: ; Bai et al.

Original: COVID-19 prediction Cao et al.
Revised: COVID-19 prediction Cao et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: (2022), etc.
Revised: (2022), etc.

Original: Recently, deep neural networks, such as long short-term memory (LSTM) Hochreiter & Schmidhuber (1997), convolutional neural network (CNN)
Revised: Recently, deep neural networks, such as long short-term memory (LSTM) Hochreiter & Schmidhuber (1997), convolutional neural network (CNN)

Original: Borovykh et al.
Revised: Borovykh et al.

Original: (2017), Transformer Vaswani et al.
Revised: (2017), Transformer Vaswani et al.

Original: (2017), have dominated MTS modeling.
Revised: (2017), have dominated MTS modeling.

Original: In particular, graph neural networks (GNNs) have demonstrated promising performance on MTS forecasting with their essential capability to capture the complex couplings between time-series variables.
Revised: In particular, graph neural networks (GNNs) have demonstrated promising performance on MTS forecasting with their essential capability to capture the complex couplings between time-series variables.

Original: Some studies enable to adaptively learn the graph for MTS forecasting even without an explicit graph structure, e.g., by node similarity Mateos et al.
Revised: Some studies enable to adaptively learn the graph for MTS forecasting even without an explicit graph structure, e.g., by node similarity Mateos et al.

Original: ; Bai et al.
Revised: ; Bai et al.

Original: ; Wu et al.
Revised: ; Wu et al.

Original: and/or self-attention mechanism Cao et al.
Revised: and/or self-attention mechanism Cao et al.

Original: Despite the success of GNNs on MTS forecasting, three practical challenges are eagerly demanded to address: 1) the dependencies between each pair of time series variables are generally non-static, which demands full dependencies modeling with all possible lags; 2) a high-efficiency dense graph learning method is demanded to replace the high-cost operators of graph convolutions and attention (with quadratic time complexity with the graph size); 3) the graph over MTS varies with different temporal interactions, which demands an efficient dynamic structure encoding method.
Revised: Despite the success of GNNs on MTS forecasting, three practical challenges are eagerly demanded to address: 1) the dependencies between each pair of time series variables are generally non-static, which demands full dependencies modeling with all possible lags; 2) a high-efficiency dense graph learning method is demanded to replace the high-cost operators of graph convolutions and attention (with quadratic time complexity of the graph size); 3) the graph over MTS varies with different temporal interactions, which demands an efficient dynamic structure encoding method.

Original: In view of the increasing leverage of the convolution theorem in Fourier theory to design high-efficiency neural units (operators) to replace costly attention computations Guibas et al.

Original: (2022), we are inspired to perform graph computations in the Fourier space and provide a potential alternative for processing supra-graphs jointly attending to spatial-temporal dependencies.

Original: Different from most GNN-based MTS that constructs graphs to model the spatial correlation between variables (i.e. nodes) Bai et al.
Revised: In this paper, different from most GNN-based methods that construct graphs to model the spatial correlation between variables (i.e. nodes) Bai et al.

Original: ; Wu et al.
Revised: ; Wu et al.

Original: (2019; 2020), we attempt to build a supra-graph that sheds light on the "high-resolution" correlations between any two variables at any two timestamps  and largely enhances the expressiveness on non-static spatial-temporal dependencies.
Revised: (2019; 2020), we attempt to build a supra-graph that sheds light on the "high-resolution" correlations between any two variables at any two timestamps (i.e., fine-grained spatial-temporal dependencies), and largely enhances the expressiveness on non-static spatial-temporal dependencies.

Original: Obviously, the supra-graph will heavily increase the computational complexity of GNN-based model, then a high-efficiency learning method is required to reduce the cost for model training.
Revised: Obviously, the supra-graph will heavily increase the computational complexity of GNN-based model, then a high-efficiency learning method is required to reduce the cost for model training.

Original: Inspired by
Revised: Inspired by

Original: Fourier Neural Operator (FNO) Li et al. (2021), we transform graph convolutions in the time domain to much lower-complexity matrix multiplication in the frequency domain by defining an efficient  Fourier Graph Shift Operator (FGSO).
Revised: Fourier Neural Operator (FNO) Li et al. (2021), we reformulate the graph convolution (time domain) to much lower-complexity matrix multiplication in the frequency domain by leveraging an efficient and newly-defined Fourier Graph Shift Operator (FGSO).

Original: In addition, it is necessary to consider the multiple iterations (layers) of graph convolution to expand receptive neighbors and mix the diffusion information in the graph.
Revised: In addition, it is necessary to consider the multiple iterations (layers) of graph convolutions to expand receptive neighbors and mix the diffusion information in the graph.

Original: To capture time-varying diffusion over the supra-graph, we introduce the edge-varying graph filter, expressed as a polynomial of multiple graph shift operators (GSOs), to weigh the graph edges differently from different iterations. We then reformulate the edge-varying graph filter with  FGSOs in the frequency domain to reduce the computational cost.
Revised: To capture time-varying diffusion over the supra-graph, we introduce the edge-varying graph filter to weight the graph edges differently from different iterations and reformulate the edge-varying graph filter with multiple frequency-invariant FGSOs in the frequency domain to reduce the computational cost of graph convolutions.

Original: Accordingly, we construct a complex-valued feed forward network, dubbed as Edge-Varying Fourier Graph Networks (EV-FGN), stacked with multiple FGSOs to perform high-efficiency multi-layer graph convolutions in the Fourier space.
Revised: Finally, a novel Edge-Varying Fourier Graph Networks (EV-FGN) is designed for MTS analysis, which is stacked with multiple FGSOs to perform high-efficient multi-layer graph convolutions in the Fourier space.

Original: The main contributions of this paper are summarized as follows:
Revised: The main contributions of this paper are summarized as follows:

Original: • We adaptively learn a supra-graph, representing non-static correlations between any two variables at any two timestamps, to capture high-resolution spatial-temporal dependencies.
Revised: • We adaptively learn a supra-graph, representing non-static correlations between any two variables at any two timestamps, to capture high-resolution spatial-temporal dependencies.

Original: • To efficiently compute graph convolutions over the supra-graph, we define FGSO that has the capacity of scale-free learning parameters in the Fourier space.
Revised: • To efficiently compute graph convolutions over the supra-graph, we reformulate the graph convolutions in the Fourier space by leveraging FGSO. To the best of our knowledge, this work makes the first step to reformulate the graph convolutions in the Fourier space.

Original: • We design  EV-FGN evolved from edge varying graph filters  to capture the time-varying variable dependencies where FGSOs are used to reduce the complexity of graph convolutions.
Revised: • We design a novel network EV-FGN evolved from edge-varying graph filters for MTS analysis to capture the time-varying variable dependencies in the Fourier space.

Revised: This study makes the first attempt to design a complex-valued feed-forward network in the Fourier space to efficiently compute multi-layer graph convolutions.

Original: • Extensive experimental results on seven MTS datasets demonstrate that EV-FGN achieves state-ofthe-art performance with high efficiency and fewer parameters.
Revised: • Extensive experimental results on seven MTS datasets demonstrate that EV-FGN achieves state-ofthe-art performance with high efficiency and fewer parameters.

Original: Multifaceted visualizations further interpret the efficacy of EV-FGN in graph representation learning for MTS forecasting.
Revised: Multifaceted visualizations further interpret the efficacy of EV-FGN in graph representation learning for MTS forecasting.

Original:  R ELATED W ORKS 2. M ULTIVARIATE T IME S ERIES F ORECASTING
Revised:  R ELATED W ORKS 2. M ULTIVARIATE T IME S ERIES F ORECASTING

Original: Classic time series forecasting methods are linear models, such as VAR Watson (1993), ARIMA
Revised: Classic time series forecasting methods are linear models, such as VAR Watson (1993), ARIMA

Original: Asteriou & Hall (2011), state space model (SSM) Hyndman et al.
Revised: Asteriou & Hall (2011) and state space model (SSM) Hyndman et al.

Original: Recently, deep learning based methods Lai et al.
Revised: Recently, deep learning based methods Lai et al.

Original: Sen et al.
Revised: Sen et al.

Original: ; Zhou et al.
Revised: ; Zhou et al.

Original: (2021) have dominated MTS forecasting due to their capability of fitting any complex nonlinear correlations Lim & Zohren (2021).
Revised: (2021) have dominated MTS forecasting due to their capability of fitting any complex nonlinear correlations Lim & Zohren (2021).

Original: MTS with GNN .
Revised: MTS with GNN .

Original: More recently, MTS have embraced GNN Wu et al.
Revised: More recently, MTS have embraced GNN Wu et al.

Original: ; Bai et al.
Revised: ; Bai et al.

Original: ; Wu et al.
Revised: ; Wu et al.

Original: ; Yu et al.
Revised: ; Yu et al.

Original: (2018b)
Revised: (2018b)

Original: ; Chen et al.
Revised: ; Chen et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: (2018) due to their best capability of modeling structural dependencies between variables.
Revised: (2018) due to their best capability of modeling structural dependencies between variables.

Original: Most of these models, such as STGCN Yu et al.
Revised: Most of these models, such as STGCN Yu et al.

Original: (2018b)
Revised: (2018b)

Original: , DCRNN Li et al.
Revised: , DCRNN Li et al.

Original: and TAMP-S2GCNets Chen et al.
Revised: and TAMP-S2GCNets Chen et al.

Original: (2022), require a pre-defined graph structure which is usually unknown in most cases.
Revised: (2022), require a pre-defined graph structure which is usually unknown in most cases.

Original: In recent years, some GNN-based works
Revised: In recent years, some GNN-based works

Original: Kipf et al.
Revised: Kipf et al.

Original: ; Deng & Hooi (2021) account for the dynamic dependencies due to network design such as the time-varying attention Deng & Hooi (2021).
Revised: ; Deng & Hooi (2021) account for the dynamic dependencies due to network design such as the time-varying attention Deng & Hooi (2021).

Original: In comparison, our proposed model captures the dynamic dependencies leveraging the high-resolution correlation in the supra-graph without introducing specific networks.
Revised: In comparison, our proposed model captures the dynamic dependencies leveraging the high-resolution correlation in the supra-graph without introducing specific networks.

Original: MTS with Fourier transform .
Revised: MTS with Fourier transform .

Original: Recently, increasing MTS forecasting models have introduced the
Revised: Recently, increasing MTS forecasting models have introduced the

Original: Fourier theory into neural networks as high-efficiency convolution operators Guibas et al.
Revised: Fourier theory into neural networks as high-efficiency convolution operators Guibas et al.

Original: Chi et al.
Revised: Chi et al.

Original: SFM Zhang et al.
Revised: SFM Zhang et al.

Original: (2017) decomposes the hidden state of LSTM into multiple frequencies by discrete Fourier transform (DFT).
Revised: (2017) decomposes the hidden state of LSTM into multiple frequencies by discrete Fourier transform (DFT).

Original: mWDN
Revised: mWDN

Original: Wang et al.
Revised: Wang et al.

Original: (2018) decomposes the time series into multilevel sub-series by discrete wavelet decomposition and feeds them to LSTM network, respectively.
Revised: (2018) decomposes the time series into multilevel sub-series by discrete wavelet decomposition and feeds them to LSTM network, respectively.

Original: ATFN Yang et al.
Revised: ATFN Yang et al.

Original: (2022) utilizes a time-domain block to learn the trending feature of complicated non-stationary time series and a frequency-domain block to capture dynamic and complicated periodic patterns of time series data.
Revised: (2022) utilizes a time-domain block to learn the trending feature of complicated non-stationary time series and a frequency-domain block to capture dynamic and complicated periodic patterns of time series data.

Revised: FEDformer Zhou et al.

Revised: (2022) proposes an attention mechanism with low-rank approximation in frequency and a mixture of expert decomposition to control the distribution shift.

Original: However, these models only capture temporal dependencies in the frequency domain.
Revised: However, these models only capture temporal dependencies in the frequency domain.

Original: StemGNN Cao et al.
Revised: StemGNN Cao et al.

Original: (2020) takes the advantages of both interseries correlations and temporal dependencies by modeling them in the spectral domain, however, it captures the temporal and spatial dependencies separately.
Revised: (2020) takes the advantages of both inter-series correlations and temporal dependencies by modeling them in the spectral domain, but, it captures the temporal and spatial dependencies separately.

Original: Different from these works, our model enables to jointly encode spatial-temporal dependencies in the Fourier space. 
Revised: Unlike these efforts, our model is able to jointly encode spatial-temporal dependencies in the Fourier space. Input I npu t E m b e dd i n g 2 D

Revised: FF T

Original: 2. G RAPH S HIFT O PERATOR
Revised: Fourier Space 2 D I FF T FF N

Revised: Edge Varying Fourier Graph Network Edge Varying Fourier Graph Network

Revised: FGSO + …

Revised: FGSO

Revised: Output FGSO 𝑋 𝐗 𝒳 𝒮 1 𝒳 𝒮𝒮𝒮 𝐾 𝒮 1 𝒮 2 𝒳 ෑ 𝑖=

Revised: 𝐾 𝒮 𝑖 𝒳

Original:  Graph shift operators (GSOs) (e.g., the adjacency matrix and the Laplacian matrix) are a general set of linear operators which are used to encode neighbourhood topologies in the graph.
Revised: Time Space Time Space 𝐗 𝚿 2. G RAPH S HIFT O PERATOR Graph shift operators (GSOs) (e.g., the adjacency matrix and the Laplacian matrix) are a general set of linear operators which are used to encode neighbourhood topologies in the graph.

Original: Klicpera et al.
Revised: Klicpera et al.

Original: (2019) shows that applying the varying GSOs in the message passing step of GNNs can lead to significant improvement of performance.
Revised: (2019) shows that applying the varying GSOs in the message passing step of GNNs can lead to significant improvement of performance.

Original: Dasoulas et al.
Revised: Dasoulas et al.

Original: (2021) proposes a parameterized graph shift operator to automatically adapt to networks with varying sparsity.
Revised: (2021) proposes a parameterized graph shift operator to automatically adapt to networks with varying sparsity.

Original: Isufi et al.
Revised: Isufi et al.

Original: (2021) allows different nodes to use different GSOs to weigh the information of different neighbors.
Revised: (2021) allows different nodes to use different GSOs to weight the information of different neighbors.

Original: Hadou et al.
Revised: Hadou et al.

Original: (2022) introduces a linear composition of the graph shift operator and time-shift operator to design space-time filters for time-varying graph signals.
Revised: (2022) introduces a linear composition of the graph shift operator and time-shift operator to design space-time filters for time-varying graph signals.

Original: Inspired by these works, in this paper we design a varying parameterized graph shift operator in Fourier space.
Revised: Inspired by these works, in this paper we design a varying parameterized graph shift operator in Fourier space.

Original: 2. F OURIER N EURAL O PERATOR
Revised: 2. F OURIER N EURAL O PERATOR

Original: Different from classical neural networks which learn mappings between finite-dimensional Euclidean spaces, neural operators learn mappings between infinite-dimensional function spaces Kovachki et al.
Revised: Different from classical neural networks which learn mappings between finite-dimensional Euclidean spaces, neural operators learn mappings between infinite-dimensional function spaces Kovachki et al.

Original: (2021b).
Revised: (2021b).

Original: Fourier neural operators (FNOs), currently the most promising  of the neural operators, are universal, in the sense that they can approximate any continuous operator to the desired accuracy
Revised: Fourier neural operators (FNOs), currently the most promising one of the neural operators, are universal, in the sense that they can approximate any continuous operator to the desired accuracy

Original: Kovachki et al.
Revised: Kovachki et al.

Original: (2021a).
Revised: (2021a).

Original: Li et al.
Revised: Li et al.

Original: (2021) formulates a new neural operator by parameterizing the integral kernel directly in the Fourier space, allowing for an expressive and efficient architecture for partial differential equations.
Revised: (2021) formulates a new neural operator by parameterizing the integral kernel directly in the Fourier space, allowing for an expressive and efficient architecture for partial differential equations.

Original: Guibas et al.
Revised: Guibas et al.

Original: (2022) proposes an efficient token mixer that learns to mix in the Fourier domain which is  principled architectural modifications to FNO.
Revised: (2022) proposes an efficient token mixer that learns to mix in the Fourier domain which is a principled architectural modification to FNO.

Original: In this paper, we learn a Fourier graph shift operator by leveraging the Fourier Neural operator.
Revised: In this paper, we learn a Fourier graph shift operator by leveraging the Fourier Neural operator.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: Let us denote the entire MTS raw data as X ∈ R N × L with N variables and L timestamps.
Revised: Let us denote the entire MTS raw data as X ∈ R N × L with N variables and L timestamps.

Original: Under the rolling setting, we have window-sized time-series inputs with T timestamps, i.e., { X | X ⊂ X , X ∈ R N × T } .
Revised: Under the rolling setting, we have window-sized time-series inputs with T timestamps, i.e., { X | X ⊂ X , X ∈ R N × T } .

Original: Accordingly, we formulate the problem of MTS forecasting as learning the spatialtemporal dependencies simultaneously on a supra-graph G =
Revised: Accordingly, we formulate the problem of MTS forecasting as learning the spatialtemporal dependencies simultaneously on a supra-graph G =

Original: ( X, S ) attributed to each X .
Revised: ( X, S ) attributed to each X .

Original: The supra-graph G contains N ∗ T nodes that represent values of each variable at each timestamp in
Revised: The supra-graph G contains N ∗ T nodes that represent values of each variable at each timestamp in

Original: X , and S ∈ R ( N ∗ T ) ×
Revised: X , and S ∈ R ( N ∗ T ) ×

Original: ( N ∗ T ) is a graph shift operator (GSO) representing the connection structure of G .
Revised: ( N ∗ T ) is a graph shift operator (GSO) representing the connection structure of G .

Original: Since the underlying graph is unknown in most MTS scenarios, we assume all nodes in the supra-graph are connected with each other, i.e., a fully-connected graph, and perform graph convolutions on the graph to learn spatial-temporal representation.
Revised: Since the underlying graph is unknown in most MTS scenarios, we assume all nodes in the supra-graph are connected with each other, i.e., a fully-connected graph, and perform graph convolutions on the graph to learn spatial-temporal representation.

Original: Then, given the observed values of previous T steps at timestamp t , i.e., X t − T : t ∈ R N × T , the task of multi-step multivariate time series forecasting is to predict the values of N variables for next τ steps denoted as ˆ X t +1: t + τ ∈ R N × τ on the supra-graph G , formulated as follows:
Revised: Then, given the observed values of previous T steps at timestamp t , i.e., X t − T : t ∈ R N × T , the task of multi-step multivariate time series forecasting is to predict the values of N variables for next τ steps denoted as ˆ X t +1: t + τ ∈ R N × τ on the supra-graph G , formulated as follows:

Original: [Equation]
Revised: [Equation]

Original: where F is the forecasting model with parameters Θ .
Revised: where F is the forecasting model with parameters Θ .

Original: 3. O VERALL A RCHITECTURE
Revised: 3. O VERALL A RCHITECTURE

Original: The overall architecture of our model is illustrated in Fig.
Revised: The overall architecture of our model is illustrated in Fig.

Original: Given input data X ∈ R N × T , first we embed the data into embeddings X ∈ R N × T × d by assigning a d -dimension vector for each node using an embedding matrix Φ ∈ R N × T × d , i.e., X = X × Φ .
Revised: Given input data X ∈ R N × T , first we embed the data into embeddings X ∈ R N × T × d by assigning a d -dimension vector for each node using an embedding matrix Φ ∈ R N × T × d , i.e., X = X × Φ .

Original: Instead of directly learning the huge embedding matrix, we introduce two small parameter matrices: 1) a variable embedding matrix ϕ v ∈ R N × 1 × d , and 2) a temporal embedding matrix ϕ u ∈ R 1 × T ×
Revised: Instead of directly learning the huge embedding matrix, we introduce two small parameter matrices: 1) a variable embedding matrix ϕ v ∈ R N × 1 × d , and 2) a temporal embedding matrix ϕ u ∈ R 1 × T ×

Original: d to factorize Φ , i.e., Φ = ϕ v × ϕ u .
Revised: d to factorize Φ , i.e., Φ = ϕ v × ϕ u .

Original: Subsequently, we perform 2D discrete Fourier transform (DFT) on each discrete N × T spatialtemporal plane of the embeddings X and obtain the frequency input X :
Revised: Subsequently, we perform 2D discrete Fourier transform (DFT) on each discrete N × T spatialtemporal plane of the embeddings X and obtain the frequency input X :

Original: = DFT( X ) ∈
Revised: = DFT( X ) ∈

Original: C N × T × d .
Revised: C N × T × d .

Original: [Equation]

Original: We then feed X to K -layer
Revised: We then feed X to K -layer

Original: Edge-Varying Fourier Graph Networks (denoted as Ψ K ) to perform graph convolutions for capturing the spatial-temporal dependencies simultaneously in the Fourier space.
Revised: Edge-Varying Fourier Graph Networks (denoted as Ψ K ) to perform graph convolutions for capturing the spatial-temporal dependencies simultaneously in the Fourier space.

Original: To make predictions in time domain, we perform 2D inverse Fourier transform to generate representations X Ψ := IDFT(Ψ K ( X ))
Revised: To make predictions in time domain, we perform 2D inverse Fourier transform to generate representations X Ψ := IDFT(Ψ K ( X ))

Original: ∈ R N × T × d in the time domain.
Revised: ∈ R N × T × d in the time domain.

Original: The representation is then fed to two-layer feed-forward networks (FFN, see more details in Appendix E.4) parameterized with weights and biases denoted as ϕ ff to make predictions for future τ steps ˆ X ∈ R N × τ by one forward procedure.
Revised: The representation is then fed to two-layer feed-forward networks (FFN, see more details in Appendix F.4) parameterized with weights and biases denoted as ϕ ff to make predictions for future τ steps ˆ X ∈ R N × τ by one forward procedure.

Original: The L2 loss function for multi-step forecasting can be formulated as:
Revised: The L2 loss function for multi-step forecasting can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: with parameters Θ = { ϕ v , ϕ u , ϕ ff , Ψ K } and the groudtruth X t +1: t + τ ⊂ X at timestamp t.
Revised: with parameters Θ = { ϕ v , ϕ u , ϕ ff , Ψ K } and the groudtruth X t +1: t + τ ⊂ X at timestamp t

Original:  3. F OURIER G RAPH S HIFT O PERATOR
Revised: . 3. F OURIER G RAPH S HIFT O PERATOR

Original: According to the discrete signal processing on graphs Sandryhaila & Moura (2013), a graph shift operator (GSO) is defined as a general family of operators which enable the diffusion of information over graph structures Gama et al.
Revised: According to the discrete signal processing on graphs Sandryhaila & Moura (2013), a graph shift operator (GSO) is defined as a general family of operators which enables the diffusion of information over graph structures Gama et al.

Original: ; Dasoulas et al.
Revised: ; Dasoulas et al.

Original: Definition 1 ( Graph Shift Operator ) .
Revised: Definition 1 ( Graph Shift Operator ) .

Original: Given a graph G with n nodes, a matrix S ∈ R n × n is called a Graph Shift Operator (GSO) if it satisfies S ij = 0 if i ̸ = j and nodes i, j are not connected.
Revised: Given a graph G with n nodes, a matrix S ∈ R n × n is called a Graph Shift Operator (GSO) if it satisfies S ij = 0 if i ̸ = j and nodes i, j are not connected.

Original: The graph shift operator includes the adjacency, Laplacian matrices and their normalisations as instances of its class and represents the connection structure of the graph.
Revised: The graph shift operator includes the adjacency, Laplacian matrices and their normalisations as instances of its class, and represents the connection structure of the graph.

Original: Accordingly, given the graph G attributed to X ∈ R n × d , a generally form of spatial-based graph convolution is defined as
Revised: Accordingly, given the graph G attributed to X ∈ R n × d (corresponding to the supra-graph with n = N ∗ T ), a general form of spatial-based graph convolution is defined as

Original: [Equation]
Revised: [Equation]

Original: with the parameter matrix W ∈ R d × d .
Revised: with the parameter matrix W ∈ R d × d .

Original: Regarding S as n × n scores, we can define a matrix-valued kernel κ : [ n ] × [ n ] −→ R d
Revised: Regarding S as n × n scores, we can define a matrix-valued kernel κ : [ n ] × [ n ] −→ R d

Original: ×
Revised: ×

Original: d with κ [ i, j ] =
Revised: d with κ [ i, j ] =

Original: S ij ◦ W , where [ n ] = { 1 , 2 , · · · , n } .
Revised: S ij ◦ W , where [ n ] = { 1 , 2 , · · · , n } .

Original: Then the graph convolution can be viewed as a kernel summation.
Revised: Then the graph convolution can be viewed as a kernel summation.

Original: [Equation]
Revised: [Equation]

Original: In the special case of the Green’s kernel κ [ i, j ] = κ
Revised: In the special case of the Green’s kernel κ [ i, j ] = κ

Original: [ i − j ] , we can rewrite the kernel summation 
Revised: [ i − j ] , we can rewrite the kernel summation [Equation]

Original: [Equation]
Revised: [Equation]

Original: with X ∗ κ denotes the convolution of discrete sequences X and κ .
Revised: with X ∗ κ denotes the convolution of discrete sequences X and κ .

Original: According to the convolution theorem Katznelson (1970) (see Appendix B), the graph convolution is rewritten as
Revised: According to the convolution theorem Katznelson (1970) (see Appendix B), the graph convolution is rewritten as

Original: [Equation]
Revised: [Equation]

Original: where F and F − 1 denote the discrete Fourier transform (DFT) and its inverse (IDFT), respectively.
Revised: where F and F − 1 denote the discrete Fourier transform (DFT) and its inverse (IDFT), respectively.

Original: The multiplication in the Fourier space is a lower-complexity computation compared to the graph convolution, and DFT can be efficiently implemented by the fast Fourier transform (FFT).
Revised: The multiplication in the Fourier space is a lower-complexity computation compared to the graph convolution, and DFT can be efficiently implemented by the fast Fourier transform (FFT).

Original: Definition 2 ( Fourier Graph Shift Operator ) .
Revised: Definition 2 ( Fourier Graph Shift Operator ) .

Original: Given a graph G =
Revised: Given a graph G =

Original: ( X, S ) with input X ∈ R n × d and GSO S ∈ R n × n and the weight matrix W ∈ R d × d , the graph convolution is formulated as
Revised: ( X, S ) with input X ∈ R n × d and GSO S ∈ R n × n and the weight matrix W ∈ R d × d , the graph convolution is formulated as

Original: [Equation]
Revised: [Equation]

Original: where F denotes DFT, satisfies κ [ i, j ] = κ
Revised: where F denotes DFT, satisfies κ [ i, j ] = κ

Original: [ i − j ] , and × n is matrix multiplication on dimensions except that of n .
Revised: [ i − j ] , and × n is matrix multiplication on dimensions except that of n .

Original: We define S :
Revised: We define S :

Original: = F ( κ ) ∈ C n × d × d as a Fourier graph shift operator (FGSO).
Revised: = F ( κ ) ∈ C n × d × d as a Fourier graph shift operator (FGSO).

Original: In particular, turning to our case of the fully-connected supra-graph G with an all-one GSO S ∈ { 1 } n × n , it yields the  kernel κ [ i, j ] =
Revised: In particular, turning to our case of the fully-connected supra-graph G with an all-one GSO S ∈ { 1 } n × n , it yields the space-invariant kernel κ [ i, j ] =

Original: S ij ◦ W = W 
Revised: S ij ◦ W = W and F ( SXW ) =

Original: and then the kernel κ is space-invariant.
Revised: F ( X ) F ( κ ) .

Original: Accordingly, we can parameterize FGSO S with a complex-valued matrix C d × d which is spaceinvariant and is computationally low costly compared to a varying kernel resulting a parameterized matrix of C n × d × d .
Revised: Accordingly, we can parameterize FGSO S with a complex-valued matrix C d × d which is frequencyinvariant and is computationally low costly compared to a varying kernel resulting a parameterized matrix of C n × d × d .

Original: Furthermore, we can extend Definition 2 to 2D discrete space, i.e., from [ n ] to [ N ] × [ T ] , corresponding to the finite discrete spatial-temporal space of multivariate time series. 
Revised: Furthermore, we can extend Definition 2 to 2D discrete space, i.e., from [ n ] to [ N ] × [ T ] , corresponding to the finite discrete spatial-temporal space of multivariate time series. See Appendix C for more explanations on the frequency-invariant FGSO and the extension to 2D domain.

Original: Remarks .
Revised: Remarks .

Original: Compared with GSO and FNO, space-invariant FGSO has several advantages.
Revised: Compared with FGSO and FNO, frequency-invariant FGSO has several advantages.

Original: Assume a graph with n nodes and the embedding dimension d ( d ≤ n ).
Revised: Assume a graph with n nodes and the embedding dimension d ( d ≤ n ).

Original: 1) Efficiency : the time complexity of space-invariant FGSO is O ( nd log n + nd 2 ) for DFT, IDFT and the matrix multiplication compared with that of O ( n 2 d + nd 2 ) on a GSO. 2) Scale-free parameters : space-invariant FGSO strategically shares O ( d 2 ) parameters for each node and the parameter volume is agnostic to the data scale, while the parameter count of FNO is O ( nd 2 ) .
Revised: 1) Efficiency : the time complexity of frequency-invariant FGSO is O ( nd log n + nd 2 ) for DFT, IDFT and the matrix multiplication compared with that of O ( n 2 d + nd 2 ) on a GSO. 2) Scale-free parameters : frequencyinvariant FGSO strategically shares O ( d 2 ) parameters for each node and the parameter volume is agnostic to the data scale, while the parameter count of FNO is O ( nd 2 ) .

Original: 3. E DGE -V ARYING F OURIER G RAPH N ETWORKS
Revised: 3. E DGE -V ARYING F OURIER G RAPH N ETWORKS

Original: Graph filters as core operations in signal processing are linear transformations expressed as polynomials of the graph shift operator Isufi et al.
Revised: Graph filters as core operations in signal processing are linear transformations expressed as polynomials of the graph shift operator Isufi et al.

Original: ; Mateos et al.
Revised: ; Mateos et al.

Original: and can be used to exactly model graph convolutions and capture multi-order diffusion on graph structures Segarra et al.
Revised: and can be used to exactly model graph convolutions and capture multi-order diffusion on graph structures Segarra et al.

Original: To capture the time-varying counterparts and adopt different weights to weigh the information of different neighbors in each diffusion order, the edge-variant graph filters are defined as follows Isufi et al.
Revised: To capture the time-varying counterparts and adopt different weights to weight the information of different neighbors in each diffusion order, the edge-variant graph filters are defined as follows Isufi et al.

Original: (2021); Segarra et al.
Revised: (2021); Segarra et al.

Original: (2017): given GSO S ∈ R n × n corresponding to a graph with n nodes,
Revised: (2017): given GSO S ∈ R n × n corresponding to a graph with n nodes,

Original: [Equation]
Revised: [Equation]

Original: where S 0 denotes the identity matrix, { S k } Kk =1 is a collection of K edge-weighting GSOs sharing the sparsity pattern of S , and S k ∈ R n × n corresponds to the k -th diffusion step.
Revised: where S 0 denotes the identity matrix, { S k } Kk =1 is a collection of K edge-weighting GSOs sharing the sparsity pattern of S , and S k ∈ R n × n corresponds to the k -th diffusion step.

Original: The edge-variant graph filters are proved effective to yield a highly discriminative model and lay the foundation for the unification of GCNs and GATs Isufi et al.
Revised: The edge-variant graph filters are proved effective to yield a highly discriminative model and lay the foundation for the unification of GCNs and GATs Isufi et al.

Original: We can extend Equation 7 to H EV and reformulate the multi-order graph convolution in a recursive form.
Revised: We can extend Equation 7 to H EV and reformulate the multi-order graph convolution in a recursive form, where we omit the weight W for conciseness.

Original: Proposition 1.
Revised: Proposition 1.

Original: Given a graph input X ∈ R n × d , the K -order graph convolution under the edgevariant graph filter H EV =
Revised: Given a graph input X ∈ R n × d , the K -order graph convolution under the edgevariant graph filter H EV =

Original: (cid:80)
Revised: (cid:80)

Original: Kk =

Revised: Kk =

Original: S k
Revised: S k

Original: with { S k ∈ R n × n } Kk =0 is reformulated as follows:
Revised: with { S k ∈ R n × n } Kk =0 is reformulated as follows:

Original: [Equation]
Revised: [Equation]

Original: where S k ∈ C n ×
Revised: where S k ∈ C n ×

Original: d × d is the k -th FGSO satisfying F ( S k X )
Revised: d × d is the k -th FGSO satisfying F ( S k X )

Original: = F ( X )  S k , S 0 is the identity matrix, and F and F − 1 denote the discrete Fourier transform and its reverse, respectively.
Revised: = F ( X ) × n S k , S 0 is the identity matrix, and F and F − 1 denote the discrete Fourier transform and its reverse, respectively.

Original: Proposition 1 proved in Appendix C.1 states that we can rewrite the multi-order graph convolution corresponding to H EV as a summation of a recursive multiplication of individual FGSO in the Fourier space.
Revised: Proposition 1 proved in Appendix D.1 states that we can rewrite the multi-order graph convolution corresponding to H EV as a summation of a recursive multiplication of individual FGSO in the Fourier space.

Original: Considering our case of the supra-graph G , we can similarly adopt K space-invariant FGSOs and parameterize each FGSO S k with a complex-valued matrix C d × d .
Revised: Corresponding to our case of the supra-graph G , we can similarly adopt K frequency-invariant FGSOs and parameterize each FGSO S k with a complex-valued matrix C d × d .

Original: This saves a large amount of computation costs and results in a concise form:
Revised: This saves a large amount of computation costs and results in a concise form:

Original: [Equation]
Revised: [Equation]

Original: The recursive composition has a nice property that S 0: k = S 0: k − 1 S k , which inspires us to design a complex-valued feed forward network with S k being the complex weights for the k -th layer.
Revised: The recursive composition has a nice property that S 0: k = S 0: k − 1 S k , which inspires us to design a complex-valued feed forward network with S k being the complex weights for the k -th layer.

Original: However, both the FGSO and edge-variant graph filter are linear transformations, limiting the capability of modeling non-linear information diffusion on graphs.
Revised: However, both the FGSO and edge-variant graph filter are linear transformations, limiting the capability of modeling non-linear information diffusion on graphs.

Original: Following by the convention in GNNs, we introduce the non-linear activation and biases to reformulate the k -the layer as follows:
Revised: Following by the convention in GNNs, we introduce the non-linear activation and biases to reformulate the k -the layer as follows:

Original: [Equation]
Revised: [Equation]

Original: with the complex weight S k ∈ C d × d , biases b k ∈ C d and the activation function σ .
Revised: with the complex weight S k ∈ C d × d , biases b k ∈ C d and the activation function σ .

Original: Accordingly, we design an edge-varying Fourier graph network (EV-FGN) in the Fourier space according to Equations 9 and 11, as shown in Fig.
Revised: Accordingly, we design an edge-varying Fourier graph network (EV-FGN) in the Fourier space according to Equations 9 and 11, as shown in Fig.

Original: Accordingly, the K -layer EV-FGN Ψ K is formulated:
Revised: Accordingly, the K -layer EV-FGN Ψ K is formulated:

Original: [Equation]
Revised: [Equation]

Original: where X 0 :
Revised: where X 0 :

Original: = F ( X ) , {
Revised: = F ( X ) , {

Original: S k ∈ C
Revised: S k ∈ C

Original: d × d } Kk =1 and { b k ∈ C
Revised: d × d } Kk =1 and { b k ∈ C

Original: d } Kk =1 are complex-valued parameters.
Revised: d } Kk =1 are complex-valued parameters.

Original: The frequency output Ψ K ( X ) is then transformed via IDFT, followed with feed forward networks (see
Revised: The frequency output Ψ K ( X ) is then transformed via IDFT, followed with feed forward networks (see

Original: Appendix E.4 for more details) to make multi-step forecasting in the time domain.
Revised: Appendix F.4 for more details) to make multi-step forecasting in the time domain.

Original: Remarks .
Revised: Remarks .

Original: EV-FGN efficiently learns edge- and neighbor-dependent weights to compute multi-order graph convolutions on one fly in the Fourier space with scale-free parameter volume.
Revised: EV-FGN efficiently learns edge- and neighbor-dependent weights to compute multi-order graph convolutions on one fly in the Fourier space with scale-free parameter volume.

Original: For K iterations of graph convolutions, GCNs have a general time complexity of O
Revised: For K iterations of graph convolutions, GCNs have a general time complexity of O

Original: ( Kn 2 d + Knd 2 ) , and FNO needs
Revised: ( Kn 2 d + Knd 2 ) , and FNO needs

Original: O ( Knd log n + Knd 2 ) , and EV-FGN achieves a time complexity of O ( nd log n + Knd 2 + Knd ) .
Revised: O ( Knd log n + Knd 2 ) , and EV-FGN achieves a time complexity of O ( nd log n + Knd 2 + Knd ) .

Original: EV-FGN saves time costs from FNO in Fourier transforms and is much more efficient than GCNs, especially in our case with n = NT for a given raw input X ∈ R N × T .
Revised: EV-FGN saves time costs from FNO in Fourier transforms and is much more efficient than GCNs, especially in our case with n = NT for a given raw input X ∈ R N × T .

Original: See Appendix D for more details about the relations and differences between EV-FGN with FNO, adaptive FNO, and GNNs.
Revised: See Appendix E for more details about the relations and differences between EV-FGN with FNO, adaptive FNO, and GNNs.

Original:  E XPERIMENTS 4. S ETUP
Revised:  E XPERIMENTS 4. S ETUP

Original: Datasets .
Revised: Datasets .

Original: We select seven representative datasets from different application scenarios for evaluation, including traffic, energy, web traffic, electrocardiogram, and COVID-19.
Revised: We select seven representative datasets from different application scenarios for evaluation, including traffic, energy, web traffic, electrocardiogram, and COVID-19.

Original: These datasets are summarized in Table 1.
Revised: These datasets are summarized in Table 1.

Original: All datasets are normalized using the min-max normalization.
Revised: All datasets are normalized using the min-max normalization.

Original: Except the
Revised: Except the

Original: COVID-19 dataset, we split the other datasets into training, validation, and test sets with the ratio of 7:2:1 in chronological order.
Revised: COVID-19 dataset, we split the other datasets into training, validation, and test sets with the ratio of 7:2:1 in chronological order.

Original: For the COVID-19 dataset, the ratio is 6:2:2. 
Revised: For the COVID-19 dataset, the ratio is 6:2:2. [Table] Solar [Table]

Revised: Traffic

Original: [Table]
Revised: [Table]

Revised: Electricity

Revised: COVIDMETR-LA

Revised: Samples

Original:  Baselines .
Revised: Variables Granularity 1hour 1day 1hour15min 1day 5min Start time 01/01/ 01/07/ Baselines .

Original: We compare the forecasting performance of our EV-FGN with other representative and
Revised: We compare the forecasting performance of our EV-FGN with other representative and

Original: SOTA models on the seven datasets, including VAR Watson (1993), SFM Zhang et al.
Revised: SOTA models on the seven datasets, including VAR Watson (1993), SFM Zhang et al.

Original: , LSTNet
Revised: , LSTNet

Original: Lai et al.
Revised: Lai et al.

Original: (2018), TCN Bai et al.
Revised: (2018), TCN Bai et al.

Original: (2018), GraphWaveNet Wu et al.
Revised: (2018), GraphWaveNet Wu et al.

Original: , DeepGLO Sen et al.
Revised: , DeepGLO Sen et al.

Original: StemGNN Cao et al.
Revised: StemGNN Cao et al.

Original: MTGNN Wu et al.
Revised: MTGNN Wu et al.

Original: (2020), AGCRN Bai et al.
Revised: (2020), AGCRN Bai et al.

Original: (2020) and Reformer Kitaev et al.
Revised: (2020), Reformer Kitaev et al.

Original: Informer Zhou et al.
Revised: Informer Zhou et al.

Original: (2021), Autoformer Wu et al.
Revised: (2021), Autoformer Wu et al.

Original: FEDformer Zhou et al.
Revised: FEDformer Zhou et al.

Revised: (2022), and CoST Woo et al.

Original: In addition, we compare EV-FGN with SOTA TAMP-S2GCNets Chen et al.
Revised: In addition, we compare EV-FGN with SOTA TAMP-S2GCNets Chen et al.

Original:  DCRNN Li et al.
Revised: , DCRNN Li et al.

Original: and
Revised: and

Original: STGCN Yu et al.
Revised: STGCN Yu et al.

Original: (2018a), which need pre-defined graph structures.
Revised: (2018a), which need pre-defined graph structures.

Original: Experimental Settings .
Revised: Experimental Settings .

Original: All experiments are implemented in Python in Pytorch 1.8 (SFM in Keras) and conducted on one NVIDIA RTX 3080 card.
Revised: All experiments are implemented in Python in Pytorch 1.8 (SFM in Keras) and conducted on one NVIDIA RTX 3080 card.

Original: Our model is trained using RMSProp with a learning rate of 0 . 00001 and MSELoss (Mean Squared Error) as the loss function.
Revised: Our model is trained using RMSProp with a learning rate of 0 . 00001 and MSELoss (Mean Squared Error) as the loss function.

Original: The best parameters for all comparative models are chosen through careful parameter tuning on the validation set.
Revised: The best parameters for all comparative models are chosen through careful parameter tuning on the validation set.

Original: We use
Revised: We use

Original: MAE (Mean Absolute Errors), RMSE (Root Mean Squared Errors), and MAPE (Mean Absolute
Revised: MAE (Mean Absolute Errors), RMSE (Root Mean Squared Errors), and MAPE (Mean Absolute

Original: Percentage Error) to measure the performance.
Revised: Percentage Error) to measure the performance.

Original: More details about the datasets, baselines and experimental settings can be found in Appendix E.
Revised: More details about the datasets, baselines and experimental settings can be found in Appendix F.

Original: 4. R ESULTS
Revised: 4. R ESULTS

Original: We summarize the evaluation results in Table 2, and more results can be found in Appendix F.
Revised: We summarize the evaluation results in Table 2, and more results can be found in Appendix G.

Original: Generally, our model EV-FGN establishes a new state-of-the-art on all datasets.
Revised: Generally, our model EV-FGN establishes a new state-of-the-art on all datasets.

Original: On average, EV-FGN improves 8 .
Revised: On average, EV-FGN improves 8 .

Original: 8% on MAE and 10 .
Revised: 8% on MAE and 10 .

Original: 5% on RMSE compared to the best baseline for all datasets.
Revised: 5% on RMSE compared to the best baseline for all datasets.

Original: Among these baselines, Reformer, Informer, Autoformer and FEDformer are transformer-based models that achieve competitive performance on Electricity and COVID-19 datasets since those models are competent in capturing temporal dependencies.
Revised: Among these baselines, Reformer, Informer, Autoformer and FEDformer are transformer-based models that achieve competitive performance on Electricity and COVID-19 datasets since those models are competent in capturing temporal dependencies.

Original: However, they are defective in capturing the spatial dependencies explicitly.
Revised: However, they are defective in capturing the spatial dependencies explicitly.

Original: GraphWaveNet, MTGNN, StemGNN and AGCRN are GNN-based models that show promising performances on Wiki, Traffic, Solar and ECG datasets due to their high capability in handling spatial dependencies among variables.
Revised: GraphWaveNet, MTGNN, StemGNN and AGCRN are GNN-based models that show promising performances on Wiki, Traffic, Solar and ECG datasets due to their high capability in handling spatial dependencies among variables.

Original: However, they are limited to simultaneously capturing spatial-temporal dependencies.
Revised: However, they are limited to simultaneously capturing spatial-temporal dependencies.

Original: EV-FGN significantly outperforms the baseline models since it learns comprehensive spatial-temporal dependencies simultaneously and attends to time-varying dependencies among variables.
Revised: EV-FGN significantly outperforms the baseline models since it learns comprehensive spatial-temporal dependencies simultaneously and attends to time-varying dependencies among variables.

Original: In Appendix F, we further report the results on four datasets and show the comparison between EV-FGN with those models requiring pre-defined graph structures.
Revised: In Appendix G [Table] we further report the results on four datasets and show the comparison between EV-FGN with those models requiring pre-defined graph structures [Table]

Original: [Table]
Revised: [Table]

Revised: 191.95 SFM Zhang et al.

Revised: 76.08 LSTNet Lai et al.

Revised: 89.04 TCN Bai et al.

Revised: 151.78 DeepGLO Sen et al.

Revised: 75.19 Reformer Kitaev et al.

Revised: 132.78 Informer Zhou et al.

Revised: 155.55 Autoformer Wu

Revised: [Table]

Revised: al. (2021) 0.

Revised: 136.24 FEDformer Zhou et al.

Revised: 134.45 GraphWaveNet Wu et al.

Revised: 100.83 StemGNN Cao et al.

Revised: 141.01 MTGNN Wu et al.

Revised: [Table]

Original: . A NALYSIS
Revised: 4. A NALYSIS

Original: Efficiency Analysis .
Revised: Efficiency Analysis .

Original: We investigate the parameter volumes and training time costs of EV-FGN,
Revised: We investigate the parameter volumes and training time costs of EV-FGN,

Original: StemGNN, AGCRN, GraphWaveNet and MTGNN on two representative datasets Wiki and Traffic.
Revised: StemGNN, AGCRN, GraphWaveNet and MTGNN on two representative datasets (Wiki and Traffic).

Original: We report the parameter volumes and the average time costs of five rounds of experiments in Table 3.
Revised: We report the parameter volumes and the average time costs of five rounds of experiments in Table 3.

Original: In terms of parameter volumes, EV-FGN requires the least volume of parameters among the comparative models.
Revised: In terms of parameter volumes, EV-FGN requires the least volume of parameters among the comparative models.

Original: Specifically, it achieves 32.2% and 9.5% parameter reduction over GraphWaveNet on
Revised: Specifically, it achieves 32.2% and 9.5% parameter reduction over GraphWaveNet on

Original: Traffic and Wiki datasets, respectively.
Revised: Traffic and Wiki datasets, respectively.

Original: This is highly attributed that EV-FGN has shared scale-free parameters for each node.
Revised: This is highly attributed that EV-FGN has shared scale-free parameters for each node.

Original: Regarding the training time, EV-FGN runs much faster than all baseline models, and it shows 5.8% and 23.3% efficiency improvements over the fast baseline GraphWaveNet on Traffic and Wiki datasets, respectively.
Revised: Regarding the training time, EV-FGN runs much faster than all baseline models, and it shows 5.8% and 23.3% efficiency improvements over the fast baseline GraphWaveNet on Traffic and Wiki datasets, respectively.

Original: Besides, the variable count of Wiki dataset is about twice larger than that of Traffic dataset, but EV-FGN shows larger efficiency gaps with the baselines.
Revised: Besides, the variable count of Wiki dataset is about twice larger than that of Traffic dataset, but EV-FGN shows larger efficiency gaps with the baselines.

Original: These results demonstrate that EV-FGN shows high efficiency in computing graph convolutions and is scalable to large datasets with large graphs.
Revised: These results demonstrate that EV-FGN shows high efficiency in computing graph convolutions and is scalable to large datasets with large graphs.

Original: Significantly , the supra-graph in EV-FGN has N × T nodes, which is much larger than the graphs (with N nodes) in the baselines.[Table]
Revised: Significantly , the supra-graph in EV-FGN has N × T nodes, which is much larger than the graphs (with N nodes) in the baselines.

Original: Ablation study .
Revised: Ablation study .

Original: We conduct the ablation study on METR-LA dataset to evaluate the contribution of different components of our model.
Revised: We conduct the ablation study on METR-LA dataset to evaluate the contribution of different components of our model.

Original: The results shown in Table 4 verify the effectiveness of  each component.
Revised: The results shown in Table 4 verify the effectiveness of [Table] each component.

Original: Specifically, w/o Embedding shows the significance of performing embedding to improve model generalization.
Revised: Specifically, w/o Embedding shows the significance of performing embedding to improve model generalization.

Original: w/o Dynamic Filter using the same graph shift operator verifies the effectiveness of applying different graph shift operators in capturing time-varying dependencies.
Revised: w/o Dynamic Filter using the same graph shift operator verifies the effectiveness of applying different graph shift operators in capturing time-varying dependencies.

Original: In addition, w/o Residual represents EV-FGN without the K = 0 layer, while w/o Summation adopts the last order (layer) output X K as the output of EV-FGN.
Revised: In addition, w/o Residual represents EV-FGN without the K = 0 layer, while w/o Summation adopts the last order (layer) output X K as the output of EV-FGN.

Original: These results demonstrate the importance of high-order diffusion and the contribution of multi-order diffusion.
Revised: These results demonstrate the importance of high-order diffusion and the contribution of multi-order diffusion.

Original: More results and analysis of the ablation study are provided in Appendix G.1.
Revised: More results and analysis of the ablation study are provided in Appendix H. [Table] [Table]

Original: [Table]
Revised: [Table]

Original:  V ISUALIZATION
Revised:  V ISUALIZATION

Original: To better investigate the spatial-temporal representation learned by EV-FGN, we conduct more visualization experiments on METR-LA and COVID-19 datasets.
Revised: To better investigate the spatial-temporal representation learned by EV-FGN [Table] we conduct more visualization experiments on METR-LA and COVID-19 datasets.

Original: More details about the way of visualization can be found in Appendix E.5.
Revised: More details about the way of visualization can be found in Appendix F.5.

Original: Visualization of spatial representations learned by EV-FGN .
Revised: Visualization of spatial representations learned by EV-FGN .

Original: We produce the visualization of the generated adjacency matrix according to the learned representation from EV-FGN on the METR-LA dataset.
Revised: We produce the visualization of the generated adjacency matrix according to the learned representation from EV-FGN on the METR-LA dataset.

Original: Specifically, we randomly select 20 detectors and visualize their corresponding adjacency matrix via heat map, as shown in Fig.
Revised: Specifically, we randomly select 20 detectors and visualize their corresponding adjacency matrix via heat map, as shown in Fig.

Original: Correlating the adjacency matrix with the real road map, we observe: 1) the detectors (7, 8, 9, 11, 13, 18, and 19) are very close in physical distance, corresponding to the high values of their correlations with each other in the heat map; 2) the detectors 4, 14 andhave small overall correlation values since they are far from other detectors; 3) however , compared with detectors 14 and 16, the detector 4 has slightly higher correlation values to other detectors e.g., 7, 8, 9, which is attributed that although they are far apart, the detectors 4, 7, 8, 9 are on the same road.
Revised: Correlating the adjacency matrix with the real road map, we observe: 1) the detectors (7, 8, 9, 11, 13, 18, and 19) are very close in physical distance, corresponding to the high values of their correlations with each other in the heat map; 2) the detectors 4, 14 andhave small overall correlation values since they are far from other detectors; 3) however , compared with detectors 14 and 16, the detector 4 has slightly higher correlation values to other detectors e.g., 7, 8, 9, which is attributed that although they are far apart, the detectors 4, 7, 8, 9 are on the same road.

Original: The results verify the effectiveness of EV-FGN in learning highly interpretative correlations.
Revised: The results verify the effectiveness of EV-FGN in learning highly interpretative correlations.

Original: Visualization on four consecutive days .
Revised: Visualization on four consecutive days .

Original: Furthermore, we conduct experiments to visualize the adjacency matrix of 10 randomly-selected counties on four consecutive days on the COVID-19 dataset.
Revised: Furthermore, we conduct experiments to visualize the adjacency matrix of 10 randomly-selected counties on four consecutive days on the COVID-19 dataset.

Original: The visualization results via heat map are shown in Fig.
Revised: The visualization results via heat map are shown in Fig.

Original: From the figure, we can observe that
Revised: From the figure, we can observe that

Original: EV-FGN learns clear spatial patterns that show continuous evolution in the time dimension.
Revised: EV-FGN learns clear spatial patterns that show continuous evolution in the time dimension.

Original: This is because EV-FGN highlights the edge-varying design and attends to the time-varying variability of the supra-graph.
Revised: This is because EV-FGN highlights the edge-varying design and attends to the time-varying variability of the supra-graph.

Original: These results verify that our model enjoys the feasibility of exploiting the time-varying dependencies among variables.
Revised: These results verify that our model enjoys the feasibility of exploiting the time-varying dependencies among variables.

Original: Visualization of EV-FGN diffusion process .
Revised: Visualization of EV-FGN diffusion process .

Original: To understand how EV-FGN works, we analyze the frequency input of each layer.
Revised: To understand how EV-FGN works, we analyze the frequency input of each layer.

Original: We choose 10 counties from COVID-19 dataset and visualize their adjacency matrices at two different timestamps, as shown in Fig. 4.
Revised: We choose 10 counties from COVID-19 dataset and visualize their adjacency matrices at two different timestamps, as shown in Fig. 4.

Original: From left to right, the results correspond to X 0 , · · · , X 3 respectively.
Revised: From left to right, the results correspond to X 0 , · · · , X 3 respectively.

Original: From the top, we can find that as the number of layers increases, some correlation values are reduced, indicating that some correlations are filtered out.
Revised: From the top, we can find that as the number of layers increases, some correlation values are reduced, indicating that some correlations are filtered out.

Original: In contrast, the bottom case illustrates some correlations are enhanced as the number of layers increases.
Revised: In contrast, the bottom case illustrates some correlations are enhanced as the number of layers increases.

Original: These results show that EV-FGN can adaptively and effectively capture important patterns while removing noises to a learn discriminative model.
Revised: These results show that EV-FGN can adaptively and effectively capture important patterns while removing noises to a learn discriminative model.

Original: More visualizations are provided in Appendix G.2.
Revised: More visualizations are provided in Appendix H.2.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we define a Fourier graph shift operator (FGSO) and construct the efficient edge-varying
Revised: In this paper, we define a Fourier graph shift operator (FGSO) and construct the efficient edge-varying

Original: Fourier graph networks (EV-FGN) for MTS forecasting.
Revised: Fourier graph networks (EV-FGN) for MTS forecasting.

Original: EV-FGN is adopted to simultaneously capture high-resolution spatial-temporal dependencies and account for time-varying variable dependencies.
Revised: EV-FGN is adopted to simultaneously capture high-resolution spatial-temporal dependencies and account for time-varying variable dependencies.

Original: This study makes the first attempt to design a complex-valued feed-forward network in the Fourier space for efficiently computing multi-layer graph convolutions.
Revised: This study makes the first attempt to design a complex-valued feed-forward network in the Fourier space for efficiently computing multi-layer graph convolutions.

Original: Extensive experiments demonstrate that EV-FGN achieves state-of-the-art performances with higher efficiency and fewer parameters and shows high interpretability in graph representation learning.
Revised: Extensive experiments demonstrate that EV-FGN achieves state-of-the-art performances with higher efficiency and fewer parameters and shows high interpretability in graph representation learning.

Original: This study sheds light on efficiently calculating graph operations in Fourier space by learning a Fourier graph shift operator.
Revised: This study sheds light on efficiently calculating graph operations in Fourier space by learning a Fourier graph shift operator.

Original: A N OTATION
Revised: A N OTATION

Original: [Table]
Revised: [Table]

Original: B C ONVOLUTION T HEOREM
Revised: B C ONVOLUTION T HEOREM

Original: The convolution theorem Katznelson (1970) is one of the most important property of Fourier transform.
Revised: The convolution theorem Katznelson (1970) is one of the most important property of Fourier transform.

Original: It states the Fourier transform of a convolution of two signals equals the pointwise product of their
Revised: It states the Fourier transform of a convolution of two signals equals the pointwise product of their

Original: Fourier transforms in the frequency domain.
Revised: Fourier transforms in the frequency domain.

Original: Given a signal x [ n ] and a filter h [ n ] , the convolution theorem can be defined as follows:
Revised: Given a signal x [ n ] and a filter h [ n ] [Table] the convolution theorem can be defined as follows:

Original: [Equation]
Revised: [Equation]

Original: (13) where ( x ∗ h )
Revised: (13) where ( x ∗ h )

Original: [ n ] = (cid:80) N − 1 m =

Revised: [ n ] = (cid:80) N − 1 m =

Original: h [ m ] x
Revised: h [ m ] x

Original: [( n − m ) N ] denotes the convolution of x and h , ( n − m ) N denotes ( n − m ) modulo N
Revised: [( n − m ) N ] denotes the convolution of x and h , ( n − m ) N denotes ( n − m ) modulo N

Original: [Table]
Revised: [Table]

Original: and F ( x ) and F ( h ) denote discrete Fourier transform of x [ n ] and h [ n ] , respectively.
Revised: and F ( x ) and F ( h ) denote discrete Fourier transform of x [ n ] and h [ n ] , respectively.

Original: C P ROOFS
Revised: C E XPLANATIONS

Original: C.
Revised: C.

Revised: T

Revised: HE INTERPRETATION OF F REQUENCY - INVARIANT FGSO

Revised: In addition to reducing parameter volumes and saving computation costs, the frequency-invariant parameterized FGSO is empirically proved effective to improve model generalization.

Revised: As we mentioned above, we perform EV-FGN over the input embeddings and adopt the frequency-invariant

Original: P ROOF OF
Revised: FGSOs. Relatively to directly adopting the frequency-variant FGSOs ( C n × d × d ), we subtly “factorize” the frequency-variant parameterization to the time domain (i.e., the embedding matrix Φ ∈ R n × d ) and the frequency domain (i.e., frequency-invariant FGSO C d × d ). In the time domain, we embed the raw MTS inputs to improve the model learning capability, while we learn the same transformation (FGSO) for all N ∗ T frequency points in the frequency domain (similar to CNN with shared-weight convolution kernels or filters that slide along input features). Note that the frequency spectrum in the frequency domain has a global view of which each frequency point attends to all variables or timestamps.

Revised: This treatment in EV-FGN guarantees the model capacity and is empirically proved superior over the treatment without embeddings and with frequency-variant FGSO (please refer to the Ablation study for detailed results).

Revised: Although the frequency-variant parameterization may be more powerful and flexible than the frequency-invariant one, it introduces more parameters in the frequency domain, especially for multi-layer EV-FGN, and may not obtain superior performance due to inadequate training or overfitting.

Revised: C. E XPLANATION TO THE EXTENSION OF D EFINITION 2 TO 2D DOMAIN

Revised: Recall Eqs.

Revised: 4 and 5,

Revised: [Equation]

Revised: When we extend the equations to 2D domain, i.e., from [ n ] to [ N ] × [ T ] , it means performing a kernel summation/graph convolution over the discrete spatial-temporal space corresponding to all nodes in the supra-graph.

Revised: Obviously, these computations of the kernel summation can be easily extended to 2D domain.

Revised: Similarly, according to the convolution theorem, we can obtain a 2D-version of Eq. 6:

Revised: Eq. 6 (graph convolution): O ( X )( i ) =

Revised: F − 1 ( F ( X ) F ( κ ))

Revised: ( i ) ∀ i ∈

Revised: [ N ] × [ T ] with F and F − 1 denote the 2D discrete Fourier transform and its inverse, respectively.

Revised: Accordingly, when extending Definition 2 to 2D domain, F denotes the 2D discrete Fourier transform.

Revised: Therefore, given input embeddings X ∈ R N × T × d , we perform 2D discrete Fourier transform on each discrete

Revised: N × T spatial-temporal plane of the embeddings to obtain the frequency spectrum, and then feed the frequency input into K-layer EV-FGN followed by two-layer (real-valued) feed-forward networks to generate multi-step forecasting (see Section 3.1 for more details).

Revised: Note that we adopt the frequencyinvariant FGSO ( d × d ) in the EV-FGN where the feed-forward computations act on the embedding dimension, i.e., d .

Revised: In addition, when extending Definition 2 to 2D domain, the frequency-invariant

Revised: FGSO is invariant to both frequency components derived from the spatial dimension ( N ) and time dimension ( T ) respectively.

Original:  P ROPOSITION
Revised: C. I NTERPRETATION OF OMITTING THE WEIGHT MATRIX W IN P ROPOSITION

Revised: This treatment of omitting the weight matrix W in Eq. 9 is feasible since the weight matrix W can be absorbed in the embedding input, precisely the embedding matrix parameters. Note that we feed the input embeddings into EV-FGN (refer to Fig.

Revised: 1 and Section 3.1).

Revised: In addition, this treatment will not reduce the capability of EV-FGN intuitively since we adopt edge-varying filters in EV-FGN.

Revised: Note that traditional GCNs adopt different weight matrices (regarding the model capability) but the same GSO (e.g., adjacency and Laplacian matrices regarding the given graph structure) in different diffusion orders.

Revised: Differently, there is no pre-given graph structure in MTS forecasting scenarios, therefore we adopt the edge-varying filters, i.e., varying GSOs in EV-FGN, which does not reduce the model capability and achieves desirable performance empirically.

Revised: D P ROOFS D. P ROOF OF P ROPOSITION

Original: The proof aims to expand the graph convolution corresponding to H EV using a set of FGSOs in the
Revised: The proof aims to expand the graph convolution corresponding to H EV using a set of FGSOs in the

Original: Fourier space.
Revised: Fourier space.

Original: According to the concise form of Equation 7, i.e.,
Revised: According to the concise form of Equation 7, i.e.,

Original: [Equation]
Revised: [Equation]

Original: where F denotes the discrete Fourier transform (DFT) and we omit the weight matrix W for convenience (we can treat W as an identity matrix), it yields:
Revised: where F denotes the discrete Fourier transform (DFT) and we omit the weight matrix W for convenience (we can treat W as an identity matrix), it yields:

Original: [Equation]
Revised: [Equation]

Original: where { S i } Ki =0 is a set of GSOs, and {S i } Ki =0 is a set of FGSOs corresponding to { S i } Ki =0 individually.
Revised: where { S i } Ki =0 is a set of GSOs, and {S i } Ki =0 is a set of FGSOs corresponding to { S i } Ki =0 individually.

Original: Thus, the edge-varying graph filter H EV can be rewritten as
Revised: Thus, the edge-varying graph filter H EV can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: Accordingly, we have
Revised: Accordingly, we have

Original: [Equation]
Revised: [Equation]

Original: Proved.
Revised: Proved.

Original: D C OMPARED WITH O THER N ETWORKS D. GNN
Revised: E C OMPARED WITH O THER N ETWORKS E. GNN

Original: Graph Convolutional Networks .
Revised: Graph Convolutional Networks .

Original: Graph convolutional networks (GCNs) depend on the Laplacian eigenbasis to perform the multi-order graph convolutions over a given graph structure.
Revised: Graph convolutional networks (GCNs) depend on the Laplacian eigenbasis to perform the multi-order graph convolutions over a given graph structure.

Original: Compared with GCNs, EV-FGN as an efficient alternative to multi-order graph convolutions has three main differences: 1) No eigendecompositions or similar costly matrix operations are required.
Revised: Compared with GCNs, EV-FGN as an efficient alternative to multi-order graph convolutions has three main differences: 1) No eigendecompositions or similar costly matrix operations are required.

Original: EV-FGN transforms the input into Fourier domain by discrete Fourier transform (DFT) instead of graph Fourier transform (GFT); 2) Explicitly assigning various importance to nodes of the same neighborhood with different diffusion steps.
Revised: EV-FGN transforms the input into Fourier domain by discrete Fourier transform (DFT) instead of graph Fourier transform (GFT); 2) Explicitly assigning various importance to nodes of the same neighborhood with different diffusion steps.

Original: EV-FGN adopts different Fourier graph shift operators S in different diffusion steps corresponding to the time-varying dependencies among nodes; 3) EV-FGN is invariant to the discretization N , T .
Revised: EV-FGN adopts different Fourier graph shift operators S in different diffusion steps corresponding to the time-varying dependencies among nodes; 3) EV-FGN is invariant to the discretization N , T .

Original: It parameterizes the graph convolution via Fourier bases invariant graph structure and graph scale.
Revised: It parameterizes the graph convolution via Fourier bases invariant graph structure and graph scale.

Original: Graph Attention Networks .
Revised: Graph Attention Networks .

Original: Graph attention networks (GATs) are non-spectral attention-based graph neural networks.
Revised: Graph attention networks (GATs) are non-spectral attention-based graph neural networks.

Original: GATs use node representations to calculate the attention weights (i.e., edge weights) varying with different graph attention layers.
Revised: GATs use node representations to calculate the attention weights (i.e., edge weights) varying with different graph attention layers.

Original: Accordingly, both GATs and EV-FGN do not depend on eigendecompositions and adopt varying edge weights with different diffusion steps (layers).
Revised: Accordingly, both GATs and EV-FGN do not depend on eigendecompositions and adopt varying edge weights with different diffusion steps (layers).

Original: However, EV-FGN can efficiently perform graph convolutions in the Fourier space.
Revised: However, EV-FGN can efficiently perform graph convolutions in the Fourier space.

Original: For a complete graph, the time complexity of the attention calculation of K layers is proportional to Knwhere n is the number of nodes, while a K -layer EV-FGN infers the graph structure in Fourier space with the time complexity proportional to n log n .
Revised: For a complete graph, the time complexity of the attention calculation of K layers is proportional to Knwhere n is the number of nodes, while a K -layer EV-FGN infers the graph structure in Fourier space with the time complexity proportional to n log n .

Original: In addition, compared with GATs that implicitly achieve edge-varying weights with different layers, EV-FGN adopts different FGSOs in different diffusion steps explicitly.
Revised: In addition, compared with GATs that implicitly achieve edge-varying weights with different layers, EV-FGN adopts different FGSOs in different diffusion steps explicitly.

Original: D. FNO
Revised: E. FNO

Original: Inspired by Fourier neural operator (FNO) Li et al.
Revised: Inspired by Fourier neural operator (FNO) Li et al.

Original: (2021) that computes the global convolutions in the Fourier space, we elaborately design EV-FGN to compute the multi-order graph convolutions in the Fourier space.
Revised: (2021) that computes the global convolutions in the Fourier space, we elaborately design EV-FGN to compute the multi-order graph convolutions in the Fourier space.

Original: Both FNO and EV-FGN replace the time-consuming graph/global convolutions in time domain with the efficient spectral convolution in the Fourier space according to the convolution theorem.
Revised: Both FNO and EV-FGN replace the time-consuming graph/global convolutions in time domain with the efficient spectral convolution in the Fourier space according to the convolution theorem.

Original: However, FNO and EV-FGN are quite different in the network architecture.
Revised: However, FNO and EV-FGN are quite different in the network architecture.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5, FNO consists of a stack of Fourier layers where each Fourier layer serially performs 1) DFT to obtain the spectrum of the input, 2) then the spectral convolution in the Fourier space, 3) and finally IDFT to transform the output to the time domain.
Revised: 5, FNO consists of a stack of Fourier layers where each Fourier layer serially performs 1) DFT to obtain the spectrum of the input, 2) then the spectral convolution in the Fourier space, 3) and finally IDFT to transform the output to the time domain.

Original: Accordingly, FNO needs K pairs of DFT and IDFT for K
Revised: Accordingly, FNO needs K pairs of DFT and IDFT for K

Original: Fourier layers in FNO.
Revised: Fourier layers in FNO.

Original: In contrast, EV-FGN with just a pair of DFT and IDFT performs multi-order (multi-layer) graph convolutions on one fly via stacking multiple FGSOs in Fourier space, as shown in Fig.
Revised: In contrast, EV-FGN with just a pair of DFT and IDFT performs multi-order (multi-layer) graph convolutions on one fly via stacking multiple FGSOs in Fourier space, as shown in Fig.

Original: In addition, adaptive Fourier neural operator (AFNO) Guibas et al.
Revised: In addition, adaptive Fourier neural operator (AFNO) Guibas et al.

Original: (2022) is a variant of FNO.
Revised: (2022) is a variant of FNO.

Original: It is a neural unit and used in a plug-and-play fashion as an alternative to self-attention to reduce the quadratic complexity.
Revised: It is a neural unit and used in a plug-and-play fashion as an alternative to self-attention to reduce the quadratic complexity.

Original: Similarly, it requires a pair of DFT and IDFT to accomplish the self-attention computation.
Revised: Similarly, it requires a pair of DFT and IDFT to accomplish the self-attention computation.

Original: In contrast, our proposed EV-FGN is a neural network with a set of FGSOs in a well-designed connection.
Revised: In contrast, our proposed EV-FGN is a neural network with a set of FGSOs in a well-designed connection.

Original: E E XPERIMENT D ETAILS E. D ATASETS
Revised: F E XPERIMENT D ETAILS F. D ATASETS

Original: Solar 1 : This dataset is about the solar power collected by National Renewable Energy Laboratory.
Revised: Solar 1 : This dataset is about the solar power collected by National Renewable Energy Laboratory.

Original: We choose the power plant data points in Florida as the data set which contains 593 points.
Revised: We choose the power plant data points in Florida as the data set which contains 593 points.

Original: The data is collected from 2006/01/01 to 2016/12/31 with the sampling interval of every 1 hour.
Revised: The data is collected from 2006/01/01 to 2016/12/31 with the sampling interval of every 1 hour.

Original: Wiki 2 : This dataset contains a number of daily views of different Wikipedia articles and is collected from 2015/7/1 to 2016/12/31.
Revised: Wiki 2 : This dataset contains a number of daily views of different Wikipedia articles and is collected from 2015/7/1 to 2016/12/31.

Original: It consists of approximately 145 k time series and we randomly choose 2 k from them as our experimental data set.
Revised: It consists of approximately 145 k time series and we randomly choose 2 k from them as our experimental data set.

Original: Traffic 3 : This dataset contains hourly traffic data from 963 San Francisco freeway car lanes.
Revised: Traffic 3 : This dataset contains hourly traffic data from 963 San Francisco freeway car lanes.

Original: The traffic data are collected since 2015/01/01 with the sampling interval of every 1 hour.
Revised: The traffic data are collected since 2015/01/01 with the sampling interval of every 1 hour.

Original: ECG 4 : This dataset is about Electrocardiogram(ECG) from the UCR time-series classification archive
Revised: ECG 4 : This dataset is about Electrocardiogram(ECG) from the UCR time-series classification archive

Original: Dau et al.
Revised: Dau et al.

Original: It contains 140 nodes and each node has a length of 5000.
Revised: It contains 140 nodes and each node has a length of 5000.

Original: Electricity 5 : This dataset contains electricity consumption of 370 clients and is collected since 2011/01/01.
Revised: Electricity 5 : This dataset contains electricity consumption of 370 clients and is collected since 2011/01/01.

Original: Data sampling interval is every 15 minutes.
Revised: Data sampling interval is every 15 minutes.

Original: COVID-19 6 :
Revised: COVID-19 6 :

Original: This dataset is about COVID-19 hospitalization in the U.S. states of California (CA) from 01/02/2020 to 31/12/2020 provided by the Johns Hopkins University with the sampling interval of every one day.
Revised: This dataset is about COVID-19 hospitalization in the U.S. states of California (CA) from 01/02/2020 to 31/12/2020 provided by the Johns Hopkins University with the sampling interval of every one day.

Original: METR-LA 7 : This dataset contains traffic information collected from loop detectors in the highway of Los Angeles County.
Revised: METR- : This dataset contains traffic information collected from loop detectors in the highway of Los Angeles County.

Original: It contains 207 sensors which is from 01/03/2012 to 30/06/2012 and the data sampling interval is every 5 minutes.
Revised: It contains 207 sensors which is from 01/03/2012 to 30/06/2012 and the data sampling interval is every 5 minutes

Original: E. B ASELINES
Revised: F. B ASELINES

Original: VAR Watson (1993): VAR is a classic linear autoregressive model.
Revised: VAR Watson (1993): VAR is a classic linear autoregressive model.

Original: We use Statsmodels library ( https://www.statsmodels.org ) which is a python package that provides statistical computations to realize the VAR.
Revised: We use Statsmodels library ( https://www.statsmodels.org ) which is a python package that provides statistical computations to realize the VAR.

Original: DeepGLO Sen et al.
Revised: DeepGLO Sen et al.

Original: (2019): DeepGLO models the relationships among variables by matrix factorization and employs a temporal convolution neural network to introduce non-linear relationships.
Revised: (2019): DeepGLO models the relationships among variables by matrix factorization and employs a temporal convolution neural network to introduce non-linear relationships.

Original: We download the source code from: https://github.com/rajatsen91/deepglo .
Revised: We download the source code from: https://github.com/rajatsen91/deepglo .

Original: We use the default setting as our experimental settings for wiki, electricity and traffic datasets.
Revised: We use the default setting as our experimental settings for wiki, electricity and traffic datasets.

Original: For covid datasets, the vertical and horizontal batch size is set to 64, the rank of global model is set to 64, the number of channels is set to [32, 32, 32, 1], and the period is set to 7.
Revised: For covid datasets, the vertical and horizontal batch size is set to 64, the rank of global model is set to 64, the number of channels is set to [32, 32, 32, 1], and the period is set to 7.

Original: LSTNet Lai et al.
Revised: LSTNet Lai et al.

Original: LSTNet uses a CNN to capture inter-variable relationships and a RNN to discover long-term patterns.
Revised: LSTNet uses a CNN to capture inter-variable relationships and a RNN to discover long-term patterns.

Original: We download the source code from: https://github.com/ laiguokun/LSTNet .
Revised: We download the source code from: https://github.com/ laiguokun/LSTNet .

Original: In our experiment, we use the default settings where the CNN hidden units is 100, the kernel size of the CNN layers is 4, the dropout is 0.2, the RNN hidden units is 100, the number of RNN hidden layers is 1, the learning rate is 0.001 and the optimizer is Adam.
Revised: In our experiment, we use the default settings where the CNN hidden units is 100, the kernel size of the CNN layers is 4, the dropout is 0.2, the RNN hidden units is 100, the number of RNN hidden layers is 1, the learning rate is 0.001 and the optimizer is Adam.

Original: TCN Bai et al.
Revised: TCN Bai et al.

Original: (2018): TCN is a causal convolution model for regression prediction.
Revised: (2018): TCN is a causal convolution model for regression prediction.

Original: We download the source code from: https://github.com/locuslab/TCN .
Revised: We download the source code from: https://github.com/locuslab/TCN .

Original: We utilize the same configuration as the polyphonic music task exampled in the open source code where the dropout is 0.25, the kernel size is 5, the hidden units is 150, the number of levels is 4 and the optimizer is Adam.
Revised: We utilize the same configuration as the polyphonic music task exampled in the open source code where the dropout is 0.25, the kernel size is 5, the hidden units is 150, the number of levels is 4 and the optimizer is Adam.

Original: Reformer Kitaev et al.
Revised: Reformer Kitaev et al.

Original: Reformer combines the modeling capacity of a Transformer with an architecture that can be executed efficiently on long sequences and with small memory use.
Revised: Reformer combines the modeling capacity of a Transformer with an architecture that can be executed efficiently on long sequences and with small memory use.

Original: We download the source code from: https://github.com/thuml/Autoformer .
Revised: We download the source code from: https://github.com/thuml/Autoformer .

Original: We use the recommended settings as the experimental settings.
Revised: We use the recommended settings as the experimental settings.

Original: Informer Zhou et al.
Revised: Informer Zhou et al.

Original: Informer leverages an efficient self-attention mechanism to encode the dependencies among variables.
Revised: Informer leverages an efficient self-attention mechanism to encode the dependencies among variables.

Original: We download the source code from: https://github.com/ zhouhaoyi/Informer2020 .
Revised: We download the source code from: https://github.com/ zhouhaoyi/Informer2020 .

Original: We use the default settings as our experimental settings where the dropout is 0.05, the number of encoder layers is 2, the number of decoder layers is 1, the learning rate is 0.0001, and the optimizer is Adam.
Revised: We use the default settings as our experimental settings where the dropout is 0.05, the number of encoder layers is 2, the number of decoder layers is 1, the learning rate is 0.0001, and the optimizer is Adam.

Original: Autoformer Wu et al.
Revised: Autoformer Wu et al.

Original: Autoformer proposes a decomposition architecture by embedding the series decomposition block as an inner operator, which can progressively aggregate the long-term trend part from intermediate prediction.
Revised: Autoformer proposes a decomposition architecture by embedding the series decomposition block as an inner operator, which can progressively aggregate the long-term trend part from intermediate prediction.

Original: We download the source code from: https://github. com/thuml/Autoformer .
Revised: We download the source code from: https://github. com/thuml/Autoformer .

Original: We use the recommended settings as our experimental settings withencoder layers and 1 decoder layer.
Revised: We use the recommended settings as our experimental settings withencoder layers and 1 decoder layer.

Original: FEDformer Zhou et al.
Revised: FEDformer Zhou et al.

Original: (2022): FEDformer proposes an attention mechanism with low-rank approximation in frequency and a mixture of experts decomposition to control the distribution shifting.
Revised: (2022): FEDformer proposes an attention mechanism with low-rank approximation in frequency and a mixture of experts decomposition to control the distribution shifting.

Original: We download the source code from: https://github.com/MAZiqing/FEDformer .
Revised: We download the source code from: https://github.com/MAZiqing/FEDformer .

Original: We use
Revised: We use

Original: FEB-f as the Frequency Enhanced Block and select the random mode with 64 as the experimental mode.
Revised: FEB-f as the Frequency Enhanced Block and select the random mode with 64 as the experimental mode.

Original: SFM Zhang et al.
Revised: SFM Zhang et al.

Original: On the basis of the LSTM model, SFM introduces a series of different frequency components in the cell states.
Revised: On the basis of the LSTM model, SFM introduces a series of different frequency components in the cell states.

Original: We download the source code from: https://github. com/z331565360/State-Frequency-Memory-stock-prediction .
Revised: We download the source code from: https://github. com/z331565360/State-Frequency-Memory-stock-prediction .

Original: We use the default settings as the authors recommended where the learning rate is 0.01, the frequency dimension is 10, the hidden dimension is 10 and the optimizer is RMSProp.
Revised: We use the default settings as the authors recommended where the learning rate is 0.01, the frequency dimension is 10, the hidden dimension is 10 and the optimizer is RMSProp.

Original: StemGNN Cao et al.
Revised: StemGNN Cao et al.

Original: (2020): StemGNN leverages GFT and DFT to capture dependencies among variables in the frequency domain.
Revised: (2020): StemGNN leverages GFT and DFT to capture dependencies among variables in the frequency domain.

Original: We download the source code from: https://github.com/ microsoft/StemGNN .
Revised: We download the source code from: https://github.com/ microsoft/StemGNN .

Original: We use the default setting of stemGNN as our experiment setting where the optimizer is RMSProp, the learning rate is 0.0001, the stacked layers is 5, and the dropout rate is 0.5.
Revised: We use the default setting of stemGNN as our experiment setting where the optimizer is RMSProp, the learning rate is 0.0001, the stacked layers is 5, and the dropout rate is 0.5.

Original: MTGNN Wu et al.
Revised: MTGNN Wu et al.

Original: (2020): MTGNN proposes an effective method to exploit the inherent dependency relationships among multiple time series.
Revised: (2020): MTGNN proposes an effective method to exploit the inherent dependency relationships among multiple time series.

Original: We download the source code from: https://github. com/nnzhan/MTGNN .
Revised: We download the source code from: https://github. com/nnzhan/MTGNN .

Original: Because the experimental datasets have no static features, we set the parameter load_static_feature to false.
Revised: Because the experimental datasets have no static features, we set the parameter load_static_feature to false.

Original: We construct the graph by the adaptive adjacency matrix and add the graph convolution layer.
Revised: We construct the graph by the adaptive adjacency matrix and add the graph convolution layer.

Original: Regarding other parameters, we adopt the default settings.
Revised: Regarding other parameters, we adopt the default settings.

Original: GraphWaveNet Wu et al.
Revised: GraphWaveNet Wu et al.

Original: (2019): GraphWaveNet introduces an adaptive dependency matrix learning to capture the hidden spatial dependency.
Revised: (2019): GraphWaveNet introduces an adaptive dependency matrix learning to capture the hidden spatial dependency.

Original: We download the source code from: https://github. com/nnzhan/Graph-WaveNet .
Revised: We download the source code from: https://github. com/nnzhan/Graph-WaveNet .

Original: Since our datasets have no prior defined graph structures, we use only adaptive adjacent matrix.
Revised: Since our datasets have no prior defined graph structures, we use only adaptive adjacent matrix.

Original: We add a graph convolution layer and randomly initialize the adjacent matrix.
Revised: We add a graph convolution layer and randomly initialize the adjacent matrix.

Original: We adopt the default setting as our experimental settings where the learning rate is 0.001, the dropout is 0.3, the number of epoch is 50, and the optimizer is Adam.
Revised: We adopt the default setting as our experimental settings where the learning rate is 0.001, the dropout is 0.3, the number of epoch is 50, and the optimizer is Adam.

Original: AGCRN Bai et al.
Revised: AGCRN Bai et al.

Original: (2020): AGCRN proposes a data-adaptive graph generation module for discovering spatial correlations from data.
Revised: (2020): AGCRN proposes a data-adaptive graph generation module for discovering spatial correlations from data.

Original: We download the source code from: https://github.com/
Revised: We download the source code from: https://github.com/

Original: LeiBAI/AGCRN .
Revised: LeiBAI/AGCRN .

Original: We use the default settings as our experimental settings where the embedding dimension is 10, learning rate is 0.003, and the optimizer is Adam.
Revised: We use the default settings as our experimental settings where the embedding dimension is 10, learning rate is 0.003, and the optimizer is Adam.

Original: TAMP-S2GCNets Chen et al.
Revised: TAMP-S2GCNets Chen et al.

Original: (2022): TAMP-S2GCNets explores the utility of MP to enhance knowledge representation mechanisms within the time-aware DL paradigm.
Revised: (2022): TAMP-S2GCNets explores the utility of MP to enhance knowledge representation mechanisms within the time-aware DL paradigm.

Original: We download the source code from: https://www.dropbox.com/sh/n0ajd5l0tdeyb80/AABGn-ejfV1YtRwjf_
Revised: We download the source code from: https://www.dropbox.com/sh/n0ajd5l0tdeyb80/AABGn-ejfV1YtRwjf_

Original: L0AOsNa?dl=0 .
Revised: L0AOsNa?dl=0 .

Original: TAMP-S2GCNets requires predefined graph topology and we use the California State topology provided by the source code as input.
Revised: TAMP-S2GCNets requires predefined graph topology and we use the California State topology provided by the source code as input.

Original: We adopt the default settings as our experimental settings on COVID-19.
Revised: We adopt the default settings as our experimental settings on COVID-19.

Original: DCRNN Li et al.
Revised: DCRNN Li et al.

Original: (2018): DCRNN uses bidirectional graph random walk to model spatial dependency and recurrent neural network to capture the temporal dynamics.
Revised: (2018): DCRNN uses bidirectional graph random walk to model spatial dependency and recurrent neural network to capture the temporal dynamics.

Original: We download the source code from : https://github.com/liyaguang/DCRNN .
Revised: We download the source code from : https://github.com/liyaguang/DCRNN .

Original: We use the default settings as our experimental settings with the batch size is 64, the learning rate is 0 .
Revised: We use the default settings as our experimental settings with the batch size is 64, the learning rate is 0 .

Original: 01 , the input dimension is 2 and the optimizer is Adam.
Revised: 01 , the input dimension is 2 and the optimizer is Adam.

Original: DCRNN repuires a pre-defined graph structures and we use the adjacency matrix as the pre-defined structures provided by METR-LA dataset.
Revised: DCRNN repuires a pre-defined graph structures and we use the adjacency matrix as the pre-defined structures provided by METR-LA dataset.

Original: STGCN Yu et al.
Revised: STGCN Yu et al.

Original: (2018a): STGCN integrates graph convolution and gated temporal convolution through spatial-temporal convolutional blocks.
Revised: (2018a): STGCN integrates graph convolution and gated temporal convolution through spatial-temporal convolutional blocks.

Original: We download the source code from: https: //github.com/
Revised: We download the source code from: https: //github.com/

Original: VeritasYin/STGCN_IJCAI-18 .
Revised: VeritasYin/STGCN_IJCAI-18 .

Original: We use the default settings as our experimental settings where the batch size is 50, the learning rate is 0 .
Revised: We use the default settings as our experimental settings where the batch size is 50, the learning rate is 0 .

Original: 001 and the optimizer is Adam.
Revised: 001 and the optimizer is Adam.

Original: STGCN requires a pre-defined graph structures and we leverage the adjacency matrix as the predefined structures provided by METR-LA dataset. 
Revised: STGCN requires a pre-defined graph structures and we leverage the adjacency matrix as the predefined structures provided by METR-LA dataset. CoST Woo et al.

Revised: (2022): CoST separates the representation learning and downstream forecasting task and proposes a contrastive learning framework that learns disentangled season-trend representations for time series forecasting tasks.

Revised: We download the source code from: https://github.com/ salesforce/CoST .

Revised: We set the representation dimension to 320, a learning rate to 0.001 and the batch size to 32.

Revised: Inputs are min-max normalization, we perform a 70/20/10 train/validation/test split for METR-LA dataset and 60/20/20 for COVID-19 dataset.

Original: E. E VALUATION M ETRICS
Revised: F. E VALUATION M ETRICS

Original: We use MAE (Mean Absolute Error), RMSE (Root Mean Square Error), and MAPE (Mean Absolute
Revised: We use MAE (Mean Absolute Error), RMSE (Root Mean Square Error), and MAPE (Mean Absolute

Original: Percentage Error)
Revised: Percentage Error)

Original: as the evaluation metrics in the experiments.
Revised: as the evaluation metrics in the experiments.

Original: Specifically, given the groudtruth X t +1: t + τ ∈ R N × τ and the predictions ˆ X t +1: t + τ ∈ R N × τ for future τ steps at timestamp t , the metrics are defined as follows: 
Revised: Specifically, given the groudtruth X t +1: t + τ ∈ R N × τ and the predictions ˆ X t +1: t + τ ∈ R N × τ for future τ steps at timestamp t , the metrics are defined as follows: [Equation]

Original: [Equation]
Revised: [Equation]

Original: with x ij ∈ X t +1:
Revised: with x ij ∈ X t +

Original: t +τ and ˆ x ij ∈
Revised: +

Original: ˆ  Xt +1: t + τ .
Revised: ˆ x ij ∈ ˆ X +1: t + τ .

Original: E. E XPERIMENTAL S ETTINGS
Revised: F. E XPERIMENTAL S ETTINGS

Original: We summarize the implementation details of the proposed EV-FGN as follows.
Revised: We summarize the implementation details of the proposed EV-FGN as follows.

Original: Note that the details of the baselines are introduced in their corresponding descriptions (see Section E.2).
Revised: Note that the details of the baselines are introduced in their corresponding descriptions (see Section F.2).

Original: Network details.
Revised: Network details.

Original: The fully connected feed-forward network (FFN) consists of three linear transformations with LeakyReLU activations in between.
Revised: The fully connected feed-forward network (FFN) consists of three linear transformations with LeakyReLU activations in between.

Original: The FFN is formulated as follows:
Revised: The FFN is formulated as follows:

Original: [Equation]
Revised: [Equation]

Original:  where W
Revised: ) where W

Original: [Equation]
Revised: [Equation]

Original:  and W 3 ∈ R d ffn 2 × τ are the weights of the three layers respectively, and b 1 ∈ R d ffn 1 , b 2 ∈ R d ffnand b 3 ∈ R τ are the biases of the three layers respectively.
Revised: [Equation] and W 3 ∈ R d ffn 2 × τ are the weights of the three layers respectively, and b 1 ∈ R d ffn 1 , b 2 ∈ R d ffnand b 3 ∈ R τ are the biases of the three layers respectively.

Original: Here, d ffnand d ffnare the dimensions of the three layers.
Revised: Here, d ffnand d ffnare the dimensions of the three layers.

Original: In addition, we adopt a
Revised: In addition, we adopt a

Original: ReLU activation function in Equation 11. 
Revised: ReLU activation function in Equation 11. [Table]

Original: Training details.
Revised: Training details.

Original: We carefully tune the hyperparameters, including the embedding size, batch size, d ffnand d ffn, on the validation set and choose the settings with the best performance for[Table] [Table] [Table]
Revised: We carefully tune the hyperparameters, including the embedding size, batch size, d ffnand d ffn, on the validation set and choose the settings with the best performance for

Original: [Table]

Original: [Table]

Original: EV-FGN on different datasets.
Revised: EV-FGN on different datasets.

Original: Specifically, the embedding size and batch size are tuned over { 32 , 64 , 128 , 256 , 512 } and { 2 , 4 , 8 , 16 , 32 , 64 , 128 } respectively.
Revised: Specifically, the embedding size and batch size are tuned over { 32 , 64 , 128 , 256 , 512 } and { 2 , 4 , 8 , 16 , 32 , 64 , 128 } respectively.

Original: For the COVID-19 dataset, the embedding size is 256, and the batch size is set to 4.
Revised: For the COVID-19 dataset, the embedding size is 256, and the batch size is set to 4.

Original: For the Traffic, Solar and Wiki datasets, the embedding size is 128, and the batch size is set to 2.
Revised: For the Traffic, Solar and Wiki datasets, the embedding size is 128, and the batch size is set to 2.

Original: For the METR-LA, ECG and Electricity datasets, the embedding size is 128, and the batch size is set to 32.
Revised: For the METR-LA, ECG and Electricity datasets, the embedding size is 128, and the batch size is set to 32.

Original: Note that the supra-graph connecting all nodes is a fully-connected graph, indicating that any spatial order is feasible.
Revised: Note that the supra-graph connecting all nodes is a fully-connected graph, indicating that any spatial order is feasible.

Original: Therefore, although we perform 2D DFT in EV-FGN, none spatial order in the spatial space is necessary for performing
Revised: Therefore, although we perform 2D DFT in EV-FGN, none spatial order in the spatial space is necessary for performing

Original: EV-FGN, and we directly adopt the raw dataset for experiments.
Revised: EV-FGN, and we directly adopt the raw dataset for experiments.

Original: To reduce the number of parameters, we adopt a linear transform to map the original time domain representation X Ψ ∈
Revised: To reduce the number of parameters, we adopt a linear transform to map the original time domain representation X Ψ ∈

Original: R N × T ×
Revised: R N × T ×

Original: d to a low-dimensional tensor X Ψ ∈ R N × l × d with l < T .
Revised: d to a low-dimensional tensor X Ψ ∈ R N × l × d with l < T .

Original: We then reshape X
Revised: We then reshape X

Original: Ψ ∈
Revised: Ψ ∈

Original: R
Revised: R

Original: N
Revised: N

Original: × ( ld ) and feed it to FFN.
Revised: × ( ld ) and feed it to FFN.

Original: We perform grid search on the dimensions of
Revised: We perform grid search on the dimensions of

Original: FFN, i.e., d ffnand d ffn, over { 32 , 64 , 128 , 256 , 512 } and tune the intermediate dimension l over { 2 , 4 , 6 , 8 , 12 } .
Revised: FFN, i.e., d ffnand d ffn, over { 32 , 64 , 128 , 256 , 512 } and tune the intermediate dimension l over { 2 , 4 , 6 , 8 , 12 } .

Original: The settings of the three hyperparameters over all datasets are shown in Table 6.
Revised: The settings of the three hyperparameters over all datasets are shown in Table 6.

Original: Finally, we set the diffusion step K = 3 for all datasets.
Revised: Finally, we set the diffusion step K = 3 for all datasets.

Original: E. D ETAILS FOR V ISUALIZATION E XPERIMENTS
Revised: F. D ETAILS FOR V ISUALIZATION E XPERIMENTS

Original: To verify the effectiveness of EV-FGN in learning the spatial-temporal dependencies on the fullyconnected supra-graph, we obtain the output of EV-FGN as the node representation, denoted as
Revised: To verify the effectiveness of EV-FGN in learning the spatial-temporal dependencies on the fullyconnected supra-graph, we obtain the output of EV-FGN as the node representation, denoted as

Original: R = IDFT(Ψ K ( X ))
Revised: R = IDFT(Ψ K ( X ))

Original: ∈ R N × T × d with inverse discrete Fourier transform ( IDFT ) and K -layer EVFGN Ψ K .
Revised: ∈ R N × T × d with inverse discrete Fourier transform ( IDFT ) and K -layer EVFGN Ψ K .

Original: Then, we visualize the adjacency matrix A calculated based the flatten node representation
Revised: Then, we visualize the adjacency matrix A calculated based the flatten node representation

Original: R ∈ R
Revised: R ∈ R

Original: NT × d , formulated as A = RR T ∈ R NT × NT , to show the variable correlations.
Revised: NT × d , formulated as A = RR T ∈ R NT × NT , to show the variable correlations.

Original: Note that A is normalized via A / max( A ) .
Revised: Note that A is normalized via A / max( A ) .

Original: Since it is not feasible to directly visualize the huge adjacency matrix
Revised: Since it is not feasible to directly visualize the huge adjacency matrix

Original: A of the supra-graph, we visualize its different subgraphs in Figures 3, 4, 5, and 10 to better verify the learned spatial-temporal information on the supra-graph from different perspectives.
Revised: A of the supra-graph, we visualize its different subgraphs in Figures 3, 4, 5, and 10 to better verify the learned spatial-temporal information on the supra-graph from different perspectives.

Original: We further compare the sub adjacency with the real road map (generated by the google ) to verify the learned dependencies between different detectors.
Revised: We further compare the sub adjacency with the real road map (generated by the google map tool) to verify the learned dependencies between different detectors.

Original: Figure 4.
Revised: Figure 4.

Original: On the COVID-19 dataset, we randomly choose 10 counties out of N = 55 counties and obtain their four sub adjacency matrices of four consecutive days for visualization.
Revised: On the COVID-19 dataset, we randomly choose 10 counties out of N = 55 counties and obtain their four sub adjacency matrices of four consecutive days for visualization.

Original: Each of the four sub adjacency matrices R 10 × 10 embodies the dependencies between counties in one day.
Revised: Each of the four sub adjacency matrices R 10 × 10 embodies the dependencies between counties in one day.

Original: Figurereflects the time-varying dependencies between counties (i.e., variables).
Revised: Figurereflects the time-varying dependencies between counties (i.e., variables).

Original: Figure 5.
Revised: Figure 5.

Original: Since we adopt a 3 -layer EV-FGN, we can calculate four adjacency matrices based on the input X of EV-FGN and the outputs of each layer in EV-FGN, i.e., X 1 , X 2 , X 3 .
Revised: Since we adopt a 3 -layer EV-FGN, we can calculate four adjacency matrices based on the input X of EV-FGN and the outputs of each layer in EV-FGN, i.e., X 1 , X 2 , X 3 .

Original: Following the way of visualization in Figure 4, we select 10 counties and two timestamps on the four adjacency matrices for visualization.
Revised: Following the way of visualization in Figure 4, we select 10 counties and two timestamps on the four adjacency matrices for visualization.

Original: Figure 5 shows the effects of each layer of EV-FGN in filtering or enhancing variable correlations.
Revised: Figure 5 shows the effects of each layer of EV-FGN in filtering or enhancing variable correlations.

Original: Figure 10.
Revised: Figure 10.

Original: We select 8 counties and visualize the correlations between 12 consecutive time steps for each selected county respectively.
Revised: We select 8 counties and visualize the correlations between 12 consecutive time steps for each selected county respectively.

Original: Figure 10 reflects the temporal correlations within each variable.[Table]
Revised: Figure 10 reflects the temporal correlations within each variable.

Original: F M ORE R ESULTS
Revised: G M ORE R ESULTS

Original: To further evaluate the performance of our model EV-FGN in multi-step forecasting

Original: [Table]
Revised: [Table]

Original:  we conduct more experiments on the COVID-19, Wiki [Table] and METR-LA datasets, respectively.
Revised: To further evaluate the performance of our model EV-FGN in multi-step forecasting, we conduct more experiments on the COVID-19, Wiki, ECG, and METR-LA datasets, respectively.

Original: Note that the COVID-19 and METR-LA datasets have predefined graph topologies.
Revised: Note that the COVID-19 and METR-LA datasets have predefined graph topologies.

Original: We compare EV-FGN with other GNN-based MTS models (including StemGNN, AGCRN, GraphWaveNet, MTGNN and TAMP-S2GCNets)  on the COVID-19 dataset under different prediction lengths, and the results are shown in Table 7.
Revised: We compare EV-FGN with other GNN-based MTS models (including StemGNN, AGCRN, GraphWaveNet, MTGNN and TAMP-S2GCNets) and representation learning model (CoST) on the COVID-19 dataset under different prediction lengths, and the results are shown in Table 7.

Original: From the table, we can find that EV-FGN achieves the best MAE and RMSE on all the prediction lengths.
Revised: From the table, we can find that EV-FGN achieves the best MAE and RMSE on all the prediction lengths.

Original: On average, EV-FGN has 30.0% and 27.9% improvement on MAE and RMSE respectively over the best baseline, i.e.,TAMP-S2GCNets.
Revised: On average, EV-FGN has 30.0% and 27.9% improvement on MAE and RMSE respectively over the best baseline, i.e.,

Original:  Among these models, TAMP-S2GCNets requiring a pre-defined graph topology achieves competitive performance since it enhances the resultant graph learning mechanisms with a multi-persistence.
Revised: TAMP-S2GCNets. Among these models, TAMP-S2GCNets requiring a pre-defined graph topology achieves competitive performance since it enhances the resultant graph learning mechanisms with a multi-persistence.

Original: However, it constructs the graph in the spatial dimension, while our model EV-FGN adaptively learns a supra-graph connecting any two variables at any two timestamps, which is effective and more powerful to capture high-resolution spatial-temporal dependencies.[Table] In addition
Revised: However, it constructs the graph in the spatial dimension, while our model EV-FGN adaptively learns a supra-graph connecting any two variables at any two timestamps, which is effective and more powerful to capture high-resolution spatial-temporal dependencies.

Original: [Table]
Revised: [Table]

Original:  we compare our model EV-FGN with five neural MTS models (including StemGNN,
Revised: In addition, we compare our model EV-FGN with five neural MTS models (including StemGNN,

Original: AGCRN, GraphWaveNet, MTGNN and Informer) on Wiki dataset under different prediction lengths, and the results are shown in Table 8.From the table
Revised: AGCRN, GraphWaveNet, MTGNN and Informer) on Wiki dataset under different prediction lengths, and the results are shown in Table 8.

Original: [Table] we observe that EV-FGN outperforms other models on MAE, RMSE and MAPE metrics for all the prediction lengths.
Revised: From the table, we observe that EV-FGN outperforms other models on MAE, RMSE and MAPE metrics for all the prediction lengths.

Original: On average, EV-FGN improves MAE, RMSE and MAPE by 6.8%, 3.2% and 22.9%, respectively.
Revised: On average, EV-FGN improves MAE, RMSE and MAPE by 6.8%, 3.2% and 22.9%, respectively.

Original: Among these models,
Revised: Among these models,

Original: AGCRN shows promising performances since it captures the spatial and temporal correlations adaptively.
Revised: AGCRN shows promising performances since it captures the spatial and temporal correlations adaptively.

Original: However, it fails to simultaneously capture spatial-temporal dependencies, limiting its forecasting performance.
Revised: However, it fails to simultaneously capture spatial-temporal dependencies, limiting its forecasting performance.

Original: In contrast, our model learns a supra-graph to capture comprehensive spatial-temporal dependencies simultaneously for multivariate time series forecasting.Finally
Revised: In contrast, our model learns a supra-graph to capture comprehensive spatial-temporal dependencies simultaneously for multivariate time series forecasting.

Original: [Table]
Revised: [Table]

Original:  we compare our model EV-FGN with seven neural MTS models (including STGCN, DCRNN,
Revised: Finally, we compare our model EV-FGN with seven neural MTS models (including STGCN, DCRNN,

Original: StemGNN, AGCRN, GraphWaveNet, MTGNN and Informer) on the METR-LA dataset, and the results are shown in Table 9.
Revised: StemGNN, AGCRN, GraphWaveNet, MTGNN, Informer and CoST) on the METR-LA dataset, and the results are shown in Table 9.

Original: On average, we improve 5 .
Revised: On average, we improve 5 .

Original: 7% on MAE and 2 .
Revised: 7% on MAE and 2 .

Original: 5% on RMSE.
Revised: 5% on RMSE.

Original: Among these models, StemGNN achieves competitive performance because it combines GFT to capture the spatial dependencies and DFT to capture the temporal dependencies.
Revised: Among these models, StemGNN achieves competitive performance because it combines GFT to capture the spatial dependencies and DFT to capture the temporal dependencies.

Original: However, it is also limited to simultaneously capturing spatial-temporal dependencies.
Revised: However, it is also limited to simultaneously capturing spatial-temporal dependencies.

Revised: CoST learns disentangled seasonal-trend representations for time series forecasting via contrastive learning and obtains competitive results.

Revised: But, our model still outperforms CoST.

Revised: Because, compared with CoST, our model not only can learn the dynamic temporal representations, but also capture the discriminative spatial representations.

Original: Besides, STGCN and DCRNN require pre-defined graph structures.
Revised: Besides, STGCN and DCRNN require pre-defined graph structures.

Original: But StemGNN and our model outperform them for all steps, and AGCRN outperforms them when the prediction lengths are 9 and 12.
Revised: But StemGNN and our model outperform them for all steps, and AGCRN outperforms them when the prediction lengths areand 12.

Original: This also shows that a novel adaptive graph learning can precisely capture the hidden spatial dependency.
Revised: This also shows that a novel adaptive graph learning can precisely capture the hidden spatial dependency.

Original: In addition, we compare[Table] [Figure] EV-FGN with the baseline models under the different prediction lengths on the ECG dataset, as shown in Fig.
Revised: In addition, we compare EV-FGN with the baseline models under the different prediction lengths on the ECG dataset, as shown in Fig.

Original: It reports that EV-FGN achieves the best performances (MAE, RMSE and MAPE) for all prediction lengths.
Revised: It reports that EV-FGN achieves the best performances (MAE, RMSE and MAPE) for all prediction lengths.

Original: [Figure]
Revised: [Figure]

Original: G M ORE A NALYSES G. A NALYSES
Revised: H M ORE A NALYSES H. A NALYSES

Original: [Table]

Original: Parameter Analysis .
Revised: Parameter Analysis .

Original: We evaluate the forecasting performance of our model EV-FGN under different diffusion steps on the COVID-19 dataset, as illustrated in Table 10.
Revised: We evaluate the forecasting performance of our model EV-FGN under different diffusion steps on the COVID-19 dataset, as illustrated in Table 10.

Original: The table shows that EV-FGN achieves increasingly better performance from K = 1 to K = 4 and achieves the best results when
Revised: The table shows that EV-FGN achieves increasingly better performance from K = 1 to K = 4 and achieves the best results when

Original: K = 3 .
Revised: K = 3 .

Original: With the further increase of K , EV-FGN obtains inferior performance.
Revised: With the further increase of K , EV-FGN obtains inferior performance.

Original: The results indicate that high-order diffusion information is beneficial for improving the forecasting accuracy, but the  diffusion information may gradually weaken the effect or even bring noises to forecasting with the increase of the order.
Revised: The results indicate that high-order diffusion information is beneficial for improving the forecasting accuracy, but the [Table] diffusion information may gradually weaken the effect or even bring noises to forecasting with the increase of the order.

Original: In addition, we conduct extensive experiments on the ECG dataset to analyze the effect of the input length and the embedding dimension d , as shown in Fig.
Revised: In addition, we conduct extensive experiments on the ECG dataset to analyze the effect of the input length and the embedding dimension d , as shown in Fig.

Original: 7 and Fig. 8, respectively.
Revised: 7 and Fig. 8, respectively.

Original: Fig.
Revised: Fig.

Original: 7 shows that the performance (including RMSE and MAPE) of EV-FGN gets better as the input length increases, indicating that EV-FGN can learn a comprehensive supra-graph from long MTS inputs to capture the spatial and temporal dependencies.
Revised: 7 shows that the performance (including RMSE and MAPE) of EV-FGN gets better as the input length increases, indicating that EV-FGN can learn a comprehensive supra-graph from long MTS inputs to capture the spatial and temporal dependencies.

Original: Moreover, Fig. 8 shows that the performance (RMSE and MAPE) first increases and then decreases with the increase of the embedding size, which is attributed that a large embedding size improves the fitting ability of EV-FGN but it may easily lead to the overfitting issue especially when the embedding size is too large.
Revised: Moreover, Fig. 8 shows that the performance (RMSE and MAPE) first increases and then decreases with the increase of the embedding size, which is attributed that a large embedding size improves the fitting ability of EV-FGN but it may easily lead to the overfitting issue especially when the embedding size is too large.

Original: [Figure]
Revised: [Figure]

Original: We further conduct experiments on the Wiki dataset to investigate the performance of EV-FGN under different graph sizes.
Revised: We further conduct experiments on the Wiki dataset to investigate the performance of EV-FGN under different graph sizes.

Original: The results are shown in Fig.
Revised: The results are shown in Fig.

Original: 9, where Fig.
Revised: 9, where Fig.

Original: 9(a), Fig. 9(b) and Fig.
Revised: 9(a), Fig. 9(b) and Fig.

Original: 9(c) show
Revised: 9(c) show

Original: MAE, RMSE and MAPE at the different number of nodes, respectively.
Revised: MAE, RMSE and MAPE at the different number of nodes, respectively.

Original: From these figures, we observe that EV-FGN keeps a leading edge over the other state-of-the-art MTS models as the number of nodes increases.
Revised: From these figures, we observe that EV-FGN keeps a leading edge over the other state-of-the-art MTS models as the number of nodes increases.

Original: The results demonstrate the superiority and scalability of EV-FGN on large scale datasets.[Table]
Revised: The results demonstrate the superiority and scalability of EV-FGN on large scale datasets.

Original: Ablation Study .
Revised: Ablation Study .

Original: We provide more details about each variant used in this section and Section 4.3.
Revised: We provide more details about each variant used in this section and Section 4.3.

Original: • w/o Embedding .
Revised: • w/o Embedding .

Original: A variant of EV-FGN feeds the raw MTS input instead of its embeddings into the graph convolution in the Fourier space. 
Revised: A variant of EV-FGN feeds the raw MTS input instead of its embeddings into the graph convolution in the Fourier space. [Table]

Original: • w/o Dynamic Filter .
Revised: • w/o Dynamic Filter .

Original: A variant of EV-FGN uses the same FGSO for all diffusion steps instead of applying different FGSOs in different diffusion steps.
Revised: A variant of EV-FGN uses the same FGSO for all diffusion steps instead of applying different FGSOs in different diffusion steps.

Original: It corresponds to a vanilla graph filter.
Revised: It corresponds to a vanilla graph filter.

Original: • w/o Residual .
Revised: • w/o Residual .

Original: A variant of EV-FGN does not have the K = 0 layer output, i [Table] e., X , in the summation.
Revised: A variant of EV-FGN does not have the K = 0 layer output, i.e., X , in the summation.

Original: • w/o Summation .
Revised: • w/o Summation .

Original: A variant of EV-FGN adopts the last order (layer) output X K as the final frequency output of the EV-FGN.
Revised: A variant of EV-FGN adopts the last order (layer) output X K as the final frequency output of the EV-FGN.

Original: We conduct another ablation study on the COVID-19 dataset to further investigate the effects of the different components of our EV-FGN.
Revised: We conduct another ablation study on the COVID-19 dataset to further investigate the effects of the different components of our EV-FGN.

Original: The results are shown in Table 11, which confirms the results in Table 4 and further verifies the effectiveness of each component in EV-FGN.
Revised: The results are shown in Table 11, which confirms the results in Table 4 and further verifies the effectiveness of each component in EV-FGN.

Original: Both Table 11 and
Revised: Both Table 11 and

Original: Table 4 report that the embedding and dynamic filter in EV-FGN contribute more than the design of residual and summation to the state-of-the-art performance of EV-FGN.
Revised: Table 4 report that the embedding and dynamic filter in EV-FGN contribute more than the design of residual and summation to the state-of-the-art performance of EV-FGN.

Original: [Figure]
Revised: [Figure]

Original: Ordering of the time series .
Revised: Ordering of the time series .

Original: Note that the supra-graph connecting all nodes is a fully-connected graph, indicating that any spatial order is feasible.
Revised: Note that the supra-graph connecting all nodes is a fully-connected graph, indicating that any spatial order is feasible.

Original: Then how could we perform 2D DFT to achieve the graph convolution?
Revised: Then how could we perform 2D DFT to achieve the graph convolution?

Original: First, the graph convolution on the supra-graph can be viewed as a kernel summation (cf. Eq.4, i.e., O ( X )[ i ] = (cid:80) nj =1 X
Revised: First, the graph convolution on the supra-graph can be viewed as a kernel summation (cf. Eq.4, i.e., O ( X )[ i ] = (cid:80) nj =1 X

Original: [ j ] κ [ i, j ] , ∀ j ∈ [ n ] ) and does not depend on a specific spatial order.
Revised: [ j ] κ [ i, j ] , ∀ j ∈ [ n ] ) and does not depend on a specific spatial order.

Original: From Eq.4 to Eq.5, we extend the kernel summation to a kernel integral in the continuous spatial-temporal space and introduce a special kernel, i.e., the shift-invariant Green’s kernel κ ( i, j )
Revised: From Eq.4 to Eq.5, we extend the kernel summation to a kernel integral in the continuous spatial-temporal space and introduce a special kernel, i.e., the shift-invariant Green’s kernel κ ( i, j )

Original: =
Revised: =

Original: κ ( i − j ) .
Revised: κ ( i − j ) .

Original: According to the convolution theorem, we can reformulate the graph convolution with continuous Fourier transform (i.e., Eq.6 O ( X )( i ) =
Revised: According to the convolution theorem, we can reformulate the graph convolution with continuous Fourier transform (i.e., Eq.6 O ( X )( i ) =

Original: F − 1 ( F ( X ) F ( κ ))
Revised: F − 1 ( F ( X ) F ( κ ))

Original: ( i ) , ∀ i ∈ D ) and then apply Eq.6 to the finite discrete spatial-temporal space (i.e., the supra-graph).
Revised: ( i ) , ∀ i ∈ D ) and then apply Eq.6 to the finite discrete spatial-temporal space (i.e., the supra-graph).

Original: Accordingly, we can obtain the Definition 2 and Eq.7 reformulating the graph convolution with 2D DFT.
Revised: Accordingly, we can obtain the Definition 2 and Eq.7 reformulating the graph convolution with 2D DFT.

Original: To understand the learning paradigm of EV-FGN, we can recall the self-attention mechanism as an analogy that does not need any assumption on datapoint order and introduce extra position embeddings for capturing temporal patterns.
Revised: To understand the learning paradigm of EV-FGN, we can recall the self-attention mechanism as an analogy that does not need any assumption on datapoint order and introduce extra position embeddings for capturing temporal patterns.

Original: EV-FGN performs 2D DFT on each discrete N × T spatial-temporal plane of the embeddings X instead of the raw data X .
Revised: EV-FGN performs 2D DFT on each discrete N × T spatial-temporal plane of the embeddings X instead of the raw data X .

Original: A key insight underpinning
Revised: A key insight underpinning

Original: EV-FGN is to introduce variable embeddings and temporal position embeddings to equip EV-FGN with a sense of variable correlations (spatial patterns and temporal patterns).
Revised: EV-FGN is to introduce variable embeddings and temporal position embeddings to equip EV-FGN with a sense of variable correlations (spatial patterns and temporal patterns).

Original: To verify the claim, we conducted experiments on the dataset ECG.
Revised: To verify the claim, we conducted experiments on the dataset ECG.

Original: Specifically, we randomly shuffle the order of time-series variables of the raw data five times and evaluate our EV-FGN on each shuffled data.
Revised: Specifically, we randomly shuffle the order of time-series variables of the raw data five times and evaluate our EV-FGN on each shuffled data.

Original: The results are reported in the following table.
Revised: The results are reported in the following table.

Original: Note that we can also conduct shuffling experiments on temporal order via performing the same shuffling scheme over each window-sized time-series input, which will get the same conclusion.
Revised: Note that we can also conduct shuffling experiments on temporal order via performing the same shuffling scheme over each window-sized time-series input, which will get the same conclusion.

Original: We will provide discussions about this issue in the paper.

Original: [Table]
Revised: [Table]

Original: G. V ISUALIZATIONS
Revised: H. V ISUALIZATIONS

Original: To demonstrate the ability of our EV-FGN in jointly learning spatial-temporal dependencies, we visualize the temporal adjacency matrix of different variables.
Revised: To demonstrate the ability of our EV-FGN in jointly learning spatial-temporal dependencies, we visualize the temporal adjacency matrix of different variables.

Original: Note that the spatial adjacency matrices of different days are reported in Fig.
Revised: Note that the spatial adjacency matrices of different days are reported in Fig.

Original: Specifically, we randomly select 8 counties from the COVID19 dataset and calculate the correlations of 12 consecutive time steps for each county.
Revised: Specifically, we randomly select 8 counties from the COVID19 dataset and calculate the correlations of 12 consecutive time steps for each county.

Original: Then we visualize the adjacency matrix via heat map, and the results are shown in Fig.
Revised: Then we visualize the adjacency matrix via heat map, and the results are shown in Fig.

Original: 10 where N denotes the index of the country (variable).
Revised: 10 where N denotes the index of the country (variable).

Original: From the figure, we observe that EV-FGN learns clear and specific temporal patterns for each county.
Revised: From the figure, we observe that EV-FGN learns clear and specific temporal patterns for each county.

Original: These results show that our EV-FGN can not only learn highly interpretable spatial correlations (see Fig.
Revised: These results show that our EV-FGN can not only learn highly interpretable spatial correlations (see Fig.

Original: 2 and Fig. 3), but also capture discriminative temporal patterns.
Revised: 2 and Fig. 3), but also capture discriminative temporal patterns.