Multivariate Time-Series Forecasting with Temporal Polynomial Graph Neural Networks
Abstract
Modeling multivariate time series (MTS) is critical in modern intelligent systems.
The accurate forecast of MTS data is still challenging due to the complicated latent variable correlation.
Recent works apply the Graph Neural Networks (GNNs) to the task, with the basic idea of representing the correlation as a static graph.
However, predicting with a static graph causes signiﬁcant bias because the correlation is time-varying in the real-world MTS data.
Besides, there is no gap analysis between the actual correlation and the learned one in their works to validate the effectiveness.
This paper proposes a temporal polynomial graph neural network (TPGNN) for accurate MTS forecasting, which represents the dynamic variable correlation as a temporal matrix polynomial in two steps.
First, we capture the overall correlation with a static matrix basis.
Then, we use a set of time-varying coefﬁcients and the matrix basis to construct a matrix polynomial for each time step.
The constructed result empirically captures the precise dynamic correlation of six synthetic MTS datasets generated by a non-repeating random walk model.
Moreover, the theoretical analysis shows that TPGNN can achieve perfect approximation under a commutative condition.
We conduct extensive experiments on two trafﬁc datasets with prior structure and four benchmark datasets.
The results indicate that TPGNN achieves the state-of-the-art on both short-term and long-term MTS forecastings.
Introduction
The wide deployment of sensors in modern societies records tremendous time-series data, which boosts the application of energy dispatch, trafﬁc control, etc.
Commonly, there are numerous distributed sensors in a monitoring system, e.g., the temperature control system of a computing center.
Univariate time-series data recorded by these sensors formulate the multivariate time-series (MTS) data.
Forecasting over MTS data has been widely studied [4,21,41] in recent years, as it provides essential information for strategy formulation and resource schedule [30,52].
One basic premise of modeling the MTS data is that the variables interact with each other, which is fulﬁlled in most cases.
Therefore, capturing the variable correlation is essential for MTS forecasting.
Previous deep-learning-based works like FC-LSTM [36] and TPA-LSTM [34] model the correlation with an implicit recurrent process, which is inefﬁcient in prediction and hard to optimize.
The development of graph neural networks (GNNs) [17] brings an innovative way to capture the variable dependence.
Recent works [12,25,44] regard the MTS data as a series of graph signals and process them with GNNs, where the nodes are variables and the edge weight quantiﬁes the dependence.
Although GNNs-based methods achieve promising efﬁciency and accuracy in MTS forecasting,
[Figure]
there are two notable challenges to the current approaches.
The MTS data have no prior variable dependence, which is required for determining the edge weight in graph construction.
The existent methods compute the weight according to the physical distance among sensors [23,35,44] or utilize parameter matrices to learn the weight from MTS data [27,41,42].
As a result, the edge weights are constant in their methods.
However, according to the observations from the real-world dataset, the variable dependence changes with time.
Therefore, using static-weight graphs in prediction causes signiﬁcant bias.
Figure 1 illustrates the ﬁrst challenge with a trafﬁc dataset (PEMS-D7).
It demonstrates that the correlation has distinct patterns in different periods.
Besides, three staticgraph-based methods [41,42,44] show error increases between two consecutive periods, which is caused by dependence variation.
Many researchers point out a gap between the actual variable dependence and the constructed correlation [25, 42, 49], i.e., the edge weight does not accurately describe the variable correlation.
Although they propose various methods to bridge the gap, there is no theoretical/empirical analysis of the gap.
Therefore, the method’s effectiveness is still unknown.
To address the above two challenges, we propose Temporal Polynomial Graph Neural Network (TPGNN), a novel GNNs-based forecasting model for MTS data.
TPGNN has an encoder-decoder structure, we encode historical MTS data with a novel GNN module and make predictions by decoding the encoded results auto-regressively.
The core of TPGNN is a temporal polynomial graph (TPG) module that learns the dynamic variable dependence end-to-end.
The main idea is to represent the correlation as a time-varying matrix polynomial.
Firstly, we deﬁne an adjacency matrix basis A to capture the overall dependence for each variable pair.
Then, we use a matrix polynomial of A to represent the correlation of each time step, where the coefﬁcients are determined by time.
To generate coefﬁcients for MTS data of arbitrary length, we propose to use a set of cyclic timestamp embedding to index the time step.
As a result, TPGNN captures the dynamic correlation with a temporal matrix polynomial, which solves the ﬁrst challenge effectively.
For the second challenge, we analyze the approximation error of the TPG module and provide a theoretical bound on the error.
Furthermore, we evaluate the existent dependence learning methods on six synthetic MTS datasets with ground-truth variable dependence.
The main contributions of the work are as follows:
• We propose a novel dependence learning module to capture MTS data’s dynamic correlation, which represents the variable dependence as a temporal matrix polynomial.
The resultant model achieves state-of-the-art forecasting performance on 5 of 6 benchmark datasets with signiﬁcant improvements.
• We investigate the dependence approximation gap on six synthetic datasets with different levels of dependence complexity.
On average, the proposed TPGNN outperforms the best baseline by 23 .
41% on the approximation error.
A theoretical result further demonstrates the dependence learning ability of TPGNN.
Related Works
MTS Forecasting There are two main categories of methods for MTS forecasting: implicitdependence approaches and structural-dependence approaches.
One representative method of the ﬁrst category is LSTNet [20], it captures the variable dependence with convolution over variables.
The structural-dependence approaches represent the variable correlation as explicit graphs and predict with GNNs.
Graph Wavenet [42] (GWN) attempts to learn the variable correlation with static node representations and captures the temporal pattern with convolution neural networks (CNNs).
MTGNN [41] follows the idea of GWN to learn variable correlation statically and proposes a new graph convolution module for MTS forecasting.
Although works like GMAN [50], SLC [49], and
StemGNN [4] model a dynamic correlation among variables with self-attention, the learned graph is sensitive to the input, causing a signiﬁcant forecasting variance.
Our construction is based on a set of time-variant coefﬁcients, the result is independent of the input and robust in forecasting.
Some works that focus on learning time-varying graph structure [26,33,45].
However, these graph structures are not captured from the forecast task directly.
They thus cause bias and increase the computation cost.
There are some transformer-based works [43,46,48], but their methods do not thoroughly discuss the dynamic-dependence issues.
GNNs and Polynomial Graph Filters (PGFs) GNNs are based on the message-passing mechanism.
One representative work is GraphSAGE [14], which learns a sample-aggregation function to generate the node embedding.
Michael et al.
[6] generalize the convolutional neural networks (CNNs) to the graph structure.
Many ﬁlters of GNNs belong to PGFs, and Gama et al.
[11] discussed the connections.
In order to incorporate the node feature of high-order neighbors, many works introduce PGFs to their methods.
APPNP [18] and SGC [40] adopt ﬁxed-weight PGFs and use hyperparameters to deﬁne the coefﬁcients.
However, the constructed ﬁxed-weight PGFs are biased because the coefﬁcients are task-irrelevant.
GPR-GNN [5] constructs a PGF from the graph dataset, where the coefﬁcients are learnable.
Besides, there are also some PGFs-based methods for MTS data learning [13,15,16].
Although these works are competitive in capturing graph structure, they fail in learning the dynamic correlation, we illustrate the point in Section 5.4.
Moreover, the missing/noisy/incomplete adjacency matrix causes difﬁculties in applying these methods.
Our approach introduces a self-adaptive graph to the PGFs construction, which improves the dependence capturing ability and the model versatility.
The Framework of TPGNN
In this section, we ﬁrst deﬁne the MTS forecasting task in the context of graph signal processing.
We then introduce the TPG module and illustrate the inference pipeline of our framework.
3.1 Problem Deﬁntion
At time step t , the MTS data is a signal set G ( t ) =
{V , E ( t ) , X ( t ) , W ( t ) } .
The node set V ( |V| = N ) contains N variables of MTS data, E ( t ) is the edge set, X ( t ) ∈ R N × 1 records the signal of N variables, and W ( t ) ∈ R N × N is a weighted adjacency matrix.
If variables i and j are dependent, then an edge connects the two nodes, and entry W ( t ) ij indicates their correlation.
Otherwise, W ( t ) ij equals to zero.
Given T observations of the signal sets, our problem is to ﬁnd a function F that can forecast the variable state of the subsequent T (cid:48) signals:
[Equation]
The setting regards the graph structure as a latent feature of MTS data, and the forecast target is the same as the standard setting.
Prevailing works [25,41,42] regard W ( t ) as a time-invariant matrix, while in our work, W ( t ) changes with the time.
3.2 Represent the Correlation as a Temporal Matrix Polynomial
To capture E and W t for MTS data, we construct a series of matrix polynomials that share a matrix basis.
The TPG module controls the inﬂuence of each polynomial term temporally with a set of adaptive coefﬁcients.
As a result, we can approximate a wide range of dynamic graph structures.
Counstructing the adjacency matrix basis.
We ﬁrstly deﬁne an initial adjacency matrix A ∈ R N × N following the self-adaptive graph proposed by Wu et al.
They learn variable embeddings
E ∈ R N × c ( c is the embedding dimension) from MTS data and deﬁne the variable dependence with embedding similarity.
Besides, they further remove weak connections with ReLU( · ) and normalize the result with SoftMax( · ) .
[Equation]
For MTS data that have a prior structure, e.g., the physical distance among sensors, we denote the corresponding adjacency matrix as W ∈ R N × N .
To incorporate W into the encoding process, we ﬁrst calculate its symmetric normalized Laplacian [17] L = D − 12 ( I + W ) D − 12 ( I is the identity matrix and D is the degree matrix), then combine the two results to deﬁne the initial adjacency matrix:
[Equation]
Generating the temporal polynomial coefﬁcients.
At time step t , we represent the variable dependence as a K -order matrix polynomial of A :
[Equation]
Due to the polynomial coefﬁcients being time-variant, W ( t ) shows dynamic behavior.
However, learning coefﬁcients for every time step is impractical because the sequence length grows inﬁnitely.
Fortunately, the periodicity property is common in MTS data [8, 29].
Therefore, we propose to use a set of cyclic timestamp embeddings to index the time step.
Firstly, we deﬁne T p timestamp embeddings ( e (1) ts , . . .
, e ( T p ) ts ) ∈
R D e , where T p is the cycle and D e is the embedding dimension.
Following a cyclic order, we then assign the ( t % T p ) -th embedding to time step t , where % is the remainder operator.
Finally, we generate the coefﬁcients for time step t by the e ( t % T p ) ts .
As shown in Figure 2 (1), Given T historical graph signals start from time step t , we calculate the coefﬁcients a ∈ R K +1 for each graph signal with a coefﬁcient matrix W c ∈ R D e × ( K +1) .
[Equation]
According to Equation 4, each a ( t ) deﬁnes a K -order matrix polynomial of A , representing variable dependence of time step t .
Instead of predicting with T different matrix polynomials, we use an average result to improve the prediction efﬁciency and robustness.
Due to the linearity of the polynomial, the average polynomial is equivalent to the polynomial with average coefﬁcients.
We calculate the ﬁnal coefﬁcients ¯ a with another parameter matrix W a ∈ R T × 1 (average matrix).
[Equation] [Equation]
In Figure 2 (2), we illusrate the TPG module’s propagation process with ¯ a .
Let X ∈ R N × D e denote the matrix signal embedding,
W k ∈ R D
e × D e , k = 0 , . . .
, K denote the model parameter matrices, || · || F denote the Frobenius norm of matrix, we derive the hidden feature Z ( t ) ∈ R N × D e as follows:
[Equation]
TPG module follows the diffusion graph convolution layer [23] to introduce ( K + 1) parameter matrices for enhancing representation diversity.
Besides, we decouple the parameter matrix norm and coefﬁcient by normalizing W k with a factor of 1 || W k || F , since W k with a large norm increases the k -th term’s contribution.
As a result, ¯ a completely controls the contribution of each term.
3.3 Inference Pipeline
In Figure 2 (3), TPGNN has an encoder-decoder structure.
First, we derive the historical data embedding with a linear transformation, then feed the results into the encoder.
The encoder generates data encoding Z ( t : t + T − 1) ∈ R T × N × D e for time step t to t + T − 1 with the TPG module and temporal attention layer.
The temporal attention layer follows the design of Transformer [37], which uses self-attention to capture intra-series patterns for each variable.
Finally, the decoder forecasts with the data encoding and temporal attention layer.
For the ﬁrst prediction of time step t + T , the decoder queries the data encoding with a learnable beginning of sentence (BOS) token E BOS ∈ R N × D e .
The query result represents the ﬁrst prediction, denoted as E ( t + T ) X ∈ R N × D e .
We use a prediction matrix
W pred ∈ R D e × 1 to get the forecasting result ˜ X ( t + T ) ∈
R N × 1 .
[Equation]
After generating the subsequent k query results ( E ( t + T ) X , . .
, E ( t + T + k − 1) X ) , the decoder derives the ( k + 1) -th result E ( t + T + k ) X by querying
Z ( t : t + T − 1) with the latest L max query results.
The forecast basically follows an auto-regressive (AR) mechanism, where L max is the maximum query length.
[Equation]
The above process continues until getting the subsequent T (cid:48) forecasts, such AR mechanism helps
TPGNN to capture long-term intra-series dependence with informative context.
Theoretical Properties of TPGNN
To explore the dependence learning capability of TPGNN, we focus on analyzing the theoretical gap between the optimal graph structure and the TPG module’s approximation in this section.
TPGNN is able to achieve perfect approximation.
Unlike the real function, the approximation of graph structure largely depends on the topology property of the target graph, where the lower bound of the approximation error is nonzero in many cases.
However, TPGNN mitigates the gap for a wide range of dynamic graph structures with a small group of parameters, which leads to a robust and accurate forecasting result.
The approximation capability of TPGNN is concluded as the following result, we show the detailed statement and proofs in Appendix A.9 due to the space limitation.
Theorem 1 .
Let G (1) , . .
, G ( T )
∈ R N × N be the symmetric normalized Laplacian of the optimal graph structure for time step 1 to T , A ∈ R N × N is the initial adjacency matrix of TPGNN.
We model these Laplacians with Equation 4, and the corresponding approximation error e (1: T ) =
T Σ T t =1 || W ( t ) − G ( t )
|| 2 F , where the || · || F is the Frobenius norm of the matrix.
Suppose all the matrices are symmetric (undirected graph), G i G j = G j G
i , ∀ i, j , A has N different singular values, and the polynomial’s order is large enough.
Then we have the following estimation for e (1: T ) .
[Equation]
[Equation]
where E t || G ( t ) ||
2 F = 1 T Σ Tt =1 || G ( t )
||
2 F is the average norm of the Laplacians, λ min , λ max ∈ [0 , 1] are the minimum and maximum eigenvalues of a constant matrix.
If A is commutative with G ( t ) , then λ max = λ min = 1 , i.e., we achieve a uniform perfect approximation.
Although the zero error depends on the commutative condition, A is learnable with Equation 2, which relaxes the restriction.
Moreover, TPGNN empirically achieves the best approximation compared with other methods.
Experiments
[Table]
5.1 Experimental Setup
We validate the performance of TPGNN on two tasks: single-step and multi-step MTS forecasting.
In
Table 1, we present the statistics of six benchmark datasets.
The ﬁrst four benchmark datasets [2,19] aim to predict a single future step; the last two datasets [23,44] record a distance matrix among sensors, where the goal is to forecast multiple future steps.
More details about the datasets are given in the
AppendixA.3.
For single-step prediction, the baselines are VAR-MLP [47], GP [31], RNN-GRU [41],
LSTNet [20], TPA-LSTM [34], and MTGNN [41].
MTGNN ﬁrstly introduces GNNs to general
MTS data and achieves state-of-the-art (SOTA) performance; other baselines are representative real-world datasets, and models with high CORR and low RSE are preferred.
TPGNN outperforms other models on three of the ﬁrst four datasets.
For the remaining two datasets, we predict the next 15, 30, and 60 minutes based on one-hour observation
[Table]
and the three metrics represent the prediction error.
The results show that
TPGNN achieves state-of-the-art performance on both short-term and long-term predictions.
[Table]
non-GNNs-based methods.
For multi-step prediction, the baselines are ARIMA [23],FC-LSTM [36],
DCRNN [22], STGCN [44], StemGNN [4], Graph WaveNet, Informer [51], and MTGNN
; all the baselines are GNN-based except ARIMA, FC-LSTM and Informer.
These GNNs-based baselines are representative methods for MTS data with a prior structure.
Besides, the Informer is a novel
Transformer-based model that achieves SOTA performance on long-term forecasting.
We list the paramter scale of each model in Appendix A.4 to show our method is light-weight.
We use ﬁve evaluation metrics [41]: Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE),
Root Mean Squared Error (RMSE), Root Relative Squared Error (RRSE), and Empirical Correlation
Coefﬁcient (CORR).
For CORR, higher values are better.
For the other metrics, lower values are better.
We tune the hyperparameters on the validation dataset, the results are presented in the Appendix A.7.
We divide the dataset into three parts for training, validation, and testing with a ratio of 7:1:2.
More details about the baselines, metrics, and experiment settings are in Appendix A.4A.5A.6.
5.2 Main Results
Table 2 summarizes the experimental results of TPGNN.
Generally, TPGNN achieves the stateof-the-art on most of the datasets.
Our framework has an on-par performance for the single-step forecasting task with the SOTA methods like MTGNN and TPA-LSTM.
Moreover, TPGNN makes a signiﬁcant improvement on Trafﬁc and Electricity datasets.
On the Solar-Energy, TPGNN lowers
RSE by 1 . 61% , 18 .
08% and increases CORR by 1 .
93% over the horizons of 12, 24.
Our method improves the CORR by 2 .
21% on average over the four horizons of the Trafﬁc.
Moreover, we lower the RSE by 15 .
82% on average over the four horizons of Electricity.
Our method fails to achieve SOTA performance on the exchange-rate data.
We think the main reason is the small sample size, which causes difﬁculties in capturing the dynamic variable dependence.
The second row of
Table 2 concludes the results of the multi-step forecasting task, which demonstrates that TPGNN achieves SOTA performance on the two datasets with prior structure.
On the PEMS-BAY, TPGNN lowers the MAPE by 5 .
47% on average over the three horizons.
Besides, The RMSE is reduced by 4 . 30% , 2 .
41% over the horizons of 3 and 6.
TPGNN reduces the MAE/MAPE by 8 . 04% / 6 .
61% on average over the three horizons on the PEMS-D7 dataset.
The RMSE is lowered by 2 .
23% over the horizon of 6 and 12.
Although the prior structure is provided to each baseline, the results on the last two trafﬁc datasets demonstrate that TPGNN can capture more precise variable dependence and intra-series patterns than other GNNs-based methods.
5.3 Ablation Experiments
[Table]
The PEMS-D7 includes a prior structure and has complex inter-series and intra-series dependence, we conduct ablation experiments on the PEMS-D7 to evaluate the effectiveness of each design in our method.
Due to the importance and complexity of the long-term prediction, we set the horizon as 12.
More results of other horizons can be found in Appendix A.8.
There are ﬁve experiments in total, we repeat each experiment 5 times and report the average and standard deviation (std).
w/o
TPG : TPGNN without the TPG module.
We replace the graph constructed by TPG with the prior structure of PEMS-D7.
w/o dynamic : TPGNN without the time-varying coefﬁcients.
We adopt a set of learnable static coefﬁcients to construct the graph.
w/o overview : TPGNN does not use average coefﬁcients in Equation 6.
We predict with T graphs deﬁned by Equation 5.
w/o normalize :
TPGNN without the normalization factor in Equation 7.
w/o K-matrices : TPGNN without multiple parameter matrices in Equation 7.
We use an identical normalized matrix to derive the hidden feature.
Table 3 summarizes the experimental results, which indicates that all the designs are indispensable.
The result of w/o TPG illustrates the TPG module’s critical contribution to our method.
Although we replace the graph constructed by the TPG module with the prior structure given by PEMS-D7, the performance has a steep decline by 13 .
23% on the three metrics.
w/o dynamic follows the strategy of GPR-GNN [5] that uses static coefﬁcients to capture graph structure from data.
However, the results indicate the importance of introducing time-variant coefﬁcients, which
[Figure]
signiﬁcantly improve the long-term prediction accuracy by 7 . 04% on average.
w/o overview and w/o K-matrices show large variance compared with other results
[Table]
indicating the two modules are essential for improving model robustness.
Besides, the w/o overview’s performance is close to the w/o dynamic, demonstrating the necessity of using the average coefﬁcients.
w/o normalize has comparable performance to the TPGNN, possibly due to the weight decay causing similar norms.
Nevertheless, it demonstrates that the normalization helps TPGNN capture the variable dependence.
[Table]
5.4 Study of the Graph Structure Approximation Gap
Synthetic data.
To investigate the empirical gap between the optimal graph structure and the approximation result, we propose to generate MTS data with ground-truth dynamic dependence using a non-repeating random walk (NRW) model [7], which is widely adopted in time-series data generation [1, 32].
At time step t , we synthesize the matrix signal X ( t ) ∈
R N × 1 with a dynamic weighted adjacency matrix W ( t ) by X ( t ) =
N ( W ( t − 1) X ( t − 1) , σ ) , where N ( · , · ) is the normal distribution, σ ∈ R controls the variance, and X (0) is sampled from a discrete uniform distribution.
We deﬁne W ( t ) by traversing N w constant matrices ( G (1) , . . .
, G ( N w ) ) in a cyclic order, where each
G ( i ) is a Laplacian of sparsiﬁed random adjacency matrix.
Given a period of T p , T p ≥ N w , we partition the period into N w even intervals of length T s , and each interval shares one of the matrices, i.e., W ( t ) =
G ([( t % T p ) /T
s ]) , where [ · ] is the rounding function.
We point out that N w controls the complexity of dynamic dependence, and N w = 1 corresponds to a static-dependence case.
To increase the data diversity, we randomly initialize the X ( t ) for every T p step.
We set T p = 120 , σ = 0 .in
our evaluation and generate six MTS data of length 2400 with N w = 1 , . .
Besides, each
[Figure]
synthetic dataset is divided into three parts for training, validation, and testing with a ratio of 7:1:2.
More details about the synthetic algorithm, conﬁguration, and synthetic results are in Appendix A.3.
Predicting with different graph approximation methods.
We evaluate the dependence capturing ability of different methods on the six synthetic datasets, where the task is the next-step prediction based on current variable states.
Suppose that ˜ W ( t ) ∈ R N × N , t = 0 , . .
, T − 1 is the learned adjacency matrix series, we measure the approximation gap with a Mean Frobenius Error (MFE) metric, deﬁned by MFE = 1 TN 2 Σ T − 1 t =
|| W t − ˜ W ( t ) || F .
Besides, we use MAE/RMSE to measure the forecasting accuracy.
The baselines are Graph Wavenet (GWN), MTGNN, GPR-GNN, and
Self-attention.
The ﬁrst three methods are static-dependence-based.
Self-attention constructs a dynamic graph, which is adopted in many works like GMAN [50] and StemGNN [4].
We follow the implementation of StemGNN to learn the graph structure with a GRU module and self-attention operator in the evaluation.
The polynomial order K is ﬁxed as 2 in the evaluation, we repeat each experiment 5 times and report the average/std, more details about the settings are in Appenedix A.6.
The main results are summarized in Figure 3 (a) and Table 4, demonstrating that TPGNN outperforms other baselines on the six synthetic datasets.
In Table 4, TPGNN reduces the graph approximation error by 23 .
41% on average over the six datasets.
The ﬁrst chart of Figure 3 (a) also illustrates the improvement, we consistently achieve the best approximation under different N w .
Moreover, the
MFE of TPGNN has a descending trend with the increase of N w while other methods’ approximation errors are increasing.
We think the TPG module overﬁts on datasets with small N w , since their behavior tends to have static dependence.
TPGNN achieves the lowest prediction error on average, the improvement in the accuracy is slight compared with MFE because the task is a simple next-step prediction.
Furthermore, we observe that the ranking on MFE is identical to the MAE/RMSE, which demonstrates the importance of capturing precise variable dependence for MTS forecasting.
The effect of polynomial order.
We further investigate the relation between approximation error and the polynomial order K , where the K ranges from 1 to 4 , and K = 1 corresponds to a linear approximation.
The ﬁrst two charts of Figure 3 (b) present the average MFE and corresponding std, respectively.
The results show that the approximation variance increases signiﬁcantly with K , especially when K = 4 .
Moreover, TPGNN-K4 has a large MFE on the six datasets compared with other conﬁgurations.
Therefore, we think a high-order matrix polynomial impairs both the precision and robustness of the dependence capturing ability of TPGNN.
We point out that the observation is non-trivial since GPR-GNN achieves good results with a large K ( K ≥ 10 ) on normal graph datasets, possibly due to the difference between MTS forecasting and node classiﬁcation.
The third chart shows the prediction error, it also validates that model with low MFE has a low prediction error.
Case study on the real-world dataset.
Due to the absence of ground-truth dependence, we illustrate the correlation modeling ability of TPGNN on real-world datasets with an example of PEMS-D7.
We ﬁnd three highly correlated nodes in the learned result but independent in the prior structure, where the node IDs are 60, 93, and 218.
In Figure 4 (a), we mark the physical location of the three nodes according to PEMS [3].
The three nodes are on the same road, which illustrates that TPGNN can learn reliable dependence from data.
In Figure 4 (b), we show the learned correlation of pairs (60 , 218) and (93 , 218) .
We observe that (93 , 218) has a stable dependence and (60 , 218) ’s correlation increases in the evening, the difference possibly caused by the branch road between 218 and 60.
Conclusion
We proposes a novel GNNs-based method for MTS forecasting, TPGNN, which represents the variable dependence as a temporal polynomial matrix.
The results on six real-world datasets illustrate the outstanding forecasting performance of TPGNN.
Besides, extensive experiments on synthetic datasets demonstrate that our method achieves the best dependence approximation than other baselines.
Limitations and future works Although TPGNN achieves outstanding performance on MTS forecasting and dependence capturing, our methods have limitations.
First, the dependence captured by TPGNN is not a strict causal structure.
Therefore, some correlations are unreliable and decrease the model’s robustness.
Second, MTS forecasting in the real world is affected by rare events like weather disasters, which may signiﬁcantly impair prediction accuracy.
The future works focus on causal learning and transfer learning [9,24,28,38] on MTS data to solve the above limitations.
Acknowledgment
The work of the authors was supported by the National Natural Science Foundation of China (No.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] see Section(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] see Appendix A. (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] see Appendix A. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] see Appendix A. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] see Appendix A. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] see Section 5.3 5.4 and Appendix A. (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] see Appendix A. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] see Appendix A. (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[Yes] see Appendix A. (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[Yes] see Appendix A. 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] not applicable (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] not applicable (c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A] not applicable
[-Ensure: Synthetic MTS data X ∈ R T × N 1: Generate a random orthogonal matrix P ∈ R N × N ; 2: for i = 
-]
[-Symmetric normalized Laplacian of G i 11: end for 12: X = 0 ∈ R T × N 13: T l = T/T p 14: for t = 1 → T do 15: if t-1% T p =
-]
[-Kk =
-]
[-Let E = G − Σ Kk =
-]
[-Lemma 
-]