Multivariate Time-Series Forecasting with Temporal Polynomial Graph Neural Networks
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Modeling multivariate time series (MTS) is critical in modern intelligent systems.
The accurate forecast of MTS data is still challenging due to the complicated latentvariable correlation.
Recent works apply the Graph Neural Networks (GNNs) to thetask, with the basic idea of representing the correlation as a static graph.
However,predicting with a static graph causes signiﬁcant bias because the correlation istime-varying in the real-world MTS data.
Besides, there is no gap analysis betweenthe actual correlation and the learned one in their works to validate the effectiveness.
This paper proposes a temporal polynomial graph neural network (TPGNN) foraccurate MTS forecasting, which represents the dynamic variable correlation as atemporal matrix polynomial in two steps.
First, we capture the overall correlationwith a static matrix basis.
Then, we use a set of time-varying coefﬁcients and thematrix basis to construct a matrix polynomial for each time step.
The constructedresult empirically captures the precise dynamic correlation of six synthetic MTSdatasets generated by a non-repeating random walk model.
Moreover, the theoretical analysis shows that TPGNN can achieve perfect approximation under acommutative condition.
We conduct extensive experiments on two trafﬁc datasetswith prior structure and four benchmark datasets.
The results indicate that TPGNNachieves the state-of-the-art on both short-term and long-term MTS forecastings.
Introduction
The wide deployment of sensors in modern societies records tremendous time-series data, whichboosts the application of energy dispatch, trafﬁc control, etc.
Commonly, there are numerousdistributed sensors in a monitoring system, e g., the temperature control system of a computing center.
Univariate time-series data recorded by these sensors formulate the multivariate time-series (MTS)data.
Forecasting over MTS data has been widely studied [4,19,39] in recent years as it providesessential information for strategy formulation and resource schedule [27,47].
One basic premise of modeling the MTS data is that the variables interact with each other, which isfulﬁlled in most cases.
Therefore, capturing the variable correlation is essential for MTS forecasting.
Previous deep-learning-based works like FC-LSTM [36] and TPA-LSTM [33] model the correlationwith an implicit recurrent process, which is inefﬁcient in prediction and hard to optimize.
Thedevelopment of graph neural networks (GNNs) [15] brings an innovative way to capture the variabledependence.
Recent works [10,22,41] regard the MTS data as a series of graph signals and processthem with GNNs, where the nodes are variables and the edge weight quantiﬁes the dependence.
Although GNNs-based methods achieve promising efﬁciency and accuracy in MTS forecasting,there are two notable challenges to the current approaches.
The MTS data have no prior variabledependence, which is required for determining the edge weight in graph construction.
The existentmethods compute the weight according to the physical distance among sensors [21,35,41] or utilize
[Figure]
parameter matrices to learn the weight from MTS data [24,39,40].
As a result, the edge weightsare constant in their methods.
However, according to the observations from the real-world dataset,the variable dependence changes with time.
Therefore, using static-weight graphs in predictioncauses signiﬁcant bias.
Figure 1 illustrates the ﬁrst challenge with a trafﬁc dataset (PEMS-D7).
It demonstrates that the correlation has distinct patterns in different periods.
Besides, three staticgraph-based methods [39–41] show error increases between two consecutive periods, which iscaused by dependence variation.
Many researchers point out a gap between the actual variabledependence and the constructed correlation [22, 40, 44], i.e., the edge weight does not accuratelydescribe the variable correlation.
Although they propose various methods to bridge the gap, there isno theoretical/empirical analysis of the gap.
Therefore, the method’s effectiveness is still unknown.
To address the above two challenges, we propose Temporal Polynomial Graph Neural Network(TPGNN), a novel GNNs-based forecasting model for MTS data.
TPGNN has an encoder-decoderstructure, we encode historical MTS data with a novel GNN module and make predictions by decodingthe encoded results auto-regressively.
The core of TPGNN is a temporal polynomial graph (TPG)module that learns the dynamic variable dependence end-to-end.
The main idea is to represent thecorrelation as a time-varying matrix polynomial.
Firstly, we deﬁne an adjacency matrix basis
Ato capture the overall dependence for each variable pair.
Then, we use a matrix polynomial of Ato represent the correlation of each time step, where the coefﬁcients are determined by time.
Togenerate coefﬁcients for MTS data of arbitrary length, we propose to use a set of cyclic timestampembedding to index the time step.
As a result, TPGNN captures the dynamic correlation with atemporal matrix polynomial, which solves the ﬁrst challenge effectively.
For the second challenge,we analyze the approximation error of the TPG module and provide a theoretical bound on the error.
Furthermore, we evaluate the existent dependence learning methods on six synthetic MTS datasetswith ground-truth variable dependence.
The main contributions of the work are as follows:
• We propose a novel dependence learning module to capture MTS data’s dynamic correlation,which represents the variable dependence as a temporal matrix polynomial.
The resultantmodel achieves state-of-the-art forecasting performance on 5 of 6 benchmark datasets withsigniﬁcant improvements.•
We investigate the dependence approximation gap on six synthetic datasets with differentlevels of dependence complexity.
On average, the proposed TPGNN outperforms the bestbaseline by 23 .
41% on the approximation error.
A theoretical result further demonstrates
the dependence learning ability of TPGNN.
Related Works
MTS Forecasting There are two main categories of methods for MTS forecasting: implicitdependence approaches and structural-dependence approaches.
One representative method of the ﬁrst category is LSTNet [18], it captures the variable dependence with convolution over variables.
The structural-dependence approaches represent the variable correlation as explicit graphs and predict with GNNs.
Graph Wavenet [40] (GWN) attempts to learn the variable correlation with staticnode representations and captures the temporal pattern with convolution neural networks (CNNs).
MTGNN [39] follows the idea of GWN to learn variable correlation statically and proposes a newgraph convolution module for MTS forecasting.
Although works like GMAN [45], SLC [44], and
StemGNN [4] model a dynamic correlation among variables with self-attention, the learned graph issensitive to the input, causing a signiﬁcant forecasting variance.
Our construction is based on a set oftime-variant coefﬁcients, the result is independent of the input and robust in forecasting.
Some worksthat focus on learning time-varying graph structure [23,32,42]. However, these graph structures arenot captured from the forecast task directly.
They thus cause bias and increase the computation cost.
GNNs and Polynomial Graph Filters (PGFs) GNNs are based on the message-passing mechanism.
One representative work is GraphSAGE [12], which learns a sample-aggregation function to generatethe node embedding.
Michael et al. [6] generalize the convolutional neural networks (CNNs) to thegraph structure. Many ﬁlters of GNNs belong to PGFs, and Gama et al.
[9] discussed the connections. In order to incorporate the node feature of high-order neighbors, many works introduce PGFs to theirmethods.
APPNP [16] and SGC [38] adopt ﬁxed-weight PGFs and use hyperparameters to deﬁnethe coefﬁcients.
However, the constructed ﬁxed-weight PGFs are biased because the coefﬁcientsare task-irrelevant.
GPR-GNN [5] constructs a PGF from the graph dataset, where the coefﬁcientsare learnable.
Besides, there are also some PGFs-based methods for MTS data learning [11,13,14].
Although these works are competitive in capturing graph structure, they fail in learning the dynamiccorrelation, we illustrate the point in Section 5.4.
Moreover, the missing/noisy/incomplete adjacencymatrix causes difﬁculties in applying these methods.
Our approach introduces a self-adaptive graphto the PGFs construction, which improves the dependence capturing ability and the model versatility.
The Framework of TPGNN
[Figure]
We then introduce the module and illustrate the inference pipeline of
3.1 Problem Deﬁntion
[Figure]
At time step t , the MTS data is a signal set G ( t ) =
{V , E ( t ) , X ( t ) , W ( t ) } .
The node set V ( |V| = N )contains N variables of MTS data, E ( t ) is the edge set, X ( t ) ∈ R N × 1 records the signal of Nvariables, and W ( t )
∈ R N × N is a weighted adjacency matrix.
variables i and j are dependent, then an edge connects the two nodes, and entry W ( t ) ij indicates their correlation.
Otherwise, W ( t ) ijequals to zero.
Given T observations of the signal sets, our problem is to ﬁnd a function F that canforecast the variable state of the subsequent T (cid:48) signals:
[Equation]
The setting regards the graph structure as a latent feature of MTS data, and the forecast target is thesame as the standard setting.
Prevailing works [22,39,40] regard W ( t ) as a time-invariant matrix,while in our work, W ( t ) changes with the time.
3.2 Represent the Correlation as a Temporal Matrix Polynomial
To capture E and W t for MTS data, we construct a series of matrix polynomials that share a matrixbasis.
The TPG module controls the inﬂuence of each polynomial term temporally with a set ofadaptive coefﬁcients.
As a result, we can approximate a wide range of dynamic graph structures.
Counstructing the adjacency matrix basis.
We ﬁrstly deﬁne an initial adjacency matrix A ∈ 113 R N × N following the self-adaptive graph proposed by Wu et al.
They learn variable embeddings
E ∈ R N × c ( c is the embedding dimension) from MTS data and deﬁne the variable dependence withembedding similarity.
Besides, they further remove weak connections with ReLU( · ) and normalizethe result with SoftMax( · ) .
[Equation]
For MTS data that have a prior structure, e.g., the physical distance among sensors, we denote thecorresponding adjacency matrix as W ∈ R N × N .
To incorporate W into the encoding process, weﬁrst calculate its symmetric normalized Laplacian [15] L = D − 12 ( I + W ) D − {+
+} ( I is the identitymatrix and D is the degree matrix), then combine the two results to deﬁne the initial adjacency matrix:
[Equation]
Generating the temporal polynomial coefﬁcients.
At time step t , we represent the variable dependence as a K -order matrix polynomial of A :
[Equation]
Due to the polynomial coefﬁcients being time-variant, W ( t ) shows dynamic behavior.
However,learning coefﬁcients for every time step is impractical because the sequence length grows inﬁnitely.
Fortunately, the periodicity property is common in MTS data [8, 26].
Therefore, we propose touse a set of cyclic timestamp embeddings to index the time step.
Firstly, we deﬁne T p timestampembeddings ( e (1) ts , . . .
, e ( T p ) ts ) ∈
R D e , where T p is the cycle and D e is the embedding dimension.
Following a cyclic order, we then assign the ( t % T p ) -th embedding to time step t , where % is theremainder operator.
Finally, we generate the coefﬁcients for time step t by the e ( t % T p ) ts .
As shownin Figure 2 (1), Given T historical graph signals start from time step t , we calculate the coefﬁcientsa ∈ R K +1 for each graph signal with a coefﬁcient matrix W c ∈ R D e ×
( K +1) .
[Equation]
According to Equation 4, each a ( t ) deﬁnes a K -order matrix polynomial of A , representing variabledependence of time step t .
Instead of predicting with T different matrix polynomials, we usean average result to improve the prediction efﬁciency and robustness.
Due to the linearity of thepolynomial, the average polynomial is equivalent to the polynomial with average coefﬁcients.
Wecalculate the ﬁnal coefﬁcients ¯ a with another parameter matrix W a ∈ R T × 1 (average matrix).
[Equation]
In Figure 2 (2), we illusrate the TPG module’s propagation process with ¯ a .
Let X ∈ R N × D e denotethe matrix signal embedding,
W k ∈ R D
e × D e , k = 0 , . . .
, K denote the model parameter matrices,|| · || F denote the Frobenius norm of matrix, we derive the hidden feature Z ( t ) ∈ R N × D e as follows:
[Equation]
TPG module follows the diffusion graph convolution layer [21] to introduce ( K + 1) parametermatrices for enhancing representation diversity.
Besides, we decouple the parameter matrix norm andcoefﬁcient by normalizing W k with a factor of 1 || W k || F , since W k with a large norm increases thek -th term’s contribution.
As a result, ¯ a completely controls the contribution of each term.
3.3 Inference Pipeline
In Figure 2 (3), TPGNN has an encoder-decoder structure.
First, we derive the historical dataembedding with a linear transformation, then feed the results into the encoder.
The encoder generatesdata encoding Z ( t : t + T − 1) ∈ R T × N × D e for time step t to t + T − 1 with the TPG module andtemporal attention layer.
The temporal attention layer follows the design of Transformer [37], whichuses self-attention to capture intra-series patterns for each variable.
Finally, the decoder forecasts withthe data encoding and temporal attention layer.
For the ﬁrst prediction of time step t + T , the decoderqueries the data encoding with a learnable beginning of sentence (BOS) token E BOS ∈ R N × D e .
Thequery result represents the ﬁrst prediction, denoted as E ( t + T ) X ∈ R N × D e .
We use a prediction matrix
W pred ∈ R D e × 1 to get the forecasting result ˜ X ( t + T ) ∈
R N × 1 .
[Equation]
After generating the subsequent k query results ( E ( t + T ) X , . .
, E ( t + T + k − 1) X ) , the decoder derives the( k + 1) -th result E ( t + T + k ) X by querying
Z ( t : t + T − 1) with the latest L max query results.
The forecastbasically follows an auto-regressive (AR) mechanism, where L max is the maximum query length.
[Equation]
The above process continues until getting the subsequent T (cid:48) forecasts, such AR mechanism helps
TPGNN to capture long-term intra-series dependence with informative context.
Theoretical Properties of TPGNN
To explore the dependence learning capability of TPGNN, we focus on analyzing the theoretical gapbetween the optimal graph structure and the TPG module’s approximation in this section.
TPGNN is able to achieve perfect approximation.
Unlike the real function, the approximation ofgraph structure largely depends on the topology property of the target graph, where the lower boundof the approximation error is nonzero in many cases.
However, TPGNN mitigates the gap for a widerange of dynamic graph structures with a small group of parameters, which leads to a robust andaccurate forecasting result.
The approximation capability of TPGNN is concluded as the followingresult, we show the detailed statement and proofs in Appendix A.9 due to the space limitation.
Theorem 1 .
Let G (1) , . .
, G ( T )
∈ R N × N be the symmetric normalized Laplacian of the optimalgraph structure for time step 1 to T , A ∈ R N × N is the initial adjacency matrix of TPGNN.
Wemodel these Laplacians with Equation 4, and the corresponding approximation error e (1: T )
=1 T Σ Tt =1 || W ( t ) − G
( t )
|| 2 F , where the || · || F is the Frobenius norm of the matrix.
Suppose all thematrices are symmetric (undirected graph), G i G j = G j G i , ∀ i, j , A has N different singular values,and the polynomial’s order is large enough.
Then we have the following estimation for e (1: T ) .
[Equation]
where E t || G ( t ) ||
2 F = 1 T Σ Tt =1 || G ( t )
||
2 F is the average norm of the Laplacians, λ min , λ max ∈ [0 , 1]are the minimum and maximum eigenvalues of a constant matrix.
If A is commutative with G ( t ) ,then λ max = λ min = 1 , i.e., we achieve a uniform perfect approximation.
Although the zero errordepends on the commutative condition, A is learnable with Equation 2, which relaxes the restriction.
Moreover, TPGNN empirically achieves the best approximation compared with other methods.
Experiments
5.1 Experimental Setup
We validate the performance of TPGNN on two tasks: single-step and multi-step MTS forecasting.
In
Table 1, we present the statistics of six benchmark datasets.
The ﬁrst four benchmark datasets [2,17]
[Table]
aim to predict a single future step; the last two datasets [21,41] record a distance matrix among sensors,where the goal is to forecast multiple future steps.
More details about the datasets are given in the
AppendixA.3.
For single-step prediction, the baselines are VAR-MLP [43], GP [28], RNN-GRU [39],
LSTNet [18], TPA-LSTM [33], and MTGNN [39].
MTGNN ﬁrstly introduces GNNs to general
MTS data and achieves state-of-the-art (SOTA) performance; other baselines are representativenon-GNNs-based methods.
For multi-step prediction, the baselines are ARIMA [21],FC-LSTM [36],
DCRNN [20], STGCN [41], StemGNN [4], Graph WaveNet, Informer [46], and MTGNN
; all thebaselines are GNN-based except ARIMA, FC-LSTM and Informer.
These GNNs-based baselinesare representative methods for MTS data with a prior structure.
Besides, the Informer is a novel Transformer-based model that achieves SOTA performance on long-term forecasting.
We list theparamter scale of each model in Appendix A.4 to show our method is light-weight.
We use ﬁveevaluation metrics [39]: Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE),
Root Mean Squared Error (RMSE), Root Relative Squared Error (RRSE), and Empirical Correlation Coefﬁcient (CORR).
For CORR, higher values are better.
For the other metrics, lower values are better.
We tune the hyperparameters on the validation dataset, the results are presented in the Appendix A.7.
We divide the dataset into three parts for training, validation, and testing with a ratio of 7:1:2.
Moredetails about the baselines, metrics, and experiment settings are in Appendix A.4A.5A.6.
[Table]
5.2 Main Results
Table 2 summarizes the experimental results of TPGNN.
Generally, TPGNN achieves the stateof-the-art on most of the datasets.
Our framework has an on-par performance for the single-stepforecasting task with the SOTA methods like MTGNN and TPA-LSTM.
Moreover, TPGNN makes asigniﬁcant improvement on Trafﬁc and Electricity datasets.
On the Solar-Energy, TPGNN lowers
RSE by 1 . 61% , 18 .
08% and increases CORR by 1 .
93% over the horizons of 12, 24.
Ourmethod improves the CORR by 2 .
21% on average over the four horizons of the Trafﬁc.
Moreover,we lower the RSE by 15 .
82% on average over the four horizons of Electricity.
Our method fails toachieve SOTA performance on the exchange-rate data.
We think the main reason is the small samplesize, which causes difﬁculties in capturing the dynamic variable dependence.
The second row of
Table 2 concludes the results of the multi-step forecasting task, which demonstrates that TPGNNachieves SOTA performance on the two datasets with prior structure.
On the PEMS-BAY, TPGNNlowers the MAPE by 5 .
47% on average over the three horizons.
Besides, The RMSE is reduced by4 .
41% over the horizons of 3 and 6.
TPGNN reduces the MAE/MAPE by 8 . 04% / 6 .
61% onaverage over the three horizons on the PEMS-D7 dataset.
The RMSE is lowered by 2 .
23%over the horizon of 6 and 12.
Although the prior structure is provided to each baseline, the results onthe last two trafﬁc datasets demonstrate that TPGNN can capture more precise variable dependenceand intra-series patterns than other GNNs-based methods.
5.3 Ablation Experiments
[Table]
The PEMS-D7 includes a prior structure and has complex inter-series and intra-series dependence,we conduct ablation experiments on the PEMS-D7 to evaluate the effectiveness of each design inour method.
Due to the importance and complexity of the long-term prediction, we set the horizonas 12.
More results of other horizons can be found in Appendix A.8.
There are ﬁve experiments intotal, we repeat each experiment 5 times and report the average and standard deviation (std).
w/o
TPG : TPGNN without the TPG module.
We replace the graph constructed by TPG with the priorstructure of PEMS-D7.
w/o dynamic : TPGNN without the time-varying coefﬁcients.
We adopta set of learnable static coefﬁcients to construct the graph.
w/o overview : TPGNN does not useaverage coefﬁcients in Equation 6.
We predict with T graphs deﬁned by Equation 5.
w/o normalize :
TPGNN without the normalization factor in Equation 7.
w/o K-matrices : TPGNN without multipleparameter matrices in Equation 7.
We use an identical normalized matrix to derive the hidden feature.
Table 3 summarizes the experimental results, which indicates that all the designs are indispensable.
The result of w/o TPG illustrates the TPG module’s critical contribution to our method.
Althoughwe replace the graph constructed by the TPG module with the prior structure given by PEMS-D7,the performance has a steep decline by 13 .
23% on the three metrics.
w/o dynamicfollows the strategy of GPR-GNN [5] that uses static coefﬁcients to capture graph structure fromdata.
However, the results indicate the importance of introducing time-variant coefﬁcients, whichsigniﬁcantly improve the long-term prediction accuracy by 7 . 04% on average.
w/o overview andw/o K-matrices show large variance compared with other results, indicating the two modules areessential for improving model robustness.
Besides, the w/o overview’s performance is close to thew/o dynamic, demonstrating the necessity of using the average coefﬁcients.
w/o normalize hascomparable performance to the TPGNN, possibly due to the weight decay causing similar norms.
Nevertheless, it demonstrates that the normalization helps TPGNN capture the variable dependence. [Table] [Figure]
5.4 Study of the Graph Structure Approximation Gap
Synthetic data.
To investigate the empirical gap between the optimal graph structure and theapproximation result, we propose to generate MTS data with ground-truth dynamic dependenceusing a non-repeating random walk (NRW) model [7], which is widely adopted in time-series datageneration [1, 31].
At time step t , we synthesize the matrix signal X ( t ) ∈
R N × 1 with a dynamicweighted adjacency matrix W ( t ) by X ( t ) =
N ( W ( t − 1) X ( t − 1) , σ ) , where N ( · , · ) is the normaldistribution, σ ∈ R controls the variance, and X (0) is sampled from a discrete uniform distribution.
We deﬁne W ( t ) by traversing N w constant matrices ( G (1) , . . .
, G ( N w ) ) in a cyclic order, where each
G ( i ) is a Laplacian of sparsiﬁed random adjacency matrix.
Given a period of T p , T p ≥ N w , wepartition the period into N w even intervals of length T s , and each interval shares one of the matrices,i.e., W ( t ) =
G ([( t % T p ) /T
s ]) , where [ · ] is the rounding function.
We point out that N w controls thecomplexity of dynamic dependence, and N w = 1 corresponds to a static-dependence case.
To increasethe data diversity, we randomly initialize the X ( t ) for every T p step.
We set T p = 120 , σ = 0 .
001in our evaluation and generate six MTS data of length 2400 with N w = 1 , . .
Besides, eachsynthetic dataset is divided into three parts for training, validation, and testing with a ratio of 7:1:2.
More details about the synthetic algorithm, conﬁguration, and synthetic results are in Appendix A.3.
Predicting with different graph approximation methods.
We evaluate the dependence capturingability of different methods on the six synthetic datasets, where the task is the next-step predictionbased on current variable states.
Suppose that ˜ W ( t ) ∈ R N × N , t = 0 , . .
, T − 1 is the learnedadjacency matrix series, we measure the approximation gap with a Mean Frobenius Error (MFE)metric, deﬁned by MFE = 1 TN 2 Σ T − 1 t {+=
+}
|| W t − ˜ W ( t ) || F .
Besides, we use MAE/RMSE to measurethe forecasting accuracy.
The baselines are Graph Wavenet (GWN), MTGNN, GPR-GNN, and
Self-attention.
The ﬁrst three methods are static-dependence-based.
Self-attention constructs adynamic graph, which is adopted in many works like GMAN [45] and StemGNN [4].
We followthe implementation of StemGNN to learn the graph structure with a GRU module and self-attentionoperator in the evaluation.
The polynomial order K is ﬁxed as 2 in the evaluation, we repeat eachexperiment 5 times and report the average/std, more details about the settings are in Appenedix A.6.
[Figure]
The main results are summarized in Figure 3 (a) and Table 4, demonstrating that TPGNN outperformsother baselines on the six synthetic datasets.
In Table 4, TPGNN reduces the graph approximationerror by 23 .
41% on average over the six datasets.
The ﬁrst chart of Figure 3 (a) also illustrates theimprovement, we consistently achieve the best approximation under different N w .
Moreover, the
MFE of TPGNN has a descending trend with the increase of N w while other methods’ approximationerrors are increasing.
We think the TPG module overﬁts on datasets with small N w , since theirbehavior tends to have static dependence.
TPGNN achieves the lowest prediction error on average,the improvement in the accuracy is slight compared with MFE because the task is a simple next-stepprediction.
Furthermore, we observe that the ranking on MFE is identical to the MAE/RMSE, whichdemonstrates the importance of capturing precise variable dependence for MTS forecasting.
The effect of polynomial order.
We further investigate the relation between approximation errorand the polynomial order K , where the K ranges from 1 to 4 , and K = 1 corresponds to a linearapproximation.
The ﬁrst two charts of Figure 3 (b) present the average MFE and correspondingstd, respectively.
The results show that the approximation variance increases signiﬁcantly with K ,especially when K = 4 .
Moreover, TPGNN-K4 has a large MFE on the six datasets compared withother conﬁgurations.
Therefore, we think a high-order matrix polynomial impairs both the precisionand robustness of the dependence capturing ability of TPGNN.
We point out that the observation isnon-trivial since GPR-GNN achieves good results with a large K ( K ≥ 10 ) on normal graph datasets,possibly due to the difference between MTS forecasting and node classiﬁcation.
The third chartshows the prediction error, it also validates that model with low MFE has a low prediction error.
Case study on the real-world dataset.
Due to the absence of ground-truth dependence, we illustratethe correlation modeling ability of TPGNN on real-world datasets with an example of PEMS-D7.
Weﬁnd three highly correlated nodes in the learned result but independent in the prior structure, wherethe node IDs are 60, 93, and 218.
In Figure 4 (a), we mark the physical location of the three nodesaccording to PEMS [3].
The three nodes are on the same road, which illustrates that TPGNN can learnreliable dependence from data.
In Figure 4 (b), we show the learned correlation of pairs (60 , 218) and(93 , 218) .
We observe that (93 , 218) has a stable dependence and (60 , 218) ’s correlation increases inthe evening, the difference possibly caused by the branch road between 218 and 60.
Conclusion
We proposes a novel GNNs-based method for MTS forecasting, TPGNN, which represents the variable dependence as a temporal polynomial matrix.
The results on six real-world datasets illustrate theoutstanding forecasting performance of TPGNN.
Besides, extensive experiments on synthetic datasetsdemonstrate that our method achieves the best dependence approximation than other baselines.
Limitations and future works Although TPGNN achieves outstanding performance on MTSforecasting and dependence capturing, our methods have limitations.
First, the dependence capturedby TPGNN is not a strict causal structure.
Therefore, some correlations are unreliable and decreasethe model’s robustness.
Second, MTS forecasting in the real world is affected by rare events likeweather disasters, which may signiﬁcantly impair prediction accuracy.
The future works focus oncausal learning and transfer learning on MTS data to solve the above limitations.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] see Section [-
 (c)-]6(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] see
Appendix A.1(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes](b) Did you include complete proofs of all theoretical results?
[Yes] see Appendix A.93.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)?
[Yes] see Ap- pendix A.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] see Appendix A.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] see Section 5.3 5.4 and
Appendix A.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] see Appendix A.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes] see Appendix A.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[Yes] see Appendix A.
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[Yes] see Appendix A.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A] not applicable
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A] not applicable
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation