S IM ST: A GNN-F REE S PATIO -T EMPORAL L EARNING F RAMEWORK FOR T RAFFIC F ORECASTING
Anonymous authors
Paper under double-blind review
A BSTRACT
Traffic forecasting is a crucial and challenging problem in smart city efforts.
Spatio-Temporal Graph Neural Networks (STGNNs) have demonstrated great promise and become the de facto solution in this field.
While successful, they require the message passing scheme of GNNs to construct spatial dependencies between nodes, and thus inevitably inherit the notorious inefficiency of GNNs.
Given these facts, in this paper, we propose a simple yet effective GNN-free spatio-temporal learning framework, entitled SimST.
Specifically, our framework replaces GNNs with two feasible and efficient spatial context injectors, which provide proximity and position information, respectively.
SimST is also compatible with various temporal encoding backbones and involves a tailored training strategy.
We conduct extensive experiments on five popular traffic benchmarks to assess the capability of SimST in terms of effectiveness and efficiency.
Experimental results show that such a simple baseline performs surprisingly well.
Using much fewer parameters, SimST not only achieves comparable or better performance than more sophisticated state-of-the-art STGNNs, but also obtains substantial throughput improvements.
I NTRODUCTION
Urban traffic forecasting has emerged as one of the most important components of Intelligent Transportation Systems (ITS).
Given the historical traffic data collected from sensors over road networks, the task is to predict future traffic conditions (e.g., traffic speed, flow), which provides insights for improving urban planning and traffic management (Zheng et al., 2014).
Spatio-Temporal Graph
Neural Networks (STGNNs) have become the de facto most popular tool for traffic forecasting, in which sequential models such as Temporal Convolution Networks (TCN) or Recurrent Neural Networks (RNN) are applied for modeling temporal dependencies (Yu et al., 2018; Li et al., 2018; Pan et al., 2019; Wu et al., 2020), and Graph Neural Networks (GNNs) (Kipf & Welling, 2017; Defferrard et al., 2016) are utilized to capture spatial correlations among different locations, by first aggregating features from sensors’ (nodes) neighbors, and then transforming the aggregated representations via a feed-forward network.
A fundamental problem for using GNNs in traffic modeling is how to establish a graph structure.
After revisiting the STGNNs proposed in recent years, we find that the mainstream approach to build such structures is using either a predefined and sparse adjacency matrix constructed from the distances of the road network between sensors (Yu et al., 2018;
Li et al., 2018; Song et al., 2020), or an adaptive adjacency matrix that is typically built by matrix multiplication of two node embedding tables, producing a fully-connected graph (Wu et al., 2019; Bai et al., 2020; Han et al., 2021).
Due to the capacity of modeling latent spatial relations, adaptive matrix usually produces superior performance than the predefined one (Wu et al., 2019).
Recently, more sophisticated enhancements to spatial learning modules have been proposed to improve predictive performance.
For example, Fang et al.
; Choi et al.
coupled GNNs with neural ordinary differential equations to produce continuous layers for modeling long-range spatio-temporal dependencies.
However, standard GNNs are applied as the basic unit in their methods and they still fall into the above two categories: Fang et al.
(2021) use a predefined matrix and Choi et al.
(2022) apply an adaptive matrix.
While these approaches can successfully improve forecasting performance, they rely heavily on
GNNs for performing the expensive message passing step and thus inevitably inherit or even ex-
GWNET
AGCRN
STGODE
STGNCDE
GMSDR
SimST-WN
SimST-GRU
SimST-CT    800018.
GWNET
AGCRN
STGODE
STGNCDE
GMSDR
SimST-WN
SimST-GRU
SimST-CT 5k 10k14.
Throughput Throughput
M AE on P e
M S D
M AE on P e
M S D
P a r a m ( K )
Throughput means the number of samples the network can process in a second during inference. acerbate GNNs’ notorious inefficiencies (Chen et al., 2021a; Zhang et al., 2022).
Specifically, the predefined matrix inherits the inefficiency of GNNs and has the complexity of O ( E ) , where E is the number of edges.
The adaptive adjacency matrix that has preferable capability aggravates the issue, i.e., it uses a fully-connected graph structure to perform message passing, leading to O ( N 2 ) computational complexity during feature aggregation, where N denotes the number of sensors.
Consequently, the bottleneck from GNNs may hinder the applications of STGNNs in large-scale sensor networks, such as metropolitan cities where traffic forecasting systems are urgently needed.
Although there are plenty of efforts in the graph domain to improve the efficiency of GNNs, such as via graph simplification approaches (Hamilton et al., 2017; Zeng et al., 2020), the related studies in STGNNs are still scarce to the best of our knowledge, as such simplification usually leads to information loss and performance degradation (Wu et al., 2020).
In this study, we aim to investigate a more challenging problem – how to alleviate the inefficiency of STGNNs while preserving or even improving the learning capacity.
Considering the information loss in graph simplification, we approach the problem from a complementary perspective, i.e., a model view to tackle this problem.
In other words, we aim to change the mindset that we must use GNNs for spatial modeling, and deliver a solution that trims the explosive complexity but remains competitive in accuracy.
Present work The key challenge after removing GNNs is how to enable nodes to interact with each other to establish relationships between them, since now all nodes are processed individually by the model.
In this paper, we propose SimST , a simple yet effective spatio-temporal learning framework for traffic forecasting.
Specifically, we propose two feasible and efficient ways to inject spatial context into the model: (1) Spatial position injector.
To allow the model to distinguish between different nodes, we replace GNNs by injecting the model with unique representations of nodes, which are obtained by a non-linear transformation of low-dimensional node embeddings.
During end-to-end training, these node-specific embeddings implicitly capture latent spatial correlations in a data-driven manner and can be viewed as personalized features of each node.
(2) Spatial proximity injector.
Considering that spatial correlations are localized in traffic and closer neighbors usually have greater impact on the anchor node (Li et al., 2018), we incorporate neighbor information by explicitly concatenating the historical observations of each node’s neighbors in the model input.
Moreover, SimST makes no assumption about the temporal encoding backbones and can be easily adapted to models such as GRU (Chung et al., 2014), WaveNet (Oord et al., 2016), and Transformer (Vaswani et al., 2017).
We also devise a tailored training strategy for SimST to further improve performance.
This strategy provides greater flexibility in choosing batch size and increases sample diversity during batch formation.
In summary, our contributions lie in three aspects.
• We present SimST, a conceptually simple yet effective framework.
It replaces GNNs with the proposed spatial injectors that have linear complexity w.r.t N during training and inference.
It is also compatible with various temporal encoders and applies a customized training strategy.
• Extensive experiments demonstrate that SimST performs surprisingly well on traffic forecasting benchmarks, which is a baseline result absent in the literature.
The results also show that message passing is not necessary for effective spatial modelling in traffic forecasting, which calls into question the prevailing practice in this field and opens up new research possibilities.
• As shown in Figure 1, SimST achieves strong empirical results in terms of both accuracy and throughput, while using much fewer parameters.
For example, SimST-WN obtains comparable results to STGNCDE, but is 36 × and 34 × faster than it on PeMSD4 and PeMSD8, respectively.
P RELIMINARIES
We define a sensor network as a graph G =
( V , E , A ) , where V is a set of nodes and |V| = N ,
E is a set of edges and |E| =
E , and A ∈ R N × N is a weighted adjacency matrix.
Following Li et al.
(2018), the entries w ij in A represent the proximity between nodes and are calculated using a thresholded Gaussian kernel (Shuman et al., 2013):
[Equation]
where dist( v i , v j ) denotes the road network distance between sensors, σ is the standard deviation of distances, and r is a threshold.
The graph G has a unique feature matrix X t ∈ R N × F at each time step t , where F is the feature dimensionality.
The features consist of a target attribute (e.g., traffic speed or flow) and other auxiliary information, such as time of day (Wu et al., 2019).
In traffic forecasting, we aim to learn a function f to predict the target attribute of the future T f steps based on T h historical frames:
[Equation] [Equation]
) where X ( t − T h ): t ∈ R T h × N × F indicates the observations from the time step t − T h to t (excluding the right endpoint), and Y t :( t + T f ) ∈ R T f × N × 1 denotes the T f -step ahead predictions.
M ETHODOLOGY
As a typical spatio-temporal data mining task, traffic forecasting requires modelling both the spatial and temporal aspects.
Figure 2 depicts the overall architecture of the proposed SimST framework, which consists of three major components.
We commence by introducing three commonly used temporal encoders to establish the temporal correlations in Section 3.1.
We then describe how spatial dependencies are captured by the two proposed spatial context injectors instead of using GNNs in
Section 3.2.
Finally, we illustrate a tailored training strategy for SimST in Section 3.3.
3. T EMPORAL E NCODING B ACKBONES
There are several viable options for building the temporal dependencies of a node’s history.
We take three basic and popular backbones in this study.
They are GRU (Chung et al., 2014) (RNN-based),
WaveNet (Oord et al., 2016)
(TCN-based), and Causal Transformer (Transformer-based).
We first apply an input MLP to map X ∈ R T h × N × F to the input representation H in ∈ R T h × N × D m , where D m is the model’s hidden dimension size.
Then, this representation is fed into the temporal encoder.
For GRU and WaveNet, they compress the temporal dimension to 1, generating the output representations H T ∈ R N × D m , which can be seen as a temporal summary of each node.
In contrast, for Causal Transformers, the output representations H T ∈ R T h × N × D m still have the same shape as H in .
Here, we take the representation of the last time step as the temporal summary.
Moreover, since the standard Transformer is aware of all input time steps during self-attention operations and the traffic status at the current step is in fact not conditioned on its future, we follow
WaveNet to introduce causality into the attention mechanism.
This ensures that the model cannot violate the temporal order of inputs; such causality can be easily implemented by masking the specific entries in the attention map.
3. S PATIAL C ONTEXT I NJECTORS
The inefficiency of GNNs has been extensively discussed within the graph domain (Chiang et al., 2019; Chen et al., 2021a; Zhang et al., 2022).
Utilizing GNNs to explicitly set up spatial correlations via message passing, STGNNs also suffer from flagrant inefficiencies.
In this work, we propose two simple alternatives to allow each node to exploit spatial information without message passing.
[Figure]
Spatial Position Injector Inspired by positional embeddings in Transformers, we randomly initialize a low-dimensional node embedding table E ∈ R N × D n , where D n is the node embedding size.
This can be seen as the personalized features possessed by each node.
Apart from this, all other trainable parameters in SimST are shared across nodes.
We then apply a non-linear transformation to map E to high-dimensional representations H S ∈ R N × D m and concatenate H S with H T to form the final spatio-temporal representations H ST ∈ R N × 2 D m , which will be fed into the output
MLP to produce the forecasting results.
Formally, we have:
[Equation]
During end-to-end training, the embedding table learns the latent spatial relationships between nodes in a data-driven manner.
For example, two distant nodes may be inherently correlated because they are both located around residential areas.
Then, they may share similar traffic patterns and similar inputs.
Such long-range dependencies can be implicitly learned by the model and reflected in their node embeddings’ similarity.
A further study of the learned embeddings is shown in Section 4.5.
Spatial Proximity Injector
According to Tobler’s First Law of Geography, near things are more related than distant things.
We note that the spatial position injector does not consider the information from the physical neighbors of each node.
To complement this, in this component we inject neighbor information by concatenating the histories of each node’s neighbors in the model input.
Specifically, we utilize the predefined adjacency matrix A to obtain the neighbor information.
We also consider the neighbors in the reversed direction, i.e., the entries in A T , so that the model can perceive the impact from the upstream and downstream traffic directions.
Since closer neighbors usually have greater correlation with the anchor node (Li et al., 2018), for each node v i , we concatenate the historical observations from its top- k nearest neighbors with the original observations to highlight their influence.
In addition, we also combine the average of the histories of all v
i ’s hop-one neighbors, allowing the anchor node to learn about its local context.
We simplify the time period notation from ( t − T h ) : t to T p here.
Formally,
[Equation]
where v u 1 , ..., v u k and avg denote v
i ’s top-
k neighbors and average of hop-one neighbors in A .
The prime in the notation (e.g., v u ′ 1 ) indicates neighbors from A T .
Note that this injector operates in the data preprocessing stage, so it has no influence on model training and inference.
Complexity Comparison We provide a complexity comparison between the spatial learning modules here.
As GCN (Kipf & Welling, 2017) is the widely adopted form of GNNs in spatiotemporal models, we take it as a baseline.
Suppose a L -layer GCN with fixed hidden features
D m are applied.
For STGNNs applying the predefined adjacency matrix, the time complexity is
O ( LED m + LND 2 m ) .
When applying the adaptive adjacency matrix that can boost performance, the complexity becomes O
( LN 2 D m + LND 2 m ) .
In SimST, the complexity of position injector is
O ( ND n D m ) , and the complexity of proximity injector is O ( NR ) , where R is the average degree of nodes.
It is obvious that SimST has lower complexity and is capable of supporting traffic forecasting applications on large-scale sensor networks.
3. T RAINING S TRATEGY FOR S IM ST
When training an STGNN, the input data are usually a four-dimensional tensor X ( t − T h ): t ∈ R B × T h × N × F , where B denotes the set batch size.
That said, the entire graph is required to be incorporated into a batch, so that the nodes can interact with each other via message passing.
However, we identify two main issues with this strategy.
First, since SimST processes each node individually, the actual batch size B ∗ is equal to B × N .
For example, there are 307 nodes in the PeMSD4 dataset (Song et al., 2020), and usually B is set to 64, then B ∗ = 19648 .
Such a large number makes the gradient too exact and affects the effectiveness of stochastic gradient decent.
Also, when N is large, it can easily lead to out-of-memory problems.
Second, we argue that the sample diversity when forming mini-batches is not enough: for example, the traffic flow of all nodes from 8 a.m. to 9 a.m. on Sept. 28 will always appear in the same batch.
To alleviate the above issues, we simply combine the batch and node dimensions to form a large pool, and then sample batches in this pool during training.
Thus, the input data to SimST are
X ( t − T h ): t ∈ R B ∗ × T
h
× F and B ∗ ≪ B × N .
The benefits are two-fold.
First, our solution provides flexibility to choose the actual batch size, which can be disproportionate to N , thus allowing for smaller values like 256.
This property allows SimST to have low GPU memory costs and to easily scale to large graphs.
Second, we provide more sample diversity and randomness when creating the batches.
E XPERIMENTS 4. E XPERIMENTAL S ETUP
Datasets We conduct experiments on commonly used traffic benchmarks.
The details are presented in Table 1.
All traffic readings are aggregated into 5-minute windows, resulting in 288 data points per day.
Note that the values in traffic flow datasets are generally much larger than those in speed-related datasets.
Following Li et al.
; Song et al.
; Choi et al.
(2022), we use the 12-step historical data to predict the next 12 steps.
The datasets are divided into three parts for training, validation, and test with a ratio of 6:2:2 on PeMSD4, PeMSD7, PeMSD8, and 7:1: on LA and BAY.
Z-score normalization is applied to the input data for fast training.
We build the predefined adjacency matrix of each dataset by using road network distances with the thresholded
Gaussian kernel method (Shuman et al., 2013).
The threshold r is set to 0 for PeMSD4, PeMSD7,
PeMSD8, and 0.1 for LA and BAY.
More information are provided in Appendix A.
[Table]
Baselines We compare SimST with the following baselines.
Historical Average (HA) (Pan et al., 2012) and Vector Autoregression (VAR)
(Toda, 1991) are traditional methods.
DCRNN (Li et al., 2018), STGCN (Yu et al., 2018) , ASTGCN (Guo et al., 2019), GWNET (Wu et al., 2019), STSGCN (Song et al., 2020), and AGCRN (Bai et al., 2020) are well-known STGNNs.
We also incorporate methods that are published recently, namely, STGODE (Fang et al., 2021), STGNCDE (Choi et al., 2022), and GMSDR (Liu et al., 2022).
Implementation Details SimST is implemented with PyTorch 1.12.
There are three variants of
SimST based on the applied temporal encoders.
We describe the specific configurations for different variants in Appendix A.2.
The following settings are the same for all variants.
We train our method via the Adam optimizer with an initial learning rate of 0.001 and a weight decay of 0.0001.
We set the maximum epochs to 150 and use an early stop strategy with the patience of 20.
The batch size is 1,024.
The node dimension D n is 20.
The top- k nearest neighbors is set to 3 for LA and BAY, and 0 for PeMSD4, PeMSD7, and PeMSD8 (due to limited available edges). More details of k are provided in Appendix A.4.
For baselines, we run their codes based on the recommended configurations if their accuracy is not known for a dataset.
If known, we use their officially reported accuracy.
Experiments are repeated five times with different seeds on an NVIDIA RTX A6000 GPU.
Evaluation Metrics We adopt three common metrics in forecasting tasks to evaluate the model performance, including mean absolute error (MAE), root mean squared error (RMSE), and mean absolute percentage error (MAPE).
For efficiency measurement, the commonly used floating points of operations (FLOPs) cannot reflect the “real” running speed of methods because it ignores the effects of parallelization.
Hence, we use a more reasonable metric, throughput per second (TPS), to measure the efficiency in this study.
In particular, TPS indicates the average number of samples the network can process in one second during inference.
[Table] [Table]
4. P ERFORMANCE C OMPARISONS
In this subsection
[Table]
we conduct model comparisons between SimST variants and state-of-the-art
STGNNs on five real-world traffic benchmarks.
The three variants of SimST are SimST-WaveNet (WN), SimST-GRU, and SimST-Causal Transformer (CT).
As shown in Table 2 and 3, all variants of SimST achieve comparable or better performance than all competing baselines on the three evaluation metrics.
This is a surprising discovery, since using GNNs to model spatial dependencies is a common practice in traffic forecasting.
The results also demonstrate that: (1) SimST is temporalmodel-agnostic, and (2) it is viable and sufficient to use the proposed spatial context injectors to establish spatial correlations between nodes.
We next compare only among SimST variants: note that for fairness we ensure that the variants have a similar number of parameters (around 150k).
From the table
[Table]
we observe that SimST-CT performs better than the other two variants in 3 of thedatasets, which can be attributed to Transformers’ strong learning capability.
At the same time
[Table]
it is also surprising that integrating SimST with a simple GRU can achieve competitive performance, particularly on PeMSD7 and BAY.
Additionally, we observe that: (1) STGNNs approaches surpass
HA and VAR by a large margin due to their greater learning capacity.
The capability of methods that apply the adaptive matrix, such as GWNET and AGCRN, have been overlooked.
They have achieved similar performance to the recently proposed approaches
[Table] [Table]
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
4. E FFICIENCY C OMPARISONS
In this part, we select the top-performing approaches and compare their efficiency with SimST variants.
It can be seen from Table 4 that all SimST variants are significantly more efficient than baselines, thanks to their simple model architecture and linear time complexity w.r.t N .
Specifically, comparing between TCN-based models, SimST-WN has around 45% fewer parameters than
GWNET, but achieves 3.3 × - 5.6 × higher TPS.
For RNN-based methods, SimST-GRU is 3.7 × - 6.1 × and 19 × - 26 × faster than AGCRN and GMSDR, respectively.
STGODE and STGNCDE are the recently published methods, which are based on neural ordinary differential equations.
Though achieving competitive performance, they suffer from non-ignorable inefficiency problem.
SimSTWN has comparable performance to them, but is 11 × - 17 × and 30 × - 39 × faster.
Among the variants, SimST-WN achieves the highest TPS due to WaveNet’s superior parallelism capability.
SimST-GRU also achieves good TPS results, which we attribute to an optimized implementation in the PyTorch library.
See Appendix A.3 for more results on efficiency comparisons.
4. A BLATION S TUDIES
We have verified the superiority of SimST against STGNNs baselines.
Next, we select SimST-CT and SimST-GRU as the representatives due to their preferable performance and conduct a series of ablation studies on one flow-based dataset PeMSD4 and one speed-related dataset LA.
Effects of spatial injectors.
To study the effects of two spatial injectors, we consider the following three variants for comparisons: (1) w/o Position : we turn off the injection of spatial position representations.
(2) w/o Proximity : for each node, we only input the histories of itself.
(3) CT/GRU: we do not inject any spatial information.
The results are shown in Figure 3.
First, we find that removing position injector leads to a significant degradation on MAE for both SimST-CT and SimST-GRU on both datasets, revealing the great importance of distinguishing different nodes in the models.
Second, the benefit of proximity injector is marginal on the PeMSD4 dataset.
The reason is that the average degree of nodes in PeMSD4 is 1.1 and thus the available neighbor information is very limited.
In contrast, the average degree of nodes in the LA dataset is 7.3, where the proximity injector can have a larger impact on performance.
The results on the PeMSD4 dataset also suggest that on some real-world datasets, even without knowing the neighbor information, the model can achieve very competitive result by only perceiving the latent relationships provided by the position injector.
T e s t
M [Figure] E (a) PeMSD
CT w/o Position w/o Proximity
SimST-CT 2.
(b) LA      (c) PeMSD GRU w/o Position w/o Proximity
SimST-GRU 2.
(d) LA
Effects of training strategy.
We examine the influence of the batch size number in Figure 4 (a) and (b).
Generally, both SimST-CT and SimST-GRU reach their best performance when setting B ∗ to 1,024 on the two datasets, revealing the necessity of applying small B ∗ .
The influence of using a large batch size such as 19,648, is more significant on the PeMSD4 dataset.
Besides, we notice that SimST-CT only occupies around 2GB of memory on GPU across all datasets.
In contrast, the memory cost of STGNNs usually grows as N increases.
For example, on the dataset with the highest
N , PeMSD7 (883 nodes), and the lowest N , PeMSD8 (170 nodes), GWNET requires around 11GB and 3GB of memory, respectively.
Furthermore, we investigate the effects of different batch forming methods in Table 5, where a SimST-CT model is trained by the strategies applied in STGNNs and
SimST, respectively.
It can be seen that under a similar actual batch size B ∗ , the strategy of SimST surpasses the counterpart on both datasets.
This result verifies our speculation in Section 3.3 that offering more sample diversity during batch establishment can improve performance.
[Figure]
[Table]
Effects of node dimension size.
We assess the effects of the node dimension size in Figure 4 (c) and (d).
We observe that both SimST-CT and SimST-GRU are not sensitive to the changes of the dimension size on both datasets.
The results also suggest that a small dimension such as 16, is sufficient to record the latent correlations between nodes.
4. C ASE S TUDY
We further conduct a case study to investigate the effectiveness of the learned node embeddings.
Figure 5 (a) shows the embedding visualization by t-SNE (Van der Maaten & Hinton, 2008).
Due to page limits, we randomly select a region where node 43, 82, and 130 are located in.
We also plot the nodes’ raw traffic data in sub-figure (b) and (c).
It can be seen that the three nodes are not only similar in the embedding space, but also share similar input traffic patterns.
Moreover, by examining the adjacency matrix, we know that node 43’s neighbors include node 81, 82, and 158.
However in (a), two neighbors 81 and 158 are far from node 43.
Also, we can see from (b) and (c) that the patterns of 81 and 158 have significant discrepancy to node 43.
These results indicate that the learned node embeddings can reflect the similarity in the input space.
Moreover, given the fact that our node embeddings are not generated from input features, we conclude that the node embeddings have learned the inherent relations between nodes and SimST does work in a data-driven manner, i.e., nodes with similar patterns are likely to have similar node embeddings.
[Figure]
M ORE R ELATED W ORK
Traffic forecasting is a crucial application in smart city efforts.
In recent years, Spatial-Temporal
Graph Neural Networks (STGNNs) have become the most widely used tools for predicting traffic (Cao et al., 2020; Li & Zhu, 2021;
Chen et al., 2021b; 2022).
Generally, they integrate GNNs with either RNNs or TCNs to capture the spatial and temporal dependencies in traffic data.
For example, DCRNN (Li et al., 2018) considered traffic flow as a diffusion process and combined a novel diffusion convolution with GRU.
The following efforts (Pan et al., 2019; Bai et al., 2020;
Fang et al., 2021; Liu et al., 2022) were also based on RNNs.
To improve training speed and enjoy parallel computation, plenty of works (Wu et al., 2019; 2020; Li & Zhu, 2021; Han et al., 2021) replaced RNNs with dilated causal convolution operations.
A prerequisite for using GNNs effectively, however, is to have a high-quality adjacency matrix.
In traffic forecasting, there are mainly two approaches, namely predefined adjacency matrix and adaptive adjacency matrix.
Specifically, the former is usually a sparse matrix constructed from road network distances, and the latter is a dense matrix that records the pairwise relationships of nodes.
For a comparison between the two matrices, researchers have pointed out that the predefined matrix is heuristic and cannot reflect the genuine dependencies between nodes, which negatively affects model performance (Wu et al., 2019; Bai et al., 2020).
Indeed, we notice that the models applying the adaptive matrix achieve superior performance (Wu et al., 2019; Bai et al., 2020; Choi et al., 2022).
Though successful, the adaptive matrix discards the sparsity of the graph and incurs quadratic computational cost, making it difficult for models using it to scale to large-scale traffic applications.
In addtion, we note that there are also methods that use attention mechanism for spatial learning (Zheng et al., 2020).
But they also suffer from quadratic time complexity.
In this study, we propose
SimST to alleviate this inefficiency issue, which removes the inefficient GNNs component and thus has linear time complexity.
C ONCLUSION AND F UTURE W ORK
In this paper, we propose SimST, a simple, effective, and efficient spatio-temporal learning framework for traffic forecasting.
It replaces GNNs with two proposed spatial injectors, involves a tailored training strategy, and is temporal-model-agnostic.
Our study suggests that message passing is not the only effective way of modelling spatial relations in traffic forecasting; hence, we hope to spur the development of new model designs with both high efficiency and effectiveness in the community.
We also would like to point out several potential future directions.
First, the node-specific representations in SimST are not varying over time.
Thus, a possible direction for improving performance is to make it change dynamically with time.
Second, though SimST can generalize to newly added sensors easily, it still needs to be fine-tuned over the new nodes and is not in a fully inductive manner.
A future study may focus on developing a completely inductive spatio-temporal model.
Third, our finding and analysis in this work are limited to traffic forecasting currently, a potential future work would be to expand SimST to other spatio-temporal applications such as air quality prediction.
A A PPENDIX A. D ATASETS
We conduct experiments on the traffic datasets of PeMSD4 1 , PeMSD7 1 , PeMSD8 1 , LA 2 and BAY 2 .
• PEMS-04 (Song et al., 2020):
The dataset refers to the traffic flow data in San Francisco Bay Area.
There are 307 sensors and the period of data ranges from Jan. 1 - Feb. 28, 2018.
• PEMS-07 (Song et al., 2020):
This dataset involves traffic flow readings collected from 883 sensors, and the time range is from May 1 - Aug. 6, 2017.
• PEMS-08 (Song et al., 2020):
The dataset contains traffic flow information collected fromsensors in the San Bernardino area from Jul. 1 - Aug. 31, 2016.
• LA (Li et al., 2018):
The dataset records the traffic speed information collected from 207 loop detectors in the highway of Los Angeles County, ranging from Mar. 1 - Jun. 27, 2012.
• BAY (Li et al., 2018):
This dataset contains traffic speed information from 325 sensors in the Bay Area.
It has 6 months of data ranging from Jan. 1 - Jun. 30, 2017.
We use one-hour (12-step) historical data to predict the future one-hour (12-step) values.
The datasets are split into three parts for training, validation, and testing with a ratio of 6:2:2 on PeMSD4,
PeMSD7, and PeMSD8, and 7:1:2 on LA and BAY.
The statistics of data partitions is in Table 6.
[Table]
A. C ONFIGURATIONS OF S IM
[Table]
[Table]
T RAINING AND I NFERENCE T IME C OMPARISONS
In Table
[Table]
we provide the training and inference time per epoch of SimST and baselines, to enable straightforward efficiency comparisons.
It can be seen that SimST variants have significantly shorter inference time.
For the training efficiency, GWNET and AGCRN generally have shorter training time than SimST.
The reason is that SimST applies a small actual batch size B ∗ = 1 , 024 .
[Table]
A. I MPACT OF T OP - K N EIGHBORS
In Figure
[Table]
we show the effects of k on the LA dataset within the range of { 0, 1, 2, 3, 4 } .
For the
PeMSD4 dataset, it is not necessary to do so because the average degree of nodes is only 1.1.
[Figure]
[Figure]
[Figure]
L EARNING C URVE C OMPARISONS
In Figure
[Figure]
we compare the learning curves between some baselines and the variants of SimST.
It can be seen that all SimST variants exhibit very fast convergence rates, dropping to a small validation MAE within a few epochs.
In addition, we also observe that the standard deviation of SimST is generally smaller than those in baselines, indicating the additional stability provided by our approach.
[Figure]
A. V ISUALIZATION
To have a better sense of the capacity of our method
[Figure]
in this part
[Figure]
we randomly select two nodes and visualize the forecasting results of the SimST-CT model and the corresponding ground truths.
[Figure]
[Figure] [Figure]