Original: D IFFUSION C ONVOLUTIONAL R ECURRENT N EURAL N ETWORK : D ATA -D RIVEN T RAFFIC F ORECASTING
Revised: D IFFUSION C ONVOLUTIONAL R ECURRENT N EURAL N ETWORK : D ATA -D RIVEN T RAFFIC F ORECASTING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Spatiotemporal forecasting has various applications in neuroscience, climate and transportation domain.
Revised: Spatiotemporal forecasting has various applications in neuroscience, climate and transportation domain.

Original: Trafﬁc forecasting is one canonical example of such learning task.
Revised: Trafﬁc forecasting is one canonical example of such learning task.

Original: The task is challenging due to (1) complex spatial dependency on road networks, (2) non-linear temporal dynamics with changing road conditions and (3) inherent difﬁculty of long-term forecasting.
Revised: The task is challenging due to (1) complex spatial dependency on road networks, (2) non-linear temporal dynamics with changing road conditions and (3) inherent difﬁculty of long-term forecasting.

Original: To address these challenges, we propose to model the trafﬁc ﬂow as a diffusion process on a directed graph and introduce Diffusion Convolutional Recurrent Neural Network (DCRNN), a deep learning framework for trafﬁc forecasting that incorporates both spatial and temporal dependency in the trafﬁc ﬂow.
Revised: To address these challenges, we propose to model the trafﬁc ﬂow as a diffusion process on a directed graph and introduce Diffusion Convolutional Recurrent Neural Network (DCRNN), a deep learning framework for trafﬁc forecasting that incorporates both spatial and temporal dependency in the trafﬁc ﬂow.

Original: Speciﬁcally, DCRNN captures the spatial dependency using bidirectional random walks on the graph, and the temporal dependency using the encoder-decoder architecture with scheduled sampling.
Revised: Speciﬁcally, DCRNN captures the spatial dependency using bidirectional random walks on the graph, and the temporal dependency using the encoder-decoder architecture with scheduled sampling.

Original: We evaluate the framework on two real-world large scale road network trafﬁc datasets and observe consistent improvement of 12% - 15% over state-of-the-art baselines.
Revised: We evaluate the framework on two real-world large scale road network trafﬁc datasets and observe consistent improvement of 12% - 15% over state-of-the-art baselines.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Spatiotemporal forecasting is a crucial task for a learning system that operates in a dynamic environment.
Revised: Spatiotemporal forecasting is a crucial task for a learning system that operates in a dynamic environment.

Original: It has a wide range of applications from autonomous vehicles operations, to energy and smart grid optimization, to logistics and supply chain management.
Revised: It has a wide range of applications from autonomous vehicles operations, to energy and smart grid optimization, to logistics and supply chain management.

Original: In this paper, we study one important task: trafﬁc forecasting on road networks, the core component of the intelligent transportation systems.
Revised: In this paper, we study one important task: trafﬁc forecasting on road networks, the core component of the intelligent transportation systems.

Original: The goal of trafﬁc forecasting is to predict the future trafﬁc speeds of a sensor network given historic trafﬁc speeds and the underlying road networks.
Revised: The goal of trafﬁc forecasting is to predict the future trafﬁc speeds of a sensor network given historic trafﬁc speeds and the underlying road networks

Original:  similar to road 2 as they locate in the same highway
Revised: (1) Trafﬁc speed in road 1 are similar to road 2 as they locate in the same highway.

Original: (2) Road 1 and road 3 locate in the opposite directions of the highway.
Revised: (2) Road 1 and road 3 locate in the opposite directions of the highway.

Original: Though close to each other in the Euclidean space, their road network distance is large, and their trafﬁc speeds differ signiﬁcantly.
Revised: Though close to each other in the Euclidean space, their road network distance is large, and their trafﬁc speeds differ signiﬁcantly.

Original: This task is challenging mainly due to the complex spatiotemporal dependencies and inherent difﬁculty in the long term forecasting.
Revised: This task is challenging mainly due to the complex spatiotemporal dependencies and inherent difﬁculty in the long term forecasting.

Original: On the one hand, trafﬁc time series demonstrate strong temporal dynamics .
Revised: On the one hand, trafﬁc time series demonstrate strong temporal dynamics .

Original: Recurring incidents such as rush hours or accidents can cause nonstationarity, making it difﬁcult to forecast longterm.
Revised: Recurring incidents such as rush hours or accidents can cause nonstationarity, making it difﬁcult to forecast longterm.

Original: On the other hand, sensors on the road network contain complex yet unique spatial correlations .
Revised: On the other hand, sensors on the road network contain complex yet unique spatial correlations .

Original: Figure 1 illustrates an example.
Revised: Figure 1 illustrates an example.

Original: Roadsegments 1 and  2 are correlated,  road 3 is not.
Revised: Road 1 and road 2 are correlated, while road 1 and road 3 are not.

Original: Although  1 and 3 are close in the Euclidean space, they demonstrate very different behaviors.
Revised: Although road 1 and roadare close in the Euclidean space, they demonstrate very different behaviors.

Original: Moreover, the future trafﬁc speed is inﬂuenced more by the downstream trafﬁc than the upstream one.
Revised: Moreover, the future trafﬁc speed is inﬂuenced more by the downstream trafﬁc than the upstream one.

Original: This shows that the spatial structure in trafﬁc is non-Euclidean and directional.
Revised: This means that the spatial structure in trafﬁc is nonEuclidean and directional.

Original: Trafﬁc forecasting has been studied for decades, falling into two main categories: knowledgedriven approach and data-driven approach.
Revised: Trafﬁc forecasting has been studied for decades, falling into two main categories: knowledgedriven approach and data-driven approach.

Original: In transportation and operational research, knowledge-driven methods usually apply queuing theory and simulate user behaviors in trafﬁc (Cascetta, 2013).
Revised: In transportation and operational research, knowledgedriven methods usually apply queuing theory and simulate user behaviors in trafﬁc (Cascetta, 2013).

Original: In time series community, data-driven methods such as autoregressive integrated moving average (ARIMA) model and Kalman ﬁltering remain popular (Liu et al., 2011; Lippi et al., 2013).
Revised: In time series community, data-driven methods such as Auto-Regressive Integrated Moving Average (ARIMA) model and Kalman ﬁltering remain popular (Liu et al., 2011; Lippi et al., 2013).

Original: However, simple time series models usually rely on the stationarity assumption, which is often violated by the trafﬁc data.
Revised: However, simple time series models usually rely on the stationarity assumption, which is often violated by the trafﬁc data.

Original: Most recently, deep learning models for trafﬁc forecasting have been developed in Lv et al.
Revised: Most recently, deep learning models for trafﬁc forecasting have been developed in Lv et al.

Original: Yu et al.
Revised: Yu et al.

Original: (2017b), but without considering the spatial structure.
Revised: (2017b), but without considering the spatial structure.

Original: Wu & Tan (2016) and Ma et al.
Revised: Wu & Tan (2016) and Ma et al.

Original: (2017) model the spatial correlation with Convolutional Neural Networks (CNN), but the spatial structure is in the Euclidean space (e.g., 2D images).
Revised: (2017) model the spatial correlation with Convolutional Neural Networks (CNN), but the spatial structure is in the Euclidean space (e.g., 2D images).

Original: Bruna et al.
Revised: Bruna et al.

Original: (2014), Defferrard et al.
Revised: (2014), Defferrard et al.

Original: (2016) studied graph convolution, but only for undirected graphs.
Revised: (2016) studied graph convolution, but only for undirected graphs.

Original: In this work, we represent the pair-wise spatial correlations between trafﬁc sensors using a directed graph with sensors as nodes  and  road network distances as edge weights.
Revised: In this work, we represent the pair-wise spatial correlations between trafﬁc sensors using a directed graph whose nodes are sensors and edge weights denote proximity between the sensor pairs measured by the road network distance.

Original: We model the dynamics of the trafﬁc ﬂow as a diffusion process and propose the diffusion convolution operation to capture the spatial dependency. 
Revised: We model the dynamics of the trafﬁc ﬂow as a diffusion process and propose the diffusion convolution operation to capture the spatial dependency. We further propose

Original: Our Diffusion Convolutional Recurrent Neural Network (DCRNN)  integrates diffusion convolution ,  sequence to sequence architecture and the scheduled sampling technique.
Revised: Diffusion Convolutional Recurrent Neural Network (DCRNN) that integrates diffusion convolution , the sequence to sequence architecture and the scheduled sampling technique.

Original: When evaluated on real-world trafﬁc datasets, DCRNN consistently outperforms state-of-the-art trafﬁc forecasting baselines by a large margin.
Revised: When evaluated on realworld trafﬁc datasets, DCRNN consistently outperforms state-of-the-art trafﬁc forecasting baselines by a large margin.

Original: In summary:
Revised: In summary:

Original: • We study  trafﬁc forecasting problem andpropose to model the spatial dependency of trafﬁc using diffusion-based convolution on a  graph.
Revised: • We study the trafﬁc forecasting problem and model the spatial dependency of trafﬁc as a diffusion process on a directed graph.

Original: This formulation of convolution on the directed graph has an intuitive interpretation and can be efﬁciently calculated.
Revised: We propose diffusion convolution , which has an intuitive interpretation and can be computed efﬁciently.

Original: • We propose Diffusion Convolutional Recurrent Neural Network (DCRNN), a holistic approach that captures both spatial and temporal dependencies among time series using diffusion convolution and  sequence to sequence learning framework together with scheduled sampling.
Revised: • We propose Diffusion Convolutional Recurrent Neural Network (DCRNN), a holistic approach that captures both spatial and temporal dependencies among time series using diffusion convolution and the sequence to sequence learning framework together with scheduled sampling.

Original: DCRNN is not limited to transportation and is readily applicable to other spatiotemporal forecasting tasks.
Revised: DCRNN is not limited to transportation and is readily applicable to other spatiotemporal forecasting tasks.

Original: • We conducted extensive experiments on two large-scale real-world datasets, and the proposed approach obtains signiﬁcant improvement over state-of-the-art baseline methods.
Revised: • We conducted extensive experiments on two large-scale real-world datasets, and the proposed approach obtains signiﬁcant improvement over state-of-the-art baseline methods.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: We formalize the learning problem of spatiotemporal trafﬁc forecasting and describe how to model the dependency structures using diffusion convolutional recurrent neural network .
Revised: We formalize the learning problem of spatiotemporal trafﬁc forecasting and describe how to model the dependency structures using diffusion convolutional recurrent neural network .

Original: 2. T RAFFIC F ORECASTING P ROBLEM
Revised: 2. T RAFFIC F ORECASTING P ROBLEM

Original: The goal of trafﬁc forecasting is to predict the future trafﬁc speed given previously observed trafﬁc ﬂow from N correlated sensors on the road network.
Revised: The goal of trafﬁc forecasting is to predict the future trafﬁc speed given previously observed trafﬁc ﬂow from N correlated sensors on the road network.

Original: We can represent the sensor network as a weighted directed graph G =
Revised: We can represent the sensor network as a weighted directed graph G =

Original: ( V , E , W ) , where V is a set of nodes |V| =
Revised: ( V , E , W ) , where V is a set of nodes |V| =

Original: N , E is a set of edges and
Revised: N , E is a set of edges and

Original: W ∈ R N × N is a weighted adjacency matrix representing the nodes proximity (e.g., a function of their road network distance).
Revised: W ∈ R N × N is a weighted adjacency matrix representing the nodes proximity (e.g., a function of their road network distance).

Original: Denote the trafﬁc signal on  a graph at time t as X( t ) ∈ R N × P , where P is the number of features  (e.g., velocity, volume).
Revised: Denote the trafﬁc ﬂow observed on G as a graph signal X ∈ R N × P , where P is the number of features of each node (e.g., velocity, volume).

Original: The trafﬁc forecasting problem aims to learn a function h ( · ) that maps T (cid:48) historical observations to the next T time steps, given a graph G :
Revised: Let X ( t ) represent the graph signal observed at time t , the trafﬁc forecasting problem aims to learn a function h ( · ) that maps T (cid:48) historical graph signals to future T graph signals, given a graph G :

Original: [Equation]
Revised: [Equation]

Original: Note that the graph G is ﬁxed and the signals at each node X are dynamic.

Original: 2. S PATIAL D EPENDENCY M ODELING
Revised: 2. S PATIAL D EPENDENCY M ODELING

Original: We model the spatial dependency by relating trafﬁc ﬂow to a diffusion process, which explicitly captures the stochastic nature of trafﬁc dynamics.
Revised: We model the spatial dependency by relating trafﬁc ﬂow to a diffusion process, which explicitly captures the stochastic nature of trafﬁc dynamics.

Original: This diffusion process is characterized by an initial probability α ∈ [0 , 1] , and a state transition matrix D − 1 O W .
Revised: This diffusion process is characterized by a random walk on G with restart probability α ∈ [0 , 1] , and a state transition matrix D − 1 O W .

Original: HereD O = diag
Revised: Here

Original: ( W 1 ) is the out-degree diagonal matrix, and 1 ∈ R N denotes the all one vector.
Revised: D O = diag( W 1 ) is the out-degree diagonal matrix, and 1 ∈ R N denotes the all one vector.

Original: After many time steps, such Markov process converges to a stationary distribution P ∈ R N × N whose i th row P i, : ∈ R N represents the likelihood of diffusion from node v i ∈ V , hence the proximity w.r.t the node v i .
Revised: After many time steps, such Markov process converges to a stationary distribution P ∈ R N × N whose i th row P i, : ∈ R N represents the likelihood of diffusion from node v i ∈ V , hence the proximity w.r.t. the node v i .

Original: The following Lemma provides a closed form solution for the stationary distribution.
Revised: The following Lemma provides a closed form solution for the stationary distribution.

Original: Lemma 2.1.
Revised: Lemma 2.1.

Original: (Teng et al., 2016)
Revised: (Teng et al., 2016)

Original: The stationary distribution of the diffusion process can be represented as a weighted combination of inﬁnite random walks on the graph, and be calculated in closed form:
Revised: The stationary distribution of the diffusion process can be represented as a weighted combination of inﬁnite random walks on the graph, and be calculated in closed form:

Original: [Equation]
Revised: [Equation]

Original: where k is the diffusion step.
Revised: where k is the diffusion step.

Original: In practice, we use a ﬁnite K -step truncation of the diffusion process and assign a trainable weight to each step.
Revised: In practice, we use a ﬁnite K -step truncation of the diffusion process and assign a trainable weight to each step.

Original: We also include the reversed direction diffusion process, such that the bidirectional diffusion offers the model more ﬂexibility to capture the inﬂuence from both the upstream and the downstream trafﬁc.
Revised: We also include the reversed direction diffusion process, such that the bidirectional diffusion offers the model more ﬂexibility to capture the inﬂuence from both the upstream and the downstream trafﬁc.

Original: Diffusion Convolution The  diffusion convolution operation over a graph signal X: ,i ∈ R N  and a ﬁlter f θ is deﬁned as:
Revised: Diffusion Convolution The resulted diffusion convolution operation over a graph signal X ∈ R N × P and a ﬁlter f θ is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where θ ∈ R K × 2 are the parameters for the ﬁlter and D − 1 O W , D − 1 I W (cid:124) represent thestate transition matrices for outgoing and incoming ﬂow respectively.
Revised: where θ ∈ R K × 2 are the parameters for the ﬁlter and D − 1 O W , D − 1 I W (cid:124) represent the transition matrices of the diffusion process and the reverse one, respectively.

Original: In general, computing the convolution can be expensive.
Revised: In general, computing the convolution can be expensive.

Original: However, if G is sparse, Equation 2 can be calculated efﬁciently using O ( K ) recursive sparse-dense matrix multiplication with total time complexity O
Revised: However, if G is sparse, Equation 2 can be calculated efﬁciently using O ( K ) recursive sparse-dense matrix multiplication with total time complexity O

Original: ( K |E| )
Revised: ( K |E| )

Original: (cid:28) O ( N 2 ) .
Revised: (cid:28) O ( N 2 ) .

Original:  Diffusion Convolutional Layer With the convolution operation deﬁned in Equation 2, we can build a diffusion convolutional layer that maps P -dimensional features to Q -dimensional outputs.
Revised: See Appendix B for more detail. Diffusion Convolutional Layer With the convolution operation deﬁned in Equation 2, we can build a diffusion convolutional layer that maps P -dimensional features to Q -dimensional outputs.

Original: Denote the parameter tensor as Θ ∈ R Q × P × K × 2 =
Revised: Denote the parameter tensor as Θ ∈ R Q × P × K × 2 =

Original: [ θ ] q,p , where Θ q,p, : , : ∈ R K × 2 parameterizes the convolutional ﬁlter for  p th input and  q th output.
Revised: [ θ ] q,p , where Θ q,p, : , : ∈ R K × 2 parameterizes the convolutional ﬁlter for the p th input and the q th output.

Original: The diffusion convolutional layer is thus:
Revised: The diffusion convolutional layer is thus:

Original: [Equation]
Revised: [Equation]

Original:  where X ∈ R N × P is the input, H ∈ R N × Q is the output, { f Θ q,p,, : } are the ﬁlters and a is the activation function (e.g. Sigmoid, ReLU).
Revised: q ∈ [Equation] [Equation] where X ∈ R N × P is the input, H ∈ R N × Q is the output, { f Θ q,p,, : } are the ﬁlters and a is the activation function (e.g., ReLU, Sigmoid).

Original: Diffusion convolutional layer can learn the representations forany graph structured data and we can train it using stochastic gradient based method.
Revised: Diffusion convolutional layer learns the representations for graph structured data and we can train it using stochastic gradient based method.

Original: Relation with Spectral Graph Convolution Diffusion convolution is deﬁned on  directed  graphs.
Revised: Relation with Spectral Graph Convolution Diffusion convolution is deﬁned on both directed and undirected graphs.

Original: When applied to undirected graphs, we show that many existing graph structured convolutional operations including the popular spectral graph convolution, i.e., ChebNet (Defferrard et al., 2016), can be considered as a special case  (up to a similarity transformation).
Revised: When applied to undirected graphs, we show that many existing graph structured convolutional operations including the popular spectral graph convolution, i.e., ChebNet (Defferrard et al., 2016), can be considered as a special case of diffusion convolution (up to a similarity transformation).

Original: Let D denote the degree matrix, and L = D − 12 ( D − W ) D − 12 be the normalized graph Laplacian, the following Proposition demonstrates the connection.
Revised: Let D denote the degree matrix, and L = D − 12 ( D − W ) D − 12 be the normalized graph Laplacian, the following Proposition demonstrates the connection.

Original: Proposition 2.2.
Revised: Proposition 2.2.

Original: The spectral graph convolution deﬁned as
Revised: The spectral graph convolution deﬁned as

Original: [Equation]
Revised: [Equation]

Original: with eigenvalue decomposition L = Φ Λ Φ (cid:124) and F ( θ )
Revised: with eigenvalue decomposition L = Φ Λ Φ (cid:124) and F ( θ )

Original: =
Revised: =

Original: (cid:80) K − 1θ k Λ k , is equivalent to graph diffusion convolution up to a similarity transformation, when the graph G is undirected.
Revised: (cid:80) K − 1θ k Λ k , is equivalent to graph diffusion convolution up to a similarity transformation, when the graph G is undirected.

Original: Proof.
Revised: Proof.

Original: See Appendix 5.2.
Revised: See Appendix C.

Original: 2. T EMPORAL D YNAMICS M ODELING
Revised: 2. T EMPORAL D YNAMICS M ODELING

Original: We leverage the recurrent neural networks (RNNs) to model the temporal dependency.
Revised: We leverage the recurrent neural networks (RNNs) to model the temporal dependency.

Original: In particular, we use Gated Recurrent Units (GRU) (Chung et al., 2014), which is a simple yet powerful variant of
Revised: In particular, we use Gated Recurrent Units (GRU) (Chung et al., 2014), which is a simple yet powerful variant of

Original: RNNs.
Revised: RNNs.

Original: We replace the matrix multiplications in GRU with thegraph diffusion convolution .
Revised: We replace the matrix multiplications in GRU with the diffusion convolution , which leads to our proposed Diffusion Convolutional Gated Recurrent Unit (DCGRU).

Original: [Equation]
Revised: [Equation]

Original: [Figure] where X ( t ) , H ( t ) denote the input and output of at time t , r ( t ) , u ( t ) are reset gate and update gate at time t  respectively.
Revised: where X ( t ) , H ( t ) denote the input and output of at time t , r ( t ) , u ( t ) are reset gate and update gate at time t , respectively.

Original: (cid:63) G denotes the diffusion convolution deﬁned in Equation 2 and Θ r , Θ u , Θ C are parameters for the corresponding ﬁlters.
Revised: (cid:63) G denotes the diffusion convolution deﬁned in Equation 2 and Θ r , Θ u , Θ C are parameters for the corresponding ﬁlters.

Original: Similar to GRU, DCGRU can be used to build recurrent neural network layers and be trained using backpropagation through time.
Revised: Similar to GRU, DCGRU can be used to build recurrent neural network layers and be trained using backpropagation through time.

Original: In multiple-step ahead forecasting, we employ the Sequence to Sequence architecture (Sutskever et al., 2014), The graph diffusion convolution is applied to both the encoder and the decoder.
Revised: In multiple step ahead forecasting, we employ the Sequence to Sequence architecture (Sutskever et al., 2014). Both the encoder and the decoder are recurrent neural networks with DCGRU.

Original: During training, we feed the historical time series into the encoder and use its ﬁnal states as the initial states for the  decoder. 
Revised: During training, we feed the historical time series into the encoder and use its ﬁnal states to initialize the [Figure] decoder. The decoder generates predictions given previous ground truth observations .

Original: At testing time, ground truth observations are replaced by predictions generated by the model itself.
Revised: At testing time, ground truth observations are replaced by predictions generated by the model itself.

Revised: The discrepancy between the input distributions of training and testing can cause degraded performance.

Original: We integrate scheduled sampling (Bengio et al., 2015) into the model, where we feed the model with either the ground truth  with probability (cid:15) i or the output of the model with probability 1 − (cid:15)
Revised: To mitigate this issue, we integrate scheduled sampling (Bengio et al., 2015) into the model, where we feed the model with either the ground truth observation with probability (cid:15) i or the prediction by the model with probability 1 − (cid:15)

Original: i at the i th iteration.
Revised: i at the i th iteration.

Original: During the training process, (cid:15) i gradually decreases to 0 to allow the model to learn the testing distribution.
Revised: During the training process, (cid:15) i gradually decreases to 0 to allow the model to learn the testing distribution.

Original: With both spatial and temporal modeling, we build a Diffusion Convolutional Recurrent Neural
Revised: With both spatial and temporal modeling, we build a Diffusion Convolutional Recurrent Neural

Original: Network (DCRNN).
Revised: Network (DCRNN).

Original: The model architecture of DCRNN is shown in Figure 2.
Revised: The model architecture of DCRNN is shown in Figure 2.

Original: The entire network is trained by maximizing the likelihood of generating the target future time series using backpropagation through time and scheduled sampling.
Revised: The entire network is trained by maximizing the likelihood of generating the target future time series using backpropagation through time.

Original: DCRNN is able to capture spatiotemporal dependencies among time series and can be applied to various spatiotemporal forecasting problem.
Revised: DCRNN is able to capture spatiotemporal dependencies among time series and can be applied to various spatiotemporal forecasting problems.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Trafﬁc forecasting is a classic problem in transportation and operational research which are primarily based on queuing theory and simulations (Drew, 1968).
Revised: Trafﬁc forecasting is a classic problem in transportation and operational research which are primarily based on queuing theory and simulations (Drew, 1968).

Original: Data-driven approaches for trafﬁc forecasting have received considerable attention,  details can be found in a recent survey paper (Vlahogianni et al., 2014) and the references therein.
Revised: Data-driven approaches for trafﬁc forecasting have received considerable attention, and more details can be found in a recent survey paper (Vlahogianni et al., 2014) and the references therein.

Original: However, existing machine learning models either impose strong stationary assumptions on the data (e.g., auto-regressive model) or fail to account for highly non-linear temporal dependency (e.g., latent space model Yu et al.
Revised: However, existing machine learning models either impose strong stationary assumptions on the data (e.g., auto-regressive model) or fail to account for highly non-linear temporal dependency (e.g., latent space model Yu et al.

Original: ; Deng et al.
Revised: ; Deng et al.

Original: Deep learning models deliver new promise for time series forecasting problem.
Revised: Deep learning models deliver new promise for time series forecasting problem.

Original: For example,  Yu et al.
Revised: For example, in Yu et al.

Original: (2017b)  studyunivariate time series forecasting using deep Recurrent Neural Networks (RNN).
Revised: (2017b) ; Laptev et al. (2017), the authors study time series forecasting using deep Recurrent Neural Networks (RNN).

Original: Convolutional

Original:  Neural Networks (CNN) have also been applied to trafﬁc forecasting.
Revised: Convolutional Neural Networks (CNN) have also been applied to trafﬁc forecasting.

Original: Zhang et al.
Revised: Zhang et al.

Original: (2016; 2017) convert the road network to a regular 2-D grid and apply traditional CNN to predict crowd ﬂow.
Revised: (2016; 2017) convert the road network to a regular 2-D grid and apply traditional CNN to predict crowd ﬂow.

Revised: Cheng et al.

Revised: (2017) propose DeepTransport which models the spatial dependency by explicitly collecting upstream and downstream neighborhood roads for each individual road and then conduct convolution on these neighborhoods respectively.

Original: Recently, CNN has been generalized to arbitrary graphs based on the spectral graph theory.
Revised: Recently, CNN has been generalized to arbitrary graphs based on the spectral graph theory.

Original: Graph convolutional neural networks (GCN) are ﬁrst introduced in Bruna et al.
Revised: Graph convolutional neural networks (GCN) are ﬁrst introduced in Bruna et al.

Original: (2014), which bridges the spectral graph theory and deep neural networks.
Revised: (2014), which bridges the spectral graph theory and deep neural networks.

Original: Defferrard et al.
Revised: Defferrard et al.

Original: (2016) propose ChebNet which improves GCN with fast localized convolutions ﬁlters.
Revised: (2016) propose ChebNet which improves GCN with fast localized convolutions ﬁlters.

Revised: Kipf & Welling (2017) simplify ChebNet and achieve state-of-the-art performance in semi-supervised classiﬁcation tasks.

Original: Seo et al.
Revised: Seo et al.

Original: (2016) combine ChebNet with Recurrent Neural Networks (RNN) for structured sequence modeling.
Revised: (2016) combine ChebNet with Recurrent Neural Networks (RNN) for structured sequence modeling.

Original: Yu et al.
Revised: Yu et al.

Original: (2017a) model the sensor network as a undirected graph and applied ChebNet and convolutional sequence model (Gehring et al., 2017) to do forecasting.
Revised: (2017a) model the sensor network as a undirected graph and applied ChebNet and convolutional sequence model (Gehring et al., 2017) to do forecasting.

Original: One limitation of the mentioned spectral based convolutions is that they generally require the graph to be undirected to calculate meaningful  spectral decomposition.
Revised: One limitation of the mentioned spectral based convolutions is that they generally require the graph to be undirected to calculate meaningful [Table] spectral decomposition.

Original: Going from spectral domain to vertex domain, Atwood & Towsley (2016) propose diffusion-convolutional neural network (DCNN) which deﬁnes convolution as a diffusion process across each node in a graph-structured input. 
Revised: Going from spectral domain to vertex domain, Atwood & Towsley (2016) propose diffusion-convolutional neural network (DCNN) which deﬁnes convolution as a diffusion process across each node in a graph-structured input. Hechtlinger et al. (2017) propose GraphCNN to generalize convolution to graph by convolving every node with its p nearest neighbors.

Original: However, it does not consider the temporal dynamics and mainly deal with static graph settings.
Revised: However, both these methods do not consider the temporal dynamics and mainly deal with static graph settings.

Original: Our approach is different from all those methods due to both the problem settings and the formulation of the graph convolution.
Revised: Our approach is different from all those methods due to both the problem settings and the formulation of the convolution on the graph.

Original: We model the sensor network as a weighted directed graph which is more realistic than grid or undirected graph.
Revised: We model the sensor network as a weighted directed graph which is more realistic than grid or undirected graph.

Original: Besides, the proposed convolution is deﬁned using bidirectional graph random walk and is further integrated with the sequence to sequence learning framework as well as the scheduled sampling to model the long-term temporal dependency.
Revised: Besides, the proposed convolution is deﬁned using bidirectional graph random walk and is further integrated with the sequence to sequence learning framework as well as the scheduled sampling to model the long-term temporal dependency [Table]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We conduct experiments on two real-world large-scale datasets: (1) METR-LA This trafﬁc dataset contains trafﬁc information collected from loop detectors in the highway of Los Angeles County (Jagadish et al., 2014).
Revised: We conduct experiments on two real-world large-scale datasets: (1) METR-LA This trafﬁc dataset contains trafﬁc information collected from loop detectors in the highway of Los Angeles County (Jagadish et al., 2014).

Original: We select 207 sensors and collect 4 months of data ranging from Mar 1st 2012 to Jun 30th 2012.
Revised: We select 207 sensors and collect 4 months of data ranging from Mar 1stto Jun 30th 2012 for the experiment.

Original: (2) PEMS-BAY
Revised: (2) PEMS-BAY

Original: This trafﬁc dataset is collected by CaliforniaTransportation Agencies (CalTrans)
Revised: This trafﬁc dataset is collected by California

Original:  Performance Measurement System (PeMS).
Revised: Transportation Agencies (CalTrans) Performance Measurement System (PeMS).

Original: We select 325 sensors in the Bay Area and collect the data  from Jan 1st 2017 to May 31th 2017 to conduct experiments.
Revised: We selectsensors in the Bay Area and collect 6 months of data ranging from Jan 1st 2017 to May 31th 2017 for the experiment.

Original: The sensor distributions of both datasets are visualized in Figure 8 in the Appendix.
Revised: The sensor distributions of both datasets are visualized in Figure 8 in the Appendix.

Original: In both of those datasets, we aggregate trafﬁc speed readings into 5 minutes windows, and apply
Revised: In both of those datasets, we aggregate trafﬁc speed readings into 5 minutes windows, and apply

Original: Z-Score normalization.
Revised: Z-Score normalization.

Original: 70% of data is used for training, 20% are used for testing while the remaining 10% for validation.
Revised: 70% of data is used for training, 20% are used for testing while the remaining 10% for validation.

Original: To construct the sensor graph, weﬁrst compute the pairwise road network distances  andthen build the adjacency matrix using thresholded Gaussian kernel (Shuman et al., 2013).
Revised: To construct the sensor graph, we compute the pairwise road network distances between sensors and build the adjacency matrix using thresholded Gaussian kernel (Shuman et al., 2013).

Original: [Equation] where W ij represents the edge weight between sensor v i and sensor v j , dist( v i , v j ) denotes the road network distance from sensor v i to sensor v j .
Revised: W ij = exp (cid:16) − dist( v i ,v j ) 2 σ(cid:17) if dist( v i , v j ) ≤ κ , otherwise 0 , where W ij represents the edge weight between sensor v i and sensor v j , dist( v i , v j ) denotes the road network distance from sensor v i to sensor v j .

Original: σ is the standard deviation of distances and κ is the threshold.
Revised: σ is the standard deviation of distances and κ is the threshold.

Original: 4. E XPERIMENTAL S ETTINGS
Revised: 4. E XPERIMENTAL S ETTINGS

Original: Baselines We compare DCRNN  with widely used time series regression models, including (1)
Revised: Baselines We compare DCRNN 1 with widely used time series regression models, including (1)

Original: HA: Historical Average, which models the trafﬁc ﬂow as a seasonal process, and uses weighted average of previous seasons as the prediction; (2) ARIMA kal : Auto-Regressive Integrated Moving
Revised: HA: Historical Average, which models the trafﬁc ﬂow as a seasonal process, and uses weighted [Figure]

Original: Average model with Kalman ﬁlter which is widelyused in time series prediction; (3) VAR: Vector
Revised: '&5111R&RQY '&5118QL&RQY '&51 [Figure] average of previous seasons as the prediction is widely

Original: Auto-Regression (Hamilton, 1994). (4) SVR: Support Vector Regression which uses linear support vector machine for regression task; Thefollowing deep neural network based approaches are also included: (5) Feed forward Neural network (FNN): Feed forward neural network with two hidden layers and L2 regularization. (6) Recurrent Neural Network with fully connected LSTM hidden units (FC-LSTM)
Revised: The

Original: (Sutskever et al., 2014). All neural network based approaches are implemented using Tensorﬂow (Abadi et al., 2016), and trained using the Adam optimizer with learning rate annealing.
Revised: All neural network based approaches are implemented using Tensorﬂow (Abadi et al., 2016), and trained using the Adam optimizer with learning rate annealing.

Original: The best hyperparameters are chosen using the Tree-structured Parzen Estimator (TPE) (Bergstra et al., 2011) on the validation dataset.
Revised: The best hyperparameters are chosen using the Tree-structured Parzen Estimator (TPE) (Bergstra et al., 2011) on the validation dataset.

Original: Detailed parameter settings for DCRNN as well as baselines are available in Appendix 5.3.
Revised: Detailed parameter settings for DCRNN as well as baselines are available in Appendix E.

Original: 4. T RAFFIC F ORECASTING P ERFORMANCE C OMPARISON
Revised: 4. T RAFFIC F ORECASTING P ERFORMANCE C OMPARISON

Original: Table 1 shows the comparison of different approaches for 15 minutes, 30 minutes and 1 hour ahead forecasting on both datasets.
Revised: Table 1 shows the comparison of different approaches for 15 minutes, 30 minutes and 1 hour ahead forecasting on both datasets.

Original: These methods are evaluated based on three commonly used metrics in trafﬁc forecasting, including (1) Mean Absolute Error (MAE), (2) Mean Absolute Percentage[Table] [Table] Error (MAPE), and (3) Root Mean Squared Error (RMSE)
Revised: These methods are evaluated based on three commonly used metrics in trafﬁc forecasting, including (1) Mean Absolute Error (MAE), (2) Mean Absolute Percentage Error (MAPE), and (3) Root Mean Squared Error (RMSE).

Original: Note that, missing values are excluded in calculating these metrics.
Revised: Missing values are excluded in calculating these metrics.

Original: Detailed formulations of these metrics are provided in Appendix 5.3.1.
Revised: Detailed formulations of these metrics are provided in Appendix E.2.

Original: We observe the following phenomenon in both of these datasets.
Revised: We observe the following phenomenon in both of these datasets.

Original: (1) RNN-based methods, including 
Revised: (1) RNN-based methods, including FC-LSTM and

Original: FC-LSTM and DCRNN, generally outperform other baselines which emphasizes the importance of modeling the temporal dependency.
Revised: DCRNN, generally outperform other baselines which emphasizes the importance of modeling the temporal dependency.

Original: (2) DCRNN achieves the best performance regarding all the metrics for all forecasting horizons, which suggests the effectiveness of spatiotemporal dependency modeling.
Revised: (2) DCRNN achieves the best performance regarding all the metrics for all forecasting horizons, which suggests the effectiveness of spatiotemporal dependency modeling.

Original: (3) Deep neural network based method including FNN, FC-LSTM and DCRNN, tends to have better performance than linear baselines for long-term forecasting, e.g., 1 hour ahead.
Revised: Deep neural network based methods including FNN, FC-LSTM and DCRNN, tend to have better performance than linear baselines for long-term forecasting, e.g., 1 hour ahead.

Original: This is because the temporal dependency becomes increasingly non-linear with the growth of the horizon.
Revised: This is because the temporal dependency becomes increasingly non-linear with the growth of the horizon.

Original: Note that, as the historical average method does not depend on short-term data, its performance is invariant to the small increases in the forecasting horizon.
Revised: Besides, as the historical average method does not depend on short-term data, its performance is invariant to the small increases in the forecasting horizon.

Original: Note that, trafﬁc forecasting on the METR-LA (Los Angeles, which is known for its complicated trafﬁc conditions) dataset is more challenging than that in the PEMS-BAY (Bay Area) dataset.
Revised: Note that, trafﬁc forecasting on the METR-LA (Los Angeles, which is known for its complicated trafﬁc conditions) dataset is more challenging than that in the PEMS-BAY (Bay Area) dataset.

Original: Thus we use METR-LA as the default dataset forthe following experiments.
Revised: Thus we use METR-LA as the default dataset for following experiments.

Original: 4. E FFECT OF SPATIAL DEPENDENCY MODELING
Revised: 4. E FFECT OF SPATIAL DEPENDENCY MODELING

Original: To further investigate the effect of spatial dependency modeling, we compare DCRNN with the following variants: (1) DCRNN-NoConv, which ignores spatial dependency by replacing the random walk matrix with the  identity matrix.
Revised: To further investigate the effect of spatial dependency modeling, we compare DCRNN with the following variants: (1) DCRNN-NoConv, which ignores spatial dependency by replacing the transition matrices in the diffusion convolution (Equation 2) with identity matrices.

Original: This essentially means the forecasting of a sensor can be only be inferred from its  historical readings; (2) DCRNN-UniConv,which only uses the forward random walk for diffusion convolution; Figure 3 shows the learning curves of these three models with roughly the same number of parameters.
Revised: This essentially means the forecasting of a sensor can be only be inferred from its own historical readings; (2) DCRNN-UniConv,

Original: Without diffusion convolution, DCRNN-NoConv has much higher validation error.
Revised: [Table]

Original: Moreover, DCRNN achieves the lowest validation error which shows the effectiveness of using bidirectional random walk.
Revised: sequence sequence framework and sched sampling, achieves the lowest MAE on the validation dataset.

Original: The intuition is that the bidirectional random [Figure]
Revised: The advantage becomes more clear with the increase of the forecasting horizon

Original:         [Figure] RQ
Revised: DCRNN generates smooth prediction and is usually better at predict the start and end of peak hours.

Original: [Figure] $( '&5111R&RQY '&5118QL&RQY '&51 [Figure] [Figure]
Revised: which only uses the forward random walk transition matrix for diffusion convolution; Figure 3 shows the learning curves of these three models with roughly the same number of parameters.

Original: comes more clear with the increase of the forecasting horizon
Revised: Without diffusion convolution, DCRNN-NoConv has much higher validation error.

Original:  DCRNN generates smooth prediction and is usually better at predict the  of  walk gives the model the ability and ﬂexibility to capture the inﬂuence from both the upstream and the downstream.
Revised: Moreover, DCRNN achieves the lowest validation error which shows the effectiveness of using bidirectional random walk. The intuition is that the bidirectional random walk gives the model the ability and ﬂexibility to capture the inﬂuence from both the upstream and the downstream trafﬁc.

Original: To investigate the effect of graph construction, we construct a undirected graph by setting ˆ W ij = ˆ W ji = max( W ij , W ji ) , where ˆ W is the new symmetric weight matrix.
Revised: To investigate the effect of graph construction, we construct a undirected graph by setting (cid:99) W ij = (cid:99) W ji = max( W ij , W ji ) , where (cid:99) W is the new symmetric weight matrix.

Original: Then we develop a variant of DCRNN denotes GCRNN, which uses the sequence to sequence learning with ChebNet graph convolution (Equation 4).
Revised: Then we develop a variant of DCRNN denotes GCRNN, which uses the sequence to sequence learning with ChebNet graph convolution (Equation 5) with roughly the same amount of parameters.

Original: Table 2 shows the comparison between DCRNN and GCRNN in METRLA dataset.
Revised: Table 2 shows the comparison between DCRNN and GCRNN in the METR-LA dataset.

Original: DCRNN consistently outperforms GCRNN.
Revised: DCRNN consistently outperforms GCRNN.

Original: The intuition is that directed graph better captures the asymmetric correlation between trafﬁc sensors.
Revised: The intuition is that directed graph better captures the asymmetric correlation between trafﬁc sensors.

Original: Figure 4 shows the effects of different parameters.
Revised: Figure 4 shows the effects of different parameters.

Original: K roughly corresponds  the reception ﬁeld of the ﬁlter while the number of units corresponds to the number of ﬁlters.
Revised: K roughly corresponds to the size of ﬁlters’ reception ﬁelds while the number of units corresponds to the number of ﬁlters.

Original: Larger K enables the model to capture broader spatial dependency at the cost of increasing learning complexity.
Revised: Larger K enables the model to capture broader spatial dependency at the cost of increasing learning complexity.

Original: We observe that with the increase of K , the error on the validation dataset ﬁrst quickly decrease, and then slightly increase.
Revised: We observe that with the increase of K , the error on the validation dataset ﬁrst quickly decrease, and then slightly increase.

Original: Similar behavior is observed for varying the number of units.center Max Min
Revised: Similar behavior is observed for varying the number of units.

Original:  E FFECT OF TEMPORAL DEPENDENCY MODELING
Revised: 4. E FFECT OF TEMPORAL DEPENDENCY MODELING

Original: To evaluate the effect  temporal modeling including the sequence to sequence framework as well as the scheduled sampling mechanism, we further design four variants of DCRNN: (1) DCNN: in which we concatenate the historical observations as a ﬁxed length vector and feed it into stackedto diffusion convolutional layers  predict the future time series.
Revised: To evaluate the effect of temporal modeling including the sequence to sequence framework as well as the scheduled sampling mechanism, we further design three variants of DCRNN: (1) DCNN: in which we concatenate the historical observations as a ﬁxed length vector and feed it into stacked diffusion convolutional layers to predict the future time series.

Original: Wethen train a single model  one step ahead prediction, and feed the previous prediction into the model as input to perform multiple steps ahead prediction.
Revised: We train a single model for one step ahead prediction, and feed the previous prediction into the model as input to perform multiple steps ahead prediction.

Original: (2) DCRNN-SEQ: which uses the encoder-decoder sequence to sequence learning framework to perform multiple steps ahead forecasting (3) DCRNN to DCRNN-SEQ 
Revised: (2) DCRNN-SEQ: which uses the encoder-decoder sequence to sequence learning framework to perform multiple steps ahead forecasting. (3) DCRNN: similar to DCRNN-SEQ except for adding scheduled sampling. center

Original:  Figure 5 shows the comparison of those four methods with regards to MAE for different forecasting horizons.
Revised: Max Min Figure 5 shows the comparison of those four methods with regards to MAE for different forecasting horizons.

Original: We observe that: (1) DCRNN-SEQ outperforms DCNN by a large margin which conforms the importance of modeling temporal dependency.
Revised: We observe that: (1) DCRNN-SEQ outperforms DCNN by a large margin which conforms the importance of modeling temporal dependency.

Original: (2) DCRNN achieves the best result, and its superiority becomes more evident with the increase of the forecasting horizon.
Revised: (2) DCRNN achieves the best result, and its superiority becomes more evident with the increase of the forecasting horizon.

Original: This is mainly because the model is trained to deal with its mistakes during multiple steps ahead prediction and thus suffers less from the problem of error propagation.
Revised: This is mainly because the model is trained to deal with its mistakes during multiple steps ahead prediction and thus suffers less from the problem of error propagation.

Original: We also train a model that always been fed its output as input for multiple steps ahead prediction.
Revised: We also train a model that always been fed its output as input for multiple steps ahead prediction.

Original: However, its performance is much worse than all the three variants which emphasizes the importance of scheduled sampling.
Revised: However, its performance is much worse than all the three variants which emphasizes the importance of scheduled sampling.

Original: 4. M ODEL I NTERPRETATION
Revised: 4. M ODEL I NTERPRETATION

Original: To better understand the model, we visualize forecasting results as well as learned ﬁlters.
Revised: To better understand the model, we visualize forecasting results as well as learned ﬁlters.

Original: Figureshows the visualization of 1 hour ahead forecasting.
Revised: Figureshows the visualization of 1 hour ahead forecasting.

Original: We have the following observations: (1) DCRNN generates smooth prediction of the meaneven when frequent oscillation exists in the trafﬁc signal (as shown in Figure 6(a)).
Revised: We have the following observations: (1) DCRNN generates smooth prediction of the mean when small oscillation exists in the trafﬁc speeds (Figure 6(a)).

Original: This reﬂects the robustness of the model.
Revised: This reﬂects the robustness of the model.

Original: (2) DCRNN is more likely to accurately predict abrupt changes in the trafﬁc speed than baseline methods (e.g., FC-LSTM).
Revised: (2) DCRNN is more likely to accurately predict abrupt changes in the trafﬁc speed than baseline methods (e.g., FC-LSTM).

Original: As shown in Figure 6(b), DCRNN is often able to predict the start and the end of the peak hours.
Revised: As shown in Figure 6(b), DCRNN predicts the start and the end of the peak hours.

Original: This is because DCRNN captures the spatial dependency, and is able to utilize the speed changes in neighborhood sensors for more accurate forecasting.
Revised: This is because DCRNN captures the spatial dependency, and is able to utilize the speed changes in neighborhood sensors for more accurate forecasting.

Original: Figure 7 visualizes examples of learned ﬁlters centered at different nodes.
Revised: Figure 7 visualizes examples of learned ﬁlters centered at different nodes.

Original: The star denotes the center, and colors denote the weights.
Revised: The star denotes the center, and colors denote the weights.

Original: We can observe that (1) weights are well localized around the center, and (2) the weights diffuse based on road network distance.
Revised: We can observe that (1) weights are well localized around the center, and (2) the weights diffuse based on road network distance.

Original: More visualizations are provided in Appendix 5.4.
Revised: More visualizations are provided in Appendix F.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we formulated the trafﬁc prediction on road network as a structured time series forecasting problem, and proposed the diffusion convolutional recurrent neural network that captures the spatiotemporal dependencies.
Revised: In this paper, we formulated the trafﬁc prediction on road network as a spatiotemporal forecasting problem, and proposed the diffusion convolutional recurrent neural network that captures the spatiotemporal dependencies.

Original: Speciﬁcally,  bidirectional graph random walkwas used to model spatial dependency and recurrent neural networkwas adopted to capture the temporal dynamics.
Revised: Speciﬁcally, we use bidirectional graph random walk to model spatial dependency and recurrent neural network to capture the temporal dynamics.

Original: We further integrated the encoder-decoder architecture and  scheduled sampling technique to improve the performance for long-term forecasting.
Revised: We further integrated the encoder-decoder architecture and the scheduled sampling technique to improve the performance for long-term forecasting.

Original: When evaluated on two large-scale real-world trafﬁc data, our approach obtained signiﬁcantly better prediction than baselines.
Revised: When evaluated on two large-scale real-world trafﬁc datasets, our approach obtained signiﬁcantly better prediction than baselines.

Original: For future work, we will investigate the following two aspects (1) applying the proposed model to other spatial-temporal forecasting tasks; ( how to model the spatiotemporal dependency when the underlying graph structure is evolving, e.g., the K nearest neighbor graph for moving objects. 
Revised: For future work, we will investigate the following two aspects (1) applying the proposed model to other spatial-temporal forecasting tasks; (2) modeling the spatiotemporal dependency when the underlying graph structure is evolving, e.g., the K nearest neighbor graph for moving objects. A CKNOWLEDGMENTS This research has been funded in part by NSF grants CNS-1461963, IIS-1254206, IIS-1539608, Caltrans-65A0533, the USC Integrated Media Systems Center (IMSC), and the USC METRANS

Original:  A PPENDIX 5. N OTATION
Revised: Transportation Center. Any opinions, ﬁndings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reﬂect the views of any of the sponsors such as NSF. Also, the authors would like to thank Shang-Hua Teng, Dehua Cheng and Siyang Li for helpful discussions and comments. A PPENDIX A N OTATION

Original: [Table]
Revised: [Table]

Original: Table 3 summarizes the main notation used in the paper.
Revised: Table 3 summarizes the main notations used in the paper.

Revised: B E FFICIENT C ALCULATION OF E QUATION

Revised: Equation 2 can be decomposed into two parts with the same time complexity,

Revised: i

Revised: [Table]

Revised: e., one part with

Revised: D − 1 O W and the other part with D − 1 I W (cid:124) .

Revised: Thus we will only show the time complexity of the ﬁrst part.

Revised: Let T k ( x )

Revised: =

Revised: (cid:0) D − 1 O W (cid:1) k x

Revised: [Table]

Revised: The ﬁrst part of Equation 2 can be rewritten as

Revised: [Equation]

Revised: As T k +1 ( x ) =

Revised: D − 1 O W T k ( x ) and D − 1 O W is sparse

Revised: [Table]

Revised: it is easy to see that Equation 4 can be calculated using O ( K ) recursive sparse-dense matrix multiplication each with time complexity

Revised: O ( |E| ) .

Revised: Consequently, the time complexities of both Equation 2 and Equation 4 are O

Revised: ( K |E| ) .

Revised: For dense graph, we may use spectral sparsiﬁcation (Cheng et al., 2015) to make it sparse.

Original: 5. R ELATION WITH S PECTRAL G RAPH C ONVOLUTION
Revised: C R ELATION WITH S PECTRAL G RAPH C ONVOLUTION

Original: Proof.
Revised: Proof.

Original: The spectral graph convolution utilizes the concept of normalized graph Laplacian L =
Revised: The spectral graph convolution utilizes the concept of normalized graph Laplacian L =

Original: D − 12 ( D − W ) D − 12 = 
Revised: D − [Equation] 12 = [Equation]

Original: ΦΛ Φ (cid:124) . ChebNet parametrizes f θ to be a K order polynomial of Λ
Revised: Φ

Original: [Table]
Revised: [Equation]

Original:  and calculate it using stable Chebyshev polynomial basis.[Equation] [Equation] where T 0 ( x ) =
Revised: ChebNet parametrizes f θ to be a K order polynomial of Λ , and calculates it using stable Chebyshev polynomial basis.

Original: [Table]
Revised: [Equation]

Original:  T 1 ( x ) = x, T k ( x ) =
Revised: where T 0 ( x ) = 1 , T 1 ( x ) =

Original:  xT k − 1 ( x ) − T k − 2 ( x ) are the basis of the Cheyshev polynomial.
Revised: x, T k ( x ) = xT k − 1 ( x ) − T k − 2 ( x ) are the basis of the Cheyshev polynomial.

Original: Let λ max denotes the largest eigenvalue of L ,  ˜ L = 2 λ max L − I represents a rescaling of the graph Laplacian that maps the eigenvalues from [0 , λ max ] to [ − 1 , 1] since Chebyshev polynomial forms an orthogonal basis in [ − 1 , 1] .
Revised: Let λ max denote the largest eigenvalue of L , and ˜ L = 2 λ max L − I represents a rescaling of the graph Laplacian that maps the eigenvalues from [0 , λ max ] to [ − 1 , 1] since Chebyshev polynomial forms an orthogonal basis in [ − 1 , 1] .

Original: Equation 4 can be considered as a polynomial of ˜ L and we will show that the output of ChebNet Convolution is similar to the output of diffusion convolution up to constant scaling factor.
Revised: Equation 5 can be considered as a polynomial of ˜ L and we will show that the output of ChebNet Convolution is similar to the output of diffusion convolution up to constant scaling factor.

Original: Assume λ max = 

Revised: Assume λ max =

Original: and D I =
Revised: and D I =

Original: D O = D for undirected graph.
Revised: D O = D for undirected graph.

Original: [Equation]
Revised: [Equation]

Original: ˜ L is similar to the negative random walk  matrix, thus the output of Equation 4 is also similar to the output of Equation 2 up to constant scaling factor. 
Revised: ˜ L is similar to the negative random walk transition matrix, thus the output of Equation 5 is also similar to the output of Equation 2 up to constant scaling factor. D M ORE R ELATED W ORK AND D ISCUSSION Xie et al.

Original: 5. D ETAILED E XPERIMENTAL S ETTINGS
Revised: (2010) introduce a Gaussian processes (GPs) based method. GPs are hard to scale to the large dataset and are generally not suitable for relatively long-term trafﬁc prediction like 1 hour (i.e.,12 steps ahead), as the variance can be accumulated and becomes extremely large. Cai et al. (2016) propose to use spatiotemporal nearest neighbor for trafﬁc forecasting (ST-KNN).

Revised: Though ST-KNN considers both the spatial and the temporal dependencies, it has the following drawbacks.

Revised: As shown in Fusco et al.

Revised: (2016), ST-KNN performs independent forecasting for each individual road.

Revised: The prediction of a road is a weighted combination of its own historical trafﬁc speeds.

Revised: This makes it hard for ST-KNN to fully utilize information from neighbors.

Revised: Besides, ST-KNN is a non-parametric approach and each road is modeled and calculated separately (Cai et al., 2016), which makes it hard to generalize to unseen situations and to scale to large datasets.

Revised: Finally, in ST-KNN, all the similarities are calculated using hand-designed metrics with few learnable parameters, and this may limit its representational power.

Revised: Cheng et al.

Revised: (2017) propose DeepTransport which models the spatial dependency by explicitly collecting certain number of upstream and downstream roads for each individual road and then conduct convolution on these roads respectively.

Revised: Comparing with Cheng et al.

Revised: (2017), DCRNN models the spatial dependency in a more systematic way, i.e., generalizing convolution to the trafﬁc sensor graph based on the diffusion nature of trafﬁc.

Revised: Besides, we derive DCRNN from the property of random walk and show that the popular spectral convolution ChebNet is a special case of our method.

Revised: The proposed approach is also related to graph embedding techniques, e.g., Deepwalk (Perozzi et al., 2014), node2vec (Grover & Leskovec, 2016) which learn a low dimension representation for each node in the graph.

Revised: DCRNN also learns a representation for each node.

Revised: The learned representations capture both the spatial and the temporal dependency and at the same time are optimized with regarding to the objective, e.g., future trafﬁc speeds.

Revised: E D ETAILED E XPERIMENTAL S ETTINGS

Original: HA Historical Average,  the  period used is 1 week, and the prediction is based on aggregated data from previous weeks.
Revised: HA Historical Average, which models the trafﬁc ﬂow as a seasonal process, and uses weighted average of previous seasons as the prediction. The period used is 1 week, and the prediction is based on aggregated data from previous weeks.

Original: For example, the prediction for this Wednesday is the averaged trafﬁc speeds from last four Wednesdays.
Revised: For example, the prediction for this Wednesday is the averaged trafﬁc speeds from last four Wednesdays.

Original: As the historical average method does not depend on short-term data, its performance is invariant to the small increases in the forecasting horizon
Revised: As the historical average method does not depend on short-term data, its performance is invariant to the small increases in the forecasting horizon

Original: ARIMA  with Kalman Filter the orders are (3, 0, 1),  implemented using the statsmodel python package.
Revised: ARIMA kal : Auto-Regressive Integrated Moving Average model with Kalman ﬁlter. The orders are (3, 0, 1), and the model is implemented using the statsmodel python package.

Original: VAR Vector Auto-regressive model with lags equals to 3,  implemented using the statsmodel python package.
Revised: VAR Vector Auto-regressive model (Hamilton, 1994). The number of lags is set to 3, and the model is implemented using the statsmodel python package.

Original: SVR Linear Support Vector Regression, the penalty term C = 0 .
Revised: SVR Linear Support Vector Regression, the penalty term C = 0 .

Original: 1 , the number of historical observation is 5.
Revised: 1 , the number of historical observation is 5.

Revised: The following deep neural network based approaches are also included.

Original: FNN the network has 2 hiddenfully connected layers, each layer contains 256 units.
Revised: FNN Feed forward neural network with two hidden layers, each layer contains 256 units.

Original: The initial learning rate is 1 e − 3 , and reduces to 110 every 20 epochs starting at the 50th epochs.
Revised: The initial learning rate is 1 e − 3 , and reduces to 110 every 20 epochs starting at the 50th epochs.

Original: In addition, for all hidden layers, dropout with ratio 0.5 and L2 weight decay 1e-2 is used.
Revised: In addition, for all hidden layers, dropout with ratio 0.5 and L2 weight decay 1 e − 2 is used.

Original: The model is trained with batch size 64 and MAE as the loss function.
Revised: The model is trained with batch size 64 and MAE as the loss function.

Original: Early stop is performed by monitoring the validation error.
Revised: Early stop is performed by monitoring the validation error.

Original: FC-LSTM  Encoder-decoder framework using LSTM with peephole.
Revised: FC-LSTM The Encoder-decoder framework using LSTM with peephole (Sutskever et al., 2014).

Original: Both  encoder and  decoder contain two recurrent layers.
Revised: Both the encoder and the decoder contain two recurrent layers.

Original: In each recurrent layer, there are256 LSTM units, L1 weight decay is 2e-5, L2 weight decay 5 e − 4 .
Revised: In each recurrent layer, there are LSTM units, L1 weight decay is 2 e − 5 , L2 weight decay 5 e − 4 .

Original: The model is trained with batch size 64 and loss function MAE.
Revised: The model is trained with batch size 64 and loss function MAE.

Original: The initial learning rate is 1e-4 and reduces to 110 every 10 epochs starting from the 20th epochs.
Revised: The initial learning rate is 1e-4 and reduces to 1 10 every 10 epochs starting from the 20th epochs.

Original: Early stop is performed by monitoring the validation error.
Revised: Early stop is performed by monitoring the validation error.

Original: DCRNN :
Revised: DCRNN :

Original: Diffusion Convolutional Recurrent Neural Network.
Revised: Diffusion Convolutional Recurrent Neural Network.

Original: Both encoder and decoder contain two recurrent layers.
Revised: Both encoder and decoder contain two recurrent layers.

Original: In each recurrent layer, there are 64 units, the initial learning rate is 1e-2, and reduces to 110 every 10 epochs starting at the 20th epoch and early stopping on the validation dataset is used.
Revised: In each recurrent layer, there are 64 units, the initial learning rate is 1 e − 2 , and reduces to 110 every 10 epochs starting at the 20th epoch and early stopping on the validation dataset is used.

Original: Besides, the maximum steps of random walks, i.e., K , is set to 3.
Revised: Besides, the maximum steps of random walks, i.e., K , is set to 3.

Original: For scheduled sampling, the thresholded inverse sigmoid function is used as the probability decay:
Revised: For scheduled sampling, the thresholded inverse sigmoid function is used as the probability decay:

Original: [Equation]
Revised: [Equation]

Original: where i is the number of iterations while τ are parameters to control the speed of convergence.
Revised: where i is the number of iterations while τ are parameters to control the speed of convergence.

Original: τ is set to 3,000 in the experiments.
Revised: τ is set to 3,000 in the experiments.

Original: 5.3. M ETRICS
Revised: The implementation is available in https://github.com/ liyaguang/DCRNN E. D ATASET We conduct experiments on two real-world large-scale datasets: • METR-LA This trafﬁc dataset contains trafﬁc information collected from loop detectors in the highway of Los Angeles County (Jagadish et al., 2014).

Revised: We select 207 sensors and collect 4 months of data ranging from Mar 1st 2012 to Jun 30th 2012 for the experiment.

Revised: The total number of observed trafﬁc data points is 6,519,002.

Revised: • PEMS-BAY

Revised: This trafﬁc dataset is collected by California Transportation Agencies (CalTrans)

Revised: Performance Measurement System (PeMS).

Revised: We select 325 sensors in the Bay Area and collect 6 months of data ranging from Jan 1st 2017 to May 31th 2017 for the experiment.

Revised: The total number of observed trafﬁc data points is 16,937,179.

Revised: The sensor distributions of both datasets are visualized in Figure 8.

Revised: In both of those datasets, we aggregate trafﬁc speed readings into 5 minutes windows, and apply

Revised: Z-Score normalization.

Revised: 70% of data is used for training, 20% are used for testing while the remaining 10% for validation.

Revised: To construct the sensor graph, we compute the pairwise road network distances between sensors and build the adjacency matrix using thresholded Gaussian kernel (Shuman et al., 2013).

Revised: [Equation]

Revised: where W ij represents the edge weight between sensor v i and sensor v j , dist( v i , v j ) denotes the road network distance from sensor v i to sensor v j .

Original:  Suppose x = x 1 , · · · , x n represents the ground truth, ˆ x = ˆ x 1 , · · · , ˆ x n represents the predicted values, and Ω denotes the indices of observed samples, the metrics are deﬁned as follows. 
Revised: σ is the standard deviation of distances and κ is the threshold. E. M ETRICS Suppose x = x 1 , · · · , x n represents the ground truth, ˆ x = ˆ x 1 , · · · , ˆ x n represents the predicted values, and Ω denotes the indices of observed samples, the metrics are deﬁned as follows. [Equation]

Original: Root Mean Square Error (RMSE)
Revised: Mean Absolute Percentage Error (MAPE)

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: Mean Absolute Error (MAE)

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: Figure 9 visualizes the correlations between a sensor and its neighborhoods for different forecasting horizons. We observe that the correlations are localized and closer neighborhoods usually have larger relevance, and the magnitude of correlation quickly decay with the increase of distance which is consistent with the diffusion process on the graph.
Revised: [Equation] [Figure]