# A Sparse Gating Convolutional Recurrent Network for Traffic Flow Prediction

Authors: Xiaohui Huang; Jie Tang; Zhiying Peng; Zhiyi Chen; Hui Zeng
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

With the continuous development of deep learning, more and more huge deep learning models are developed by researchers, which leads to an exponential increase of the parameters of models. Therein, the convolutional recurrent network as a type of widely used deep learning method is often employed to handle spatiotemporal data, e.g., traffic data. However, because of the large number of parameters in the model, the convolutional recurrent network needs to consume a lot of computing resources and time in the training process. To reduce the consumption of resources, we propose a sparse convolutional recurrent network with a sparse gating mechanism that is able to reduce the complexity of the network by an improved gate unit while keeping the performance of the model. We evaluate the performance of our proposed network on traffic flow datasets, and the experimental results show that the parameters of the model are significantly reduced under the condition of similar prediction accuracy compared with the traditional convolutional recurrent network.


---

## Body

## 1. Introduction

Deep learning [1] has achieved great success in various fields, e.g., computer vision [2, 3] and natural language processing [4], and it has also been introduced into the field of traffic flow prediction by some scholars [5–7]. Meanwhile, with the development of the economy, the transportation network in recent years has also achieved rapid expansion. However, traffic congestion is becoming an increasingly serious issue. The traffic flow prediction of different key points in the transportation network is the basis for solving the issue. To improve the accuracy of traffic flow prediction, various complex deep networks are developed, which leads to an overwhelming increase of parameters and resource consumption. Convolutional long short-term memory (ConvLSTM) network was proposed by Shi et al. [8] for precipitation nowcasting, which introduced the convolutional operation into the fully connected LSTM [9]. Ballas et al. [10] proposed a convolutional gated recurrent unit (ConvGRU) for video representation, which uses a convolutional structure to capture spatial features. Then, they are applied to traffic flow prediction by many researchers. For example, Zonoozi et al. [11] proposed a ConvGRU method based on the periodic characteristics of the traffic flow data, which employs ConvGRU to extract the spatiotemporal representation of input data. Zang et al. [12] developed a deep learning approach for traffic speed prediction, which uses the ConvLSTM network to extract the temporal and spatial dependence of historical traffic data.Since convolutional recurrent network has great advantages in extracting temporal and spatial information hidden in spatiotemporal data simultaneously, the method has been naturally applied to solve the prediction problem of traffic flow. However, the convolutional recurrent network has a large number of parameters and requires more resources in the training and testing process. Therefore, it is necessary to compress the convolutional recurrent network by reducing the amount of its parameters while the model is utilized to solve the prediction problem of traffic flow, especially on some resource-limited devices. In this paper, we propose a sparse convolutional recurrent network by using the sparse gate in ConvLSTM and ConvGRU to reduce the resource requirement. The contributions of this paper are summarized as follows:(i)
Based on ConvLSTM and ConvGRU, we develop four sparse convolutional recurrent networks: SConvLSTM, SConvLSTM+, SConvGRU, and SConvGRU+. The numbers of the parameters of our proposed networks are significantly reduced as opposed to their original versions (ConvLSTM and ConvGRU) on the basis of keeping the performance of the algorithms.(ii)
In the original gating unit of ConvGRU and ConvLSTM, we design a sparse gating mechanism, which reduces the parameters of the networks (ConvGRU and ConvLSTM) by reducing the input data of the gating unit.(iii)
We evaluate our proposed network on three real traffic flow datasets. Compared with the baseline model, our proposed network reduces the resource consumption and saves the training time under the condition of competitive prediction accuracy.The outline of this paper is as follows: section2 describes the related work of traffic flow prediction and sparse neural network. Then, the notation and problem definition are given in Section 3. Section 4 describes our proposed model (sparse convolutional recurrent network) in detail. In addition, Section 5 gives specific parameter settings and experimental analysis based on three real traffic flow datasets. Finally, in Section 6, we summarize the work of this paper and make an outlook for future work.


## 2. Related Work

### 2.1. Traffic Flow Prediction

In essence, traffic flow prediction is a time series forecasting problem, which predicts the future traffic value based on historical observations. Generally speaking, traffic flow forecasting methods can be divided into two categories: traditional prediction approaches and deep learning-based prediction approaches.The first category is the traditional forecasting methods. HA [13] is a classic traffic flow prediction method, which uses the historical average as the prediction value of the next time interval. However, it can only estimate the traffic data of future time intervals and cannot capture the correlation between different time intervals. ARMA [14] prediction is a time series forecasting method for processing stationary series data, which builds the model to predict future values based on the autoregression (AR) model and the moving average (MA) model. ARIMA [15] is a traditional time series prediction approach used to deal with nonstationary series data. Firstly, the input data is converted into stationary series data by the kth order difference, and then the ARMA method is used to build a prediction model. The disadvantage of them is that they cannot capture the nonlinear characteristics of traffic data. VAR [16] was proposed by Sims et al. in 1980 for time series prediction, which uses the form of simultaneous equations to obtain the relationship between different traffic flows. In addition, the variant methods derived from them are SARIMA [17], SARIMAX [18], VARMA [19], and so on. The limitation of all these traditional methods is that it is difficult to capture the complex spatial and temporal features of traffic flow data because of the limited capacity of the model.The second category is deep learning-based prediction approaches. Deep learning-based methods have a strong ability in dealing with nonlinear structural data and complex spatiotemporal series data. Therefore, these methods are introduced into predicting traffic flow by many researchers. Fu et al. [20] first employed the LSTM and GRU networks in the field of traffic flow forecasting, and the experimental results on the PeMS dataset prove that the prediction performance is better than the ARIMA model. To capture the hidden patterns of traffic flow data, Dai et al. [21] proposed a DeepTrend model based on a fully connected neural network and an LSTM network. Kang et al. [22] used the LSTM network to analyze the impact of different forms of traffic data on the prediction results, and the experimental results show that more external information is helpful to improve the performance of the model. Luo et al. [23] developed a deep learning method based on K-Nearest Neighbors (KNN) model [24] and the LSTM network. The experimental results on the real-time traffic flow dataset show that it achieves better performance than the existing prediction model. To capture the proximity, periodicity, and trend of traffic data, Wang et al. [25] proposed a deep learning model based on a convolutional recurrent network, which can effectively extract temporal dependence and spatial dependence. Chen et al. [26] developed a deep learning method based on Convolutional Neural Network (CNN) [27], LSTM network, and a fully connected neural network. The results show that the prediction accuracy is improved compared with LSTM and its variants. To improve the accuracy of taxi demand prediction, Li et al. [28] proposed a model based on deep learning, which uses a ConvLSTM network to capture spatiotemporal features.


### 2.2. Sparse Method of Neural Network

The deepening in the number of layers of the neural network brings about an increase in parameters and resource consumption. Therefore, it is necessary to compress the neural network under the premise of no significant performance degradation.To improve the training speed and generalization ability of the network, Louizos et al. [29] developed a sparse method using Lo regularization, which reduces the model complexity by pruning parameters. Liu et al. [30] compressed the model by reducing the number of redundant parameters. The experimental results prove that the speed of the training is effectively improved under the condition of minimizing the loss of accuracy. To improve computational performance and reduce the transmission of redundant data, Mukkara et al. [31] proposed a sparse convolutional neural network, which uses a strategy of pruning zero-valued weights to compress parameters. Dettmers et al. [32] proposed a sparse momentum to improve the training speed of deep neural networks. The experimental results on MNIST, CIFAR-10, and ImageNet datasets show that the sparse momentum achieves the most advanced sparse performance. To solve the problem that performance is limited by resources, Alford et al. [33] developed a pruning-based sparse neural network, which prunes the low weight of the trained densely connected network. Luo et al. [34] proposed a compression method for deep neural networks, which reduces the parameters of the network by pruning filters. In this paper, we develop a sparse gating mechanism for convolutional recurrent neural networks, which compresses the model by reducing the input data of the gating mechanism.


## 2.1. Traffic Flow Prediction

In essence, traffic flow prediction is a time series forecasting problem, which predicts the future traffic value based on historical observations. Generally speaking, traffic flow forecasting methods can be divided into two categories: traditional prediction approaches and deep learning-based prediction approaches.The first category is the traditional forecasting methods. HA [13] is a classic traffic flow prediction method, which uses the historical average as the prediction value of the next time interval. However, it can only estimate the traffic data of future time intervals and cannot capture the correlation between different time intervals. ARMA [14] prediction is a time series forecasting method for processing stationary series data, which builds the model to predict future values based on the autoregression (AR) model and the moving average (MA) model. ARIMA [15] is a traditional time series prediction approach used to deal with nonstationary series data. Firstly, the input data is converted into stationary series data by the kth order difference, and then the ARMA method is used to build a prediction model. The disadvantage of them is that they cannot capture the nonlinear characteristics of traffic data. VAR [16] was proposed by Sims et al. in 1980 for time series prediction, which uses the form of simultaneous equations to obtain the relationship between different traffic flows. In addition, the variant methods derived from them are SARIMA [17], SARIMAX [18], VARMA [19], and so on. The limitation of all these traditional methods is that it is difficult to capture the complex spatial and temporal features of traffic flow data because of the limited capacity of the model.The second category is deep learning-based prediction approaches. Deep learning-based methods have a strong ability in dealing with nonlinear structural data and complex spatiotemporal series data. Therefore, these methods are introduced into predicting traffic flow by many researchers. Fu et al. [20] first employed the LSTM and GRU networks in the field of traffic flow forecasting, and the experimental results on the PeMS dataset prove that the prediction performance is better than the ARIMA model. To capture the hidden patterns of traffic flow data, Dai et al. [21] proposed a DeepTrend model based on a fully connected neural network and an LSTM network. Kang et al. [22] used the LSTM network to analyze the impact of different forms of traffic data on the prediction results, and the experimental results show that more external information is helpful to improve the performance of the model. Luo et al. [23] developed a deep learning method based on K-Nearest Neighbors (KNN) model [24] and the LSTM network. The experimental results on the real-time traffic flow dataset show that it achieves better performance than the existing prediction model. To capture the proximity, periodicity, and trend of traffic data, Wang et al. [25] proposed a deep learning model based on a convolutional recurrent network, which can effectively extract temporal dependence and spatial dependence. Chen et al. [26] developed a deep learning method based on Convolutional Neural Network (CNN) [27], LSTM network, and a fully connected neural network. The results show that the prediction accuracy is improved compared with LSTM and its variants. To improve the accuracy of taxi demand prediction, Li et al. [28] proposed a model based on deep learning, which uses a ConvLSTM network to capture spatiotemporal features.


## 2.2. Sparse Method of Neural Network

The deepening in the number of layers of the neural network brings about an increase in parameters and resource consumption. Therefore, it is necessary to compress the neural network under the premise of no significant performance degradation.To improve the training speed and generalization ability of the network, Louizos et al. [29] developed a sparse method using Lo regularization, which reduces the model complexity by pruning parameters. Liu et al. [30] compressed the model by reducing the number of redundant parameters. The experimental results prove that the speed of the training is effectively improved under the condition of minimizing the loss of accuracy. To improve computational performance and reduce the transmission of redundant data, Mukkara et al. [31] proposed a sparse convolutional neural network, which uses a strategy of pruning zero-valued weights to compress parameters. Dettmers et al. [32] proposed a sparse momentum to improve the training speed of deep neural networks. The experimental results on MNIST, CIFAR-10, and ImageNet datasets show that the sparse momentum achieves the most advanced sparse performance. To solve the problem that performance is limited by resources, Alford et al. [33] developed a pruning-based sparse neural network, which prunes the low weight of the trained densely connected network. Luo et al. [34] proposed a compression method for deep neural networks, which reduces the parameters of the network by pruning filters. In this paper, we develop a sparse gating mechanism for convolutional recurrent neural networks, which compresses the model by reducing the input data of the gating mechanism.


## 3. Notations and Problem Definition

### 3.1. Grid Map

As shown in Figure1, we partition a city as an M×N grid map based on longitude and latitude [35], where a grid i.j indicates that this area is located in the ith row and the jth column in the map.Figure 1
Grid-based map segmentation.


### 3.2. Inflow/Outflow Matrix

Given thetth time interval, for the region gi,j, the inflow is defined as the sum of the crowd flow from other regions to the region gi,j at the tth time interval. The outflow is defined as the sum of the crowd flow from the region gi,j to other regions at the tth time interval. In this way, we can obtain the values of inflow and outflow of the region gi,j at the tth time interval. Then, using the same method for all areas, the values of inflow and outflow of the tth time interval can be obtained [33]. Figure 2 shows the inflow and outflow of all areas in a certain time interval in Beijing. Therefore, the inflow and outflow matrices at any time interval can be expressed as a tensor. X∈R2×m×n.Figure 2
Examples of inflow matrix (a) and outflow matrix (b) in Beijing.
(a)(b)We useΓ to represent all the crowd trajectories at the tth time interval st,et, and each trajectory is represented by a tuple. TrPstart,Pend,Tstart,Tend, which means that this trajectory starts at the region Pstart at time interval Tstart, and ends at the region Pend at time interval Tend. Then, the computing formula for inflow and outflow of the region gi,j can be defined as follows:(1)xt,ijin=∑Tr∈ΓPstart∉gi,j∧Pend∈gi,j∧Tstart∈st,et,xt,ijout=∑Tr∈ΓPstart∈gi,j∧Pend∈gi,j∧Tstart∉st,et,where Tr is a trajectory. xt,ijin/xt,ijout denotes the inflow/outflow in the region gi,j at the tth time interval. st represents the start time of the tth time interval. et represents the end time of the tth time interval, and ⋅ denotes the cardinality of a set.


### 3.3. Inflow/Outflow Prediction Problem

As shown in Figure3, traffic flow prediction is to predict the values of the traffic flow of the next k time intervals based on a series of the historical data of the previous l time intervals. The formula is as follows:(2)Xt−l+1,Xt−l,…,Xt⟶fX^t+1,X^t+2,…,X^t+k,where f denotes a flow prediction model, Xt−l+1,…,Xt represents the observation values of the previous l time interval, and X^t+1,X^t+2,…,X^t+k represents the predicted values of k time intervals in the future.Figure 3
Traffic flow prediction.


## 3.1. Grid Map

As shown in Figure1, we partition a city as an M×N grid map based on longitude and latitude [35], where a grid i.j indicates that this area is located in the ith row and the jth column in the map.Figure 1
Grid-based map segmentation.


## 3.2. Inflow/Outflow Matrix

Given thetth time interval, for the region gi,j, the inflow is defined as the sum of the crowd flow from other regions to the region gi,j at the tth time interval. The outflow is defined as the sum of the crowd flow from the region gi,j to other regions at the tth time interval. In this way, we can obtain the values of inflow and outflow of the region gi,j at the tth time interval. Then, using the same method for all areas, the values of inflow and outflow of the tth time interval can be obtained [33]. Figure 2 shows the inflow and outflow of all areas in a certain time interval in Beijing. Therefore, the inflow and outflow matrices at any time interval can be expressed as a tensor. X∈R2×m×n.Figure 2
Examples of inflow matrix (a) and outflow matrix (b) in Beijing.
(a)(b)We useΓ to represent all the crowd trajectories at the tth time interval st,et, and each trajectory is represented by a tuple. TrPstart,Pend,Tstart,Tend, which means that this trajectory starts at the region Pstart at time interval Tstart, and ends at the region Pend at time interval Tend. Then, the computing formula for inflow and outflow of the region gi,j can be defined as follows:(1)xt,ijin=∑Tr∈ΓPstart∉gi,j∧Pend∈gi,j∧Tstart∈st,et,xt,ijout=∑Tr∈ΓPstart∈gi,j∧Pend∈gi,j∧Tstart∉st,et,where Tr is a trajectory. xt,ijin/xt,ijout denotes the inflow/outflow in the region gi,j at the tth time interval. st represents the start time of the tth time interval. et represents the end time of the tth time interval, and ⋅ denotes the cardinality of a set.


## 3.3. Inflow/Outflow Prediction Problem

As shown in Figure3, traffic flow prediction is to predict the values of the traffic flow of the next k time intervals based on a series of the historical data of the previous l time intervals. The formula is as follows:(2)Xt−l+1,Xt−l,…,Xt⟶fX^t+1,X^t+2,…,X^t+k,where f denotes a flow prediction model, Xt−l+1,…,Xt represents the observation values of the previous l time interval, and X^t+1,X^t+2,…,X^t+k represents the predicted values of k time intervals in the future.Figure 3
Traffic flow prediction.


## 4. Sparse Convolutional Recurrent Neural Network

In this section, we introduce the details of the sparse convolutional recurrent neural network. There are two popular forms of convolutional recurrent neural networks: convolutional long short-term memory network and convolutional gated recurrent unit network. Based on the idea of eliminating redundant parameters in the sparse neural network methods, for convolutional long short-term memory network, we develop the sparse convolutional LSTM network (SConvLSTM) and SConvLSTM+. For the convolutional gated recurrent unit network, we develop the sparse convolutional GRU network (SConvGRU) and SConvGRU+.


### 4.1. Inflow/Outflow Prediction Problem

Figure4(a) is the convolutional long short-term memory unit proposed by Shi et al. [8], which is composed of a main line part and some gating units. The input of the ConvLSTM unit is the cell state Ct−1 and the hidden state Ht−1 of the previous time interval, and the feature matrix xt of the current time interval. The output is the cell state Ct and the hidden state Ht of the current time interval. The main line part is mainly about the update of the cell state and hidden state, and the specific update equation is as follows:(3)c˜=gWc⊛xt+Uc⊛ht−1+bc,ct=ft⊙ct−1+it⊙c˜,ht=ot⊙gct,where ⊙ represents the Hadamard product. ⊛ represents the convolutional operation. ct is the updated equation about the cell state. c˜ is the updated equation about the input state. ht is the updated equation about the hidden state. g represents the Tanh function. ft,it,ot represent the forget gate, the input gate, the output gate, respectively. xt denotes the input data of the tth time interval. Wc,Uc,bc are the learning parameters.Figure 4
(a) ConvLSTM unit. (b) SConvLSTM unit.
(a)(b)The gating unit is made of a forget gate, an input gate, and an output gate. The forget gate determines the degree of the forgetting ofCt−1. The input gate controls the degree of the input state to the cell state, and the output gate determines the degree of the cell state Ct to the output state ht. The gating mechanism of the ConvLSTM unit uses the hidden state ht−1 of the previous time interval, the input xt of the current time interval, and the bias as input. Then, the updated formula of the gating mechanism is as follows:(4)it=σWi⊛xt+Ui⊛ht−1+bi,ft=σWf⊛xt+Uf⊛ht−1+bf,ot=σWo⊛xt+Uo⊛ht−1+bo,where σ denotes the Sigmoid function. W,U,b represent the learning parameter.The SConvLSTM unit is shown in Figure4(b). We introduce a sparse gating mechanism to cut out the redundant parameters of the network. The main line part of the SConvLSTM unit is consistent with the ConvLSTM unit. The input of the gating mechanism in the ConvLSTM unit includes Ht−1,xt and the bias b, however, the input of the gating mechanism in the SConvLSTM unit is Ht−1 and the bias b. In this way, we can effectively reduce the complexity of the model while ensuring that xt is propagated to subsequent sequences in the input update formula c˜.In Section5, our experimental results on three traffic datasets verify this conclusion. The update formula of the gating mechanism in the SConvLSTM unit is as follows:(5)it=σUi⊛ht−1+bi,ft=σUf⊛ht−1+bf,ot=σUo⊛ht−1+bo.The SConvLSTM + unit removes the biasb based on the SConvLSTM unit. The amount of network parameters is further reduced. The update formula of the gating mechanism in the SConvLSTM + unit is as follows:(6)it=σUi⊛ht−1,ft=σUf⊛ht−1,ot=σUo⊛ht−1.


### 4.2. Sparse Convolutional GRU (SConvGRU) Network

As shown in Figure5(a), the ConvGRU unit is proposed by Ballas et al. [10] in 2016 to solve the problem of video representation. It is also composed of a main line part and gating units. Compared with ConvLSTM, it has fewer gating mechanisms and parameters, and it reduces the transmission of cell state in subsequent networks. The input of the ConvGRU unit includes the input xt of the current time interval and the hidden state ht−1 of the previous time interval, and the output is the hidden state ht of the current time interval. The main line part is about the update of the hidden state. The updated formula is as follows:(7)ht˜=gWh⊛xt+Uh⊛rt⊙ht−1+bh,ht=1−zt⊙ht−1+zt⊙ht˜.where zt,rt represent the updating gate and the reset gate, respectively. ht is the updating the hidden state. ht˜ represents the input update. W and b are the learning parameters.Figure 5
(a) ConvGRU unit. (b) SConvGRU unit.
(a)(b)The gating units are composed of an update gate and a reset gate. The reset gate determines the importance of the hidden stateht−1 of the previous time interval to the input update ht˜, and the update gate determines the influence of the current input state xt on ht. Like ConvLSTM, the input of the gating mechanism of the ConvGRU unit includes the input state xt of the current time interval, the hidden state ht−1 of the previous time interval, and the bias $b$. The specific update equation is as follows:(8)zt=σWz⊛xt+Uz⊛ht−1+bz,rt=σWr⊛xt+Ur⊛ht−1+br,where W, U, and b are the learning parameters.The SConvGRU unit is shown in Figure5(b). Like the SConvLSTM unit, we introduce a sparse gating mechanism to reduce the parameters of the reset gate and the update gate. The main line part of SConvGRU is consistent with ConvGRU, and the input of the gating mechanism is composed of the hidden state ht−1 of the previous time interval and bias b. The current input state xt is brought into the subsequent spatiotemporal sequence propagation by the input update formula. Compared with ConvGRU, SConvGRU can help to effectively reduce the number of parameters and the training time of the network while keeping a similar prediction accuracy. The updated equation of the gating mechanism in the SConvGRU unit is as follows:(9)zt=σUz⊛ht−1+bz,rt=σUr⊛ht−1+br,where U and b are the learning parameters.The SConvGRU + unit is based on the SConvGRU unit, and we further remove the biasb to reduce the amount of network parameters. The updated formula of the gating mechanism in the SConvGRU + unit is as follows:(10)zt=σUz⊛ht−1,rt=σUr⊛ht−1,where U is the learning parameter.


### 4.3. The Overall Framework of Traffic Flow Prediction

In this paper, we use the four algorithms: SConvLSTM, SConvLSTM+, SConvGRU, and SConvGRU + to forecast traffic flow. The overall framework of traffic flow prediction is shown in Figure6, in which the input of the model is the historical traffic flow matrix. Firstly, we use a multilayer convolutional neural network to extract the spatial dependence of the traffic data. Then, a multilayer network built by the above-mentioned units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) is used to capture the time dependence while further extracting spatial features. In addition, a multilayer deconvolutional neural network is employed to obtain a prediction matrix with the same dimension as the input.Figure 6
Traffic flow prediction model.X1,X2,…,Xt represents the historical traffic flow data. I1,I2,…,It denotes the output of the multilayer convolutional neural network. H1,H2,…,Ht represents the output of the multilayer network composed of the above-mentioned units. Convs denotes the multilayer convolutional neural network, and Deconvs denotes the multilayer deconvolutional neural network.


## 4.1. Inflow/Outflow Prediction Problem

Figure4(a) is the convolutional long short-term memory unit proposed by Shi et al. [8], which is composed of a main line part and some gating units. The input of the ConvLSTM unit is the cell state Ct−1 and the hidden state Ht−1 of the previous time interval, and the feature matrix xt of the current time interval. The output is the cell state Ct and the hidden state Ht of the current time interval. The main line part is mainly about the update of the cell state and hidden state, and the specific update equation is as follows:(3)c˜=gWc⊛xt+Uc⊛ht−1+bc,ct=ft⊙ct−1+it⊙c˜,ht=ot⊙gct,where ⊙ represents the Hadamard product. ⊛ represents the convolutional operation. ct is the updated equation about the cell state. c˜ is the updated equation about the input state. ht is the updated equation about the hidden state. g represents the Tanh function. ft,it,ot represent the forget gate, the input gate, the output gate, respectively. xt denotes the input data of the tth time interval. Wc,Uc,bc are the learning parameters.Figure 4
(a) ConvLSTM unit. (b) SConvLSTM unit.
(a)(b)The gating unit is made of a forget gate, an input gate, and an output gate. The forget gate determines the degree of the forgetting ofCt−1. The input gate controls the degree of the input state to the cell state, and the output gate determines the degree of the cell state Ct to the output state ht. The gating mechanism of the ConvLSTM unit uses the hidden state ht−1 of the previous time interval, the input xt of the current time interval, and the bias as input. Then, the updated formula of the gating mechanism is as follows:(4)it=σWi⊛xt+Ui⊛ht−1+bi,ft=σWf⊛xt+Uf⊛ht−1+bf,ot=σWo⊛xt+Uo⊛ht−1+bo,where σ denotes the Sigmoid function. W,U,b represent the learning parameter.The SConvLSTM unit is shown in Figure4(b). We introduce a sparse gating mechanism to cut out the redundant parameters of the network. The main line part of the SConvLSTM unit is consistent with the ConvLSTM unit. The input of the gating mechanism in the ConvLSTM unit includes Ht−1,xt and the bias b, however, the input of the gating mechanism in the SConvLSTM unit is Ht−1 and the bias b. In this way, we can effectively reduce the complexity of the model while ensuring that xt is propagated to subsequent sequences in the input update formula c˜.In Section5, our experimental results on three traffic datasets verify this conclusion. The update formula of the gating mechanism in the SConvLSTM unit is as follows:(5)it=σUi⊛ht−1+bi,ft=σUf⊛ht−1+bf,ot=σUo⊛ht−1+bo.The SConvLSTM + unit removes the biasb based on the SConvLSTM unit. The amount of network parameters is further reduced. The update formula of the gating mechanism in the SConvLSTM + unit is as follows:(6)it=σUi⊛ht−1,ft=σUf⊛ht−1,ot=σUo⊛ht−1.


## 4.2. Sparse Convolutional GRU (SConvGRU) Network

As shown in Figure5(a), the ConvGRU unit is proposed by Ballas et al. [10] in 2016 to solve the problem of video representation. It is also composed of a main line part and gating units. Compared with ConvLSTM, it has fewer gating mechanisms and parameters, and it reduces the transmission of cell state in subsequent networks. The input of the ConvGRU unit includes the input xt of the current time interval and the hidden state ht−1 of the previous time interval, and the output is the hidden state ht of the current time interval. The main line part is about the update of the hidden state. The updated formula is as follows:(7)ht˜=gWh⊛xt+Uh⊛rt⊙ht−1+bh,ht=1−zt⊙ht−1+zt⊙ht˜.where zt,rt represent the updating gate and the reset gate, respectively. ht is the updating the hidden state. ht˜ represents the input update. W and b are the learning parameters.Figure 5
(a) ConvGRU unit. (b) SConvGRU unit.
(a)(b)The gating units are composed of an update gate and a reset gate. The reset gate determines the importance of the hidden stateht−1 of the previous time interval to the input update ht˜, and the update gate determines the influence of the current input state xt on ht. Like ConvLSTM, the input of the gating mechanism of the ConvGRU unit includes the input state xt of the current time interval, the hidden state ht−1 of the previous time interval, and the bias $b$. The specific update equation is as follows:(8)zt=σWz⊛xt+Uz⊛ht−1+bz,rt=σWr⊛xt+Ur⊛ht−1+br,where W, U, and b are the learning parameters.The SConvGRU unit is shown in Figure5(b). Like the SConvLSTM unit, we introduce a sparse gating mechanism to reduce the parameters of the reset gate and the update gate. The main line part of SConvGRU is consistent with ConvGRU, and the input of the gating mechanism is composed of the hidden state ht−1 of the previous time interval and bias b. The current input state xt is brought into the subsequent spatiotemporal sequence propagation by the input update formula. Compared with ConvGRU, SConvGRU can help to effectively reduce the number of parameters and the training time of the network while keeping a similar prediction accuracy. The updated equation of the gating mechanism in the SConvGRU unit is as follows:(9)zt=σUz⊛ht−1+bz,rt=σUr⊛ht−1+br,where U and b are the learning parameters.The SConvGRU + unit is based on the SConvGRU unit, and we further remove the biasb to reduce the amount of network parameters. The updated formula of the gating mechanism in the SConvGRU + unit is as follows:(10)zt=σUz⊛ht−1,rt=σUr⊛ht−1,where U is the learning parameter.


## 4.3. The Overall Framework of Traffic Flow Prediction

In this paper, we use the four algorithms: SConvLSTM, SConvLSTM+, SConvGRU, and SConvGRU + to forecast traffic flow. The overall framework of traffic flow prediction is shown in Figure6, in which the input of the model is the historical traffic flow matrix. Firstly, we use a multilayer convolutional neural network to extract the spatial dependence of the traffic data. Then, a multilayer network built by the above-mentioned units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) is used to capture the time dependence while further extracting spatial features. In addition, a multilayer deconvolutional neural network is employed to obtain a prediction matrix with the same dimension as the input.Figure 6
Traffic flow prediction model.X1,X2,…,Xt represents the historical traffic flow data. I1,I2,…,It denotes the output of the multilayer convolutional neural network. H1,H2,…,Ht represents the output of the multilayer network composed of the above-mentioned units. Convs denotes the multilayer convolutional neural network, and Deconvs denotes the multilayer deconvolutional neural network.


## 5. Experiment

In this section, we evaluate our proposed method based on three real-world traffic flow datasets (TaxiBJ, BikeNYC, and TaxiNYC). Then, we describe our experimental process in detail from the aspects of datasets description, model comparison, evaluation metric, hyperparameter setting, and experimental analysis.


### 5.1. Datasets Description

As shown in Table1, The TaxiBJ dataset is collected from Beijing taxi GPS data from 07/01/2013 to 10/30/2013. Before the experiment, we, firstly, divide the city into 32×32 areas, set the time interval to 30 minutes, and count the traffic flow matrix within each 30 minutes. Furthermore, the data are divided into three subsets: the training sets is the data from 07/01/2013 to 10/20/2013, the validation set is the data from 10/21/2013 to 10/25/2013, and the test set is the data from 10/26/2013 to 10/30/2013.Table 1
The details of the datasets.
DatasetTaxiBJTaxiNYCBikeNYCStart time7/1/20131/1/20151/1/2016End time10/30/20133/1/20156/30/2016Training set7/1/2013–10/20/20131/1/2015–2/9/20151/1/2016–6/10/2016Validation set10/21/2013–10/25/20132/10/2015–2/19/20156/11/2016–6/20/2016Test set10/26/2013–10/30/20132/20/2015–3/1/20156/21/2016–6/30/2016Time interval0.50.51 (hour)Grid map size(32, 32)(10, 20)(16, 16)The TaxiNYC dataset comes from the New York Taxi System from 01/01/2015 to 03/01/2015. For this dataset, we divide the city into10×20 areas and set the time interval to 0.5 hour. In the data, we set the data from 01/01/2015 to 02/09/2015 and from 02/10/2015 to 02/19/2015 as the training set and validation set in the experiment, respectively. The rest of the data is used as the test set in the experiment.The BikeNYC dataset comes from the New York Citi bike system from 01/01/2016 to 06/30/2016. For this dataset, we, firstly, divide the city into16×16 areas and set the time interval to 1 hour. We set the training set, validation set, and test set to the data from 01/01/2016 to 6/10/2016, from 06/11/2016 to 06/20/2016, and from 06/21/2016 to 06/30/2016, respectively.


### 5.2. Baseline

In this paper, we employ the subsequent model to compare with our proposed model.(i)
HA [13]: HA is a classic time series forecasting method, which predicts the traffic flow using historical average(ii)
ARMA [14]: ARMA is a time series forecasting method for stationary series data, which is based on the AR model and MA model to predict future traffic flow(iii)
VAR [6]: VAR uses the form of simultaneous equations to obtain the linear relationship between different traffic flows(iv)
ConvLSTM [8]: ConvLSTM is a deep learning approach for spatiotemporal sequence prediction, which introduces the convolutional structure into the fully connected LSTM(v)
ConvGRU [10]: ConvGRU employs convolutional operation to capture spatial features while preserving the structure of the gated recurrent unit


### 5.3. Evaluation Metric

In the experiment, we predict the traffic flow of the next time interval based on the previous 10-steps of historical traffic data. We use the root mean square error (RMSE) as an evaluation metric to evaluate the performance of our proposed model. The formula is as follows:(11)RMSE=1N∑i∑jxij−xij^2,where xij denotes the ground truth of the region gi,j, xij^ represents the predicted value of the region gi,j, and N represents the number of prediction areas.


### 5.4. Experiment on TaxiBJ Dataset

#### 5.4.1. Hyperparameter Setting

For the TaxiBJ dataset, a Min-Max normalization is used to scale the data to0,1 at first. The overall framework of the model is shown in Section 4.3, and the detailed settings are as follows: a two-layer convolutional neural network is employed to extract the spatial features of the data. After each convolutional operation, we use batch normalization and the rectified linear unit (ReLU). Furthermore, the number of layers of the units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) is set to 1. In addition, a two-layer deconvolutional neural network is used to obtain the predicted value of the same dimension as the real value. The detailed description of each module is shown in Table 2.Table 2
The details of module on TaxiBJ dataset.
HyperparameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+CNN layers222222Number of filters in CNN8 (first layer), 16 (second layer)Kernel size in CNN(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Stride in CNN1 (first layer), 2 (second layer)Unit layers111111Hidden channels of unit646464646464Kernel size in unit(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Input of gating mechanismht−1,xt,bht−1,bht−1ht−1,xt,bht−1,bht−1DCNN layers222222Number of filters in DCNN8 (first layer), 2 (second layer)Kernel size in DCNN(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Stride in DCNN2 (first layer), 1 (second layer)Batch size161616161616Timestep101010101010Epoch300300300300300300OptimizerAdam [36]AdamAdamAdamAdamAdamLearning rate0.00020.00020.00020.00020.00020.0002StrategyEarly-stoppingEarly-stoppingEarly-stoppingEarly-stoppingEarly-stoppingEarly-stopping


#### 5.4.2. Convergence Analysis

Figures7(a) and 7(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From the figure, we can see that with the increase of training times, the RMSE on the training set and the validation set gradually decreases, and finally converges. Furthermore, we can see that the SConvGRU + model converges to a lower RMSE on the validation set, compared to the ConvGRU model and SConvGRU model. In addition, on the training set, the three models maintain a consistent convergence trend. In summary, the convergence speed and performance of our proposed methods do not reduce under the condition of compressing the original models.Figure 7
The loss curve of the model on the TaxiBJ dataset. (a) ConvGRU, SConvGRU, and SConvGRU + on the training set. (b) ConvGRU, SConvGRU, and SConvGRU + on the validation set. (c) ConvLSTM, SConvLSTM, and SConvLSTM + on the training set. (d) ConvLSTM, SConvLSTM, and SConvLSTM + on the validation set.
(a)(b)(c)(d)Figures7(c) and 7(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model on the training set and validation set. We can see from the figure that the three models basically maintain consistent convergence performance on the training set and validation set. Similarly, the model achieves convergence on the validation set. In a word, the convergence performance of our proposed models does not reduce.


#### 5.4.3. The Comparative Results of Different Models

Table3 describes the RMSE comparison between the traditional model and our proposed model on the test set. First of all, we can see that the deep learning method is better than the traditional predictive model, which shows that the deep learning method can better capture the nonlinear characteristics of the data. Compared with the ConvGRU model, the RMSE of the SConvGRU + model increases by 1.8%. This result shows that the parameters of the model are effectively reduced under the condition of losing limited prediction accuracy. In addition, compared with ConvLSTM, SConvLSTM + improves the prediction accuracy of the model by 4.5% while reducing the number of model parameters. In general, SConvLSTM achieves the best performance on the TaxiBJ dataset. This result shows the effectiveness of the sparse convolutional recurrent network.Table 3
The comparative results of different models.
ModelRMSEHA97.348ARMA21.324VAR22.945ConvGRU19.290SConvGRU19.484SConvGRU+19.632ConvLSTM19.637SConvLSTM19.270SConvLSTM+18.751Bold represents the best results.


#### 5.4.4. The Rate of Model Compression

Table4 shows the comparison of the rate of model’s compression. We use ConvGRU and ConvLSTM as the baselines to calculate the rate of parameter reduction of our proposed models. Compared with ConvGRU, the rates of parameter reduction of SConvGRU and SConvGRU + are 13.3% and 13.5%, respectively. Compared with ConvLSTM, the rates of the parameter reduction of SConvLSTM and SConvLSTM + are 15% and 15.1%, respectively. When the same hyperparameters in Table 2 are set, the number of the parameters of ConvLSTM, SConvLSTM, and SConvLSTM + are more than ConvGRU, SConvGRU, and SConvGRU+, respectively. The reason for this result is that ConvLSTM, SConvLSTM, and SConvLSTM + have more gating mechanism than ConvGRU, SConvGRU, and SConvGRU+.Table 4
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU1384320SConvGRU12000013.3%SConvGRU+11980813.5%ConvLSTM1845760SConvLSTM15692815%SConvLSTM+15667215.1%


### 5.5. Experiment on TaxiNYC Dataset

#### 5.5.1. Hyperparameter Setting

For the TaxiNYC dataset, similarly, we first use the Min-Max normalization approach to scale the original data to0,1. The framework of the model is shown in Section 4.3. Different to the setting on the TaxiBJ dataset, the number of layers of the units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) are set to 2, and the hidden channels of ConvGRU unit, SConvGRU unit, and SConvGRU + unit are set to 32 (first layer) and 64 (second layer). Furthermore, the hidden channels of ConvLSTM unit, SConvLSTM unit, and SConvLSTM + unit are set to 32 (first layer) and 32 (second layer). Then, the detailed settings are shown in Table 5. In addition, other hyperparameter settings are shown in Table 2, which are the same as those of the TaxiBJ dataset.Table 5
The details of the module on the TaxiNYC dataset.
HyperparameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+Kernel size in CNN(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)Stride in CNN1 (first layer), 1 (second layer)Unit layers222222Hidden channels of unit32 (first layer), 64 (second layer)32 (first layer), 32 (second layer)Kernel size in DCNN(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)Stride in DCNN1 (first layer), 1 (second layer)


#### 5.5.2. Convergence Analysis

Figures8(a) and 8(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From the figure, it can be seen that the three models maintain a consistent convergence curve. Furthermore, RMSE gradually decreases with the increase of epoch on the training set and validation set and finally converges.Figure 8
The loss curve of the model on the TaxiNYC dataset. (a): ConvGRU, SConvGRU, and SConvGRU + on the training set. (b) ConvGRU, SConvGRU, and SConvGRU + on the validation set. (c) ConvLSTM, SConvLSTM, and SConvLSTM + on the training set. (d) ConvLSTM, SConvLSTM, and SConvLSTM + on the validation set.
(a)(b)(c)(d)Figures8(c) and 8(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model. Firstly, the model achieves convergence on the validation set. Then, their convergence on the training set and validation set is basically the same, and the baseline model on the validation set converges to a lower value.


#### 5.5.3. The Comparative Results of Different Models

From Table6, in the traditional model, VAR achieved a good result of 14.306. Compared with ConvGRU, the RMSE of SConvGRU+ is increased by 0.07. Similarly, compared with ConvLSTM, the RMSE of SConvLSTM+ is increased by 0.029. This result shows that the parameters of the model are effectively reduced under the condition of losing limited prediction accuracy. On the whole, the prediction results of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model are better than the ConvGRU model, SConvGRU model, and SConvGRU + model. This result shows that ConvLSTM and sparse ConvLSTM can better capture the nonlinear structures on this dataset.Table 6
The comparative results of different models.
ModelRMSEHA39.764ARMA15.658VAR14.306ConvGRU12.851SConvGRU13.059SConvGRU+12.921ConvLSTM12.373SConvLSTM12.554SConvLSTM+12.402Bold represents the best results.


#### 5.5.4. The Rate of Model Compression

The rate of compression on different models are shown in Table7 on the TaxiNYC dataset. Because of the different number of hidden channels, the number of parameters of ConvLSTM-based models is less than that of ConvGRU-based models. Compared with the ConvGRU model, the compression rates of the SConvGRU model and the SConvGRU + model are 22.2% and 22.3%, respectively. Compared with the ConvLSTM model, the compression rates of the SConvLSTM model and the SConvLSTM + model are 32.1% and 32.3%, respectively.Table 7
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU2076480SConvGRU16156822.2%SConvGRU+16128022.3%ConvLSTM1292800SConvLSTM8780832.1%SConvLSTM+8755232.3%


### 5.6. Experiment on BikeNYC Dataset

#### 5.6.1. Hyperparameter Setting

Similarly, the overall framework of the model is shown in Section4.3, which consists of a two-layer convolutional neural network, a two-layer unit, and a two-layer deconvolutional neural network. In addition, the specific details of the framework are shown in Table 8. Other hyperparameter settings are shown in Table 2, which are the same as those in the TaxiBJ dataset.Table 8
The details of the module on the TaxiNYC dataset.
ParameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+Stride in CNN1 (first layer), 1 (second layer)Unit layers222222Stride in DCNN1 (first layer), 1 (second layer)


#### 5.6.2. Convergence Analysis

Figures9(a) and 9(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From Figure 9(a), we can see that the RMSE of the model gradually decreases with the increase of epoch. Furthermore, we can find that the baseline (ConvGRU) is reduced to a lower RMSE compared to SConvGRU and SConvGRU+ in the training process. However, ConvGRU has worse performance in the validation set compared with our proposed models. From Figure 9(b), we can see that the RMSE of the model decreases first and then increases with the increase of the epoch. The reason for this result may be that the model is overfitting. In addition, the SConvGRU model and the SConvGRU + model achieve better performance on the validation set, which proves that the prediction accuracy is not reduced under the condition of reducing the number of parameters.Figure 9
The loss curve of the model on the training set and validation set on the BikeNYC dataset. (a, b) ConvGRU, SConvGRU, and SConvGRU+. (c, d) ConvLSTM, SConvLSTM, and SConvLSTM+.
(a)(b)(c)(d)Figures9(c) and 9(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model. Like Figures 9(a) and 9(b), we can see that a lower RMSE on the training set can be gained by ConvLSTM compared to SConvLSTM and SConvLSTM+, however, ConvLSTM on the validation set shows worse performance than SConvLSTM and SConvLSTM+. The reason for this result may be that when training epoches exceed a certain number, the model is overfitting. On the validation set, the SConvLSTM model and the SConvLSTM + model achieve a lower RMSE.


#### 5.6.3. The Comparative Results of Different Models

In the BikeNYC dataset, it can be seen from Table9 that SConvLSTM achieves the best performance. Compared with the ConvGRU model, the SConvGRU model and the SConvGRU + model achieve a lower RMSE. Similarly, compared with the ConvLSTM model, the SConvLSTM model and the SConvLSTM + model also achieve better performance. This result proves that under the condition of reducing the parameters of the model, the prediction effect of the model is improved at the same time. The reason for this result maybe that part of the parameters of the gating mechanism in ConvLSTM and ConvGRU are redundant. Furthermore, on the whole, compared with the ConvGRU model, the SConvGRU model, and the SConvGRU + model, the ConvLSTM model, the SConvLSTM model, and SConvLSTM + model achieve better RMSE, which shows that the ConvLSTM model and the sparse long short-term memory network have a stronger ability to capture the spatiotemporal information.Table 9
The comparative results of different models.
ModelRMSEHA11.968ARMA15.463VAR6.478ConvGRU5.547SConvGRU5.084SConvGRU+5.259ConvLSTM4.994SConvLSTM4.852SConvLSTM+4.973Bold represents the best results.


#### 5.6.4. The Rate of Model Compression

Table10 shows the rate of compression on different models on the BikeNYC dataset. The ConvGRU model and ConvLSTM model are used as the benchmark to measure the rate of compression of our proposed model. Compared with the ConvGRU model, the compression rates of the SConvGRU model and the SConvGRU + model are 25.6% and 25.7%, respectively. Compared to the ConvLSTM model, the compression rates of the SConvLSTM model and the SConvLSTM + model are 28.8% and 28.9%, respectively.Table 10
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU3598080SConvGRU26764825.6%SConvGRU+26726425.7%ConvLSTM4797440SConvLSTM34150428.8%SConvLSTM+34099228.9%


## 5.1. Datasets Description

As shown in Table1, The TaxiBJ dataset is collected from Beijing taxi GPS data from 07/01/2013 to 10/30/2013. Before the experiment, we, firstly, divide the city into 32×32 areas, set the time interval to 30 minutes, and count the traffic flow matrix within each 30 minutes. Furthermore, the data are divided into three subsets: the training sets is the data from 07/01/2013 to 10/20/2013, the validation set is the data from 10/21/2013 to 10/25/2013, and the test set is the data from 10/26/2013 to 10/30/2013.Table 1
The details of the datasets.
DatasetTaxiBJTaxiNYCBikeNYCStart time7/1/20131/1/20151/1/2016End time10/30/20133/1/20156/30/2016Training set7/1/2013–10/20/20131/1/2015–2/9/20151/1/2016–6/10/2016Validation set10/21/2013–10/25/20132/10/2015–2/19/20156/11/2016–6/20/2016Test set10/26/2013–10/30/20132/20/2015–3/1/20156/21/2016–6/30/2016Time interval0.50.51 (hour)Grid map size(32, 32)(10, 20)(16, 16)The TaxiNYC dataset comes from the New York Taxi System from 01/01/2015 to 03/01/2015. For this dataset, we divide the city into10×20 areas and set the time interval to 0.5 hour. In the data, we set the data from 01/01/2015 to 02/09/2015 and from 02/10/2015 to 02/19/2015 as the training set and validation set in the experiment, respectively. The rest of the data is used as the test set in the experiment.The BikeNYC dataset comes from the New York Citi bike system from 01/01/2016 to 06/30/2016. For this dataset, we, firstly, divide the city into16×16 areas and set the time interval to 1 hour. We set the training set, validation set, and test set to the data from 01/01/2016 to 6/10/2016, from 06/11/2016 to 06/20/2016, and from 06/21/2016 to 06/30/2016, respectively.


## 5.2. Baseline

In this paper, we employ the subsequent model to compare with our proposed model.(i)
HA [13]: HA is a classic time series forecasting method, which predicts the traffic flow using historical average(ii)
ARMA [14]: ARMA is a time series forecasting method for stationary series data, which is based on the AR model and MA model to predict future traffic flow(iii)
VAR [6]: VAR uses the form of simultaneous equations to obtain the linear relationship between different traffic flows(iv)
ConvLSTM [8]: ConvLSTM is a deep learning approach for spatiotemporal sequence prediction, which introduces the convolutional structure into the fully connected LSTM(v)
ConvGRU [10]: ConvGRU employs convolutional operation to capture spatial features while preserving the structure of the gated recurrent unit


## 5.3. Evaluation Metric

In the experiment, we predict the traffic flow of the next time interval based on the previous 10-steps of historical traffic data. We use the root mean square error (RMSE) as an evaluation metric to evaluate the performance of our proposed model. The formula is as follows:(11)RMSE=1N∑i∑jxij−xij^2,where xij denotes the ground truth of the region gi,j, xij^ represents the predicted value of the region gi,j, and N represents the number of prediction areas.


## 5.4. Experiment on TaxiBJ Dataset

### 5.4.1. Hyperparameter Setting

For the TaxiBJ dataset, a Min-Max normalization is used to scale the data to0,1 at first. The overall framework of the model is shown in Section 4.3, and the detailed settings are as follows: a two-layer convolutional neural network is employed to extract the spatial features of the data. After each convolutional operation, we use batch normalization and the rectified linear unit (ReLU). Furthermore, the number of layers of the units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) is set to 1. In addition, a two-layer deconvolutional neural network is used to obtain the predicted value of the same dimension as the real value. The detailed description of each module is shown in Table 2.Table 2
The details of module on TaxiBJ dataset.
HyperparameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+CNN layers222222Number of filters in CNN8 (first layer), 16 (second layer)Kernel size in CNN(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Stride in CNN1 (first layer), 2 (second layer)Unit layers111111Hidden channels of unit646464646464Kernel size in unit(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Input of gating mechanismht−1,xt,bht−1,bht−1ht−1,xt,bht−1,bht−1DCNN layers222222Number of filters in DCNN8 (first layer), 2 (second layer)Kernel size in DCNN(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Stride in DCNN2 (first layer), 1 (second layer)Batch size161616161616Timestep101010101010Epoch300300300300300300OptimizerAdam [36]AdamAdamAdamAdamAdamLearning rate0.00020.00020.00020.00020.00020.0002StrategyEarly-stoppingEarly-stoppingEarly-stoppingEarly-stoppingEarly-stoppingEarly-stopping


### 5.4.2. Convergence Analysis

Figures7(a) and 7(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From the figure, we can see that with the increase of training times, the RMSE on the training set and the validation set gradually decreases, and finally converges. Furthermore, we can see that the SConvGRU + model converges to a lower RMSE on the validation set, compared to the ConvGRU model and SConvGRU model. In addition, on the training set, the three models maintain a consistent convergence trend. In summary, the convergence speed and performance of our proposed methods do not reduce under the condition of compressing the original models.Figure 7
The loss curve of the model on the TaxiBJ dataset. (a) ConvGRU, SConvGRU, and SConvGRU + on the training set. (b) ConvGRU, SConvGRU, and SConvGRU + on the validation set. (c) ConvLSTM, SConvLSTM, and SConvLSTM + on the training set. (d) ConvLSTM, SConvLSTM, and SConvLSTM + on the validation set.
(a)(b)(c)(d)Figures7(c) and 7(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model on the training set and validation set. We can see from the figure that the three models basically maintain consistent convergence performance on the training set and validation set. Similarly, the model achieves convergence on the validation set. In a word, the convergence performance of our proposed models does not reduce.


### 5.4.3. The Comparative Results of Different Models

Table3 describes the RMSE comparison between the traditional model and our proposed model on the test set. First of all, we can see that the deep learning method is better than the traditional predictive model, which shows that the deep learning method can better capture the nonlinear characteristics of the data. Compared with the ConvGRU model, the RMSE of the SConvGRU + model increases by 1.8%. This result shows that the parameters of the model are effectively reduced under the condition of losing limited prediction accuracy. In addition, compared with ConvLSTM, SConvLSTM + improves the prediction accuracy of the model by 4.5% while reducing the number of model parameters. In general, SConvLSTM achieves the best performance on the TaxiBJ dataset. This result shows the effectiveness of the sparse convolutional recurrent network.Table 3
The comparative results of different models.
ModelRMSEHA97.348ARMA21.324VAR22.945ConvGRU19.290SConvGRU19.484SConvGRU+19.632ConvLSTM19.637SConvLSTM19.270SConvLSTM+18.751Bold represents the best results.


### 5.4.4. The Rate of Model Compression

Table4 shows the comparison of the rate of model’s compression. We use ConvGRU and ConvLSTM as the baselines to calculate the rate of parameter reduction of our proposed models. Compared with ConvGRU, the rates of parameter reduction of SConvGRU and SConvGRU + are 13.3% and 13.5%, respectively. Compared with ConvLSTM, the rates of the parameter reduction of SConvLSTM and SConvLSTM + are 15% and 15.1%, respectively. When the same hyperparameters in Table 2 are set, the number of the parameters of ConvLSTM, SConvLSTM, and SConvLSTM + are more than ConvGRU, SConvGRU, and SConvGRU+, respectively. The reason for this result is that ConvLSTM, SConvLSTM, and SConvLSTM + have more gating mechanism than ConvGRU, SConvGRU, and SConvGRU+.Table 4
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU1384320SConvGRU12000013.3%SConvGRU+11980813.5%ConvLSTM1845760SConvLSTM15692815%SConvLSTM+15667215.1%


## 5.4.1. Hyperparameter Setting

For the TaxiBJ dataset, a Min-Max normalization is used to scale the data to0,1 at first. The overall framework of the model is shown in Section 4.3, and the detailed settings are as follows: a two-layer convolutional neural network is employed to extract the spatial features of the data. After each convolutional operation, we use batch normalization and the rectified linear unit (ReLU). Furthermore, the number of layers of the units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) is set to 1. In addition, a two-layer deconvolutional neural network is used to obtain the predicted value of the same dimension as the real value. The detailed description of each module is shown in Table 2.Table 2
The details of module on TaxiBJ dataset.
HyperparameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+CNN layers222222Number of filters in CNN8 (first layer), 16 (second layer)Kernel size in CNN(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Stride in CNN1 (first layer), 2 (second layer)Unit layers111111Hidden channels of unit646464646464Kernel size in unit(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Input of gating mechanismht−1,xt,bht−1,bht−1ht−1,xt,bht−1,bht−1DCNN layers222222Number of filters in DCNN8 (first layer), 2 (second layer)Kernel size in DCNN(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)(3, 3)Stride in DCNN2 (first layer), 1 (second layer)Batch size161616161616Timestep101010101010Epoch300300300300300300OptimizerAdam [36]AdamAdamAdamAdamAdamLearning rate0.00020.00020.00020.00020.00020.0002StrategyEarly-stoppingEarly-stoppingEarly-stoppingEarly-stoppingEarly-stoppingEarly-stopping


## 5.4.2. Convergence Analysis

Figures7(a) and 7(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From the figure, we can see that with the increase of training times, the RMSE on the training set and the validation set gradually decreases, and finally converges. Furthermore, we can see that the SConvGRU + model converges to a lower RMSE on the validation set, compared to the ConvGRU model and SConvGRU model. In addition, on the training set, the three models maintain a consistent convergence trend. In summary, the convergence speed and performance of our proposed methods do not reduce under the condition of compressing the original models.Figure 7
The loss curve of the model on the TaxiBJ dataset. (a) ConvGRU, SConvGRU, and SConvGRU + on the training set. (b) ConvGRU, SConvGRU, and SConvGRU + on the validation set. (c) ConvLSTM, SConvLSTM, and SConvLSTM + on the training set. (d) ConvLSTM, SConvLSTM, and SConvLSTM + on the validation set.
(a)(b)(c)(d)Figures7(c) and 7(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model on the training set and validation set. We can see from the figure that the three models basically maintain consistent convergence performance on the training set and validation set. Similarly, the model achieves convergence on the validation set. In a word, the convergence performance of our proposed models does not reduce.


## 5.4.3. The Comparative Results of Different Models

Table3 describes the RMSE comparison between the traditional model and our proposed model on the test set. First of all, we can see that the deep learning method is better than the traditional predictive model, which shows that the deep learning method can better capture the nonlinear characteristics of the data. Compared with the ConvGRU model, the RMSE of the SConvGRU + model increases by 1.8%. This result shows that the parameters of the model are effectively reduced under the condition of losing limited prediction accuracy. In addition, compared with ConvLSTM, SConvLSTM + improves the prediction accuracy of the model by 4.5% while reducing the number of model parameters. In general, SConvLSTM achieves the best performance on the TaxiBJ dataset. This result shows the effectiveness of the sparse convolutional recurrent network.Table 3
The comparative results of different models.
ModelRMSEHA97.348ARMA21.324VAR22.945ConvGRU19.290SConvGRU19.484SConvGRU+19.632ConvLSTM19.637SConvLSTM19.270SConvLSTM+18.751Bold represents the best results.


## 5.4.4. The Rate of Model Compression

Table4 shows the comparison of the rate of model’s compression. We use ConvGRU and ConvLSTM as the baselines to calculate the rate of parameter reduction of our proposed models. Compared with ConvGRU, the rates of parameter reduction of SConvGRU and SConvGRU + are 13.3% and 13.5%, respectively. Compared with ConvLSTM, the rates of the parameter reduction of SConvLSTM and SConvLSTM + are 15% and 15.1%, respectively. When the same hyperparameters in Table 2 are set, the number of the parameters of ConvLSTM, SConvLSTM, and SConvLSTM + are more than ConvGRU, SConvGRU, and SConvGRU+, respectively. The reason for this result is that ConvLSTM, SConvLSTM, and SConvLSTM + have more gating mechanism than ConvGRU, SConvGRU, and SConvGRU+.Table 4
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU1384320SConvGRU12000013.3%SConvGRU+11980813.5%ConvLSTM1845760SConvLSTM15692815%SConvLSTM+15667215.1%


## 5.5. Experiment on TaxiNYC Dataset

### 5.5.1. Hyperparameter Setting

For the TaxiNYC dataset, similarly, we first use the Min-Max normalization approach to scale the original data to0,1. The framework of the model is shown in Section 4.3. Different to the setting on the TaxiBJ dataset, the number of layers of the units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) are set to 2, and the hidden channels of ConvGRU unit, SConvGRU unit, and SConvGRU + unit are set to 32 (first layer) and 64 (second layer). Furthermore, the hidden channels of ConvLSTM unit, SConvLSTM unit, and SConvLSTM + unit are set to 32 (first layer) and 32 (second layer). Then, the detailed settings are shown in Table 5. In addition, other hyperparameter settings are shown in Table 2, which are the same as those of the TaxiBJ dataset.Table 5
The details of the module on the TaxiNYC dataset.
HyperparameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+Kernel size in CNN(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)Stride in CNN1 (first layer), 1 (second layer)Unit layers222222Hidden channels of unit32 (first layer), 64 (second layer)32 (first layer), 32 (second layer)Kernel size in DCNN(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)Stride in DCNN1 (first layer), 1 (second layer)


### 5.5.2. Convergence Analysis

Figures8(a) and 8(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From the figure, it can be seen that the three models maintain a consistent convergence curve. Furthermore, RMSE gradually decreases with the increase of epoch on the training set and validation set and finally converges.Figure 8
The loss curve of the model on the TaxiNYC dataset. (a): ConvGRU, SConvGRU, and SConvGRU + on the training set. (b) ConvGRU, SConvGRU, and SConvGRU + on the validation set. (c) ConvLSTM, SConvLSTM, and SConvLSTM + on the training set. (d) ConvLSTM, SConvLSTM, and SConvLSTM + on the validation set.
(a)(b)(c)(d)Figures8(c) and 8(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model. Firstly, the model achieves convergence on the validation set. Then, their convergence on the training set and validation set is basically the same, and the baseline model on the validation set converges to a lower value.


### 5.5.3. The Comparative Results of Different Models

From Table6, in the traditional model, VAR achieved a good result of 14.306. Compared with ConvGRU, the RMSE of SConvGRU+ is increased by 0.07. Similarly, compared with ConvLSTM, the RMSE of SConvLSTM+ is increased by 0.029. This result shows that the parameters of the model are effectively reduced under the condition of losing limited prediction accuracy. On the whole, the prediction results of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model are better than the ConvGRU model, SConvGRU model, and SConvGRU + model. This result shows that ConvLSTM and sparse ConvLSTM can better capture the nonlinear structures on this dataset.Table 6
The comparative results of different models.
ModelRMSEHA39.764ARMA15.658VAR14.306ConvGRU12.851SConvGRU13.059SConvGRU+12.921ConvLSTM12.373SConvLSTM12.554SConvLSTM+12.402Bold represents the best results.


### 5.5.4. The Rate of Model Compression

The rate of compression on different models are shown in Table7 on the TaxiNYC dataset. Because of the different number of hidden channels, the number of parameters of ConvLSTM-based models is less than that of ConvGRU-based models. Compared with the ConvGRU model, the compression rates of the SConvGRU model and the SConvGRU + model are 22.2% and 22.3%, respectively. Compared with the ConvLSTM model, the compression rates of the SConvLSTM model and the SConvLSTM + model are 32.1% and 32.3%, respectively.Table 7
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU2076480SConvGRU16156822.2%SConvGRU+16128022.3%ConvLSTM1292800SConvLSTM8780832.1%SConvLSTM+8755232.3%


## 5.5.1. Hyperparameter Setting

For the TaxiNYC dataset, similarly, we first use the Min-Max normalization approach to scale the original data to0,1. The framework of the model is shown in Section 4.3. Different to the setting on the TaxiBJ dataset, the number of layers of the units (SConvLSTM unit, SConvLSTM + unit, SConvGRU unit, and SConvGRU + unit) are set to 2, and the hidden channels of ConvGRU unit, SConvGRU unit, and SConvGRU + unit are set to 32 (first layer) and 64 (second layer). Furthermore, the hidden channels of ConvLSTM unit, SConvLSTM unit, and SConvLSTM + unit are set to 32 (first layer) and 32 (second layer). Then, the detailed settings are shown in Table 5. In addition, other hyperparameter settings are shown in Table 2, which are the same as those of the TaxiBJ dataset.Table 5
The details of the module on the TaxiNYC dataset.
HyperparameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+Kernel size in CNN(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)Stride in CNN1 (first layer), 1 (second layer)Unit layers222222Hidden channels of unit32 (first layer), 64 (second layer)32 (first layer), 32 (second layer)Kernel size in DCNN(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)(3, 5)Stride in DCNN1 (first layer), 1 (second layer)


## 5.5.2. Convergence Analysis

Figures8(a) and 8(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From the figure, it can be seen that the three models maintain a consistent convergence curve. Furthermore, RMSE gradually decreases with the increase of epoch on the training set and validation set and finally converges.Figure 8
The loss curve of the model on the TaxiNYC dataset. (a): ConvGRU, SConvGRU, and SConvGRU + on the training set. (b) ConvGRU, SConvGRU, and SConvGRU + on the validation set. (c) ConvLSTM, SConvLSTM, and SConvLSTM + on the training set. (d) ConvLSTM, SConvLSTM, and SConvLSTM + on the validation set.
(a)(b)(c)(d)Figures8(c) and 8(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model. Firstly, the model achieves convergence on the validation set. Then, their convergence on the training set and validation set is basically the same, and the baseline model on the validation set converges to a lower value.


## 5.5.3. The Comparative Results of Different Models

From Table6, in the traditional model, VAR achieved a good result of 14.306. Compared with ConvGRU, the RMSE of SConvGRU+ is increased by 0.07. Similarly, compared with ConvLSTM, the RMSE of SConvLSTM+ is increased by 0.029. This result shows that the parameters of the model are effectively reduced under the condition of losing limited prediction accuracy. On the whole, the prediction results of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model are better than the ConvGRU model, SConvGRU model, and SConvGRU + model. This result shows that ConvLSTM and sparse ConvLSTM can better capture the nonlinear structures on this dataset.Table 6
The comparative results of different models.
ModelRMSEHA39.764ARMA15.658VAR14.306ConvGRU12.851SConvGRU13.059SConvGRU+12.921ConvLSTM12.373SConvLSTM12.554SConvLSTM+12.402Bold represents the best results.


## 5.5.4. The Rate of Model Compression

The rate of compression on different models are shown in Table7 on the TaxiNYC dataset. Because of the different number of hidden channels, the number of parameters of ConvLSTM-based models is less than that of ConvGRU-based models. Compared with the ConvGRU model, the compression rates of the SConvGRU model and the SConvGRU + model are 22.2% and 22.3%, respectively. Compared with the ConvLSTM model, the compression rates of the SConvLSTM model and the SConvLSTM + model are 32.1% and 32.3%, respectively.Table 7
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU2076480SConvGRU16156822.2%SConvGRU+16128022.3%ConvLSTM1292800SConvLSTM8780832.1%SConvLSTM+8755232.3%


## 5.6. Experiment on BikeNYC Dataset

### 5.6.1. Hyperparameter Setting

Similarly, the overall framework of the model is shown in Section4.3, which consists of a two-layer convolutional neural network, a two-layer unit, and a two-layer deconvolutional neural network. In addition, the specific details of the framework are shown in Table 8. Other hyperparameter settings are shown in Table 2, which are the same as those in the TaxiBJ dataset.Table 8
The details of the module on the TaxiNYC dataset.
ParameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+Stride in CNN1 (first layer), 1 (second layer)Unit layers222222Stride in DCNN1 (first layer), 1 (second layer)


### 5.6.2. Convergence Analysis

Figures9(a) and 9(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From Figure 9(a), we can see that the RMSE of the model gradually decreases with the increase of epoch. Furthermore, we can find that the baseline (ConvGRU) is reduced to a lower RMSE compared to SConvGRU and SConvGRU+ in the training process. However, ConvGRU has worse performance in the validation set compared with our proposed models. From Figure 9(b), we can see that the RMSE of the model decreases first and then increases with the increase of the epoch. The reason for this result may be that the model is overfitting. In addition, the SConvGRU model and the SConvGRU + model achieve better performance on the validation set, which proves that the prediction accuracy is not reduced under the condition of reducing the number of parameters.Figure 9
The loss curve of the model on the training set and validation set on the BikeNYC dataset. (a, b) ConvGRU, SConvGRU, and SConvGRU+. (c, d) ConvLSTM, SConvLSTM, and SConvLSTM+.
(a)(b)(c)(d)Figures9(c) and 9(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model. Like Figures 9(a) and 9(b), we can see that a lower RMSE on the training set can be gained by ConvLSTM compared to SConvLSTM and SConvLSTM+, however, ConvLSTM on the validation set shows worse performance than SConvLSTM and SConvLSTM+. The reason for this result may be that when training epoches exceed a certain number, the model is overfitting. On the validation set, the SConvLSTM model and the SConvLSTM + model achieve a lower RMSE.


### 5.6.3. The Comparative Results of Different Models

In the BikeNYC dataset, it can be seen from Table9 that SConvLSTM achieves the best performance. Compared with the ConvGRU model, the SConvGRU model and the SConvGRU + model achieve a lower RMSE. Similarly, compared with the ConvLSTM model, the SConvLSTM model and the SConvLSTM + model also achieve better performance. This result proves that under the condition of reducing the parameters of the model, the prediction effect of the model is improved at the same time. The reason for this result maybe that part of the parameters of the gating mechanism in ConvLSTM and ConvGRU are redundant. Furthermore, on the whole, compared with the ConvGRU model, the SConvGRU model, and the SConvGRU + model, the ConvLSTM model, the SConvLSTM model, and SConvLSTM + model achieve better RMSE, which shows that the ConvLSTM model and the sparse long short-term memory network have a stronger ability to capture the spatiotemporal information.Table 9
The comparative results of different models.
ModelRMSEHA11.968ARMA15.463VAR6.478ConvGRU5.547SConvGRU5.084SConvGRU+5.259ConvLSTM4.994SConvLSTM4.852SConvLSTM+4.973Bold represents the best results.


### 5.6.4. The Rate of Model Compression

Table10 shows the rate of compression on different models on the BikeNYC dataset. The ConvGRU model and ConvLSTM model are used as the benchmark to measure the rate of compression of our proposed model. Compared with the ConvGRU model, the compression rates of the SConvGRU model and the SConvGRU + model are 25.6% and 25.7%, respectively. Compared to the ConvLSTM model, the compression rates of the SConvLSTM model and the SConvLSTM + model are 28.8% and 28.9%, respectively.Table 10
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU3598080SConvGRU26764825.6%SConvGRU+26726425.7%ConvLSTM4797440SConvLSTM34150428.8%SConvLSTM+34099228.9%


## 5.6.1. Hyperparameter Setting

Similarly, the overall framework of the model is shown in Section4.3, which consists of a two-layer convolutional neural network, a two-layer unit, and a two-layer deconvolutional neural network. In addition, the specific details of the framework are shown in Table 8. Other hyperparameter settings are shown in Table 2, which are the same as those in the TaxiBJ dataset.Table 8
The details of the module on the TaxiNYC dataset.
ParameterConvGRUSConvGRUSConvGRU+ConvLSTMSConvLSTMSConvLSTM+Stride in CNN1 (first layer), 1 (second layer)Unit layers222222Stride in DCNN1 (first layer), 1 (second layer)


## 5.6.2. Convergence Analysis

Figures9(a) and 9(b) are the loss curves of the ConvGRU model, SConvGRU model, and SConvGRU + model on the training set and validation set. From Figure 9(a), we can see that the RMSE of the model gradually decreases with the increase of epoch. Furthermore, we can find that the baseline (ConvGRU) is reduced to a lower RMSE compared to SConvGRU and SConvGRU+ in the training process. However, ConvGRU has worse performance in the validation set compared with our proposed models. From Figure 9(b), we can see that the RMSE of the model decreases first and then increases with the increase of the epoch. The reason for this result may be that the model is overfitting. In addition, the SConvGRU model and the SConvGRU + model achieve better performance on the validation set, which proves that the prediction accuracy is not reduced under the condition of reducing the number of parameters.Figure 9
The loss curve of the model on the training set and validation set on the BikeNYC dataset. (a, b) ConvGRU, SConvGRU, and SConvGRU+. (c, d) ConvLSTM, SConvLSTM, and SConvLSTM+.
(a)(b)(c)(d)Figures9(c) and 9(d) are the loss curves of the ConvLSTM model, SConvLSTM model, and SConvLSTM + model. Like Figures 9(a) and 9(b), we can see that a lower RMSE on the training set can be gained by ConvLSTM compared to SConvLSTM and SConvLSTM+, however, ConvLSTM on the validation set shows worse performance than SConvLSTM and SConvLSTM+. The reason for this result may be that when training epoches exceed a certain number, the model is overfitting. On the validation set, the SConvLSTM model and the SConvLSTM + model achieve a lower RMSE.


## 5.6.3. The Comparative Results of Different Models

In the BikeNYC dataset, it can be seen from Table9 that SConvLSTM achieves the best performance. Compared with the ConvGRU model, the SConvGRU model and the SConvGRU + model achieve a lower RMSE. Similarly, compared with the ConvLSTM model, the SConvLSTM model and the SConvLSTM + model also achieve better performance. This result proves that under the condition of reducing the parameters of the model, the prediction effect of the model is improved at the same time. The reason for this result maybe that part of the parameters of the gating mechanism in ConvLSTM and ConvGRU are redundant. Furthermore, on the whole, compared with the ConvGRU model, the SConvGRU model, and the SConvGRU + model, the ConvLSTM model, the SConvLSTM model, and SConvLSTM + model achieve better RMSE, which shows that the ConvLSTM model and the sparse long short-term memory network have a stronger ability to capture the spatiotemporal information.Table 9
The comparative results of different models.
ModelRMSEHA11.968ARMA15.463VAR6.478ConvGRU5.547SConvGRU5.084SConvGRU+5.259ConvLSTM4.994SConvLSTM4.852SConvLSTM+4.973Bold represents the best results.


## 5.6.4. The Rate of Model Compression

Table10 shows the rate of compression on different models on the BikeNYC dataset. The ConvGRU model and ConvLSTM model are used as the benchmark to measure the rate of compression of our proposed model. Compared with the ConvGRU model, the compression rates of the SConvGRU model and the SConvGRU + model are 25.6% and 25.7%, respectively. Compared to the ConvLSTM model, the compression rates of the SConvLSTM model and the SConvLSTM + model are 28.8% and 28.9%, respectively.Table 10
The rate of model compression.
ModelParameterRate of parameter reductionConvGRU3598080SConvGRU26764825.6%SConvGRU+26726425.7%ConvLSTM4797440SConvLSTM34150428.8%SConvLSTM+34099228.9%


## 6. Conclusion and Future Work

In this paper, we study how to compress the convolutional recurrent networks while keeping the competitive results with traditional algorithms. To solve this problem, we propose a sparse convolutional recurrent network framework, in which a sparse gating mechanism is developed. For ConvGRU, we develop the SConvGRU unit and SConvGRU + unit. For ConvLSTM, we develop the SConvLSTM unit and SConvLSTM + unit. Based on three real traffic flow datasets, the experimental results prove that our proposed methods are able to effectively reduce the number of model parameters while keeping the prediction accuracy.



---

*Source: 6446941-2022-02-22.xml*