# Learning Spatial-Temporal Features of Ride-Hailing Services with Fusion Convolutional Networks

Authors: Dapeng Zhang; Feng Xiao; Gang Kou; Jian Luo; Fan Yang
Journal: Journal of Advanced Transportation
 (2023)

---

## Abstract

Accurate and reliable taxi demand prediction is of great importance for intelligent planning and management in the transportation system. To collectively forecast the taxi demand in all regions of a city, many existing studies focus on the capturing of spatial and temporal correlations among regions but ignore the local statistical differences throughout the geographical layout of a city. This limits the further improvement of prediction accuracy. In this paper, we propose a new deep learning framework, called the locally connected spatial-temporal fully convolutional neural network ( LC-ST-FCN), to learn the spatial-temporal correlations and local statistical differences among regions simultaneously. We evaluate the proposed model on a real dataset from a ride-hailing service platform (DiDi Chuxing) and observe significant improvements compared with a bunch of baseline models. Besides, we further explore the working mechanism of the proposed model by visualizing its feature extraction processes. The visualization results showed that our approach can better localize and capture useful features from spatial-related regions.


---

## Body

## 1. Introduction

A taxi is one of the key public passenger transports for daily travel activities in urban cities. Recently, ride-hailing service platforms, such as Didi, Uber, and Lyft, have grown tremendously, which provide convenient ride services by connecting passengers and drivers via smartphone apps. Ride-hailing services have various advantages compared to taxis but still suffer from the spatial and temporal imbalance between the supply and demand [1–3]. To provide more convenient and efficient services, ride-hailing demand forecasting is of great importance. Accurate ride-hailing demand prediction can help enhance user experience, increase taxi utilization, and optimize traffic efficiency.In recent years, ride-hailing demand prediction has attracted a wide range of research interests. To realize more accurate and reliable demand predictions, both traditional time-series and machine learning models are thoroughly studied by researchers [4–12]. Among these works, different deep learning architectures have been developed to model the complex spatial and temporal dependencies among regions. Specifically, the convolutional neural network (CNN) and graph neural network (GNN)-based modules are used to capture spatial dependencies, while the recurrent and recursive network (e.g., RNN and LSTM)-based modules are used to learn temporal dependencies [13–15].Most of the existing studies focus on how to model the spatial and temporal properties of the demand data but ignore the local statistical differences among regions. However, due to the local statistics varying from region to region, the parameter-sharing scheme at the spatial level may not be valid. For example, radial cities often have a typical central structure, and it is inappropriate to use the same parameters to predict the demands for both the central and peripheral areas of the city.On the other hand, the models that process the temporal information by RNNs (or LSTMs) may not be able to accurately model the connections between spatial and temporal dynamics, and the training of the recurrent and recursive networks is time consuming. In addition to the recurrent architectures, experiments in many studies indicate that convolutional operations can be even more suitable for sequence modeling after appropriate modifications [7, 16–19].In this paper, we design a fully convolutional architecture named LC-ST-FCN (locally connected spatial-temporal fully convolutional neural network) to address the issues mentioned above. Specifically, in the LC-ST-FCN model, the 3D convolutional operations are used to extract features from the spatial and temporal dimensions simultaneously. The locally connected convolutional layers are adopted to deal with the local statistical differences among regions. Due to the fully convolutional architecture of the LC-ST-FCN, the spatial coordinates of regions are maintained throughout the process, and no spatial information is lost between layers. Hence, the prediction error of each region can be delivered to the previous layers independently during the backpropagation training stage and optimize the corresponding model parameters of the region.We evaluate the proposed model on a real dataset from a ride-hailing service platform (DiDi Chuxing) and observe significant improvements compared with a bunch of baseline models. Besides the predictive performance, we visualize the feature maps produced by the LC-ST-FCN model to investigate the working mechanism of the proposed model. The analysis results illustrate that (1) the fully convolutional architecture can accurately localize the most related regions for each target region, (2) the spatiotemporal features extracted by 3D convolutional layers are more powerful and suitable for the subsequent layers to learn, and (3) the locally connected layers can deal with the issue of local statistical differences.The remainder of the paper is organized as follows: Section2 reviews the existing literature. Section 3 describes the LC-ST-FCN model in detail. Section 4 compares the predictive performance between the proposed approach and the baseline models based on the real-world dataset extracted from DiDi Chuxing. Section 5 analyzes the spatial correlations among regions and the effectiveness of model structures. Section 6 concludes the study and proposes future research directions.


## 2. Related Work

With the rapid development of electronic sensors and wireless communication technology, real-time demand prediction using the spatial and temporal data collected by the Internet and mobile terminals has become a research hotspot in the field of transportation. For the consideration of the temporal dependencies among the traffic data in various time intervals, a series of time-series [4, 5, 11, 20] and machine learning [21, 22] methods are adopted to solve traffic prediction problems. However, these time-series methods are not capable of handling spatial dependencies, which limits their performance.In recent years, deep learning methods are increasingly being applied to transportation research and significantly improve the accuracy of many traffic prediction tasks, including traffic flow prediction, traffic congestion forecasting, and traffic demand prediction [23–25].To achieve accurate region-based forecasting, a number of deep learning models were designed to model complex spatiotemporal information, and state-of-the-art results were achieved [6–10, 12]. The research in [19] studied the predictive performance of deep neural networks (DNNs) on taxi demand forecasting problems, and their experimental results show that DNNs indeed outperform most traditional machine learning techniques, but such superior results can only be achieved with proper design of the right DNN architecture, where domain knowledge plays a key role. Recently, graph-based models, especially spectral-based graph convolutional networks, have attracted a lot of attention due to the effective representation of the graph structure data [26–29]. However, when it comes to the topic of taxi demand forecasting, graph-based models heavily rely on the graph structures generated based on artificial experience. Most of the previous studies have focused on the combinations of CNNs and RNNs, where RNN architectures were utilized to model temporal dependencies. The experimental results in [18] indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets.Different from the abovementioned previous studies, 3D convolutional operations are employed to simultaneously capture the spatial-temporal dependencies in this paper. 3D convolution-related models have achieved superior performance on many types of video analysis tasks, such as action, scene, and object recognition [17, 30, 31]; since for video analysis tasks, it is very important to effectively extract the spatiotemporal features from adjacent video frames. Inspired by the findings on face verification [32, 33], in our work, locally connected convolutional layers are used at the end of the model to obtain the final prediction results, without parameter sharing.


## 3. LC-ST-FCN Model

In this study, we partition a city into anI×J grid map based on the longitude and latitude, where a grid cell denotes a region. We divide the observation time period (e.g. one month) into T time intervals with the interval length Δt (e.g., 10 minutes). At the time interval t, the order demand generated in the region i,j is xti,j, and the demand matrix of the entire I×J region is Xt∈ℝI×J.Our goal is to predict how many orders will emerge during the future periodt∗Δt,t+1∗Δt in each region. That is, given the historical observations X0,X1,…,Xt−1, Xt is predicted. The LC-ST-FCN model shown in Figure 1 is developed to achieve this goal. Each input sample of our model is generated from historical observations and consists of two parts, the recent dependent sequence and the period-dependent sequence. We concatenate them together along the time dimension as one tensor Xtin. 3D convolutional operations are used to fuse the spatial and temporal information into multiple time intervals. Since 3D convolution is a natural extension of 2D convolution, features from both spatial and temporal dimensions can be learned simultaneously. Then, multiple 2D convolutional layers are used to extract and encode features from low to high level, and from local to global. With the increase in network depth, the receptive field of neurons also increases, allowing an increasing amount of spatial information to be learned. In particular, locally connected convolutional layers are used at the end of the model to obtain the final prediction results, without parameter sharing.Figure 1
End-to-end implementation of the LC-ST-FCN model.According to the findings in 2D CNNs [34], small receptive fields of 3 × 3 convolutional kernels with deeper architectures yield the best results. Hence, in our LC-ST-FCN model, we fix the spatial receptive field to 3×3 and vary only the temporal depth of 3D convolutional filters. The 3D convolutional filters have access to information across all input demand matrices after several layers, depending on the depth of the input data. The important components and training methods of the LC-ST-FCN model are introduced in the following sections.


### 3.1. Input: 3D Volume

As a class of attractive deep models for automated spatial feature construction, CNNs have been primarily applied to 2D images. However, for short-term passenger demand forecasting, it is also important to capture the temporal information in multiple adjacent or periodic time intervals. For instance, the traffic demand is expected to be high during peak hours and to be low during the off-peak hours of the day, and the traffic demand patterns of weekdays are very different from those of the weekend. Therefore, we construct the following input data forms, which integrate the spatial-temporal information of multiple time intervals into one 3D volume, with one demand matrix for each time interval:(1)Xtin=Xtrec;Xtper,Xtin∈ℝI×J×2τ,Xtrec=Xt,Xt−1,…,Xt−τ+1,Xtrec∈ℝI×J×τ,Xtper=Xt−l+1,Xt−l,…,Xt−l−τ,Xtper∈ℝI×J×τ,where τ is the number of time intervals in Xtrec and Xtper, Xtrec represents the recent term, Xtper represents the period term, and l denotes the number of time intervals between Xtrec and Xtper. In urban cities, demand patterns always show the characteristics of periodicity. The reasons for the change in demand patterns on consecutive workdays can be very complicated. For example, the tail number limited policy has been implemented in many cities in China, and people are more likely to take a taxi when their cars are restricted. Thus, we use both current and historical data to improve the predictive performance of the model. As shown in Figure 2, each input instance used by the LC-ST-FCN model is a 3D volume containing 2τ time intervals, with each time interval corresponding to a matrix of I×J grid values.Figure 2
The 3D volume constructed to describe the spatial and temporal information of demand data.In this paper, the additive model [35] is used to explore periodicity, and we find that the demand pattern of a day is very similar to that of the same day a week ago. By decomposing the demand data with l=144 (e.g., number of time intervals in a day) and l=1008 (e.g., number of time intervals in a week), it can be observed that, when l=1008, the trend term is almost linear and the variance of the residual term is much smaller than l=144 (as shown in Figure 3). Hence, we select l=1008 to construct our training 3D tensor for training.Figure 3
Temporal dependencies of demand data. (a) Trend term. (b) Residual term.
(a)(b)


### 3.2. The Fusion of 3D and 2D Convolutions

In 2D CNNs, convolutions are applied to the 2D feature maps to compute features from spatial dimensions only. When the 2D CNN is applied to the demand forecasting problem, it is desirable to capture the temporal information in multiple time intervals, e.g., trends and periods. Although 2D CNNs can also take multiple time intervals as the input, after the first convolution layer, temporal information is collapsed completely. Hence, it is difficult to effectively extract temporal information by 2D convolutional operations. Compared to 2D convolution, 3D convolution has the ability to model the temporal information better, due to that 3D convolution preserves the temporal information of the input by maintaining the 3D volume as the output.In this section, we first introduce the implementation of 2D convolutional operations and further define the calculation of 3D convolutional operations to show their differences. For the sake of simplicity and flexibility, we denote the input of these two operations asXin∈ℝI×J×2τ. In the 2D convolutional layer, a group of filters ℱ2Dmm=1M with size ℝH×H×2τ and stride 1 are used to extract local features from the input 3D volume:(2)S2Dt,m=σW2Dm∗Xint+b2Dm,S2Dt,m∈ℝI×J×1,S2Dt=S2Dt,1,…,S2Dt,m,…,S2Dt,M,S2Dt∈ℝI×J×M,where “∗” denotes the convolutional operation, performing element-wise multiplications between W2Dm and Xint at any position when sliding the filter ℱ2Dm across the width and height dimensions, σ represents a nonlinear activation function (e.g., ReLU), W2Dm is the weight matrix of the filter ℱ2Dm with b2Dm as the bias term, and M is the number of filters in each group. For each filter ℱ2Dm, after convolutional operations, its output S2Dt,m will compress the temporal dimension of Xint from ℝM to ℝ1.On the other hand, in the 3D convolutional layer, a group of filtersℱ3Dmm=1M with size ℝH×H×F and stride 1 are also used to extract local features from the input 3D volume. Specifically, the depth size F of each ℱ3Dm is less than 2τ, which enables the filters in the 3D convolutional layer to perform convolutional operations along the temporal dimension. Thus, the local temporal dependencies of the input can be extracted.(3)Xint=Xint,1,…,Xint,p,…,Xint,2τ−F+1,Xint,p∈ℝI×J×F,S3Dt,p,m=σW3Dm∗Xint,p+b3Dm,S3Dt,p,m∈ℝI×J×1,S3Dt=S3Dt,1,1⋯S3Dt,1,M⋮⋱⋮S3Dt,2τ−F+1,1⋯S3Dt,2τ−F+1,M,where W3Dm is the weight matrix of the filter ℱ3Dm with b3Dm as the bias term and Xint,p is a data patch obtained by sliding over the temporal dimension. Given the temporal size of Xintis 2τ, 2τ−F+1 patches can be produced. S3Dt,p,m denotes the output of ℱ3Dm on one patch p, and the output of ℱ3Dm on all the patches is S3Dt∈ℝI×J×2τ−F+1. Since the outputs produced by the 3D convolutional operations are 3-dimensional feature maps rather than 2-dimensional feature maps, it is more suitable to extract the temporal dependencies among the input. By stacking multiple 3D convolutional layers together, they can extract spatial and temporal features from local to global.


### 3.3. Locally Connected Convolutional Layer

The setting of parameter sharing in 2D convolutional layers assumes local features. For example, if a horizontal boundary in some parts of the image is regarded as important, then it will be equally useful in other places. However, the parameter sharing assumption may not be applicable to demand forecasting problems as local features vary from region to region. For example, radial cities often have a typical central structure, and it is inappropriate to use the same parameters to predict the demands for both the central and peripheral areas of the city.In the LC-ST-FCN model, we relax the assumption of parameter sharing in 2D convolution layers by using the locally connected layers at the end of the model. In this way, the prediction value of each region is obtained by a different set of parameters, and the prediction error of each region can be delivered to the previous layers independently during the backpropagation training process. As shown in Figure4(b), in locally connected convolutional layers, the filters in different spatial locations have different parameters. Compared with fully connected layers (Figure 4(a)) or 2D convolutional layers, locally connected convolution layers can simultaneously maintain local statistics and spatial coordinates.Figure 4
Schematic diagram showing the transformation of fully connected layers into locally connected convolution layers to output a heatmap. (a) The fully connected layers take spatial inputs and produce nonspatial outputs. (b) Transforming fully connected layers into locally connected convolutional layers to avoid spatial information loss.
(a)(b)


### 3.4. Algorithm and Optimization

The LC-ST-FCN model can be trained by minimizing the mean squared error between the estimated demandY^t and the real demand Yt. The objective function is shown in Equation (4), where w and b are both learnable parameters.(4)minW,bY−Y^22.Algorithm1 outlines the LC-ST-FCN training process. We first construct the training instance from the historical observations (lines 1–8), where D is the set of training instances. Then, the LC-ST-FCN model is trained via backpropagation and the adaptive subgradient method (AdaGrad) (lines 9–28). Under the parameter-sharing scheme of 2D and 3D convolutional layers, the same wm and bm are used when we slide each filter m across the width, height, and depth of each input instance in one minibatch (line 16). For each filter m, in locally connected convolutional layers, different wm′ and bm′ will be used in different locations when we slide each filter across each input instance in one minibatch (line 22).Algorithm 1: LC-ST-FCN training algorithm.
Input: Historical demand of each region: X0,X2,…,Xt−1; lengths of input data sequence: 2τ; length of the period interval: l.Output: Learned LC-ST-FCN model.(1)
//construct training instance:(2)
D←∅(3)
for all the available time interval t1≤t≤ndo(4)
Xtrec=Xt,Xt−1,…,Xt−τ+1,Xtrec∈ℝI×J×τ(5)
Xtper=Xt−l+1,Xt−l,…,Xt−l−τ,Xtper∈ℝI×J×τ(6)
//Xtin is the target at time t(7)
put a training instanceXtin=Xtrec;Xtper into D(8)
end for(9)
//Training:(10)
repeat(11)
Initialize the biases and weights at each layer;(12)
Sample minibatch fromD randomly;(13)
if in 2D or 3D convolution layers then(14)
for filters m∈Mdo(15)
//Parameter sharing(16)
optimize learnable parameterswm and bm(17)
end for(18)
end if(19)
if in locally connected convolution layers then(20)
for filters m∈Mdo(21)
//Without parameter sharing(22)
optimize learnable parameterswm′ and bm′(23)
end for(24)
end if(25)
Calculate the stochastic gradient by minimizing the objective function (4);(26)
Update the parameters via backpropagation;(27)
untilstopping criteria are met


## 3.1. Input: 3D Volume

As a class of attractive deep models for automated spatial feature construction, CNNs have been primarily applied to 2D images. However, for short-term passenger demand forecasting, it is also important to capture the temporal information in multiple adjacent or periodic time intervals. For instance, the traffic demand is expected to be high during peak hours and to be low during the off-peak hours of the day, and the traffic demand patterns of weekdays are very different from those of the weekend. Therefore, we construct the following input data forms, which integrate the spatial-temporal information of multiple time intervals into one 3D volume, with one demand matrix for each time interval:(1)Xtin=Xtrec;Xtper,Xtin∈ℝI×J×2τ,Xtrec=Xt,Xt−1,…,Xt−τ+1,Xtrec∈ℝI×J×τ,Xtper=Xt−l+1,Xt−l,…,Xt−l−τ,Xtper∈ℝI×J×τ,where τ is the number of time intervals in Xtrec and Xtper, Xtrec represents the recent term, Xtper represents the period term, and l denotes the number of time intervals between Xtrec and Xtper. In urban cities, demand patterns always show the characteristics of periodicity. The reasons for the change in demand patterns on consecutive workdays can be very complicated. For example, the tail number limited policy has been implemented in many cities in China, and people are more likely to take a taxi when their cars are restricted. Thus, we use both current and historical data to improve the predictive performance of the model. As shown in Figure 2, each input instance used by the LC-ST-FCN model is a 3D volume containing 2τ time intervals, with each time interval corresponding to a matrix of I×J grid values.Figure 2
The 3D volume constructed to describe the spatial and temporal information of demand data.In this paper, the additive model [35] is used to explore periodicity, and we find that the demand pattern of a day is very similar to that of the same day a week ago. By decomposing the demand data with l=144 (e.g., number of time intervals in a day) and l=1008 (e.g., number of time intervals in a week), it can be observed that, when l=1008, the trend term is almost linear and the variance of the residual term is much smaller than l=144 (as shown in Figure 3). Hence, we select l=1008 to construct our training 3D tensor for training.Figure 3
Temporal dependencies of demand data. (a) Trend term. (b) Residual term.
(a)(b)


## 3.2. The Fusion of 3D and 2D Convolutions

In 2D CNNs, convolutions are applied to the 2D feature maps to compute features from spatial dimensions only. When the 2D CNN is applied to the demand forecasting problem, it is desirable to capture the temporal information in multiple time intervals, e.g., trends and periods. Although 2D CNNs can also take multiple time intervals as the input, after the first convolution layer, temporal information is collapsed completely. Hence, it is difficult to effectively extract temporal information by 2D convolutional operations. Compared to 2D convolution, 3D convolution has the ability to model the temporal information better, due to that 3D convolution preserves the temporal information of the input by maintaining the 3D volume as the output.In this section, we first introduce the implementation of 2D convolutional operations and further define the calculation of 3D convolutional operations to show their differences. For the sake of simplicity and flexibility, we denote the input of these two operations asXin∈ℝI×J×2τ. In the 2D convolutional layer, a group of filters ℱ2Dmm=1M with size ℝH×H×2τ and stride 1 are used to extract local features from the input 3D volume:(2)S2Dt,m=σW2Dm∗Xint+b2Dm,S2Dt,m∈ℝI×J×1,S2Dt=S2Dt,1,…,S2Dt,m,…,S2Dt,M,S2Dt∈ℝI×J×M,where “∗” denotes the convolutional operation, performing element-wise multiplications between W2Dm and Xint at any position when sliding the filter ℱ2Dm across the width and height dimensions, σ represents a nonlinear activation function (e.g., ReLU), W2Dm is the weight matrix of the filter ℱ2Dm with b2Dm as the bias term, and M is the number of filters in each group. For each filter ℱ2Dm, after convolutional operations, its output S2Dt,m will compress the temporal dimension of Xint from ℝM to ℝ1.On the other hand, in the 3D convolutional layer, a group of filtersℱ3Dmm=1M with size ℝH×H×F and stride 1 are also used to extract local features from the input 3D volume. Specifically, the depth size F of each ℱ3Dm is less than 2τ, which enables the filters in the 3D convolutional layer to perform convolutional operations along the temporal dimension. Thus, the local temporal dependencies of the input can be extracted.(3)Xint=Xint,1,…,Xint,p,…,Xint,2τ−F+1,Xint,p∈ℝI×J×F,S3Dt,p,m=σW3Dm∗Xint,p+b3Dm,S3Dt,p,m∈ℝI×J×1,S3Dt=S3Dt,1,1⋯S3Dt,1,M⋮⋱⋮S3Dt,2τ−F+1,1⋯S3Dt,2τ−F+1,M,where W3Dm is the weight matrix of the filter ℱ3Dm with b3Dm as the bias term and Xint,p is a data patch obtained by sliding over the temporal dimension. Given the temporal size of Xintis 2τ, 2τ−F+1 patches can be produced. S3Dt,p,m denotes the output of ℱ3Dm on one patch p, and the output of ℱ3Dm on all the patches is S3Dt∈ℝI×J×2τ−F+1. Since the outputs produced by the 3D convolutional operations are 3-dimensional feature maps rather than 2-dimensional feature maps, it is more suitable to extract the temporal dependencies among the input. By stacking multiple 3D convolutional layers together, they can extract spatial and temporal features from local to global.


## 3.3. Locally Connected Convolutional Layer

The setting of parameter sharing in 2D convolutional layers assumes local features. For example, if a horizontal boundary in some parts of the image is regarded as important, then it will be equally useful in other places. However, the parameter sharing assumption may not be applicable to demand forecasting problems as local features vary from region to region. For example, radial cities often have a typical central structure, and it is inappropriate to use the same parameters to predict the demands for both the central and peripheral areas of the city.In the LC-ST-FCN model, we relax the assumption of parameter sharing in 2D convolution layers by using the locally connected layers at the end of the model. In this way, the prediction value of each region is obtained by a different set of parameters, and the prediction error of each region can be delivered to the previous layers independently during the backpropagation training process. As shown in Figure4(b), in locally connected convolutional layers, the filters in different spatial locations have different parameters. Compared with fully connected layers (Figure 4(a)) or 2D convolutional layers, locally connected convolution layers can simultaneously maintain local statistics and spatial coordinates.Figure 4
Schematic diagram showing the transformation of fully connected layers into locally connected convolution layers to output a heatmap. (a) The fully connected layers take spatial inputs and produce nonspatial outputs. (b) Transforming fully connected layers into locally connected convolutional layers to avoid spatial information loss.
(a)(b)


## 3.4. Algorithm and Optimization

The LC-ST-FCN model can be trained by minimizing the mean squared error between the estimated demandY^t and the real demand Yt. The objective function is shown in Equation (4), where w and b are both learnable parameters.(4)minW,bY−Y^22.Algorithm1 outlines the LC-ST-FCN training process. We first construct the training instance from the historical observations (lines 1–8), where D is the set of training instances. Then, the LC-ST-FCN model is trained via backpropagation and the adaptive subgradient method (AdaGrad) (lines 9–28). Under the parameter-sharing scheme of 2D and 3D convolutional layers, the same wm and bm are used when we slide each filter m across the width, height, and depth of each input instance in one minibatch (line 16). For each filter m, in locally connected convolutional layers, different wm′ and bm′ will be used in different locations when we slide each filter across each input instance in one minibatch (line 22).Algorithm 1: LC-ST-FCN training algorithm.
Input: Historical demand of each region: X0,X2,…,Xt−1; lengths of input data sequence: 2τ; length of the period interval: l.Output: Learned LC-ST-FCN model.(1)
//construct training instance:(2)
D←∅(3)
for all the available time interval t1≤t≤ndo(4)
Xtrec=Xt,Xt−1,…,Xt−τ+1,Xtrec∈ℝI×J×τ(5)
Xtper=Xt−l+1,Xt−l,…,Xt−l−τ,Xtper∈ℝI×J×τ(6)
//Xtin is the target at time t(7)
put a training instanceXtin=Xtrec;Xtper into D(8)
end for(9)
//Training:(10)
repeat(11)
Initialize the biases and weights at each layer;(12)
Sample minibatch fromD randomly;(13)
if in 2D or 3D convolution layers then(14)
for filters m∈Mdo(15)
//Parameter sharing(16)
optimize learnable parameterswm and bm(17)
end for(18)
end if(19)
if in locally connected convolution layers then(20)
for filters m∈Mdo(21)
//Without parameter sharing(22)
optimize learnable parameterswm′ and bm′(23)
end for(24)
end if(25)
Calculate the stochastic gradient by minimizing the objective function (4);(26)
Update the parameters via backpropagation;(27)
untilstopping criteria are met


## 4. Experiments

We perform experiments using a dataset from DiDi Chuxing (https://gaia.didichuxing.com), which is the largest ride-hailing service platform in China. The dataset includes customer requests in Chengdu, China, containing the request time, longitude, and latitude. After the raw data are cleaned, the dataset containing 7,031,022 requests within 103.85–104.30 longitude to the east and 30.48–30.87 latitude to the north is used in our experiments. All the requests are partitioned into 10-minute time intervals, and the investigated area is partitioned into 16×16 grids. Considering the periodicity of the data, we select the first three weeks of the dataset as a training set and the remaining nine days as an independent testing set. In the experiment, we set τ=10. Correspondingly, we use four 3D convolutional layers with a filter depth of F=3,5,7,8 to extract the spatial-temporal information from inputs. The number of the 2D and locally connected convolutional layers is 4 and 2, respectively.To evaluate the performance of the LC-ST-FCN model, we select a bunch of different models as benchmarks, where all the models are trained and validated with the same training set and test set. The baseline models are as follows:(1)
Additive Model. The time-series additive model is often used to decompose a time series into deterministic and nondeterministic components [35].(2)
Random Forest. The random forest is a powerful boosting tree-based method and is widely used in data mining applications. We train a random forest regressor with 200 trees (each limited to maximum of 20 nodes).(3)
ANN (Artificial Neural Network). We compare our method with a neural network of four fully connected layers. The numbers of hidden units are 128, 128, 64, and 64, respectively.(4)
CNN. The structure of the CNN is comprised of eight 2D convolutional layers and two fully connected layers with 256 outputs. The numbers of filters for convolutional layers from 1 to 8 is 32, 16, 16, 32, 32, 64, 64, and 32, respectively.(5)
LSTM. The LSTM is commonly adopted in sequence modeling due to its ability to capture both short- and long-term temporal dependencies. The LSTM model contains four LSTM layers with 128, 128, 64, and 64 hidden units each.(6)
Conv-LSTM. The Conv-LSTM structure can simultaneously learn spatial correlations and temporal dependencies [6]. The structure of the Conv-LSTM is comprised of four Conv-LSTM layers and two 2D convolutional layers.(7)
FCL-Net. FCL-Net [8] is one of the most powerful short-term passenger demand forecasting models. Conv-LSTM layers and convolutional operators are employed to capture the characteristics of spatiotemporal information. Each input consists of two terms, historical demand intensity and travel time rate, and is fed into two separate Conv-LSTM architectures.(8)
ST-ResNet. ST-ResNet [7] is a deep learning-based approach for traffic prediction. The CNN is used to extract features from historical images.(9)
LC-ST-FCN. The structure of LC-ST-FCN is comprised of four 3D convolutional layers, four 2D convolutional layers, and two locally connected convolutional layers.(10)
LC-LSTM-FCN. The combination of the convolutional layers and LSTM layers has been proved to be an effective structure in many tasks. The structure of LC-LSTM-FCN is comprised of four LSTM layers, four 2D convolutional layers, and two locally connected convolutional layers.(11)
LC-FCN. The structure of LC-FCN is comprised of eight 2D convolutional layers and two locally connected convolutional layers, where 2D convolutional layers are applied to the input to generate feature maps instead of 3D convolutional layers. Thus, the temporal dimension of input collapses after the first 2D convolutional layer.(12)
FCN. The structure of the FCN is comprised of ten 2D convolutional layers, where the 2D convolutional layer is used to obtain the final predictions.We compare the proposed model against all the baseline models in terms of three widely adopted metrics, including the root mean squared error (RMSE), mean absolute percentage error (MAPE), and modified MAPE (SMAPE), which are defined as follows:(5)RMSE=1n∑i,j∈I,Jyi,j−y^i,j2,MAPE=1n∑i,j∈I,Jyi,j−y^i,jyi,j+ε,SMAPE=2n∑i,j∈I,Jyi,j−y^i,jyi,j+y^i,j,where yi,j and y^i,j are the ground truth and prediction value of demands in the region i,j at the time interval t, while ε=1 is used to avoid the division by zero (otherwise, the MAPE equals to infinite when yi,j=0).We train deep learning models, except ST-ResNet and FCL-Net, by implementing standard backpropagation on feedforward nets using the adaptive subgradient method (AdaGrad), and the initial learning rate is 0.01. The early stop is used, and the maximum number of training epochs is set to 100, while the batch size is set to 10. The training parameters of ST-ResNet and FCL-Net are consistent with [7, 8].The best-performing model on the validation set was chosen for comparison and evaluation. In Table1, the LC-ST-FCN model obtains the best results (i.e., the bolded values) on RMSE, MAPE, and SMAPE. Compared with ST-ResNet and FCL-Net, the LC-ST-FCN utilizes 3D convolutional layers to fuse the temporal components (e.g., short-term and periodic). In addition, the LC-ST-FCN considers the impact of local statistical differences which have been ignored in ST-ResNet and FCL-Net. To evaluate the effectiveness of the architecture of the proposed model, we compare it with other variants from three aspects. First, the LC-ST-FCN has better predictive performance than LC-FCN and LC-LSTM-FCN, which indicates that 3D convolutional operations capture the spatiotemporal information better than 2D convolutional operations and LSTM. Second, compared with the FCN, the predictive performance of LC-FCN is improved by simply replacing the last two 2 fully connected layers with locally connected convolutional layers. This implies that the local statistical differences among regions should be properly concerned. Moreover, both the LC-FCN and CNN are able to capture the local statistical differences among regions (the last two layers of them do not share parameters), but the LC-FCN performs better than the CNN. The reason may be that the fully convolutional architecture enables the model to focus on the neighboring regions of each target region by maintaining the spatial coordinates between the input and output.Table 1
Comparison of the predictive performance.
ModelRMSEMAPE (%)SMAPE (%)Additive model1.8624.7829.92Random forest1.9625.5232.34ANN2.0424.4329.06CNN1.8225.7832.22LSTM1.8226.9535.76Conv-LSTM1.8326.0632.92FCL-Net1.7725.0532.48ST-ResNet1.7524.8131.30FCN1.7827.3633.85LC-FCN1.6922.9827.70LC-LSTM-FCN1.7823.0028.36LC-ST-FCN1.6722.4027.69The bold values represent the best results given by the metrics (i.e., RMSE, MAPE, and SMAPE).The validation error of the LC-ST-FCN and other deep learning models are recorded after each training epoch. In Figure5(a), it can be observed that the training curve of the LC-ST-FCN drops quickly in the several initial epochs and then converges to a stable range, which indicates that the LC-ST-FCN model has a lower risk of overfitting. The training curves of the models with the convolutional LSTM architecture fluctuate markedly. On the other hand, the value of τ is very critical. We perform the sensitivity analysis on these two parameters. We train the LC-ST-FCN model under different values of τ with 100 training epochs. As shown in Figure 5(b), the RMSE of the LC-ST-FCN gradually decreases when the value of τ increases, but there is no significant improvement when the value exceeds 10. Besides, the training time linearly increases with an increase in the value of τ. Considering both the RMSE and computation time, we choose the value to be 10 in our experiments.Figure 5
Experimental analysis. (a) Training convergence curves. (b) Sensitivity analysis ofτ.
(a)(b)


## 5. Analysis and Visualization

### 5.1. Spatial Correlations among Regions

The LC-ST-FCN model is developed to address the unique challenges of the passenger demand forecasting problem. Our model adopts the fully convolutional structure based on the assumption that learning the correlations among regions is of great importance. To validate the assumption, we calculate the pairwise Pearson correlation between regions by the following equation:(6)CorrYi,j,Yi′,j′=EYi,j−EYi,jTYi′,j′−EYi′,j′EYi,j−EYi,j2EYi′,j′−EYi′,j′2,where Yi,j and Yi′,j′ are the demand time-series data of two regions. Figure 6(a) shows the Pearson correlations among all 256 regions, where the coordinates of each pixel point are obtained by computing i∗16+j (flatten the original grid regions) and the value of each pixel point represents the pairwise Pearson correlation coefficient. In Figure 6(a), we can see that the spatial correlation between each region and its surroundings does exist, but it has varying degrees. To further understand the relationship between the spatial correlation and the spatial distance, we calculate the grid distance (i−i′2+j−j′2) between regions shown in Figure 6(b). As shown in Figure 6(b), by and large, with an increase in the grid distance, the degree of correlation declines regularly. According to the correlation analysis results, we can conclude that there exist spatial correlations among regions, especially among neighboring regions. However, the Pearson correlation coefficient can only be used to measure the linear relationship between two variables, and the correlation among regions may be nonlinear. To better explain the spatial correlation among regions and reveal the working mechanism of the proposed LC-ST-FCN model, we perform a direct analysis of the spatial-temporal features and the activation maps produced by the LC-ST-FCN in the next section.Figure 6
The analysis of the spatial correlations among regions. (a) The pairwise Pearson correlation coefficient between regions. (b) The relationship between the grid distance and the Pearson correlation coefficient.
(a)(b)


### 5.2. Evaluation of Model Structures

The experimental results show that the LC-ST-FCN model can realize accurate regional demand forecasting based on historical observations, which illustrates the effectiveness of the LC-ST-FCN. Besides the predictive performance, it is equally important to explore the reasons why the proposed model is more effective than other methods, especially other CNN-based models.The motivations for building the model structure of the LC-ST-FCN are as follows.First, for short-term passenger demand forecasting, detailed predictions in all regions of a city depend on both global and local features, which need to be encoded using spatial coordinates. Therefore, we adopt the fully convolutional architecture to maintain spatial coordinates.Second, before 2D convolutional layers, using the 3D convolutional operations to fuse the spatial and temporal information in multiple time intervals can enable the network to learn more powerful features.Thirdly, since the local statistics of regional demand data are not invariant, the spatial stationarity assumption of convolution cannot hold. Therefore, in the LC-ST-FCN model, the locally connected layers are utilized to obtain the final predictions.To illustrate the effectiveness of the LC-ST-FCN model, it is necessary to investigate how the model gets the predictions from the input data. Meanwhile, whether each component of the model is working as we expected. To realize these purposes, we first sample one input data from our testing data set. As shown in Figure7, the sampled input data include two parts, the recent term and the period term.Figure 7
The visualization of the sampled input data.Then, we feed the sample data into the LC-ST-FCN and other models, including the LC-FCN, FCN, and CNN, in its control group. The comparison of their model structures is shown in Table2. According to the type of layers, we divide each model into three blocks. To find out how different models transform the input data to obtain their prediction results, we visualize the activation values of their hidden layers.Table 2
Comparison of model structures.
ModelBlock 1 (1–4 layers)Block 2 (5–8 layers)Block 3 (9-10 layers)FCN architectureLC-ST-FCN3D convolution2D convolutionLocally connectedYesLC-FCN2D convolution2D convolutionLocally connectedYesFCN2D convolution2D convolution2D convolutionYesCNN2D convolution2D convolutionFully connectedNoGiven the sample data, region 8-8 (grid number) is maximally activated by the output layer (Block 3) of all these four models. Taking region 8-8 as a target, we can correspondingly find out which parts of the input data are utilized by each model to obtain the final prediction of the target region.The results reported in Figure8 correspond to the hidden layers of each model listed in Table 2. For visualization and comparison purposes, we normalize the activation value between 0 and 1. Each heatmap image is generated by the activation maps of the corresponding hidden layer, and the brighter colors represent higher activation values. Besides each heatmap image, the scatter diagram is used to compare the spatial correlations detected by the Pearson correlation analysis and each network structure. In each scatter diagram, the horizontal axis represents the pairwise Pearson correlation between the target region and the others (including itself), and the vertical axis represents the activation value of each region.Figure 8
The activation maps of hidden layers.From the results of Figure8, we can observe as follows:(1)
From the heatmap images of Block 2 (the third column in Figure8), we can find that the regions localized by the CNN model are different from those by the three other models. The CNN model activated more regions than the others (see the highlighted areas). According to the corresponding scatter diagram, many regions that have a weak linear correlation with the target region are also activated by the CNN model. Different from the CNN model, the regions activated by FCN-based models are significantly related to the Pearson correlation analysis results. Intuitively, the predictive performance will suffer if the model pays too much attention to weakly related regions, and the experimental results also confirm this view (shown in Table 1). Moreover, the relationships between the two variables shown in the scatter diagrams are not linear, indicating that the neural networks can capture not only the linear but also the nonlinear relationships among regions.In Table2, except for the CNN model, the other three models are all fully convolutional. Therefore, we can conclude that it is necessary to maintain spatial coordinates when we aim for detailed predictions in all regions of a city, and the fully convolutional architecture is very effective.(2)
In the first column of Figure8, we can observe that the regions concerned by each model are almost the same. However, after 4 layers, the output of Block 1 (the second column of Figure 8) shows a great difference, where the output of the LC-ST-FCN model in this block is obtained by the 3D convolutional layer and the others are obtained by the 2D convolutional layer. From the comparison of the diagrams in the second column, it is interesting to observe that the two variables described in the diagram of the LC-ST-FCN model are still highly related to each other, but in the other three diagrams, the relationship between these two variables becomes hard to detect. The very reason for the situation mentioned above lies in that the 3D convolution layers and 2D convolutional layers used in this paper adopt different strategies to extract spatial-temporal features. The spatial correlation between regions is defined by the relationship between their time-series data, but the temporal information is collapsed after 2D convolutional layers. Consequently, the relationship between regions is also lost. Moreover, we also try another variant of the LC-ST-FCN by changing the order of 3D layers and 2D layers. As a result, the RMSE increases from 1.66 to 1.69. Thus, we can infer that the 3D convolutional layers have the ability to preserve temporal information better.(3)
To deal with the local statistical differences among regions, besides the features that are important for all the regions, the model should have the ability to capture features that are only useful for limited regions. More specifically, the objective function of the model needs to pay attention to region-wise errors. By putting the locally connected convolutional layers at the end of the model, the prediction value of each region is obtained by a different set of parameters, and the prediction error of each region can be delivered to the previous layers independently during the backpropagation training process. Thus, the use of locally connected convolutional layers can enable the LC-ST-FCN model to capture the statistics of different regions by enforcing the model to focus on the prediction errors of each region.In the third column of Figure8, we can see that the scatter diagrams of the first three models are very similar, which means that the useful regions concerned by these three models are almost the same. However, based on the activation values output by the last layer of Block 2 of these three models (without normalization), we can find that the models with locally connected convolutional layers can activate these useful regions more effectively (as shown in Figure 9). Therefore, we can conclude that locally connected layers play an important role in dealing with local statistical differences and activating useful regions.Figure 9
The activation values with and without locally connected convolutional layers.


## 5.1. Spatial Correlations among Regions

The LC-ST-FCN model is developed to address the unique challenges of the passenger demand forecasting problem. Our model adopts the fully convolutional structure based on the assumption that learning the correlations among regions is of great importance. To validate the assumption, we calculate the pairwise Pearson correlation between regions by the following equation:(6)CorrYi,j,Yi′,j′=EYi,j−EYi,jTYi′,j′−EYi′,j′EYi,j−EYi,j2EYi′,j′−EYi′,j′2,where Yi,j and Yi′,j′ are the demand time-series data of two regions. Figure 6(a) shows the Pearson correlations among all 256 regions, where the coordinates of each pixel point are obtained by computing i∗16+j (flatten the original grid regions) and the value of each pixel point represents the pairwise Pearson correlation coefficient. In Figure 6(a), we can see that the spatial correlation between each region and its surroundings does exist, but it has varying degrees. To further understand the relationship between the spatial correlation and the spatial distance, we calculate the grid distance (i−i′2+j−j′2) between regions shown in Figure 6(b). As shown in Figure 6(b), by and large, with an increase in the grid distance, the degree of correlation declines regularly. According to the correlation analysis results, we can conclude that there exist spatial correlations among regions, especially among neighboring regions. However, the Pearson correlation coefficient can only be used to measure the linear relationship between two variables, and the correlation among regions may be nonlinear. To better explain the spatial correlation among regions and reveal the working mechanism of the proposed LC-ST-FCN model, we perform a direct analysis of the spatial-temporal features and the activation maps produced by the LC-ST-FCN in the next section.Figure 6
The analysis of the spatial correlations among regions. (a) The pairwise Pearson correlation coefficient between regions. (b) The relationship between the grid distance and the Pearson correlation coefficient.
(a)(b)


## 5.2. Evaluation of Model Structures

The experimental results show that the LC-ST-FCN model can realize accurate regional demand forecasting based on historical observations, which illustrates the effectiveness of the LC-ST-FCN. Besides the predictive performance, it is equally important to explore the reasons why the proposed model is more effective than other methods, especially other CNN-based models.The motivations for building the model structure of the LC-ST-FCN are as follows.First, for short-term passenger demand forecasting, detailed predictions in all regions of a city depend on both global and local features, which need to be encoded using spatial coordinates. Therefore, we adopt the fully convolutional architecture to maintain spatial coordinates.Second, before 2D convolutional layers, using the 3D convolutional operations to fuse the spatial and temporal information in multiple time intervals can enable the network to learn more powerful features.Thirdly, since the local statistics of regional demand data are not invariant, the spatial stationarity assumption of convolution cannot hold. Therefore, in the LC-ST-FCN model, the locally connected layers are utilized to obtain the final predictions.To illustrate the effectiveness of the LC-ST-FCN model, it is necessary to investigate how the model gets the predictions from the input data. Meanwhile, whether each component of the model is working as we expected. To realize these purposes, we first sample one input data from our testing data set. As shown in Figure7, the sampled input data include two parts, the recent term and the period term.Figure 7
The visualization of the sampled input data.Then, we feed the sample data into the LC-ST-FCN and other models, including the LC-FCN, FCN, and CNN, in its control group. The comparison of their model structures is shown in Table2. According to the type of layers, we divide each model into three blocks. To find out how different models transform the input data to obtain their prediction results, we visualize the activation values of their hidden layers.Table 2
Comparison of model structures.
ModelBlock 1 (1–4 layers)Block 2 (5–8 layers)Block 3 (9-10 layers)FCN architectureLC-ST-FCN3D convolution2D convolutionLocally connectedYesLC-FCN2D convolution2D convolutionLocally connectedYesFCN2D convolution2D convolution2D convolutionYesCNN2D convolution2D convolutionFully connectedNoGiven the sample data, region 8-8 (grid number) is maximally activated by the output layer (Block 3) of all these four models. Taking region 8-8 as a target, we can correspondingly find out which parts of the input data are utilized by each model to obtain the final prediction of the target region.The results reported in Figure8 correspond to the hidden layers of each model listed in Table 2. For visualization and comparison purposes, we normalize the activation value between 0 and 1. Each heatmap image is generated by the activation maps of the corresponding hidden layer, and the brighter colors represent higher activation values. Besides each heatmap image, the scatter diagram is used to compare the spatial correlations detected by the Pearson correlation analysis and each network structure. In each scatter diagram, the horizontal axis represents the pairwise Pearson correlation between the target region and the others (including itself), and the vertical axis represents the activation value of each region.Figure 8
The activation maps of hidden layers.From the results of Figure8, we can observe as follows:(1)
From the heatmap images of Block 2 (the third column in Figure8), we can find that the regions localized by the CNN model are different from those by the three other models. The CNN model activated more regions than the others (see the highlighted areas). According to the corresponding scatter diagram, many regions that have a weak linear correlation with the target region are also activated by the CNN model. Different from the CNN model, the regions activated by FCN-based models are significantly related to the Pearson correlation analysis results. Intuitively, the predictive performance will suffer if the model pays too much attention to weakly related regions, and the experimental results also confirm this view (shown in Table 1). Moreover, the relationships between the two variables shown in the scatter diagrams are not linear, indicating that the neural networks can capture not only the linear but also the nonlinear relationships among regions.In Table2, except for the CNN model, the other three models are all fully convolutional. Therefore, we can conclude that it is necessary to maintain spatial coordinates when we aim for detailed predictions in all regions of a city, and the fully convolutional architecture is very effective.(2)
In the first column of Figure8, we can observe that the regions concerned by each model are almost the same. However, after 4 layers, the output of Block 1 (the second column of Figure 8) shows a great difference, where the output of the LC-ST-FCN model in this block is obtained by the 3D convolutional layer and the others are obtained by the 2D convolutional layer. From the comparison of the diagrams in the second column, it is interesting to observe that the two variables described in the diagram of the LC-ST-FCN model are still highly related to each other, but in the other three diagrams, the relationship between these two variables becomes hard to detect. The very reason for the situation mentioned above lies in that the 3D convolution layers and 2D convolutional layers used in this paper adopt different strategies to extract spatial-temporal features. The spatial correlation between regions is defined by the relationship between their time-series data, but the temporal information is collapsed after 2D convolutional layers. Consequently, the relationship between regions is also lost. Moreover, we also try another variant of the LC-ST-FCN by changing the order of 3D layers and 2D layers. As a result, the RMSE increases from 1.66 to 1.69. Thus, we can infer that the 3D convolutional layers have the ability to preserve temporal information better.(3)
To deal with the local statistical differences among regions, besides the features that are important for all the regions, the model should have the ability to capture features that are only useful for limited regions. More specifically, the objective function of the model needs to pay attention to region-wise errors. By putting the locally connected convolutional layers at the end of the model, the prediction value of each region is obtained by a different set of parameters, and the prediction error of each region can be delivered to the previous layers independently during the backpropagation training process. Thus, the use of locally connected convolutional layers can enable the LC-ST-FCN model to capture the statistics of different regions by enforcing the model to focus on the prediction errors of each region.In the third column of Figure8, we can see that the scatter diagrams of the first three models are very similar, which means that the useful regions concerned by these three models are almost the same. However, based on the activation values output by the last layer of Block 2 of these three models (without normalization), we can find that the models with locally connected convolutional layers can activate these useful regions more effectively (as shown in Figure 9). Therefore, we can conclude that locally connected layers play an important role in dealing with local statistical differences and activating useful regions.Figure 9
The activation values with and without locally connected convolutional layers.


## 6. Conclusions

In this paper, we proposed a fusion convolutional architecture named LC-ST-FCN to model the distinct features of the ride-hailing demand forecasting problem, including spatial-temporal dependencies, spatial coordinates, and local statistical differences. Besides the higher prediction accuracy, we reveal the working mechanism of the proposed model on the taxi demand data. The advantages and disadvantages of various deep learning architectures in addressing the challenges of the demand forecasting problem, including the capturing of spatiotemporal dependencies, learning of local statistical differences, and ability to localize most related regions, are systematically evaluated. A real dataset from the DiDiChuxing platform is used for model evaluation and comparison. In the experiments, our model outperforms all the benchmark models in terms of three widely adopted metrics. In this paper, the training data of our proposed model are generated by the grid-based partition, which is carried out independently of the model training. In the future work, we expect to explore a learnable partition algorithm that has the ability of automatic region partition by learning the demand statistics of ride-hailing service platforms.



---

*Source: 4427638-2023-01-18.xml*