# Intercity Online Car-Hailing Travel Demand Prediction via a Spatiotemporal Transformer Method
###### Abstract
This enables LSTM models to be more efficient at capturing longer time dependencies and resolves the problem of vanishing and/or exploding gradient problems in the original RNN to a certain extent. This characteristic makes LSTM models a popular option in short-term traffic forecasting problems [10; 11; 12]. [PERSON] et al. [13] employed an LSTM network to predict traffic under extreme conditions, proving LSTM's capability in modeling sequential dependency. [PERSON] et al. [14] used LSTM that considered semantic similarity among different regions to make predictions. However, these sequential models have limited scalability for longer time dependencies where their memorization power declines [7]. Furthermore, sequential models are good at temporal dependencies but have no mechanism to deal with the spatial dependencies in traffic data. In practice, LSTM is often combined with models that capture spatial features, such as convolutional neural networks (CNNs), to form hybrid models. Such a hybrid model is then able to capture the temporal-spatial characteristics of traffic data. CNNs have shown outstanding performance in modeling local and shift-variant features [15]. [PERSON] et al. [16] further integrated an RNN and CNN, whereby the latter was in charge of spatial features and used LSTM to capture temporal features. The long- and short-term temporal network (LSTNet), proposed by [PERSON] et al. [17], is another example of LSTM and CNN integration that demonstrated significant performance improvements for long- and short-time series prediction tasks. However, since the number of hidden layers grows linearly with increasing sequences, the scalability of such models is limited for longer input sequences [18]. Deeper layers also affect the efficiency of CNNs for capturing dependencies in longer sequences [19]. After a review of related works, the utilization of spatiotemporal features is key to forecasting travel demand with high accuracy. However, all the above research mainly focused on intracity travel demand forecasting, and few studies have focused on the intercity travel demand prediction problem. In addition, compared to intracity travel demand prediction, two challenges need to be solved for intercity travel demand forecasting. First, it requires capturing a long time-horizon correlation. Second, the correlation among adjacent regions may not be obvious, and many regions have high correlations with some far away regions. To conduct accurate intercity travel demand prediction, we first utilized a transformer model to capture longer time-horizon correlations. A transformer is a popular model proposed by [PERSON] et al. [19]. It is an attention-based model that no longer requires recursive feeding of sequential data, as RNN-based models do. Therefore, a transformer can preserve sequence orders and is more computationally efficient than RNNs. Strategies such as multiheaded attention and positional encoding help a transformer attain significant success in machine translation tasks [20; 21]. Machine translation and traffic prediction are formulated similarly. Machine translation is a seq-2-seq learning task that aims to translate a source sentence into a target sentence [22], whereas traffic prediction tries to use historical data as an indicator of future traffic conditions. Hence, the time steps in historical traffic data can be similar to the position index of each word in the input sentences in a machine translation task [23]. For instance, [PERSON] et al. [23] utilized a transformer to capture temporal dependencies in traffic data. To tackle the second challenge, a spatial transformer was introduced that armed the transformer with spatial transformation capabilities [24]. If we employ a CNN model for spatial dependency extraction, the representation of correlations between two distanced regions usually requires multiple hidden layers. The spatial transformer can capture the correlation between two regions and is independent of the distance between regions.