## 4 Material and Method  
A transformer is an architecture transforming one sequence into another. All of these processes are done with the help of the encoder and decoder. However, as compared to the previously existing sequence-to-sequence models, the transformer is different from RNN, and it belongs to the non-RNN family. The attention mechanisms are the entire backbone of the transformer, which makes it possible for the target access any part of the sequences regardless of its distance [45; 46]. In essence, the structure of the transformer is organized into encoder and decoder modules, as shown in Figure 2. Similar encoder modules are stacked at the bottom of the stacked decoder modules. The construction of each encoder module is composed of a position-wise feed-forward layer and self-attention layer, while the decoder module has one additional layer as compared to the encoder part (the encoder-decoder attention layer). The encoder and decoder parts are bridged by a layer inserted between the self-attention and feed-forward layers. During transformation, we embed trajectory sequences of each driver similar to sequences of words/tokens. Inside the transformer, the multi-head attention layers in a sequence from multiple aspects (heads) attach different importance to the words/token. The output belongs to different heads to aggregate all the information that is concatenated and passed through a linear transformation. The encoder-decoder attention layers and self-attention layers are the same as the attention mechanism behind the scenes [47]. As an example, we take self-attention. On the sequences of token as trajectories, it operates as \(T=(P_{1},P_{2},\dots,P_{n})\), each of which is prepared by a random vector and passed through a linear transformation after being updated by using a weighted sum of any other words. The weights are assigned by their similarities, which is also called the attention score. We take the update of \(T_{i}\) as an example. \[y_{i}=\Sigma_{j=1}^{n}a_{ij}(T_{j}W_{V}) \tag{1}\]  
Figure 1: (**Left**) The whole city graph divided into the CRS i.e., latitude and longitude; and (**Right**) the LBSN-based driver trajectory as customer pick-up and drop-off. Shaded lines show the driver trajectory, while blue, red, and green nodes show the pick-up/drop-off points. In the above equation, \(y_{i}\) is the updated of \(T_{i}\) and \(a_{ij}\) is the attention score. We can measure the similarity between \(T_{i}\) and \(T_{j}\) as:  
\[a_{ij}=\frac{exp(C_{ij})}{\Sigma_{K=1}^{n}exp(C_{ik})} \tag{2}\]  
where we measure the compatibility \(C_{ij}\) of two linearly transformed \(T_{i}\) and \(T_{j}\). We calculate the scaled dot product as:  
\[C_{ij}=\frac{(T_{i}W_{Q})(T_{j}W_{K})^{T}}{\sqrt{h}} \tag{3}\]  
To strengthen the expressiveness of the transformer, three linear transformation matrices are represented as \(W_{V}\), \(W_{K}\), and \(W_{Q}\), with \(h\) the dimension of the output. The three linear transformation matrices are equal to W.  
There are five components of our deep wide model DWSTTNs, where each component works as follows. 1. We designed novel spatial and temporal transformers. In our approach, both transformers are based on a multi-modal embedding module to embed the spatial-temporal features jointly. As the encoder part, spatial and temporal transformer embedding layers convert the input token to the dimensional vectors. The input element, through the spatial and temporal transformer, make the information ordered through the positional encoding part. The purpose of positional embedding is to preserve the sequential order in a sequence. We can define mathematically positional encoding as follows. \[pos_{embedding(pos,2)}=sin(\frac{pos}{10000^{\frac{2}{h}}})\] (4) \[pos_{embedding(pos,2+1)}=cos(\frac{pos}{10000^{\frac{2}{h}}})\] (5)  
Figure 2: Main framework: (**Left**) the encoder part; and (**Right**) the decoder part. In the above equation, \(i\) is the dimension of the position embedding and \(h\) is the size of the hidden layers.