# Exploring Prior Knowledge from Human Mobility Patterns for POI Recommendation
## 3 Methodology
### Human Mobility Pattern Discovery
#### 3.1.2 Human Mobility Pattern Discovery Architecture
\(\mathbf{X}\mathbf{W}\) linearly changes the feature vector of the node, \(\mathbf{\hat{\lambda}}\mathbf{X}\mathbf{W}\) propagates the transformed node feature to neighbors, and \(\mathbf{\hat{D}}^{-1}\mathbf{\hat{\lambda}}\mathbf{X}\mathbf{W}\) normalizes the feature received by the node. In this paper, the graph convolutional neural network [32; 33] to which each time slice belongs shares the parameter weight, and only one graph neural network is trained and maintained. For the input backpropagation gradients of different time slices, we use the sum-average processing. The human mobility pattern extraction strategy learned by the graph neural network is applicable to all time slices [34]. Since the input data of the coding module are a sequence composed of the representation vectors of urban human mobility patterns under a single time slice instead of the whole graph representation, we use the feedforward neural network [35] module to realize the function of the ReadOut module in DGI [36] and obtain the hidden vector representation of the whole graph. In the encoder-decoder module based on the multi-head attention mechanism, we refer to the encoder-decoder structure of Transformer [37]. We use a sequence of representation vectors of human mobility patterns as input representations and target data representations of downstream tasks as output representations. The attention mechanism can be represented by the following formula:  
\[Attention(\mathbf{Q},\mathbf{K},\mathbf{V})=softmax(\frac{\mathbf{Q}\mathbf{K}^{T}}{\sqrt{d_{k}}}) \mathbf{V} \tag{5}\]  
We set the original attention module to be replicated \(h\) times, i.e., \(h\) head attention. We input the same data to the multi-head attention module to obtain \(h\) single-head attention results. We then spliced the multi-head results and transformed them through a linear layer to obtain the fused feature output [38]. The multi-head attention formula is as follows:  
\[MultiHead(\mathbf{Q},\mathbf{K},\mathbf{V})=Concat(head_{1},head_{2},\ldots,head_{h_{1}}) \mathbf{W}^{o} \tag{6}\]  
We used the human-mobility-pattern-extraction module to replace the original list of embedding in the transformer. The representation is directly extracted from the original data, and the output module is replaced by the downstream-task feature-extraction module. The upsampling module is used to restore the predicted representation vector to the same structure as the real data. In this paper, we use the deconvolution network as the upsampling module and generate a map of the urban traffic congestion situation from the predicted representation vector \(\mathbf{\hat{emb}}\in\mathbb{R}^{d}\), denoted as \(\mathbf{\hat{S}}\in\mathbb{R}^{r\times r}\). We calculate the average speed of \(r\times r\) nodes in the time period, and the average speed of each node in each time slice can be regarded as a map snapshot of urban traffic congestion. We denote the snapshot of the congested state of the city-market traffic corresponding to the time slice \(t\) as \(\mathbf{S}_{t}\in\mathbb{R}^{r\times r}\). We thereby formulate the task of urban traffic congestion state assessment. Given the spatio-temporal graph sequence of traffic flow as \(G_{seq}=\{G_{1},G_{2},\ldots,G_{t}\}\), the corresponding function distribution of each region as \(\mathbf{A}\in\mathbb{R}^{r^{2}\times d_{f_{i}}}\), the sequence of urban traffic congestion state as \(\mathbf{A}\in\mathbb{R}^{r^{2}\times d_{f_{i}}}\), and the prediction of urban traffic congestion state in time slice \(t\) as \(\mathbf{S}_{t}\), let the output result of the overall framework be \(\mathbf{\hat{S}}_{t}\), and expect \(\mathbf{\hat{S}}_{t}\) to be as close as possible to \(\mathbf{S}_{t}\). Then, the optimization objective of the overall framework is  
\[L=\|\mathbf{\hat{S}}_{t}-\mathbf{S}_{t}\| \tag{7}\]In order to adapt to the data characteristics, we used the convolutional neural network in the downstream task feature extraction module of this paper and the deconvolutional neural network in the upsampling module.