# Exploring Prior Knowledge from Human Mobility Patterns for POI Recommendation
## 3 Methodology
### Human Mobility Pattern Discovery
#### 3.1.2 Human Mobility Pattern Discovery Architecture  
The human mobility patterns discovery framework uses the prediction of downstream tasks to pre-train the human-mobility-pattern-extraction module, so that the human-mobility-pattern-extraction module obtains the ability to represent the spatio-temporal pattern under the specific city data distribution. At the same time, the prediction results of the downstream tasks themselves can help to test the representation effect of human mobility patterns. The overall architecture is shown in Figure 1. The human mobility pattern-extraction framework has three main parts, the human-mobility-pattern-extraction module, the encoding and decoding module based on the multi-head attention mechanism, and the upsampling module adapted to the downstream task. The three modules form a unified end-to-end model, and the resulting intermediate hidden layer vectors contain the spatio-temporal contextual relations of the human mobility pattern. The human-mobility-pattern-extraction module is composed of a graph convolutional neural network with shared weights and a feedforward neural network [19]. Given a graph \(G(V,E)\), where \(V\) and \(E\) are the sets of points and edges of the graph, respectively, with the point set size \(n=|V|\) and the edge set size \(m=|E|\). The adjacency matrix of graph \(G\) is \(A\in\mathbb{R}^{n\times n}\). Assuming that there are connected edges between nodes \(i\) and \(j\), then \(A  
So far, we can derive the corresponding inverse transform \(\mathbf{x}=\mathbf{U}\mathbf{\hat{x}}\) of the Fourier transform \(\mathbf{\hat{x}}=\mathbf{U}^{T}\mathbf{x}\) of the graph signal. The convolution form of the graph signal \(x\) and the convolution kernel \(h\) on graph \(G\) can be obtained, as shown in Formula (2). \[\mathbf{x}*\mathbf{h}=\mathbf{U}\cdot diag(\mathbf{\hat{h}})\cdot\mathbf{U}^{T}\mathbf{x} \tag{2}\]  
We replace \(diag(\mathbf{\hat{h}})\) above with \(\sum_{k=0}^{K}\alpha_{k}\Lambda^{k}\), where \(\{\alpha_{k}\}_{k=0}^{K}\) is a learnable parameter. So far, we have obtained a graph convolutional neural network model that does not require feature decomposition and has a computational complexity of \(O(n)\):  
\[ \begin{split} y&=\sigma(\mathbf{U}\cdot(\sum_{k=0}^{K} \alpha_{k}\Lambda^{k})\cdot\mathbf{U}^{T}\mathbf{x})\\ &=\sigma(\sum_{k=0}^{K}\alpha_{k}\mathbf{L}^{k}\mathbf{x})\end{split}  \tag{3}\]  
In this module, the graph signal refers to the functional attribute distribution vector of the urban area, and the adjacency matrix is the spatio-temporal graph of the flow between urban areas. We set the matrix after rasterization of the city to have \(r\times r\) areas, the attribute vector of each area to have \(d_{fi}\) dimensions, and the area representation dimension after  
Figure 1: Human-mobility-pattern-discovery architecture. The human mobility pattern extraction module consists of a convolutional neural network with a shared weight graph and a feedforward neural network. The encoding and decoding module based on the multi-head attention mechanism refers to the encoding and decoding structure of Transformer [31]. The upsampling module uses a deconvolutional network. graph convolution to be \(d_{f_{o}}\). The graph neural network framework used in this module can be simplified as follows:  
\[\mathbf{Y}=\sigma(\mathbf{\hat{D}}^{-1}\mathbf{\hat{\lambda}}\mathbf{X}\mathbf{W}) \tag{4}\]  
where \(\mathbf{X}\in\mathbb{R}^{r^{2}\times d_{f_{i}}}\) is the city function attribute matrix, \(\mathbf{A}\in\mathbb{R}^{r^{2}\times r^{2}}\) is the traffic adjacency matrix, \(\mathbf{\hat{A}}=\mathbf{A}+\mathbf{I}\), \(\mathbf{\hat{D}}\) is the degree matrix corresponding to \(\mathbf{\hat{A}}\), and \(\mathbf{W}\in\mathbb{R}^{d_{f_{i}}\times d_{f_{o}}}\) is the network parameter to be learned.