# Heterogeneous Graph Structure Learning for Next Point-of-Interest Recommendation
## 4 Heterogeneous Graph Structure Learning Model
### Heterogeneous Graph Contrastive Learning
#### 4.2.2 Heterogeneous Graph Representation Learning  
In the previous section, we constructed different subgraphs based on various meta-paths. To better model the higher-order relations in the graph, this paper employs graph neural networks to encode these subgraphs. Before using neural networks for encoding, we first generated a representation vector for each user and POI based on their ID, denoted as \(\mathrm{e}_{u}\in\mathrm{R}^{d},\mathrm{e}_{p}\in\mathrm{R}^{d}\), where \(d\) represents the embedding dimension of the vector. Based on \(\mathrm{e}_{u}\) and \(\mathrm{e}_{p}\), the initial embedding matrices for all users and POIs are \(\mathrm{E}_{u}^{0}\in\mathrm{R}^{M\times d},\mathrm{E}_{p}^{0}\in\mathrm{R}^{N \times d}\), respectively, where \(M\) and \(N\) represent the number of users and POIs. To better distinguish different types of entities, this paper employs a self-gating mechanism [52] to further initialize the representation vectors of the input meta-path subgraphs, as follows:  
\[\mathrm{E}_{uu}^{0}=\mathrm{E}_{u}^{0}\bigodot\sigma\Big{(}\mathrm{E}_{u}^{0 }\mathrm{W}_{u}+\mathrm{b}_{u}\Big{)};\mathrm{E}_{pp}^{0}=\mathrm{E}_{p}^{0} \bigodot\sigma\Big{(}\mathrm{E}_{p}^{0}\mathrm{W}_{p}+\mathrm{b}_{p}\Big{)} \tag{1}\]where \(\mathrm{E}_{uu}^{0}\in\mathrm{R}^{M\times d}\) is the input of \(\mathrm{G}_{uu1}\) and \(\mathrm{G}_{uu1}\), \(\mathrm{E}_{pp}^{0}\in\mathrm{R}^{N\times d}\) is the input of \(\mathrm{G}_{pp1}\) and \(\mathrm{G}_{pp2}\), \(\sigma\) is the sigmoid nonlinear activation function, \(\mathrm{W}_{u}\), \(\mathrm{W}_{p}\), \(\mathrm{b}_{u}\), and \(\mathrm{b}_{p}\) are learnable parameters, and \(\odot\) denotes element-wise multiplication. We can obtain \(\mathrm{E}_{uu}^{0}\) and \(\mathrm{E}_{pp}^{0}\) by (1), which will be used as the input to the meta-path subgraph encoders, while the input of \(\mathrm{G}_{up}\) is the initial representation \(\mathrm{E}_{u}^{0}\) and \(\mathrm{E}_{p}^{0}\). After obtaining the initial representation vectors, a graph neural network is used to further extract higher-order features for meta-path subgraphs \(\mathrm{G}_{uu1}\), \(\mathrm{G}_{uu2}\), \(\mathrm{G}_{pp1}\), and \(\mathrm{G}_{pp2}\). We use different graph encoders for representation learning:  
\[ \begin{split} E_{u1}^{l}=\mathrm{GNN}\Big{(}A_{uu1},E_{u1}^{l-1} \Big{)},& E_{u2}^{l}=\mathrm{GNN}\Big{(}A_{uu2},E_{u2}^{l-1} \Big{)}\\ E_{p1}^{l}=\mathrm{GNN}\Big{(}A_{pp1},E_{p2}^{l-1}\Big{)},& E_{p2}^{l}=\mathrm{GNN}\Big{(}A_{pp1},E_{p2}^{l-1} \Big{)}\end{split}  \tag{2}\]  
Among them, GNN represents the encoder based on a graph neural network. In our work, GIN [53] is used as the encoder to process meta-path subgraph representation learning. \(\mathrm{E}_{u1}^{l}\), \(\mathrm{E}_{u2}^{l}\), \(\mathrm{E}_{p1}^{l}\), and \(\mathrm{E}_{p2}^{l}\) are the node representations after the aggregation of the graph neural network at the \(l\)-th layer, and the initialization representations at the \(0\)-th layer are \(\mathrm{E}_{u1}^{0}\), \(\mathrm{E}_{u2}^{0}\), \(\mathrm{E}_{p1}^{0}\), and \(\mathrm{E}_{p2}^{0}\) from (1). After \(l\) iterations, we take the output of the last layer as the final result, where \(l\) is the number of layers of the graph encoders. The final output \(\mathrm{E}_{u1}\), \(\mathrm{E}_{u2}\), \(\mathrm{E}_{p1}\), and \(\mathrm{E}_{p2}\) are used as the input for subsequent contrastive learning and graph structure learning. For \(G_{up}\), we use LightGCN [54] to obtain the representation vectors:  
\[\mathrm{E}_{u0},\mathrm{E}_{p0}=\mathrm{LightGCN}\Big{(}A_{up},\mathrm{E}_{u }^{0}\Big{|}\Big{|}\mathrm{E}_{p}^{0}\Big{)} \tag{3}\]  
Specifically, we have the following:  
\[ \begin{split} e_{u}^{l}=\sum_{p\in N_{u}}\frac{1}{\sqrt{\left|N_{ u}\right|\left|N_{p}\right|}}e_{p}^{l-1},& e_{p}^{l}=\sum_{u\in N_{p}}\frac{1}{\sqrt{\left|N_{p}\right| \left|N_{u}\right|}}e_{u}^{l-1}\end{split}  \tag{4}\]  
where \(N_{u}\) and \(N_{p}\) denote the first-order neighbors of user \(u\) and POI \(p\).