# Hierarchical Self-Supervised Learning for Knowledge-Aware Recommendation
## 4 Methodology
### Triple-Graph Masked Autoencoder
#### 4.1.1 Node Connection Reconstruction  
We design the attention-based node connection masking strategy hierarchically and divide the reconstruction into two tasks: edge reconstruction and degree restoration, aiming to unveil the underlying semantic information. Masking. We adopt a generative paradigm, wherein a proportion of edges with high weights in the KG are selectively masked. The main objective is to reconstruct these masked edges, allowing the model to learn crucial information. T-GMAE selects the top-\(n\) edges with the highest weights as the masked node connections. To this end, given a triplet \((i,r_{ij},j)\), we employ an attention mechanism [31] to calculate the weight of relation \(r_{ij}\) between the head entity \(i\) and the tail entity \(j\) within the KG, evaluating the significance of \(r_{ij}\) and its associated triplets. Hence, we formulate the attention-based weighting function as follows:  
\[\alpha_{i,j}=\sigma\left(\frac{\mathbf{e}_{i}\mathbf{W}^{\text{Q}}\Big{(} \mathbf{e}_{j}\mathbf{W}^{\text{K}}\odot\mathbf{r}_{ij}\Big{)}^{\top}}{\sqrt{ d_{k}}}\right)\beta_{i} \tag{1}\]  
where \(\mathbf{e}_{i}\), \(\mathbf{e}_{j}\), and \(\mathbf{r}_{ij}\) denote the embedding of entity \(i\) and entity \(j\), along with the relation between them, respectively. \(\mathbf{W}^{\text{Q}}\) and \(\mathbf{W}^{\text{K}}\) denote the weights transformation matrix. \(\sqrt{d_{k}}\) denotes the scale factor and \(\sigma\) denotes the softmax function. We denote the node connections that are masked and remained as \(\mathcal{R}_{mask}\) and \(\mathcal{R}_{remain}\), respectively, where \(\mathcal{R}_{mask}\cup\mathcal{R}_{remain}=\mathcal{R}\). By doing so, we obtain the masked knowledge graph \(\mathcal{G}_{m}\) as the input data:  
\[\mathcal{G}_{m}=(\mathcal{E},\mathcal{R}_{remain}),\mathcal{R}_{remain}= \mathcal{R}-\mathcal{R}_{mask} \tag{2}\]  
Edge reconstruction. We utilize a traditional encoder-decoder architecture to reasoning node connections that are as close as possible to the original structure and encode essential information from the knowledge graph to low-dimensional representations. To be more specific, we use the GNN encoder, such as GCN [53], GraphSAGE [54], HGCN [55], to encode \(\mathcal{G}_{m}\). During the neighbor aggregation process, we integrate the weights of the edges into the embedding learning of entities:  
\[\mathbf{e}_{i}^{(l)}=\frac{1}{|\mathcal{N}_{i}|}\sum_{(i,r_{ij},j)\in \mathcal{N}_{i}}\alpha_{ij}\mathbf{r}_{ij}\odot\mathbf{e}_{j}^{(l-1)} \tag{3}\]  
where \(\mathcal{N}_{i}\) denotes the subgraph centered around the node \(i\) and containing all its first-order neighbors. The node representation can be updated by incorporating the embeddings of both itself and its neighboring nodes, as depicted by  
\[\mathbf{e}_{j}^{(l+1)}=\text{Com}\Big{(}\mathbf{e}_{j}^{(l)},\text{Agg}\Big{(} \{\mathbf{e}_{i}^{(l)}\,|\,i\in\mathcal{N}_{i}\}\Big{)}\Big{)} \tag{4}\]  
where the function Agg is used to aggregate the embedding vectors of the neighboring nodes. The combination function Com refers to an operation that combines the aggregated neighbor information with its own from the previous layer. It is worth pointing out that the final representation is obtained by summing the aggregated embeddings from different layers. The decoder is responsible for mapping the node representations back to input embeddings for the reconstruction task. We aim to reconstruct both edges and degrees about the masked node connections and we utilize the GNN as the edge decoder. Note that the edge reconstruction loss function is devised by minimizing the dot product logarithmic loss of KG triplets:  
\[\mathcal{L}_{edge}=\sum_{(i,r,j)\in\mathcal{G}_{mask}}-\text{log}\big{(} \sigma(\mathbf{e}_{i}^{\top}\cdot(\mathbf{e}_{j}\odot\mathbf{r}_{ij}))\big{)} \tag{5}\]Degree restoration. To fully capture the KG's structure, we introduce a degree restoration task.