# Commonsense-Guided Inductive Relation Prediction with Dual Attention Mechanism
## 3 Methodology
### Foundation Framework  
In this subsection, we introduce the general model of inductive relation predictions. **Neighboring Relational Feature Extraction. ** The neighboring relational feature model consists of two parts: subgraph extraction and node initialization. The local graph neighborhood of a specific triple in the KGs contained the logical evidence needed to infer the relationship between the target nodes; as such, the first step was to extract the subgraphs around the target nodes that contain complete neighbor relationships to initialize the node features. For the subgraph extraction, the target triple \((u,r_{t},v)\) was identified and the enclosing subgraph \(G(u,r_{t},v)\) around the target triple was extracted. More details can be found in the GraIL paper [15]. Then, regarding node initialization--considering that the inductive inference method cannot utilize the node attribute features--the node initial features were obtained by extracting the node's positional features and adjacency features. The details can be found in [21]. Figure 2: The overall framework of CNIA, which consists of the following parts: (1) extraction of the subgraphs with complete neighboring relations and initialization of the node features; (2) the feeding of subgraphs into the SNN with dual attention mechanisms to learn representations and extract neighboring relational path features; (3) commonsense-based re-ranking; (4) maximizing the MI between subgraph to graph to model the neighboring relations in a global way. **Subgraph Representaton Learning. ** As the main component of the foundation framework, this stage includes two parts: (1) obtaining the representation of subgraph entity nodes by the subgraph neural networks; (2) extracting and modeling the neighboring relational paths across target triples. For subgraph modeling, the subgraph \(G(u,r_{t},v)\) of the target triad \((u,r_{t},v)\) was first input into the GNN. SNRI defines the update strategy of the node features at each layer of the neural network. Inspired by CoMPILE [17], it feeds all node embeddings \(\mathbf{H}_{L}\) of the last layer to a gated recurrent unit to increase the expressive ability of the network. Finally, to obtain the representation of subgraph \(G\), it uses an average readout function as follows:  
\[\mathbf{h}_{G}=\frac{1}{|V_{G}|}\sum_{i\in V_{G}}\mathbf{h}_{i}^{L}. \tag{1}\]  
The neighboring relational path modeling aims to obtain valuable information from the relationship paths of \((u,v)\) as this helps to predict the type of relationship between \(u\) and \(v\). The specific procedure can be found in reference [21]. The subgraph representation \(\mathbf{h}_{G}\) and neighboring relational path representation \(\mathbf{p}_{G}\) were concatenated as the final representation of subgraph \(\mathbf{s}_{G}\) as follows:  
\[\mathbf{s}_{G}=[\mathbf{h}_{G}\oplus\mathbf{p}_{G}]. \tag{2}\]  
The score function of target triple is  
\[Score(u,r_{t},v)=\mathbf{W}_{s}\left[\mathbf{h}_{u}^{L}\oplus\mathbf{h}_{u}^{ L}\oplus\mathbf{e}_{r_{t}}^{L}\oplus\mathbf{s}_{G}\right], \tag{3}\]  
where \(\mathbf{h}_{u}^{L},\mathbf{h}_{u}^{L}\), and \(\mathbf{e}_{r_{t}}^{L}\) represent the embeddings of the target nodes \(u\), \(v\), and target relation \(r_{t}\) at the L layer of the SNN. **Supervised training. ** A loss function for supervised learning was constructed:  
\[L_{\text{sup}}=\sum_{(u,r_{t},v)\in G}\max\left(0,Score(u^{\prime},r^{\prime},v^{\prime})-Score(u,r_{t},v)+\theta\right), \tag{4}\]  
where \((u,r_{t},v)\) and \((u^{\prime},r^{\prime},v^{\prime})\) refer to the positive and negative samples, and \(\theta\) is the margin hyperparameter.