## 3 Materials and Methods
### Relgraph Attention Network  
Knowledge graph embedding (KGE) represents entities and relations in a continuous space called _embeddings_. A scoring function based on these embeddings can be defined to score triplets \((h,r,t),h,t\in\mathcal{E},r\in\mathcal{R}\). The embeddings are trained to ensure that observed facts in the KG have higher scores than unobserved ones. Well-trained embeddings are typically used for tasks like KGC and rule mining. According to the definition provided in Section 4, we have transformed the knowledge graph into \(|\mathcal{R}|\) entity graphs \(\{\mathcal{G}_{n}|r_{a}\in\mathcal{R}\}\), each containing \(|\mathcal{E}|\) vertices representing entity meta-nodes. Additionally, we have \(|\mathcal{E}|\) relation graphs \(\{\mathcal{RG}_{i}|e_{i}\in\mathcal{E}\}\), each containing \(|\mathcal{R}|\) vertices representing predicate meta-nodes. In the entity graph, the relations are defined by the predicates present in the KG, while in the relation graph, the relations are defined by entity-links. To perform tasks, such as KGC, Relgraph utilizes two interconnected graph attention networks to train embeddings for entities, predicates, and entity links. These networks are referred to as the _Entity GAT_ (graph attention network) and the _Relation GAT_ (relation graph attention network). The two GATs share the same embedding of predicates. **Entity GAT. ** The Entity GAT corresponding to entity graph \(\mathcal{G}_{k}\) is responsible for analyzing the message passing process of Relgraph and the attention between vertices \(emn_{ijk}\) and \(emn_{j,k}\) (entity meta-nodes representing the influence of entity \(e_{i},e_{j}\) on the predicate \(r_{k}\), respectively). We denote \(e_{ijk}\) as the component of \(emn_{i,k}\) corresponding to the vertex \(emn_{j,k}\). Matrix \(F\in\mathbb{R}^{|\mathcal{E}|\times d}\) represents the input features, where \(d\) is the dimension of the feature. Each row \(f_{i}=F_{i:}\) represents the embedding of entity \(e_{i}\). Similarly, matrix \(P\in\mathbb{R}^{|\mathcal{R}|\times d}\) represents features of the predicates, and each row \(p_{k}=P_{k:}\) represents the embedding of predicate \(r_{k}\). We denote \(f_{ijk}\) as the embedding of \(e_{ijk}\). In accordance with [16], the entity GAT uses the following aggregation function to update the embedding of entities and predicates:  
\[f_{ijk}=W_{f}[f_{i}|f_{j}|p_{k}], \tag{3}\]  
\[b_{ijk}=ReLu(W_{b}f_{ijk}), \tag{4}\]  
\[a_{ijk}=\frac{exp(b_{ijk})}{\sum_{n\in N_{i}}\sum_{r\in R_{in}}exp(b_{inr})}, \tag{5}\]\[\hat{f}_{i}=\sigma(\sum_{j\in N_{i}}\sum_{k\in R_{ij}}a_{ijk}f_{ijk}), \tag{6}\]  
\[F^{\prime}=W_{F}F+\hat{F}, \tag{7}\]  
\[P^{\prime}=W_{P}P, \tag{8}\]  
where \(p_{k}\) is the embedding of predicate \(r_{k}\) determined by \(\big{(}e_{j},r_{k},e_{i}\big{)}\in\mathcal{K}\), neighbor degree \(N_{i}=\big{\{}j|e_{j}\in\mathcal{E},r_{k}\in\mathcal{R}_{r}\big{(}e_{j},r_{k}, e_{i}\big{)}\in\mathcal{K}\big{\}}\) is the number set of \(e_{i}\)'s one-hop neighboring vertices, and \(N_{i}\) also includes \(i\) (i.e., there is a self-loop); \(R_{ij}=\big{\{}k|e_{i}\in\mathcal{E},e_{j}\in\mathcal{E},\big{(}e_{j},r_{k}, e_{i}\big{)}\in\mathcal{K}\big{\}}\) is the number set of predicates that have appeared in all facts with \(e_{i}\) as the head entity and \(e_{j}\) as the tail entity; \(a_{ijk}\) is the attention weight between the target vertex \(emn_{i,k}\) and the neighboring vertex \(emn_{j,k}\), which is generated by applying softmax to the values computed by \(b_{ijk}\). The parameters \(W_{f}\), \(W_{b}\), \(W_{F}\), and \(W_{P}\) are trainable parameters of the attention function. **Relation GAT. ** Dually, for each relation graph \(\mathcal{RG}_{s}\) in the Relgraph, to comprehend the message-passing process and the attention between vertices \(pmm_{us}\) and \(pmm_{vs}\) (predicate meta-nodes that symbolize the impact of predicates \(r_{u},r_{v}\) on entity \(e_{s}\), respectively), we utilize \(r_{us}\) as the component of \(pmn_{us}\) at vertex \(pmm_{vs}\).