A Unified Joint Approach with Topological Context Learning and Rule Augmentation for Knowledge Graph Completion (2024)

Abstract: Knowledge graph completion (KGC) task is to infer the missing knowledge in the knowledge graph based on known factual triples. However, present KGC approaches still face the following two challenges. Those methods perform simple linear update on relation representation, and only local neighborhood information is aggregated, which makes it difficult to capture logic semantic between relations and global topological context information. To tackle the above challenges, we propose a unified joint approach with Topological Context learning and Rule Augmentation (TCRA) for KGC. The TCRA framework consists of an entity topological context learning mechanism based on dual-branch hierarchical graph attention network, and a relation rule context learning mechanism based on Rule-Transformer and rule-to-relation aggregator. The former mechanism encodes the topological structure features of entities, aggregates the local neighborhood topological context information of entities on the three levels (entity, relation and triple), and build clusters of global head or tail entities related to the same relation. It can capture the local and global topological context information of entities related to the same relation. The latter mechanism introduces chain-like Horn rules as the context information of relations, and encodes the logical semantic of relations to enrich the relation representation. Experimental performances on three benchmark datasets FB15k-237, WN18RR and Kinship indicate the effectiveness and superiority of our proposed approach. The codes are publicly available.1

heading: 1 Introduction
text: Currently, large-scale knowledge graphs (KGs) such as Freebase (Bollacker et al., 2008), DBpedia (Lehmann et al., 2015) and NELL (Mitchell
* Corresponding author. 1https://github.com/starlet122/TCRA.
et al., 2018) have been widely used in knowledgeintensive applications including semantic retrieval, question answering, and recommendation systems. Specially, knowledge graphs provide strong technical support for applications such as public opinion monitoring, intelligent decision-making, and credit assessment. However, knowledge graphs constructed in a manual or automated way are usually incomplete and sparse. Knowledge graph completion (KGC) task addressed in this paper is a vital technique to infer new knowledge and complete missing entities and relations within knowledge graphs.
At present, KGC approaches are mainly divided into methods based on neural reasoning, symbolic reasoning, and neural-symbolic reasoning (Zhang et al., 2021). The first family techniques (Sun et al., 2019; Yang et al., 2015; Dettmers et al., 2018; Schlichtkrull et al., 2018; Zhang et al., 2022; Li et al., 2022) first embed entities and relations into a low-dimensional dense vector space. And then they measure the plausibility of unobserved triples by calculating their scores in the continuous space. The methods in the second family (Galárraga et al., 2015; Yang et al., 2017; Qu et al., 2020; Cheng et al., 2022, 2023) deduce general logical rules from knowledge graphs, and apply the logical rules to infer missing facts, which can achieve explainable reasoning. The third family approaches (Cheng et al., 2021; Lin et al., 2021; Tang et al., 2023) fuse the former two kinds of methods, and have become a mainstream technology in the fields of natural language processing and knowledge graph construction.
However, existing KGC methods still face the following problems: (a) the existing KGC approaches based on graph neural network (GNN) only perform simple linear updates on the relation representation, which can not capture the correlation information between the relations in KGs. (b) Those methods only aggregate local neighborhood
1 13686
information. Even if message passing is applied multiple times, it is difficult for nodes that are far apart in KGs to effectively transfer information. Hence, they cannot capture global topological context information.
To solve the above problems, we propose a unified joint representation learning approach with Topological Context learning and Rule Augmentation (TCRA) for KGC task. The TCRA model adopts an encoder-decoder architecture. The encoder consists of two modules: the entity topology context learning module based on a dualbranch hierarchical graph attention network and the relation rule context learning module based on Rule-Transformer and rule-to-relation aggregator, which are used to generate embeddings of entities and relations, respectively.
Architecturally, the entity context learning module leverages hierarchical graph attention network (H-GAT) to aggregate local neighborhood structure information to generate local embeddings of entities. Parallelly, it builds global head entity or tail entity clusters related to the same relation to capture global semantic associations between entities. In addition, the relation context learning module leverages chain-like Horn rules as context information of relations, and designs the RuleTransformer to encode the logical semantic information of the relations in rule body relation sequences to generate the representation of rule bodies. Further, the rule-to-relation aggregator fuses the rule body representations related to the rule head relations, thereby updating the representations of the rule head relations.
The contributions of this paper are summarized as follows:
• A unified joint representation learning approach with topological context learning and rule augmentation is proposed for KGC task. On the one hand, our TCRA can effectively model the local topological structure information of entities, and capture the global structural characteristics of entities in the form of entity clusters for enriching the context representation of entities. On the other hand, logical rules are introduced to mine logical semantic association of relations, and those rules are explicitly modeled to constrain relation embeddings for enhancing the representation of relations.
• We design an entity topological context learn-
ing mechanism based on dual-branch hierarchical graph attention network. That mechanism can capture the local structural features of entities by modelling the neighborhood information at three levels of entities, relations, and triples. Moreover, the cluster of global head or tail entities related to the same relation is introduced to capture the global structural characteristics of entities by using cluster encoder and cluster-to-entity aggregator.
• A relation rule context learning mechanism based on Rule-Transformer and rule-torelation aggregator is developed. That mechanism can effectively capture the logical association between relations in the rule body, and simultaneously aggregate the rule body representation associated with a certain rule head relation for enriching the representation of the relations.
• Extensive comparative experiments on three benchmark datasets FB15k-237, WN18RR and Kinship show that our model TCRA outperforms the state-of-the-art methods.

heading: 2 Related Work
text: KGC approaches can be roughly classified into methods based on neural reasoning, symbolic reasoning, and neural-symbolic reasoning.
Neural reasoning is also known as knowledge graph embedding (KGE) or knowledge graph representation learning. Further, the present neural reasoning techniques can roughly fall into three categories: translational distance based models, semantic matching based models, deep learning based models. The basic idea of the translational distance models is to regard relations as conversion factors between head and tail entities (Bordes et al., 2013; Sun et al., 2019). The semantic matching models calculate the confidence of triples by measuring the similarity of the underlying semantics between entities and relations (Nickel et al., 2011; Yang et al., 2015). The deep learning models have stronger representation and generalization ability than the former two kinds of methods. Recently, GNN is utilized to fulfill the KGC task (Schlichtkrull et al., 2018; Vashishth et al., 2019; Li et al., 2022), which can learn the topological structure of KGs in an end-to-end manner. SEGNN (Li et al., 2022) introduced different levels of semantic evidence to explain the extrapolation
2
ability of KGE model and explicitly treated each semantic evidence as a different neighbor pattern.
The methods based on symbolic reasoning deduce general logical rules from the knowledge graph, and then apply the logical rules to infer the missing facts, which have good interpretability. Many rule mining approaches have been developed to extract rules from large-scale KGs, such as AMIE+ (Galárraga et al., 2015), RNNLogic (Qu et al., 2020), RLogic (Cheng et al., 2022) and NCRL (Cheng et al., 2023).
Recent techniques used in knowledge graph reasoning combine neural reasoning and symbolic reasoning. One of the fused methods is the symbol-driven neural reasoning method, which utilizes logic to enhance embedding. Specifically, the fusion mechanism can be classified into the following three types of ones. The first family of mechanisms including KALE (Guo et al., 2016) and RUGE (Guo et al., 2018) are to use logical rules as additional regularization for KGE training. The second kind of methods such as IterE (Zhang et al., 2019) and UniKER (Cheng et al., 2021) are to employ logical rules to obtain new hidden triples and generate additional triples for KGE training. The third kind of approaches such as RPJE (Niu et al., 2020), RulE (Tang et al., 2023) are to leverage logical rules as additional information to enhance KGs representation learning. This paper focuses on the symbol-driven neural reasoning method to solve KGC task.

heading: 3 Methodology

heading: 3.1 Problem Formulation
text: A knowledge graph G can be represented as a set of factual triples in the form of G = {(h, r, t)|h, r ∈ E, r ∈ R}, where E denotes a set of entities, R is a set of relations, h means a head entity, r is a relation, and t is a tail entity. KGC task means to predict the remaining missing element given two elements of a triple, including predicting the tail entity t in ⟨h, r, ?⟩ given h and r, or predicting the head entity h in ⟨?, r, t⟩ given r and t, where ? is the element to be inferred.

heading: 3.2 Overview of TCRA Framework
text: Our TCRA model for knowledge graph completion adopts the encoder-decoder architecture. The encoder consists of two modules: the entity topology context learning module based on a dualbranch hierarchical graph attention network and
the relation rule context learning module based on Rule-Transformer and rule-to-relation aggregator. The former module encodes the topological structure characteristics of entities. On the one hand, it aggregates three levels of local neighborhood topological context of entities including entity, relation and triple levels. On the other hand, global head or tail entity clusters related to the same relation are built to capture global semantic associations between entities. Thereby, we generate embedding representation of entities, which can capture the local structural features and the global topological context of entities. The latter module encodes the causal association information between relations, and introduces chain-like Horn rules as the context information of relations to learn the relation embedding, which aims to encode the logical semantic information of relation. ConvE is chosen as the decoder. The overall architecture of our TCRA model is shown in Figure 1.

heading: 3.3 Entity Topology Context Learning
text: A dual-branch hierarchical graph attention network is designed to learn entity topological context of entities. H-GAT is exploited to aggregate local neighborhood topological structure information to generate local embeddings of entities. Additionally, the global head entity clusters or tail entity clusters associated with the same relation is introduced to capture the global association information between entities.

heading: 3.3.1 Local Branch: Hierarchical Graph Attention Network
text: Inspired by the works in Li et al. (2022), the neighborhood topological context information on the three levels of entity, relation, and triple are jointly modeled as information sources of complementary reinforcing local neighbor structure to obtain richer local structure features of entities. H-GAT is leveraged to generate those three-levels local representations of the central entity. Further, the three embeddings are integrated to generate the updated embedding of that entity. In addition, a multi-layer iterative aggregation mechanism is introduced to capture multi-hop neighbor information and deep interactions between different levels of information. We will explain the single aggregation layer.
For the entity-level neighborhood information, we use senti for the entity-level local representation
3
of the central entity ei, which is computed as:
senti = σ
  ∑
(ej ,rj)∈Ni αentij Wentej
  , (1)
where Ni denotes the set of neighbor entities and connecting relation of ei in train set, Went ∈ RdG×dG is the linear transformation matrix, dG is the embedding dimension of entities and relations in G. ej is the embedding representation of ej . σ is a non-linear activation function. αentij is an aggregation attention score of neighbor entity ej , which is calculated as follows:
αentij = exp
( eTj ei )
∑ (ek,rk)∈Ni exp ( eTk ei ) . (2)
The relation-level neighborhood information sreli is computed as:
sreli = σ
  ∑
(ej ,rj)∈Ni αrelij Wrelrj
  , (3)
αrelij = exp
( rTj ei )
∑ (ek,rk)∈Ni exp ( rTk ei ) , (4)
where rj is the embedding representation of rj .
The triple-level neighborhood information strii is calculated as:
strii = σ
  ∑
(ej ,rj)∈Ni αtriij Wtriφ(ej , rj)
  , (5)
αtriij = exp
( φ(ej , rj) Tei )
∑ (ek,rk)∈Ni exp ( φ(ej , rj) Tei ) , (6)
where φ(ej , rj) = ej ∗ rj . A multi-layer iterative aggregation approach is adopted to integrate the neighborhood information on levels of entity, relation, and triple. At each layer l, the three-levels embeddings ( senti
)l,( sreli )l and ( strii )l with original input embedding eli are integrated to be feed into the next layer. In first layer, the embedding e1i is initialized. Then, after K layers aggregation, eKi is regarded as the final entity local topological context embedding elocali .
el+1i = e l i + ( senti )l + ( sreli )l + ( strii )l , (7)
elocali = e K i . (8)
4

heading: 3.3.2 Global Branch: Cluster Encoder and Cluster-to-Entity Aggregator
text: We observed that entities with the same relation exist global topological associations with each other in KGs. Thereby, the factual triples containing the same relation are constructed into a cluster of triples for capturing global topological context information between entities. Further, each relation has a corresponding head entity cluster Zhri and a tail entity cluster Ztri . Hence, the cluster triple can be represented as Cri = {(Zhri , ri, Ztri) | ri ∈ R}.
First, we collect the representations of entities contained in each entity cluster and generate the entity cluster representation, as shown in Eq.(9).
U = D−1H T EWc, (9)
where U ∈ R2|R|×dS is the entity cluster embedding matrix, dS is the embedding dimension of entity clusters. H ∈ R|E|×2|R| is the incidence matrix, which shows whether an entity belongs to a cluster. D ∈ R2|R|×2|R| is a diagonal matrix of cluster degree, i.e. the number of entities contained in clusters, and Djj = ∑ iHij . E ∈ R|E|×dG is the entity embedding matrix. Wc ∈ RdG×dS is a trainable weight matrix.
Then, we design a cluster-to-entity aggregator to aggregate the representation of the entity cluster to which the entity ei belongs, and generate the global context representation eglobali of the entity ei,
eglobali = σ ( 1∑ j hi,j hTi UWg ) , (10)
where hi ∈ R2|R| is an incidence vector, which indicates whether the entity ei belongs to cluster j by taking values 0 or 1. Wg ∈ RdS×dG is a trainable weight matrix. Finally, the entity embedding is calculated as eouti = e local i + e global i .

heading: 3.4 Relation Rule Context Learning
text: The relation rule context learning module is intended to encode the logical semantic features of relations. Horn rule is a special first-order logic rule in the language of symbolic logic, typically is expressed in the form of “body ⇒ head”. The body of a Horn rule is defined as a connection normal form (CNF), which connects a set of predicates through logical connectives. Here, the head is a single predicate. With regard to relation context learning, chain-like Horn rules are mined in the following form:
rh (x, y) ⇐ rb1 (x, z1)∧· · ·∧rbn (zn−1, y) . (11)
Here, rb1 (x, z1) ∧ · · · ∧ rbn (zn−1, y) is the rule body, rh (x, y) is the rule head. Integrating rule head and rule body, we denote a Horn rule as (Rb, rh), where the rule body Rb = [rb1 , . . . , rbn ]. The rule body of a chain-like Horn rule can be regarded as a relation sequence. In addition, the length of a Horn rule refers to the number of predicates (or atoms) in its body.
Chain-like Horn rules are introduced to capture logical semantic of relations. Thereby, a RuleTransformer is designed to mine the interactive information between relations in the rule body by encoding the rule body. It includes N Transformer encoder layers. We then aggregate the rule body embeddings to update the representation of the corresponding rule head relation. For example, the relation r3 in Figure 1 has three rule body relation sequences.
Nr3 = {R1, R2, R3} = {(r1, r2, r4), (r4, r5), (r7, r8, r9)}. (12)
Logical rules focus on capturing the semantic constraints among different types of relations, which have the rich semantic intension correlations. Inspired by this observation, the relation interaction layers are designed to learn the semantic associations of the relations in the rule body and update the representation of each relation in the rule body. At first, for the rule (R1, r3), i.e., r1 ∧ r2 ∧ r4 ⇒ r3, the relation interaction layers use the first former N -1 Transformer encoder layers and adopt multi-head self-attention mechanism. The input ER1 of the relation interaction layers is the representation matrix of the rule body sequence R1, and the output E′R1 is the encoding matrix of R1,
E′R1 = transformer encoders (ER1) , (13)
where ER1 ,E ′ R1 ∈ Rl×dG . Secondly, the rule encoding layer employes the N -th Transformer encoder layer and adopts a multi-head cross-attention mechanism to generate the representation of the rule body for r3,
eoutRi = transformer encoder ( E′Ri , r3 ) . (14)
Finally, we develop a rule-to-relation aggregator to aggregate the rule body representation related to the rule head relation ri, introduce the rule head association matrix, and update the representation of the rule head relation ri,
ri = σ ( 1∑ j hi,j hTi BWr ) , (15)
5
where H ∈ R|B|×2|R| is the incidence matrix, which indicates whether the rule body b corresponds to the rule head relation r. hb,r = 1 if b =⇒ r otherwise hb,r = 0. |B| represents the number of rules. hi ∈ R|B| is an incidence vector, which indicates whether the rule body bj corresponds to the rule head relation ri. B ∈ R|B|×dR is the rule body embedding matrix, which is output by the rule encoding layer. dR is the embedding dimension of rule bodies. Wr ∈ RdR×dG is a trainable weight matrix.

heading: 3.5 Decoder
text: We leverage the embeddings of entities and relations to perform KGC task. ConvE is chosen as the decoder, which uses a 2D convolutional neural network to match the query (h, r) and the answer t. The ConvE’s scoring function is shown in Eq.(16):
f(h, r, t) = σ(f(σ([h̃; r̃]) ∗ ψ))Wd)t, (16)
where h̃ and r̃ represent the 2D tensor corresponding h and r respectively, ∗ denotes the convolution operator. ψ represents a set of convolution kernels. f(·) is the vectorized function. Wd is a learnable weight matrix. σ is the ReLU activation function.
The standard binary cross-entropy loss function is chosen and label smoothing is adopted:
L = − 1 N
∑
t
(ti log(pi) + (1− ti) log(1− pi)),
(17) where N denotes the number of entities, ti is the true label of triple i, and pi is the corresponding prediction score.

heading: 4 Experiments

heading: 4.1 Experiment Settings
text: Datasets and Rules. In order to evaluate the performance of our proposed method on the KGC task, we have conducted comparative experiments on three public benchmark datasets, including FB15k-237 (Toutanova and Chen, 2015), WN18RR (Dettmers et al., 2018), and Kinship (Kok and Domingos, 2007). The statistics of datasets is given in Table 1. For rule mining, RNNLogic (Qu et al., 2020) is chosen as our rule mining tool. In addition, we have also conducted experiments to demonstrate the impact of the selection of decoders (ConvE and DistMult) and rule miners (RNNLogic and RLogic). Additional experimental details are listed in Appendix A.1.1.
Baselines. We compare our TCRA model with a comprehensive suite of baselines, including three classes of models: (a) KGE-based models: RotatE (Sun et al., 2019), DistMult (Yang et al., 2015), ConvE (Dettmers et al., 2018), RGCN (Schlichtkrull et al., 2018), SACN (Shang et al., 2019), CompGCN (Vashishth et al., 2019), LTE-KGE (Zhang et al., 2022), SE-GNN (Li et al., 2022), KGT5 (Saxena et al., 2022), (b) Rule learning-based models: AMIE+ (Galárraga et al., 2015), NeuralLP (Yang et al., 2017), RNNLogic (Qu et al., 2020), RLogic (Cheng et al., 2022), NCRL (Cheng et al., 2023), (c) joint models: RuleIC (Lin et al., 2021), RulE (Tang et al., 2023).
Evaluation metrics. KGC performance can be evaluated through five common evaluation metrics: MR (the Mean Rank), MRR (Mean Reciprocal Rank), Hits@N for N is 1,3 and 10 (the proportion of correct entity rankings in top-N).

heading: 4.2 Experimental Results

heading: 4.2.1 Performance Comparison
text: As shown in Table 2 and Table 3, TCRA achieves competitive performance on FB15k-237, WN18RR and Kinship. Especially on Kinship, we obtain the highest MRR, Hits@1, and Hits@3 that are 1.8%, 2.5%, and 1.7% higher than the best baseline, respectively.
Comparison with KGE-based and rule learning-based models. Our TCRA is essentially a rule-enhanced GNN-based KGE model. (a) TCRA was first compared with traditional KGE models, including the translational distance model RotatE, the semantic matching model DistMult, and the CNN-based model ConvE. Table 2 and Table 3 show that TCRA achieves the higher performance than those of RotatE, DistMult and ConvE, indicating the effectiveness of our GNN architecture. (b) We further compare TCRA with the GNN-based KGE models including R-GCN, SACN, CompGCN, LTE-KGE and SE-GNN. The performance of TCRA is superior to those of all
6
other models about MRR, Hits@1, Hits@3, and Hits@10 on FB15k-237 and WN18RR, while it outperforms all other models about MRR, Hits@1 and Hits@3 on Kinship. Those experimental results in Table 2 and Table 3 demonstrate the effectiveness of our proposed entity topology context learning and the relation context learning mechanism. The former mechanism solves the problem of effective message transmission between adjacent distant entities to supplement rich topological information. Within the latter mechanism, log-
ical rules are introduced as relation context to capture the logical semantic characteristics of relations. (c) The performance of TCRA is better than the rule learning-based model including AMIE+, NeuralLP, RNNLogic, RLogic and NCRL in most cases. This can be attributed to the fact that TCRA jointly learns logical rules and topological structures, which can encode the knowledge graph in a more complementary way and obtain better KGC performance.
Comparison with joint models. Compared with joint models including Rule-IC and RulE, TCRA achieves the convincing performance. Specifically, TCRA obtain the best performance on FB15k-237 and Kinship, and better performance than Rule-IC on WN18RR. The reason is that the logical rules and topology structures in WN18RR and Kinship contain more complementary information.

heading: 4.2.2 Ablation Study
text: We have conducted ablation experiments to demonstrate how Cluster Encoder and cluster-toentity Aggregator (abbreviated as CEA) and Relation Rule Context learning (abbreviated as RRC) contribute to the overall performance of TCRA. “w/o CEA” means that CEA is removed from TCRA, while “w/o RRC” denotes that RRC is removed from TCRA. “w/o CEA&RRC” represents that TCRA simultaneously removes CEA
7
and RRC. "CEA&Rule-Transformer" means that TCRA simultaneously eliminates CEA and RuleTransformer. Table 4 summarizes the results of our ablation experiments on three datasets.
For WN18RR and Kinship, these four ablation models have decreased significantly on MRR, Hits@1, and Hits@3 metrics. The ablation experiments not only indicate the effectiveness of the proposed global aggregation and relation rule context learning mechanism, but also show the two modules CEA and RRC complementarily contribute to our TCRA model. In particularly, the performance of "w/o CEA&RRC" is worse than that of "w/o CEA&Rule-Transformer", which supports the effectiveness and correctness of logical rules.

heading: 4.2.3 Hyperparameter Sensitivity Analysis
text: This subsection analyzes the sensitivity of hyperparameters used in TCRA, including number of rules and encoder layers in Rule-Transformer. The corresponding experimental results are illustrated in Figure 2. Additional hyperparameter experiments about learning rate and embedding size are provided in the appendix A.1.2.
First, we analyze the effects of different number of rules on the performance of our model. We randomly select a fixed number of rules from the mined logical rules for training. For FB15k237, the overall trend of performance is flat. For WN18RR, the performance of TCRA generally shows a decline first and then an increase. For Kinship, four curves show a trend of first falling and then rising.
Further, we investigated the sensitivity of the number of encoder layers used in the RuleTransformer, and its value range of N is in {2,3,4,5,6,7}. The performance of TCRA shows a trend of rising first and then falling on all three datasets. All in all, our model TCRA is not sensitive to number of rules and encoder layers in the Rule-Transformer.

heading: 5 Conclusion and Future Work
text: In this paper, a unied joint approach with Topological Context learning and Rule Augmentation (TCRA) has been proposed to perform knowledge graph completion, which can infer missing knowledge in a more complementary way. The designed entity topological context learning mechanism based on dual-branch hierarchical graph attention network can capture local structural features of entities at three levels, and incorporate global entity cluster to capture the global structural characteristics of entities. Meanwhile, the relation rule context learning mechanism based on Rule-Transformer and rule-to-relation aggregator is developed to facilitate adequate interaction between relations by leveraging chain-like Horn rules, which can capture the logical semantic of relations to enrich the relation representation. Ex-
8
tensive experiments demonstrate that the proposed TCRA model outperforms present methods. In the future, we will further design an iterative manner to implement rule-enhanced KGE methods on this basis, so as to achieve full complementarity between logical rules and KGE.
Limitations
Knowledge graphs contain rich interrelated knowledge, including structural association and semantic association. The models based on GNNs are easily affected by the sparsity of local structures. To alleviate the local sparsity, we build clusters of global head or tail entities related to the same relation to learn global structure association about entities. In addition, the utilization of logical rules increases the risk of being affected by noisy data in KGs. We make the attempt to randomly select a fixed number of rules to participate in training to reduce risks.
Ethics Statement
We used the publicly available datasets FB15k237, WN18RR, and Kinship to train and evaluate KGC models, and there is no ethical issue.

heading: Acknowledgements
text: The work is supported by the National Natural Science Foundation of China (62072039).

heading: A Appendix
text: A.1 Experiments A.1.1 Implementation Details. We train our model using the Adam optimizer and tune the model hyperparameters via grid search on the validation set. Specifically, the hyperparameter ranges are as follows: the learning rate is chosen between 0.0001 and 0.01, and the embedding size is chosen between 100 and 500. Pytorch is used to implement the TCRA model. We conduct all experiments on the Ubuntu system with 3090 GPU.
A.1.2 Hyperparameter Experiments This subsection analyzes the sensitivity of hyperparameters used in our model TCRA, including learning rate and embedding size. The extensive experiments of hyperparameters on FB15k-237, WN18RR, and Kinship have been coducted. The corresponding experimental results are illustrated in Figure 3.
We explore the effects of different learning rates on the performance of our TCRA model. The performance of TCRA shows a tiny fluctuation. It is seen that TCRA obtains the highest performance on FB15k-237, WN18RR, and Kinship when learning rate is 4e-4, 3e-3, and 5e-3, espectively.
In addition, the value of the embedding size is taken in {100,200,300,400,500}. It is seen that TCRA obtains the highest performance on FB15k237, WN18RR, and Kinship when embedding size is 400, 300, and 400, espectively. It shows a trend of rising along the increasing of embedding size
10
on all three datasets FB15k-237, WN18RR, and Kinship. All in all, our model TCRA is sensitive to embedding size and not sensitive to learning rate.
11

heading: Future Work
text: In the future, we will further design an iterative manner to implement rule-enhanced KGE methods on this basis, so as to achieve full complementarity between logical rules and KGE. Limitations
Knowledge graphs contain rich interrelated knowledge, including structural association and semantic association. The models based on GNNs are easily affected by the sparsity of local structures. To alleviate the local sparsity, we build clusters of global head or tail entities related to the same relation to learn global structure association about entities. In addition, the utilization of logical rules increases the risk of being affected by noisy data in KGs. We make the attempt to randomly select a fixed number of rules to participate in training to reduce risks