Towards Enhancing Relational Rules for Knowledge Graph Link Prediction (2023)

Abstract: Graph neural networks (GNNs) have shown promising performance for knowledge graph reasoning. A recent variant of GNN called progressive relational graph neural network (PRGNN), utilizes relational rules to infer missing knowledge in relational digraphs and achieves notable results. However, during reasoning with PRGNN, two important properties are often overlooked: (1) the sequentiality of relation composition, where the order of combining different relations affects the semantics of the relational rules, and (2) the lagged entity information propagation, where the transmission speed of required information lags behind the appearance speed of new entities. Ignoring these properties leads to incorrect relational rule learning and decreased reasoning accuracy. To address these issues, we propose a novel knowledge graph reasoning approach, the Relational rUle eNhanced Graph Neural Network (RUN-GNN). Specifically, RUN-GNN employs a query related fusion gate unit to model the sequentiality of relation composition and utilizes a buffering update mechanism to alleviate the negative effect of lagged entity information propagation, resulting in higher-quality relational rule learning. Experimental results on multiple datasets demonstrate the superiority of RUN-GNN is superior on both transductive and inductive link prediction tasks.

## introduction knowledge graph (kg) such as freebase @xcite , nell @xcite and @xcite , is essentially a semantic network used to store and organize knowledge, which is extensively applied in a variety of scenarios, such as question answering @xcite , recommendation systems @xcite , semantic search (berant and liang, 2014), etc. each of these kgs, how-ever, faces the problem of incompleteness, making it difficult to provide effective knowledge services for downstream applications. as a result, kg reasoning, known as link prediction in kg, is proposed to automatically complete the missing knowledge and has attracted much attention from researchers. various methods for link prediction are explored to facilitate reasoning for missing knowledge. previous methods such as transe @xcite and @xcite perform reasoning by learning and utilizing distributed representations of entities and relations in triples. since these methods can only capture the features of a single triple, some methods such as minerva @xcite and m-walk @xcite , are proposed to mine the path semantic information composed of triples. to further learn the semantic association of graph structures, methods such as compgcn @xcite and ke-gcn @xcite use graph neural networks (gnns) to aggregate neighbor information. the recent progressive relational graph neural network (prgnn) such as nbfnet @xcite and red-gnn @xcite , is a kind of advanced kg link prediction method that learns relational rules to infer missing knowledge. relational rules refer to the horn clause that consists only of ordered relations, such as has_f ather_in_law(a, c) ← has_wif e(a, b) ∧ has_f ather @xcite . in prgnn-based methods, the representation of each entity encodes the information of the specific relational digraph (r-digraph) @xcite . yet, we observe that the sequentiality of relation composition and lagged entity information propagation would affect the quality of encoded relational rule during inference, and the aforementioned methods lack the ability to deal with these two properties. the sequentiality of relation composition means that the order of combining different relations affects the semantics of the relational rules. for example, both the relational rules has_f ather(a, c) ← has_sister(a, b) ∧ has_f ather(b, c) and has_aunt(a, c) ← has_f ather(a, b) ∧ has_sister(b, c) contain the two same relations has_sister and has_f ather. the combination of two different relations in a different order can lead to different conclusions. however, existing prgnn-based methods face limitations in encoding such sequential information due to the standard practice of using addition and multiplication operations to integrate representations of entities and relations. consequently, these methods are prone to misinterpreting the encoded relational rule information, which ultimately restricts their reasoning ability. the phenomenon of lagged entity information propagation refers to the situation where the transmission speed of required information lags behind the appearance speed of new entities. the prgnn updates entity representations by propagating information on a growing subgraph, with entities in the subgraph considered as candidate answer entities. consequently, as prgnn undergoes forward propagation, new candidate answer entities continually emerge. however, these new entities often possess limited relational path information, while a substantial amount of crucial information remains trapped within the old candidate answer entities without being promptly conveyed to the new entities. taking figure 1 as an example, suppose we want to infer who andrew's mother-in-law is. the correct answer entity mia is a new candidate answer entity in figure 1 (d ) that can only obtain information about the shortest relational path ---→ r 2 , r 1 from andrew to mia. the true key information ---→ r 1 , r 3 , however, remains in the old candidate answer entity ethan. therefore, it becomes challenging to make the correct judgment for the given question. this phenomenon makes the model more prone to learning erroneous relation rules during the training phase and producing incorrect conclusions during the inference phase. to address the properties mentioned above, we propose a novel kg link prediction method called relational rule enhanced graph neural network(run-gnn), equipped with two strategies for enhancing relational rules. the first enhancement strategy involves employing a query related fusion gate unit to update the representation of relational rules. this allows run-gnn to better encode various combination patterns among relations and identify the most valuable relational rules from a pool of candidate rules. the second proposed strategy is the utilization of a buffering update mechanism. this involves adding a buffer module to the prgnn encoder, allowing the transmission of lagged information to the entities that require it. by doing so, the candidate answer entities promptly receive the necessary information, thus reducing the risk of the model learning incorrect rules. both of the above strategies allow the method to learn and utilize enhanced relational rules effectively, thereby improving the performance of the method. briefly, the main contributions of this paper are as follows: ## experiments to demonstrate the effectiveness of run-gnn, many experiments under inductive and transductive tasks are conducted for kg link prediction on multiple benchmark datasets. ## conclusion this paper attributes prgnn's reasoning capability to learning relational rules and identifies two issues with existing methods. firstly, we identify the issue of ignoring the sequentiality of relation composition, which causes the model to become confused with different relational rules. to address this issue, we propose qrfgu. additionally, we identify the issue of lagged entity information propagation, which can lead to erroneous rule learning. a buffering update mechanism is introduced to mitigate this problem. we then combine these two relation rule enhancement strategies to propose run-gnn , which achieves state-of-the-art performance on knowledge graph link prediction tasks. in the future, we plan to investigate more efficient link prediction methods based on prgnn to improve their practicality.