SCE : Semantic Consistency Enhanced Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning (2025)

Abstract: Multi-hop reasoning with reinforcement learning has proven effective in discovering inference paths in incomplete knowledge graphs. However, a major challenge remains: spurious paths (incorrect reasoning paths that accidentally lead to correct answers) often arise due to reward mechanisms that prioritize final results over reasoning quality. While existing approaches attempt to mitigate this issue using external rules, they often neglect the internal semantic consistency between the target triple and the intermediate triples along the reasoning path. In this paper, we propose a novel framework, \textbf{S} emantic \textbf{C} onsistency \textbf{E} nhanced Reinforcement Learning (SCE), which incorporates semantic consistency into the reward function to guide multi-hop reasoning. Experimental results demonstrate that SCE outperforms strong baseline methods and facilitates the discovery of more interpretable reasoning paths.

## introduction knowledge graphs (kgs) are structured collections of factual triples @xcite @xcite that have been widely applied in downstream natural language processing (nlp) tasks, such as recommendation systems @xcite @xcite and question answering @xcite . however, the inherent incompleteness of most kgs, due to missing facts, limits their effectiveness in these applications @xcite . knowledge graph reasoning (kgr) has emerged as a promising solution to infer missing information. embedding-based methods @xcite aim to learn vector representations of entities and relations, capturing implicit relationships in high-dimensional spaces. while these approaches yield strong performance in link prediction tasks, they lack interpretability, which is crucial for understanding model decisions @xcite . to address this issue, rulebased methods @xcite have been proposed to mine logical rules @xcite , offering improved explainability. however, they often suffer from poor generalization and high computational cost @xcite . multi-hop reasoning methods (mrms) @xcite @xcite @xcite , which incorporate reinforcement learning (rl), strike a balance between performance and interpretability. these methods model knowledge inference as a sequential decision-making process. for instance, given a query (tom, educatedat, ?), a reasoning agent may correctly reach the answer mit via a semantically coherent path (a) in figure 1 . however, the same answer could also be reached via an unrelated path (b), if the reward is based solely on the final answer. this issue-where incorrect reasoning leads to correct answers-is known as the spurious path problem, a persistent challenge in mrms @xcite . this stems from reward functions that focus on factual accuracy while overlooking the semantic plausibility of the reasoning path @xcite . several methods have been proposed to mitigate this issue, including action dropout @xcite , high-quality rule mining @xcite , and spuriousness metrics @xcite . however, these approaches typically rely on external rules and fail to leverage the internal semantic consistency among triples within a reasoning path. in this work, we propose a novel solution centered on enhancing the semantic consistency of reasoning paths. as illustrated in figure 1 , the triples in path (a)-(tom, student of, mary) and (mary, works at, mit)-are more semantically aligned with the target triple (tom, educatedat, mit) than those in path (b)-(tom, is friends with, jerry) and @xcite . this semantic consistency is influenced by both the relations and the entities involved. we define semantic consistency functions to measure such consistency and incorporate them into the learning process. our contributions are summarized as follows: ## related work embedding-based methods such as transe @xcite , transr @xcite , and complex @xcite have been widely adopted due to their efficiency and strong predictive performance. these methods predict missing triples in kgs by embedding entities and relations into low-dimensional vector spaces, utilizing similarity or distance between embedding vectors. however, the lack of interpretability in these approaches hinders their practical applications. to address this limitation, several hybrid approaches have emerged that aim to combine the strengths of embedding-based models with interpretable reasoning techniques. for instance, rlogic @xcite learns predicate embeddings while incorporating recursive logical rules to enhance reasoning capability. the multi-hop method @xcite employs pre-trained embeddings to generate soft rewards, alleviating issues of sparse supervision and false negatives in reinforcement learning-based multi-hop reasoning. rf2 @xcite constructs a dynamic reasoning framework grounded on transe embeddings to further boost performance. similarly, pt-mh @xcite leverages pre-trained embeddings to represent entities and relations, thereby improving reasoning accuracy. in parallel, many researchers have explored the incorporation of external rules to improve the performance of interpretable methods. for example, chatrule @xcite takes advantage of the semantic understanding capabilities of large language models to efficiently extract logical rules and perform reasoning. rarl @xcite incorporates high-quality rules as prior knowledge, guiding the model to select more reasonable reasoning paths. ps-agent @xcite introduces the path spuriousness (ps) metric to quantify the reliability of a reasoning path and integrates it into the reward function to steer the agent toward more reasonable paths. while these approaches significantly improve reasoning accuracy and interpretability by leveraging external signals, they often overlook the internal semantic consistency between triples. this omission results in the persistent generation of spurious paths (paths that are logically weak but yield correct answers) undermining the transparency of the reasoning process @xcite . although prior work, such as action dropout in multi-hop @xcite , rule-based filtering in rarl @xcite , and the use of ps metrics in ps-agent @xcite , have attempted to alleviate this issue, their reliance on external resources remains a limitation. in contrast, our work focuses on the semantic consistency between triples, emphasizing internal consistency in the reasoning process. by modeling and leveraging the semantic consistency between intermediate and target triples, our method encourages the discovery of more interpretable and semantically grounded reasoning paths. ## preliminaries in this section, we review the definition of the multihop reasoning problem ( §3.1) and introduce the formulation of reinforcement learning ( §3.2). ## methodology building on the observations presented in the introduction section, that reasonable multi-hop reasoning paths often exhibit strong semantic consistency among triples, we propose a semantic consistency framework (sc) to quantitatively evaluate this consistency. the proposed framework consists of two complementary components: relation consistency and entity consistency. ## experiments we introduce the benchmark datasets, baseline models, hyperparameters, and evaluation protocol ( §5.1). we compare our model with baselines ( §5.2), analyze its interpretability ( §5.3), evaluate performance across different relation types ( §5.4), and assess runtime efficiency relative to external rule-based methods ( §5.5). we conclude with case studies that qualitatively demonstrate the effectiveness of our approach ( §5.6). ## conclusion we propose semantic consistency enhanced reinforcement learning, a multi-hop reasoning framework that enhances the interpretability of reasoning paths. sce models semantic consistency between triples by decomposing it into two complementary components: relation consistency and entity consistency. these components are integrated into a unified consistency score, which serves as the basis for a reward function that guides the agent toward more semantically coherent reasoning paths. extensive experiments demonstrate that sce improves both triple prediction accuracy and path interpretability, while maintaining significantly lower computational overhead compared to methods relying on external rules.