Beyond Completion: A Foundation Model for General Knowledge Graph Reasoning (2025)

Abstract: In natural language processing (NLP) and computer vision (CV), the successful application of foundation models across diverse tasks has demonstrated their remarkable potential. However, despite the rich structural and textual information embedded in knowledge graphs (KGs), existing research of foundation model for KG has primarily focused on their structural aspects, with most efforts restricted to in-KG tasks (e.g., knowledge graph completion, KGC). This limitation has hindered progress in addressing more challenging out-of-KG tasks. In this paper, we introduce MERRY, a foundation model for general knowledge graph reasoning, and investigate its performance across two task categories: in-KG reasoning tasks (e.g., KGC) and out-of-KG tasks (e.g., KG question answering, KGQA). We not only utilize the structural information, but also the textual information in KGs. Specifically, we propose a multi-perspective Conditional Message Passing (CMP) encoding architecture to bridge the gap between textual and structural modalities, enabling their seamless integration. Additionally, we introduce a dynamic residual fusion module to selectively retain relevant textual information and a flexible edge scoring mechanism to adapt to diverse downstream tasks. Comprehensive evaluations on 28 datasets demonstrate that MERRY outperforms existing baselines in most scenarios, showcasing strong reasoning capabilities within KGs and excellent generalization to out-of-KG tasks such as KGQA.

## introduction knowledge graphs (kgs) are structured knowledge bases that represent entities and their relationships, providing a foundation for reasoning and information retrieval in various real-world domains. with their rich entity representations and rigorous logical connections, kgs have become integral to applications such as classification @xcite , † corresponding author recommendation @xcite , knowledge retrieval @xcite , and qa systems @xcite , as well as knowledge-grounded llm alignment @xcite . recently, foundation models in nlp and cv @xcite ; chatgpt and barnes (2023); @xcite ; @xcite have demonstrated significant advancements in transfer learning, enabling improved performance across datasets and tasks. inspired by these successes, researchers have developed foundational models for kgs that aim to generalize across datasets and adapt to diverse reasoning tasks. kgs naturally encompass both structural and textual information, yet existing research has predominantly focused on leveraging their structural aspects, with relatively limited attention to the textual modality @xcite @xcite @xcite . however, fully utilizing both modalities is crucial, as textual information provides contextual knowledge that complements structural representations. this integration is particularly important for downstream applications such as commonsense reasoning and kgqa, where the combination of relational and contextual knowledge significantly enhances task performance @xcite @xcite . in addition, prior work has largely been restricted to in-kg reasoning tasks, such as kg completion (kgc), and has not adequately addressed the challenges posed by out-of-kg reasoning tasks, such as kgqa. out-of-kg tasks require models to generalize beyond the explicit structure of kgs, incorporating both modalities to handle more complex reasoning scenarios. overcoming these limitations involves addressing three key challenges in model design: (1) mitigating the semantic disparity between textual and structural information to facilitate effective integration; (2) balancing the contributions of textual and structural modalities to suit diverse task requirements, particularly for reasoning beyond kgs; and to address these challenges, we propose the multi-perspective reasoning system, merry, a universal knowledge graph reasoning framework. merry integrates textual and structural information through a global structural semantic encoding module (gcmp), designed to reconcile their semantic differences. to enhance adaptability, we introduce a dynamic text-adaptive fusion module (dtaf) that selectively preserves essential textual information, facilitating effective application across a range of tasks. furthermore, we develop a flexible edge scoring mechanism that adjusts adaptively to meet the specific requirements of downstream tasks, thereby enhancing the model's transferability across diverse reasoning scenarios. both in-kg (zero-shot kgc) and out-of-kg (kgqa) tasks are evaluated in our merry. results across 28 datasets demonstrate that merry consistently outperforms multiple benchmark models in both tasks, highlighting its robust generalization and adaptability. our codes are released to the github foot_0 . the main contributions of this paper are as follows: recent efforts leverage pre-training paradigms from nlp and cv. for example, ultra @xcite identifies meta-topology types in kg structures, enabling zero-shot transfer through dataset-agnostic representations of entities and relations. nevertheless, it remains limited to structural information and does not incorporate textual modalities, which are critical for contextual reasoning. moreover, it focuses exclusively on in-kg reasoning tasks, neglecting out-of-kg tasks. ## task definition a kg with textual information is defined as @xmath0, r, t , d}, where e and r are the set of entities and relations, d is the set of textual descriptions for entities and relations. the set of factual triples in the kg is denoted as @xmath1, r, e t )| e h , e @xmath2, @xmath3, where e h , e @xmath4. inductive kgc. kg completion (kgc) task aims to predict the correct entity e from the given kg g for query (h, r, ?) or (?, r, t). in particular, inductive kgc tasks aim to train a score function based on the train kg g @xmath5, r tr , t tr , d tr }. considering the different inductive settings of the test kg g @xmath6, r te , t te , d te }, we can categorize the evaluation into: (1) kg containing only unseen entities, which satisfies e tr ̸ = e te and r @xmath7; (2) kg containing both unseen entities and unseen relations, which satisfies e tr ̸ = e te and r tr ̸ = r te . ## methodology a detailed breakdown of merry's components is presented in this section, as illustrated in figure 1 . merry adopts an encoder-decoder architecture, and its processing can be formalized as follows: in the encoding phase, merry encodes the graph structure to derive its structural representation (section 4.2) and explores strategies to effectively integrate textual and structural information (section 4.3). a multi-perspective fusion module further enhances this process, enabling robust feature integration while preserving key textual semantics (section 4.4). additionally, we employ a flexible edge scoring mechanism to adapt to different tasks (section 4.5). in the decoding phase, a flexible cross-attention decoder facilitates adaptation to diverse downstream tasks, including zero-shot kgc and kgqa. ## experiments we evaluate merry on 28 datasets across two tasks: inductive knowledge graph completion (kgc) and knowledge graph question answering (kgqa). our evaluation focuses on the following research questions: rq1: how effective is merry in reasoning for in-kg tasks under a zero-shot setting? rq2: can merry effectively transfer and generalize to out-of-kg tasks? rq3: what is the impact of key components on the performance of merry? rq4: how do key hyperparameters affect the performance of merry? ## conclusion in this paper, we introduced merry, a general knowledge graph reasoning framework that bridges textual and structural modalities through multichannel cmp encoding and multi-perspective dynamic fusion mechanisms. additionally, we proposed a flexible edge scoring mechanism to adapt to diverse downstream tasks. experiments across 28 datasets demonstrate merry's strong generalization capabilities in in-kg tasks, such as zeroshot kgc, and its adaptability to out-of-kg tasks, such as kgqa, highlighting its potential as a unified framework for reasoning across in-kg and out-of-kg tasks.