Title: Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Knowledge Graph

Graph Foundation Model

Graph Message Passing
Abstract: Inspired by the success of large language models, there is a trend toward developing graph foundation models to conduct diverse downstream tasks in various domains. However, current models often require extra fine-tuning to apply their learned structural and semantic representations to new graphs, which limits their versatility. Recent breakthroughs in zero-shot inductive reasoning on knowledge graphs (KGs), offer us a new perspective on extending KG reasoning to general graph applications. In this paper, we introduce SCR, a unified graph reasoning framework designed to train on knowledge graphs and effectively generalize across a wide range of graph tasks and domains. We begin by designing the task-specific KG structures to establish a unified topology for different task formats. Then we propose semantic-conditioned message passing, a novel mechanism addressing the inherent semantic isolation in traditional KG reasoning, by jointly modeling structural and semantic invariance patterns in graph representations. Evaluated on 38 diverse datasets spanning node-, link-, and graph-level tasks, SCR achieves substantial performance gains over existing foundation models and supervised baselines, demonstrating its remarkable efficacy and adaptability.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel graph foundation model (GFM) framework called SCR, which aims to enhance the transferability of knowledge graph (KG) reasoning to general graph learning tasks. The authors introduce a semantic-conditioned message passing mechanism to address the semantic isolation issue in traditional KG reasoning. Extensive experiments across diverse graph datasets demonstrate the effectiveness and adaptability of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\n### Paper Strength\n1. The paper introduces a novel graph reasoning framework, SCR, which adapts the principles of knowledge graph reasoning to general graphs learning tasks like node classification, link prediction, and graph classification.\n2. The proposed semantic-conditioned message passing mechanism effectively considers the node semantics, addressing the semantic isolation issue in previous KG foundation models.\n3. The paper conducts extensive experiments on diverse graph datasets, covering node-level, link-level, and graph-level tasks across multiple domains, showcasing the effectiveness and generalizability of the proposed approach.\n\n### Paper Weakness\n1.  The paper argues that existing attempts to apply node semantic features into the $Init()$ function would lead to the violation of target node distinguishability following previous research [1]. Motivated by this, SCR proposes a novel Semantic-injected INIT Function, and incorporates semantic features in a late-fusion manner (Eq.7). However, the paper does not provide sufficient comparison with other attempts that incorporate semantic features and also hold the target node distinguishability. Meanwhile, the semantic isolation issue should be further analyzed to enhance the motivation of the proposed method. \n\nFor example, in the Section 4.1 and Appendix C.3 of [1], the authors discuss that the $Init^3()= \\mathcal{1}_{u=v} * (z_q + \\epsilon_u )$ can be used to incorporate node semantic feature $\\epsilon_u$ while maintaining the target node distinguishability by adding query-dependent vector $z_q$. This can be considered as an early-fusion method and is easier to implement than the proposed SCR. The paper should provide more details on why the proposed late-fusion method is necessary and how it outperforms other methods that also incorporate semantic features [2,3]. \n\nBesides, the original embedding dimension of the semantic features is 768, which is much larger than the dimension of the node features (64). Such a large dimension gap may lead to a significant loss of semantic features during the feature fusion process and impair the performance as shown in Figure 2. Can the authors further conduct an analysis on the impact of the dimension gap between the semantic features and node features on the performance of the model to justify the necessity of the proposed method?\n\n2. The proposed SCMP only considers the semantics of the node. How does the proposed method handle the semantics of the edges [4]? This can be quite important for tasks like link prediction, where the edge semantics can provide additional information for reasoning. Moreover, for node classification on homogeneous graphs, there is only one type of edge, which makes the relation graph a fully-connected graph and provides no meaningful information for learning the relation representation.\n\n3. The proposed method shows a great zero-shot performance on node classification and graph classification tasks. However, the performance of the model decreases after a few-shot fine-tuning. The paper does not provide sufficient analysis on why the performance drops after a few-shot fine-tuning. Is the model trained from scratch, or is it fine-tuned from a pre-trained model? If it is fine-tuned from a pre-trained model, how does the pre-training and fine-tuning affect the performance of the model? \n\n\n[1] Huang, Xingyue, et al. \"A theory of link prediction via relational weisfeiler-leman on knowledge graphs.\" Advances in Neural Information Processing Systems 36 (2023): 19714-19748.   \n[2] Combining Structure and Text: Learning Representations for Reasoning on Graphs, https://openreview.net/forum?id=hJ8OQAiTrl   \n[3] Hua, Yin, et al. \"Beyond Completion: A Foundation Model for General Knowledge Graph Reasoning.\" arXiv preprint arXiv:2505.21926 (2025).   \n[4] Arun, Arvindh, et al. \"SEMMA: A Semantic Aware Knowledge Graph Foundation Model.\" arXiv preprint arXiv:2505.20422 (2025).\n\nQUESTIONS:\n1. Can you compare the performance of the proposed SCR with other methods that also incorporate semantic features into the conditioned message passing framework to show the necessity of the proposed late-fusion method? Especially, when setting the dimension of the node features closer to the dimension of the semantic features and using Init^3 fusion, would the trend in Figure 2 still hold?\n\n2. In the homogeneous graphs node classification, are there only two types of edges (i.e., original edges and label edges)? How does the proposed method generate meaningful relation representations from the relation graph for reasoning? \n\n3. Why does the performance of the model drop after a few-shot fine-tuning? Is the model trained from scratch, or is it fine-tuned from a pre-trained model? Would fine-tuning from a pre-trained model improve the performance of the model?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study broadens the scope of knowledge graph foundation models by enabling their application to both node-level and graph-level tasks. To support this expansion, it introduces a unified topological framework that accommodates diverse task formats and develops a semantic-conditional message passing mechanism to address the challenge of semantic isolation. Experimental results across link-level, node-level, and graph-level datasets confirm the model’s strong generalization capabilities.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The primary contribution of this work lies in extending the capabilities of Knowledge Graph Foundation Models (KGFMs) beyond traditional link prediction, enabling their application to both node-level and graph-level tasks.\n\n- The introduced modules effectively mitigate the identified semantic isolation challenge.\n\n- Extensive experimental results validate the model’s effectiveness across diverse datasets and task types.\n\nWeaknesses:\n\n- The link prediction performance on transductive datasets does not demonstrate a significant advantage compared to ULTRA.\n\n- The reliance on Semantic Conditional Message Passing (SCMP) may constrain the model’s reasoning ability on knowledge graphs that lack rich textual features. Besides, in the introduction, a brief explanation of \"semantic features\" should precede the discussion of the \"semantic isolation issue,\" rather than deferring this explanation to the appendix. This would significantly enhance clarity and readability. Notably, in knowledge graph reasoning, semantic features are not strictly essential—as indicated by the relatively modest performance difference between SCR and ULTRA in Table 1. However, these features are often indispensable in certain node-level and graph-level tasks, which likely underpins the emergence of the semantic isolation issue in this work. I encourage the authors to revise this section to reflect this context more clearly.\n\nQUESTIONS:\n- Can SCR be applied to knowledge graphs (KGs) that lack textual features?\n\n- Does SCR face limitations when scaling to large KGs? Specifically, is it capable of handling large-scale transductive datasets?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces **SCR (Semantic Conditional Reasoner)**, a framework that leverages *inductive knowledge graph (KG) reasoning* as a foundation for general-purpose graph learning. Instead of pretraining on general graphs directly, the authors propose transforming diverse graph tasks (like node or graph classification) into KG reasoning problems using task-specific KG structures. A key challenge they address is the *semantic isolation problem*, where existing KG reasoning models struggle to integrate node features effectively. To overcome this, SCR introduces **Semantic Conditional Message Passing (SCMP)**, which combines structural and semantic information via a new initialization scheme and non-parametric semantic encoding. The framework supports semantic generalization by using feature-agnostic unification and modular pretraining with various semantic sources. By training exclusively on KGs, SCR can perform zero-shot inference across multiple graph domains and tasks without fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**  \n1. The paper proposes a unified framework that reformulates multiple graph tasks as inductive link prediction problems on knowledge graphs (KGs), offering a different angle on generalizing Graph Foundation Models (GFMs).  \n2. The method demonstrates competitive results across standard benchmarks.\n\n**Weaknesses**  \n1. Graph-level and node-level tasks often require modeling local neighborhoods, community structures, or global graph semantics—factors not typically present in KG link prediction tasks. Without pretraining on such structures, it is unclear how the learned representations transfer effectively.\n2. The comparison with ULTRA is potentially unfair. ULTRA is trained solely on KGs and not designed for general graph tasks. In contrast, the proposed method is designed a broader range, giving it a task diversity advantage. \n3. The model performs better on homophilic graphs than heterophilic ones, suggesting it relies heavily on feature similarity or local neighborhood consistency. This raises concerns about its robustness across structural regimes, which are common in real-world graphs.  \n4. Over-smoothing and over-squashing are known issues in message-passing-based models, especially in deep architectures or long-range dependency tasks. The paper does not provide any discussion or evaluation on these issues. Benchmarks like LRGB or synthetic long-range tasks would help assess whether the model is scalable in such settings.\n\nQUESTIONS:\n1. Section \"CMP-based Backbone Model\" is crucial in understanding the paper, but it is not carefully written. Could the author elaborate more, such as what is \"conditioned on the query\" or the initialization process of $h_v$ ?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SCR, a novel graph foundation model designed to address the challenge of transferring knowledge from knowledge graphs (KGs) to general graph domains and tasks without requiring extra fine-tuning. Inspired by recent advances in zero-shot KG reasoning, SCR unifies diverse graph tasks (node, link, and graph classification) into an inductive KG reasoning framework. The authors propose a task-specific KG transformation that reframes general graph problems as KG reasoning, and introduce Semantic Conditional Message Passing (SCMP), which integrates both structural and semantic information to overcome the \"semantic isolation\" problem in traditional KG models. Extensive experiments on 38 datasets across multiple domains demonstrate that SCR achieves strong zero-shot and few-shot generalization, outperforming existing graph foundation models and supervised baselines on a range of inductive reasoning and classification tasks. The paper provides detailed ablations and discussions on the impact of semantic features, model components, scalability, and limitations.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths\n  - The paper provides a comprehensive empirical evaluation across 38 datasets spanning node-level, link-level, and graph-level tasks. This broad experimental scope convincingly demonstrates the generalizability and robustness of the proposed approach across a wide range of domains and graph types.\n  - The motivation for bridging knowledge graph reasoning and general graph foundation models is well-articulated, and the paper offers a thorough description of the SCR framework and its components. The unified transformation of graph tasks into KG reasoning is clearly presented, and the introduction of Semantic Conditional Message Passing (SCMP) is well-explained. Most methodological details are accessible, with only minor sections that could benefit from further clarification.\n\n- Weaknesses\n  - The main figure intended to illustrate the SCR framework does not significantly aid understanding of the method. As presented, it lacks clarity and does not effectively convey the model’s workflow or the intuition behind key components, which could hinder readers’ comprehension.\n  - The main innovation appears to be a practical and effective integration of node semantic information based on the ULTRA framework. The introduction of virtual nodes to convert various tasks into reasoning tasks on graphs primarily serves this integration, which to some extent reduces the novelty of this work.\n  - The largest performance gains are observed in graph classification tasks. However, it remains unclear to what extent the improvement is attributable to the introduction of semantic information. If such ablation studies have not been conducted, additional experiments isolating the effect of semantic features on graph classification would strengthen the claims. Additionally, did SCR (3g), ULTRA (3g) and ProLINK(3g) consistently use the same KGs?\n  - The methodology section could be better organized. Currently, the lack of an initial high-level overview of the entire approach makes it difficult for readers to grasp the core ideas before delving into technical details. This affects the overall readability and the depth at which readers can understand the method.\n\nQUESTIONS:\nPlease see weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}