Title: Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph Machine Learning

Federated Graph Learning

Graph Foundation Model
Abstract: Recent advances in graph machine learning have shifted to data-centric paradigms, driven by two emerging research fields: 
    (1) Federated graph learning (FGL) facilitates multi-client collaboration but struggles with data and task heterogeneity, resulting in limited practicality;
    (2) Graph foundation model (GFM) enables desirable domain generalization but is typically confined to single-machine training, neglecting the potential of cross-silo data and computational resources.
    It is evident that these two paradigms are complementary, and their integration offers substantial advantages.
    Motivated by this, we present a pioneering study about the federated graph foundation model (FedGFM), a novel decentralized GFM training paradigm. Despite the promising vision of FedGFM, knowledge entanglement has emerged as a critical challenge, where multi-domain knowledge is encoded into indistinguishable representations, thereby limiting downstream adaptation.

To this end, we propose FedGFM+, an effective FedGFM framework with two key modules to mitigate knowledge entanglement in a dual-pronged manner. 
(1) AncDAI: From a global perspective, we introduce a novel anchor-based domain-aware initialization strategy. Before pre-training, each client encodes its local graph into a domain-specific prototypes, which serve as semantic anchors in the representation space. Around each anchor, we construct synthetic embeddings to initialize the global model. We theoretically show that these prototypes are distinguishable across domains, and the initialization provides a strong inductive bias that facilitates disentanglement of domain-specific knowledge. 
(2) AdaDPP: From a local perspective, during pre-training, each client independently learns a lightweight graph prompt that captures domain semantic preferences. During fine-tuning, prompts from all clients are aggregated into an adaptive domain-sensitive prompt pool, from which the GFM selects relevant prompts to augment the target graph’s attributes, thereby improving the downstream adaptation. FedGFM+ is extensively evaluated on 8 diverse benchmarks spanning multiple domains and tasks, outperforming 20 baselines from isolated supervised learning, FGL, and federated variants of centralized GFM paradigms.

Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement

Abstract

Recent advances in graph machine learning have shifted to data-centric paradigms, driven by two emerging research fields: (1) Federated graph learning (FGL) facilitates multi-client collaboration but struggles with data and task heterogeneity, resulting in limited practicality; (2) Graph foundation model (GFM) enables desirable domain generalization but is typically confined to single-machine training, neglecting the potential of cross-silo data and computational resources. It is evident that these two paradigms are complementary, and their integration offers substantial advantages. Motivated by this, we present a pioneering study about the federated graph foundation model (FedGFM), a novel decentralized GFM training paradigm. Despite the promising vision of FedGFM, knowledge entanglement has emerged as a critical challenge, where multi-domain knowledge is encoded into indistinguishable representations, thereby limiting downstream adaptation.

To this end, we propose FedGFM+, an effective FedGFM framework with two key modules to mitigate knowledge entanglement in a dual-pronged manner. (1) AncDAI : From a global perspective, we introduce a novel anc hor-based d omaina ware i nitialization strategy. Before pre-training, each client encodes its local graph into a domain-specific prototypes, which serve as semantic anchors in the representation space. Around each anchor, we construct synthetic embeddings to initialize the global model. We theoretically show that these prototypes are distinguishable across domains, and the initialization provides a strong inductive bias that facilitates disentanglement of domain-specific knowledge. (2) AdaDPP : From a local perspective, during pre-training, each client independently learns a lightweight graph prompt that captures domain semantic preferences. During finetuning, prompts from all clients are aggregated into an ada ptive d omain-sensitive p rompt p ool, from which the GFM selects relevant prompts to augment the target graphs attributes, thereby improving the downstream adaptation. FedGFM+ is extensively evaluated on 8 diverse benchmarks spanning multiple domains and tasks, outperforming 20 baselines from isolated supervised learning, FGL, and federated variants of centralized GFM paradigms.

(a) Limitations of FGL Approaches

(b) Limitations of GFM Approaches

Multi-Domain Data Isolation

Silo 1

Silo 3

Silo 2

Social

Network

Protein

Network

Co-Purchase

Network

Data Processing

TF-IDF

Data Collection

BOW

Node-level

FGL

Subgraph-level

FGL

Task Heterogeneity

Graph-level

FGL

(c) A Naive FedGFM Paradigm

Client 1

Client 3

Client 2

Local Data

Collection

Standard

Processing

Node/Edge

Text

Description

Node/Edge

Text

Description

LM

Local

Pre-Training

Local

Model

Attributed

Graph

Local Latent

Representation

Local

Model

Local

Model

Local

Model

Graph

Foundation

Model

Client 1

Execution

Server

Execution

Cross-Domain & Cross-Task

General representation

Model

Aggregation

Task-oriented

Fine-Tuning

Learning System

Silo 1

Silo 3

Silo 2

Cross-Silo Storage & Computation Neglect

Upload

Upload

Upload

Broadcast

Clients

二

000

TXI

TXT

52

A

+

T

000

GPL

Y

010

TXT

A:

1 Introduction

Recent advances in computational capabilities have sparked a data-centric paradigm shift in deep learning. Moving beyond an exclusive reliance on architectural innovations, the AI community now prioritizes large-scale data utilization, as evidenced by the success of GPT-4 [1] in language processing and Sora [30] in vision tasks. This data-centric scaling trend also extends to graph machine learning, where two learning paradigms are gaining prominence (1) Federated graph learning (FGL) enables cross-silo graph collaboration; (2) Graph foundation models (GFM) promote multi-domain graph generalization. However, both of them face practical deployment limitations.

Two limitations hinder FGL from achieving cross-domain and cross-task collaboration, as illustrated in Fig. 1 (a): (1) Data Heterogeneity. Due to diverse data sources and processing methods, client graphs often differ in feature dimension, label space, and topology pattern. As a result, most FGL methods are confined to collaboration across subsets of a single dataset [62, 26, 20]. While GCFL+ [53] and FedStar [36] enable limited cross-domain collaboration via domain-aware client clustering or feature-agnostic parameter sharing, they are only applicable to graph-level tasks and lack the ability to capture cross-domain general knowledge at the feature level. (2) Task Heterogeneity. Existing FGL assumes uniform graph granularity and downstream tasks across clients, enforcing one of three settings: node-level (ego-networks for node classification/link prediction), subgraphlevel (induced subgraphs from a global graph for node classification/link prediction), or graph-level (graph sets for classification/regression) [16]. As a result, existing FGL approaches often adopt taskspecific designs in both model architectures and training algorithms, which significantly limits their ability to support collaboration across multi-task graph data.

Meanwhile, existing GFM studies face the following two limitations, as illustrated in Fig. 1 (b): (1) Multi-Domain Data Isolation. Training generalizable GFMs requires diverse graph data spanning multiple domains, like social networks, molecular structures, etc. Although a number of public graph datasets are available, they remain limited in both scale and diversity. In contrast, real-world graph data is expected to continuously grow in volume and variety, yet it is often distributed across institutions and isolated in data silos due to privacy regulations or commercial competition. This renders existing centralized GFM approaches increasingly infeasible. (2) Cross-Silo Storage and Computation Neglect. Although current GFMs require significantly fewer storage and computation resources than their NLP or vision counterparts, which makes them feasible within a single institution, centralized training frameworks inherently fail to leverage the vast yet fragmented storage and computation capacities distributed across multiple silos in real-world deployments. This under-utilization results in non-trivial opportunity costs, such as redundant resource provisioning and sub-optimal training efficiency.

Fortunately, FGL and GFM exhibit a naturally complementary relationship. Specifically, FGL equips GFM with a decentralized training paradigm that supports learning across distributed silos while efficiently utilizing cross-silo storage and computational resources. In contrast, GFM enhances FGL by offering unified feature encoding and a pre-training followed by fine-tuning framework, thereby facilitating generalized collaboration across diverse graph domains and task types. To this end, we introduce Federated Graph Foundation Model (FedGFM), a novel and practical paradigm designed for training GFM over decentralized, cross-domain, and cross-task graphs. As illustrated in

Fig. 1 (c), the FedGFM paradigm follows a pipeline that begins with federated pre-training and proceeds with fine-tuning. During the federated pre-training phase, each client performs self-supervised learning on its private graph to acquire domain-specific representations. The server then aggregates these local models to construct a global model that captures generalizable topological and semantic patterns. The global model is subsequently broadcast to clients as the initialization for the next round of federated pre-training. This iterative process continues across multiple rounds of federated communication. In the fine-tuning phase, the global model is treated as a graph foundation model and is further adapted to specific downstream tasks through supervised learning.

To establish an effective FedGFM framework, our work begins with an empirical investigation (Sec. 3), assessing its feasibility and revealing a non-trivial challenge. Specifically, (1) From a feasibility perspective, FedGFM faces stringent communication constraints, as frequent transmission of large-scale model parameters or gradients is often impractical in real-world federated deployments. This limitation calls for a lightweight yet expressive model architecture. Fortunately, the graph vector quantization-variational auto-encoder (gVQ-VAE), which is widely used as the backbone in centralized GFM, presents a promising solution. It has been extensively validated for its ability to jointly encode graph structures and text attributes into discrete, semantically meaningful representations [41, 43], making it well-suited for multi-domain pre-training. Meanwhile, its lightweight design naturally aligns with the communication-efficiency requirements of FedGFM. (2) However, naively distributing the pre-training of gVQ-VAE across local clients in a federated setting introduces a critical challenge we term knowledge entanglement. Unlike centralized training, federated pre-training operates on multiple isolated, domain-specific graphs, each with distinct data distributions. Each client's local trained model tend to overfit their domain-specific data without alignment across clients. Consequently, the aggregated global GFM encodes multi-domain graphs into indistinguishable representations and further limits its downstream generalization.

Building upon these insights, we present an effective FedGFM framework named FedGFM+, which involves two key modules to mitigate knowledge entanglement in a dual-pronged manner: (1) AncDAI : From a global perspective, we introduce a novel anc hor-based d omaina ware i nitialization strategy. Before pre-training, each client encodes its local graph into a domain-specific prototype, which serve as semantic anchors in the representation space. Around each anchor, we construct synthetic embeddings to initialize the global model. We theoretically show that these domain prototypes are distinguishable across domains, and the initialization provides a strong inductive bias that naturally facilitates encourages separation among knowledge representations from different domains. (2) AdaDPP : From a local perspective, during the pre-training stage, each client independently learns and retains a lightweight, domain-sensitive prompt that captures its local semantic preferences, without participating in federated aggregation. In the fine-tuning stage, these prompts are assembled into an ada ptive d omain-sensitive p rompt p ool. For a given target graph, the model selects and incorporates the most relevant prompts from the pool based on its semantic characteristics. These prompts serve as domain-specific priors that condition the GFMs representations, thereby enabling adaptive exploitation of domain knowledge and facilitating improved adaption to downstream tasks.

Our Contributions. (1) Problem Identification. To the best of our knowledge, this is the first exploration of the FedGFM paradigm, which organically combines FGL and GFM to offer a practical solution for training graph foundation model across silos with diverse graph domain and tasks. (2) In-depth Investigation. (Sec. 3) We conduct an in-depth empirical investigation for FedGFM, assessing its feasibility and revealing a non-trivial challenges named knowledge entanglement, providing valuable insights for its development. (3) Novel Framework. (Sec. 4) We propose a novel and effective FedGFM framework named FedGFM+, which employs two key modules to address the knowledge entanglement challenge, including AncDAI from the global perspective and AdaDPP from the local perspective. (4) State-of-the-art Performance. (Sec. 5) Extensive experimental results on graph learning with 8 cross-task and cross-domain datasets demonstrate the superiority of FedGFM+ compared with 20 baselines, including 5 isolated supervised learning methods, 10 FGL techniques, and 5 federated variants of centralized GFM training strategies.

2 Preliminaries and Problem Formalization

Text-Attributed Graph. Consider a text-attributed graph G = ( V , E ) , where V is the set of nodes and E is the set of edges. Each node v i ∈ V and edge e i ∈ E may be associated with a textual description, which is encoded into a semantic vector using a specific embedding technique (e.g.,

bag-of-words, pre-trained language models). Depending on the downstream task, the graph may be equipped with supervision signals at different levels: node-level labels (for node classification), edgelevel labels (for edge classification or link prediction), or graph-level labels (for graph classification).

Graph Vector Quantization-Variational Auto-Encoder as GFM Backbone. Most recent GFMs adopt gVQ-VAEs as the trainable GNN. This backbone enables the joint encoding of topology and textual attributes into a discrete embedding space with clear semantic boundaries, making it particularly suitable for multi-domain GFM pre-training. Specifically, (1) G ′ = ( V , E , X ) → Encoder → Embeddings : To ensure generality in arbitrary inputs, the Encoder can be instantiated as any reasonable GNN capable of incorporating both node and edge features to generate informative embeddings z ∈ R d . (2) Embeddings → Codebook → Quan. Emb. : To establish clear semantic boundaries, the Codebook C transforms continuous embeddings z into discrete embeddings e ∈ R d (Quan. Emb. z q ∈ R d ) via similarity retrieval-based vector quantization:

(3) Quan. Emb. → Decoder → G ′ r = ( V , E r , X r ) : To enable the self-supervised training, gVQVAEs follow an autoencoder framework, where gradients are computed by the discrepancy between the reconstructed graph G ′ r and the original input graph G ′ , thereby updating the Encoder and Codebook. Notably, the trainable components of the Encoder and the Codebook are the weighted matrix and the discrete embeddings { e 1 , . . . , e K } , which together constitute the trainable GFM embedding function parameterized by f θ . Meanwhile, to construct end-to-end gradient flow, the straightthrough estimator (STE) [4] is used to approximate gradients by bypassing the non-differentiable quantization step. Formally, the gVQ-VAE is pre-trained via optimizing loss function as follows:

where sg [ · ] represents the stop-gradient operator, n denotes the number of nodes, z i represents the i -th node embedding produced by the GNN encoder, z q i denotes its quantized embedding obtained by retrieving the codebook, and ˆ x i denotes the reconstructed node attributes projected via MLP-based decoders, i.e., ˆ x i = δ ( z q i ) , γ is the scaling factor. More details and related works about gVQ-VAE are presented in Appendix A.

Problem Formalization of FedGFM. For FedGFM, there is a trusted central server and K clients. The subgraphs or graph collections of the client present a relationship such as subgraph-level decentralization or graph-level decentralization (see Appendix. C.2 for more details about data settings). To unify the representation, we regard the graph data held by k -th client as S k , where |S k | = 1 for subgraph-level decentralization. The proposed FedGFM paradigm follows a federated pre-trainingfine-tuning process. For the Federated Pre-Training phase, each client conducts self-supervised training to optimize its local model based on its local graph, and the server aggregates multiple local models to obtain a global graph foundation model. Consider adapting the widely-used FedAvg [32] aggregation strategy in federated learning for vision tasks within the FedGFM framework, the federated pre-training process unfolds as follows: (1) Initialization: At the first communication round ( r =1 ), the central server sets the local model parameters of K clients to the global parameters, i.e., Θ k ← Θ g ∀ k . (2) Local Updates: Each local model performs training on the current local data G k to minimize the self-supervised loss L ( G k ; Θ k ) , and then updating the parameters: Θ k ← Θ k -η ∇L . (3) Global Aggregation: After local training, the server aggregates the local knowledge with respect to the number of training instances, i.e., Θ g ← N k N ∑ K k =1 Θ k with N = ∑ k N k , and distributes the global parameters Θ g to local clients selected at the next round. This process iterates between steps 2 and 3 until reaching the final round R . This iterative cycle continues until the completion of the last round ( r = R ), facilitating collaborative GFM training by parameter sharing without the exchange of private local data. For the Fine-Tuning phase, FedGFM first loads and freezes the pretrained global model from the central server as GFM, then uses available graph supervision signals to fine-tune the task heads to adapt to specific downstream graph tasks.

3 Empirical Investigation

In this section, we present an in-depth empirical study of the FedGFM paradigm, organized around two key questions from different perspectives. Q1 : From the perspective of Feasibility , is FedGFM practical for real-world deployment? Q2 : From the perspective of Effectiveness , what are the main bottlenecks that limit the effectiveness of a naive FedGFM implementation?

To address Q1 , we survey several representative foundation models to quantify their parameter scales, and summarize the results in Table 1. Notably, compared with foundation models in language and vision domains, graph foundation models (GFMs) are significantly more lightweight in terms of parameter size. This suggests that federated pre-training of GFMs is communication-efficient and practically feasible. Among all surveyed GFMs, we further observe that two gVQ-VAE-based methods, GFT [43] and GQT [41], exhibit the smallest parameter scales. This highlights the advantage of the gVQ-VAE architecture in achieving a lightweight yet

(a) Degree Frequency

(b) Representation Similarity

WN18RR

HIV

1

5

10

15

20

25

Frequency (%)

10

20

30

Cosine Similarity

Raw Feat.

GFT Emb.

GFT* Emb.

0.4

0.8

1.0

0.6

0.2

0

WN18RR-HIV

expressive design, making it particularly suitable for FedGFM settings. More related works about GFM are presented in Appendix A.

To address Q2 , we conduct a simple yet illustrative visualization experiment, aiming to reveal the bottlenecks that limit the effectiveness of naive FedGFM. Building on the insight of Q1 , we implement naive federated variants of GFT [43] (denoted as GFT ∗ ), and evaluate GFT and GFT ∗ on three datasets: Cora [56], WN18RR [12], and HIV [47], covering different domains (citation networks, knowledge graphs, and molecular graphs).

The empirical results are presented in Fig. 2. Specifically, panel (a) illustrates the node degree distributions of the Cora, WN18RR, and HIV datasets (restricted to the first 30 degrees starting from 1 for visual clarity), while panel (b) reports the inter-domain cosine similarity among the three datasets, computed in three different representation spaces: (1) the average initial node features, (2) the average node embeddings learned by GFT, and (3) those learned by GFT ∗ . This comparison reveals how well each model distinguishes multi-domain knowledge during representation learning. As observed, the three datasets differ markedly in both topological structure and initial feature distributions. Despite such heterogeneity, centralized GFT pretraining produces a graph foundation model that generates embeddings with clear domain-specific distinctions. This indicates effective preservation of inter-domain variability through joint optimization. In contrast, the embeddings learned by GFT ∗ under decentralized federated pretraining show near-unity inter-domain similarity, reflecting a collapse of domain specificity caused by the absence of coordinated global optimization. We term this the knowledge entanglement , a non-trivial challenge to resolve for effective FedGFM design.

Gradient Backward

(Straight-Through)

Graph

Encoder

…

Node

Embeddings

…

Quantized

Node

Embedding

Query

Assign

Input

Gradient Backward

Codebook

1

2

3

4

N

…

…

Self-Supervised

Graph Reconstruction

Feature

Reconstruction

Topology

Reconstruction

Before Pre-Training

Language

Encoding

Embedding

Averaging

Codebook

1

2

3

4

N

…

…

Anchor-Based Domain-Aware Initialization (AncDAI)

Gaussian Perturbation

Init.

Local

Attributed

Graph

During Pre-Training

Local Graph

Prompts

(a) Federated Pre-Training Stage of FedGFM+

Client 1

Client 3

Client 2

Global

GFM

Upload

Broadcast

Broadcast

Upload

Text

Attributed

Graphs

Client 1

Client 2

Client 3

Vector

Attributed

Graphs

GNN

Encoding

Upload to

Server

Adaptive

Domain-Sensitive

Prompt Pool (AdaDPP)

Before Fine-Tuning

During Fine-Tuning

Global

GFM

Target

Graph

Download

From

Clients

(b) Fine-Tuning Stage of FedGFM+

Local

Graph Prompts

Ensemble

Client 1

Client 3

Client 2

Augmentation

AdaDPP

Augmentation

Server

Download

From Server

Task

Head

Input

Frozen

Trainable

AdaDPP

000

2

中

288

0

D

田

田

田

田

QQ：

。。。

由

610

田

田

田

田

田

↑田

田

田

田

229

田

田

4 Methods

In this section, we introduce the proposed FedGFM+ framework. We first provide an overview of FedGFM+ in Fig. 3. At its core, FedGFM+ adopts a federated pre-training and fine-tuning paradigm. During each communication round of pre-training, clients leverage a local gVQ-VAE encoder to perform self-supervised graph reconstruction, capturing domain-specific semantics. The resulting local models are uploaded to the server for aggregation, yielding an updated global model. The global model is subsequently broadcast to clients as the initialization for the next round of federated pre-training. In the fine-tuning stage, this global model serves as a general-purpose GFM encoder, while a task-specific prediction head is optimized for downstream tasks. Moreover, FedGFM+ introduces two key modules to mitigate the knowledge entanglement challenges: (1) AncDAI : Before pre-training, FedGFM+ employs a novel anchor-based domain-aware initialization strategy to initialize the global codebook, providing a strong inductive bias that facilitates disentanglement of domain-specific knowledge. (2) AdaDPP : During pre-training, each client independently learns a lightweight graph prompt that imbues the GFM with its own domain semantic preferences. During fine-tuning, prompts from all clients are aggregated into an adaptive domain-sensitive prompt pool, from which the GFM selects relevant prompts to augment the target graph attributes, thereby improving the downstream adaptation. Below we introduce these two modules in detail.

4.1 Anchor-Based Domain-Aware Initialization

As discussed in Section 3, naive FedGFM suffers from knowledge entanglement, where representations from different domains collapse into indistinguishable embeddings. To mitigate this, from a global perspective, we aim to endow the global model with a strong inductive bias that explicitly encourages the separation of domain-specific semantics.

Before federated pre-training, to capture domain-specific knowledge, we introduce a domain prototype extraction mechanism, which models intrinsic patterns in the graph topology and node attributes of the local graph and summarizes them into a compact, unified-dimensional vector representation. Specifically, for the k -th client with a local graph G k = ( V k , E k ) , node features X k and adjancency matrix A k , we first compute the node embeddings Z k as follows:

where θ glb denotes the initialized global model parameter broadcast to all clients. The domain prototype p k is then obtained by mean-pooling over node embeddings:

We theoretically demonstrate that, even under a randomly initialized model with shared parameters, the domain prototypesobtained via averaging the encoded node representationsremain distinguishable across clients. This separability stems from intrinsic discrepancies in node features and graph topologies among domains, and can be formally bounded (Appendix B Theorem. B.1).

Each client subsequently uploads its prototype to the central server. To steer the global model toward learning domain-aware representations, we treat these prototypes as semantic anchors and

synthesize local neighborhoods in the embedding space via controlled perturbations. Specifically, for each anchor p k , a set of perturbed embeddings { ˜ p k i } H i =1 is generated as:

where ϵ i is sampled from a standard Gaussian distribution, and σ is a noise scaling factor that ensures numerical stability. Notably, the number of synthetic embeddings H is uniformly allocated across prototypes, depending on the number of the learnable codebook tokens in the global model.

Finally, the synthetic embeddings aggregated from all domains are used to initialize the codebook C of the global model, i.e., C ← Init ( ∪ k { ˜ p k i } H i =1 ) . We further provide a theoretical analysis (Appendix B Theorem. B.2) to demonstrate that this initialization introduces a structured inductive bias, which not only facilitates disentangled representation learning across diverse domains but also stabilizes optimization during the early stages of federated pretraining.

4.2 Adaptive Domain-Sensitive Prompt Pool

Moreover, to address knowledge entanglement from the local perspective, we introduce a novel prompt learning-based mechanism. During the pre-training stage, each client independently learns and retains domain-specific prompts and is excluded from federated aggregation. During the finetuning stage, these prompts serve as semantic priors that condition the GFM's representations, facilitating improved adaptation to diverse downstream tasks.

Concretely, during federated pre-training, each client maintains a set of learnable prompt tokens embedded in its local graphs feature space. For the k -th client, this prompt set is denoted as Φ k = { ϕ k i } λ i =1 , where λ is the number of prompts and F the feature dimensionality. Given the local graph G k = ( V k , E k ) and node features { x k i } v i ∈V k , node representations are enhanced by a weighted combination of prompts, with attention weights computed via λ learnable linear projections:

where α k j reflects the relevance of the j -th prompt to node v i , and w k j is the corresponding learnable projection vector. These prompts and projection weights are optimized together with the local GNN backbone through a self-supervised graph reconstruction task, as described in Eq. 2.

During the fine-tuning stage, we downloads the global model as GFM, which encodes generalizable cross-domain knowledge. In parallel, it collects all locally learned prompts and associated projection weights to construct a adaptive domain-aware prompt pool, denoted as ρ = { ϕ j i } λ,K i =1 ,j =1 and w = [ w 1 , . . . , w K ] . Given a target graph G tgt = ( V tgt , E tgt ) , node features are augmented using this prompt pool. For each node v i ∈ V tgt with feature x tgt i , the enhanced representation is computed as:

As a result, FedGFM+ effectively capitalizes on domain-specific prompts acquired during pretraining, substantially improving its adaptability to heterogeneous domains and diverse downstream tasks in the fine-tuning phase.

5 Experiments

In this section, we present a comprehensive evaluation of FedGFM+. We begin by introducing the experimental setup (Sec.5.1), and then seek to answer the following research questions: Q1 : After task-specific fine-tuning, does the GFM trained by FedGFM+ consistently outperform (1) isolated supervised learning techniques, (2) state-of-the-art FGL baselines, and (3) naive federated variants of centralized GFM strategies across node-, edge-, and graph-level prediction tasks (Sec.5.2)? Q2 : How does each individual module contribute to the overall performance of FedGFM+ (Sec.5.3)? Q3 : Is FedGFM+ robust to changes in hyperparameter configurations (Sec.5.4)? In addition to the main evaluation, we further investigate the few-shot generalization ability ( Q4 ) in Appendix D.

5.1 Experimental Setup

To evaluate the effectiveness of FedGFM+, we conduct experiments on 8 benchmark graph datasets spanning a range of domains and covering three key tasks: node classification (Citation Networks: Cora, PubMed [56], and OGB-Arxiv [19]; Hyper-Link Networks: WikiCS [33]), edge classification (Knowledge Graphs: FB15K237 [37] and WN18RR [12]), and graph classification (Molecule Graphs: HIV, PCBA [47]). Each dataset is partitioned into 3 clients to simulate decentralized scenarios, and we report the average test performance (accuracy or AUC) across clients. We compare FedGFM+ against three baseline categories: (1) Isolated Supervised Models, trained independently on each client, including a linear layer, GCN, GAT, GraphSAGE, and GIN; (2) FL/FGL Approaches, including general-purpose methods like FedAvg and MOON, and task-specific methods such as FedSage+, Fed-PUB, FedGTA, FedTAD, FGSSL, FGGP, GCFL+, and FedStar; and (3) Federated Variants of centralized GFM training strategies (OFA, GFT, UniGraph, GQT, GraphCLIP). More experimental details are provided in Appendix C.

5.2 Performance Comparison (Answers for Q1)

To answer Q1 , we compare FedGFM+ with a range of competitive baselines, evaluating each configuration over 3 independent runs without fixed seeds. As summarized in Table 2, FedGFM+ consistently achieves superior performance across all datasets and downstream tasks.

Comparison with Isolated Supervised Learning. FedGFM+ consistently outperforms supervised backbones, confirming its strong cross-domain and cross-task generalization. Specifically, it improves over the best baselines by at least 2.70% in node classification, 2.18% in edge classification, and 3.09% in graph classification, demonstrating superior transferability and robustness.

Comparison with FL/FGL Methods. As discussed in Section 1, existing FL/FGL methods are limited by data/task heterogeneity and reliance on task-specific information, restricting its training and evaluation scenarios. In contrast, as observed, FedGFM+ consistently outperforms by enabling broad cross- domain and task collaboration that captures general structural and semantic knowledge.

Comparison with Federated Variants of Centralized GFM. As observed, naive federated GFM models often suffer from knowledge entanglement, leading to them even below isolated supervised baselines (i.e., negative transfer). In contrast, FedGFM+ effectively addresses these issues via its design (i.e., AncDAI and AdaDPP), enabling efficient downstream adaptation.

5.3 Ablation Study (Answer for Q2)

To address Q2 , we analyze FedGFM+s two key modules. AncDAI guides the initialization of learnable tokens in the global gVQ-VAE codebook, while AdaDPP is applied during fine-tuning to improve adaptability to domain- and task-specific variations. An ablation study on 8 datasets (Table 3) shows that removing both modules degrades performance. Notably, excluding AncDAI causes a larger drop than excluding AdaDPP, highlighting AncDAIs crucial role in reducing knowledge entanglement and boosting generalization. In summary , both are vital for FedGFM+s effectiveness.

5.4 Sensitivity Analysis (Answer for Q3)

To address Q3 , we perform a sensitivity analysis on key hyperparameters in FedGFM+. As a pre-trainingfine-tuning framework, it involves many hyperparameters; here we focus on those in our core modules. For AncDAI, we vary the number of learnable tokens in the global gVQ-VAE codebook. For AdaDPP, we vary the number of learnable prompts per client. Results are shown in Fig. 4: (a) AncDAI maintains stable perfor-

128

WN18RR

HIV

256

Number of Tokens

(a) Sensitivity of AncDAI

1

2

3

4

5

Number of Prompts

70

75

80

85

90

Test ACC or AUC (%)

WN18RR

HIV

(b) Sensitivity of AdaDPP

mance under different codebook sizes, indicating robust domain initialization; (b) AdaDPP performs

90

85

80

75

70

Test ACC or AUC (%)

512

well with few prompts, and is insensitive to prompt number. Overall, FedGFM+ shows strong robustness to key hyperparameters.

6 Limitation

While FedGFM+ adopts randomly initialized encoders and decentralized optimization to mitigate privacy leakage, we acknowledge that the exchange of high-level representations (e.g., prototypes and prompts) may still expose partial semantic information. A thorough privacy analysis, including the investigation of potential leakage pathways and the development of a threat model, remains an important direction for future work. Incorporating formal privacy guarantees would further strengthen the robustness of our approach in practical federated settings.

7 Conclusion

This paper initiates the study of Federated Graph Foundation Models (FedGFM), aiming to train a unified graph model with domain and task generalization under decentralized settings. By integrating the complementary strengths of Federated Graph Learning (FGL) and centralized Graph Foundation Models (GFM) training strategies, FedGFM alleviates the limitations of both paradigms. Empirical analysis reveals a key challenge, knowledge entanglement, which limits the effectiveness of naive federated adaptations of centralized GFM training. To address this, we propose FedGFM+, a dual-perspective framework incorporating AncDAI and AdaDPP. Experimental results demonstrate the superior performance and generalization ability of FedGFM+.

Acknowledgments

This work was supported in part by the National Key R&D Program of China under Grant 2023YFB3001900, in part by the Shenzhen Science and Technology Program under Grant KJZD20230923113901004, in part by the National Natural Science Foundation of China under Grant 62572501.

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD, 2019.

Jinheon Baek, Wonyong Jeong, Jiongdao Jin, Jaehong Yoon, and Sung Ju Hwang. Personalized subgraph federated learning. 2023.

Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013.

Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10):P10008, 2008.

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020.

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013.

Chuan Chen, Weibo Hu, Ziyue Xu, and Zibin Zheng. Fedgl: federated graph learning framework with global self-supervision. arXiv preprint arXiv:2105.03170, 2021.

Jialin Chen, Haolan Zuo, Haoyu Peter Wang, Siqi Miao, Pan Li, and Rex Ying. Gfse: A foundational model for graph structural encoding.

Jinsong Chen, Chenyang Li, GaiChao Li, John E Hopcroft, and Kun He. Rethinking tokenized graph transformers for node classification. arXiv preprint arXiv:2502.08101, 2025.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1113, 2023.

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.

Dongqi Fu, Wenxuan Bao, Ross Maciejewski, Hanghang Tong, and Jingrui He. Privacypreserving graph machine learning from data to computation: A survey. ACM SIGKDD Explorations Newsletter, 25(1):54-72, 2023.

Xingbo Fu, Binchi Zhang, Yushun Dong, Chen Chen, and Jundong Li. Federated graph machine learning: A survey of concepts, techniques, and applications. ACM SIGKDD Explorations Newsletter, 24(2):32-47, 2022.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems, NeurIPS, 2017.

Chaoyang He, Keshav Balasubramanian, Emir Ceyani, Carl Yang, Han Xie, Lichao Sun, Lifang He, Liangwei Yang, S Yu Philip, Yu Rong, et al. Fedgraphnn: A federated learning benchmark system for graph neural networks. In International Conference on Learning Representations, ICLR Workshop on Distributed and Private Machine Learning, 2021.

Yufei He, Yuan Sui, Xiaoxin He, and Bryan Hooi. Unigraph: Learning a unified cross-domain foundation model for text-attributed graphs. arXiv preprint arXiv:2402.13630, 2024.

Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, and Bryan Hooi. Unigraph2: Learning a unified embedding space to bind multimodal graphs. In Proceedings of the ACM on Web Conference 2025, pages 1759-1770, 2025.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems, NeurIPS, 2020.

Wenke Huang, Guancheng Wan, Mang Ye, and Bo Du. Federated graph semantic and structural learning. arXiv preprint arXiv:2406.18937, 2024.

Xinke Jiang, Rihong Qiu, Yongxin Xu, Yichen Zhu, Ruizhe Zhang, Yuchen Fang, Chu Xu, Junfeng Zhao, and Yasha Wang. Ragraph: A general retrieval-augmented graph learning framework. Advances in Neural Information Processing Systems, 37:29948-29985, 2024.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, ICLR, 2017.

Lecheng Kong, Jiarui Feng, Hao Liu, Chengsong Huang, Jiaxin Huang, Yixin Chen, and Muhan Zhang. Gofa: A generative one-for-all model for joint graph language modeling, 2024.

Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2021.

Xunkai Li, Zhengyu Wu, Wentao Zhang, Henan Sun, Rong-Hua Li, and Guoren Wang. Adafgl: A new paradigm for federated node classification with topology heterogeneity. arXiv preprint arXiv:2401.11750, 2024.

Xunkai Li, Zhengyu Wu, Wentao Zhang, Yinlin Zhu, Rong-Hua Li, and Guoren Wang. Fedgta: Topology-aware averaging for federated graph learning. arXiv preprint arXiv:2401.11755, 2024.

Xunkai Li, Yinlin Zhu, Boyang Pang, Guochen Yan, Yeyu Yan, Zening Li, Zhengyu Wu, Wentao Zhang, Rong-Hua Li, and Guoren Wang. Openfgl: A comprehensive benchmarks for federated graph learning. arXiv preprint arXiv:2408.16288, 2024.

Tianqianjin Lin, Pengwei Yan, Kaisong Song, Zhuoren Jiang, Yangyang Kang, Jun Lin, Weikang Yuan, Junjie Cao, Changlong Sun, and Xiaozhong Liu. Langgfm: A large language model alone can be a powerful graph foundation model. arXiv preprint arXiv:2410.14961, 2024.

Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. arXiv preprint arXiv:2310.00149, 2023.

Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, and Lichao Sun. Sora: A review on background, technology, limitations, and opportunities of large vision models, 2024.

Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan. Gfm-rag: Graph foundation model for retrieval augmented generation. arXiv preprint arXiv:2502.01113, 2025.

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. Artificial Intelligence and Statistics, 2017.

Péter Mernyei and C˘ at˘ alina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks. arXiv preprint arXiv:2007.02901, 2020.

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bertnetworks, 2019.

Li Sun, Zhenhao Huang, Suyang Zhou, Qiqi Wan, Hao Peng, and Philip Yu. Riemanngfm: Learning a graph foundation model from riemannian geometry. In Proceedings of the ACM on Web Conference 2025, pages 1154-1165, 2025.

Yue Tan, Yixin Liu, Guodong Long, Jing Jiang, Qinghua Lu, and Chengqi Zhang. Federated learning on non-iid graphs via structural knowledge sharing. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 9953-9961, 2023.

Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1499-1509, 2015.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.

Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, ICLR, 2018.

Guancheng Wan, Wenke Huang, and Mang Ye. Federated graph learning under domain shift with generalizable prototypes. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 15429-15437, 2024.

Limei Wang, Kaveh Hassani, Si Zhang, Dongqi Fu, Baichuan Yuan, Weilin Cong, Zhigang Hua, Hao Wu, Ning Yao, and Bo Long. Learning graph quantized tokenizers. In The Thirteenth International Conference on Learning Representations.

Shuo Wang, Bokui Wang, Zhixiang Shen, Boyan Deng, and Zhao Kang. Multi-domain graph foundation models: Robust knowledge transfer via topology alignment. International Conference on Machine Learning, ICML, 2025.

Zehong Wang, Zheyuan Zhang, Nitesh Chawla, Chuxu Zhang, and Yanfang Ye. Gft: Graph foundation model with transferable tree vocabulary. Advances in Neural Information Processing Systems, 37:107403-107443, 2024.

Zehong Wang, Zheyuan Zhang, Tianyi Ma, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. Towards graph foundation models: Learning generalities across graphs via task-trees.

Zhen Wang, Weirui Kuang, Yuexiang Xie, Liuyi Yao, Yaliang Li, Bolin Ding, and Jingren Zhou. Federatedscope-gnn: Towards a unified, comprehensive and efficient package for federated graph learning. Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD, 2022.

Zhihao Wen and Yuan Fang. Augmenting low-resource text classification with graph-grounded pre-training and prompting. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR, 2023.

Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: a benchmark for molecular machine learning. Chemical science, 9(2):513-530, 2018.

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4-24, 2020.

Lianghao Xia and Chao Huang. Anygraph: Graph foundation model in the wild. arXiv preprint arXiv:2408.10700, 2024.

Lianghao Xia and Chao Huang. Anygraph: Graph foundation model in the wild. arXiv preprint arXiv:2408.10700, 2024.

Lianghao Xia, Ben Kao, and Chao Huang. Opengraph: Towards open graph foundation models. arXiv preprint arXiv:2403.01121, 2024.

Lianghao Xia, Ben Kao, and Chao Huang. Opengraph: Towards open graph foundation models. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024.

Han Xie, Jing Ma, Li Xiong, and Carl Yang. Federated graph classification over non-iid graphs. Advances in Neural Information Processing Systems, NeurIPS, 2021.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? 2019.

Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. Federated learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 13(3):1-207, 2019.

Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning, ICML, 2016.

Yuhang Yao, Weizhao Jin, Srivatsan Ravi, and Carlee Joe-Wong. Fedgcn: Convergencecommunication tradeoffs in federated training of graph convolutional networks. Advances in neural information processing systems, 36, 2024.

Xingtong Yu, Zechuan Gong, Chang Zhou, Yuan Fang, and Hui Zhang. Samgpt: Textfree graph foundation model for multi-domain pre-training and cross-domain adaptation. In Proceedings of the ACM on Web Conference 2025, pages 1142-1153, 2025.

Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Text-free multi-domain graph pre-training: Toward graph foundation models. arXiv preprint arXiv:2405.13934, 2024.

Huanding Zhang, Tao Shen, Fei Wu, Mingyang Yin, Hongxia Yang, and Chao Wu. Federated graph learning-a position paper. arXiv preprint arXiv:2105.11099, 2021.

Ke Zhang, Lichao Sun, Bolin Ding, Siu Ming Yiu, and Carl Yang. Deep efficient private neighbor generation for subgraph federated learning. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pages 806-814. SIAM, 2024.

Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu Ming Yiu. Subgraph federated learning with missing neighbor generation. Advances in Neural Information Processing Systems, NeurIPS, 2021.

Taolin Zhang, Chengyuan Mai, Yaomin Chang, Chuan Chen, Lin Shu, and Zibin Zheng. Fedego: privacy-preserving personalized federated graph learning with ego-graphs. ACM Transactions on Knowledge Discovery from Data, 18(2):1-27, 2023.

Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI Open, 1:57-81, 2020.

Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang, Kaixiong Zhou, and Yongfeng Zhang. Llm as gnn: Graph vocabulary learning for text-attributed graph foundation models. arXiv preprint arXiv:2503.03313, 2025.

Yinlin Zhu, Xunkai Li, Zhengyu Wu, Di Wu, Miao Hu, and Rong-Hua Li. Fedtad: topologyaware data-free knowledge distillation for subgraph federated learning. arXiv preprint arXiv:2404.14061, 2024.

Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs. In Proceedings of the ACM on Web Conference 2025, pages 2183-2197, 2025.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The brief descriptions of our proposed method and main contributions are provided at the end of the introduction, along with the motivations and overall performance analysis discussed in both the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]

Justification: We discuss the limitation of the proposed FedGFM+ framework in Sec. 6. Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide proofs for each introduced theorem in the paper and clearly present the assumptions, which can be found in Appendix B.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have provided detailed experimental implementation details in the Appendix regarding how we fine-tuned each dataset or method to achieve its optimal performance.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Yes, we have included the source code in the supplementary materials to enable interested researchers to reproduce the experimental results presented in our paper with sufficient guidance.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The training details can be found in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provide all our performances with error bars.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please refer to Appendix C for details regarding to our computing resources.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We are hereby comply to the guidelines illustrated in the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no social impact associated with the work we performed and presented in this paper.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper does not pose such risks for being misused for malicious intents.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: For all existing works mentioned in our paper, whether for illustrative purposes or as baselines for performance comparison, we have provided proper citations and references to acknowledge their contributions.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: For the purpose of evaluating our methods, we have developed new code and included it in the supplementary materials when submitting our work to OpenReview. This work does not introduce any new assets such as datasets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We do not include such research or experiments in our work.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: LLM is not an important, original, or non-standard component of the core methods in this research.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Appendix: Table of Content

A More Related Works

Graph Neural Networks. Earlier research on deep graph learning extends convolution to handle graphs [7] but comes with notable parameter counts. To this end, GCN [22] simplifies graph convolution by utilizing a 1-order Chebyshev filter to capture local neighborhood information. Moreover, GAT [39] adopts graph attention, allowing weighted aggregation. GraphSAGE [15] introduces a variety of learnable aggregation functions for performing message aggregation. Moreover, GIN [54] aims to preserve structural information maximally and theoretically proves its discriminative power matches the Weisfeiler-Lehman graph isomorphism test. Further details on GNN research can be found in surveys [48, 64].

Federated Graph Learning. Motivated by the success of federated learning in computer vision and natural language processing [55] and the demand for distributed graph learning, FGL has gained increasing attention. From the data and task perspectives, FGL studies are categorized into three settings: (1) Graph-level FGL , where each client collects multiple graphs for graph-level tasks, like graph classification. The main challenge is avoiding interference between clients' graph datasets, especially in multi-domain settings. For example, GCFL+ [53] introduces a GNN gradient patternaware technique for dynamic client clustering to reduce conflicts from structural and feature heterogeneity. (2) Subgraph-level FGL , where each client holds a subgraph of a global graph for node-level tasks like node classification. The key challenges are subgraph heterogeneity and missing edges . Fed-PUB [3] addresses heterogeneity by enhancing local GNNs with random graph embeddings and personalized sparse masks for selective aggregation. FedGTA [26] encodes topology into smoothing confidence and graph moments to improve model aggregation. Other studies [25, 20, 40, 66] also achieve strong results on this challenge. To address missing edges, FedSage+ [62] integrates node representations, topology, and labels across subgraphs, training a neighbor generator to restore missing links and achieve robust subgraph-FL. Other works [8, 57, 61] also excel in this area. (3) Node-level FGL , where each client collects one or multiple ego-networks for node- and edge-level tasks. From the perspective of data format and task, Node-level FGL can be seen as a special case of Subgraph-level FGL. Notably, the application scenarios of Node-level FGL usually involve strict privacy constraints, and representative methods include FedEgo [63]. Detailed insights into FGL research are available in surveys [14, 60, 13] and benchmark studies [16, 45, 27].

Language-Oriented GFMs [28, 65, 17, 23]. These approaches transform graph structures into linearized textual sequences by encoding nodes and edges using syntactically structured templates. The resulting representations can then be processed by token-based encoderstypically LLMsthat are pre-trained on vast corpora of natural language. This approach allows for seamless integration with existing LLM infrastructure and leverages the powerful contextual understanding capabilities developed through natural language processing (NLP). In more detail, during the pre-training phase, these models optimize the parameters of the embedding functionoften realized as an LLMthrough conventional NLP objectives such as next-token prediction or masked language modeling. These objectives encourage the model to learn coherent semantic representations from the flattened graph text, effectively transferring linguistic inductive biases to graph representation learning. However, despite their ability to inherit the expressive power of LLMs, language-oriented GFMs face intrinsic limitations. The transformation from graph to text inevitably introduces information loss, especially concerning structural properties such as node connectivity and subgraph patterns. Moreover, this flattening process may distort the original graph topology in ways that are not easily reversible, thereby affecting downstream tasks that rely on accurate structural reasoning. Additionally, scalability becomes a concern due to the growing length of textual sequences with increasing graph size, which may lead to inefficiencies in both computation and memory usage.

Graph-Oriented GFMs [50, 59, 31, 9, 52, 29, 44, 42, 43, 46, 67, 35, 58]. These approaches aim to preserve both the semantic richness of textual attributes and the integrity of graph topology through purpose-built architectures. These models typically adopt a hybrid design, wherein a frozen LLM is used to extract high-quality textual embeddings from node and edge features, while a trainable GNN component handles the aggregation and propagation of information across the graph structure. This dual-component architecture enables the model to benefit from the strong language understanding capabilities of LLMs without compromising the fidelity of graph structure. The GNN backbone ensures that topological relationships are explicitly modeled, allowing for effective message passing and relational reasoning. During the pre-training stage, graph-oriented GFMs often incorporate selfsupervised learning strategies, such as graph reconstruction or contrastive learning objectives, which

help the model capture invariant and transferable representations across diverse domains. These tasks encourage the model to learn a unified representation space where both textual and structural semantics are coherently aligned, leading to better generalization on downstream tasks involving heterogeneous graph data. By preserving the native structure of graphs and leveraging the representational power of modern neural architectures, graph-oriented GFMs offer a promising direction toward building robust and scalable foundation models for graph-centric machine learning.

B Theoretical Proof

In this section, we provide theoretical analysis for the distinguishability of domain prototypes under random initialization (Theorem. B.1) and the semantic separability of gVQ-VAE codebooks initialized via AncDAI (Theorem. B.2).

Theorem B.1 (Domain Prototype Distinguishability) . Let G a = ( V a , E a ) and G b = ( V b , E b ) denote local graphs from two clients belonging to different domains, with node features X a , X b ∈ R n × d and adjacency matrices A a , A b ∈ R n × n . Let f glb θ be the parameters of a randomly initialized L -layer global GNN-Encoder, which is broadcast to all clients for local initialization. The domain prototype is computed with Eqs. (3) and (4):

Then, there exists a constant α > 0 , whose value depends on the architecture and depth L of GNNEncoder), such that:

Proof. Let z a i = f glb θ ( A a , X a ) i and z b i = f glb θ ( A b , X b ) i denote the representations of node i obtained from a frozen GNN applied to graphs a and b , respectively. Here, the GNN parameters θ are randomly initialized and held fixed. Under this setting, the GNN's computations can be interpreted as performing random but deterministic linear transformations and message passing operations. Leveraging the linearity of expectation and the independence of random initialization, the expected squared Euclidean distance between the resulting node prototypes can be expressed as:

Theorem B.2 (Semantic Separability of AncDAI-Initialized Codebook) . Let { p k } K k =1 be the set of domain prototypes uploaded from K clients. For each prototype p k , we generate a set of perturbed vectors via Eq. 5:

̸

Let C perturb and C rand be codebooks constructed respectively from perturbed prototypes and from standard Gaussian initialization. Then for any two domains a = b and respective node embeddings z a , z b (drawn from f θ ( A , X ) ), we have:

̸

Proof. We adopt a quantization function based on cosine similarity:

̸

which assigns each embedding to the codebook vector with the smallest angular distance.

Assume that the domain prototypes { p k } K k =1 satisfy a minimal angular separation:

̸

The perturbed codebook C perturb is formed by adding isotropic Gaussian noise σ ϵ to each prototype, with ϵ ∼ N ( 0 , 1 ) . For sufficiently small σ , the perturbations preserve the cluster structure, yielding distinct codebook clusters separated by angles close to δ .

Node embeddings z a and z b sampled from different domains concentrate in neighborhoods around their respective prototypes. Formally, with high probability,

for some small ϵ > 0 . Then. by the triangle inequality on the unit sphere,

which implies that embeddings from distinct domains remain well-separated.

Therefore, the probability that z a and z b are assigned to the same codeword under C perturb is bounded above by the probability that perturbations cause cluster overlap, which is small for sufficiently small σ . In contrast, a random codebook C rand sampled isotropically from a standard Gaussian lacks such separation, and embeddings from different domains have a higher probability of being assigned the same codeword. Thus, we combines these observations and proof that:

̸

C More Detailed Experimental Setup

C.1 Dataset

We utilize 8 datasets from various domains and tasks, as detailed in Table 4.

C.2 Data Processing

Our data processing process can be illustrated as Fig. 5, consisting of two steps: Step 1: Language Encoding. We use Sentence-Bert [34] to uniformly encode text attribute graph datasets in different fields to uniformly convert node and edge text into 768-dimensional vectorized representations; and Step 2: Data Decentralization Simulation. Real-world graph data is inherently collected by multiple institutions, resulting in naturally decentralized data distributions. Prior studies in FGL categorize such decentralization into three canonical levels [16]: (1) node-level, where each client maintains ego-networks extracted from a global graph; (2) subgraph-level, where each client collects

̸

-

-

Stepglyph[suppress]1

: glyph[suppress]

Languageglyph[suppress]E ncoding

Stepglyph[suppress]2

: glyph[suppress]

Dataglyph[suppress]De centralizationglyph[suppress]Simulat ion

S ingleglyph[suppress]G rap h

(Node/E dgeglyph[suppress]

CL Sglyph[suppress]Dataset)

Multipleglyph[suppress]Grap h s

(Graph

CL Sglyph[suppress]Dataset)

Louvain

Split

Uniform

Randomglyph[suppress]S p lit

Clie ntglyph[suppress]1

xglyph[suppress]N

xglyph[suppress]3glyph[suppress]N

Node/Edge

Text

Description

Sentence

Clie ntglyph[suppress]2

Clie ntglyph[suppress]3

Clie ntglyph[suppress]1

Clie ntglyph[suppress]2

Clie ntglyph[suppress]3

xglyph[suppress]N

xglyph[suppress]N

Node/Edge

Attribute

Anyglyph[suppress]S p ecifglyph[suppress] G r a p h glyph[suppress] Dataset

(Globalglyph[suppress]Da tasetglyph[suppress]w /o glyph[suppress] S plit

)

a local subgraph induced from a broader graph topology; and (3) graph-level, where each client independently gathers a set of graphs from a larger collection. Notably, the node-level setting can be regarded as a special case of subgraph-level decentralization. Hence, we focus on the latter two in this work. Specifically, under the subgraph-level setting, the implicit global graph G = ( V , E ) has multiple substructures independently collected by different clients. The k -th client locally collects a subgraph G k = ( V k , E k ) such that V k ⊊ V and E k ⊊ E ; Under the graph-level setting, the k -th client independently collects a subset of graphs S k from an implicit broader collection S = { G i } i M =1 , i.e., S k ⊊ S . To simulate these decentralized scenarios in our experiments, we adopt two partitioning strategies: the Louvain algorithm [5] for simulating subgraph-level decentralization, and random allocation for graph-level decentralization, both of which is widely used in various FGL studies [27].

Finally, the default train/validation/test splits used in the fine-tuning stage are summarized in Table. 5. Notably, due to the distributed nature of federated settings, the training set proportion is typically much higher than in centralized graph learning paradigms. This splitting strategy has been widely adopted in prior works [27].

C.3 Baselines

Since this paper is the first to explore FedGFM, we transfer baselines from adjacent fields. Specifically, in our experiments, we evaluate 20 baselines, which can be summarized into 3 categories. The detailed descriptions of these baselines are as follows:

Isolated Supervised Learning. These methods train individual supervised models on each client without federated communication. They serve as a reference for evaluating negative transfer and the benefits of federated learning. The models in this category include a linear layer, GCN [22], GAT [39], GraphSAGE [15], and GIN [54];

GCN [22] is a classical model in graph neural networks, which captures graph structure through spectral convolutions based on the normalized graph Laplacian. By aggregating information from neighboring nodes, it enables efficient node classification and handles graph data in a computationally effective manner. The use of the Laplacian matrix simplifies the convolution operation, making it a foundational approach in graph representation learning.

-

GAT [39] draws inspiration from the success of attention mechanisms in natural language processing, introducing a novel graph attention mechanism that allows nodes to dynamically focus on the most relevant neighbors. This attention-based aggregation enables more adaptive learning.

GraphSAGE [15] extends graph neural networks by introducing a sampling-based message passing mechanism, which allows for scalable neighborhood aggregation. This approach is particularly wellsuited for inductive learning, as it can efficiently generalize to unseen nodes by sampling a fixed-size neighborhood during training. The use of different learnable aggregation functions further enhances scalability, enabling the model to handle large graphs effectively.

GIN [54] is designed to preserve graph structural information and has been shown to be as expressive as the Weisfeiler-Lehman graph isomorphism test in distinguishing graph structures. Notably, GIN is usually more suitable for graph-level tasks.

(2) FGL Approaches. We evaluate various representatives FL/FGL baselines, including two FL methods desinged for FL with vision tasks (FedAvg [32], MOON [24]), and subgraph-level FGL techniques (FedSage+ [62], Fed-PUB [3], FedGTA [26], FedTAD [66], FGSLL [20], FGGP [40]) and graph-level FGL methods (GCFL [53] and FedStar [36]). The detailed descriptions of these baselines are as follows:

FedAvg [32] is a simple yet effective method in FL for the vision and language field, enabling decentralized model training while preserving data privacy. A central server distributes the global model to clients for local updates. The server then aggregates the clients' local models to form a new global model, which is broadcast to all clients to update their local models in the next round.

MOON [24] is a representative FL method originally developed for the vision domain. It leverages contrastive learning at the model level to align local and global representations, thereby mitigating performance degradation caused by data heterogeneity across clients.

FedSage+ [62] integrates node features, link structures, and labels using a GraphSAGE [15] model with FedAvg [32] for FGL over local subgraphs (i.e., subgraph-level FGL). It also introduces a neighbor generator to handle cross-client missing links, improving robustness and ensuring a more comprehensive graph representation.

Fed-PUB [3] is a personalized subgraph-level FGL framework that improves local GNNs without relying on a global model. It measures inter-client similarity using functional embeddings derived from random graph inputs, enabling weighted aggregation at the server. A client-specific sparse mask further guides personalized parameter updates, facilitating subgraph-aware local adaptation.

FedGTA [26] integrates large-scale graph learning into FGL by having clients encode topology and node attributes, compute local smoothing confidence and mixed feature moments, and share them with the server. The server aggregates personalized models using smoothing confidence as aggregation weights.

FedTAD [66] is a subgraph-level FGL method that computes topology-aware node embeddings to estimate class-wise knowledge reliability. This guidance enables the server to perform data-free knowledge distillation, transferring reliable knowledge from local clients to the global model.

FGSSL [20] is a subgraph-level FGL technique, which addresses client drift by aligning node-level semantics and preserving graph-level structures. It employs contrastive objectives to align nodes of the same class while separating different classes, and distills global relational knowledge into local models via similarity distributions.

FGGP [40] is a subgraph-level FGL approach, which decomposes the global model into two tiers connected via prototypes. At the classifier level, class prototypes replace traditional classifiers for better discriminability; at the feature level, contrastive learning injects global knowledge into prototypes to enhance generalization.

GCFL+ [53] is a graph-level FGL framework that clusters clients based on GNN gradient patterns to address structural and feature heterogeneity. It further improves stability through gradient sequencebased clustering using dynamic time warping, enhancing both clustering quality and robustness.

FedStar [36] enables graph-level FGL by decoupling structure and feature learning. Clients share domain-invariant structural embeddings via an independent encoder, while learning personalized features locally, reducing feature misalignment and improving transferability.

(3) Federated Variants of Centralized GFM Approaches. These baselines adapt state-of-the-art centralized GFM training strategies to the federated setting. Specifically, we include OFA [29], GFT [43], UniGraph [17], GQT [41], and GraphCLIP [67]. In their original centralized versions, these methods perform pre-training on all available data at a central learning system using selfsupervised objectives. Their federated counterparts distribute this pre-training process across clients. Specifically, for our experiments, in each communication round of the pre-training phase, each local client deploys the corresponding framework based on its own local data and performs 2 epoch optimization. Subsequently, all trainable parameters will be uploaded to the server, and the parameters will be averaged to obtain the global model, which will be broadcast to all clients as the starting point for the next round of local optimization.

OFA ∗ [29] is a representative training paradigm for GFM, aiming to learn generalizable representations over cross-domain and cross-task textual attributed graphs. It first standardizes the description of nodes and edges via carefully designed language model prompts, transforming any textual attributed graph into a unified vectorized representation. Additionally, OFA introduces NODES-OFINTEREST prompts to unify various graph tasks within a single modeling framework.

GFT ∗ [43] treats computation trees derived from message passing as transferable patterns over graphs. Based on this insight, it adopts a gVQ-VAE architecture to map computation trees into discrete codebook representations. Through self-supervised reconstruction on cross-domain graphs during pre-training, it learns a generalizable GFM with strong cross-graph transferability.

UniGraph ∗ [17] is a GFM training framework that encodes heterogeneous graphs, including those without inherent textual features, into unified textual representations to support cross-domain transferability. It adopts a cascaded architecture of language models and GNNs to jointly capture semantic and structural information. UniGraph further introduces a Masked Graph Modeling objective for large-scale self-supervised pre-training and applies graph instruction tuning with LLMs to enhance zero-shot and few-shot generalization.

GQT ∗ [41] introduces a novel graph quantized tokenizer that decouples tokenizer training from Transformer training, leveraging multi-task graph self-supervised learning to produce robust and generalizable graph tokens. By using the residual Vector Quantization technique, GQT learns hierarchical discrete tokens, reducing memory requirements and enhancing generalization.

GraphCLIP ∗ [67] addresses key challenges in text-attributed graphs, including heavy reliance on label information and limited cross-domain transferability. It introduces a self-supervised contrastive pretraining method using graph-summary pairs curated with the help of LLMs. By leveraging invariant learning, GraphCLIP enhances zero-shot transferability and proposes a graph prompt tuning technique for few-shot learning, mitigating catastrophic forgetting.

C.4 Model Architecture

For Isolated Supervised Learning Methods , we adopt a two-layer architecture with 64 hidden units. For FL/FGL Methods , if a method does not specify a custom architecture, we select the backbone based on the downstream task: GraphSAGE is used for node and edge classification, while GIN is employed for graph classification. For Federated Variants of Centralized GFM Methods , we follow the backbone choices reported in the original papers. For FedGFM+ , we employ a gVQ-VAE as the backbone for both client-side local models and the server-side global model. The encoder is a 2-layer GraphSAGE-based graph convolutional network that jointly encodes node and edge features from the input graph G = ( V, E ) . All layersincluding input, hidden, and outputare set to 768 dimensions, matching the Sentence-BERT [34] representations of node and edge attributes. The encoder outputs node embeddings Z ∈ R N × 768 , where N is the number of nodes. These embeddings are then quantized via a multi-head gVQ-VAE codebook using cosine similarity for nearest-neighbor retrieval. The codebook comprises 4 heads, each containing 128 learnable tokens. A shared linear projection is applied to aggregate the multi-head outputs into the final quantized representation. In addition to the backbone network, FedGFM+ also introduces multiple light-weight learnable graph hints for each client. By default, we learn 3 local graph prompts for each client. Finally, for task-specific heads used during GFM fine-tuning, we follow the original design if specified in the corresponding dataset paper. Otherwise, for node classification, we apply a single-layer MLP to predict node labels from node embeddings; for edge classification, we average the embeddings of the two nodes to form the edge representation and apply a single-layer

C

L

Trainingglyph[suppress]/glyph[suppress]Pre

Clie ntglyph[suppress]1

xglyph[suppress]N

Clie ntglyph[suppress]2

Clie ntglyph[suppress]3

Clie ntglyph[suppress]3K

- 2

Clie ntglyph[suppress]3K

- 1

Clie ntglyph[suppress]3K

xglyph[suppress]N

xglyph[suppress]N

1-

th

Globalglyph[suppress]Da taset

(Node/E dgeglyph[suppress]

-•

S glyph[suppress]Task)

th

G lobalglyph[suppress]Da tasetglyph[suppress]

(Graph

S glyph[suppress]Task)

xglyph[suppress]3glyph[suppress]N

.glyph[suppress].glyph[suppress] .

Isolatedglyph[suppress] S u pervisedglyph[suppress]Le arnin g

(Trainingglyph[suppress]with glyph[suppress]S ing leglyph[suppress]Clien t)

FG Lglyph[suppress]Approaches

(Trainingglyph[suppress]with glyph[suppress]3 glyph[suppress]C lien tglyph[suppress] o f glyph[suppress]S pec if glyph[suppress] G lo b a lglyph[suppress] Da ta s et)

FedGFM+glyph[suppress] &glyph[suppress] F ed erated glyph[suppress]Varia ntsglyph[suppress]of glyph[suppress] C entra lized glyph[suppress]G F Mglyph[suppress]Ap proa ches

(Trainingglyph[suppress]ac ros glyph[suppress]Allglyph[suppress]Kglyph[suppress] G lob a lglyph[suppress] Da ta s ets )

Clientglyph[suppress]1

Model

Clie ntglyph[suppress]2

Model

Clie ntglyph[suppress]3

Model

glyph[suppress]

FedGFM+

glyph[suppress]Federatedglyph[suppress] Variantsglyph[suppress]o fglyph[suppress]Ce ntraliz edglyph[suppress]G FM glyph[suppress]App r o ac hesglyph[suppress]

Yes

Graph

Foundation

Model

Fine

Tuning

(Isolatedglyph[suppress]Su pervisedglyph[suppress] L earnin g

glyph[suppress]

glyph[suppress] FG Lglyph[suppress]Approaches)

Evaluationglyph[suppress]Re s u ltglyph[suppress]fo r glyph[suppress]

1-

thglyph[suppress]

Globalglyph[suppress]Da taset

Loc alglyph[suppress] Evaluation

Localglyph[suppress]Evaluation

Loc alglyph[suppress] Evaluation

-

trainingglyph[suppress]S tage

/

Evaluationglyph[suppress]/glyph[suppress] Fine

-

N

o

/

Tuningglyph[suppress]+glyph[suppress]Eval uationglyph[suppress]Stage

MLP; for multi-task graph classification, we perform mean pooling over node embeddings to obtain a graph-level representation, which is fed into a MLP to predict binary labels for each tasks.

C.5 Training and Evaluation Illustration

We illustrate the training and evaluation processes for all baselines and FedGFM+ in Fig. 6, with detailed descriptions as follows:

Training / Pretraining Stage. For the Isolated Supervised Learning Baselines , each client trains a model independently from scratch, using only its own local graph(s), without any collaboration or information exchange. For the FL/FGL Baselines , we run a FL/FGL algorithm among every 3 clients from the same global dataset. For example, the Cora dataset is split using the Louvain algorithm into clients 1, 2, and 3, and subgraph-level FGL algorithms such as FedGTA are then applied among these clients. Notably, as mentioned in Sec. 1, due to the heterogeneity of data and tasks, most FL/FGL algorithms can only be simulated among different shards of the same dataset. Moreover, existing FGL algorithms cannot be applied simultaneously to the three tasks of node classification, edge classification, and graph classification. For FedGFM+ and Federated Variants of Centralized GFM Baselines , all clients participate in federated pre-training together, which enables extensive collaboration among graph datasets from multiple fields.

Evaluation / Fine-Tuning + Evaluation Stage. For each dataset, we evaluate the performance on the test sets of the three clients associated with it, and report the mean and variance of the resulting metrics. For node and edge classification tasks, we use Accuracy (ACC) as the evaluation metric, while for graph classification tasks, we adopt the Area Under the Receiver Operating Characteristic Curve (AUC-ROC). To assess the performance of each individual client under different settings, we follow distinct evaluation protocols. For Isolated Supervised Learning baselines, we directly evaluate each clients local model without any collaboration. For FL and FGL baselines, we evaluate each clients model after training the global model for two communication rounds. For FedGFM+ and the federated variants of centralized GFM baselines, we first attach a task-specific header and then fine-tune the model using each clients local graph before evaluation.

C.6 Hyperparameters

For Isolated Supervised Learning Baselines , we perform 1,000 epochs of local training with early stopping based on validation performance. For FL/FGL Baselines , we conduct 100 communication rounds, where each round includes 2 local training epochs. We use the Adam optimizer with a learning rate of 1 × 10 -2 , weight decay of 5 × 10 -4 , and dropout rate of 0 . 5 . For federated variants of centralized GFM Baselines , we adopt the hyperparameter configurations reported in their original papers whenever available. When unspecified, we employ automated hyperparameter optimization using the Optuna framework [2]. Federated pre-training is carried out for 50 communication rounds, each consisting of 2 local pre-training epochs. For our proposed FedGFM+ framework, we fix the learning rate for pre-training to 1 × 10 -4 . During fine-tuning, we perform a grid search over learning rates in { 10 -5 , 10 -4 , 10 -3 , 10 -2 , 10 -1 } for each dataset. The weight decay is fixed to 5 × 10 -4 , and the batch size is set to 1,024. Federated pre-training is conducted for 25 communication rounds, with 2 local epochs per round.

K

-

-

C

L

I

s

-

?

C.7 Experimental Environment

The experimental machine is an Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz and NVIDIA A100 with 80GB memory and CUDA 12.4. The operating system is Ubuntu 22.04.5 with 251GB memory.

D Few-shot Learning Results

We perform a few-shot evaluation across a range of downstream tasks. Specifically, for node and edge classification tasks, we constrain the number of labeled samples per class to at most 2. For graph classification tasks, however, we do not report few-shot performance, as each graph instance is associated with multi-dimensional labels, making few-shot evaluation non-trivial. The experimental results are summarized as Table. 6

As observed, FedGFM+ consistently outperforms naive federated adaptations of centralized GFM training strategies across all evaluated settings. By integrating the AncDAI and AdaDPP modules, FedGFM+ effectively constructs domain-aware semantic priors that enhance generalization to downstream tasks in heterogeneous domains, even with limited fine-tuning labels. Despite these gains, it is important to note that FedGFM+ still falls short of its own performance under scenarios with abundant labeled data.

Peer reviews:
comments: SUMMARY:
This paper is trying to solve question containing federated graph learning and graph foundation models. To address the knowledge entanglement challenge, FedGFM+ is proposed to solve it from the global and the local perspective. The authors conduct cross-task and cross-domain experiments to demonstrate the effectiveness of the approach.

STRENGTHS AND WEAKNESSES:
Strengths:

(1) Figure 1 is a good diagram to demonstrate the comparison.

(2) They conduct extensive experiments to verify their framework.

Weakness:

(1) The motivation is not clear. The combination of two terms seems not convincing as the motivation.

(2) The authors try hard to include too many aspects in one paper, which confuses the reader a lot. I understand that the data heterogeneity, task heterogeneity, multi-domain data isolation, cross-silo storage and computation neglect are the limitations for the two terms respectively. But the statements from Line 64 to 78 are far-fetched and strained.

(3) This framework contains multiple modules. But most of them are not novel if we treat them individually, for example, the prototype and personalized prompt.

QUESTIONS:
1, Could you please clarify the motivation of the proposed approach? From the methodology to the applications.

2, Could you please identify which individual technique is innovatively proposed in this paper?

3, Although I read that you categorize the baselines roughly, could you please clarify a little bit more about how you select the baselines?

LIMITATIONS:
1, I suggest the authors to think deeper about the KEY motivation of the proposed approach and present it more clearly.

2, Also, one or two focused points deserves to be explored comprehensively.
TITLE: Official Review
CLARITY: 2
IMPACT: 1
ORIGINALITY: 2
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper delves into the Federated Graph Foundation Model (FedGFM), which seamlessly integrates two areas within graph machine learning: Federated Graph Learning (FGL) and Graph Foundation Models (GFM). The paper offers a practical overview of the challenges associated with both FGL and GFM in real-world applications, while emphasizing their complementary strengths. FedGFM is introduced as a novel framework for building graph foundation models by harnessing distributed datasets from multiple parties and domains, along with decentralized computational resources. A central issue identified in this context is knowledge entanglement, where global models struggle to separate domain-specific insights into distinct representations. To tackle this, the authors propose FedGFM+, which incorporates the AncDAI and AdaDPP modules. Comprehensive experiments across various datasets validate the practicality and performance of FedGFM+.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This work presents FedGFM, a novel framework combining Federated Graph Learning (FGL) with Graph Foundation Models (GFM). It leverages multi-domain data and distributed resources, marking the first study in this area.\n\n2. FedGFM+ builds on FedGFM, addressing communication challenges and knowledge entanglement with robust empirical methods.\n\n3. Experiments on eight datasets across tasks validate its effectiveness, with baselines including GNNs, FGL variants, and federated GFM adaptations.\n\nWeakness:\n\n1. In the context of VQ-VAE or Graph-VQ-VAE, the concept of vocabulary collapse [1, 2] refers to a situation where only a small subset of tokens in the codebook is frequently accessed, while many are rarely or never used. How does this differ from the knowledge entanglement issue discussed in this paper? Please elaborate on the distinction.\n\n2.  What is the reason for structuring the FedGFM paradigm’s pre-training phase as iterative rounds of federated communication, while the fine-tuning phase is carried out independently by each client?\n\n3. Provide further explanation regarding the fairness of comparisons made between the FGL baselines and FedGFM+.\n\n4. Could you clarify the meaning and implications of Figure 2 in more detail?\n\n[1] Zheng, C., & Vedaldi, A. (2023). Online clustered codebook. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 22798-22807).\n[2] Wang, Z., Zhang, Z., Chawla, N., Zhang, C., & Ye, Y. (2024). Gft: Graph foundation model with transferable tree vocabulary. Advances in Neural Information Processing Systems, 37, 107403-107443.\n\nQUESTIONS:\nSee weakness\n\nLIMITATIONS:\nSee weakness", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates Federated Graph Foundation Models (FedGFM) and proposes a unified training framework, FedGFM+, that aims to combine the complementary advantages of Federated Graph Learning (FGL) and Graph Foundation Models (GFM) and mitigate their respective limitations. The authors identify and discuss key limitations in both areas and introduce two novel components (AncDAI and AdaDPP) to mitigate the issue of knowledge entanglement across domains. The experimental evaluation covers eight graph datasets, multiple downstream tasks, and diverse baselines, demonstrating the consistent superiority of FedGFM+.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n- This paper presents a novel and practically relevant study grounded in real-world scenarios about Federated Graph Foundation Models (FedGFM), proposing a training paradigm that unifies decentralized graph data sources with the goal of learning a generalizable graph foundation model.\n- The paper provides a diagnosis of core limitations in existing Federated Graph Learning (FGL) and Graph Foundation Model (GFM) approaches. Based on these insights, the authors further proposed the FedGFM+ framework, which organically combines FGL and GFM and complementarily addresses the limitations of both.\n- The authors provide an empirical analysis that evaluates FedGFM from the perspective of feasibility, effectiveness, and further experimentally reveal the non-trivial challenge faced by simple federated variants of the naive centralized GFM training strategy, namely, knowledge entanglement.\n- The experiments are comprehensive, and the performance is promising.\n- Overall, the manuscript is well-structured with well-designed illustration figures and tables, and clear technical explanations.\n- The authors have provided the source code of this paper.\n\n### Weaknesses:\n- It would be beneficial for the authors to further clarify the workings of the proposed FedGFM+ framework by providing detailed algorithmic pseudo code. A more formal description would aid readers who are less familiar with FGL or GFM in fully understanding the methodology.\n- In the empirical analysis section, the experiment on the knowledge entanglement problem measured the encoding differences of a simple federated variant of the centralized GFM training strategy GFT for graph datasets in different domains; I suggest that the authors can supplement the encoding results of FedGFM+ to better illustrate that FedGFM+ can indeed alleviate the knowledge entanglement problem.\n- The author divides different datasets into 3 clients. I suggest that the author further explore the performance of tasks divided into other numbers of clients. For example, does dividing Cora into 5 clients affect the performance of the learned graph-based model?\n\nQUESTIONS:\nPlease refer to the weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces FedGFM, a federated learning framework for training graph foundation models (GFMs). A key challenge identified is knowledge entanglement, where representations from different domains become indistinguishable, limiting downstream performance. To address this, the authors propose FedGFM+, which extends FedGFM with two modules: (1) AncDAI, a global anchor-based domain-aware initialization that encodes domain-specific prototypes before pretraining to ensure separable representations across domains; and (2) AdaDPP, an adaptive domain-sensitive prompt pool used during fine-tuning to enhance domain-specific adaptation via learned prompts from each client. The authors support their method with theoretical analysis and demonstrate improved performance across multiple datasets compared to strong baselines, including domain-specific and federated models.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**  \n\t1.\tThe idea of applying federated learning to train graph foundation models is important and addresses a fundamental need in graph machine learning.  \n\t2.\tThe paper identifies a key challenge in federated graph foundation models—knowledge entanglement—and supports this with a clear and relevant experimental analysis.  \n\t3.\tThe proposed solutions are backed by both theoretical analysis and strong empirical results.  \n\t4.\tThe overall structure and flow of the paper are logical and easy to follow.  \n\n**Weaknesses:**  \n\t1.\tAlthough Theorem B.1 shows that prototypes are distinguishable even under random initialization, the paper does not provide any quantitative estimation of how separated they are. Including such an estimate, perhaps with a simple example, would help illustrate this point more clearly.  \n\t2.\tA similar concern applies to Theorem B.2. While the theorem suggests that the codebook is more likely to assign distinct tokens to different domains, it’s unclear how likely this is in practice. Quantitative or empirical evidence would strengthen the argument.  \n\t3.\tTheorem B.2 is described as general, but the proof only focuses on cosine similarity. It would be helpful to clarify whether the conclusions hold under other similarity metrics.  \n\t4.\tSince all domain prototypes and prompts (from Sections 4.1 and 4.2) are shared with the server, there should at least be a discussion of the potential privacy risks associated with this sharing.  \n\t5.\tBased on Equation (7), all prompts appear to have the same dimensionality as the node features, which implies that all node features across graphs must have the same dimension. This is a strong assumption and could limit the applicability of the method to heterogeneous graphs.  \n\t6.\tBecause each prompt has the same dimension as the node features and requires its own projection weights, the resulting objective function becomes high-dimensional and potentially expensive to optimize. This can increase both computation and communication costs, especially in a decentralized setting. More justification and analysis of this overhead would be beneficial.  \n\t7.\tWhile the experimental results are thorough and span multiple tasks and domains, the paper does not report any comparison of computation or communication costs.  \n\t8.\tIntroducing synthetic prototypes to the codebook increases its size, thereby enlarging the search space for the optimization in Equation (1). This added complexity should be discussed.\n\nQUESTIONS:\n1.\tHow is the model trained to ensure well-separated prototypes across domains?  \n2.\tCan you provide a pseudo-code or algorithm sketch for FedGFM+?  \n3.\tIs it possible to evaluate or estimate the privacy leakage of the method?  \n4.\tHow does the model perform on domains with limited or imbalanced data?  \n5.\tWhat happens to performance if the data is randomly partitioned or if the number of clients increases?\n\nLIMITATIONS:\n1.\tThe scalability of the method with respect to the number of domains is not discussed.  \n2.\tThe potential privacy leakage due to sharing prototypes and prompts is not analyzed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}