Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement
Authors: Yinlin Zhu
Xunkai Li
Jishuo Jia
Miao Hu
Di Wu
Meikang Qiu
Venue: NeurIPS 2025 poster ()
Keywords: Graph Machine Learning
Federated Graph Learning
Graph Foundation Model
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Recent advances in graph machine learning have shifted to data-centric paradigms, driven by two emerging research fields: 
    (1) Federated graph learning (FGL) facilitates multi-client collaboration but struggles with data and task heterogeneity, resulting in limited practicality;
    (2) Graph foundation model (GFM) enables desirable domain generalization but is typically confined to single-machine training, neglecting the potential of cross-silo data and computational resources.
    It is evident that these two paradigms are complementary, and their integration offers substantial advantages.
    Motivated by this, we present a pioneering study about the federated graph foundation model (FedGFM), a novel decentralized GFM training paradigm. Despite the promising vision of FedGFM, knowledge entanglement has emerged as a critical challenge, where multi-domain knowledge is encoded into indistinguishable representations, thereby limiting downstream adaptation.

To this end, we propose FedGFM+, an effective FedGFM framework with two key modules to mitigate knowledge entanglement in a dual-pronged manner. 
(1) AncDAI: From a global perspective, we introduce a novel anchor-based domain-aware initialization strategy. Before pre-training, each client encodes its local graph into a domain-specific prototypes, which serve as semantic anchors in the representation space. Around each anchor, we construct synthetic embeddings to initialize the global model. We theoretically show that these prototypes are distinguishable across domains, and the initialization provides a strong inductive bias that facilitates disentanglement of domain-specific knowledge. 
(2) AdaDPP: From a local perspective, during pre-training, each client independently learns a lightweight graph prompt that captures domain semantic preferences. During fine-tuning, prompts from all clients are aggregated into an adaptive domain-sensitive prompt pool, from which the GFM selects relevant prompts to augment the target graph’s attributes, thereby improving the downstream adaptation. FedGFM+ is extensively evaluated on 8 diverse benchmarks spanning multiple domains and tasks, outperforming 20 baselines from isolated supervised learning, FGL, and federated variants of centralized GFM paradigms.

Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglement
Yinlin Zhu' Xunkai Jishuo Jia², Miao Hu', Di Wu Meikang Qiu
Sun Yat-sen University, Guangzhou, China
2 Beijing Institute of Technology, Beijing, China Shandong University, Weihai, China Augusta University, Augusta, Georgia, USA
zhuylin27@mai122 12. sysu.edu..uuuuuu...uuuuuuuuuuu edu cn, cs. xunkai 11 1i@gmail.....m.m com, 1ashooia44700mmall. com humiao5@mail. sysu. edu. en, wud127 1 sysu. edu. en, glumeikan@@yahoo.comm
Abstract
Recent advances in graph machine learning have shifted data paradigms, driven by two emerging research fields: (1) Federated graph learning (FGL) fa- cilitates multi-client collaboration but struggles with data and task heterogeneity, resulting in limited practicality; (2) Graph foundation model (GFM) enables de- sirable domain generalization but is typically confined to single-machine training, neglecting the potential of cross-silo data and computational resources. It is ev- ident that these two paradigms are complementary, and their integration offers substantial advantages. Motivated by this, present pioneering study about the federated graph foundation model (FedGFM), a novel decentralized GFM training paradigm. Despite the promising vision of FedGFM, knowledge entanglement has emerged as a critical challenge, where multi-domain knowledge is encoded into indistinguishable representations, thereby limiting downstream adaptation.
To this end, propose FedGFM+, effective FedGFM framework with key modules to mitigate knowledge entanglement in a dual-pronged manner. (1) AncDAI: From global perspective, we introduce novel anchor-based domain- aware initialization strategy. Before pre-training, each client encodes its local graph into a domain-specific prototypes, which serve semantic anchors in the representation space. Around each anchor, we construct synthetic embeddings initialize the global model. We theoretically show that these prototypes are distinguishable across domains, and the initialization provides strong inductive bias that facilitates disentanglement of domain-specifie knowledge. (2) AdaDPP: From local perspective, during pre-training, each client independently learns a lightweight graph prompt that captures domain semantic preferences. During fine- tuning, prompts from all clients aggregated into adaptive prompt pool, from which the GFM selects relevant prompts to augment the target graphs attributes, thereby improving the downstream adaptation. FedGFM+ is extensively evaluated on diverse benchmarks spanning multiple domains and tasks, outperforming 20 baselines from isolated supervised learning, FGL, and federated variants of centralized GFM paradigms.
'Equal contribution. Cooresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(6) Limitationns of FöL Approaches
(c) A Nave FodGFM Paradigm
Date Colleciin
buna Feecastey
Senver
Modil
Sockd
Pratesl Chrhar
Node leve
Greah
Jone
Data Heterogeneity
Task Heterogencity
Limitatiors Approaches
Learnig System
Sto
Clien Exscuricr
LACA
Locel Data Collection
Stondard Processing
Local
Multi-Domoin Data Isolation
Crozs Storage Computation Neglect
Pre- Trannng
Figure I: Comparison of the FGL, GFM, and naive FedGFM paradigm. (a) Limitations of FGL ap- proaches: (b) Limitations of GFM approaches; (c) A naive FedGFM paradigm organically combines the complementary strengths of FGL. and GFM to overcome their respective limitations.
Introduction
Recent advances in computational capabilities have sparked a data-centric paradigm shift in deep learning. Moving beyond an exclusive reliance on architectural innovations, the Al community now prioritizes large-scale data utilization, as evidenced by the success of GPT-4 [M]] in language pro- cessing and Sora [60] in vision tasks. This data-centric scaling trend also extends to graph machine learning, where two learning paradigms are gaining prominence (1) Federated graph learning (FGL) enables cross-silo graph collaboration; (2) Graph foundation models (GFM) promote multi-domain graph generalization. However, both of them face practical deployment limitations.
Two limitations hinder FGL from achieving cross-domain and cross-tass task collaboration, as illustrated in Fig. M (a): (1) Data Heterogeneity. Due to diverse data sources and processing methods, client graphs often differ in feature dimension, label space, and topology pattern. As a result, most FGL methods are confined to collaboration across subsets of single dataset [622, 22,, 20J.. While GCFL+ [53] and FedStar [36] enable limited cross- collaboration via domain- aware client clustering or feature-agnostic parameter sharing, they are only applicable to graph- tasks and lack the ability to capture cross cross-domain general knowledge at the feature level. (2) Task Heterogeneity. Existing FGL assumes uniform graph granularity and downstream tasks across clients, enforcing one of three settings: node-level (ego-networks for node classification/lind prediction), subgraph- level (induced subgraphs from a global graph for node classification/ling prediction), or graph-level (graph sets for classification/rerresse specific designs in both model architectures and training algorithms, which significantly limits their ability to support collaboration across multi-task graph data.
[IG]] As result, existing FGL approaches often adopt task-
Meanwhile, existing GFM studies face the following two limitations, as illustrated in Fig. M (b): (I) Multi-Domain Data Isolation. Training generalizable GFMs requires diverse graph data span- ning multiple domains, like social networks, molecular structures, etc. Although a number of public graph datasets are available, they remain limited in both scale and diversity. In contrast, real-world graph data is expected to continuously grow volume and variety, yet it is often distributed across institutions and isolated in data silos due to privacy regulations or commercial competition. This renders existing centralized GFM approaches increasingly infeasible. (2) Cross-Silo Storage and Computation Neglect. Although current GFMs require significantly fewer storage and computa- tion resources than their NLP or vision counterparts, which makes them feasible within a single institution, centralized training frameworks inherently fail to leverage the vast yet fragmented stor- age and computation capacities distributed across multiple silos in real-world deployments. This under-utilization results in non-trivial opportunity costs, such as redundant resource provisioning and sub-optimal training efficiency.
Fortunately, FGL and GFM exhibit a naturally complementary relationship. Specifically, FGL equips GFM with a decentralized training paradigm that supports learning across distributed silos while efficiently utilizing cross-silo storage and computational resources. In contrast, GFM enhances FGL by offering unified feature encoding and pre-training followed by fine-tuning framework, thereby facilitating generalized collaboration across diverse graph domains and task types. To this end, we introduce Federated Graph Foundation Model (FedGFM), a novel and practical paradigm designed for training GFM over decentralized, cross-domain, and cross-task graphs. As illustrated in

---

Fig. M (c), the FedGFM paradigm follows pipeline that begins with federated pre-training and pro- ceeds with fine-tuning, During the federated pre-training phase, each client performs self-suuprrvsed learning on its private graph to acquire domain-specite representations. The server then aggregates these local models to construct a global model that captures generalizable topologiaal and seman- tic patterns. The global model is subsequently broadcast to clients as the initialization for the next round of federated pre-training. This iterative process continues across multiple rounds of federated communication. In the fine-tuning phase, the global model is treated graph foundation model and is further adapted to specific downstream tasks through supervised learning.
To establish an effective FedGFM framework, our work begins with an empirical investigation (Sec. E), assessing its feasibility and revealing a non-trivial challenge. Specifically, (1) From a feasi- bility perspective, FedGFM faces stringent communication constraints, as frequent transmission of large-scale model parameters or gradients often impractical real-world federated deployments. This limitation calls for a lightweight yet expressive model architecture. Fortunately, the graph vector coantizattio-variatiinnn
auto-encoder (gVQ-VABE which is widely used as the backbone in centralized GFM, presents promising solution. It has been extensively validated for its ability to jointly encode graph structures and text attributes into discrete, semantically meaningful represen- tations [MI] 833, making it well-suited for multi-domain pre-training. Meanwhile, its lightweight design naturally aligns with the naively distributing the pre-training of EVQ-VAE across local clients in a federated setting intro- duces a critical challenge we term knowledge entanglement. Unlike centralized training, federated pre-training operates multiple isolated, fomain--spccifi graphs, each with distinct data distribu- tions. Each client's local trained model tend to overfit their domain-specific data without alignment across clients. Consequently, the aggregated global GFM encodes multi-domain graphs into indis- tinguishable representations and further limits its downstream generalization.
wwwaimiiaatoo-ffiiinnnne requirements of FedGFM. (2) However,
Building upon these insights, we present effective FedGFM framework named FedGFM+, which involves two key modules to mitigate knowledge entanglement cDAI: From global perspective, we introduce a novel anchor-based domain-aware initialization strategy. Before pre- vhe-training each client encodes its local graph into a domain-specifie prototype, which serve as semantic anchors in the representation space. Around each anchor, construct syn- thetic embeddings to initialize the global model. We theoretically show that these domain prototypes distinguishable across domains, and the initialization provides strong inductive bias that natu- rally facilitates encourages separation among knowledge representations from different domains. (2) AdaDPP: From local perspective, during the pre-training stage, each client independently learns and retains a lightweight, domain-sensitive prompt that captures its local semantic preferences, with- out participating federated aggregation. In the fine-tuning stage, these prompts are assembled into an adaptive domain-sensitive prompt pool. For a given target graph, the model selects and incorpo- rates the most relevant prompts from the pool based on its semantic characteristics. These prompts serve as domain-specific priors that condition the GFMs representations, thereby enabling adaptive exploitation of domain knowledge and facilitating improved adaption to downstream tasks.
dual-pronged manner: (1) An-
Our Contributions. (1) Problem Identification. To the best of our knowledge, this is the first exploration of the FedGFM paradigm, which organically combines FGL and GFM offer prac- tical solution for training graph foundation model across silos with diverse graph domain and tasks. (2) In-depth Investigation. (Sec. 2) We conduct an in-depth empirical investigation for FedGFM, assessing its feasibility and revealing a non-trivial challenges named knowledge entanglement, pro- viding valuable insights for its development. (3) Novel Framework. (Sec. H) We propose novel and effective FedGFM framework named FedGFM+, which employs two key modules to address the knowledge entanglement challenge, including AncDAI from the global perspective and AdaDPP from the local perspective. (4) State-of-the-art Performance. (Sec. E Extensive experimental re- sults on graph learning with 8 cross-task and cross-domain datasets demonstrate the superiority of FedGFM+ compared with 20 baselines, including 5 isolated supervised learning methods, 10 FGL techniques, and 5 federated variants of centralized GFM training strategies.
2 Preliminaries and Problem Formalization
Text-Attribute Graph. Consider text-atrributed graph G = (V, E), where V is the set of nodes and & is the set of edges. Each node v; E V and edge e E E may be associated with a textual description, which is encoded into a semantic vector using specific embedding technique (e.g..
3

---

bag-of-words, pre- pre-trained language models). Depending on the downstream task, the graph may be equipped with supervision signals different levels: node-level labels (for node classification), edge- level labels (for edge classification or link prediction), or graph-level labels (for graph classification).
Graph Vector Quantization- Variatioonn Auto-Encoder GFM Backhone. Most recent GFMs adopt gVQ VAQS as the trainable GNN. This backbone enables the joint encoding of topology and textual attributes into a discrete embedding space with clear semantic boundaries, making it partieu- larly suitable for multi-domain GFM pre-training. Specifically, (1) g (V) + Encoder - Embeddings: To ensure generality in arbitrary inputs, the Encoder can be instantiated as any reason- able GNN capable of incorporating both node and edge features to generate informative embeddings E R. (2) Embeddings + Codebook + Quan. Emb.: To establish clear semantic boundaries, the Codebook C transforms continuous embeddings 2 into discrete embeddings E IRd (Quan. Emb.
Bq E TRe) via similarity strieval-based vector quantization:
2q t- Cj. j = arg min ||2 eill2, C (2,,60,,.. ex, c,EC
(1)
(3) Quan. Emb. + Decoder + G (V,), To enable the self seff-supervised training, gVQ- VAEs follow an autoencoder framework, where gradients are computed by the discrepancy between the reconstructed graph Gú and the original input graph G', thereby updating the Encoder and Code- book. Notably, the trainable components of the Encoder and the Codebook are the weighted matrix and the discrete embeddings (e1,...eek) ex). which together constitute the trainable GFM embed- ding function parameterized by fa- Meanwhile, to construct end-to-end gradient flow, the straight- through estimator (STE) [3]] is used to approximate gradients by bypassing the non-diffcrentiable quantization step. Formally, the gVQ-VAE is pre-trained via optimizing loss function follows:
Cpretrain = =feeat + Lropo +
Σ1 sg[zi] - =z.11 1122+
+
23 1=1
sgl sglza.1112.
fjeat n1
Σ
||:a| )",
Liapo =
- -IAA-o (xx 113
|
i=1
where sg[:] represents the stop-gradient operator, denotes the number of nodes, z; represents the i- th node embedding produced by the GNN encoder, 2g. denotes its quantized embedding obtained by retrieving the codebook, and itz denotes the reconstructed node attributes projected via MLP-based decoders, i.e., . 6(2g)). )- y is the scaling factor. More details and related works about gVQ- VVQ-VEE presented in Appendix A.
Problem Formalization of FedGFM. For FedGFM, there is a trusted central server and K clients. The subgraphs or graph collections of the client present a relationship such as subgraph-level decen- tralization or graph-level decentralization (see Appendix. C2 for more details about data settings). To unify the representation, we regard the graph data held by k-tth client as Sk. where [ for subgraph-level decentralization. The proposed FedGFM paradigm follows federated pre-training- fine-tuning process. For the Federated Pre-Traime Pre-Training phase, each client conducts self-supervised training optimize its local model based on its local graph, and the server aggregates multiple local models to obtain a global graph foundation model. Consider adapting the widely-used FedAvg [62] aggregation strategy in federated learning for vision tasks within the FedGFM framework, the fed- erated pre-training process unfolds as follows: (1) Initialization: At the first communication round (r 1). the central server sets the local model parameters of K clients to the global parameters, i.e., # t @ Ak. (2) Local Updates: Each local model performs training on the current local data G to minimize the self- sulf-supervised loss L(G"; @*), and then updating the parameters: O* +- O - V
(3) Global Aggregation: After local training, the server aggregates the local knowledge with respect
N Σ O with N = Σ. Nr. and distributes the global parameters O to local clients selected at the next round. This process iterates between steps 2 and 3 until reaching the final round R. This iterative cycle continues until the completion of the last round (r = R), facilitating collaborative GFM training by parameter sharing without the exchange of private local data. For the Fine- TimeTTuning phase, FedGFM first loads and freezes the pre- trained global model from the central server as GFM, then uses available graph supervision signals to fine-tune the task heads to adapt to specific downstream graph tasks.
to the number of training instances, i.e., O +- N₂

---

3 Empirical Investigation
In this section, we present an in-depth empirical study of the FedGFM paradigm, organized around two key questions from different perspectives. QI: From the perspective of Feasibility, is FedGFM practical for real-world deployment? Q2: From the perspective of Effectiveness, what are the main bottlenecks that limit the effectiveness of a naive FedGFM implementation?
Table 1: Comparison of parameter sizes between graph foundation models and those in the language and vision fields. Parameter counts are shown above each method name. *** indicates an upper bound. Graph, Language and vision models are highlighted in red, yellow and blue, respectively.
16.8M
20M
20M
40M
GnyGlil 22
GFSE !
SwayKI IC
Opentiaph
25M DFA [
10M'
5M* GQT BX
180M Unigraph
GOIA AA
7M GPT [+]
10M
150M
31 31.64M* UniGmap22
RAGmpt [X]
GrapbCLIP aR
7B Llam22 7B [S]]
175B GPT X
540B* "
1B DINOv2 (X
PaLM ICII
To address Q1, survey several repre- sentative foundation models to quantify their parameter scales, and summarize the results in Table M. Notably, compared with foundation models in language and vision domains, graph foundation models (GFMs) are significantly more lightweight in terms of parameter size. This sug- gests that federated pre-training of GFMs is communication-efficin feasible. Among all surveyed GFMs, we further observe that two gVQ VAE-based methods, GFT [HS]] and GQT [MI), exhibit the smallest parameter scales. This high- lights the advantage of the VVQ-VAE ar- chitecture in achieving a lightweight yet expressive design, making particularly suitable for FedGFM settings. More related works about GFM are presented in Appendix [A.
GFF*
Cosine Similarity
Frequency (%)
0.8
04
20
25
Cie-WN:SK Coratitv WNISHI-HIV
(n) Degree Frequency
(b) Representation Similarity
and practically
Figure 2: Empirical analysis on three graph datasets: Cora, WNISRR, and HIV. (a) Comparison of topologi- cal patterns in terms of degree distribution. (b) Average cosine similarity of original node features and node em- beddings encoded by GFT and GFT", respectively.
To address Q2,
we conduct a simple yet illustrative visualization experiment, aiming to reveal the bottlenecks that limit the effectiveness of naive FedGFM. Building on the insight of Q1, we imple- ment naive federated variants of GFT [+ (denoted as GFT"), and evaluate GFT and GFT* on three datasets: Cora [56], WN1 1RR [02]] and HIV [77]] covering different domains (citation networks,
knowledge graphs, and molecular graphs).
The empirical results are presented in Fig. E. Specifically, panel (a) illustrates the node degree distri- butions of the Cora, WNISRR, and HIV datasets (restricted the first 30 degrees starting from 1 for visual clarity), while panel (b) reports the inter-domain cosine similarity among the three datasets, computed in three different representation spaces: (I) the average initial node features, (2) the av- erage node embeddings learned by GFT, and (3) those learned by GFT*. This comparison reveals how well each model distinguishes multi-domain knowledge during representation learning. As observed, the three datasets differ markedly in both topological structure and initial feature distribu- tions. Despite such heterogeneity, centralized GFT pretraining produces graph foundation model that generates embeddings with clear domain-spectie tion of inter-domain variability through joint optimization. In contrast, the embeddings learned by GFT under decentralized federated pretraining show near-unity inter-domain similarity, reflecting a collapse of domain specificity caused by the absence of coordinated global optimization. We term this the knowledge entanglement, a non-trivial challenge to resolve for effective FedGFM design.
distinctions. This indicates effective preserva-
5

---

(o) Federated Tre-Traning Stags of FedoFW
(b) Fire Turing Stage FiddFW-
Enbeddag
Upicdd to
Dawtioaa Fran
Zen.
Enceppiee
Clert
Vestor Singls
Node Erbeeiiigs
Concir Proheppa
AGD
Sated Duma www.icaatioonn....ooo.o (AR
Codkbbok
dieen
MA
Local
Prompts
Grogges
Smoclent
DPG
Prozue
Grogh
Madio
Ind
Ruciratuccion
braeleed
Tranehh
Quary
Phe
Sncodes
Enput
Aftr Butee
Graph
(Sraggh Thruash
Goudinn Baskwoodd
Quertixd Brbedd kg
Tepolog El
Figure 3: Overview of the proposed FedGFM+ framework.
4 Methods
In this section, we introduce the proposed FedGFM+ framework. We first provide overview of FedGFM+ in Fig. BL At core, FedGFM+ adopts a federated pre-training and fine-tuning paradigm. During each communication round of pre-training, clients leverage local gVQ-VAE encoder to perform self-supervised graph reconstruction, capturing domain-specific semantics. The resulting local models are uploaded the server for aggregation, yielding an updated global model. The global model is subsequently broadcast to clients as the initialization for the next round of federated pre-training. In the tine-tuning stage, this global model serves as general-purpont GFM encoder, while a task-specific prediction head is optimized for downstream tasks. Moreover, FedGFM+ in- troduces two key modules to mitigate the knowledge entanglement challenges: (1) AneDAI: Before pre-training, FedGFM+ employs a novel anchor-based domain-aware initialization strategy to ini- tialize the global codebook, providing a strong inductive bias that facilitates disentanglement of domain-specific knowledge. (2) AdaDPP: During pre-training, each client independently learns lightweight graph prompt that imbues the GFM with its own domain semantic preferences. Dur- ing fine-tuning, prompts from all clients are aggregated into an adaptive domain-sensitive prompt pool, from which the GFM selects relevant prompts to augment the target graph attributes, thereby improving the downstream adaptation. Below we introduce these two modules in detail.
4.1 Anchor-Based Domain-Aware Initialization
As discussed in Section EL naive FedGFM suffers from knowledge entanglement, where represen- tations from different domains collapse into indistinguishable embeddings. To mitigate this, from a global perspective, we aim to endow the global model with a strong inductive bias that explicitly encourages the separation of domain-specific semantics.
Before federated pre-training, to capture domain-specific knowledge, we introduce a domain proto- type extraction mechanism, which models intrinsic patterns in the graph topology and node attributes of the local graph and summarizes them into a compact, mified-dimensionn vector representation. Specifically, for the k-th client with a local graph G = (V*, 1)*,E*). node features X= & and adjancency
matrix A R,, we first compute the node embeddings z as follows:
z = foom (x* A+
(3)
where Balb denotes the initialized global model parameter broadcast to all clients. The domain pro- totype d is then obtained by mean-pooling over node embeddings:
p k
Σ
z
D
PEVE
We theoretically demonstrate that, even under a randomly initialized model with shared parameters, the domain prototypesobtained via averaging the encoded node representationsremain distinguish- able across clients. This separability stems from intrinsic discrepancies in node features and graph topologies among domains, and can be formally bounded (Appendix [ Theorem. B.
Each client subsequently uploads its prototype to the central server. To steer the global model toward learning domain-aware representations, treat these prototypes as semantic anchors and

---

synthesize local neighborhoods in the embedding space via controlled perturbations. Specifically, for each anchor p". set of perturbed embeddings {P. fi=1
ff is generated as:
d =p + ocis C ~N,11.
i=
(5)
where e; is sampled from standard Gaussian distribution, and is noise scaling factor that ensures numerical stability. Notably, the number of synthetic embeddings H is uniformly allocated across prototypes, depending on the number of the learnable codebook tokens in the global model.
Finally, the synthetic embeddings aggregated from all domains are used to initialize the codebook C of the global model, i.e., C Init(Ux: [ ( We further provide theoretical analysis (Ap- pendix B Theorem. (+2) to demonstrate that this initialization introduces a structured inductive bias, which not only facilitates disentangled representation learning across diverse domains but also sta- bilizes optimization during the early stages of federated pretraining.
4.2 Adaptive Domain-Sensitive Prompt Pool
Moreover, address knowledge entanglement from the local perspective, introduce novel prompt learning- mechanism. During the pre-training stage, each client independently learns and retains domain-specifi prompts and excluded from federated aggregation. During the fine- tuning stage, these prompts serve as semantic priors that condition the GFM's representations, facil- itating improved adaptation to diverse downstream tasks.
Concretely, during federated pre-training, each client maintains a set of learnable prompt tokens embedded in its local graphs feature space. For the k-th client, this prompt set is denoted as & where is the number of prompts and F the feature dimensionality. Given the local graph 1A Ji=1" (V*,&* c*) and node features {- Jo,eve, node representations are enhanced by a weighted combination of prompts, with attention weights computed via À learnable linear projections:
G
Σe e(w))
where o reflects the relevance of the j-th prompt to node ni and w is the corresponding learnable projection vector. These prompts and projection weights are optimized together with the local GNN backbone through a self-supervised graph reconstruction task, described in Eq. ?
During the fine-tuning stage, we downloads the global model as GFM, which encodes generalizable cross-domain knowledge. In parallel, it collects all locally learned prompts and associated projection X. weights to construct a adaptive domain-aware prompt pool, denoted as p { and w (\atg, E node features are augmented using this
[w ...,, wK]. Given target graph G prompt pool. For each node U, Vist with feature x,, the enhanced representation is computed as:
K
A
(W))) o?
P
As a result, FedGFM+ effectively capitalizes on domain-specifie prompts acquired during pre- training, substantially improving its adaptability to heterogeneous domains and diverse downstream tasks in the fine-tuning phase.
Experiments
In this section, we present a comprehensive evaluation of FedGFM+. We begin by introducing the experimental setup (.). and then seek to answer the following research questions: Q1: After task-specific fine-tuning, does the GFM trained by FedGFM+ consistently outperform (1) isolated supervised learning techniques, (2) state-ofi satte-of-the-an FGL baselines, and (3) naive federated variants of centralized GFM strategies across node-, edge-, and graph-level prediction tasks (Se)), Q2: How does each individual module contribute to the overall performance of FedGFM+ (S3)...3) Q3: Is FedGFM+ robust to changes in hyperparameter configurations (Sec In addition to the main evaluation, further investigate the few-shot generalization ability (Q4) in Appendix D.
7

---

Table 2: Performance comparison of FedGFM+ and baselines. Best results of each baseline category are underline. *** denotes federated variants centralized GFM. "N/A' denotes task inapplicabil- ity. Node, edge, and graph classification datasets marked in red, yellow, and blue, respectively.
Dataset
Cora
PubMed
OGB-arxiv
WikiCS
FB15K237
WNISRR
HIV
PCBA
Method
Lincar
73.44
85.11
67.55
74.38
72.05
84,33
65.48
57.71 0.22
0.08
0.34
0.23
GCN
80.17
84.70
72.50
77.24
71.24
82.27
65.37
63.41
0.35
0.22
#: 0.24
+ 0.16
#: 0.30
=:: 0.18
0.51
# 0.20
GAT [52
81.09
84.47
71.34
77.59
73.07
85.52
65.68 02
64.83
GraphSAGE [1K]
80.52
85 20
72.78
77.63
72.10
82,98
65.
66.42
0.24
0.24
3: 0.31
15 0:21
d: 0.38
0.22
0.27
4: 0.14
GIN
78.45
83.61
70.74
70.06 0.14
80.25
66.30
68.83
#: 0.30
FedAvg
81.45
85.22
71.53
77.67
73.14
83.55
66.05
68.52
0.28
MOON M1
81.72
85.84
72.50
77.54
73.20
83.64
67.10
69.81
0.38
+ 0.21
0.41
: 0.24
2:1.11
0.45
0.26
0.30
FedSage+
82.15
86.37
72.80
78. 64 1
73.17
82.95
N/A
N/A
0.28
0.15
0.11
0.22
0.16
Ted-PUB [
81.98
86.51
73.15
78.32
72.84
83.79
N/A
N/A
0.29
0.13
FedGTA A4
82.41
87 10
73.28
78.60
N/A
N/A
N/A
N/A
0.33
0.25
0.14
+: 0.24
FedTAD 66
82.24
72.50
78.22
N/A
N/A
N/A
FGSSL [0]]
81.55
85.60
73.33
76.25
N/A
N/A
N/A
N/A
0.49
+: 0.21
22 0:15
1: 0.24
FGGP
82.03
85.10
74.19
76.44
N/A
N/A
N/A
N/A
=: 0.13
= 0.37
d: 0.05
+ 0.18
GCFL+
./A
N/A
N/A
N/A
N/A
N/A
67.51
0.14
71.95
0.28
FodStar [3
N/A
N/A
N/A
N/A
N/A
N/A
67.82
71.27 2 0.11
2 0.21
OFA [
80.04
85.30
73.12
78.55
72.88
84.28
67.00
71.05
= 0.33
= 0.29
2: 0:25
:: 0.37
= 0.36
=: 0.49
=: 0.19
:: 0.28
GFT 3K
81.07
73.19
73.52
86.30
66.32
72.81
UniGraph"
81.53
86.07
72.94
78.47
73.80
86,
67. 17.24
73.51
0.18
0:15
1: 0.22
0:48
0.51
0.74
GQT
81.92
85.59
74.07
77.52
73.40
85.66
67.93
73.22
=: 0.26
=6 0.37
0:47
4: 0.25
4: 0.11
=: 0.29
0.24
:: 0.30
GraphCT.IP P7
82.33
84,95
73.55
78.14
72.95
84,92
67.31
73.40
0.5.1
0.29
FedGFM+ (Ours)
83.79
88.52
76.31
80.70
75.25
89.25
69.39
77.68
0.18
0.34
0.22
5.1 Experimental Setup
To evaluate the effectiveness of FedGFM+, conduct experiments 8 benchmark graph datasets spanning a range of domains and covering three key tasks: node classification (Citation Networks: Cora, PubMed [_6], and OGB-Arxiv [T]] Hyper-Link Networks: WikiCS [B3]), edge classifica- tion (Knowledge Graphs: FB 15K237
[77 and WN18RR (I21),, and graph classification (Molecule Graphs: HIV, PCBA [[Z]). Each dataset partitioned into 3 clients simulate decentralized sce-
narios, and report the average test performance (accuracy or AUC) across clients. We compare FedGFM+ against three baseline categories: (1) Isolated Supervised Models, trained independently on each client, including linear layer, GCN, GAT, GraphSAGE, and GIN; (2) FL/FGL Approaches, including general-purposs methods like FedAvg and MOON, and task-specific methods such as FedSage+, Fed-PUB, FedGTA, FedTAD, FGSSL, FGGP, GCFL+, and FedStar: and (3) Federated Variants of centralized GFM training strategies (OFA, GFT, UniGraph, GQT, GraphCLIIP). More experimental details are provided in Appendix Q

---

5.2 Performance Comparison (Answers for Q1)
To answer QI, we compare FedGFM+ with a range of competitive baselines, evaluating each con- figuration over 3 independent without fixed seeds. As summarized in Table 2, FedGFM+ con- sistently achieves superior performance across all datasets and downstream tasks.
Comparison with Isolated Supervised Learning. FedGFM+ consistently outperforms supervised backbones, confirming its strong cross-domain and cross-task generalization. Specifically, im- proves over the best baselines by at least 2.70% in node classification, 2.18% in edge classification, and 3.09% in graph classification, demonstrating superior transferability and robustmess.
Comparison with FL/FGL Methods. As discussed in Section MI existing FL/FGL methods are limited by data/task heterogeneity and reliance on task-specific information, restricting its training and evaluation scenarios. In contrast, observed, FedGFM+ consistently outperforms by enabling broad cross- domain and task collaboration that captures general structural and semantic knowledge.
Comparison with Federated Variants of Centralized GFM. As observed, naive federated GFM models often suffer from knowledge entanglement, leading to them even below isolated supervised baselines (i.e., negative transfer). In contrast, FedGFM+ effectively addresses these issues via its design (i.e., AncDAI and AdaDPP), enabling efficient downstream adaptation.
5.3 Ablation Study (Answer for Q2)
To address Q2, we analyze FedGFM+s two key modules. AncDAI guides the initialization of learn- able tokens the global gVQ VAE codebook, while AdaDPP is applied during fine-tuning to im- prove adaptability to domain- and task-specific variations. An ablation study on 8 datasets (Table B) shows that removing both modules degrades performance. Notably, excluding AneDAI causes larger drop than excluding AdaDPP, highlighting AncDAIs crucial role in reducing knowledge en- tanglement and boosting generalization. summary, both are vital for FedGFM+s effectiveness.
Table 3: Ablation study results for FedGFM+. Node, edge, and graph classification datasets are marked in red, yellow, and blue, respectively.
Datasct
Cora
PubMed
OGB-
WIKICS
FB15K237
WNISRR
HIV
PCBA
Method
FedGFM+ w/o. AncDAI
81,55
85.56
75.19
78.05
73.08
87.61
67.52
74.81
FedGEM+ AdaDPP
83.17
87.42 + 0.26
75.83
77.64
74.59
88.
67.84 0..2
76.72
0.18
* 0.27
+ 0.14
0.26
+: 0,20
0.10
FedGEM+
83.79
75.25
89.25
69.39
77.68
0.13
5.4 Sensitivity Analysis (Answer for Q3)
To address Q3, we perform a sensitivity analysis on key hy- perparameters in FedGFM+. As
pre-trainingfnn-tuuing frame-
Test Test ACC or AUC (96)
Test (96)
85
work, involves many hyper-
parameters; here we focus on those in our core modules. For
80
25
WN1BRR
25
WN18RR
AncDAI, ber of learnable tokens in the
vary the num-
HIV
HIV
70
20
global AVQ-VAE codebook. For AdaDPP, we vary the number of learnable prompts per client. Re- sults are shown in Fig. H (a) AneDAl maintains stable perfor-
128
256
512
Number of Takens
Number of Prompts
(a) Sensitivity AucDAI
(b) Sensitivity of AdaDPP
Figure 4: Sensitivity analysis results for FedGFM+
mance under different codebook sizes, indicating robust domain initialization; (b) AdaDPP performs

---

well with few prompts, and is insensitive to prompt number. Overall, FedGFM+ shows strong ro- bustmess key hyperparameters.
Limitation
While + adopts randomly initialized encoders and decentralized optimization mitigate privacy leakage, we acknowledge that the exchange high-level representations (c.g., prototypes and prompts) may still expose partial semantic information. A thorough privacy analysis, including the investigation of potential leakage pathways and the development of threat model, remains an important direction for future work. Incorporating formal privacy guarantees would further strengthen the robustness of our approach in practical federated settings.
Conclusion
This paper initiates the study of Federated Graph Foundation Models (FedGFM), aiming to train a unified graph model with domain and task generalization under decentralized settings. By inte- grating the complementary strengths of Federated Graph Learning (FGL) and centralized Graph Foundation Models (GFM) training strategies, FedGFM alleviates the limitations of both paradigms. Empirical analysis reveals key challenge, knowledge entanglement, which limits the effectiveness of naive federated adaptations of centralized GFM training. To address this, we propose FedGFM+, dual-perspective framework incorporating AncDAI and AdaDPP. Experimental results demonstrate the superior performance and generalization ability of FedGFM+.
Acknowledgments
This work was supported in part by the National Key R&D Program of China under Grant 2023YFB3001900, in part by the Shenzhen Science and Technology Program under Grant KJZDD2023022111300000 139001044 in part by the National Natural Science Foundation China under Grant 62572501.
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tige Akkaya, Florencia Leoni Aleman, Diego Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[2] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Op- tuna: A next next-generation hyperparameter optimization framework. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD, 2019.
[3] Jinheon Baek, Wonyong Jeong, Jiongdao Jin, Jachong Yoon, and Sung Ju Hwang. Personalized subgraph federated learning. 2023.
[4] Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradi- ents through stochastic neurons for conditional computation. arXiv preprint arv: Mtt.133083332 2013.
[5] Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 1008(10),P10008, 2008
[6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models few-shot learn- ers, 2020.
[7] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks graphs. arXiv preprint arXiv: 131111122622 1316.220,,,... 2013.
[8] Chuan Chen, Weibo Hu, Ziyue Xu, and Zibin Zheng. Fedgl: federated graph learning frame- work with global self-supervision. arXiv preprint arXiv:2105.03170. 2021.
[9] Jialin Chen, Haolan Zuo, Haoyu Peter Wang, Siqi Miao, Pan Li, and Rex Ying. Gfse: A foundational model for graph structural encoding.
[10] Jinsong Chen, Chenyang Li, GaiChao Li, John E Hopcroft, and Kun He. Rethinking tokenized graph transformers for node classification. arXiv preprint arXiv:2502.08101 2025.
[11) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1- 113, 2023.
[12] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
[13] Dongqi Fu, Wenxuan Bao, Ross Maciejewski, Hanghang Tong, and Jingrui He. Privacy- preserving graph machine learning from data to computation: A survey. ACM SIGKDD Explorations Newsletter, 25(1):54-72, 2023.
[14] Xingbo Fu, Binchi Zhang, Yushun Dong, Chen Chen, and Jundong Li. Federated graph machine learning: A survey of concepts, techniques, and applications. Explorations Newsletter, 24(2):32-47, 2022.
ACM SIGKDD
[15] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems, NeurlPS, 2017.
[16] Chaoyang He, Keshav Balasubramanian, Emir Ceyani, Carl Yang, Han Xie, Lichao Sun, Lifang He, Liangwei Yang, S Yu Philip, Yu Rong, et al. Fedgraphon: A federated learn- ing benchmark system for graph neural networks. International Conference on Learning Representations, ICLR Workshop on Distributed and Private Machine Learning, 2021.
11

---

[17] Yufei He, Yuan Sui, Xiaoxin He, and Bryan Hooi. Unigraph: Learning unified cross ross-domain foundation model for text-attributed graphs. arXiv preprint arXiv:2402.13630, 2024.
[18] Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, and Bryan Hooi. Unigraph2: Learning unified embedding space bind multimodal graphs. In Proceedings of the ACM Web Conference 2025, pages 1759-1770, 2025.
[19] Weihua Hu, Matthias Fey, Marinka Zitnik. Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems, NeurIPS, 2020.
[20] Wenke Huang. Guancheng Wan, Mang Ye, and Bo Du. Federated graph semantic and structural learning. arviv preprint Xiv::22406. 1837,, 2024.
[21] Xinke Jiang, Rihong Qiu, Yongxin Xu, Yichen Zhu, Ruizhe Zhang, Yuchen Fang, Chu Xu, Jun- feng Zhao, and Yasha Wang. Ragraph: A general retrieval-uugmented graph learning frame- work. Advances in Neural Information Processing Systems, 37:29948-299 29985, 2024.
[22] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, ICLR, 2017.
[23] Lecheng Kong, Jiarui Feng, Hao Liu, Chengsong Huang, Jiaxin Huang, Yixin Chen, and Muhan Zhang. Gofa: A generative one-for-all model for joint graph language modeling, 2024.
[24] Qinbin Li, Bingsheng Hc, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
CVPR, 2021.
[25] Xunkai Li. Zhengyu Wu, Wentao Zhang, Henan Sun, Rong-Hua Li, and Guoren Wang. Adafgl: A new paradigm for federated node classification with topology heterogeneity. arXiv preprint arXiv:2401.1 11750, 2024.
[26] Xunkai Li, Zhengyu Wu, Wentao Zhang, Yinlin Zhu, Rong-Hua Li, and Guoren Wang. Fedgta: Topology-aware averaging for federated graph learning. arXiv preprint 15555 2024.
[27] Xunkai Li, Yinlin Zhu, Boyang Pang, Guochen Yan, Yeyu Yan, Zening Li, Zhengyu Wu, Wen- tao Zhang, Rong-Hua Li, and Guoren Wang. Openfgl: A comprehensive benchmarks for federated graph learning. arXiv preprint arXiv:2408.I 16288, 2024.
[28] Tianqiamjin Lin, pengwei Yan, Kaisong Song. Zhuoren Jiang, Yangyang Kang, Jun Lin, Weikang Yuan, Junjie Cao, Changiong Sun, and Xiaozhong Liu. Langgfm: A large language model alone can be powerful graph foundation model. arXiv preprint arXiv:2410.1490 149611, 2024.
[29] Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. arXiy preprint arXiv:2310.00149 2023.
[30] Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, and Lichao Sun. Sora: A review background, technology, limitations, and opportunities of large vision models, 2024.
[31] Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan. Gfmm-aag Graph foundation model for retrieval augmented generation. arXiv preprint arXiv:2502.01113, 2025.
[32] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Ar- cas. Communication-effiiinn learning of deep networks from decentralized data. Artificial Intelligence and Statisties, 2017.
[33] Péter Mernyei and Cătälina Cangea. Wiki-cs: wikipedia-based benchmark for graph neural networks. arXiv preprint 20Xv:2007.02901,, 2020.
12

---

[34] Nils Reimers and Iryna Gurevych. Sentence-bert. Sentence embeddings using siamese bert- networks, 2019.
[35] Li Sun, Zhenhao Huang, Suyang Zhou, Qiqi Wan, Hao Peng, and Philip Yu. Riemanngfm: Learning graph foundation model from riemannian geometry. In Proceedings of the ACM on Web Conference 2025, pages 1 1154-1165, 2025.
[36] Yue Tan, Yixin Liu, Guodong Long, Jing Jiang, Qinghua Lu, and Chengqi Zhang. Federated learning on non-iid graphs via structural knowledge sharing. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 9963,9961, 2023.
[37] Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 20155 conference on empirical methods in natural language processing.
pages 1499-1509, 2015.
[38) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint www.:2037.0988... 2023.
[39] Petar Veličkovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, ICLR, 2018.
[40] Guancheng Wan, Wenke Huang, and Mang Ye. Federated graph learning under domain shift with generalizable prototypes. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 15429-15437 2024.
[41] Limei Wang, Kaveh Hassani, Zhang, Dongqi Fu, Baichuan Yuan, Weilin Cong, Zhigang Hua, Hao Wu, Ning Yao, and Bo Long. Learning graph quantized tokenizers. In The Thirteenth International Conference on Learning Representations.
[42] Shuo Wang, Bokui Wang, Zhixiang Shen, Boyan Deng, and Zhao Kang. Multi-domain graph foundation models: Robust knowledge transfer via topology alignment. International Conference on Machine Learning. ICML, 2025.
[43] Zehong Wang. Zheyuan Zhang. Nitesh Chawla, Chuxu Zhang, and Yanfang Ye. Gft: Graph foundation model with transferable tree vocabulary. Advances in Neural Information Processing Systems, 37:107403-107443, 2024.
[44] Zehong Wang, Zheyuan Zhang, Tianyi Ma, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. Towards graph foundation models: Learning generalities across graphs via task-trees.
[45] Zhen Wang, Weirui Kuang, Yuexiang Xie, Liuyi Yao, Yaliang Li, Bolin Ding, and Jingren Zhou, Federatedscope-gnnt Towards unified, comprehensive and efficient package for feder- ated graph learning. Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD, 2022.
[46] Zhihao Wen and Yuan Fang. Augmenting low-resource text classification with graph- grouuddd pre-training and prompting. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR, 2023.
[47] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl I Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, 9(2):513-330, 2018.
[48] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEBE transactions on neural networks and learning systems, 32(1):4-244 22, 2020.
[49] Lianghao Xia and Chao Huang. Anygraph: Graph foundation model in the wild, arXiv preprint arXiv:2408. 10700, 2024.
13

---

[50] Lianghao Xia and Chao Huang, Anygraph: Graph foundation model in the wild. arXiv preprint arXiv:2408.1077 10700, 2024.
[51] Lianghao Xia, Ben Kao, and Chao Huang. Opengraph: Towards open graph foundation models. arXiv preprint arXiv:2403.0111 2024.
[52]
Lianghao Xia, Ben Kao, and Chao Huang. Opengraph: Towards open graph foundation models. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024.
[53] Han Xie, Jing Ma, Li Xiong, and Carl Yang. Federated graph classification over non-iid graphs. Advances in Neural Information Processing Systems, NeurIPS, 2021.
[54] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? 2019.
[55] Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. Federated learn- ing. Synthesis Lectures on Artificial Intelligence and Machine Learning, 13()):1207,, 2019,
[56] Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learn- ing with graph embeddings. In International Conference Machine Learning, ICML, 2016.
[57] Yuhang Yao, Weizhao Jin, Srivatsan Ravi, and Carlee Joe-Wong. Fedgen: Convergence- communication tradeoffs in federated training of graph convolutional networks. Advances in neural information processing systems, 36, 2024.
[58] Xingtong Yu, Zechuan Gong, Chang Zhou, Yuan Fang, and Hui Zhang. Samgpt: Text- free graph foundation model for multi-domain pre-training and cross-domain adaptation. In Proceedings of the ACM on Web Conference 2025, pages 1142-1153, 2025.
[59] Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Text-free multi-domain graph pre-training: Toward graph foundation models. arXiv preprint 1.XXi:2440,139344 2024.
[60] Huanding Zhang, Tao Shen, Fei Wu, Mingyang Yin, Hongxia Yang. and Chao Wu. Federated graph learning-a position paper. arXiv preprint 1.XXiv22005.1099) 2021.
[61] Ke Zhang, Lichao Sun, Bolin Ding, Siu Ming Yiu, and Carl Yang. Deep efficient private neigh- bor generation for subgraph federated learning. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pages 806-814, 814. SIAM, 2024.
[62] Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu Ming Yiu. Subgraph federated learn- ing with missing neighbor generation. Advances in Neural Information Processing Systems, NeurIPS, 2021.
[63] Taolin Zhang, Chengyuan Mai, Yaomin Chang, Chuan Chen, Lin Shu, and Zibin Zheng. Fedego: privacy-preseriigg personalized federated graph learning with ego-graphs. ACM Transactions on Knowledge Discovery from Data, 18(2):1-27, 2023.
[64] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI Open, 1:57-81, 2020.
[65] Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang, Kaixiong Zhou, and Yongfeng Zhang. Llm as gnn: Graph vocabulary learning for text-attributed graph foundation models. arXiv preprint 203333550.03313, 2025.
[66] Yinlin Zhu, Xunkai Li, Zhengyu Wu, Wu, Miao Hu, and Rong-Hua Li. Fedtad: topology- aware data-free knowledge distillation for subgraph federated learning. arXiv preprint arXiv:2404.1 14061, 2024.
[67] Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs. In Proceedings of the ACM on Web Conference 2025, pages 2183-2197, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The brief descriptions of our proposed method and main contributions provided at the end of the introduction, along with the motivations and overall performance analysis discussed both the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation of the proposed FedGFM+ framework in Sec.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-tex system might not be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach ad- dress problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: We provide proofs for each introduced theorem the paper and clearly present the assumptions, which can be found in Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear the main paper or supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple-
mented by formal proof' provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided detailed experimental implementation details in the Ap- pendix regarding how we fine-tuned each dataset or method achieve its optimal perfor- mance.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Yes, we have included the source code in the supplementary materials to enable interested researchers to reproduce the experimental results presented in our paper with sufficient guidance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (ttttps:/nniss.ce)
for more details.
While we encourage the release of code and data, understand that this might be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (ittps:)
//nips. http://cccc...iiciuieeeee/ee///e /Codeidue ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specifyy all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training details be found in Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
We provide all our performances with error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the enror bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative еπτог rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to Appendix O for details regarding to our computing resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpp:///oerrp..cccaoaaattttttet httppb/liic/tticcccceeeeeeeesese
Answer: [Yes]
Justification:
We are hereby comply to the guidelines illustrated the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no social impact associated with the work we performed and pre- sented in this paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e-g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not pose such risks for being misused for malicious intents.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: For all existing works mentioned in our paper, whether for illustrative pur- poses as baselines for performance comparison, we have provided proper citations and references to acknowledge their contributions.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
19

---

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the pack- age should be provided. For popular datasets, haperswithcods com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the cense of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [Yes]
Justification: For the purpose of evaluating our methods, we have developed new code and included it in the supplementary materials when submitting our work OpenReview. This work does not introduce any new assets such as datasets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification:
We do not include such research or experiments in our work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but the main contri- bution of the paper involves human subjects, then as much detail possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is not an important, original, or non-standard component of the core methods in this research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
21

---

Appendix: Table of Content
A More Kelated Works
23
T'horretccal Proof
24
C More Detailed Experimental Setup
25
C.1
Dafasel
25
C.2 Data Processing
25
C.3 Basclmes
26
C.4
Model Architecture
28
C.5
Traiinng Evaluation Illustration
29
C.6
lyperparameters
29
C.7
Experimental Environment
30
D
Few-shof Learning Results
30
22

---

A More Related Works
Graph Neural Networks. Earlier research on deep graph learning extends convolution to handle
graphs [+] but comes with notable parameter counts. To this end, GCN [2 simplifies graph convo-
lution by utilizing a l-order Chebyshev filter to capture local neighborhood information. Moreover, GAT [399] adopts graph attention, allowing weighted aggregation. GraphSAGE [T5]] introduces variety of learnable aggregation functions for performing message aggregation. Moreover, GIN [54]] aims to preserve structural information maximally and theoretically proves its discriminative power matches the Weisfeiler-Lehee Lehmman graph isomorphism test. Further details on GNN research can be found in surveys [18. b4]]
Federated Graph Learning. Motivated by the success of federated learning in computer vision and natural language processing [НS] and the demand for distributed graph learning, FGL has gained increasing attention. From the data and task perspectives, FGL studies are categorized into three set- tings: (1) Graph-level FGL, where each client collects multiple graphs for graph-level tasks, like graph classification. The main challenge is avoiding interference between clients' graph datasets, especially in multi-domain settings. For example, GCFL+ [53] introduces a GNN gradient pattern- aware technique for dynamic client clustering to reduce conflicts from structural and feature het- erogeneity. (2) Subgraph-level FGL, where each client holds subgraph of a global graph for node-level tasks like node classification. The key challenges are subgraph heterogeneity and miss- ing edges. Fed-PUB [[]] addresses heterogeneity by enhancing local GNNs with random graph embeddings and personalized sparse masks for seleetive aggregation. FedGTA [26] encodes topol- ogy into smoothing confidence and graph moments to improve model aggregation. Other stud- ies [255 21 40, 66 also achieve strong results on this challenge. To address missing edges, Fed- Sage+ [62] integrates node representations, topology, and labels across subgraphs, training neigh- bor generator to restore missing links and achieve robust subgraph-FL. Other works [8, 577, [1] also excel in this area. (3) Node-level FGL, where each client collects one or multiple ego-networks for node- and edge-level tasks. From the perspective of data format and task, Node-level FGL can be seen as a special case of Subgraph-Ievel FGL. Notably, the application scenarios of Node-level FGL usually involve strict privacy constraints, and representative methods include FedEgo [6:3]. Detailed insights into FGL research are available in surveys
and benchmark studies [06. MM
Language-Orieen Oriented GFMs [28. 63 I7 23J. These approaches transform graph structures into linearized textual sequences by encoding nodes and edges using syntactically structured templates. The resulting representations can then be processed by token-based encoderstypically LLMsthat are pre-trained on vast corpora of natural language. This approach allows for seamless integration with existing LLM infrastructure and leverages the powerful contextual understanding capabilities de- veloped through natural language processing (NLP). In more detail, during the pre-training phase, these models optimize the parameters of the embedding functionoften realized as an LLMthrough conventional NLP objectives such as next-token prediction or masked language modeling. These objectives encourage the model to learn coherent semantic representations from the flattened graph text, effectively transferring linguistic inductive biases to graph representation learning. However, despite their ability to inherit the expressive power of LLMs, Language-orient GFMs face intrinsic limitations. The transformation from graph to text inevitably introduces information loss, especially concerning structural properties such as node connectivity and subgraph patterns. Moreover, this flattening process may distort the original graph topology in ways that are not easily reversible, thereby affecting downstream tasks that rely on accurate structural reasoning. Additionally, scala- bility becomes a concern due to the growing length of textual sequences with increasing graph size, which may lead to inefficiencies in both computation and memory usage.
Graph-Oriented GFMs [S0, 5y
588] These approaches aim to preserve both the semantic richness of textual attributes and the integrity of graph topology through purpose-built architectures. These models typically adopt a hybrid design, wherein a frozen LLM is used to extract high-quality textual embeddings from node and edge features, while a trainable GNN component handles the aggregation and propagation of information across the graph structure. This dual-component architecture enables the model to benefit from the strong language understanding capabilities of LLMs without compromising the fidelity of graph structure. The GNN backbone ensures that topological relationships are explicitly modeled, allowing for effective message passing and relational reasoning. During the pre-training stage, graph-oriented GFMS often incorporate self- supervised learning strategies, such as graph reconstruction or contrastive learning objectives, which
52, 2A
23

---

help the model capture invariant and transferable representations across diverse domains. These tasks encourage the model to learn a unified representation space where both textual and structural semantics are coherently aligned, leading to better generalization on downstream tasks involving heterogeneous graph data. By preserving the native structure of graphs and leveraging the repre- sentational power of modern neural architectures, graph-oriented GFMs offer promising direction toward building robust and scalable foundation models for graph-centric machine learning.
B Theoretical Proof
In this section, we provide theoretical analysis for the distinguishabiliyy of domain prototypes under random initialization (Theorem. B.II) and the semantic separability of gvQ-vA QAE codebooks initial- ized via AncDAI (Theorem.
Theorem B.1 (Domain Prototype Distinguishabiltyy) Let G (V=,E") and G = -V=, Eh) de-
note local graphs from two clients belonging to different domains, with node features X", xx,X" Raxd and adjacency matrices A E RRax, Let 18 be the parameters of a randomly initial- which is broadcast to all clients for local initialization. The
ized L-layer global domain prototype is computed with Eqs. (G) and (E))
R
p
(A" X"),,
p
-
18th (A",
(8)
Then, there exists constant a > O, whose value depends on the architecture and depth L of GNN- Encoder), such that:
Eg - 1P'12 2a -e- 1 -
(9)
Proof. Let 2 = 10th (A" X"); and x f0lh (A )x denote the representations of node i ob- tained from a frozen GNN applied to graphs a and b, respectively. Here, the GNN parameters randomly initialized and held fixed. Under this setting, the GNN's computations preted as performing random but deterministic linear transformations and message passing opera- tions. Leveraging the linearity of expectation and the independence of random initialization, the expected squared Euclidean distance between the resulting node prototypes can be expressed as:
be inter-
Eg ||pe
10112] = Eg
22
21²)
(10)
2 n i=1
Σ Eg
2 - -
(11)
2 a. (|Xx
x0 + ||A - A²122)
(12)
Theorem B.2 (Semantic Separability of AncDAl-Initialized Codebook). Let {p* be the set of domain prototypes uploaded from K clients. For each prototype pº, we generate a set of perturbed vectors via Eq. B
(13) Let Cpernub and Crand be codebooks constructed respectively from perturbed prototypes and from standard Gaussian initialization. Then for any domains + b and respective node embeddings z", z from fo(A, X)). we have:
D 1 = oe,
oe,, &~N(O,,1), i=
IP [coode(z": Cpernubb ) # code(z": cpernnbb]] > P [code(z"; crund) + code(z"; code(z":crmnd)) (14)
i.e. the perturbation innitiliiidd codebook yields higher domain- separability.
Proof. We adopt quantization function based on cosine similarity:
TTc.
code(z; C) = arg max eEc
|z||||||||22

---

which assigns each embedding to the codebook vector with the smallest angular distance. Assume that the domain prototypes {p*
{p*)
k= satisfy minimal angular separation:
(p*) T ||pd | 8
min arccos azth
The perturbed codebook (perturb formed by adding isotropic Gaussian noise ae to each prototype, with (0, 1). For sufficiently small the perturbations preserve the cluster structure, yielding distinct codebook clusters separated by angles close
Node embeddings z" and z" sampled from different domains concentrate in neighborhoods around their respective prototypes. Formally, with high probability,
(z") pa
(1(2) (z")
arccos
|| p
arccos
| |2
for some small e > 0. Then. by the triangle inequality on the unit sphere,
(z"))
arccos
2 8 2e,
||
which implies that embeddings from distinct domains remain rell-separated.
Therefore, the probability that z" and z" are assigned to the same codeword under Cperturb is bounded above by the probability that perturbations cause cluster overlap. which is small for suf- ficiently small a. In contrast, a random codebook Crand sampled isotropically from standard Gaussian lacks such separation, and embeddings from different domains have higher probability of being assigned the same codeword. Thus, we combines these observations and proof that:
PP [ooee(z"; C (perturb) * code(z') [Perturb)] 2 P [code(z"; Crandd) * code(z"; craud)].
C
More Detailed Experimental Setup
C.1 Dataset
Table 4: The statistics of evaluated datasets in our experiments.
Dataset
Domain
Task
# Graphs
Avg. #Nodes
Avg. #Edges
# Classes
Cora
Citation
Node
1
2,708
10,556
7
PubMed
Citation
Node
1
19,717
44,338
3
Arxiv
Citation
Node
1
169.343
1,166,243
40
WikiCS
Hyper link
Node
1
11,701
216, 123
10
FB15K237
Knowledge
Link
14,541
310,116
237
WN18RR
Knowledge
Link
1
40,943
93,003
11
PCBA
Molecule
Graph
437,929 41,127
26.0
28.1
128
HIV
Molecule
Graph
25.5
27.5
2
We utilize 8 datasets from various domains and tasks, as detailed in Table
C.2 Data Processing
Our data processing process can be illustrated as Fig. G. consisting of two steps: Step 1: Language Encoding. We use Sentence-Ber [[4]] uniformly encode attribute graph datasets in differ- ent fields to uniformly convert node and edge text into 768 088-dimensional vectorized representations; and Step Data Decentralization Simulation. Real-world graph data inherently collected by multiple institutions, resulting in naturally decentralized data distributions. Prior studies in FGL categorize such decentralization into three canonical levels [[6]]: (1) node-level, where each client maintains ego-networks extracted from global graph; (2) subgraph-level, where each client collects
25

---

SpliT
Any Specific Grush bataset Detaset Spiit)
Single (Nocke Medee/Elge-a -415 Datoset)
Clinn
Client
Uniform Spitt
Nedwllbdge
decw/dda Attributc
Mustiple Graphs (Grapi--115 Datoset)
Clien
Step 1: Longuage Encading
Step 2:
Data Decentralization Simulation
Figure 5: Data processing pipeline to simulate decentralized multi-domain and task graphs.
local subgraph induced from broader graph topology; and (3) graph-level, where each client in- dependently gathers a of graphs from a larger collection. Notably, the node-level setting can be regarded as a special case of subgraph-leve decentralization. Hence, we focus on the latter two in this work. Specifically, under the subgraph-leve setting, the implicit global graph G = (V,E) has multiple substructures independently collected by different clients. The k-th client locally collects a subgraph G2 = (Va, such that V& Ç and & Ç E; Under the graph-level setting, the k-th client independently collects subset of graphs Sg from an implicit broader collection {G, 1=1* i.e., Sy $ S. To simulate these decentralized scenarios in our experiments, we adopt two partition- ing strategies: the Louvain algorithm [5]] for simulating subgraph-level decentralization, and random allocation for graph-level decentralization, both of which is widely used in various FGL. studies [27]].
Finally, the default train/validation/eest splits used in the fine-tuning stage are summarized in Table. Notably, due to the distributed nature of federated settings, the training set proportion is typically much higher than in centralized graph learning paradigms. This splitting strategy has been widely adopted in prior works [27].
Table 5: Train/Validation/ Tesst splits for different datasets
Dataset
Train Split
Validation Split
Test Split
Cora
5%
20%
40%
PubMed
60%
20%
20%
WikiCS
80%
10%
10%
Arxiv
80%
10%
10%
WNI8RR
80%
10%
10%
FB15k237
80%
10%
10%
ChemHIV
80%
10%
10%
ChemPCBA
80%
10%
10%
C.3 Baselines
Since this paper is the first to explore FedGFM, we transfer baselines from adjacent fields. Specifi- cally, in our experiments, evaluate 20 baselines, which can be summarized into categories. The detailed descriptions of these baselines are as follows:
(1) Isolated Supervised Learning. These methods train individual supervised models each client without federated communication. They serve as a reference for evaluating negative transfer and the benefits of federated learning. The models in this category include linear layer, GCN [[2]]
GAT [39]] GraphSAGE [LS]], and GIN [54]];
GCN 2 is a classical model in graph neural networks, which captures graph structure through spectral convolutions based on normalized graph Laplacian. By aggregating information from neighboring nodes, it enables efficient node classification and handles graph data in computation- ally effective manner. The use the Laplacian matrix simplifies the convolution operation, making it foundational approach in graph representation learning.
26

---

GAT [89] draws inspiration from the success of attention mechanisms in natural language process- ing, introducing a novel graph attention mechanism that allows nodes dynamically focus on the most relevant neighbors. This attention- based aggregation enables more adaptive learning.
GraphSAGE [15] extends graph neural networks by introducing a sampling- based message passing mechanism, which allows for scalable neighborhood aggregation. This approach is particularly well- suited for inductive learning, as it can efficiently generalize unseen nodes by sampling fixed-size neighborhood during training. The use of different learnable aggregation functions further enhances scalability, enabling the model to handle large graphs effectively.
GIN [لع|] is designed to preserve graph structural information and has been shown to be as expressive as the Weisfeiler- Lehmann graph isomorphism test in distinguishing graph structures. Notably, GIN is usually more suitable for graph-level tasks.
(2) FGL Approaches. We evaluate various representatives FL/FGL baselines, including two FL methods desinged for FL with vision tasks (FedAvg [32]] MOON [244],, and subgraph-Ievel FGL techniques (FedSage+ [62], Fed-PUB [3]. FedGTA [26]], FedTAD [66], FGSLL [20], FGGP [H]]
and graph-level FGI. methods (GCFL [5XS]] and FedStar [36]). The detailed descriptions of these baselines are as follows:
FedAvg [12]] a simple yet effective method in FL for the vision and language field, enabling decentralized model training while preserving data privacy. A central server distributes the global model to clients for local updates. The server then aggregates the clients' local models form new global model, which is broadcast to all clients to update their local models in the next round.
MOON | is representative FL method originally developed for the vision domain. It leverages contrastive learning at the model level to align local and global representations, thereby mitigating performance degradation caused by data heterogeneity across clients.
FedSage+ [62]] integrates node features, link structures, and labels using GraphSAGE [LS]]] model with FedAvg [+>]]] for FGL over local subgraphs (i.e., subgraph-level FGL). It also introduces neighbor generator to handle cross-client missing links, improving robustness and ensuring a more comprehensive graph representation.
Fed-PUB [E]] is a personalized subgraph- FGL framework that improves local GNNs without relying on global model. It measures inter-client similarity using functional embeddings derived from random graph inputs, enabling weighted aggregation at the server. A client-specific sparse mask further guides personalized parameter updates, facilitating subgraph-aware local adaptation.
FedGTA [26] integrates large-scale graph learning into FGL by having clients encode topology and node attributes, compute local smoothing confidence and mixed feature moments, and share them with the server. The server aggregates personalized models using smoothing confidence as aggregation weights.
FedTAD [b6]] is subgraph- level FGL method that computes topology- aware node embeddings to estimate class-wise wise knowledge reliability. This guidance enables the server to perform data-free knowledge distillation, transferring reliable knowledge from local clients to the global model.
FGSSL [20] subgraph-level FGL technique, which addresses client drift by aligning nove-level semantics and preserving graph-level structures. It employs contrastive objectives to align nodes of the same class while separating different classes, and distills global relational knowledge into local models via similarity distributions.
FGGP [O] is subgraph-level FGL approach, which decomposes the global model into two tiers connected via prototypes. At the classifier level, class prototypes replace traditional classifiers for better discriminability; at the feature level, contrastive learning injects global knowledge into proto- types to enhance generalization.
GCFL+ [5:3]] is graph-level FGL framework that clusters clients based on GNN gradient patterns to address structural and feature heterogeneity. It further improves stability through gradient sequence- based clustering using dynamic time warping, enhancing both clustering quality and robustness.
FedStar [36] enables graph-level FGL by decoupling structure and feature learning. Clients share domain-invariant structural embeddings via an independent encoder, while learning personalized features locally, reducing feature misalignment and improving transferability.
27

---

(3) Federated Variants of Centralized GFM Approaches. These baselines adapt state-of-the-ar centralized GFM training strategies to the federated setting. Specifically, we include OFA [29]] GFT [+ UniGraph [[Z]], GQT [H] and GraphCIIIP [67]] In their original centralized versions, these methods perform pre-training on all available data at a central learning system using self- supervised objectives. Their federated counterparts distribute this pre-training process across clients. Specifically, for our experiments, in each communication round of the pre-training phase, each lo- cal client deploys the corresponding framework based on its own local data and performs 2 epoch optimization. Subsequently, all trainable parameters will be uploaded to the server, and the parame- ters will be averaged to obtain the global model, which will be broadcast to all clients as the starting point for the next round of local optimization.
OFA P is representative training paradigm for GFM, aiming to learn generalizable represen- tations over cross-domain and cross-task textual attributed graphs. It first standardizes the descrip- tion of nodes and edges via carefully designed language model prompts, transforming any textual attributed graph into a unified vectorized representation. Additionally, OFA introduces NODES-OF. INTEREST prompts to unify various graph tasks within a single modeling framework.
GFT [ treats computation trees derived from message passing as transferable patterns over graphs. Based on this insight, it adopts a <VQ-VAI architecture to map computation trees into discrete codebook representations. Through self-supervised reconstruction on cross-domain graphs during pre- pre-training, it learns a generalizable GFM with strong cross-gras graph transferability.
UniGraph" [177] is GFM training framework that encodes heterogeneous graphs, including those without inherent textual features, into unified textual representations to support cross-domain trans- ferability. It adopts cascaded architecture of language models and GNNs to jointly capture seman- tic and structural information. UniGraph further introduces a Masked Graph Modeling objective for large-scale self-supervised pre-training and applies graph instruction tuning with LLMs to enhance zero-shot and few-shot generalization.
GQT || introduces novel graph quantized tokenizer that decouples tokenizer training from Transformer training, leveraging multi-task graph self-supervised learning to produce robust and generalizable graph tokens. By using the residual Vector Quantization technique, GQT learns hier- archical discrete tokens, reducing memory requirements and enhancing generalization.
GraphCLIP* [67]] addresses key challenges in text-attributed graphs, including heavy reliance on la- bel information and limited cross-domain transferability. It introduces self-super contrastive pretraining method using graph-summary pairs curated with the help of LLMs. By leveraging in- variant learning, GraphCLIP enhances zero-shot transferability and proposes graph prompt tuning technique for few-shot learning, mitigating catastrophic forgetting.
C.4 Model Architecture
For Isolated Supervised Learning Methods, we adopt a two-layer architecture with 64 hidden units. For FL/FGL Methods, if method does not specify custom architecture, we select the backbone based on the downstream task: GraphSAGE is used for node and edge classification, while GIN is employed for graph classification. For Federated Variants of Centralized GFM Methods, we follow the backbone choices reported in the original papers. For FedGFM+, we employ a gVQ-VAE model. The encoder is a 2-layer TraphSAGE-baasd codes node and edge features from the input graph G = (V, E). All layersincluding input, hidden, and outputare set to 768 dimensions, matching the Sentence-BFRRT [[_|]] representations of node and edge attributes. The encoder outputs node embeddings Z R X66 where N is the number of nodes. These embeddings are then quantized via a multi-head g gVQ-VAE codebook using cosine similarity for nearest-neighhor retrieval. The codebook comprises 4 heads, each containing 128 learnable tokens. A shared linear projection is applied to aggregate the multi-head outputs into the final quantized representation. addition to the backbone network, FedGFM+ also introduces mul- tiple light-weight learnable graph hints for each client. By default, we learn 3 local graph prompts for each client. Finally, for task-specific heads used during GFM fine-tuning, we follow the orig- inal design if specified in the corresponding dataset paper. Otherwise, for node classification, we apply a single-layer MLP to predict node labels from node embeddings; for edge classification, we average the embeddings of the two nodes to form the edge representation and apply single-layer
the backbone for both client-side local models and the server-side global
graph convolutional network that jointly en-
28

---

la Fictitt / Felernad Vireeta of Corrdi wwW.tganccher
1ith diaad Bonwet
X
E-fi Bioul botoour (Droph Toak)
Sigh Fonnetice
(Iat sted superited Learn Approcheo)
Taving
KN
Cley
Cle MIDK
Char1
awwn
Chem
Olan 3K-2
CANTaRI
Hadd
Tockted Sganwerr Learnigg
FRi, Approakee
Gibbol FedSFWr- Rellvatte Vorvatte Destriizeed GFM Approocks
Elouctics Reault for 2-th dicba Cotnuct
(Trosag
Training Pre. training Stage
Evoluation / Fine- Tuning Evaluation Stage
Figure 6: Illustration of the pipeline about the traing and evaluation stage for isolated supervised learning, FGL approaches, FedGFM+, and federated variants of centralized GFM approaches.
MLP; for multi-task graph classification, we perform mean pooling over node embeddings to obtain graph-level representation, which fed into a MLP to predict binary labels for each tasks.
C.5 Training and Evaluation Illustration
We illustrate the training and evaluation processes for all baselines and FedGFM+ in Fig. E with detailed descriptions as follows:
Training/ Pretraining Stage. For the Isolated Supervised Learning Baselines, each client trains model independently from scratch, using only its own local graph(s), without any collaboration or information exchange. For the FL/FGL Baselines, we run a FL/FGL algorithm among every 3 clients from the same global dataset. For example, the Cora dataset is split using the Louvain algorithm into clients 1, 2, and 3, and subgraph-Ievel FGL algorithms such as FedGTA are then applied among these clients. Notably, as mentioned in Sec. MI due the heterogeneity of data and tasks, most FL/FGI algorithms can only be simulated among different shards of the same dataset. Moreover, existing FGL. algorithms cannot be applied simultaneously classification, edge classification, and graph classification. For FedGFM+ and Federated Variants of Centralized GFM Baselines, all clients participate in federated pre-training together, which enables extensive collaboration among graph datasets from multiple fields.
Braining/Pretrainnngg
the three tasks of node
Evaluation Wine-Tuuing + Evaluation Stage. For each dataset, we evaluate the performance on the test sets of the three clients associated with it, and report the mean and variance of the resulting metrics. For node and edge classification tasks, we use Accuracy (ACC) as the evaluation metric, while for graph classification tasks, adopt the Area Under the Receiver Operating Characteristic Curve (AUC-ROC). To assess the performance of each individual client under different settings, we follow distinct evaluation protocols. For Isolated Supervised Learning baselines, we directly evaluate each clients local model without any collaboration. For FL and FGL baselines, we evaluate each clients model after training the global model for two communication rounds. For FedGFM+ and the federated variants of centralized GFM baselines, we first attach task-specific header and then fine-tune the model using each clients local graph before evaluation.
C.6 Hyperparameters
For Isolated Supervised Learning Baselines, we perform 1,000 epochs of local training with early stopping based on validation performance. For FLL/FII Baselines, we conduct 100 communication rounds, where each round includes local training epochs. We use the Adam optimizer with a learn- ing rate of 1 x 10-2, weight decay of 5 x 10 and dropout rate of 0.5. For federated variants of centralized GFM Baselines, we adopt the hyperparameter configurations reported in their original papers whenever available. When unspecified, we employ automated hyperparameter optimization using the Optuna framework [2]. Federated pre-training carried out for 50 communication rounds, each consisting of 2 local pre- pre-training epochs. For our proposed FedGFM+ framework, we fix the learning rate for pre-training to x 10-4, During fine-tuning, we perform grid search over learning rates in (10-5 10 10-3, 10-2 10-1) for each dataset. The weight decay is fixed to 5 x 10-1 and the batch size is set to 1.024. Federated pre-training is conducted for 25 communication rounds, with 2 local epochs per round.
29

---

C.7 Experimental Environment
The experimental machine an Intel(R) Xeon(R) Gold 6240 CPU 2.60GHz and NVIDIA A with 80GB memory and CUDA 12.4. The operating system is Ubuntu 22.04.5 with 251GB memory.
D Few-shot Learning Results
perform few-shot evaluation across range of downstream tasks. Specifically, node and edge classification tasks, constrain the number of labeled samples per class to at most 2. For graph classification tasks, however, we do not report few-shot performance, as each graph instance is associated with multi-ddimnssionl labels, making few-shot evaluation non-trivial. The experimental results are summarized as Table. g
Table 6: 2-shot Performance comparison of FedGFM+ and baselines. denotes federated variants of centralized GFM. 'N/A' denotes task inapplicability. Node and edge classification datasets are marked in red and yellow, respectively.
Dataset
Cora
PubMed
OGB-arxiv
WikiCS
FB15K237
WNISRR
Method
OFA *
54.31
45.29
20.56
40.05
19.72
31,28
+ 0.18
0.26
0.42
+ 0.10
0.33
4 0.20
GFT
52.16
44.71
18.31
37.42
17.49
29.55
+ 0.39
# 0.10
0.22
= 0.56
+ 0.24
+ 0.1
UniGraph
54.22
46.41
19.88
39.46
18.45
31.53
+ 0.27
#: 0.50
0.15
# 0.17
+: 0.36
# 0.20
GQT 81
52.45
45.28
20.10
39.25
20.40
30.08
- 0.18
1 0.26
1031
- 0.42
+ 0.18
$ 0.14
GraphCLIP 67
55.31
44.25
20.39
38.58
20.58
31.42
#: 0.12
0.36
0.17
+ 0.16
0.28
# 0.45
FedGFM+ (Ours)
58.33
50.19
21.34
43.35
21.94
33.64 1 0.42
+ 0.42
0.23
0.15
+ 0.39
0.17
As observed, FedGFM+ consistently outperforms naive federated adaptations of centralized GFM training strategies across all evaluated settings. By integrating the AneDAl AdaDPP modules, FedGFM+ effectively constructs domain-aware semantic priors that enhance generalization to down- stream tasks in heterogeneous domains, even with limited fine-tuning labels. Despite these gains, it is important to note that FedGFM+ still falls short of its own performance under scenarios with abundant labeled data.
30


Reviews:
[{"review_id": "WqcOruWjaO", "reviewer": "Reviewer_GZh8", "summary": "The paper introduces FedGFM, a federated learning framework for training graph foundation models (GFMs). A key challenge identified is knowledge entanglement, where representations from different domains become indistinguishable, limiting downstream performance. To address this, the authors propose FedGFM+, which extends FedGFM with two modules: (1) AncDAI, a global anchor-based domain-aware initialization that encodes domain-specific prototypes before pretraining to ensure separable representations across domains; and (2) AdaDPP, an adaptive domain-sensitive prompt pool used during fine-tuning to enhance domain-specific adaptation via learned prompts from each client. The authors support their method with theoretical analysis and demonstrate improved performance across multiple datasets compared to strong baselines, including domain-specific and federated models.", "questions": "1.\tHow is the model trained to ensure well-separated prototypes across domains?  \n2.\tCan you provide a pseudo-code or algorithm sketch for FedGFM+?  \n3.\tIs it possible to evaluate or estimate the privacy leakage of the method?  \n4.\tHow does the model perform on domains with limited or imbalanced data?  \n5.\tWhat happens to performance if the data is randomly partitioned or if the number of clients increases?", "limitations": "1.\tThe scalability of the method with respect to the number of domains is not discussed.  \n2.\tThe potential privacy leakage due to sharing prototypes and prompts is not analyzed.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**  \n\t1.\tThe idea of applying federated learning to train graph foundation models is important and addresses a fundamental need in graph machine learning.  \n\t2.\tThe paper identifies a key challenge in federated graph foundation models—knowledge entanglement—and supports this with a clear and relevant experimental analysis.  \n\t3.\tThe proposed solutions are backed by both theoretical analysis and strong empirical results.  \n\t4.\tThe overall structure and flow of the paper are logical and easy to follow.  \n\n**Weaknesses:**  \n\t1.\tAlthough Theorem B.1 shows that prototypes are distinguishable even under random initialization, the paper does not provide any quantitative estimation of how separated they are. Including such an estimate, perhaps with a simple example, would help illustrate this point more clearly.  \n\t2.\tA similar concern applies to Theorem B.2. While the theorem suggests that the codebook is more likely to assign distinct tokens to different domains, it’s unclear how likely this is in practice. Quantitative or empirical evidence would strengthen the argument.  \n\t3.\tTheorem B.2 is described as general, but the proof only focuses on cosine similarity. It would be helpful to clarify whether the conclusions hold under other similarity metrics.  \n\t4.\tSince all domain prototypes and prompts (from Sections 4.1 and 4.2) are shared with the server, there should at least be a discussion of the potential privacy risks associated with this sharing.  \n\t5.\tBased on Equation (7), all prompts appear to have the same dimensionality as the node features, which implies that all node features across graphs must have the same dimension. This is a strong assumption and could limit the applicability of the method to heterogeneous graphs.  \n\t6.\tBecause each prompt has the same dimension as the node features and requires its own projection weights, the resulting objective function becomes high-dimensional and potentially expensive to optimize. This can increase both computation and communication costs, especially in a decentralized setting. More justification and analysis of this overhead would be beneficial.  \n\t7.\tWhile the experimental results are thorough and span multiple tasks and domains, the paper does not report any comparison of computation or communication costs.  \n\t8.\tIntroducing synthetic prototypes to the codebook increases its size, thereby enlarging the search space for the optimization in Equation (1). This added complexity should be discussed.", "was_revised": true, "final_justification": "After carefully considering the rebuttal and follow-up clarifications, as well as the discussions, I summarize my assessment as follows: \n \n**Resolved Issues:**  \n\n- The authors added explicit details on the computation and communication costs (48 MB per client per round, ~9.6 s local execution time).\n- They clarified the applicability of the framework to textual-attributed graphs. However, they acknowledged the limitation for more general heterogeneous cases.\n- They explained their partitioning strategy, noting Louvain for subgraph-level FL and uniform random for graph-level datasets.  Although I still believe random partitioning is an important challenge for subgraph-FL problems.\n\n**Unresolved Issues:**  \n\n- Privacy risks: Although the authors now list the exchanged information and potential risks, no quantitative analysis or mitigation strategies are provided. In a federated learning setting, this remains a central limitation.\n- Scalability/robustness: While communication numbers are given, there is no analysis of how these costs scale with more clients, larger models, or longer training. Random partitioning robustness is still not fully explored.\n\n**Weighting:**    \n\n- The paper’s motivation, novelty, and strong empirical results remain its key strengths.\n- The unresolved privacy and robustness aspects are important, but do not fully undermine the technical contribution.\n- Overall, the paper makes a valuable step in defining and addressing the FedGFM paradigm, and I maintain my positive borderline-accept score.", "rebuttal": "We sincerely thank you for your comments, and appreciate that you acknowledged the strengths of our work. Here we initially address all concerns below:\n\n---\n\n\n\n**Question 1**: Theorem B.1 and B.2 lack quantitative estimation or empirical evidence.\n\nThank you for your insightful feedback. We hope to address your concerns.\n\nFor Theorem B.1 (Domain Prototype Distinguishability), prior work has shown that even randomly initialized encoders can produce distinguishable embeddings for multi-domain data, especially in the text domain [1]. This supports our assumption that distinguishability can emerge from structural biases and feature heterogeneity.\n\nFor Theorem B.2 (Semantic Separability of the AncDAI-Initialized Codebook), we have added empirical evidence demonstrating that FedGFM+ encodes multi-domain knowledge into semantically separable codebook representations. Please refer to our response to Question 2 from reviewer cFwZ.\n\n---\n\n\n\n**Question 2**: Whether the theorem B.2 hold under other similarity metrics?\n\nThank you for your comments. Theorem B.2 remains valid for other similarity metrics (e.g., Euclidean distance), as long as the codebook’s query strategy is adapted accordingly. Since the codebook selects discrete tokens based on encoder outputs, the similarity metric used in the query must match the one assumed in the theorem. In our case, Theorem B.2 is based on cosine similarity, which is also the metric used in our codebook querying.\n\n\n\n---\n\n**Question 3**: Lack of discussion about the potential privacy risks associated with shared domain prototypes and prompts.\n\nWe sincerely thank you for your comments and agree that a deeper privacy risk analysis of the shared domain prototypes and prompts in FedGFM+ would provide stronger safeguards. However, we respectfully emphasize that we did not upload any raw local data; instead, it was encoded using a randomly initialized model, which helps reduce the risk of privacy leakage to some extent. It is also important to note that privacy preservation is not the primary focus of this work. Our main contribution lies in defining a new paradigm named federated graph foundation model (FedGFM). In future revisions, we plan to include a theoretical analysis and explore the use of noise injection on the uploaded domain prototypes to enhance privacy protection.\n\n\n\n---\n\n**Question 4**: Eq. (7) results in same feature dimension for  all nodes, which may limit the applicability of the method to heterogeneous graphs.\n\n\n\nThank you for your comments. Equation 7 illustrates the feature augmentation process when applying the pre-trained GFM and personalized prompt pool to a specific target graph during fine-tuning. We would like to clarify that the statement \"all nodes must have the same feature dimension\" does not imply that our method is unsuitable for heterogeneous graphs. On the contrary, our approach is applicable to all textual attribute graphs with textual descriptions for nodes and edges. We ensure a consistent feature dimension by encoding all textual descriptions into unified vector representations using a shared pre-trained language model (e.g., Sentence-BERT, as used in this paper).\n\n\n\n---\n\n**Question 5**: Need more justification and analysis of computation and communication costs about  would prompts and projections.\n\nThank you for your comment. Our prompts are very lightweight and contain far fewer parameters than the backbone network. Although each prompt shares the same feature dimension as the node features, the total number of prompt parameters remains minimal, resulting in negligible additional computation and communication overhead in the federated setting. Specifically, in our experiments, for each client, we use only 3 learnable prompts (shape: 3 × 768) along with a projection matrix (shape: 768 × 3), which is extremely lightweight.\n\n\n\n---\n\n\n\n**Question 6**: How is the model trained to ensure well-separated prototypes across domains?\n\nThank you for your comments. We respectfully clarify that FedGFM+ relies on the AncDAI module to yield distinguishable domain prototypes for graphs from different domains. As outlined in Section 4.1 and Theorem B.1, prototype computation is independent of a pre-trained encoder; it only presupposes that every client employs an identically randomly initialized encoder.\n\n\n\n---\n\n\n\n**Question 7**: provide a pseudo-code or algorithm sketch for FedGFM+.\n\nWe sincerely appreciate your constructive comments. Due to word limit, please see our rebuttal to **Question 1** from reviewer **cFwZ**.\n\n\n\n---\n\n\n\n**Question 8**: How does the model perform on domains with limited or imbalanced data?\n\n\n\nThank you for your constructive feedback. We respectfully remind you that the original manuscript briefly discussed the performance of FedGFM+ on limited data and domain imbalanced data. Specifically:\n\n- ***Limited Data***: In Appendix D of the original manuscript, we evaluated the performance of FedGFM+ in the few-shot setting. As observed, FedGFM+ consistently outperforms naive federated adaptations of centralized GFM training strategies across all evaluated settings\n- ***Imbalanced Data***: We’d like to clarify that our original experiments were conducted on data with imbalanced domains (see Appendix C.1 Table 4 and C.2 Table 5). The graph classification dataset involves significantly more nodes than the node classification dataset, based on local codebook access counts. To address this, we use a fixed batch size and number of batches with random sampling during pre-training, ensuring each client sees similar improvement per training round. Our results show that FedGFM+ handles domain imbalance effectively. We will include a clearer explanation of this training strategy in the revised version.\n\n\n\n---\n\n**Question 9**: What happens to performance if the data is randomly partitioned or if the number of clients increases?\n\nWe sincerely appreciate your constructive comments. Due to word limit, please see our rebuttal to **Question 3** from reviewer **cFwZ**."}, {"review_id": "aBzLHjngkm", "reviewer": "Reviewer_cFwZ", "summary": "This paper investigates Federated Graph Foundation Models (FedGFM) and proposes a unified training framework, FedGFM+, that aims to combine the complementary advantages of Federated Graph Learning (FGL) and Graph Foundation Models (GFM) and mitigate their respective limitations. The authors identify and discuss key limitations in both areas and introduce two novel components (AncDAI and AdaDPP) to mitigate the issue of knowledge entanglement across domains. The experimental evaluation covers eight graph datasets, multiple downstream tasks, and diverse baselines, demonstrating the consistent superiority of FedGFM+.", "questions": "Please refer to the weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths:\n- This paper presents a novel and practically relevant study grounded in real-world scenarios about Federated Graph Foundation Models (FedGFM), proposing a training paradigm that unifies decentralized graph data sources with the goal of learning a generalizable graph foundation model.\n- The paper provides a diagnosis of core limitations in existing Federated Graph Learning (FGL) and Graph Foundation Model (GFM) approaches. Based on these insights, the authors further proposed the FedGFM+ framework, which organically combines FGL and GFM and complementarily addresses the limitations of both.\n- The authors provide an empirical analysis that evaluates FedGFM from the perspective of feasibility, effectiveness, and further experimentally reveal the non-trivial challenge faced by simple federated variants of the naive centralized GFM training strategy, namely, knowledge entanglement.\n- The experiments are comprehensive, and the performance is promising.\n- Overall, the manuscript is well-structured with well-designed illustration figures and tables, and clear technical explanations.\n- The authors have provided the source code of this paper.\n\n### Weaknesses:\n- It would be beneficial for the authors to further clarify the workings of the proposed FedGFM+ framework by providing detailed algorithmic pseudo code. A more formal description would aid readers who are less familiar with FGL or GFM in fully understanding the methodology.\n- In the empirical analysis section, the experiment on the knowledge entanglement problem measured the encoding differences of a simple federated variant of the centralized GFM training strategy GFT for graph datasets in different domains; I suggest that the authors can supplement the encoding results of FedGFM+ to better illustrate that FedGFM+ can indeed alleviate the knowledge entanglement problem.\n- The author divides different datasets into 3 clients. I suggest that the author further explore the performance of tasks divided into other numbers of clients. For example, does dividing Cora into 5 clients affect the performance of the learned graph-based model?", "was_revised": true, "final_justification": "Most of my initial concerns have been addressed properly during the rebuttal. Given the core contributions of this work. I will raise my score to 5.", "rebuttal": "We sincerely thank you for your constructive comments, and we will initially address all of your concerns below:\n\n---\n\n**Question 1**: Lack of formal algorithmic pseudo code for FedGFM+\n\nWe fully agree with this constructive feedback and apologize for the oversight. Here, we present the pseudo code for the federated pre-training phase and isolated fine-tuning phase of FedGFM+ in Algorithms 1 and 2, respectively. We are committed to revising this content to provide a more rigorous and formal exposition of the FedGFM+ approach in the revision stage.\n\n\n**Algorithm 1 - Federated Pre-Training Phase of FedGFM+**\n\n**Input**: Number of clients $K$, each client's local graph $G^k$, initialized global gVQ-VAE parameters $\\Theta^{g}$, each client's local learnable prompt parameter $\\Phi^k$ and projection vector $w^k$, number of communication rounds $T$, number of local epochs $E$, noise scaling factor $\\sigma$.\n\n\n\n**Output**: learned global GFM $\\Theta^{g}$, domain-aware prompt pool $\\rho$ and projection pool $w$\n\n\n\n***for*** each client $k$ in $1,...,K$ ***do***\n\n------------ Initialize local gVQ-VAE with global gVQ-VAE, $\\Theta^k \\leftarrow \\Theta^g$.\n\n------------ compute domain prototype $p^k$ via Eqs. (3), (4)   \n\n------------ upload domain prototype to central server\n\n***end***\n\n\n\ncompute perturbed embeddings and initialize global codebook via Eq. (5)\n\n\n\n***for*** each round $t$ ***in*** $1,...,T$ ***do*** \n\n------------ ***for*** each client $k$ in $1,...,K$ ***do***\n\n------------------------- replace local gVQ-VAE with global gVQ-VAE, $\\Theta^k_t \\leftarrow \\Theta^g_t$\n\n------------------------- ***for*** local epoch $e$ in $1,...,E$ ***do***\n\n------------------------------------- local graph feature prompting via Eq. (6)\n\n------------------------------------- local pre-training to update local gVQ-VAE $\\Theta^k_t$ and learnable prompt $\\Phi^k$, $w^k$ via Eq. (2)\n\n------------------------- ***end***\n\n------------------------- upload gVQ-VAE parameters $\\Theta^k_t$ to central server\n\n------------ ***end***\n\n------------ aggregate local  gVQ-VAE parameters to update global  gVQ-VAE, $\\Theta^g_{t+1} \\leftarrow \\frac{N_k}{N}\\sum_{k=1}^{K}\\Theta^k_t$\n\n***end***\n\n\n--- \n\n\n**Algorithm 2 - Isolated Fine-Tuning Phase of FedGFM+**\n\n**Input**: Target graph $G$, learned global GFM $\\Theta^{g}$, domain-aware prompt pool $\\rho$ and projection pool $w$, number of fine-tuning epochs $E^t$, task head $W$.\n\n\n\n**Output**: fine-tuned global GFM $\\Theta^{g}$ and task head $W$.\n\n***for*** each epoch $e$ ***in*** $1,...,E^t$ ***do*** \n\n------------ target graph feature prompting via Eq. (7)\n\n------------ downstream task-specific optimization to fine-tune global GFM $\\Theta^{g}$ and $W$.\n\n***end***\n\n\n\n\n---\n\n**Question 2**: Missing encoding results to demonstrate how FedGFM+ alleviates the knowledge entanglement problem.\n\nThank you for your valuable feedback. In Section 3, we show the similarity of initial node features and GFT/GFT* embeddings for Cora, WN18RR, and HIV, which highlights the knowledge entanglement issue in FedGFM—where representations from different domains become indistinguishable. As suggested, we further report FedGFM+ embeddings on these datasets. The results are summarized in Table. 1:\n\n**Table. 1: Comparison of the distinguishability of FedGFM+ and baseline embeddings on cross-domain datasets.**\n|                | Cora-WN18RR | Cora-HIV | WN18RR-HIV |\n| -------------- | ----------- | -------- | ---------- |\n| **Raw Feat.**      | 0.53        | 0.41     | 0.49       |\n| **GFT**            | 0.72        | 0.64     | 0.71       |\n| **GFT***           | 0.98        | 0.92     | 0.96       |\n| **FedGFM+ (Ours)** | 0.66        | 0.52     | 0.65       |\n\nAs observed, FedGFM+ produces significantly lower embedding similarity across the three domains compared to GFT and GFT*, indicating that its AncDAI and AdaDPP modules effectively help encode multi-domain semantics into distinguishable representations, thereby mitigating the knowledge entanglement issue.\n\n---\n\n**Question 3**: How does FedGFM+ perform under increased number of clients?\n\nWe appreciate the insightful suggestion. Following your advice, we evaluated FedGFM+ and 5 representative baselines, including FedAvg, Fed-PUB, GCFL+, OFA\\*, and GFT\\* on the Cora, PubMed, WikiCS, OGB-arxiv, FB15K237, WN18RR, HIV and PCBA datasets, each evaluated under both 3-client and 5-client partitions. The results are summarized in **Table. 2**.\n\n **Table. 2: Performance comparison between FedGFM+ and baselines under 5 client partitions.**\n\n|         | Cora  | PubMed | OGB-arxiv | WikiCS | FB15K237 | WB18RR | HIV   | PCBA  |\n| ------- | ----- | ------ | --------- | ------ | -------- | ------ | ----- | ----- |\n| FedAvg  | 67.42 | 70.45  | 66.30     | 64.52  | 65.28    | 65.39  | 60.42 | 63.55 |\n| Fed-PUB | 69.20 | 73.18  | 67.22     | 65.37  | 67.50    | 66.20  | N/A   | N/A   |\n| GCFL+   | N/A   | N/A    | N/A       | N/A    | N/A      | N/A    | 60.49 | 63.83 |\n| OFA*    | 68.42 | 72.35  | 67.40     | 63.28  | 67.25    | 64.82  | 59.45 | 63.81 |\n| GFT\\*   | 69.88 | 73.52  | 66.59     | 66.73  | 68.53    | 65.38  | 58.31 | 64.24 |\n| FedGFM+ | 72.42 | 76.12  | 70.24     | 69.24  | 71.42    | 68.40  | 62.18 | 66.04 |\n\nAs observed, FedGFM+ **consistently outperforms** all baselines. Compared to the 3-client setting in Section 3, all methods show performance drops under 5-client partitioning, which is a common phenomenon in federated learning when data becomes more sparsely distributed (increased client heterogeneity).\n\n\n---"}, {"review_id": "9LihDugxfI", "reviewer": "Reviewer_v6fQ", "summary": "This paper delves into the Federated Graph Foundation Model (FedGFM), which seamlessly integrates two areas within graph machine learning: Federated Graph Learning (FGL) and Graph Foundation Models (GFM). The paper offers a practical overview of the challenges associated with both FGL and GFM in real-world applications, while emphasizing their complementary strengths. FedGFM is introduced as a novel framework for building graph foundation models by harnessing distributed datasets from multiple parties and domains, along with decentralized computational resources. A central issue identified in this context is knowledge entanglement, where global models struggle to separate domain-specific insights into distinct representations. To tackle this, the authors propose FedGFM+, which incorporates the AncDAI and AdaDPP modules. Comprehensive experiments across various datasets validate the practicality and performance of FedGFM+.", "questions": "See weakness", "limitations": "See weakness", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. This work presents FedGFM, a novel framework combining Federated Graph Learning (FGL) with Graph Foundation Models (GFM). It leverages multi-domain data and distributed resources, marking the first study in this area.\n\n2. FedGFM+ builds on FedGFM, addressing communication challenges and knowledge entanglement with robust empirical methods.\n\n3. Experiments on eight datasets across tasks validate its effectiveness, with baselines including GNNs, FGL variants, and federated GFM adaptations.\n\nWeakness:\n\n1. In the context of VQ-VAE or Graph-VQ-VAE, the concept of vocabulary collapse [1, 2] refers to a situation where only a small subset of tokens in the codebook is frequently accessed, while many are rarely or never used. How does this differ from the knowledge entanglement issue discussed in this paper? Please elaborate on the distinction.\n\n2.  What is the reason for structuring the FedGFM paradigm’s pre-training phase as iterative rounds of federated communication, while the fine-tuning phase is carried out independently by each client?\n\n3. Provide further explanation regarding the fairness of comparisons made between the FGL baselines and FedGFM+.\n\n4. Could you clarify the meaning and implications of Figure 2 in more detail?\n\n[1] Zheng, C., & Vedaldi, A. (2023). Online clustered codebook. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 22798-22807).\n[2] Wang, Z., Zhang, Z., Chawla, N., Zhang, C., & Ye, Y. (2024). Gft: Graph foundation model with transferable tree vocabulary. Advances in Neural Information Processing Systems, 37, 107403-107443.", "was_revised": true, "final_justification": "Most of my concerns have solved", "rebuttal": "We sincerely thank you for your constructive comments, and initially address all of your concerns below:\n\n---\n\n**Question 1**: What is the difference between knowledge entanglement in FedGFM and vocabulary collapse in VQ-VAE?\n\nWe appreciate the opportunity to clarify the distinction between the two challenges.\n\n- ***Knowledge Entanglement*** is a unique challenge in FedGFM caused by the lack of centralized optimization over heterogeneous, domain-specific data. This leads to different semantics being mapped to the same token positions, resulting in conflicting updates during server-side aggregation. For example, token ID 1 may represent financial concepts on one client and biomedical knowledge on another. These conflicts degrade the quality of the aggregated global model and cause domain confusion downstream.\n\n- ***Vocabulary Collapse*** is a common issue in VQ-VAE, where only a few codebook tokens dominate usage while others remain inactive. This self-reinforcing dynamic occurs because tokens slightly closer to inputs are selected and updated more often, further increasing their dominance. Consequently, token utilization decreases and representational diversity suffers.\n\n\n---\n\n**Question 2**: Why is the federation mechanism not applied during the fine-tuning stage?\n\nThank you for your comment. FedGFM consists of two stages. During pre-training, clients act as data holders, collaboratively training a unified and generalizable graph foundation model via federated communication over distributed, domain-specific datasets. In contrast, during fine-tuning, clients serve as model users or task adapters, working independently without federation. Notably, the fine-tuning data often differs from the pre-training data, reflecting practical deployment scenarios where distributed coordination is **unnecessary or impractical**. Based on the above considerations, federated communication is employed during the pre-training stage, but not during the fine-tuning stage.\n\n\n---\n\n**Question 3**: Explain the fairness of the comparison between FGL baselines and FedGFM+\n\n\nThank you for the question. We appreciate the opportunity to clarify the fairness of our comparison.\n\nAll methods (including FedGFM+ and the FGL baselines) are evaluated under the same federated protocol, with consistent feature pre-processing, identical data splits, and shared optimization settings. The only difference lies in the modeling paradigm: FedGFM+ adopts a federated pre-training and fine-tuning framework, while FGL baselines follow a task-specific federated training approach. This paradigm shift allows FedGFM+ to enable generalized collaboration across domains and tasks (see Lines 64–78), whereas FGL baselines are inherently limited to single-task, task-bound collaboration (see Lines 38–50).\n\n**In summary**, the performance **superiority of FedGFM+ arises from its modeling innovations**, not from any unfair advantage in experimental setup. We therefore consider the comparison between FedGFM+ and FGL baselines to be **fair and meaningful**.\n\n---\n\n**Question 4**: Clarify the meaning of Figure 2 in more detail.\n\nFigure 2 is designed to illustrate the core challenge of the FedGFM paradigm, named knowledge entanglement.\n\nSpecifically, we consider three datasets from distinct domains and tasks: Cora (citation network, node classification), WN18RR (knowledge graph, edge classification), and HIV (molecular graph, graph classification). As shown in Figure 2(a), these datasets exhibit significantly different structural patterns, while Figure 2(b) ('Raw Feat.') shows notable feature disparities across domains.\n\nWe then compare GFT and its naive federated variant GFT\\*:\n\n- **GFT**, trained centrally across all datasets (jointly optimized);\n\n- **GFT\\***, trained in a decentralized manner following the FedGFM setting.\n\nAs shown in Figure 2(b), the embeddings produced by GFT ('GFT Emb.') preserve clear domain-specific separability, while those produced by GFT\\* ('GFT\\* Emb.') collapse into indistinguishable representations.\n\nThis result highlights a fundamental limitation in the FedGFM setting: knowledge entanglement, where multi-domain knowledge is encoded into indistinguishable representations, thereby limiting downstream adaptation.\n\n---"}, {"review_id": "DKAiqHVInO", "reviewer": "Reviewer_PsLJ", "summary": "This paper is trying to solve question containing federated graph learning and graph foundation models. To address the knowledge entanglement challenge, FedGFM+ is proposed to solve it from the global and the local perspective. The authors conduct cross-task and cross-domain experiments to demonstrate the effectiveness of the approach.", "questions": "1, Could you please clarify the motivation of the proposed approach? From the methodology to the applications.\n\n2, Could you please identify which individual technique is innovatively proposed in this paper?\n\n3, Although I read that you categorize the baselines roughly, could you please clarify a little bit more about how you select the baselines?", "limitations": "1, I suggest the authors to think deeper about the KEY motivation of the proposed approach and present it more clearly.\n\n2, Also, one or two focused points deserves to be explored comprehensively.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n(1) Figure 1 is a good diagram to demonstrate the comparison.\n\n(2) They conduct extensive experiments to verify their framework.\n\nWeakness:\n\n(1) The motivation is not clear. The combination of two terms seems not convincing as the motivation.\n\n(2) The authors try hard to include too many aspects in one paper, which confuses the reader a lot. I understand that the data heterogeneity, task heterogeneity, multi-domain data isolation, cross-silo storage and computation neglect are the limitations for the two terms respectively. But the statements from Line 64 to 78 are far-fetched and strained.\n\n(3) This framework contains multiple modules. But most of them are not novel if we treat them individually, for example, the prototype and personalized prompt.", "was_revised": true, "final_justification": "The authors have addressed my concerns of baseline selection. But I am concerned about the KEY novelty of each module and the potential issues of privacy leakage. I do not think I can raise my score to positive at this moment.", "rebuttal": "We sincerely thank you for your comments, and hope to address your concerns as follows:\n\n\n---\n\n**Question 1**: The motivation of the FedGFM paradigm (combining FGL and GFM) is not convincing.\n\nWe respectfully highlight that FedGFM is not a superficial field combination, but a practical paradigm that leverages the complementary strengths of FGL and GFM.\n\n- The advantage of FGL is that it can achieve cross-silo data collaboration, computing and storage resource collaboration under the scenario of privacy protection. However, its disadvantage is that the scope of collaboration is extremely limited (across small number of fields or even different shards of a single dataset) due to the data heterogeneity (including features, space, topology) and task heterogeneity (bound to specific model architecture, algorithm process, and optimization goals).\n\n- On the contrary, the advantage of GFM is that the language model encoding unifies the data format, while the pre-training mechanism is task-format-agnostic. The disadvantage is that the actual training of GFM faces the difficulty of centrally collecting multi-domain datasets (graphs from different domains are collected by different institutions and are not allowed to be made centralized and public) and the opportunity cost of wasting multi-party cross-silo computing and storage resources.\n\n**Why centralized GFMs are not enough to fully meet real-world needs**: Existing centralized GFM studies are typically trained on datasets from a small range of domains (e.g., 4 domains for OFA[1] and GFT[2]), which is significantly fewer than those used in vision or NLP foundation models. However, building a generalized graph foundation model likely requires access to graphs from a much broader range of domains. In reality, such data (e.g., financial networks, disease graphs) are often privacy-sensitive and siloed across institutions, where legal, ethical, and commercial constraints prohibit direct data sharing. As a result, aggregating these graphs for centralized GFM training is not only impractical but often infeasible.\n\nFedGFM addresses this limitation by aligning GFM’s task-agnostic modeling capabilities with FGL’s decentralized training infrastructure, enabling scalable foundation model pre-training across siloed, heterogeneous, and sensitive domains, which is the core motivation behind FedGFM.\n\n---\n\n**Question 2**: The various modules of the proposed framework FedGFM+ lack technical novelty.\n\nWe respectfully argue that the core novelty of this work lies in **paradigm innovation**: to the best of our knowledge, this is the first work to identify and systematically characterize the complementary relationship between FGL and GFM, leading to the proposal of a new paradigm, FedGFM. We further conduct an in-depth empirical analysis to uncover a paradigm-specific challenge named knowledge entanglement.\n\nWithin this new paradigm, the **novelty of technical design** should be evaluated in terms of how our method addresses the above challenge. While certain components like prototypes and personalized prompts have been widely used in various machine learning research, our design motivations and functionalities are uniquely aligned with the needs of FedGFM, rather than being direct adaptations from any FGL, GFM, or VQ-VAE studies. Below, we provide a detailed clarification of the novel aspects of each module:\n\n\n- ***Anchor-Based Domain-Aware Initialization (AncDAI)***: In FedGFM, where centralized optimization of multi-domain knowledge is infeasible, clients from different domains may encode distinct semantics into the same codebook tokens. When these locally updated codebooks are aggregated on the server, such semantic conflicts can compromise the global representation quality. To mitigate this, AncDAI leverages each client’s local domain prototype as a semantic anchor to initialize the global codebook, encouraging early separation of domain-specific semantics and reducing conflict during aggregation. **This design is unique to FedGFM paradigm**, as existing GFM or VQ-VAE models operate in centralized settings, where token semantics can be aligned through joint optimization, eliminating the need for such initialization strategies.\n\n\n\n\n\n- ***Adaptive Domain-Sensitive Prompt Pool (AdaDPP)***: To mitigate knowledge entanglement in FedGFM from the local perspective, AdaDPP learns domain-specific prompts during pre-training and adaptively injects them during fine-tuning. These prompts provide domain-aware guidance that modulates the shared encoder’s behavior, enabling localized adaptation while reducing cross-domain interference. Unlike prior GFM or FGL methods that rely on a shared encoder without domain-specific adaptation, AdaDPP introduces explicit domain-aware modulation to preserve semantic boundaries across domains.\n\n---\n\n**Question 3**: Need more explanation about the baseline selection strategy and the reason behind it.\n\nWe sincerely apologize for any possible oversights. Here we provide further explanations about the reasons for the setting and selection of each category baseline:\n\n- ***Isolated Supervised Learning***: These models are trained independently on each client without collaboration, serving as the performance lower bound under non-federated and non-pre-training settings. They help assess the benefit of FedGFM over traditional per-graph training. Model choices are also aligned with task types: GCN, GAT, and GraphSAGE for node/edge classification, and GIN for graph classification.\n\n\n\n- ***FL/FGL Methods***: These approaches support only limited collaboration across domains and are often tailored to specific downstream tasks, limiting their generality. In contrast, FedGFM enables broad, scalable collaboration across heterogeneous domains and tasks. The performance comparison with FL/FGL baselines further highlights the effectiveness of generalized, task-agnostic pretraining over narrow, task-specific federated learning.\n\n- ***Federated Variants of Centralized GFM***: These methods adapt centralized GFM architectures to federated settings by performing multi-round parameter aggregation across clients. While they enable federated pretraining, they do not account for the knowledge entanglement challenge arising from cross-domain heterogeneity. In contrast, FedGFM+ is explicitly designed to mitigate such entanglement, and comparisons with these baselines highlight its superior ability to preserve domain-specific semantics and improve performance under the FedGFM paradigm.\n\n\nFinally, we kindly remind you that detailed descriptions of each baseline catagories and specific methods can be found in Appendix C.3. We promise you that we will add these supplementary content to the appendix during the revision phase.\n\n\n---\n\n**References**\n\n[1] Liu, H., Feng, J., Kong, L., Liang, N., Tao, D., Chen, Y., & Zhang, M. One For All: Towards Training One Graph Model For All Classification Tasks. In The Twelfth International Conference on Learning Representations.\n\n[2] Wang, Z., Zhang, Z., Chawla, N., Zhang, C., & Ye, Y. (2024). Gft: Graph foundation model with transferable tree vocabulary. Advances in Neural Information Processing Systems, 37, 107403-107443."}]

Author rebuttal:

Meta-review: After carefully reading the submission, the reviewers' comments, and the authors' detailed rebuttal, I recommend acceptance.

The paper presents FedGFM, a novel paradigm for building GFMs by leveraging decentralized data and computational resources from multiple clients. The authors identify a core challenge in this setting, which they term "knowledge entanglement," where a globally aggregated model struggles to maintain domain-specific representations, thereby hampering downstream performance on diverse local tasks. To address this, the paper proposes an enhanced framework, which introduces two novel components, AncDAI and AdaDPP. These are designed to ensure that representations from different clients remain distinguishable during the pretraining stage and to help the model adapt more effectively to local downstream tasks during fine-tuning. Many reviewers acknowledged the novelty of the problem formulation and the proposed method. The submission is backed by a comprehensive and convincing experimental evaluation, demonstrating consistent, superior performance across diverse datasets and tasks against a well-chosen set of baselines.

However, one reviewer found the motivation for combining the two fields somewhat strained and the overall presentation potentially confusing due to the number of concepts involved. The narrative connecting the high-level motivation to the specific technical solution could be sharpened to be more compelling for all readers. Additionally, the privacy discussion, added during the rebuttal, would be more impactful if it were fully integrated into the paper with a more formal treatment of potential vulnerabilities. Similarly, a quantitative analysis of the computation and communication overhead of the proposed modules would help practitioners assess the method's real-world feasibility.
Decision: Accept (poster)