GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning
Authors: Haonan Yuan
Qingyun Sun
Junhua Shi
Xingcheng Fu
Bryan Hooi
Jianxin Li
Philip S. Yu
Venue: NeurIPS 2025 poster ()
Keywords: graph foundation models
multi-domain graph pre-training
graph transfer learning
graph prompt fine-tuning
Primary area: deep_learning
TL;DR: We propose a novel generative graph vocabularies for robust GFM fine-tuning framework named GRAVER that tackles fine-tuning instability via generative augmentations with transferrable patterns.

Abstract: Inspired by the remarkable success of foundation models in language and vision, Graph Foundation Models (GFMs) hold significant promise for broad applicability across diverse graph tasks and domains. However, existing GFMs struggle with unstable few-shot fine-tuning, where both performance and adaptation efficiency exhibit significant fluctuations caused by the randomness in the support sample selection and structural discrepancies between the pre-trained and target graphs. How to fine-tune GFMs robustly and efficiently to enable trustworthy knowledge transfer across domains and tasks is the major challenge. In this paper, we propose **GRAVER**, a novel **G**enerative g**RA**ph **V**ocabulari**E**s for **R**obust GFM fine-tuning framework that tackles the aforementioned instability via generative augmentations. Specifically, to identify transferable units, we analyze and extract key class-specific subgraph patterns by ego-graph disentanglement and validate their transferability both theoretically and empirically. To enable effective pre-training across diverse domains, we leverage a universal task template based on ego-graph similarity and construct graph vocabularies via graphon-based generative experts. To facilitate robust and efficient prompt fine-tuning, we grave the support samples with in-context vocabularies, where the lightweight MoE-CoE network attentively routes knowledge from source domains. Extensive experiments demonstrate the superiority of GRAVER over **effectiveness**, **robustness**, and **efficiency** on downstream few-shot node and graph classification tasks compared with **15** state-of-the-art baselines.

GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning
Haonan Yuan', Qingyun Sun'; Junhua Shi', Xingcheng Fu², Bryan Hooi
Jianxin Li Philip S. Yu
SKLCCSE, School of Computer Science and Engineering, Beihang University
2 Key Lab of Education Blockchain and Intelligent Technology, Guangxi Normal University School of Computing, National University of Singapore *Department of Computer Science, University of Illinois, Chicago
{yuanhn, sunqy, ,hi jhijuna,, 1ijxx/buaaa..... edu. fuxc0gxnu. edu.eduu... cn, bhooi bhooi@comp mus edu. psyu@uic.edu
Abstract
Inspired by the remarkable success of foundation models in language and vision, Graph Foundation Models (GFMs) hold significant promise for broad applicability across diverse graph tasks and domains. However, existing GFMs struggle with unstable few-shot fine-tuning, where both performance and adaptation efficiency exhibit significant fluctuations caused by the randomness in the support sample selection and structural discrepancies between the pre-trained and target graphs. How to fine-tune GFMs robustly and efficiently to enable trustworthy knowledge transfer across domains and tasks is the major challenge. In this paper, we propose GRAVER, novel Generative gRAph VocabulariEs for Robust GFM fine-tuning framework that tackles the aforementioned instability via generative augmentations. Specifically, to identify transferable units, we analyze and extract key class-specific subgraph patterns by ego-graph disentanglement and validate their transferability both theoretically and empirically. To enable effective pre-training across diverse domains, we leverage a universal task template based on ego-graph similarity and construct graph vocabularies via graphon-based generative experts. To facilitate ro- bust and efficient prompt fine-tuning, we grave the support samples with in-context vocabularies, where the lightweight MoE-CoE network attentively routes knowl- edge from source domains. Extensive experiments demonstrate the superiority GRAVER over effectiveness, robustness. and efficiency downstream few-shot node and graph classification tasks compared with 15 state-of-the-art baselines.
Introduction
Graph-structured data is pervasive across a wide range of domains. including social networks [13, 127]. recommendation systems 14, 100], and bioinformatics [50. 17]. Graphs are capable of modeling complex relationships and attributed interactions between entities, making them powerful abstraction for solving real-world problems, such as fraud detection [60, 128], molecular property prediction [70, 135], and protein interaction modeling [34, 15]. While foundation models such GPTs [1] and Vision Transformers [38] have achieved impressive generalization capabilities in language and vision through acaeescale pre-training, they struggle when applied directly to graphs due their Euclidean nature, structural heterogeneity, and task diversity inherent graph data [52, 46, 63, 37]. To address this gap, Graph Foundation Models (GFMs) [64, 75, 149] have emerged with the goal of enabling general-purpee learning on graph-structured data. Instead of
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Multi-Domain Pre-training Phase
Few-Shot Cross-Domain Fine-tuning Phase
triangle
Bad Case
Loss
Case Good Casc
house
Good Case
Accuracy
square
triangle
ladder
random selection + high deviation
Figure 1: Case study of the fine-tuning instability. When support samples share structural patterns with pre-trained graphs (e.g., triangle), fine-tuning is efficient and stable (Good Case). In contrast, mismatched patterns (e.g., ladder) lead poor convergence and lower accuracy (Bad Case). Loss and accuracy curves illustrate that random support selection causes high performance deviation, highlighting the need for structure-award aware augmentation.
relying on task- susk-specific deep graph models, ideal GFMs are trained once on diverse graph datasets and then adapted with minimal supervision to a variety of downstream tasks [76, 5]. Such a trretrain-then-fiettee
paradigm [86, 23, 42] holds the promise of unifying graph learning across domains by providing a reusable model architecture that generalizes structural knowledge at scale [58, 83, 122, 136, 126].
Existing research on GFMs has addressed prominent challenges like architecture scalability [78, 42], domain alignment [47, 136], prompt design [83, 45), etc. Nevertheless, GFMs encounter a more fundamental and pervasive issue: robustness and efficiency of few-shot prompt fine-tuning. Specifically, the existing GFMs exhibit significant instability and inefficiency, whereby fine-tuning performance fluctuates heavily due randomness in the selection of limited support samples and structural discrepancies between pre-trained and target graph domains. We conducted an illustrative case study in Figure highlight this issue. A synthetic cross-domain node classification setting is constructed with motif- graphs. The findings reveal consistent trend: when the randomly selected support nodes exhibit structural patterns similar to those in pre-training (e.g., triangle A. fine-tuning converges faster and yields higher accuracy (Good Case). In contrast, the mismatched structures (e.g., ladder > lead to slower convergence and degraded performance (Bad Case). This phenomenon is intuitive and commonly observed in practice. Such instability severely undermines the practical applicability and reliability of GFMs in yeal-world scenarios, where dependable knowledge transfer and rapid adaptation to new tasks and domains are essential.
Research Question: How can Graph Foundation Models be fine-tuned robustly and efficiently under random support sets to enable trustworthy knowledge transfer across diverse domains and tasks?
Recent studies attempt to improve GFM's adaptation ability. One line of work focuses on pre-training with transferable patterns [83, 7, 148, 97]. The most relevant GFT [98] proposes a computation- tree-based vocabulary to enable pattern reuse across domains. However, the proposed vocabulary is limited to tree-shaped structures, restricting transferability in multi-domain settings. Another line of research emphasizes generative augmentation [137, 84, 10, 96], which interpolates structures to simulate new training examples. Yet, they often generate task-agnostic structures that misalign with target domains and do not condition on the few-shot support samples, limiting their informativeness. A third group of works [122, 120, 199 introduces modular pre- pre-training with weighted source tokens as prompts to facilitate domain- aware transfer. While effective in improving cross-domain adaptation, these strategies still fall short in scenarios where only a handful of labeled target samples are available. Notably, none of these aforementioned paradigms simultaneously address the dual challenges of generating transferable patterns and ensuring stable fine-tuning under arbitrary few-shot support sets.
To tackle this dilemma, we propose GRAVER, novel Generative gRAph VocabulariEs for Robust GFM fine-tuning framework. GRAVER directly addresses the instability inherent in few-shot fine- tuning through in-context subgraph augmentations. Specifically: 1 We first analyze and validate the key transferable patterns from an ego-graph disentanglement perspective. rigorously exploring their transferability thcoretical foundations and empirical utility across multiple domains. @ By leveraging this insight, we establish adaptive graph vocabularies through graphon-based generative experts in the multi-domain pre-training stage. @ At the final fine-tuning stage, GRAVER enhances randomly selected support samples by embedding context-specific generative vocabularies guided by lightweight MoE-CoE network. which selectively routes pertinent structural and semantic knowledge from relevant source domains and prevents negative transfer. Our contributions are:
2

---

We propose novel trustworthy GFM with robust and efficient prompt fine-tuning under multi- domain pre-training and cross-domain adaptation. To the best our knowledge, this the first GFM with generative graph vocabularies to in in-context augment support sets for stable adaptation.
We design graphon-based generative experts to synthesize structure-feature vocabularies aligned with transferable subgraphs, and introduce a MoE- routing mechanism to dynamically assemble source- -orree knowledge for context-aware augmentation during fine-tuning.
Extensive experiments demonstrate its superior effectiveness, robustness, and efficiency on down- stream few-shot node and graph classification compared with 15 ofate-of-the-art baselines.
2
Related Work
2.1 Multi-Domain Pre-trained Graph Foundation Models (Appendix G.1)
Graph foundation models have made rapid progress by using self-supervised pre-training to extract transferable patterns from large-scale graphs [57, 29, 104]. Pre-training tasks such as node masking, edge prediction, and contrastive learning have enabled methods like GCC [69]. GPT-GNN [30], GPPT [82], L2P-GNN [61], and others [105, 36, 109, 8, 7, 113] to serve as the strong initializations for downstream tasks. However, most of these methods assume the structural or semantic similarity between pre-training and target domains [32, 135], making them sensitive to distribution shifts.
To address this, recent studies have investigated cross-domain graph learning, aiming to transfer knowledge between dissimilar domains. Models such as GCOPE [136], OMOG [54], PGPRec [1 12], UniAug [88], and UniGraph [23] focus on learning domain-invarian representations, While effective for single-soure source transfer, these methods often struggle to generalize across multiple heterogeneous domains and may rely task-specific assumptions that limit scalability.
Toward multi-domain foundation models, recent work explores unified pre-training strategies across diverse domains. OFA [54] and HiGPT [87] use LLMs to encode node information as text, which improves alignment but restricts applicability to text-attributed graphs [95, 85]. GCOPE [136] introduces virtual domain bridges but lacks semantic alignment. MDGPT [123] incorporates domain tokens and SVD-based projection, though it does not ensure semantic consistency across domains.
2.2 Graph Foundation Models Fine-tuning with Prompt (Appendix G.2)
Fine-tuning the pre-trained graph foundation models is a widely adopted strategy to adapt general representations to downstream tasks [130, 84, 139, 108, 85, 115, 33, 35, 129]. Traditional methods typically update full model or selected components, which can be effective but often incur high costs.
To improve fine-tuning efficiency, warameter-efficien fine-tuning (PEFT) techniques update a small set of parameters [143, 89, 19, 106] or introduce lightweight modules [49, 147, 110]. These ap- proaches preserve most pre-trained knowledge while reducing computational load. However, their generalization across domains remains limited due to the difficulty of handling domain shifts.
Prompt-based fine-tuning offers an alternative by conditioning the model on learned input prompts rather than altering parameters [55, 141, 140, 11]. In graph learning, prompt tuning leverages structured embeddings to guide the model toward task-specific patterns [82, 145, 77, 132]. achieving better robustness and adaptability with minimal updates. Based on this paradigm, numerous graph prompt methods have been developed [82, 83, 58, 18, 145, 14, 31, 123, 118, 149], which incorporate learnable prompts highlight structural or semantic cues, improving transferability across tasks.
2.3 GFM Pre-training with Transferable Patterns (Appendix G.3)
Graph Foundation Models aim to capture structural patterns that generalize across diverse domains and tasks [69, 142, 83, 7, 148, 97, 96]. A key challenge is defining transferable subgraph units for pre-training, since graphs do not contain obvious tokens like words or image patches.
One approach addresses this by identifying common substructures such as motifs or computation trees as a reusable graph vocabulary [142, 146, 47, 138, 144, 97, 117]. For example, GFT [98] encodes each node's computation tree and discretizes it through vector quantization. This enables the model to represent graphs in terms of recurring structural units, which improves generalization across domains and mitigates knowledge conflicts during transfer.
3

---

Prompt
Vocabulary Bank
Graph Learnel = (Gmiph Neural Networks)
LEI D
samples
G7
Gruphon Experts
Turget Assemble Vecabulary
[14] 20 ())])
augnent
structure graphon W.A [0, 1/2 [0.1]
(uqport sample)
Souren Domaie Aligumecat
20de Aa blgieio Paeeddr Phenng Nownd howeerl
Retrieved Raw Test
Durumarr Normitsan this edeggay soeo borooaen paws aniicil....
LLMa-nha Text
Routing Network o(-)
LLMI-
"roor"
(AA))
(b) feature graphon
"huppry"
W [0. [] R
WA
"affic"
Domain
Co-purchase Domain
Web Link Domain
x
(1) Multi-Domain Pre-training
(2) Vocabulary Modeling
(3) Cross-Domain Fine-tuning
Figure 2: Framework of GRAVER. (1) aligns semantics via LLM-enhanced raw text, and disentangles
ego-graph into transferable patterns. (2) models structure- feature tokens with graphon experts. (3)
assembles class-aware vocabularies to augment support samples for robust cross-domain fine-tuning.
Another approach focuses on mining frequent motifs during pre-training [135, 4, 133]. MICRO- Graph [133] trains GNNs using motif-based contrastive objectives, while MGSSL [135] reconstructs motif segments to learn interpretable structural blocks, achieving strong performance in molecular prediction tasks. Other methods learn universal structural patterns from synthetic graphs [113].
2.4 GFM Fine-tuning with Generative Augmentations (Appendix G.4)
To improve the adaptability of pre-trained graph models, recent work explores generative augmenta- tion techniques that synthesize new data aligned with the target graph distribution [30, 137, 84, 115, 86, 111]. Unlike traditional perturbations on edges or features, these approaches aim to produce structurally meaningful examples support robust fine-tuning.
Recent efforts have explored generative strategies to enhance the robustness and adaptability of graph foundation models. One line of work leverages graphons, continuous funetions that define graph distributions [2. 68, 72, 12, 71, 84]. For instance, G-Mixup [21] interpolates between class- wise graphons to produce realistic hybrid graphs that improve generalization and noise robustness. Another direction introduces generative fine-tuning objectives to mitigate the mismatch between pre-training and downstream distributions [48, 84, 9, 53], such G-Tuning [84], which preserves the generative structure of downstream graphs to avoid overfitting. In addition, subgraph-leve augmentations [3, 131, 41, 101, 24, 22] have been proposed help models decouple subgraph semantics from global context; for example, [57] replaces subgraph environments during training to improve interpretability and robustness in molecular tasks. Complementing these efforts, mixture- of-experts architectures(27, 94, 59, 99, 20, 123] such as GraphMETRO [99] deploy multiple expert networks specialized on distinct structural regimes, with routing mechanisms adaptively select experts for diverse input distributions.
3 Preliminary
Notations. A graph is denoted as G = (V,E), where V is the node set and E is the edge set. Let A N be the adjacency matrix and X NR be the node fcature matrix, where = | is the number of nodes and d; denotes input feature dimension. H means hidden embeddings of X.
Pre-training and Few-shot Fine-tuning, Given source graphs {G$) G from multi-domains (DS) E DS with their labels (Y'} )oS the pre-training goal is to train graph learner h = on multi-domain graph datasets composed of a GNN-based graph encoder ff and discriminator g, after which the pre-training parameter @* is frozen. During fine-tuning, given a set of target graphs (GT) E g from target domain {DT} E ET (seen or unseen) with m randomly selected labels "YTEY" (support set) under the m-shot setting (m & - No), the goal is to adapt the pre-trained GFM to predict the labels of the unlabeled samples (query set) correctly by learnable graph prompts.

---

Proposed GFM Framework: GRAVER
In this section, we elaborate on the proposed GRAVER with its framework illustrated in Figure 2.
4.1 Pre-training with Transferable Graph Vocabulary
A central challenge in constructing transferable graph vocabulary for robust GFM fine-tuning lies in identifying subgraph patterns (graph vocabulary) that generalize across tasks and domains.
Molti-Domain Alignment. Graphs in different domains present heterogeneous features with incon- sistent dimensions and semantics. To establish a unified representation space, we introduce a set of shared aligners that consequently unify dimension- and semantic- saiae features. Given graph G = (V,, E,) (i & n)) with feature matrix X E ERNxd from source domain, the aligners perform:
X E R W (F(x:) |LL ()))): for all G E {(G),
(1)
where LLM(-) implemented by retrieving ceaturerrelttd raw texts and enhancing class-awar semantics by large language models (GPT-4 and BertTokenizer) [47]. FF implemented by truncated SVD [80] aligns feature dimensions, followed by learnable projection W, that normalizes semantics.
Factor-aware Ego-Graph Disentanglement. Motivated by the observation that localized graph patterns often emerge from multiple latent semantic factors [44], such social affiliation, topic preference, structural role, etc., each factor-specific subgraph forms a distinct semantic unit, analogous vocabulary in linguistics. Treating these units holistically during pressage-passingg and aggregation may obscure the unique semantics of each vocabulary, significantly weakening the discriminability and transferability of pre-trained knowledge. To explicitly identify transferable patterns and prepare a "vocabulary bank" for modeling, we decompose the ego-graph g. of a given node u into K factor- aware subgraphs (vocabularies) with a differentiable neighbor soft-routing mechanism [62, 56].
Denote the multi-channel encoder as fe: g. + {h E ] where each channel & encodes a disentangled factor of the ego-neighbors. The i-th soft-routing iteration aims to find an attention:
ae x Softmaxx ((hale u,k v.k )/T); and h (()) o (W J X bx) (WT (W/
(9)
h((0))
s.t. Ex=
K
bx)ll2, iff
+ bx) >> p.
(3)
where r is the temperature, Wa and b are projections. The routed neighbors form K vocabularies:
s(t)) K (() { j==1"
Sa,k
s(t)
(y(t) u,k e())), Uu fo N(u) (22
(4)
The updated h is the concatenation of that from each disentangled ego-graph over iterations:
K
S(T)
h u,k
h + Σ eEv(!)
: S(1)
(e)
S(t)
(5)
Promotion. To ensure each channel captures distinct semantic factors, we derive mutual information regularizer between each disentangled vocabulary, which we implemented by leveraging outrassive-bassd variational upper bound with the InfoNCE estimator [67]]
RM =>
T(hS h..))
Proof C.I
Enes
log (Softmax, ((h //3)))) ) (6)
where B is batch of sampled nodes from source domains. Intuitively, RVn constrains vocabularies to be non-aligned across nodes, thus discouraging shared semantics and keeping distinguished attributes.
Vocabulary Transferability Justification. To investigate how different vocabulary combinations contribute to pre-training, we evaluate their transferability by measuring the stability of encoder f-
Proposition (Vocabulary Transferability). Given any nodes u, v, and assume || [l.
x:
& e.
The semantic discrepancies A
f(g
)ll is upper bounded by:
[a] Proof C22
min HEESK
Σ K
h
[6] Proof C.3
EVK
CoLWLL
4pT
where II denotes a permutation over channels, ()(-) is slack function controls bound tightness. Co. Tow, and byg are Lipschitz constants for activation function a, weight matrix W, and cosine similarity (..) (inner product), respectively. p is the lower bound of T---orm in Eq. (3).
5

---

Proposition I states that the upper bound of semantic discrepancy is governed by a specific combina- tion of vocabularies. This demonstrates that disentangled ego-graphs can function as independent and minimal semantic units, facilitating effective knowledge transfer and providing us with theoretical foundations for vocabulary-based generative augmentations.
Objectives of Sell-supervised Pre-training. Inspired by prior studies [58], we adopt fully self- supervised contrastive link prediction as the pre-training task. To address task heterogeneity, we build upon a universal task template based on ego-graph semantic similarity, which unifies both link prediction with node- or graph-level classification tasks under shared framework.
Specifically, we construct a set of quadruples (u, v", ; _ .u) sampled non redundannyyy from {G*), where vt, v are the positive and negative neighbor that link (do not link) to u. The binary label y denotes the link existence and is determined by the pairwise similarity discriminator The objective is encourage higher semantic similarity between and its positive neighbor h while suppressing that with the negative neighbor he- The training objective formalized as:
MLP((
exp(g(he )/T)
Spr O H =
Σ log (a,ut,o-)
+ Σ,
RMI (8)
exp(o(h,, h )/T) + explo(he h (/-)//)
where À is the trade-off hyperparameter. O' is fixed once pre-training converges.
4.2 Vocabulary Modeling with Generative Graphon Experts
We denote each vocabulary extracted from the pre-training dataset G tuple (A,, D.. GO), where A; and X; represent its adjacency and node feature matrix, 3/ is class label. To enhance the generalization of disentangled vocabularies, enabling efficient and robust generative augmentation during fine-tuning we establish a "vocabulary bank" equipped with two complementary graphon experts: WA dedicated to modeling vocabularies by structure tokens and W to feature tokens.
Structure Token Modeling. For each class e E {1, ,C}, we collect adjacencies (A, ( each disentangled vocabulary, and approximate the structure token via nonparametric graphon:
No for
1 No No E (1)))
W:A: [0, 1J2 + [0, 1], W=
(v)],
(9)
where n;: [0, 1] + (1, (1..... |4,l] measure- peeasure-preserving mapping estimated via empirical node ordering (e.g., node degree). |A.] denotes the number of nodes in this vocabulary structure token, and we align each vocabulary with the same number of nodes by utilizing node padding [21, 7].
Feature Token Modeling, Simultancously, for each class c, we estimate feature token graphon to model the distribution of node features conditioned on latent positions (coordinates):
W [0, 1] + RR, W: N E No
(mr().):
(10)
where m, is shared with W:A. This joint token-level parameterization establishes critical foundation for a hierarchical collaboration mechanism introduced in the next section. By coherently generating both structure and feature tokens, the unified modeling facilitates more stable knowledge transfer.
Conditional Vocabulary Generation. linguistics, a word typically consists of root and affixes, where the root conveys semantics, and the affixes determine its grammatical or functional properties. In similar vein, graph vocabulary follows comparable "root + affix" paradigm: root (structures) determines its structural semantics (triad, grid, tree, etc.), and affix (features) represents its domain- specific propertics. For instance, a triad can indicate a stable social relationship, but in molccular contexts, it can indicate chemical instability. Based on this understanding, next illustrate how to generate a graph vocabulary conditioned on a given class within a specific domain.
For dataset GS, given a class label e, we can sample (A, X) as a new synthetic graph vocabulary by first drawing n' latent positions: u Un 24[0, 1), where U is the uniform distribution. Them:
A Bern (u,, u;)): [t,
W.,(,:))
(11)
where Bern denotes the Bernoulli distribution. The generated vocabulary inherits both the structural priors and semantic traits of the specified class and domain. We depict this process in Figure (right).
We theoretically demonstrate that the generated vocabularies converge in distribution to their empirical distribution, ensuring transferability and semantic consistency. We formulate this in Proposition 2.

---

amp Proposition 2 (Generation Distributional Convergence). Let ge. be the empirical distribution over n' -node vocabulary subgraphs of class c, collected from the disentangled vocabulary bank:
Be
cmp
E 8 - No
1
1
No l each (A. x E( (0,1)) RR' xd
xn'
(12)
where )6) denotes Dirac measure, representing a point mass probability distribution, Assume the true underlying structure and feature functions W.,, W are bounded, Lipschitz, and satisfy permulation equivariance. If the estimators W W.A. W converge uniformly in Lxx. norm, then the
total variation (TV) distance between ge
(Ao: X and ge vanishes as No 0o:
g. TV 0. (Proof C.4)
(13)
4.3 Fine-tuning with Augmented Support Samples
Given any sample (node or graph) G. = E.) )) (i & m) with feature X E B.Nidd from the target domain D3 E }D"}, utilize the shared LLM(.), aligners W, F to obtain unified X E IR
MoE-CoE Network for Selective Augmentation. Fine-tuning GFMs under few-shot settings is highly non-trivial. The scarcity of labeled samples makes it difficult to adapt large pre-trained models by tuning limited parameters, often resulting in unstable performance and overfitting. To alleviate this, we explicitly incorporate transferable knowledge from source domains while avoiding negative transfer, through a hierarchical routing mechanism termed as MoE-CoE. We propose a two-stage architecture consisting of a cotture-of-Experrt (MoE) layer and Collaboration-o--Exppeee layer, which together determine how pre-trained vocabulary tokens are reused for target adaptation.
(CoE)
MoE addresses the question of where to route by selecting relevant domains, where routing weights are assigned across n source-specific vocabulary banks to enable coarse-level domain alignment. CoE addresses the question of how to compose by promoting collaboration among class-level token experts within each selected domain, which allows multiple tokens to contribute jointly, capturing complementary semantics and enhancing representation coverage. Specifically, assigned weights are:
SM E R" Softmax (W <(X? o({)), Sc E ER = Softmax (W (d(X/ I1))),
(14)
where WM M and Wc are projections, (()) is a learnable network. The composed vocabulary is:
del
> sh. 1E=-
21 ST.
ge
= G! g
(15)
1=1
where @ denotes augmenting the support sample G; with the generated vocabulary by overlapping on the max-degre max-degree node for pearrgattionenhannee alignment. To mitigate load imbalance and encourage sparse activation in the MoE-CoE routing architecture, the weight assignment vectors Sy. Se are optimized in supervised manner by minimizing their selective uncertainty:
LMot-CoE(SS Sc) = = 7 ST, (og()
ST. log(S..c).
(16)
c=1
Objectives of Few-shot Fine-tuning. Motivated by prompt learning [6], we introduce learnable graph prompts as few-shot adapters to retrieve relevant knowledge:
H 1 (Pn @ G.),
(17)
where is graph prompt initialized with tunable parameter Ω optimized over m-shot augmented support samples. The graph prompt serves lightweight adapter, selectively injecting transferable semantics while preserving domain specificity. To support both node- and graph-level classification, we adopt unified task template across pre-training and fine-tuning based on ego-graph similarity.
Given m (m < Em No) rocabulary-augmentte support samples ((G )m where H
denotes the node (for node task) or ego-graph embeddings (for graph task), the fine-tuning objective transformed into determining the similarity between the query sample and class prototype embedding:
exp (g(H HT)/T)
(st: =-
Σ
log
(18)
Ex.e(YT) exp (9(H7 ( HTTLT
(G
where H.. is the class prototype for samples in class Y, The overall fine-tuning objective is: Lin = Lot + μ LO (SM. Sc),
(19)
where μ is the trade-off hyperparameter.
7

---

5 Experiments
In this section, conduct extensive experiments evaluate effectiveness, efficiency, and robustness of the proposed GRAVER*. We focus on the following research questions:
RQI: How effective and robust is the few-shot classification fine-tuning? (D> Section 5.2)
RQ2: Which module contributes most to the classification performance? (D> Section 5.3)
RQ3: How time-efficient in the few-shot fine-tuning phase? (t> Section 5.4)
RQ4: How LLMs enhance domain alignment to enable zero-shot transferability? (D> Section 5.5)
RQS: How sensitive is the performance to hyperparameter fluctuations? (D> Section 5.6)
5.1
Experimental Settings
Datasets. To highlight the multi-domain pre-training capability of GFMs, we select seven benchmark graph datasets from three distinct domains. It is worth noting that this differs from conventional experimental settings, which typically treat a single dataset as a domain. Specifically:
Citation Domain: Cora [65], CiteSeer [16], PubMed [73], and the large-scale ogbn- axiv [28].
Co-purchase Domain: ogbn-Tech and ogbn-Home from large-scale co-purchase network [28].
Web Link Domain: Ciki-CS [66], a hyperlink network constructed from a subset of Wikipedia.
Baselines. We compare GRAVER with 15 state-of- thetthe-rt baselines from four primary categories.
Vanilla Graph Neural Networks: including GCN [40] and GAT [91] without pre-training.
Self-Supervised Graph Pre-training: GCC [69], DGI [92], InfoGraph [81], GraphCL [116],
DSSL [102], and GraphACL [103], which utilize self-supervised objectives to pre-train and finetune.
Prompt-based Graph Fine-tuning: we select pioneering prompt-based fine-tuning baselines
including GPPT [82), GraphPrompt [58], GPF [14], and ProNoG [121].
Multi-Domain Graph Pre-training: we select the most closely related baselines on
graph pre-training and finetuning, including GCOPE [136], MDGPT [123], and SAMGPT [119].
Multi-Domain Pre-training Settings. We mainly focus on the challenging task of adapting GFMs to unseen datasets and domains in downstream applications. To better characterize the setting, we explicitly distinguish between cross-dataset and cross-domain adaptation:
pre-training and fine-tuning on different datasets, which are from the same domain. Specifically, take Citation, Co-purchase, and Web Link as the pre-training domains. When each dataset is used as the target alternatively, the remainings are collectively used as the source datasets. Cross-Domain: extends to a more challenging scenario, where the target dataset belongs to unseen domain different from the source domain. Specifically, we alternately set Citation, Co-purchase, and Web Link as the target domain, while the remaining two domains are the source.
Few-shot Fine-tuning Settings. We evaluate node and graph classification under the m- setting, where labeled samples per class are randomly selected. As each dataset contains single large graph, we follow prior work [61, 118, 123] to extract ego-graphs centered on target nodes for graph classification, assigning labels based on the central nodes. Accuracy (Acc.) is used for evaluation.
5.2 RQ1: Few-Shot Node and Graph Classification
One-Shot Classification (Table 1). O Overall, GRAVER achieves the best accuracy, demonstrating clear superiority. Specifically, it outperforms the runner-up with average gains of 2.8% (node) and 3.2% (graph), which are consistent across both dataset and domain transfer scenarios, indicating effective source knowledge utilization even under extreme one-shot conditions. @ Compared to strong basclines like SAMGPT and MDGPT, GRAVER achieves significantly higher performance by over 4% in both tasks. The performance gap widens further in harder domains like ogbn- aaniivariivvvvii and Wiki-CS, demonstrating its broad applicability. Notably, GRAVER still leads with large margins the more challenging cross-domain settings due to domain shift. @ On robustness, GRAVER consistently exhibits an obviously smaller standard deviation compared to baselines with an average relative decrease of 54.0% (node) and 54.6% (graph). This suggests GRAVER maintains stable performance regardless of the random support set selection, thanks to the generative vocabulary augmentations, and is beneficial for real-world deployment. I Additional results under the five-shot setting, presented in Appendix F.I, consistently exhibit the same performance trend as observed in the one-shot scenario.
Codes are available https: htw.ttuuw..ccmmiiignntttttteeee
8

---

Table 1: Accuracy (% std for 20 runs) of one-shot classification. Best results are presented in bold and the runner-ups are underlined. CR = Cora, CS = CiteSeer. PM = PubMed, arxiv
Tech = ogbn-tech, Home = ononn---ec home, Wiki = Wiki- CS. Color denotes domain.
Datsset
Cross-Domain
Source
CS
PM
CR
PM
CR
CS
CR
CS
PK
CR
CS
CR
CS
Home Tech
Home
Wiki
Home
Wiki
Home
Miki
Home
Miki
PM
Wiki
PM Home
Miki
Model/ / Target
CR
CS
PK
Tech
Home
Wiki
arXiv
Node Classification
GCN (bb.) [40]
28.40 a 4.62
29. 29.25 a 3.39
61.59 5.13
53.89 4 3.35
16.74.. 2.53
28.58 5.39
GAT 191]
29.72
29.31 3.47
40.51
$1.8&5222
51.70
36.24 a
29.03 5.75
GCC [69]
32.47 4.55
32.78
3.26
64.74 3.78
55.36 5.86
37.66 + 3.80
30.42 + 5.54
DG1 [92]
30.77 3.92
31.41
39.97
63.76
53.31
39.20
32.15
GraphCt [1]6]]
33.64 + 5.75
28.20 + 3.13
39.03 8
62.44
6.55
$1.55 + 8.20
38.05 +
31,81 + 5.04
DSSL [102]
29.76 4 1.55
30,844
39.99 6.29
61.27 5:21
51.90
37.58 6.78
28.144...17
GraphACL [103]
a.61
6.69 33.88
42.73 5.26
66.70 x 4.29
58.01 a 6.32
40.94 + 6.65
35.57 4.29
GPPT [82]
3 5.74
31.78 4.61
41.97
65.81 6.56
57.8 + 5,36
40.97 5.41
34,22 + 6.51
GraphPrompt [58]
38.02 6.52
34.57
70.11 6.66
60.99
7.19
44.02 6.74
40.74
GPF [14]
40.01 8.21
40.07 + 7.64
47.58 5.883
70.07 6.55
59.05 + 5.85
44.62 + 8.68
40.13 + 5.07
ProNoG [121]
43.67 6:33
40.34 7.11
51.35
73.49 7.34
62.77 6.10
45.65 6.44
41.15 11.5+4..2 4.21
GOOPE [136]
36.27
40.42 1.64
44.75
$1.26.15.99
60.39 a 6.08
41.80 1.96
40,00 6.53
MDCPT [123]
42.55 6.84
37.92 7.18
51.03
72.10
7.12
62.69
7.29
45.93 6.74
43.33
SAMCPT [119]
4.57
38.65 à 6.35
4
73.60 7.55
64.32 4 7.02
46.03 a 6.98
45,27 4 5.05
GRAVER (ours)
48.00 2.52
41.13
55.30 3.03
77.62
67.12 3.18
49.33 2.44
49.25
Graph Classification
GON (bb.)
40.07 a 4.78
29.53 5.74
45.25 + 7.32 41.03 + 5.77
81.16 4,56
58.82 à 2.48
38.46 4.47
46.05 a 4.42
GAT [91]
36.01 5.09
25.98+ + 7.84
77.75 4.92
59.12 5,07
38.49 5.11
40.80
6.87
InfoGraph [81]
42.18 5.17
30.18 1 1.11
49.10
81.62
63.31 3.14
40.98
1.24
43.96 1 6.18
GraphCt. [I [6]
39.56
32.61 6.54
47.71 + 7.04
82.27+
63.77
46.
3.10
47.61
5.02
[192
41,46 7.91
32.92 5.24
47.28
7.71
83.66 1.59
60.56 5.39
41,09
5.18
47.50 5.20
GraphACL [103]
40.60 5.00
33.17 + 7:53
50.48
5.77
80.98 3.78
62.38 + 8.05
45.01
46.43 + 6.32
GraphPrompt 158]
41.71 3:445
38.76 + 7.06
50.61 a 6.39
8-423
61.37 5.64
43.98 a 8.22
47.98 5.40
GPF [14]
41.25 9.41
38. 18.18+88.8 8.22 39.87
47.83 8.17
83.72+5542
67.63 + 4,72
47.16+
47.27 4.16
ProNoo [121]
51.72
54.13
8.155
68.95
47.20
52.11
GCOPE [136]
55.91 +,337
40.99 + 9:01
54.36 (1.6+880)
82.58
67.45 4,39
49.26
62,90 + 3.64
MDGPT [123]
52.77 6.771
41.01
55.47
83.91
70,46
50.24 7.49
59.29 1.25
SANGPT [119]
53.25 + 4.28
42.39 + 7.28
57.66 6.34
85.08 4 6.39
73.69 5.06
52.10 3.65
62.28 a 4.05
GRAVER (ours)
59.20
45.50 a: 2.00
61570
87.33 1.84
75.46
55,90
67.10
m-Shot Classification (Figure 3). To assess performance under varying m, we compare with three
strongest baselines from Table 1: GCOPE, MDGPT, and SAMGPT. Results show: O GRAVER consistently outperforms three baselines across nearly all m-shot settings. @ The advantage is most pronounced under low-shot conditions (m & 4), where the generative vocabulary introduces semantically diverse and transferable patterns to enhance adaptation. As m increases, the relative gain narrows due to reduced augmentation benefits and increasing noise. @ Similar trends are observed in both cross- dataset and cross domain scenarios, confirming GRAVER's s generalizability across transfer settings. 4 Additional results of graph classification in Appendix F.I further support these findings.
Robustness against Random Noise (Figure 4). To evaluate robustness under perturbations, we add Gaussian feature noise and random edge modifications support sets, controlled by parameters X and X₂. Results show: GRAVER maintains superior performance under both feature and structure noise, showing robustness to both support sample variability and direct corruption. @ While baselines degrade more rapidly with increasing noise, especially in structure, GRAVER shows only gradual decline, or even slight increase, highlighting the stabilizing effect of vocabulary-guided augmentation. @ The widening performance gap under higher noise levels indicates that the MoE-CoE architecture effectively filters out irrelevant patterns and reinforces useful knowledge for robust fine-tuning.
5.3 RQ2: Ablation Study
We conduct ablation studies on three core components of GRAVER: the semantic-independene pro- motion (SIP), the vocabulary augmentation (VA), and the MoE-CoE network (MC). Results (Figure 5) show that: Removing SIP leads to minor yet consistent drop, indicating that encouraging semantic diversity across vocabulary channels is beneficial, though not dominant. @ Disabling VA causes a major decline, highlighting the importance of generative subgraphs in enhancing support quality under low-shot conditions. O Replacing MC with uniform fusion results in significant degradation, confirming that selective routing is crucial for injecting relevant knowledge while suppressing noise. @ Additional results on other datasets in Appendix F.2 demonstrate similar conclusions.

---

GCOPE MDGPT
SAMGPT
GRAVER (ours)
GCOPE MDGPT
SAMGPT --- GRAVER (ouns)
Cora (atasa-dataset)
ogbar Xiv (cross domain)
Feature Noise
Structure Noise
Accuracy (%)
70
Accuracy (%)
70
60
50
50
40
40
50
3
6 7
9
10
5
6
8
9
10
0
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
Number
Number
A
Figure 3: m--hoo node classification.
Figure 4: 5-shot graph classification (Wiki-CS)
Time Efficiency (episodes)
GPU Memory Efficiency (GB)
GRAVER
(
SIP
(+) VA
(-) MC
Node Classification
Graph Classification
Accurncy (%)
80
GRAVER
GRAVER.
Accuray (%)
70-
x
80
SAMGPI
SAMGET
20
OCOPE
65
75-
60
60
70
one-shot
five-shot
onc-shot
five-shot
100
20
Figure Ablation studies (ogbn-Home)
Figure Efficiency analysis (ogbn-Tech),
Node Classification
Graph Classifiiation
Original Feature
(+) Raw Text
(++) LLM Enhanced
CateSear (cross dataset)
Wilri-CB (cross domain)
ecurney (%
mero-shot one-shott
2 2
sive-shot
6.7
60
0.25
Accuracy (%)
Accuracy (36)
-1
3.0
0.0
Figure 7: Analysis on LLM (node classification). Figure 8: Hyperparameter sensitiiityy (PubMed).
5.4 RQ3: Efficiency Evaluation for Fine-tuning
In addition to the theoretical complexity analysis presented in Appendix B, we empirically analyze the time and memory efficiency of one-shot node classification, where the results (Figure show that: GRAVER converges in nearly half the episodes compared to baselines while achieving higher accuracy, demonstrating the efficiency of its generative vocabulary and lightweight MoE-CoE routing design. 2 Though GRAVER consumes more GPU memory due to maintaining and routing richer vocabulary tokens, the trade-off is acceptable given its superior performance and faster convergence.
5.5
RQ4: LLM-Enhanced Multi-Domain Alignment
We enhance node features by incorporating raw text and LLM LLM-generated class-aware descriptions. Results (Figure 7) indicate that, raw text alone brings substantial improvement in zero-shot settings, as it serves as Nomain-agnostic semantic bridge when no labeled data is available. This highlights the crucial role of textual information in aligning feature spaces across domains purely through inherent semantics. Building on this, LLM-enhanced features further boost low-shot performance by injecting flassdiicriminative signals. These improvements are consistent across both cross-dataset and cross-domain tasks, underscoring the effectiveness of text-based feature enhancement.
5.6 RQ5: Hyperparameter Sensitivity Investigation
We evaluate the sensitivity of GRAVER to two important hyperparameters: À (for RMI in Eq. (8) and (for CatE-CaE in Eq. (19)). Results (Figure 8) show that accuracy reveals only mild fluctuations within a broad range values for both node and graph classification tasks. This indicates GRAVER robust to hyperparameter combinations. We provide detailed hyperparameter settings in Appendix E.
6 Conclusion
We propose GRAVER, trustworthy graph foundation model that enables robust and efficient fine- tuning under multi-domain pre-training and cross-domain adaptation. To our knowledge, this the first framework to leverage generative graph vocabularies for in-context support set augmentation to enhance adaptation stability. GRAVER introduces graphon-based generative experts to synthesize structure-feature vocabularies aligned with transferable subgraphs, and employs a hierarchical MoE- CoE to dynamically assemble source-domain knowledge for context-aware augmentation. Extensive experiments demonstrate its multi-task superiority over 15 state-of-the-ar baselines.
10

---

Acknowledgments
The corresponding author is Qingyun Sun. This work is supported part by NSFC under grants No.623B2010, No.62225202, and No.62302023, by NSF under grants III-2106758 and POSE- 2346158, by the Fundamental Research Funds for the Central Universities, and by the Academic Excellence Foundation of BUAA for PhD Students. We extend our sincere thanks to all authors for their valuable contributions.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad,, Ilge Akkaya, Florencia Leoni Aleman. Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. GPT-4 technical report. arXiv preprint arXiv:2303.08774 2023.
[2] Edo M Airoldi, Thiago B Costa, and Stanley Chan. Stochastic blockmodel approximation of graphon: Theory and consistent estimation. NeurIPS, 26, 2013.
(3] Emily Alsentzer, Samuel Finlayson, Michelle Li, and Marinka Zitmik. Subgraph neural networks, NewrlPS, 73:8017-8029, 2020.
[4] Maciej Besta, Raphael Grob, Cesare Miglioli, Nicola Berold, Grzegorz Kwasniewski. Gabriel Gjini, Raghavendra Kanakagiri. Saleh Ashkboos, Lukas Gianinazzi, Nikoli Dryden, et al. Motif prediction with graph neural networks. In KDD, pages 35-45, 2022.
[5] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney Arx, Michael Bernstein, Jeannette Bohg. Antoine Bosselut, Emma Brunskill, et al. the opportunities and risks of foundation models. arXiv preprint arXiv:2108. 072.0888, 2021.
[6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurlPS, 13:1877-1901, 2020.
[7] Yuxuan Cao, Jiarong Xu, Carl Yang, Jiaan Wang. Yunchao Zhang, Chunping Wang, Lei Chen, and Yang Yang. When to pre-train graph neural networks? from data generation perspective! In KDD, pages 142-153, 2023.
[8] Ke-Jia Chen, Jiajun Zhang, Linpu Jiang, Yunyun Wang. and Yuxuan Dai. Pre-training on dynamic graph neural networks. Neurocomputing, 500:679-687, 2022.
[9] Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, et Text-space graph foundation models: Comprehensive henchmarks and new insights. NeurlPS, 37:7464-7492. 2024.
[10] Yao Cheng. Yige Zhao, Jianxiang Yu. and Xiang Li. Boosting graph foundation model from structural perspective. arXiv preprint arXiv:2407.199/ 2024.
[11] Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li. Learning prompt for open-vocabulary object detection with vision-language model. In CVPR, pages 14084-14093, 2022.
[12] Yutai Duan, Jie Liu, Shaowei Chen. Liyi Chen, and Jianhua Wu. G-Prompt: Graphon-based prompt tuning for graph classification. Information Processing & Management, 10363, 2024.
[13] Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric 7hao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In WWW, pages 417-426, 2019.
[14] Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. NeurIPS, 36, 2024.
[15] Ziqi Gao, Chenran Jiang, Jiawen Zhang, Xiaosen Jiang, Lanqing Li, Peilin Zhao, Huanming Yang, Yong Huang. and Jia Li. Hierarchical graph learning for protein-protein interaction. Nature Communications, 14(1):1093, 2023,
[16] Lee Giles, Kurt D Bollacker, and Steve Lawrence. CiteSeer: An automatic citation indexing system. In Proceedings of the Third ACM Conference on Digital libraries, pages 89-98, 1998.
[17] Vladimir Gligorijević, P Douglas Renfrew, Tomasz Kosciolek, Julia Koehler Leman, Daniel Berenberg. Tommi Vatanen, Chris Chandler, Bryn C Taylor. Ian M Fisk, Hera Vlamakis, et al. Structure-base protein function prediction using graph convolutional networks. Nature Communications, 12(1):3168, 2021.
11

---

[18] Chenghua Gong, Xiang Li, Jianxiang Yu, Cheng Yao, Jiaqi Tan, Chengcheng Yu, and Dawei Yin. Prompt tuning for multi-view graph contrastive learning. arXiv preprint arXiv:2310. 10362, 2023.
[19] Anchun Gui, Jingiang Ye, and Han Xiao. G-Adapter: Towards structure-awe aware parameter- efffccien transfer learning for graph transformer networks. AAAI,
12226 12234, 2024.
[20] Zihao Guo, Qingyun Sun, Haonan Yuan, Xingcheng Fu, Min Zhou, Yisen Gao, and Jianxin Li. Graph- MoRE: Mitigating topologiaal heterogeneity via mixture of riemannian experts. AAAI, volume 39, pages 1754-11762, 2025.
[21] Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu, G-Mixup: Graph data augmentation for graph classification. In ICML. pages 8230-8248. PMLR, 2022.
[22] Yufei He. Zhenyu Hou, Yukuo Cen, Feng He, Xu Cheng, and Bryan Hooi. Generalizing graph transformers across diverse graphs and tasks via pre-training on industrial-scale data, arXiv preprint arXiv:2407.03953, 2024.
[23] Yufei He, Yuan Sui, Xiaoxin He, and Bryan Hooi. UniGraph: Leaming a unified cross- cross-domale domain foundation model for rext-attributed graphs. In KDD, pages 448 45984459.220 2025.
[24] Van Thuy Hoang and O-Joun Lec. Pre-training graph neural networks on molecules by using subgraph- conditioned graph information bottleneck. In AAAI, volume 39, pages 17204-1753 17213, 2025.
[25] Wassily Hoeffding. Probability inequalities for sums of bounded random variables. The collected works of Wassily Hoeffding, pages 409-426, 1994.
[26] Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjic Wang, and Jie Tang. Graph- MAÉ: Self-supervised masked graph autoencoders. In KDD, pages 594-604, 2022.
[27] Fenyu Hu, Liping Wang, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. GraphDIVE: Graph classification by mixture of diverse experts. In IJCAL, pages 2080-200 2086, 2022.
[28] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. NeurlPS, 33:22118- 22133, 2020.
[29] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In ICLR, 2020.
[30] Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai- Wei Chang, and Yizhou Sun. GPT-GNN: Generative pre-training of graph neural networks. In KDD, pages 1857-1867, 2020.
[31] Qian Huang. Hongyu Ren, Peng Chen, Gregor Kržmane, Daniel Zeng, Percy Liang, and Jure Leskovec. PRODIGY: Enabling in-context learning over graphs. NeurIPS, 36. 2024.
[32] Qian Huang, Hongyu Ren. and Jure Leskovec. Few-shot relational reasoning via connection subgraph pretraining. NewrlPS, 35:6397-6409, 2022.
[33] Renhong Huang. Jiarong Xu, Xin Jiang, Chenglu Pan. Zhiming Yang, Chunping Wang. and Yang Yang. Measuring task similarity and its implication in fine-tuning graph neural networks. AAAI, 18(11) 12625, 2024.
[34] Kanchan Jha, Sripamn Saha, and Hiteshi Singh. Prediction of protein-proteen interaction using graph neural networks. Scientific Reports, 12(1):8360, 2022.
[35] Bo Jiang, Hao Wu, Beibei Wang, Jin Tang, and Bin Luo. Reliable and compact graph fine-tuning via graphsparse prompting. arXiv preprint arXiv:2410.21 17449, 2024.
[36] Xunqiang Jiang, Yuanfu Lu, Yuan Fang, and Chuan Shi. Contrastive pre-training of gnns on heterogeneous graphs. In CIKM, pages 903-812, 2021.
[37] Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. Large language models on graphs: A comprehensive survey. IEEE TKDE, 2024.
[38] Salman Khan, Muzammal Nascer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey, ACM Computing Surveys, 54(10s): 41, 2022.
[39] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
12

---

[40] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
[41] Xiangzhe Kong, Wenbing Huang. Zhixing Tan, and Yang Liu. Molecule generation by principal subgraph mining and assembling. NeurIPS, 35:2550-2563, 2022.
[42] Divyansha Lachi, Mchdi Azabou, Vinam Arora, and Eva Dyer. GraphFM: A scalable framework for multi-graph pretraining. arXiv preprint MXxx:2447.119000 2024.
[43] Jackoo Lee. Hyunjae Kim, Jongsun Lee, and Sungroh Yoon. Transfer learning for deep learning on graph-structured data. In AAAI, volume 31, 2017.
[44] Haoyang Li, Xin Wang. Ziwei Zhang. Zehuan Yuan, Hang Li, and Wenwu Zhu. Disentangled contrastive learning on graphs. NeurIPS, 34:21872 21884, 2021.
[45] Jia Li, Xiangguo Sun, Yuhan Li, Zhixun Li. Hong Cheng, and Jeffrey Xu Yu. Graph intelligence with large language models and prompt learning. KDD, pages 6545-6554, 2024.
[46] Yuhan Li, Zhixun Li, Peisong Wang. Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. A survey of graph meets large language model: progress and future directions. In IJCAI, pages 8123-8131, 2024.
[47] Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li. ZeroG: Investigating cross-dataset scoo-shot transferabilityy in graphs. In KDD, pages 1725- 1735, 2024.
[48] Yujia Li, Chenjie Gu. Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. Graph matching networks for learning the similarity of graph structured objects. In ICMI, pages 3835-3845. PMLR, 2019.
[49] Mingkai Lin, Wenzhong Li, Xiaobin Hong, and Sanglu Lu. Scalable multi-source pre-training for graph neural networks. In ACM MM, pages 1292- 1201, 2024.
[50] Xuan Lin, Zhe Quan, Zhi-Jie Wang, Tengfei Ma, and Xiangxiang Zeng. KGNN: knowledge graph neural network for drug-drug interaction prediction. In IJCAI, pages 2739-2745, 2021.
[51] Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In ICLR, 2024.
[52] Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip Yu, ct al. Towards graph foundation models: A survey and beyond. arXiv preprint arXiv:2310. 18829, 2023.
[53] Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S Yu, et al. Graph foundation models: Concepts, opportunities and challenges. IEFE TPAMI, 2025.
[54] Jingzhe Liu, Haitao Mao, Zhikai Chen, Wenqi Fan, Mingxuan Ju, Tong Zhao, Neil Shah, and Jiliang Tang. One model for one graph: A new perspective for pretraining with cross-domain graphs. arXiv preprint arX 2024.
[55] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9): 2023.
[56] Yanbei Liu, Xiao Wang, Shu Wu, and Zhitao Xiao. Independence promoted graph disentangled networks. In AAAI, volume 34, pages 4916-4923, 2020.
[57] Yixin Liu, Ming Jin, Shirui Pan, Chuan Zhou, Yu Zheng, Feng Xia, and S Yu Philip. Graph self-supervised learning: A survey, IEEE TKDE, (5(6))5879-5900 2022.
[58] Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. GraphPrompt: Unifying pre-training and downstream tasks for graph neural networks. In WWW. pages 417-428, 2023.
[59] Zheyuan Liu, Chunhui Zhang, Yijun Tian, Erchi Zhang, Chao Huang, Yanfang Ye, and Chuxu Zhang. Fair graph representation learning via diverse mixture-of-experts. In WWW, pages 28-38, 2023.
[60] Zhiwei Liu, Yingtong Dou, Philip S Yu, Yutong Deng, and Hao Peng. Alleviating the inconsistency problem of applying graph neural network to fraud detection. In SIGIR, pages 1569-1572, 2020.
[61] Yuanfu Lu, Xunqiang Jiang, Yuan Fang, and Chuan Shi. Learning to pre-train graph neural networks. AAAI 32(5):4277-47777 4284, 2021.
13

---

[62] Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang. and Wenwu Zhu. Disentangled graph convolutional networks. In ICML, pages 4212-4221. PMLR, 2019.
[63] Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma. Tong Zhao, Neil Shah, Michael Galkin, and Jiliang Tang. Graph foundation models. arXiv preprint ar221:240..02216, 2024.
[64] Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Mikhail Galkin, and Jiliang Tang. Position: Graph foundation models are already here. In ICML, 2024.
[65] Andrew Kachites McCallum, Kamal Nigam, Jason Rennic, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127-163, 2000.
[66] Péter Mernyei and Catälina Cangea. Wiki-CS: A wikipedia-bast based benchmark for graph neural networks. arXiv preprint 2xXi:2007.022901 2020.
[67] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Icarning with contrastive predictive coding. arXiv preprint
2018.
[68] Francesca Parise and Asuman Ozdaglar. Graphon games. In Proceedings of the 2019 ACM Conference on Economics and Computation, pages 457-458, 2019.
[69] Jiezhong Qiu, Qibin Chen, Yuxiao Dong. Jing Zhang, Hongxia Yang. Ming Ding, Kuansan Wang, and Jie Tang. GCC: Graph contrastive coding for graph neural network pre-training. In KDD, pages I 1150-1160, 2020.
[70] Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. NeurIPS, 33:12559- 12571, 2020.
[71] Luana Ruiz, Luiz Chamon, and Alejandro Ribeiro. Graphon neural networks and the transferabilliyy of graph neural networks. NeurIPS, 33: 13:1702-1712, 1722.
[72] Sayan Saha and Sanghamitra Bandyopadhyay. for graph neural networks using graphons. TMLR, 2024.
Generating model-level explanations
[73] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al magazine, 29(3);93 93, 2008.
[74] Eli Shamir and Joel Speneer. Sharp concentration of the chromatic number on random graphs G..p- Combinatorica, 7:121-129, 1987.
[75] Chuan Shi, Junze Chen, Jiawei Liu, and Cheng Yang. Graph foundation model. Frontiers of Computer Science, 18(6), 2024.
[76] Chuan Shi, Cheng Yang, Yuan Fang, Lichao Sun, and Philip S Yu. Lecture-style tutorial: Towards graph foundation models. In WWW, pages 1264-1267, 2024.
[77] Reza Shirkavand and Heng Huang. Deep prompt tuning for graph transformers. arXiv preprint arXiv:2309. 10131, 2023.
[78] Razieh Shirzadkhani, Tran Gia Bao Ngo, Kiarash Shamsi, Shenyang Huang, Farimah Poursafaci, Poupak Azad, Reihanch Rabbany, Baris Coskunuzer, Guillaume Rabusseau, and Cuneyt Gurean Akcora. Towards neural scaling laws for foundation models on temporal graphs. arXiv preprint arXiv:2406. 10426, 2024.
[79] Lin Song, Peter Langfelder, and Steve Horvath. Comparison of co-expression measures: mutual informa- tion, correlation, and model based indices. BMC bioinformatics, 13:1-21, 2012.
[80] Gilbert W Stewart. On the early history of the singular value decomposition. SIAM Review, 3544):5511666 1993.
[81] Yun Sun, Jordan Hoffman, Vikas Verma, and Jian Tang. InfoGraph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In ICI.R, 2020.
[82] Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. GPPT: Graph pre-training and prompt tuning generalize graph neural networks. In KDD, pages 1717-1727, 2022.
[83] Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. In KDD, pages 2120-2131, 2023.
[84] Yifei Sun, Qi Zhu, Yang Yang. Chunping Wang, Tianyu Fan, Jiajun Zhu, and Lei Chen. Fine-tuning graph neural networks by preserving graph generative patterns. AAAI, 3888)905539906 2024.
14

---

[85] Yanchao Tan, Zihao Zhou, Hang Lv. Weiming Liu, and Carl Yang. WalkLM: A uniform language model fine-tuning framework for attributed graph embedding. NeurIPS, 36, 2024.
[86] Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. GraphGPT: Graph instruction tuning for large language models. In SIGIR, pages 491-500, 2024.
[87] Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. HiGPT: Heteroge- neous graph language model. In KDD, pages 2842-2853, 2024.
[88] Wenzhuo Tang, Haitao Mao, Danial Dervovic, Ivan Brugere, Saumitra Mishra, Yuying Xie, and Jil- iang Tang. Cross-domain graph data scaling: A showcase with diffusion models. arXiv preprint 0111:2406.01199 2024.
[89] Shiyu Tian, Yangyang Luo, Tianze Xu, Caixia Yuan, Huixing Jiang, Chen Wei, and Xiaojie Wang. KG-Adapter: Enabling knowledge graph integration in large language models through parameter-efficient fine-tuning. In ACL Findings, pages 3813 13-388888, 2024.
[90] Howard G Tucker, A generalization of the Seivenko-Cantelly theorem. The Annals of Mathematical Statistics, 30(3):828 830, 1959.
[91] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengi. Graph attention networks. In ICLR, 2018.
[92] Petar Veličković, William Fedus, William L. Hamilton, Pietro Lid, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In ICLR, 2019.
[93] Nicolas Veyrat-Charvillon and François-Xavite Standaert. Mutual information analysis: how, when and why? In International workshop on cryptographic hardware and embedded systems, pages 429-443. Springer, 2009.
[94] Haotao Wang, Ziyu Jiang, Yuning You, Yan Han, Gaowen Liu, Jayanth Srinivasa, Ramana Kompella, Zhangyang Wang. et al. Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. NeurlPS,
2023.
[95] Heng Wang, Shangbin Feng, Tianxing Hc, Zhaoxuan Tan, Xiaochuang Han, and Yulia Tsvetkov. Can language models solve graph problems in natural language? NeurlPS, 36, 2024.
[96] Shuo Wang. Bokui Wang. Zhixiang Shea, Boyan Deng, and Zhao Kang. Multi- graph foundation models: Robust knowledge transfer via topology alignment. In ICML, 2025.
[97] Yuxiang Wang, Wenqi Fan, Suhang Wang. and Yao Ma. Towards graph foundation models: A transfer- ability perspective. arXiv preprint arXiv:2503.0936. 2025.
[98] Zehong Wang. Zheyuan Zhang, Nitesh Chawla, Chuxu Zhang, and Yansang Ye. GFT: Graph foundation model with transferable tree vocabulary. NeurlPS, 37:107403-107443, 2024.
[99] Shirley Wu, Kaidi Cao, Bruno Ribeiro, James Zou, and Jure Leskovec. GraphMETRO: Mitigating complex graph distribution shifts via mixture of aligned experts. NeurlPS, 37:9358-9933 9387, 2024.
[100] Shu Wa, Yuyuan Tang, Yanqiao Zhu, Liang Wang. Xing Xie, and Tieniu Tan. Session-based recommen- dation with graph neural networks. AAAI, 33(01):346-333, 2019.
[101] Yucheng Wu, Leye Wang, Xiao Han, and Han-Jia Ye. Graph contrastive learning with cohesive subgraph awareness. In WWW, pages 629 099-600,220 2024.
[102] Teng Xiao, Zhengyu Chen, Zhimeng Guo, Zeyang Zhuang, and Suhang Wang. Decoupled self-supervised lcarning for graphs. NeurlPS, 35:620-634. 2022.
[103] Teng Xiao, Huaisheng Zhu, Zhengyu Chen, and Suhang Wang. Simple and asymmetric graph contrastive learning without augmentations. NewrIPS, 36, 2024.
[104] Yaochen Xie, Zhao Xu, Jingtun Zhang, Zhengyang Wang, and Shuiwang Ji. soff-supervieed learning of graph neural networks: A unified review. IEEE TPAMI, 45(2):2412 242222941222299 2022.
[105] Jiarong Xu, Renhong Huang, Xin Jiang, Yuxuan Can, Carl Yang, Chunping Wang, and Yang Yang. Better with less: A data-active perspective on pre-training graph neural networks. NewrIPS, 36:56946-56978. 2023.
[106] Rui Xue, Xipeng Shen, Ruozhou Yu, and Xiaorui Liu, Efficient large language models fine-tuning on graphs. arXiv preprint arXiv:2312.047377 2023. 202..
15

---

[107] Pengwei Yan, Kaisong Song. Zhuoren Jiang, Yangyang Kang, Tianqianjin Lin. Changlong Sun, and Xiaozhong Liu. Empowering dual-level graph self-supervised pretraining with motif discovery. In AAAI, volume 38. pages 9223-9231, 2024.
[108] Yuchen Yan, Peiyan Zhang, Zheng Fang, and Qingqing Long. Inductive graph alignment prompt: Bridging the gap hetween graph pre-training and inductive fine-tuning from spectral perspective. In wwW, pages 4328 4339, 2024.
[109] Yaming Yang. Ziyu Guan, Zhe Wang. Wei Zhao, Cai Xu, Weigang Lu, and Jianbin Huang. Self-super heterogeneous graph pre-training based on structural clustering. NeurIPS, 35:16962 169744, 2022.
[110] Yuhao Yang, Lianghao Xia, Da Luo, Kangyi Lin, and Chao Huang. GraphPro: Graph pre-training and prompt learning for recommendation. In WWW, pages 3690-3699, 2024.
[1II] Ruosong Yc, Caiqi Zhang, Runhui Wang. Shuyuan Xu, and Yongteng Zhang. Language is all graph needs. In EACL Fingdings, pages 1955-1973, 2024.
[112] Zixuan Yi, ladh Ounis, and Craig Macdonald. Contrastive graph prompt-tuning for cross-domain recommendation. ACM TOIS, 42(2):1-28, 2023.
[113] Jun Yin, Chaozhuo Li, Hao Yan, Jianxun Lian, and Senzhang Wang. Train once and explain everywhere: Pre-training interpretable graph neural networks. NeurlPS, 36:35277-35299, 2023.
[114] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In KDD, pages 974-983. 2018.
[115] Wangyang Ying, Dongjic Wang, Xuanming Hu, Yuanchun Zhou, Charu C Aggarwal, and Yanjie Fu. Unsupervised generative feature transformation via graph contrastive pre- pre-training and multi-objective fine-tuning. In KDD. pages 3966-3976, 2024.
[116] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. NeurlPS, 33:5812-5823, 2020.
[117] Qing Yu, Lixin Zou, Xiangyang Luo, Xiangyu Zhao, and Chenliang Li. Uniform graph pre-training and prompting for transferable recommendation. ACM TOIS, 2025.
[118] Xingtong Yu, Yuan Fang, Zemin Liu, and Ximming Zhang. HGPrompt: Bridging homogeneous and heterogeneous graphs for few-shot prompt learning. AAAI, (8(15) 2024.
[119] Xingtong Yu, Zechuan Gong, Chang Zhou, Yuan Fang, and Hui Zhang. SAMGPT: Text-free graph foundation model for multi-domain pre-training and cross-domain adaptation. In WWW. pages 1142-1153, 2025.
[120] Xingtong Yu, Zhenghao Liu, Yuan Fang, Zemin Liu, Sihong Chen, and Xinming Zhang. Generalized graph prompt: Toward unification of pre-training and downstream tasks on graphs. IEEE TKDE, 2024.
[121] Xingtong Yu, Jie Zhang, Yuan Fang, and Renhe Jiang. Non-homophilic graph pre-training and prompt learning. In KDD, 2025.
[122] Xingtong Yu. Chang Zhou, Yuan Fang, and Xinming Zhang, MultiGPrompt for multi-task pre-training and prompting graphs In WWW, pages 515-526, 2024.
[123] Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Text-free multi-domain graph pre-training: Toward graph foundation models. arXiv preprint arXiv:2405.1392 2024.
[124] Haonan Yuan, Qingyun Sun, Xingcheng Fu, Cheng Ji, and Jianxin Li. Dynamic graph information bottleneck. In WWW, pages 469-480 (19-480,,70 2024.
[125] Нaonan Yuan, Qingyun Sun, Xingeheng Fu, Ziwei Zhang, Cheng Ji, Hao Peng, and Jianxin Li. Environment-aware dynamic graph learning for out-of-distribution generalization. NeurlPS, 36:49715- 49747, 2023.
[126] Haonan Yuan, Qingyun Sun, Junhua Shi, Xingcheng Fu, Bryan Hooi, Jianxin Li, and Philip S Yu. How much can transfer? BRIDGE: Bounded multi-domain graph foundation model with gencralization guarantees. In JCML, 2025.
[127] Haonan Yuan, Qingyun Sun, Zhaonan Wang, Xingcheng Fu, Cheng Ji, Yongjian Wang. Bo Jin, and Jianxin Li. DG-Mamba: Robust and efficient dynamic graph structure learning with selective state space models. In AAAI, volume 39, pages 22272 22280, 2025.
16

---

[128] Ge Zhang. Zhao Li. Jiaming Huang, Jia Wu, Chuan Zhou, Jian Yang, and Jianliang Gao. eFraudCom: An e-commerce fraud detection system via competitive graph neural networks. ACM TOIS, 40(3):1-29. 2022.
[129] Jiawei Zhang. Haopeng Zhang, Congying Xia, and Li Sun. Graph-BERT: Only attention is needed for learning graph representations. arXiv preprini arXiv:2001 055000, 2020.
[130] Jiying Zhang, Xi Xiao, Long-Kai Huang. Yu Rong, and Yatao Bian. Fine-tuning graph neural networks via graph topology induced optimal transport. In I/CAI, pages 3730-3736, 2022.
[131] Ke Zhang, Carl Yang, Xiaoxiao Li. Lichao Sun, and Siu Ming Yiu. Subgraph federated learning with missing neighhor generation. NeurlPS, 34:6671-6682, 2021. 2021.
[132] Peiyan Zhang, Yuchen Yan, Zhang, Liying Kang. Chaozhuo Li, Feiran Huang, Senzhang Wang. and Sunghun Kim. GPT4Rec: Graph prompt tuning for streaming recommendation. In SIGIR, pages 7774-1784, 2024.
[133] Shichang Zhang, Ziniu Hu, Arjun Subramonian, and Yizhou Sun. Motif-driven contrastive learning of graph representations. IEEE TKDE, 36(8):4063-40)) 4075, 2024.
[134) Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xiaochuang Han, Tianxing He, and Yulia Tsvetkov. Can Ilm graph reasoning generalize beyond pattern memorization? In EMNLP Findings, pages 2289 2005, 2024.
[135] Zaixi Zhang. Qi Liu, Hao Wang. Chenggiang Lu, and Chee-Kong Lee. Motif-based graph self-supervised learning for molecular property prediction. NeurIPS, 34:15870-15882, 2021.
[136] Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li. All in and one for all: A simple yet effective method towards cross- graph pretraining. In KDD, pages 4443 4454, 2024.
[137] Qifang Zhao, Weidong Ren, Tianyu Li, Xiaoxiao Xu, and Hong Liu. GraphGPT: Graph learning with generative pre-trained transformers. arXiv preprint arXiv:2401.005: 2023.
[138] Tianxiang Zhao, Dongsheng Luo, Xiang Zhang, and Suhang Wang. Multi-source unsupervised domain adaptation on graphs with transferability modeling. In KDD, pages 4479-4489, 2024.
[139] Wang Zhili, Di Shimin, Chen Lei, and Zhou Xiaofang. Search to fine-tune pre-trained graph neural networks for graph-level tasks. In ICDE, pages 2805-2819, IEEE, 2024.
[140] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In CVPR, pages 16816 16816-16825, 2022.
[141] Kaiyang Zhou, Jingkang Yang. Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-languan models. IJCV, 130(9):2337-233 2244, 2022.
[142] Qi Zhu, Carl Yang, Yidan Xu, Haonan Wang, Chao Zhang. and Jiawei Han. Transfer learning of graph neural networks with ego-graph information maximization. NewrIPS, 111176661779 2021.
[143] Qi Zhu, Da Zheng, Xiang Song, Shichang Zhang, Bowen Jin, Yizhou Sun, and George Karypis. Parameter- efficient tuning large language models for graph representation learning. arXiv preprint arXiv:2404. 18277,, 2024.
[144] Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang. Kaixiong Zhou, and Yongfeng Zhang. LLM as GNN: Graph vocabulary learning for text-attributed graph foundation models. arXiv preprint arXiv:2503.033:
2025.
[145] Yun Zhu, Jianhao Guo, and Siliang Tang. SGL-PT: A strong graph leamer with graph prompt tuning. arXiv preprint 2rXiv:2302.12444 2023.
[146] Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. GraphCLIP: Enhancing transfcrability graph foundation models for text-attributed graphs In WWW, pages 2183 2197, 2025.
[147] Yun Zhu, Yaoke Wang, Haizhou Shi, and Siliang Tang. Efficient tuning and inference for large language models on textual graphs. In IJCAI, pages 5734-57 5742 2024.
[148] Yun Zhu, Yaoke Wang, Haizhou Shi, Zhenshuo Zhang, Dian Jiao, and Siliang Tang. GraphiControl: Adding conditional control to universal graph pre-trained models for graph domain transfer learning. In WWW, pages 539-550. 2024.
[149] Chenyi Zi, Haihong Zhao, Xiangguo Sun, Yiqing Lin, Hong Cheng, and Jia Li. ProG: A graph prompt learning benchmark. NerulPS, 2024.
17

---

Appendix
Table of Contents
A Notations
19
B Algorithm and Complexity Analysis
20
Proofs
23
C.I Proof: the Upper Bound of Rin
23
C.2 Proof: Inequity [a] in Proposition
24
C33 Proof: Inequity [6] in Proposition 1
25
C,4 Proof: Generation Distributional Convergence (Proposition 2)
26
D Experiment Details
28
D.1 Datasets Details
28
Baseline Details
29
D.3 Experimental Setting Details
30
E
Implementation Details
33
E.1 Implementation Details of GRAVER
33
E.2
Implementation Details of Baselines
33
E.3
Hardware and Software Configurations
33
F
Additional Results
34
F.1
Additional Results: Few-Shot Node and Graph Classification
34
F2
Additional Results: Ablation Study
35
F3
Additional Results: Efficiency Evaluation for Fine-tuning
35
F4
Additional Results: Node Embedding Visualization
36
Additional Related Work
37
Multi-Domain Pre-trained Graph Foundation Models
Nod.............................
37
G.2 Graph Foundation Models Fine-tunie Fine-tuning with Prompt
.................
37
G.3 GFM Pre-training with Transferable Patterss
38
G.4 GFM Fine-tuning with Generative Augmentations
38
Limitations and Broader Impact
39
18

---

A Notations
Notations
Descriptions
== (V,E)
Graph G with node set V and edge set &.
(0a) egs. {GT)
E gT
Graphs sampled from source domain (S) and target domains (7).
(Y&) =JS {YT
E
yT
Corresponding graph lahels in source domain (S) and larget domains (T).
(D DS, {DT) E
DDT
Source domains and targeted domains.
AC A0,0,1 (0, 10.1)NxN, I)NNN
Adjacency matrix A, where N; is the number of nodes.
RERM
Node feature matrix X. with N, nodes and d, input feature dimensions. Adjacency matrix of disentangled and generated vocabulary with n' nodes. Node feature matrix of disentangled and generated vocabulary with h dim. Source domain graph features, aligned features, and single node features. Encoded source domain graph features and node features.
A. AE E Ra'xn
X. Ra'xA
X X X E R-xa
H E kR
X7, XT, 87 E B-xd
Target domain graph features, aligned features, and single node features. Encoded target domain graph features and node features.
H RK-
gu (Bun)
Ego-graph and its K disentangled vocabularies of the central node Observed, generated, and weighted mixed generated vocabularies.
g g&a, g
fe fe(:): Rd + R K-h
Graph vocabulary encoder with learnable parameter O
g(.): K- x R k-
Graph discriminator compused of an inner product (+-)) and a binary MI.. The graph learner compound of encoder f(-) and binary discriminator g(-)
h() gof
F(): Rd H Rd
Domain feature alignment function from d; dimension to d.
Non-linear activation function (PReLU). Learnable routing network in the MoE-CoE network.
6()
(())
The slack function controls bound tightness.
II())
The set of permutation functions (disentangle channel mappings).
T()
The measure-preserving mapping estimated via empirical node ordering.
T(.)
The Shamnon Mutual Information term.
(())
The neighbor nodes of (I-hop by default).
DRL.
The Kullback-Leebles divergence.
Dancc
The matching distance between disentangled channels.
Dry
The Total Variation (TV) distance between vocabularies. The nonparametric estimated structure token graphon.
W:A [0, 1]2 1]
WX 1 +RR* Rd
The nonparametric estimated feature token graphon. Assignment weight vector for the MoE and CoB network, respectively. Learnable weight matrix for non-linear projection.
Sv. Sc
W. WM. Wc
The non-linear projection bias.
Pa
Graph prompl with learnable parameter Ω.
The routing attention (weights) from node to channel (factor) kt
The augmentation operations by overlapping with the generated vocabulary. The number disentanglement routing iterations.
C.
The Lipschitz constant for the activation function The Lipschitz constant for weight matrix W.
lw
b,
The Lipschitz constant for the cosine similarity (..))
The semantic discrepancics of [f( f(g" ll2-
5
The Dirac measure, representing point mass probability distribution. The upper bound of | x 1|2.
c
P
The lower bound of Læ-norm in Eq. (3). The temperature parameter.
T
A
The trade-off hyperparameter for RMI in Eq. (8).
The trade-off hyperparameter for LxoE-COE in Eq. (19).
RMI
The mutual information regularizer between each disentangled vocabulary.
The pre-training loss function.
LMoE-Cob
Loss function for the MoE- CoE network.
Las
Loss function for the downstream classification.
Lin
Loss function for the fine-tuning phase few-shot training.
19

---

B Algorithm and Complexity Analysis
The pre-training pipeline of GRAVER illustrated in Algorithm I, and the fine-tuning pipeline is shown in Algorithm 2. Next, we theoretically analyze their computational complexity, respectively.
Algorithm Pre-training pipeline of GRAVER.
Input: n source graphs {G=} g from multi-domain {D$) DS, where each of the graph G is associate with its feature matrix X R and adjacency matrix A E R x.N:: Feature dimension aligner F(-); Domain semantic aligners W,; Learnable weight matrix W j. for disentangle channel projection: Number of the latent factors (channels) K; Number of the routing iterations T;; Number of the pre-training epochs E;: Temperature parameter т; Hyperparameter À for trading- off the mutual information regularization term RMI.
Output: Pre-trained graph encoder and discriminator g(f (-)): Structure token graphon
W.A and feature token graphon W for each source domain G and class
Initialize all parameters randomly; for 2, ..., Ej do
// domain feature alignment
Aligned graph feature: X + Eq. (1) for each G E {G$) with F and W;;
// ego-graph disentanglent ement
for / 1,2, ..., and k 1, ..., K do
Initialize projected embedding h +- Eq. (3) for each node u; Routing attention for neighbor node v to k-th channel (factor): a a + Eq. (2); Disentangled ego-graphs (vocabularies): g s(()) + Eq. (4) for each node u; Updated node disentangled embeddings: (t)(t) + Eq. (5) for each node u;
10
11
Calculate the mutual information regularizer: Mi +- Eq. (6) for each node u; end
12
13
// sulf-vsuer pre-training
14
Calculate the overall pre-training loss: fue Eq. (8); Update + by minimizing fre and back-propagation;
15
16
if e Ej then
17
// establish the vocabulary bank
Modeling structure token graphon with observed vocabularies: WAA <- Eq. (9); Modeling feature token graphon with observed vocabularies: W <- Eq. (10);
18
19
20
end
21 end
Complexity Analysis. We analyze the complexity of each part Algorithm follows.
bimension-wise feature alignment: We implement the aligner function F: RR + Rd by the
truncated singular value decomposition (SVD) [80]. We omit this computational complexity as can be pre- pre-processed without increasing the runtime computational burdens.
Semantic-wise feature alignment: We implemented each aligner W, E Rdxd by a randomly initialized learnable non-linear projection matrix, where the complexity takes O(nd)).
Ego-graph disentanglement: We first project the aligned feature of each node to K embedding space, which takes the complexity V.n/V. Kd),,h), where [V] is the average number nodes in each of the pre-training graphs. Next, the neighbor routing iterations take the complexity of O (n V. [T] (w)) where [N(w) is the number of neighbor nodes of of u. Then, the node disentangled embedding updating takes the complexity of the same
For brevity, denote the average of V. as n,, and the average of [N())) I as n2. Thus, the total complexity of the ego-graph disentanglement takes Onn, Kh(d Tnz)).
Pair-wise mutual information regularizer: For pair of factors (i,j), the inner product takes the complexity of ((BB)). For each node, we can enumerate |B] K(K 1) factor pairs. Then, the total complexity accounting each node takes the complexity of O | K(K 1)h).
20

---

Suppose there are Rs direct node neighbors, indirect neighbors (nodes that are connected) averaged for each node the source domain graphs, we can sample n Wa n4 quadruples, which is utilized to formalize the self- self-supervised pre-training contrastive loss. Its computational complexity takes ((nn (ng + n4)
Pre-training loss calculation:
Graphon experts estimation: We implement mT(-) by ordering nodes with degree, which takes the complexity of ((Nan' n')n') for each graph G and class c. For each coordinate (ú, v) E {1..... n²} J. the estimation takes the complexity of (NN,) for n'² coordinate pairs. Thus, the total complexity takes O(nC(NNo log n' + N), where C is the average number of classes.
The overall computational complexity for the pre-training phase is:
O(nd²) + OOn, Kh(d+ Tn2)) + (nn | K(K 1)h) + O (ns + n4) Kh)
+ *O(nC(Nor log + Non?)). (B.1)
As n1 > nI, n1 >> 122, n2 = n2. n1 = n4. and K, T. [B], C. Ne. d. h, n' are relatively small constants, the overall computational complexity is approximately reduced quadratic with respect to the number of pre-training nodes, which demonstrates the proposed GRAVER on par with other state-of-the-ar graph multi-domain pre-training GFMs.
Algorithm 2: Fine-tuning pipeline of GRAVER.
Input: m nodes class-balanced sampled from graph GT EGT (for node classification) or graphs {G" } GT class-balanced sampled from target domain {DT) } E DT (for graph classification), where each the graph G? is associate with feature matrix X; E R N;xd; and adjacency matrix A E x.;;>,;; Pre-trainee graph encoder and discriminator g (-)); Structure token graphon W-A and feature token graphon W for each source domain G and class c: Learnable weight matrix WM M and Wc: Weight assignment vectors SM and Sc: Learnable graph prompts Ps: Number of the maximum fine-tuning episodes E2; Temperature parameter т; Hyperparameter μ for trading-off the MoE-CoE loss term. Output: Fine-tuned graph encoder and discriminator g(fā(-) with s Initialize all parameters randomly;
for e = 1, .., E2 do
// domain feature ali alignment
Aligned graph feature: X + Eq. (1) for the given GT (node classification) or
X - Eq. (1) for each G7 (GT] (graph classification);
// MoE-CoE network initialization and optimization
Initialize assigned routing weights: Sc Eq. (14), SM +- Eq. (14);
Calculate the MoE-CoE loss: LMME.CoE +- Eq. (16);
// support samples augmentation
Vocabulary generation: gen
+- Eq. (11) for each source dataset and class;
Vocabulary composition: ggnn +- Eq. (15) for each supporting samples;
10
11
Vocabulary overlapping: GT + Eq. (15) for each supporting samples;
12
Encode target domain node or graph with graph prompts: H." <- Eq. (17);
13
Calculate downstream classification loss: (s, + Eq. (18);
// optimization
15
Calculate overall fune-tuning loss: Lin <- Eq. (19);
16
Update Si by minimizing Lin and baok-propag
17 end
Complexity Analysis. We analyze the complexity of each part Algorithm follows.
Dimension-wise feature alignment: We apply the same shared aligner function F: Rd: - Rd which is implemented by the truncated singular value decomposition (SVD) [80]. Similarly, omit its complexity as it can be preprocessed without increasing the runtime computational burdens. Semantic-wise feature alignment: We implemented each aligner W, E Rdxd by a randomly initialized learnable non-linear projection matrix, where the complexity takes O(VV (node clas-
21

---

sification) and (m V. (graph classification). Denote the average of [V] as 71, the complexity for this step takes ( d²) for node classification, and (mn,, for graph classification.
MoE-CoE weights initialization: For Sy, the matrix multiplication and Softmax operation takes the complexity of (mn, d) for each support samples, while Se takes the complexity of O(nn (d- h)C)) for each support samples. Thus, the total complexity takes ((mnn1d(C + 1) + mmn1hC).
MoE-CoE self-supervised optimization: Calculating the uncertainty of SM takes the complexity of On""11,, and that of Sc takes C?), Thus, the total complexity takes O (n + C²)).
Support samples augmentation: For each source dataset and class, the vocabulary generation takes the complexity of O(mnC(n2 + n'h)). Then, the vocabulary composition takes the complexity of (mmhi( + 1)). Next, the vocabulary overlapping takes the complexity of O(mn' log n') Thus, the total complexity of this step takes OmmmC(n2 + n'h) + mnh(C + 1) + mmn' log n').
Graph prompt insertion: We initialize graph prompt with learnable parameter 12 over the aligned features, which takes the complexity of O(nid) d) for node classification and (mm1jd) for graph classification.
Augmented support sample encoding: Similar to the pre-training phase, we first project the aligned feature of each node to K embedding space, which takes the complexity of [(n) - mnn) dh) for node classification, and (mi(n, + dh)) for graph classification. Next, the neighbor routing iterations take the complexity of ((m + mmn' hT))) for node classification, and ((nnn f22 + mmn'TT Kh) for graph classification, where 02 denotes the average number of node neighbors. Then, the node disentangled embedding updating takes the same complexity as the last step. The total complexity takes O((n; + mn') K dh- + (ning + mn')T'' h) for node classification, and O(m(n) + n²) Kdhh + (nuing + mnn mn))Tkh) for graph classification.
Fine-tuning classification loss: Suppose there are n. classes in total, the contrastive loss takes the computational complexity of O(mm,Kh)) for both node and graph classification.
The overall computational complexity for the fine-tuning phase of node classification takes:
d + OC + 1) + mc) + O(nn + )²)) + O'mne(n' 2 + n'hi)
+ +mnh(C + 1) + mn' logn') + dm + O(nn + mn') + (nin2 + mn' ma')TKK KT)
+ O(mmKKh) (B.2)
The overall computational complexity for the fine-tuning phase of graph classification takes:
O(maid²) )d + (mmnn,d(C + 1) + mC)) + O(nn (n + C²)) + Ommc((n 2 +''h)
+ +mnh(C + 1) + mn' logn() + ((m, + O(m(n +n')
+ (nn + mn) (TKh) + Omm,,Kh).
(B.3)
As n1 > n. m, n', and K, T, C, d, h, ne are relatively small constants, the overall complexity is reduced quasi-linear complexity, scaling as log with the number of nodes in target graphs.
is worth mentioning that we temporarily defer the theoretical analysis of space complexity. and instead assess the empirical computational cost through efficiency evaluation. As shown in our exper- iments (see Appendix D for setup details), the pre-training phase of GRAVER exhibits comparable time and memory consumption to strong baselines. Although the fine-tuning phase incurs moder- ately higher overhead, this can be effectively alleviated via optimized training strategies, enabling a favorable trade-off between performance and efficiency.
To summarize, the proposed GRAVER achieves an efficient training paradigm through a carefully structured pipeline. The pre-training phase, though involving multiple steps such as disentangled representation construction and graphon expert estimation, scales approximately quadratically with the number of nodes in the source graphs. In contrast, the fine-tuning phase benefits from streamlined and modular design that leverages learned graph vocabularies and MoB-CoE routing, thereby reducing the overall complexity to quasi-linear time with respect to the target graph size. This efficiency, together with our empirical results, demonstrates that the proposed GRAVER maintains strong scalability and practicality for scage multi-domain graph learning tasks as scalable GFM.
22

---

C Proofs
C.1 Proof: the Upper Bound of RM
We first restate RM for reference.
Eq. (6):
RM =
[h b &
EuEB - log (Softmax, ((h /T)
IEKGER
Proof. We provide a detailed derivation showing how the mutual information between two disentan- gled representations can be upper bounded using the InfoNCE loss.
Step-1: From Mutual Information to InfoNCE. The mutual information between two random variables X and Y defined the Kullback Leibler (DKL) divergence between their joint distribution p(x.y) and the product of marginals p(x)
T(X; Y)= (x(x) y) ())(()))(())) Fpx(.)) log p(x,y)
(C.1)
p(x)p(y)
In practice, Eq. (C.1) intractable to compute for continuous high-dimensional variables because requires access to the true densities p(x,y) and p(x) The InfoNCE estimator [67] provides tractable lower bound for I [X; Y), based on noise-contrastive learning setup that distinguishes true positive pairs from negative samples.
Step-2: InfoNCE Objective. Assume sample [8] examples {(x,, y where each yx) p(x. y) is drawn from the true joint distribution. One positive pair is drawn from p(x. y). and [B] negative pairs sampled independently from the marginal p(x)p(y). The InfoNCE objective is defined as:
s s(x, y)
LIBINNC = =E log
(C.2)
s(x,y) + - 181-1 s(x,y,)
where y( is a positive scoring function, typically chosen as exp (x, y) /T) with temperature
Step-3: Mutual Information Upper Bound. Assume that the positive sample (x. y) ~ p(x, y). and that the negatives (yī) ~ p(y) are i.i.d. From [67], we can rewrite the InfoNCE loss in expectation form over the true joint distribution and the sampling distribution:
s(x,y)
Ci -Ep(x.))
log Ey s(x
(C.3)
where the sum in the denominator includes the positive y and [B] negative samples y' ~ p(y). The optimal discriminator in this noise contrastive setup yields the bound:
T(X: Y) & log [ CI
(C.4)
This bound becomes tight as |B| oo, then InfoNCE provides a tractable surrogate for minimizing mutual information by replacing x with h and with h respectively.
Step-4: Batch- Batch-Based InfoNCE and Independence Regularization. In our setting, consider the batch B of size [B] and two latent factor factor-specific representations h,,- h over the same node
u. The positive pair is (h h5 while negative samples are the cross channel representations
Joyt from other nodes in the batch. Then, the InfoNCE objective becomes:
exp((n 2,6: /T) - -log Ever
(t,f)
= EEEEB
(C.5)
InfONCE
(55,)/T)
log (Softmax,, ((h nE /T) ]
A EnEB
(C.6)
Hence, by the inequality derived above:
((,)) [h :b & log |3|
LifoNCE
(C.7)
23

---

By discarding the constant log [B], we obtain a looser but practical upper bound:
Softmmxx, ((b hT) [_))
& EuEE - log
(C.8)
Σ 1(
b bggg) <
Σ
BuEB
- log (Softmax, ((h (ns)//) /T/ |-m)) (C.9)
KEKSEKK
KEKJEK
This concludes the proof.
C.2 Proof: Inequity [a] in Proposition
We first restate the inequity [a] in Proposition for reference.
Proposition 1 (Inequity [al):
K Σ
h (000)))))))))))))()) | + (R
= - f(;)) <
-1((
min NESK
h
Proof. We next provide proof of the semantic discrepancies upper bound via subgraph matching and mutual information constraints.
Step-1: Minimal Matching Distance. Since the per-channel routing is independently conducted for each node, the semantic order of channels may be aligned across nodes. Therefore, we consider the minimal matching distance between their vocabularies (subgraph embeddings):
K
D mmatch
min ITES
b - h 1
(C.10)
1=1
where II Sy denotes a permutation over channels.
Slack. To encourage disentanglement, we introduce an information- theoretic regularization RMi during training to minimize inter-channel mutual information. Ideally, small RMI implies that each channel captures a distinct and independent semantic factor. However, due optimization limitations, residual dependencies across channels may still exist. These dependencies introduce cross- coosnch interaction terms in the final embedding space, making the overall semantic discrepancies larger than the matching term alone. To formally capture this effect, we introduce slack function ((RM)). upper bounding the contribution from such interactions.
Step-2: Information-theored
To obtain tractable analytical form, adopt linear upper bound of the form:
vb (RMi) & B RMI
(C.11)
where 3 > 0 is model-dependent constant that reflects how sensitive the final semantic discrepancies are to cross-channel redundancy. This linear approximation is justified under mild distributional assumptions. Specifically, when the per-channel embeddings approximately follow Gaussian or smooth distributions, the single mutual information term between two channels can be bounded in terms of their correlation:
I S h
log I-p" p2
(C.12)
where / the Pearson correlation coefficient. The equality can be satisfied only if (hS hr,)) ~ NV is jointly Gaussian with variances of of and covariance po; This inequality is commonly used in the literature as conservative estimate of mutual information [93, 79], since direct computation of [h u by,))) is intractable for arbitrary distributions. We omit its proof for simplicity.
Conversely, high mutual information implies strong coupling (large covariance) between channels, which directly contributes the cross terms in the semantic discrepancies calculation. Thus, using linear upper bound allows to quantify this slack without explicitly modeling the full joint statistics.
24

---

Step-3: Upper Bound on Semantic Diserepancy. Combining the matching difference with the information-theon theoretic slack, we obtain the following upper bound:
A [f (g.))
<
[ match + (R
(C.13)
K
min nESK km
h h [I((()))
[ + (R 2.0)
(C.14)
This concludes the proof.
C.3 Proof: Inequity [6) in Proposition
We first restate the inequity [6] in Proposition for reference.
Proposition (Inequity [b]):
K
| (R
(CoLuweg 4pr
min HESK
h
h
<VVK
Proof. We achieve the proof first by further relaxing the conditions in Proof C.2.
Step-1: Further Relaxation of the inequity [a]. From Eq. (C. 10), we can define the optimal channel matching be derived with the optimal permutation function as:
II arg min ThE
Σ I h m
(C.15)
Then, can reach more relaxed upper bound:
h h (a)
Σ
h
(C.16)
A=1
In this way, we can rewrite the inequality [a] in Proposition C.2 as:
K
2 | (R
min Σ h S
uuk
h (R
(C.17)
HESK
Next, continue proof with respect to the RHS of Eq. (C.17).
Step-2: Semantic Discrepancy of Single Channel. Assuming || function weight matrix W. and cosine similarity (---) (inner product) are Lipschitz continuous with constants C., Iw, and I.,, respectively. p the lower bound of L2-norm Eq. (3). Then, for each single channel, the semantic discrepancies in projected features satisfy:
x3 112 & e. The activation
< C. Lo
CoLw
(C.18)
where the normalization lower bound ensures numerical stability by preventing the denominator from approaching 0. Next, we discuss the discrepancy in the channel routing probability.
The routing attention Op->k is determined by the cosine similarity (h h
exp (hh ()TT) ap O Σn exp (hS u.k. h w/TT))
(Eq, (2))
(C.19)
Differentiating with respect to (h a. h S..)) we obtain the sensitivity of routing attention:
aa
ap (p
(C.20)
S/ u,k:
h )
25

---

Then, the discrepancy in routing attention is bounded by:
Lg
122 (since ap-sk(1 ap-sk) E }
aap+t au-kkl
<
h
(C.21)
4T
Thus, the semantic discrepancy after routing is:
Co T
S(T) h u,k
S(T)) h 1 40T
(C.22)
Step-3: Semantic Discrepancy for K Channels. Integrating Eq. (C.22) into RHS of Eq. (C.17);
CoLWLy
h h (RMi)
Σe
(
(C.23)
4pT
Co CoLWLS
cCVK
(C.24)
4pr
Considering 0 in Eq. (C.12), which implies the variables linearly uncorrelated with no shared information. Then, the mutual information regularization term Ru.a - 0. We reach:
(ColaTIn)
h k he (R
< KvK
(C.25)
4pT
By combining Eq. (C.17) and Eq. (C.25), conclude the proof.
C.4
Proof: Generation Distributional Convergence (Proposition 2)
We first introduce a lemma on the graph estimator consistency.
Lemma 1 (Consistency of Vocabulary Graphon Estimators). Assume that the graph vocabulary
samples are drawn i.i.d. from a latent generative model with the true underlying structure graphon function W. [0. 1 > [0. 1] and the feature graphon function W x [0, 1] + IRd both of which are bounded and Lipschitz continuous. Further, assume that the alignment mappings n. are consistent with the true latent positions up vanishing approximation error. Then, as No + c0. the estimators satisfy uniform convergence:
W.W W. lo +0
W.
W
- 0.
(C.26)
Proof. Let u. v E [0, 1] be two arbitrary latent positions. We consider the estimation error:
(w, W. (u, 2)
1 No i=1
Σ
N.
A (0) (T)(
Wa Wa,(u, (u..u))
(C.27)
We decompose this difference into bias and variance terms. The random variable 1 (w), TT('] is a Bernoulli variable with success probability (u, + 1 (u, v), where u; (u, v) captures the alignment approximation error and latent variability. Under the alignment consistency assumption, we have (u, vi] < Vay for all with 14," 0 as the number of nodes in a vocabulary n' +
Applying Hoeffding's 's inequality [25], we bound the deviation for fixed (u, v):
p( [ u, v) E W (u, v)] > w) & 2 exp 2N
(C.28)
Since the domain [0, 1]2 is compact, and A is Lipschitz, cover the domain with finite w-net of size (1/w²²), and apply the union bound. Then, for any O, with probability at least 1 - m
( (11((1/m)
sup (u, VyA"(u, b) < val +0 0,2
(C.29)
No
26

---

where Vi accounts for alignment enror. Therefore, as No + oo, we conclude that:
|| W:Ar|
11 +
(C.30)
The same argument applies W Since the feature graphon is rector-valued and assumed Lipschitz, and the feature vectors are bounded, we apply Azuma- Hoeff-io inequality [74] coordinate- wise to each dimension of Rd and take union bound. Thus:
W 11 - 0,
(C.31)
N₂ oc. This completes the proof.
Based on Lemma 1, we then prove Proposition 2. We first restate it for reference.
Proposition 2 (Generation Distributional Convergence) Let ge over n' -node vocabulary subgraphs of class c, collected from the disentangled vocabulary bank:
cmp be the empirical distribution
No
each (A xx))) E00,1 So''x xn'
gp'xd
Bc
No
('! 2m1
where d()) denotes Dirac measure, representing a point mass probability distribution. Assume the true underlying structure and feature functions W.A., Wx** are bounded, Lipschitz, and satisfy permutation equivariance. If the estimators W.A, W converge uniformly in Los norm, then the total variation (TV) distance between ge gem
= (A.r Xo) and g:
emp vanishes as No + 00:
gen
Be I + 0.
cmp
Proof. Let U1," Un be i.i.d. latent positions drawn from (( 1]. Denote latent vector u (u,, ... unx) E [o, 1jm'. We define the vocabulary generation process via the mapping: +; u r> (A, The convergence assumption on W.A and W are formally established in Lemma 1.
Step-1: Total Variation Distance Decomposition. To study the convergence of geth to the empirical distribution ge smmp, we introduce limiting (true) distribution g: induced by the true graphons W.. W We then decompose the total variation distance based on the triangle inequality as:
|g
ge
cmp ITv < I| -g. llg. - g. "8:""///vv
(C.32)
We prove that both terms vanish No 0o, using functional convergence of the graphon estimators.
Step-2: Convergence of geen >> g.. We consider the pushforward measure of latent positions u through the mapping ((W) that uses estimated graphons W.,, W. versus the mapping p(*) that uses the true graphons W. W Since Aii. are conditionally independent Bernoulli trials, and the Bernoulli distribution is Lipschitz with respect to its parameter, have for every fixed u:
Drv (sv u), u)) <
W. W. uz) W. (u, (u;,u)))
(C.33)
where 7 be the induced measure over adjacency matrices. By uniform convergence of W W. this sum is bounded by O(n AAA) where A !|W- W.+ |l 0. Similarly, for features:
x
x
= W (w:) x (u:) ||, < I w Il
(C.34)
Hence, via coupling over u1 we construct a measure-preserving map such that:
gen g:llTv ((. +' 0,
(C.35)
where 14 IWW. WAA 1o. A = | - x and C > 0 is constant.
Step-3: Convergence of cmp + g. By assumption, the vocabulary bank ((A {())) are i..d samples from g. Since the sample space Z,, E (0. 1 xn', R.n xd is Polish space (product of compact discrete and Fuclidean space), the Clivenko-Cantell theorem [90] applies. Therefore:
.mp g. | - 0.
(C.36)
This completes the proof.
27

---

D Experiment Details
In this section, we present experimental details, The dataset statistics are summarized in Table D.I.
Table D.I: Statistics of the multi-domain graph dataset.
Feature Dimensions
Dataset
Domain
Nodes
Edges
Classes
Avg. # Deg.
Cora [65]
Citation
2,708
5,429
1,433
4.00
CiteSeer [16]
Citation
3,186
4,277
3,703
6
2.57
Pubłed [73]
Citation
19,717
44,338
500
3
4.50
ogbn-arXiv [28]
Citation
169,343
1,166,243
128
40
13.77
ogbn-Tech [28]
Co-purchase
47,428
2,077,241
100
3
87.60
ogbn-Home [28]
Co-purchase
9,790
131,841
100
5
26.93
Wiki-CS [66]
Web Link
11,701
216,123
300
10
36.94
D.1
Datasets Details
To emphasize multi-domain pre-training, we select seven benchmark graph datasets across three distinct domains, in contrast to conventional settings where each dataset is treated as an independent domain. These datasets vary in structure, scale, and feature distribution, providing a diverse and challenging testbed for evaluating pre-trained graph models.
This domain comprises four widely used citation network benchmarks. Each dataset consists of single graph, where nodes represent academic papers, edges denote citation relationships, and node labels correspond the category or research topic of each paper.
Citation Domain.
Cora [65]: ICC BY 4.0] This dataset contains 2,708 machine Cearning-relatee рapers from 7 research fields. Each paper is represented by a -433-dimensional bag-of-words feature vector, where each dimension indicates specific term. The graph includes 5,429 citations, resulting in moderately sparse structure. Due to its small size and clear class boundaries, Cora is widely used for semi-supervised node classification benchmarks.
CiteSeer [16]: [CC BY-NC-SA 3.0] This dataset comprises 3,186 publications categorized into fields. CiteSeer represents each node with 3, 3.703-dimensional sparse feature vector capturing word occurrences. The graph contains 4,277 citations and exhibits a sparser structure with fewer average node degrees compared to Cora. The dataset is considered more challenging due to its noisy labels and lower graph connectivity.
PubMed [73]: [License Unspecified] This biomedical citation network consists of 19,717 research articles on diabetes, divided into classes. Fach node is described by dense 500-dimensional feature vector based on TF-IDF statistics of medical terms. With 44,338 citations, PubMed features a significantly denser graph structure. Its larger scale and domain specificity make it common benchmark for evaluating scalable graph learning methods.
ogon-an arXiv [28]: [ODC-BY] large-scale citation network comprising 169,343 computer science papers from arXiv. Each node is associated with a 28-dimensional feature vector, generated by averaging word2vec embeddings of the paper's title and abstract. The graph contains 1,166.243 citations, and node labels span 40 subject areas. ogbn agbn-arXiv arxiv is widely used for benchmarking scalable graph learning methods on real- weal-world data.
Co-purchase Domain. This domain includes two networks derived from the ogbn-Product [28], capturing relationships between items frequently co-purchased on Amazon. These datasets are widely used for evaluating recommendation and product classification tasks. Nodes represent products, edges denote co-purchase relationships, and node labels correspond to product categories.
ogbn-Tech [28]: [Amazon License] A co-purchase sub-network of ogbn-Product [28] related to technology with 47,428 nodes, 2,077,241 edges, and categories. Each node represents product, edges indicate co-purchase relationships, and node labels correspond to product categories. ogbn-Tech is commonly used for testing recommendation algorithms in the tech product domain.
28

---

Ogbn-Home [28]: [Amazon License] A co- porchcccaee sub-network of ogbn [28] related home usage consisting 9,790 nodes, 131,841 edges, and 6 categories. Each node represents product, edges denote co-purchase relationships, and node labels correspond to product categories. ogbn-Home useful for evaluating recommendation systems in the home goods domain.
Web Link Domain. This domain includes Wiki-CS [66][MIT License], a comprehensive collection of Wikipedia entries with 11,701 nodes and 216,123 web links, which is categorized into 10 distinct areas of computer science. The graph contains nodes representing articles, with edges denoting citation relationships between them. Each node is associated with a feature vector generated from the article's s textual content. Wiki-CS is widely used for evaluating graph learning methods, particularly for tasks that involve knowledge construction and nomain-specinfe knowledge extraction.
D.2 Baseline Details
We benchmark GRAVER against 15 state-of-the-aa methods spanning four major categories: vanilla graph neural networks, self-supervised graph pre-training, prompt-based graph fine-tuning, and multi-domain pre-training frameworks. This comprehensive comparison highlights the effectiveness and robustness of our approach across diverse paradigms.
Vanilla Graph Neural Networks. This category comprises standard graph neural networks trained from scratch on downstream tasks without any pre-training. These models serve fundamental baselines for evaluating representation learning performance.
GCN [40] (backbone): A spectral-based method that captures local graph structures via layer- layer-wise neighborhood aggregation. We adopt GCN as a backbone for both node and graph classification. GAT [91]: graph neural network that leverages attention mechanisms to assign adaptive weights to neighbors, enabling selective aggregation of relevant information. We apply GAT to both node and graph classification tasks.
Self-Supervised Graph Pre-training. This category includes baselines that learn transferable representations by optimizing auxiliary tasks without annotations. By capturing intrinsic structural and semantic patterns during pre-training, these models significantly enhance downstream performance.
GCC [69]: It learns transferable structural representations through subgraph instance diserimina- tion across multiple graphs. By leveraging contrastive learning, GCC captures domain-invarian topological patterns, enhancing generalization to unseen graphs and improving performance in out-of-domain scenarios. It is applied to node classifications.
DGI [92]: It maximizes mutual information between local patch and global summary, encouraging node representations to retain graph-wide context. Unlike relying on random walks, it well-suited for both transductive and inductive settings. It is applied to node classifications.
InfoGraph [81]: It focuses on learning graph-level representations by maximizing mutual informa- tion between entire graphs and hierarchical substructures. This design enables effective encoding of both structural and semantic features. It is applied to graph classifications.
GraphCL [116]: It introduces contrastive learning with a suite of graph augmentations, such as node dropping, edge perturbation, and subgraph sampling, etc. These transformations promote the learning of robust and transferable representations under various supervision regimes. It is applied to both node and graph classifications.
DSSL [102]: It addresses the limitations of homophily assumptions by introducing a latent variable model that decouples semantic content from graph structure. This enables the capture of diverse neighborhood patterns, leading to improved adaptability across homophilic and heterophilic graphs. It is applied both node and graph classifications.
GraphACL [103]: proposes an asymmetric contrastive learning method tailored for heterophilic graphs. By removing the reliance handcrafted augmentations and homophily priors, it effectively models both local interactions and monophily-bur based similarities. It is applied to both node and graph classifications.
Prompt-based Graph Fine-tuning. This category includes baselines that inject learnable prompts into pre-trained graph models to steer task-specific adaptation. By introducing lightweight, tunable instructions, prompt-based fine-tuning enhances the model's flexibility and improves knowledge transfer to diverse downstream tasks.
29

---

GPPT [82]: It bridges pre- training and downstream adaptation by introducing token-pan prompts that recast node classification into edge prediction. It employs masked edge prediction as a pre- training task and maintains this objective format during fine-tuning, thereby reducing the task discrepancy and enabling effective prompt-guided transfer. It is applied to node classifications.
GraphPrompt [58]: It formulates both pre-training and downstream tasks within a unified prompt- ing framework, driven by learnable instructions. This design narrows the objective gap while enhancing label efficiency and transferability. The method provides a general-purpose interface for applying prompts to various graph tasks. It is applied to both node and graph classifications.
GPF [14]: It presents a universal prompt-based tuning approach compatible with arbitrary pre- trained GNNs. By injecting prompts directly into imput feature space, it enables flexible downstream adaptation without architectural constraints. GPF consistently outperforms full-model fine-tuning and tailored prompt strategies. It is applied to both node and graph classifications.
ProNoG [121]: It targets the challenge of non-homophilic graphs by introducing a conditional network that dynamically adjusts prompt representations based on node-level structural cues. This allows the model capture both homophilic and heterophilic patterns, achieving strong generalization across woald graph scenarios. It is applied to both node and graph classifications.
Multi-Domain Graph Pre-training. This category includes the most directly related methods to ours, aiming to learn graph representations that generalize across heterogeneous domains. By exploiting shared structural and semantic patterns, these methods enhance the transferability and robustness of pre-traaee rre-trained models in multi-domaie settings.
GCOPE (1 [136]: It proposes a unified graph pre-training framework that aggregates multiple datasets distill common structural patterns across domains. By explicitly mitigating negative transfer, which is a common challenge in cross-domain scenarios, it facilitates more effective few-shot adaptation. This method advances the development of general-purpose graph foundation models tailored for multi-domain settings. It is applied to both node and graph classifications.
MDGPT [123]: It introduces a text-independent multi-domain pre-training framework that aligns heterogeneous graph domains through learnable domain tokens. It further incorporates a dual- prompt strategy, which comprises unifying and mixing prompts to adaptively reconcile domain gaps during fine-tuning. This design enables effective transfer by minimizing semantic conflicts and enhancing the utility of source knowledge. It is applied to both node and graph classifications.
SAMGPT [119]: It tackles cross-domain adaptation in text-free graphs by aligning structural patterns across domains, in contrast to MDGPT, which focuses on feature aligmment using domain tokens. It introduces structure tokens to unify graph aggregation during pre-training, and employs dual prompts to transfer both generalizable and domain-specine structural knowledge. It is applied to both node and graph classifications.
D.3
Experimental Setting Details
D.3.1 Detailed Settings for Section 5.2 (RQ1)
This section examines the effectiveness and robustness of GRAVER in transferring knowledge from multiple source domains to unseen target datasets under the few-shot learning settings. i.e. (C-way) m-shot scenarios (where & m & 10). GRAVER first pre-trained on source-domain graphs and then fine-tuned with only a limited number of labeled samples from the target domain. To mitigate the instability of fine-tuning and build a trustworthy GFM, we propose a generative augmentation strategy that enriches support samples based on graph vocabularies. This approach ensures both high quality and stability of GFM fine-tuning under any random sample selection.
m-shot Fine-tuning Settings. In each experiment, we randomly sample m labeled instances (nodes or graphs) per class to form the support set used for fine-tuning, while the remaining labeled samples serve as the query set for evaluation. To evaluate robustness and mitigate sampling bias, we perform 20 independent runs with different random seeds, reporting their mean accuracy and standard deviation. We evaluate GRAVER on both node and graph classification tasks. For graph classification, we construct level datasets by extracting 2-hop ego-graphs centered on each node, It is worth noting that GCC, DGI, and GPPT are designed exclusively for node-level classification and are thus only included in node-level comparisons, whereas InfoGraph, an extension of DGI, is evaluated solely graph-level classification.
30

---

Table D.2: Statistics of the multi-domain graph dataset.
Pre-training
Fine-tuning and Downstream Task
Model
multi-
transferable
LLM-
self-supervised cross- multi- zero-shot few-shot in-context
data
domain
pulterm modeling
enhanced feature
pre-training domain task learning learning learning augmentation
CCX (bb.) [40]
GAT [91]
G0C 6001
DCI
Inf InfoGraph [81]
GraphCT. [1
DSSL [102]
GraphACL [103]
GPPT [82]
GraphPrompt [58]
GPF [14]
ProNoC [121]
GCOPE [136]
MDGPT [123]
355
SAMGPT [119]
GRAVER (ours)
Robustness against Random Noise Settings. To further evaluate the robustness of GRAVER under perturbations during fine-tuning, we introduce random noise into the support sets from two com- plementary aspects: feature noise and structure noise. For feature noise, we add random Gaussian perturbations to each dimension of node features for all support samples, following the formulation 7p.r- C where denotes the reference amplitude of the original features, 6g N(0, Gaussian noise, and ' E {0.2, 0.4, 0.6, 0.8) controls the noise intensity. For structure noise, randomly modify the support graph structure by simultaneously deleting and adding X (%) of the edges, with the same range of d values. These controlled perturbations allow us to systematically assess the stability and noise tolerance of GRAVER in few-shot adaptation scenarios.
standard
mass--Daaset and Pross-Domain Settings. We consider two types of domain adaptation scenarios. For cross cross-dataset transfer, the target dataset is unseen during pre-training but comes from the same domain as the source datasets. For cross-domain transfer, the target dataset originates from a different domain altogether. It worth noting that, while some prior works on cross-domain graph pre-training and fine-tuning [136, 123, 119] treat each dataset as a separate domain, we argue that such treatment overly coarse. Instead, we define domains based on broader semantic or application categories. As result, our cross-datasel setting corresponds to what previous literature often calls cross-domain, whereas our cross-domain setting reflects a higher-level shift across distinct fields.
D.3.2 Detailed Settings for Section 5.3 (RQ2)
We investigate contributions of GRAVER'S 's three key components
GRAVER (w/o SIP). This variant removes the semantic independence promotion (SIP) module, which regularizes mutual information minimization between disentangled vocabulary channels. Specifically, we eliminate the mutual information regularizer RMI in Eq. 8 (set À = O), thus removing the constraint that encourages each factor-specific ego-subgraph to capture distinct semantics. This variant evaluates the necessity of encouraging vocabulary disentanglement in the pre-training stage.
GRAVER (w/o VA). This variant disables the vocabulary augmentation (VA) mechanism during fine-tuning. Specifically, we remove the generative vocabulary gi in Eq. (15) and skip the support augmentation step G. @g. Fine-tuning is conducted directly on raw support graphs G? without in- place augmentation. This ablation tests the utility of injecting class-conditioned generative subgraphs to enrich the few-shot supervision.
GRAVER (w/o MC). This variant replaces the hierarchical MoE-CoE routing network (MC) with uniform composition mechanism. Specifically, we set the MoE routing weights SM and CoE routing weights Sç in Eq. (14) uniform distributions, i.e., SM [1/n.... ... 1/n) and Sc = [1/Na, N.). This variant examines whether learned adaptive routing necessary, or whether naive equal-weight fusion suffices for knowledge reuse.
31

---

All variants are evaluated on the ogbn ogbn-Home dataset dross-d under one-shot and five-shot settings for both node and graph classification. We report the accuracy (%) averaged over 20 runs, along with the standard deviation to measure robustness. Other settings, such as pre-trained weights, support-query splits, and training schedules, are kept identical to ensure fair comparison.
D.3.3 Detailed Settings for Section 5.4 (RQ3)
Convergence Efficiency. We measure the number of training episodes required for the model to converge under the one-shot node classification setting. The convergence is defined as accuracy increase for 50 consecutive episodes. All models are initialized with the same pre-trained weights and run on the same query set split for fairness.
Memory Usage. We monitor the peak GPU memory consumption (GB) during fine-tuning using NVIDIA's nvidia-smi smi tool. Measurements are taken over 20 repeated runs and averaged. Analysis includes memory used by prompt embeddings, vocabulary augmentation, routing networks, and model forward/backward passes.
Target Dataset. We perform one-shot node classification on the ogbn-Tech dataset for all efficiency evaluations due to its large graph size and complex feature space. The setting reflects real-world high-load scenarios, making it suitable for assessing practical efficiency.
D.3.4 Detailed Settings for Section 5.5 (RQ4)
(+) Raw Text. For each node in graphs, we retrieve associated raw textual metadata (e.g., paper titles, product names, or page summaries, depending on the specific domain) following [51, 47, 54, 9]. We tokenize the raw text using a pre-trained
SeTT-Base-nncsee tokenizer, and transform it into dense vector by taking the mean-pooling over all output token embeddings. This vector is then concatenated with the original node features to form an enriched representation.
(++) LLM Enhanced. Building the raw text, we further incorporate class-aware textual informa- tion. Specifically, for each node, we append short class-level description (e.g., a node in the "Graph Neural Networks" class may be appended with "This paper belongs to the topic of graph neural networks.") to its raw text. These descriptions are generated manually or retrieved from pre-defined templates, and are consistent within each class. The concatenated text is then encoded with the same SERT-Base-Uncart tokenizer as above. The resulting embedding is concatenated with the original features, similarly to the raw text setting.
Evaluation Protocol and Datasets. In both (+) and (++) settings, the obtained textual embeddings directly concatenated to the original node feature vectors before alignment. This results in an extended feature space that blends structural and semantic cues. We compare three settings: original structural features, raw text-enhanced features (+), and LLM-enhanced features (++). For all settings, the model architecture and routing modules remain unchanged. Especially, in the zero-shot setting, no fine-tuning is performed, and predictions target domain. In the one-shot and five-shot settings, textual features are used alongside support samples for fine-tuning. We report results on CiteSeer (cross-dataset) and Wiki-CS (cross-domain), which are representative benchmarks with rich textual content. The zero-shot setting is evaluated by removing all supervision in the target domain and relying solely on aligned textual semantics.
made directly using the pre- treitriined encoder on the
D.3.5 Detailed Settings for Section 5.6 (RQ5)
Hyperparameter Ranges. For both and μ. we sweep over the values {0, 0.2, 0.4, 0.6, 0.8). forming 5x5 grid in the hyperparameter space. This results in 25 configurations, each evaluated independently. To provide a smooth and interpretable view of sensitivity trends, we apply bilinear interpolation over the 5x5 accuracy matrix to obtain a denser and continuous accuracy surface. Figure clearly illustrates how model performance varies with respect to hyperparameter changes.
Evaluation Protocol. For each configuration, we perform five-shot classification on the PubMed dataset for both node and graph tasks. We report the accuracy averaged over random support-query splits per configuration to ensure statistical reliability. The results suggest that GRAVER maintains stable performance over a broad range of and μ values, with only mild fluctuations accuracy. This indicates that the model is relatively robust to hyperparameter tuning.
32

---

E Implementation Details
In this section, we provide implementation details of GRAVER and baselines.
E.1
Implementation Details of GRAVER
Pre-training. We pre-train GRAVER for up to 10,000 epochs, with early stopping applied if training loss does not decrease for 50 consecutive epochs ensure efficiency without compromising perfor- mance. The aligned feature dimension is set to 64, and the hidden dimension of the encoder is 256. The number of disentangled channels (factors) K is to 3, and the number of convolution layers is set to 2. The trade-off hyperparameter À is tuned within the range of 1. For optimization, we adopt the Adam optimizer [39], with the learning rate and weight decay selected from the range of le-5 to le-2 via grid search on the validation set. All model parameters are randomly initialized to eliminate potential bias from pretrained weights.
the number of routing iterations 7 is set
Fine-tuning, We fine-tune GRAVER for up to 1,000 epochs, using early stopping based on training accuracy with patience of 50 epochs. All architectural and optimization hyperparameters remain consistent with pre-training. During vocabulary-based augmentation, we generate 15 nodes per class overlap with the support set. The trade-off hyperparameter μí is tuned within [0, I| to balance the CoF-MoE objectives. Model parameters are randomly initialized to ensure fair adaptation.
E.2 Implementation Details of Baselines
We provide the baseline implementations with their respective licenses follows.
GCN [40]: [MIT License]
GAT [91]: [MIT License] http:////ttttt.o/oo.....e//////a
GCC [69]: [MIT License] httpss//githubbccm///D////...
DGI [92]: [MIT License] hitccu//tttt..eegtttto..////e//a
GraphCL http://ww/iiniiiiggo////////o//a
InfoGraph [81]: [License
[MIT http:///aaaaarrrra////a///aaaaaa
DSSL [102]: [Apache-2.0 License]
https://gá http::/wwwa.aaaacccoooo/cceaaaaa
GraphACL [103]: [License httpa///saaiiiii/t/////////a//aa
GPPT [82]: [License httpa//w.oaoooaaa/o///////////oa
GraphPrompt httpa//wwaaiiia/a/o/////////////
GPF [14]: |
ProNoG htwpennwhgggggreggtaoooooo////rr
[License httpa//phcico//oo////////ooo///a
GCOPE [136]: [License Unspecified] http:r//gethbb.omm////chhhaa/aaa
MDGPT [123]: [License J http:://aaaopoooss.opppp.cceeees
SAMGPT [119]: [License Unspecified] http:://gtttbb.cmm//eeeeeeee/
For all baselines, we follow the recommended hyperparameters from the original papers or official implementations. unavailable suboptimal, we apply careful tuning for best performance. To ensure fairness, we standardize key settings (e.g., hidden dimensions, layers) to match our method, so that performance differences reflect model design rather than external factors.
E.3 Hardware and Software Configurations
We conduct the experiments with the following hardware and software configurations.
Operating System: Ubuntu 20.04 LTS.
CPU: Intel(R) Xeon(R) Platinum 8358 CPU@2.60GHZ with 1TB DDR4 of Memory,
GPU: NVIDIA Tesla A100 SMX4 with 80GB of Memory.
Software: CUDA 10.1, Python 3.8.12, Torchh 1.9.1, PyTorch Geometrie 2.0.1.
'nttpa://github com/ppytorec/pytorcch btt /nttps://gi http://ggithuu..oo//orooee http///g-team/pytrrcc.ggeo
33

---

Table F.I: Accuracy (% + std for 20 runs) of five-shot node classification. Best results are presented in bold and the runner-ups are underlined. CR = Cora, Ci teSeer, PM = PubMed, arXiv Wiki = Wiki-cS. Color denotes domain.
ogbn-arXiv Tech = ogbn-tech, Home ogbn-home,
Cross- Dataset
Pross-Domain
Source
CS
PK
CR
PM
CR
CS
CR
C8
PM
CR
CS
CR
CS
Home Tech
Home
Wik1
Home
Wik1
Home
Miki
Home
Wiki
PM
Wiki
PM Home
W1x1
Model/ / Target
CR
CS
PM
Tech
Home
Wiki
arXiv
GCN (bb.) [40]
50.19 4,86
89 = 5.42
50.64 + 7.46
63.44 5.18
58.24 4.27
47.37 = 6.06
35.79 5.06 36,74 4.82
GAT [91]
49.03 7.91
46.11 + 5.12
52.19 + 6.29
64.06 + 6.34
$7.40 5.40
47.10 + 5,42
GCC [69]
49.81 + 8.88
45. 6.40
53.03 + 6.53
63.77 5.62
57.94 4.02
45.25 + 6.92
36.88
3.72
DGI [92]
49,89 6.64
46.52 02.5222 + 7:10
53.63 + 7.112
65.75 6.90
6.90
$7.07 6.49
46.27
4.02
38.28
5.41
GraphCL [116]
$3.17
48.76
54.65
69.71
56.20
47.05
40.41
4.57
DSSL [102]
48.36 a 6,99
48 48 = 7.53
54.44 x 8.08
67.12 a. 5.83
5.05 56.05
49.36 x 4.80
38.67 a 3.67
GraphACL. [103]
54,50 7.33
50.41 6.72
55.13 4.77
67.12 5,45
59.38
51.62
44.22 5.96
GPPT [82]
$1,50 5.24
48.48 + 6.02
55.22 + 7.85
64.36 4.98
2.534
50.75 + 3.32
42,06 + 4.33
GraphPrompt [58]
$4.94
52.53
58.03
69.22
61.599
53.50
49.18 4.5%
GPF [14]
54,21 8.39
52.89 4 7.38
56.38 5.47
6.375
61.41 5.57
53.43 5:29
49.17 1.99
ProNoo [121]
6
54.93 6:48
57.83 7.71
77.70
64.98 1 7:47
53.24 4.339
54,27
3.91
GCOPE [136]
55.61 6.40
56.92 + 5.77
53.64 +
74.96 4.11
62.88 6.93
54.03 = 5.04
$0.77 * 6.51
MDCPT [123]
62.71 5.998
55.85 3:27
58.72 4 6.23
4.115
6
55.08 4.771
54,05 4,06
SAMGPT [1 [1]
64.55 6.72
56.43 a 4.69
59.05 +: 5.97
79.11 =
68.96 4.90
55.95 a 3.34
56.60 5.02
GRAVER (ours)
68.68 2.82
58.10
61.63 2.54
8
70.43 2.30
57.37 2.21
59.37
Table F.2: Accuracy (% + std for 20 runs) of five-shot graph classification. Best results are presented in bold and the runner-ups are underlined. CR = Cora, CS Ci teeeeer, PM = PubMed, arXiv ogbn-arXiv, Tech = ogbn- Home ogbn oobn-hhme, Wiki Wiki-CS. Color denotes domain.
Cross-Datasee
Cross- Dossi-
Source
CS
PK
CS
CS PN Home
Home Tech
Home
Wiki
Home
Wiki
Home Miki
Home
Miki
PK
Wiki
Miki
Model/ Target
CR
CS
PH
Tech
Home
Wiki
arXiv
GON (bb.) [40]
52,93 1.08
43.91 1. 5.866
1
77.52 a 6.08
61.78.5.88
40.17 8.330
41.84 5.57
GAT [9]]]
49.62 5.09
45.25
7:28
7:30 54.49
79.77 6.41
61.45 6.83
39.44
5.28
34.64 4.07
InfoGraph [81]
54.65 a 4.93
47.21 5.16
59.70 7.12
78.85
61.94 a 3.99
43.01 1 4.32
40.05 1 5.83
GraphCL [1]6]
55.22
46.41
3,80
59.94
5.44
$1.38
62.75 6.53
44.78 5.50
40.98 1 5.99
1192
53,92 55005
47.49 5:18
62.43 3.61
78. 18 7.23
62.07 + 7.17
51.67 6.68
46.67 +: 3.55
GraphACL [103]
56.19 + 7.08
45.18 6.14
60.10 + 7.08
81.50 5.82
69.69 + 2.53
48.92 7.40
46.93 193++17 7.40
GraphPrompt [58]
53,83 5.19
50.47 + 4.21
64.38 6.40
83.31 5.31
73.77 4.53
55.64
6.13
56.31 4.81
GPF [14]
54.79 4.73
53.90 5.81
63.08 6.36
5.44 84.79
73.13 1:3+++..3 7.38
4,89 56.54
55.43 5.76
ProNo6 [121]
62.09
57.64 7.57
66.44
84.17
6.11
59.52
58.08
GCOPE [136]
63,86 4.7.5
58 20 1 5.78 59.32
66.39 3.71
85.99 4.17
12.10155.3
58.42 1 5.56
58. 14.44.3 3.20
MDGPT [123]
60
67.60
84.85 3.39
74.30
61.99
61.15
SANGPT [119]
68.29 + 3.36
62.08 + 5.66
67.94 + 4.56
85.27 + 4.62
75.83 + 4,56
63.09 2.69
63.82 + 3.41.
GRAVER (ours)
71.07
65.45
89.32 +
79.18
65.85
65.28
F
Additional Results
In this section, we provide additional experiment results.
E1 Additional Results: Few-Shot Node and Graph Classification
Five-Shot Classification (Table F. and Table F2). We further evaluate GR AVER under the five-shot settings. Results consistently demonstrate that GRAVER maintains superior performance across both node and graph classification tasks in cross-dataset and cross-domain scenarios, extending the positive trends that are observed in the one-shot setting. Specifically, GRAVER achieves significant performance advantages over baselines, marked by notably higher accuracy and substantially reduced variance, emphasizing its robustness against the randomness inherent in few-shot support set selection. While the performance gap narrows slightly owing to improved baseline performance with increased labeled data, GRAVER continues to outperform competitors. These findings highlight its consistent effectiveness in leveraging generative graph vocabularies to robustly and efficiently adapt to limited labeled data. Overall, the additional results provided here reinforce the reliability and superiority of GRAVER in shot learning scenarios.
34

---

GCOPE MDGPT
SAMGPT
GRAVER (ours)
GCOPE
MDGPT
SAMGPT
GRAVER ER (ours)
(eross-datasee)
oggon Melin-axXivv (orossdomai)
Noise
Structure
60
Accuracy (%g
Accuracy (%)
70
63
50
50-
60
40
50
40
10
Number of shots
Number of sbots
Ag
Figure F.1: m-shot graph classification.
Figure F:2: 1-shot graph classification (Wiki-CS).
GRAVER (-) SIP
(-)
VA
(-)
MC
Time Efficiency (episodes)
GPU Memory Efficiency (GB)
GRAV ER
Node Classification
Graph Classification
50
50-
Accurncy (%)
SORAVER
Accuracy (%)
SMMPP
40
MOGPT
40-
60
GCOPE
G8OPE
40-
M
30
30-
080-shot
five-shot
one-éhot
five-ahot
200
300
20
30
Figure F.:: Ablation studies (CiteSeer).
Figure F.4: Efficiency analysis (ogbn-arXiy)
m-Shot Classification (Figure F.1). We additionally analyze the performance under varying shot set- tings for graph classification tasks. Results further corroborate the conclusions drawn from Figure F.1. Specifically, the performance margin remains evident in the low-shot regime (m & 5), underscor- ing GRAVER's ability inject meaningful generative vocabularies even with minimal supervision. Notably, in the cross-domain case, where domain shifts pose a significant challenge, GRAVER'S margin remains prominent throughout, highlighting its superior generalization and robustness. These findings reinforce our core claim: generative vocabulary augmentation effectively enhances few-shot adaptation in both low-resource and more relaxed settings.
Robustness against Random Noise (Figure E2). Compared to the five-shot setting, the one-shot setting reveals
pronounced performance drop in the presence of noise for baseline methods, In contrast, GRAVER exhibits consistently high accuracy and reduced degradation across all noise levels. Under feature noise, GRAVER shows graceful performance decay, while other models experience sharp declines. Under structure noise, GRAVER not only maintains stability but occasionally improves, likely benefiting from structural diversity introduced by generative augmentation. This contrast highlights GRAVER'S 's robustness advantage, especially in the extremely low-shot regime.
F.2
Additional Results: Ablation Study
Figure F.3 extends the ablation study to the CiteSeer (cross-dataset) setting. Compared to results on oome (cross-domain), the relative contributions of the three core components remain consistent. Specifically, removing VA causes the most substantial performance drop in both node and graph classification, underscoring its centrality to effective adaptation under extreme supervision scarcity. The degradation from removing MC is also evident, indicating the importance of selective knowledge routing. Notably, the performance gaps between full GRAVER and its ablated variants are larger on CiteSeer than ogbn-Home, particularly in the five-shot setting. This suggests that GRAVER's mechanisms are especially effective when dealing with smaller, more homogeneous graphs.
F.3 Additional Results: Efficiency Evaluation for Fine-tuning
Figure F.4 supplements the efficiency analysis on the large-scale -exvva dataset. Compared to the earlier findings on ogbn -Tech, GRAVER continues to demonstrate strong senciincy--erffommaae trade-offs. It consistently converges in fewer episodes and achieves higher accuracy with less variance than baselines. Although GRAVER incurs higher GPU memory usage, its performance gain per memory unit remains the highest among all methods. Notably, the gap in time efficiency is even more pronounced on ogbn-arXiv, suggesting that GR GRAVER' 's generative vocabulary and routing design not only stabilize training but also accelerate convergence on large and heterogeneous graphs.
35

---

Ditydeer
rghr-lioms
MERT-CS
53
-43
-30
-10
30
130
-230
20
60
-73
-30
-25
(a) Pre-trained embeddings.
(b) Unfinetuned on Cora.
(c) Fine-tuned on Cora.
Figure F5;: Visualization of node embeddings for I-shot node classification (Cora, cross-dataset)
Dora
CSSASMr
aghn lime
Silhouette Score: -0.071
75
53
50-
50
50
-00
105
-50
(a) Pre-trained embeddings.
(b) Unfinetuned on Wiki-CS.
(c) Fine-tuned on Wiki-CS.
Figure F6: Visualization of node embeddings for 1-shot node classification (Wiki-CS, cross-domain).
F.4
Additional Results: Node Embedding Visualization
Figure F.5 and Figure F.6 provide a qualitative comparison of node embeddings for one-shot node classification under cross-dataset (Cora) and cross rross-domain (Wiki-CS) settings, respectively. Across both tasks, we observe consistent improvements in the clustering structure of the embeddings after fine-tuning with GRAVER.
In both settings, the pre-trained embeddings exhibit clear separation among nodes from different source domains, which supports the effectiveness of GRAVER'S 's multi-domale pre-training design. Specifically, the use of LLM LLM-ennanced semantic alignment and shared dimension-wis. projection enables graphs with heterogeneous features to be embedded into unified latent space. Furthermore, the ego-graph disentanglement ensures that factor-specific patterns are preserved across domains. As result, nodes from different graphs are aligned without sacrificing their structural diversity.
However, despite strong domain-level alignment, the pre-trained embeddings lack strong class- level separability, as evidenced by the relatively low silhouette scores (e.g., 0.0238 on Cora and -0.0471 on Wiki-CS), Silhouette scores quantitatively measure the class separability embeddings, where higher values indicate tighter intra-class cohesion and clearer inter-class boundaries. After applying one-shot fine-tuning, silhouctte scores increase in both scenarios (e.g., to 0.0325 on Cora and 0.00518 18 Wiki- showing that GRAVER is able to adapt pre-trained embeddings into more dissriiicriiiiative structures with minimal supervision. The improvement is more pronounced in the cross-dataset case (Cora), while still non-trivial under the harder cross-domain setting (Wiki-CS), where semantic and structural gaps are larger.
In summary, GRAVER's 's pre-training effectively aligns nodes across domains by combining LLM- based semantie projection and ego-graph disentanglement, leading to structured but class-agnostic embeddings. The low silhouette scores reflect poor class separability at this stage. After fine-tuning, the introduction of class-aware vocabulary improves silhouette scores, indicating enhanced class discriminability on top of the well-aligned multi-domain space.
36

---

G Additional Related Work
G.1 Multi-Domain Pre-trained Graph Foundation Models
Graph foundation models have rapidly progressed by leveraging self-supervised pre-training to capture universal patterns in graph-structured data [57, 29, 104, 124]. These models aim to distill transferable knowledge from large-scale graphs through tasks such as node feature masking, edge reconstruction, or contrastive instance discrimination. Notable examples include GCC [69], GPT- GNN [30], GPPT [82]. L2P-GNN [61], and others [105, 36, 109, 8, 7, 7,113, 113, 20]. After pre-training, these foundation models are fine-tuned or prompted for specific downstream tasks. However, most existing models are developed under the assumption that the pre- pre-training and downstream domains share similar structural or semantic distributions, often involving homogeneous subgraphs [32] or graphs within narrow domain [135]. As such, these approaches are vulnerable to domain shift and tend to underperform when directly transferred to out-of-distribution graphs.
To overcome this limitation, recent efforts have explored cross-domain graph learning, where the goal is to adapt knowledge from one domain to another dissimilar domain. Representative models in this direction include GCOPE [136], OMOG [54], PGPRec [112], UniAug [88], and UniGraph [23], which focus on extracting Nomain-invariant representations to reduce the distribution gap. Despite their effectiveness in single-source transfer settings, these models often lack the capacity to generalize across multiple diverse domains simultaneously. Furthermore, many are designed with see-ispccfiic or domain sppccifc assumptions, limiting their scalability as universal graph foundation models.
To advance toward multi-domain graph foundation models, several recent works explore pre-training strategies that can encode graph knowledge across heterogeneous domains. OFA [54] and HiGPT [87], for example, introduce LLMM-uided frameworks that convert node information into natural language for alignment. While promising, these methods are restricted to text-attributed graphs [95, 85] and cannot handle seneral-purpose structural graphs. Other approaches, such as GCOPE [136], insert virtual nodes to serve as domain bridges, yet these do not achieve semantic feature alignment. MDGPT [123] proposes the use of domain tokens to align multi-domain representations, but employs SVD-based alignment that overlooks semantic consistency across domains.
G.2 Graph Foundation Models Fine-tuning with Prompt
Fine-tuning pre-trained graph foundation models is dominant strategy for adapting general-purpose representations to downstream applications. Conventional fine-tuning methods typically update either the full parameter set or selected components of the model to optimize for specific tasks [130, 84, 139, 108, 85, 115, 33, 35, 129, 125]. While such approaches can yield strong task-specific performance, they are often resource-intensive and prone to overfitting.
To address these limitations, parameter-efficient fine-tuning (PEFT) methods have been proposed. Instead of retraining the entire model, PEFT techniques update only small subset of parameters [143, 89, 19, 106] or incorporate lightweight modules [49, 147, 1 110] that can be trained with significantly fewer resources. These strategies retain much of the model's pre-trained knowledge while improving adaptability and reducing computational overhead. Despite their advantages, PEFT still faces difficulties in generalizing across domains, where task variability and domain shift may compromise their effectiveness due to the inability to handle inter-domain discrepancies.
In parallel, prompt-based fine-tuning is a promising alternative. Originating from natural language processing [55, 141, 140, 11], prompt learning reframes task adaptation as conditioning the model via learned input tokens or templates, rather than altering model parameters. For graph learning, this paradigm has been extended through the design of graph prompts, which are structured inputs embeddings that guide the behavior of pre-trained models toward downstream objectives [82, 145, 77, 132]. Prompt tuning enables efficient task adaptation with minimal parameter updates, thus mitigating overfitting and supporting better generalization, especially in few-shot settings.
Building upon this foundation, variety of prompt learning methods for graphs have been proposed. These include GPPT [82], All in One [83]. GraphPrompt [58]. Self-Pro [18], SGL-PT [145], GPF.
Plus [14], PRODIGY [31], MDGPT [123], HGPROMPT [118], and ProG [149], among others. These models introduce learnable prompts into architectures, allowing them to emphasize salient topological patterns or semantic features relevant to specific tasks, which has shown promise in
improving task transfcrability and enhancing model robustness structurally complex graphs.
37

---

Nevertheless, current graph prompt methods often exhibit limited scalability in multi-domain settings. Many of them are tailored to single-domain tasks or require manual sppaciic prompt designs, restricting their flexibility when faced with diverse or evolving graph distributions. Moreover, the field still lacks formal understanding of prompt learning the context of graph structures, in which most methods are empirically driven, without theoretical guarantees or generalization analysis.
G.3 GFM Pre-training with Transferable Patterns
Graph Foundation Models aim to capture structural patterns generalizable across diverse domains and tasks [69, 142, 83, 7, 148, 97, 96]. A core challenge is defining transferable subgraph patterns for pre-training since graphs lack obvious "tokens" like words or patches.
One line of work addresses this by identifying common substructures (motifs, computation trees, etc.) as a graph vocabulary that can be learned and reused [142. 146, 47, 138, 144, 97, 117]. For example, Wang et al. [98] propose GFT, which treats each node's 's computation tree (the tree unrolled from one node's message-passee paassaggppssinggggg neighborhood) as token. By vector-quantizing these computation trees into a discrete vocabulary, GFT's s pre- pre-training encourages the model to represent graphs in terms of recurring structural patterns. This approach improved cross-domain generalization and reduced negative knowledge transfer in experiments, illustrating that encoding graphs via transferable pattern "tokens" can serve as a promising pre-training strategy.
Another perspective directly learns frequent subgraph motifs during pre-training [135, 4, 133, 107]. Zhang al. [133] introduce motif-driven contrastive framework MICRO-Graph that automatically mines significant subgraph patterns from large graphs. MICRO-Graphe clusters similar subgraphs into motif slots and trains GNN via graph-to-subgrapy contrastive learning. By pulling together representations of graphs sharing a motif and pushing apart those without, the GNN learns motif- aware embeddings. This motif-based pre-training yielded improved transfer downstream tasks like molecular property prediction, as motifs (e.g.. functional groups in molecules) carry semantic patterns that generalize across graphs. Similarly, MGSSL [135] incorporates a motif generation objective: a GNN is trained to reconstruct masked substructures and generate likely motif segments within graphs. By capturing the "building blocks" of graphs, MGSSL's multi-level pre-training (at both atom and motif levels) significantly boosted downstream molecular classification performance.
Beyond motifs, researchers have explored universal structural pattern learning for cross-task trans- fer [43, 142, 148, 134]. For instance, Yin et al. [113] present π T-GNN, which is pre-trained on synthetic graphs with known explanatory subgraphs. T-GNN uses a dedicated structural pattern learn- ing module to extract diverse, universal subgraph patterns that occur across graph types, which are then integrated as a comprehensive graph representation. The pre-trained æ r-GNN can be fine-tuned on real datasets, where it not only improves accuracy but also generalizes its interpretability to new tasks. Notably, a single π GNNN pre-trained on graph-level tasks achieved state-of-the art explanation quality even node classification tasks, indicating that the "universal structural patterns" it learned are transferable across task formats.
Early self-supervised graph pre-training methods (e.g., context prediction or graph-level contrastive learning) also implicitly encourage the learning of generalizable structure [26]. However, these often rely on dataset-specific augmentations or predictive tasks. In contrast, the recent trend is to explicitly encode structural commonalities, whether via subgraph vocabulary [98], motif discovery [133], or synthetic pattern pre-training [1 13], to build graph foundation models with stronger out-of-distribution robustness. These approaches collectively demonstrate that transferable graph patterns (from small motifs up to rooted computation trees) form powerful basis for general- sereral-purpose graph learning.
G.4
GFM Fine-tuning with Generative Augmentations
After pre-training, adapting graph models to specific downstream tasks can be further enhanced by generative data augmentation techniques [30, 137, 7, 84, 115, 86, 111]. Rather than only tuning on the limited observed graph data, recent works use graph generators to synthesize additional training examples or to adjust the model to the target domain's 's underlying graph distribution. Such generative augmentations go beyond traditional edge or node perturbations by creating new realistic graph structures, often in a context-awar manner.
38

---

One prominent approach is to leverage graphons. which are continuous graph generation functions [2, 68, 72. 12. 71, 84]. Han et al. [21] propose G-Mixup, which adapts Mixup to graphs by interpolating between the underlying graphons of different classes. By augmenting with inter-class hybrid graphs, G-Mixup substantially improved GNN generalization and robustness against noisy labels. The use of a generative graph model is key: it produces realistic graphs that capture smooth transitions between structural patterns of different classes, something unattainable by random edge swaps.
While G-Mixup creates new input graphs, other work focuses on generative fine-tuning objectives to better fit pre-trained model the target graph data [48, 84, 9. 53]. A recent example G- G-Tuning by Sun et al. [84], which addresses the issue of structural mismatch between pre-training and downstream graphs. They identify that a pre- traitre GNN might overfit to patterss from its pre-training graphs and struggle to capture a new graph's 's structure if the generative processes differ. G- G-Tuning tackles this by explicitly preserving the "generative patterns" of the downstream graph during fine-tuning. shows that treating fine-tuning as fitting generative graph model helps retain broad generalization ability while adapting to new structures.
Another category of generative augmentation focuses on subgraph generation and replacement [3, 131, 41, 101, 24, 22]. These methods generate new sub-components of graphs to enrich training data or to regularize model explanations. For instance, Liu et al. [57] introduce an environment replacement as part of graph rationalization framework. By training the GNN on both original and these augmented examples, the model learns to identify the subgraph's 's influence independent of a specific context. Such context-aware subgraph augmentations improve model robustness and interpretability, as the GNN sees multiple generated environments for the same substructure. The success in molecular property prediction tasks demonstrates that generatively augmenting graph contexts can help the model generalize the learned subgraph patterns beyond the limited surroundings.
More broadly, mixture-of-experts (MoE) have been applied to handle diverse generative regimes [27, 94, 59, 99, 20, 123]. Rather than a single generator, multiple generative "experts" can be trained, each specializing in certain structural pattern or data distribution. Wu et al. [99] develop GraphMETRO, an MoE- based GNN that implicitly performs augmentation by routing each graph through an expert tuned its distributional subdomain. Each expert produces a representation focusing on a particular structural shift, and a gating network combines their outputs. This idea could be extended by having each expert generate slight variant of the input and then ensembling the predictions.
In summary, generative augmentations for graph fine-tuning go beyond traditional techniques like edge drops feature noise by synthesizing structurally meaningful examples or aligning with target distributions. This provides richer training signals that better capture task-specific graph patterns. Empirical studies show that approaches such as graphon-based sampling [21], subgraph replacement [57], and generative alignment [84] consistently improve robustness and generalization. These trends underscore the growing interplay between graph generation and representation learning. enabling more adaptable and data-efficient graph foundation models.
H Limitations and Broader Impact
Limitations. While GRAVER demonstrates strong performance, several peripheral aspects remain to be explored. The vocabulary construction relies on class-aware supervision, which may limit applicability in unsupervised or open-world scenarios. Extending toward unsupervised or weakly supervised vocabulary discovery could enhance generality. In addition, the method is mainly evaluated on classifications, and its effectiveness on more complex reasoning settings, such as link prediction or anomaly detection, remains unclear. Finally, the reuse and compression of disentangled vocabularies across tasks have not been thoroughly examined, which may offer further gains in scalability.
Broader Impact. This work contributes to the growing field of GFMs by addressing a core challenge in robust and efficient few-shot adaptation. The proposed generative vocabulary framework has the potential to improve the stability and scalability of GFM deployment in real-world applications such as scientific discovery, bioinformatics, and recommendation systems, where labeled data often scarce. By enabling more reliable cross-domain generalization, GRAVER may facilitate broader access to graph-bascd intelligence capabilities across meseureesourced domains. We do not foresee any direct negative societal impacts, as the proposed method does not involve human subjects, sensitive data, or high-risk generative content. Instead, it aims to strengthen the foundation of trustworthy and generalizable graph learning.
39

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly articulate the main contributions, in- cluding generative augmentation with transferable vocabularies, graphon-based experts for structure-feature modeling, and robust fine-tuning via MoE-CoFE routing. These claims are substantiated by theoretical results (e.g., Proposition 1 and Proposition 2) and extensive experiments (Section 5). The scope and limitations are also reasonably aligned.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The paper includes dedicated discussion on limitations in Appendix H. It acknowledges the dependency on class- awsse supervision during vocabulary construction and generation, which may hinder applicability in unsupervised or open open-world settings. Limitations on large-scale domains with highly dynamic distributions are also implied.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
40

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
The paper provides theoretical assumptions and formal proofs in Appendix for key propositions, such as the vocabulary transferability bound (Proposition 1) and the convergence of generated vocabularies (Proposition 2). Each proof is rigorously derived with necessary assumptions clearly stated (e.g., Lipschitz continuity, permutation invariance).
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper discloses sufficient experimental details across Sections 5, and references to Appendix D and Appendix E for more settings. The authors provide a public anonymous code repository, which further supports reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
41

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The authors share an anonymous code repository thttps://anony::
4open wcweeceee//eciecccccc httee///GRA/R)) which includes code for reproducing key experiment results. Dataset usage and access instructions are also provided or referenced from prior works.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/CodeSoomiisseeeeeee ssssssosiccyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The paper thoroughly specifies experimental details Section 5. Appendix D and Appendix E, including baselines, datasets, pre-training/fine-tue settings, evaluation protocols, hyperparameter selection, and the optimization strategies applied, etc.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
42

---

Justification: All main accuracy results are reported with standard deviation across 20 runs, and robustness noise and hyperparameters analyzed in Figure and Figure The error bars (std.) and run randomness (few-shot sample selection) is made explicit.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots. The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The paper includes empirical runtime and GPU memory evaluations in Sec- tion 5.4 and Appendix F, where convergence time and memory usage are compared with basclines. The compute resouree setups are also listed in Appendix E.3.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //teer//////ss//// cctppblic//aacccccccccccccccctct wwwhiissoeeeeeeeeeeeee line?
Answer: [Yes]
Justification: The paper complies with the NeurIPS Code of Ethics. uses publicly available datasets with proper citation and does not involve human subjects sensitive data. There is no evidence ethical concerns model design or usage.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
43

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses societal impacts in the broader context of trustworthy graph foundation models in Appendix H. It highlights positive implications in real-world de- ployment under domain shift and acknowledges potential negative applications if generative vocabularies are misused for adversarial structure injection.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e-g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deg.lyeemmm of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper does not release high-risk models (e.g., LLMs, scraped datasets). Although it involves generative components, they constrained and do not pose identifiable dual-use risks requiring explicit safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
44

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Datasets and baselines are properly cited and used under appropriate licenses listed in Appendix E.2. The code base will be accompanied by license information once the paper is accepted for publication.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces GRAVER new framework asset, comprising the disentangled vocabulary banks, graphon-base generative experts, and MoE-CoB routing mechanism. It is well-documented and supported by anonymized code release.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No human subjects or crowdsourced tasks are involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
45

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No IRB approval is needed human subject research was conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: The paper explicitly mentions the use of LLMs (e.g.. GPT-4, BERT tokenizer) for feature alignment and class-aware text enhancement (Section 4.1), which constitutes non-standard component of the methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: // /ttp///eer/////// tteerrps.cc//onfffiee 220ecce/25///2 for what should or should be described.
46


Reviews:
[{"review_id": "FYtkYASEWJ", "reviewer": "Reviewer_v4BW", "summary": "This paper introduces GRAVER, a framework that leverages graphon-based generative modeling to build graph vocabularies for GFMs. It incorporates a MoE-CoE module to enhance the robustness and efficiency of prompt-based fine-tuning. Extensive experiments on few-shot node and graph classification demonstrate that GRAVER consistently outperforms state-of-the-art baselines.", "questions": "While some ideas are novel and the paper is detailed, it gives the impression of overselling its contributions through mathematical and technical complexity. The above weaknesses need to be addressed.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- Well-written with clear structure, making it easy to follow the technical flow.\n- The authors demonstrate strong familiarity with the field, providing detailed explanations and context throughout the paper.\n- The authors pose theoretically grounded architectural motivation, making the contributions feel purposeful.\n- The idea of using graphon-based token (vocabulary) generation is particularly appealing to me.\n- Extensive experiments and diverse directions of analyses are provided.\n\n## Weaknesses\n- Although presented as a minor component, the use of LLM-based alignment appears very powerful; so powerful that one suspects simply designing strong LLM-enhanced features for other SOTA baselines may yield comparable performance to GRAVER, which raises the question of whether the complex GRAVER architecture is truly necessary or if the LLM enhanced features are the main driver.\n- MoE-CoE, while presented with a fancy name, essentially appears to be an \"attention-weighted sum using softmax-normalized weights across domains and classes,\" and the authors should adopt clearer terminology and technical descriptions to reflect its actual simplicity.\n- It is unclear whether Proposition 1 truly supports the claim that \"the upper bound of semantic discrepancy is governed by a specific combination of vocabularies,\" as the formulation seems to relate only to the number of disentangled factors, regardless of whether SIP is well-trained. The logical link to \"disentangled ego-graphs functioning as independent and minimal semantic units\" also remains vague and make a huge logical gap.\n- Adding to Proposition 1, the empirical result \"Removing SIP leads to a minor yet consistent drop ... encouraging semantic diversity across vocabulary channels is beneficial, though not dominant” implies that Proposition 1 may not be well aligned with practical observations.\n- It is unclear why “ogbn-product” was split into \"tech\" and \"home”. Plus, this renaming should be clarified to ensure readers understand it originated from ogbn-product. (I was so confused)", "was_revised": true, "final_justification": "1. Despite the remaining concerns, the authors resolve most of them. So, I will raise my score slightly.\n\n2. The concerns were resolved through subsequent discussions. It remains to be seen whether these changes will be reflected in the actual paper.", "rebuttal": "We sincerely thank the reviewer for their appreciation of our paper’s clear structure, well-grounded motivation, and especially the novelty of the graphon-based vocabulary generation. Our detailed responses are as follows.\n\n---\n\n> **Q1: The performance improvements may mainly stem from LLM-enhanced features.**\n\n**A1:** We thank the reviewer for raising this important concern.\n\n- **Feature-enhancement was applied equally to all baselines.** To ensure a fair comparison, we reconfirm that we used the same LLM-based node feature enhancement (as described in Eq. (1), lines 120~126) consistently **across all datasets** evaluated on all baselines. This ensures that performance gains are attributable to GRAVER’s design, not differences in feature processing.\n- **Using LLM-enhanced features is a widely-adopted practice in recent GFM studies.** Recent GFM works [1-3] (such as ZeroG [1], cited in line 125 where the LLM-enhanced tricks are introduced) regularly use LLMs to enhance node features on text-attributed graphs. These works differ in how LLMs are integrated, but all rely on LLMs’ generalization strength. In our case, this enhancement serves as a standard input processing step, not our major contribution.\n- **Retrieval of original texts contributes more significantly than LLM enhancement.**\nOur ablation in Section 5.5 (Figure 7) separates the effects of raw-text retrieval and LLM enhancement. Results show that **retrieving original node texts contributes more to performance**, particularly for zero-shot adaptation. This highlights the importance of recovering full semantic context (often lost in original features obtained by, *e.g.*, bag-of-words) and aligns with our goal of enabling robust cross-domain GFM.\n- **Additional ablation experiment (GRAVER without any textual enhancement).** To further isolate GRAVER’s contribution, we introduce a new ablation using the **original, non-enhanced node features** (*i.e.*, no text or LLM). Under this strict setting (1-shot node classification), GRAVER still outperforms strong baselines, confirming the effectiveness of its generative vocabulary and hierarchical routing modules. **Results:**\n\n|  | `CiteSeer` (cross-dataset) | `Wiki-CS` (cross-domain) |\n| --- | --- | --- |\n| **GCOPE** | 30.25 | 37.50 |\n| **MDGPT** | 33.15 | 40.22 |\n| **SAMGPT** | 35.67 | 42.64 |\n| **GRAVER** (ours) | **37.81*** | **45.90*** |\n\n---\n\n> **Q2: The MoE-CoE routing mechanism may be functionally simple, despite being presented with complex naming.**\n\n**A2:** Thank you for raising this insightful point. While MoE-CoE involves softmax-based weighting, it differs from that in several key ways:\n\n- **Selective Sparsity vs. Uniform Aggregation.** Unlike typical softmax fusion that softly aggregates all inputs, MoE-CoE is designed to enforce selective sparsity. Through domain- and class-specific projections ($\\mathbf{W}_ {\\text{M}}$, $\\mathbf{W}_{\\text{C}}$ in Eq. (14)), it suppresses irrelevant sources, resulting in targeted and meaningful composition rather than uniform blending.\n- **Adaptive Routing Structure vs. Static Weighting.** Traditional softmax fusion often uses static attention over fixed candidates. In contrast, MoE-CoE dynamically computes routing weights conditioned on each target support instance, enabling fine-grained, task-aware augmentation.\n- **Hierarchical Decision Logic vs. Single-layer Fusion.** MoE-CoE explicitly separates domain-level and class-level routing, which makes the overall decision process more interpretable and modular, especially important when scaling to larger, noisier, or multi-task graph settings.\n\nWe chose the term “MoE-CoE” to reflect this **structured design** and to emphasize GRAVER’s role as a flexible framework, not just a standalone model. We will clarify these distinctions to avoid any misunderstanding.\n\n---\n\n> **Q3: Proposition 1 does not clearly support the claim that the upper bound of semantic discrepancy is governed by a specific vocabulary combination.**\n\n**A3:** We sincerely appreciate this insightful comment. Indeed, at first glance, the final form of the upper bound (Eq. (7) in Proposition 1) appears to depend solely on the number of disentangled factors ($K $). **However, this simplified form is the final theoretical conclusion after several derivation steps.** Importantly, during the derivation (Proof in Appendix C.2~C.3), the mutual information regularization term ($\\mathcal{R}_{\\text{MI}}$, Eq. (6)), induced by the Semantic Independence Promotion (SIP), **explicitly appears** and **constrains the intermediate upper bound** (Eq. (7), [a]).\n\nSpecifically, our key theoretical insight is that by constraining $\\mathcal{R}_ {\\text{MI}}$ during pre-training, the semantic discrepancy ($\\Delta$) can be effectively bounded. **We do not directly optimize this simplified upper bound during training.** Instead, we incorporate the mutual information regularizer $\\mathcal{R}_ {\\text{MI}}$ directly into our pre-training objective (Eq. (8)), implicitly controlling the semantic discrepancy between vocabularies. In other words, the simplified form of the upper bound (depending only on $K$) is precisely as have already explicitly controlled vocabulary independence through SIP ($\\mathcal{R}_{\\text{MI}}$).\n\n---\n\n> **Q4: The logical connection between “disentangled ego-graphs” and “independent, minimal semantic units” is unclear.**\n\n**A4:** In Section 4.1, we explicitly decompose each node’s neighborhood (ego-graph) into multiple disentangled subgraphs, each capturing a distinct semantic aspect. Proposition 1 then theoretically demonstrates that if these subgraphs (or vocabularies) are semantically independent (each vocabulary encodes clearly distinct semantics), the overall semantic representation becomes more stable and robust to minor feature variations.\n\nTherefore, the phrase “independent and minimal semantic units” emphasizes that each disentangled subgraph ideally encodes a unique, non-redundant piece of semantic information. Minimizing semantic redundancy between these vocabularies (enforced by SIP and its mutual-information regularizer $\\mathcal{R}_ {\\text{MI}}$) is crucial for effectively bounding the semantic discrepancy and facilitating stable knowledge transfer across tasks and domains. \n\n---\n\n> **Q5: Empirical results suggest Proposition 1’s theoretical claims might not align with observed practical outcomes.**\n\n**A5:** We appreciate this critical observation. There indeed appears to be a **misunderstanding** stemming from the simplified final expression of the bound in Proposition 1 (Eq. (7)), which depends only on the number of disentangled factors. **If taken literally,** reducing $K$ would minimize the semantic discrepancy, which contradicts the empirical observation that increasing semantic diversity (*i.e.*, larger $K$) improves performance.\n\nWe clarify this misunderstanding as follows: the derived bound (Eq. (7)) explicitly relies on the assumption that semantic independence across vocabularies (enforced via SIP and the regularizer $\\mathcal{R}_{\\text{MI}}$) has already been achieved during pre-training (we also explain this in **A3**). Therefore, Proposition 1 is **not intended as a direct optimization target** (*i.e.*, reducing $K$), but rather as theoretical justification for why controlling semantic redundancy (via SIP) stabilizes the semantic representation.\n\nIn practice, moderate semantic diversity (larger $K$) indeed proves beneficial by capturing richer semantics, as demonstrated by our empirical observations. Thus, there is no contradiction: Proposition 1 provides theoretical backing for controlling semantic independence through $\\mathcal{R}_\\text{MI}$, rather than advocating directly reducing $K$ (given a certain $K$, the upper bound of $\\Delta$ is confirmed).\n\n---\n\n> **Q6: The reason for splitting “ogbn-product” into “tech” and “home” is unclear, and the renaming causes confusion about the dataset’s origin.**\n\n**A6:** We appreciate the reviewer highlighting this confusion. The `ogbn-Tech` and `ogbn-Home` datasets used in our experiments are split from the original `ogbn-Products` dataset, following the same protocol used in related work ZeroG [1].\n\nDue to space constraints, we briefly referenced this in the main text, and we provide full details in Appendix D.1. As described on line 928, we created the two subsets to be **non-overlapping in class space**, which allows us to **simulate realistic in-domain adaptation under class-level shifts**, a setting that better reflects generalization within a single structural distribution.\n\nIn our revision, we will explicitly clarify in the main text to avoid confusion and ensure dataset provenance is transparent.\n\n---\n\n> **Q7: The contributions may feel overstated due to dense mathematical and technical presentation, despite the paper’s novelty and detail.**\n\n**A7:** We sincerely thank the reviewer for acknowledging the novelty of our ideas and the level of detail in the paper. We are especially encouraged by the time and effort taken to carefully engage with both the theoretical and empirical aspects of our work.\n\nThat said, we understand the concern regarding potential over-packaging of technical content. Our intention was to provide clear theoretical grounding for key design choices, not to overstate their complexity. We will be mindful in refining the presentation to ensure that the contributions are communicated with clarity and appropriate emphasis.\n\n---\n\n**We hope our responses have clarified the theoretical and empirical intentions behind our design. If the above points address your concerns, we would greatly appreciate a reconsideration of your score!**\n\n---\n\n> **Refs:**  \n> [1] ZeroG: Investigating cross-dataset zero-shot transferability in graphs, KDD 2024.  \n> [2] One for All: Towards training one graph model for all classification tasks, ICLR 2024.  \n> [3] Exploring the potential of large language models (LLMs) in learning on graphs, KDD 2024."}, {"review_id": "Z87blAcz5m", "reviewer": "Reviewer_92Wv", "summary": "This work presents a novel framework designed to address the challenge of instability and inefficiency when fine-tuning Graph Foundation Models on few-shot tasks.", "questions": "- Can the model handle the potential conflicts in scenarios of semantic-structural mismatch?\n- When facing domains with varying structural differences, how does the routing decision of the MoE-CoE network change with structural heterogeneity?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n- The authors design a novel augmentation strategy. The core idea of creating generative graph vocabularies for in-context support set augmentation is an interesting and direct approach to mitigating the randomness inherent in few-shot learning scenarios.\n- GRAVER builds upon solid theoretical concepts, utilizing ego-graph disentanglement to identify transferable patterns and graphon-based models to construct generative vocabularies, which adds significant technical depth.\n- A comprehensive evaluation of 15 state-of-the-art baselines shows that it systematically addresses research questions covering effectiveness, robustness, and efficiency.\n\nWeaknesses\n- Potential for semantic-structural mismatch. The model uses an LLM to align semantics from text while using separate graph-based methods to handle structure. This creates a potential failure point if the textual descriptions do not faithfully represent the underlying graph topology or node attributes. In cases of ambiguous, inaccurate, or misleading text, the LLM-guided semantic alignment could conflict with the structural realities of the graph, leading the model to generate irrelevant or even harmful augmentations.\n- Missing pre-training cost analysis. The paper emphasizes the efficiency of the fine-tuning phase but omits any discussion of the computational overhead required for the multi-domain pre-training and vocabulary construction. This information is crucial for assessing the overall practicality of the approach.\n- Lack of measurement of structural heterogeneity across domains. The three domains divided in this paper may appear intuitively different, but it is necessary to clarify the extent of structural heterogeneity between these domains. Otherwise, it is difficult to determine whether success stems from the ability to handle truly distinct graph structures or from performance in structurally similar domains.", "was_revised": true, "final_justification": "The authors' rebuttals have addressed most of the concerns, and I have no more questions. I will keep my score unchanged.", "rebuttal": "We sincerely thank the reviewer for the positive and detailed feedback. We are especially encouraged by the recognition of our augmentation strategy, the theoretical depth behind ego-graph disentanglement and vocabulary generation, and the systematic evaluation. Below, we address each of the concerns raised.\n\n---\n\n> **Q1: Risk of semantic-structural mismatch due to using LLM for semantic alignment while structure is modeled independently.**\n\n**A1:** We appreciate the reviewer’s thoughtful observation. GRAVER explicitly addresses this issue through both the design of the semantic enhancement pipeline and the separation of routing versus content generation:\n\n- **LLM-enhanced semantics are strictly derived from retrieved raw texts.** As shown in the illustrative example (Figure 2), our LLM is applied to raw titles and abstracts retrieved for each node to generate **class-aware but semantically consistent** descriptions. We do **not** use hallucinated or externally constructed semantics. This ensures that enhanced text does not drift from the original node meaning, avoiding the risk of introducing misleading semantics.\n- **Textual semantics guide vocabulary selection, not structural generation.** GRAVER’s vocabulary is generated entirely from disentangled ego-graphs via graphon modeling (Section 4.1), independent of any semantic signal. The LLM-enhanced embeddings are only used to compute attention over these pre-generated subgraphs (Section 4.2). This clear decoupling ensures that even if semantic descriptions are noisy, the underlying graph augmentations remain structurally valid and transferable.\n- **Fallback routing ensures robustness in low-quality text scenarios.** In cases where the semantic input is missing, ambiguous, or misleading, GRAVER falls back to CoE routing, which leverages intra-class structural patterns for vocabulary selection. This fallback mechanism is empirically validated in Appendix D.3.4, where routing performance remains stable under partial or ablated semantic inputs.\n\nTaken together, GRAVER is robust to semantic-structural mismatch by: **(1)** grounding semantic enhancement in original text, **(2)** separating routing from graph construction, and **(3)** using structure-only fallback when necessary. We will make these points clearer in Section 4.2 and include the illustrative example to aid understanding.\n\n---\n\n> **Q2: Missing pre-training cost analysis. While fine-tuning is efficient, the cost of multi-domain pre-training and vocabulary construction is unclear.**\n\n**A2:** We appreciate the reviewer’s concern. While GRAVER focuses on improving **fine-tuning efficiency**, we agree that understanding pre-training complexity is important, and we clarify the following:\n\n- **We have explicitly analyzed the computational complexity of pre-training.** In Appendix B, we provide a detailed breakdown of the time and space complexity for both the pre-training stage (Algorithm 1) and fine-tuning stage (Algorithm 2). The pre-training involves subgraph disentanglement and conditional graphon fitting, both of which are lightweight and performed once per source domain. The complexity scales with the number of source domains and the size of tokenized subgraphs, which remain small.\n- **Efficiency at pre-training is not the core objective of GRAVER.** Like other Graph Foundation Models [1-5] and large models for languages and vision, the community has reached a consensus that the pre-training phase is a **one-time cost**. Our focus is on **designing a modular framework** that enables **robust and efficient fine-tuning**, which is the real bottleneck in the real-world scenarios.\n- **Practicality comes from reusable vocabulary and high adaptation efficiency.** Once pre-trained, GRAVER reuses a shared vocabulary across domains and tasks. This allows rapid downstream tuning, as demonstrated in our speedup results (Section 5.4, Figure 6). In settings with numerous or evolving tasks, this efficiency gain is far more critical than saving pre-training time.\n\nWe will clarify this perspective and move the complexity analysis reference to the main text for better visibility.\n\n---\n\n> **Q3: Lack of measurement of structural heterogeneity across domains. The paper does not clarify whether domains differ significantly in structure.**\n\n**A3:** We appreciate the reviewer’s thoughtful comment. GRAVER is designed precisely to address the challenge of **structural heterogeneity across domains**, which is a core motivation for its generative vocabulary design and hierarchical routing mechanism. \n\n- **Clear domain-level structural differences exist across datasets.** We categorize datasets into three domains: *Citation*, *Co-purchase*, and *Web Link*. Although this categorization may appear intuitive, we reinforce it using measurable structural statistics, including average degree, node homophily, and class count. These are reported below (also detailed in Table D.1 and Appendix D):\n\n| Dataset | Domain | Avg. # Deg. | Homophily |\n| --- | --- | --- | --- |\n| `Cora` | Citation | 4.00 | 0.81 |\n| `CiteSeer` | Citation | 2.57 | 0.74 |\n| `PubMed` | Citation | 4.50 | 0.80 |\n| `ogbn-arXiv` | Citation | 13.77 | 0.65 |\n| `ogbn-Tech` | Co-purchase | 87.60 | 0.87 |\n| `ogbn-Home` | Co-purchase | 26.93 | 0.73 |\n| `Wiki-CS` | Web Link | 36.94 | 0.65 |\n\nThese statistics show clear structural divergence:\n\n**(1)** *Citation Domain:* tend to be **sparser**, with **lower average degree** and **moderate homophily**.\n\n**(2)** *Co-purchase Domain:* **highly dense**, with **higher degrees and homophily**, and reflect **different linking principles** (co-buying patterns rather than citations).\n\n**(3)** *Web Link Domain:* its edges are based on hyperlink co-occurrence, and the **structural connectivity is medium-dense with flatter hierarchy**.\n\n- **Structural heterogeneity is central to the motivation of GRAVER.** Unlike approaches that rely on matching structures across domains, GRAVER embraces heterogeneity by using **generative vocabulary subgraphs** as **structure-agnostic, transferable prompts**. This avoids brittle assumptions about topology alignment. Our hierarchical routing (MoE + CoE) learns to dynamically select suitable vocabularies depending on **local task context**, instead of forcing hard alignment between structurally divergent graphs.\n- **GRAVER’s empirical success confirms its robustness to structural gap.** Our results in cross-domain and zero-shot settings (*e.g.*, `PubMed` → `ogbn-Tech`, or `WikiCS` → `ogbn-arXiv`) demonstrate that the model can transfer knowledge across domains with **substantial topological discrepancy**. This confirms that the design does not exploit structural similarity, but rather overcomes **topological mismatch** through prompt generalization and adaptation.\n\nWe hope this response clarifies the structural heterogeneity across domains and strengthens the justification for the design and evaluation protocol of GRAVER.\n\n---\n\n> **Q4: Potential concern about semantic-structural mismatch when using LLM-enhanced features.**\n\n**A4:** Thank you for raising this concern. GRAVER is explicitly designed to mitigate semantic-structural mismatch through a decoupled architecture: we retrieve raw node-associated texts and encode them with a pre-trained LLM to ensure semantic prompts are faithful to node attributes, while structural augmentation is separately modeled via disentangled ego-graphs. The MoE-CoE routing maintains this separation by handling semantic and structural cues independently. As shown in Section 5.5 (Figure 7), GRAVER remains robust under weak textual alignment and benefits further from high-quality semantics, confirming the effectiveness of this design.\n\n---\n\n> **Q5: How the MoE-CoE routing mechanism responds to structural heterogeneity across domains.**\n\n**A5:** GRAVER is designed to **explicitly address domain-level structural heterogeneity** through its hierarchical routing mechanism: the MoE layer assigns weights across domains, while the CoE layer routes over subgraph prototypes within each domain. To evaluate how routing behavior changes with structural divergence, we illustrate the routing weights of MoE layer under cross-dataset and cross-domain settings. **Results:**\n\n|  | Cross-Dataset | Cross-Domain |\n| --- | --- | --- |\n| Source | `Cora` + `PubMed` + `ogbn-Home` + `Wiki-CS`  | `Cora` + `CiteSeer` + `PubMed` + `Wiki-CS` |\n| Target | `Cora` | `ogbn-Home` |\n| MoE Routing Weights | [**0.435732**, **0.411561**, 0.065877, 0.086830] | [0.241328, 0.194376, 0.199192, 0.365104] |\n\nIn the **cross-dataset** setting, the routing weights concentrate on `Cora` **(0.44)** and `PubMed` **(0.41)** (datasets with similar structures), while assigning much lower weights to `ogbn-Home` and `Wiki-CS`. This reflects MoE’s preference for structurally aligned sources.\n\nIn contrast, in the **cross-domain** case targeting `ogbn-Home`, a structurally distinct dataset, the routing becomes **more balanced across all domains** (0.19~0.36), indicating no clear structural match. This suggests GRAVER **aggregates knowledge from diverse sources** to support adaptation when structural alignment is weak.\n\nOverall, the results verify that GRAVER’s MoE routing mechanism **adapts to structural heterogeneity** by emphasizing aligned sources or diversifying when necessary.\n\n---\n\n**We sincerely thank the reviewer again for the thoughtful feedback. We hope our clarifications have addressed all concerns.**\n\n---\n\n> **Refs:**  \n> [1] ZeroG: Investigating cross-dataset zero-shot transferability in graphs, KDD 2024.  \n> [2] One for All: Towards training one graph model for all classification tasks, ICLR 2024.  \n> [3] How much can transfer? BRIDGE: Bounded multi-domain graph foundation model with generalization guarantees, ICML 2025.  \n> [4] AutoGFM: Automated graph foundation model with adaptive architecture customization, ICML 2025.  \n> [5] Multi-domain graph foundation models: Robust knowledge transfer via topology alignment, ICML 2025."}, {"review_id": "tj7N53vJIH", "reviewer": "Reviewer_m8FB", "summary": "This paper proposes GRAVER, a framework for robust few-shot fine-tuning of GFMs. It constructs generative graph vocabularies via ego-graph disentanglement and graphon modeling, and integrates them using a MoE-CoE routing network to enhance support sets. GRAVER improves stability and performance across multi-domain tasks.", "questions": "1. Why choose graphons over learnable generative models? Have you compared graphon-based vocabularies to alternatives like GraphVAEs, autoregressive generators, or graph diffusion models in terms of transferability and fidelity?\n2. Are vocabularies general or domain-specific? Can a vocabulary generated from one domain (e.g., citation networks) effectively help in a structurally different one (e.g., co-purchase or molecular graphs)? Some cross-domain reuse metrics would be valuable.\n3. What happens if the vocabulary is misaligned or noisy? If the generated subgraphs do not match the target domain well, does the MoE-CoE mechanism effectively filter them out, or does performance degrade significantly?\n4. For other questions, please refer to the Weaknesses.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The instability of GFM fine-tuning in few-shot settings is a well-acknowledged issue, and the paper addresses it head-on.\n2. Using ego-graph disentanglement combined with graphon modeling provides a novel way to build transferable subgraph units, going beyond static motif-based vocabularies.\n3. The model maintains stable performance under structural and feature perturbations, showing that its generative vocabulary helps resist noise in both the topology and attributes. It provides mathematical bounds on transferability and convergence of generated vocabularies, giving formal justification for the design.\n\nWeaknesses:\n\n1. The framework involves multiple heavy components: LLM-enhanced features, ego-graph disentanglement, graphon estimation, MoE-CoE routing, which may be hard to reproduce, extend, or deploy in real-world systems.\n2. While the model assumes vocabulary transfer, it lacks empirical analysis of how well vocabularies learned in one domain generalize to another. Are they reusable or domain-specific?\n3. Although the paper includes several propositions and proofs, they remain abstract and are not linked back to practical model design or interpretation. Bounds exist but don’t yield intuitive guidance for design or practical decisions.", "was_revised": true, "final_justification": "Most of my concerns have been solved. My score keeps the same, as it have already revealved my positive evaluation towards its acceptance.", "rebuttal": "We sincerely thank the reviewer for recognizing the importance of fine-tuning instability in GFMs, and for appreciating our use of ego‑graph disentanglement, graphon modeling, and both theoretical and empirical validations. Our detailed response is as follows.\n\n---\n\n> **Q1: The multi-component design of GRAVER may hinder reproducibility and real-world deployment.**\n\n**A1:** We acknowledge the reviewer’s concern and clarify that GRAVER is structured as a modular and reproducible pipeline. Each component is lightweight, well-isolated, and intentionally designed to support extensibility:\n\n- **LLM-enhanced features are derived using open-source toolchains.** we retrieve raw node-associated texts and process them via pre-trained models. No fine-tuning or training of the LLM is required, ensuring ease of use.\n- **Ego-graph disentanglement is a non-learnable and deterministic decomposition process** that extracts $K$ structural factors per node via topological similarity and structural entropy, without any training or tuning overhead.\n- **Graphon estimation is conducted on a small number of factor-aligned subgraphs** using empirical neighborhood averaging, making it stateless and reusable. No complex optimization or learning is involved.\n- **MoE-CoE routing involves only two small learnable vectors per task**, applied during fine-tuning. The design is deliberately simple and plug-in compatible with transformer decoders.\n\nFurthermore, we emphasize that all pre-training and vocabulary generation are conducted offline, making fine-tuning fast, memory-efficient, and readily deployable in practice. Our released code includes full scripts and default settings for all modules to ensure reproducibility.\n\n---\n\n> **Q2: The generalizability of vocabularies across domains is unclear.**\n\n**A2:** Thank you for raising this important point. Our vocabularies are **domain-specific during pre-training**, but become **reusable across tasks within that domain** during fine-tuning. This design aligns with GRAVER's modular \"pretrain-then-finetune\" paradigm. Each domain undergoes preprocessing to construct a vocabulary using disentangled ego-graphs and graphon modeling. Once generated, these vocabulary tokens are **shared across all tasks in that domain**, including **zero-shot** and **cross-dataset** scenarios, as demonstrated throughout our experimental settings (Section 5.1~5.4). This approach inherently validates that our vocabularies, though initially domain-specific, possess strong reusability and generalization capacity across diverse downstream tasks.\n\nFurthermore, to confirm that the routing network indeed utilizes domain-specific vocabularies effectively, we illustrate the routing weights of MoE layer under cross-dataset and cross-domain settings. **Results:**\n\n|  | Cross-Dataset | Cross-Domain |\n| --- | --- | --- |\n| Source | `Cora` + `PubMed` + `ogbn-Home` + `Wiki-CS`  | `Cora` + `CiteSeer` + `PubMed` + `Wiki-CS` |\n| Target | `Cora` | `ogbn-Home` |\n| MoE Routing Weights | [**0.435732**, **0.411561**, 0.065877, 0.086830] | [0.241328, 0.194376, 0.199192, 0.365104] |\n\nIn the **cross-dataset** setting, the routing weights concentrate on `Cora` **(0.44)** and `PubMed` **(0.41)** (datasets with similar structures), while assigning much lower weights to `ogbn-Home` and `Wiki-CS`. This demonstrates how the routing network adaptively prioritizes vocabularies from structurally aligned sources.\n\nIn contrast, in the **cross-domain** case targeting `ogbn-Home`, a structurally distinct dataset, the routing becomes **more balanced across all domains** (0.19~0.36), indicating no clear structural match. This confirms that GRAVER's routing mechanism can automatically adjust based on the structural divergence between source and target domains, **drawing knowledge from multiple diverse sources** when structural alignment is weak.\n\nThe results show that MoE adaptively selects vocabulary tokens most relevant to the target domain or task, demonstrating that GRAVER not only supports vocabulary reuse, but also dynamically mitigates potential mismatch through informed routing. We will clarify this distinction more explicitly in the revised version.\n\n---\n\n> **Q3: Theoretical bounds in the paper remain abstract and are not clearly connected to model design or practical interpretation.**\n\n**A3:** We appreciate this feedback and agree that theoretical insights are most valuable when they inform practical design. In GRAVER, the **propositions serve as theoretical justifications** for key architectural decisions:\n\n- **Proposition 1** provides a bound on semantic discrepancy across vocabulary tokens, showing that the bound is governed by the **number of disentangled semantic factors (**$K$**)** and the mutual information regularizer $\\mathcal{R}_{\\text{MI}}$. This directly motivates the use of **semantic information preservation (SIP)** during pretraining to ensure well-separated vocabulary channels, which improves transferability. The corresponding empirical observation: removing SIP leads to a performance drop, aligns with this bound.\n- **Proposition 2** provides convergence guarantees for the **graphon-based generation** of vocabulary tokens, ensuring that the learned distributions faithfully capture ego-graph structures. This supports the use of **graphon modeling over more black-box neural generators** (*e.g.*, VAEs or diffusion), especially under low-data regimes.\n\nTo make these links clearer, we have now added direct references from our model design choices back to the propositions in the appendix. While the bounds may appear abstract, they serve to **constrain the design space and ensure that each component: disentanglement, generation, routing, has theoretical grounding**, guiding our practical implementation.\n\n---\n\n> **Q4: Why use graphons instead of neural generative models (GraphVAEs, diffusion, autoregressive)? Any comparisons in terms of transferability and fidelity?**\n\n**A4:** We chose **graphons** as the generative backbone for vocabulary construction due to their **nonparametric, data-efficient nature** and **theoretical interpretability**. Unlike GraphVAEs or diffusion-based generators, which require large training sets to generalize well and often involve high variance in generation, graphons allow **explicit modeling of graph distributional structure** with **provable convergence** (see Proposition 2). This makes them particularly well-suited for **few-shot and zero-shot** settings, where structural prior needs to be distilled from limited examples.\n\nWhile we did not perform exhaustive comparisons with all generative baselines, we emphasize that GRAVER’s vocabulary is designed not for raw graph generation, but for producing **task-adaptive, transferable subgraphs** under domain shift. Graphon-based generation gives us **explicit control over structural smoothness and heterogeneity**, aligning well with our routing and prompt integration mechanisms.\n\n---\n\n> **Q5: Are vocabularies general or domain-specific? Can vocabularies generated in one domain generalize to structurally different ones?**\n\n**A5:** Please refer to our response to **Q2**. \n\nIn short, the vocabularies in GRAVER are fundamentally **domain-specific**, as they are constructed from disentangled ego-graphs and graphons fitted to the local structures observed in each domain during pre-training. However, once pre-trained, these vocabularies are **reusable across tasks and datasets within the same domain**, supporting efficient few-shot adaptation.\n\nImportantly, all our **cross-dataset and cross-domain experiments** (Section 5.2) are conducted **under the vocabulary reuse setting**: we do not re-train or regenerate vocabularies in the target domain. This empirically verifies that the pre-trained vocabularies, though learned within source domains, **retain sufficient generality** to support downstream adaptation in unseen domains.\n\nThe additional results in **A2** show that the model dynamically reweights vocabulary channels depending on the domain and task, indicating effective **domain-aware vocabulary reuse**. While we agree that cross-domain reuse metrics are an interesting direction, our current results already demonstrate strong generalization behavior, suggesting that the learned vocabularies, while structurally grounded in source domains, are **transferable enough** to support adaptation to distinct domains.\n\n---\n\n> **Q6: What happens if the vocabulary is misaligned or noisy? Can the MoE-CoE mechanism effectively filter out mismatched subgraphs?**\n\n**A6:** GRAVER is explicitly designed to handle vocabulary mismatch through its hierarchical routing architecture. The **MoE layer** operates at the inter-domain level, selecting relevant vocabulary sets based on the target domain’s global characteristics, while the **CoE layer** works intra-domain, selecting the most semantically aligned subgraphs within a vocabulary set. This two-level gating mechanism helps the model **dynamically downweight irrelevant or noisy patterns**, even when vocabulary and target domain structures are not perfectly aligned.\n\nIn essence, while vocabulary mismatch may introduce noise, the MoE-CoE mechanism provides **resilience through selective routing**, ensuring that only the most compatible augmentations contribute to downstream prediction.\n\n---\n\n**We sincerely thank the reviewer for the thoughtful and constructive feedback. We hope our responses have addressed all concerns.**"}, {"review_id": "2MiSG4xNK0", "reviewer": "Reviewer_HK8Q", "summary": "This paper proposes a graph foundation model named **GRAVER**. The authors argue that existing graph foundation models often underperform in few-shot fine-tuning scenarios, primarily due to the randomness in support set selection and discrepancies between pretraining and target graph structures. To address these challenges, GRAVER leverages **generative augmentation**.\n\nSpecifically, the model identifies transferable components by extracting **class-specific subgraph patterns**. Furthermore, it constructs a unified task template based on **ego-graph similarity**, and builds a **graph vocabulary** using generative experts grounded in graph primitives. The effectiveness of GRAVER is validated through extensive experiments across multiple datasets.", "questions": "Please address the weaknesses listed in the previous section.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The authors conducted extensive experiments to demonstrate the effectiveness of the proposed model.\n\n2. Through a series of ablation studies, the authors demonstrate the effectiveness of each component of the proposed model.\n\n3. The authors demonstrate the effectiveness of the proposed model not only through empirical results but also through mathematical analysis.\n\n\n**Weaknesses**\n\n1. The authors incorporate too many techniques in this paper, which makes the overall presentation appear unfocused. As a result, it becomes unclear to readers which components are original contributions and which are adapted from prior work. The authors are encouraged to emphasize the key techniques proposed in this paper and clearly distinguish them from existing methods.\n\n2. The authors focus on the few-shot fine-tuning setting; however, it is unclear why traditional **few-shot graph learning** works are not discussed. The authors are encouraged to include a dedicated subsection in the related work section comparing their method with existing approaches in few-shot graph learning. For example, works such as **[1], [2], and [3]** in **few-shot node classification**, and **[4], [5], and [6]** in **few-shot graph classification** should be considered for a more comprehensive and contextualized discussion.\n\n[1] Meta-gnn: On few-shot node classification in graph meta-learning. \\\n[2] Few-shot node classification on attributed networks with graph meta-learning. \\\n[3] Dual-level Mixup for Graph Few-shot Learning with Fewer Tasks. \\\n[4] Adaptive-step graph meta-learner for few-shot graph classification. \\\n[5] Faith: Few-shot graph classification with hierarchical task graphs. \\\n[6] A simple but effective approach for unsupervised few-shot graph classification.\n\n3. The pretraining graph datasets used in this work are generally small in scale. Evaluating the model on large-scale datasets with millions of nodes would enhance its practical applicability, as real-world graphs often contain hundreds of millions of nodes.", "was_revised": true, "final_justification": "The authors’ response has partially addressed my concerns; therefore, I still maintain a positive attitude toward this paper.", "rebuttal": "We thank the reviewer for the positive and constructive feedback. We are encouraged that the reviewer acknowledges the novelty and thoroughness of our method, the rigor of our ablation studies, and the balance between theoretical grounding and empirical validation. Below, we address the raised concerns in detail.\n\n---\n\n> **Q1: The paper includes too many techniques, making the presentation feel unfocused and blurring the line between original contributions and existing works.**\n\n**A1:** We thank the reviewer for this insightful comment. GRAVER indeed designs multiple components, but all are organized around a unified goal: **building and reusing a transferable graph vocabulary to support robust graph foundation models.** To avoid possible confusion, we clarify the core contributions of GRAVER as follows:\n\n- **Generative Graph Vocabulary Construction.** GRAVER proposes a new paradigm where class-specific subgraph patterns are extracted and disentangled into semantic primitives, forming a reusable vocabulary of graph tokens. This vocabulary captures transferable structural and semantic features across domains.\n- **Vocabulary-Guided Prompt Augmentation.** Rather than relying on raw node features or full-graph encoding, GRAVER constructs prompts by selectively assembling vocabulary tokens based on task-specific prototypes. This design improves sample efficiency and enhances generalization in few-shot settings.\n- **Hierarchical Routing with MoE-CoE.** To adaptively reuse vocabulary tokens across domains and classes, GRAVER introduces a two-level routing mechanism. The first level selects domain-relevant experts (MoE), and the second level refines the token composition within each class (CoE), enhancing both robustness and specificity.\n\nOther components, such as feature enhancement or retrieval strategies, serve as auxiliary modules to support this vocabulary-centric workflow and are aligned with prior literature, which also have been properly cited. In the revision, we will emphasize the above core contributions more clearly in the method overview and contribution summary.\n\n---\n\n> **Q2: The paper lacks discussion of traditional few-shot graph learning methods.**\n\n**A2:** Thank the reviewer for the thoughtful suggestion. We emphasize that GRAVER targets a different goal:\n\n- **Few-shot fine-tuning is a standard setting in GFM literature.** We adopt the few-shot fine-tuning setting following recent GFM studies, where the goal is to assess cross-domain generalization with limited task-specific supervision. This is a widely used protocol for evaluating foundation models’ adaptability in both vision and language domains, and increasingly in graph learning.\n- **GRAVER is fundamentally distinct from traditional few-shot graph learning.** The objective of few-shot graph learning works is to design models specifically optimized for task-level generalization under few-shot constraints, often via episodic training or meta-learning. In contrast, GRAVER is not designed for task-level few-shot learning, but rather as a **foundation model** that can generalize across graph domains and tasks, with few-shot fine-tuning merely serving as a test modality.\n- **From the GFM perspective, zero-shot generalization is the ultimate goal.** Especially with the integration of textual and LLM-enhanced knowledge, the focus of GFM is shifting toward **zero-shot adaptability** across unseen domains. In this context, few-shot fine-tuning serves as a middle ground to evaluate adaptability under minimal supervision, but is not the design target of GRAVER.\n\nAlso, we acknowledge that the current draft lacks an explicit discussion of traditional few-shot graph learning works, we will include the following subsection in the revised manuscript:\n\n> **Related Work: Few-shot Graph Learning.** (Additional)  \n> Few-shot graph learning addresses the challenge of learning on graphs with extremely limited labeled data per class, and has been widely studied in both node classification and graph classification. Inspired by meta-learning successes in vision and text domains [1], early studies adapted meta-learning to graphs but encountered unique challenges due to their relational and non-Euclidean structure. To address this, researchers developed graph-specific meta-learning frameworks based on Graph Neural Networks (GNNs) for fast adaptation to new classes with minimal supervision [1].  \n> **Few-shot node classification.** Early methods combined GNNs with optimization-based meta-learning. Meta-GNN [1] incorporated Model-Agnostic Meta-Learning (MAML), training on simulated tasks to learn transferable initializations for novel classes. Metric-based alternatives such as Graph Prototypical Networks (GPN) [2] created class prototypes in embedding space to classify query nodes efficiently. G-Meta [3] enhanced this by building local subgraphs and using second-order optimization to capture task-specific structure. Meta-GPS [4] introduced learnable scaling and shifting transformations to refine prototype representations, further improving task adaptability under a MAML framework. To cope with limited task diversity, SMILE [5] proposed dual-level mixup within and across tasks to synthesize new nodes and even novel tasks, thereby alleviating overfitting and improving generalization.  \n> **Few-shot graph classification.** Optimization-based methods have proven effective. Adaptive-Step Graph Meta-Learner [6] dynamically adjusted the gradient steps in the inner loop through a trainable controller, capturing discriminative substructures of unseen classes more effectively. FAITH [7] modeled inter-task relationships via a hierarchical task graph, organizing tasks, classes, and graphs into a three-level structure, and selecting related tasks using a loss-guided sampling strategy. These methods demonstrated strong performance, particularly when labeled training graphs were sparse.  \n> Recent advances have also explored unsupervised strategies. SMART [8] eliminates meta-learning entirely, pre-training a GNN on large-scale unlabeled graphs using self-supervised objectives and a novel graph mixup technique. During few-shot adaptation, it applies prompt tuning by freezing the encoder and updating a small set of task-specific parameters. This lightweight adaptation yields strong results, often surpassing supervised meta-learners, and highlights the promise of leveraging large unlabeled datasets for few-shot graph learning.\n\n---\n\n> **Q3: The pre-training datasets are relatively small. Evaluating on larger graphs with hundreds of millions of nodes would better demonstrate real-world applicability.**\n\n**A3:** We appreciate the reviewer’s suggestion regarding scalability. We agree that scaling graph foundation models (GFMs) to massive real-world graphs is an important direction. However, we intend to clarify several contextual points:\n\n- **GFM research is still in its formative stage.** The community has not yet converged on standardized benchmarks, architecture formations, or evaluation protocols for GFMs. Current efforts primarily focus on understanding generalization across domains and tasks, especially in few-shot and zero-shot settings. To our knowledge, **no existing GFM studies have evaluated models at the scale of hundreds of millions of nodes**.\n- **Our dataset scale already surpasses prior GFM works.** Compared to recent GFM literatures [9-13], our pre-training corpus reaches **over 250,000 nodes and 3.5 million edges** across multiple graph datasets. This places GRAVER **among the largest-scale settings** used for GFM pre-training in existing academic works.\n- **Scalability remains a core challenge under constrained resources.** As detailed in Appendix E.3, our experiments were conducted on an 80GB A100 GPU. Scaling to graphs in the hundreds of millions is beyond the scope of current infrastructure, and remains a critical open challenge in graph learning community. That said, we view this as a vital future direction for GFM deployment in real-world applications.\n\nWe will make these clarifications explicit in the revised manuscript.\n\n---\n\n**We sincerely thank the reviewer for the constructive comments. We have clarified the core contributions, distinguished our setting from traditional few-shot methods, and discussed scalability concerns. We hope these responses address the concerns and would greatly appreciate a score revision!**\n\n---\n\n> **Refs:**  \n> [1] Meta-GNN: On few-shot node classification in graph meta-learning, CIKM 2019.  \n> [2] Graph prototypical networks for few-shot learning on attributed networks, CIKM 2020.  \n> [3] Graph meta learning via local subgraphs, NeurIPS 2020.  \n> [4] Few-shot node classification on attributed networks with graph meta-learning, SIGIR 2022.  \n> [5] Dual-level mixup for graph few-shot learning with fewer tasks, WWW 2025.  \n> [6] Adaptive-step graph meta-learner for few-shot graph classification, CIKM 2020.  \n> [7] FAITH: Few-shot graph classification with hierarchical task graphs, IJCAI 2022.  \n> [8] A simple but effective approach for unsupervised few-shot graph classification, WWW 2024.  \n> [9] ZeroG: Investigating cross-dataset zero-shot transferability in graphs, KDD 2024.  \n> [10] One for All: Towards training one graph model for all classification tasks, ICLR 2024.  \n> [11] How much can transfer? BRIDGE: Bounded multi-domain graph foundation model with generalization guarantees, ICML 2025.  \n> [12] AutoGFM: Automated graph foundation model with adaptive architecture customization, ICML 2025.  \n> [13] Multi-domain graph foundation models: Robust knowledge transfer via topology alignment, ICML 2025."}]

Author rebuttal:

Meta-review: The paper proposes a generative graph vocabulary construction based on ego-graph disentanglement and graphon modeling to create transferable subgraph patterns. These vocabularies are integrated via a hierarchical Mixture-of-Experts (MoE) and Class-of-Experts (CoE) routing mechanism to enhance robustness in cross-domain and few-shot node/graph classification tasks. The authors claim that GRAVER outperforms 15 state-of-the-art baselines in effectiveness, robustness, and efficiency, supported by extensive experiments and theoretical analyses (Propositions 1 and 2) that ensure semantic stability and transferability.

Reviewers point some weakness of the paper, including:

Scalability Concerns: The evaluation is limited to graphs with up to 250,000 nodes, which, while substantial for GFM research, does not address real-world graphs with millions of nodes;

Limited Discussion of Traditional Few-Shot Methods: The initial submission lacked a comparison with traditional few-shot graph learning approaches, though this was addressed in the rebuttal with a proposed related work section.

Empirical Validation of Vocabulary Transfer: While the authors claim vocabulary reusability, more explicit cross-domain reuse metrics could strengthen the evidence

The authors’ rebuttal was thorough, addressing most concerns with additional experiments, clarifications, and commitments to revise the manuscript for clarity and transparency. The consensus among reviewers leaned toward acceptance.
Decision: Accept (poster)