Title: GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning
Venue: NeurIPS 2025
Accepted: yes
Keywords: graph foundation models

multi-domain graph pre-training

graph transfer learning

graph prompt fine-tuning
Abstract: Inspired by the remarkable success of foundation models in language and vision, Graph Foundation Models (GFMs) hold significant promise for broad applicability across diverse graph tasks and domains. However, existing GFMs struggle with unstable few-shot fine-tuning, where both performance and adaptation efficiency exhibit significant fluctuations caused by the randomness in the support sample selection and structural discrepancies between the pre-trained and target graphs. How to fine-tune GFMs robustly and efficiently to enable trustworthy knowledge transfer across domains and tasks is the major challenge. In this paper, we propose **GRAVER**, a novel **G**enerative g**RA**ph **V**ocabulari**E**s for **R**obust GFM fine-tuning framework that tackles the aforementioned instability via generative augmentations. Specifically, to identify transferable units, we analyze and extract key class-specific subgraph patterns by ego-graph disentanglement and validate their transferability both theoretically and empirically. To enable effective pre-training across diverse domains, we leverage a universal task template based on ego-graph similarity and construct graph vocabularies via graphon-based generative experts. To facilitate robust and efficient prompt fine-tuning, we grave the support samples with in-context vocabularies, where the lightweight MoE-CoE network attentively routes knowledge from source domains. Extensive experiments demonstrate the superiority of GRAVER over **effectiveness**, **robustness**, and **efficiency** on downstream few-shot node and graph classification tasks compared with **15** state-of-the-art baselines.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a graph foundation model named **GRAVER**. The authors argue that existing graph foundation models often underperform in few-shot fine-tuning scenarios, primarily due to the randomness in support set selection and discrepancies between pretraining and target graph structures. To address these challenges, GRAVER leverages **generative augmentation**.\n\nSpecifically, the model identifies transferable components by extracting **class-specific subgraph patterns**. Furthermore, it constructs a unified task template based on **ego-graph similarity**, and builds a **graph vocabulary** using generative experts grounded in graph primitives. The effectiveness of GRAVER is validated through extensive experiments across multiple datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The authors conducted extensive experiments to demonstrate the effectiveness of the proposed model.\n\n2. Through a series of ablation studies, the authors demonstrate the effectiveness of each component of the proposed model.\n\n3. The authors demonstrate the effectiveness of the proposed model not only through empirical results but also through mathematical analysis.\n\n\n**Weaknesses**\n\n1. The authors incorporate too many techniques in this paper, which makes the overall presentation appear unfocused. As a result, it becomes unclear to readers which components are original contributions and which are adapted from prior work. The authors are encouraged to emphasize the key techniques proposed in this paper and clearly distinguish them from existing methods.\n\n2. The authors focus on the few-shot fine-tuning setting; however, it is unclear why traditional **few-shot graph learning** works are not discussed. The authors are encouraged to include a dedicated subsection in the related work section comparing their method with existing approaches in few-shot graph learning. For example, works such as **[1], [2], and [3]** in **few-shot node classification**, and **[4], [5], and [6]** in **few-shot graph classification** should be considered for a more comprehensive and contextualized discussion.\n\n[1] Meta-gnn: On few-shot node classification in graph meta-learning. \\\n[2] Few-shot node classification on attributed networks with graph meta-learning. \\\n[3] Dual-level Mixup for Graph Few-shot Learning with Fewer Tasks. \\\n[4] Adaptive-step graph meta-learner for few-shot graph classification. \\\n[5] Faith: Few-shot graph classification with hierarchical task graphs. \\\n[6] A simple but effective approach for unsupervised few-shot graph classification.\n\n3. The pretraining graph datasets used in this work are generally small in scale. Evaluating the model on large-scale datasets with millions of nodes would enhance its practical applicability, as real-world graphs often contain hundreds of millions of nodes.\n\nQUESTIONS:\nPlease address the weaknesses listed in the previous section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes GRAVER, a framework for robust few-shot fine-tuning of GFMs. It constructs generative graph vocabularies via ego-graph disentanglement and graphon modeling, and integrates them using a MoE-CoE routing network to enhance support sets. GRAVER improves stability and performance across multi-domain tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The instability of GFM fine-tuning in few-shot settings is a well-acknowledged issue, and the paper addresses it head-on.\n2. Using ego-graph disentanglement combined with graphon modeling provides a novel way to build transferable subgraph units, going beyond static motif-based vocabularies.\n3. The model maintains stable performance under structural and feature perturbations, showing that its generative vocabulary helps resist noise in both the topology and attributes. It provides mathematical bounds on transferability and convergence of generated vocabularies, giving formal justification for the design.\n\nWeaknesses:\n\n1. The framework involves multiple heavy components: LLM-enhanced features, ego-graph disentanglement, graphon estimation, MoE-CoE routing, which may be hard to reproduce, extend, or deploy in real-world systems.\n2. While the model assumes vocabulary transfer, it lacks empirical analysis of how well vocabularies learned in one domain generalize to another. Are they reusable or domain-specific?\n3. Although the paper includes several propositions and proofs, they remain abstract and are not linked back to practical model design or interpretation. Bounds exist but don’t yield intuitive guidance for design or practical decisions.\n\nQUESTIONS:\n1. Why choose graphons over learnable generative models? Have you compared graphon-based vocabularies to alternatives like GraphVAEs, autoregressive generators, or graph diffusion models in terms of transferability and fidelity?\n2. Are vocabularies general or domain-specific? Can a vocabulary generated from one domain (e.g., citation networks) effectively help in a structurally different one (e.g., co-purchase or molecular graphs)? Some cross-domain reuse metrics would be valuable.\n3. What happens if the vocabulary is misaligned or noisy? If the generated subgraphs do not match the target domain well, does the MoE-CoE mechanism effectively filter them out, or does performance degrade significantly?\n4. For other questions, please refer to the Weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents a novel framework designed to address the challenge of instability and inefficiency when fine-tuning Graph Foundation Models on few-shot tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The authors design a novel augmentation strategy. The core idea of creating generative graph vocabularies for in-context support set augmentation is an interesting and direct approach to mitigating the randomness inherent in few-shot learning scenarios.\n- GRAVER builds upon solid theoretical concepts, utilizing ego-graph disentanglement to identify transferable patterns and graphon-based models to construct generative vocabularies, which adds significant technical depth.\n- A comprehensive evaluation of 15 state-of-the-art baselines shows that it systematically addresses research questions covering effectiveness, robustness, and efficiency.\n\nWeaknesses\n- Potential for semantic-structural mismatch. The model uses an LLM to align semantics from text while using separate graph-based methods to handle structure. This creates a potential failure point if the textual descriptions do not faithfully represent the underlying graph topology or node attributes. In cases of ambiguous, inaccurate, or misleading text, the LLM-guided semantic alignment could conflict with the structural realities of the graph, leading the model to generate irrelevant or even harmful augmentations.\n- Missing pre-training cost analysis. The paper emphasizes the efficiency of the fine-tuning phase but omits any discussion of the computational overhead required for the multi-domain pre-training and vocabulary construction. This information is crucial for assessing the overall practicality of the approach.\n- Lack of measurement of structural heterogeneity across domains. The three domains divided in this paper may appear intuitively different, but it is necessary to clarify the extent of structural heterogeneity between these domains. Otherwise, it is difficult to determine whether success stems from the ability to handle truly distinct graph structures or from performance in structurally similar domains.\n\nQUESTIONS:\n- Can the model handle the potential conflicts in scenarios of semantic-structural mismatch?\n- When facing domains with varying structural differences, how does the routing decision of the MoE-CoE network change with structural heterogeneity?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces GRAVER, a framework that leverages graphon-based generative modeling to build graph vocabularies for GFMs. It incorporates a MoE-CoE module to enhance the robustness and efficiency of prompt-based fine-tuning. Extensive experiments on few-shot node and graph classification demonstrate that GRAVER consistently outperforms state-of-the-art baselines.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- Well-written with clear structure, making it easy to follow the technical flow.\n- The authors demonstrate strong familiarity with the field, providing detailed explanations and context throughout the paper.\n- The authors pose theoretically grounded architectural motivation, making the contributions feel purposeful.\n- The idea of using graphon-based token (vocabulary) generation is particularly appealing to me.\n- Extensive experiments and diverse directions of analyses are provided.\n\n## Weaknesses\n- Although presented as a minor component, the use of LLM-based alignment appears very powerful; so powerful that one suspects simply designing strong LLM-enhanced features for other SOTA baselines may yield comparable performance to GRAVER, which raises the question of whether the complex GRAVER architecture is truly necessary or if the LLM enhanced features are the main driver.\n- MoE-CoE, while presented with a fancy name, essentially appears to be an \"attention-weighted sum using softmax-normalized weights across domains and classes,\" and the authors should adopt clearer terminology and technical descriptions to reflect its actual simplicity.\n- It is unclear whether Proposition 1 truly supports the claim that \"the upper bound of semantic discrepancy is governed by a specific combination of vocabularies,\" as the formulation seems to relate only to the number of disentangled factors, regardless of whether SIP is well-trained. The logical link to \"disentangled ego-graphs functioning as independent and minimal semantic units\" also remains vague and make a huge logical gap.\n- Adding to Proposition 1, the empirical result \"Removing SIP leads to a minor yet consistent drop ... encouraging semantic diversity across vocabulary channels is beneficial, though not dominant” implies that Proposition 1 may not be well aligned with practical observations.\n- It is unclear why “ogbn-product” was split into \"tech\" and \"home”. Plus, this renaming should be clarified to ensure readers understand it originated from ogbn-product. (I was so confused)\n\nQUESTIONS:\nWhile some ideas are novel and the paper is detailed, it gives the impression of overselling its contributions through mathematical and technical complexity. The above weaknesses need to be addressed.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}