Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Models
Authors: Li Sun
Zhenhao Huang
Silei Chen
Lanxu Yang
Junda Ye
Sen Su
Philip S. Yu
Venue: ICLR 2026 Oral (2026)
Keywords: Multi-domain graph pre-training
graph neural network
graph foundation model
Riemannian geometry
Primary area: learning on graphs and other geometries & topologies
TL;DR: From differential geometry perspective, we present a novel framework that merges multi-domain graphs into a unified, smooth manifold with geometric consistency, enabling quantifiable transferability and geometric scaling behavior.

Abstract: Multi-domain graph pre-training integrates knowledge from diverse domains to enhance performance in the target domains, which is crucial for building graph foundation models. Despite initial success, existing solutions often fall short of answering a fundamental question: how is knowledge integrated or transferred across domains? This theoretical limitation motivates us to rethink the consistency and transferability between the pre-trained model and target domains. In this paper, we propose a fresh differential geometry perspective, whose core idea is to merge any graph dataset into a unified, smooth Riemannian manifold, enabling a systematic understanding of knowledge integration and transfer. To achieve this, our key contribution is the theoretical establishment of neural manifold gluing,
which first characterizes local geometry using an adaptive orthogonal frame and then “glues” the local pieces together into a coherent whole. Building on this theory, we present the GraphGlue framework, which supports batched pre-training with EMA prototyping and provides a transferability measure based on geometric consistence. Extensive experiments demonstrate its superior performance across diverse graph domains. Moreover, we empirically validated GraphGlue’s geometric scaling law, showing that larger quantities of datasets improve model transferability by producing a smoother manifold.

Published as a conference paper at ICLR 2026
MULTI-DOMAIN RIEMANNIAN GRAPH GLUING FOR BUILDING GRAPH FOUNDATION MODELS
Li Sun's Zhenhao Huang?, Silei Chen?, Lanxu Yang", Junda Ye', Sen Su', Philip S. Yu²
BBeijing University of Posts and Telecommunications, Beijing 100876, China North China Electric Power University, Beijing 102206, China 3 University of Illinois Chicago. IL, USA
ABSTRACT
Multi-domain graph pre-training integrates knowledge from diverse domains enhance performance in the target domains, which is crucial for building graph foundation models. Despite initial success, existing solutions often fall short of answering a fundamental question: how is knowledge integraled or transferred across domains? This theoretical limitation motivates us to rethink the consis- tency and transferability between model pre-training and domain adaptation. In this paper. we propose fresh Riemannian geometry perspective, whose core idea is to merge any graph dataset into a unified, smooth Riemannian mani- fold, enabling a systematic understanding of knowledge integration and transfer. To achieve this, key contribution is the theoretical establishment of neural manifold gluing, which first characterizes local geometry using an adaptive or- thogonal frame and then "glues" the local pieces together into coherent whole. Building on this theory, we present the GRAPHGLUE framework, which sup- ports batched pre-training with EMA prototyping and provides transferability measure based on geometric consistence. Extensive experiments demonstrate its superior performance across diverse graph domains. Moreover, empirically validated GRAPHGLUE" geometric scaling law, showing that larger quantities of datasets improve model transferability by producing smoother manifold. Codes are available at httpt::://// http//gitttt cttp///eeeaaaaaaaaaaaaaaaaaaaaaa
INTRODUCTION
Foundation models have revolutionized the representation learning in natural language processing Bommasani et al. (202)); Brown et al. (2020); Devlin et al. (2019) and computer vision Dosovit- skiy et al. (2020) by integrating multi-domain knowledge during pre-training and transferring it to target domains. Graph-structured data are ubiquitous non- Con-Euclidean structures in real-world appli- cations, ranging from social network analysis Zhou et (2020): Sharma et al. (2024) to molecular design Guo et al.) (2022); WWan et al.] (2023). Hence, recent efforts have been made to replicate the success of the foundation model in the field of graphs, achieving multi-domain pre-training and cross-domain transfer learning for graphs.
Multi-domain graph pre-training is challenging given the significant semantic heterogeneity across different domains, such as social networks and biological molecules. In the literature, one line of work extracts multi-domain knowledge via Large Language Models (LLMs), leveraging the well- pretrained textual semantics but remaining limited to text-attributed graphs Zhu et al. (2025); Xia al.) (2024); Tang al.) (2024); Ren al. (2024); [Chnn et al. (2024). However, many real graphs lack explicit textual attributes. Moreover, textual annotation labor-intensive and may introduce hallucinations through LLM generation.
Rather than being tied to textual information, multi-domain pre-training for text-free graphs has gar- nered increasing attention recently. series of methods seek learn shared or invariant knowledge during pre-training using graph codebooks Wangg et al. (2024); Sun et al. (2025b); Jiang et al. (2024);
"Corresponding Author: Li Sun, Isun @ bupt.edu.cn

---

Published as a conference paper at ICLR 2026
Bo et al. 2025), motifs Sun et al.
2025b). computation trees Wang et al. [0024 2025c etc. Mean-
while, advanced adaptation techniques are introduced to improve the downstream tasks, c.g., domain tokens Yu al. (2025a); Jiao et al. (2025); Yuan et al. (2025); Wang al. (2025a) and in-context learning Huang et al. (2023): Liu et al. (2024). While existing solutions have achieved encouraging results, fundamental question remains inadequately addressed: how is knowledge integrated transferred across domains? The theoretical underpinnings in this context remain underexplored. Though Wang et al. (2024); Zhang al. (2024); Ruiz al. (2020) give similarity measures across different domains, they do not frame model pre-traing pre-training and domain adaptation within a consistent framework. This gap limits its ability to assess transfer difficulty, especially for the unseen graphs.
Thus, we are motivated to rethink the consistency and transferability to target domains.
In this paper, we propose fresh differen- tial geometry perspective, whose core is the integration of any graph dataset into a uni- fied, smooth Riemannian manifold, provid- ing rigorous foundation for systematically analyzing knowledge integration and transfer. To achieve this, we introduce a new theory
Computers
Readtt
FB15K_217 257
NIV
Neural Manifold Gluing, whose intuitive idea is to first characterize the local geom- etry, and then "glue" these local pieces to- gether into coherent whole. Specifically, we propose a sparse perturbation and adaptive orthogonal frame to learn the local geometry. Gluing local pieces achieved through met- ric compatibility along the edges (Theorem 4.5) and triangle triviality with respect the concept of holonomy (Theorem (..). Finally, we smooth the manifold by controlling the change ratio of volume elements (Theorem 4.9). enhancing knowledge transport along the manifold.
Figure 1: An illustration of manifold gluing. The domains are distinguished by colors.
Building on the theory established above, we design pre-traniing-adappttion framework named GRAPHGLUE, which extends local geometry to the global scale. During pre-training, we incorporate an Exponential Moving Average (EMA) prototyping before gluing, which distinguishes domain semantics through different locations on the manifold and efficiently handles large-scale graphs in batched manner. In the adaptation phase, GRAPHGLUE employs learnable prompts and a Riemannian Mixture-oo-EExprts,, while gluing tar- get domains to the pre-trained manifold, ensuring geometric consistency. A Geometric Transfer Metric (GTM) is naturally defined by metric compatibility to quantify transfer difficulty. More- over, GRAPIGLUEE exhibits geometric scaling law: larger quantities of graph datasets produce smoother manifold, thereby improving model transferability.
In summary, key contributions are listed as follows. 1. Problem. We investigate the theoretical underpinnings of multi-domain graph pre-training, and study a foundational problem of how knowl- edge integrated and transferred across different domains. 2. Theory. We introduce fresh differ- ential geometry perspective for systematically understanding knowledge transfer, and propose the theory of neural manifold gluing, which consistently integrates multi-domain graphs into a unified, smooth Riemannian manifold via "gluing". 3. Methodology. We propose a GRAPIGLUE frame- work based the above theory, which supports batched pre-training for large-scale graphs and in- corporates natural metric to quantify its transfcrability. 4. Experiment. We evaluate GRAPHGLUE in cross dross-n transfer learning and empirically demonstrate its geometric scaling law.
2 RELATED WORK
Graph Foundation Models Graph Foundation Models (GFMs) aim to provide pre-trainable, general-purpose deep learning architectures for graphs Wang et al. (2025b); Liu et al. (2023). Re- cently, the capabilities of Large Language Models (LLMMs) have extended text-attributed graphs
2

---

Published as a conference paper at ICLR 2026
Zhu et al. (2025): :: Xia et al. 2024); Tang et al. (2024): Ren al. (2024): Chen et al. (2024). Also,
GFMs have been developed for various specialized domains, such as knowledge graphs Huang et al.
2025
Luo et al. (2025), recommender systems Wu et al. (2025). and molecular graphs Xia et al. 2023 Sypetkowski et al. (2024). Given the prevalence of text-free graphs, recent efforts have general-uurpose purpose models via multi-domain pre-training /Zhao et al. 2025)
focused on building general
Multi-domain Graph Pre-training In graph pre-training, Graph Neural Networks (GNNs) are trained by self-supervised learning- either generative Hou al. (2022) or contrastive Veličkovid et al.] (2019); QQu et al.] (2020). In light of the semantic heterogeneity across different domain, several methods have been proposed to learn shared or invariant knowledge Yuan et al. (2025); [Chnn et al. 2225); Wang al. (2025a). Despite the encouraging results, the theoretical foundations of how knowledge is integrated and transferred remain underexplored.
Graph Fine-tuning and Prompt Learning The alignment of pre-trained models with down- stream tasks necessitates an adaptation phase, which is roughly categorized into two paradigms: 1) Graph fine-tuning adapts the model behavior using limited target-domain data (un al. (2024d),
and recent advances introduce parameter-efficien fine-tuning methods such as low-rank adaptation Yang et al. (2025b). 2) Graph prompting keeps pre-trained parameters frozen and enhances perfor-
mance by inserting learnable prompt vectors Yu et al. (2025a;; Liu et al. (2023); Sun et al. (2022b);
Fang et al. (2023). Yet, how quantify the transfer effort target domains remains an open issue.
Riemannian Graph Representation Learning Most existing Riemannian models are tailored to specific tasks Chami et al. 2019); Grover et al. (2025); [Bachmann et al. (2020); Gu et al.) (2019). Recently, (Sun et al. (2025b design new GNN backbone the product manifold for GFM. In con- trast, our focus lies on developing a framework for multi-domain pre- vve-training, and on constructing general manifold, rather specific ones. (Full related work is provided in Appendix E
NOTATIONS AND PRELIMINARIES
This part briefly reviews the key concepts of Riemannian manifold, frame and holonomy, and then states multi-domain pre-training where reconsider its consistency and transferability from fresh differential geometry perspective. Important notations are summarized in Appendix A.
Riemannian Geometry Riemannian geometry provides an elegant framework for studying graphs and structures. A Riemannian manifold (M, G) with dimension M is smooth manifold M en- dowed with Riemannian metric tensor G. Each point p E .M ties to a tangent space T, and its volume element is the determinant of the Riemannian metric tensor, denoted as [G(p). The coordinate chart of tangent space is denoted as (U. xM)) Ricci curvature Ric( X, Y) governs the change ratio of volume elements along the geodesic. The concept of holonomy describes the changes of a tangent vector traversing a closed curve. Rigorous elaborations are in Appendix
Cartan's Method of Moving Frame This renowned method Tron et al. 2024 offers principled way to study manifold geometry with frame. Though Élie Cartan laid the mathematical principle, its deep learning methodology remains largely unexplored. Our work seeks to bridge this gap.
Multi-domain Graph Pre-training In this context, deep learning architecture is first pre-trained on different source domains and then adapted to a target domain. A graph is described as G = (V. E) with feature matrix X E R where V and & denote the node set and edge set, respectively. We consider collection of K graphs S S
SK from I, domains DD {D1, D2 D2,.... DD). A model fe (GNN(.)) is pre-traines pre-trained on the graph dataset G with an encoder GNN(-), after which the pre-trained parameters {O,, (Nx) are frozen. The encoder is implemented with popular graph neural networks such as GCN Kipf & Welling (007)). Given a graph G' of the target domain Dt, the pre- model can generate informative representations for G' with slight adaptation. Note that the target domain can be seen D' E D or unseen D' & D during pre-training Unlike existing solutions, our goal is to design transferable graph model with a principled interpretation.
S
3

---

Published as a conference paper at ICLR 2026
4 THEORY: CONSTRUCTING A UNIFIED, SMOOTH MANIFOLD
Existing solutions often lack a principled framework to interpret how knowledge is integrated or transferred across domains. To fill this gap, we introduce differential geometry perspective for multi-domain graph pre-training. The core of our approach is the construction of a pre-trainable, unified, and smooth Riemannian manifold, which provides a rigorous foundation for systemati- cally analyzing knowledge integration and transfer. In the literature, Riemannian graph representa- tion learning primarily studies the specific manifolds, e.g., hyperbolic spaces Chami et al. (2019);
Yang et al. (2025a), spherical spaces Liu et al. (2022), and product manifolds Gu et al. (0119) However, constructing a general manifold underlying multi-domain graphs remains unexplored.
To achieve this, we establish a novel theory neural manifold gluing, whose intuitive idea is to first characterize the local geometry, and then "glue" these local pieces together to form a unified, smooth Riemannian manifold. Derivations and proofs of our establishment are provided in Appendix B
4.1 LEARNING LOCAL GEOMETRY WITH ADAPTIVE ORTHOGONAL FRAME
In a Riemannian manifold, the local geometry at given point is characterized by its tangent space. Going beyond the classic Cartan's 's method Tron al. (2024), we present deep learning approach to infer the basis of the tangent space. Specifically, we introduce (ks, - perturbation, mimicking the directional derivative Dof = lim,-- f(pt-r))f(p) to generate set of tangent vectors at the given point, after which an adaptive orthogonal frame is applied to form the basis of the tangent space. Note that the perturbation is attached with parametric fGNN in our establishment.
Definition 4.1 ((k, -sssaree perturbation). Given a graph perturbation set that consists of M nodes PP- = {pp:} with parameters {pe). for G = E), the perturbed graph is denoted as G (Й, = G @ = (VU (pm) J E U (Uimn.m pPm)) pp)), R.M ".m..-..=mm) where (Ui,Pm) is edge weighted by an attentive function h( pm) Vie are k nodes selected based on top-k h(x,,, Pmm)
Definition 4.2 (Adaptive Orthogonal Frame, AOF). With tangent vectors generated by the above perturbation and a graph encoder fONN, after QR-decomposition with length recovery, the adaptive orthogonal frame is {w, x+) + wm E ) for every representation x There exists dual frame {mm) such that 6Th (wn) = 8mt, where dul is the Kronecker delta.
We show that the aforementioned length recovery of the basis is important, since the length of the tangent vector, describing the space deformation, is upper-bounded by the perturbation. In fact, the angles and lengths of the basis vectors reflect how the space stretched and twisted, respectively.
Theorem 4.3 (Upper bound of Tangent Vector Length, Appendix B.I.. Given connected G with N nodes, the adjacency matrix A. the Laplacian L, and the feature matrix of perturbation
nodes P. apply (k, M -sparse perturbation to G, suppose N
kM s, where 0 is small, and
added edge weights satisfy Σ, h(vx,P)) = =1 Then, the upper bound || w. S (1 + []+ P|| holds, where wT is the component of w., determined by perturbation.
Thus, the local metric at each point is derived through the basis vectors of its tangent space. In particular, given the representation of Gi 2; Rd. the coordinates U; in neighborhood around z;. and the learned dual frame fm the local metric tensor G; on U, takes the form of G (wm wn (*) = (6)
(m (0 X 0¹), where Imi( (z:) (wm. w ( Equivalently, the matrix form of G is written as
G ( w diag( (og([|w))) 1|., ||wM| 112,
(1)
(1 RdxM The inner product w.r.t. G; is given as (, v) : u." G,v for tangent vectors u, E U.
with the basis of tangent space formulated as W = (w) ..... w
4.2 GLUING LOCAL PIECES TO FORM A SMOOTH MANIFOLD
Given a set of isolated Riemannian manifolds (M = (2()) Ty Tzou U.. G. we are devoted to gluing them together to construct unified, smooth Riemannian manifold with global metric. In a nutshell, These local pieces are connected through the edges and triangles with the concept of holonomy, after which the constructed manifold is smoothed by controlling the Ricci curvature.
4

---

Published as a conference paper at ICLR 2026
Gluing, We begin with the compatibillity of metric along edges, which is necessary for the existence of a global metric. According to Edelsbrunner & Harer, Chung. the gluing boundary can be defined by the adjacency in graph topology. To preserve compatibility, we perform tangent translation along an edge (i. E &, referred to as edge tangent translation, to transform the local metrics. We show that it ensures metric compatibility along edge, and is proven to induce a global metric. In addition, its computational complexity is reduced to (M)) with the QR-decomposition above.
Definition 4.4 (Edge Tangent Translation). Given an edge (i.j) E E, the tangent spaces of its two endpoints T g U, and T aiss and the Riemannian metric of gu,, denoted as G the edge tangent translation is defined as linear map p(-.); Tgu: Tu0 U, on edge (i.j) E E as
plis) = GG/1/² (c/ (C. 1/2 G
(2)
Theorem 4.5 (Tangent Edge Translation as Isometry, Appendix B.2) tion in Definition 4.4 is the optimal solution of
The tangent edge transla-
min PEGL(Ar) GT || Gi GillE=
(3)
where GL denotes the general linear group, such that Gp(Pii p(lD) = Gu v), which induces an isometry g155) between manifold boundaries aU; and J
Theorem 4.6 (Existence of Global Metric, Appendix B.3). Let ((G (pb.)) (as)ee) be local metrics and tangent edge translations. There exists unique global continuous metric G on (U)) NN U, such that the restriction of Glu: = for all
==1
The edge tangent translations connect gluing boundaries in accordance to Theorem 4.5 and 4.6 However, when gluing along higher-order motifs, such as triangles and cycles, some offsets may occur when going round trips, so that gluing boundaries are not well aligned. In other words, although the glued manifold is connected, it is not yet continuous everywhere. To address this issue, we introduce the concept of holonomy, describing how the tangent vector changes when traversing along a closed curve, and define holonomy map measure the changes. We show that, when the holonomy map of triangles is trivial, the offset at the gluing boundaries is eliminated.
Definition 4.7 (Holonomy Map and Holonomy Luss). Let Z (9) denote the real vector space of l-yyclss graph G = EV) under symmetric difference. For any cycle C (io, io....,.,,1 is io), its holonomy map is defined as the composition of transport maps along the path,
H(C)== II pliedee)) E GLM(M).
(4)
The collection : (p(1)) } is said to be trivial if (C) is the identity map for VC E (9). Given the set of all triangles Aijk ((1,, vy), (uz, vx)), the corresponding holonomy loss is formulated as
Choto (9) = [AI
Ex,,
pikks) p(t p(i.) - -
(5)
Asyx
Theorem 4.8 (Triangle Triviality, Appendix B.4]. If every edge belongs to at least one triangle, and HTT) I for all triangular cycles TT in 9, then (CC)= for all cycles C E Z.)
Smoothing. So far, the glued manifold has achieved C* continuity, but C continuity is required to yield a smooth global metric and to eliminate "fold" that hinders knowledge transport along the manifold. To bridge this gap, visit the concept of Ricci curvature, kind of C² continuity the manifold, which governs the rate of changes of the volume element along the geodesic. Never- appensiveePetersen
theless, calculating Ricci curvature is rather expensive Petersen (016)); Ollivier (0007). Instead, we propose an alternative of volume change ratio between two endpoints, which is shown to sufficiently determine whether the geodesic is "convex" or "concave".
Theorem 4.9 (Ricci Curvature Estimation, Appendix B.5). Given graph G (V,E) and an edge (i.j) E ε, let 200, E M be the corresponding embedded points, and 7 [0, 1] + M
be the unit-speed geodesic connecting them, i.e., (0)) = =(),, 7(1) = =(,), The sign of the Ricci
curvature along ty can be estimated by the ratio of metric determinants:
det G.
1
(12(), 200) = det G, 2 1 Ric(t).
(6)
3
5

---

Published as a conference paper at ICLR 2026
Domain
Generate Tangeet Space
A O
Fronen
AOF
EMA Riemannian Protodyprs
Manifold Skeleton
Pretotypes
Domain
Durain
Unified Manifold
proto
glue
Domain
Domain
92
Target Domain
Lad
Transferabilityy Metric
AOF
GTM AH + AC
Figure 2: An Illustration of GRAPHGLUE Framework.
Accordingly, the volume element det G; varies smoothly along the path of length k implying that the Ricci curvature changes continuously along that path, referred Log- neg-Determinant k-order smoothness. Thus, we can investigate the k-order smoothness with scalar field of volume element, and formulate a Ricci curvature loss which encourages the glued manifold to be smooth.
Definition 4.10 (k-order Smoothness and Curvature Loss). Define 9: log det G; scalar field over G. representing the logarithmic volume density at node U,. We say the manifold structure exhibits log-determinant smoothness if | gl|2, where L is the (normalized) Laplacian of In light of computational efficiency in practice, define the curvalure loss function of 2-order smoothness as follows,
BR minimizes the graph Dirichlet energy: Enilgg]
Lcwr (G)
Σ AAI
[og(ris) log(r5))1
Aish
Geometric Scaling Law Consequently, any graph datasets are merged into a unified, smooth Ric- mannian manifold, allowing to study knowledge transfer within the framework of differential geometry. As the quantities of graphs increase, (F, F.G.P P) approximates an ideal manifold, and thus deduce a geometric scaling law that larger quantities of datasets improve model transferability with a smoother manifold, which is empirically validated in Sec. 66.2
Theorem 4.11 (Gluing into a Smooth Manifold, Appendix B.6). For any graph dataset G, if bog-determinant co-order smooth, and P is trivial with induced metric meeric-preserving diffeomorphism 6, then (F, G, P) glues smooth Riemannian manifold (F, G), where
(U)) U.
GRAPHGLUE: GEOMETRIC MULTI-DOMAIN GRAPH PRE-TRAINING
Building on our theory of neural manifold gluing, we present a novel vverraining-adaptation frame- work, GRAPHGL.UE, as illustrated in Fig. The pre-training first learns the local geometry and then glues these local pieces together as introduced in Sec. 4 Moreover, before gluing, an Expo- nential Moving Average (EMA) prototyping is proposed to distinguish domain semantics through different locations on the manifold, while enabling batched pre-training to efficiently handle large- scale graphs. Then, we leverage prompt adaptation and Riemannian Mfxture-of-Experte (MoE), while gluing the target domain to the pre-trained manifold for geometric consistency. A Geometric Transfer Metric (GTM) is naturally induced measure the transfer difficulty. The overall procedure is summarized in Algorithm
5.1 PRE-TRAINING WITH EMA PROTOTYPING
For multi-domain source graphs $ = [S,,..., Sx }. we associate each graph with a Riemannian prototype, which is given tuple of global location and Riemannian metrics, (2St, log Go

---

Published as a conference paper at ICLR 2026
[Sal Eges, =9, S3ul Eges, log log G( The challenges of Riemannian prototyping are dual: computation efficiency for large-scale graphs, and semantics distinction across different domains. To address the first challenge, we develop an EMA for Riemannian prototyping. For each batch, we perform the following updating rules,
(
z Su 82 + (1 (3)
[B&
1
Σ
GEB,
20
(8)
log Sx t 3 log Sh + (1 31 13x E log G(29),
where B E (0, 1) is a momentum coefficient, and log means matrix logarithm. This EMA update ensures that (z log Go2) gradually converge to the stable average value throughout pre-training Morales-Broton et al. (2024): Izmailov et al.] (2019). Note that the metric matrix belongs to a symmetric positive-dehnite manifold, and we utilize the log update, different from the traditional ones. To address the second challenge, we incorporate sample-prototype contrastive loss that encourages graph prototypes to be well separated on the manifold, distinguishing domain semantics.
K
exp(sim(29 2x
Eproto (9) =
log
(10)
K
exp(sim(=9 2 //T
5.2 CONSISTENT ADAPTATION & QUANTIFIABLE TRANSFERABILITY
GRAPIGLUE employs prompt adaptation and Riemannian MoE to generate representations, while emphasize geometric consistency between the pre- pre-trained manifold and target graphs by "glu- ing". To be specific, for a target sample GT, we first infer the global coordinates and local metric through prompting. With the coordinates zT, local metric G² and basis vectors of the tangent space WT (w] ,..., ....,,,]r] given by the pre-trained model, we introduce a learnable prompt matrix Q E Raxd The global coordinates is adapted as zdddpe = Q= Note that the metric adaptation is challenging owing to the orthogonal requirement of basis vectors, Thus, in- stead of prompting the pre-trained local metric, we apply the prompt matrix Q to WT and the adapted local metric is derived as Gudapt diag (I|@w] 11 ..., Qw" where w² are the basis vectors undergoing the proposed adaptive orthogonal frame. Second, to ensure consistency, the pre-trained Riemannian manifold F, where Riemannian proto-
we glue the target sample types are treated as the anchors to align the target. In particular, we construct a transfer graph Go by connecting the target to its k--nearsst prototypes, and apply Lawto and Learv (Go) proposed in Sec. 4 penalizing non-trivial holonomy and abrupt volume changes, respectively. Third, we present a Riemmanian MoE where each Riemannian prototype (2S², log Goe) serves as an ex- pert and its weight is given by gating function 3x gk (zdapi Güdlup)). This MoE generates log Galign = Σ K
> Bx log Go,, summarized from the experts. Accordingly, we obtain the final rep- resentation Busk T=;;; log Gudapt log G"liga], where Zusk E Rd+2M since log Gudapt log Galign are both diagonal matrix that can be vectorized. The overall adaptation loss is given as
Ludup =
Lissk (Zunki Bmnsk) + >Lglue: Eglue (9o + Learv (Go).
(11)
where À balances task spsc-ipe learning with consistency, and yrask is the label of downstream task.
On Transferability Within the framework of differential geometry, we are able to systematically analyze knowledge transfer across different domains, and transfer effort of GRAPHGLUE is natu- rally measured by the geometric compatibility. We introduce Geometric Transfer Metric (GTM) which is defined as the minimal geometric deformation required to merge the target G into the are-trained manifold F without disrupting its learned local geometry. GTM is computed along with the adaptation and decomposes into two interpretable components as follows,
GTM(GT: :S) = AH + AC, AH = (hota(Go) AC = Leurv
(12)
1. Holonomy disagreement AH. It measures how the holonomy map deviates from identity along paths connecting the target to its nearest prototype, interpreted as the "twisting" induced by g?
2. Curvature disagreement AC. It is computed as the discrepancy between the volume element /det G, indicating the dismatch with respect to Ricci curvature according to Theorem 4.9 The natural interpretation is given as the "bending" abrupt change in local volume.
7

---

Published as a conference paper at ICLR 2026
Accordingly, low GTM means that the target is seamlessly integrated F with trivial deforma- tion, implying high transferability; in contrast, a high value shows that the target is geometrically alien, thus requiring significant effort to fit the geometry of F. Different from similarity measures between source and target domains Wang et al. )2024), GTM examines the geometric consistency from GRAPHGLUE itself, and provides an interpretable assessment of transfer difficulty.
Further Insight The generalization error is related to the smoothness of the model objective Bartlett et al. (2017); [Scaman & Virmaux (2018). In fact, GRAPHGL.UE controls the smoothness by inducing smooth global metric. Specifically, Lholo guarantees the topological continuity of glu- ing boundaries, while Leurv achieves k-order smooth by log-determinant smoothness in Definition 4.10 similar to CCarneecki et al. (2017). The complexity analysis is provided in Appendix D.2 D.3
EXPERIMENTS
We conduct experiments six representative domains to evaluate cross-domain transfer learning performance. Also, we examine the transferability measure (GTM), geometric scaling law, the effect of incorporating graphs of distinct semantics, and the geometric interpretation. Ablation study, hyperparameter sensitivity and performance on heterophilic graphs are in Appendix G.2
G.3
6.1 EXPERIMENTAL SETUPS
Datasets Baselines We carefully select 6 representative benchmark datasets, covering various domains: academic citation network Arxiv, product co- co-purchase graph Computers social network Reddit, knowledge graph 7BB15k2233 27, and benchmarks on bioinformatics PROTEINS and chemoinformatics HIV. We compare GRAPHGLUE against baselines from 3 main categories: (1) Supervised GNNs: GCN Kipf & Welling (0017), GraphSAGi Hamilton et al. (2017) ) and GIN Xu et al.] (0019). (2) Self-Supervised GNNs: DGI Veličković et al. (2019), Graph Hou et al.
2002)). and GCC Qiu et al. (2020). (3) Graph Foundation Models: PRODIG Huang et al. (2023).
GFT Wang et al. (2024), RAGraph Jiang et al. (2024). SAMGPT Yu et al. (2025a), GCOPE Zhao
et al. (2024). and MDGFM Wang et al. (2025a). Detailed descriptions are specified in Appendix F
Evaluation Protocol Our evaluation adopts a lcave-one-out cross-domain sctup, where models are pre-trained on five source datasets and fine-tuned on single held-out target dataset. We use a few-shot fine-tuning setting, leveraging & labeled samples per class (k E {1,5}) from the target task for adaptation. The remaining target data is randomly split into 10% for validation and 90% for testing. We evaluate performance on three tasks: node/edge classification measured by ACC and graph classification measured by AUC. All reported results are the average of 10 independent runs.
6.2 RESULTS AND DISCUSSION
Main Results on Cross-domain Transfer Learning As shown in Table 1 the empirical results demonstrate the superior effectiveness of GRAPHGLUE in challenging few-shot scenarios. For in- stance, in the 1-shot setting, it outperforms the strongest basclines on Comput ers and Reddit by significant margins of 4.9% and 2.3%. respectively. This strong performance is often maintained as more data becomes available. In the 5-shot setting on the Reddit t dataset, GRAPHGI.U achieves 85.0% ACC, exceeding the runner-up by 4.6%. These results suggest that the geometric construc- tion of GRAPHGLUE enhances the model performance, and we will demonstrate additional benefits of the constructed smooth manifold in the following parts.
Ablation study on the effectiveness of proposed Leurv and Lhoto are provided in Appendix G showing that both gluing via holonomy and smoothing via Ricci curvature are important to downstream tasks.
On Transferability Measure This part shows how the proposed measure of GMT aligns with the transfer effort of the pre-trained model. To this end, we pre-train the model in Arxiv, v, Reddit, FB1 52.237, PROTEINS, and HIV datasets, then conduct transfer settings on Computers with 2000 epochs. In this case, holonomy loss vanishes rapidly during training, and thus we investigate the curvature loss in Figure where z-axis is the training epoch. We plot the test task loss of cross- entropy for the classification task the y-axis the left. In the top of Figure 3 we find that, as
8

---

Published as a conference paper at ICLR 2026
Table 1: Performance of cross-domain transfer on various downstream tasks, reported mean + std over 10 runs. The highest result is bolded, and the runner-up is underlined.
Node Classification
Link Classification
Graph Classification
Model
Arxiv
Computers
Reddit
FB 5k 237
PROT K INS
1-shot
5-shol
1-shot
5-shol
I-shot
5-shot
I-shot
5-shot
1-shot
5-shot
GCN
12.6
27.6*2.1
33.8
65.744.2
11.1#2.1
28.32L.
2
52. 12.4218
50.1
55.0aa.
GraphSAGE
14.6+3.7
26.1+22
35.4+8.2
66.7+4.4
14.6+2
22.2=111
35.7=2.
58.9+13
58.94277
60.4+1a
GIN
11.2+2.0
26.0+2.4
44.7
69.5+a.
18,5
29.0=16
38.2a2.
63.7+17
54.2+11.5
58.8+5.0
GCC
12.6+2.0
26.8+2.1
31.8a6.1
62.6+a.1
51.7=5.6
65.2=1.
47 47.8=1.
73.6s12
59.2+7.9
61.2+a..
DGI
13.3+8.3
27.1+2.3
35.2+7.5
61.0+3.2
60.0+44.
62.7=22
42.5=2.0
68
53.1
58.3
GraphMAE
12.6+1.7
27.6+2.1
33.8
65.744.2
11.1+2.1
28.3e1.
51.=11
77.2=1.0
60.1#18.0
05.0a9.9
PRODIGY
28.4+2.2
33.G+2.8
45.3+1.1
52.7+3..
35.6+3.2
42.3=2.
53.5=1.0
72.1
48.925.4
55.2+447
GFT
26.5+2.
36.7+1.9
54.6+4..
69.1.
58.8+2.5
06.2=1.
58.0=1.5
79.1=1.8
55.4
02.1
RAGruph
18.7
32.321.7
46.22433
62.34a.7
52.5+34
63.021.
52.1=30
64.522.5
51.45.1
58.6
SAMGPT
24.1::3.8
34.4122
47.627.4
60.8133.
62.8
75.1
57 67.4
77.622.
52.415.1
59.1
GCOPE
26.5#5.5
39.1=1.9
54.5+t9.1
72.2+2.8
02.7+4.5
80.4=0.7
58.2=2.
79.3+2.2
55.1+3.5
64.8+2.4
MDGFM
26.0+:2.4
32.2
46.628.4
64.015.3
64.8+3.3
76.521.7
56.1
77.622.
53.415.3
57.7
GRAPHGLUE
28. 8 37.0+2.3
59. 59.5
73.2
67.1+aa
85, 0
59.7 7+8.2 81.5=22
59.8
65 65.3+124
Task Loss vs Curvature Loss
Curvature Loss Oscillation Decay flog Scale)
Toss
Curvature Love
Roiling Shd
Osoilation =lu
Osgiligton Amolltus
Trand
3.50
0.10
3.25
0.00
1.00
0.06
0.64
0.02
0.50
Epoch
Epoch
Figure 3: GTM Test Task Loss.
curvature loss decreases and converges, the test task loss exhibits the same pattern, and it suggests that GMT measures the effort of training the pre-trained model in transfer setting. Morcover, at the bottom of Figure it shows another feature of curvature loss that the convergence of its oscillation amplitude implies the convergence of the test task loss, which meets the theory in Keskar et al. (007) Czarnecki et al. (2017).
Case Study We conduct an interesting case study to examine the effect of including semantically distinct data during pre-training. To this end, we incremen- tally expand Reddit-only pre- pre-training with the distinct PROTEING and HIV datasets, and consistently evaluate on Reddit under the 1-shot setting. As shown in Fig- ure GRAPHGLUE achieves steady improvement with the inclusion of each dataset. In contrast, GCOPE suffers from negative transfer and results in possible performance decline. This result provides evidence that GRAPHGL.EE effectively incorporate knowledge from even vastly different domains to enhance its capabilities.
GrashGlue GCOPS
64.33
ACC (%)
53,32
58,26
54.35
only
Modits Besok Pre-training Corpus
Figure 4: Effect of including distinct domains during pre-training.
On Geometric Scaling Law We validate the geomet- ric scaling law by enlarging the quantities of pre-training datasets. Specifically, we show the few-shot performance on Computers and Reddit in Fig- ure [5] where the original datasets are same that in Figure Bl denoted as +0, and we incre- mentally incorporate Pubmed, Photo, FacebookPagee2en age. WordNet 18RR MUATG and Lipophilicity in order, referred to as +1, +2, +3, +4, +5 and +6, respectively. In the 1- shot setting, average accuracy rises steadily while transfer loss drops consistently, both well-fitted by logarithmic functions (blue curves), and thus it exhibits clear scaling laws. 5-shot performance remains more stable (red curves), with only marginal gains in accuracy and slight reduction in loss. The insight is that, under extreme data scarcity (1-shot), the performance is highly sensitive

---

Published as a conference paper at ICLR 2026
Auseangh ACC (99
80
90
1.4
1.22
70
Translen
1.5
80
2.0
1.0
0.8 0.0.
60
1.0
70
0.6
50
+1 +2 +3. #4 +5
+1
12
13
14
+5
16
+1 +2 +3. e4
+0
+2
14
Pre-training Dttasets Size
Pre-training Datasets Size
Pre-training Datasets Sizee
Pre-training Datasets Sizee
(a) Computers
(b) Reddit
Figure 5: Geometric scaling law on (a) Computers and (b) Reddit datasets.
the pre-trained model's capacity, the expressive power of the learned manifold, while more labeled samples restrain such scaling effect. The observed logarithmic scaling supports our claim on the scaling law.
Visualization & Geometric Interpreta- tion To illustrate our intuition, we vi- sualize a 3D per-trained manifold on the 6 datasets in Figure 6 where the con- figuration is detailed in Appendix G We observe that the Reaasets-RRedit (so- cial network), Arxiv (citation network), Computers (e-commerce network), and exhibit
eghn-arziv
Computers
Reddit
F915K_207 237
PROTEINS
IVV
FB15k.237 (knowledge graph) grapb)-exhibit substantial semantic overlap while retain- ing the difference. Their corresponding re- gions on the manifold lie in close prox- imity, sometimes intermingling owing to shared semantics, yet remain distinguish- able. The two chemistry-related datasets (PROTEINS and HIV) are well-separated from the others the learned manifold. That is, the proposed neural manifold glu- ing captures the complicated domain mantics. Also, the smoothness is generally ensured, facilitating knowledge transport along the manifold. The visualization un- derscores our framework's 's ability to unify diverse domains into coherent geometric structure, which forms the foundation for effective cross- domain transfer.
Figure 6: Visualization of the pre-trained manifold from 6 datasets.
CONCLUSION
This work present the first multi-domain graph pre-training framework through the lens of Rieman- nian geometry, enabling the merging of arbitrary graph datasets into unified, smooth Riemannian manifold and facilitating a principled understanding of knowledge transfer across different graphs The theoretical contribution lies in the establishment of neural manifold gluing, which "glues" the local pieces together into a coherent whole. Building on this theory, we introduce the GRAPHGLUE framework, supporting the batched pre- pre-training and providing means measure its transferability. Furthermore, we empirically validate the geometric scaling law of GRAPHGLUE.
ACKNOWLEDGEMENT
This work is supported in part by NSFC under grants 62202164. Philip S. Yu is supported in part by NSF under grants 111-2106758, and POSE-2346158.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Gregor Bachmann, Gary Bécigneul, and Octavian-Fugen Ganea. Constant curvature graph convo- lutional networks. In Proceedings of the 37th International Conference Machine Learning (ICML), volume 119, pp. 486-496, 2020.
Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalizee normalized margin bounds for neural networks. In Advances in Neural Information Processing Systems 31 (NeurIPS), vol- ume 30, pp. 6240-6249, 2017.
Beatrice Bevilacqua, Joshua Robinson, Jure Leskovec, and Bruno Ribeiro. Holographic node repre- sentations: Pre-training task -ggnostc node embeddings. In Proceedings of the 13th International Conference on Learning Representations (ICLR), 2025.
Jianyuan Bo, Hao Wu, and Yuan Fang. Quantizing text-attributed graphs for semantic-structural integration. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 (KDD), pp. 107-118, 2025.
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ B. Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeamnette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen Creel, Jared Quiney Davis, Dorottya Demszky, Chris Donahue, Moussa Doumbouya, Esin Dur- mus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tat- sunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho. Jenny Hong, Kyle Hsu. Jing Huang. Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Ro- hith Kuditipudi, and et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2/08.. 07258, 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and et Language models are few-shot learners. Advances in neural information processing systems (NeurlPS), 33: 33:7 1901, 2020.
Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. In Advances in Neural Information Processing Systems 31 (NeurIPS), volume 32, 2019.
Haibo Chen, Xin Wang, Zeyang Zhang, Haoyang Li, Ling Feng, and Wenwu Zhu. AutoGFM: Automated graph foundation model with adaptive architecture customization. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.
Runjin Chen, Tong Zhao, Ajay Kumar Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.081: 02240070.71, 2024.
Fan R. K. Chung. Spectral Graph Theory. Number 92 in Regional Conference Series in Mathemat- ics. American Mathematical Society, reprint edition. ISBN 978-0-8218-0315-8.
Wojeiech M. Czarnecki, Simon Osindero, Max Jaderberg, Grzegorz Swirszez, and Razvan Pascanu. Sobolev training for neural networks. In Advances in Neural Information Processing Systems 31 (NeurIPS), volume 30, 2017.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding- In Proceedings of the 2019 Conference of the North American chapter of the Association for Computational Linguistics (NAACL), pp- 4171-4186, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint 1:Xiv:2010.11922 2020.
H. Edelsbrunner and Harer. Computational Topology: An Introduction. Applied Mathematics. American Mathematical Society. ISBN 978-0-8218-4922555
11

---

Published as a conference paper at ICLR 2026
Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. In Advances in Neural Information Processing Systems 37 (NeurIPS), 2023.
Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2016.
Karish Grover, Geoffrey J. Gordon, and Christos Faloutsos. CurvGAD: Leveraging curvature for enhanced graph anomaly detection. Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.
Albert Gu, Frederic Sala, Beliz Gunel, and Christopher Ré. Learning mixed-curvature representa- tions in product spaces. In Proceedings of the 7th International Conference on Learning Repre- sentations (ICLR), 2019.
Zhichun Guo, Kehan Guo, Bozhao Nan, Yijun Tian, Roshni G Iyer, Yihong Ma, Olaf Wiest, Xian- gliang Zhang, Wei Wang, Chuxu Zhang, et al. Graph-based molecular representation learning. arXiv preprini arXiv:2207.044880
2022.
William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems 29 (NeurIPS), 2017.
Allen Hatcher. Algebraic Topology. Cambridge University Press, 2002. ISBN 978-0-521-79540-1L
Morris W. Hirsch. Differential Topology, volume 33 of Graduate Texts in Mathematics. Springer, 1976. ISBN 978-1-4684-9451-88 978-1-4684-9449-55
doi: 10. 10000799881-4684-445555
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 594-604, 2022.
Qian Huang, Hongyu Ren. Peng Chen, Gregor Kržmanc, Daniel Zeng. Percy Liang, and Jure Leskovec. Prodigy: Enabling in-context learning over graphs. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pp. 16302-16317, 2023.
Xingyue Huang, Pablo Barcelo, Michael M. Bronstein, Ismail IIkan Ceylan, Mikhail Galkin, Juan Reutter, and Miguel Romero Orth. How expressive are knowledge graph foundation models? In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization, 2019.
Xinke Jiang, Rihong Qiu, Yongxin Xu, Yichen Zhu, Ruizhe Zhang, Yuchen Fang, Chu Xu, Junfeng Zhao, and Yasha Wang. Ragraph: A general retrieval-augmentt -ugmented graph learning framework. Advances in Neural Information Processing Systems (NeurIPS),
2024.
Pengfei Jiao, Jialong Ni, Di Jin, Xuan Guo, Huan Liu, Hongjiang Chen, and Yanxian Bi. Hgmp: Heterogeneous graph multi-task prompt learning. In Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI), pp. 2982-2990, 2025.
Ce Ju and Cuntai Guan. Graph neural networks on spd manifolds for motor imagery classification: A perspective from the time- frequency analysis. IEEE Transactions on Neural Networks and Learning Systems, 35, 2024.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Pe- Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
Diederík P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
12

---

Published as a conference paper at ICLR 2026
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In Proceedings of the 12th International Conference on Learning Representations. OpenReview.net, 2024.
Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S Yu, et al. Graph foundation models: Concepts. opportunities and challenges. JEEE Transactions on Pattern Analysis and Machine Intelligence 2025.
Yi Liu, Limei Wang. Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In Proceedings of the 10th International Conference Learming Representations (ICIR). 2022.
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In Proceedings of the 32nd ACM Web Conference (WWW), 2023.
Iya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In Pro- ceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan. Gfm--aag Graph foundation model for retrieval augmented generation, 2025.
Daniel Morales-Broute Brotons, Thijs Vogels, and Hadrien Hendrikx. Exponential moving average of weights in deep learning: Dynamics and benefits. Transactions on Machine Learning Research, 2024.
Yann Ollivier. Ricci curvature of markov chains on metric spaces, 2007.
Peter Petersen.
Riemannian Geometry, volume 171 of Graduate Texts Mathematics. Springer International Publishing, 2016. ISBN 978-3-319-26652-7 978-3-319-26654-11 doi: 10.1007/
978-3-319-26654-1
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. Gee: Graph contrastive coding for graph neural network pre-training. arXiv preprint arXiv:2006.00 09963, 2020.
Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh Chawla, and Chao Huang. survey of large language models for graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discov- ery and Data Mining (KDD). pp. 6616-6626, 2024.
Luana Ruiz, Luiz Chamon, and Alejandro Ribeiro. Graphon neural networks and the transferability of graph neural networks. In Advances in Neural Information Processing Systems, volume 33, pp. 1702-1712. Curran Associates, Inc., 2020.
Kevin Scaman and Aladin Virmaux. Lipschitz regularity of deep neural networks: analysis and effi- cient estimation. In Advances in Neural Information Processing Systems 32 (NeurIPS), NIPS 18, pp. 3839-3848, 2018
Kartik Sharma, Yeon-Chang Lee, Sivagami Nambi, Aditya Salian, Shlok Shah, Sang- Wook Kim, and Srijan Kumar. A survey of graph neural networks for social recommender systems. ACM Computing Surveys, 56(10):1-34, 2024.
Li Sun, Zhongbao Zhang, Junda Ye, Hao Peng, Jiawei Zhang, Sen Su, and Philip Yu. self- supervised mixed-curvature graph neural network. In Proceedings of the 36th AAAL, pp. 4146- 4155, 2022a.
Li Sun, Zhenhao Huang, Hua Wu, Junda Ye. Hao Peng, Zhengtao Yu, and Philip S. Yu. Deepricci: Self-supervised graph structure- ceneturr--eeture co-refinement for alleviating over wverssiussiin. In Proceed- ings of the 23rd ICDM, pp. 558-5556 2023a.
Li Sun, Feiyang Wang, Junda Ye, Hao Peng, and Philip S. Yu. Congregate: contrastive graph clustering in curvature spaces. In Proceedings of the 32nd IJCAI, pp. 2305, 2023b.
13

---

Published as a conference paper at ICLR 2026
Li Sun, Zhenhao Huang, Hao Peng, Yujie Wang. Chunyang Liu, and Philip S. Yu. Lsenet: Lorentz structural entropy neural network for deep graph clustering, In Proceedings of the 41st ICML, pp. 2024a.
Li Sun, Zhenhao Huang. Qiqi Wan, Hao Peng, and Philip S. Yu. Spiking graph neural network on riemannian manifolds. In Advances in NearIPS, 2024b.
Li Sun, Zhenhao Huang, Zixi Wang, Feiyang Wang, Hao Peng, and Philip S. Yu. Motif-aware riemannian graph neural network with generative-cootrastive learning. In Proceedings of the 38th AAAI, pp. 9044-9052, 2024c.
Li Sun, Zhenhao Huang, Ming Zhang, and Philip S. Yu. Deeper with riemannian geometry: Over- coming oversmoothing and oversquashing for graph foundation models. In Advances NeurIPS, 2025a.
Li Sun, Zhenhao Huang, Suyang Zhou, Qiqi Wan, Hao Peng, and Philip S. Yu. Riemanngfm: Learning a graph foundation model from riemannian geometry. Proceedings of the ACM on Web Conference 2025 (WWW), pp. 1154-1165, 2025b.
Li Sun, Zhenhao Huang, Yujie Wang, Hongbo Lv, Chuanyang Liu, Hao Peng, and Philip S. Yu. Asil: Augmented structural information learning for deep graph clustering in hyperbolic space. IEEE Transactions Pattern Analysis and Machine Intelligence, pp. 1-18. 2026a.
Li Sun, Ming Zhang, Wenxin Jin, Zhongtian Sun, Zhenhao Huang, Hao Peng, Sen Su, and Philip S. hypergraph neural networks with riemannian local exchanger. In Pro-
Yu. Heterophily- ceedings of the ACM Web Conference (WWW), 2026b.
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), KDD '22, pp. 1717-1727, 2022b.
Yifei Sun, Qi Zhu, Yang Yang, Chunping Wang, Tianyu Fan, Jiajun Zhu, and Lei Chen. Fine-tuning graph neural networks by preserving graph generative patterns. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI), volume 38, pp. 9053-9061, 2024d.
Maciej Sypetkowski, Frederik Wenkel, Farimah Poursafaei, Nia Dickson, Karush Suri, Philip Frad- kin, and Dominique Beaini. On the scalability of GNNs for molecular graphs. In Advances in Neural Information Processing Systems 38 (NeurIPS), 2024.
Jiabin Tang. Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 491-500, 2024.
Eliot Tron, Rita Fioresi, Nicolas Couellan, and Stéphane Puechmorel. Cartan moving frames and the data manifolds. CoRR, abs/2409.1 2057, 2024.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, (86):25579228) 220799926,5222 2008
Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In Proceedings of the 7th International Conference on Learning Representations (ICLR), 2019.
Shuo Wang, Bokui Wang, Zhixiang Shen, Boyan Deng, and Zhao Kang. Multi-domain graph foun- dation models: Robust knowledge transfer via topology alignment. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025a.
Yuyang Wang, Zijie Li, and Amir Barati Farimani. Graph neural networks for molecules. In Machine learning in molecular sciences, pp. 21-66. Springer, 2023.
Zehong Wang. Zheyuan Zhang, Nitesh Chawla, Chuxu Zhang, and Yanfang Ye. Gft: Graph founda- tion model with transferable tree vocabulary. Advances in Neural Information Processing Systems (NeurIPS), 107443, 2024.
14

---

Published as a conference paper at ICLR 2026
Zehong Wang, Zheyuan Liu, Tianyi Ma, Jiazheng Li, Zheyuan Zhang, Xingbo Fu, Yiyang Li, Zhengqing Yuan, Wei Song. Yijun Ma, et al. Graph foundation models: A comprehensive survey. arXiv preprint wrXh::2505.5511 2025b.
Zehong Wang, Zheyuan Zhang. Tianyi Ma, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. To- wards graph foundation models: Learning generalities across graphs via task-trees. In Proceed- ings of the 42nd International Conference on Machine Learning (ICML), 2025c.
Grady Wright and Bengt Fornberg. Scattered node compact finite difference-type formulas gener- ated from radial basis functions. In Computational Methods, pp. 1391-1395, Dordrecht, 2006.
Bin Wu, Yihang Wang, Yuanhao Zeng, Jiawei Liu, Jiashu Zhao, Cheng Yang, Yawen Li, Long Xia, Dawei Yin, and Chuan Shi. Graph foundation models for recommendation: A comprehensive survey. 2025.
Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z. Li. Mole-BERT: Rethinking pre-training graph neural networks for molecules. In Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023.
Lianghao Xia, Ben Kao, and Chao Huang. Opengraph: Towards open graph foundation models. In Findings of Empirical Methods in Natural Language Processing (EMNLP), pp. 2365-2266222222 2379, 2024.
Bo Xiong. Shichao Zhu, Nico Potyka, Shirui Pan, Chuan Zhou, and Steffen Staab. Pseudo- riemannian graph convolutional networks. In Advances in neural information processing systems (NeurIPS), 2022.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In Proceedings of the 7th International Conference on Learning Representations (ICLR), 2019.
Menglin Yang, Min Zhou, Tong Zhang, Jiahong Liu, Zhihao Li, Lujia Pan, Hui Xiong, and Irwin King. Hyperbolic graph neural networks: A review of methods and applications, 2025a.
Zhe-Rui Yang, Jindong Han, Chang-Dong Wang, and Hao Liu. Graphlora: Structure- -ware con- trastive low-rank adaptation for cross-graph transfer learning. In Proceedings of the 3 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. I, pp. 1785-1796, 2025b.
Xingtong Yu, Yuan Fang, Zemin Liu, and Xinming Zhang. Hgprompt: bridging homogeneous and heterogeneous graphs for few-shot prompt learning. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI), 2024a.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Multigprompt for multi-task pre- training and prompting graphs. In Proceedings of the 33rd ACM Web Conference (WWW), pp- 515-526, 2024b. ISBN 9798400701719.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Text-free multi-domain graph pre- training: Toward graph foundation models, 2024c.
Xingtong Yu, Zechuan Gong, Chang Zhou, Yuan Fang, and Hui Zhang. Samgpt: Text-free graph foundation model for multi-domain pre-training and cross-domain adaptation. In Proceedings of the ACM on Web Conference 2025 (WWW), pp. 1142-1153, 2025a.
Xingtong Yu, Jie Zhang, Yuan Fang, and Renhe Jiang. non-homophilic graph pre-training and prompt learning. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.I (KDD), pp. 1854-1854, 2025b.
Haonan Yuan, Qingyun Sun, Junhua Shi, Xingeheng Fu, Bryan Hooi, Jianxin Li, and Philip S. Yu. How much can transfer? BRIDGE: Bounded multi-domain graph foundation model with generalization guarantees. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.
Bohang Zhang, Jingehu Gai, Yiheng Du, Qiwei Ye, Di He, and Liwei Wang. Beyond weisfeiler- lehman: A quantitative framework for GNN expressiveness. In Proceedings of the 12th Interna- fional Conference on Learning Representations, 2024.
15

---

Published as a conference paper at ICLR 2026
Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li. All in one and one for all: A simple yet effective method towards cross- dooss- graph pretraining. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 14443-4454. 2024.
Zihao Zhao, Xinlong Zhai, Jinyu Yang, and Chuan Shi. Towards text-free graph foundation models: Rethinking multi-domain graph contrastive learning. In arXiv, 2025.
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changeheng Li, and Maosong Sun. Graph neural networks: A review of methods and applica- tions. AI Open, 1:57-81, 2020.
Xi Zhu, Haochen Xue, Ziwei Zhao, Wujiang Xu, Jingyuan Huang, Minghao Guo, Qifan Wang, Kaixiong Zhou, and Yongfeng Zhang. Llm as gnn: Graph vocabulary learning for text-attributed graph foundation models. arXiv preprint arXiv:2503.0333 2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX: TABLE OF CONTENT
A
Notations
19
Proofs
20
B.1
Proof of Theorem 4.3
20
B.2
Proof of Theorem 4.5
B.3
Proof of Theorem 4.6
22
................................
B,4
Proof of Theorem 4.8 and Clarification
24
B.5
Proof of Theorem 4.9
24
B.6
Proof of Theorem 4.11
25
C
Background: Differential Geometry on Riemannian Manifolds
26
C.1
Riemannian Manifold: The Continuous Setting
26
C.2
Levi-Civita Connection and Parallel Transport
26
C.3
Curvature and Holonomy
26
C.4
Ricci Curvature and Volume Change
26
C.5
Smoothness and Harmonic Functions
27
C.6
Cartan's Method of Moving Frame
27
C.7
Connection to Our Framework
27
D
Algorithms
29
D.1
Multi-Domain Pre-training
D.2
Complexity Analysis
29
D.3
Complexity Comparison with Other GFMs
E
Related Work
30
E.1
Graph Foundation Models
E.2
Multi-domain Graph Pre-training
30
E.3
Graph Fine-tuning and Prompt Learning
E.4
Riemannian Graph Representation Learning
31
Empirical Details
31
F11
Dataset Description
F2
Baselines
32
F33
Implementation notes
33
G
Additional Results....
34
G.1
Supplementary Results
34
G.2
Comprehensive Ablation Study
G.3
Hyperparameter Sensitivity Analysis
35
G.4
Results on Heterophilic Graphs
36
G.5
Visualization of Manifold Gluing
36
H
Reprodueibility Statement
40
Ethics Statement
41
17

---

Published as a conference paper at ICLR 2026
J Declaration of LLM Usage
41
K Broader Impact
18

---

Published as a conference paper at ICLR 2026
A NOTATIONS
Table 2: Notation and Description
Notation
Description
M
A smooth manifold
G p
Riemannian metric tensor
A point in M
T..M
The tangent space point on M
G())
The volume element
(U, xt .... x
A coordinate chart of tangent space M
(0.)
The standard frame of TMM
Ric (ic,X.Y)
Ricci curvature for vector fields X A graph with node set V and edge set E feature matrix with node set V
9 = (V.E)
X X RIXF
g
The graph of target domain D
G [g), G
G
A collection of K graphs from L domains ID
D (D), D2
D
L domains
fe (GNN(-))
pretrained model on the graph dataset with an encoder GNN(-)
1 GON
The pre-training parameters
faNN: G + Rdd
A differentiable encoder from manifold G into the Euclidean space The set of tangent vectors of graph
Wm DoS H(C) EDi E,(V..
The directional derivative G
The holonomy map cycle C The graph Dirichlet energy of function
The h-hopp neighborhood centered at v; with node set V, and edge set &, within this subgraph
G. (Dm, Em) PIM, d) Ug
The augmented graph
The Adaptive Orthogonal Frame
neighborhood around (i)
(
The basis of Tg generated from AFB
Gi(u.
The local Riemannian metric defined on U;
SMXM
The set of positive-defintt matrix The source graph datasets the Riemannian prototypes for each source graph dataset The prototype-level contrastive loss
(SI, Sk)
S2 log (.S))
p(s)
The tangent edge translation
A
The set of all triangles Aijk ((v,, 0y). (2), 0x)) The holonomy loss
(hal(9)
((2(1), 2()
The overall sign of the Ricci curvature along the geodesic 7t) between z; and zj The curvature loss regularizing the change of curvature by controlling the volume change ratio
Lear(())
GT (VT,ET
An unseen graph
Wrldupi Giddup
The adaptive tangent vectors and adaptive metric The prompt matrix
log Gulign
The aligned log-metric to give K -dddiennsional weighted vector The adaptation loss
Cadop
X
The balance coefficient of task loss and gluing loss
AH
Holonomy disagreement
AC
Curvature disagreement
19

---

Published as a conference paper at ICLR 2026
B PROOFS
B.1 PROOF OF THEOREM 4.3
Theorem 4.3 (Upper bound of Tangent Vector Length) Given a connected with N nodes, A, L the adjacency matrix and Laplacian of G, and P the feature matrix of perturbation nodes. Apply (k, J-ppars perturbation 9, suppose &M N E, where small, and added edge weights
satisfy 21 (()s, 1. Then,
w (1++
holds, where wh is the component of wm determined by perturbation.
Proof. We denote the weighted matrix N&M that consists of h(e,,p) the row summation
TH = H1M- Then, the perturbed adjacency matrix Áis is
E
IM
The perturbed Laplacian is
diag(TH)
-H
(RI++MX(N++M)
(13)
H
IM
Let the d-dimensional graph signal F e: g M)xd in heat diffusion a perturbed graph & be
F(L) = exp( (()) 0,
(14)
F())
X
E
(R(
(15)
P
By the linearity of the heat equation, we can divide F(t) into two parts:
F(t) = exp( tL)
X
+ exp( exxp(ttt) +b)
0
(16)
P
We denote
X
Fosse = exp( -tL)
(17)
0
F Feen(t) = exp(
(18)
P
We are only concerned about ppn (t) since it reflects how perturbation P affects other nodes. Observe from the construction of L, the affected nodes non-zero elements in rn- Let
S supp FH C N}, that 5 = S < kM. We can extract the corresponding part of
L:
Liocal
Ls
Hss
E E((++M)((++0)),
(19)
H
IM
where
Lg (L + diag(rr)))
Hg = H [s.].
Then we have
S eexp(-LLond
0s
(t)
F Np (t) F Fpert, M
P
[1:8]
(20)
exp((tLLnoa)
0.5
P
[S+1:$+M
20

---

Published as a conference paper at ICLR 2026
where S RRXS is index projection matrix such that (5)(i.)} 1 if Bly]. To simplify the notation, let
(()) = exp(-I
=
Kss(t) (s KMs(t) (tM(())
ER(S+3))((S5a))
(21)
We can find
Ne (1) SKsu
(22)
F pent (t) K (MM
(23)
As return to Eq. T6). we obtain
FN (f) = (buse.((t) + SKSM (t)P. Fas (t) (tave.M()) + KMM (t)P: where Fbsse.( (t) (base F (t)) Fbase (N We simply consider the global mean pooling operation that we obtain
(24)
(25)
=(1) N 1NF((() E RM
(26)
wm (t) = (t) xt)) E R
(27)
where (t) = Fas (m) the m-th row of FM. Similarly, we can still divide wm. (t) into two parts as
wm (1) = (t) - (t) + (term affects by X);
(28)
where fP. is the m-th row of KMSss (t) and
1 1 1 WKS SM (t) (t)P= zt(t)
N (1s (SM(())PP
(29)
N
Let att) & KKaas(t)ss M then we have 2P(t) a (t)P. We denote wP, (t) fPh (t)
=t)),, then
wh (t) (KMM(t)P) [m] NO (t) (t) a at))) P. (30)
Let b(t) = MMM [m..l a()), we have wP (1) = b (()P.
Since
RM N
S < kM eN, obtain
S
$ S $ s,
||a(t)]| = N [1 K (t))l
max K (KsM(t)) 43
(31)
N
which means
/ (1) S Pa (1+e)||
(32)
since each element finite, and the diagonal elements of the heat kernel matrix 1 while the other elements are less than 1. Then, we complete the proof.
B.2 PROOF OF THEOREM 4.5
Theorem 4.5 (Edge Tangent Translation as Isometry) The tangent edge translation in Definition 4.4 the optimal solution of
min P
(33)
where GL denotes the general linear group, such that (P(ii)) p(i)) = G. v), which induces isometry (1555) between O and aU,-
21

---

Published as a conference paper at ICLR 2026
Proof. We prove that the tangent edge translation p(i)) =G
(c.
1/2
G/1/2
uniquely minimizes || PP Gills over P E GL(M) and induces an isometry.
Let Q /.2'P. P. Then PT PT P Q" Q, and the objective becomes:
min QEGL(M)
||Q Q G.
(34)
The minimum is achieved when Q Q G since the Frobenius nomn is strictly convex over SPD matrices. Thus, Q = G//² R for orthogonal R. and minimal norm occurs at I, giving Q' =²
From Q = G P. we get candidate Po = G G!? However, this is not symmetric in G., G3 unless they commute. To ensure geometric consistency and symmetry, instead use the metric geometric mean:
=GG
plis) = ( (c/""///))))) G7,1
(35)
Then, compute
p(ij) plid) G,p(i)) = 1/2 2 (c (c."", 1/2 Gj G, Gj Gj (.) 1/2
G
(36)
- 1/2 (c, Thus, G,(Plil u, p()))) u p(.i) G2 C u G Gi(u, v), ((.i) is isom- etry.
( G. 07113 G
(37)
All isometric maps satisfy PT G.P G., and form the set (pl.) RGG,R = G.}. The Frobenius norm =T P) is minimized when Gj P symmetric. Then, we have
G, G,p(i,) = c./² (c. (c./G..))//)) 1/2
(38)
Hence, p(ij) is the minimum norm isometry, and thus the global minimizer of the original Frobe- nius problem (since the constraint is active and satisfied exactly).
Since p((.)) Tem U. + +T 2o U3 is linear isometry, and assuming smooth compatibility of charts near z( 2(,), can lift p((.) via the exponential map (or local parametrization) to local dif- feomorphism ((id)) oU;- - aU, such that of (f.5) p(i.), which the differential of a diffeo- (.
morphism and preserves metric. Hence ((i)) a isometry.
B.3 PROOF OF THEOREM 4.6
Theorem 4.6 Existence of Global Metric) Let (I has)es) be local metrics and tan- gent edge translations. There exists unique global continuous metric G on (U+) Us such that Glu, f. all i.
Proof. We aim to construct global continuous Riemannian metric G on the space F= U U. where each U, is an open subset of IRd, and the overlaps Ug n U, are non-empty for (i,j) E E. By assumption, we have local Riemannian metric G, each U, and tangent edge translations p(i.)) T20 U; + Tuss U, satisfying
P G, G(td) = G
(39)
which ensures that p(i.) is an isometry between (Tz Us, G.) and (Ta G2).
Let us define a topological space FF = U. Un with topology induced by the Euclidean topology on each U,. For each pair (i,j) let (ه(tي)) OU; + aU, be diffeomorphism whose differential
22

---

Published as a conference paper at ICLR 2026
at the shared boundary point 20)) is precisely p(ij), Since p(tJ) is an isometry, it preserves inner products, so (1i-5) is local isometry near (11)
Now, we consider that, let M₁ U,, M2 Uj, Ni aU .N2 jU,, and : Wi -> W2 be the diffeomorphism induced by p(i.), Now we introduce the following lemma to complete the proof.
Lemma B.1 (Gluing Manifolds via Boundary Isometries Hirsch (1976), Let M1 and M2 be smooth M-dimensional manifolds with boundary, and let N1 OM,, N2 C 0M2 be closed, smoothly embedded (M - 1) 1)-dimensional submanifold of their respective boundaries. Suppose diffeomorphism such that its differential @a. TaN1 To(n)N2 extends to an
N22 + isometry
Pa M, + M2
(40)
between the Riemannian metrics G1 on Mi and G2 on M2, ie.
P G2(o()))P& Gi(x), Yo E N1.
(41)
Then, the topological spuce Mi Up M2 obtained by identifying Ni with No via & admits a unique smooth structure such that:
1. The inclusions Mi Mi Us M2 and M₂ Mi Us M2 are smooth embeddings;
2. The Riemannian metrics G1 and G2 extend to a continuous Riemannian metric on Mi Us M2.
Moreover, this smooth structure is unique up to a diffeomorphism that fixes NG N2 point-wise.
By the Lemma B. there exists a smooth structure on the glued space M₁ Ug M2 that arises from identifying Ni with N2 via ø. Moreover, this smooth structure is unique up to diffeomorphism fixing M1 = N₁.
Applying this construction iteratively over all edges (i,j) E E. we can glue all charts U/ together along their boundaries using the maps d(bJ), resulting in globally defined topological space F equipped with a smooth structure.
On each U, already have Riemannian metric G, We now define global metric G on M by setting Glu,, = G, To ensure that G is well-defined on overlaps U; NU,, we must verify that the values agree under coordinate changes.
Let u E T2FF) for z E U. NU. In the chart U,, u is represented as u, E Tuu and in Uj. as Uj = p(,1), E ETan U,. Then:
G, u;) = u, G,u. Gy(uj,u, = G,u, (pl.)), TG (P(.l))..)
(42)
But by the isometry condition:
p(t.) pl..TTG, G,P(,,) - u, G,u, u, p(iJ)) G Glp.0 u, G,
(43)
Thus, (u,, u:) Gy u;), so the metric value is independent of the chart. Hence, G' is well-defined on M.
Since each G; is continuous on Ug, and the transition maps p(i)) are smooth, the metric G' is continuous across overlaps.
Uniqueness follows from the fact that any other metric & agreeing with G, on each U, must coin- cide with G on overlaps due to the isometry constraint. Thus, G is the unique continuous metric extending G; consistently.
Therefore, under the given assumptions, there exists unique continuous Riemannian metric G' on U Uj such that Glu; G; for all completing the proof.
23

---

Published as a conference paper at ICLR 2026
B.4 PROOF OF THEOREM 4.8 AND CLARIFICATION
Theorem 4.8 (Triangle Triviality) !f every edge belongs to at least one triangle, and (T) = for all triangular cycles T in G, then (C) I for all cycles C E (g).
Proof: Under the assumption that every edge lies in at least one triangle, the cycle space 2 is generated by triangular cycles (see, e-g. the simplicial/cellula homology discussion in (Hatcher) 2002] Section 2.1), where I-cycles are generated by boundaries of 2-simplices
here, triangles). Since the holonomy map H 2, (9) + (L(M) is multiplicative and trivial on generators (i.e.. (T) I for all triangles T), it follows that H (C) I for all C E (g))
Note that every edge belonging to at least one triangle is not the assumption of Theorem 4.8. This theorem states that, if every edge belongs at least one triangle, triangles are already sufficient to construct the coherent manifold described in this work. It means that there is no need for exploring any higher-order motifs, but the triangle coverage is not necessary condition of manifold gluing.
We clarify that GraphGlue does not need to add synthetic motifs. Since GraphGlue aims to approx- imate a smooth manifold, the closed triple paths (strict triangles) benefit the approximation process. As we consider that sample closed triangle paths may be impossible in large-scale graphs or trke-like graphs, the triangle holonomy regularization gives computationally efficient way to approximate a "perfect gluing." In practice, we only sample two adjacent edges to approximate strict triangles (at the end of Appendix D.1), and the computation of Leurv only relies on two adjacent edges.
B.5 PROOF OF THEOREM 4
(Ricci Curvature Estimation) Given graph 9 EV,E) and an edge (1,J) E, let 20, 2(16 .M be the corresponding embedded points, and let y [0, 1] M be the unit-speed geodesic connecting them, i.e., 70) 2(0), 711 ( The sign of the Ricci curvature along can be estimated by the ratio of metric determinants:
Theorem 4.9
2(i),
det G;
1
(z = det G3 = 1 Ric(t).
(44)
3
Proof. We work in Gaussian normal coordinates centered at 21)) (0). aligned with the geodesic
(t). In these coordinates, the element of metric tensor giş (t) = 9tg (t)) admits the following
Taylor expansion near = 0 (see, Petersen (0166))
g (t) d.j
Raga(= 2 t + O(t))
(45)
where Rikjs denotes the components of the Riemann curvature tensor at 200,, and = -() is the initial tangent vector.
Let g(t) det( Since g(0) = det(d,y) 1, we compute the expansion of g(t) using the Jacobi formula for the derivative of a determinant:
d
d
log.((!) (1) dl 9))(()
(46)
di
At 0. (0) 8y and (0) 0 (since first-order terms vanish in normal coordinates). Diffcrentiating again:
d de log g())
80 d2
die 9 (t)
(-2 2 Raantt - 2 ==
= 8'd 8's
Ruring --anntt - (47)
2
Thus, expanding log (1) to second order:
logg(1) - ( Rie())
(48)
and exponentiating:
g()) exp Ric()))))++ =1 Ric(t) 12 0(1))
(49)
24

---

Published as a conference paper at ICLR 2026
Now, evaluate at t (i.e., 200) 7()), assuming the higher-order terms remain negligible (which holds if either the curvature bounded and the edge length is small, or if we consider the leading-order behavior):
det ===9(1) ~ Ric(f), det G; = z(0) = 1.
(50)
Therefore, the ratio satisfies:
det G3
1
1 # =1+ 3 Rio(+)) O
r( ())
~
(51)
det G3
3
Ric(t)
where the last step uses (1 - ~ + for small x. However, since we are only interested in the sign of Ric(f), and under the assumption that [ < 1, we may directly approximate:
det G; det G;
= Ric(t),
(52)
by matching leading- aaddngg-rder order terms in the reciprocal expansion (equivalently, approximating det G, P 1- 3 Ric implies det G./ det G5 = 1+ Ric, but since det G. = 1, the direct expansion of det G3 gives the sign relation).
Thus, we conclude:
If Ric(f)) > 0, then det G3 < del G; + r(z(1), 26)) < 1.
If Ric(f) < 0, then det G2 > det G: + r(x(0, 200) > 1.
If Ric(f) 0, then del G ~ del G; r(z(0) 26)) ~ 1.
This establishes the correspondence between the sign of Ricci curvature and the metric volume ratio, as claimed.
B.6 PROOF OF THEOREM
Theorem 4.11 (Glue to a Global Riemannian Manifold) For the set of all graph data G, if is log-determinant co-order smooth, and is trivial with induced metric-preserving diffeomorphism 6, then G. P) glues smooth Riemannian manifold (F, G), where == (U+) 1=1 U,
Proof. We construct the global manifold structure in three steps, leveraging the established compo- nents:
(1) Trivial holonomy + path-independent parallel transport. By Theorem 4.8 and Definition
4.7 the triviality of on cycles implies that the tangent edge translations pO.) detine flat
connection on the graph. Consequently, the induced diffeomorphisms ((ا-) (from Theorem 4.5) are compatible across higher-order overlaps: for any two paths from U; to Uz. the composed gluing maps agree. This ensures the cocycle condition for manifold gluing.
(2) Global metric existence. By Theorem 4.6 (Existence of Global Metric), the pairwise isometric identifications ((i.)) - now globally consistent due to trivial holonomy - allow us to glue the Ug Us equipped with a unique continuous Riemannian
charts {U,} into topological space
metric G such that U. = G
(3) Smoothness from log-det co-order smoothness. By Definition 4.10 the scalar field g, log det G, minimizes [ gll for all k 2 1, which implies g is in the kernel of all powers of L iee, is infinitely smooth over the graph. Since L* 0 for all only if y is constant on connected components (under mild graph connectivity), and since det. G; metric determinants vary smoothly (in fact, constantly, if the graph is connected). Combined with the smoothness of the transition maps ((6-)) (which are isometries, hence (o,), this ensures that the metric tensor G is smooth in overlapping charts. Thus, (F, G) is smooth Riemannian manifold.
exp(2g:). it follows that the
Therefore, the triple (F, G.P). under the given conditions, glues consistently to form the smooth Riemannian manifold (F, G).
25

---

Published as a conference paper at ICLR 2026
C BACKGROUND: DIFFERENTIAL GEOMETRY ON RIEMANNIAN MANIFOLDS
This appendix provides the necessary background on continuous Riemannian geometry, which forms the theoretical foundation for claim that MERGE learns smooth, intrinsic manifold in the latent space. While our implementation operates on discrete graphs and neural networks, we argue that the learned structure approximates true, continuous Riemannian manifold due to the smoothness of the GNN encoder. We emphasize concepts relevant to Sections 5 and 6 of the main text.
C.I RIEMANNIAN MANIFOLD: THE CONTINUOUS SETTING
A Riemannian manifold (M, g) is smooth (typically (*) topological manifold M of dimension M, endowed with Riemannian metric tensor g. At each point p M, the metric 8p symmetric, positive-definite bilinear form defined on the tangent space Tp.M:
Ep T.M x TpM R.
(53)
The metric Ep allows to compute lengths of tangent vectors, angles between them, and volumes of regions on M.. In local coordinates (x² .... ad around p, the metric is represented by matrix (p) [at where 915(2) (p) = (a, 02), and (0: - is the coordinate basis of T...
The volume element at p is given by dV, / det (p) dx A... dx The scalar field f(p) log det (p) is called the logarithmic volume density. manifold said to be -mmoohh the components 9ij are -iififccrentiabl functions of the coordinates.
C.2 LEVI-CIVITA CONNECTION AND PARALLEL TRANSPORT
A connection V M defines how differentiate vector fields along curves, enabling the notion of parallel transport. The unique connection compatible with the metric and torsion-free is called the Levi-Civita connection. It is characterized by two properties:
1. Metric Compatibility: For any vector fields X, Y, on M,
x(,, Z) = VV 2) (V, Vx
(54)
This means parallel transport preserves inner products (and thus lengths and angles). V X [X,Y]. where [5,.] is the Lie bracket.
2. Torsion-Free: X
Given a smooth curve (t) [a, b] M, vector field (tt) along 7 is parallel transported if V v( (t) = O. The parallel transport map PT, M - M M is the linear isometry that maps a vector at the start of the curve its
version at the end.
C.3 CURVATURE AND HOLONOMY
The failure of parallel transport be path-independent is measured by the curvature tensor R, (1, 3) -eenor defined as:
R R(,Y))Z V Vixyy
(55)
R 0 everywhere, the manifold flat, and parallel transport depends only on the endpoints, not the path.
For closed loop (cycle) starting and ending at p. the composition of parallel transports along C yields linear transformation H(C): T M, called the holonomy of C. If H(C) id for all loops C, then the curvature vanishes (R = 0), and the manifold is flat. Conversely, if R # 0, then HCC) o id for some non-contractible loop.
C.4 RICCI CURVATURE AND VOLUME CHANGE
The Ricci curvature Ric is (0, -e-soo obtained by contracting the curvature tensor: Rie(X, Y) EM R(e, X, Y,e,), where {e:) is orthonormal basis.
26

---

Published as a conference paper at ICLR 2026
On geodesic (t) with unit speed (()), the Ricci curvature governs the rate of change of the volume element along the geodesic. Gaussian normal coordinates centered on y(0), the determinant of the metric satisfies the following expansion for small t:
det G = - (i)( O(t)).
(56)
Thus, the ratio of volume elements between two nearby points = (0) and g = 7(t) approxi- mately:
v det G( 21 (i)()
(57)
det G(p)
6
This implies:
1. Ric > 0: Volume shrinks along the geodesic (elliptic/positive curvature region).
2. Ric < 0: Volume expands along the geodesic (hyperbolic/negatiie curvature region).
3. Ric 0: Volume is locally preserved (flat region). This relationship underpins our use of the metric volume ratio del G./ det G, as a proxy for estimating Ricci curvature along graph edges.
C.5 SMOOTINESS AND HARMONIC FUNCTIONS
scalar function / .M + R is harmonic satisfies Aʃ = O, where A the Laplace- Beltrami operator. On compact manifold without boundary, harmonic functions constant. More importantly, solutions to A = O are infinitely differentiable (C**) by elliptic regularity theory.
In the context of framework, minimizing the Dirichlet energy E(syee(Ji fj)² over the graph Gdat is a discrete approximation to minimizing JM IIV Minimizing this energy drives f= log det G toward harmonic function on the underlying continuous manifold. By elliptic regularity, this ensures that the log-volum density f is smooth, implying the metric G has contin- uous first derivatives (C*). This justifies our assumption that the learned manifold is geometrically well-behaved, free from pathological singularities.
C.6 CARTAN'S METHIO OF MOVING FRAME
The renowned Cartan's Method Tron al. 2024) offers principled way explore the geometry of Riemannian manifolds, establishing a profound connection between differential calculus and ge- ometry. Specifically, Élli Cartan introduces the concept of frame to characterize the local geometry, which is then extended to a global manifold through moving frame". Although Élie Cartan laid the mathematical principle, its deep learning theory and methodology remain largely unexplored. Our work seeks to bridge this gap.
C.7 CONNECTION TO OUR FRAMEWORK
Our work does assume a pre-existing manifold. Instead, we posit that the embedding space d induced by smooth GNN faNN contains low-dimensional submanifold M. whose intrinsic geometry encodes the generalizable rules of graph data. The Adaptive Frame Bank (AFB) samples the local tangent spaces M. The optimal isometric alignment (Theorem 5.6) approximates the Levi-Civita connection's action between sampled points. The cycle-consistency loss enforces trivial holonomy, mimicking flatness. The log-determinant smoothness regularization drives the volume element toward harmonicity. Together, these components constitute a learning procedure that con- structs continuous, smooth, nearly-flat Riemannian manifold M within the latent space of neural network, using only discrete graph samples and their embeddings. The graph structure Gam serves sampling mesh not the domain of geometry.
27

---

Published as a conference paper at ICLR 2026
Algorithm Training Procedure for GRAPHGLUE
Require:
Epoch index e, optimizer, datasets Dmxx: Deingle, Mmutt with data name mappings.
Ensure: Updated model parameters O
// Stage 1: Mix Training for Local Construction
Initial Liocal 0
2: for each batch B in Dmxx do
3:
Z. WW GRAPHGLUE(B))
4: Ciocal < ContrastiveLLoss Z)
5:
if 2 warmup epochs then
6:
Sproto < Prototypel ooss Z, data_name)
7:
Glocal f- Local + Cproto
8:
end if
9:
VoCloeal + Backward (Lowal)
10:
OptimizerStep()
11:
Update Prototypes with Z, W in Eq. (
end for
// Stage 2: Mix Training for Global Manifold Skeleton
for each batch mix do
14:
Z. W + GRAPHGLUE(B))
Ekmn Cross-Dataset KNN Graph( Z, data name)
16:
T <- Pample/rringglePatsss(
Number_Same Sampled, T sample)
17:
Cgo0 +0
18:
for 1 = 1 to sample do
19:
Lgeo += GeometricLosss W, T(I) in Eq. and Eq.
20:
end for
21:
Egeo f- -Lyon/
22:
V VoLyeo <- Backward((Lge))
23:
OptimizerStep()
end for
// Stage 3: Refine Local Manifold Structure For Each Dataset
for each dataset D. in siingle do
26:
Load graph data G, with edge set E T. + SampleTriang TampleeTiaaglllPaahs(,,,, Number Sampled, Treal)
28:
for - 1 to Ticcal do Construct mini-graph batch B, from Tx(1]
29:
30:
z, Ztan GRAPHGLUE(BB))
31:
Listine f GeometricLoss(WW W, Ta(t) in Eq. 77) and Eq.
32:
V VoLrefine Backward(Crriin,
33:
OptimizerStep()
34:
end for
35: end for
36: for each dataset D,,, in mutti do
37:
for each batch (B) in D,, do
Z. W +- GRAPHGLUE(B) Exmm <- Intra-Dataset KNN Graph( (
39:
T SampleTrianglePathssee
Number Sampled, T Tample))
41:
for t 1 to sample do Lgco += GeometricLoss(W W, T(t)) in Eq. and Eq.
42:
43:
end for
44:
Egeo f C
45:
VoLgoo & Bacckward
46:
OptimizerStep()
47:
end for end for
48:
return Optimized Model parameters e
28

---

Published as a conference paper at ICLR 2026
D ALGORITHMS
D.1 MULTI-DOMAIN PRE-TRAINING
The training procedure given in Algorithm which consists of the data loader for pre-processing.
To use unified interface, process all the graph datasets at the graph level, which means each data sample in the dataset a graph. Taking Reddit for instance, we extract 2-hop neighborhood ego-subgraph as a data sample for each node, and store the global edge index.
As we have many datasets from multiple domains, we need to build a mixture graph dataset loader that can iteratively load a batch of data from different datasets. For each batch, we uniformly sample from all source graph datasets.
During training, in each epoch, we first build locality recognition using graph contrastive learning Veličković et al. (2019): Qiu et al. (2020) that distinguishes the different semantics from different graph datasets. Meanwhile,
will update the Riemannian prototypes using EMA Izmailov et al.
2019) Morales-Brotons et al. (2024) for each dataset as in Eq. S After warm-up epochs, we
still cumple-prototypess contrastive learning that guarantees the prototypes are truly around the center of each dataset distribution. Second, we build cross-dataset KNN graph that builds a rough skeleton of the manifold, and learn from the regularization Eq. (5) and Eq. 7 Finally, we refine
the region for each dataset. We load every dataset and compute the geometric regularization, like the second step.
Here, since sampling triangles from a graph costs many computational resources, especially for large-scale graphs, we replace with sampling pairs of adjacent edges for effective implementation.
D.2 COMPLEXITY ANALYSIS
We list the cost of the key modules of GraphGlue in Table 3 where is the batch size, the number of graph samples in a batch; [V], [E] are the average nodes/edges per graph in a batch; d is hidden di- mension, setting 512 commonly. M is number of nodes TP in (k: M -spaaee perturbation, also the dimension of the manifold, commonly set to 32. k, is number of selected top-k, nodes in the sparse perturbation. T, is number of sampled triangle paths, NOT all triangles. For more effectiveness, we sample pairs of adjacent edges to approximate closed triangle paths.
Table 3: Computational and memory complexity of each module in GraphGlue.
Module
Computational complexity
Memory complexity
(k, M)-Sparse Perturbation
Okk MB)
OBM + d)) O(BMd) O(BM)
Adaptive Orthogonal Frame Matrix form of metric tensor
O | + M²)d) O(BMd) (T)
Cholo and Leurv
O(T.)
Riemannian prototypes and Epoto Riemannian MoE
O(KBd + K(d + MM) O(KBd)
O(K + M)) O(KB)
Thus, the total computational cost in pretraining phase is +| M K)d T.M), and the adaption (per graph) costs (([V] + [E]dd + K(d + M) TTM.. That is, in *GraphGlue* scales linearly with respect to the graph size. In our experiment, we pretrain the model on large-scale datasets, e.g., ogbn-arxiv and Reddit.
D.3 COMPLEXITY COMPARISON WITH OTHER GFMS
We compare the proposed GraphGlue to other GFM in pretraining and adaptation phases regarding the total computational cost. The results are summarized in Table
Notes
PRODIGY: In-context learning requires full attention over prompt and query nodes;
29

---

Published as a conference paper at ICLR 2026
Table Comparison of computational complexity across graph few-shot learning methods.
Model
Pretraining
Adaptation (per graph sample)
PRODIGY
(
O((VV + Fl)d + [V]3)
GFT
([B([]] + El)d BTh)
[([[V] [E]]d Th)
RAGraph
O + El)d + [d
O([V] + [E]]d + - d)
SAMGPT
O(B(V + [El]d + Bk,d)
(((V) + [E]]d d kpd)
GCOPE
O(B(V) + [E]d BKad)
O([V] + [E] K.d)
MDGFM
O(B([]] [E]] B(VP)
O((V E|)d [VI)
GraphGlue
(B((V]+|E) + + K)d+T.M) O((())1B11d+ (ddd+) M)
GFT: T: number of trees, h: tree height; tree construction adds overhead;
RAGraph: [E,]: retrieved edges from external library;
SAMGPT: ky: number of structure tokens, kp: prompt tokens;
GCOPE: Kg: number of virtual coordinators;
MDGFM: Graph Structure Learning (GSL) involves dense adjacency refinement;
GraphGlue: M 32, T, < [
Furthermore, we compare the memory cost to GCOPE and MDGFM on six datasets. [1, 2, 3, 6] denotes that we incrementally include ogbn-arxiv, computers, FB15k-237, Reddit, PROTEINS, HIV in the pretraining dataset. Under the setting of 512 batch size. [10, 10] neighbor sampler size, d = 512. Results on GPU memory cost (GB) are collected in Table
Table 5: Memory Cost. Lower values indicate better efficiency.
Model
5
GCOPE
18.39
19.11
21.12
OOM
OOM
OOM
MDGFM
19.71
21.67
29.35
0OM
OOM
OOM
GraphGlue
12.53
15.07
15.73
16.87
28.67
29.21
E RELATED WORK
E. GRAPH FOUNDATION MODELS
Graph Foundation Models (GFMs) aim to provide pre-trainable, general-purpose deep learning ar- chitectures for graph-structured data Wang al. (2025b); Liu al. (2025). Recently, researchers have extended the capabilities of Large Language Models (LLMs) to text attr-bbttibbtedd graphs, enabling
cross-domain transfer learning through textual descriptions Zhu et al. (2025); Xia al. (2024): Tang
et al. (2024); Ren et al. (2024); Chen et al. (2024). Additionally, GFMs have been developed for various specialized domains, such as knowledge graphs Huang et al. (2025); Luo et al. (2025). recommender systems Wu et al. (2025), and molecular graphs Xia et al. (2023); Sypetkkwski et al.
(2024). Given the prevalence text-free graphs, recent efforts have focused on constructing general-
purpose models via multi-domain pre-training Yuan al. (2025): Chen al. (2025). Wang et al. 2025a).
E.2 MULTI-DOMAIN GRAPHI PRE-TRAINING
In graph pre-training, Graph Neural Networks (GNNs) are trained by self-supervised learn- ing-either generative Hou al. (0022) or contrastive Veličković et al. (2019); Qiu et al. (2020). In light of the semantic heterogeneity across different domains, several methods have been proposed
to learn shared or invariant knowledge Yuan et al. (2025); Chen et al. (2025); Wang al. (2025a).
Despite the encouraging results, the theoretical foundations of how knowledge is integrated and transferred remain underexplored.
30

---

Published as a conference paper at ICLR 2026
In graph pre-training, Graph Neural Networks (GNNs) are trained using self-supervised learn- ing either generative Hou et al. (2022) or contrastive Veličković et al. (2019); Qiu et al. (2020) to capture intrinsic semantics from unlabeled data. While traditional pre-training typically operates within a single domain, multi-domain graph pre-training has recently attracted growing interest. However, integrating knowledge across diverse domains remains challenging due to significant mantic heterogeneity. Several methods have been proposed to learn shared or invariant knowledge using advanced techniques Yuan et al. (2025); Chen et al. (2025): Wang et al. (2025a). For instance, Chen et al. (2025) addresses architecture inconsistency by using disentangled learning to adaptively customize network architectures based on invariant graph patterns. Meanwhile, Yuan al. (2025) aligns multi-domain features with domain-invariant aligners and uses graph spectral-based error bound to theoretically guide knowledge transfer. Despite the encouraging results, the theoretical foundations of how knowledge is integrated and transferred in this context remain underexplored.
E.3 GRAPH FINE-TUNING AND PROMPT LEARNING
The alignment of pre-trained graph models with downstream tasks necessitates an adaptation phase, and existing adaptation methods can be roughly categorized into two paradigms: graph fine-tuning and prompt learning. Concretely, graph fine-tuning adapts the model behavior using limited target- domain data Sun et al. (0244);; Wang et al. 2024 2025c). For example. Sun et al. (2024d) fine- tunes the entire model downstream data. A more common strategy is keep the majority of the pre-trained parameters frozen and only train a simple classification head, a technique employed by models like Zhao al. (2024); Liu et (2024). [Bevilaqqua al. (2025) proposes a unique adapta- tion strategy: it freezes the large, pre-trained expansion map and only trains a smaller reduction map and task task-specific head for the new task. And recent advances introducing parameter-efficient fine- tuning methods such as low-rank adaptation Yang et al. (2025b). On the contrary, graph prompting keeps the pre-trained parameters frozen and enhances performance by inserting learnable prompt vectors Yu et al.] (2025a); Liu et al. (2023); Yu et al. (2024a]b) 2025b). For instance, [Lu et al. (2023) unifies tasks under a subgraph similarity template and employs a learnable vector to guide the READOUT function. Other approaches generate more adaptive prompts, such as PRONOG Yu ct al. (2025b), which uses a conditional network to create node-specific prompts for non-homophilic graphs, and PRODIGY Huang et al. (2023). which formulates novel prompt graph for in-context learning. To tackle more complex scenarios, several works have developed dual aual-prompting mecha- nisms. Jiao al. (205); Yu et al. (2024a) introduce feature prompt and a heterogeneity prompt to bridge the gap between homogeneous and heterogeneous graphs. Yu et al. (2024c) leverages a composed prompt for task-specific knowledge and open prompt for global knowledge from mul- tiple pre-training tasks. Similarly, Yu et al. (2025a) designs holistic and specific structural prompts for cross-domain adaptation. Yet, how quantity the transfer effort the target domain remains an open issue.
E.4 RIEMANNIAN GRAPH REPRESENTATION LEARNING
In recent years, Riemannian manifolds have emerged as promising alternative to traditional Eu-
clidean spaces in graph representation learning Sun et al. (2026b 2025a 2024a b. 2026a 2023b a 2024c). Most existing Riemannian models are tailored to specific tasks Grover et al. (2025). and
often leverage the particular manifolds, such as the hyperbolic space Chami et al. (2019 Yang et al.
(025a),, the spherical space Liu et al. (2022), the symmetric positive definite manifold Ju & Guan
2024), and their products Gu et al. (2019), Bachmann et al. (2020); Sun et al. (2022a) and quotients Xiong al. (2022). Very recently, Sun et al. (2025b) introduces a structural vocabulary and designs graph foundation model. In
a new GNN backbone on the product manifold for contrast to backbone architecture design, focus lies in developing framework for multi-domain pre-training and characterizing a general manifold that underlies diverse graphs.
F EMPIRICAL DETAILS
F.1 DATASET DESCRIPTION
This section provides a detailed description of the 12 benchmark datasets used in our experiments. For summary of their statisties, please refer Table 6
31

---

Published as a conference paper at ICLR 2026
Table 6: Statistics of 12 datasets used in our experiment.
Domain
Dataset
Task
# Graphs
Avg. #Nodes
Avg. #Edges
# Classes
PubMed
Node
19,717
88,648
Citation
Arxiv
Node
169,343
1. 1,166,243
Computers
Node
13,752
491,722
Co-purchase
Photo
Node
7,650
238,162
Reddit
Node
232,965
114,615,99
Social Network
FacebookPagePPse
Node
22,470
342,004
FB15K 237
Edge
14,541
310.116
Knowledge Graph
WordNet18:2 8RRR PROTEINS MUTAG HIV Lipophilicity
Edge
40,943
93,003
Graph
1.113
39.1
145.6
Bioinformatics
Graph
17.9
39.6
Graph 41,127
25.5
Molecule
Graph 4,200
27.0
59.0
Our experiments utilize a diverse set of 12 benchmark datasets. For citation networks, we include PubMed, where nodes represent scientific publications, and the task classify their category, as well as Arxiv, a large-scale network for academic paper classification. In the co-purchase domain, both Computers and Photo are sourced from Amazon; in these graphs, nodes are products, edges signify frequent no-purchases, and the task is to predict product categories. For social net- works, Reddit is constructed from posts, with goal predicting post's community, while FacebookPa gePage consists of official pages with edges mutual likes, and the task is to clas- sify the page's category. Our knowledge graph datasets include WordNet BRR, where the task is to classify the semantic relation between synsets, and FB1SK 237, used to predict the relation type between entities. Finally, for graph-level classification, we use several benchmarks: PROTEINS and MUTAG are bioinformatics datasets for binary classification, with the latter predicting compound mutagenicity; similarly, HIV and Lipophi licity are molecular datasets for binary classification tasks that predict molecular properties.
F.2 BASELINES
We evaluate our model against comprehensive set of basclines from three main categories: Super- vised GNNs, Self-Supervised GNNs, and Graph Foundation Models.
Supervised GNNs
This category includes foundational GNNs that are trained from scratch supervised manner for a specific downstream task.
GCN Kipf & Welling (0177) is a widely used GNN model that generates node represen- tations by aggregating information from local node neighborhoods. It employs mean- pooling approach for neighborhood aggregation to integrate information from adjacent nodes.
GraphSAGE Hamilton al. (017) is inductive representation learning framework de- signed for large graphs. utilizes a mean-poos pean-pooling propagation rule and often employs a neighborhood sampling approach scale efficiently large-scale graphs.
GIN Xu al. (2019) is state-of-the-art GNN that is commonly used as powerful super- vised baseline, particularly for graph classification tasks.
Self-Supervised GNNs These methods first pre-train a GNN encoder on unlabeled graph data using self-supervised objectives and are then fine-tuned for downstream tasks. They represent the predominant pre-training paradigm in graph machine learning.
DGI VVelčković et al. (2099) learns node representations by maximizing the mutual in- formation between local patch representations and a global graph summary vector. Its contrastive objective is notably not based random walks.
GraphMAE Hou et al. 2022) operates by masking portion of node features and training a GNN-based architecture to reconstruct them. utilizes a scaled cosine enror for recon- struction improve training robustness.
32

---

Published as a conference paper at ICLR 2026
GCC Qiu et al. (2020) is a self-supervised pre-training framework designed to capture transferable structural representations across multiple networks, Its pre- pre-training task is sub- graph instance discrimination, using contrastive learning to distinguish between augmented views of node's local subgraph and those from other nodes.
Graph Foundation Models This group comprises recent, large-scale models pre-trained on di- verse datasets and fine-tuned for strong generalization. They are the most direct competitors to our work and represent the current state-of-the-art.
PRODIGY Huung et al. 2023) enables in-context learning over graphs by formulating tasks with a novel prompt graph representation. This structure connects prompt examples with queries, allowing the model to perform new tasks without updating its parameters.
GFT Wang et (2024) rethinks transferable patterns as computation trees derived from the GNN message-passe passing process. It uses a tree reconstruction task for pre- pre-training and unifies downstream tasks as tree classification.
RAGraph Jiang et al. (2024) a retrieval-augmentt framework that improves GNN gen- eralization by retrieving knowledge from an external library of toy graphs. The retrieved information is injected into the target graph using a message-passing prompt mechanism to enhance performance.
SAMGPT Yu et al. (2025a) text-free graph foundation model for multi-domain pre- training and cross tross-domain adaptation. It uses learnable structure tokens to harmonize struc- tural differences across domains during pre-training and dual prompts to adapt knowledge new target domains.
GCOPE Zhao et al. (2024) mitigates negative transfer during cross-domain pre-training by introducing coordinators, which are virtual nodes that act bridges between disparate graph datasets. This approach helps create a unified representation from multiple graphs.
MDGFM Wang et al.) (2025a) focuses on achieving robust knowledge transfer through topology alignment. It employs Graph Structure Learning (GSL) module to refine graph structures, reduce noise, and learn domain-invariant knowledge.
F.3 IMPLEMENTATION NOTES
Our primary framework is a leave-one-out cross-domain evaluation. We pre-train models on five source datasets and evaluate them on a single held-out target dataset. This protocol applies to Self- Supervised GNNs and Graph Foundation Models. In contrast, supervised GNNs are not pre-trained and are instead trained directly from scratch on the target task. All downstream evaluations use few-shot fine-tuning setting. For the pre-trained models, we use only k labeled samples per class from the target task for fine-tuning. In experiments, k is set to 1 and 5. After setting aside these training samples, the remaining data is randomly split into a validation set (10%) and a test set (90%) ). We evaluate performance across three downstream tasks: node classification, Link Classification, and graph classification. For node and Link Classification, we use Accuracy (ACC) as the evaluation metric. For graph classification, we use Area Under the Curve (AUC). To ensure robust results, the final reported score for each experiment is the average over 10 runs with different random data splits.
For pretraining, we extract the 2-hop ego-graph with 10 neighbors each hop for single graph datasets and adopt 2-layer GCN Kipf & Welling (2017) as backbone model. The dimension of the manifold, or the number of virtual nodes (k, M -sparee perturbation, set M 32 with A 15. For the KNN construction for mixed data training in Algorithm and multi-graph datasets training, we also set k = 15. The dropout rate 0.1 and the learning rate is 1e-4 The model input dimension is 128. For different datasets, unify the input dimension by random projection
SVD. For the & Leskoved (2016) to get the node embeddings. The hidden dimension is 512. The temperature in contrastive learning is 1.0. The optimizer is Adam Kingma & Ba (2015). with a cosine annealing schedule Loshchilov & Hutter (2017).
knowledge graph datasets, we use VoddeVeec/rrovve
Table to Table 12 show the hyperparameters in few-shot transferring: the learning rate Ir. dropout rate drop, the KNN number & between prototypes and target graph data points, and the balance coefficient A. We adopt the classifier head with only linear layer for the node graph classification
33

---

Published as a conference paper at ICLR 2026
task. For the link classification task, simply adopt bilinear layer as classifier. The gated function for Riemannian MoE is an MLP with layers.
Table 7:
for s-shot and 5-shot cross-domain transfer on Arxiv.
Table 8: Plyper-paramedd for 1-shot and 5-shot cross-domain transfer on Computers.
Ir
drop
Ir
drop
k
X
1-shot
le-3
0.1
1.0
1-shot
le-3
0.2
3
1.0
5-shot
le
0.15
1.0
5 Shot
1e
0.2
3
1.0
Table 9: Hyper-paramen parameters for 1-shot and 5-shot cross-domain transfer on Reddit
Table 10: Syppe--arammter for 1-shot and 5-shot cross-domain transfer on FB1 237.
Ir
drop
Ir
drop
k
X
1-shot
le
3
0.1
0.1
1-shot
1e=4 4
0.5
3
0.5
5-shot
le-3
0.15
0.1
5-shot
1e=1 0.5
3
0.5
Table 11: parame-arsmeters for 1-shot and 5-shot cross- drmmi- transfer on PROTEINS.
Table 12:
for 1-shot and 5-shot cross-domain transfer on HIV.
Ir
drop
Ir
drop
k
X
1-shot
1e
3
0.1
2.0
1-shot
1e
0.1
2
2.0
5-shot
1e
3
0.15
2.0
5-shot
1e
0.15
2
2.0
G ADDITIONAL RESULTS
G.1 SUPPLEMENTARY RESULTS
We provide additional empirical results to further validate our framework. We present comprehen- sive results for both cross-domale transfer [1n and intra-domain transfer (Table and 21 in few-shot settings. Furthermore, an ablation study in Table (2 demonstrates the effectiveness of the key components of our model. We also include an additional visualization the pre-trained manifold in Figure 8 where first project the 511-dimensiion van der Maaten & Hinton (2008) and then apply RBF interpolation Wright Fornberg (2006) to generate smooth surface that approximates the learned global Riemannian manifold.
embeddings into 3-D using t-SNE
G.2 COMPREHENSIVE ABLATION STUDY
We conduct a further ablation study to verify the effectiveness of EMA, prototype loss and Rieman- nian MoE. Specifically, we introduce variants of GraphGlue, described as follows:
"w/o EMA" means that we replace EMA with the common average of batch of embed- dings;
"w/o Eprota" " means pretraining without prototype loss;
"w/o Riemannian MoE" means that during adaption, Riemannian MoB module replaced by typical prompting scheme.
In Table 13 both results on 1-shot setting and 5-shot setting demonstrate the effectiveness of the proposed components.
34

---

Published as a conference paper at ICLR 2026
Table 13: Ablation study of GraphGlue's 's key components.
Variants
Arxiv
Computers
Reddit
FB15k.231
PROTEINS
HIV
w/o EMA
15.46:1..41
30.84199.0
7.43=2.37
35.90=17.44 40
58.4812.56
52.62::2.41
I-shot
w/o L. proto
9.57-+4.56
31.24 10.36
37.90-19.34
43.59++8.
58,49++2.60
54. 24.10-22
GRAPHGLUE
29.73.12.56
67.13
68.42.14.68
60.8911.11
69.12.14.19
$8.53
w/o EMA
16.08-2.113
34,9017.77
11,53::2,26
40.21=12.07
59,85:4.533
53.87:2.443
S-shot
w/o L. proto
15.26419.91
36.63=11..4
16.13414.14
66.56=16.42
61.64:6.28
55.50=2.70
GRAPHGLUE
39.98.1..67
74.15 1.338
10.69
79.52.11.75
73.94.1
62.18
G.3 HYPERPARAMETER SENSITIVITY ANALYSIS
For AOF, investigate the hyperparameter sensitivity on the neighborhood size and the number of nodes M in (k, -sparsse perturbation. Results are shown in Table 14 and 15
Table 14: 1-shot results under different settings.
(a) Analysis on & (M 32).
k
Arxiv
Computers
Reddit
FB15k
PROTEINS
HIV
2
18.6413.10 49.80112.41
6..04
31.63:1..19
57.8013.05
53.07-1..02
5
17.16+3.08
43.76=10.78
68.57=6.76
21.10+22.47
59.49=2.62
52.04+3.01
10
15.29:12.69
46.21+11.110
21.03:+2.89
45.51
58.10
54.42:1..16
15
29.73:2.56
61.03:17.13
48.42:14.68
60.89:2.11
69.1244.19
58.5318.20
30
20.23++2.83
15.39+
49.57+4.80
38.85+19.62
54.36
54.49++3.33
60
18.2042.63
10.88=10.22
7..76
31.83+16.41
58.2443.34
51.6443.44
(b) Analysis on M (k: = 15).
M
Arxiv
Computers
Reddit
FB15k_233
PROTEINS
HIV
19.8943.79
42.66=11.17
60.57+4.48
45.16=5.85
56.9914.64
52.58+3.68
8
22.64:2..51
46.2419.12
61.7345.02
54.8019.57
58.77+1.92
52.3242.94
16
27.84+1.477
14.92
62.73+1.97
57.09+7.444
55.3445.68
54.18
32
29.73
61.03.17.13
6..42.14.68
60.89.1..11
69.12.1..19
58.53
Table 15; 5-shot results under different settings.
(a) on (M 32).
k
Arxiv
Computers
Reddit
FB15k 237
PROTEINS
HIV
2
29.49+3.114
70.82+3.114
80.46+ 1.21
36.24+..11
59.42+2.03
56.10+22.76
5
23.95
67.74.1..02
67.4118.774
34.26:114.50
60.45.1.005
54.76:12.33
10
25.86=7.17
70.59+18.03
80.4910.59
18.01
60.8643.08
55.6944.23
15
1.67
7..18
84.89
7.75
73.94
62.18
30
33.00+11.63
27.65429.35
64.5746.84
42.88-11.11
62.0942.455
54.63+3.01
32.59+1.57 60
68.08+1.66
85.24+00.2
45.42+8.09
60.40+1.35
54.01++.669
(b) Analysis M (k: 15).
M
Arxiv
Computers
Reddit
FB15k 237
PROTEINS
HIV
4
3..42
64.66+13..8 13.87
7.98
4..83
56.27++1.62 17.78
8
1..0
1.925+1.99
78.82+1.79
13.18
59.70+1..2
54.6442.35
16
37.78+:5.779
13.22
74.92+2.41
70.11+11.89
60.62+13.66
57.55+2..5
32
39.98-11667
74.15+22.38
84.89+00.68
79.52+1.77
73.94+2.38
62.18+22.50
35

---

Published as a conference paper at ICLR 2026
G.4 RESULTS ON HETEROPHILIC GRAPHS
We demonstrate the performance of GraphGlue on several benchmarking heterophilic graphs (Amazon-ratings, Roman-empire, Texas and Wisconsin). The results in Table 16 and 17
Table 16: Performance under different shot settings with pretrained on ogbn-arxiv, Reddit, Comput- ers, FB15k PROTEINS, and HIV.
Method
Amazon-Ratings
Roman-empire
Texas
Wisconsin
GCOPE
28.65:15.82
11.44=1991
33.19-6.62
31.22:16.85
1-shot
MDGFM
29.53++3.45
14.51+2.08
34.63++1.70
35.11=10.555
GraphGlue
31.16+3.56
16.23+3.00
35.16+20.43
37.95 10.91
GCOPE
30.06:1..11
16.00+11.29
36.31+10.14
38.21+2.96
5-shot
MDGFM
30.42++..80
17.15+1.66
48.33=6.36
47.46+44.8
GraphGlue
32.17=2.91
18.50=1.07
50.88-11..9
49.71+8.000
Table 17: Performance under different shot settings with pretrained on 8 datasets (including Amazon-Ratings and Roman-Enan -omann-Emprre..
Method Amazon-Rating
Roman-empire
Texas
Wisconsin
GCOPE
29.03:14.17
13.14422.36
33.82:17.39
30.08:15.13
1-shot
MDGFM
27.01
14.11
36.02-19.54
33.28
GraphGlue
34.12:12.57
18.1942.51
38.65
40.17 10.09
GCOPE
32.83+3.226
16.98
41.33
43.74:13.19
5-shot
MDGFM
32.54
16.77+1.92
48.10:17.26
46.62+44.16
GraphGlue
36.26-3.09
20.6711.34
52.60
51.4917.97
G.5 VISUALIZATION OF MANIFOLD GLUING
Manifold gluing aims to glue local pieces into smooth surface, whose process is described as follows.
First, we construct local geometry on each patch using (k, -sparse perturbation like drawing a coordinate grid;
Then, when two graphs share similar structures, we "glue" their grids together along over- lapping regions, ensuring no stretching or twisting (via the isometry of Def. 4.4 and holon- omy of Eq. 5);
Finally, we smooth the entire surface so that curvature changes gradually-forming a unified manifold where knowledge flows naturally across domains.
In addition, we visualize toy example of the aforementioned process in Figure
36

---

Published as a conference paper at ICLR 2026
Local geometry construction
Isometry and holonomy regularization
k-order smoothing regularization
Figure 7: Visualization of the pre-trained manifold from 6 datasets.
Table 18: Performance of cross-domain transfer on various downstream tasks in the 1-shot setting, reported as mean std over 10 runs. The highest result is bolded, and the runner-up underlined.
Node Classification
Link Classification
Graph Classification
Model
Arxiv
Computers
Reddit
FB15k_237
PROTEINS
HIV
GCN
11.67
8944.933
11.15
32.1 11.337
50.11+13.07
72.56=5.30
GraphS AGE GIN
14.68
35.47=8.29
14.6942.31
35.74+2.1
58.99=2.79
56. 78=3.75
11.20+2.03
44.77
18.53+1.89
38.2542.55
13.22+13.50
52.63=7.47
GCC
12.65+2.08
34.82+6.1
54.78+5..6
47.84+1.96
59.20=7.97
52. 22.63=3.66
DGI
13.32+3.35
35. 35.26+77.55
60.088+4.90
42.50+2.08
53.18=8844
52. 822-0.2.55
GraphMAE
12.61
33.89 86
11,15
51.3411.87
60.11 13.007
52.
PRODIGY
28.45+2.20
15.32:44.10
35.67 1:3. 20
53.50+1.0r
48.90
41.78
GFT
26.59+2.45
54. 14.6552:4.08
58. 38.87:22.58 1.2.3
58.07:11.39
55.41
58.94
RAGraph 18.71
46.2 1:4.37
52.56133. 68
52.1813.04 14.00
51.4225.18
54.26
SAMGPT
24.15133.81
47.61
62.85 62.85.14.22
57 147.441..46
52.42.3.15
55.48
GCOPE
26.52+5.56
54.55
62.76:14.55
58.25++2.07
55. 15.1993..0
58.93=2.60
MDGEM
26.05:2.400
46.6829.45
64.88
56. 56.11:1.66
53.41
51.46=2..5
GRAPHIGLUE
28.8845.22
59.50+7.05
.7.12433.3
59.75+6.27
59.87=4.85
60. 20.
37

---

Published as a conference paper at ICLR 2026
Table 19: Performance of cross-domain transfer on various downstream tasks in the 5-shot setting, reported as mean std over 10 runs. The highest result is bolded, and the runner-up is underlined.
Node Classification
Link Classification
Graph Classification
Model
Arxiv
Computers
Reddit
FB15k.233
PROTEINS
HIV
GCN
27 13
65.78
28.36=1.01
52.43-+-..8
55.04 9.99
47. 47.81+:3 $:4.991
GraphSAGE
26.18:2221
66. 4..4
22.27=1117
58.91+1.52
60, 10
50.59
GIN
26 26.06:2242
69.51 13.50
29.03=1.66
63.7611.79
58.8715.06
49.1224.85
GCC
26.84+2.14
62. 02.63+33.00
1..2
73.69+1.24
61.20++300
17.4
DGI
27.1842.33
61.02+3.200
62.72-2.21
68.32+1.46
53.34
22.234444
GraphMAE
27.68.42.13
65. 7..44.2
28.36=1.01
77.25411.07
65.04
57.81
PRODIGY
33.67:2..0
52. 02.788:3..0 60
42.34=2.90
72.17:1699
35.2311477
48, 18,655::3.. 80
GFT
36.7811.92
69. 133156
66,28
19.1311.68
62.18 18.1..3959
57. 57.68
RAGraph
32.35++1.78
62.38
6..08=1.32
64..22+22.77
58.62+22.80
56.32+3.44
SAMGPT
34.42+22.5
60.87 43.64
75.12 12
77.63+22.1
59. 14...4:2.60
37.63+2.87
GCOPE
39.18=1.96
22.27:1 2.284
80.45
79..3
64.85
58.47#1822
MDGEM
32.28
64.08
76.55=1.72
77.6742.00
57.79#3.42
55.79+3.16
GRAPHGLUE 37.0212.33
73. 20.70
85.05
81.51
65.3211..5
61.55:12.66
Table 20: Performance of intra-domain transfer on various downstream tasks in the 1-shot setting, reported as mean + std over 10 runs. The highest result is bolded, and the runner-up is underlined.
Node Classification
Link Classification
Graph Classification
Model
Arxiv
Computers
Reddit
FB15k.237
PROTEINS
EIV
GCN
12.61+1.75
33.89
11.15=2.14
32.11+2.37
60.11+13.07
52.56+5.30
GraphSAGE
14.68.3.78
35.4718.29
a2.6
35.7412.19
68.99
56.78
GIN
11:2042.00
44.77+6.02
18. 18.53
38.25442.56
64.22413.50
52.63
GCC
12.6512.08
64.8226.11 18
(4.7825.66 64
47.80
59.20
52.63113.66
DGI
13.32+3.35 35.26+7.66
60.08=4.80
12.56++2.00
53.18+8.44
52.80+7.53
GraphMAH
12.61+1.76
33.89
11.15=2.14
51.34+1.87
60.11+13.07
52.56+55.30
PRODIGY
28.4542220
45.32+44.10
35.67
53.5041.oz
48,90
41.78++4.50
GFT
28.83+1.76
53. 13.94 47
G3. (3..0322.34 .44
59.43-10.87
63. 13.54444.98
38.17755.70
RAGraph
20.53:2215
60.39
59.9112.79
52.09 12.557
52.831
55.73113.00
SAMGPT
25.88+3.58
55.31 +0.67
63.05=3.77
28.75++2.16
61.59+22.99
22.38
GCOPE
27.41+4.77
58.24+7.48
65.07=3377
58.33+1.79
68.55
60. 60.67=22 27:42222.2
MDGEM
10.76+2.04
43. 43.222
64.38=3.11
58.32+1.71
37.79+11.51
53.03=33.88
GRAPHGLUE
29.73=22.6
61.03:77:1
68.42114.68
60.89
69.1214419
58.53+8.20
Table 21: Performance of intra-domain transfer on various downstream tasks in the 5-shot setting, reported as mean std over 10 runs. The highest result is bolded, and the runner-up is underlined.
Node Classification
I Link Classification
Graph Classification
Model
Arxiv
Computers
Reddit
FB15k 237
PROTEINS
BIV
GCN
27.68
44.78
28.36=1.11
12.413+1.87
65.0449.m 98
37.81+3.91
GraphSAGE
18
44.45
22.27=1.17
58.91+1.02
70.45+1.8
60.59+0.70
GIN
26.06+22.22
69.51 23.50
29.03=1166
63.7641.73
68.8745.06
59. 194-444
GCC
26.8412.14 14
02.63-23.. 16
65.21 21.56
73.6 73.6411.25
64.20 09
58.3412.10
DGI
27.18422.3
61.02:33.0 20
62.72=2.21
68.32+1447
53.34 63.34416.5 27
52.23 22.233::.0 49
GraphMAE
27. 27.68+2..1
65.78+44.20
28.36=1.01
77.25+
65.04+9..0 49.99
57.81
PRODIGY
33.67
52.78. 22.78.1.3.00
42.34
72.1716.91
55.23.14.70
48.65
GFT
39.02.41.39
73.41 4:3.21
7..37=1145
79.25:40.94
74.63 14.69942.4
61.03+44.45
RAGraph
35.74+1.46
61.98
66.30=0.75
67.86+1.60
62.52+3.
59. 29..3322.55
SAMGPT
38.1441.87
64.68 14..6822..55
1..8
78.7642.33
70.48
59.09:2
GCOPE
39.45.11.2
73.06.12.0
02.12
78. 19..9911.87
73. 73.26.5
60.05
MDGFM
19.17+2.38
68. 19.00.0
81.27=1.23
78.24
65.95488.2
54.73+4.37
GRAPHGLUE
$9.98
74.15 1:333
84.8911..45
79.52.1.75
69.744.2.00
62.18
38

---

Published as a conference paper at ICLR 2026
Table 22: Ablation study of GRAPHGILUE's key components.
Variants
Arxiv
Computers
Reddit
FB15K.237
PROTEINS
HIV
w/o Leury
22.33
49.631..1
64.3815,12
63.12
61.21
50.34
I-shot
w/o Cadt
27.14:3.62
56.3924.16
65.9314.33
54.85
53.23:24448
54.83=2.11 15
GRAPHGLUE
28.88-5.22
59.50.1705
67. 67.12:13.39
59. 39.75:15.22 27
55.8714.85
60. 60.2213.00
w/o Leury
29.1713.14
85.3388
74. 14.19.11.2
69.3313399
58.77.14.35
57.13.3.30
5-shot
w/o Casie
35.77+264
2..1642..4
79. 10.11:33 1:3.47
74.02:0.5P
58.74
54.1224.19
GRAPHGLUE
37.02:22.3
73.22 23.29.20770
85.05=1.17
81.51:2.31
65.321:245
61.55:12.66
ogbn-arxiv
Computers
Reddit
FB15K_233 237
PROTEINS
HIV
Figure Visualization of the pre-trained manifold from 6 datasets.
39

---

Published as a conference paper at ICLR 2026
H REPRODUCIBILITY STATEMENT
This part provides the reproducibility statement claims, theory assumptions and proofs, empiri- cal result reproducibility, empirical setting/details, empirical statistical significance, open access to data/code, computation resources, code of ethics, safeguards, licenses for existing assets, new assets, crowdsourcing and research with human subjects, declaration of LLM usage, and broader impacts.
1. Claims. Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Yes. Main claims made in the abstract and introduction reflect the contributions in Sections
4, 5 and 6.
2. Theory assumptions and proofs. For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Yes. The theoretical results including the assumptions are clearly stated in Theorems in this paper, while the complete and correct proofs are provided in Appendix B.
3. Empirical result reproducibility. Does the paper fully disclose all the information needed reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided not)?
Yes. Key information is introduced in the subsection of "Evaluation Protocals", and further details are disclosed in Appendix F entitled "Empirical Details".
4. Empirical setting/details. Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Yes. Specifications are provided in Appendix F entitled "Empirical Details", and the full details are included in the code.
5. Empirical statistical significance. Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experi- ments?
Yes. In the experiment, each case undergoes 10 independent runs, and report the mean with the error bar of standard derivations.
6. Open access to data and code. Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results? Yes. Codes and data are available at the anonymous GitHub link with sufficient instructions.
7. Computation resources. For each experiment, does the paper provide sufficient infor- mation on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Yes. The computer resources for the evaluation are described in Appendix F entitled "Em- pirical Details".
8. Code of ethics. Does the research conducted in the paper conform, in every respect, with the ICLR Code of Ethics https:://clr,.. wtt.cll..ccpuuluc ccc/publl ic/oode0fff i ics? Yes. We confirm that the research conducted the paper conform, in every respect, with the ICLR Code of Ethics.
9. Safeguards. Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Not available.
10. Licenses for existing assets. Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Yes. The original papers that produced the code package or dataset are properly cited in this submission.
40

---

Published as a conference paper at ICLR 2026
11. New assets, Are new assets introduced in the paper well documented and is the documen- tation provided alongside the assets?
Yes. The documentation is provided alongside the Codes of the proposed model.
12. Crowdsourcing and research with human subjects. For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Not available. This paper does not involve crowdsourcing nor research with human sub- jects.
13. Institutional review board (IRB) approvals equivalent for research with human subjects. Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Not available. This paper does not involve crowdsourcing nor research with human sub- jects.
14. Declaration of LLM usage. Does the paper describe the usage of LLMSs (especially when it is an important, original, or non-standard component of the core methods this re- search)?
Yes. LLM is used to polish writing only, and include a section of "Declaration of LLM Usage" the Appendix.
15. Broader impacts. Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Yes. Both potential positive societal impacts and negative societal impacts are included in the section of "Broader Impact and Limitations" in the Appendix.
I ETHICS STATEMENT
We confirm that the research conducted in the paper conform, in every respect, with the ICLR Code of Ethics http::tttt:/tttett httpp//ccccccc//cooootttttt
DECLARATION OF LLM USAGE
Large Language Model (LLM) used polish writing. Concretely, we refine the textual contents in Section 1 (Introduction) and Section 7 (Conclusion) with LLM..
K BROADER IMPACT
Our work brings together two previously separate domains - multi-domaal graph pre-training and differential geometry. Our constructions taking in multi-domain graphs with unified, smooth Rie- mannian manifold, thus enabling the solid tools of differential geometry systematically understand the knowledge integration and transfer across graphs. Theoretically, we develop the neural mani- fold gluing that makes the differential geometry principles implementable through deep learning. In practice, the proposed pre- pre-trainin training model paves the way to build a powerful graph foundation model with better generality and quantifiable transferability.
Positive societal impacts lie the transfcrability and generality of the proposed graph pre-training model, allowing for the analysis more complicated real-world graphs. None of negative societal impacts we feel must be specifically highlighted.
41


Reviews:
[{"review_id": "Zw6xvN1iuH", "reviewer": "Reviewer_w9Kq", "summary": "This paper focuses on how knowledge is theoretically integrated and transferred during multi-domain graph pre-training, which is a fundamental and underexplored problem in the field of graph foundation models (GFMs). Authors propose a novel perspective from differential geometry, with the core idea of gluing diverse graph datasets into a unified Riemannian manifold. Moreover, authors empirically validate GRAPHGLUE’s geometric scaling law and show that larger quantities of datasets could improve the transferability of models by producing a smoother manifold.", "questions": "Please see the weakness.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- This paper is good in originality and introduces a principled and powerful theoretical framework from differential geometry. This Neural Manifold Gluing concept provides a new, systematic, and theoretically sound language to model knowledge integration in graphs, which is a conceptual advance for the GFM field.\n- The experiments provide strong support for the theory. The Geometric Scaling Law experiment shows that 1-shot accuracy improves and transfer loss decreases as more datasets are added, which is a powerful validation of the smoother manifold hypothesis. Furthermore, the case study in Figure 5 shows that GRAPHGLUE benefits from adding semantically distinct domains in mitigating negative transfer is a strong practical result.", "weaknesses": "- The proposed theory assumes that all source and target domains can be glued into a single smooth manifold. However, it is unclear how the model would perform if a new domain is fundamentally geometrically incompatible. For example, what if a new domain possesses a markedly different intrinsic dimensionality? Therefore, further discussion on the limitations and potential failure cases would be valuable for practical use. \n- Several key design choices in the GRAPHGLUE framework are not ablated. In particular, what is the impact of the EMA prototyping and the L_proto loss? How critical is the Riemannian MoE during adaptation compared to a simpler prompting scheme? I expect a more comprehensive ablation that provides deeper comprehension regarding the impact of individual components within the full framework. \n- The proposed framework involves several non-trivial operations. For instance, (k, M)-sparse perturbation, maintaining an Adaptive Orthogonal Frame (AOF), and calculating gluing losses (L_holo, L_curv). I am curious about the computational cost and memory complexity of the overall pre-training and adaptation process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "psws3K42SH", "reviewer": "Reviewer_RdxQ", "summary": "This paper introduces GraphGlue, a novel framework for multi-domain graph pre-training that leverages principles from differential geometry to address the long-standing challenge of knowledge transfer across diverse graph domains. By conceptualizing graphs as pieces of a unified Riemannian manifold, the authors provide a systematic approach to ensure that these domains are seamlessly “glued” together, preserving both local and global geometric consistency. The framework employs an Adaptive Orthogonal Frame (AOF) to model local graph structure, then uses holonomy-based regularization to smoothly align these pieces into a coherent whole.\n\nOne of the key contributions of this work is the Geometric Transfer Metric (GTM), which quantifies the transfer difficulty between pre-trained models and target domains. The experiments show that GraphGlue achieves superior transferability and scalability across multiple graph domains, outperforming existing methods. The paper also introduces a new empirical insight: geometric scaling law, which suggests that increasing the number of domains during pre-training leads to a smoother manifold and better generalization.", "questions": "1. See weaknesses.\n2. Could you provide a clear logical chain explaining the connections and roles of the numerous theorems and lemmas? **This is my main concern**, and I hope the authors can primarily address this question within a limited number of characters.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper introduces a novel approach to multi-domain graph pre-training by treating the graphs as local pieces of a larger, unified Riemannian manifold. This fresh perspective allows for a more systematic understanding of knowledge transfer across domains, which is a critical issue in graph foundation models.\n2. The concept of “neural manifold gluing” is well-formulated, using differential geometry to tie together multiple domains. The method employs an Adaptive Orthogonal Frame (AOF) to model local geometry and uses holonomy and curvature constraints to ensure smooth global alignment. This provides both theoretical depth and practical utility.\n3. The experiments cover a range of graph domains and demonstrate that the proposed method outperforms previous models in terms of transferability. The geometric scaling law showing that increasing the number of domains improves transferability is both a novel and insightful contribution to the field.\n4. Convincing theoretical analyses.", "weaknesses": "1. The method heavily relies on triangle-based holonomy for graph gluing. However, in sparse graphs or those with few cycles, this assumption may not hold, limiting the approach's applicability. Further analysis of the method’s behavior on sparse or acyclic graphs is needed.\n2. The paper introduces the AOF for local geometry estimation, but does not provide sufficient analysis on how sensitive the results are to the choice of hyperparameters like perturbation strength and neighborhood size.\n3. Some parts of the paper writing needs to be polished.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8TzRodaKm8", "reviewer": "Reviewer_9yab", "summary": "This paper presents a novel differential-geometry view of multi-domain graph pretraining and domain adaptation, which the authors call a neural manifold gluing framework. Locally the model learns tangent-space bases and local metrics via an Adaptive Orthogonal Frame (AOF) and a (k, M)-sparse perturbation; globally the framework “glues” local pieces using edge tangent translation and holonomy/curvature-based smoothing to induce a smooth Riemannian metric across domains. The method includes practical engineering (EMA-based prototyping and a Riemannian MoE) to scale to large multi-graph settings. Experiments on multiple datasets and leave-one-domain-out few-shot transfer show consistent improvements over baselines and an empirical “geometric scaling law.”", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Strong theoretical foundation and unification. The paper provides a mathematically solid perspective, casting multi-domain pretraining and domain adaptation as a manifold-gluing problem; this is novel and helps unify several previously disparate ideas about metric compatibility, holonomy, and transferability. Theorem-level results and carefully defined operators (e.g., edge tangent translation) make the theoretical contribution convincing.\n\n2. Practical scaling via an EMA strategy. The use of EMA prototypes and the design choices to make the framework operate on large graphs are valuable. These engineering choices materially improve the applicability of multi-domain graph learning to realistic, large-scale settings.\n\n3. Comprehensive experiments and diverse datasets. The authors evaluate across a variety of datasets and tasks (leave-one-domain-out few-shot settings, ablations, scaling-law experiments). The empirical section is thorough and supports the main claims.", "weaknesses": "1. The idea of “manifold gluing” may depend on the number of domains and on how many pairwise/collective gluing operations are needed. The paper briefly remarks that a QR-based subroutine reduces complexity, but it lacks a clear, end-to-end complexity and empirical runtime/memory analysis showing how cost scales with the number of domains, graph size, and manifold dimension.\n2. The paper states (line ~1532) “For pretraining, we extract the 2-hop ego-graph with 10 neighbors each hop for single graph datasets and adopt a 2-layer GCN...” Please clarify: how are those 10 neighbors chosen? Are they random uniform samples among neighbors, the top-10 by degree/score, or chosen by some importance sampling / structural heuristic? This will inform whether the pretraining pipeline’s effectiveness depends on a particular sampling heuristic.\n3. Many of the node-classification datasets used appear to be homophilic. The manifold gluing assumptions may rely implicitly on local coherence of labels/structure; it is unclear whether the proposed framework adapts to heterophilic graphs. We suggest that the authors either include experiments on heterophilic benchmarks (or a small toy study) or provide a discussion of expected behavior if heterophilic graphs are included in pretraining.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X08xAlpCxp", "reviewer": "Reviewer_5PYX", "summary": "The paper proposes a differential-geometry view of multi-domain graph pre-training: neural manifold gluing that merges graphs from diverse domains into a unified smooth Riemannian manifold. Locally, geometry is estimated via a (k, M)-sparse perturbation and an Adaptive Orthogonal Frame (AOF) to form tangent-space bases and local metrics. Globally, pieces are “glued” by edge tangent translation (isometries) with holonomy (triangle) triviality and further curvature-based smoothing to promote a smooth global metric. Experiments over six domains show strong few-shot transfer and a geometric scaling law: adding more pre-training datasets yields smoother manifolds and better transfer.", "questions": "1. Triangle coverage & sparsity: How does performance degrade when the underlying graph (or inter-domain scaffold) has few triangles, so triangle holonomy regularization is weak or inapplicable? Do you back off to cycle-based approximations or add synthetic motifs? (Additionally, some theorems are based on the assumption that every edge belongs to at least one triangle, e.g., Theorem 4.8)\n2. How sensitive is the learned manifold geometry to the hyperparameters of the AOF module (e.g., k and perturbation scale)?\n3. Could you provide more intuition or examples to help readers understand how the “manifold gluing” process works in practice?\n4. From which previously introduced equations or lemmas is **equation (30)** derived?\n5. See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Conceptual originality & unification: Frames multi-domain GFM pre-training/adaptation in a single geometric framework with local-to-global gluing, tying transferability to metric compatibility, holonomy, and Ricci-related smoothness—offering interpretable levers rather than only heuristics.\n2. Experiments span diverse graph domains and demonstrate solid performance and scalability. The observed geometric scaling law is an insightful and interpretable empirical finding that supports the theoretical motivation.\n3. The proposed framework is well structured, moving clearly from local metric learning (AOF) to global manifold construction (holonomy and curvature smoothing) and then to practical pre-training and adaptation.", "weaknesses": "1. Although the method is conceptually rich, the explanation is dense, with long mathematical expressions and limited intuition. The main framework diagram is visually cluttered and could better emphasize the data flow between modules. Simplifying the narrative and improving the visuals would greatly enhance clarity.\n2. The paper lacks a clear analysis of computational cost and training scalability — given that manifold operations such as matrix logarithm and curvature regularization can be expensive, a runtime or memory comparison would help readers assess practicality for large-scale deployment.\n3. The method relies on geometric assumptions such as triangle-based holonomy and curvature smoothness, but many real graphs are sparse or irregular, with very few closed loops. It remains unclear how well the model behaves when these geometric constraints cannot be fully satisfied. Adding experiments or visualizations on sparse or highly heterogeneous graphs would make the theory–practice connection more convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)