Modality-free Graph In-context Alignment
Authors: Wei Zhuo
Siqiang Luo
Venue: ICLR 2026 Oral (2026)
Keywords: Graph foundation model
In-context learning
Pretraining
Primary area: learning on graphs and other geometries & topologies
TL;DR: A pretraining framework that enables in-context learning on graph-structured data without modality assumptions.

Abstract: In-context learning (ICL) converts static encoders into task-conditioned reasoners, enabling adaptation to new data from just a few examples without updating pretrained parameters. This capability is essential for graph foundation models (GFMs) to approach LLM-level generality. Yet current GFMs struggle with cross-domain alignment, typically relying on modality-specific encoders that fail when graphs are pre-vectorized or raw data is inaccessible. In this paper, we introduce **M**odality-**F**ree **G**raph **I**n-context **A**lignment (MF-GIA), a framework that makes a pretrained graph encoder promptable for few-shot prediction across heterogeneous domains without modality assumptions. MF-GIA captures domain characteristics through gradient fingerprints, which parameterize lightweight transformations that align pre-encoded features and indexed labels into unified semantic spaces. During pretraining, a dual prompt-aware attention mechanism with episodic objective learns to match queries against aligned support examples to establish prompt-based reasoning capabilities. At inference, MF-GIA performs parameter-update-free adaptation using only a few-shot support set to trigger cross-domain alignment and enable immediate prediction on unseen domains. Experiments demonstrate that MF-GIA achieves superior few-shot performance across diverse graph domains and strong generalization to unseen domains. The code is available at https://github.com/JhuoW/MF-GIA.

Published as a conference paper at ICLR 2026
MODALITY-FREB IN-CNTENTT ALIGNMENT
Wei Zhuo, Siqiang Luo Nanyang Technological University
fwei.zhug siqiang. Luo]entu..diie edu.
ABSTRACT
In-context learning (ICL) converts static encoders into task-conditioned reason- ers, enabling adaptation to new data from just a few examples without updating pretrained parameters. This capability essential for graph foundation models (GFMs) to approach LLM-level generality. Yet current GFMs struggle with cross- domain alignment, typically relying on modality-specific
specific encoders that fail when graphs are pre-vectorized raw data is inaccessible. In this paper, we introduce Modality-Free Free Graph In-context Alignment (MF-GIA), framework that makes a pretrained graph encoder promptable for few-shot prediction across heterogeneous domains without modality assumptions. MF-GIA captures domain characteristics through gradient fingerprints, which parameterize lightweight transformations that align pre-encoded features and indexed labels into unified semantic spaces. During pretraining, dual prompt-aware attention mechanism with episodic objective learns to match queries against aligned support examples to establish prompt-based reasoning capabilities. At inference, MF-GIA performs parameter-update--ree adaptation using only few-shot support set to trigger cross- domsan alignment and enable immediate prediction on unseen domains. Experiments demonstrate that MF-GIA achieves superior few-shot performance across diverse graph do- mains and strong generalization unseen domains. The code is available stttp::/ggtthu com/ JhuOW/MF-GGA
INTRODUCTION
Table 1: Comparison of methods with respect to the three main criteria of true ICL.
The remarkable success of Large Language Models (LLMs) has fundamentally revolution- ized AI, with in-context learning (Brown et al.
2020, Zhang al.] [023] Lu et al.) (002) emerg- ing pivotal capability that enables these models to adapt to new tasks through mere expo- sure to few demonstration examples, without any parameter updates like fine-tuning. This paradigm shift, from task-specific fine-tuning to prompt-based adaptation, naturally sparks profound question for the graph learning com-
Mithed
Poa-Treming Fece
Damsis Aligament
Misidit-Fha:
GNNN
GFT Win wir.
Unigmph mueen mireut
AuscCFF (olx
Groph
MI-GIA
munity: Can we achieve similar foundation-level generality for graph-structured data? Unlike sequential text where context flows naturally, graphs encode complex topological patterns, multi- hop dependencies, and heterogeneous node and edge attributions that demand fundamentally new approaches to demonstration selection, prompt design, and reasoning.
Achieving true graph in-context lcarning demands three fundamental criteria that remain elusive in existing methods. First, post-training-free inference essential for genuine ICI. where models must adapt to new tasks through demonstrations alone, without fine-tuning or learnable prompt engineering. Second, cross-domain alignment enables single model to reason across diverse graph types within a unified semantic space, mirroring LLMs' domain-agnostie capabilities. Third, modality-frie operation ensures that the model can process arbitrary pre-encoded graphs without requesting data, crucial for the heterogeneous domains of real-world graphs. As shown in Table I prior approaches fall short of meeting all three criteria at once: self-supervised GNNs (You al. 2020 Qiu al. 2020) and GFMs like All in One (Sun al. 2023). GCOPE /Zhao al.

---

Published as a conference paper at ICLR 2026
2024), and GFT WWang et al. 2024b) compromise ICL through required post-training downstream graphs, where All in One trains task -psk-iiec prompts while GCOPE and GFT require fine-tuning; GPF (Fang al. 2023) and Prodigy (Huang et al., 2023) lack cross-domain alignment, limiting their generalization on graphs from unseen domains; recent advances like UniGraph (He et al.] 2025a). GraphAlign (Hou et al. 2024), and OFA (Liu al. 2024a) achieve alignment and post-training-
inference, yet sacrifice modality freedom by requiring conversion to a single unified modality (e.g., text-attributed graphs) for alignment, making them inapplicable when raw data are inaccessible or when graphs are already pre-encoded by domain-specifer pipelines. More related work is discussed in Appendix[A]
In this work, we present Modality-free Graph In-context Alignment (MF-GIA), the first GFM to achieve all three criteria for true in-context learning on graphs. Our key insight that the interaction between graph and shared frozen encoder reveals its domain characteristics, which can be captured by a gradient fingerprint: a single-step parameter update that encodes how features, labels, and structure jointly influence the model. This fingerprint drives lightweight domain-conditione transformations that align pre-encoded features and graph-local label IDs into unified semantic spaces, where related domains occupy neighboring subspaces while preserving intra-domain geometry, thereby achieving modality free domain alignment. The aligned features and labels are then processed by Dual Prompt-Aware Attention (DPAA) optimized with an episodic objective that learns match queries against support examples. This approach establishes prompt-based in-context reasoning that simulates the few-shot scenarios faced at test time. At inference, given a few labeled examples as prompts, MF-GIA computes the fingerprint, instantiates the aligners, and performs parameter- update-frre prediction on unseen domains. Experiments across diverse benchmarks demonstrate that MF-GIA excels at few-shot node-level tasks, generalizes to entirely unseen domains without additional training, and transfers seamlessly to edge-level tasks. These results bring GFMs closer to the universal in-context learning capabilities exhibited by LLMs.
2 PRELIMINARIES
Following the ICL setup of the pioneering work Prodigy (Huang et al. 2023), we study few-shot, prompt-based node and edge classification. In this section, formalize graph ICL as episodic classification over graphs and introduce a modality-free alignment perspective that standardizes features and labels across domains.
2.1 GRAPH IN-CONTEXT LEARNING
Let G = (G1, G2, .... G denote a collection of M graphs drawn from heterogeneous domains, where each graph G. (V,, Ei, X;, Y,) comprises node set V,, edge set E, C V, V., node features X,
_..II[] " RiK] and labels Y. The node features X; may exist in domain-specific formats (c.g., dense vectors, categorical attributes, IDs), with potentially different dimensions d, To pretrain a universal model across these graphs with a common input width, following (Yu et al. 2025 Zhao et al., 2024), we first unify feature dimensions to do by applying SVD on each X; < (V) E V. dd. Depending on the task, Y, is either node-label vector Y = 1, 1,....... C - 1 an edge-label vector y. (0, 1, Gr 1)12:1, where Crude and C. denote the number of node and edge classes in G,, respectively. universal GNN encoder fo dRd + d maps graph items (nodes edges) d-dimensional representations. For node classification, the representation of u E V4 is h fo fo; G;) E Rdd For edge classification, we analogously obtain h, fe(e; G,) for e E, using endpoint features and structure as needed. We use the generic symbol w to denote an item (w = or w e).
Given G as a pretraining corpus with M graphs and target graph Gnew (Vnew; Enew, Xnew. Y Koow) from an unseen domain with Cnew classes, graph in-context learning aims to classify graph items in Gnew using a few labeled examples per class as in-context demonstrations, without updating model parameters. Formally, the graph ICL operates in two phases. During pretraining, we learn unified model Ma containing l labeled graph items per class from Gnew as prompts, the model predicts labels for query items Q = {q q E Gnew \S5 as:
+ y on the corpus G. At test time, given support set S = {(w). My) by)
9n = -M (g, Gnew,,)) Vg Q
(1)
2

---

Published as a conference paper at ICLR 2026
Modality-free Alignment
Episodic Pre-training
In-Context Prediction
Capter
Socal
Z
Grav
Trainable
Ulrrsketed
+
Domain-aligned Features
62
C3
nnew
Modality-free Aligument
Domain Space
Domain-uligned
GNN
Labels
Domain-Connitionn Tronsformstions
Episode Learning
Prompt
Query
* GNN
Prompt
8 (Frence Mareu
m-way
Prediction
Unifled Space
shoe
(
Mely
Query
DPAA
9y
Miler
Zuew
The pretraining graphs are mapped to unified space via domain-conditiones transformations. Domain descriptors ensure similar domains occupy neighboring subspaces. (Middle) Episodic Pretraining: The model learns from vr-way k--hoot episodes using domain- features and labels. The DPAA mechanism matches queries to classes using only prompts as context. (Right) In-context Prediction: For an unseen graph, the frozen model performs few-shot classification using the support
Figure Overview of ME-GIA. (Left) Modality-free Alignment:
prompt.
where the pretrained model MM is parameterized by &. Crucially, & remains frozen during inference, so the model leverages the in-context demonstrations in S to adapt the domain without fine-tuning. For example, consider Me pretrained on citation and E-commerce networks. When tested on social network Gnew from unseen domain, the model fine-tuning. Instead, we provide support set containing a few labeled users from each class. By leveraging these in-context demonstrations as prompts, Mp identifies patterns between the support examples and query users to classify the remaining users, all while keeping its parameters frozen.
classify users in Gnew without
Episodic Meta Learning. paradigm (Vinyals et al. 2016 Li et al. (019). Specifically, for each pretraining graph G, E G, we construct m-way k-shot episodes by sampling m classes and examples per class as support set S, with additional samples queries Q. The model Mi. consumes SG,,S) as the prompt and is optimized to maximize the likelihood of the ground-trun truth labels on Q:
To enable in-context adaptation, we adopt an episodic training
min E +
Σ IEE
log (Vg S,
|Q]
This episodic formulation teaches the model to recognize patterns from limited examples. By pretraining on numerous episodes that simulate the few-shot scenarios encountered at test time, the model acquires the capacity to perform in-context reasoning. At inference, this enables adaptation to new domains through few-shot prompts alone, with all pretrained parameters remaining frozen.
2.2 MODALITY-FREE ALIGNMENT
The pretraining graphs in G often differ in both input modalities and label systems. Features range from dense vectors categorical attributes or domain-specific identifiers. Likewise, label spaces are graph-local and vary in cardinality, with no global alignment across domains. These heterogeneities make direct in-context transfer across graphs challenging.
Recent GFMs (Wang et al. 2024b He et al., 2025a Liu et al. 2024a) attempt to unify graphs from heterogeneous domains with Attribbut Graphs (TAGs), which convert all features and labels to natural language, then map them with language models into shared semantic spaces. However, this approach has fundamental limitations. Real-world graph data is typically already vectorized through domain-specific methods, such word2vee (Mikolov al. 2013) for documents, molecular
3

---

Published as a conference paper at ICLR 2026
fingerprints (Rogers & Hahn 2010) for compounds, and user behavior embeddings (Pan & Ding 2019). Converting these optimized representations to text and back introduces information loss and computational overhead. Furthermore, privacy constraints often restrict access to raw data, and data providers usually release only pre-encoded datasets, making modality-awart conversions infeasible in sensitive domains. Instead, adopt modality-free alignment perspective, which aligns graphs directly in their existing representations without modality-awar conversion.
Definition 1. (Modality-free Alignment) Let {(G;, L.)) be graphs from M domains, whose item features are already pre-encoded by (unknown) domain-specifie pipelines, X; E R.d,, and whose labels have been indexed by 14- = {0, ...
.C " -1.. }. modality-free alignment is a domain-
conditioned transformation system T - (xcfte XClube)) with:
i=1
K2teaa - TR. (feature alignment) and K. Tu - RRd (label alignment)
(3)
that maps domain-specifte features and label IDs directly into a unified d-dimensional feature space and label space, respectively, without reconstructing or converting to any intermediate modality. The transformations should be conditioned on the domain descriptors ei E IRde that capture domain characteristics of Gú, such that for any two domain i, j.
-Kcta x l|e; ezll and ||Cllbel Kcllbeel| x [lel -eyl-
This ensures that similar domains with close descriptors e; Rej produce similar transformations, causing their aligned features and labels to occupy neighboring subspaces in the unified space.
Fig. leet illustrates the idea intuitively. Modality-free alignment maps every graph into a unified semantic space according to domain relationships: graphs from related domains (e.g., two citation networks G₁ and G2) have similar domain descriptors and thus map to neighboring subspaces, whereas unrelated domains (social network G3) sit far away. The domain-conditioned transformations Kcfar and xclahel project each graph's ' pre-encoded features and indexed labels into unified feature and label spaces, preserving intra-domain semantics while enabling cross-domain transfer. This is essential because numerically similar feature vectors from different domains can carry entirely different meanings (each domain's encoder defines its own coordinate system), and indexed label IDs [0, 1, 2, ... are reused with domain-spectio differences by calibrating features and labels via the domain descriptor, unifying them in shared spaces without requiring any knowledge of the original data modality like TAGs.
semantics. Modality-fiee alignment reconciles these
3 MF-GIA: MODALITY-FREE GRAPH IN-CONTEXT ALIGNMENT
In this section, we present MF-GIA for enabling in-context learning across heterogeneous graph domains without modality-specifee priors. MF-GIA addresses the fundamental challenge of aligning graphs with incompatible feature spaces and label systems through three key components: (1) domain embedder encodes domain characteristics, (2) fomain-conditioned alignment maps pre-encoded features and indexed labels to unified spaces, and (3) episodic pretraining realizes few-shot adaptation during pretraining. We then describe in-context inference on graphs from unseen domains.
3.1 DOMAIN EMBEDDER
Reliable domain embeddings are the pivot of MF-GIA: they summarize graphs' domain characteristics, parame- terize the comain-conditioned aligners (Kffet, Klabel), and ensure that graphs with related domains are mapped to neighboring subspaces while preserving intra-/cross- do- main semantics. Prior work represents graph domains using learnable tokens, but depends on external signals, such as domain labels (Yu et al. 2025) modality metadata (He et al. 2025a). which are often unavailable in practice. We instead induce the domain embeddings [e directly from each graph's 's intrinsic properties by capturing the interactions between the graph and shared encoder, without external signals. Starting from a single shared weight initialization 6n E Id xd for a one-layer GNN encoder followed by fixed all-ones projection matrix Laxc, from embedding label space as shown
GNN
1axc:
G
one gradient step
Figure 2: Domain embedder.
4

---

Published as a conference paper at ICLR 2026
in Fig. take single gradient step each pretraining graph G; G as = = VoC, (0o). where 1) 18 a small learning rate uniformly set to 0.01 and L; is the task loss w.r.t. the available labels gradient fingerprint that
on G.. The resulting single-step displacement A0; 8; 0, serves captures how the graph's 's features, labels, and structure jointly influence the shared encoder. Intuitively, graphs with similar gradient patterns likely to come from related domains, making A0, a natural descriptor of domain-level information. To obtain compact domain embeddings, we project these fingerprints through learnable domain embedder feu IRd,xd Rd:;
C fes (A MLP (Flatten (Conv2D (A8:)) R
(5)
where the fingerprint A0; E [dd xd is treated as single-channel image to be embedded. The embedder f is trained preserve domain relationships by minimizing:
La Σ (IAA, - - - -eellleeeg - G.GGGE
(6)
so that pairwise relationships among graphs are retained in the embedding space. This approach naturally captures domain characteristies without domain labels modality priors, the gradient pattern inherently reflects the unique way each domain's 's data distribution interacts with the shared model initialization.
The domain embedding induced by the gradient fingerprint is central to MF-GIA, and the subsequent alignment operations established on To justify its effectiveness, we provide a theoretical analysis showing that this embedding faithfully preserves domain characteristics (Proof in Appendix B.I..
Theorem 3.1. Let G; and G3 be graphs sampled from domains DD and Dj respectively, with corresponding gradient fingerprints 20,, 20, E Rd.xd computed using task loss C, and Lj (e.g.. cross-entropy). The domain embedder feu produces domain embeddings e; and ez. Assuming every task loss L is the domain embeddings preserve domain relationships:
with respect to model parameters, and few has Lipschitz constant La.
||a -e--ella S (D),
where Wa(:s)) measures inherent distance between two domains, and C a constant.
This upper bound ensures that if two domains are inherently similar, their embeddings learned by fou. will be close in the embedding space, while dissimilar domains produce distant embeddings.
In-context Domain Embedding. For downstream graph Gnew from unseen domain. compute its domain embedding using the same fingerprinting process. Given a few labeled items = {w,, 9:) } as a -wew-way k-shot prompt from Gnew- we perform a single gradient step from the same initialization 06. which is a component of the pretraining model M (0, E +), as Onew On on S), where Lnew is computed using only the prompt S. The in-context domain embedding of Gnew then computed by passing the gradient fingerprint through the pretrained domain embedder:
Enew Sou (Onew 0o)
(8)
This process automatically captures Gnew's 's characteristics and positions within the learned domain space. Since the domain embedder for has been trained to preserve domain relationships during pretraining, it naturally maps the new graph's 's fingerprint to an appropriate location based on the knowledge learned from existing domains.
3.2 DOMAIN-CONDITIONE ALIGNMENT
With the domain embedding e; for G, E 9, MF-GIA instantiates two lightweight transformations (KCeat, K.aael)) aligners that respectively align G,'s item features and graph-local label IDs into unified semantic spaces. Because the transformations are conditioned on e, related domains with nearby e; induce similar transformations and occupy neighboring subspaces after alignment, while dissimilar domains remain separated. Here, we detail the feature and label alignment mechanisms and their application during in-context inference.
5

---

Published as a conference paper at ICLR 2026
3.2.1 FEATURE ALIGNMENT
For each pretraining graph G3 we learn a domain- feature transformation IC mapping pre-encoded item features to a unified feature space. Given an item w E G; with its feature Iw E R we first obtain its base representation via shared GNN encoder fo. whose first-layer weight matrix is initialized from the stored 0g:
h = fo(w, G4) R
Then we apply Feature-wise Linear Modulation (FiLM) (Perez et al. 2018) to generate domain- conditioned transformations from the domain embedding:
(7: feat (ofoat) Semm (ei);
feat ffoat
E Rd
Ti
(10)
2,,w = K./cal (hi,w) = Tite hi + B!
where Sorn RRd. - Red is two-layer MLP that outputs scale T (with SoftPlus head for positivity) and shift (8/ea parameters, @ denotes element-wist product, and 2i,0 is the aligned feature for w in G, The FiLM-based transformation X/(ea is affine, so it calibrates scales and offset across domains to map features to domain- sppcificcp subspace determined by e,, while preserving the intra- domain geometry already present in hi Formally, the alignment satisfies (Proof in Appendix B.2):
Property 1. If dnn is L Lipschittz continuous, then Il% -o --,7 ll2 + 110 - -ff l|2 S Llle: - e,ll2 for two graph G; and G5, so nearby domains yield similar feature transforms and thus neighboring subspaces in the unified feature space.
The domain- conditond transformations {K! Mi=1 parameterized by shared Oteat are learned jointly over all pre-training graphs G using their domain embeddings fer Mr form unified and general feature space. Together with the GNN encoder fe. they constitute part of the pretrained model M (i.e.,Q 0, Oteat 4)
In-context Feature Alignment. At test time, for downstream graph Gnew with domain embed- ding Cnew computed via the pretrained domain embedder, we generate its alignment parameters (Teew new
S (Pnow). For any item Gnow. its aligned feature is computed as:
Znow au
feat Tnw
fo Gnew) + gfeat
(11)
3.2.2 LABEL ALIGNMENT
Domarn-gneste glatel
Demaitsspecitic Label Dianbutions
M
Different graphs have their own label systems, leading to label IDs across domains having in- consistent semantics, such as label ID rep- resenting different concepts across graphs. To reconcile graph-local label systems, we main- tain a shared label base Elabel IR TmLxxx with Lmax = max; C₁, where each row Blahel serves as a Nomain-agnostic label prototype for ID I, initial- ized as Elabel ~ (0, Ia). Given domain embedding e; of G,, we instantiate a domain-conditioned label transformation with FiLM, architecturally identical to the feature-side transformation, that maps to scale and shift parameters for label alignment:
Figure Domain-conditioned label alignment.
(7rabee , /alabel) Sonna (ei), ,
Jabel galabll RRd
(12)
11,11 K Kalbell (Ex)
Jabel
Elabel + glabel 1 T, (0, , 1),
7
where ferm RRd + 2d is a two-layer MLP with SoftPlus for Label, u4,7 is the aligned label embeddings conditioned on for label I. As illustrated in Fig. this mechanism transforms single comain-agnoste distribution into domain-specific label distributions. The shared base Elabel provides common reference, while FiLM parameters shift and scale these prototypes based domain characteristics, ensuring semantically distinct labels occupy different subspaces in a unified label space even when sharing the same ID. few is component of the pretrained model M (olaee E b)

---

Published as a conference paper at ICLR 2026
In-context Label Alignment. For graph Gnew with Cnew classes, compute label align- ments using the pretrained transformation (Tnew for Gnw are:
libel
glabel) new
= fehma (Cnow) The aligned label embeddings
Unew /
Vnew
label
E
+
new
LE Cnew -
(13)
which yields domain-aware label prototypes that compatible with the unified feature space and ready for shw-shot matching.
3.3 EPISODIC PRETRAINING
MF-GIA is pretrained with an episodic, prompt-based objective that teaches the model to match aligned item features aligned label prototypes, mimicking the few-shot scenarios encountered during inference.
For each pretraining graph G; E 9, construct m-way k-shot episodes simulate in-context learning scenarios. Specifically, in each episode, we select m classes and sample k labeled items
per class form a support set S U { {wo)) ((e) } أ-ر where w/)) is the j-th item of the
e-th selected class and ((e) Vj is its label ID. We also sample T items per class for the query set
T
Using the domain embedding we compute aligned item features with Eq. (10) and aligned label prototypes with Eq. (12) yielding F,,0f0 KC!-a (Mrnugo): ). 21.0.
Q U c= (())
1=1
Kffeat
11 (1aa
and ui,ree Xclabel (Elett)). The prompt-query pairs become:
Prompt S {< u.
Query Q { {z cc(mm].eeiT]"
(14)
cE(m)
Recalling the episodic meta learning objective in Eq. which requires matching queries classes using only the prompt, we propose a Dual Prompt-Aware Attention (DPAA) mechanism. allows queries to attend to prompt examples but prevents prompts from interacting with each other, strictly
following the principle of in-context learning. Specifically, let 7mm = ( (,,,n!m)) (m))
R be the matrix of row-stacked support features and Upmr [u,, [u, Rmxx be the label prototype matrix. DPAA consists of two single-query attention layers, one feature-side and one label-side, both sharing the same projection matrices W K, Wy pdxd For aligned query feature Zi.q E Q from G; the feature-side attention computes:
Kteal
Zpint W K:
Vieat
Zpmnt W V:
Zigg We
feal (K feat
(15)
oul 21.0
softmax
Veaa
Vd
where the attended representation "i.g aggregates features from prompt examples relevant to the query. In other words, Eq. (15) aims to use the support features Zomt to prompt the query feature zi.g- out for the query. zi.g producing the prompt-conditioned feature ²i.q learnable function fa Rd + Rd, which is also prompted by the support set. Thus, the label-side
out
out is then projected label space via
attention lets the query interact with label prototypes:
K label
Upmt WK,
label
-puut WV, Wv,
label fa((za))
Qlabel(( Klabely T
(16)
softmax
Vilabe!
Analogous to LLMs, where prompt examples guide task completion, here (Zumt, Upmit serves as the few-shot demonstrations, ži,q as the query to be answered, and 2 as the prompt-conditioned intermediate, and w as the answer produced from the prompt for the task objective 21.q- Thus, the pretraining objective is to build the matching between u and the ground-trute label. The final prediction is obtained by scoring the query's 's prompted representation against all label prototypes:
8ig9 out (Upmit) T E m
(17)
7

---

Published as a conference paper at ICLR 2026
where $1,g contains the per-class scores for the query item 9. For each episode from G3, minimize the cross-entropy loss over all queries in Q:
T
exp
eepisade (Gi)
log
(18)
m.
Σド= exp 1
1=1
denotes the score where t > 0 a temperature that controls the sharpness of the softmax, 2,
of the ground-trute -ruth class for the query q: (c) all pretraining graphs:
The complete pretraining loss aggregates episodes across
peecrrin Peg.snBeeiisse----- (episode (G.))
(19)
Note that the domain embedder fox, is optimized with Lde prior episodic pretraining and then kept fixed. Overall, the pretraining model Ms comprises the frozen encoder initialization fog= the domain embedder fou the domain-conddtioned transformation fer. and fōrea the DPAA projection matrices and the projection head fn. This episodic regime trains the model to leverage prompt examples for prediction, establishing the feature-label matching capability essential for ICL on unseen domains.
3.4 IN-CONTEXT PREDICTION ON UNSEEN DOMAINS
At test time, ME-GIA freezes all pretrained parameters. Given unseen graph Gnew together with
Cnew -we k-shot support set S {(w;, )=)) as prompts, we compute the in-context domain embedding Enew using the gradient fingerprint and the pretrained domain embedder described in Section 3.1 With the pretrained lomain-conditioned transformations, any item w Gnew is mapped
to its aligned feature Znow,w w via Eq. (II.. For label IDs [ E Lnew = {0, (0..... Caew - 1}. the aligned
label prototypes are [Unow / 1e Lan obtained by Eq. 13p. We then form the prompt matrices Zhen
(kC and U [Mnee [ (.,,.............. Nnew [[Cro.)]
E B To
neww,n! mo! new, u (Cres)
make a prediction on a query item q, we apply the pretrained DPAA on it. Specifically, the feature-
side attention produces the prompt-conditioned feature "new.q by Eq. 15), which is then fed to the
label-side attention Eq. (16) to yield uuu
The final scores and prediction are:
new.gq
(Upmnt) E R.CC,
Spew.q "now
(20)
0g
arg max jE [Ceo
Snew.q [i].
This inference procedure is parameter-update-free w.r.t. the pretrained model M, and the same pipeline applies node or edge items by letting w. range over Vnew or F Frew. The detailed algorithms and complexity analysis of MF-GIA are provided in Appendix
To fully exploit the sparse few-shot support labels and the topology of Gnew during inference, we further enhance prototype construction and prediction refinement beyond DPAA. Specifically, we construct graph-aware class prototypes by propagating soft label distributions initialized from the support set through the graph structure using label propagation, which enriches the prototypes with neighborhood context to compensate for the sparsity of the k-shot support. For query classification, we employ an adaptive distance metric that combines cosine similarity and inverse Fuclidean distance, weighted per query by (aa((=ew.g)), enabling the metric adapt local feature geometry. Finally, predictions are iteratively refined through semi-supervised label propagation, where query pseudo- labels are updated by blending propagated and current distributions while keeping support labels fixed, enforcing graph-level consistency in the final output.
4 EXPERIMENTS
4.1
EXPERIMENTAL SETUP
We employ cross- cross-domain graph datasets to evaluate MF-GIA, which is pretrained exclusively on node classification tasks using four datasets: WikiCS (web link), PubMed and ogbn-Arxiv (citation), and Amazon Amazon--atings (e-commerce rating). These datasets are pre-encoded with heterogeneous feature spaces and label systems, enabling to learn domain alignment without modality priors.
8

---

Published as a conference paper at ICLR 2026
Table 2: Few-shot node classification accuracy (%) with standard deviation over 10 runs, Best and second-best results are shown in bold and underlined. "_" denotes datasets where only encoded features and indexed labels are available, making modality dependet models inapplicable.
way
agte Pradacs-- way
Cherpaans way
Plynced way
Bleg way
Metaod
T-cber
T-dver
Tochee
Trdve
5-4vi
T-cher
5-dve
T-ther
S-dee
T-tbet
Tolkee
Subee
GKN
42.38 42.38
42.
8.7
39 39.50
40 MM
15.4
15.28am
72.15
43.22g0
GAT
46. 12
4377
7:00
8.50 8.5
37.33544
16.93aa
18.2244
46.27amm
52.47a40
56 M042
GrapeisALa
41.30uu
7.30
K.9Pau
1/4
38.89
40.56.m
71,95u
77.36
CradMME
42:41mm
43:56
44.2-am
8,56
9,37
40.80am
42 42.02mm
43.55mm
18,3
90.35
43.5
S1.3
02 14aae
DGG
1010)..
19.97-4
75 76.00-
960kur
Crapert.
77.00
EIXHI:
4m30
46:4
11
18.75
11
45.0
47.46-1m
14.0
6200
43.8
11:3
12
430
Allia
42.6pcm
46.0
56.22am
CET
41 10uuu
43.3am
13.5
11
1405.r0
15
47.3k
41.0
Proday
40.3
4222-m
44.2
9,5
10
13,46
40.20
41 (Boua
45 82
61.26am
11.58aa 30.5 M0
53.46
DES
35
31.00mh
14
7:98
Criphslge
MF-GIA
47.6
97.38
63.0..m
16.6ase
19.66.1m
32
41.89
46 Willu
53.7
88 Mauce
6266
67.3a
Table 3: Few-shot edge classification accuracy (%) with standard deviation over 20 episodes.
Eacyshood
Leaina BG
Leand RG
S-alet
Lolos
3-daa
5-abet
Lohat
S-aao
3olst
GCN
82 25.5
34.00uc
15.12am
N0.5aam
26.04.2
55.7amm
56 Hamm
60.05.m
33. 4azm
42.57am
3405as
36.33m
31 10a0
CepeSAGE
85
8505
1600.am
71 1.Ma
75
19.45cc
56 Mluu
Blur
60.72.m0
13,05am
50 3041as
34,35m
29/22s
DGi
3167.09
76 16
80.17.01
9695um
61:31
11.1
GrephMAE
$4.00voo
95.12ow
18/8-am
73,35-mm
79.02.mm
$3
97.4
59.66cc
61/82-94
35.42420
41.1641
44 44.5+
16.3
30.7400s
32 22.5-1pp
Poodgy
87 50um
15.5
64.90
St S.30ur
99 58
62.0.am
4637as
47.88 77.8um
38.6amm
34.5
GFT
87 67
3500..
16.77ax
79.15am
61 11.88..
61.12 61.12am
68 6.53
35.33mm
15.0
CraphAbgg
$4922.0
76 28.14:20
54
59:38-49
47.4
19
21,90
3055
MI-GIA
99.42.40
39/04-00
37.57-40
91.34-10
$1.38-4x
18st
62.13-mm
58.64.a00
64.54ccc
08.06.um
38.37am
33.124x
For downstream evaluation, we test on both node-level and edge-level classification tasks across seen and unseen domains. For node classification, we evaluate on Cora (citation), and unseen domains including oron-crod and Computers (e-commerce product), Physics (co-authorship), and BlogCatalog (social media). For link classification, we assess performance on two knowledge graphs (KGs) from different domains predict the relation types: FB15K237 (encyclopedic) and WNI8RR (lexical), which represent entirely new tasks not encountered in pretraining. To unify the task formulation, we transform the edge-level task into the node-level task by converting edges to nodes in line graph, as detailed in Appendix D This task allows us to evaluate our model's generalization capability on an entirely new task and domains not seen during pretraining. More information about baseline configurations, datasets, and implementation details is provided in Appendix B
4.2 IN-CONTEXT LEARNING RESULTS
Table demonstratss that MF-GIA achieves state-of-the of-the-art node classification results across diverse graph domains. Remarkably, MF-GIA reaches 63. 63.98% on Cora with 5-shot prompting, which is an 1 11.34% absolute improvement over the second-best baseline. Across all 15 configurations, MF-GIA consistently outperforms existing methods with an average margin of 4.2%, despite using pure prompt- based inference without any parameter updates. In contrast, methods like GFT and AutoGFM require extensive fine-tuning on target domains yet still achieve inferior results. This superiority reveals a fundamental insight: when equipped with proper cross-domain alignment, ICL beats fine-tuning, The critical role of alignment is also evident when comparing MF-GIA with Prodigy, which is a true ICL model without domain alignment. MF-GIA consistently outperforms Prodigy unseen domains, demonstrating that domain embeddings capture domain characteristics for successful cross-domain transfer. Recent modality-dependder GFMs fail on graphs without raw text data (marked "_"), while MF-GIA operates universally on any pre-encoded graphs. Moreover, as shown in Table 3 MF-GIA excels at edge-level tasks, an entirely new task formulation never encountered during pretraining. It demonstrates that MF-GIA captures generalizable patterns for in-context reasoning rather than memorizing dataset/task specific features. On WN18RR dataset with a 10-way setting, our MF-GIA does not surpass the state-of-the-ee baselines GFT and AutoGFM, achieving third-best performance across all shot settings. It is because MF-GIA is pretrained exclusively on node classification tasks, while WNI RNR is a dataset for edge-level tasks, which is entirely different task formulation never encountered during pretraining. We deliberately evaluate on this dataset to assess our model's s generalization capacity to unseen tasks, we believe genuine graph foundation model should generalize not only to unseen domains but also to unseen task types. While GFT and AutoGFM achieve superior performance on WN18RR- 10way, they are pretrained on both node-level and

---

Published as a conference paper at ICLR 2026
Table 4: Effect of core components.
Method
Cora
Computers
Physies
71-
1-shot
5-shol
1-shol
5-shot
I-shot
5-shol
GraphSAGE#FT
40.50
42.40
35.89
40.58
67.12
77.36
Feal. Align.
42.78
45.26
37.86
45.97
75.83
85.29
Label Alig.
43.96
49.16
37.93
47.00
76.54
87.66
DPAA eepised:
47.64
63.98
41.49
53.71
79.12
88.92
Figure 4: Backbone selections.
edge-level tasks. Therefore, edge classification is not an unseen task for these baselines, so their performance advantage does not necessarily demonstrate stronger cross-task generalization.
4.3 MODEL ANALYSIS
Effect of Core Components. We analyze the contribution of each component in MF-GIA, starting from its GraphSAGE backbone. GraphSAGE+FT is pretrained the datasets and fine-tuned on support sets of test graphs. Adding domain embedder with FiLM-based feature alignment (+Feat. Align.) improves cross-domain adaptability. Extending alignment the label space (++Label Align.) further boosts performance by unifying class indices across graphs. Finally, incorporating DPAA with an episodic objective yields the full MF-GIA, which achieves the largest gains across datasets and shots. Table @ shows clear step-wise improvement, underscoring that both domain- conditioned alignment and prompt-aware reasoning are crucial for effective graph ICI.
Effect of Episodic Inference. ICL can be achieved through two paradigms: episodic meta beta-learning, which unifies pretraining and inference by training the model to perform inference episodes (MF- GIA and Prodigy), and supervised pretraining with test-time proto- type construction, where class prototypes are built from support sets and queries are classified by proximity (GraphAlign). As shown in Table 5 episodic inference (MF-GIA) consistently outperforms the supervised variant (MF-GIAsup).
Table 5: Effect of ICL scheme.
Computers
Physies
ME-GIAup MF-GIA
38.73
75.26 79.12
41.49
Effect of Backbone GNNs. In MF-GIA, we adopt GraphSAGE as the default backbone. Fig. shows MF-GIA exhibits minor accuracy fluctuations across different GNN backbones under 1-shot settings, demonstrating that MF-GIA is robust to backbone selections. More analytical results are provided in Appendix(F
5 CONCLUSION
We introduced MF-GIA, a pretraining framework for graph neural networks that enables in-context learning across heterogeneous domains without relying modality assumptions. By capturing do- main characteristics via gradient fingerprints and aligning pre-encoded features and graph-local labels through
transformations, MF-GIA supports parameter-update- update-free adaptation from few-shot prompts. This design overcomes key limitations of existing GFMs by removing the need for post-training fine-tuning and modality- conversions. Experiments demonstrate strong performance on both seen and unseen domains, with seamless transfer to new tasks.
Future Directions. Beyond our current design, MF-GIA opens up several promising avenues for future work. One direction is to couple our gradient fingerprints with LLMs to generate semantic domain descriptions, enabling human-interpretable summaries of latent domain characteristics and more transparent cross-domain reasoning. Another is to leverage these fingerprints to automatically discover latent domain structure from large unlabeled graph collections, moving from manually curated domains to data-driven domain decomposition. We believe these extensions will further enhance the interpretability, automation, and scalability of modality-free graph foundation models.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This research is supported by the National Research Foundation, Singapore under its Frontier CRP Grant TIKF-F-CRP-2224-0005
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the National Research Foundation, Singapore.
REFERENCES
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Haibo Chen, Xin Wang, Zeyang Zhang, Haoyang Li, Ling Feng, and Wenwu Zhu. AutoGEM: Automated graph foundation model with adaptive architecture customization. In Forty-second International Conference on Machine Learning, 2025. URL
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, al. Exploring the potential of large language models (Ims)) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25(2):42-61, 2024a.
Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, et al. Text-space graph foundation models: Comprehensive benchmarks and new insights. Advances in Neural Information Processing Systems, 37:7464-7492. 2024b.
Yuanning Cui, Zequn Sun, and Wei Hu. A prompt-based knowledge graph foundation model for universal in-context reasoning. Advances in Neural Information Processing 3ystems,,7:7005551 37:7095-7124, 2024.
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. Advances in Neural Information Processing Systems, 36:52464-52489, 2023.
Yi Fang, Dongzhe Fan, Sirui Ding, Ninghao Liu, and Qiaoyu Tan. Uniglm: Training one unified language model for text-attributed graphs embedding. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, pp. 973-981, 2025.
Tao Feng, Yexin Wu, Guanyu Lin, and Jiaxuan You. Graph world model. In Forty-second Interna- lional Conference on Machine Learning, 2025. URL /RR./https://opeeevieee wwtpev/wwwwweeeff.memee.mmm
id=xjTrT] Bbrc
Chelsea Finn, Pieter Abbeel, and Sergey Levine Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135. PMLR, 2017.
Mikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang. and Zhaocheng Zhu. Towards foundation models for knowledge graph reasoning. In The Twelfth International Conference Learning Representations, 2024. URL http:/:://////evvi/eeeeee
net/ forommid:=jWoooooO 9
Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What transformers learn in-context? a case study of simple function classes. Advances in neural information processing systems, 35:30583-30595 2022.
11

---

Published as a conference paper at ICLR 2026
Qi Guo. Leiyu Wang. Yidong Wang. Wei Ye, and Shikun Zhang. What makes good order of examples in in-context learning. In Findings of the Association for Computational Linguistics: ACL pp. 14892-14904, 2024.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Yufei He, Yuan Sui, Xiaoxin He, and Bryan Hooi. Unigraph: Learning a unified cross-domain foundation model for text text-attributed graphs. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. I, pp. 148-459, 2025a.
Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, and Bryan Hooi. Unigraph2: Learning unified embedding space to bind multimodal graphs. In Proceedings of the ACM on Web Conference 2025, pp. 1159-1770707 2025b.
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: supf-supe masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pp. 604, 2022.
Zhenyu Hou, Haozhan Li, Yukuo Cen, Jie Tang, and Yuxiao Dong. Graphalign: Pretraining one graph neural network on multiple graphs via feature alignment. arXiv preprint arXiv:2406.02953, 2024.
Weihua Hu, Matthias Fey, Marinka Zitmik, Yuxiao Dong. Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning graphs. Advances in neural information processing systems, 33:22118 18-2233,,, 2020.
Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. OGB- LSC: A large-scale challenge for machine learning on graphs. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL
net/ http:://pppnereeeeneneooooooooee
Qian Huang, Hongyu Ren, Peng Chen, Gregor Kržmanc, Daniel Zeng, Percy S Liang, and Jure Leskovec. Prodigy: Enabling in-context learning over graphs. Advances in Neural Information Processing Systems, 16166022-16317, 2023.
Xingyue Huang, Pablo Barcelo, Michael M. Bronstein, Ismail Ilkan Ceylan, Mikhail Galkin, Juan L Reutter, and Miguel Romero Orth. How expressive knowledge graph founda- tion models? In Forty-second International Conference on Machine Learning, 2025. URL https:// //tprevv//peereveee net forum Pid=mEddUeLtaa
Pengfei Jiao, Jialong Ni, Di Jin, Xuan Guo, Huan Liu, Hongjiang Chen, and Yanxian Bi. Hgmp: Heterogeneous graph multi-task prompt learning. arXiv preprini arXiv:2507.07405. 2025.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. networs In International Conference Learning Representations, 2017. URL httppppp////ppeeeeeeeee
net (t-
Lecheng Kong, Jiarui Feng, Hao Liu, Chengsong Huang, Jiaxin Huang, Yixin Chen, and Muhan Zhang. GOFA: A generative one-for-all model for joint graph language modeling. In The Thirleenth International Conference Learning Representations, 2025.
Jure Leskovec and Rok Sosič. Snap: A general network analysis and graph-mining library. ACM Transactions on Intelligent Systems and Technology (TIST), 8(1):1-20, 2016.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficien prompt tuning. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2021.
Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, and Timothy M Hospedales. Episodic training for domain generalization. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1446-1455, 2019.
12

---

Published as a conference paper at ICLR 2026
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In The Twelfth International Conference Learning Representations, 2024a.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, arXiv preprint arXiv: / 1692, 2019.
Yixin Liu, Shiyuan Li, Yu Zheng. Qingfeng Chen, Chengqi Zhang, and Shirui Pan. Are: A generalist graph anomaly detector with in-context learning. Advances in Neural Information Processing Systems, 77:5077--50804 2024b.
Zemin Liu, Xingtong Yu. Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In Proceedings of the ACM web conference 2023, pp. 417-428, 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05100 2017.
Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Frozen pretrained transformers as universal computation engines. In Proceedings of the AAAI conference on artificial intelligence. volume 36, pp. 1628-7636, 2022.
Man Luo, Xin Xu, Yue Liu, Panupong Pasupat, and Mchran Kazemi. In-context learning with retrieved demonstrations for language models: A survey. Transactions Machine Learn- ing Research, 2024. ISSN 2835-8856. URL NCPo8ZhQPa Survey Certification.
net. faaee//foruute forum? ?id=
Péter Mernyei and Cätälina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks. arXiv preprint arXiv:2007.02901, 2020.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representa- tions in vector space. arXiv preprint 3011
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. MctalCL: Learning to learn in context. In NAACL-HLT, 2022.
Shimei Pan and Tao Ding. Social media-based user embedding: A literature review. arXiv preprint arXiv:1907.00725, 2019.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with general conditioning layer. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018
Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of GNNs under heterophily: Are we really making progress? In The Eleventh International Conference on Learning Representations, 2023. URL https
net./ //
Chengwei Qin, Aston Zhang, Chen Chen, Anirudh Dagar, and Wenming Ye. In-context learning with iterative demonstration selection. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 7441-7455, 2024.
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. Gec: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2020.
Nils Reimers and Iryna Gurevych. Sentence-bert, Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLE) pp. 3982. Association for Computational Linguistics, 2019.
13

---

Published as a conference paper at ICLR 2026
Ruifeng Ren and Yong Liu. Towards understanding how transformers learn in-context through representation learning lens. Advances in Neural Information Processing Systems, 37:892-9333 2024.
David Rogers and Mathew Hahn. Extended-coomecting fingerprints. Journal of chemical information and modeling. 50(5):742-754. 2010.
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning retrieve prompts for in-context learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2655-2677, 2022.
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günneeaann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv: 1.05888 2018.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning, Advances in neural information processing systems, 30, 2017.
Xiangguo Sun, Hong Cheng, Jía Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2120-2131, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXhv:2307.09288 2023.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liō, and Yoshua Bengio. Graph Attention Networks. International Conference on Learning Representations, 2018.
URL
net. / forum d iKCZ
Petar Veličković, William Fedus, William L. Hamilton, Pictro Liò, Yoshua Bengio, and R Devon Hjelm. Deep Graph Infomax. In International Conference Learning Representations, 2019.
URL /tRp:ittpp::/opeeeeeveww net. http://pnneeeeeeee.eeeeeoooommme forrmmid==rii9iiii
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016.
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pp. 35151-35174. PMLR, 2023.
Kai Wang, Siqiang Luo, Caihua Shan, and Yifei Shen. Towards graph foundation models: Training on knowledge graphs enables transfcrability to general graphs. arXiv preprint 2xXi::2410.12699 2024a.
Zehong Wang. Zheyuan Zhang, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. Gft: Graph foundation model with transferable tree vocabulary. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024b.
Renchi Yang, Jieming Shi, Xiaokui Xiao, Yin Yang, Sourav S Bhowmick, and Juncheng Liu. Pane: scalable and effective attributed network embedding. The VLDB journal, 1277-1227 2023.
Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning. pp. 40-48. PMLR, 2016.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33: 5812-5823, 2020.
Xingtong Yu, Zechuan Gong, Chang Zhou, Yuan Fang. and Hui Zhang. Samgpt: Text-free graph foundation model for multi-domain pre- gre-training and cross-domain adaptation. In Proceedings of the ACM Web Conference 2025, pp. 1142-1153, 2025.
14

---

Published as a conference paper at ICLR 2026
Haonan Yuan, Qingyun Sun, Junhua Shi, Xingcheng Fu, Bryan Hooi, Jianxin Li, and Philip S. Yu. How much can transfer? BRIDGE: Bounded multi-domain graph foundation model with generalization guarantees. In socty- International Conference on Machine Learning, 2025. URL htw.hhttss:/pppnreeieeee net/ d=b JDKZ3Roax
Ruiqi Zhang, Spencer Frei, and Peter Bartlett Trained transformers learn linear models in-context. Journal of Machine Learning Research, 25(49):1-55, 2024.
Yiming Zhang, Shi Feng, and Chenhao Tan. Active example selection for in-context learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9134-9148, 2022.
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. In The Eleventh International Conference Learning Representations, 2023.
net,
8GF JUHKr
Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li. All in one and one for all: A simple yet effective method towards cross-domain graph pretraining. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4443 4454, 2024.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. International conference on machine learning, pp- 12697-12706. PMLR, 2021.
Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs. In Proceedings of the ACM Web Conference 2025, pp. 2183-2197, 2025.
15

---

Published as a conference paper at ICLR 2026
A RELATED WORK
In-context Learning. The modern paradigm of in-context learning (ICL) emerged with GPT- 3 (Brown et al. 2020). which demonstrated that large autoregressive transformers can adapt to new tasks using only a few labeled demonstrations, without any parameter updates. This breakthrough sparked extensive research along multiple dimensions. From theoretical perspective, subsequent work has clarified the fundamental mechanisms underlying ICL. Several studies establish connections between ICL and classical meta learning frameworks, drawing parallels to metric-based few-shot methods, including Matching Networks (Vinyals et al.) (0016), Prototypical Networks (Snell et al. 2017), and Model-Agnostic Meta Learning (Finn et 2017). More recent theoretical analyses reveal that transformers can implement gradient descent algorithms within their forward pass (Von Oswald
al. 2023: Ren & Liu 2024, Zhang et al., 2024), effectively learning to optimize in-context. Complementary work by (Garg et al. [022] demonstrates that transformers can learn entire function classes from context, providing an alternative computational perspective on ICL capabilities. On the methodological front, researchers have developed techniques to enhance ICL performance through improved prompt engineering. Min et al. (2022) introduce MetalCL, which explicitly trains models to perform in-context learning. Practical advances focus demonstration selection and ordering: retrieval- -aaed methods identify optimal examples (Rubin et al. 2022;] Luo et al.) 2024). while active selection strategies iteratively refine the demonstration set (Zhang et al. 2022 Qin al. 2024). The sensitivity of ICL to prompt construction has motivated calibration techniques (Zhao et al.) (021))
and continuous prompt optimization (Lester et al. [022]) with recent work revealing substantial impacts from demonstration ordering (Guo et al. (002)). However, extending ICL to graph-structured data presents unique challenges due to the heterogeneous nature of graph domains and the complex interplay between topology. features, and labels.
Graph Foundation Models. Graph Foundation Models (GFMs) have evolved from early self- supervised methods like GraphCL You et al. 2020) and GraphMAE (Hou et al. (022) toward
comprehensive cross-domain and cross-task generalization. This evolution follows three primary research directions. First, cross-domain unification is achieved by using text-attributed graphs (TAGs) as a universal modality. Specifically, OFA (Liu et al., (024a). GOFA (Kong et al.) 2025). and UniGraph (He al.. (025a) convert graphs to textual representations, then adopt LLMs and GNNs to learn semantic and structural information, respectively. UniGLM (Fang et al.) (025) trains unified language models over multiple TAGs, GraphCLIP (Zhu et al. (025) aligns graph summaries with language via contrastive learning, and (Wang et al. (2024b) introduces transfcrable tree vocabularies. AutoGFM (Chen et al., 2025) studies the automatically adapting architectures to different TAGs. In contrast, text-free methods avoid modality conversion. For example, SAMGPT Yuu et al. 2025)
employs learnable domain tokens for domain alignment, and GCOPE ZZaa et al.) 2024) connects disparate graphs with virtual nodes to enable cross-domain pretraining. Second, for prompt-based adaptation, GraphPrompt (Liu et al., 2023) and All in One (Sun et al. 2023) unify pretraining and downstream tasks via learnable prompts; Prodigy (Huang et al. 2023) enables graph ICL with prompt graphs and shows few-zero transfer to unseen graphs: ARC (Liu et al. (2024) achieves generalist anomaly detection via contextual cues. Knowledge graph models (Wang al. (0224a) have particularly benefited from this paradigm. For example, ULTRA (Galkin et al. 2024) learns universal relational representations, KG-ICL (Cui al. 2024) frames reasoning as prompting, and theoretical analysis (Huang et al., (0255) links expressivity to learned relation motifs. Third, for multimodal extensions, emerging work extends GFMs beyond single modalities. UniGraph2 (He et al. 2025b) unifies text and visual features with graph structure, while Graph World Model (Feng et al. 2025) integrates graph-structure states for planning and control. These advances establish a trajectory toward GFMs that function as universal encoders and reasoners across domains, tasks, and modalities.
Our ME-GIA framework is particularly suited to privacy- mrisayy-onntrriieed settings where schemas and modalities differ across organizations but raw features cannot be shared, such cross-organizationn fraud detection over transaction graphs from different banks and multi-hospital patient networks with heterogeneous EHR systems. In such scenarios, each party can locally encode its graph, while MF-GIA uses gradient fingerprints and lightweight domain aligners to align these heterogeneous domains without any modality assumptions, such as TAGs. This complements recent trends in GFMs toward realistic cross-domain deployment, including eext-based cross- cross-domain models (Chen et al. 2024b) and graph prompt optimization frameworks like HGMP (Jiao et al. 2025). all of which
16

---

Published as a conference paper at ICLR 2026
highlight the growing need for privacy-preservine preserving, modality agnostic foundation models in practical applications.
B PROOFS
B.1
PROOF OF THEOREM 3.1
The theorem shows that the domain embedder sox acts distance-preserving map from the domain space to the embedding space. To prove it, we first give a formal definition the graph domain and graph distance.
Definition (Graph Domain). A graph domain Dj is characterized by joint distribution P. Y) over graphs G (V, X) and labels Y, where the feature distribution P((X), label distribution PYY,), and structure distribution P. (VV) jointly define the domain characteristics.
Definition 3 (Graph Distance). For two graphs G; = (Vi, Es; Xi) and Gj = =V,, Xj) with normalized adjacency matrices A, and Aj. we define the graph distance as dG (Gi, Gj)
|| - 1 for comparison).
x IIAA; Ajllp, where assume Vi [V,] (padding with isolated nodes necessary
Then we define the domain distance used in Eq. 7 measure the inherent similarity between domains.
Definition (Domain Distance). The distance between two domains Dj and Dj is measured by the Wasserstein distance:
W2 (D., Dj)
inf kED(P..Ps)
(B(G,,G)) (,)(
(21)
where I (P,, denotes all joint distributions with marginals P. and Pj. dg(G,, Gj) is a graph distance metric that captures both feature and structural differences between graphs.
Before proving this theorem, we establish a technical lemma that characterizes the properties of gradient fingerprints.
Lemma B.1. For graph G = (V, X, Y) and the one-layer GNN encoder initialization d idd the gradient of the task loss at Og can be decomposed as:
1
VoL G)
X A V
diag(g) 1d.
(22)
where A is the normalized adjacency matrix. == 9vvl E V is vector of per-node loss gradients with gv V., (hws yu), is the node-level loss function, and la is the all-ones vector
gvv]
Proof. Consider the forward pass of a one-layer GNN with weight matrix 0 e RRoxdd
H (AXA),
(23)
where a is the nonlinear activation function (e.g., ReLU). The task loss over the graph is:
C(O, G) = V Σ e eh (hr,yu),
(24)
where hy is the representation of which is the v-th row of H, and is the node-level loss function (e.g., cross-entropy). Computing the gradient with respect to 0 via the chain rule:
VeL G)
M Σ
Voh Vn. (ho,yo).
(25)
For the gradient of h. w.r.t. 0, it can be represented as:
Vahu =
Σ
AvaI'
@ ((AX0).).
(26)
(EN(v)
17

---

Published as a conference paper at ICLR 2026
At initialization 8u, assuming ReLU activation with appropriate initialization ensuring positive pre-activations, we have o' ((AXtu),) = 1a. Therefore, the gradient can represented as:
VoL (00. G)
M
1
Σ
Σ
AvuTu
9u:
(27)
EEV
(EE((H)
where 9. Vb,l (hgs Yo) is the gradient of the loss with respect to node v's representation. This can be rewritten in matrix form as:
(0. 1
A diag(g) ld-
(28)
[V
For undirected graphs with symmetric normalized adjacency matrices, AT A, yielding:
1 [vi X .ddag(g)) la.
VoL (6u. G)
(29)
which completes the proof.
Building on the above definitions and lemmas, we present the proof of Theorem 3.1
Proof. Since the gradient fingerprint for G; sampled from domain Di, gradient fingerprint is defined
Ad = 0o VVCC (0o, Gi).
(30)
For two graphs G, and Gj from two domains, we have:
A0 A0,- = -1 (VeC, (fo, G,) (0o, Gz)).
(31)
Then the Frobenius norm can be represented as:
|A: ~0,
VV (0o, G₂) VoC, (0u, Gý)lp
(32)
Based on the Lemma B. and assuming VV Vyl (we can pad with isolated nodes if necessary) for simplicity, the gradient difference becomes:
VoL: (Bo. G.) - VoLj (6u. Gý) = [ A diag (g:) - - A, diag (g,) 1a:
(33)
where gi = [9.,1.912, I.m] E R" is the vector of per-node loss gradient with Ri.v V VA, (hi,r Vi,v) for node v in G,.. Let M, X,A, A, diag (gi) E Idd, and Mj
x] A, diag (g,) E Rd.xn taking into Eq. 33) and computing the Frobenius norm, we have:
o|V,, (0o, G₂) VoC, (0o, G = n ||M, - M2)) 1all2-
(34)
Since ||all2 = Vd, with the property of operator norm, the following inequality holds:
|| [M, M,) Lall2 |M, M, ||1all2 = ||M; Myllop Vd.
(35)
where || llop denotes the operator norm. To bound ||M, M, Ilop decompose it adding and subtracting intermediate terms
M, - M, = x A; diag(g:) - XJ A, diag(g,)
(X, X,) A; diag (g:) X. (A; Aj) diag(g:) + x A (diag(g,) diag(g,)) ))-
(36)
Taking the operator norm and using the triangle inequality Eq. 36 M, -M,llap with:
bound the
M - M, S (X, x,) ||. ||A:llap ||diag (g: llap
+ || ll (I Ay)ll ||diag (g:) (g:))lop
(37)
+ ||x !l || ||diag (g:) diag (g:)
18

---

Published as a conference paper at ICLR 2026
For diagonal matrix, the operator norm is the maximum absolute value of its diagonal entries. Therefore, for the diagonal matrix of the per-node gradient vector diag(g,) of graph Gú, its operator norm can be computed by ||diag (gx))log = maX 19t Based on the assumption in Theorem 3.1 that the task loss L is Lust-mmooth with respect model parameters, it means that its gradient is Lips-hips continuous. Let the task loss L be a cross-entropy loss with C, classes, the
exp
node-level loss on node v in G3 is E (hi Vv) = - logg Ee
The gradient on is
Ji.v V As.. A( We) Pv Ty.. where Pv softmax(h4,, encoding of the ground truth label of v. Therefore, the following inequality holds:
[0, 1] jc. and ry. is the one-hot
119..1122 llp Tv. 1l2 S lppll22 + ll l12 S svv++
(38)
Since we are considering Ii.v as a scalar after projection, we have:
19t,el S vG++=
(39)
Thus, (X: x,) op
we have |ddiag (g: (g:))lop Σ C.. Based on the nature of the operator norm, we have
|
X; op S |X; X5llø>
X 1pp | B (bounded by fea-
ture norms), and ||A:|lap. Ajllop I 1 where A is the normalized adjacency matrix, we get:
[M; - MM,-ll I E |X X + BC; A; Ayllo + B ||diag (g:) diag (g, Mlap (40)
For the last term of Eq. 440) since the task loss is -euu--smoott the node-level loss & inherits smoothness with Lipschitz gradient constant Lt. Given node e with the same index in G; and Gj- the gradient difference between gi and g, is ||diag(g:) - diag(g)) - max, 19
95.el- Since
gi.v V h (hi,v ) and the gradient is Lipschitz, we have:
11 - g1.0|| S Lellha, hg.ell-
(41)
23 the activation function of)) is typically ReLU, In the one-layer message passing defined in Eq. which is I-Lipschitz. Thus, we have:
[lhi - hg.0ol ll S II[(A,X, A X, )...1 S (AAXX - A,x,) .lll 110o1,
(42)
where the node e in A,X, - A, can be rewritten as (A,X,), - (A,X,), = En. ALuuZT
En A Asout j. which can be further bounded by:
IAA -AXX-AAXX A,X,) Moll = [I(A;
- XX,II $ (A, - A,). 1o + |I(A)) X, llop-
(43)
Since A is normalized where [I(A)).ll = 1 and thus (I(A)) ) l|2 S 1. Taking Eq. we can bound the difference between hipv and 11 as:
43
into Eq. (42).
| ا < ||6ol| (IICA, As)all B |X Xs1lap)
(44)
To get a uniform bound over all nodes, we have:
max | - ال S |1061| (I]A, - -,llA IX; X211z),
(45)
which gives us:
||daag (g:) diag (g) S 10oololl (B ||A
Asllo + +| - x1y1)))
(46)
Since Lt, On. and B are fixed, we can use L = Lt |00|| max( B, 1) to combine them. There, Eq. 46) can be rewritten as:
|diag (g:) - diag (g,) ll.p S L (IX - X,l |A, -1)1))
(47)
To measure the inherently distance between two graphs, we define the graph distance as dG =
X
X,1IF + ||A, - A,IIF as Definition Then, taking the graph distance and Eq. (47) into
Eq.
40 we have:
|M, M, ol S dg (Gi, Gz),
(48)
19

---

Published as a conference paper at ICLR 2026
where C B.L' Recall Eq. (32). Eq. (3 and Eq. (35 we can integrate the used
constants, such vd and !, into C. Then the difference between the gradient fingerprints of two domains is bounded as:
|00; A0 S n do (G,,G,) G2).
(49)
For the graph distance dg, we have:
dg GG) = X - X,رl + |A, A,ll $ v2v |x, - x, 1l |A, - A
(50)
The Wasserstein distance can be rewritten as:
W/ (D;, Dj) = inf
P3)))..P)) V Bica,o,)un (d. G,)
(51)
For the optimal coupling '² that achieves the Wasserstein distance, by Jensen's 's inequality for the concave square root function, we have:
B [dg (G,, Gj)] VBIGLGGS [dB (G;, G,)] Wa W2 (D)
(52)
Combining Eq. 52 and Eq. (9)) the bound of |A:; A0j1 P can be expressed as:
| A6, FF I n Eldg(Gi, G₁)] s n W)(D;,D)),
(53)
Eq. (53) shows that the similarity between gradient fingerprints can effectively reflect the domain similarity. Recall the domain embedder defined in Eq. ( we can conduct the Lipschitz analysis on First, the Conv2D with kernel weights W comy satisfy Conv2D(x) - Conv2D(y)) I ||W conv llop || w/l. the Flatten operation preserves norms ||Flatten( A) || =I Al[F, and the MLP
with R- R-layer weight matrices W1- W.... Wa satisfy || MLP(x)) (MLP(z)-MLPP MMLP(M) (v) S II |Wrllop - vll-
Thus, the overall Lipschitz constant of the domain embedder fo.. is:
Lae
W cony llap
II
W llp.
(54)
Thus, the following Lipschitz inequality holds:
le, e,l12
[l (AO:) - fae (ا0,)11, S Lae |00, - A0,1 p.
(55)
We can substitute Eq.
53) into Eq. 55) and merge all constants into Ö, we can get:
||e - eyl12 e < Ō (2,,,,
(56)
This shows that the domain embedding function preserves domain relationships: similar domains with small W2 map to nearby embeddings with small ||e; - cyll2-
B.2 PROOF OF PROPERTY
Proof. By definition, fer. being L -Lipsschitz means that for all e, el e Rd.
|1 (e) - Iéw (e)) S L lle e'1l2-
(57)
Applying this with e; and e' = ej for domain embeddings of G; and Gj. have:
fera (es) - Sérc (E;) = (7 leat (e:) 7 feal (e,) gleat (es) - gleal (e;)) = (Ar, A9).
(58)
Then using the definition of || 11,,1 gives:
||>l|2 || Il.frra (e:) - Sorm (ej) S - lle: Ezll2
(59)
which exactly the claimed inequality in Property transformation for its domain i on an item is K:feat I + Rd:
For the graph G,, the domain-conditiones
feat 7
4 h + 8f
(60)
20

---

Published as a conference paper at ICLR 2026
Then for any h RR, the following inequality holds:
||c] (h)
- (b)]l2 - 11(7% ma ?5 Jeal
+ (Bfcat Bl 11
< (1 feat T
Fent
3 1b112 + Sfrer
g 11
(61)
2 || ob |1
2/11 112 + ||
8
- |1lll2 ||
Fear
112 + ||ofoor
5 112-
Combining Eq. 59) and Eq. (61) yiclds:
|| (h) - ( max {11b112, le; -eyll2
(62)
Thus, as domains move closer in the embedding space, their induced feature transforms move closer uniformly on any set of bounded ||A/|2,, so the collections {K? (hi,w)) hn and (Xc) (hz occupy neighboring subspaces in the unified feature space.
C ALGORIHTMMS AND COMPLEXITY ANALYSIS
C.I ALGORITHMS
Algorithm I presents the episodic pretraining procedure for MF-GIA. Algorith@@deetail parameter- update-free in-context inference on unseen graphs. At a high level, pretraining teaches the model to (1) extract gradient-fingrprinn domain embedding from small support set, (2) map that embedding to dondinnconditioned feature and label transforms that place heterogeneous graphs in shared space, and (3) perform prompt-aware matching between queries and aligned supports. At test time, we reuse the same pipeline with frozen parameters, computing the domain embedding and transforms on-the-ff fly from a few labeled examples only.
C.2 COMPLEXITY ANALYSIS
Let G, (V,, E,, Xi, Yi) be pretraining graph with [V] nodes and |Eg| edges, GNN encoder width d, input width do, and domain embedding width de. Offline, the domain embedder com- putes a single gradient fingerprint per graph by one forward-backward through the shared J-layer GNN fo from On. which costs OJ[(Ea + [VD). d) per G;. Each fingerprint A0; E Rd.xd then embedded via Sew. giving O (d.dod) time. After caching 20, training Seu. with the pairwise metrie-preserving loss adds O de per epoch for M pretraining graphs. In each episode on G, with m-way k-shot support and T' queries per way, generating FiLM parameters for domain-conditioned transformations (1 sp(ca = for (e;) and label, (slubel) = f (e:) costs O (d,d), and applying Kleat to (mk + mT) item embeddings and X.label to m label rows of the shared base Elabel E RLaaxx Xd costs O(mkk: + mmT)d + +md). The time complexity of DPAA is O (mmk + m)d² 1d + m&T' (d3 + mkd + md)), which typically secondary the encoder when m and & are fhw-shot. Typically, time complexity can be represented as O (EM (LE + [V]]) d dod,e) which is linear in graph size.
have J.T. IIA < [Val, [E,, d, do de, therefore the overall
D TASK UNIFICATION
With a bit of notation abuse, in knowledge graphs FB15K237 and WNISRR, link classification aims predict the relation type r for given triple (h, r),,, where h and are head and tail entities, respectively. To leverage our node classification framework for this task, we transform the link classification problem into node classification through line graph construction. Given knowledge graph G = (V, E, R, X) with entities V. edges E, relation types R. and node feature matrix X, we construct a line graph LG(G)
(Vic, ÉLG, XLG) ). where each edge (h, t) E E becomes node v. E ViG. Two nodes Ce,, Uz, E VLG and are connected if the corresponding edges e,, Ej share a common entity (head or tail). Formally, the edge set ELG defined as:
FLG { (Vey Vey) :44 (hi, ri, (4), Eg (h2 rs. rg,4)) (2) (he 4)) (his()) # 0}
(63)
21

---

Published as a conference paper at ICLR 2026
Algorithm 1: MF-GIA Pretraining
Input Pretraining graphs G = {G, (V,, F., X, Y;) i==i unified item dim dg: embedding dim d; GNN encoder fo with stored initialization dg (kept frozen); base label table Elabel E IR famux xd, Domain embedder Sou: FiLM aligners fémm Sonaa DPAA params WK. Wv, We and head fo: Episode spec (m-way, k-shot, T queries), temperature т, learning rate
Output: Pretrained model Ma = {80: few few.fan fern: forma WK, Wv. WQ: fa).
(Optional) feature unification. For each X,, map to Rd via SVD.
// unify dims 11 gradient fingerprints
Stage A: Train domain embedder Seu
for i = ,,..., .M do
Compute one gradient step G, from 0o: 0, f - Store fingerprint Ad; + 0; - 8g.
end
repeat
e, fou for all i.
Lde + Ers (IAA; ~0 - |le eyl12)²
10 Update föu. by descending VLde.
11 until converged
Freeze San (and keep So at 0o).
Stage B: Episodic pretraining with DPAA (encoder init frozen).
14 for episodes do
15 Sample graph G3 and an m--ay k-shot support set S plus T queries per class the query set Q
e, - for (AO,); feat Sffeat) + fenn (g.label, glabel) + (e)m // Aligned features / labels
16
For any item w, h + foo (w, Gi), 21, t g hy. + 8fem
17
For each class I used in the episode, u4,1 + % label & Ellabel + glabel
18
Form Z E (mm)) from support {2t,cc} and Upmi E mm from {u,1).
19
// Dual Prompt-Award Attention (single-query attention)
29
for each query item with class do
Kleat Z W K, Vleat + Z Wv, Queat + Zigg We.
21
22
ouu "i
+ softmax
Khai
Va Vlabel
Vea
23
K label < Up
W
Upmit Wv, Qlabel + fr (2
L. < softmax
Vlabel
Val
25
81,9 t wigh (UPmt)
E mR
Lepisode
f-
log
exp( 0 (cl//) 2 exp( 11/T)
26
end
Update [/ S Elabel WK, Wv, W WQ, Sa) to minimize the mean query loss pretrain of the episode.
27
end
For each node v. VLG in the line graph corresponding to edge = (h, r, t), we construct features by aggregating the embeddings of the connected entities and relation as Zo. [[r]] where [-II-] denotes concatenation. The concatenated features are then projected to the unified dimension using PCA to maintain consistency with the node classification framework. After transformation, link classification becomes a node classification problem on the line graph, where each node (representing an edge in the original graph) needs be classified into of R relation types.
22

---

Published as a conference paper at ICLR 2026
Algorithm 2: MF-GIA Test-time In-context Inference (Parameter-Uipdate- w.r.t. M Input Frozen pretrained & from Algorithm !
unseen graph Gnew = (Vnew: Enew Xnew)
with Cnew classes;
-ay k-shot support set S {(0,, yj) 15)) C queries Q C Gnew S. [NEQ.
// In-context domain embedding (from support only) Compute one-step fingerprint from Oo on S: Onew t - (0o; S); Cuew t- few (One - -0))
// Domain-conditionee
alignment for Gnew (Thear (Ston) +- fénn (Cnew); (n (Sabat)
+ f (Cnew)
For any item w: know, S0g (w, Gnew); know.
Tuew
② Hnnwww uo + new
For l = O, .,.,, Cnew 1: unew,l
label Inew
Elabel + new
Form Zhne I (kCCuc ]xd from {Znew,u so (M.a))S and Uhmt Cuew from (Unew.
// Dual Trompt-Awase for each query q E Q do Kleat f
Attention inference (no parameter update)
Znew W K, Vieat
Wv, Q Znew,o WQ.
zout "new.g
softmax
RKaTT
V
va Vlahel U
K lahell new WK
new Wv. Qlabel + new.nn
10
2 Dnww,
f softmax
Vd
Vlabel
8q
new.y
UUmm T; Mg
arg max, 8 8g1i).
12
end
Table 6: Dataset statisties.
Usage
Dataset
Domain
Task
#Nodes
#Edges
# Classes
Pretrain
WikiCS
Web link
Node
11,701
216,123
10
PubMed
Citation
Node
19,717
44,338
3
ogbn-Arxiv
Citation
Node
169,343
1,166,243
40
Amazon-ratinne
E-commerce (Ratings)
Node
24,492
93,050
5
Evaluation
Cora
Citation
Node
11,701
216,123
ogbn-Products
E-commerce (Product Category)
Node
2,449,029
61,859,140
47
Computers
E-commerce (Product Category)
Node
13,752
491,722
10
Physics
Co-authorship
Node
34,493
495,924
5
BlogCatalog
Social Media
Node
5,196
343,486
6
FB F5155237
Encyclopedic KG
Link
14,541
310,116
237
WNI8RR
Lexical KG
Link
40,943
93,003
11
E EXPERIMENTAL DETAILS
E. DATASETS
We pretrain MF-GIA on four source graphs: WikiCS (Mernyei & Cangea 2020). PubMed (Yang et al.
2016).
ogbn-Arxiv (Hu al. [2200, and Amazon-ratings (Leskovec Sosič 2016 Platonov et
2023 The pretrained model is evaluated on five held-out graphs on node-level tasks: Cora Yang
al. 2016), oron-crs (Hu et al. 2020). Computers (Shchur et al. 2018), Physics (Shchur et al. 2018), and BlogCatalog (Yang et al. 2023). spanning citation, e-commerce, co-authorship. and social media domains. The pretrained model is also evaluated on edge-level tasks on FB15K237 (Bordes al. 2013) and WNISRR (Dettmers et al. 2018), which are knowledge graphs from encyclopedic and lexical domains to predict relation types. Dataset statistics are summarized in Table 6 Note Lebn-Producte and Computers
E-commerce networks, they form distinct domains: Amazon-ratings labeled by average user rating per item, whereas ogbn-Products and Computers use product-category labels. We therefore treat them separate domains.
that although Amazon-ratinge
23

---

Published as a conference paper at ICLR 2026
0.7
1.8
0.6
8 1.6
Epoch
(a) Training loss
(b) Validation accuracy.
Figure 5: Pretraining curves of MF-GIA.
E.2 BASBLINE CONFIGURATIONS
We compare MF-GIA against two categories of baselines: (1) Traditional GNNs: GCN (Kipf &
Welling 2017), GAT (Veličković et al. 2018), , and GraphSAGE (Hamilton et al. 2017); (2) Self-
supervised GNNs: GraphMAE (Hou et al. 2022). DGI (Veličković et al.) 2019), and GraphCL YYou
et al. 2020); (3) GFM with post-training: GCOPE (Zhao al. 2024). GFT (Wang et al. 2024b),
AutoGFM (Chen al. 2025), , GPF Fang al. 2023 and All One (Sun et al. 2023); (4) GFM
with ICL: Prodigy (Huang et al. 2023 OFA (Liu ct al. 2024a). and GraphAlign (Hou et al. 2024).
For traditional GNNs and Self-Supervised Methods, pretrain the same four datasets MF-GIA. Since these models lack in-context learning capabilities, we fine-tune them on the support set and evaluate the query set for each episode. For GFMs with post-training (no ICL), we consider two pretraining regimes and report the stronger results: (1) pretraining on the same four graphs as MF-GIA, and (2) pretraining on the datasets used by the methods' official implementations. For Prodigy, we compare two variants: one pretrained on our datasets and another on MAG240M (Hu models
al. )021) as the original work, reporting the better result. For modality (OFA, GraphAlign), which require text-attributed graphs (TAGs) and cannot operate on pre-encoded features, we use their original TAG datasets and implementations. To ensure fairness, all baselines follow the same episode protocol (identical m-way, k-shot support/query splits), use comparable backbones when applicable, and tune hyperparameters on validation episodes within the authors' recommended ranges.
E.3 IMPLEMENTATION
We evaluate MF-GIA under the few-shot learning paradigm without any fine-tuning. For each test graph, we randomly sample k-shot support sets, where = {1, 3, 5}. and evaluate on the remaining nodes. For node classification tasks, measure classification accuracy and report the mean performance across 10 independent trials, each with randomly sampled support/query splits to ensure robustness of our results. For edge classification tasks, focus relation type prediction and conduct evaluation over 20 episodes to account for variance in the few-shot sampling process.
For the dimension alignment, domain embedder uses a 2-layer CNN followed by a 1-layer MLP to project gradient fingerprints into 64-dimensional domain embeddings. For feature alignment, we also employ I-layer GNN encoder with hidden dimension 64, followed by FiLM-based transformations. The label alignment uses shared label base dimension I.max 64. The DPAA mechanism consists of attention layer with 1 head each. During pretraining, we use episodic meta learning with 10-way 5-shot tasks sampled from the pretraining graphs. We train for 10000 episodes using AdamW (Loshchilov & Hutter 2017) with learning rate 0.005 and weight decay 0.0005. For gradient fingerprint computation, we use fixed learning rate = 0.01 for single-step updates. Fig. 5 illlustraees the pretraining dynamics of MF-GIA. The training loss exhibits stable convergence, while validation accuracy shows consistent improvement before platcauing at approximately 6000 epochs, indicating effective model convergence.
SVD unify all graphs' feature dimensions d, 64. The
24

---

Published as a conference paper at ICLR 2026
Table Effect of pretraining dataset composition on few-shot node classification accuracy (%). We systematically vary domain coverage from single to full four-domain pretraining. Results are 5-shot accuracy averaged over 20 episodes. Best results are bold.
Configuration
Pretraining Datasets
Domain Coveruge
In-Domain
Out-of-Doomann
Avg OOD
Overall
Cora
Products
Physics
BlogCatalog
PubMed
Citation
42.76
12:43
68.15
41.28
40.62
40.91
Single
Araiy WIKCCS
Citation
48.31
13.86
76.42
43.67
44.65
45.57
Web
29.23
14.72
65,83
45.91
42.15
41.67
Amazon
E-commnerce
35:47
17.28
61.26
38.74
39,09
38.19
PuiMMod + Anri
Citaiin
52. 14
14.92
79.87
47.35
47.38
48.57
WikiS Amazon
Web E-commerze
41.68
18.35
67.42
48.26
44.68
43.93
PubMMdd WikIUS
Citation Web
Arxiv Amazon
R-commerce
36
19.14
78.53
46.72
PubsMee + Anxiv WikiCS
who E-commerce
56.47
16.82
83.74
51.38
51.65
$2.85
Poll Three
PubModd+ Arxiv Amazon
w/o Web
57.82
20.67
85.91
52.16
52.91
54.14
PubMMed WikiCS Amazon
w/o Citation (Arxiv)
77.62
51.56
Arxiv WikiCS Amazon
wlo Citation (PubMedd
59.21
21.38
86.73
60:47
56.19
56.96
All Four Ditasets
Complete
63.98
22.61
88.92
67.31
59.61
60.73
F MORE EXPERIMENTS
F.1 EFFECT OF DOMAIN DIVERSITY IN PRETRAINING
Table 7 demonstrates that domain diversity is critical for MF-GIA to construct an effective domain embedding space that enables robust alignment of downstream graphs. Single-dataset pretraining yields limited performance, where Arxiv shows the strongest individual results due to its diverse academic content, providing richer domain signals. When combining two datasets, cross-domain pairs (e.g.. Arxiv + Amazon: 48 48.69%) perform comparably to same-domain pairs (PubMed + Arxiv: 48.57%) despite Amazon's 's weaker individual performance, indicating that structural diversity helps the domain embedder learn more generalizable alignment patterns beyond surface-level similaritics. This effect amplifies with three datasets, where the configuration without PubMed (Arxiv WikiCS + Amazon) achieves 56.96% notably outperforming the configuration without Arxiv (51.56%% .56%), despite Arxiv's superior standalone performance. This counterintuitive result reveals that once sufficient domain signals are captured, maintaining diverse structural patterns (Web link, E-commerce) becomes more valuable than redundant citation networks for constructing a comprehensive domain space. The full four-datase model achieves optimal performance (60.73%) with remarkable generalization to entirely unseen domains like BlogCatalog (67.31%) validating that comprehensive domain coverage during pretraining enables the domain embedder to map novel graphs into appropriate subspaces of the learned domain space. These results confirm that MF-GIA's domain-conditioned transformations require diverse pretraining to establish a rich domain embedding space where graphs from any domain, seen or unseen, can be effectively mapped and aligned based on their intrinsic characteristics.
F.2 EFFECT OF GRAPH PRE-ENCODER
Table 8: Performance of MF-GIA on Cora with different feature encodings.
While the raw text data of the Cora dataset has been made available by Chen et al. (2024a), pop- ular end-to-end GNN models typically utilize the pre-encoded version from Yang et al. (2016). which employs bag-of-words (BoW) encoding for node features. -odaaity-dependent GFMs such as GFT, AutoGFM, and OFA require ac- cess to raw text data and rely on specific lan- guage models, such as Sentence Transformer
Pre-encoder
1-shot
3-shot
5-shot
BoW
47.64
57.38
63.98
ST
48.37
62.79
68.54
ROBERTa
48.24
61.53
69.85
LLaMa2-7B
47.93
58.64
65.33
(ST) (Reimers & Gurevych 2019), LLaMa2-
7B Touvron et al. (2023), RoBERTa (iu et al. 2019). to align the semantic space between the
Cora dataset and their pretraining graphs. In contrast, MF-GIA demonstrates modality freedom, i.e.,
it operates directly graphs with arbitrary pre-encoded features without requiring knowledge of the encoding method. Whether a graph has been encoded using ofg-of--wrdd: advanced language models,
25

---

Published as a conference paper at ICLR 2026
even unknown proprietary encoders, MF-GIA can seamlessly adapt to these features. The results in Table 2 employ the public bag-of-worde features for MF-GIA. To validate our modality-free claim, in Table 8 conduct additional experiments applying pretrained model (without re- re-pretraining) to the graph pre-encoded through different pipelines. The results show that MF-GIA maintains its effectiveness regardless of the underlying feature encoding, confirming its ability to generalize across diverse pre-processing methods and enabling practical deployment in scenarios where encoding
details are unknown heterogeneous.
F.3 DIMENSION UNIFICATION
For GFMs, feature dimensions unified before feeding data into the model because GFMs require inputs in a consistent format. We adopt SVD-based unification to achieve a fair comparison with baselines which isolates the contribution of our core innovation from preprocessing effects. Dedicated methods such as learnable pre-unifiers, which can be seamlessly integrated into our MF-GIA framework. Table shows that our framework remains effective and compatible with advanced dimension unification techniques.
Aligner (DIA) (Yuan et al. (025) provide more expressive and
Table 9: Performance on MF-GIA with expressive dimension unification component.
Computers
Physics
BlogCatalog
Method
1-shot
5-shot
I-shot
5-shot
1-shot
5-shot
ME-GIA
41.49
53.71
79. 12
88.92
49.46
67.31
MF-GIA (w. DIA)
41.96
54.60
80.15
89.74
48.37
66.53
F.4 MORE PRETRAINING TASKS
Table 10: Performance of MF-GIA with additional pretraining tasks (5-shot).
In this work, we pretrain model small number of datasets to showcase its ability to adapt diverse unseen domains and task types. The four datasets we use are among the widely adopted. publicly available, and easily accessi- ble graph benchmarks. This design ensures that anyone can reproduce or extend our pretraining process without the need for extensive dataset cura- tion, as common benchmarks already provide sufficient diversity to pretrain our model effectively. Here, we expand pretraining beyond node classification by adding link existence prediction and re-pretrain the model, denoted as MF-GIA (w. link), and report the inference results under the 5-shot setting in Table I0, It shows that a broader pretraining corpus yields consistent improvements on unseen domains.
ogba-Produce
Computers
FB FB15K237
ME-GIA MF-GIA link)
22.61
53.71
91.38
24.59
$6.60
92.36
F.5 STABILITY OF GRADIENT FINGERPRINTS
Table 11: Stability of gradient fingerprints.
The stability of the gradient fingerprints is dependent on the size of the support set. From Lemma B.I the gradient decomposes VoC (0,, G' reveals that the fingerprint aggregates informa-
Datasets
Avg. Cosine Similarity
Std. Dev.
Min. Similarity
Physics BlogCatalog
0.931
0.015
0.896
x A diag(g) 12, which
0.917
0.021
0,873
tion from domain specific components including feature distribution X. graph structure A and label distribution g. As the support set size increases, the law of large numbers ensures that these aggregated statistics converge to their population expectations, making the gradient fingerprints increasingly stable. Here, we examine the fingerprint stability under 5-shot setting. Take Physics and BlogCatalog datasets as examples, we randomly sample 20 different 5-shot support sets from all classes, and compute fingerprints AO for each support set and feed it into the domain embedder to get the domain embedding. Then we can measure the stability by computing the pairwise cosine similarity between all domain embedding pairs from the same graph. The results are shown in Table II where the high average cosine similarity and low standard deviation demonstrate that different support sets from the same domain produce highly consistent domain embeddings.
26

---

Published as a conference paper at ICLR 2026
Table 12: Effect of on different datasets (5-shot).
oghn-Products
Computers
FB15K237
0.05
21.45
52.10
91.82
0.2 (Default)
22.61
53.71
91.38
0.5
22.60
53.93
90.45
22.15
53.29
90.27
Table 13: Sensitivity of DPAA configurations
Computers
Physies
BlogCatalog
I-layer. 1-head (Shared)
53.71
88.92
67.31
llaaerr 1-head (Separated)
51.25
88.10
67.05
1 I-layer, 4-head
52.47
86.58
66.15
2-layer, 1-head
54.05
88.53
67.46
2-layer, 4-head
53.39
87.52
66.83
F.6 SENSITIVITY ANALYSIS
Sensitivity of the Temperature т. The temperature parameter 7 in Eq. 118 controls the sharpness of the softmax distribution over class predictions. We conduct systematic experiments varying (0.05, 0.2, 0.5, 1} across multiple datasets in Table We can find that the default T = 0.2 provides optimal balance, but the model is highly sensitive this hyperparameter within reasonable range [0.2, 1]. However, as foundation model, all trainable parameters and hyperparameters must remain fixed across all datasets and tasks. Thus, set 0.2 uniformly for all experiments both during pretraining and in-context inference without any task- and dataset-specific tuning.
Sensitivity DPAA Configurations For the DPAA depth and number heads, we directly adopt single-layer and single-head design for complexity and generalization considerations. Adding more layers and heads would increase computational and memory costs and could introduce a higher risk of overfitting, whereas
goal is to keep the learning process efficient, lightweight, yet expressive. The results Table 2 and Table 3 already show that this simplest DPAA configuration outperforms baselines, which highlights the architectural strength of our approach rather than relying heavy over-para parameterizaaion. Table shows that the 1-layer 1-head and 2-layer 1-head settings achieve the best performance, with I-layer I-head offering the best trade-off between efficiency and effectiveness. Therefore, we adopt the I-layer 1-head configuration our experiments. Moreover, matrices Wr and Wv in this table. The shared setting outperforms the separate setting, possibly because the latter introduces more parameters and is more prone to over-fitting, and shared parameters can effectively improve efficiency.
the unified default setting for all datasets conduct an ablation study comparing shared and separate weight
G THE USE OF LARGE LANGUAGE MODELS (LLMs)
In this paper, LLMs have been used solely for polishing the writing and identifying typographical errors. No LLMs were used for generating research content, analysis, or conclusions.
27


Reviews:
[{"review_id": "oVVdgd6r46", "reviewer": "Reviewer_Dtat", "summary": "The paper introduces MF-GIA aimed at doing few-shot / in-context node classification on new graphs without fine-tuning. The key idea is: for each target graph, first build a domain embedding to capture the graph’s style/distribution; then use this embedding to generate alignment/FiLM transforms that map (i) node features and (ii) label IDs into a shared semantic space, even when the original features aren’t text-based. On top of this, a DPAA module performs in-context matching between query nodes and the support set. so the model can adapt at test time just by seeing a few labeled nodes, with gradient-free inference.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- It doesn’t assume raw text or a specific encoder. as long as features are vectors, the domain-conditioned aligner can normalize them, which makes it more practical for real graph platforms. \n\n- Adaptation is done by in-context attention + generated aligners, so deployment on new graphs is lightweight and gradient-free.\n\n- Using a domain embedding to steer FiLM-style transforms lets the model cope with cross-graph distribution shift (different feature spaces, different label vocab)", "weaknesses": "- Forcing all graphs to a fixed width via SVD could destroy domain-specific structure and cross-domain comparability before alignment.\n\n- A single small-step gradient from a shared init can be noisy/sensitive to loss scaling; the theory assumes smoothness/Lipschitzness and gives an upper bound but not tight guarantees for practical separability. \n\n-  Pretraining uses only four node-classification datasets; broader modalities/tasks would better justify foundation-level generality.\n\n- The episodic objective introduces a temperature τ, but robustness/ablation for τ and DPAA projections isn’t detailed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dwaN098DXk", "reviewer": "Reviewer_pj4V", "summary": "This paper introduces MF-GIA, a modality-free in-context learning framework for graph-structured data. It enables frozen, pretrained graph encoders to perform few-shot classification across heterogeneous domains without requiring raw modality inputs or task-specific fine-tuning. The method relies on gradient fingerprints to characterize domains and uses domain-conditioned transformations to align pre-encoded features and labels into a unified space for cross-domain reasoning.", "questions": "**Q1:** How stable are the gradient fingerprints across variations in the support set?\n\n**Q2:** Why is a *single* gradient step chosen to compute the gradient fingerprint?\n\n**Q3:** Can you elaborate on why the episodic training scheme outperforms supervised pretraining followed by prototype matching?\n\n\n**Q4:** What exactly is the nature of the modality heterogeneity in the pretraining and test datasets?\n\n\n**Q5:** How does the domain embedder perform when exposed to domains that are semantically distant from any in the pretraining set?\n\n\n**Q6:** What is the computational overhead of generating gradient fingerprints at test time, particularly for large graphs or many-shot settings?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "S1: The use of gradient fingerprints to derive domain embeddings and guide feature/label alignment is novel and well-motivated.\n\nS2: The model meets all three desired criteria—modality-free, post-training free, and cross-domain alignment—which most prior works fall short of.\n\nS3: MF-GIA significantly outperforms competitive baselines on both node and edge classification tasks in few-shot settings, including unseen domains and tasks.\n\nS4: The architecture, especially the use of Dual Prompt-Aware Attention, is carefully designed and theoretically grounded.\nScalable and Flexible: The approach accommodates pre-encoded data and avoids reliance on raw modalities, making it broadly applicable in practical scenarios.", "weaknesses": "To be honest, I did not find very serious weaknesses in this submission. Below are just some suggestions that may lead to a more solid work.\n\n\n- Suggestion 1:The robustness of gradient fingerprints across noisy or low-quality domains is not deeply examined.\n- Suggestion 2: While results suggest episodic training is beneficial, more direct comparison with alternative meta-learning strategies would strengthen the claim.\n- Suggestion 3: While the framework is motivated by practical constraints (e.g., privacy, modality), the paper could benefit from an application case study demonstrating real-world utility, for example, RecSys (Adaptive Coordinators and Prompts on Heterogeneous Graphs for Cross-Domain Recommendations. SIGIR 2025), Protein (Protein Multimer Structure Prediction via PPI-guided Prompt Learning. ICLR 2024), Urban (Urban Region Pre-training and Prompting: A Graph-based Approach. KDD'25,  Boundary Prompting: Elastic Urban Region Representation via Graph-based Spatial Tokenization. arXiv.), drug (DDIPrompt: Drug-Drug Interaction Event Prediction based on Graph Prompt Learning. CIKM 2024), cross domain (All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining. SIGKDD 24), and theory basis (Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis. ICML 2025.). In summary, I think the authors could discuss more and the introduced related work seems to be not sufficient to capture the latest application trends in this area. \n- Suggestion 4: The conclusion should explore concrete future research directions. The authors are encouraged to expand their discussion with pointers to promising avenues such as: Task-agnostic generalization using prompt pools across evolving graphs\n(see \"ProG: A Graph Prompt Learning Benchmark\" – NeurIPS 2024), Large language model integration for socio-semantic graph understanding (see \"When LLM Meets Hypergraph: A Sociological Analysis on Personality via Online Social Networks\" – CIKM 2025)\nTheoretical perspectives on data operation views in graph prompting (see \"Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis\" – ICML 2025)\n\n\nKindly note that all the above mentioned works are just for the author's information, not a mandatory request of citing them. Overall, this is a strong, well-written, and original submission that advances the state of graph in-context learning. I recommend acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iUISTmghFR", "reviewer": "Reviewer_e9tY", "summary": "The paper introduces a Modality-Free Graph In-context Alignment (MF-GIA) framework that enables pretrained graph encoders to perform few-shot, in-context learning across heterogeneous domains without relying on raw graph data. By leveraging gradient fingerprints and a dual prompt-aware attention mechanism, MF-GIA aligns pre-encoded features into a unified semantic space.", "questions": "1. In the label alignment module, what if the number of classes in the unseen graph is less than $L_{max}$ that is determined by the available (training and test) graphs? In this scenario, it seems to restrict the capability of label alignment.\n2. In table 2, “–” denotes datasets where only encoded features and indexed labels are available, making modality-dependent models inapplicable. Why do the authors report the performance of OFA and GraphAlign on ogn-Products dataset while do not report the results of AutoGFM?\n3. In the proposed DPPA, two attention layers share the same projection matrices $W_k$ and $W_v$. Have you tried to not share the parameters? It seems that sharing the same matrices enforces the identical matching in both the feature embedding space and the label space. Intuitively, the cross-graphs matching in these two spaces should not be identical.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The proposed method achieves good performance on the few-shot node classification task across 5 datasets in m-way k-shot settings.\n2. This paper provides the theoretical analysis for the proposed feature-alignment. \n3. The presentation of this paper is good and the paper is easy to follow.", "weaknesses": "1. In Theorem 3.1, the authors fail to explicitly define how Wasserstein distance $W_2(\\cdot,\\cdot)$ is used to quantify the distance between two domains. This omission makes it unclear what assumptions are made about the underlying distributions, and whether the bound holds under general conditions. Furthermore, in Definition 3, the authors do not provide a formal definition of the graph distance metric $d_g(G_i, G_j)$, leaving readers uncertain about what graph properties are used for similarity measurement. While the proof later specifies the way to measure graph distance, presenting it only in the proof creates the illusion that the theoretical statements are self-contained and fully justified, when the definitions are essential for interpreting the results. A more rigorous treatment would explicitly define all metrics and assumptions in the main text before presenting the theorem.\n2. The term \"modality-free\" is somehow tricky. From terminology perspective, a modality-free model treats all input data under a unified representation or processing framework, regardless of its original modality. However, simply not having access to raw data does not automatically make a model or setting modality-free.\n3. In table 3, MF-GIA performs worse on WN18RR dataset and a large performance gap between MF-GIA and the baseline methods. However, the authors do not discuss why the proposed method fails on this dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mHShbhUF5D", "reviewer": "Reviewer_LCqo", "summary": "This paper proposes MF-GIA, a modality-free in-context learning framework for heterogeneous pre-encoded graphs. The key idea is to derive a single-step gradient fingerprint from a frozen GNN initialization and use it as a domain descriptor. This domain embedding modulates lightweight FiLM-based feature and label alignment modules. The method is trained via episodic tasks and performs test-time adaptation by computing a new gradient fingerprint on the support set—without updating model parameters. A dual prompt-aware attention (DPAA) module performs few-shot matching.", "questions": "Please refer to the above weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Novel idea: Leveraging a single-step gradient displacement as a domain signature to instantiate FiLM alignment is an interesting and original approach for graph ICL.\nAmbitious scope: The method targets heterogeneous graphs and attempts to unify node-level and edge-level tasks without modality conversion.", "weaknesses": "W1. Gradient-fingerprint robustness not validated\nThe central mechanism assumes that a single-step gradient displacement Δθis a robust domain descriptor. However, the paper provides no sensitivity analysis to different initialization seeds, learning rates, or step counts (1 vs. 2+). If the fingerprint is unstable or dominated by noise, the entire alignment process can break down.\n\nW2. Domain embedding loss collapse risk\nThe pairwise distance-preserving loss used to train the domain embedder lacks explicit normalization or regularization, and no experiments test whether embedding collapse or trivial scaling happens. While scalability is not a major concern given the small number of pretraining domains, training stability and meaningful domain structure need to be demonstrated.\n\nW3. Misleading “gradient-free” terminology & lack of inference-overhead reporting\nThe paper repeatedly claims “gradient-free adaptation,” yet inference requires a backward pass to compute Δθ_\"new\" . This distinction matters for deployment and comparisons against genuinely gradient-free ICL methods. The paper should clarify the terminology (e.g., “parameter-update-free”) and report inference-time overhead (FLOPs, memory, latency).\n\nW4. Label alignment is fragile to label-ID permutation. The FiLM-modulated label prototype table indexes shared label embeddings by label ID. This implicitly assumes consistent label semantics across domains. In practice, label IDs are arbitrary. The paper lacks tests for label-ID permutation invariance or alternative mechanisms to infer label semantics from the support set.\n\nW5. Lack of architectural ablations\nNo ablation comparing Conv2D vs. more canonical designs (Flatten+MLP, pooling) for Δθembedding. No ablation on DPAA depth or number of heads.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)