Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors
Authors: Jeongwhan Choi
Jongwoo Kim
Woosung Kang
Noseong Park
Venue: ICLR 2026 Poster (2026)
Keywords: graph machine learning
node classification
Primary area: learning on graphs and other geometries & topologies

Abstract: One of the most challenging problems in graph machine learning is generalizing across graphs with diverse properties. Graph neural networks (GNNs) face a fundamental limitation: they require separate training for each new graph, preventing universal generalization across diverse graph datasets. A critical challenge facing GNNs lies in their reliance on labeled training data for each individual graph, a requirement that hinders the capacity for universal node classification due to the heterogeneity inherent in graphs --- differences in homophily levels, community structures, and feature distributions across datasets. Inspired by the success of large language models (LLMs) that achieve in-context learning through massive-scale pre-training on diverse datasets, we introduce NodePFN. This universal node classification method generalizes to arbitrary graphs without graph-specific training. NodePFN learns posterior predictive distributions (PPDs) by training only on thousands of synthetic graphs generated from carefully designed priors. Our synthetic graph generation covers real-world graphs through the use of random networks with controllable homophily levels and structural causal models for complex feature-label relationships. We develop a dual-branch architecture combining context-query attention mechanisms with local message passing to enable graph-aware in-context learning. Extensive evaluation on 23 benchmarks demonstrates that a single pre-trained NodePFN achieves 71.27\% average accuracy. These results validate that universal graph learning patterns can be effectively learned from synthetic priors, establishing a new paradigm for generalization in node classification.

Published as a conference paper at ICLR 2026
LEARNING POSTERIOR PREDICTIVE DISTRIBUTIONS FOR NODE CLASSIFICATION FROM SYNTHETIC GRAPH PRIORS
Jeongwhan Choi Jongwoo Kim,
Woosung Kang,
Noseong Park
KAIST Daejcon, Republic Korea
ABSTRACT
One of the most challenging problems in graph machine learning is generalizing across graphs with diverse properties. Graph neural networks (GNNs) face fun- damental limitation: they require separate training for each new graph, preventing universal generalization across diverse graph datasets. A critical challenge facing GNNs lies in their reliance on labeled training data for each individual graph, a requirement that hinders the capacity for universal node classification due to the heterogeneity inherent in graphs differences in homophily levels, community structures, and feature distributions across datasets. Inspired by the success large language models (LLMs) that achieve in-context learning through massive- scale pre-training on diverse datasets, we introduce NodePFN. This universal node classification method generalizes to arbitrary graphs without graph-specific training. NodePFN learns posterior predictive distributions (PPDs) by training only on thousands of synthetic graphs generated from carefully designed priors. Our synthetic graph generation covers real-world graphs through the use of ran- dom networks with controllable homophily levels and structural causal models for complex feature-label relationships. We develop a dual-branch architecture combining context-query attention mechanisms with local message passing to enable graph-aware in-context learning. Extensive evaluation on 23 benchmarks demonstrates that single pre-trained NodePFN achieves 71.27% average accuracy. These results validate that universal graph learning patterns be effectively learned from synthetic priors, establishing a new paradigm for generalization node classification.
INTRODUCTION
Graph neural networks (GNNs) have achieved success in tasks on graph-structured data prevalent in
chemistry (Gilmer et al. 2017 Hamilton 2020), recommender systems (Ying et al. 2018 He et al.)
2020). biology (Bongini et al. 2022), social sciences (Kipf & Welling 2017 Qiu et al. 2018), etc,
by learning to aggregate neighborhood information through message passing. However, GNNs still
have the limitation that, for node classification (Kipf & Welling 2017; Bresson & Laurent 2017
Hamilton et al. 2017 Klicpera et al. 2019 Zhou et al. 2020 Choi et al. 2023 Luan et al. 2023
Kim al.]] 2025| Choi et al. 2025b). separate GNN models must be trained for the labeled nodes
of each new graph. This dependence on graph-specifie training makes generalization across graphs with different properties challenging. The core issue is that real-world graphs exhibit vastly different structural properties varying homophily levels, community structures and features, and degree distributions among datasets. GNNs struggle to handle this diversity without dataset-specific training.
The success of foundation models, particularly large language models (LLMs) (Brown et al. 2020 Touvron et al., 2203) Achiam et al. 2023) comes from their training paradigm of learning generaliz- able patterns from massive and diverse datasets. This enables these models to perform in-context learning, adapting to new datasets without parameter updates by learned patterns during pre-training.
"Equal contribution.
1 "orresponniig Author.

---

Published as a conference paper at ICLR 2026
Real-world Graphs
Synthetic Graphs
Real-world Graphs
Graph A D.1
Graph B
Graph =
Graph
n?
In 0.9
0.9
Train
Train
Train
Inference
GNN A
GNN
NodePFN
NodePFN
Predict A
Predict B
Predict Synthetic
Predict
Predict
(a) Standard GNNs
(b) Our proposed NodePFN
Figure 1: NodePEN enables universal node classification. (a) Each real-world graph requires its own trained GNN model. (b) Pre- tre-training on synthetic graphs sampled from controlled priors (varying homophily (h) from 0. to 0.9) produces a single model capable generalization arbitrary graphs
In manner analogous the capacity of LLMs to adapt to new samples with only context examples, we propose graph model that performs node classification on arbitrary graph datasets. This implies that a single pre-trained model could perform node classification on arbitrary graph datasets without needing to be trained specifically for that dataset.
Recent studies have explored applying L.LMs to graphs (Li et al., 2024a Chen et al. 2024b a et al. 2024c Tang et al. 2024] Liu et al. 2024). However, LLMMs primarily trained on textual data, are better suited for capturing semantic content rather than learning the structural patterns that govern node classification framework, still requires training on specific source datasets, with performance varying significantly based on the training dataset choice.
Li
diverse graph topologies. While Zhao et al. (2025) introduce fully inductive
We propose a different approach by training on synthetic graphs that systematically cover the diversity of real-world graphs (see FFig. T). The key insight is learning the posterior predictive distribution (PPD) from synthetic priors. Recently, prior-fitted networks (PFNs) have demonstrated that models trained on synthetic data from carefully designed priors can approximate PPDs for new datasets in single forward pass (Müller et al., 2022, Hollmann et al. (023). This approach enables in-context learning. That is, the model learns extract patterns from context examples (labeled nodes) and apply them to query points (unlabeled nodes), enabling immediate prediction without gradient updates. We extend this PFN paradigm to graphs by designing synthetic graph priors that sy systematically control homophily levels, community structures, and feature-labe relationships. We aim to design model that predicts the label distribution of query nodes based on labeled context nodes in real graphs, by learning PPD from various synthetic graphs.
We introduce NodePEN learning PPDs for node classification from synthetic graph priors. During training, we generate thousands of diverse synthetic graphs, leveraging methods that control class homophily and community levels to ensure that they include a range of network characteristics found in real-world benchmarks.
Experimental evaluation on 23 real-world benchmarks shows that NodePEN achieves competitive performance. Our approach outperforms both homophily and heterophily graph benchmarks, surpassing GNN baselines. These extensive experiments validate that the patterns governing node classification can de facto be learned from synthetic priors.
The contributions of our proposed NodePFN are summarized as follows.
To the best of our knowledge, we are the first to extend the PFN paradigm to graphs, demonstrating that PPDs for node classification can be learned from synthetic graph prior distributions without requiring actual training data. (Section 3).
We design a comprehensive synthetic graph prior by using random networks, incorporating levels of homophily, community structure, and leature-label relationships. (Section 3.2)
Our code is available here: http:///w/////////oo///////////a /Nodeee
2

---

Published as a conference paper at ICLR 2026
To enable learning graph-aware context from both labeled examples and topological structure, developed a novel dual-branch architecture combining a context-query attention mechanism with local message passing (Section 3.3).
We demonstrate universal node classification across 23 diverse real-world benchmarks using a single pre-trained model, achieving average accuracy of 71.27% and strong performance of 65. 14% on challenging heterophily graphs where traditional GNNs struggle (Section 4).
PRELIMINARIES
In this section, we introduce posterior predictive distribution and prior-data fitted networks. Then, we address the notation used in our study and node classification
2.1 POSTERIOR PREDICTIVE DISTRIBUTION IN SUPERVISED LEARNING
In supervised learning, the goal is predict labels for unlabeled data points using labeled training samples. Given a training set D Duain {x,, y:) and test set Piest = {x we aim to predict labels for the test set. In the Bayesian framework, we model the conditional distribution p(y) x Φ) with parameters treated as random variables with prior p(ø). The goal is to predict labels for test point Xtest using the posterior predictive distribution (PPD):
p(Yiesi Xeest Dirain) = / p p(Yresa Xtese ()p(()(Dmmnn))
where the posterior distribution follows Bayes' rule:
n P(rl x p II I((:)x;))
(2)
If the hypothesis class includes the true conditional distribution, there exists a o* such that ply x; 6²) Pirue ( (y| xx) for all yx, then the PPD results in optimal prediction.
2.2 PRIOR- DATA FITTED NETWORKS
Prior-data fitted networks (PFNs) Müüller al. 2022) learn approximation of the PPD from the
training data using neural networks. Instead of computing the integral in/Eq. (I) at test time, PFNs trained synthetic datasets sampled from prior (D) to learn:
fo (Xeets Duran) + p(Yresi Xiett Dwain).
During training, we sample synthetic datasets from prior p(D). Each dataset is split into training and test sets. The PFN So with parameters is trained to minimize the expected loss:
(0)) ED |- log go (Yreal Xiest, D Durin))
where go is the neural network's approximation of the true PPD. By training on synthetic datasets, the model learns to extract relevant patterns from context samples and apply them to new queries.
This approach allows the model perform inference at time single forward pass without gradient updates, given new dataset. Through implicit Bayesian inference, the network learns to marginalize over parameter uncertainty.
2.3 GNNs FOR NODE CLASSIFICATION AND THEIR LIMITATIONS
In the node classification problem, given graph (V,8) with node feature matrix X <d where d is the feature dimension, adjacency matrix A {0, 1)/1|(D) and set of labeled nodes Virain C. with their corresponding labels Yeess the goal is to predict labels Yies for the unlabeled node set V V Virain

---

Published as a conference paper at ICLR 2026
Homophily in Node Classification. The success of GNNs is believed to be rooted in the homophily assumption [McPherson et al.) 2001), which implies that connected nodes tend to share similar attributes (Hamilton 2020). This provides additional useful information in the aggregated features compared to the original node features, and the effectiveness of node classification can he determined by the level of edge homophily (Luan et al., 2023 Zhu et al., 2020a). which measures the tendency of connected nodes to share the same class label. The level of homophily, h, falls within the range of [0, 1J, with value closer I, strong homophily, implying that GNNs are more likely to outperform than non- gon-graph models, and vice versa.
2.4 RANDOM GRAPH MODELS
We consider 2 random networks, Erdós-Rényi (ER) model (Erdös & Rényi 1959) generates graphs where each edge appears independently with probability Per- This creates graphs with binomial degree distributions and no inherent community structure. Stochastic block models (SBMs) (Holland et al.) 1983) control commu- nity structure through different connection prob- abilities within and between groups. Contextual SBMs (cSBMs) (Binkiewicz et al.] (017)) ex- tend SBM by relating community membership to node labels and allow control over homophily.
(a)
(b)
(c)
(d)
Figure 2: Network examples with various graph priors used in NodePFN training. (a) Sparse ER. (b) Dense ER. (c) Low homophily cSBM. (d) High homophily cSBM. For simplicity, class color- coded nodes are not shown in ER.
3 NODEPEN: PRIOR-FITTED NETWORKS FOR NODE CLASSIFICATION
We introduce NodePFN, prior- fitted network that learns to approximate PPDs for node classification from synthetic graph data (see Fig. 3p. Unlike traditional GNNs that require task-specific training, NodePFN performs in-context learning on arbitrary graphs in a single forward pass.
3.1 LEARNING FROM SYNTHETIC GRAPH PRIORS
we train a neural network fo to approximate posterior predictive distributions for node classification. During training, we sample synthetic graphs G p(9) and learn to predict query node labels from context examples:
Given the PPD framework from Section 2
fo : (Xest, Durin 9) + p(yreal D 9),
(5)
where Dirin {Xv. Yv) v E Virain} contains labeled training nodes. This formulation naturally induces in-context learning: the model learns to extract patterns from training nodes and apply them to test nodes.
3.2 SYNTHETIC GRAPH PRIORS
As shown in Fig. 3(a) our approach begins with sampling diverse synthetic graph priors that capture the broad spectrum of structural patterns found in real-world networks.
Feature-Label Relationships via Causal Models. We generate feature-label relationships using structural causal models (SCMs) (Peters et al., 2017 Pearl 2209) instantiated as random MLPs. For each graph, sample MLP architecture and convert to a DAG by dropping random connections. Gaussian noise propagates through this network to produce node features X from intermediate layers and labels y from later layers, creating complex non-linear dependencies. Importantly, for cSBM graphs, these generated labels determine the community assignments, which in turn control the graph structure through the homophily parameter h.
Graph Structure Generation. We two random network models shown in Fig. (i) cSBMs generate graphs with controlled community structure and homophily. We sample the homophily level from 0.1 to 0.9. The cSBM creates edges with intra-community probability pm and inter-community
4

---

train combine Published as a conference paper at ICLR 2026
Syrthatic Graph Priors
(9CM)
Synthelic Graph ( (v.6))
Graph A
Graph (Sreai (V.6))
(Srva (V.6))
Labeling
(Piren)
(Dlead) (Unisbeled
Processing
Processing
Context (llrann
Context (Dtran)
Query (Due))
Context (Prai
Query (Doert)
Contextual Stochaste (:58M
Erdbs-Rámi Rånyl
NodePFN
NodePFN
Mexsugee
Adtnttinn
HC Lotttn
H((11mppe)
Predict
Predict
Synthelic Graphs
H
(a) Prior Fitting Parse
(b) Pretraining on NodePEN
(c) Inference on Real Graph
Figure 3: NodePFN overview. (a) Generation of diverse synthetic graph priors with varying struc- tural properties. (b) Dual-branch architecture combining local message passing with context-query attention for in-context learning. (c) Inference on real-world graphs via the pre-trained NodePFN without task-specific re-training.
probability Pour such that h - Pin/ (pin + Pomt). This control over homophily allows to generate graphs ranging from strong homophily to heterophily. (ii) ER networks provide unstructured baseline graphs where edges appear independently with probability Per- This ensures the model learns beyond community-baie -basud patterns. The distribution for Per generates graphs with varying densities, from sparse dense networks. During training, we sample from both networks to ensure comprehensive coverage of graph structures encountered in practice.
3.3
MODEL. ARCHITECTURE
Each NodePEN layer consists of two parallel branches that process information complementarily, as shown in Fig. 3(b)
Context-Query Attention Branch. Following the PFN design (Müller et al. 2022), we use asym- metric attention patterns to enable in-context learning. The initial representations H (0)
embeddings both features and labels, while (0) uses only feature embeddings (detailed imple-
mentation in Appendix B.5). For Vizann with observed labels, self-attention allows them to build
comprehensive understanding of the label distribution:
(f+1) SelfAttention
(6)
train
urinn
train
train
For test nodes Viests cross- attention to training nodes enables leveraging the learned patterns:
H
H Lat1))) = CrossAttention (H
(
train
where the attention functions follow the standard formulation. We employ multiple attention heads with outputs concatenated and linearly projected. This asymmetry ensures test nodes leverage training information without influencing each other's 's predictions.
Local MPNN Branch. In parallel, message passing aggregates neighborhood information to capture local graph topology:
HC (++ (11mm))))
MPNN(H (9)
(8)
where À D 1/2 AD is the symmetrically normalized adjacency matrix and D is a degree matrix. This branch captures structural patterns critical for classification regardless train/test splits. In our framework, GCN (Kipf & Welling 2017) for the local MPNN branch.
D-2
5

---

Published as a conference paper at ICLR 2026
Layer Fusion.
The parallel branches merge with the input via residual connections:
(++1) LayerNorm(H + (amm)
H ((++)))mmn))
This design enables NodePFN to simultaneously learn from labeled examples via attention and local graph structure via message passing.
3.4
How TO TRAIN
We train NodePEN approximate the PPD by minimizing the expected cross-entropy synthetic graphs sampled from our prior:
[
(()) = ED-p(D)
Vv.c log (o Xu, Duain 9)
(10)
Viestl
DE VEVuu cm)
where is the number of classes, Ye,e the one-hot hot encoded label for node and class c, and fo is our neural approximation to the true PPD from Eq. (1) For each synthetic graph G, we randomly partition nodes into Viain and Viest-
3.5 How TO INFERENCE
As shown in Fig. 3(c) NodePFN performs direct prediction on a real-world graph Greal with own training-test split. Given labeled nodes Vinin with Dimin = {(x1, W:) : E Vinin] and unlabeled nodes Vist. the model computes predictions
single forward pass.
Given a real-world graph, we perform a preprocessing step (Appendix B.6 on the graph and its features. Then, the model processes the graph through L NodePFN layers and outputs the PPD:
f3 X.: Draii Grea) softmax (orth
(11)
for each test node Viea. Training nodes incorporate label information through concatenation with features, while test nodes use only features. This provides calibrated uncertainty estimates as the model has learned to approximate the true PPD during training. Importantly, gradient updates or fine-tuning are required the pre-trained model generalizes directly to new graphs
4 EXPERIMENTS
In this section, we present experiments to evaluate the performance of our proposed NodePFN. We begin by detailing the experimental settings. Next, we investigate the following research questions:
(RQL. Does our NodePFN perform well on various controlled homophily synthetic graphs?
(RQ2.) Does our NodePFN generalize well for node classification on real-world benchmarks?
(RQ3.) How does the performance of NodePFN compare against training-frre methods?
(RQ4.) Does NodcPFN perform well compared to the baseline for structural node classification?
(RQ5.) How do components contribute to NodePFN's effectiveness?
4.1 (RQ1.) CONTROLLED SYNTHETIC GRAPHS
Setup. To evaluate the classification capability on various homophily ratios. we the synthetic Cora generator Li et al. (202T) The detailed synthetic datasets are in Appendix
100
Test Accuracy
MLP
GCN
NodePFN
80
60
Results. Fig. shows the mean test accuracy. MLP maintains its test accuracy for all homophily rates. GCN and GAT perform poorly at low homophily rates. Our NodePFN has the best overall trend without sudden drops. Our prior data contribute to its stable accuracy for both homophily and heterophily settings compared with other models.
40
0.00
0.25
0.50
0.75
1.00
Homophily Ratio
Figure 4: Experiments on the syn- thetic Cora Dataset.

---

Published as a conference paper at ICLR 2026
Table Performance comparison on homophily and heterophily real-world benchmark datasets. We report the average accuracy and ranking on each type of dataset, as well as the overall values.
GraphAny (Producct))
GraphAny (Arxiv)
GraphAny (Wisconsin)
GraphAny (Cora)
Datasct
MLP
GCN
GAT
NodePFN
AirBrazil
23,08
42.31+7.98
57.69+14.75
34.61+16.34
34.61
36.15-+16.68
33.07
75.38++1.88
AirEU
21.25
41,88:3.00
32.50:33.45
41.75=6.84
41 11.50+6.50
41. 41.13-6602
40.50+701
1..00++1121
AirUS
22.88:1146
46,491181
48.47:14.17
43.57=2.07
13.
43.86.1:.44
43.46:1::40
61.66.1:0.31
Cora
48.42400.33
81.40-4.700
81.70++1.43
79.36=0.25
0..38
77.82+11115
80.18+0012
82.06-1-0.29
Homaphily Graphs
Ciieszer
44,40+0.44
63.40:20.33
69.10
07.94
68. 68.34+00.25
67.50 67.50+:0.4
68, 68.90:::00
67.30+4-08
Pubmed
69.5011:.79
76.60:10.32
77.30:10.60
0..54=0034
7..36110117
71.46:1:0.30
76.60:1:031
78.00:10.24
WikiC's
72.72-40043
79.1244045
79.27-+0.00
75.01=0.54
7..95+00.61
73.77-+0.83
75.98-1:.50 74.39:400.1
Amazon- Phoxo
68.20-4-0.8
91,88-+0.29
91.86:+1.07
90.64=0.82
90.60--0882
90.18-+:0.91
90.14::0552
90.53-+-113
Amazon-Comp
58.28:122.88
85,83:1.0.6
87.01:10.00
82.90=1.225
13,04:1:.24
82.00:11.14
82.99:1:222
81, 81.42.10:.18
DBLP
56.27-4:0.62
73.02-4:2222
13.87
7..62
7..90
70.13-t:0.77
41.73
74.71:1-0.29
Coauthor CS
85.88:2099
91.83::0.71
88, 79:
90 0,556
00.45
90.85:2065
90.47:1:033
91.55::-0.12
Coauthor Physies
87.43:1:1.98
93.93:10.37
93.01:1:.899
0.55
92.69:20.52
92.54:10.33
92.70.1:0.5
93.43:1:0113
Docuer
54.24:1:2115
53.69:1:2229
55.99:13.78
$2.11
$2.13-13:02
51.98
53.45:10..5
Average Accuracy
56.43
73.05
74.39
71.09
71.14
70.86
71
77.39
Average Ranking
7.62
4,92
4.54
4.46
4.31
4.31
4.15
1.69
Cornell
67.57:15.06
35.14:1611
35,14:13.52
64.86=0,00
1..94:1:448
66.49
19.8
2..89:12.76
Texas
48.65444.01
51.35-+221
54.05+22.41
73.52=2296
72.97
+3.51++121
+.889+148
76.22-47.55
Wisconsin
66.67:13.11
17.25:1:.64
52.9411.10 10
65.89 2..89
65. 3.22
71.77
61.18:1008
79.22:1:6.97
Heterophily Graphs
Chameloon
38.87:1:2211
41.31:13.05
39.83:12.10
49.45=4.20
37.40
36.67:1.532
37.99:14.54
50.11:133.00
Actir
33.95:000
28
27.30:20.22
28.99=0.61
28.6020.21
20.51
27.91 16
32.99:::10
Minesweeper
80,00:10:00
81.12:10.37
80.08:10:44
10.27=0.16
80.30 80.30:2..13
80.
80.46:10:15
20.66:10
Tolokers
78.16:10.02
79.93:10.10
78.50:10.55
78.18:0.03
78.18110.44
78.24:1:0.3
78.20:1:0002
78.61
Amazon- Ratings
47,90+t0:45
47.35::026
47.18:1042
42.70=0.10
42.74=20.12
42.57:20.34
42.84:2:004
44.68-2-0.8
Questions
97.33:2006
15:10::
97.1 1::02
97. 10=0.011
97 0..09220.02
10:000
97. 03
97. 97.02:1:000
Squirrel
35.55:1:0:8
38.67:1:1.84
38.78:12.39
28.92:22.8
37.73
36.76:13.55
37.25 1:265
43.40:1:1.08
Average Accuracy
58.17
58.84
39.11
61.39
60.71
61.62
60.56
65.14
Average Ranking
7.20
6.80
6.60
4.90
4.70
4.50
4.60
1.70
Avg. Accuracy
$7.30
66.63
67.67
66.24
65,93
66.24
66.00
71.27
Avg. Ranking
7.41
5.86
5.57
4.68
4.50
4.40
4.38
1.70
4.2 (RQ2.) EXPERIMENTS ON REAL- GRAPH BENCHMARKS
Setup. We evaluate on 23 benchmark datasets for node classification. We compare against MLP, GCN (Kipf & Welling 2017). GAT (Veličković et al.. 2018), and GraphAny (Zhao et al., 2025) models. If there are reported results from Zhao et al. (202)), we directly adopt the reported results, otherwise, we run experiments with their optimal setting. More detailed dataset and evaluation settings are provided in Appendices Aland B.7
presents comprehensive comparison of our results on 23 datasets. The results show that NodePFN achieves the best overall average accuracy of 71.27% using only single pre-trained model. In contrast, GraphAny models require training on each specific dataset but still underperform NodePFN. On homophily and heterophily datasets, NodePFN achieves the highest average accuracy. Moreover, GraphAny models show inconsistent performance depending on the characteristics of the training dataset. GraphAny (Cora) performs well on homophily graphs but worse on Wisconsin, one of the heterogeneous graphs. In contrast, NodePFN consistently performs well both graph types without requiring dataset-specific training.
Results. Table
4.3 (RQ3.) COMPARISON WITH TRAINING-FREE METHODS
Setup. NodePFN can be compared with several training-free methods, which can be tested directly without training steps. We compare closed-form so- lution methods that use pseudo-inverse operations to solve node classification lem (Zhao et al., (025). The methods include the "Linear" model that predicts directly without graph
Table 2: Training-fiee free models vs NodePFN.
Method
Cora
Puhmed
Wisconsin
Lincar
52.80:10.00
59: 3::00:000 80.00-2.15
$..35
78.20:20.00
02
57.64-1407
46.03=6.86
a regression prob-
HGC
22.5011..00
46.32.1.00
64.32-255
6.35
LabelProp
60.30:10.00
63 04.004
16.08-2155
23.53-5.51
TFONN
60.03
40.04-0001
14.51=300
19.91=6.10
NodePEN 82.0620.20 78:0020:24
81 31.18:570 76.22=7:55
7

---

Published as a conference paper at ICLR 2026
Test Accuracy (%)
100
Table 3: Ablation studies
75
Mewnr: 71.2%
NodePFN components.
50
Hean 15.5%
Ablation
Cora
Wisconsin
Tolokers
wo ER
81.26
78.82
77.30
w/o cSBM
80.62
80.39
77.18
TSBPFN
NodaPFN
TahPFN
53.10
72.94
78.18
Figure 5: Structural node clas- sification results (GraphAny trained on Wisconsin).
Figure 6: Comparison of ac- curacy distributions between TabPFN and NodePFN.
NodePFN-L6 NodePFN-Seq
53.10
72.94
78.00
80.64
78.82
77.88
NodePEN
82.06
81.18
78.61
convolutions, and closed-form models using SGC (Wu al.
2019) and high-pass filter graph convolutions (HGC) (Chien et al. 20221; Luan et al. 2022). We also include label propagation ("LaabelProp") (Zhu & Ghahramani 2002 and TE-GNNs (Sato) 2024).
the results demonstrate NodePEN's superior performance all datasets compared to training-free baselines. Our NodePFN consistently outperforms all baseline methods while using single pre- rre-trained model. This demonstrates that the learned inductive bias of NodePFN surpasses analytical closed-form solutions and highlights the value of the pre- pre-trained trained approach for generalizable node classification.
Results. In Table 2
4.4 (RQ4.) STRUCTURAL NODE CLASSIHICATION
Setup. We evaluate NodePFN on Airport datasets (Ribeiro et al. 2017). where the goal is to predict the "structural role" of each node based only on the network topology without node features (Cui itt al. 2022). Node features are provided as one-hot encoded node identifiers. This setting evaluates whether NodePFN can learn structural roles when forced to rely primarily on topological patterns. We include Node2 Vec (Grover & Leskovec] (016) and LINE (Tang et al.) 2015) additional baselines, since these methods specialize in structural embedding.
the results show that NodePFN outperforms all baselines. This suggests that NodePFN learns robust structural patterns that generalize beyond node features and effectively identifies meaningful node properties well as structural roles based network topology.
Results. As shown in Fig. 5]
4.5
(RQ.5) ABLATION STUDIES
demonstrates the robustness of NodePFN's s design through ablation studies. Removing ER Networks or cSBM shows minimal performance degradation: for homophily datasets, Cora, cSBM removal causes slight drops, while for heterophily datasets such as Wisconsin and Tolokers (with 10 features and 0.5 homophily ratio), the impact is negligible. This indicates that these priors adapt well to different graph characteristics. The architectural variant NodePFN-L6, with reduced model capacity from M9.01M to 14.80M parameters, shows performance drops on Cora. This suggests that sufficient model capacity is important for learning patterns in highly homophily datasets. NodePFN- Seq with sequential processing maintains competitive performance, validating the effectiveness of both parallel and sequential architectures for combining structural information.
Table 3
We also compare TabPFN, since when all graph-specific priors and MPNN components are removed, proposed NodePFN can be reduced TabPFN, As shown in Fig. 6 NodePFN outperforms TabPFN on all datasets (see Appendix for full results). At the same time, TabPFN shows wider variance and lower overall accuracy, confirming the necessity of graph-aware modeling over treating nodes as independent tabular data.
RELATED WORK
Prior-data Fitted Networks. Müller et al. (2022) introduced PFNs and proved that a Transformer
trained on tasks drawn from prior approximate PPDs from in-context examples. Following this work, |agler (2023) shows how PFNs approximate PPD and why they can still learn at inference.
and this paradigm has been adapted specialized domains. TabPFN (Hollmann et al. 2023 2025
8

---

Published as a conference paper at ICLR 2026
demonstrates that carefully designed synthetic priors can yield state-of-the- oh-the-art performance on small tabular datasets. Also, the PFN has been adapted to time-series forecasting (Dooley et al. 2023 and Hoo et al. 20244] 2025) analyzes time series via feature engineering and encodes temporal pattems as tabular features. Concurrently, TabPFN-GN (Choi et al. 2025a) converts graph data into tabular features for direct use with TabPFN, but its reliance feature engineering limits performance on heterophily graphs (cf. Section 4.5)
Graph Foundation Models. Recent works primarily leverage L.LMs for zero-shot learning. GraphGPT (Tang et al. 2024), GraphI.LM (Chai et al., 2023), and LLAGA (Chen et al. 2024a)
convert graphs to text descriptions, while frameworks that use text-attributed graph datasets (Li
et al. 2024c). such as OFA (Liu et al. (2024), use LLMs to encode node features. More recently, GOFA (Kong et al. 2024), Graph-R (Wu et al. 2025). and ZeroG (Li et al. 2024b) extend this line modeling, explicit reasoning for zero-shot learning, and cross-datase transferabiliity, respectively. These approaches leverage LLMs' strengths and limitations, including their dependency on textual attributes. In contrast, our approach requires no LLMs and works with arbitrary node features.
of work by exploring joint
GNNs for Node Classification. Node classification is a classical graph machine learning task on which GNNs have recently achieved strong results. GCN (Kipf & Welling (017). Graph- SAGE (Hamilton et al. 2017), and GAT (Veličković et al. 2018) established the foundation of GNNs by showing strong performance on homophily graph datasets. Additionally, neighborhood aggrega- tion of GNNs shows stable performance on homophily graph benchmark datasets but struggles with heterophily graphs (Pei et al. 2020). Sato (2024) proposes training- free GNNs for node classification, but they are always suboptimal for GNN performance and have limitations that make them inapplica- ble to heterophily graphs (see Section 4.3). As GNNs may not dominate all graph networks, zero-shot approaches leveraging pretrained models such as TabPFN can bypass this architecture search.
6 DISCUSSION
Although our primary objective is to demonstrate that universal node classification can be achieved via synthetic graph priors, the proposed NodePFN has limitations. NodePFN currently requires fixed maximum class numbers (tested up to 20 classes) and feature dimensions during training, and the attention mechanism's 's quadratic complexity restricts applicability to large-scale graphs. We leave these limitations for future work.
Despite these limitations, NodePFN's pre-training paradigm offers significant advantages. Although the model requires computational resources to pre-train on approximately 250,000 synthetic graphs (see (Appeddxx C). this investment is amortized across all subsequent inference tasks. This con- trasts with conventional GNNs that require retraining for each new dataset. Extensive experiments demonstrate that NodePFN achieves universal node classification, thereby justifying this initial computational overhead.
This universal applicability stems from our focus on learning structural patterns from synthetic priors. Unlike recent graph foundation models that rely text-attributed graphs and LLMs (Tang et al., 2024; Chai et al., 20233 Liu et al., (024) (as discussed in Section 5), NodePFN operates on graphs with arbitrary numerical features without requiring semantic understanding.
7 CONCLUDING REMARKS
We presented NodePFN, the first extension of PFNs to graphs, showing that universal node clas- sification can be learned from synthetic graph priors. A single NodePFN model demonstrates an average accuracy of 71.27% 23 benchmarks, particularly outperforming standard GNNs het- erophily graphs. This work represents a new paradigm for graph machine learning through synthetic pre-training, validating that universal patterns can be learned without real-world training data.
We discussed the limitations of NodePFN in ieeetioo 6. Future work could address these limitations by exploring efficient attention mechanisms for massive graphs and investigating hybrid approaches that combine our structural pattern learning with semantic processing from text-attributed graphs.

---

Published as a conference paper at ICLR 2026
ACKNWOLEDGEMENTS
This work was partly supported by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grants funded by the Korean government (MSIT) (No. RS-2024- 00457882, AT Research Hub Project; No. RS-2025-25442144, LG AI STAR Talent Development Program for Leading Large-Scale Generative AI Models in the Physical AI Domain), and the Korea Advanced Institute of Science and Technology (KAIST) grant funded by the Korea government (MSIT) (No. G04240001, Physics-inspired Deep Learning).
ETHICAL STATEMENTS
In terms the broader impact of this research on society, we do not see the very negative impacts that might be expected.
USE OF LLMS
In accordance with ICLR 2026 policy, acknowledge the of LLMs in the preparation of this paper. To achieve perfect grammar and better expression and translation, we use Google Translate (Google
and DeepL (DeepL SE 2025) improve some text. DeepL has an LLM-powered feature built in.
REPRODUCIBILITY STATEMENT
To ensure reproducibility and completeness, we have included appendices in this paper. We also report the model architecture, all training hyperparameters, and the hardware specifications for our experiments in Appendix B. The synthetic graph prior generation and all associated hyperparameters are described in Appendices B.3 and The source code can found in the following: https 7/githubs cwm wtt.//////wwwwa/cocoooooooo NodePFN
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprini MXXi::303308772: 2023.
Albert-LászIó Barabási and Réka Albert. Emergence of scaling in random networks. science, 286 (5439):509-512, 1999.
Norbert Binkiewicz, Joshua T Vogelstein, and Karl Rohe. Covariate-assisted spectral clustering. Biometrika, 104(2):361-377, 2017.
Deyu Bo, Xiao Wang, Chuan Shi, and Huawei Shen. Beyond low-frequency information in graph convolutional networks. In AAAI, 2021.
Pietro Bongini, Niccold Pancino, Franco Scarselli, and Monica Bianchini. Biognn: how graph neural networks can solve biological problems. In Artificial Intelligence and Machine Learning for Healthcare: Vol. 1: Image and Data Analytics, pp. 2-1-231. Springer, 2022.
Xavier Bresson and Thomas Laurent arXiv:1711.07533, 2017.
Residual
gated
graph convnets.
arXiv preprint
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Ziwei Chai, Tianjie Zhang, Liang Wu. Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang. Graphllm: Boosting graph reasoning ability of large language model. arXiv preprint mrXiv:2310.05844 2023.
10

---

Published as a conference paper at ICLR 2026
Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.08170, 2024a.
Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song. Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, et al. Text-space graph foundation models: Comprehensive benchmarks and new insights. Advances Neural Information Processing Systems, 37:7464-7492, 2024b.
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized PageRank graph neural network. In Proceedings of the International Conference Learning Representations (ICLR), 2021a.
Eli Chien, Jie Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In Proceedings of the 9th International Conference on Learning Representations (ICLR), 2021b.
Jeongwhan Choi, Seoyoung Hong, Noseong Park, and Sung-Bae Cho. Gread: Graph neural reaction- diffusion networks. In International Conference on Machine Learning (ICML), pp. 5722-5747. PMLR, 2023.
Jeongwhan Choi, Woosung Kang, Minseo Kim, Jongwoo Kim, and Noseong Park. Can tabpfn com- pete with gnns for node classification via graph tabularization? arXiv preprint arXiv:2512.08798, 2025a.
Jeongwhan Choi, Seungjun Park, Sumin Park, Sung-Bae Cho, and Noscong Park. Are graph transformers necessary? efficient long-range message passing with fractal nodes in mpnns. arXiv preprint 1r..iv:2511.3300 2025b.
Hejie Cui, Zijie Lu, Pan Li, and Carl Yang. On positional and structural node features for graph neural networks non-attributed graphs. In Proceedings the 31 ACM International Conference Information & Knowledge Management, pp. 3898-3902, 2022.
DeepL SE. Deepl translate: The world's most accu- rate translator, 2025. URL https:/ www deepl. httpl//ccc///aaasaat..ooo Last accessed 25 Sep 2026.
Samuel Dooley, Gurnoor Singh Khurana, Chirag Mohapatra, Siddartha V Naidu, and Colin White. Forecastpfn: Synthetically-traine zero-shot forecasting. Advances Neural Information Process- ing Systems, 36:2403-2426, 2023.
Paul Erdös and Alfréd Rényi. random graphs i. Publ. math. debrecen, 6(290-297):1,, 1959.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George F. Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning, 2017.
Google LLC. Google translate, 2025. URL cessed 25 Sep 2026.
Last ac-
Aditya Grover and Jure Leskovec. Node2vec: Scalable feature learning for networks. In KDD, pp. 855-864 2016.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 2017.
William L. Hamilton. Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3):1-159, 2020.
Xiangnan He, Kuan Deng, Xiang Wang. Yan Li, YongDong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In SIGIR, 2020.
Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First
steps. Social networks, 5(2):109-137, 1983.
11

---

Published as a conference paper at ICLR 2026
Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. TabPEN: A transformer that solves small tabular classification problems in a second. In The Eleventh International Conference Learning Representations, 2023. bttttps://openreveew....
iid=cp5Pvc 6w8
Noah Hollmann, Samuel Müller, Lennart Purucker. Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirmeisterr, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319-327) 2025.
Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. The tabular foundation model tabpfn outperforms specialized time series forecasting models based on simple features. In NewrIPS Workshop Time Series in the Age of Large Models, 2024.
Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. From tables time: How tabpfn-v2 outperforms specialized time series forecasting models. arXiv preprint arXiv:2501.02945, 2025.
Jongwoo Kim, Seongyeub Chu, Hyeongmin Park, Bryan Wong, Keejun Han, and Mun Yong Yi. Leveraging multi-facet paths for heterogeneous graph representation learning. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, pp. 1334-1343, 2025.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
Lecheng Kong, Jiarui Feng, Hao Liu, Chengsong Huang, Jiaxin Huang, Yixin Chen, and Muhan Zhang. Gofa: generative one-for-all model for joint graph language modeling. arXiv preprint arXiv:2407.09709 2024.
Xunkai Li, Zhengyu Wu, Jiayi Wu, Hanwen Cui, Jishuo Jia, Rong-Hua Li, and Guoren Wang. Graph learning in the era of Ilms: A survey from the perspective of data, models, and tasks. arXiv preprint arXiv:2412.12456. 2024a.
Yaxin Li, Wei Jin, Han Xu, and Jiliang Tang. Deeprobust: a platform for adversarial attacks and defenses. In AAAI, pp. 16078-16080, 2021.
Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li. Zerog: Investigating cross-dataset zero-shot transferability in graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1725-1735, 2024b.
Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Victor W Chan, and Jia Li. Glbench: A comprehensive benchmark for graph with large language models. Advances in Neural Information Processing Systems, 37:42349 42368, 2024c.
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In The Twelfth International Conference on Learning Representations, 2024. /RRLAttts///peerreee
Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao- Ween Chang, and Doina Precup. Revisiting heterophily for graph neural networks. Advances in neural information processing systems, 35:1362-1111 1361-13757 2022.
Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao-Wen Chang, Jie Fu, Jure Leskovec, and Doina Precup. When do graph neural networks help with node classification? investigating the homophily principle on node distinguishability. Advances in Neural Information Processing Systems, (1-28744-2876) 2023.
Miller McPherson, Lynn Smith-Lovin and James M Cook. Birds of a feather: Homophily in social networks. Annual review of sociology, (7(1):1154444 2001.
12

---

Published as a conference paper at ICLR 2026
Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do bayesian inference. In International Conference on Learning Representations, 2022.
Thomas Nagler. Statistical foundations of prior-data fitted networks. In International Conference on Machine Learning, pp. 25660-25676. PMLR, 2023.
Judea Pearl. Causality. Cambridge university press, 2009.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei,, and Bo Yang. Geom-gen: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287, 2020.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: fouudatio
and learning algorithms. The MIT press, 2017.
Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of GNNs under heterophily: Are we really making progress? In The Eleventh International Conference Learning Representations, 2023a. URL https
/ppenrevieww net ftr.moorummmmm/eoommma Pittmmiditttee Job-rrunnn-dttttto----o------- -
Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look the evaluation of gnns under heterophily: Are we really making progress? arXiv preprint arXiv:2302. 1640, 2023b.
Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and Jie Tang. Deepinf: Social influence prediction with deep learning. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp.
2018.
Leonardo FR Ribeiro, Pedro HP Saverese, and Daniel R Figueiredo. struc2vee: Learning node representations from structural identity. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 385-394, 2017.
Benedek Rozemberezki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding. Journal of Complex Networks, 9(2):cnab014, 4, 2021.
Ryoma Sato. Training-free Graph Neural Networks and the Power of Labels as Features, August 2024. URL http::tttp:xii..//a///aa/// org www./arg/aaaaaoaoooooa/.oaaaaaaa 2404. 19288 19288 arXiv:2404.19288 [cs].
Jiabin Tang, Yuhao Yang. Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 491-500, 2024.
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large- scale information network embedding. In The Wehooff (former WWW), pp. 1067-1077, 2015.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 2rXIv:2302.13971, 202.,
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. Proceedings of the International Conference on Learning Representations (ICLR), 2018
Felix Wu. Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q Weinberger. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning (ICML), pp. 6861-6871, 2019.
Yicong Wu, Guangyue Lu, Yuan Zuo, Huarong Zhang, and Junjie Wu. Graph-r1: Incentivizing the zero-shot graph learning capability in ilms via explicit reasoning. arXiv preprint arXiv:2508.17387. 2025.
Ruosong Ye, Caiqi Zhang, Runhui Wang. Shuyuan Xu, and Yongfeng Zhang. Language is all a graph needs. arXiv preprint arXiv:2308.071.44 2023.
13

---

Published as a conference paper at ICLR 2026
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. KDD, 2018
Jianan Zhao, Le Zhuo, Yikang Shen, Meng Qu, Kai Liu, Michael Bronstein, Zhaocheng Zhu, and
Jian Tang. Graphtext: Graph reasoning in space. arXiv preprint arXiv:2310.01089, 2023.
Jianan Zhao, Zhaocheng Zhu, Mikhail Galkin, Hesham Mostafa, Michael M. Bronstein, and Jian Tang. Fully-inductive node classification on arbitrary graphs. In The Thirteenth International Conference on Learning Representations, 2025. /tlLittss////peeeeeeee f
id=1Qpt43cqhg
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. Al open, 1:57-81, 2020.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in neural information processing systems, 33:7793-7804, 2020a.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In Advances in Neural Information Processing Systems, 2020b.
Xiaojin Thu and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propaga- tion. In Technical Report CMU-CALDD-02-007, Carnegie Mellon University, 2002.
14

---

Published as a conference paper at ICLR 2026
Supplementary Materials for "Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors"
A DETAILS OF DATASETS
The Synthetic Cora Network The synthetic Cora dataset is provided by (Zhu et al. 2020b). Zhu ctt al. (2020b) generate graphs for a target homophily level using a modified preferential attachment process. We sample nodes, edges, and features from Cora to create synthetic graph with desired homophily and feature/label distribution. In Table we summarize the properties of the synthetic Cora networks we used.
Table The detailed information of the synthetic Cora. All levels of homophily have the same number of features (1, (1,433), nodes (1,480), edges (5,936), and classes (5).
Homophily
Avg. Degree
Max. Degree
Min. Degree
0.0
3.98
84.33
1.67
0.1
3.98
71.33
2,00
0.2
3.98
73.33
1.67
0.3
3.98
70.00
2.00
0.4
3,98
77.67
2.00
0.5
3.98
76.33
2.00
0.6
3.98
76.00
1.67
0.7
3.98
67.67
2.00
0.8
58.00
1.67
0.9
3.98
58.00
1.67
1.0
3.98
51.00
2,00
Real-world Graph Datasets. We list the dataset statistics we used in Tables 5 5] 6 We use 23 benchmark datasets for node classification. Following prior work, these include both 13 homophily graphs (Kipf& & Welling 2017 Rozemberczki et al.) 2021) (e.g.. Cora, Citeseer, Pubmed, WikiCS) and 10 heterophilous graphs (Pei et al./ 2020 Platonov et al. 2023b) (e.g., Cornell, Texas, Squirrel, Actor). For Chameleon and Squirrel, we use filtered datasets from Platonov et al. (202aa). We also report the clustering coefficients of each graph dataset.
Table Homophily dataset statisties for node classification benchmarks.
Dataset
#Nodes
#Edges
#Features #Classes #Laabels Coeff.
Train/Val/Test (%)
AirBrazil
131
1,074
N/A
4
80
0.6364
11.19/1.1/19
AirEU
1,190
5,995
N/A
0.5393
20. 101.111//440 8/40.
AirUS
10,008
13,599
N/A
80
0.5011
6.7/46.6/46.
Cora
2,708
10,556
1,433
7
140
0.2407
5.2/18.5/36.9
Citeseer
3.327
9,104
3,703
6
120
0.1435
3.6/15.0/30.
Pubmed
19,717
88,648
500
60
0.0602
0.3/2.5/5.1
WikiCS
11,701
431,206
300
10
0.4660
5.0015.1/49.9
Amazon-Phhen Photo
7.650
238,162
745
160
0.4101
2.1/49.0/4 0/99.0
Amazon-Comp
13,752
491,722
767
10
200
0.3513
1.5/49
DBLP
17,716
105,734
1,639
80
0.1344
0.5/49
Coauthor-CS
18,333
163,788
6,805
15
300
0.3425
1.6/49 2/49.2
Coauthor- Coauthor-Physics
34,493
495,924
8,415
100
0.3776
0.3/49. 9/49.9
Deezer
28,281
185,504
128
2
40
0,1412
0.1/49.9/49.9
15

---

Published as a conference paper at ICLR 2026
Table Heterophily dataset statistics for 10 benchmarks.
Dataset
#Nodes
#Edges
#Features #Classes #Labels Coeff.
Train/Val/Test (%)
Cornell
183
554
1703
5
87
0.1671
47. 47.5/32.2/20
Texas
183
558
1703
87
0.1979
47.5/31.7/20.2
Wisconsin
251
1703
0.2077
47.8/31.9/22 9/20.3
Chameleon
890
8854
2325
445
0.5769
00.0/5.00/5.5
Actor
7600
30,019
932
S
3648
0.0802
48.0/32.0/20.0
Minesweeper
10,000
78,804
5000
0,4355
50.0/25 0/25.0
Tolokers
11,758
1,038,000
5879
0,5329
50.0/25.0/25.
Amazon-Ratine Ratings
24,492
186,
300
12,246
0,5816
00.0/25.0/555
Questions
48,921
307,080
301
2
24,460
0.0307
50.0/25.0/25. 0/25.0
Squirel
2223
46,998
2089
5
1053
0.4631
50.0/25.0/25.0
B DETAILED EXPERIMENTAL SETTINGS
B.1 HARDWARF AND SOFTWARE SPECIFICATIONS
Our implementation is developed on top of the TABPFN-V framework. All experiments were performed using the following software and hardware environments: UBUNTU 21.04 LTS, PYTHON 3.10.16, 16, PYTORCH 1.12.11 PYTORCH GEOMETRIC 2.3.1, TORCH-SCATTER 2.1.2, TORCH-SPARSE 0.6.18, 18, NUMPY 1.26.4, NETWORKX 3.3, SCIKIT-LEARN 1.4.0, CUDA 12.3, NVIDIA Driver 550.54.14, 19 CPU, NVIDIA RTX 6000.
B.2 TRAINING SETUP
The model configuration of NodePEN is summarized in Table 7
In total, the model contains approximately 29.1M trainable parameters. We trained NodePFN for total of 30 epochs, each epoch comprising up to 1,024 steps (245,760 steps total) with a batch size of 8 (see more hyperparameters Table 8 The total training required approximately 6 GPU hours single NVIDIA RTX A6000.
Table 7: Model configuration of NodePEN.
Hyperparameter
Value
Embedding dimension
512
Number of layers
12
Number of attention heads
4
Dropout
0.0
Table 8: Training hyperparameters for NodePFN.
Hyperparameter
Value
Epochs
30
Steps per epoch
1024
Batch size
8
Embedding size
512
Learning rate
{1.5 10-5,
5 10
Optimizer
AdamW
B.3 DETAILS NODEPEN PRIOR
Structural Causal Models (SCM) We adopt the optimal sampling distributions from TabPFN (Holl mann 2023 for our SCM prior ensure robust feature-label relationships. Following TabPFN's
framework each SCM is generated by:
qit htthh//cw//ttiaaaaa/p/peeeeeeeee
16

---

Published as a conference paper at ICLR 2026
Sampling MLP layers feem ~ p(esem) and hidden size heem p(hsem) from discretized noisy log-normal distributions
Creating a layered graph structure and randomly dropping edges to form a DAG
Selecting feature nodes and one label node from the causal graph
Using activation functions sampled from Tanh, LeakyReLU, ELU, Identity
Applying noisy log-normal noise distributions with beta-distributed dropout rates
This generates complex non-linear feature-label dependencies while maintaining the causal structure that has proven effective for tabular data modeling.
Contextual SBM. Our contextual SBM generates community-structure graphs with controllable homophily levels:
Sample homophily rate 0.0.1,0.9) and intra-community probability Pin (00011,0.1)
Compute inter-community probability as Pout Pin x (1 h) Assign nodes communities based their labels from the SCM
Generate edge probabilities using power distributions: probs,,; Power(5) X Pout fori * for inter-community edges, with diagonal values Pout + power(2, size) (pin - Pout) for intra-community connections
Create symmetric probability matrix and generate edges via the stochastic block model
This approach ensures that network topology and node labels are inherently related through the homophily parameter.
ER networks provide unstructured baseline graphs complementing the community- based patterns:
ER Network.
Sample edge probability Per ((0.01,0.05)
Generate edges independently with probability Eij Bernoulli(pe))
Creates graphs with binomial degree distributions and no inherent community structure
The combination of cSBMs (50% of training graphs) and ER networks (50% of training graphs) ensures comprehensive coverage of graph structures from community-based patterns to random connectivity.
B.4 FLEXIBLE ENCODER FOR VARIABLE NODE FEATURE DIMENSIONS
Our NodePFN model designed to handle graphs with varying node feature dimensionalities up to a pre-defined maximum capacity. This is achieved through a flexible input encoder that standardizes the feature vectors before they are processed by the main architecture.
When graph is provided where the node features have dimension d that is less than the maximum supported dimension of the model, dmaax each node's feature vector is first extended to dmaz dimensions by appending zero-padding. Then, to ensure that the zero-padding process does not systematically alter the input's scale or introduce bias, we apply a normalization factor to the padded vector. This mechanism ensures that feature vectors from different graph datasets are processed consistent scale, enabling our single pre-traie rreitrained model to generalize effectively across wide range of struccurd data.
B.5 FEATURE AND LABEL EMBEDDINGS IN IMPLEMENTATION
We employ learnable linear transformations Wx E ddxdua and Wy E Rd to project features and labels the embedding dimension d, where diea is the feature dimension and C is the number
17

---

Published as a conference paper at ICLR 2026
of classes. Following TabPFN-v1's implementation, element-wise addition rather than concatenation to combine features and labels:
irain W
Y W
X W
train
where aain and Y vain are the feature and label matrices for training nodes. This additive approact [ maintains constant dimensionality and enables the model learn complementary representations different subspaces of the embedding vector. During inference, real-world labels are first converted canonical integers before applying Wy.
B.6 INFERENCE IMPLEMENTATION DETAILS
We describe the data preprocessing methods used in the inference stage below.
Graph Structure Preprocessing. We convert the adjacency matrix of given graph into a normal- ized adjacency matrix.
Feature Preprocessing. Adopting the ensemble approach from Hollmann et al. (023)) we employ method that alters the order and scaling of features within the imput context. This integrates form of ensemble technique, using a fixed number of 32 multiple inputs, with the subsequent prediction results being aggregated. When real-world graphs have features exceeding the capacity our NodePFN, we apply truncated SVD for dimensionality reduction. We also optionally apply feature smoothing using sum aggregation by edge connectivity for enhanced feature quality.
B.7 EXPERIMENTAL SETTINGS FOR NODE CLASSIFICATION
Evaluation Protocol. For homophily datasets, we follow the semi-supervised setting of Kipf & Welling (201)) for Cora, Citeseer, and Pubmed (20 nodes per class for training, 500 validation, 1000 test), while for WikiCS, we follow the splits RRozebberzzki al. (2021), I. and the remaining datasets follow the GraphAny (Zhao al., (025) protocol (20 nodes class for training, the rest split evenly into validation and test). For heterophily datasets, we use the predefined split masks provided in
Pei et al. (2020) and Platonov et al. (203b). For Chameleon and Squirrel, we use filtered datasets
from Platonov et al. 2023a).
Search Space of Hyperparameters. We report search space of hyperparameters used in experiments in Table 9 Note that we use the default hyperparameter of 32 ensembles.
Table 9: Homophily dataset hyperparameters for node classification.
Hyperparam.
Range
Components
(10,15,20,25,30)
# Smoothing Steps
(0. 1, 3,
Optimal Hyperparameters. We report our optimal hyperparameters used in our experiments in We do not use truncated SVD on Tolokers and Minesweeper.
Tables 10 and 11
18

---

Published as a conference paper at ICLR 2026
Table 10: Homophily dataset hyperparameters for node classification.
Dataset
# Components
Smoothing Steps
AirBrazil
25
3
AirEU
25
1
AirUS
25
3
Cora
15
4
Citeseer
15
2
Puhmed
2
WikiCS
15
2
Amazon-Phin Phozo
15
3
Amazon-Comp
15
3
DBLP
25
2
Coauthor-CS
2
Coauthor-Physic
15
0
Deezer
25
2
Table 11: Heterophily dataset hyperparameters for node classification.
Dataset
# Components
Smonthing steps
Cornell
15
Texas
Wisconsin
Chamelcon
25
Actor
35
Minesweeper
Tolokers
Laazoon-Rating
25
Questions
25
Squirrel
15
19

---

Published as a conference paper at ICLR 2026
C PRIOR DATA SCALE ANALYSIS
We analyze the impact of the number of synthetic prior graphs on NodePEN performance and address the question of data efficiency in PFN training. This analysis aims to understand the trade-ofi off between computational cost and
85
Accuracy
performance gains when scaling prior data.
In our pretraining, each training iteration generates new synthetic graphs based our random network prior, which creates diverse structural patterns. Fig. reveals sev- eral key insights: (1) Particularly evident in the heterophily Texas dataset, its performance increases substantially with more prior data. The accuracy improves from approxi- mately 53% to 76%. (2) For Cora, NodePEN shows more modest gains, and this suggests that patterns with higher homophily rates are effectively learned via cSBMs. (3) The performance improvement peaks approximately 250,000 prior graphs.
65
60
55
Cora
Texas
50
50000
100000 150000 200000 250000 Number of Prior Graphs
Figure 7: Impact of Prior Data Scale on NodePFN Performance
While generating synthetic prior data incurs an initial computational cost, this expense amortized across all future inference tasks. Unlike traditional GNNs, which require retraining for each new graph, NodePFN enables immediate inference on arbitrary real-world graphs via one-time training on synthetic priors. Given the universal applicability of the resulting model, the computational investment comprehensive prior generation proves worth it.
D COMPARISON WITH TABPEN AND NODEPEN
In In[Appendix D and Table 13 we report all results from Fig. 6lin Section 4.5 We used the TabPFN-v1 framework for comparison with TabPFN, and to ensure fair comparison, we employed pre- trained model using the same number of prior data points as ours. For TabPFN, we use the default hyperparameter of 32 ensembles, and we also enable feature subsampling for feature preprocessing.
Table 12:
Comparison between TabPFN and NodePEN on homophily datasets (accuracy %).
Table 13:
Comparison between TabPFN and NodePFN on heterophily datasets (accuracy %).
Dataset
TabPFN
NodePFN
AirBrazil
52.31
75.38
Dataset
TabPFN
NodePFN
AirEU
53.62
57.00
Coreell
55.68
71.89
AirUS
53.55
61.66
Texas
62.70
76.22
Cora
53.10
82.06
Wisconsin
72.94
79.22
Citeseer
32,94
67.30
Chameleon
37.54
50.13
Pubmed
65.04
78.00
Actor
25.84
32.99
WikiCS
62.25
75.98
Minesweeper
79.86
80.66
Amazon-Photo
74.77
90.53
Tolokers
78.18
78.61
Amazon-Comp
50.66
81.42
Maazznn-Rating
21.64
44.68
DBLP
60.69
74.71
Questions
90.09
97.02
Co-author CS
48.32
91.55
Squirrel
31.84
43.40
Co-author Physics
63,41
93.43
Deezer
49.83
53.45
Average
53.47
65.14
Average
55.62
77.39
20

---

Published as a conference paper at ICLR 2026
E THEORETICAL DISCUSSION
The original PFN framework Mülllr et al. 2022) establishes that Transformers can approximate posterior predictive distributions (PPDs) by minimizing the Prior-Data negative log-likelihood. Specifically, Müller et al. (002 Insight 1) shows that this loss equals the expected cross- ross-entropy between the true PPD and its approximation, while Müller al. (2022) Corollary 1.2) guarantees convergence to the exact posterior under the infinite-capacity assumption, provided the architecture respects the exchangeability of the conditioning dataset D. In practice, this requíres the architecture to be permutation equivariant with respect to the ordering of training examples.
Remark E. (Preservation of PFN Guarantees in NodePFN). The dual-branch architecture of NodePEN maintains permutation equivariance because: the attention branch uses self-attention and cross-attention operations that are inherently permutation equivariant, the MPNN branch uses aggregation functions that are permutation equivariant, and their additive fusion (Eq. (9)) preserves this property. Therefore, NodePFN satisfies the exchangeability requirement of Müller al. (2022)
Corollary 1.2) and converges to the posterior.
This theoretical guarantee ensures that while the MPNN branch enriches the model with structural information, the fundamental Bayesian convergence properties of PFN remain intact.
F COMPARISON WITH HETEROPHILY-SPECCINN GNNS
To further validate NodePFN's effectivvnness on heterophily graphs, we compare against GNNs specifically designed for heterophily: H2GCN (Zhu al. 2020b), GPRGNN (Chien et al., 2021a),
and FAGCN (Bo et al. 2021). As shown in Table 14 NodePFN achieves the best performance on 7 out of datasets despite using no real-world training data, while these specialized methods require dataset-specific training with carefully designed aggregation schemes for heterophily.
Notably, NodePFN shows improvements on Chameleon and Squirrel. The competitive performance on Texas and Actor within 1% of the best methods further confirms that learning from diverse synthetic graphs with controlled homophily provides generalization on the heterophily spectrum without requiring architectural modifications or datasct-specific tuning.
Table 14: Comparison of NodePEN with H2GCN, GPRGNN, FAGCN (accuracy %).
Dataset Chameleon
Squirrel
Cornell
Texas
Actor
Wisconsin A. A.Ratings
Co.CS
Co.Physies
35.10 65.77 6.50 76.58 :: 1.56 35.86 1.03 1.12 40.87 88.45 92.86 GPRGNN 39.69+1.15 38.95
H2GCN 41.07#265
40.54 65.77+136 a3 75.21 4:408 42.23
91.49 1039 92.76+0.20
FAGCN 37.24
36.78
60.38 68.44-1.78
34.87 72.02+504 44.12 91 92.34
NodePFN 50.1323.0 43.40=1..8 21.89 76.22
32.99 79.22:s37 44.68 91.55 93.43
G COMPARISON WITH LLM-BASED GRAPH METHODS ON GLBENCH
Following the experimental setting of recent L.L.Mbased graph methods, we conduct the supervised node classification experiments on all the datasets in GLBench (Li et al. 2024c
NodePFN achieves competitive or superior performance compared to LLM-based graph foundation models without requiring text descriptions or language model dependencies. While LLM- methods leverage pre- rre-trained language knowledge, NodePFN leverages pre-trained patterns from massive synthetic prior data.
H COMPARISON WITH TABPEN USING SMOOTHED FEATURES
Tables and 17 present comprehensive comparisons between TabPFN-vI (Holmmann al. 2023 with smoothed features and NodePFN across homophily and heterophily datasets. The smoothed
https:://github..cmm/ http:://tttttbbb.ooo///meeessses
21

---

Published as a conference paper at ICLR 2026
Table 15: Accuracy under the supervised setting of GLBench (Li 2024c). Best and second-best are highlighted.
Dataset
Cora
Citeseer
Pubmed
WikiCS
InstructGI.M Ye et al. 2023
69.10
51.87
71.26
45.73
GraphTTxt (Zhao ct al. 2023
76.21
59.43
75.11
67.35
LLaGA (Chen al. 2024a)
74.42
55.73
68.82
73.88
OFA (Liu et al. 2024
75.24
73.04
75.61
77.34
NodePFN
76.38
63.08
68. 18.18
76.29
features baseline applies non-parametric feature aggregation before feeding node representations into TabPFN-vI's official checkpoin effectively incorporating local neighborhood information without explicit graph structure modeling. On homophily datasets (Table 16). NodePFN demonstrates consistent improvements. The advantages become even more pronounced on heterophily datasets (Table 17p. where NodePFN substantially outperforms the smoothed feature baseline on Cornell, Texas, and Wisconsin. These results validate that NodePFN's explicit modeling graph topology through its dual-branch architecture provides meaningful improvements over simple feature smooth- ing approaches. Note that TabPFN-vI's limitation to 10 classes prevents evaluation on datasets like Co-author CS, whereas NodePFN supports up to 20 classes.
Table 16: Comparison between TabPFN with smoothed features and NodePFN homophily datasets (accuracy %).
Table 17: Comparison between TabPFN-v1 with smoothed features and NodePFN heterophily datasets (accuracy %).
TabPFN-vI (smoothed features)
Dataset
NodePFN
TabPFN-vI (smoothed features)
AirBrazil
67.69
75.38
Dataset
NodePFN
AirEU
55,62
57.00
Corell
42.16
71.89
AirUS
59.60
61.66
Texas
56.22
76.22
Cora
74.06
82.06
Wisconsin
51.37
79.22
Citeseer
51.16
67.30
Chameleon
41.42
50.13
Pubmed
75.96
78.00
Actor
25.29
32.99
WikiCS
74.90
75.98
Minesweeper
80.07
80.66
Amazon-Phokt
83.69
90.53
Tolokers
78.05
78.61
Amazon-Coma
75.61
81.42
Amazon-Rating
44.24
44.68
DBLP
69.20
74.71
Questions
97.02
97.02
Co-author CS
N/A
91.55
Squirrel
40.42
43.40
Co-author Physics
87.93
93.43
Deezer
48.17
53.45
I EXTENSIVE ABLATIONS ON SYNTHETIC PRIOR DESIGN
Graph generation models and homophily distribution. Table 18 shows ablation results different graph generation models and homophily distributions. We compare our approach against various alternatives including using only ER graphs, only cSBM graphs with restricted full homophily ranges, and only Barabási-Albert (BA) networks (Barabási & Albert 1999). As shown in Table 18, the results reveal several key insights. First, restricting training specific homophily ranges leads to performance degradation. This shows the necessity of covering the full homophily spectrum generalize across diverse real-world graphs. Second, training exclusively Barabási- Albert networks - which explicitly model plwer-law degree distributions - shows inconsistent performance. This suggests that pawer-law topology alone provides insufficient structural diversity for universal graph learning. Third, using only ER or only cSBM graphs underperforms the combined approach, validating that both graph types contribute complementary inductive biases.
22

---

Published as a conference paper at ICLR 2026
Table 18: Ablation study on graph generation models and homophily distributions.
Ablation
Cora
Wisconsin
Tolokers
Only ER
80.62
80.39
77.18
Only cSBM (0.1-0.3)
79.89
79.98
77.65
Only cSBM (0.7-0.9)
80.42
78.20
77.23
Only cSBM (full range)
81,26
78.82
77.30
Only BA (Barabási-Albert)
74.18
80.57
74.63
NodePEN (ER+cSBM)
82.06
81.18
78.61
ER/SBBM ratio analysis. Table 19 examines how the mixture ratio between ER and cSBM graphs affects performance. We varied the proportion of ER graphs from 0% (cSBM only) to 100% (ER only). The balanced 50/50 ratio consistently achieves optimal or near-optimal performance across all homophily regimes. Notably, heterophilic Wisconsin benefits from higher ER ratios (80-50% range), likely because ER's unbiased topology provides crucial structural diversity for heterophilic learning, while homophilic Cora shows more robustness to varying ratios. These results show that ER's 's unbiased topology and cSBM's community structure provide complementary inductive biases essential for universal graph learning.
Table 19: Ablation study ER/SBBM mixture ratio.
ER Ratio
Cora
Wisconsin
Tolokers
100%
(Only ER)
80.62
80.39
77.18
80%
80.90
81.30
77.20
50% (NodePFN)
82.06
81.18
78.61
20%
82.01
78.75
78.10
0% (Only cSBM)
81.26
78.82
77.30
J ARCHITECTURAL ABLATIONS
We provide comprehensive ablation studies on NodePEN's dual-branch architecture demonstrate the necessity and contribution of each component. Table 20 shows results for different architectural variants compared to the full NodePFN model.
Removing the MPNN branch causes substantial degradation on both homophilic Cora and heterophilic Wisconsin, demonstrating that the MPNN provides essential structural inductive biases that pure attention cannot capture. NodePFN-Seq underperforms the parallel design. Reducing model capacity layers (NodePFN-L6) causes failure on Cora, demonstrating that sufficient depth is important for learning diverse patterns from many synthetic priors. These ablations validate that the MPNN provides structural biases, and their parallel combination enables optimal integration, and deep architecture is required.
Table 20: Ablation study on architectural design choices.
Ablation
Cora
Wisconsin
Tolokers
NodePFN-L6
53.10
72.94
78.00
NodePFN-Seq
80.64
78.82
77.88
NodePEN w/o MPNN
75.50
70.10
78.09
NodePFN (Full)
82.06
81.18
78.61
K STATISTICS OF SYNTHETIC PRIOR DATASETS
We provide detailed statistics the synthetic graphs used for pre-training NodePFN. Our training set consists of approximately 250,000 graphs generated over 30 epochs, with each graph sampled from
23

---

Published as a conference paper at ICLR 2026
mixture of ER networks and cSBM with varying homophily ratios. All synthetic graphs are fixed at 1,024 nodes to balance computational efficiency with sufficient structural complexity for learning meaningful patterns. The number of classes per graph varies from 1 to 20, and edge counts vary based on the underlying generation model and density parameters. On average, each graph contains 12,706.4 edges and 8.79 classes.
Fig. 8ishows the distributions of edges and classes across all synthetic graphs. The edge distribution Fig. 8(a)) exhibits normal distribution centered around 10,000-15,000 edges, with long tail extending to sparser graphs. This diversity reflects the combination of sparse ER networks, which generate fewer edges on average, and dense cSBM communities, which create higher edge densities within communities. The class distribution (Fig. 8(b) shows coverage across the full range of 1-20 classes, with slightly higher frequency for graphs with fewer classes.
sdgar
17:00
17000
(a) Edge count distribution
(b) Class count distribution
Figure 8: Distribution of synthetic prior datasets used for pre-training
L COMPUTATIONAL COMPLEXITY
We provide formal complexity analysis for NodePFN. For graph with N nodes, |E] edges, d- dimensional features, and L MPNN layers, the MPNN branch requires O(LEd) operations for message passing and aggregation, identical to standard GCN complexity. The Transformer branch computes attention over all nodes, requiring O(N²d) operations for attention computation and ((N)) for feed-forward layers. The total per-graph complexity is therefore OLEEd N2d).
Table 21 shows comprehensive runtime comparison between GCN and NodePFN across all 23 bench- mark datasets. While NodePFN achieves superior average accuracy and ranking, it also demonstrates remarkable computational efficiency in terms of total deployment cost. GCN requires cumulative training time of 188 seconds plus 12.35 seconds for inference across all datasets. Importantly, this represents single training run per dataset with fixed hyperparameters - in practice, achieving competitive performance typically requires multiple hyperparameter tuning attempts, potentially multiplying this cost by 5 to 10 times more. In contrast, NodePFN requires only 47.78 seconds total representing a 4 times speedup in total time-to-deployment The one-time pre-training cost (6 GPU hours) is amortized across unlimited datasets, eliminating repetitive per-dataset optimization and making it increasingly efficient as more graphs are processed.
Table 21: Runtime efficiency comparison
23 Benchmark Datasets
GCN
NodePFN
Avg. Accuracy Avg. Ranking
66.63
71.27
5.86
1.70
Total Train Runtime
188
47.78
Total Predict Runtime (s)
12.35
24


Reviews:
[{"review_id": "7opXsApSSk", "reviewer": "Reviewer_ogba", "summary": "The authors propose NodePFN, a foundation model for node classification on arbitrary graphs. Following recent results on foundation models for tabular data, whose main insight is training models on synthetically generated data that resembles relationships present in real-world data, the authors propose to train a GNN on synthetically generated graphs. In order to accommodate for the variable number of node features and labels across different datasets, as well as to achieve in-context learning for new graphs, the authors propose an architecture based on self-attention. Experiments indicate that NodePFN effectively generalizes to real world datasets, surpasing the performance of supervised methods that require data-specific training.", "questions": "1. Can you please provide further information about the statistics of the generated datasets?\n2. The appendix states that NodePFN is trained with one synthetic graph per epoch, and that a batch size of 8 is used. What does a \"batch\" mean in this context? In the simplest setting, training a GNN requires using the full graph (i.e. the complete feature and adjacency matrices) in order to do an unbiased forward pass. \n3. Can you please clarify how NodePFN handles input features whose number varies across datasets?\n4. Do you have any thoughts on the generalization properties of NodePFN and its robustness to distribution shift? It would be interesting to know how confident you are that it will work well for every graph in the wild.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper addresses an important limitation of supervised methods that require training from scratch, or at best fine-tuning for every new dataset, thus reducing the cost of deploying predictive models on graphs.\n2. The paper introduces well-motivated synthetic data generation methods that cover diverse network topologies, and an architecture for learning in context intended to be pre-trained with synthetic data.\n3. The experiments are comprehensive, demonstrating the effectiveness of NodePFN agains a broad range of baselines and in ablation experiments.", "weaknesses": "1. The novelty of the work is slightly limited, as it is a natural extension of well-known results from TabPFN and its follow-up works, to the graph domain.\n2. Important details about the generated data are missing. How many graphs in total were generated? What is the number of nodes and edges in them?\n3. NodePFN looks like a costly model for training and inference. Some indication of training cost is given in the appendix (\"6 GPU hours\"), but a formal derivation of complexity is missing from the paper. This is especially important considering that the method requires a new architecture that differs from established ones like the GCN.\n4. Important details about the architecture are not clear. Can NodePFN handle node features, especially a variable number? Section 6 states that this is a limitation, requiring a maximum number of dimensions, but it is not clear how this is dealt with. Does it involve some form of padding along the feature dimension?\n5. A sometimes implicit yet strong motivation for works such as TabPFN, and in this case NodePFN, is that generating a collection of datasets containing complex relations between variables that is large and random enough allows pretraining models that generalize to real data. The question here is: how do we know what is large and random enough to guarantee generalization? Is NodePFN good because our knowledge about the datasets in Table 1 allows us to generate data like them? And if so, how robust is it to distribution shift?\n6. The architecture can in principle be fine-tuned but these experiments are not considered in the paper. It would be interesting to know how much of an improvement this can bring.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eRhXQWhIwu", "reviewer": "Reviewer_WYNZ", "summary": "This paper introduces NodePFN, extending the Prior-Fitted Network (PFN) paradigm to graph neural networks. The key innovation is training a single model on thousands of synthetic graphs generated from carefully designed priors (random networks with controllable homophily and structural causal models) to learn posterior predictive distributions for node classification. This enables the model to perform inference on arbitrary real-world graphs without task-specific training, achieving 71.27% average accuracy across 23 benchmarks.", "questions": "1. How does performance degrade as graphs exceed the training distribution in terms of size, feature dimensions, or class numbers? Can you provide scaling experiments?\n2. Can you provide more extensive ablations on the synthetic prior design? How do different homophily distributions, SCM architectures, or graph generation models affect performance?\n3. Why does NodePFN sometimes underperform GraphAny models trained on specific datasets (e.g., Amazon-Comp: 81.42% vs 83.04%)? What patterns are difficult to capture with synthetic priors?\n4. Could you compare against more recent heterophily-specific GNN methods and explain when NodePFN's universal approach is preferable to specialized architectures?\n5. Is there a way to adapt or fine-tune NodePFN for specific graphs while maintaining most of the universal knowledge? Maybe this could address performance gaps on specific datasets.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The extension of PFNs to graphs represents a genuinely new direction. The idea of learning universal node classification patterns from synthetic priors rather than requiring graph-specific training is novel and addresses a fundamental limitation of current GNNs.\n- The paper provides theoretical analysis (Theorems in Section 2.3) showing how graph structure affects node features' impact on GNN performance, motivating why synthetic priors with controlled properties can capture real-world patterns.\n- Testing on 23 diverse benchmarks, including both homophilic and heterophilic graph,s demonstrates wide applicability. The 71.27% average accuracy with a single model is impressive, particularly the 65.14% on heterophilic graphs where traditional GNNs struggle.\n- The combination of contextual SBMs with controllable homophily (0.1-0.9) and structural causal models for feature-label relationships shows thoughtful prior design that systematically covers graph diversity.\n- The \"train once, deploy everywhere\" paradigm offers practical benefits, removing the need for dataset-specific training while maintaining competitive performance.", "weaknesses": "- The quadratic complexity of attention mechanisms and fixed constraints on class numbers (max 20) and feature dimensions significantly limit applicability to real-world large-scale graphs. This is acknowledged but not addressed.\n- While the paper describes the priors used, it lacks ablation studies on prior design choices. Why these specific distributions? How sensitive is performance to prior specification? The connection between prior properties and downstream performance needs deeper investigation.\n- The paper mentions ~250,000 synthetic graphs and 6 GPU hours of training, but doesn't provide detailed computational comparisons with training multiple task-specific GNNs. The amortization argument needs quantitative support.\n- The paper primarily compares against basic GNNs (GCN, GAT) and GraphAny. Comparisons with more recent heterophily-specific methods (e.g., H2GCN, GPRGNN) and other universal graph methods would strengthen the evaluation.\n- What determines when NodePFN will succeed or fail? The paper doesn't clearly characterize the conditions under which synthetic priors successfully capture real-world patterns or provide failure case analysis.\n- The dual-branch architecture combining attention and message passing is described superficially. Ablations on architectural choices and their contributions are limited (only Table 3 provides some ablations).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2ftqlyC4z2", "reviewer": "Reviewer_f51f", "summary": "NodePFN pretrains on procedurally generated graphs—features/labels from a random SCM and structure from cSBM/ER—to learn the posterior predictive distribution for node labels. At test time it takes any real graph plus a few labeled nodes and, in a single forward pass (no fine-tuning), outputs calibrated label distributions for the rest, achieving competitive accuracy across 23 benchmarks and varying homophily.", "questions": "See weakness.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Studying prior-fitting paradigm on relational data is valuable and interesting\n2. Experimental results are good", "weaknesses": "Major:\n1. One major concern is that the method part writing is unclear. For example, 1. how is the feature and structure treated? Is it like that you first generate the random features and labels, then labels are used as community, and then use community to generate structures (for SBM). 2. How are labels used in the whole pipeline? I even don't see any label encoder module in the paper. \n2. The structure of the model is very different from the original PFN. Transformer is put on top of the MPNN. Can you give some theoretical insights on why this is valid? For this part, I think some theoretical discussions are missing since the theory for PFN no longer holds here. \n3. Some baselines are missing. I'm wondering how this method compares to first use some non-parametric aggregation to get aggregated features and then utilize TabPFN for prediction. \n4. I doubt the scalability of this architecture since you put heavy self-attention ahead of message passing. Also in the experiment part, there's no large-scale datasets. \n\nMinor:\n1. The writing of abstract is not clear. Why training on labels a fundamental limitation?\n2. I don't think this work is inspired from LLM. Philosophy of prior fitting is very different from LLM, and it's also not possible to have a unified vocabulary akin to LLM. I suggest motivating from the prior fitting perspective. \n3. Line 44: What is graph heterogeneity? Graph and tasks are different concepts. Tasks determine labels. \n4. Line 74: Graphany does have designs for heterogeneity. In the aggregation stage, it has filter like (I-A).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pZeatJSckX", "reviewer": "Reviewer_M3ir", "summary": "The paper proposes NodePFN, which extends PFNs to graphs. It learns posterior predictive distributions for node classification by pretraining only on synthetic graphs drawn from priors (ER and contextual SBM with controlled homophily; SCM-generated feature–label mechanisms). A dual-branch architecture mixes context–query attention (for in-context learning from labeled nodes) with local message passing (GCN). A single pretrained model (no per-graph training) shows strong performance across 23 benchmarks, with both homophily and heterophily graphs.", "questions": "1. How does performance change when real graphs exhibit power-law degrees, overlapping/temporal communities, or motif biases besides ER/cSBM?\n2. How does the model compare to the SOTA model for heterophily graphs? What could be the possible reason if the model performs worse?\n3. Could the authors provide more analysis on how the synthetic training graphs are generated? For example, will the sampling distribution affect the model performance? And how does ER graph (when varying its percentage in the training set) affect the model performance? And what is the size distribution of the training graphs?\n4. Overall, I think the paper proposes a possibly promising path to solve the cross-dataset node prediction task. **I will raise the score if the above concerns are settled.**", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It proposes a novel PFN-style, training-free inference framework. The model can learn synthetic priors and predict in one forward pass.\n2. The model trained on synthetic graphs shows strong performance on both homophily and heterophily benchmarks. \n3. The analysis of performance on different datasets and the ablation study are comprehensive.", "weaknesses": "1. Priors are limited to ER and cSBM with SCM-driven features. Real graphs often exhibit heavy-tailed degrees, motifs, assortativity/disassortativity. Whether these properties might affect the model performance is not discussed.\n2. Real-world SOTA GNNs tailored to heterophily, label-efficient settings, or inductive protocols aren’t comprehensively covered.\n3. Pretraining requires ~250k synthetic graphs, but there’s no wall-clock/peak-memory vs. baselines or cost-vs-benefit analysis; only a conceptual argument that training amortizes across tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)