Fully-inductive Node Classification on Arbitrary Graphs
Authors: Jianan Zhao
Zhaocheng Zhu
Mikhail Galkin
Hesham Mostafa
Michael M. Bronstein
Jian Tang
Venue: ICLR 2025 Poster (2025)
Keywords: node classification
inductive generalization
Primary area: learning on graphs and other geometries & topologies
TL;DR: We propose GraphAny, the first fully-inductive node classification model that generalizes to any graph with arbitrary structure, feature and label spaces.

Abstract: One fundamental challenge in graph machine learning is generalizing to new graphs. Many existing methods following the inductive setup can generalize to test graphs with new structures, but assuming the feature and label spaces remain the same as the training ones. 
This paper introduces a fully-inductive setup, where models should perform inference on arbitrary test graphs with new structures, feature and label spaces. We propose GraphAny as the first attempt at this challenging setup. GraphAny models inference on a new graph as an analytical solution to a LinearGNN, which can be naturally applied to graphs with any feature and label spaces. To further build a stronger model with learning capacity, we fuse multiple LinearGNN predictions with learned inductive attention scores. Specifically, the attention module is carefully parameterized as a function of the entropy-normalized distance features between pairs of LinearGNN predictions to ensure generalization to new graphs. Empirically, GraphAny trained on a single Wisconsin dataset with only 120 labeled nodes can generalize to 30 new graphs with an average accuracy of 67.26%, surpassing not only all inductive baselines, but also strong transductive methods trained separately on each of the 30 test graphs.

Published as a conference paper at ICLR 2025
FULLY-INDUCTIVE NODE CLASSIFICATION ON ARBITRARY GRAPHS
Jianan Zhao"... Zhaocheng Zhu Mikhail Galkin Hesham Mostafa Michael Bronstein Jian Tang 1.7,
1 Milla- Québee AI Institute, UUniversité de Montréal, 'Google Research "Intel Labs University of Oxford, AITHYRA, 'HEC Montréal, *CIFAR AI Chair
ABSTRACT
One fundamental challenge in graph machine learning is generalizing to new graphs. Many existing methods following the inductive setup can generalize to test graphs with new structures, but assuming the feature and label spaces remain the same as the training ones. This paper introduces a fully-inductive setup, where models should perform inference arbivary test graphs with new structures, feature and label spaces. We propose GraphAny as the first attempt at this challenging setup. GraphAny models inference LinearGNN, which can be naturally applied to graphs with any feature and label spaces. To further build stronger model with learning capacity, we fuse multiple LiNearGNN predictions with learned inductive attention scores. Specifically, the function of the entropy- normalized
graph as analytical solution
attention module is carefully parameterized distance features between pairs of LinearGNN predictions to ensure generalization new graphs. Empirically. GraphAny trained on a single Wisconsin dataset with only 120 labeled nodes can generalize to 30 new graphs with average accuracy of 67.26%, surpassing not only all inductive baselines, but also strong transductive methods trained separately on each of the 30 test graphs.
INTRODUCTION
Avg best acc. on 31 graphs
GraphAny
One of the most important requirements for ma- chine learning models is the ability to generalize to new data. Models with better generalization abilities are able to perform better on unseen data
GraphAny Wisconsin
Anel
GAT
and tasks, which is a key property for foundation
Trained on:
models (Achiam et al.) 2023; Team et al.] 2023 Touvron al.) (203)) that are designed accom- plish a wide range of downstream tasks. For graphs, generalization is challenging since different graphs usually have different structures and associated at- tributes. Ideally, graph machine learning models expected to accommodate this difference and learn functions that are applicable to all graphs.
Graph
GCN
66.5
Graphs
57.5
MLP
1000
Total training labeled nodes (k)
Figure I: Average performance 31 datasets. GraphAny is trained on a single dataset (Wis- consin or Arxiv) and performs inductive infer- ence on any graph. The other methods have to be trained on each dataset.
Many previous works graphs (Hamilton et al. 2017 Hang et al., 222T Qu et al. 2022) Jang et al. 2026) consider generalization in the inductive setup, where models are supposed to perform inference test graphs with new structures different from the training ones. However, these works rely the assumption that the training and test graphs share the same feature and label spaces, which limits their applications to graphs like to have a model that generalizes to arbitrary graphs, involving new structures, new dimensions
fixed domain (e.g. social networks, citation networks). Ideally, we would
Equual contribution. Code release: Work done while Intel Labs

---

Published as a conference paper at ICLR 2025
Structure
Features
Observed Labels
Unobserved Labels
predict
Pre-training
G (V,8)
X R Mlxd
RPPa/x
predict
Inference
G' = (V,E)
RIV
Yi Pl
Figure 2: dully-inductive node classification: Trained graph G. fully-inductive model should generalize to any new graph G' with new feature and label spaces without additional training.
and semantics for their feature and label spaces without additional training. We name this more general and practical setting as the fully-inductive setup (visualized in Figure 2
The fully-inductive setup particularly challenging for existing graph machine learning models for two reasons: (1) Existing models learn transformations specific to the dimension, type, and structure of the features and labels used in the training. and cannot perform inference on feature and label spaces that are different from the training ones. This requires us to develop a new model architecture arbitrary feature and label spaces. (2) Existing models learn functions specific to the training graph and cannot generalize to new graphs. This calls for an inductive function that can generalize to any graph once it is trained. Despite these challenges, is always possible to cheat the fully-inductive setup by training separate instance of existing models for each test dataset. We emphasize that this solution does not solve the fully-inductive setup. However, this cheating solution can regarded as strong baseline for dally-inductiv models, since it additionally leverages back propagation on the labeled nodes and hyperparameter tuning on the validation data of the test datasets.
Our Contributions. We propose GraphAny solve node classification in the fully-inductive setup. GraphAny consists of two components: LinearGNNs that perform inference on new feature and label spaces without training steps, and inductive attention module based entropy-normalizee distance features that ensure generalization to new graphs. Specifically, our LinearGNN models the mapping between node features and labels
non-parameterie graph convolution followed by linear layer, whose parameters are determined in analytical form without requiring explicit training steps. While single LinearGNN model may be far from optimal for many graphs, we employ multiple LinearGNN models with different graph convolution operators and learn an attention vector to adaptively fuse their predictions. The attention vector is carefully parameterized as a function of distance features between the predictions of LinearGNNs, which guarantees the model to be invariant to the permutations feature and label dimensions. To further improve the generalization ability of our model, we propose entropy normalization to rectify the distance feature distribution to fixed entropy, which reduces the effect of different label dimensions. By combining both modules, GraphAny learns to combine the LinearGNNs' predictions for each node based on their prediction distributions, which reflects statistics of its local structure, and generalizes to new graphs.
To summarize, the contribution of GraphAny are four folds:
We introduce the fully-inductive setup for generalization across arbitrary graphs. Fully-inductive setup is more general and practical than the conventional inductive setup, allowing knowledge transfer across diverse domains, such as from knowledge graphs to e-commerce graphs
We devise LinearGNN, an analytical solution for node classification that enables efficient inductive inference any new graph without the need for gradient descent.
2

---

Published as a conference paper at ICLR 2025
We identify two necessary properties for fully-inductive generalization: permutation invariance and dimensional robustness w.r.t. the feature and label spaces. Towards these goals, we design an inductive attention module that satisfies these properties and generalizes to new graphs.
By combining LinearGNN and the inductive attention module, present GraphAny, the first fully-inductive model for node classification. Across 31 datasets, GraphAny outperforms strong transductive baselines trained separately on each test dataset and achieves 2.95x speedup.
2 RELATED WORK
Inductive Node Classification. Depending on the test graphs which models generalize to, tasks on graphs can be categorized into transductive and inductive setups. In the transductive (semi-supervised) node classification setup (Kipf and Welling] 2017). test nodes belong to the same training graph the model was trained on. In the inductive setup (Hamilton et al. 2017). the test graph might be different. The majority of GNNs aiming work in the inductive setup use known node labels as features. HHang et al. (202T) introduced Collective Learning GNNs with iterative prediction refinement. Jang et al. (2023) applied a diffusion model for such prediction refinement. Structured Proxy Networks (Qu et 2022) combined GNNs and conditional random fields (CRF) in the structured prediction framework. However, the train-test setup in all those works is limited to different partitions of the same bigger graph, that is, they cannot generalize unseen graphs with different input feature dimensions and number of classes. To the best of our knowledge, GraphAny is the first approach for fully-inde fully-inductive node classification, which generalizes to unseen graphs with arbitrary feature and label spaces.
Labels as Features. Addanki et al. (002)) demonstrated that node labels used as features yield noticeable improvements in the transductive node classification setup on MAG-240M in OGB- LSC (Hu et al.) 2021a). Sato (202)) used labels as features for training-free free transductive node classification with specitic GNN weight initialization strategy mimicking label propagation (hence only applicable to homophilic graphs). In homophilic graphs, node label largely depends on the labels of its close neighbors whereas in heterophilic graphs, node label does not depend on the neighboring nodes. GraphAny supports both homophilic and heterophilic graphs in both transductive and inductive setups.
Connections to Graph Foundation Models. Graph foundation models (GFMs), which aim to develop a single model transferable to new graphs and diverse graph tasks, have gamnered significant attention in the graph learning community. The core challenge in designing a GFM is achieving generalization across graphs with varying input and output spaces, which requires identifying an invariant feature space that transfers effectively across graphs (Mao et al., 2024). While fine-tuning a pre-trained model on a new graph is feasible (Zhu et al., 2024), we focus on the more challenging fully-inductive setting, where the model must generalize to unseen graphs without additional training.
In scenarios where graphs share a common feature space, such as in molecular learning (Kläser et al. 2024 Kováes et al.) 2023 Zhang et al. 2023) (Sypetkowski et al. 2024 (Shoghi et al.) 2024) and material design (Batatia et al.] 2024). GNNs can be applied to the shared feature space of atoms, acting as GFMs. For non-featurized graphs, GNNs equipped with labeling tricks (Zhang al. 2021
have shown promise as generalizable link predictors in homogeneous graphs (Dong et al. 2024 and
multi-rclational knowledge graphs (Galkin et al. 2024). Another line of research (Zhao al. [023
Tang et al., 2023 Chai et al. 2223 Fatemi et al. 2024 Perozzi et al. [024 Liu et al.) 2024) focuses
on Text-attributed graphs (Yan et al. (023)) where features are represented as, or converted into, text.
Here, large language models (LLMs) serve as universal featurizers by verbalizing the (sub)graph structure and the associated task in natural language. However, it is unclear whether the sequential nature of L.Ms is suitable for graphs with permutation invariance.
In summary, while existing GFMs have shown promising results, their generalization capabilities often rely on strong assumptions over the training and test graphs, particularly the presence of shared input space. In contrast, GraphAny is the first attempt to the more general and challenging problem of generalizing across arbitrary graphs. This broader scope opens up new possibilities for transferring knowledge between different graph domains, such as from knowledge graphs to e-commerce graphs. Additionally, we believe our proposed model, along with essential generalization properties like permutation invariance and dimensional robustness, provide a solid foundation for future research on powerful and versatile GFMs.
3

---

Published as a conference paper at ICLR 2025
Inductive Attention
(A. x Y..
Graph Conv.
Pseudo-inverse
Predictions
Y
Input graph with features observed labels
LinearGNNs
Painwise
Final Prediction
LIX
1x1
non-learnable
Attention fs
GraphAny
leamable
Entropy
Normalization
The GraphAny Architecture
Figure 3: Overview of GraphAny: LinearGNNs are used to perform non- non- con-parametric predictions and derives the mntropy-normalizer distance features. The final prediction is generated by fusing multiple LinearGNN predictions on each node with an attention learned based on the distance features.
GRAPHANY: FULLY-INDUCTIVE NODE CLASSIFICATION ON ANY GRAPH
Our goal is to devise fully-inductive model that can perform inductive inference any graph with arbitrary feature and label spaces, typically different from the ones associated with the training graph (Figure 2p. Here we propose such solution GraphAny, which consists of two main components (Figure 3 LinearGNN and an attention module. Each LinearGNN provides a basic solution to inductive inference on new graphs with arbitrary feature and label spaces, while the attention module leams to combine multiple LinearGNNs based on inductive features that generalize to new graphs
Formally, in a cemi-supervised node classification task, we are given a graph 9 (V,E). typically represented as an adjacency matrix A, and node features X E M set of labeled nodes V. and their labels Yr. Rivxxxe, where is the number of unique label classes. The goal of node classification to predict the labels Y for all the unlabeled nodes Vu the conventional transductive learning setup, this is performed by training GNN (e.g. GCN (Kipl) and Welling, 2017), GAT (Velickovic et al. 2018)) on the subset of labeled nodes using standard backpropagation requiring multiple gradient steps. Such a GNN assumes the full graph to be given and typically does not generalize to a new graph out of the box without some forms re-training or fine-tuning. Conversely, a fully-inductive model is expected to predict the labels Y for any graph without expensive gradient steps. Furthermore, when a new graph provided, it might have different dimensionality d' of the features and number of class labels, c', which can also appear in an arbitrary permuted order.
V2 in the graph. In
3.1 INDUCTIVE INFERENCE WITH LINEARGNNS
A key idea of this paper is to use simple GNN models whose parameters can be expressed analytically, Following existing works that simplifies GNN (Wu al. 2019 Zhu and Koniusz [022] Yoo et 2023) by removing non-linearity, we leverage graph convolutions to process node features, followed by linear layer to predict the labels,
(aftmax(FFW))
where --=A*X Rivixd is the processed features and W Rdxe the weight of the linear layer. Originally, the parameters of most existing GNNs are trained to minimize a cross-entropy loss on node classification, which does not have analytical solution and requires gradient descent to learn the weights. Alternatively, we propose to use a meen-squared error loss for optimizing the weights:
W = arg min |Í Yul.3
W
where use Y, denote model predictions the of labeled nodes. The benefit of this approximation is that now we have an analytical solution for the optimal weights W
W F YL,
(3)
where F7, is the pscudo inverse of Fr, and the model prediction is given by:
& F FF.YL.
4

---

Published as a conference paper at ICLR 2025
We term this architecture LinearGNN, as it approximates the prediction of linear GNNs (like SGC (Wu et al. 2019)) with a single forward pass. Its main advantage is that it requires no training, which computationally efficient (see Section 3.3) Note that the derivation of LinearGNNs is independent of the graph convolution operation, and can be applied to other linear convolution operations well. We stress that while do not expeet LinearGNNs outperform existing transductive models on node classification, they provide a simple basic module for inductive inference. In Section 3.2 we will see how combine multiple LinearGNNs create a stronger model with inductive attention that generalizes to new graphs.
makes significantly
3.2 LEARNING INDUCTIVE ATTENTION OVER LINEARGNN PREDICTIONS
Although LinearGNNs provide basic solutions to inductive inference on new graphs, the learned weights still transductive, i.e. specific to the training graph, and do not capture transferable knowledge that can be applied to unseen graphs. Besides, our experiments (Figure suggest that different graphs may require LinearGNNs with convolution operations. Hence, natural way to incorporate fully-inductive learning is to add an inductive atlention module over a set of multiple
LinearGNNs, Let (1) (2) ...., au denote the node-level attention over LinearGNNs. We generate the final prediction as a combination of all LinearGNN predictions:
Yu Σ
1 au
(5)
While there are various ways to parameterize this attention module, finding inductive solution is non-trivial since neural networks can easily fit the information specific the training graph. We no- tice a necessary property for fully-inductive func- tions that it should be robust transformations on features and labels, such as permutation or masking on some dimensions (Figure 4. This requires our attention module, to be permutation invariant and robust to dimension changes, which motivates our design of distance features and en- tropy normalization respectively.
Penmutation
Masking
[V]
IM
Figure 4: Transformations on graph features and labels: permutation (left), masking (right),
Permutation-Invaaee Attention with Distance Features. We would like to design an attention module that is permutation invariant (Bronstein et al. 2021) along the data dimension Consider new graph generated by permuting feature and label dimensions of the training graph. We expect our attention output to be invariant to these permutations in order to generate the same prediction as for the unpermutated training set.
Our idea is to construct set of permutation-invariiat invariant features, such that any attention module we build on top of these features becomes intitattiooniviiian.... Formally, the permutation matrices for feature and label dimensions are P Rdxd and Q E Rexe respectively, function f is (data) permutation-ins iontattttionnnvriianttt if:
(XP. = YL).
In our LinearGNN, the prediction Ý. as a function the new graph, is invariant to the feature permutation and equivariant the label permutation since it has the following analytical form:
{XP.Y = FLQ.Q.
Deriving feature that invariant to the label permutation requires us to cancel Q with its inverse Q A straightforward solution is to use a dot product feature for predictions on each node:
YÝ FF
(8)
It is important to distinguish between domain symmetry (in this case, permutation of the nodes of the graph) and data symmetry (permutation of the feature and label dimensions). Invariance to domain symmetry (node permutations) is provided by design in I. LinearGNN.

---

Published as a conference paper at ICLR 2025
which is invariant to both feature and permuuttoont matrices P and Q. Generally, any feature that is a linear combination of dot products between LinearGNN predictions is also permutation invariant, e.g. Euclidean distance or -nasnn-Shnnnon divergence. For a set of LinearGNN predictions
Du y6² .... "66 on a single node u, we construct the following t(t - 1) permutation-invaricc features to capture their squared distances:
! 1 " 11 10 - 12
We include detailed proof' in Appendix A An advantage of such corrmutationiivariian features is that any model taking them as imput is also permutation invariant, allowing us to use a simple model, such multi-layer perceptrons (MLP), to predict the attention scores over different LinearGNNs.
LiverbG Liber
Uneer50C3
Linaatiig
Line-5012
Leeadooc
User
UlwersSc3
urear
8-
Liear Linaaniiaa Liear
E-
Lisertics
Uirwnicc
Unswmoo22
Liseshti2
500 0uh
0:19
9.7x
1.30
510
525
0:40
3.1
110
0.91
0.75
10b
0:31 00h
6.6 3.0
Wiscorein (5 aesses)
Co 17 clyooed)
Roman 118 desses)
Ande (40 dissed)
Prodect (47 dassed
RulCony 170 dessed)
Linar
Liea
Uner5503
Ubau5021
Unwer20c
UneeSGGC
Uneriicc2
LisurtiC
Uneer GC2
UnewriCCs
Uneen OC2
Lineati022
2-
8.0
815
853
6.0
635
8:58
015 353
0.5
6.0
0:30 0.55
3.3
0.50
0.22 1000
8.00 0.2
6,50 6is
210
Wike
18.00
18.
Figure 5: Comparison of Euclidean distances (the first row) and entropy-normalized (the second row) features between five channels: F = X (Linear). F = AX (LinearSGCI), F = X (LinearSGC2), F (I A)X (LinearHGCI) and F (I A) (LineaHHGC2) with A denoting the row normalized adjaceny matrix. noroopy-normalized features are on the same scale and exhibit transferrable patterns across datasets.
Robust Dimension Generalization with Entropy Normalization. While the distance features for inductive attention ensure a permuutaionniivariian attention module, the distance features are known to suffer from the curse of dimensionality, where distances between vectors with larger dimensions have smaller scales (Beyer et al., 1999). This will hamper the generalization performance when the dimensions of label spaces vary across training and inference graphs (e.g. training on 7 classes for Cora and inference on 70 classes for FullCora). Empirically, as shown in the first row of Figure [5] the scale of Euclidean distance distributions decreases drastically when the number of classes increases, hampering the generalization ability of the attention module trained on a single graph.
naive approach is to normalize the distance-feature distributions using a hyperparameter (e.g.. temperature). However, due to the varying neighbor patterns across graphs (or even nodes (Luan ctt al. (0222), a single hyperparameter may not be suitable for all nodes and graphs. Instead, we propose an adaptive solution that normalizes distance features to consistent scale. To achieve this, we employ entropy normalization, a technique commonly used in manifold learning (Hinton and Roweis 2002 van der Maaten and Hinton 2008) to adaptively determine the similarity features. For node u. the asymmetric similarity feature between LinearGN'N predictions i and j is defined as:
exp(- 11
(1)
0(00 12/2(00 ( (2)
(10)
Σ exp(
1 U 1 (")
where o is the standard deviation of an isotropic multivariate Gaussian, determined by matching the entropy of distance distributions p(() (ja(i 1) 1 j t)} fixed hyperparameter H.
Since the similarity features are derived from distance features, they are also permutation semmutationinvariian
to the feature and label dimensions of the graph. Intuitively, this imposes soft constraint on the

---

Published as a conference paper at ICLR 2025
number of LinearGNN predictions considered similar ÿu significantly reducing the gap between training and test features. As shown in the second row of Figure [ entropy-normalized feature distributions are on consistent scales across datasets. Additionally, we observe that different types of homophilic graphs (c.g.. the citation graph Cora and the e-commerce graph Product) exhibit similar entropy-normed features. We will further verify the effectiveness of entropy normalization empirically in Section 4.4
3.3 EFFICIENT TRAINING AND FULLY-INDUCTIVE INFERENCE
In this section, we summarize how GraphAny utilizes the techniques introduced in the previous section and derive efficient fully-inductive node classification model. As shown in Figure B given any graph, GraphAny first utilizes t LinearGNNs to provide basic predictions with different channels. Then, entropy-normalizer features are computed based on the distances between these predictions, leading to t(( - 1) --dimensional features, Further, an inductive attention module So (e.g. MLP) is used compute the attention scores for fusing different predictions into final prediction (Eq. 5 Since the only trainable module of GnyphAny lies in the inductive attention module fo R TR. which is independent to feature dimension d and label dimension c, GraphAny enjoys fully-inductive inference on any graph with arbitrary feature and label dimensions.
One advantage of GraphAny is that it is more efficient than conventional graph neural networks (e.g. GCN (Kipf and Welling 2017), which is due two reasons. First, LinearGNN leverages con-parameteric graph convolution operations, which can be preprocessed and cached for all nodes on a ggaphh, reducing the optimization complexity to O(VV) compared with the complexity of O(EE)) for standard GNNs. Second, once trained, GraphAny is ready to generalize to arbitrary graphs, eliminating the need for gradient descent on test graphs.
Table 1 shows the time complexity and total wall time of GCN, LinearGNN and GraphAny. The total wall time considers all training and inference time on 31 graphs. Even without any speed optimization in our implementation, GraphAny is 2.95x faster than the optimized DGL's (Wang) jet al. 2020) GCN implementation in total time. We believe the speedup can even larger with dedicate implementations of GraphAny.
Table 1: Comparison of time complexity and wall time. Note that GCN has to be trained individually on each the 31 graphs while GraphAny only needs training graph.
Total Wall Time (31 graphs)
Model
Pre-processing
Optimization
Inference
GCN
O(EE)
O(E)
18.80 min 1.25 min (15.04x)
LinearGNN
O(EE)
o(Vil)
o(Vel
GraphAny
O(EE)
O(V2))
O(Vol
6.37 min (2.95x)
4 EXPERIMENTS
In this section, we evaluate the performance of GraphAny against both transductive and inductive methods on 31 node classification datasets (details in Appendix B. We visualize the attention of GraphAny on different datasets, shedding light on what inductive knowledge our model has learned (Section 4.3). To provide comprehensive understanding of the proposed techniques of GraphAny, we further conduct ablation studies on the enoraie-normalized feature and attention parameterization in Section 4.4 More training and implementation details are provided Appendix C
4.1 EXPERIMENTAL SETUP
Datasets. We have compiled a diverse collection of 31 node classification datasets from three sources: PyG (Fey and Lenssen 2019), DGL (Wang et al.) 2020), and OGB (Hu et al. 202 These datasets encompass a wide range of graph types including academic collaboration networks, social networks, e-commerce networks and knowledge graphs, with sizes varying from few hundreds to few millions of nodes. The number of classes across these datasets ranges from 2 to 70. Detailed statistics for each dataset provided in Appendix B
7

---

Published as a conference paper at ICLR 2025
100
Model
80
Grophley (Wlccosed) rain nodes 611,77 baikko mone
GON (S11 1828 mamming nodes
(S11u02 irariigg modes
Teat
Questons
CoPhyeiie
Reddll
Cacs
AmaPhoto
AmzComp
Minsswopper
Loof MAris
Tolokers
Cire
BlogCatolog
Pubmedd
WkicS
Toxod
Wissondin
DBLP
Citesser
Ganall
Roman
Will
Products
Chamelean
Aodiv
Cur
Deszer
Airus
AmdRating
Squinl
AlEUU
Actor
Average
Dataset
Figure 6: Inductive test accuracy (%) of GraphAny pre-trained using 120 labeled nodes of the Wisconsin dataset on 30 diverse graphs. Baseline methods are trained individually each graph (51lk labeled nodes in total). GraphAny is slightly better than the basclines in average performance.
Implementation Details. For GraphAny, employ 5 LinearGNNs with different graph convolution operations: F = X (Linear), F AX (LinearSGC1), = A (LinearSGC2), F= (I X (LinearHGCI) and F matrix, which cover identical, low-pass and high-pass spectral filters. In our experiments, we consider 4 GraphAny models trained separately on datasets respectively: Cora (homophilic, small), Wisconsin (heterophilic, small), Arxiv (homophilic, medium), and Products (homophilic, large). The remaining 27 datasets are held from these training sets, ensuring that the evaluations on these datasets are conducted in fully-inductive manner. More implementation details can be found at Appendix C
2 (LinearHGC2) with A denoting the normalized adjaceny
Baselines. As there are no existing fully-inductive node classification baselines, we include non- parametric methods (models without learnable parameters) like label propagation (Zhu and Ghahra mani 2002) and the five LinearGNNs used in GraphAny. While these methods perform inductive inference, they do not transfer knowledge across graphs Additionally, we compare GraphAny with transductive models, including MLP, GCN (Kipf and Welling 2017), and GAT (Velickovic jet al. (2018). These models are trained separately on each dataset and serve as strong baselines for inductive models, as they benefit from backpropagation labeled nodes and hyperparameter tuning on validation sets of the test dataset.
4.2 PERFORMANCE OF INDUCTIVE NODE CLASSIFICATION
Table presents the results GraphAny and various baselines on 31 node classification datasets (complete results for each dataset are provided in Appendix D). Our proposed LinearGNNs, despite being non-parametric, demonstrate competitive performance. Notably, LinearSGC2, a linear model with two-hop graph convolution layer, achieves only 2.1% lower accuracy than GCN, which aligns with previous findings that SGC performs comparably to GCN (Wu et al., 2019). Moreover, LinearSGC2 leverages an analytical solution for inference, making approximately 15x faster than training a GCN from scratch on each dataset (see Table []. Additionally, we observe that the optimal LinearGNN model differs across datasets, highlighting that no single graph convolution kernel is universally effective for all graphs.
As for GraphAny, which is trained on just of the 31 graphs, it significantly outperforms Lin- carGNNs and even slightly surpasses transductive baselines that are individually trained on all 31 graphs. This improvement is primarily driven by inductive generalization, as GraphAny achieves its strongest performance on the 27 held-out (fully-inductive) datasets rather than the 4 training (transductive) datasets. A closer examination of Figure 6 shows that GraphAny performs well on both homophilic and heterophilic graphs in an inductive manner. We attribute this to the inductive attention module, which adaptively fuses predictions from different graph convolution kernels for each node. Interestingly, trained on small datasets (c.g.. Cora and Wisconsin) and large datasets (c.g., Arxiv and Products). We hypothesize that even small datasets contain sufficiently diverse local node patterns (e.g., homophily
also observe minimal performance differences between GraphAny when
8

---

Published as a conference paper at ICLR 2025
Table 2: Main experiment results (test accuracy %).
Held Out Avg. (27 graphs)
Total Avg. (31 graphs)
Category
Method
Cora
Wisconsin
Arxiv
Products
MLP
48.42anaa
66.67=351
55, 23.50=023
61.06=00s
57.09
57.20
Transductive
GCN
81.40:0000
L64
71.74=029
75.79
65.55
66.55
GAT
81.70=1443
52.94a310
73.65-a11
=9.45
65.31
67.03
LabelProp
60.30:a000
16.08:215
0,98+0.00
74.50=000
50.73
49.01
Linear
22.80.000
80,00
46.79.000
42. 10g000
57.91 20:43
$7.59_0a4
Non-parameterie
LincarSGCI
74.30:0.00
45.49
55.33-000
56.58=000
62.69=024
62.08-0.4
LinearSGC2
78.20:0000
2m
59.58:00
62.92:000
64.38
64,41=005
LineaHHC1
22.50:0000
64.32=2215
22.92=000
15.00=0000
=0.01
36,26-023
LinearHGC2
23,80:000
56,08
20.65:000
13.39:0000
35.62.00
34.70=055
GruphAny (Cora)
80.18 1013
61.18250s
58.62201s
11.60
67
67.00
Inductive (training set)
GraphAny (Wisconsin)
17.82:1115
71.77=599
57.79:0056
60.28g0..0
67.31 2o.3s
67.26_000
GraphAny (Arxiv)
19
65.10
58.682017
61.31 20:00
67.65
67.46_0m
GraphAny (Products)
79.36:023
65.89:223
$8.58:0011
61.19:0.23
67.66:0.9
0.34
Normalized
LinerrGNN Resuts
1.0 Graph/eey
Attention
52.
H
Graphhmy-Aoo
Hits00 1-0 25.
Hit085200.77
0.23
0:0
596
1:3
-cov
0.:0
8:11
0:11
0:6
-ure
0.00
0.06
1011
0:43
FCcr
0.23
027
5.75
1:91
FGom
0:13
5:10
8:30
6:12
33c
rCom
022
0.34
0.
D0.P
0.95
0460
5:70
5.81
1.02
D3LP
8:1
816
0:27
8:17
032
0.40
201P
0.61
0.6
0.0
8.35
0.27
Adeera
0:31
0.2
E/b
5:21
1.25
Wi55
0:15
612
9:21
036
WAGS
0.01
10:6
0.38
CA
Prokuce
0.34
0:21
6:05
1.28
Phoduds
10:13
815
8:23
8:79
100
Protuda
0.16
0.14
022
00:30
10:44
Chomawon
0:47
5:44
1.02
Chemelan
0::11
8:15
8:33
Chomiees
0.00
3.00
0:14
0:33
0.0
00:1
5:27
5:37
1:25
0.07
E.0
6:27
8:17
0.00
10.44
Quaters
0:23
0:5
3.4
1,00
1.25
1.22
-07
0.00
$:00
8:29
0:78
0.7
0:36
0.50
Queaiora
13:0
0.05
0.00
0114
12.34
0:45
0.42
9.9
Anddenp
0:33
0.35
5.79
1.05
1.25
ArucComa
0:11
611
0.20
AnsCCmm
0.04
0.06
0.28
0.25
And Toon
3.70
3.0
1/12
0.24
ArrFPhan
811
$17
0:28
0:20
0.
AexpPhoss
3.
0.58
11:5
0.30
13:0
Wh
0:45
2:00
1:25
0.34
Wic
12
$:13
8:29
0.19
031
We
3.05
3.05
B21
0.33
Aathoul
0:43
5.25
1,32
AkSrust
6:17
021
Altrraa
0.05
0.05
0.42
1.02
0.97
10:18
0:2
8:30
0:54
0.16
3.16
1121
01:55
1.45
0.96
Squ mol
0111
8:10
03:3
8:17
1:30
0.04
3.04
18:00
0.44
Minaswopper
0:72
0.5
5:27
1:05
-64
Vireovspper
0.07
6:14
8:23
242
0.15
Wiwnwwezer
0.65
00:3
0.25
Comell
1:00
8:91
10.02
Ouroll
0:33
6:12
Camiel
1118
1.14
10:9
10:35
-0.1
Toxzs
0:93
1::0
1/00
Toeas
0.11
0:12
1100
1.06
0.34
Deeas
0:92
2:55
1:20
1.34
Desce
6:17
Deeaar
0.11
0.34
Roman
1:55
2.00
1:30
0.70
Ronan
0.00
8.10
0:12
2,5
Rontar
0.04
0.04
0.13
10:28
1100
11002
Lineer
5GC1
SIC2
11001
1922
Uneer
9901
5OC2
11222
Liner
SCC1
5002
Figure 7: Normalized performance of LinearGNNS (left; best as 1.00) and attention weights of GraphAny trained on Wisconsin (middle) and Arxiv (right) respectively. The best and second best LinearGNN performance and attention weights for each dataset are highlighted with red and purple rectangles respectively. The learned inductive attention of GraphAny successfully identifies the best-performing LinearGNN for most datasets.
and heterophily) (Luan et al.) 2022). enabling GraphAny to learn an effective node-level attention mechanism that generalizes well with a limited number of nodes (e.g., 120). A detailed analysis of the attention module is provided in Section 4.3
4.3 VISUALIZATION OF THE INDUCTIVE ATTENTION
To understand how LinearGNNs are combined in GraphAny by the inductive attention, we visualize the attention weights of GraphAny (Wisconsin) and GraphAny (Arxiv) on all datasets, averaged across nodes. For reference, we also visualize the performance of each individual LinearGNN on all datasets. As shown in Figure 7 we can see that half of the datasets are homophilic with LinearSGC2 being the optimal LinearGNN, while the other half prefers LinearHGC1, Linear or LinearSGCI. In most cases, GraphAny successfully identifies the optimal LinearGNN within its top-2 attention weights, with Hits @2 2 being 0.65 and 0.77 for GraphAny trained on Wisconsin and Arxiv respectively.
We hypothesis that this amazing inductive performance comes from the inductive entropy normed distance feature we derived, where homophilic and heterophilic graphs share different patterns (see the second row in Figure 5) Interestingly, there is a distinction between the attention distributions when training on different datasets: GraphAny-Wisconsinn Wisconsin leads a relatively balanced distribution of attention across 5 LinearGNNs. while Graph-Arxiv prefers more focused distribution of attention, favoring low- aw-pass filters like LinearSGCI and LinearSGC2. This reflects the nature of these training sets: As shown in the left part of Figure 7 all LinearGNN channels in Wisconsin reasonably good,
9

---

Published as a conference paper at ICLR 2025
Erthorn-4
Noda
Gragh
Transdactioe
Thareductive Accuracy
Intuctivn That Acouwayy
50.5
Wiler Ayrg M
39
58.6
66
$8.4
Accuracy
58.5
you
65
50
64
57.4
57.2
500
1000
1500
2000
500
1000
1500
2000
500
1000
1900
2000
500
1000
1500
2000
Training Babch
Rattch
Trwining flatch
Ratch
Figure 8: Performance of different distance fea- Figure 9: Performance of different attention pa- tures with and without entropy normalization. rameterizations.
indicating diverse message-passingg pattern exists, but Arxiv is homophilic dataset where Linear, LinearSGCI and LinearSGC2 have better performance [T These different message-passing patterns are learned and used by GraphAny to generate inductive attention scores.
4.4 ABLATION STUDIES
Entropy Normalization. A key component of GraphAny is the entropy normalization technique applied to distance features. As discussed in Figure [ entropy normalization ensures that the generated features are of consistent scale. Here, we further evaluate entropy normalization from a performance perspective. Figure S demonstrates that unnormalized features, such as Euclidean distance and Jensen- Shannon divergence, yield better test performance the transductive setting. However, their performance in the inductive setting decreases as training progresses, indicating that the model overfits to transductive information in the features, such as feature scale. In contrast, distance features normalized to the same entropy H (denoted as EntNorm-J/ H/ in the figure) achieve stable convergence in both transductive and inductive settings. Additionally, entropy normalization shows robustness to the choice of the hyperparameter entropy value H, with different selections resulting in similar convergence rates and performance.
Jensen-Shannon
Attention Parameterization. It known that the optimal wessage-passing patterns vary for different graphs Zhu et al., 2020) or even different nodes (Luan et al. 2022). Therefore, GraphAny utilizes node-level attention to adaptively combine different LinearGNN predictions. Here consider two variants of attention parameterization: (1) Graph-level attention uses distance features averaged over all nodes in a training batch, which results in the same attention for all nodes in a training batch, losing the personalization for each node. (2) Transductive attention directly parameterizes attention 9 plots the
weights as t-dimensional vector and assumes they can transfer new graphs. Figure transductive and inductive test performance curves for different attention parameterizations. We notice that transductive attention does not even learn anything useful, given its performance is worse than a single LinearSGC2 model (see Table 2 Comparing node-level attention and graph-level attention, we can see that node-level attention converges slower than graph-level attention, but results in better performance in both transductive and inductive settings. This suggests the effectiveness of learning fine- mnne-graind attention based on the local information cach node.
CONCLUSION
In this paper, we propose GraphAny, the first fully-inductive node classification model capable of performing inference any graph with arbitrary feature or label space. GraphAny is composed of two core components: LinearGNNs and an inductive attention module. LinearGNN's enable efficient inductive inference unseen graphs, while the inductive attention module learns adaptively aggregate predictions from multiple LinearGNNs. Trained on a single graph, GraphAny demonstrates strong generalization 30 new graphs, even surpassing the average performance of transductive models that are trained separately on each dataset.
10

---

Published as a conference paper at ICLR 2025
ACKNOLEDGEMENT
The authors would like to thank Dinghuai Zhang for his helpful discussions and comments. This project is supported by the Intel-MILA partnership program, the Natural Sciences and Engineering Research Council (NSERC) Discovery Grant, the Canada CIFAR Al Chair Program, collaboration grants between Microsoft Research and Mila, Tencent AI Lab Rhino-Bird Gift Fund and a NRC Collaborative R&D Project (A14DCORE-D) Fundamental Research Project grant 11--2009-333319977777 EPSRC Turing AI World-Leading Research Fellowship No. EP/X040062/1 and EPSRC AI Hub No. EP/Y028872/1. The computation resource of this project is supported by Mila, Calcul Québec and the Digital Research Alliance of Canada
This project was also partially funded by IVADO
M.B. is partially supported by the
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 2303:087777 2023.
Ravichandra Addanki, Peter W Battaglia, David Budden, Andreea Deac, Jonathan Godwin, Thomas Keck, Wai Lok Sibon Li, Alvaro Sanchez- -oozzlzzG, Jacklynn Stott, Shantanu Thakoor, et al. Large-scale graph representation learning with very deep gnns and self-supervision. arXiv preprint arXiv:2107.09422. 2021.
Ilyes Batatia, Philipp Benner, Yuan Chiang, Alin M. Elena, Dávid P. Kováes, Janosh Riebesell, Xavier R. Advincula, Mark Asta, Matthew Avaylon, William J. Baldwin, Fabian Berger, Noam Bernstein, Arghya Bhowmik, Samuel M. Blau, Vlad Cärare, James Darby, Sandip De, Fla- viano Della Pia. Volker L. Deringer, Rokas Elijošius, Zakariya El-Machachi, Fabio Falcioni, Edvin Fako, Andrea C. Ferrari, Annalena Genreith- Schheitt-Sehriieeer. Janine George, Rhys A. Goodall, Clare P. Grey, Petr Grigorev, Shuang Han, Will Handley, Hendrik H. Heenen, Kersti Hermansson, Christian Holm, Jad Jaafar, Stephan Hofmann, Konstantin S. Jakob, Hyunwook Jung, Venkat Kapil, Aaron D. Kaplan, Nima Karimitari, James R. Kermode, Namu Kroupa, Jolla Kullgren, Matthew C. Kuner, Domantas Kuryla, Guoda Liepuoniute, Johannes T. Margraf, loan-Bogdes Bogdan Magdāu, Angelos Michaelides, J. Harry Moore, Aakash A. Naik, Samuel P. Niblett, Sam Walton Norwood, Niamh O'Neill, Christoph Ortner, Kristin A. Persson, Karsten Reuter, Andrew S. Rosen, Lars L.. Schaaf, Christoph Schran, Benjamin X. Shi, Eric Sivonxay, Tamás K. Stenezel, Viktor Svahn, Christopher Sutton, Thomas D. Swinburne, Jules Tilly, Cas van der Oord, Eszter Varga-Umbriche Tejs Vegge, Martin Vondrák, Yangshuai Wang, William C. Witt, Fabian Zills, and Gábor Csányi. A foundation model for atomistic materials chemistry. arXiv preprint arXiv:2401.00 000009,,, 2024.
Kevin S. Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. When "nearest neighbor" meaningful? In Catriel Beeri and Peter Buneman, editors, Database Theory ICDT '99, 7th International Conference, Jerusalem, Israel, January 10-12, 1999, Proceedings, volume 1540 of Lecture Notes in Computer Science, pages 217-235. Springer, 1999, doi: 10. 10.100773-54044925777 _15.
Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. International Conference on Learning Representations, 2018.
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint 11t::2204413477 2021.
Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang. Graphllm: Boosting graph reasoning ability of large language model. arXiv preprint arXiv:2310.05845, 2023.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS, 2016.
Kaiwen Dong, Haitao Mao, Zhichun Guo, and Nitesh V. Chawla. Universal link predictor by in-context learning graphs. arXiv preprint arXiv:2402.07738. 2024.
11

---

Published as a conference paper at ICLR 2025
Bahare Fatemi, Jonathan Halerow, and Bryan Perozzi. Talk like graph: Encoding graphs for large language models. In The Twelfth International Conference on Learning Representations, 2024.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric, 2019.
Mikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang, and Zhaocheng Zhu. Towards foundation models for knowledge graph reasoning. In The Twelfth International Conference Learning Representations, 2024.
William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.
Mengyue Hang. Jennifer Neville, and Bruno Ribeiro. A collective learning framework to boost gnn expressiveness for node classification. Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 4040 4050. PMLR, 2021.
Geoffrey E Hinton and Sam Roweis. Stochastic neighbor embedding. Advances in neural information processing systems, 15, 2002.
Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. Ogb-Ise: large-scale challenge for machine learning graphs. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 2021a.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning graphs, 2021b.
Hyosoon Jang, Seonghyun Park, Sangwoo Mo, and Sungsoo Ahn. Diffusion probabilistic models for structured node classification. Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR,, 2017.
Kerstin Kläser, Blažej Banaszewski, Samuel Maddrell-Mander, Callum McLean, Luis Müller, Ali Parviz, Shenyang Huang, and Andrew Fitzgibbon. MiniMol: A parameter-efficient foundation model for molecular learning. arXiv preprint 2xXx::2404414986 2024.
Dávid Péter Kováes, Harry Moore, Nicholas J. Browning, Ilyes Batatia, Joshua T. Horton, Venkat Kapil, William C. Witt, Ioan-Bol Joan-Bogdan Magdäu, Daniel J. Cole, and Gábor Csányi. Mace-off23: Transferable machine learning force fields for organic molecules. arXiv preprint arXiv:2312.15211, 2023.
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In The Twelfth International Conference Learning Representations, 2024.
Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wor Wen Chang, and Doina Precup. Revisiting heterophily for graph neural networks. In NeurlPS, 2022.
Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma. Tong Zhao, Neil Shah, Mikhail Galkin, and Jiliang Tang. Graph foundation models. arXiv preprint arXiv:2402.02216. 2024.
Péter Mernyei and Cätälina Cangea. Wiki-cs: A wikipedia-basee benchmark for graph neural networks. arXiv preprint avXiv:2007.02901, 2020.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen: Geometric graph convolutional networks. In ICLR, 2020.
Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, and Jonathan Halcrow. Let your graph do the talking: Encoding structured data for 1lms. arXiv preprint 202.0:2402.05862.. 2024.
Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at evaluation of gnns under heterophily: Are we really making progress? In The Eleventh International Conference Learning Representations, 2023.
12

---

Published as a conference paper at ICLR 2025
Meng Qu, Huiyu Cai, and Jian Tang. Neural structured prediction for inductive node classification. In International Conference on Learning Representations, 2022.
Leonardo F.R. Ribeiro, Pedro H.P. Saverese, and Daniel R. Figueiredo. struc2vec: Learning node representations from structural identity. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, 2017.
Benedek Rozemberezki, Carl Allen, and Rik Sarkar. Journal of Complex Networks, 9(2), 2021.
Multi-Scale Attributed Node Embedding.
Ryoma Sato. Training-free free graph neural networks and the power of labels as features. arXiv preprint arXiv:2404. (90004//9288 /9288, 2024.
Oleksandr Shchur, Maximilian Mumme, Alcksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprini arXiv: /8/ 1.05868, 2018.
Nima Shoghi, Adeesh Kolluru, John R. Kitchin, Zachary Ward Ulissi, C. Lawrence Zitnick, and Brandon M Wood. From molecules to materials: Pre-training large generalizable models for atomic property prediction. In The Twelfth International Conference on Learning Representations, 2024.
Maciej Sypetkowski, Frederik Wenkel, Farimah Poursafaei, Nia Dickson, Karush Suri, Philip Frad- kin, and Dominique Beaini. On the scalability of gnns for molecular graphs. arXiv preprint arXiv:2404. 1568, 2024.
Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. arXiv preprint arXiv:2310.13023 2023.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Ilama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):2579-2600 2008
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.
Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:7909.000 0155, 2020.
Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Weinberger. Simplifying graph convolutional networks. In ICML, 2019.
Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, Weiwei Deng, Qi Zhang. Lichao Sun, Xing Xie, and Senzhang Wang. A comprehensive study on text-attributed graphs: Benchmarking and rethinking. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 17238-17264. Curran Associates, Inc., 2023.
Renchi Yang, Jieming Shi, Xiaokui Xiao, Yin Yang. Sourav S Bhowmick, and Juncheng Liu, Pane: scalable and effective attributed network embedding. The VLDB Journal, (2(6):1237-133) 2023.
Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In Proceedings of The 33rd International Conference on Machine Learning. pages 40-48. PMLR, 2016.
13

---

Published as a conference paper at ICLR 2025
Jaemin Yoo, Meng-Chich Chich Lee, Shubhranshu Shekhar, and Christos Faloutsos. Less is more: Slimg for accurate, robust, and interpretable graph mining. In KDD, pages 3128-3122 3139. ACM, 2023.
Duo Zhang. Xinzijian Liu, Xiangyu Zhang, Chengqian Zhang, Chun Cai, Hangrui Bi, Yiming Du, Xuejian Qin, Jiameng Huang, Bowen Li, Yifan Shan, Jinzhe Zeng, Yuzhi Zhang, Siyuan Liu, Yifan Li, Junhan Chang, Xinyan Wang, Shuo Zhou, Jianchuan Liu, Xiaoshan Luo, Zhenyu Wang, Wanrun Jiang. Jing Wu, Yudi Yang. Jiyuan Yang. Manyi Yang. Fu- Fu-Qian Qiang Gong, Linshuang Zhang, Mengchao Shi, Fu-Zhi Dai, Darrin M. York, Shi Liu, Tong Zhu, Zhicheng Zhong, Jian Lv, Jun Cheng, Weile Jia, Mohan Chen, Guolin Ke, Weinan E, Linfeng Zhang, and Han Wang. Dpa-2: Towards universal large atomic model for molecular and material simulation. arXiv preprint arXiv:2312.1549,, 2023.
Muhan Zhang. Pan Li, Yinglong Xia, Kai Wang, and Long Jin. Labeling trick: A theory of using graph neural networks for multi-node representation learning. In Advances in Neural Information Processing Systems, volume 34, pages 9061-9073, 2021.
Jianan Zhao, Le Zhuo, Yikang Shen, Meng Qu, Kai Liu, Michael Bronstein, Zhaocheng Zhu, and Jian Tang. Graphtext: Graph reasoning in text space. arXiv preprint arXiv:2310.01089, 2023.
Hao Zhu and Piotr Koniusz. Simple spectral graph convolution. In ICLR. OpenReview.net, 2021.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In NIPS, 2020.
Xiaojin Zhu and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propaga- tion. Technical Report CMU-CALD-02-107, 2002.
Yun Zhu, Yakke Wang, Haizhou Shi, Zhenshuo Zhang, Dian Jiao, and Siliang Tang. Graphcontrol: Adding conditional control to universal graph pre-trained models for graph domain transfer learning. In WWW, 2024.
14

---

Published as a conference paper at ICLR 2025
A PROOF OF FEATURE AND LABEL PERMUTATION INVARIANCE
In this section, we prove that the proposed distance-based features have the desiring property of feature- and permmtttnn invariance.
A.1 LABEL-PERMUTATION INVARIANCE
Here, show the label-permutation invariance of the distance of LinearGNN predictions. Le. for any two LinearGNNs and j. the (squared) distances between the permuted predictions, denoted as | (0)
)1 and the distances between the original predictions, i.c. | g are the same.
Consider permuting the label dimension, i.. right multiplicate permutation matrix Q E IRe the LinearGNN prediction y = FFF Yz. (Eq 4 The permutated label logits Y can be expressed as:
FF
(11)
Given the linearity of matrix multiplication, we have:
==
(12)
Thus, label-permutation equivariance is proved. Since the permutation matrix Q is orthonormal (i.e. Q = D), pairwise distances between permuted logits remain the same as those between the original logits.
10
11
( Q" "0!l + G Q² Q00 20 Q" Q"
y (T (( (T) g( - (1) gD)
(13)
= -10.0 - 112
Hence, the distance-based feature label permutation-invarian..
A.2 PEATURE-PERMUTATTON INVARIANCE
Now, let's consider permuting the feature dimension by right multiplication with a permutation matrix P Idddd resulting in permuted features labeled nodes denoted F2 P. Since the permutation matrix P orthonormal, the pseudoinverse of F, P.PPo is:
(14)
Substituting it into the equation the predictions using permuted feature are:
F = F = FF =Y
(15)
Here, PPT I where I E Rdxdd is the identity matrix, hence proving that the predictions with Peature-permutation are identical the original predictions Y. That the predictions are feature- permutation invariant. Therefore, the distance features are also feature-permutation invariant.
B DATASETS
We collect diverse collection of 31 node classification datasets from three sources: PyG (Fey and Lenssen 2019), DGL (Wang et al., 2020), and OGB (Hu et al.) 2021b). We follow the default split if there given one, otherwise,
the standard semi-supervised setting (Kipf and Welling) 2017) where 20 nodes are randomly selected as training nodes for each label. The detailed dataset information is summarized in Table
15

---

Published as a conference paper at ICLR 2025
Table 31 datasets used in this paper. Of those, 20 are homophilic and 11 are heterophilic.
Calegory
131
1/10 1/19
Homephilkc
Riheimesi
Heterohhllic
Heteophillic
Hetroppilic
Homophilke
Homephilke
(170.0
Hatnophikke
340
94430
Bangphilic
10856
5.218 536.
Uneseer
3703
120
3.V15 IV3L
Homoptilic
WeeCanllo
3189
120
Homoptille
Squend
5701
213075
7009
3696
48 10700
Harrophilic
Perers:
Arter
7600
30019
912
1646
45 0232 1030.0
Harmopilic
Perers
LaniFM Asis
7624
55612
126
18
360
75477777 6447.6
Hamsptilic
Rowamber Sew.ereeer
AnuPhoto
7650
238162
745
160
2.1449
Homaptilic
Minowwcpprr
10000
75904
2
5000
$0,025.002 0255.0
Heteropailic
P Pil
10.4
WIKCS
11791
431256
300
10
580
S.VIS, 1.9..
Honophilic
Mam
10.0
Tainkn
11758
10139900
10
5679
50 SOIRSS (P75.
Heterophilic
peamer 213
AnoCamp
19752
491722
767
10
200
1.9 349..5
Homaphilic
3375
DELP
17716
165734
1639
0U544 8499.
Honsphilic
Beychewel and Camrensnn
C.CS
18333
163758
6905
15
300
1040 2440
Hansphilic
Pubnee
19717
83648
300
60
0.592.955.
Hansphilic
FallCora
19733
125842
S710
70
1400
2.146
Homophilic
Bodceeski apd Co Morirni
Roman Enpaz
22662
65854
300
18
11531
SOANCS IV25.0
Heterophilic
Palorty 2.
2394
Ansoun Raings
022.0
Hitznophilk:
Deczer
28381
9499
Bonephilli
CaPlysco
Honophillc
Quetions
24460
025.0
Haterophilic
Wall Alre
1166243
Honophilic
Redde
233965
05/N10.222399
Honophilic
Hangtto
196615
3,0/1,6904
Homophilic
20
C IMPLEMENTATION DETAILS
All experiments were conducted using five different random seeds: (0, 1, 2, 3, 4). The best hyperparameters were selected based on the validation accuracy. The runtime measurements presented in Table were performed on an NVIDIA Quadro RTX 8000 GPU with CUDA version 12.2, supported by an AMD EPYC 7502 32-Core Processor that features 64 cores and a maximum clock speed of 2.5 GHz. All GraphAny experiments can be conducted on single GPU with 20GB GPU memory and 50GB CPU memory.
C.I GRAPHANY IMPLEMENTATION
Table 4: Summary of hyperparameters of GraphAny on different datasets.
Dataset
# Batches
Learning Rate
Hidden Dimension
# MLP Layers
Entropy
Cora
500
0.0002
64
1
2
Wisconsin
1000
0.0002
32
2
Arxiv 1000
0.0002
128
2
Products 1000
0.0002
128
2
1
We optimize the attention module using standard cross-entropy loss on the labeled nodes. In each training batch (batch size set as 128 for all experiments), we randomly sample two disjoint sets of nodes from the labeled nodes VL: Vier and Vinget. Vear is used for performing inference using LinearGNNs, and Vurget is utilized separation is intended to prevent the attention module from learning trivial solutions unless the number of labeled nodes too low to allow for meaningful split. Empirically, as the final attention weights of GraphAny mostly focus Linear, LinearSGCI, and LinearSGC2 (Figure 7 mask the attention for LinearHGC1 and LincarHGC2 to achieve faster convergence.
compute the loss and update the attention module. This
The hyperparameter search space for GraphAny is relatively small, we fixed the batch size as 128 and varied the number training batches with options of 500, 1000, and 1500; explored hidden dimensions of 32, 64, and 128; tested configurations with 1, 2. and MLP layers; and set the fixed entropy value H at and The optimal settings derived from this hyperparameter search space are detailed in Table 4
16

---

Published as a conference paper at ICLR 2025
C.2 BASELINE IMPLEMENTATION
We utilize the Deep Graph Library (DGL) (Wang et al. 2020) implementations of GCN and GAT for baseline models. To optimize their performance, we conducted a comprehensive hyperparameter tuning using a grid search strategy on each dataset. The search space included the following parameters: number of epochs fixed 400; hidden dimensions explored were 64, 128. and 256; number of layers tested included 1, 2, and 3; and the learning rates considered were 0.0002, 0.0005, 0.001, and 0.002.
For label propagation, we use the DGL's implementation and use grid search to find the best model with search space defined as follows: number of propagation hops of 2, and of 0.1, 0.3, 0.5, 0.7, and 0.9.
D COMPLETE RESULTS
Table 5 provides all results 31 datasets for MLP, GCN, GAT baselines trained on each dataset from scratch and four GraphAny models trained only on one graph.
Table 5: Per-dataset results of all baselines and four GraphAny models
GraphAny (Products)
GraphAny (Arxiv)
GraphAny (Wisconsin)
GraphAny (Cora)
Dataset
MLP
GCN
GAT
Actor
30
28.55
22
28.9910.61
28. 28.6020.21
10.55 10:55
16
AirBrazil
23.08
42.31
57.69+14.75
34.61+16.54
34.61
36.15#16.08
33.07=16668
AirEU
21.25=2.11
41.88=3.00
32,50+845
41.75.t6.84
41.50+6.50
41.13++602
40.50=701
AirUS
22.88=1.46
46.49=1151
48.47+4.17
43.57
43.64+1..8
43.86+1144
13.46=1145
AmzComp
58.28
85.83
87.01
82.90.1.25
83.04
82,00
12.9
AmzPhoto
68.20=00
91.88
91.86+1100
90.64
90.60+0.22
90.1840991
90.14
AmzRatings
40.90
47.35
47. 17.18
42.70.10.10
42.7410.12
42.57.0.34
42.84
BlogCatalog
11.9
71.51
61.9815.99
74.73
73.63
1..69
72.52.3.22
Chameleon
36.62=087
64.69=221
67.76+0072
62.59+0087
62.59+0.6
60.09+199
61.49=188
Citeseer
44.40=004
63.40:2063
69, 10
67. 20.29
68.34
67. 07.50+0.44
68.90=007
CoCS
85.88=093
91. 01.83+1771
88.47+079
90.46+00.44
90.45+0..9
90.85
90.47
CoPhysics
87 87.43=199 1998
93.93=007
+3.01+0.89
92.66-0.52
92.69+0.52
92.54+0.5
92.70=a54
Cora
48 63.42
11.40
81.70.1.43
79.36.102
79.38
17.12
10.18
Cornell
67.57
35.14
35.1413.52
64.86.10.0
65.94
66,49
64.86=191
DBLP
56.27=0o2
73.02+222
73. 13.37
70.62
70.90
70.13
71.73=094
Deezer
54.2412.15
53.691222
55 13.79
52.09 12.75
52. 12.79
52.13
51,9812279
LastEMAsia
81.91 1095
84. 10.66
80. 10.17..0 10.44
80.60
19.47.11.3
10.8
Minesweeper
80.00=000
81. 12
80.08+0004
80.27
80.30+o.15
80. 13:009
80.46
Pubmed
69.50=117 79
76.60
77.30
20.34
76.3620.17
77.4610.0
76.60
Questions
97.33=006
97.15
97.11
97. 97.10+0mm1
97.09+0.12
97.11+0.00
97,06
Reddit
12.16
94.89=0.02
92.76+1146
90.67
90.58+0.12
91.00+0.4
90.46=a03
Roman
65.801005
15.08
43.93
10..8
64.25
64.0610.78
64.25
Squirrel
30.36-078
47.07
46.6941.4
19.67
49.70
42.3441.46
48.4
Texas
01
51.35
54,05
73.52.12.55
72.97
73.51
11,89
Tolokers
78.161002
19.93
78.50
78.18.10.1
78.1810.04
78.2410.03
78. 78.201002
Wiki
63.79.1.77
70.09
59.63
63. 22.08... 13361
62.96
61.1014.35
6.,56
Wisconsin
66.67=3.51
37.25=164
52.94+3.10
65.89 89.293
65.10+3.22
71.7745.08
61.18=508
WikiCS
43
79.12:1045
79.27:t.0
10.01
74.95+061
73.77
74.39=071
OGBN-Arxiv
55.50=023
71.74-1129
73.65+0.11
58.58+4011
58.68
57.7940.56
58. 58.62+au5
OGBN-Products
61.06=0ow
75.79=a.12
79.455+0.59
61.19400.3
61.31+0.00
60.28+0.m
61.60=a10
FullCora
33.5410.61
61.06.1024
58.95
10.33
57.25
56.2910.17
56.73
E ADDITIONAL EXPERIMENTAL RESULTS
ABLATION STUDY ON MORE GRAPH OPERATORS
In this section, further validate the effectiveness of GraphAny by gradually adding diverse types graph convolution for propagating features. Specifically, we consider three types of graph convolutions:
17

---

Published as a conference paper at ICLR 2025
GraphhnyLinennGNN
Chisyyyye
GraphAnn-PPPR
Graph/ry Meansgg
85
66
80
Performamce
Performance
55
Performamos
GraphAny MeenAgg
56
45
40
GraphAny
MeenAgg
Figure 10: Ablation study with different graph operators (test accuracy in %): GraphAny-Lineart NinearGNN (left), GraphAny- Ahhhhbyyheee (middle), and GraphAny-PPP) (right). GraphAny is trained on the Wisconsin dataset and evaluated on all 31 datasets.
GraphAAy--LinerGGNNN gradually adds the following graph convolution operators: F X (Linear), F = (I A)X (LineaHHGCI), F (I 2) (LinearHGC2), F == AX (LinearSGCI), and F A X (LineurGGC2), with Å denoting the row normalized adjaceny matrix. In this case, when increasing high-pass and low-pass predictions are gradually added.
CraphAny-Chebyshee gradually adds the Chebyshev filters (Defferrard et al. 2016), where F()) is computed recursively by:
F()) = X,
F Lx, F(t) 2i (tt-2)
(16)
and denotes the scaled and normalized Laplacian 2L_ L 1/2 A/D 1/2 Amax
In this case, when increasing high order predictions are gradually added.
GraphAny-PPR gradually adds the personalized PageRank with increasing restart ratios: [0.01, 0.25, 0.5, 0.75, 0.99). In this case, when increasing & local predictions are gradually added.
We compare the performance of GraphAny against the baseline, which averages all channels, denoted as MeanAgg. The results are shown Figure TO where we have the following observations:
GraphAny consistently outperforms the baseline MeanAgg by significant margin, demonstrat- ing effective and consistent inductive knowledge transfer across multiple datasets with different types of graph convolutions. The performance varies across graph convolution types. For LinearGNNs, low-pass graph convolutions (LinearSGCC and LinearSGC2) significantly enhance performance. In contrast, for Chebyshev graph convolutions, adding high-order convolutions reduces performance as higher-order information introduces more noise. For personalized PageRank, adding more local channels consistently improves results.
E.2 COMPARISON AGAINST ACM-GNN
To comprehensively compare GraphAny's performance, we included ACM-GNN (Luan et al. 2022) as a baseline due to its good performance on both homophilic and heterophilic graphs. However, ACM-GNN faces scalability issues that prevent its application to large graphs. Using the authors' im- plementation we encountered out-of-memory for a GPU of 40GB on four large datasets: Questions, Reddit, Arxiv, and Product. Therefore, evaluated ACM-GNN on the remaining 27 graphs, with results reported in T
Our observations
follows: Tuning ACM-GNN is highly time-consuming, requiring 672 GPU hours on 27 graphs, while GraphAny requires only 4 GPU hours, showcasing its efficiency and the advantage of inductive inference. In terms of performance, GraphAny outperforms ACM-SGC
18

---

Published as a conference paper at ICLR 2025
and is only slightly (1-2%) below ACM-GCN. However, this slight difference is not significant disadvantage for GraphAny, given the unfair advantage transductive models have by leveraging the inductive validation set for parameter and hyperparameter tuning. as well as the substantial difference in runtime.
Table 6: Experimental results of ACM-GNNs. Hyperparameter search durations: ACM-GNNs (672 GPU hours), MLPIGCN/GAT (240 GPU hours total), GraphAny (denoted as Ours, GPU hours).
Metric
MI.P
GCN
GAT
ACM-SGC
ACM-GCN
Ours-Cora
Ours- Wis
Ours-Arxiv
Ours-Product
Heldout (25 graphs)
64.01
65.11
65.23
65.56
65.56 66. 14
Overall (27 graphs)
55.07
63,83
64,26
61.82
67.79
65.58
66.02
66.
19


Reviews:
[{"review_id": "IwyEWtfHt7", "reviewer": "Reviewer_eX9V", "summary": "This paper studies the problem of fully inductive node classification, where limited parameters are learned from one small graph, and inference other unseen graphs.  The authors propose GraphAny, which consists of two components, one is a set of linearGNNs, and the other is a learnable attention MLP function. Using pseudo-inverse, LinearGNNs directly compute the node embeddings of corresponding linearGNN channels. Then a sophisticated attention technique which has properties of permutation-invariant and robust dimension generalization is used to combine these embeddings. The extensive experiments show that GraphAny gains significant improvements over the state-of-the-art methods in many datasets.", "questions": "Most of the questions and suggestions are already mentioned in the weaknesses section. I would like to mention some minor points here.\n\n1. I would like to see more graph operations used in LinearGNN instead of just X, AX, A^2X, (I-A)X, (I-A)^2X. For example, the Chebyshev polynomial operation, the PageRank operation, normalized Laplacian operation, etc. I think more operations could provide more diverse perspectives of the graph, and thus improve the performance of GraphAny at a little extra cost.\n\n2. I doubt the time complexity in Table 1, since pseudo-inverse is used in LinearGNN, which is computationally expensive up to O(n^2d), could the authors explain this?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "1. This paper proposes a novel problem setting seemingly impractical, and provides a reasonable solution to it. Previously, I was doubtful about the feasibility of graph foundation models, since unlike in NLP and CV, graph data is more universal and diverse. The information heterogeneity between different graphs may make this fully inductive setting impossible, i.e., I didn't think the knowledge in different graphs has much in common. However, the authors provide an impressive and valid solution to this problem, which is a good contribution to the community.\n\n2. The proposed method is well-motivated and well-designed. The attention module that tackles dimensionality and permutation issues is particularly novel and interesting, with strong intuition. \n\n3. The experiments are extensive and convincing. An impressive number (31) of datasets are involved in this fully-inductive setting, and the good average score of GraphAny demonstrates its effectiveness.\n\n4. The ablation study is comprehensive and insightful. The authors provide a clear understanding of the importance of each component in GraphAny.", "weaknesses": "1. (Explainability) I didn't see any explanation of one very important question: why could the knowledge learned from one graph be transferred to another unseen and unrelated graph? The authors should provide more intuitive insights on this point. From my point of view, LinearGNNs with different graph operations may serve as probes to extract different types of intrinsic knowledge from the graph, then the permutation and dimension invariant attention module could combine this knowledge in a semantic space where the common knowledge of graphs is shared. The authors should provide more insights on this point, i.e., why it works well.\n\n2. (Experiments) Although the proposed AnyGraph shows a high average performance, it is not the best in all datasets, especially in some large datasets such as Arxiv, Reddit and Products. I don't think homophily could explain this, since AnyGraph (Arxiv) also performs poorly. The authors could provide more insights on why AnyGraph fails in these datasets, and how to possibly improve it.\n\n3. (Experiments) The transductive baselines (GCN, GAT) are not strong enough. Since the benchmark contains so many datasets ranging from highly homophily to highly heterophily, baselines [1,2,3] that could fit both homophilous and heterophilous graphs should be compared. I highly recommend the authors to add some of these baselines to make the results more convincing.\n\n\n[1] Luan, S., Hua, C., Lu, Q., Zhu, J., Zhao, M., Zhang, S., ... & Precup, D. (2022). Revisiting heterophily for graph neural networks. Advances in neural information processing systems, 35, 1362-1375.\n\n[2] Lim, D., Hohne, F., Li, X., Huang, S. L., Gupta, V., Bhalerao, O., & Lim, S. N. (2021). Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. Advances in Neural Information Processing Systems, 34, 20887-20902.\n\n[3] Zhu, J., Rossi, R. A., Rao, A., Mai, T., Lipka, N., Ahmed, N. K., & Koutra, D. (2021, May). Graph neural networks with heterophily. In Proceedings of the AAAI conference on artificial intelligence (Vol. 35, No. 12, pp. 11168-11176).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "me14EZleof", "reviewer": "Reviewer_o9Mt", "summary": "This paper tackles the issue of fully inductive graph learning by introducing GraphAny. The proposed method consists of LinearGNN to preprocess the features following the idea of SGC and attention module to transform the feature.", "questions": "1. How do you get the attention score in equation 5? Do you just sum all elements in matrix $P_u^{i}$ in equation 10? What is the learnable weight in the attention module as shown in figure 3?\n\n2. Can you further explain the experimental setting in figure 5? What does the density mean? Since the value is in the range of [0, 1], is this value normalized?\n\n3. This paper mentions that it is always possible to cheat the fully-inductive setup by training a separate instance of existing models for each test dataset (in Line 75). However, the proposed LinearGNN operates like what it just said by training a linear layer with a graph convolution operation for a test graph and the authors called this LinearGNN a non-parametric solution, or preprocessing step (in Table 1). It's hard to convince the readers that the proposed method is a fully-inductive graph learning method. Can the authors clearly differentiate your approach from the \"cheating\" setup?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper tackles a great challenging fully-inductive graph learning task.\n2. This paper introduces an inductive attention module that satisfies permutation invariance properties and generalizes to new graphs.", "weaknesses": "1. The presentation of this paper needs improvement. Many details are missing in the section of methodology. \n- The authors conduct the experimental on motivating the entropy normalization, while the experimental setup in figure 5 is not explicit. It's not suggested to specify what these methods are until the section 4.1. The authors should provide more explicit explanation of the experimental setup for Figure 5.\n- It's not clear what is the learnable parameters in the attention module and how to get the attention vector $\\alpha$. A clear description of the learnable parameters in the attention module should be added.\n- It's weird to call $y_u^{(i)}$ in equations 9 and 10 as node feature and it's more proper to describe it as label vector considering its dimensionality. \n\n2. In figure 3, the authors mention that LinearGNN is non-parametric, but LinearGNN involves the learnable weight matrix W in equation 1. It's improper to claim that LInearGNN is a non-parametric solution. The authors should revise their description of LinearGNN to avoid confusion.\n\n3. This paper mentions that it is always possible to cheat the fully-inductive setup by training a separate instance of existing models for each test dataset (in Line 75). However, the proposed LinearGNN operates like what it just said by training a linear layer with a graph convolution operation for a test graph and the authors called this LinearGNN a non-parametric solution, or preprocessing step (in Table 1). It's hard to convince the readers that the proposed method is a fully-inductive graph learning method. \n\n4. Though the authors show that GraphAny has better average performance in total 31 graphs in Table 2. However, the experimental results in Table 5 shows that GAT outperforms GraphAny in 18 out of 31 graphs, which means that the proposed method does not have advantage in the fully inductive learning setting. In addition, GAT is a baseline proposed in 2018, and many recent methods can outperform GAT in most of these graphs. \n\n5. How does the different values of t influence the performance of GraphAny on different datasets? It's better to include an ablation study on the effect of t.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vamTV0qp4Z", "reviewer": "Reviewer_GtqS", "summary": "The authors focus on addressing a challenging problem: enabling GNNs to be fully-inductive across diverse datasets. They propose a model called GraphAny. Specifically, the authors employed multiple encoders (LinearGNNs) whose parameters can be obtained analytically, allowing it to generalize across datasets with different feature and label spaces. Additionally, the authors design an attention-based, learnable MLP to capture transferable graph patterns. Extensive experiments demonstrate the model's effectiveness.", "questions": "Please see above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "S1. The authors are ambitious, tackling a highly challenging and valuable problem: designing a foundational GNN model that can generalize across diverse datasets. \n\nS2. The proposed method is ingenious. The authors introduce a LinearGNN that does not require training, enabling the model to adapt to different datasets.\n\nS3. The experimental results are powerful and impressive. \n\nS4. The authors provide the complete code, along with a well-organized README file, to support their views.", "weaknesses": "W1. In fact, the proposed LinearGNNs seem to me more like a data preprocessing method which requires no learning to unify the feature and label spaces through analytical solutions.\n\nW2. Regarding W1, the authors’ statement in the Introduction that GraphAny is the first fully-inductive method seems somewhat over-claimed. According to the views in this paper, any model that can be solved analytically (i.e., without training) could also be seem as fully-inductive. Nonetheless, this point does not negate the contribution of the attention-based component to knowledge transfer.\n\nW3. The paper does not mention some recent methods capable of achieving the fully-inductive as described, such as GraphControl [1]. \n\nW4. I suggest that the author should provide the data split ratio for downstream test datasets (it is vaguely mentioned only in the appendix). This is a crucial setting, as if my understanding is correct, the proposed method requires a certain amount of ground-truth labels to analytically solve the parameters of LinearGNNs on test datasets.\n\nW5. Based on W4, the approach in this paper seems to be semi-supervised (or fine-tuned) on downstream tasks, meaning it has access to the same amount of labeled data as other semi-supervised algorithms like GCN. Moreover, GraphAny benefits from additional prior knowledge from other datasets (i.e., the pre-training phase), making it seemingly more advantageous compared to other algorithms in experimental settings. This stands in contrast to the authors' claim that other semi-supervised algorithms have an additional advantage over GraphAny in the experimental settings.\n\nIf LinearGNNs do not require any test dataset labels to solve the parameters (i.e. completely zero-shot scenario), then W4 and W5 would not hold. I strongly recommend that the authors add further explanations in the paper to improve reader comprehension.\n\n[1] GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning, WWW24.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n9QoaXxZj2", "reviewer": "Reviewer_SumM", "summary": "The paper introduces GraphAny, a model designed for fully-inductive graph learning, where models must infer on new graphs with varying structures, features, and labels. GraphAny leverages LinearGNN for analytical graph inference, adaptable to diverse graph types. By integrating multiple LinearGNN predictions using learned inductive attention, GraphAny ensures robust generalization to new graphs. Empirical results demonstrate GraphAny's effectiveness, achieving a 67.26% average accuracy on 30 new graphs with minimal training data, outperforming both inductive and transductive baselines.", "questions": "1.  Could you compare your method with the random coefficients combination of 5 different linearGNNs?\n\n2. According to your Table 2 and Figure 7, it seems that SGC1 and SGC2 occupy a dominant position( high weight and high accuracy). Could you discuss why this happens more? Could you analyze why SGC1 and SGC2 tend to get higher weights and accuracy? Does this suggest that simpler graph convolutions are more transferable? How might this insight inform future designs of inductive graph models?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. this paper raises a more general and challenging task for graph ood, that is the fully inductive node classification, which requires the model can generalize to arbitrary graphs, involving new structures, new dimensions, and semantics for their feature and label spaces.\n\n2. this paper designs a novel method GraphAny, that integrates the multiple LinearGNN predictions and learned inductive attention, which satisfies the permutation invariant and robust to dimension changes\n\n3.  the paper gives comprehensive experiments and evaluation of various datasets, showing the effectiveness of their methods.", "weaknesses": "1. The lack of baseline. This paper seems only to compare with the test-adapted GNN models as the baselines (GCN, GAT, MLP), I am not very certain if any other GNN baselines trained on the one dataset while generalizing to more datasets, such as the LLM-based GFM[1]. \n\n2. Since your method is based on the combination of 5 different linearGNNs ($ F = X$ (Linear), $F = AX$ (LinearSGC1), $F = A^2X $(LinearSGC2), $F = (I − A)X$ (LinearHGC1) and $F = (I − A)^2X$ (LinearHGC2) ), have you ever compared your method with the random coefficients combination of them? I suggest comparing GraphAny to a baseline that uses random or fixed coefficients to combine the 5 LinearGNN components. This would help isolate the benefit of the learned inductive attention mechanism.\n\n[1] One for All: Towards Training One Graph Model for All Classification Tasks", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)