Fully-inductive Node Classification on Arbitrary Graphs

## Abstract

Abstract Foundation models that can perform inference on any new task without requiring specific training have revolutionized machine learning in vision and language applications. However, applications involving graph-structured data remain a tough nut for foundation models, due to challenges in the unique feature- and label spaces associated with each graph. Traditional graph ML models such as graph neural networks (GNNs) trained on graphs cannot perform inference on a new graph with feature and label spaces different from the training ones. Furthermore, existing models learn functions specific to the training graph and cannot generalize to new graphs. In this work, we tackle these two challenges with a new foundational architecture for inductive node classification named GraphAny. GraphAny models inference on a new graph as an analytical solution to a LinearGNN, thereby solving the first challenge. To solve the second challenge, we learn attention scores for each node to fuse the predictions of multiple LinearGNNs. Specifically, the attention module is carefully parameterized as a function of the entropy-normalized distance-features between multiple LinearGNNs predictions to ensure generalization to new graphs. Empirically, GraphAny trained on the Wisconsin dataset with only 120 labeled nodes can effectively generalize to 30 new graphs with an average accuracy of 67.26% in an inductive manner, surpassing GCN and GAT trained in the supervised regime, as well as other inductive baselines.

## 1 Introduction

Figure: Figure 1: Average performance on 31 datasets. GraphAny is trained on a single dataset (Wisconsin or Arxiv) and performs inductive inference on any graph. The other transductive methods need to be trained on each dataset.
Refer to caption: x1.png

Foundation models [, , ] pre-trained on massive data have drastically changed the landscape of artificial intelligence by their ability to solve any new task conditioned on just a few demonstrations [, ]. So far, this ability has mainly been limited to modalities like text or images. An intrinsic property of these modalities is the presence of a shared input space across all tasks (e.g. a vocabulary of tokens or a patch of pixels), providing a natural basis for learning a single foundation model over a broad class of different tasks and generalizing to new ones. Graphs, however, do not possess such a property. Taking the node classification task as an example, each graph may have unique dimensions and semantics for its feature and label spaces (e.g. continuous or discrete), which prevents us from developing *graph foundation models* [] in the same way as for the above modalities.

Recently, several attempts have been made to leverage large language models (LLMs) to solve graph tasks [, , ]. While these methods demonstrate different ways of incorporating graphs into existing LLMs, they can only deal with text-attributed graphs, a subset of graphs whose node features and labels can be easily verbalized as textual sequences and fed into an LLM. For general graphs, node features and labels often do not have textual descriptions, and features may even be continuous, which often renders LLM-based approaches inadequate as graph foundational models.

Therefore, we are interested in an architecture that can *solve node classification on any new graph with any feature and label spaces* in a “native” manner. Such a goal is challenging for existing models on graphs due to two reasons: (1) Existing models learn transformations specific to the dimension, type, and structure of the features and labels used in the training, and cannot perform inference on feature and label spaces that are different from the training ones. This requires us to develop a new model architecture for arbitrary feature and label spaces. (2) Existing models learn functions specific to the training graph and cannot generalize to new graphs; this calls for an inductive function that can generalize to any graph once it is trained.

#### Our Contributions

In this work, we present GraphAny, a novel pre-trained foundation model architecture that can solve node classification on any new graph. GraphAny consists of two components: a LinearGNN that performs inference on new feature and label spaces without training steps, and an attention vector for each node based on entropy-normalized distance features that ensure generalization to new graphs. Specifically, our LinearGNN models the mapping between node features and labels as a non-parameteric graph convolution followed by a linear layer, whose parameters are determined in closed form without requiring explicit training steps. While a single LinearGNN model may be far from optimal for many graphs, we employ multiple LinearGNN models with different graph convolution operators and learn an attention vector to fuse their predictions. The attention vector is carefully parameterized as a function of distance features between the predictions of LinearGNNs, which guarantees the model to be invariant to the permutations of feature and label dimensions. To further improve the generalization ability of our model, we propose entropy normalization to rectify the distance feature distribution to a fixed entropy, which reduces the effect of different label dimensions on distance features. Intuitively, the attention vector learns to select the most effective combination of LinearGNNs for each node based on their prediction distributions, reflects statistics of its local structure (e.g., homophily metrics []), and generalizes to new graphs.

Empirically, we show that GraphAny trained on any single node classification dataset can generalize to 30 new graphs of different feature and label spaces, even surpassing the average performance of GCN and GAT baselines supervisedly trained on each dataset. We attribute this surprising performance to the attention function GraphAny learns to combine LinearGNNs, which generalizes to new graphs thanks to our careful parameterization. By visualizing the attention in GraphAny, we provide insights into how the attention selects the optimal LinearGNN and how it is affected by the choice of the training graph. We further conduct ablation studies on our entropy normalization and attention parameterization to verify their effectiveness in learning inductive attention. To summarize, the contribution of GraphAny are three folds:

- •
We propose LinearGNN, an efficient architecture for inductive node classification on any graph with arbitrary feature and label spaces.
- •
We devise an inductive attention module to fuse LinearGNNs using distance features and entropy normalization, which is invariant to feature and label permutations and robust to dimension changes.
- •
Combining LinearGNNs and the inductive attention module, GraphAny trained on a single dataset can generalize to any graph, enabling the first graph foundation model for node classification.

## 2 Related Work

Inductive Node Classification.
It is common to categorize prediction tasks into *transductive* and *inductive* setups.
In the transductive (semi-supervised) node classification setup, test nodes belong to the same training graph the model was trained on. In the inductive setup, the test graph might be different.
The majority of GNNs aiming to work in the inductive setup use known node labels as features.
Hang et al. [] introduced *Collective Learning* GNNs with iterative prediction refinement. Jang et al. [] applied a diffusion model for such prediction refinement.
Structured Proxy Networks [] combined GNNs and conditional random fields (CRF) in the *structured prediction* framework.
However, the train-test setup in all those works is limited to different partitions of the same bigger graph, that is, they cannot generalize to unseen graphs with different input feature dimensions and number of classes.
To the best of our knowledge, GraphAny is the first approach for inductive node classification to do so.

Labels as Features.
Addanki et al. [] demonstrated that node labels used as features yield noticeable improvements in the transductive node classification setup on MAG-240M in OGB-LSC [].
Sato [] used labels as features for training-free transductive node classification with a specific GNN weight initialization strategy mimicking label propagation (hence only applicable to homophilic graphs).
In homophilic graphs, node label largely depends on the labels of its close neighbors whereas in heterophilic graphs, node label does not depend on the neighboring nodes.
GraphAny supports both homophilic and heterophilic graphs in both transductive and inductive setups.

Graph Foundation Models.
Inspired by the progress in language and vision foundation models, the idea of having a single model transferable to an arbitrary graph and, possibly, to different graph-level tasks, has recently been gaining increased attention in the graph learning community.
Mao et al. [] identify the main challenge of *graph foundation models* (GFM) as finding an invariant *graph vocabulary* that transfers across different graphs.
A variety of methods [, , , , ] resort to LLMs as universal featurizers by verbalizing the (sub)graph structure and corresponding task as natural language text.
Liu et al. [] further add a GNN on top of LLM features.
However, it is unclear whether the sequential nature of LLMs is suitable for graphs with permutation invariance.

Leveraging GNNs as a backbone of GFMs has been found promising in molecular learning [, ], geometric learning for crystal structures [], and machine learning potentials [, , , ] where a unified graph vocabulary exists, that is, atoms in the periodic table.
In non-featurized graphs, GNNs with *labeling tricks* [] serve as generalizable link predictors in homogeneous graphs [] and multi-relational knowledge graphs [].
GFMs for node classification on featurized graphs present a challenge due to heterogeneous feature and label spaces – GraphAny is the first approach to tackle this problem where a single model can perform node classification on unseen homophilic and heterophilic graphs with various feature types and dimensions and number of labels.

## 3 GraphAny: A Foundation Model for Node Classification

Figure: Figure 2: The goal of graph foundation model (GFM) for node classification: A GFM trained on a pre-training graph $G$ should be able to perform inference on any new graph $G^{\prime}$ with new feature and label spaces.
Refer to caption: x2.png

Our goal is to devise a graph foundation model that can perform inductive inference on any new graph with arbitrary feature and label spaces, typically different from the ones associated with the training graph (Figure 2). Here we propose such a solution GraphAny, which consists of two main components (Figure 3): a LinearGNN and an attention module. LinearGNNs provide a basic solution to inductive inference on new graphs with arbitrary feature and label spaces, while the attention module learns to combine multiple LinearGNNs based on inductive features that generalize to new graphs.

In a semi-supervised node classification task, we are given a graph ${\mathcal{G}}=({\mathcal{V}},{\mathcal{E}})$, typically represented as an adjacency matrix ${\bm{A}}$, and node features ${\bm{X}}\in{\mathbb{R}}^{|{\mathcal{V}}|\times d}$, a set of labeled nodes ${\mathcal{V}}_{L}$ and their labels ${\bm{Y}}_{L}\in{\mathbb{R}}^{|{\mathcal{V}}_{L}|\times c}$, where $c$ is the number of unique label classes. The goal of node classification is to predict the labels $\hat{{\bm{Y}}}$
for all the unlabeled nodes ${\mathcal{V}}_{U}={\mathcal{V}}\setminus{\mathcal{V}}_{L}$ in the graph. In the conventional transductive learning setup, this is performed by training a GNN (e.g. GCN [], GAT []) on the subset of labeled nodes using standard backpropagation requiring multiple gradient steps. Such a GNN assumes the full graph to be given and typically does not generalize to a new graph “out of the box” without re-training or at least fine-tuning. Conversely, a graph foundation model is expected to predict the labels $\hat{{\bm{Y}}}$ for any graph without expensive gradient steps. Furthermore, when a new graph is provided, it might have different dimensionality $d^{\prime}$ of the features and number of class labels, $c^{\prime}$, which can also appear in an arbitrary permuted order.

### 3.1 LinearGNNs for Inductive Inference on Any Graph

Figure: Figure 3: Overview of GraphAny: LinearGNNs are used to perform non-parametric predictions and derives the entropy-normalized distance features. The final prediction is generated by fusing multiple LinearGNN predictions on each node with an attention learned based on the distance features.
Refer to caption: x3.png

A key idea of this paper is to use simple GNN models whose parameters can be expressed analytically. Among existing models, simple graph convolutional networks (SGC) [] use a non-parameteric graph convolution operation to process node features, followed by a linear layer to predict the labels,

$$ $\hat{{\bm{Y}}}=\mathrm{softmax}({\bm{F}}{\bm{W}}),$ (1) $$

where ${\bm{F}}={\bm{A}}^{k}{\bm{X}}\in{\mathbb{R}}^{|{\mathcal{V}}|\times d}$ is the processed features and ${\bm{W}}\in{\mathbb{R}}^{d\times c}$ is the weight of the linear layer. Originally, the parameters of SGC were trained to minimize a cross-entropy loss on node classification, which does not have analytical solution and requires gradient steps. However, if we assume the label distribution to be Gaussian, minimizing the cross-entropy loss is equivalent to minimizing the mean-squared error loss []:

$$ ${\bm{W}}^{*}=\operatorname*{arg\,min}_{{\bm{W}}}\|\hat{{\bm{Y}}}_{L}-{\bm{Y}}_ {L}\|^{2},$ (2) $$

where we use $\hat{{\bm{Y}}_{L}}$ to denote model predictions on the set of labeled nodes. The benefit of this approximation is that now we have a closed-form solution for optimal weights ${\bm{W}}^{*}$:

$$ ${\bm{W}}^{*}={\bm{F}}_{L}^{+}{\bm{Y}}_{L},$ (3) $$

where ${\bm{F}}_{L}^{+}$ is the pseudo inverse of ${\bm{F}}_{L}$, and the model prediction is given by:

$$ $\hat{{\bm{Y}}}={\bm{F}}{\bm{F}}_{L}^{+}{\bm{Y}}_{L}.$ (4) $$

We term this architecture LinearGNN, as it approximates the prediction of linear GNNs (like SGC) with a single forward pass. Its advantage is that it does not require training and have the same time complexity as a standard GCN at inference time (see Section 3.3). Note that the derivation of LinearGNNs is independent of the graph convolution operation, and can be applied to other linear convolution operations as well. We stress that while we don’t expect LinearGNNs to outperform existing transductive models on node classification, they provide a simple basic module for inductive inference. In Section 3.2, we will see how to combine multiple LinearGNNs to create a stronger model with inductive attention that generalizes to new graphs.

### 3.2 Learning Inductive Attention over LinearGNNs

LinearGNNs provide a basic solution to inductive inference on new graphs, but they do not learn functions from their training graphs. Besides, our experiments (Figure 6) suggest that different graphs may require LinearGNNs with different convolution operations. Hence, a natural way to incorporate learning in GraphAny is to add an attention module over a set of multiple LinearGNNs. Let $\alpha_{u}^{(1)},\alpha_{u}^{(2)},...,\alpha_{u}^{(t)}$ denote the attention over $t$ LinearGNNs. We generate the final prediction as a combination of all LinearGNN predictions:

$$ $\bar{{\bm{y}}}_{u}=\sum_{i=1}^{t}\alpha_{u}^{(i)}\hat{{\bm{y}}}_{u}^{(i)}.$ (5) $$

Figure: Figure 4: Transformations on graph features and labels: permutation (left), masking (right).
Refer to caption: x4.png

While there are various ways to parameterize this attention module, finding an inductive solution is non-trivial since neural networks can easily fit some information specific to the training graph. We notice a necessary condition for an inductive attention is that it should be robust to transformations on features and labels, such as permutation or masking on some dimensions (Figure 4). This requires our attention module to be permutation invariant and robust to dimension changes, which motivates our design of distance features and entropy normalization respectively.

Permutation-Invariant Attention with Distance Features.
We would like to design an attention module that is permutation invariant [] along the data dimension.(^1^11It is important to distinguish between domain symmetry (in this case, permutation of the nodes of the graph) and data symmetry (permutation of the feature and label dimensions). Invariance to domain symmetry (node permutations) is provided by design in our LinearGNN.) Consider a new graph generated by permuting feature and label dimensions of the training graph. We expect our attention output to be invariant to these permutations in order to generate the same prediction as for the training set.

Our idea is to construct a set of permutation-invariant features, such that any attention module we build on top of these features becomes permutation-invariant. Formally, if the permutation matrices for feature and label dimensions are ${\bm{P}}\in{\mathbb{R}}^{d\times d}$ and ${\bm{Q}}\in{\mathbb{R}}^{c\times c}$ respectively, a function $f$ is (data) permutation-invariant if:

$$ $f({\bm{X}}{\bm{P}},{\bm{Y}}_{L}{\bm{Q}})=f({\bm{X}},{\bm{Y}}_{L}).$ (6) $$

In our LinearGNN, the prediction $\hat{{\bm{Y}}}$, as a function of the new graph, is invariant to the feature permutation and equivariant to the label permutation since it has the following analytical form:

$$ $\hat{{\bm{Y}}}({\bm{X}}{\bm{P}},{\bm{Y}}_{L}{\bm{Q}})={\bm{F}}{\bm{F}}_{L}^{+} {\bm{Y}}_{L}{\bm{Q}}.$ (7) $$

Deriving a feature that is invariant to the label permutation requires us to cancel ${\bm{Q}}$ with its inverse ${\bm{Q}}^{\top}$. A straightforward solution is to use a dot product feature for predictions on each node:

$$ $\hat{{\bm{Y}}}\hat{{\bm{Y}}}^{\top}={\bm{F}}{\bm{F}}_{L}^{+}{\bm{Y}}_{L}{\bm{Y }}_{L}^{\top}({\bm{F}}_{L}^{+})^{\top}{\bm{F}}^{\top},$ (8) $$

which is invariant to both feature- and label-permutation matrices ${\bm{P}}$ and ${\bm{Q}}$. Generally, any feature that is a linear combination of dot products between LinearGNN predictions is also permutation invariant, e.g. Euclidean distance or Jensen-Shannon divergence. For a set of LinearGNN predictions $\hat{{\bm{y}}}_{u}^{(1)},\hat{{\bm{y}}}_{u}^{(2)},\cdots,\hat{{\bm{y}}}_{u}^{(
t)}$ on a single node $u$, we construct the following $t(t-1)$ permutation-invariant features to capture their squared difference:

$$ $\|\hat{{\bm{y}}}_{u}^{(1)}-\hat{{\bm{y}}}_{u}^{(2)}\|^{2},\|\hat{{\bm{y}}}_{u} ^{(1)}-\hat{{\bm{y}}}_{u}^{(3)}\|^{2},...,\|\hat{{\bm{y}}}_{u}^{(t)}-\hat{{\bm {y}}}_{u}^{(t-1)}\|^{2}.$ (9) $$

We include detailed proofs in Appendix A. An advantage of such permutation-invariant features is that any model taking them as input is also permutation invariant, allowing us to use a simple model, such as multi-layer perceptrons (MLP), to predict the attention scores over different LinearGNNs.

Robust Dimension Generalization with Entropy Normalization.
While the distance features enable us to build a permutation-invariant attention module, they still rely on the assumption that the dimensions of feature- and label-spaces do not change across graphs. In practice, the dimensions of label spaces often vary a lot across datasets (e.g. 7 classes for Cora and 70 classes for FullCora). Consequently, the distance features have substantially varying scales (e.g. 7 versus 70) for different datasets, and the maximum and minimum distance become indistinguishable when the dimensions increase []. Both seriously hamper the generalization ability of the attention module trained on a single graph. While it is possible to offset the scale with a temperature hyperparameter specified for each dataset, this is contrary to our goal of graph foundation models. Instead, we would like a solution that automatically normalizes the distance features for any label dimension.

To this end, we employ entropy normalization, a technique widely used by manifold learning methods [, ] to adaptively determine the similarity between vectors. For node $u$, the asymmetric similarity feature between LinearGNN prediction $i$ and $j$ is given by:

$$ $p_{u}(j|i)=\frac{\exp(-\|\hat{{\bm{y}}}_{u}^{(i)}-\hat{{\bm{y}}}_{u}^{(j)}\|^{ 2}/2(\sigma_{u}^{(i)})^{2})}{\sum_{k\neq i}\exp(-\|\hat{{\bm{y}}}_{u}^{(i)}- \hat{{\bm{y}}}_{u}^{(k)}\|^{2}/(\sigma_{u}^{(i)})^{2})},$ (10) $$

where $\sigma_{u}^{(i)}$ is the standard deviation of an isotropic multivariate Gaussian distribution, determined by matching the entropy of $p_{u}(j|i)$ with a hyperparameter $H$. Since the similarity features are functions of the distance features, they are also permutation invariant to the feature and label dimensions of the graph. Intuitively, this imposes a soft constraint on the number of LinearGNN predictions that are regarded as similar to the current prediction ${\bm{y}}_{u}^{(i)}$,
which significantly reduces the gap between training and test features. We will verify the effectiveness of entropy normalization in Section 4.4.

### 3.3 Analysis of Time Complexity

One advantage of GraphAny is that it is more efficient than conventional graph neural networks (e.g. GCN [], which is due to two reasons. First, LinearGNN leverages non-parameteric graph convolution operations, which can be preprocessed and cached for all nodes on a graph, reducing its inference time complexity to $O(|{\mathcal{V}}|)$. Second, the analytical solution used by LinearGNN eliminates all gradient steps, leading to a very low optimization cost for any graph $O(|{\mathcal{V}}_{L}|)$.

Table 1 shows the time complexity and total wall time of GCN, LinearGNN and GraphAny. The total wall time considers all training and inference time on 31 graphs. Even without any speed optimization in our implementation, GraphAny is 2.95$\times$ faster than GCN in total time. We believe the speedup can be even larger with dedicate implementations of GraphAny.

**Table 1: Comparison of time complexity and wall time. $P$ is the number of epochs. Note that GCN has to be trained individually on each of the 31 graphs while GraphAny only needs 1 training graph.**
| Model | Pre-processing | Optimization | Inference | Total Wall Time |
| --- | --- | --- | --- | --- |
| (31 graphs) |  |  |  |  |
| GCN | 0 | $O(|{\mathcal{E}}|P)$ | $O(|{\mathcal{E}}|)$ | 18.80 min |
| LinearGNN | $O(|{\mathcal{E}}|)$ | $O(|{\mathcal{V}}_{L}|)$ | $O(|{\mathcal{V}}|)$ | 1.25 min (15.04$\times$) |
| GraphAny | $O(|{\mathcal{E}}|)$ | $O(|{\mathcal{V}}_{L}|)$ | $O(|{\mathcal{V}}|)$ | 6.37 min (2.95$\times$) |

## 4 Experiments

In this section, we evaluate the performance of GraphAny against both transductive and inductive methods on 31 node classification datasets (Section B). We visualize the attention of GraphAny on different datasets, shedding light on what inductive knowledge our model has learned (Section 4.3). To provide a comprehensive understanding of GraphAny, we further conduct ablation studies on the proposed inductive attention and entropy normalization (Section 4.4,4.5).

### 4.1 Experimental Setup

Datasets.
We have compiled a diverse collection of 31 node classification datasets from three sources: PyG [], DGL [], and OGB []. These datasets encompass a wide range of graph types including academic collaboration networks, social networks, e-commerce networks and knowledge graphs, with sizes varying from a few hundreds to a few millions of nodes. The number of classes across these datasets ranges from 2 to 70. Detailed statistics for each dataset are provided in Appendix B.

Implementation Details. For GraphAny, we employ 5 LinearGNNs with different graph convolution operations: ${\bm{F}}={\bm{X}}$ (Linear), ${\bm{F}}={\bm{A}}{\bm{X}}$ (LinearSGC1), ${\bm{F}}={\bm{A}}^{2}{\bm{X}}$ (LinearSGC2), ${\bm{F}}=({\bm{I}}-{\bm{A}}){\bm{X}}$ (LinearHGC1) and ${\bm{F}}=({\bm{I}}-{\bm{A}})^{2}{\bm{X}}$ (LinearHGC2), which cover identical, low-pass and high-pass spectral filters. In our experiments, we consider 4 GraphAny models trained separately on 4 datasets respectively: Cora (homophilic, small), Wisconsin (heterophilic, small), Arxiv (homophilic, medium), and Products (homophilic, large). The remaining 27 datasets are held out from these training sets, ensuring that the evaluations on these datasets are conducted in a fully inductive manner. More implementation details can be found at Appendix C. Code and data is available at this link(^2^22[github.com/DeepGraphLearning/GraphAny](github.com/DeepGraphLearning/GraphAny)).

Baselines.
We compare GraphAny against conventional transductive methods, including MLP, GCN [] and GAT []. These transductive models are trained separately for each dataset and can be considered as strong baselines for evaluating inductive models. Since there have been no inductive models for node classification on any graph, we consider non-parametric methods as baselines, including label propagation (LabelProp) [] and all 5 LinearGNNs used in GraphAny. Note that these methods perform inductive inference but do not transfer any knowledge across graphs.

### 4.2 Inductive Inference on Any Graph

Figure: Figure 5: Inductive test accuracy (%) of GraphAny pre-trained using 120 labeled nodes of the Wisconsin dataset on 30 diverse graphs. Baseline methods are trained individually on each graph (511k labeled nodes in total). GraphAny is slightly better than the baselines in average performance.
Refer to caption: x5.png

Table 2 and Figure 5 show the results of GraphAny and baselines on 31 node classification datasets. We can see that our proposed LinearGNNs, with proper graph convolution operations, yield decent performance despite being non-parameteric. Specifically, LinearSGC2 is only 2.1% lower than GCN in accuracy, which coincides with the observation that SGC has performance similar to that of GCN []. However, LinearSGC2 uses an analytical solution for inference, which is about 15$\times$ faster than training a GCN from scratch on each dataset (Table  1).

By combining multiple LinearGNNs with learned attention, GraphAny further improves over LinearSGC2 by 3.0% accuracy, outperforming all transductive models. This improvement mainly comes from inductive generalization, since GraphAny achieves the best performance on the 27 held-out datasets rather than the 4 datasets we use to train GraphAny. An intriguing finding is the lack of significant difference between GraphAny trained on small datasets (e.g. Cora and Wisconsin) and large datasets (e.g. Arxiv and Products). We conjecture the reason is that even small datasets contain sufficiently diverse node patterns, allowing GraphAny to learn an effective attention mechanism.

### 4.3 Visualization of Learned Attention

Table: Table 2: Main experiment results (test accuracy %).

Figure: Figure 6: Normalized performance of LinearGNNs (left, best as 1) and attention weights of GraphAny trained on Wisconsin (middle) and Arxiv (right) respectively. The best and second best LinearGNN performance and attention weights for each dataset are highlighted with red and purple rectangles respectively. Our inductive attention can identify the best-performing LinearGNN for most datasets.
Refer to caption: x6.png

To understand how LinearGNNs are combined in GraphAny by the inductive attention, we visualize the attention weights of GraphAny (Wisconsin) and GraphAny (Arxiv) on all datasets, averaged across nodes. For reference, we also visualize the performance of each individual LinearGNN on all datasets. As shown in Figure 6, we can see that LinearSGC2 is optimal for half of the datasets, while the other half prefers LinearHGC1, Linear or LinearSGC1. In most cases, GraphAny successfully identifies the optimal LinearGNN within its top-2 attention weights, with Hits@2 being 0.65 and 0.77 for GraphAny trained on Wisconsin and Arxiv respectively.

Interestingly, there is a distinction between the attention distributions learned on the two datasets: Wisconsin leads a relatively balanced distribution of attention across 5 LinearGNNs, while Arxiv prefers a more focused distribution of attention, favoring low-pass filters like LinearSGC1 and LinearSGC2. This reflects the nature of these training sets: Wisconsin is a heterophilic dataset where all LinearGNNs are reasonably good, but Arxiv is a homophilic dataset where Linear, LinearSGC1 and LinearSGC2 have better performance (Table 2). Note that this does not necessarily mean GraphAny cannot attend to a LinearGNN that is not preferred by the training graph at node level, but the averaged attention at graph level does correlate with the distribution of the training graph.

### 4.4 Ablation Study on Entropy Normalization

One key component in GraphAny is the entropy normalization technique applied to distance features. To provide insights on how entropy normalization improves the dimension generalization ability of our method, we plot the feature distribution between the predictions of LinearSGC2 and other LinearGNNs on Cora and FullCora, which have similar semantics but different number of classes. As shown in Figure 7, there is a large difference in the scale of unnormalized feature distributions. By constrast, with entropy normalization, the feature distributions become very similar on these datasets, which facilitates inductive generalization across different datasets.

From the performance perspective, we observe a distinct trend in the generalization performance of normalized and unnormalized features. Figure 9 shows that unnormalized features, such as Euclidean distance and Jensen-Shannon divergence, provides a better test performance in the transductive setting. However, their test performance in the inductive setting diverges as training continues, indicating the model overfits the transductive information in the features, e.g. the scale of features. By contrast, entropy-normalized features achieve steady convergence in both transductive and inductive performance. Besides, the performance of entropy normalization is robust to different entropy values.

Figure: Figure 7: Cora (7 classes) vs Full Cora (70 classes). Left two plots: Without entropy normalization, there is a significant disparity in feature distributions between datasets. Right two plots: With entropy normalization applied, the disparity in feature distributions across datasets is largely reduced.
Refer to caption: x7.png

Figure: Figure 8: Performance of different distance features with and without entropy normalization.
Refer to caption: x8.png

### 4.5 Ablation Study on Attention Parameterization

Our attention is parameterized as a function of distance features between LinearGNN predictions on each node (node attention). Here we consider two variants of attention parameterization: (1) Graph attention uses distance features averaged over all nodes in a training batch, which results in the same attention for all nodes in a training batch. (2) Transductive attention directly learns attention weights as parameters similar to transductive methods, and assumes they can transfer to new graphs.

Figure 9 plots the test performance curves for different attention parameterizations. We notice that transductive attention does not even learn anything useful, given its performance is worse than a single LinearSGC2 model (cf. Table 2). Comparing node attention and graph attention, we can see that node attention converges slower than graph attention, but results in better performance in both transductive and inductive settings. This suggests the effectiveness of learning fine-grained attention for each node to fuse multiple LinearGNNs in GraphAny.

## 5 Conclusion

Limitations and Future Work.
Although GraphAny showcases the capacity of graph foundation models for node classification on any graph, there are several limitations and open questions. First, LinearGNNs have limited expressiveness and cannot fit complex functions on graphs, which may restrict the overall expressiveness of GraphAny even if we combine LinearGNNs with learned attention for each node. Second, GraphAny can only solve node classification tasks at the moment, excluding regression tasks, edge-level or graph-level tasks. Besides, it remains an open question how to apply GraphAny to relational graphs, since non-pararmetric models have never been shown effective for relational graphs. We plan to address these open questions in our future work.

Societal Impact.
GraphAny provides a efficient and effective tool for node classification on any graph. On the positive side, this largely reduces the carbon footprint of developing machine learning models on graphs. On a slightly negative side, our tool makes node classification more accessible and users can easily apply our tool to domains other than those studied in this paper.

Conclusion.
This paper presents GraphAny, a foundation model for node classification on any graph with any feature and label spaces. GraphAny consists of two main components, LinearGNNs and an inductive attention module. LinearGNNs efficiently solve inductive inference on new graphs, while the inductive attention module learns an attention to combine multiple LinearGNN predictions. By training on a single dataset, GraphAny can effectively generalize to 30 new graphs and even surpass the average performance of transductive models supervisedly trained on each dataset.

## Appendix A Proof of Feature and Label Permutation Invariance

### A.1 Label-permutation Invariance

In this section, we show the label-permutation invariance of the distance of LinearGNN predictions. I.e. for any two LinearGNNs $i$ and $j$, the (squared) distances between the permuted predictions, denoted as $\|\tilde{{\bm{y}}}_{u}^{(i)}-\tilde{{\bm{y}}}_{u}^{(j)})\|^{2}$ and the distances between the original predictions, i.e. $\|\hat{{\bm{y}}}_{u}^{(i)}-\hat{{\bm{y}}}_{u}^{(j)}\|^{2}$ are the same.

Consider permuting the label dimension, i.e. right multiplication with a permutation matrix ${\bm{Q}}\in\mathbb{R}^{c\times c}$. The permutated label logits $\tilde{{\bm{Y}}}$ can be expressed as:

$$ $\tilde{{\bm{Y}}}={\bm{F}}{\bm{F}}_{L}^{+}{\bm{Y}}_{L}{\bm{Q}}.$ (11) $$

Given the linearity of matrix multiplication, we have:

$$ $\tilde{{\bm{Y}}}={\bm{F}}{\bm{F}}_{L}^{+}({\bm{Y}}_{L}{\bm{Q}})=({\bm{F}}{\bm{ F}}_{L}^{+}{\bm{Y}}_{L}){\bm{Q}}=\hat{{\bm{Y}}}{\bm{Q}}.$ (12) $$

Thus, label-permutation equivariance is proved. As the distances between the permutation-equivariant distributions remain invariant, we have:

$$ $\displaystyle\|\tilde{{\bm{y}}}_{u}^{(i)}-\tilde{{\bm{y}}}_{u}^{(j)}\|^{2}$ $\displaystyle=\hat{{\bm{y}}}_{u}^{(i)T}{\bm{Q}}^{T}{\bm{Q}}\hat{{\bm{y}}}_{u}^ {(i)}+\hat{{\bm{y}}}_{u}^{(j)T}{\bm{Q}}^{T}{\bm{Q}}\hat{{\bm{y}}}_{u}^{(j)}-2 \hat{{\bm{y}}}_{u}^{(i)T}{\bm{Q}}^{T}{\bm{Q}}\hat{{\bm{y}}}_{u}^{(j)}$ $\displaystyle=\hat{{\bm{y}}}_{u}^{(i)T}\hat{{\bm{y}}}_{u}^{(i)}+\hat{{\bm{y}}} _{u}^{(j)T}\hat{{\bm{y}}}_{u}^{(j)}-2\hat{{\bm{y}}}_{u}^{(i)T}\hat{{\bm{y}}}_{ u}^{(j)}$ (13) $\displaystyle=\|\hat{{\bm{y}}}_{u}^{(i)}-\hat{{\bm{y}}}_{u}^{(j)}\|^{2}.$ $$

Hence, the distance-based feature is label permutation-invariant.

### A.2 Feature-permutation Invariance

Now, let’s consider permuting the feature dimension by right multiplication with a permutation matrix ${\bm{P}}\in\mathbb{R}^{d\times d}$, resulting in permuted features on labeled nodes denoted as ${\bm{F}}_{L}{\bm{P}}$. Since the permutation matrix ${\bm{P}}$ is orthonormal, the pseudoinverse of ${\bm{F}}_{L}{\bm{P}}$ is:

$$ $({\bm{F}}_{L}{\bm{P}})^{+}={\bm{P}}^{\top}{\bm{F}}_{L}^{+}.$ (14) $$

Substituting it into the equation 4, the predictions using permuted feature are:

$$ ${\bm{F}}{\bm{P}}({\bm{F}}_{L}{\bm{P}})^{+}{\bm{Y}}_{L}={\bm{F}}{\bm{P}}{\bm{P} }^{\top}{\bm{F}}_{L}^{+}{\bm{Y}}_{L}={\bm{F}}{\bm{F}}_{L}^{+}{\bm{Y}}_{L}=\hat {{\bm{Y}}}.$ (15) $$

Here, ${\bm{P}}{\bm{P}}^{\top}={\bm{I}}$ where ${\bm{I}}\in{\mathbb{R}}^{d\times d}$ is the identity matrix, hence proving that the predictions with feature-permutation are identical to the original predictions $\hat{{\bm{Y}}}$. That is, the predictions are feature-permutation invariant.

## Appendix B Datasets

We collect a diverse collection of 31 node classification datasets from three sources: PyG [], DGL [], and OGB []. We follow the default split if there is a given one, otherwise, we use the standard semi-supervised setting [] where 20 nodes are randomly selected as training nodes for each label. The detailed dataset information is summarized in Table 3.

**Table 3: 31 datasets used in this paper. Of those, 20 are homophilic and 11 are heterophilic.**
| Dataset | Nodes | Edges | Feature Dim | Num classes | Num labeled nodes | Category | Source |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Air Brazil | 131 | 1074 | 131 | 4 | 80 | Homophilic | [] |
| Cornell | 183 | 298 | 1703 | 5 | 87 | Heterophilic | [] |
| Texas | 183 | 558 | 1703 | 5 | 87 | Heterophilic | [] |
| Wisconsin | 251 | 515 | 1703 | 5 | 120 | Heterophilic | [] |
| Air EU | 399 | 5995 | 399 | 4 | 80 | Homophilic | [] |
| Air US | 1190 | 13599 | 1190 | 4 | 80 | Homophilic | [] |
| Chameleon | 2277 | 36101 | 2325 | 5 | 1092 | Heterophilic | [] |
| Wiki | 2405 | 17981 | 4973 | 17 | 340 | Homophilic | [] |
| Cora | 2708 | 10556 | 1433 | 7 | 140 | Homophilic | [] |
| Citeseer | 3327 | 9104 | 3703 | 6 | 120 | Homophilic | [] |
| BlogCatalog | 5196 | 343486 | 8189 | 6 | 120 | Homophilic | [] |
| Squirrel | 5201 | 217073 | 2089 | 5 | 2496 | Heterophilic | [] |
| Actor | 7600 | 30019 | 932 | 5 | 3648 | Heterophilic | [] |
| LastFM Asia | 7624 | 55612 | 128 | 18 | 360 | Homophilic | [] |
| AmzPhoto | 7650 | 238162 | 745 | 8 | 160 | Homophilic | [] |
| Minesweeper | 10000 | 78804 | 7 | 2 | 5000 | Heterophilic | [] |
| WikiCS | 11701 | 431726 | 300 | 10 | 580 | Homophilic | [] |
| Tolokers | 11758 | 1038000 | 10 | 2 | 5879 | Heterophilic | [] |
| AmzComp | 13752 | 491722 | 767 | 10 | 200 | Homophilic | [] |
| DBLP | 17716 | 105734 | 1639 | 4 | 80 | Homophilic | [] |
| CoCS | 18333 | 163788 | 6805 | 15 | 300 | Homophilic | [] |
| Pubmed | 19717 | 88648 | 500 | 3 | 60 | Homophilic | [] |
| FullCora | 19793 | 126842 | 8710 | 70 | 1400 | Homophilic | [] |
| Roman Empire | 22662 | 65854 | 300 | 18 | 11331 | Heterophilic | [] |
| Amazon Ratings | 24492 | 186100 | 300 | 5 | 12246 | Heterophilic | [] |
| Deezer | 28281 | 185504 | 128 | 2 | 40 | Homophilic | [] |
| CoPhysics | 34493 | 495924 | 8415 | 5 | 100 | Homophilic | [] |
| Questions | 48921 | 307080 | 301 | 2 | 24460 | Heterophilic | [] |
| Arxiv | 169343 | 1166243 | 128 | 40 | 90941 | Homophilic | [] |
| Reddit | 232965 | 114615892 | 602 | 41 | 153431 | Homophilic | [] |
| Product | 2449029 | 123718280 | 100 | 47 | 196615 | Homophilic | [] |

## Appendix C Implementation Details

All experiments were conducted using five different random seeds: {0, 1, 2, 3, 4}. The best hyperparameters were selected based on the validation accuracy. The runtime measurements presented in Table 1 were performed on an NVIDIA Quadro RTX 8000 GPU with CUDA version 12.2, supported by an AMD EPYC 7502 32-Core Processor that features 64 cores and a maximum clock speed of 2.5 GHz. All GraphAny experiments can be conducted on a single GPU with 20GB GPU memory and 50GB CPU memory.

### C.1 GraphAny Implementation

**Table 4: Summary of hyperparameters of GraphAny on different datasets.**
| Dataset | # Batches | Learning Rate | Hidden Dimension | # MLP Layers | Entropy |
| --- | --- | --- | --- | --- | --- |
| Cora | 500 | 0.0002 | 64 | 1 | 2 |
| Wisconsin | 1000 | 0.0002 | 32 | 2 | 1 |
| Arxiv | 1000 | 0.0002 | 128 | 2 | 1 |
| Products | 1000 | 0.0002 | 128 | 2 | 1 |

We optimize the attention module using standard cross-entropy loss on the labeled nodes. In each training batch (batch size set as 128 for all experiments), we randomly sample two disjoint sets of nodes from the labeled nodes ${\mathcal{V}}_{L}$: ${\mathcal{V}}_{\text{ref}}$ and ${\mathcal{V}}_{\text{target}}$. ${\mathcal{V}}_{\text{ref}}$ is used for performing inference using LinearGNNs, and ${\mathcal{V}}_{\text{target}}$ is utilized to compute the loss and update the attention module. This separation is intended to prevent the attention module from learning trivial solutions unless the number of labeled nodes is too low to allow for a meaningful split. Empirically, as the final attention weights of GraphAny mostly focus on Linear, LinearSGC1, and LinearSGC2 (Figure 6), we mask the attention for LinearHGC1 and LinearHGC2 to achieve faster convergence.

The hyperparameter search space for GraphAny is relatively small, we fixed the batch size as 128 and varied the number of training batches with options of 500, 1000, and 1500; explored hidden dimensions of 32, 64, and 128; tested configurations with 1, 2, and 3 MLP layers; and set the fixed entropy value $H$ at 1 and 2. The optimal settings derived from this hyperparameter search space are detailed in Table 4.

### C.2 Baseline Implementation

We utilize the Deep Graph Library (DGL) [] implementations of GCN and GAT for our baseline models. To optimize their performance, we conducted a comprehensive hyperparameter tuning using a grid search strategy on each dataset. The search space included the following parameters: number of epochs fixed at 400; hidden dimensions explored were 64, 128, and 256; number of layers tested included 1, 2, and 3; and the learning rates considered were 0.0002, 0.0005, 0.001, and 0.002.

For label propagation, we use the DGL’s implementation and use grid search to find the best model with a search space defined as follows: number of propagation hops of 1, 2, and 3; $\alpha$ of 0.1, 0.3, 0.5, 0.7, and 0.9.

## Appendix D Complete Results

Table 5 provides all results on 31 datasets for MLP, GCN, GAT baselines trained on each dataset from scratch and four GraphAny models trained only on one graph.

Table: Table 5: Per-dataset results of all baselines and four GraphAny models

## Peer Reviews

confidence: 4
contribution: 4
initial_rating: 6
presentation: 2
questions: Most of the questions and suggestions are already mentioned in the weaknesses section. I would like to mention some minor points here.

1. I would like to see more graph operations used in LinearGNN instead of just X, AX, A^2X, (I-A)X, (I-A)^2X. For example, the Chebyshev polynomial operation, the PageRank operation, normalized Laplacian operation, etc. I think more operations could provide more diverse perspectives of the graph, and thus improve the performance of GraphAny at a little extra cost.

2. I doubt the time complexity in Table 1, since pseudo-inverse is used in LinearGNN, which is computationally expensive up to O(n^2d), could the authors explain this?
soundness: 3
strengths: 1. This paper proposes a novel problem setting seemingly impractical, and provides a reasonable solution to it. Previously, I was doubtful about the feasibility of graph foundation models, since unlike in NLP and CV, graph data is more universal and diverse. The information heterogeneity between different graphs may make this fully inductive setting impossible, i.e., I didn't think the knowledge in different graphs has much in common. However, the authors provide an impressive and valid solution to this problem, which is a good contribution to the community.

2. The proposed method is well-motivated and well-designed. The attention module that tackles dimensionality and permutation issues is particularly novel and interesting, with strong intuition. 

3. The experiments are extensive and convincing. An impressive number (31) of datasets are involved in this fully-inductive setting, and the good average score of GraphAny demonstrates its effectiveness.

4. The ablation study is comprehensive and insightful. The authors provide a clear understanding of the importance of each component in GraphAny.
summary: This paper studies the problem of fully inductive node classification, where limited parameters are learned from one small graph, and inference other unseen graphs.  The authors propose GraphAny, which consists of two components, one is a set of linearGNNs, and the other is a learnable attention MLP function. Using pseudo-inverse, LinearGNNs directly compute the node embeddings of corresponding linearGNN channels. Then a sophisticated attention technique which has properties of permutation-invariant and robust dimension generalization is used to combine these embeddings. The extensive experiments show that GraphAny gains significant improvements over the state-of-the-art methods in many datasets.
weaknesses: 1. (Explainability) I didn't see any explanation of one very important question: why could the knowledge learned from one graph be transferred to another unseen and unrelated graph? The authors should provide more intuitive insights on this point. From my point of view, LinearGNNs with different graph operations may serve as probes to extract different types of intrinsic knowledge from the graph, then the permutation and dimension invariant attention module could combine this knowledge in a semantic space where the common knowledge of graphs is shared. The authors should provide more insights on this point, i.e., why it works well.

2. (Experiments) Although the proposed AnyGraph shows a high average performance, it is not the best in all datasets, especially in some large datasets such as Arxiv, Reddit and Products. I don't think homophily could explain this, since AnyGraph (Arxiv) also performs poorly. The authors could provide more insights on why AnyGraph fails in these datasets, and how to possibly improve it.

3. (Experiments) The transductive baselines (GCN, GAT) are not strong enough. Since the benchmark contains so many datasets ranging from highly homophily to highly heterophily, baselines [1,2,3] that could fit both homophilous and heterophilous graphs should be compared. I highly recommend the authors to add some of these baselines to make the results more convincing.


[1] Luan, S., Hua, C., Lu, Q., Zhu, J., Zhao, M., Zhang, S., ... & Precup, D. (2022). Revisiting heterophily for graph neural networks. Advances in neural information processing systems, 35, 1362-1375.

[2] Lim, D., Hohne, F., Li, X., Huang, S. L., Gupta, V., Bhalerao, O., & Lim, S. N. (2021). Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. Advances in Neural Information Processing Systems, 34, 20887-20902.

[3] Zhu, J., Rossi, R. A., Rao, A., Mai, T., Lipka, N., Ahmed, N. K., & Koutra, D. (2021, May). Graph neural networks with heterophily. In Proceedings of the AAAI conference on artificial intelligence (Vol. 35, No. 12, pp. 11168-11176).

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. How do you get the attention score in equation 5? Do you just sum all elements in matrix $P_u^{i}$ in equation 10? What is the learnable weight in the attention module as shown in figure 3?

2. Can you further explain the experimental setting in figure 5? What does the density mean? Since the value is in the range of [0, 1], is this value normalized?

3. This paper mentions that it is always possible to cheat the fully-inductive setup by training a separate instance of existing models for each test dataset (in Line 75). However, the proposed LinearGNN operates like what it just said by training a linear layer with a graph convolution operation for a test graph and the authors called this LinearGNN a non-parametric solution, or preprocessing step (in Table 1). It's hard to convince the readers that the proposed method is a fully-inductive graph learning method. Can the authors clearly differentiate your approach from the "cheating" setup?
soundness: 3
strengths: 1. This paper tackles a great challenging fully-inductive graph learning task.
2. This paper introduces an inductive attention module that satisfies permutation invariance properties and generalizes to new graphs.
summary: This paper tackles the issue of fully inductive graph learning by introducing GraphAny. The proposed method consists of LinearGNN to preprocess the features following the idea of SGC and attention module to transform the feature.
weaknesses: 1. The presentation of this paper needs improvement. Many details are missing in the section of methodology. 
- The authors conduct the experimental on motivating the entropy normalization, while the experimental setup in figure 5 is not explicit. It's not suggested to specify what these methods are until the section 4.1. The authors should provide more explicit explanation of the experimental setup for Figure 5.
- It's not clear what is the learnable parameters in the attention module and how to get the attention vector $\alpha$. A clear description of the learnable parameters in the attention module should be added.
- It's weird to call $y_u^{(i)}$ in equations 9 and 10 as node feature and it's more proper to describe it as label vector considering its dimensionality. 

2. In figure 3, the authors mention that LinearGNN is non-parametric, but LinearGNN involves the learnable weight matrix W in equation 1. It's improper to claim that LInearGNN is a non-parametric solution. The authors should revise their description of LinearGNN to avoid confusion.

3. This paper mentions that it is always possible to cheat the fully-inductive setup by training a separate instance of existing models for each test dataset (in Line 75). However, the proposed LinearGNN operates like what it just said by training a linear layer with a graph convolution operation for a test graph and the authors called this LinearGNN a non-parametric solution, or preprocessing step (in Table 1). It's hard to convince the readers that the proposed method is a fully-inductive graph learning method. 

4. Though the authors show that GraphAny has better average performance in total 31 graphs in Table 2. However, the experimental results in Table 5 shows that GAT outperforms GraphAny in 18 out of 31 graphs, which means that the proposed method does not have advantage in the fully inductive learning setting. In addition, GAT is a baseline proposed in 2018, and many recent methods can outperform GAT in most of these graphs. 

5. How does the different values of t influence the performance of GraphAny on different datasets? It's better to include an ablation study on the effect of t.

---

confidence: 4
contribution: 4
initial_rating: 6
presentation: 2
questions: Please see above.
soundness: 3
strengths: S1. The authors are ambitious, tackling a highly challenging and valuable problem: designing a foundational GNN model that can generalize across diverse datasets. 

S2. The proposed method is ingenious. The authors introduce a LinearGNN that does not require training, enabling the model to adapt to different datasets.

S3. The experimental results are powerful and impressive. 

S4. The authors provide the complete code, along with a well-organized README file, to support their views.
summary: The authors focus on addressing a challenging problem: enabling GNNs to be fully-inductive across diverse datasets. They propose a model called GraphAny. Specifically, the authors employed multiple encoders (LinearGNNs) whose parameters can be obtained analytically, allowing it to generalize across datasets with different feature and label spaces. Additionally, the authors design an attention-based, learnable MLP to capture transferable graph patterns. Extensive experiments demonstrate the model's effectiveness.
weaknesses: W1. In fact, the proposed LinearGNNs seem to me more like a data preprocessing method which requires no learning to unify the feature and label spaces through analytical solutions.

W2. Regarding W1, the authors’ statement in the Introduction that GraphAny is the first fully-inductive method seems somewhat over-claimed. According to the views in this paper, any model that can be solved analytically (i.e., without training) could also be seem as fully-inductive. Nonetheless, this point does not negate the contribution of the attention-based component to knowledge transfer.

W3. The paper does not mention some recent methods capable of achieving the fully-inductive as described, such as GraphControl [1]. 

W4. I suggest that the author should provide the data split ratio for downstream test datasets (it is vaguely mentioned only in the appendix). This is a crucial setting, as if my understanding is correct, the proposed method requires a certain amount of ground-truth labels to analytically solve the parameters of LinearGNNs on test datasets.

W5. Based on W4, the approach in this paper seems to be semi-supervised (or fine-tuned) on downstream tasks, meaning it has access to the same amount of labeled data as other semi-supervised algorithms like GCN. Moreover, GraphAny benefits from additional prior knowledge from other datasets (i.e., the pre-training phase), making it seemingly more advantageous compared to other algorithms in experimental settings. This stands in contrast to the authors' claim that other semi-supervised algorithms have an additional advantage over GraphAny in the experimental settings.

If LinearGNNs do not require any test dataset labels to solve the parameters (i.e. completely zero-shot scenario), then W4 and W5 would not hold. I strongly recommend that the authors add further explanations in the paper to improve reader comprehension.

[1] GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning, WWW24.

---

confidence: 3
contribution: 4
initial_rating: 6
presentation: 3
questions: 1.  Could you compare your method with the random coefficients combination of 5 different linearGNNs?

2. According to your Table 2 and Figure 7, it seems that SGC1 and SGC2 occupy a dominant position( high weight and high accuracy). Could you discuss why this happens more? Could you analyze why SGC1 and SGC2 tend to get higher weights and accuracy? Does this suggest that simpler graph convolutions are more transferable? How might this insight inform future designs of inductive graph models?
soundness: 3
strengths: 1. this paper raises a more general and challenging task for graph ood, that is the fully inductive node classification, which requires the model can generalize to arbitrary graphs, involving new structures, new dimensions, and semantics for their feature and label spaces.

2. this paper designs a novel method GraphAny, that integrates the multiple LinearGNN predictions and learned inductive attention, which satisfies the permutation invariant and robust to dimension changes

3.  the paper gives comprehensive experiments and evaluation of various datasets, showing the effectiveness of their methods.
summary: The paper introduces GraphAny, a model designed for fully-inductive graph learning, where models must infer on new graphs with varying structures, features, and labels. GraphAny leverages LinearGNN for analytical graph inference, adaptable to diverse graph types. By integrating multiple LinearGNN predictions using learned inductive attention, GraphAny ensures robust generalization to new graphs. Empirical results demonstrate GraphAny's effectiveness, achieving a 67.26% average accuracy on 30 new graphs with minimal training data, outperforming both inductive and transductive baselines.
weaknesses: 1. The lack of baseline. This paper seems only to compare with the test-adapted GNN models as the baselines (GCN, GAT, MLP), I am not very certain if any other GNN baselines trained on the one dataset while generalizing to more datasets, such as the LLM-based GFM[1]. 

2. Since your method is based on the combination of 5 different linearGNNs ($ F = X$ (Linear), $F = AX$ (LinearSGC1), $F = A^2X $(LinearSGC2), $F = (I − A)X$ (LinearHGC1) and $F = (I − A)^2X$ (LinearHGC2) ), have you ever compared your method with the random coefficients combination of them? I suggest comparing GraphAny to a baseline that uses random or fixed coefficients to combine the 5 LinearGNN components. This would help isolate the benefit of the learned inductive attention mechanism.

[1] One for All: Towards Training One Graph Model for All Classification Tasks

Meta-review: This paper proposes a general graph neural network that can be applied to new graphs which may have different feature and label space. Authors show that the approach outperforms a wide range of methods. While some reviewers have concerns on the selection of baseline methods and a few experimental setups, authors were able to clarify in the rebuttal. Overall, reviewers agree that the paper presents an important algorithm and is technically sound.

Decision: Accept (Poster)