iN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Authors: Nicolas Lell
Ansgar Scherp
Venue: ICML 2025 poster (2025)
Keywords: Node2Vec
Inductive
Embedding
Graph
GNN
Primary area: deep_learning->graph_neural_networks
TL;DR: We introduce iN2V, a method extending Node2Vec to generate embeddings for unseen nodes in the inductive setting.

Abstract: Shallow node embeddings like node2vec (N2V) can be used for nodes without features or to supplement existing features with structure-based information.
Embedding methods like N2V are limited in their application on new nodes, which restricts them to the transductive setting where the entire graph, including the test nodes, is available during training.
We propose inductive node2vec (iN2V), which combines a post-hoc procedure to compute embeddings for nodes unseen during training and modifications to the original N2V training procedure to prepare the embeddings for this post-hoc procedure.
We conduct experiments on several benchmark datasets and demonstrate that iN2V is an effective approach to bringing transductive embeddings to an inductive setting.
Using iN2V embeddings improves node classification by 1 point on average, with up to 6 points of improvement depending on the dataset and the number of unseen nodes.
Our iN2V is a plug-in approach to create new or enrich existing embeddings. 
It can also be combined with other embedding methods, making it a versatile approach for inductive node representation learning. 
Code to reproduce the results is available at https://github.com/Foisunt/iN2V.

V: Bringing Transductive Node Embeddings to Inductive Graphs
Nicolas Lell
Ansgar Scherp
Abstract
during training. In contrast, the transductive setting allows access to the entire graph during training, while the test node labels remain hidden. From broader perspective, the inductive setting resembles a single time step in temporal graph learning task. In temporal GNNs, the goal is to learn representations over multiple snapshots of a graph or a graph with temporal information attached to the nodes and edges, respectively (Polleres al., 2023; Longa al., 2023).
Shallow node embeddings like node2vec (N2V) can be used for nodes without features or to sup- plement existing features with structure-based in- formation. Embedding methods like N2V are limited in their application on new nodes, which restricts them to the transductive setting where the entire graph, including the test nodes, available during training. We propose inductive node2vec (iN2V), which combines a post-hoc procedure to compute embeddings for nodes unseen during training and modifications to the original N2V training procedure to prepare the embeddings for this post-hoe procedure. We conduct experiments several benchmark datasets and demonstrate that iN2V is effective approach to bringing transductive embeddings to an inductive setting. Using iN2V embeddings improves node classifi- cation by point on average, with up to 6 points of improvement depending on the dataset and the number of unseen nodes. Our iN2V is a plug-in approach to create new or enrich exist- ing embeddings. It can also be combined with other embedding methods, making it a versatile approach for inductive node representation learn- ing. Code to reproduce the results is available at https: / / htw.u..coo/Foisunttt iN2V.
Various methods exist to compute node embeddings solely from graph edges (Perozzi et al., 2014; Grover & Leskovec, 2016; Donnat et al., 2018). This is important when node features are unavailable or for enriching existing features. When using text-based embeddings such as Bag Words, some embeddings might be missing. For example, due to out-of vocabulaayy words, the Citeseer dataset (Sen al., 2008) contains 15 nodes with empty embeddings. Other reasons for new nodes include newly created social media accounts that yet lack user-provided information.
Absent node features pose a challenge for GNNs. Message- passing GNNs such as GraphS (Hamilton et al., 2017) can infer missing information from neighboring nodes. MLP-based GNNs like Graph-MLP (Hu et al., 2021) and GLNN (Zhang et al., 2022) rely solely node features, i. e., do not have message-passingg and cannot handle such cases. A common approach to addressing missing features is Feature Propagation (Rossi et al., 2022). It propagates node features along graph edges, filling in missing features while preserving existing features.
1. Introduction
We expand this idea and introduce iN2V, general and simple post-hoc approach to using trained embeddings to induce embeddings for nodes appearing in the inductive test set. We modify the training process for the popular trans- ductive embedding model node2vec (Grover & Leskovec, 2016) to foresee future embeddings. We propose a sim- ple but effective post-hoc procedure for propagating and updating embeddings for new nodes in the inductive set- ting, effectively enabling representation learning for unseen nodes. Unlike Feature Propagation (FP), our iN2V adupts the embeddings of training nodes during propagation, a crucial feature to enable the inductive setting. Since iN2V operates on N2V embeddings rather than raw node fea- tures provided by the datasets, avoids relying on external node-specific information. Furthermore, can be combined
A graph neural network (GNN) may be trained without prior knowledge about the data it will encounter after deployment. This is because, in real-world graphs, new nodes and edges appear or disappear over time. For example, papers appear in citation networks, products are added to or removed from co-purchase graphs, and users join social platforms, creating new connections. These scenarios align with the inductive setting in graph learning, where test data is entirely unseen
"Research Group Data Science and Big Data Analytics, Ulm University, Ulm, Germany. Correspondence to: Nicolas Lell <Nicolas Lelle uli-u.m.de>.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
with any existing embeddings from the datasets, enriching the node representations and improving downstream task performance. We evaluate the effectiveness of our iN2V em- beddings on range of homophilic and heterophilic datasets using the MLP and GraphSAGH models. Averaged over the other parameters, iN2V outperforms Feature Propagation by point on homophilic and 0.7 points on heterophilic datasets, 1.3 points when using MLP and 0.6 points when using GraphSAGE
similar embeddings than distant nodes, strue2vec (Ribeiro et al., 2017) focuses on neighborhood degree patterns. Ran- dom walks are done based on edge weights of fully con- nected graph, with edge weights calculated from the simi- larity of the degree distribution of the neighborhood of each node. Graph Wave (Donnat et al., 2018) treats spectral graph wavelets as distributions to provide nodes with similar struc- tural roles similar embeddings. Sub2Vec (Adhikari et al., 2018) 8) trains subgraph-level embeddings by applying para- graph2vec (Le & Mikolov, 2014) with additional random walks to better preserve the neighborhood and structural properties of the subgraphs. There are also approaches for graph-level embeddings like graph2vec (Narayanan et al., 2017). It treats subgraphs as vocabulary and applies the doc2vec skip-gram training process. Learning graph-level representations is less related to our work, which aims to learn features for nodes unseen during training.
the classification model. When using only the extended N2V embeddings, iN2V outperforms FP by 1.3 points and by 0.6 points when using both the ex- tended N2V embeddings and graph features. Finally, when using at most 20% of the nodes for embedding generation and training, iN2V outperforms FP by 1.2 points vs 0.8 points when using at least 60% of the nodes for training. In summary, our contributions are:
Introduce iN2V, simple and effective post-hoc method for extending trained node embeddings to un- seen nodes.
RDF2vec (Ristoski & Paulheim, 2016) is similar to Deep- Walk but applied to Resource Description Framework (RDF) graphs. It first converts an RDF graph into sequences and then trains word2vec (Mikolov et al., 2013) on them. There are different follow-up works for RDF2vee; for example, (Hahn & Paulheim, 2024) used RDF2vec in continual setup by sampling new walks starting from the new edges or entities. Other embedding models used for Knowledge Graphs (KGs) aim to learn not only node but also edge or relation embeddings. TransE (Bordes et al., 2013) em- bedded nodes and relations such that if there is a relation between two entities. The first entity's 's embedding added with the relation embedding is trained to be close to the sec- ond entity's ss embedding. Follow-up work (Wang et al., 2014; Trouillon et al., 2016; Sun et al., 2019) replaced the addition of real-valued vectors with other operations and other vec- tor spaces, such as Hadamard product in a complex-valle valued vector space, and investigated regularizer and the effect of inverse relations on Knowledge Graph Completion (KGC) performance (Lacroix al., 2018). ReFactorGNNs (Chen et al., 2022) try to combine the good KGC performance of these factorization-base models with the ease of fea- ture integration and inductive applicability of GNNs into a single model for KGC. FedE (Chen et al., 2021) is a fed- erated knowledge embedding framework that can use any knowledge graph embedding with multiple clients, each only having access part of the knowledge graph. This is not applicable to inductive setting, as the test graph is not seen at any time during the embeddings.
Enhance node2vec training with modifications that pre- pare their adaptability to inductive settings.
Demonstrate performance gains, showing that both the inductive extension and modified training improve classification accuracy.
Validate iN2V's s robustness, showing it remains effec- tive even when trained on only 10% of nodes; in some cases outperforming using the original dataset features.
2. Related Work
2.1. Node Embeddings without Features
Many node feature generation methods are based on ran- dom walks. DeepWalk (Perozzi et al., 2014) generates random walks from a graph, treats each random walk as a sentence, and trains word2vec (Mikolov et al., 2013) em- beddings on those random walks. Building on DeepWalk, node2vee (Grover& & Leskovec, 2016) introduces a biased random walk generator to better balance between locality and exploration by giving distinct probabilities to return to the previous node, go to node connected to the previous node, or visit a node not connected to the previous node. LINE (Tang et al., 2015) generates two sets of embeddings independently and concatenates them afterward. The first embedding optimizes that neighbors are similar, and the sec- ond embedding that nodes with many connections have sim- ilar embeddings. Another approach to node embeddings is subgraph2vec ( (Narayanan et al., 2016), which first generates rooted subgraphs for all nodes and then learns skip-gram embedding where the subgraphs of neighboring nodes are used as context for the current node. While these approaches build on the idea that neighboring nodes should have more
The Unifying Model (Jia & Benson, 2022) fits Markov Random Field to a graph, which can be similar to label propagation or a linear GCN depending on the attributes used. The model allows the sampling of new graphs from the training distribution, which differs from our task of providing embeddings to unseen nodes in graphs with no attributes.
2

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
2.2. GNNS
a)
2
3
The most well-known graph neural network is GCN (Kipf & Welling, 2017), aggregating neighbors with weights based on their degree. GraphSAGE (Hamilton al., 2017) mod- ified GCN by considering the embedding of the current node separately from the neighbor aggregation and intro- ducing sampling schemes to deal with large graphs. Other modifications of GCN used attention to assign different weights to neighbors (Velickovic et al., 2018) or make deep models easier train by adding different kinds of skip connections (Chen et al., 2020; Xu et al., 2018; San- cak et al., 2024). For efficient models, besides reducing the number of prsssage-passing layers (Wu et al., 2019), some works trained MLPs without using the edges for in- ference. Graph-MLP (Hu et al., 2021) incorporated edge information by pulling neighboring embeddings closer to- gether with a contrastive loss. GINN (Zhang et al., 2022) and NOSMOG (Tian et al., 2023) distill GNNs into MLPs. NOSMOG additionally increased robustness noise with adversarial feature augmentation and concatenates Deep- Walk embeddings to the input for capturing more structured information.
b)
GA
4
5
6
G
C)
8
11
14
9
12
15
17
10
13
16
Figure I. Three example graphs illustrate the post-hoe extensions to the white test nodes after obtaining embeddings for the gray training nodes in the inductive setup. Figure a) shows a simple example where node 2 obtains the average embedding of nodes and 3. In Figure b) node 5 got a distant embedding during embedding training, but during the post-hoc extension it should be updated to move between the embeddings of the nodes from graphs GA and Gg. Finally, Figure e) illustrates how iN2V needs only four itcrations to provide embeddings to all nodes.
Homophily is the characteristic of graph in which neigh- boring nodes share the same class. Heterophilic graphs i. e.e. graphs where neighbors usually belong to different classes, are an active area of research with work on how to measure homophily (Lim et al., 2021; Platonov et al., 2023a; Mironov & Prokhorenkova, 2024) and heterophilic datasets (Platonov et al., 2023b). Models that can better deal with heterophilic data use novel aggregations, con- sider multi-hop neighborhoods, distinguish homo- and het- erophilic edges, or make the graph more homophilic using rewiring (Zhu et al., 2020; Lim et al., 2021; Kohn et al., 2024; Al-jaa-Haija et al., 2019; Lell & Scherp, 2024; Chien et al., ; Luan et al., 2022; Bi et al., 2024).
we want to train embedding ha E TRd. In the inductive setting, only the subgraph induced by Virain is available for training the embeddings. These embeddings then need to be extended to the remaining nodes en E V Verain in the validation and test set. As we use existing benchmark datasets, our nodes also have classification labels Y NIVI and existing node features X e gV
3.2. Example
3. Inductive N2V
We motivate the different components of our post-hoc exten- sion and illustrate their effect with the three example graphs a), b), and c) in Figure 1. Training nodes are shown in gray, and test nodes in white. The latter are hidden during training in the inductive case. In graph a) it is quite straightforward that U2 should obtain the average of the embeddings of U1 and U/., In graph b), GA and GB are connected subgraphs of multiple training nodes with similar embeddings. When following the averaging idea, V/ gets an embedding that is close to the average embedding in G but skewed towards the embedding of Us. Considering that the training embed- dings were generated by N2V, the connected nodes in G' and GB got meaningful embeddings during training, while 0/ has a distant embedding as it has no neighbors in the training set and therefor only appeared as a negative sample during training. Contrary to Feature Propagation (Rossi
The principal idea of our inductive N2V (IN2V) algorithm is to simply assign each test node the average embedding of its neighbors from the training set. This is repeated for multiple iterations to also deal with test nodes with longer distances to training nodes. The N2V embedding training modified so that the embedding of training nodes are optimally prepared to induce embeddings to new nodes only seen during testing.
3.1. Notation and Formalization
Given a graph G (E, V) with node set V C N and edge set F C V x V. with disjoint training, validation, and test sets Virains Voat, Viest C V. Nv)) = {a e V (v,z) E E} is the set of neighbors of v. For each node v E V.
3

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
al., 2022), in this case, it is useful allow the adaption of input (training) embeddings. Following this line of thought while also maintaining some stability for the nodes with existing embeddings, each embedding should be a combina- tion of itself and the average neighbor embedding. When doing multiple iterations of such an averaging procedure, the embedding of vs moves in between the embeddings of nodes of GA and GB. However, too many iterations pose the challenge that all embeddings of individual nodes will converge to the average node embedding of that graph. Graph e) illustrates the challenge of extending embeddings into longer sequences of test nodes. When just averaging the neighborhood embedding for four iterations, the embed- ding of v17 = 07 v7/4 is close to zero. Feature Propagation handles this by keeping the input embeddings fixed and iter- ating many times until convergence. We already established the usefulness of adapting input embeddings in the example graph b). Therefore, we handle long sequences and high- degree nodes by considering only nodes that already have an embedding for averaging each iteration. That means that after the first iteration 0g = uy, after the second iteration Ug, 113, V10: By = 17, and so on. This leads to v17 = 07 after only four iterations.
embedding, we set e's embedding to the mean neighbor embedding (Ic). This is done for multiple iterations. After each iteration, is updated by setting entries for nodes that got an embedding to 1. We do enough iterations such that each node with a path to at least one training node gets an embedding and additional deley-many iterations to update the embeddings of nodes like vy in the example Figure I b).
3.4. Boosting Inductive Performance
We propose two different approaches which modify the training to improve the generation of inductive embeddings.
To promote embeddings that are better suited to the inductive extension of embeddings, we simulate simple version of the post-hoe extensions during training. In each epoch, some features are replaced by their mean neighborhood embedding with probability r.
Sampling-base
he = mN(()) with probability r hv else
Loss-based In addition to the sampling approach, we also introduce a loss-based approach to prepare the em- beddings for our inductive extension. When extending the trained embeddings to the inductive nodes during inference, we set the embedding of new nodes to their mean neighbor- hood embedding. The first loss promotes this relationship in the trained embeddings by pulling a node's 's own embedding closer to its mean neighborhood embedding:
3.3. Generating Inductive Embeddings
We propose an iterative algorithm to extend trained embed- dings to the unseen nodes. Let 50 be the embedding of node e after iterations of our algorithm. We use a lookup vector s E {o, 13181 with 8, being the v-th element in the vector s to keep track of which nodes already have embed- dings and use (v) (ar E N(v) 8a = 1} to denote the set of neighbors which have embedding. The mean embedding of a set of nodes S is m's = [ST Eves 1:5).
Lelose (v) = log(o (h. mMN())))
For initialization, A is set to the N2V embedding h. for training nodes u E Virain and to 0 for nodes w E V Virain
A trivial solution to minimize this loss would be to assign identical embeddings to all neighboring nodes, so we add second loss which promotes diversity in embeddings of the individual neighbors of each node:
not from the training set. The lookup vector is initialized with 8, = 1 and s = 0. Then h² calculated from (1111) by:
(v)
Σ (,WEN(o)
sim(tuu kw)
[N(P)
1
if (v) = 0 (Ia)
(0)
X 1)
(1- (1- x,)
(t-1)
if s, 1
(1b)
where sim is cosine similarity. The final loss for iN2V is
(N,())
(t-1)
else
(Ic)
((0) = (u) + a Letose + B- (u)
This means that if has no neighbor with an embedding, hy does not change (Ia). If both e and at least one neighbor of have an embedding, we calculate the convex combination of h, and the mean neighbor embedding (.) (lb). Note, for À = 1, the embedding of a node will not change once it is set, and with À= < 1 all embeddings will be updated depending on their respective neighborhoods. If v does not have an embedding but at least one neighbor has an
with hyperparameters a and B. The N2V loss Lniv is cal- culated using random walks. In each epoch, every node appears on average walks per node times walk length often in these random walks. The random walks are batched for training, but when the same node appears multiple times in a batch, the additional calculations for Cotore and Lai are redundant, To reduce these redundant calculations, we
4

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
and store embeddings for all 10 splits. Then we evaluate the embeddings using MLP and GraphSAGE (Hamilton et al., 2017). Additionally, we investigate the concatenation of the extended N2V embeddings with the original graph features.
Table 1. Graph features, nodes, edges, classes, and adjusted ho-
mophily. Upper: homophilic, bottom: heterophilic graphs.
Dataset
[
[V]
[E]
|C|
homadi
Cora
433
708
10556
0.77
Regarding the extended embeddings, we compare different setups of iN2V. The first is the "frozen" setup (A - 1), where embeddings do not change after being set. The sec- ond one is the best post-hoe setup, where À and delay are searched as hyperparameters. The third and fourth ones combine the sampling-ban based and loss-based modifications to N2V with the best post-hoc setup. We compare these results with different baselines. The comparable baselines that use the same information as iN2V are plain N2V used inductively, i. e., only training nodes have embeddings, and using Feature Propagation (Rossi et al., 2022) to extend N2V embeddings to the test nodes. Using the original graph features and training N2V embeddings in transductive setup are two more baselines. They are not directly com- parable with iN2V as they use more information but are nevertheless useful for perspective. Additional experiments with other GNNs can be found Appendix D.
CiteSeer
3 703
3327
9 104
6
0.67
PubMed
500
19717
88 648
3
0.69
Computers
767
13252
491 722
10
0.68
Photo
745
7,650
238 162
8
0.79
WikiCS
300
701
431 726
10
0.58
Actor
932
7.000
30019
5
0.01
Amazon-R.
300
242
93
5
0.14
Roman-E.
300
22 662
32 927
18
0.05
sample the nodes to calculate Leoos and Ldie independent from the random walks such that each node is the center node for these losses once per epoch.
4. Experimental Apparatus
4.1. Datasets
We use the Cora (Sen et al., 2008), CiteSeer (Sen et al., 2008), and PubMed (Namata et al., 2012) citation graphs, the Computers (Shchur et al., 2018) and Photo (Shchur et al., 2018) co-purchase graphs, and the WikiCS Wikipedia page graph (Mernyei & Cangea, 2020). These graphs are ho- mophilic, e., neighboring nodes usually share the same class. The following graphs are more heterophilic, e.e., neighboring nodes usually do not share the same class. Ac- tor (Pei et al., 2020) is Wikipedia co-occurrence graph, Amazon- ratinngs (Platonov et al., 2023b) is co-purchase graph, and Roman-empire (Platonov et al., 2023b) is a text- based graph. Table shows more details about the datasets. We report adjusted homophily accounts for class imbal- ance (Platonov et al., 2023a).
4.3. Hy Hyperparameter Optimization
We make all datasets undirected. For N2V, we use a con- text size of 10 for positive samples and 1 negative per positive sample. We use a batch size of 128 and early stopping with patience of 50 epochs. For every epoch, we sample 10 walks of length 20 per node. We do grid search over all combinations of p and q E {0.2, 1, 5), embedding size d E {64, 256). and learning rate E {0.1, 0.01, 0.001}. For the sampling based method, we try r E {0.2, 0.4, 0.6, 0.8}. The loss weights a e {o, 0.1, 1, 10) and B E {0, 0.001, 0.01, 0.1} are tuned separately from r. For Feature Propagation, we search the number of iterations in {10, 20, 40, 60).
For MLP and GraphSAGE, we use grid search for the full 10 seeds per split. We search over all combination of num- ber of layer E {I, ... 5), hidden size E (64, 512), Iearning rate E {0.01, 0.001), weight decay E {0, 0.0001, 0.01), dropout E {0.2, 0.5, 0.8). and whether to use jumping knowledge (Xu et al., 2018) connections.
4.2. Procedure
We investigate how well iN2V works for differently sized data splits into training and unseen validation or test nodes. For all datasets, we use 5 splits of different sizes that always utilize the full dataset and have validation and test set of the same size. The training set sizes are 10%, 20%, 40%, 60%, and 80%, with respective validation and test set sizes of 45%, 40%, 30%, 20%, and 10% of all nodes.
5. Results and Discussion
5.1. Key Results
First, we prepare 10 splits from different random seeds for each of the five split sizes. Then we train N2V inductively on the training subgraphs. For the N2V hyperparameter search, we use a grid search on three of the ten splits. The embed- dings from the training set are extended to the validation set using iN2V. The final N2V hyperparameters are chosen based on the validation accuracy of logistic regression on these embeddings. These hyperparameters are used to train
Comparison of iN2V vs. Baselines Table 2 shows the performance of GraphSAGE using iN2V embeddings and compares them to normal N2V in the inductive setting (test nodes have no features) and N2V embeddings extended by Feature Propagation. We also report performance on the original graph features and transductive N2V embeddings to indicate how close the baselines and iN2V are to the
5

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
ideal case of no missing nodes or features. We can see that iN2V outperforms the comparable baselines in most cases. The notable exceptions are the Actor and Roman-empire datasets, where N2V embeddings generally perform poorly.
Table 2. Accuracy of best iN2V variant vs baseline embeddings. The underlying model is GraphSAGE. Gray numbers are not di- rectly comparable as they use additional information, i. c., original features, or the transductive setup.
In some cases, using the iN2V embeddings outperforms the original features, e. g., in the 10% training splits of Cora, Computers, and Amazon- -ataing.. This implies that neigh- borhood information is more important for these datasets than the external information from the original features and that the original features do not have sufficient neighbor- hood information. Appendix B provides tables with the full results, including the 60% train split for GraphSAGE and all MLP results. We observe for some datasets that the iN2V embeddings outperform the original features for more splits when using MLP as model, as can not generate neighborhood information internally GraphSAGE does.
Percentage training data
Dataset
10%
20%
40%
80%
Cora
N2V (inductive)
42.18s.52
59.914.15
75.07
84.501.44
Feature Propagation
77.91
79. 79.482.1
81.031,85
84.132.3
IN2V (own)
78.881.45
80.94,.o 0.
83.301,m 0
84.462.09
Original features
75.272.63
83.371.17
86.231.77
87.051.20
N2V (transductive)
79.251.45
81.661.2 20
83.87
86.01
Citeseer
N2V (inductive)
34.17a.a
42.762.15
56.12..24
68.892.61
Feature Propagation
566.85..a
60.533..z
69.761.2 63.0332.21
iN2V (OWD)
57.5 97
60.78,.2v
63.02,,u0
68. 08.922.07
Original features
69.85 60.851.42
72.86o
74.931.5
76 821.77
N2V (transductive)
57.149.61
61.33
66.455.00
72.762.54
Homophilic vs Heterophilic Datasets While we gener- ally see quite good performance of models using N2V em- beddings on homophilie datasets, this is not the case for heterophilic ones. This is expected since N2V provides similar embeddings for neighbors, which matches the ho- mophilic label structure. On Actor, the models using N2V embeddings only learn to predict the largest class (2586%)) On Roman-empire, the models only predict the largest class (13.96%) when using a few training nodes and become slightly better when using most of the data during training. Roman-empire is the dataset with the lowest average de- gree in our comparison. This means the graph is prone to splitting into many disconnected subgraphs in the inductive setting. This makes this dataset challenging for training any random-walk based embeddings. It also explains the rise in performance for the largest training split. Amazon- ratings is an interesting exception to this trend because even though it has a low adjusted homophily of 0.14, N2V em- beddings seem to work as well as on the homophilic datasets and even outperform using the original features for small training splits. Table 5 paints a similar picture when using MLP instead of GraphSAGE, with the difference that even for 80% training data, the MLP does not perform better than just predicting the largest class for the Roman-empire dataset.
Pubmed
N2V (inductive)
66.02s.67
74.732.30
79.841.78
82, 62.61o..a
Feature Propagation
76.37g.62
77.72, su
80.744
82.43g.u
IN2V (own)
79.9%o.so
80.80..a a6
82. 14g.au
82.59
Original features
85.950.4
86.996.2s
08..22..u 48
89.856.50
N2V (transductive)
81.36o.a
82. 82.200..0
83 83.222..4
83. 63.666..65
Computers
N2V (inductive)
77.64g.81
84.444.s4
17.18
89.35k.77
Feature Propagation
82.799.64
866.433..a 63
89.400.45
90.877.64
IN2V (own)
88.30 36.36
89.67 so
90.844.7
91.38..51
Original features
87. 97.522.48
89. 80.76o.40 40
91.1 91.126.20
91. 50..5...
N2V (transductive)
89.18p.3s
90.16g.44
90.77a.30
91.166.50
Photo
N2V (inductive)
85.481,22 28
87-731. 40
90.98%.71
92. 92.211..9
Feature Propagation
87.43, .u
90.1 90.144,34
91.57g.a2
92.95x.7u
IN2V (own)
90.51g.72
91.70,,a 44
92.37
93. 93.08
Original features
98.740.42
94.59b.ar
95.27o.3s
95.59o.r7
N2V (transduccivee)
91, 91.29b.41
92.296. ao
92.90g..a
93.333.71
WikiCS
N2V (inductive)
67.782.0
74.22
78. 78.211..0
81.62
Feature Propagation
74.772.27
78.04
80.17
81.93, 81.931.05
iN2V (OwD)
78.91a.1
80.19
81.28
82.37,.01
Original features
80. 80.75p.64
82. 82.56g.az
84..2%..n
85.88%..7
N2V (transductive)
79.756.ar a1
80.930.0
91.88
82.8 81..0.
Actor
N2V (inductive)
25.141.09
25.566..0
25. 25.54
25.681.42
Feature Propagation
25.22, .u
25.39, us
25. 16.114..0
25.05
IN2V (own)
25.18
25.50..au
25.40g
25.79
Original features
31.77o.7
33.84g
36.488.54
36.71,.22
N2V (transductive)
25.50g.7a
25.56g.90
25. 4l..4
24.555.77
Amazon Amazon-ratings
N2V (inductive)
37.47o.a7
40.69o.ez .a
44.71
49.47
Influence of GNN Choice and N2V Modifications Ta- ble 3 shows the performance aggregated over all datasets and train splits. The first two columns show the performance impact of the loss-based and sampling-based modifications to N2V when using only those embeddings as input for the GNNs. Our post-hoc extension, which can adapt in- put embeddings, outperforms Feature Propagation and the frozen post-hoc variant. This shows that adapting existing features is an important capability of iN2v. The basse and sampling-basee based modifications to N2V training provide
Feature Propagation
38.97o.76
41.680.75
45.72g.77
50.04,.42
iN2V (own)
40.02..7
42.01o.as 48
45. 45.48
50.382.22
Original features
39.20,.00
41 41.66g.70 70
48.07o.58
57 07.34
N2V (transductive)
43. 43.69o.52 41.82p.05
46.31g.78
49. 49.800.04
Roman-empire
N2V (inductive)
13.96, as
14.10p.m .2
15.77
16.812.94
Feature Propagation
13.23,,40 .40
13.89 a4
21.97 15.49..77
IN2V (own)
13.79p.3u
13.86g 29
14.48u.63
18.55,
Original features
66.09o.75
70.28
74.41..47
82.90,.a .00
N2V (transductive)
13.82
13. 13.866
15.35 s.
27.16
6

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 3. Rffect of model and input, test accuracy averaged over all datasets and splits.
12V (tr.)
N2V lin.) frazen (1 =
post-hoc p-h losses
p-h w sampling
FP
Input Model
embed
embed MLP
|| feat. SAGE
model MLP
model GraphSAGE
MLP
SAGE
68
66
N2V (inductive)
27.91
59.45
68.72
73.51
64
Feature Propagation
62.97
63.20
73.17
74.09
test 300000
frozen (A = 1)
61.74
62.88
72.40
73.46
62
post-hoc
64.38
63.75
73.99
74.24
60
p-h w losses
64.57
63.95
74.04
74.32
58
p-h w sampling
64.42
63.83
73.95
74.02
56
Original features
71.01
74.73
71.01
74.73
20
training data percentage
60
80
20
training data percentage
40
60
80
N2 (t((tsansuuctive)
63.89
65.07
74.15
75.57
Figure 2. Influence of amount of training data. Results are aver- aged over all datascts.
small boost to performance. When we compare the results for MLP vs GraphS we see that the differences are bigger when using an MIP. Interestingly, MLP outperforms GraphSAGE when using post-hoc embeddings. This is the case because GraphSAGE's main advantage over MLP is its ability aggregate neighborhood information internally. N2V already encodes this information in the embeddings, and additionally, our post-hoc extension is similar to the aggregation performed by models like GraphSAGE. This ef- fect might also be reinforced by the fact that we use logistic regression on the embeddings for the N2V hyperparameter selection, which is closer to an MIP than to Graph- SAGE. Detailed results per dataset and split are shown in Appendix B.
These datasct-specific differences are explained by the use- fulness of N2V embeddings vs the usefulness of the original features. On Roman-empire, the graph is close to sequence of words with few additional edges; the neighborhood infor- mation encoded by N2V embeddings does not bring much useful information.
Influence of Amount of Training Data Figure 2 visu- alizes the effect of the training set size averaged over all datasets. In general, performance increases with more train- ing data. The only exception is N2V in the inductive setting when using MLP, as the test nodes do not have embeddings, and performance stays at a random guessing level. Graph- SAGE can compensate for the missing test embeddings. While starting with low performance for little training data, it is close to the methods that actually extend embeddings to test nodes for 80% training data. The MLP model with Feature Propagation frozen post-hoc (A = 1) consistently performs below the other post-hoe variants. GraphSAGE with Feature Propagation and frozen post-hoc catch up and outperform some other variants when using 80% training data. Another interesting observation is that for MI.,, the transductive N2V starts in line with the post-hoc variants but rises less when the training data increases. For GraphSAGE, transductive N2V already starts higher than the host-hoc variants and even widens the lead when the amount of train- ing data increases.
Combining N2V embeddings and Original Features For datasets that already have features, N2V embeddings can be used to provide additional structural information to GNNs. The third and fourth columns of Table 3 show our results in an aggregated way, while per dataset results can be found in Appendix C. Using both the N2V embeddings and original features as input for the models increases overall performance by about 10 points compared to just using N2V embeddings. The increase bigger for the N2V (inductive) baseline, as the test nodes do not have N2V embeddings in that scenario. When just using N2V embeddings, MLP has slightly higher average performance than GraphSAGE in most setups; this switches to GraphSAGE having a slight lead over MI.P when using both as input.
Compared to the original features baseline, GraphSAGE actually loses an average of .5 performance points when also using the N2V as input, whereas MLP gains 3 points. This again shows that MLP can benefit from more struc- tural information while GraphSAGE is already capable of aggregating this information by itself. The increase heavily depends on the dataset; see detailed per-datased results in Appendix C. Both models gain over 50 points on Roman- empire and have only smaller gains on the other datasets.
Applying of Loss and Sampling Modification with Fea- ture Propagation While our modifications to the N2V training procedure are motivated by our post-hoc exten- sion, these two parts act independently. The loss-based and sampling- based modifications change the generated N2V embeddings, and the post-hoc algorithm extends these in- dependent of their exact values. This means that instead of using our post-hoe algorithm, we can also use Feature Propagation to extend the modified N2V embeddings. Ta-
7

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
We perform ablation studies on our post-hoc method and the loss modification. For this, set the N2V hyperparameters and to 1, embeddings size to 256, and learning rate to 0.01. For a sensitivity analysis of these hyperparameters, see Appendix A.
Table Applying our loss- and sampling-based N2V modifica- tions when using Feature Propagation to extend embeddings.
the 40% training split,
model
MLP
GraphSAGE
FP
62.97
63.20
FP w loss 63.04
63.22
Figure 3 shows the effect of X and delay in our post-hoc extension, Feature Propagation, as well as a MatMul base- line that multiplies the embeddings matrix iter-many times with the adjacency matrix. When increasing iter delay, the post-hoe methods and MatMul increase to maximum, which depends on X, and then drop off again. This nicely shows the trade-off we discuss in Section 3. More iterations allow an adaption to new paths from the test split, but too many iterations lead to a convergence of all embeddings to graph average. For À 1, this is not the case as embed- dings do not change once they are set, which means that the embedding and performance are fixed for delay 2 O. As Feature Propagation keeps the training embeddings fixed, its performance increases with more iterations. Overall, we can observe that adapting training embeddings is important, as post-hoe with < and MatMul outperform both Feature Propagation and post-hoc with À
FP w sampling 62.98
62.98
ble 4 shows the results of this experiment averaged over all datasets. Feature Propagation gains around .05 points with the loss modification of the embeddings for both models. modification does not change the MLP performance, but it reduces the GraphSAGE performance by .2 points. This is lower than the average of .2 points gained with post-hoc with the losses and .06 gained by the sampling-base modification in Table and shows the syn- ergy of our post-hoc methods with the N2V modifications.
The sampling-base
5.2. Ablation Study on the Post-hoc Method
dataset - Cora
Figure 4 shows the effect of the loss weights and ,3 on logistic regression validation accuracy. As we already saw in our main results, the loss- bass- N2V modification gives a small performance improvement. The figure suggests that a has a bigger influence on the performance as long as is not too high.
78
accuras
76
74
72
70
79.7
79.7
79.7
79.7
79.7
75.6
90.2
90.2
50.2
90.2
90.2
90.1
dataset Photo
79.7
79.7
79.7
79.7
90.2
90.2
90.2
90.2
20.2
90.1
79.7
79.8
79.7
79.7
90.2
90.2
90.2
90.1
90
valication& accuracy
90.2
90.2
0.2
100
88
0.6
0.4
120.0
90.2
86
0.0
0.201
0.01
0.1
3.0
10.0
0.0
0.002
0.03
0.1
3.0
10.0
FP
MatMul
(a) Cora
(b) Photo
84
Figure 4. Validation accuracy for different weights for a and B.
dataset - WIKCS
78
valication accuracy
6. Limitations and Future Work
74
Our iN2V algorithm uses N2V and shares some of the lim- itations. N2V embeddings do not add any information for datasets where neighborhood structures alone are irrelevant. We have observed this case with the Actor and Roman- Empire datasets, where the performance when using only N2V embeddings as model input was close to predicting the largest class. Many molecule datasets consist of thousands of individual subgraphs, where the train and test split is done on a per- per-graph level. In cases where there is no path between the training and test nodes, our post-hoc extensions
72
70
68
2
10
INZV delay /tter (FP, MatMull
Figure 3. Ablation of the effect of for different delays vs Feature Propagation and MatMul.
8

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
(like Feature Propagation) cannot provide embeddings for test nodes in an inductive setting. Our method generally can only provide embeddings to test nodes with path to at least one train node. If this affects only a few nodes, then iN2V still shows good performance. We have shown this with Cora and Citeseer, which have 78 and 438 components, respectively. Some of these components have no nodes in the training set when using the small training splits.
Acknowledgements
This work was performed on the computational resource bw bwUnCCluseer funded by the Ministry of Science, Research and the Arts Walten--rrtteeerrgggg and the Universities of the State of Baden- Maden-Württeen Württemberg, Germany, within the frame- work program bwHPC
Impact Statement
As our post-hoe extension is flexible and not tied N2V, other shallow embedding methods can be used for datasets where the neighborhood structure does not provide helpful information. For example, one could use struc2vec, which focuses on the similarity of neighborhood degree distribu- tions, to obtain better embeddings for heterophilic datasets. In this case, it might be better to utilize the neighborhood- similarity-basee based graph struc2vec builds for the post-hoc algorithm. Another limitation and avenue for future work is that we used random sampling to create the dataset splits. Using random splits is important for a fair evaluation of dif- ferent embeddings and models (Shchur et al., 2018), but in the inductive case, this leads to some unconnected training nodes in the splits with few training nodes. Especially for those splits, it might be beneficial to use a biased sampler that prefers nodes with edges to already sampled nodes to obtain better-connected training sets.
This paper aims to transfer shallow embedding methods like N2V to the inductive setup, where only the graph induced by training nodes is known during training. Our work can be used to apply GNNs to new nodes in the inductive setup, for example new users in social networks or new products in co-purchase graphs, and outperforms existing methods. While our work has implications, particularly in improving the handling of nodes with missing features or in dealing with feature-less graphs, we believe that no specific societal consequences require immediate emphasis in this context.
References
Abu-BI-Haija, S., Perozzi, B., Kapoor, A., Alipourfard, N., Lerman, K., Harutyunyan, H., Steeg, G. V., and Galstyan, A. Mixhop: Higher-order graph convolu- tional architectures via sparsified neighborhood mix- ing. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California. USA, volume 97 of Proceedings of Machine Learning Research, pp. 21-29. PMLR, 2019. URL http: http://poooeeeddn..m....../ v97 hataaaaueelaaaaaa jal9a.ttm.........
Our iN2V is not limited to simple graphs; it can already deal with multi-edges and self-loops. Self-loops increase the influence of hy in Equation 1b to more than À as it also appears in N.)-- Edge weights could be incorporated into the mean neighbor embedding (v) by replacing the mean with weighted sum using normalized edge weights. For KGs, iN2V could be used is-is or by replacing N2V with a KG-focused embedding like TransE that embeds vertices and relations. When doing so, Equations 1b and lc have to be adapted to incorporate the relation embeddings.
Adhikari, B., Zhang, Y., Ramakrishnan, N., and Prakash, B. A. Sub2vec: Feature learning for subgraphs. In Phung, D. Q.. Tseng, V. S., Webb, G. I., Ho, B., Ganji, M., and Rashidi, L. (eds.), Advances in Knowledge Dis- covery and Data Mining - 22nd Pacific-Asia Confer- ence, PAKDD 2018, Melbourne, VIC, Australia, June 3-6, 2018, Proceedings, Part II, volume 10938 of Lecture Notes in Computer Science, pp. 170-182. Springer, 2018. doi: 10. 10011077/78-- / //doi.org/1...oooo
7. Conclusion
We introduced iN2V, general post-hoc extension to induce embeddings to unseen nodes in the inductive setup. We mod- ified the training algorithm of N2V to obtain embeddings better suited to this induction. Our extensive experiments on different datasets, training splits, and using different GNNs on the embeddings showed that iN2V works well and beats the comparable baselines. For some datasets and splits, iN2V even outperforms using the original graph features. In our detailed discussion and ablation, we have shown that our post-hoc extensions perform remarkably well. At the same time, the modifications of the N2V training have a smaller influence on the final performance. Our experiments also showed general limitations of N2V-based approaches for some of the heterophilic datasets, where the performance remained close to random.
URL https:
Bi, W. Du, L., Fu, Q., Wang. Y., Han, S., and Zhang, D. Make heterophilic graphs better fit GNN: A graph rewiring approach. IEEE Trans. Knowl. Data Eng., 36(12):8744-8777 2024. doi: 10.1
TKDE. 2KDE.2024.3441766. URL
10.1109/TK0E.2024.333
Bordes, A., Usunier, N., Garcfa-Durán, A., Weston, J., and Yakhnenko, O. Translating embeddings for modeling multi-relational data. In Burges, C. C.,
9

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Bottou, L., Ghahramani, Z., and Weinberger, K. Q- (eds.), Advances in Neural Information Processing Systems 26: 27th Annual Conference Neural Information Processing Systems 2013. Proceed- ings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pp. 2787-2795, 2013. URL https:/ https///roroeedn...eerr........ paper/2013/hast/
10. 10.1145/321181993222003 URL https: ptittps://ci..rgg 981 9.3220025.
Grover, A. and Leskovee, J. node2vec: Scalable fea- ture learning for networks. In Krishnapuram, B., Shah, M., Smola, A. J., Aggarwal, C. C., Shen, D., and Rastogi, R. (eds.), Proceedings the 22nd ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pp. 855-864. ACM, 2016. doi: 10. I 11114/2999922229999355 URL https:://////.... pttttps://oi.orgg /doi.org////////// 10.1145/29396/2
Abstract.html,
Chen, M., Wei, Z. Huang. Z., Ding. B., and Li, Y. Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Ma- chine Learning Research, pp. 1725-1735. PMLR, 2020. URL http:/ http:::///rooceeenggs.m.....eee pxreesss w119/chen20v...m
Hahn, S. H. and Paulheim, H. Rdf2vec embeddings for up- dateable knowledge graphs- reuse, don't 't retrain! ESWC Posters and Demos, 2024.
Hamilton, W. L., Ying, Z., and Leskovec, J. Inductive representation learning on large graphs. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Infor- mation Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp- 1024-1034, 2017. URL https: http:/:///poeeeenngggneriigeeeee paper/2017/base 5dd9db5e033da act.aa6eeebbbaee37777eeeee. Abstract.html.
Chen, M., Zhang, W., Yuan, Z., Jia, Y., and Chen, H. Fede: Embedding knowledge graphs in federated set- ting. In IJCKG'21, The 10th International Joint Con- ference Knowledge Graphs, Virtual Event, Thailand, December 6 8, 2021, pp. 80-88. ACM, 2021. doi: 041145/350022233502223 URL https:/ /doi.org///// 00.1145/3502223355222
Chen, Y., Mishra, P., Franceschi, L., Minervini, P., Stenetorp, P., and Riedel, S. Refactor gnns: Revisiting factorisation-based models from a message-passing perspective.
Hu, Y., You, H., Wang, Z., Wang, Z., Zhou, E., and Gao, Y. Graph-mip: Node classification without message passing in graph. CoRR, abs/2106.04051 2021. URL https: / //arxiv.org/bba. /11gg/aabb/1000405511
Koyejo, S., Mohamed, S., Agar- wal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Sys- tems 35: Annual Conference Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. URL http://paperr.mips.co
Jia, J. and Benson, A. R. A unifying generative model for graph learning algorithms: Label propagation, graph convolutions, and combinations. SIAM J. Math. Data Sci., 4(1):100-125, 2022. doi: 10. 10.1133721MM13955511 URL https://dbi.org/100 100.1117/2/211119311111
BGF73388225c41222222555001000003 Obstract-CCnferrenne
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In 5th International
Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https:// openreview.net/eeeeee http://wwetttttummm.eeetttaaaaa
Chien, E., Peng. J., Li, P., and Milenkovic, O. Adap- tive universal generalized pagerank graph neural net- work. In 9th International Conference on Learning Rep- resentations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https:// Formwm http://eennee/oooomooeoeeeeee/ed 11
Kohn, M., Hoffmaan, M., and Scherp, A. Edge-splitting MLP: Node classification on homophilic and heterophilic graphs without message passing. In The Third Learn- ing on Graphs Conference, 2024. URL https:// openreview.ned
Donnat, C., Zitnik, M., Hallac, D., and Leskovee, J. Learn ing structural node embeddings via diffusion wavelets. In Guo, Y. and Farooq, F. (eds.), Proceedings of the 24th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, pp. 1320-1329, ACM, 2018. doi:
21cm.
Lacroix, T., Usunier, N., and Obozinski, G. Canoni- cal tensor decomposition for knowledge base comple- tion. Dy, J. G. and Krause, A. (eds.), Proceedings
10

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
of the 35th International Conference on Machine Learn- ing, ICML. 2018, Stockholmsmüissan, Stockholm, Swe- den, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 2869-28822020 2878. PMLR, 2018. URL http:://ppptt v80/lacroix18a.htmm
Mernyei, P. and Cangea, C. Wiki-cs: A wikipedia-basee benchmark for graph neural networks. arXiv preprint arXiv:2007.02901, 2020.
Mikolov, T., Chen, K., Corrado, G., and Dean, J. Efficient es- timation of word representations vector space. In Ben- gio, Y. and LeCun, Y. (eds.), 1st International Conference Learning Representations, ICLR 2013, Scottsdale, Ari- zona, USA, May 2-4, 2013, Workshop Track Proceedings, 2013. URL http: /tatt:p//areee http//awwwiiraaeae/////////ooett abs / 1.0781...1
Le. Q. V. and Mikolov, T. Distributed representations of sentences and documents. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014, vol- ume 32 of JMLR Workshop and Conference Proceed- ings, pp. 1188-1196. JMLR.org. 2014. URL http: irproceedings..o r.press/
Mironov, M. and Prokhorenkova, L. Revisiting graph ho- mophily measures. In The Third Learning Graphs Con- ference, 2024. URL https: /tppe//eeeeeeeeee/////// JLDOLV..
Lell, N. and Scherp, A. Hyperaggregation: Aggregating over graph edges with hypernetoork.. International Joint Conference on Neural Networks, IJCNN 2024, Yokohama, Japan, June 30- July 5, 2024. pp. 1-9. IEEE, 2024.
Namata, G., London, B., Getoor, L.,, Huang, B., and EDU, U. Query-drivens active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs, 2012.
doi: 10. (000///CCNN60992222400666663
URL
https: https://doi.org/10..222 LUCNN608992022.0660000
Narayanan, A., Chandramohan, M., Chen, L., Liu, Y., and Saminathan, S. subgraph2vec: Learning distributed repre- sentations of rooted sub-graphs from large graphs CoRR, abs/1606.08922 2016. URL abs /tts/16660089228.
Lim, D., Hohne, F. Li, X., Huang, S. L., Gupta, V., Bhalerao, O., and Lim, S. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. In Ranzato, M., Beygelzimer, A.. Dauphin, Y. N.. Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021. NeurlPS 2021. Decem- ber 6-14, 2021, virtual, pp. 20887-20902, 2021. URL https: http:://pprocedinngg.eee....... paper/2021/hashi
Narayanan, A., Chandramohan, M., Venkatesan, R., Chen, L., Liu, Y., and Jaiswal, S. graph2vec: Learn- ing distributed representations of graphs. CoRR, abs/1707.05005 2017. URL abs/1707.05005
Pei, H., Wei, B., Chang, K. C.,, Lei, Y., and Yang. B. Geom-gen: Geometric graph convolutional networks. In ICLR 2020. OpenReview.net, 2020. URL https: wttp:r/wwwweeeeeeeeoouoieeeeeeee
ae81 bee_@@90eecllecccaeeeeee1113 Abstract.htm.
Longa, A., Lachi, V., Santin, G., Bianchini, M., Lepri, B., Lio, P., Scarselli, F,, and Passerini, A. Graph neural networks for temporal graphs: State of the art. open challenges, and opportunities. Trans. Mach. Learn. Res., 2023, 2023. URL https: httpt:::////peervve...... forum? peruumii=ppinn ddt.ummmdddddccMattt00110
Perozzi, B., Al-Rfou, R., and Skiena, S. Deepwalk: online learning of social representations. In Macskassy, S. A., Perlich, C., Leskovec. J., Wang, W., and Ghani, R. (eds.), The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '14, New York, NY, USA - August 24 27, 2014, pp. 701-710. ACM, 2014. doi: 10.11452602330.22233333 URL https:
Luan, S., Hua, C., Lu, Q., Zhu, J., Zhao, M., Zhang, S., Chang, X., and Precup. D. Revisiting heterophily for graph neural networks. In Koyejo. S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.). Advances in Neural Information Processing Systems 35; Annual Conference on Neural Infor- mation Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022. URL http::/ httt:://papers.ips..c
/
Platonov, O., Kuznedelev, D., Babenko, A., and Prokhorenkova, L. Characterizing graph datasets for node classification: and beyond. In Oh, A., Naumann, T., Globerson, A.. Saenko, K.,, Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Con- ference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16. 2023, 2023a. URL http:/ htt.lppp/pppess.iie.. /ttpppp.sisssscc//////
dichotomy
wttp:/ffeesssaaeee2/e//ee 09230eesc660000882779111332 CbetrracttConfferennehhttl
paper r Filees/pppe/2203 /hahh//

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
016661002fddbass833333cccooo00 Abstraatt-Conferencelltt
Shchur, O., Mumme, M., Bojchevski, A., and Günnemann, S. Pitfalls of graph neural network evaluation. CoRR, 2018. URL http://arxiv.org/ 1811.05868.
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look the evaluation of gnns under heterophily: Are really making progress? In ICLR 2023. OpenReview.net, 2023b. URL https: / h//ppenrevew.neer
Sun, Z., Deng, Z.., Nie, J., and Tang. J. Rotate: Knowl- edge graph embedding by relational rotation in complex space. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https: / //openreveew.eee Htwpe///oooooomiieeeeeeeeee
5wv.
Polleres, A., Pernisch, R., Bonifati, A., Dell'Aglio D., Dobriy, D., Dumbrava, S., Etcheverry L.. Fer- ranti, N., Hose, K., Jiménez-Ruiz, E., Lissandrini, M., Scherp, A., Tommasini, R., and Wachs, J. How Does Knowledge Evolve in Open Knowledge Graphs? Transactions on Graph Data and Knowl- edge, 1(1):11:1111:5;; 2023. doi: J...... URL URL.httpss//óórpssaaasss hett..itteeeeeeee///s////
Tang, J., Qu, M., Wang, M.. Zhang, M., Yan, J., and Mei, Q. LINE: large-scale information network em- bedding. In Gangemi, A., Leonardi, S., and Panconesi, A. (eds.), Proceedings of the 24th International Confer- ence on World Wide Web, WWW 2015, Florence, Italy, May 18-22, 2015, pp. 1067-1077. ACM, 2015. doi: 10. 10.1145/2732277.27441177 2711112777777799 URL htw.::/tt////cm /ttp:://doi../// 10. 10.114/273662772771000
Ribeiro, L. F. R., Saverese, H. P., and Figueiredo, D. R. struc2vec: Learning node representations from structural identity. In Proceedings of the 23rd ACM SIGKDD International Conference Knowledge Dis- covery and Data Mining, Halifax, NS, Canada, Au- gust 13 17. 2017, pp. 385-394. ACM, 2017. doi: 10.1145/3097983 3098061. URL htthttp:: /ttp::://do///////// 01011/////999993339033
Tian, Y., Zhang. C., Guo, Z., Zhang, X., and Chawla, N. V. Learning mips on graphs: A unified view of ef- fectiveness, robustness, and efficiency. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenRe- view.net, 2023. URL https: https///pppeeeeeeeeeeemmm forum? Sauummdd=Cs3r5K.ee
Ristoski, P. and Paulheim, H. Rdf2vec: RDF graph embed- dings for data mining. In Groth, P., Simperl, E., Gray, A. G., Sabou, M., Krötzsch, M., Lécué, F., Flöck, F., and Gil, Y. (eds.), The Semantic Web ISWC 2016 15th International Semantic Web Conference, Kobe, Japan, Oc- tober 17-21, 2016, Proceedings, Part I, volume 9981 of Lecture Notes in Computer Science, pp. 498-514, 2016. doi: 10.1007/978--3331944622-- 30. URL https: / //dbi..rgg/1..0007//////
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É. and Bouchard, G. Complex embeddings for simple link pre- diction. In Balcan, M. and Weinberger, K. Q. (eds.), Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY. USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pp. 2071-2088 JMLR.org, 2016. URL http: proceed ngs.ml r. r.press/ v48 144//rouillonn6...
319 4-0-3-6633---- 4_30.
Rossi, E., Kenlay, H., Gorinova, M. I., Chamberlain, B. P., Dong, X., and Bronstein, M. M. On the unreasonable effectiveness of feature propagation in learning on graphs with missing node features. In Rieck, B. and Pascanu, R. (eds.), Learning on Graphs Conference, LoG 2022, 9-12 December 2022. Virtual Event, volume 198 of Pro- ceedings of Machine Learning Research, pp. 11. PMLR, 2022. URL https: / httpp//ceeiinn.mm..pppe..eeeeeee v1 1198//rsss ress/rossiii 22 htmml.
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. Graph attention networks. In 6th International Conference on Learning Representa- tions, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenRe- view.net, 2018. http:t/ /eppereviee..ee//
Sancak, K., Balin, M. F., and Catalyurek, U. Do we really need complicated graph learning models? - simple but effective baseline. In The Thind Learning on Graphs Con- ference, 2024. URL https:/ httpss://ppnrreview.eess /ttp///eeeeeeeee/////e/ forum? ?= 6 4MgKEVZ.
Wang. Z., Zhang, J., Feng. J., and Chen. Z. Knowl- edge graph embedding by translating on hyperplanes. In Brodley, C. E. and Stone, P. (eds.), Proceed- ings of the Twenty-Eighth AAAI Conference on Arti- ficial Intelligence, July 27 -31, 2014, Québec City, Québec, Canada, pp. 1112-1119. AAAI Press, 2014. doi: 10.1609/AAAL 8870. URL https:// dol.org/10..609/aaa...o
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., and Eliassi-Rad. T. Collective classification in network data. Mag., 2008 doi: 10. (tt.///aa...29.......o. v29.3.2157.
12

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Wu, F., Jr., A. H. S., Zhang. T., Fifty, C., Yu, T., and Wein- berger, K. Q- Simplifying graph convolutional networks. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceed- ings of the 36th International Conference on Machine Learning, (CM1.2019) 9-15 June 2019, Long Beach, Cali- fornia, USA, volume 97 of Proceedings of Machine Learn- ing Research, pp. 6861-6871. PMLR, 2019. URL http: ml1 www.:eeediing.....ressee.ee
Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K., and Jegelka, S. Representation learning on graphs with jumping knowledge networks. In Dy, G. and Krause, A. (eds.). Proceedings of the 35th International Conference Machine Learning, ICMI. 2018, Stock- holmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Re- search, pp. 5449-5458. PMLR, 2018. URL http: //ppocceddngg.ml..pppp.
/v00/v/00///// xu 8c.html.
Xu, K., Hu, W. Leskovec, J., and Jegelka, S. How powerful are graph neural networks? In 7th Interna- tional Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenRe- view.net, 2019. URL Saammmdddryyssaaae
Zhang, S., Liu, Y., Sun, Y., and Shah, N. Graph-less neu- ral networks: Teaching old mips new tricks via distilla- tion. In The Tenth International Conference on Learn- ing Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https: // //openreviee..ee///oroo
4=4p6 5HBWPCW.
Zhu, J., Yan, Y., Zhao, L., Heimann, M., Akoglu, L., and Koutra, D. Beyond homophily in graph neural networks: Current limitations and effective designs. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.. and Lin, H. (eds.), Advances in Neural Infor- mation Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrlPS 2020, December 6-12, 2020, virtual, 2020. URL https::/pproeeeddng..euut paper/2020/hash/ B Abstract.html.
13

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Supplemental Materials
A. Hyperparameter Sensitivity
This section follows up on Section 5.2. Figure shows the sensitivity of validation accuracy to the iN2V learning rate and embedding dimension while all other hyperparameters (including X. a. 3, and r) are aggregated in the densities. We can see that the embedding dimension has the biggest influence the performance, especially for the Cora dataset. For Figure 6, which shows the sensitivity to and q, therefore fix the embedding dimension to 256.
validation acccrrcyy
0.7
0.5
learning rate
0.001
0.01
64
256
embedding dimension
(a) Cora
0.91
validation becuracy
validalion accuracy
0.76
0.89
0.74
0.88
0.72
0.87
0.70
0.86
learning rate
learning rate
0.001
0.001
0.01
0.01
0.84
0.66
256
embedding dimension
embedding dimension
(b) Photo
(c) WikiCS
Figure Sensitivity learning rate and embedding dimension.
B. Complete Result Tables
Tables and 6 show the comparison of MLP and GraphSAGE on the iN2V embeddings baselines. N2V, applied in the inductive setting, and Feature Propagation are comparable because they have access to the same information during training, while original features and N2V in the transductive setting utilize more information (the original features and all nodes during training). Tables and show MLP and GraphSAGE results when using different iN2V settings.
C. Combining Original Graph Features with Trained N2V Embeddings
As shown by related work, shallow embeddings like DeepWalk or N2V can be combined with the original graph features to improve the performance of GNNs. This is especially helpful for MLPs who do not have access to structure information when only using the default graph features (Tian et al., 2023). Tables and 10 compare the iN2V embeddings vs baselines with the original graph features concatenated to the input embeddings. Tables and 12 compare the different iN2V setups when concatenating the embeddings with the original graph features.
14

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
0.800
0.775
0.750
validation
0.700
0.2
0.675
5.0
0.2
1.0
5.0
Cora
0.92
validation accuracy
0.91
0.76
0.90
0.74
validation
0.72
0.70
0.2
0.68
0.2
5.0
0.66
5.0
0.86
0.2
1.0
5.0
0.2
1.0
5,0
(b) Photo
(c) WikiCS
Figure 6. Sensitivity to the N2V and hyperparameters.
D. Using other GNNs
The effectiveness of iN2V is not limited by the chosen GNN. To demonstrate this, we additionally performed all experiments with GAT (Velickovic al., 2018), see Tables 13, 15, 17, and 19 and with GIN (Xu et al., 2019), Tables 14, 16, 18, and 20. For GAT, we set the number of attention heads to 8, Otherwise, used the same hyperparameter tuning procedure for both GAT and GIN as for GraphSAGE.
The results for GIN and GAT are in line with the GraphSAGE results. iN2V outperforms all baselines for most splits on all datasets except Actor, where all N2V-based results amount to guessing the largest class. Averaging the iN2V results over all datasets, splits, and whether to concatenate the iN2V embedding with the original graph features, MLP remains the best model with an average accuracy of 69.37 points. GraphSAGE reaches an average of 69.25 points, GIN of 69.08 points, and GAT of 68.64 points. This shows that an MLP can outperform classical GNNs without distillation contrastive learning if enough structure information is provided with the features.
When considering all four GNNs, iN2V outperforms FP by 0.8 points on homophilic and 0.3 points on heterophilic datasets. iN2V outperforms FP by 1.3 points when using MI.P as a classification model and by 0.4 points when using message-passing GNNs. iN2V outperforms FP by 0.9 points when using only trained embeddings and by 0.3 points when using both trained embeddings and the original graph features as input. Finally, for the 10% and 20% training splits, iN2V leads over FP by 0.9 points in contrast the 0.4 point lead for the 60% and 80% training splits.
15

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 5. Comparison of best iN2V variant vs baselines; MLP ac- curacy. Gray numbers not directly comparable as they use additional information (graph features/transductive setup).
Table 6. Comparison of best iN2V variant vs baselines; Graph- SAGE accuracy. Gray numbers are not directly comparable as they use additional information (graph featurew/transductiv setup).
Percentage training data
Percentage of training data
Dataset
10%
20%
40%
60%
80%
Dataset
10%
20%
40%
60%
80%
Cora
Cora
N2V (inductive)
30.14,.0
30.07,.4
30.12,00
29.67
30.70g 14
NIV (inductive)
42.15
15.07, a0.
81.31 1.50
84.50,.4
Feature Propagation
78.54,,42
81. 11,47
82.361.01
83. 83.76..0
86. 27
Feature Propagation
77 27
79.482.19
81.03, .5
84.00,,0
84.133,ss
IN2V (owa)
79.54
83.93
81.69..6
25
IN2V (own)
78.88;
80.9 80.94.58
85.30, 09
83.73
81.462.0
Original features
66.58,m
71.34
75.94
78.78
79.23
Original features
76. 78.27
83.37
66.230 10.23,.5
66, 66.96y.46
87.05
(transdactive)
78.44
80.8 80.84,
82.82
80.75.
83.10, a6
N2V (transductive)
79.25, 45
81.66, 20
80.81g
65.24b.58
86.01,.0
Citesoer
Citeseer
N2V (inductive)
19.84
21. 13
21.14 or
21.10: 06
19 oz
N2V (inductive)
34.17 43
12.76
26.12..2
63.612.57
68. 2..
Feature Propagation
56.76, a1
60.14,.0
65.73,
68.54 na
72. 63
Feature Propagation
56.85, a1
60.53, n0
63.03
66.51a.1s
69.76,
IN2V (own)
57.8% 36
61.67
66.47,00
70.62.
73.27,25 73
IN2V (own)
57.8 9)
60.78, 18
63.02; ss
66.81,
69.9%2g.0
Original
06.00
70.33,.4
73.23
74.50_.04
76 202.00 to
Original Teatures
69 86, 42
72,80
74.93,.0
76.30,o0
70.877.m
N2V (trassiactive)
56. 132.m
60. 65
61.77.77
68.8 80:
70.09,
NIV (transductive)
57 11:
61,33, co
66. 15g op
70.14 as
72.76g.54
Pubmed
Puhmed
N2V (inducttvv))
30. 80.45
30 30.60
40 40.160.42
40.01em
39 30,30,
N2V (inducrive)
66 66-02%
74.705.30
78.84
81.866.s1
82.611..2
Feature Propagation
78. M4.44
78. 99
81 81.02ors
81.91c.64
82 9t
Feature Propugation
76. 76.37
77.726.50
80.74u.72
82.00 82.000..44
82. 82.433,
iN2V (own)
80.42, 30.42..4
81.77
82. 78.
83.19.
83. 83.26..x ar
IN2V (own)
79.93.
80.60.. 16
42.14..4
82 81.81...
82.59 m.
Orgiinal Teatures
84. 15.11..1
86.05
87, 72
88.63,
69. 10g
Original features
85 85.95p
86.0A 2s
56 52 dk
89.256
89.85% 0o
N2V (tranddactive)
81.03
82 x2.72
82.00g.
77.17
N2V (tannsductive)
82.200.5
80 80.22p.54
83.57o.50
83.06
Computers
Computers
N2V (inductive)
37.3 30..
37.33o
37.3 37
37
37. a7
NEV (inductive)
77 64
84.44 sa
17
89.21 0.
80.335 70
Feature Propagation
81.90b.s
86. 16
80.200.a5
90.27c.00
90.7loa
Feature Propagation
82.79p
88.43,.6
40.40g
90.453..4
90.877.04
IN2V (own)
88.7 47
90.04...
41.10%
91.51c..
91.56 31.55
IN2V (own)
36 ss
89. 09.677.m
90.84..37
91. 08%
91. 33..38..6
Original features
82.15
84. 68.08
85.570.3
86.00c.7
86. 50
Original features
87 52
89.76..
91. 91.12pa0
91.500 01.4Ehuss
N2V (transaactvve)
33.5%.. 54
89. 63
90.1 90.156.
90.39p 57
NIV (transductive)
89. 69.15,
90.16,. en
90.77p
91.19g.a am
91.16%.0
Photo
Photo
N2V (inductive)
25.5
24.86,
23.66,
22.96
23.15g
N2V (inductive)
85,48, as
87.73, .0
90.98
91.92
92.21p
Fenture Propagation
87.377..5
80.380.55 54
91.490..1
92.23e.5
92 6:0 77
Feature Propagation
87.43, 09
90.146.54
91 .7.02
92.480.51
92.9K%.7
iN2V (ows)
90.70..0
91.74.
92. 60,
92.86,
93.00 93.07
IN2V (own)
90.51p
91.70,
o2.37
02.62p
93.08
Original leatures
89,30 32
90. 85
91. 96
92. 12.10o
92. 92
Original leatures
93.7 1
91.5%
96.2 26.27
95.52 52
95.59%.7
N2V (transdactive)
91.230.44
91. 01.88o.47
02.67o
92.73c
42.44
NIV (transductive)
01. 01.29h.4
92 29.
92.900 92.90buas
83.01a.s7
90.33
WikiCS
WikiCS
N2V (indlactive)
22.99.41
22. es
(16
23. m2.
22. 82,.2. 25
(inductive)
67.752
74.23,15
78 78.21a.m
80.40y.4s
31.622..
Feature Propagation
73.59L
75.83
78, 53..53
801.8
80.97o.a7
Festure Propugation
74.77227
78.011.00
80.170.s
61.080.ss
81 11.98,..e
(own)
21a.0
80.37
$1.47.32
mM
82.76u as
IN2V (own)
78.9 78.91um
80. 30.19mm
81.28
$1.733mm
82. $2.37.mm
Original Teatures
76.800.64
78, 06
801440.64
81.66%
82 35
Original Teatures
80.75
82.60
54 28
85.21 0.ss
N2V (tansdaactve)
68
80.3 51
81 47.77.0 56
81. 11.886c.. 64
81. 920 06
NEV (taanductive)
79. 79.75e 41
80,93 64
$1.88
82. 30, 46
82 81.88...
Actor
Actor
N2V (inductive)
25.41
25.55 18
25.89
26.055..61
26.57, 75
NEV (inductive)
25.14, 09
25.56,
25.54
25.42, .os
25.688,.2
Feature Propagation
25.06..0
25. D0.04
25. 48
25. 18..0
25 (00
Feature Propagation
22, 29
25.39..06
25. 24.14
25.091.42
25,05,,54
IN2V (own)
25.7 25.78.. 78..0
25. 45
25.00
25. 25.144.0 mM
25. 01
IN2V (own)
25. 18.18
25.50
25. 10
25. 25.160.90
25.79
Original features
o7
37. 49
38. 75c wo
37.87
Original features
31.77om
33.846.
48.45
37.366.77
36.71..
N2V (transdactive)
25.08,00
25.34,
25.5 03.54,.0
25. 12.12,.0
23.92
N2V (transductive)
25. 25.50p 26
25.56 86
25.41
24.26,.18
24.55,.0
Amazon-ratings
Amazon-rtinns
N2V (inductive)
38.91g.u
36, 35
36.7
38.78g 50
36.5 as
NIV (inductive)
37.47o
40.69..a
41.7laas
16, as
49.47:.um
Fenture Propagation
39. 700
42. 18 al
44.90, se
46. 24g no
46. 46.32, 96
Feature Propagation
38.97 76
41,68, no
45.72
48.03, 67
50.04
IN2V (own)
41 57
64
46.56 02.52..37
51 1....
51.90 ra
IN2V (own)
40.02..3 79
42.01 48
45.4%
50. 60.35%),4 45
50.38
Original leatures
37 M0uss
21. To
47 200.
50 874.77
64 0lla
Original leatures
39 20,
41 11,06
48 0Tass
63 D.h..
57 34o.r
N2V (transdactive)
41.99 11.97am
12. as
45.5 45.57o.47
46. 46.58c.m
47. 97.97ass
NIV (transductive)
11 82, ds
43.09
36.31a7s
19.80
Roman- empire
Roman-empire
NZV (inductive)
13.34,
1..0%).2
13.50M.
13.271.07
NEV (inductie)
13. 96 ss
14.10,,52
15.77
15.981.5
16.812.04
Feature Propagation
13.83).2
13. 93.
13. 90.
13.88
13.55p ra
Feature Propogation
13 13.23, 41
13.89
16. 16.19..4 15.49p
21.97:.
IN2V (own)
13.89,
13. 90.00.mm
13. 13.79
14.23 m
14.27
IN2V (own)
13.79h 39
13.86, 29
14.48
15.99, ae
18.55, m0
Original leatures
06.
66.833 71
06. 56, 0z
Original Teatures
66 080 75
70.250 68
74.410
78.560..0 so
82,90) o0
N2V (randdactive)
13. 20,20
13. 74..0
13:906
13.98%
13.276
N2V (ransduucive))
13. 82, 22
13.86
15.351.s0
21.466,40
27.46,.0
16

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table Comparison of N2V vs different iN2V setups, MLP accu-
Table 8. Comparison of N2V vs different iN2V setups, Graph-
racy.
SAGE accuracy.
Percentage of training data
Percenlage of training data
Dataset
20%
60%
Dataset
10%
20%
40%
60%
80%
Corn
Cora
N2V (inductive)
30.04;
30.07,.0
30.122.05
29.67,
30.700,.4
N2V (inductive)
42. 18..8.
59.91.15
75.07,
81.31..6
84.50h
frozen (A ==] 1)
71.612.75
76.71
79.57..24
81.75 6s
83.51
frozen (A 1)
71.762.u
76. 76.86
80.751.77
82. 12-45
81.13
post-hoc
29.84_.2
80.28,.50
83.93
84.00,00
85.077.46
post-hoe
79. 20.
79.441.88
83.45
83.73..1
84.462.08
p-h w losses
79. 38
81.07, sx
18.99
83.75 51
85. 85.24
p-h w losses
78. 88,88,
80.50,
83.30 83.30
83.39...4
84, 14.69,.a
p-h sampling
79.57..24
.1.
82.71 02.84
84.09 94
85.5722.2
p-h w sampling
79. 1
80.941.6
82.02,.54
83.43,.0
84.02,,8
Citeseer
Citeseer
N2V (inductive)
21.43 10.8
21.11
21.10
19.82
N2V (inducrive)
34.17
42.762.
56. 12
63.6
08.80g
frozen (A - 1)
54.19 r9
59.75 xa
65.066
68.95, a7
71.59
frozen (A A 1)
56.22,m m
59.00g 16
63. 63
67.73, .33
68.92g
post-hoe
25.912.2
61.511.57
68.471...
70.62..7
7
post-hoc
57.25
60.59
62.881 so
66.811.42
69.13
p-b w losses
58.02;
61.67)
66.56,.00
70.69,.
73.009
p-h w. losses
57.25
60.78).e
62,761
67,233
08.922.r7
p-h w sampling
57. 57.88
61.40,.ax an
66. 62.
70,21, sM
73.27 mm
p-h w sampling
57. 67.88
57. 47.922..7
02,
67. 23.
68.83 88,83
Pubmed
Pubmed
N2V (inductive)
39. 19.15
39.69p.77
40.15
10.01o:71
39.30,.u
N2V (inductive)
66.02s.s
71.732a8
79. 79.811 m
81 85o85
82 61g.1.
frozen (A = 1)
76
78.4%
80.49%..c os
81.45 on
82.0 82.005
frozen (A 1)
75.99p s0
78.37 0.00
81.316.5
82.30p
82.88
post-hoc
80. 17c.s
81.77u.u
49
83, 83.12u
83.30
post-hoc
79. 93...
80 a0.40
81.80u.e
82. 82.7/o.2
82 90.00.
p-h w losses
80.42g.44
81.68
82. 74
83.45g
83.26o.5t
p-h w losses
79.76
80.75 00.75
81.910.54
82.7lo..4
82.50p.6
p-h w sampling
80. 10.422..a as
81.37
72.78
83.19p 44
83.35
p-h w sampling
79.68a.5
80.50p.a4
$2.140.44
82.81g. 21.81..00
82.88 22.94
Computers
Computers
NEV (inductive)
37.33
37 3uo.e
37.51u:75
37. 720.7
NEV (inductive)
77 77.612
S1.H
88.18
89: 89.21g..
89. 89.35
frozen (A m 1)
82.25x.75
84.08y.50
17.63
88. 88.13
88.81
frozen (A I)
85.400.54
87. 87.459.42
80.056.50
90.000.44
90.24p.39
post-huc
88.06 23.06i..u a6
89.87o.30
90.966, ao
90.90l so
91.32
post-boc
87. 67.94..4
80.62a.. as
90.666
90.77l..z
91.07p.4
p-h Josses
88.19 38.199.42
90.00 00..s
91.100.47
91. 55
91.35o.o0
p-h w losses
87.8 sa
89 670.40
90.8 4...74
91. 11.080.s6
91:21o.52
p-h w sampling
88. 70g 47
89.60 89.69h..o so
90. 90.75
91.51p is
91.55p
p-h w sampling
88, 36, se
89 19.266..5
90.533 se
91. 01.16.....
91. .3.3..3
Photo
Photo
N2V (imductive)
25.51c
24.86, 30
23.661.09
22.96 9s
23.15g
N2V (inductive)
85,48,
87.73 40
90.955 m1
91. 91.922..a
82.21p
frozen (A = 1)
86.08 .7
87. 57.72n.as
90.01a.7s
90.95
91.36
frozen (A 1)
88. 6...
90.31a.50
91.78u.47
92. 12.18
92,76
post-hoc
90 57
91.71o.36
470.0
080 ma
93.07o.
post-hoc
90.49p.o 67
91.62a.a
92.40p. na
92 93
93. 08
p-h losses
90. 70
91.7
92. 60..ss
92 22.8
93.12. 93.12..s7
p-h w losses
00.51a.
91.67 91.67ac3
92.37u.e
92.7
92, 22.95,.0
p-h w sampling
30.38g.4e 64
91. .9.99044
92. 470.64
92.70a.79
93. 1,,02
p-h w sampling
90.422.54
91. 70
92. 20.50...0
92. 12.622..s
92.8%
WikiCS
WikiCS
N2V (inductive)
22. 4.
22.96n.e
23. 03.00
23. 12.70
22.82,.00
NEV (inductive)
67.762.08
74.22,.08
78.21a.50
80.40%,
81.62p.00
Frazen (A (A-3) 1)
70.642.01
71.89h
75.48o.s
77.27us4
82.500.m 78.62
frosen (A 1)
75.62
77 77.96u.a
79.8
80.85.ums
81.55u.s
post-hoc
78.00 80.92
80. 10
81.366,05
82.03p.5t
post-hoe
78.750.74
80.07a.s
81.01o.5a
81 8...843
55..5
p-h losses
78.98
80.23a.s1
81 17.47
82.12 os
82.76,.o
p-h w losses
78.726.0
80.06
81.20o 48
81.73
82.43
p-b w sampling
79.22 21
80.37
81. 48
82. 18,
82 50
p-h w sampling
78.91
80 19.19
81.28p.s
81 12.82
82.37,
Actor
Actor
(inductive)
25.41 25.411.a9
25.351.10
25. L...
26. 26.05o.a. s1
26.57o.m
N2V (inductive)
25.14..00
25.566..00
25.544.1
25. 42...n
25.681.4
Frozen (A = 1)
25.78%
25.45
25.00,
24.62
25.03g.o
frosen (A 1)
25.18p.07
22. 22.044,.3
25. 25.40l se
24.93%
24.97).0
post-hoe
25. 25.13
25. 25.78u.m
21. M
21.93, or
25. 25.83,.0 .30
post-hoe
25. 25.10... n0...s
25 82a.
25.344.s5
21 21.89
25.39
p-h losses
25-42..1
25. 76.52
25-45p 40
25.51
p-h w losses
25.06,.06
24. S0uus
25.85
24.41l..7
25. 25.79
p-h sampling
24.96
25. 82
25. 23.60
25.
24.43,.m7
p-h w sampling
25.39
25.50u.zu
25. 33.71
16.16
24.171.0
Armzzon ratings
Amazon Amazon-rutings
NEV (inductive)
36.90 36.91o.30
36.93u.a
36.7 1o.77
36.75u 52
36.5
NEV (inductive)
37.47o.47
10.65 10.69
11.7 ss
46.99%.ns
19.17
frozen (A = 1)
39. 09.04
41.31,
44.44_.02
45.77p.44
46. 44.s1
frozen (A I)
36.70; 36.70
41.76g.01
45.48o.57
48.10
10.19..0
post hoc
41 11.336.60
43.8 33.89).0 so
46.52
40.44g
49.78,,
post-boe boc-
38.65,.00
42. 12
45.20g 70
47.9 07.93, 29
48.40,
p-h losses
43
43.690.57
46. 680.94
66
13
p-ht w losses
40.02o.
12.00l..s
45.466.50
50 49..8
50.35 50.38222
p.h sampling
41.41g
43.75
46.58,,o
49. 49.28 64
49.06p.o
p-h w sampling
40.10p
41. 41.922..8
45.316
47 47.44,.o0
48.40)
Roman-empire
Roman-empire
(imductive)
13.34, 25
13.55p.e7 on
14.00..s
13.98 40
13.27
N2V (inductive)
96,966..
14.10
15.77
15.98 ..5
16.81
Frozen (A =
13.89 26
13.90 13.90,
13.90..19
13.744
13.644.
frozen (A = 1)
13.644..
13.88%
14.488..3
14.82
17.59
post-hoe
13 90.
13.85o.30
14:23 14.01
14.27o.01
post-hoe
13.31,.0
13. S6.66.
13 33.866.11
13.8Gr.41
14.89).m 70
p-h losses
13.81c.
13.68
13.79..24
13.97p m
14.16
p-ht w losses
13.17
13.84aar
13.90h as
15.05
16.93,.1
p-h sampling
13. 20
13.844.28
13. 13.88
13.96p
13.61o.si
p-h w sampling
13.79u.s
13.50a.7
14.02a.41
15.33,.12
18.55
17

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 9. Comparison of best iN2V variant vs baselines concate- nated with the original graph features; MLP accuracy. Gray num- bers are not directly comparable as they use additional information (transductive selup).
Table 10. Comparison of best iN2V variant vs baselines concate- nated with the original graph features; GraphSAGE accuracy. Gray numbers are not directly comparable as they use additional infor- mation (transductive setup).
Percentage training data
Percentage of training data
Dataset
10%
20%
40%
60%
80%
Dataset
10%
20%
40%
60%
80%
Cora
N2V (inductive)
66.48,..8
70.54:200
75.47
76.03%..0
16.21
NZV (inductive)
a3.9va
81.02 .4
85.97,07
86.97o.00
81,38),
Feature Propagation
82, 22.04
23.40.2
K3.83,40
85.50..4
85.044,4 48
Feature Propugation
20.922 25
83-49,54
85.33,.4
87. N7.82
87.75,.12
IN2V (owa)
62.001.0
33
85 43
85.81:m7
88.49;
IN2V (own)
81.01: 42
83, 83,28.4
80 85.92,.1
86,771.4
87.000..t
Original features
66.96
71.34,7
a5.94
78.78..11
79.220 79.28,
Original features
75.272
83.37
86.23
86.95u..0
87.05
N2V (trandiactive)
81.04,
80.52,..
85.90h .o
86.93
86.13, ds
N2V (canndductiv)
62.406.7 77
84.45, 29
56.50) 01
86.85
88.30),47
Citesoar
Citeseer
N2V (inductive)
06.06
70.04,.0 as
72:33
73.58:.1
14.682 12
N2V (inductive)
68. 68.522.7
72.49..14
74.53 es
75.851.1
76.761.45
Feature Propagation
68.48,
72.13,
74.37,
75.29k
76.34,
Feature Propagation
68.00,
71.72
76.39, 74.77
76.46.
(own)
71.35oas
75.85 74.13,28
IN2V (own)
47
71.14
73.99, 47
75.771.20
75.50..o
Original features
66.60,
70.53,
73. .3
74.06.
76. 202 so
Original features
69 85,
72 18.886.0
74.93; ss
76 09.399...
76,8% 76.82..17
N2V (tansdactivv)
69. 10
72. 58,
75.94. o 7h.75
77. 37.34
NIV (transductive)
71 08. 04
73.211.32 a2
71.70u.
76.14..4
10.82).77
Pubmed
Pubmed
N2V (inductive)
63,94l,,4
85. 550
86.730
87.21c
NIV (inductive)
64.336 45
86 86.06u.5
87 32.522
87, 37,326.39
88. 88.01o.1
Feature Propagation
85 20
4:io.
39.333..0
95p oo
Feature Propagation
82 99
85.46..4
86.74u.s
87.97o.
98..9
iN2V (owa)
87.0 87.07 85,38p.3
88. 690.co
89.46
89. 900
IN2V (own)
63.02 72
85.09).5 37
87.51p
68.58 68.59).35
89. 89.39
Original features
l4.1
86. 96.05 as
87. $7.72p a
88, 63) o
89. 10 os
Original features
95, 47
86,09, o9
88.32,
25m 44
89.85, mi
N2V (trnndacciiv)
87.50
88% 700.a
80142
80, 10.88
N2V (rrnnsductive)
85 86.30p
87. 87.200 40
57. 57.14p
88.4 30
801114 0700
Computers
Computers
N2V (inductive)
80.41g.50
82, 69
84 0850.7
84.34c.or
84.72u.a7 or
NEV (inductive)
86 16.64,.. 11
90.7Mp.ss
91.335.
91.770pg.0
Feature Propagation
85.39%
87. (33
89. 14
89. 91..1. no
90. 15u re
Feature Propagation
87. 57.77o
89. 33393,, 50
91. 1..30
91 1..88.. s2
92. 13: 66
IN2V (own)
88 62
60..44
90.9H...2
IN2V (own)
3s
50.
91..4.
71,77
Original features
85.15
81.00 e..0
85.5 85.57o.ss
86. 86.00s.77
86. Sfu
Original features
87 77.52.. as
89.76, 76. o
91. 91.12
91. 91.19h..m
91. 11.500gg.
(franddactive)
80.366
01. 19
91.477.. on
91. 300 47
(ransdductive)
90 90.45 ar
91.00k). a6
922 08ho
922.35..00
922.280
Photo
Photo
N2V (inductive)
85.73
86.
90.74..0 so
90.50g 65
91.78p no
NIV (inductive)
91.97 75
53.876,10
94.94 ot
95.376 40
95.566, s1
Fenture Propagation
92.750.
913993
34.736.u
94,75c.5
95-31
Feature Progagation
93 s4
94.40%,41
95.11
95.544 84
95,733,
IN2V (own)
93 9th,
94.55 94.556mm
95 336
95. 42: 45
96. 70s 21
IN2V (own)
93. 87, 52
94. 68) so
96 20.2 es
95 5th, 42
95, (7.638,, 74
Original features
90. 00.885..0 us
91. 91.96 ab
92.41,.o 00
02 92, az
Original fentures
93.74, 48
94.59, sy
05.52 s2
95.500 97
N2V (trunsdactive)
91.07
94. 930
95. 36,
95.73c.
95.63l
NIV (transductive)
94.
94.97 ss
96. 10.50l nz
95 80, se
95.79
WiLICS
WikiCS
N2V (inductivv)
73.16
75. 6lle
76, 79..91
77.30k.
76. 77
N2V (inductive)
76.50z 34
79. 79.999.s
$2 22.23
83,76uss
84.7MM..7
Feature Propagation
78.14, 20
80. 58
82. 12.9
89.98: o
84.78, i5
Feature Propugation
79.58, as
82.09 ss
81.100ua
64.95
85.77 76
iN2V (own)
61.78
83.47e.47
84. 770.3
85.41cm
86. 26
IN2V (own)
81.50p
63. $3.38a.5
84.62u.a7
85, 250.47
85.73mm
Original features
76.85
78. 70
80. 43p.ne
81.56. na
82.38, oz
Original features
60.75m
62.56
$1.28
85.24
85.88
(tranddactive)
25
83. 73..0
84. 70..0
66.44c.e en
80 50l. 27
NIV (tansductive)
91ass
83.37our
84.00
85.00u.s1
70,,00
Actor
Actor
N2V (inductive)
35.22...
35.34,.0 o
36.640 as
38. 38.222.m m
38.08,.0
NZV (inductive)
32 32.04p
31.44.77
14.14
36.13,.5
38.08%,.o
Feature Propagation
83.97o.72
35. (Ho
36 630.70
38 222
37. 43g
Feature Propagation
32 51
33.0%
34.89,.4
36.19
36 12::25,..3
IN2V (owa)
95.15 15.00
35. 92
37 96.
37.72 m
38. 38.07:.1
IN2V (own)
32 12.12 73
33.53
35 35.38
36.039.s4
36 66..6.
Original features
35, 13
36 20
37. 79..2
38.75c.
47.57:
Original features
31.77o
33.64
36.48.54
37 38.33
36.71.2
N2V (ransdacttvv)
$2.35
34. P0g
30. 35
37.331, ko
3V.83, 45
NIV (craneductiv)
30.82, 10
$2.52 71
34.44
35.45,,1
36.54g.07
Amazon-ratngg
Amsson-rttnng
N2V (inductive)
37.36 37.366.3
37.93e
38, 30..5
38.54
38.571
N2V (inducrive)
30.144.57
41.4%..u5
44 44.98h 01
48.971.a6
50.98%.25
Feature Propagation
30.56g 79
42.99 22.92,m) oz
47.5% s6
50.90; no
53.07
Feature Propagation
39 39.50. s5
42.95, so
48.3 48.31p
51.250 ea
53.90, 11
IN2V (own)
41. 24
45. 45.644..7
51.00 51.91..40
55.96,
58.44
IN2V (own)
40.2 29
43.72,,0
50.99,84
55. 55.442..8
58. 68.70..s
Original features
80..1
41. 77.
47. 47.200.30 2e
50 50.822.
54 1000 es
Original features
39 20, 05
41 11
48 07..s
63.0th..9
57.34g.ur
N2V (transdactive)
41. a3
25, 12
50. 30.920.cm
53, 933
56.78 42
NIV (transductive)
12 58
48.31mm
61.60l
51.84
57. 17.05,.0
Roman Roman-empine
Roman oaman-empire
N2V (inductive)
62.370.51
61.35p.se
65.076..8
68.456.10
06.541.0 07
NIV (inductive)
61.71o
68.520.58
72.78p.s:
76.08u.54
$1.16,.m
Feature Propagation
62 122.1
61. 16
65.814.s5
65 01
66.31,.13
Feature Propugation
62 26, u
67 67.03p.44
71.74uu5
76.02a.75
80.31o.u7
IN2V (owa)
60.57
63.5 a3
65.36,
65.63 w
65.99,
IN2V (own)
59 29. sr
66. 66.19).5
71.50p-a
76.07a.
80.48.
Onginal features
63.46,
65.05
66. 16.477...
66.83,
66.55,
Original fentures
66. 66.09.
70.28, 68
74.41p
78. 78.560
82,90,
(mansiactive
63. se
60-08b
00-322..25
NIV (ranssiucrive)
66 06.366.27
71.000,47
18-396.44
82.41o.4s
84.070.00
18

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 11. MLP on different iN2V setups concatenated with the original graph features.
Table 12, GraphSAGE on different iN2V setups concatenated with the original graph features.
Percentage of training data
Percenlage of training data
Dataset
20%
60%
Dataset
10%
20%
40%
60%
80%
Corn
Cora
N2V (inductive)
66.48
70.54,
75.47
76.09g
76.27
N2V (inductive)
73.97a.s
81.92,.4
85.97,
86. 86.97..as
87.38g
frozen (A ==] 1)
76.83,
79.70g.44 .4
70
85.31 as
85.062.ar
frozen (A 1)
76.36
81.51,,77
85.12,.
86.620 86.62..7
87.60)
post-hoc
82.01 82.01..a8
62.53 .s
85.81
85.63
87.122.08
post-hoe
81.01
81.98
86. 86.13..08
86.551.s5
86.644.20
p-h w losses
81 88
83,27 m
86 45.83
86 27,
86. 0b..0
p-h w losses
81.45
82 822.2
85.9 85.921.37
86.35 86.35
87. 37.411,mm
p-h sampling
$1.721.20
83.97 1.36
85.37,,35
85.81
86.40
p-h w sampling
80.97,.47
83.28 83.28_,41
85.71144
80.771.42
24.2
Citeseer
Citeseer
N2V (inductive)
66.06,
70.04,.0
72.33
73.58 14
74.6%2.12
N2V (inducrive)
68.522.47
72.49.14
74.63 22
75.851.40
76.761.8
frozen (A - 1)
65.50) sa
70.19h.99 on
73.53 n3
74.35 96
76.49).m mo
frozen (A A1=1) 1)
65.62 ss
70.69
73.99 .47
75.77 1.9
77. 77.09, 77
post-hoe
66.75,.11
71.24,.00
74.13
75.850 at
76.402.2n
post-hoe
66.79L.s
71.14
73.91
76.35,.2
75.50
p-h w losses
66.98%.14
71.35
14.37
75.70,
76.499..a
p-h w. losses
67. 67.11
71.14a.s
73.51,
75.944,44
76.55
p-h w sampling
67.55 .s.
70.35
71 71.30
75.92, 10
27.09
p-h w sampling
66.69,.s
69.10,.au
73.171.54
75. 75.73
76.13
Pubmed
Pubmed
N2V (inductive)
83.94p.az
85,35m.
86, 86.733.u
87. 17.211..u sw
87.40a.
N2V (inductive)
81.33,..
86 06..5*
87.32g
87 87.322
88. 38.011.7
frozen (A= = 1)
83.40 83.400.62
86. 86.170.49
88.018g.0
88.88 oa
88.97o.m
frozen (A 1)
81.23g os
84. 84.866.5s
86.74g
87. 37.92s..
58.38p
post-hoc
87 87.05
88 0la.45
89, s9.1du
00.001..m
post-hoc
82 82.83..77
81.76a.s
86. 88.78...s
88 .9.66...
89 23
p-h w losses
45.30 46.376.40
87.07
88. 160,49
60.291o.4 44
89.62p.0e
p-h w losses
82.76o.71
85.00
86.680.4e
88.36%..7
80 0..60
p-h w sampling
85, 38
86.85
88. 69
89.466 89.46u.a
89.90u.s
p-h w sampling
83.02u.7
84.822.4
87. 11..5
88. 36.599..s
89. 19.39
Computers
Computers
NEV (inductive)
80. 80.1le
82,95u.e
81 11.088.7
81.3
81. 720.7
NEV (inductive)
86.61
89 89.06
901722..s
91. 11,333.0
91.700u.ss
frozen (A m 1)
86.00c.6s
87.9 87.99 .0
80 80.477.36
90.21 4s
90. 190,64
frozen (A - I)
88.52o.51
90.17
91.16p.55
91 11.633.00
91..0pp.z
post-hus
88.322 on
89.37o.00 36
90.466,44 41
90.37o 54
90.644.8 so
post-boc
88.8 88.83 si
20
91.47o.50
91.80p.a ne
92. 92.01l.aa
p-h Josses
88 38.3Ho..u
89.63o.8
90. 106,s9
90.68
90.89o.74
p-h w losses
88, 8...8...
90, 16o.Lo
91--33..00
91. 90.9000.s
81.85p..1
p-h w sampling
88.66 88.63,.2 na
89.75, 2o
90.677.91
90. 90.99 se
90.95,
p-h w sampling
88. 90,
90.50%
91.47
91. 01.911..c oo
91.72p.ar
Photo
Photo
N2V (inductive)
85.733 no
86,22, .22
90.7lp.e
90.90l es
91.76g no
N2V (inductive)
91.97,m
93.87 p.
94.944. 44
95.37%.0
96.56f
frozen (A = 1)
92, 92.133.3
93.6fhy.s
91. 5.5.97777
95.10g
95.3 95.32
frozen (A 1)
93.0H
91.29
95, 95.25
95.5
06.638m7
post-hoc
93. 86.
94.546.37
05.246.a0
95.31o in
95.456.c o
post-hoc
93. 44
94. 14.633.a 45
95.236.3 31
95.526.a9
p-h losses
93. 85
91. .0
95. 21g.3
95.28u
95. 35.69
p-h w losses
93.5
91.56uas
95, 14.11
95.14a
95. Auuas
p-h w sampling
93. 99,
94.55..20 20
95. 33.
95. 42. 45
95,70
p-h w sampling
93.87
04.68
95, 95.26,.. 28
95. 06..59..r
06.60
WikiCS
WikiCS
N2V (inductive)
13.16,17
Y5.00u.ra
76. 76.79...
17.300.77
76.71
NEV (inductive)
76.30..s4
79.56 20.500..ss
82. 82.230.75
83. 83.70g.es
84.79u.s7
Frazen (A- 13-1] 1)
77. 07.52...1
79.41a.m
81. 81.466ua
62.94p
83.87
frosen (A A 1)
79.29uu
81.62a.51
83.200u.7
84.44..0
45.38
post-hoe
$1.82 89.
83.34b.sa
84.37 84.37o.
85.277.49
85.626.0
post-hoe
81. 81.50a.04
83.30p.e2
84.410.51
85 43,443,.
85.97a.7
p-h losses
81.88c..0 so
83.3To.50
84.7To.5
85.41o
88.05 ea
p-h w losses
81.544.cc
$3.30g.s
84.4To 64
85.25%
86, 00,000
p-b w sampling
81. 7..88
83. 33.477.47
81. 1..11a..
85. 15
86 s6
p-h w sampling
81.36 81.36
83 38..5
81.62...
85.29k..e
85, 85.733
Actor
Actor
(inductive)
09
35.344.01
36.64oo.s
220.70 70
38.018..2
N2V (inductive)
32.0Ho.99
35. Aol.o
36.13,.a
36.05g 40
Frozen (A = 1)
33.07
34.326.97 97
36.57
37.91g
37.80
frosen (A 1)
31.29o.e
32.75
34.75o.7
36. 23, 14
36.46,
post-hoe
31.96i 78
36. 16.10
37. 13
37.75, 10
38. 98.11..u
post-hoe
32. 12
33.18g.45
31.89k.
36.041.m us
3G 06.66
p-h losses
34.97o.54
35 35.95n.0
37 10.19
37.53, oz
37.32,.2
p-h w losses
32.82..74
33 83
35.82 35.822..0
35.2
36. 76
p-h sampling
35. 15;
35.92u.5
37. 66
37.7
38.071.12
p-h w sampling
32.79t.u
33.533.s
35.38, aa
36.03..
36.58, co
Armzzon ratings
Amazon Amazon-rutings
NEV (inductivv)
37. 36
37.93u.5
30g.to
38.51p au
38.37
NEV (inductive)
39.14
11. 11.13a..5
11.99,oo .oa
15.97..15
50,98
frozen (A = 1)
39.49..78
41.98
48 18.032.84
49.49p
51.99,.2
frozen (A I)
38.76p.0
42.25 12.28
18.161.54
50.02,,ro
54.00% 14.00%
post hoc
41.24g
45.64..07 67
51 89
55.90p 77
58.69 38.69%.4
post-boe hoc
39.933.c
43.45
50.70p.51 51
55.12,
58.70p.s4
p-h losses
11.11c.00
45.36i..0
52. 4:
$5.960.s
lo.07
p-h w losses
19..99.07
43.72
54.631o.0
55.2% 58.299.05
p.h sampling
41. 11.26,
45.73
51.91g
55.90l os
51.69),00
p-h w sampling
40. 133
43 13.91,
50.99,
55.
49.54, 30
Roman-empire
Roman-empire
(imductive)
62.3
82.37o.51
64.35..39
65.677,.1
66.48%.70
66.54..07
N2V (inductive)
64.71o.c o6
68. 52
72.78o.31
76. 1G..88o.o
81.16.
Frozen (A =
60.57
63.644..1
65.36
65.98
66. 66.11
frozen (A = 1)
57,86, 21
65. 65.40a.5
70.93p.
75. 18.188.57
50
post-hoe
60.33c.as
63.49h.33
65.07o.39
65.83p an
65.971.8
post-hoe
57.94.
65.576.44
71.50p.4 43
76.07o. 09
80.48o.70
p-h losses
60.57c..u
63.59y.az
65. 28.28..34
65.81a
65.97,..2
p-ht w losses
59.29p.
66.10am
70.45
75.622.ms
99.92u..
p-h sampling
60.04c.ar
63.57o.30
66. 16,36
65.86p 59
65.99L.12
p-h w sampling
55.56
16.19
70.8%v.38
15.15
p9.91p.21
19

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 13. Comparison of best iN2V variant vs baselines; GAT accuracy. Gray numbers are not directly comparable as they use additional information (graph features/ranndductive setup).
Table 14, Comparison of best iN2V variant vs baselines; GIN ac- curacy. Gray numbers are not directly comparable as they use additional information (graph featurestransduccive sctup).
Percentage of training data
Percentage of training data
Dataset
10%
20%
40%
60%
80%
Dataset
10%
20%
40%
60%
80%
Cor
Cora
N2V (inductive)
69. 69.16,.ce
77. 14
81.97
83.47:42
85,
NIV (inductive)
68. 352
74.18
$1.76..10
63.97
85.20,
Feature Propagation
.1.90..a 29.10,.0
83.021.75
24.30..2
85, 85.312-43
Feature Propagation
78. ar 3r
81.02,.56
62.401.72
64.321.10
85.13,.
iN2V (owa)
79.70,
81. ur
83 .5.90..4
84,58
85. 68
IN2V (own)
79.58;
81. $1.21::20
84.03; 25
84.911.9
85.06
Original features
81.3 11,55,as as
84.3 14
86.011.9
88 18.23
88. 10
Original features
82 82.40;, as
84.06
86. 36.70,.3
88.081.06
87.344,.0
N2V (transdactiv))
79.20,
81.8 $1.89,1
81,98 81.16
86.94).0
N2V (tranduuctvn)
79 00.09, as
81,56, ss
83 83.25 as
85.31o.s5
86.09,.s
Citesoor
Citescer
N2V (inductive)
51.54g
58.89,uu
65.81, .s
70.12:
73.00g
N2V (inductive)
50.40s so
58.06; 75
64.802.
69.31 0s
72.91
Fenture Progagation
56.7332.0
62.29 90
06.42,,0
70.47_.10
72.79g
Feature Propagation
56.45g 41
00.722.14
85.97
70.48,.31
73.330,
IN2V (own)
58. 18
.4
71.02 67.12,.0
73.42 71.022.00
01
IN2V (own)
69, 22
67.03,.05
70. 70.74..45
72.66 72.642.21
Original Teatures
71.644.30
73. 0l
75.94
70.78..4
77. 06
Original features
71 0000
73.7 73.74
78.44s os
76.39u.77
77.18
N2V (transdactive)
37.50,.s on
66.76
18
72.61g 37
N2V (transductive)
$7.31, 5s
61.86,
66.49,17
70.48, .o5
73.24x.0
Puhmed
Pubmed
N2V (inductive)
77.35 72.7
81.74..0 t
82.83e.s
83.020 71
NIV (inductive)
76.22, as
79.78, oe
81 81.91a.5
62.78g.47
$3.07%,w 92
Feature Propagation
79.30u.47
80.S1a.m
81.87o.77
82.93o.s
83.06la r5
Feature Propagation
78.45p.s
19.19..1
$1.50psa
62.71a.42
83.09
iN2V (own)
60.65..u
81.80p.s:
82. 62.77o.sa
83.17c.u
83.34g 72
IN2V (own)
80. 51
81. $1.60,.5
82.64..7
83 33.288.55
83.07
Original leatures
85,074.42
86. 42
47.4s
88, 28.221..4
88 86u.42 42
Original Tentures
84 24
87. 37.3s
88, 88.477..0
N2V (tansdaccive))
81.32 38
82 09g
83. 41.21
83. 16
83. 5ill al
NIV (tranductive)
81 310 46
82 62.210.40 46
53 230
63, $3.170.57
83, 13.166.00
Computers
Computers
N2V (inductive)
83.442.61
87.80
90.40p
90.92,.0
91.18g
N2V (inductive)
86.06; 24
88.86, 57
90. 90.222
91.00g.sp
91.36%,
Feature Propagation
K3.00
86 05
80.850.11
90. 70c.65
91.28%
Feature Propagation
84.34p
87.8%
89 88.61p.0
90.944.39
91.083.su
IN2V (owa)
87 80
$9.199...
90 600.15
91. 1..6....
91. 00
IN2V (own)
88. 48
89 630,
90.71a.s
91. 12..2...4
91.-45..5
Original features
87.0 Me
90. 17
91. 01.336,.0 2o
91. 31.89g.a 2n
92.476 so
Original features
87 37.57
89, 176.ss
90.97o
91.43g..
91. $1.85
N2V (transdactivee
80 20.2%%.o
90. 10.355..
91.01..
91.33,
01 11.33lo os
N2V (transductive))
68 38.97
89 19.95,. m
90.71p.4
91.22 46
91.39%
Photo
Photo
N2V finductive)
m
80, es
92. 62
93.09 os
54. a7
N2V (inductive)
56. 58.95 m
90.50,.s4 64
92.19p
93. 03.133.47
93.56,.c
Feature Propagation
87.18
90.21e.::
90.05c..0
93.40u
Feature Propagation
87.68, 15
90.356 20
92.07a.5
92.866.41
90.200p.u
IN2V (own)
90.466
91. so
92. 92.77..5
93.31...
93. 18
IN2V (own)
60. st
9/2 25.277..5
922881.33
933.45o.m
Original Teattures
93. 870.0
91. 42
95. 0.3u.s
95. 27.
96. Sinas
Original features
83 o2
94, 30
96 05p.00
95 95.3/Au.s
95. 15.700..o
N2V (trasdactive)
91.37
92. 50
93. 15..e
98.69g.a oa
93.36l ss
NIV (transductive)
91. 15p go
92 22.222.. ow
02.93g..
93, 20.00y,.o. c
93.59 oo
WikiCS
WikiCS
N2V (inductive)
71.582.00
76, 91, 2e
79, 96o.rs
80.87,
82 08%g. 87
NIV (inductive)
76.68, as
78.50) 54
79.855..s
80, 60,920. so
81.99a,
Feature Propagation
428..0
77,89,
80.11..9
81.27c.m
22,
Feature Propagation
78.43,.18 76-73gar
N0.34
80.9hu.se
IN2V (owa)
78.22 21..5
79. 63
80 370.s
81.56
82. 11
IN2V (own)
78.5 51
80.03..7 m
80.960 80.96..3
81. 560..s
82 69.
Original features
80.86 sn
822 65
84. 10 on
84 ml
84 06 oz
Original features
80 00.8 sp
82. 21..00 si
53 80h
844 730 sx
85, 45,.4
N2V (transdactive)
79.5
$0.6 65
$1.966.0
82. 2225%
82 96g oe
N2V (transductivve)
79. 20. ss
80,56 79
$1.66 6.
$2.50 .s
82.55k
Actor
Actor
N2V (inductive)
25,38,
25. 21
26. 09, s6
25. 66.66.... ca
25. 28 as
N2V (inductive)
25. 93, o7
25. 84, oz
25.70, Rs
24.91 .os
24.62,
Fenture Progagation
24.97 Ls
25.00 63
25. 18.96
25.53k, 62
25.07, 32
Feature Propagation
25.20 92
25.72, 58
25.50
25. 66..6.
25.29
IN2V (own)
25.5
25.89%..
24. 800
24.98%.
25 20h
IN2V (own)
24.23 46
15
25 69
mo
25.6533
Original leatures
30 S2.a
32. 23
32, 96
33. 33.72:..
31.35%
Original features
30 20.216.r 72
31.80p sm
32 97..6
33, 70.700.m
35. 355550...
N2V (transdactive)
25.20g st
m1
24.50 24.50,ar
24.30 mm
24. 24.50p m
N2V (transductive)
25.25, m
25.37
24.15
24.86, s4
24.99%
Amazon-ratings
Amazon-mtings
N2V (inductive)
39.33o,11
43. 060 se
45.92o .s
48. 45
50. 62
N2V (inductive)
10 10.58p.27
43. 13.566.52
16. 18.299.94
19.8:1..4
53. 58.49%..o
Feature Propagation
40.97o.s4
42, 922,
46 444..7
48.187c.54
51. 21,
Feature Propogation
40.74
43.240.50
46 85p.5
50.56
588.85 53.883,.0
IN2V (owa)
41. 11.49
43. 43.58
5:
16.35 55
49.22...
51.50; 4s
IN2V (own)
41. 17 m
43.32
47.35
50.93
53.59, os
Original leatures
40.67o.10
48
46 D6.06...0
48, 75c
50 64h 70
Original Teatures
39 87
41.486.0 sa
46 30
48.181.17
52.18
N2V (tamsdactive)
41. 41.788.6
40. 99.22...
47.88 27.
40.85
51.57/23
N2V (traneductive)
41 77.77 52
44.476.4 46
48 34
50.38g 50
52.44%,
Roman--cmpire
Romsan-mmire
N2V (inductive)
13.86,
13.90
14.30. so
14.48
15.90,
N2V (inductive)
13. 13.89
13.82, so
17 17.24g
22.57,
26.06,
Feature Propagation
13.946 13.88,.2
13:94, 14.21o.37
14.32,,1
16.04.
Feature Propagation
13.89b
13.83,50
16.582o
19.742.0
24.87..0
IN2V (owa)
13.91o.10
13.00 94
16.06..0
17.49)
IN2V (own)
13. 32
13.980
18.34
20.322.67
24.94:41
Original features
52. 02.23,.. km
50. w:
68. 96..0
74.95c.4
78.89
Original features
58 01
62.58
09 08.2%
73.7
70.29%.oo
N2V (trandacctvve)
13.75p
13.63k c3
14.01..39
14.13c
13.73p
N2V (cransductive)
19.61, 07
21.1 1
24.47
27.82, am
31.55
20

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 15. Comparison of N2V vs different iN2V setups, GAT ac-
Table 16. Comparison of N2V vs different iN2V setups, GIN ac-
curacy.
curacy.
Percentage training data
Percentage of training data
Dataset
10%
20%
40%
60%
80%
Dataset
10%
20%
40%
60%
80%
Cora
Cora
(imductive)
69.166..2
77.141.54
81.97
83,47: 42
85.20
N2V (inductive)
68. 08.382.s
74.182.8
81.761.a0
83.97.39
85,20,
Frozen (A [A-1] 1)
75.96,
80.49
82.91
84.451.08
85.06
frozen (A 1)
75.572.00
79.451,a
82.231.20
83.69,.19
84.582..0
post-hoc
79.52 as
81.15 ms
83, 85.555..
84.21 a1
85.42
post-boc
79.51 ,9.42
80.66 os
as
84.10,mm
85.08la:1
p-h
79.16,.
81.261 os
83. 10..0...4
84.35 s4
85. 25.68...
p-h w losses
81. 1..211.44
83.53
83.7
85. 85.72 85.72
p-h sampling
79.70,
$1.74,.0 25
83.444.0s
84.58
85.351..0
p-h w sampling
79.58, 79.58 16
81.211.
83, 83.15p
84.91..1
85.06
Citeseer
Citeseer
(imductive)
51.54g.24
58.89by.o co
65.811.81
70.12 111
73,003
N2V (inductive)
50.40g,a o0
58.06
64.832.. 10
69.312..0
72.91
Frozen (A =
57.13,
61.47, a
66.181..
70.09..4 s4
73.242.1
frozen (A I)
56.23 56.23,.s0
59.0021s
65.571.50
09:91
72.88/2.0
post-hoe
57 19
61.381,35
67 00.0..
70.72) 6s
72.671.07
post-
57.78, .0
61.42
66.80) se
70.74..45
73.12
p-h losses
58. 13.
61.744.s
67. 43.43
70.18; x
73.42.m
p-h w losses
S7.69,
61.242.1
68.77s..0
70.8T
72.97
p-b sampling
58. 28..0s
61 .6,477
67. 12,,00
71. 02
72.97
p-h w sampling
G7.761.a0
$1.55,.0
67 00.
70.24; o2
72.64
Pubmed
Pubmed
N2V (inductive)
72.71s.
77.35p.u i
41,44
82, 22.830.a as
83.02o.74
N2V (inductive)
76.23,
79.78j.02
81.91o.52
82.730.4
$3.07p.32
Frozen (A =
79.23 79.25
80.45
82.11o.e
82.87o os
83.211..
Irosen (A = 1)
78.15p.54
80.233
81.7tlo so
82.57
post-hoe
80.57cass
81. 17.87..s
82 45
83. 27
83.31
post-hoc
80, 80.15o.s4
81. 60..Gs
82. 52.530.58
83. 39.....
22o.70
p-h losses
40.65
81.80p.51
47
47.447..4
83.375..2
p-h w losses
80. 40.449..
81.58%
82.77
83. 83.21
83.07
p-b sampling
80.7 70.71o..
81.79u.a
7T.as
83, 83,20p
83.09u.s
p-h w sampling
80. 80.42g.a
81.52a.s
82. 82.6
83.28,.s
83.10
Computers
Computers
N2V (inductive)
83. 83.442.61
87.80h ex
90. 90.10k..s
90.92 so
91.18
N2V (inductive)
86.016,.24
860.86
9012230.44
91.000g.2
91:36ia.7
frozen (A =
86.02a.79
68, 23w.36
90.00%..4
90.55p.47
91.06a.s
frozen (A 1)
86, 88.622.c4 c1
88.36a.
80.966 .sa
90.76,.a1
11.18
post-hoe
87 35
89.20h 89,20h.34
90. 28
90.89g
90.7 14.74
post-boe
88, 21.
89. 78
90.34g.ss
90.88
91.25 91.27a
p-h losses
87.770.50
80. 19
90.31 al
90.977.44
91. 130.57
p-h w losses
88. 10a.a
89 69.633.42
90. 30.744..4
91.122.44
45
p-h sampling
87.80g.al 41
89 89.32
00.600.c
91. 91.066.50
91. 91.06%.0
p-h w sampling
88. 58.43
80.600.4s
00.71a
01.322..1
91.45u.2
Photo
Photo
N2V (inductive)
86.95 7s
88.80b, 1
92. 62.
933.09g.as
93.544
N2V (inductive)
88.95,.m ne
90.50
92.19% .s
93. 13.133..7
10.56p.xx
frozen (A
89 79
90.733..0
Ala.s:
93.14g..
62..72
frozen (A 1)
89.45a..
91.04q.47
92. 92.40g.4
93.11l.s4
93, 13.15
post-hoe
90.41o.45
91.26p.50
92.5 92.5
93. 13.1
93.25
post-hoc
90. 90.60)..4 54
91. .00
92.34
92. 7..76.7
53.45g
p-h losses
90.46
91.536.
92. 37..
93.066 s6
931552,
p-ht w losses
90. 90.60u.s
91. 91.58s.
92. 92.366..4
92.81o.2
98 98.23
p-h sampling
90.19
91.71, .l.72
92.
93.311 1o.00
93. 18..t
p-h w sampling
90.559..e
91.53p.44
02.27o.45
92.60%.o
83.02
WikiCS
WikiCS
N2V (inductive)
71.08
76.91,.av
79. 19.96
80.87, os
82.0 82.06y.s7
N2V (inductive)
76.68
80.500..4
79.8% .s
00.922, so
81.94a so
froven (A- 1--1]
78.51,.0 ..
78.09)
79.56..se
82.04o.7 81,29l..4
frozen (A 1)
77.170.e
78. 11a.s
80.146.a
80.76,
82.26 22.226u
post-hoe
78.1 lo.rr
79.45..39
80.333,a0
81.000
82.32
post-hoe
78. 48.500.es
79. 98.88
80 80.89y.51
81 81.91lo.s
82 60
p-h w losses
78.00
79.55 sa
80.37o.03
81.12 as
82, 130,
p-h w lasses
78.51
79.91a.7
80.95 5o
81.566.0s
82.26
p-b sampling
10.63,..z
80. 45
81.50 81.56g.su
11
p-h w sampling
78. 62..6
80 30.033.75
80. 50.96o.5
81 9T..s
82 67o.77.
Actor
Actor
N2V (inductive)
25. 25.38,11
26. 09.00
25. 6Gu
25. 25.28
N2V (inductive)
25. 23.333.
25. 8 81..77
25.70 25.70y.m
21.91:.us
21.62,.u
froven (A-
25.233..1
25.79b.68
24.90g.o3
24.98p 84
25.20
frozen (A 1)
25.71 1o.78
25.48,.18
25.646.50
25.266.
25.32
post-hue hue
26.01..o
26. 03.033..
25.466..6
24.97 ea
25.01
pusi hoc
25. 10..0
25.5 25.583.05
25.061.54
25.18
25. 25.96uu:4
p-b w losses
25. 15.41,.m
25.8% 25.89u.s4
25. 70
21.91pss
25.50,..7
p-h w losses
25.30
25.761.01
25.94..50
25.400.15
25.532.s
p-b sampling
25.533c as
25.66, 14
25. 15.58;.19
24.91 dm
23.99 23.99
p-h w sampling
24.23,
25.52, oo
25.74a s0
25.12:
26.49p.a
Amuzon-ratings
Amazon-ratings
N2V (inductive)
39.333
43.06l..s
45.922).o
48.58p
50..7o.c2
N2V (inductive)
40.586.17
43. 13.56a.32
46 20,2 94
49.54g
53.49u.es
frozen (A= = 1)
40,22 sa
43.30h. in
48.35..5e
49. 22
51.56,.0x
frozen (A 1)
40. 16u.s
43.42oua7
47 47.35n.54
50 60.93,.7
53.59..0
post hoc
41.48% 75
43.58p.1
45.85
47.450
48.00,
posi hoc
41.26%..e
43.18p..0
45.72o.71
48.02,
48.330
p-b w losses
41. 19
43.38 43.38y.s1
45. 75
18. 18.15
19.23,
p-h w losses
41. 41.17o.7
42,94a.s
15.844.35
48.79%.
50.19, as
p-h sompling
41. 63
43. 43.49u.s0
45. 76.77..2
47.19
48.89
p-h w sampling
140.77
43.32p..s
45.756.75
47.180
19.67
Reman--mmire
Roman-empine
N2V (inductive)
13.86
13:900.24
14.30
14.48
15.901.45
N2V (inductive)
13.8 13.800.24
13.82
17.242.64
22.57
26.66
frozen (A= = 1)
13.9 13
13.94, 29
13.88
14.42 on
16.86 m
Irozen (A 1)
19.89 a1
13.98%
17.36g 30
19.72
24.94 i
post-hoc
13.81aus
13.82
14.14am7
16. ss
17.49..
post-hoe
13.890.24
13. 13.966.as
18.34p.2 28
19.222.12
21.17144
p-h w losses
13. 13
13.86p.2a
14.21
14
15.63
p-h w losses
13.89p m4
17.00l 14.17
20.32
24.88
p-h w sompling
13. 20
13. 13.78u.a4
20
14.00p
15.65u.s
p-h w sampling
13. .3.992..7
13.85aau
18.07
20.19..15
21.721.7
21

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 17. Comparison of best iN2V variant vs baselines concate- nated with the original graph features; GAT accuracy. Gray num- bers are not directly comparable as they use additional information (transductive sctup).
Table 18. Comparison best iN2V variant vs baselines concate- nated with the original graph features; GIN accuracy. Gray num- bers are not directly comparable as they use additional information (transductive sctup).
Percentage of training data
Percentage of training data
Dataset
60%
80%
Dataset
10%
20%
40%
60%
80%
Com
Cora
N2V (inductive)
80.652.34
83. 12:
86. 86.018,.41
87,25
88.27; 4s
NIV (inductiv))
81 11.553 41
84.26,.18
86.53,.1
87.10,.a
87.38,
Feature Propagation
81.561.14
84.081.7
86.49
86.12
87.561
Feature Propagation
82.54; 40
84,16
85,831.44
87.44..34
87,38,.42
iN2V (own)
$1.68,.28
84.11:.12
86. 70.70...
87.99c.
87.90, 53
IN2V (own)
81. (1.8% 23
83.80..z
96.511.12
87.92...1
87.79..4
Original features
81.35,
84.38
86.51,
88.21c.s
88.15,
Original features
62.40
84.05o m
86.70 as
68.06; .s
87.34
N2V (ransdactive)
82 .9
81. 90
86 98g
87.68..0
88. 27,
NIV (trnsducting)
82, 42g
84.22%
56 13:13,.2 2s
67.21,,00
88,31,
Citeseer
Citeseer
N2V (inductive)
71.10,
73.02
74.79h
76.93
77. 686
N2V (inductive)
71.04:
73 646.
74.50%
76.68..96
76.649.0
Feature Propagation
60.46
72.03, 48
76.30_.2
17.15
Feature Propagation
69.51
72,20.22
75.12
76.87.a.
77.60g.m
IN2V (own)
68.041,3
71.48pas
75. To.01
76.60.42
77. 17.102.0
IN2V (own)
08.47o.a
71 71.62,.1
14.31
76,76
77.36120
Original features
71.644.30
73. 23
75.94
70.78:
77. 0f
Original features
71 66 as
73.74..1
75
76.39
77.18
N2V (transdactive)
70.91.
73.12
75.89
70.71:
77.15
N2V (transductive)
70.30% sp
72.06 7M
74.69 mz
75.911.4
77.21x.0
Puhmed
Pubmed
N2V (inductive)
51.12
85. 25...l
86. 12..4
87.24c.45
87. 87.92p.a3 63
N2V (inductive)
81.01ear
85. 85.5S
15..55p..2
87.29%..r
86 05.055..s
Feature Propagation
83.93p.4
85.30
86.6 66.611.30
87.18
87. 630
Feature Propagation
84. 16, 22
85. 33.55 so
86.8Ansi
67.51g.30
88. 88.043.5
(own)
k3
860
86.446.53
87.57c.4
97.
IN2V (own)
83.75 10
84.966.3
46.79u.s.
87. 61.66
98.00l..s
Orignnal features
85.02,.0
85, 79..
87.47o
88.2 21.2
88.850
Original features
84.94.
86.03,
87 al
88.0
88. 88.476.00
N2V (ransdacctive)
85, 20.50
87. of
87 20
88 23c 5o
88. Slla.ss
NIV (tramsductive)
81.00% 40
80 00.000
86 636.6
87, 37.3
87. Fl.7
Computers
Computers
N2V (inductive)
85. .4.50
89. 10.01l.r
90 80.830..1
91.64g
92 290 s2
NIV (inductive)
85.461 ao
88. 33.331,. ss
90 90.2200.s
91.300 de
91. 91.546..s
Fenture Propagation
87.25
80.97u.a
91.250.40
91.9K3,.s
Feature Propagation
87 87.07o
80.522.5
90.955
91.499.40
88.64
IN2V (own)
88. 388.72.. 72..
90. 28
91 510.s7
91. 94
92 12
IN2V (own)
88. 60,
90 09
91. 38..8..s
91. 76
91. 800.
Original features
17
91.366.29
91. 91.69. on
92. se
Original features
87. 37.57
89, a6
90.97o.
91. 91.48..
91. 91.85
N2V (trunsdactive)
50.544.00
91. 48
02 37
92.53
02.74 74
N2V (cransducctvv)
90.044
91.0% st
91.71g
8/2.04g
91 11.5%%
Photo
Photo
N2V (inductive)
91.32)
93.
94.93. ss
95.01c 4s
95.57o
N2V (inductive)
91 26, 30
92.96k.15
95.05a sz
95.35s.0
Fenture Propagation
9G.177
94.39g at
95.09%
95,25g,44
95.816 2s
Feature Propagation
92.87o
94,24) 50
94.92
95,20% ss
95.413,00
IN2V (own)
93.80 93.82
94.71eam
95 23h
95.61c..7
IN2V (own)
93.81
94. 75..76
915, 25
95, 52.5660.. is
95.766.77
Original features
93. 476
94. 42.
95 86.026..
95.37c ai
96. 5To
Original features
93. 42
94,300,5 54
96. 05.000
95.34o.s
95.70g,e 68
N2V (trasdacctiv))
91.33
95. 12
95.63c.o
95. 95.92
N2V (transductive)
94. 91.20
94.93a.54
95.60l.ss
95.65y.a5
95.922.5
WikiCS
WikiCS
N2V (inductivv))
77.9 77
S0. 22 as
82 82.036.as
83 86.00G...
84.37
NIV (inductiv))
77 77. as
80.180,4 48
52, 13.43
83,80p. 54
81.94.,
Feature Propagation
$1.51p.c 79.690.00
83,260.75
$1.43
94.95
Festure Progagation
79.43; 1s
81.61a.71
82.96pas
84,29u.s
84.74..02
IN2V (owa)
61. 21o.s
82, a0
83. 75
84.63, 81.63c..a
11
IN2V (own)
81. o7
82 98,.2
84.26..s.
841 800.s7
85.49
Original features
80.95
82 64
81.10.au a
84.61.
Original features
60.82, sp
82, 10.00.
$3.89,
64,75
85, 85.45%.51
N2V (ransdlactvv))
81.67o.6a
83. 0o
81.000.0
81.83,
86. 22
NIV (transducting
81. 70.0
83, L6 67
84. $4.18a
84.78o.sa ss
85 65.233, 7a
Actor
N2V (inductive)
30.70p.r
31. 31.71e
33.10g.sz
33.64.
34.01s
N2V (inductive)
30.39,
32.1
33.20 12
34.33
31.50,
Feature Propagation
31.83, 30.516.12
23.10,
33.35, oo
33.99, 54
Feature Propagation
30.25p
31.6%
33.08p
33.85
34.05,.o
IN2V (own)
80..1
32.35 32.35..2
33. 18.77
33.34_.s
34.21,
IN2V (own)
30.3 07
82. 624.54
33. 133.92293
34.08
34.87..1
Original features
30 828
32.22 23
32 96
33.72..m o0
31.39, a3
Original features
30 30.21o
31 80,80. 58
32 97.05
33.70a,7
35.50,.o
N2V (transdactive)
15
31. 18
32,50,
33.35
33.67a
NIV (transductive)
26 95p
30.56 so
32.45 mi
35,35, 10
33.86
A Amaxon-ratings
Amazon-ttings
N2V (inductive)
10.230 40.23
43.0 43.04l..s
15.80 15.80y.7
48. 15..5.
51. o3
NIV (inductive)
10. 10.16u
42.10p.5
15.30u.7
18.7
53.10s.17
Feature Propagation
41.74
43. 43.58
46. 16.856.75
51.49h.s 19.51c.u
49.00g.0s
Feature Propagation
11.300.73
43. 43.90..5
47. 47.44
50.39u.ss
53.59
IN2V (owa)
41.50pmm
m2
47. 47.16,.72
51. 19
IN2V (own)
32.
43.58
47.54a.s
50 99..7
51.23,.12
Original features
40.67o
42. 48p
46.066.55
48. 75.
50.64, go
Original features
39.87o
41.48, so
45.50p
48.16,.17
52.18;
N2V (tansdacctiv))
12. 02.68..
23. 65
18. 26
18
53.05o9
NIV (transductive)
12 2l
45.17
19 19.53o
51.996.74
51.00..o
Roman- empire
Roman- Roman-empire
N2V (inductive)
00,85,
57.09
63. 17
71.36
76.53 07
N2V (inductive)
57. 57.60, 97
62.77o.4
68.75, db
73. 73.50, s0
76.17
Fenture Propagation
12.11
54.87
02.250.a8
60.300.sa
74.91po
Feature Propagation
56.49, 17
61.366.50
67.70gmo
72.280.5s
75.40_.o
IN2V (own)
43.34t.32
56. (aa.
644 400.00
69 46
74.14
IN2V (own)
52 600
59.176.30
66 800.
71.91a.54
75. 75.4666,.4
Original features
52.230.s8
74.95c es
78
Original features
58 01.33
22.58
09 09.230.44
7o.7
26.29%.00
N2V (tranddactiv)
55.500..r
62.01e
68. 44.
73.50k
76.51
N2V (crassductive)
67 12, 64
13.50l
75.71p.50
70.50%
22

---

IN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Table 19. GAT on different iN2V setups concatenated with the original graph features.
Table 20. GIN on different iN2V setups concatenated with the original graph features.
Percentage training data
Percentage of training data
Datasct
10%
20%
40%
60%
80%
Dataset
10%
20%
40%
60%
80%
Cora
Cora
(imductive)
80.652.14
83.12,.22
86.0088..a
87:25 ov
88.27
N2V (inductive)
81,53,,es
84 64.261.18
86.553..10
87.40..2
87.381.
Frozen (A [ 1)
79.19
62,24
85.443,,8
86,90)
87.93
frozen (A 1)
79.73
S2.58).4
85,44, 28
86.42,,75
87.79,.43
post-hoc
81.75 a7
83.06, 07
$8.70,.ar
86.85, 1
87.75
post-boc
81.68,aa .a
82.83 os
86.51, 12
86.922..a
87.49,
p-h losses
81. 81.78..r
83.35
87.45 1s
87. 12,m
p-ht w losses
81.58
83.341.a
86.
86.62,.a7
87 87.00),sv
p-h sampling
$1.68) as
84.111.
86. 16.11
87. 87.99g 45
88.061.5
p-h w sampling
81 31.66..41
83.50s.oz
85.96m
87.32; on
87.79
Citeseer
Citeseer
(imductive)
71.10
73.02,.00 20
74.79
76.93, os
77.63
N2V (inductive)
71.04..r
73. p3.644.00
74.56 on
76.68, as
76.64g.00
Frozen (A =
67.90,
71.18
75.11
75.71 an
76.852.0
frozen (A 1)
67. 67.49:.u
71.32
74.31, .00
76.12:
77.032.92
post-hoe
68.10).as os
71.351..0 .6
74.061.0e
76.60, is
77. 169,96
post-hoe
68.47o.50
71.681.94
74..7)
76.75
77. 36.36
p-h losses
68 68.01s.am
71.48ass
73.71...
75.80, su
76.61..0
p-ht w losses
68. 68.20..0
71. 71.621.14
74.22,
75.62;..a
76.94
p-b sampling
24
14.14
74.42,.17
75.641.3 39
77.062.45
p-h w sampling
68.590.se
71. 71.622,4
74.62
76.94.,a3
76.882
Pubmed
Puhmed
N2V (inductive)
8...2
85. 15..00.
126.64
87. 87.24o 45
87.9 87.92o.oa
N2V (inductive)
81.01. 01.57
85 a3..a
86 (55..5 422
29..29
88 050.00
Frozen (A =
82, 82.833
84.9th..4
86.35
86,896 se
8T.170.54
Irozen (A 1)
82.46,.5
45,189
88.66ly c
87.27
87.65p
post-hoe
83. 10.a1
84.70, a0
86 Ma.s
87. 87.57044
87.9 87.97
post-hoe
83, 720.77
81. 81.96
86. 76..5
87 65..6m
88 09
p-h losses
43.5
84.80p.4
130,54
87.85o.0 87.51
p-h w losses
83.644.50
84. 44.999.45
86.300.54
87.44 87.433,45
88 022.0
p-b sampling
83.4loar
$4.65m.ea
86 61o.42
87 62.
88. 18.15
p-h w sampling
83. 75o.6
84. 590gaa
86 40.46..0o
87 5...6
88. 00..0.
Computers
Computers
N2V (inductive)
85. 85.44..32
89.01a.77
90. 90.533.54
91.644..2
92.29
N2V (inductive)
85.46, m0
88.31a.s
90. 90.299..5
91.30 91.30g..a
91.54p
frozen (A =
87 87.93
90.06w.36
91. 91.300g.e
92.03p.22
92.20u.54
frozen (A 1A=11 1)
88. 120.s4
89.78
91.01o.50
91. 11.688.8
91. 000.0
post-hoe
88 37
90. 28
91. 37
91,9kpa2
92. 23..4
post-hoe
88, 22.22
91.38p. 90.10a.24
91. .8..6
91.78pm
p-h losses
48.30 08.30
90. 90.25y.35
91 47a.s.
91.80o.4s
92. 12,.00
p-h w losses
88 62o.s
91. .3...0
91. 91.76s.as
94....5.
p-h sampling
88.7 38.72,.s as
90. 13
91.51p.a7
92. 12.133:30
92. 120.97
p-h w sampling
88, 6..60o.1
90.09
91.322. s4
91.84p
91.80p.sp
Photo
Photo
N2V (inductive)
91.82)
93. 17.17
91. 13
95.01g 15
95.37p.51
N2V (inductive)
91.22 91.26, mo
92. c5
94.8% cz
95.15
M5.35 56
frozen (A
92 sl
94.16
05.01..27
95.18p a
95.48u.0
frozen (A 1)
92. 22.6/o..s
94.11a.s
94.07o.st
05.233.45
96.25u.7
post-hoe
93.76..7
94.6%o.5
16.16
95.35o
95.78 78..8
post-hoc
93.644.67
94.62..0
95.31
95.39%.
96.73g.62
p-h losses
933 933666e.s sa
94.536.3
95. 10p
95.16, in
95.42,
p-h w losses
91. 11..22,. on
94.51a.4
95. 36,114.2 3s
95.35, 67
p5.70b
p-h sampling
93.66 82.88...4 64
94.71 7l.
95. 23,,4
95.61..47
95.75%.ce
p-h w sampling
93.81
94. 75g.77
95. 95.25 sa
95.566
95. 75.76
WikiCS
WikiCS
N2V (inductive)
77.9 ar
80.22a.s5
82. 03.ss
83.0Gu
84..7
N2V (inductive)
77. 77.09L.ea
80. 80.48
82. 82.430.r7
83.80p.54
84.94p.s
froven 1--1]
o0.10g..z
81.17o.60
62.72..06
83.16o.70
83.90g.sa
frozen (A 1)
79.26 os
81.15g..0
82.966.74
84.02,.05
84.65
post-hoe
81. 21
62.00u.22
83. Mousa
84
85. 1o.m
post-hoc
81.330.
82. $2.98
12.442
84. 84.80
85.27p.4
p-h w losses
$1.02
62.622.71
83.6 33.6lo.
84.71p
84.87o.se
p-h w losses
81.26g
84.2 84.26 82.97a.70
84.0 64.911.s
p-b sampling
81 7s
82.70..st
83. 83.75
81.63
.5.29..
p-ht w sampling
81.37
83 33.05
81.180.0
81.83
45.49p.a
Actor
Actor
N2V (inductive)
30.70a.rs
31.71umm
33. 10g,ss
33. 33.G-lu.9
34.012.00
N2V (inductive)
30.39
32.1Ja.7
33:20,
34. 34.32..s
31.80,
froven (A-
29. a7
30.97o.s1
68..77
33.28 as
33.2002u0
(A 1)
20.80,.os
31.466,.00
32.08o.50
33.44,,15
34.372.04
post-hu hoe
30.42 5s
32,00h
32. 83
33.66% an
34.24,
post-bov hos
30.69,, nw
32 $2.67
33.46, 11
34.08
34.87,.11
p-b w losses
30 10.66..s
$2.35
33 18
33. 81
33. 11,.0
p-h w losses
30.3 30.57
32 G2aus
33.92
33. 33.63,,u
31.13
p-b sampling
30.51..1
31.81h, sn
33.22 w0.
33.34,. an
34.21,
p-h w sampling
30.600g.r m0
32.333
333.2% sa
33. 33.72:... .9
34.42
Amuzon-ratings
Amazan-rating
N2V (inductive)
40 40.23k.re
43.04p.co
45.80k,75
48.
51.23
N2V (inductive)
40. 16.66
42.10p,50
45.36l
48. 38.74p.s
10.10
frozen (A= = 1)
40.96i. ca
43.35
47. 17.16.
49.8 51
51.76,
frozen (A 1)
411966..c
43 825.2.
47.54..0
50.99%.re
54.23,
post hoc
41.30g
43.44..55
46.318o
48
51.48,30
post-bov hoc
41.40p oi
43.553
48.8To 57
50.27 77.
a1.74
p-b w losses
41. 50,
13. 13.42
16. 633.
19 19.08u
51. 19
p-h w losses
41 11.32
43.50
16.7 16.744..
50.533
53.21
p-h sompling
41. 11.37
43.00 62n.st
48. a9
48.9 48.971.41
p-h w sampling
41. 11.26
43. 68.28
46. 46.79u.se
20.044.,z
60.22,
Reman--emire
Roman-empire
N2V (inductive)
49.85
$7.09 57.090.56
71. 71.36 63.17
76.532.07
N2V (inductive)
57.60p.r
62.777.40
(88.75g.0
73.50p.c
76.17p60
frozen (A= = 1)
43.20, as
54.02 ss
61.61o.a3
68.65
74.19
frozen (A 1)
52. 52.23,
59.19b.a6
66.866p 16
71.916.4
75.466
post-hoc
13.31,.12
54.62,.04
61.400a.
69.16
74.14..0
post-hoc
51.27..54
58.78a.u
08.50u.st
71.8% 71.89..47
71.81ee
p-h w losses
43.26,
65.04p.30
63. fo.05
66.72pm5
73.95
p-h w losses
52.69u.n6
50.35
06. 06.27o.51
71 22.
74.97
p-h w sompling
41.55g.es
63 G8u.r
67.16, on
73.66
p-h w sampling
50.42,.us
59.17a.as
06 00.83
71. 71.84pus
74.79uau
23


Reviews:
[{"review_id": "pFRAxoP7A2", "reviewer": "Reviewer_VRfA", "summary": "This paper introduces an inductive approach to adapt shallow node embeddings for predicting unseen nodes that have at least one edge connected to the training graph. The proposed inductive node2vec method integrates post-hoc processing of node embeddings alongside corresponding adjustments to the training process to accommodate this post-hoc refinement. Extensive experiments are conducted on both homophilous and heterophilous graphs. Results demonstrate that the proposed method improves node classification performance by five percentage points.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Question: \nI would strongly recommend incorporating additional GNN baselines such as GIN and GAT, rather than relying solely on GraphSAGE.\n\nAnswer: \nThe focus of the experiments is on evaluating the trained embeddings. For this, we chose MLP and GraphSage as representative models. Other models might reach higher absolute performance numbers, but the focus was always on evaluating the trained embeddings. Nevertheless, we agree that using only one MP-GNN is limited and are currently training GIN and GAT, as suggested.\n\nFirst results suggest that for already well-performing settings (homophilic FP/iN2V), there is a 0-2% gain compared to GraphSage.\nFor very challenging settings (e.g. Actor dataset) there is no change to a small drop in performance. The biggest impact we have seen so far is that the N2V (inductive) baseline in the embedding only setting improves from 42(GraphSAGE) to 68 (GIN) / 69 (GAT) on the Cora  10% split.\n\nFull results will be added to the Appendix in the style of tables 5-12.\n\n\nQuestion: \nRelation to RefactorGNNs (NeurIPS 2022)\n\nAnswer: \nRefactorGNN tries to combine the benefits of FMs and GNNs (applicable to inductive tasks, easy feature integration) into a single model that is used for KGC.\nOur goal differs as we modify N2V to generate embeddings in the inductive case that can be used with any other GNN afterwards instead of building a singular model.\nAlso, we do not focus on KGC.\nNonetheless, this is a methodologically interesting paper that is somewhat related as it also tackles an inductive problem, and we add it to our related work.\n\n\nQuestion: \n[...] key related works in this area. For instance, periodically resetting a subset of node embeddings could serve as a useful baseline [1,2]. Additionally, exploring extensions to knowledge graphs could provide valuable insights, as efforts to inductivize transductive models have been investigated in the context of knowledge graph embeddings [1,3].\nReferences:\n[1] RefactorGNNs (NeurIPS 2022)\n[2] Resetting Embedding Layer (NeurIPS 2023)\n[3] NBFNet (NeurIPS 2021)\n\nAnswer: \n[2] occasionally resets the embedding layer of a PLM during training to improve adaptation to new languages. N2V trains an embedding for each node (in the training set) but does not have any layers after the embedding like PLMs that could be coerced to prioritize adaptability to changing embeddings. The closest connection we see to our work is to the sampling-based modification of N2V training, which during training replaces some of the trained node embeddings with the mean neighbour embedding for a single iteration.\n\nRegarding work that extends KG methods to the inductive case. We will add [1] to our RW and discuss it there, together with the suggestions from your next point.\n\n\nQuestion: \n[...] Other popular methods like [2,3,4] are also prominent in knowledge graph embeddings and should be referenced in Section 2.1.\n[1] RefactorGNNs (NeurIPS 2022)\n[2] Canonical Tensor Decomposition … ICML 2018\n[3] Convolutional 2D Knowledge Graph Embeddings AAAI 2018\n[4] Relation Prediction as an Auxiliary … (AKBC 2021)\n\nAnswer: We had to cut parts of the KG-embedding related work for the given page limit. We will add the proposed papers to the RW and slightly compact the related work.\n\n\nQuestion: \nCan the proposed method rewrite into some form of message-passing?\n\nAnswer:\nDepends on how strict one sees message-passing. The Aggregate function is a simple convolution modified to ignore neighbors without embeddings. Equation 1 defines the Update function, which takes an entity's own embedding and the aggregated neighbor embeddings to compute its updated embedding.\n\n\nQuestions For Authors:\n\nQ1: Can you explain line 134: v17 = v7/4^4? How is this derived?\nQ2: How could your method be extended to knowledge graphs?\n\nAnswer: \nQ1: This is a typo, and it should be v7/4^3. Does this answer the question, or should we explain the example in more detail?\nQ2: First, one needs to think about what exactly the task and setting are. Our method focuses on the inductive and feature-free setup. For the inductive extension, we rely on having the edges of the test set. A common task for KGs is Link Prediction, which predicts information that our method uses to generate the embeddings. Therefore, it would be a challenge to apply our method here.\nFor other tasks where the test edges/relations are available, it could be used as-is or by replacing N2V with a more KG-focused embedding method (like TransE) that embeds relations.\nWhen doing so, one would also need to adapt equations 1b and 1c not to use the average neighbor embedding, but to reflect how the relation embeddings are trained, e.g. the average over r_vw+w (+ from TransE, depending on the chosen KG embedding).", "claims_and_evidence": "1. Lack of comparison with other GNNs. The submission lacks some baseline comparisons with Graph Neural Networks (GNNs), which are well-known for their inductive reasoning capabilities, particularly since they can leverage node features. While these models may not necessarily outperform the proposed method, including them in the results table would provide valuable context and better ground the method's performance. I would strongly recommend incorporating additional GNN baselines such as GIN and GAT, rather than relying solely on GraphSAGE.\n\n2. The submission overlooks key foundational works in the literature. Notably, there have been prior efforts to inductivize node embeddings. For instance, [RefactorGNNs](https://arxiv.org/abs/2207.09980) (NeurIPS 2022) specifically focus on making shallow knowledge graph embeddings inductive by establishing connections between node embeddings and message-passing frameworks. Drawing connections between the proposed method and RefactorGNNs could offer useful insights and strengthen the theoretical grounding of the work.", "methods_and_evaluation_criteria": "Yes, the proposed method is simple and intuitive for inductivizing node embeddings. One concern is the evaluation task only consists of node classification while it might be different for inductivising node embeddings for tasks like link prediction.", "theoretical_claims": "NA", "experimental_designs_or_analyses": "Yes, I reviewed the datasets used, the dataset splitting strategy for train/validation/test sets, and the hyperparameter selection process. The experimental design and analysis appear sound and appropriate.", "relation_to_broader_scientific_literature": "The paper presents a simple yet intuitive approach for predicting unseen nodes at test time. The proposed method combines post-hoc processing with modifications to the standard training procedure, offering a meaningful contribution to the field of inductivizing shallow embeddings.  \n\nHowever, the paper overlooks some key related works in this area. For instance, periodically resetting a subset of node embeddings could serve as a useful baseline [1,2]. Additionally, exploring extensions to knowledge graphs could provide valuable insights, as efforts to inductivize transductive models have been investigated in the context of knowledge graph embeddings [1,3].  \n\n**References:** \n\n[1] [RefactorGNNs](https://arxiv.org/abs/2207.09980) (NeurIPS 2022)  \n\n[2] [Resetting Embedding Layer](https://arxiv.org/abs/2307.01163) (NeurIPS 2023)\n\n[3] [NBFNet](https://arxiv.org/abs/2106.06935) (NeurIPS 2021)", "essential_references_not_discussed": "The task of making transductive models inductive has been explored in the literature. On one dimension, the authors discuss the limitations of GNNs for inductive reasoning due to the lack of node features, which I appreciate. On the other dimension, there is deeper connection between transductive embeddings and inductive embeddings [1], which can benefit from more discussion in the revision.\n\nFor related works on knowledge graphs, Sec 2.1 touches some of them. Other popular methods like [2,3,4] are also prominent in knowledge graph embeddings and should be referenced in Section 2.1.\n\n[1] [RefactorGNNs](https://arxiv.org/abs/2207.09980) (NeurIPS 2022)  \n\n[2] [Canonical Tensor Decomposition for Knowledge Base Completion](https://arxiv.org/pdf/1806.07297.pdf) ICML 2018\n\n[3] [Convolutional 2D Knowledge Graph Embeddings](https://arxiv.org/abs/1707.01476) AAAI 2018\n\n[4] [Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations](https://openreview.net/forum?id=Qa3uS3H7-Le) (AKBC 2021)", "other_strengths_and_weaknesses": "Strengths:\n\n- The proposed method is simple yet effective in making node2vec inductive.\n\nWeaknesses:\n\n- see above in Claims and Evidence box.\n- The absence of key baselines, such as Graph Neural Networks (GNNs), limits the completeness of the evaluation. Including models like GIN and GAT would offer better insights into the method's relative performance.\n- There is little discussion on potential connections between the proposed method and established message-passing frameworks in GNNs.  For example, can the proposed method rewrite into some form of message-passing?", "other_comments_or_suggestions": "- These two sentences convey similar meanings and could be merged for conciseness:\n\"We expand on this idea and propose a general, simple post-hoc approach to turn transductive embeddings into effective inductive embeddings. We introduce iN2V, an approach to using trained embeddings to induce embeddings for nodes appearing in the inductive test set.\"\n\n- Line 246: This sentence is dense with information and may benefit from being split into two for improved clarity:\n\"Two baselines that are not directly comparable as they use more information but are nevertheless useful for perspective are using the original graph features and training N2V embeddings in a transductive setup.\"", "questions_for_authors": "Q1: Can you explain line 134: v17 = v7/4^4? How is this derived?\n\nQ2: How could your method be extended to knowledge graphs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "NA"}}, {"review_id": "WCADjAxsnu", "reviewer": "Reviewer_9oof", "summary": "The paper proposes inductive node2vec (iN2V), a procedure which updates node embeddings generated by methods like node2vec which form node embeddings based on graph topology, to account for updates to the graph topology like new nodes and edges. The core idea of the update is similar to feature propagation, but crucially, embeddings of nodes seen at training time can also be updated. Additionally, modifications to N2V are proposed, including modifications to the loss function, to make it more suitable for iN2V. On a node classification benchmark of 10 network datasets, iN2V shows overall improvement relative to N2V in an inductive evaluation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Question: [...]  make a more targeted claim than the broader claim made in the abstract and introduction that iN2V provides a lift relative to N2V in the inductive setting, \n[...]\nIn particular, a claim of gain relative to feature propagation should be specified upfront in terms of: 1) homophilic vs heterophilic dataset; 2) use of MLP vs SAGE as the classifier; and 3) whether provided features are used (concatenated). \n[...] this should be specified upfront so it is easier to evaluate the claim.\n\nAnswer:\nAveraged over the other parameters (dataset, split, model, embedding/cat) iN2V outperforms FP by the following amounts:\n+1 point on homophilic vs +0.7 on heterophilic datasets\n+1.3 when using MLP vs +0.6 when using Sage\n+1.3 when using the trained embeddings only vs +0.6 when concatenating them with the graph features\n\nAnother interesting comparison in the same direction is \n+1.2 when using few (10, 20%) training data vs +0.8 when using more (60, 80%)\n\nWe will update the claims in the Abstract and give some of these details in the Introduction.\n\n\nQuestion: [...] confidence intervals would be included for Table 3, since these results represent the performance lift in the more common setting with nodes have feature vectors.\n\nAnswer: Table 3 aggregates the performance over all datasets and splits to show a compact comparison between using only the trained embeddings and concatenating them with the existing features.\nDetails on the concatenation results including standard deviation can be found in tables 9-12 in the appendix.\n\n\nQuestion:\n1) As the authors mention, one weakness is that random splits are used to test the inductive setting, which does not provide a realistic evaluation. \n2) Perhaps temporal network datasets could be used instead.\n\nAnswer:\n1) Unless the domain really requires it, it is important to use random splits as similarly performing methods can be arbitrarily reranked by choosing a fixed split, see Pitfalls of Graph Neural Network Evaluation https://arxiv.org/abs/1811.05868\n2) We specifically focus on the inductive setting. \nWhen proposing a method for temporal datasets and claiming benefits there, one should do a thorough investigation there and deal with things like feature and class shifts, unknown classes appearing during testing that were not part of the train set, how to incorporate the data from the intermediate timesteps and catastrophic forgetting (e.g. if a class disappears for some timesteps).\nAs we want to keep our work focused on the classical inductive case, we chose not to use temporal datasets and baselines.\n\nQuestion: A prime example is \"A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations\" (Jia and Benson 2021), which also discusses other works in this area. This Unifying Model paper overlaps significantly with the core of this work's proposed algorithm, though that work provides more theoretical justification for their model and algorithms. [...]\n\nAnswer: The Unifying Model paper takes an attributed (features+labels) graph and fits their MRF to it. The focus of the paper is to show that this is similar to label propagation when using only labels as attributes and similar to a linear GCN when using the features.\nThis is a different setting than what we tackle which is to give embeddings to unseen test nodes in graphs with no features or labels.\nTheir method could be used on top of N2V to treat the N2V embeddings as attributes of the training nodes. Though this combination would be a lot more complex than FP or our proposed iN2V.\n\nWe add this paper to our related work and explain the difference in setting.\n\n\nThe related work contains broad descriptions of prior methods on node embedding methods and GNNs. It could be more focused on prior work relevant to the paper, like work involving the inductive setting and embedding propagation. The inductive setting is also related to the temporal setting, which could also be discussed.\n\nAnswer: We will adapt the related work to focus more on inductive methods, as reviewer VRfA also proposed some in the inductive knowledge graph direction. Additionally, we will add a short paragraph discussing the connection between the inductive and temporal settings.\n\n\nQuestion: Relatedly, is there any possible theoretical validation of the proposed method vs feature propagation?\n\nAnswer: We currently do not have a theoretical comparison of FP vs iN2V, but in the motivating for our method we demonstrate examples where FP is unable to adapt to the changed graph structure when new edges appear at test time vs our iN2V which can adapt to this situation. As more changes to edge structure happen with fewer training data, the bigger gain of iN2V vs FP when using less training data (+1.2 for (10, 20%) vs +0.8 for (60, 80%)) also supports this example empirically.\n\nWe incorporate the Other Comments Or Suggestions:\nRDF acronym definition; Line 231 missing space; Line 190/192 r/p", "claims_and_evidence": "I feel that the authors could make a more targeted claim than the broader claim made in the abstract and introduction that iN2V provides a lift relative to N2V in the inductive setting, as this seems unsurprising and is already achieved by the prior method of feature propagation, which can serve as a baseline. In particular, I think a claim of gain relative to feature propagation should be specified upfront in terms of: 1) homophilic vs heterophilic dataset; 2) use of MLP vs SAGE as the classifier; and 3) whether provided features are used (concatenated). The results seem to indicate a significant lift relative to N2V in most cases, and some smaller significant lift relative to feature propagation in some cases, but this should be specified upfront so it is easier to evaluate the claim.", "methods_and_evaluation_criteria": "The benchmark datasets used are reasonable and span a good variety of both homophilic and heterophilic networks. The authors commendably also include results where their proposed method, like the original N2V, performs poorly. Ideally, confidence intervals would be included for Table 3, since these results represent the performance lift in the more common setting with nodes have feature vectors.", "theoretical_claims": "There are no theoretical claims or results.", "experimental_designs_or_analyses": "The experimental design seems valid overall, and different results are provided for the cases of using provided node features or not, or testing the N2V modifications on feature propagation to separate the lift of these modifications from the core iN2V idea. Enough details in Section 4 that the experiments seem reproducible.\nAs the authors mention, one weakness is that random splits are used to test the inductive setting, which does not provide a realistic evaluation. Perhaps temporal network datasets could be used instead.", "relation_to_broader_scientific_literature": "The authors discuss how their method can be used to extend node embedding methods like N2V to the inductive setting, presenting it is an overall superior alternative to feature propagation. As discussed below, there is arguably significant overlap with some prior work in the area of graph signal processing.", "essential_references_not_discussed": "The area of graph signal processing contains many works on embedding / feature propagation that are core to the paper's thrust but not discussed here. A prime example is \"A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations\" (Jia and Benson 2021), which also discusses other works in this area. This Unifying Model paper overlaps significantly with the core of this work's proposed algorithm, though that work provides more theoretical justification for their model and algorithms. The primary difference is the focus on node classification here vs node regression in that work. In particular, this prior work also effectively allows for the embeddings of nodes seen with given embeddings to be updated at test time, so is is unclear that the core distinction of this reviewed work from feature propagation is indeed original.", "other_strengths_and_weaknesses": "- The related work contains broad descriptions of prior methods on node embedding methods and GNNs. It could be more focused on prior work relevant to the paper, like work involving the inductive setting and embedding propagation. The inductive setting is also related to the temporal setting, which could also be discussed.", "other_comments_or_suggestions": "- The \"RDF\" acronym should be defined in the text.\n- Line 231, missing space after \"GraphSAGE\"\n- Line 190, probability $p$ becomes probability $r$ in line 192", "questions_for_authors": "1. Please address the concerns in \"Claims And Evidence\": under which of these cases specifically is a significant gain claimed relative to feature propagation.\n2. Please also address the concern in \"Essential References Not Discussed\" regarding originality vs prior work in graph signal processing. Relatedly, is there any possible theoretical validation of the proposed method vs feature propagation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sCm5LffcE9", "reviewer": "Reviewer_qGa8", "summary": "This paper proposes an inductive way to learn node features from the graph structure. Specifically, it proposes to to extend node2vec to the inductive setting with the key idea being to set the embeddings of the testing nodes to be the average embeddings of their training neighbors.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Question: \nabout the induced subgraph of a training node as mentioned in section 3.1.\nEven if the testing nodes are masked, their neighborhood are still observed to some extent right (especially given figure 1)? since testing nodes may be connected to training nodes? if so then is this really inductive? I think inductive means for example splitting the graph in two, training on one side and testing on the other? Like the PPI set which has different graphs amongst which training is done on a few graphs and testing on the other graphs?\n\nAnswer:\nDuring training the validation and test nodes with all edges that contain at least one of them are removed. As you correctly state, some (how many depends on the train/val/test split) test nodes have neighbors in the training set.\nInductive means that the test nodes (features, labels, and edges) are unseen during training.\nSo splitting the graph in two, training on one side and testing on the other is inductive (for testing the whole graph is used as input for the models, the metrics are calculated on the test nodes).\nPPI is a classic inductive dataset with 24 components, where the train/val/test split is based on component.\n\n\nQuestion:\nIf I am correct for the question above, then can the proposed algorithm adapt to PPI for example? How does it perform? And what about sectioning a graph (e.g., Cora) in two or three (train, validation, test) and then reporting the performance too?\n\nAnswer:\nAs explained in the Limitation section, “our method can only provide embeddings to test nodes that have a path to at least one train node” (also applies to FP).\nSo that a node gets an embedding during the extension to the test set, it needs to have a path (that can contain nodes from both the test and val set) to at least one train node.\nThis does not have to be the case for all test nodes, as e.g. Citeseer consists of 428 components, most of which have only very few nodes. With our random splits it will happen that for some components all nodes are in the validation or test set and therefore do not get an embedding during testing.\nBut for PPI, where no nodes in the val and test set have a path to nodes from the training set, our method would not produce useful embeddings.\nTo apply it to PPI one would either have to switch the embedding model , e.g. from N2V to Struct2Vec (and use the internal fully connected Struc2Vec graph for the extension) or add additional edges to the PPI dataset, like adding edges that connect the same protein in the different components (such that the test nodes have connections to nodes in the 20 train graphs).\nThe same methods could be tried for other graphs that are split component wise like ZINC.\nThough for very good performance on these kinds of datasets (protein/molecules) GNNs have to have some additional properties like being able to distinguish and efficiently aggregate information over cycles.\n\n\nQuestion:\nMoreover is it also possible to see the performance when the number of training nodes is low say 10-5% and the validation set is low as well say 5-10% (since in the eperiments when the training set was small 10% the validation set was still large 45%)\n\nAnswer:\nFor the experiments in the paper our focus was a wide range of training sizes (some papers claim good inductive performance but then only use over 80% of the data for training) with a balanced val and test size such that the validation performance is indicative of test performance. We expect e.g. a 10-10-80 split to perform similarly to our 10-45-45 split as the amount of training data is the same, but that the hyperparameters and early stopping generalize worse from val to test set.\nNevertheless, we implemented a 10-10-80 split and are currently training (i)N2V embeddings on it and will add this as an additional section in the Appendix.\n\n\nTypos:\n\"This is repeated for multiple iterations to also deal with test nodes which longer distances to training nodes\" should be \"This is repeated for multiple iterations to also deal with test nodes with longer distances to training nodes\" ?\nIn section 3.4 under the heading Sampling-based, did you mean probability \"r\" as in the equation instead of \"p\" (see line 190).\n\nAnswer: Implemented the suggested changes.\nYes, r/p is a typo (we switched from p to r to distinguish it from the N2V p and q hyperparameters)", "claims_and_evidence": "- A question I have is about the induced subgraph of a training node as mentioned in section 3.1.\n- Even if the testing nodes are masked, their neighborhood are still observed to some extent right (especially given figure 1)? since testing nodes may be connected to training nodes? if so then is this really inductive? I think inductive means for example splitting the graph in two, training on one side and testing on the other? Like the PPI set which has different graphs amongst which training is done on a few graphs and testing on the other graphs?", "methods_and_evaluation_criteria": "- See section named \"Claims And Evidence\"\n- If I am correct for the question above, then can the proposed algorithm adapt to PPI for example? How does it perform? And what about sectioning a graph (e.g., Cora) in two or three (train, validation, test) and then reporting the performance too?\n- Moreover is it also possible to see the performance when the number of training nodes is low say 10-5% and the validation set is low as well say 5-10% (since in the eperiments when the training set was small 10% the  validation set was still large 45%)", "theoretical_claims": "See section named \"Methods And Evaluation Criteria\"", "experimental_designs_or_analyses": "See section named \"Methods And Evaluation Criteria\"", "relation_to_broader_scientific_literature": "See section named \"Methods And Evaluation Criteria\"", "essential_references_not_discussed": "No recomendations", "other_strengths_and_weaknesses": "See section named \"Methods And Evaluation Criteria\"", "other_comments_or_suggestions": "Typos:\n- \"This is repeated for multiple iterations to also deal with test nodes which longer distances to training nodes\" should be \"This is repeated for multiple iterations to also deal with test nodes with longer distances to training nodes\" ?\n- In section 3.4 under the heading Sampling-based, did you mean probability \"r\" as in the equation instead of \"p\" (see line 190).", "questions_for_authors": "See section named \"Methods And Evaluation Criteria\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes iN2V, an inductive version of node2vec for computing node embeddings. The paper has merits that the reviewers have highlighted. These include the simplicity of the approach and its effectiveness on downstream tasks. Nevertheless, the reviewers have also indicated limitations mainly related to the following points: (i) A comparison to GNN models (apart from GraphSAGE) is missing; (ii) The discussion of the related work seems insufficient. Considering the authors' responses during the rebuttal period, I'm in favor of accepting the paper. I encourage the authors to take these comments into consideration (especially the comparison to other GNNs) and revise the manuscript accordingly.
Decision: Accept (poster)