Node Embeddings via Neighbor Embeddings
Authors: Jan Niklas Böhm
Marius Keute
Alica Guzmán
Sebastian Damrich
Andrew Draganov
Dmitry Kobak
Venue: Accepted by TMLR ()

Abstract: Node embeddings are a paradigm in non-parametric graph representation learning, where graph nodes are embedded into a given vector space to enable downstream processing. State-of-the-art node-embedding algorithms, such as DeepWalk and node2vec, are based on random-walk notions of node similarity and on contrastive learning. In this work, we introduce the graph neighbor-embedding (graph NE) framework that directly pulls together embedding vectors of adjacent nodes without relying on any random walks. We show that graph NE strongly outperforms state-of-the-art node-embedding algorithms in terms of local structure preservation. Furthermore, we apply graph NE to the 2D node-embedding problem, obtaining graph t-SNE layouts that also outperform existing graph-layout algorithms.

Published in Transactions on Machine Learning Research (11/2025)
Node Embeddings via Neighbor Embeddings
Jan Niklas Böhm
mail@@jooehmm.com com
Herlic AI, University of Tübingen, Germany
Marius Keute*
Hertie AI, University of Tübingen, Germany
Alica Guzmán
Hertie AI, University of Tübingen, Germany
Sebastian Damrich
Hertie AI, University of Tübingen, Germany
Andrew Draganov
draggasoeandrruu@gmaaac.cmm
Department of Computer Science, Aarhus Universily, Denmark
Dmitry Kobak
dmitry manntry.kkoak@@nnnn-ttubinnggo
Hertie AI, University of Tübingen, Germany
Reviewed on OpenReview: https: // openreview. net/ forum? id= 8APIU9cauZ
Equal contribution
Abstract
Node embeddings are a paradigm in non-parametric graph representation learning, where graph nodes are embedded into given vector space to enable downstream processing. State-of-tho-art node-embedding algorithms, such as DeepWalk and nodc2vee, are based on random-walk notions of node similarity and on contrastive learning. In this work, we introduce the graph neighbor-embedding (graph NE) framework that directly pulls together embedding vectors of adjacent nodes without relying on any random walks. We show that graph NE strongly outperforms state-of-thh thetttttrrttt node-embedding algorithms in terms of local structure preservation. Furthermore, apply graph NE to the 2D node-embedding problem, obtaining graph t-SNE layouts that also outperform existing graph-layout algorithmss.
graph NE
node embedding
S127
InfoNCE
exp(y. T. yj ////T)
Li = - -log
En exply, yk/T)
G = (V,
2
KL divergence
(1 ll y, 11
graph
Lij = log
End 1 - Y1 2)
layout
Figure Graph -= = (V,8) embedded into 127 and I2 with graph NE. Blue denotes the attractive force between neighboring nodes and with (i.j) E, orange corresponds repulsive forces between all points.

---

Published in Transactions on Machine Learning Research (11/2025)
1 Introduction
Many real-world datasets, ranging from molecule structures to citation networks, come in the form of graphs. A graph G' is an abstract object consisting of a set of nodes V and a set of edges E between them; the nodes do not inherently belong to any specific metric space. Therefore, the field of graph representation learning has emerged with the goal of embedding the nodes into a metric space, such as IRo, so that the graph structure (neighborhoods, graph distances, etc.) is well-preserved. In this paper, we only consider non-porumelric approaches that do not use any node features.
Popular node-embedding methods like DeepWalk (Perozzi et al. 2014) and node2vec (Grover & Leskovee 2016) are based on contrastive learning and random-walk notions of node similarity, reducing the node- embedding problem to a word-embeddingg problem and then relying on the word2vec algorithm (Mikolov et al. 2013) for optimization. At the same time, node embeddings into I2 for visualization purposes known as graph layouts- are typically obtained by algorithms that simply pull together neighboring (i.c. connectod by an edge) nodes, traditionally using spring models (Fruchterman & Reingold 1991). In the context. of dimensionality reduction, the idea of pulling neighbors together has become known as neighbor embeddings through methods like t-SNE (van der Maaten & Hinton 2008) and UMAP (MeInnes et al. 2018). This raises the question: Can such beabbddd-- approaches be used for generic mode-embedding problems?
In this work, we show that neighbor-ombeddiig methods are remarkably effective for node-embeddingg problems and introduce a framework called graph neighbor embeddings (graph NE) (Figure D. Our work builds on recent literature which allows to effectively optimize neighbor embeddings in high-dimensional embedding spaces (McInnes et al., 2018 Damrich et al.) 2023 We show that graph NE outperforms DeepWalk and node2vec in terms of local structure preservation, while being conceptually simpler (no random walks are needed) and without requiring costly hyperparameter tuning. Furthermore, we show that graph NE can also be applied for 2D node embeddings (Figure I outperforming existing graph-layout methods. In short, our results demonstrate that neighbor embeddings are a powerful approach to graph reprosentation learning that beats state-of-the-art node-embedding algorithms.
2 Related work
-aramerritt node embeddings The popular DeepWalk (Perozzi et al. 2014) and node2vec (Grover & Leskoved 2016) algorithms optimize node placement in high-dimensional target space based random walks over graph. These walks treat nodes analogous to words and random-walk paths sentences, enabling the application of word-embedding techniques to learn the representation. Specifically, DeepWalk achieves this by performing random walks from each starting node and then using the word2vec algorithm MMkoolov et al. 2013) to ensure that nodes which often co-occur in these random walks are represented near one another in the embedding space. The node2vec algorithm similarly obtains node embeddings by giving graph traversals to the word2vec algorithm, but it differs from DeepWalk by defining two parameters which control the depth-first vs. breadth-first nature of the random walk. These parameters (p and q) provide an additional level of control over the community structure uncovered by the walks, with DeepWalk being a specific instantiation of node2vec when these parameters are both set to 1.
Both WelpWalk and node2vec have been widely adopted for graph-based machine learning applications, inchuding classification and link-predcction tasks (Khosla et al. 2019). Although connections have been drawn between word2vec and contrastive learning (Saunshi et al. 2019 we emphasize that the DelkWalk and node2vec algorithms are often regarded as separate from standard contrastive techniques (Grohe 2020
Instead of optimizing the embedding coordinates freely, Alvarez-Gonzaler et al. 2023 found improved performance after constraining the embedding coordinates in DeepWalk based on the graph connectivity structure. Their approach, called IGEL, allows to embed new, previously unobserved nodes, which standard non-parametric methods (including ours) do not allow. Their approach can in principle be combined with any gradient-descent based node- enleeeebeddinng method, including ours.
Parametric node embeddings and node-level graph contrastive learning Our paper is about non-parametric embeddings that only use the structure of the graph G (V,8). In contrast, parametric
2

---

Published in Transactions on Machine Learning Research (11/2025)
graph contrastive learning (GCL) methods use node feature vectors and employ neural network, usually graph convolutional network (GCN; Kipf& & Welling 2017;, to transform features into embedding vectors.
The basic principle behind contrastive learning is to learn a data representation by contrasting pairs of observations that are similar to each other (positive pairs) with those that are dissimilar to each other (negative pairs). In computer vision, positive pairs are generated via data augmentation, e.g- in SimCLR (Chen et al. 2020). GCL can be graph-level or node-level, depending on whether representations are obtained for set of graphs or for the set of nodes of a single graph. Many graph-level (e.g. You et al. 2020 and
nove-level GCL algorithms (Velickovic et al.
2019
Zhu et al. 2020b Hassani & Khasahmadij 2020 Thakoor
ct al. 2022; Zhang et al.; 2021 Zhu et al.
2021 are also based on graph augmentations, such as node
dropping or edge perturbation. A general problem with domain-agnostic graph augmentations is that they
can have unpredictable effects on graph semantics (Trivedi et al. 2022 This motivated development of augmentation-fre node-level GCL mothods, where positive pairs are pairs of nodes that are located close
to each other in terms of graph distance (Loe et al.) 2022 Li ct al. 2023 Zhang et al. 2022). Recent work argued that GCL met hods effectively pull connected nodes logether, sometlimes explicitly through their loss function, but also implicitly through the GCN architecture
(Trivedi et al.) 2022 Wang et al. 2023 Guo
ct al. 2023) A GCN can also optimize neighbor-embedding loss on node features and/or on shortest-path distances [eow et al. 2019
Note that all algorithms mentioned in this paragraph are parametric and fundamentally depend on node features. In contrast, our proposed graph NE algorithm non-parametric and operates exclusively on graph structure without requiring node features. Throughout this paper, we therefore restrict our comparisons to other non-parametric methods.
Graph layouts Graph-layout algorithus have traditionally been based on spring models, where every connected pair of nodes feels a distance-dependent attractive force F. and all pairs of nodes feel a distance- dependent repulsive force F, (force-directed graph layouts). Many algorithms can be written F, = dis and F.= = dig [Noack 2007), where di (resp. diy)) is the embedding distance between nodes i and raised to the Centheemman-Reinniin
algorithm uses a = 2, r --==1 -1 (Fruchterman & Reingold 1991); ForceAtlas2 uses a = 1, = -1 (Jacomy et al. 2014h; LinLog uses a = 0, = = (Noack 2007]. Efficient implementations can be based on Barnos-Hul approximation of the repulsive forces, as in
a-th (resp. r-th) power. For example, the
SFDP
Hu 2005). ForceAtlas2 has been shown to be related to neighbor embeddings (Böhm et al. 2022
Several recent graph-layout algorithms have been inspired by neighbor embeddings, and in particular by I-SNE qvan der Maaten & Hinton 2008). 1sNET (Kruiger et al.) 2017 applied a modified version of I-SNE to the pairwise shortest-path distances between all nodes. DRGraph (Zhu et al. 2020a) accelerated tsNET by using negative sampling (Mikolov et al.) 2013). t-FDP (Zhong et al. 2023) suggested custom F, and F, forces inspired by (-SNE and adopted the interpolation-based approximation of Linderman et al. 2019 SGtSNEpi (Pitsianis et al. 2019) is the closest method to the 2D version of our proposed graph NE algorithm. It applies t-SNE optimization to affinities derived from the graph G, but derives these affinities in a more complex way than we do, and with additional hyperparameters (Section 4.3).
There is separate set of methods which produce graph embeddings via. classical dimensionality reduction techniques. Some of these, such as Laplacian Eigenmaps (Belkin & Niyogi 2003) and Diffusion Maps (Coifman & Lafon, 2006), can be applied directly to graphs (and amount to cigendecomposition of the graph Laplacian). We use Laplacian Figenmaps in our comparisons as a representative algorithm from this family. Other approaches employ variants of multidimensional scaling on graph-derived distances (Gansner et al. 2012
Miller et al. 2023 Zhang et al.. 2023
3 Background: Neighbor-embedding framework
3.1
Neighbor embeddings
Neighbor embeddings are a family of dimensionalityy methods aiming to embed n observations from some high-dimensional metric space X into a lower-dimensional (usually two-dimensional) Fuclidean space
3

---

Published in Transactions on Machine Learning Research (11/2025)
Rd, such that neighborhood relationships between observations are preserved in the embedding space. We denote the embedding vectors y; E RR4.
One of the most popular neighbor embedding methods, i-distributed stochastic neighbor embedding (I-SNE; van der Maaten & Hinton 2008), is an extension of the earlier SNE (Hinton & Roweis 2002). t-SNE minimizes the Kullback-Leibbe divergence between the high-dimensional and low-dimensional affinilies Pij and by-
L=
Σ Péj log
Púj const- Σ Pij log qty-
4i
Both affinity matrices are defined to be symmetric, positive, and to sum to 1. The high-dimensional affinities P are computed using adaptive Gaussian kernels whose mass concentrated on nearest neighbors. Low- dimensional affinitics Q are defined in t-SNE using a t-distribution kernel with one degree of freodom, also known the Cauchy kernel:
(1+ Iy: - Oij Exp(1 + Jyy y2
(2)
In practice, t-SNE optimization can be accelerated by approximation of the repulsive force field based on Yang et al., 2013), on interpolation (Linderman et al.,
the Barnes-Hon algorithm (van der Maaten 2014
2019), or on sampling (Artemenkov & Panov 2020 2023
Damrich et al. 2023
Draganov et al. 2023 Yang et. al.,
3.2
Contrastive neighbor embeddings
The contrastive neighbor-embeedding (CNE) algorithm (Damrich et al. 2023) is a flexible dimensionality- reduction framework that replaces t-SNE's Kullback-Leibler divergence loss with contrastive losses, such as the InfoNCE loss (Jozelowicz et al. 2016 Oord et al.] 2018). This loss function is called contruslive because it is based on contrasting pairs of k-nearest neighbors and nou-neighbors in the same mini-batde and does not require global normalization like in Equation (2). As a result, the runtime of CNE scales like ((d) with the number of points and the embedding dimensionality d, unlike other existing t-SNE implementations that scale like O(n2d) (van der Maaten & Hinton 2008) ((22)) (Linderman et al. 2019)
This enables CNE to optimize high-dimensional outputs (large d). CNE with the InfoNCE loss approximates
t-SNE (Damrich et al. 2023 Ma & Collins 2018 see also Section
4.4
The InfoNCE loss is defined for one pair of k-nearest neighbors ij (positive pair) with affinity Pi as
Wij
((i,j) = -Pij log
Wij
Σk Wikk
where Wiy are non-normalized low-dimensional affinities standing in for the normalized affinities qú above. The sum in the denominator is over negative pairs ik where & can be drawn from all points in the same mini-batch apart from i and j. One mini-batch consists of b pairs of neighbors, and hence contains 2b points. Therefore, for given batch size b, the maximal value of m is 2b - 2. The larger the number of negative samples m, the better is the approximation to 1-SNE DDamrich et al. 2023). The InfoNCE loss aims to make wy large, i.c place embeddings y, and y, nearby, ij is a positive pair, and small if it is a negative one.
The wiy affinitics do not need to be normalized. When embodding into R2, they can just be defined as
Wij = (1 + lly, - 1
When using a highhdiieensional embedding space, e.g. d = 128 instead of d 2, embedding vectors are usually projected to lie on the unit sphere. For points on the unit sphere, the cosine distance and the squared Euclidean distance differ only by constant multiplicative factor, making the following definitions of wa equivalent:
yTy; |y=|| 11y -
Y: |y:
yj lys
1²/(2r)).
Wig exp
const exp
(5)
4

---

Published in Transactions on Machine Learning Research (11/2025)
where is called the temperature (by default, 0.5). Together with Equation (3) this gives the same loss function in SimCLR (Chen et al. 2020), popular contrastive learning algorithm in computer vision. Note that instead of nearest neighbors, SimCLR uses pairs of augmented images as positive pairs.
4 Graph NE: Applying the neighbor-embeeddig framework to graphs
4.1 General approach
Neighbor-embeedding algorithms employ high-dinenssonall affinities with most Pij R 0. This can be seen as a generalization of discrete nearest neighbors: if P.j is close to O, then the points are effectively dissimilar. However, almost the same visualizations can be obtained using hard nearest neighbors, i.c. simply by normalizing the symmetric kNN graph adjacency matrix A directly (Artemenkov & Panov 2020
Damrich
et al. 2023
P P A
Σ Aij.
Here, A has element A. = if is within the & nearest neighbors of x; or vice versa. This is equivalent to simply leaving out Piş from Equation (3.
Thus, even though neighbor embeddings are usually not presented as such, they can be thought of as node-embedding algorithms, specifically applied &NN graphs. During optimization, neighboring nodes (sharing a &NN edge) feel attraction, whereas all nodes feel repulsion, arising through the normalization in Equations (2 and (3)-
This suggests a simple strategy, which we call graph neighbor embedding (graph NE), for applying the neighbor embedding framework to a general graph G: obtain affinities directly from G instead of a kNN graph of some data, and then compute a -araaaaaettiic neighbor embedding on these affinities (Figure 1
4.2 High-dimensions node embeddings via graph NE
Given an unweighted graph G (V,E), its adjacency matrix A has elements A. = 1 if (i,j) E & and AG = otherwise. Since all graphs considered in this study are undirected, the adjacency matrix is a binary, symmetric square n X n matrix. order to convert the adjacency matrix into an affinity matrix suitable for neighbor embedding, we followed the simple normalization strategy in Equation (6). Then, graph NE optimizes the embedding using the contrastive InfoNCE loss function (through the CNE backend) to place
neighbors close to each other in the embedding (Section 3.2 We used the cne library (Damrich al. 2023
For all experiments with CNE we used the output dimensionality d = 128, following the DeepWalk paper, and the cosine distance (meaning the embedding vectors lie on a hypersphere, Equation 5) We set the batch size to min min(8192, [V/10}} (smaller graphs required smaller batch sizes for good convergence) and used full-batch repulsion (m = 2b - 2) for better local structure preservation (Damrich el al. 2023) The number of epochs was set to 100. We used the Adam optimizer (Kingma & Baj 2015) with learning rate 0.001. Graph NE was initialized with 128-dimensional Diffusion Maps (Coifiman & Lafon 2006), although we saw almost difference when using random initialization (Figure (4p.e). For this paper, implemented in version 0.4.0 some of the APT options shown in the code snippet below.
fron import CNE
C = CNE(
ine__odd="iiffonce" temperature=0.u 0.05, parametric=Palse ambd_dim=1d metric="cosine" batch_aize=8192, negative_samplen "full bbachbatch", optimizer=
C.fit_transfon (graph=A, init=" init="diffmaps")
Note that our method is conceptually much simpler than WeepWalk and node2vec. In both of these algorithms, random walks are used to implicitly estimate node similarity by their co-occurence, and then word2vec is employed train the embedding. Furthermore, node2vee requires per-graph hyperparameter tuning so that its aandom-walk distribution appropriately models the input graph (Grover & Leskoved, (016)). In our graph NE method, all nodes connected by edge attract each other, requiring random walks.
5

---

Published in Transactions on Machine Learning Research (11/2025)
4.3 Graph layouts via 2D graph NE
apply graph NE in 2D to obtain graph A graph layout is 2-dimensional node embedding. Therefore,
layouts. Since the main purpose for graph layouts is visualization, we use the Cauchy similarity (Equation 2 The embedding dimensionality = 2 allows us to use the KL divergence and оpenTSNE library (Poličar
et al., 2024) with default parameters for optimization. It supports Diffusion Maps for initialization (Kobak & Linderman 2021 , sets the learning rate to n to achieve good convergence (Linderman & Steinerberger 2019
Belkina 2019). and employs the FIt-SNE algorithm (Linderman et al. 2019). For this paper, we made some improvements to the spectral initialization in openTSNE.
In this setting, found the row-normalization of the adjacency matrix to perform better:
A + A
P
where Âij = Aij Σ
Aak-
2n
k=
Normalizing the adjacency matrix as in Equation (6) resulted in lower neighbor recalls and ANN accuracies, and in bllgchog-shappel embeddings with low-degree nodes pushed out to the periphery and dominating the embedding (Figure (4). Furthermore, we experimented with various inilialization schemes and found that on our graphs, random initialization performed very similar to Diffusion Maps (Figure S4b,e). This setting of graph NE can also be called graph f-SNE:
fron оpeaТiNE import TSNE import PrecomputedAffinitties
from
= A A. sun (axis=1)
= (P + P.T) 2 A. shape [o]
=
In contrast to the simple Equation 7 that we use for 2D graph NE, the closely related SGLSNEpi method
Pitsianis et al. 2019) derives the affinity matrix P from the adjacency matrix A in a more complicated way Pitsianis et al. 2024 Supplementary), Non-zero elements Ag are first weighted by the Jaccard similarity of the sets of neighbors of nodes and j, then tower-transformed to match a pre-specified row sum X, and finally divided by to yield Å. By default, X 10.
4.4 Graph NE with CNE backend approximates (-SNE backend
Node embeddings computed via cne and via openTSNE backends have the same optima:
Theorem 4.1. /adapted from Ma & Collins 2018 Let be probability distribution over S = {ij lillsii j n), so that for all pairs ij, there is path Pik.,...,,Ph, with each step having positive probabilily. Lel w0)) be a family of non-negative functions S l2z0 paramelrized by E and symametric in and j. meaning (0) = (0) Let further be a probability distribution over [m] = (I. with full support. Suppose there is some l" and some e. > O with w(f*)/c = p. Then, H" minimizes the loss
wiy(0)/5
CInfONCE (0) = -EEgapp Ex Am log wij (0)/E) + Do=i wike (0)/Ek.
(8)
and for any other minimizer á there exists with w(0)/2 = p. In particular, the minima of CIntONCE correspond one-to-one those of
DKL (p. (())/2(0) -Ej~~p log Wij (0) Z(0)
(9)
where Z(0) Eig wiy (O).
The proof can be found Appendix A
Consider this thcorem the case where p the uniform distribution of edges on connected graph, parameters 8 Bexd are simply the embedding coordinates, and ()) are the non-normalized low-dimensional affinities. Applying this theorem then shows that the InfoNCE. loss

---

Published in Transactions on Machine Learning Research (11/2025)
(when using uniform noise distribution ç, Equation 8 and the Kullback-Leibler divergence have the same minima.
This means that our graph NE framework unifies node embeddings and graph layouts. The main difference between graph NE with cne and openTSNE backends is the choice of optimization strategy for 128 and for dimensions.
5 Experimental setup
Datasets We used eight publicly available graph datasets (Table D. The first five datasets were retrieved from the Deep Graph Library (Wang et al., 2019). The arXiv and MAG dataset were retrieved from the Open Graph Benchmark (Hu et al. 2020). The MNIST ANN dataset was obtained by computing the ANN graph with k 15 on top the 50 principal components of the MNIST digit dataset (LoCun et al. 1998). Each dataset was treated an unweighted and undirected graph, where each node has class label, used only for evaluation. We restricted ourselves to graphs with labeled nodes in order to use classification accuracy one of the performance metrics. In all datasets used only the largest connected component and excluded all self-loops if present, using NetworkX (Hagberg et al. 2008 functions and selfloop edges. We did not use any node features.
Dataset
Nodes
Edges
Classes
E/N
Citeseer
2 120
7358
3.5
Table 1: Benchmark datasets. Columns: number of nodes in the largest connected component, number of undirected edges, num- ber of node classes, and the aver- age number of edges per node.
Cora
2 288
10 138
4.1
PubMed
19 717
88 648
4.5
Photo
7487
238 086
31.8
Computer
13
491 556
10
36.7
MNIST ANN
70.000
551 392
10
21.4
arXiv
169 343
2315 598
40
13.7
MAG
726 664
10778 888
349
14.8
Performance metrics We evaluated the performance using three main metrics: neighbor recall, ANN classification accuracy, and linear classification accuracy. Such metrics standard for evaluating graph embedding quality (Perazzi et al. 2014 Grover & Leskovec 2016 Zhu et al. 2020a Zhong et al. 2023 Tn addition to that, we used three further metries: a metric based on the link fink-prediction task (Appendix B top-k 2-hop neighbor recall (Appendix C. and a Spcarman correlation between shortest-path distanccs and embedding distances (on 000 random node pairs).
The neighbor recall quantifies how well local node neighborhoods are preserved in the embedding. We defined it the average fraction of each node's graph neighbors that are among the node's nearest neighbors in the embedding:
1 [V
Nold n NE..
Recall
(10)
k;
where [V] is the number nodes in the graph, No is the set of node i's graph neighbors, k; NNail] is the number of node i's graph neighbors, and NE,,, denotes the of node i's k; nearest neighbors in the embedding space. This metric does not require ground-truth classes and is similar to what commonly used in the literature to benchmark graph-layout algorithms (Kruiger et al. 2017 Zhu et al. 2020a Zhong et al. 2023). Therefore, we use this
primary metric for measuring the embedding quality.
The KNN classification accuracy quantifies local class separation in the embedding. To calculate kNN accuracy, we randomly split all nodes into a training (90% of all nodes) and a test set. (10%), and used the KleiighooreCasssifier from scikit-learn [Pedregosa et al. 2011 with 15.
We used the cosine similarity for all &NN evaluations (recall and accuracy) in d 128. CNE uses the cosine metric in its loss function (Equation 5 only cosine neighbors make sense for evaluation. DeepWalk and

---

Published in Transactions on Machine Learning Research (11/2025)
a
b
C
T= 0.05
75%
neighbor recall
50%
25%
e 0.5 10
0%
epoch
Figure 2: Learning dynamics of the 128-dimensional CNE embeddings of nodes in a stochastic-blookkmodde graph with 10 blocks. (a, b) t-SNE visualizations of the 128D CNE embeddings with 0.05, during the first epoch and after ten epochs. (c) The neighbor recall as function of the training epoch, for 0.05 and for = 0.5. Labeled points correspond to t-SNE visualizations left/right. (d, e) Same as (a, b), but for T= 0.5.
node2vec rely on word2vee, which uses dot-product similarity in the loss function, and the original paper also used cosine metric for evaluation (Mikolov et al. 2013). In our experiments cosine evaluation led to better results on average for DeepWalk and node2vec. For all &NN evaluations in d = 2, we used the Euclidean distance.
For linear accuracy we used LogisticRegression from scikit-learn with no regularization (penalty=Nonee) SAGA solver (Defazio et al., 2014) with tol=0. 01, and the same tran/tess split. We standardized all features to have unit variance, based on the training set (as this speeds up convergence of the solver).
6 Node embeddings with graph NE require low temperature
In pilot experiments, we noticed that the node-embedding performance of graph NE (in 128D, with CNE backend) was strongly affected by the temperature parameter т. To investigate it further, we synthesized a graph following the stochastic block model (SBM; Holland al.) 1983). The generated graph had 80 000 nodes in 10 clusters, with any two nodes from the same cluster having probability 2.5 10 to be connected by an edge, and any two nodes from two different clusters having probability 5 10-6 to be connected. The resulting graph has clear community structure that should be easy to recover.
CNE with the default temperature - = 0.5 achieved neareeerfect class separation but failed to retain the neighborhood structure. The neighbor recall, after reaching 13% within the first training epoch, collapsed to below 1% over the next several epochs (Figure p orange line). The I-SNE visualization of the high- dimensional embedding at the point of maximum neighbor recall showed ten compact clusters (Figure 11), but after convergence it showed nine subclusters for each of the ten classes (Figure 2 These smaller subclusters corresponded to nodes with an inter-cluster edge a specific other class. During the optimization, these nodes got 'pulled out' of their class, destroying the local structure of the embedding and leading to near-zero neighbor recall.
In contrast, CNE with a lower temperature т = 0.05 did not show this behavior. The neighbor recall was almost monotonically increasing during training, reaching 78% after 10 epochs (Figure D blue line). The t-SNE visualization showed ten compact clusters (Figure 2p), without any visible subclusters. Our interpretation is that the InfoNCE loss with low Lemperature could effectively ignore the noise in form of rare inter-class edges.
In the following experiments, we set the temperature of graph NE with CNE backend to t = 0.05 for all datasets. We have also implemented learnable temperature, making t an additional trainable parameter. We found that on all our benchmark datasets, the temperature converged towards a value in a range of [0.04,0 0.08) (Table S7). In this sotting, the evaluation results were close the results with fixed 7 = 0.05 and both of them were usually much better than with = 0.5. We report the performance for learned T, 0.5,
and J 0.05 in Tables S1 to SB.
8

---

Published in Transactions on Machine Learning Research (11/2025)
a
neighbor recall
b
kNN accuracy
linear accuracy
graph NE
100%
100%
100%
LE
75%
75%
75%
50%
50%
50%
25%
25%
25%
0%
0%
Citesser
Cofa PubMed
Photo
Comp.
MNIST
arXiv
MAG
Citeseer
Cora
PubMed
Photo
Comp.
MNIST
arXiv
MAG
Citeseer
Cora
PubMed
Photo
Comp.
MNIST
arxiv
MAG
Figure 3: Performance metrics for node embeddings: (a) neighbor recall, (b) ANN accuracy, (c) linear accuracy.
Datasets are ordered by the number of edges. For node2vec we did grid search over q E {0.25, 0.5, 1, 4} (Figure $2 and show results with the highost neighbor recall.
a
neighbor recall
b
kNN accuracy
linear accuracy
SGISNESI DRGrach
100%
100%
100%
FA2 &FDP
75%
75%
75%
50%
50%
50%
25%
25%
25%
0%
0%
0%
Citeseer
Cora PubMed
Photo
Comp
MNIST
arXiv
MAG
Citeseef
Cora
PubMed
Photo
Comp
MNIST
arXiv
MAG
Citeseer
Cora
PubMed
Photo
Comp
MNIST
arXiv
MAG
Figure 4: Performance metrics for 2D graph layouts: (a) neighbor recall, (b) ANN accuracy, (c) lincar
accuracy. See Figures
and
$3 for the corresponding layouts.
Table Neighbor recall for all methods and dalasets (in %). All values are mean + standard deviation across three training runs. The top performing method for each dimensionality is highlighted in bold. Methods in blue ours. See Table S1 for additional graph NE variants in 128D
Method
Citeseer
Cora
PubMed
Photo
Computer
MNIST
arXiv
MAG
graph NE
81.0 0.1
83.8 0.0
44.3 = 0.2
70.3 + 0.1
64.8 0.0
96.0 + 0.0
72.3 + 0.6
89.0 = 0.5
DeepWalk
0.9
67.1
32.9
50.0 0.5
4.3
0.2
0.6
60.0 0.7
128
node2vec
70.7 - 0.6
2.1110
70.1 = 0.3
50.9 0.5
4..
43.2
42.9
29.3
Laplacian E.
53.4 0.0
56.7 4 0.0
18.3 = 0.1
39.7 0.0
3.04
38.5 0.0
3.11
10.6
graph NE
71.7 2.2
66.7 0.5
25.0 0.2
46.9 0.2
41.8
40.2
36.3 +
32.3 :
SGiSNEpi
59.1 0.3
57.4
23.3 0.3
4.5
9.3
29.1
23.6
8.1 0.4
DRCraph
42.8 1.0
3..
7.5 1.1
2.1
12.
0.1
4.2
3.2
ForcnAtlas2
0.3
0.3
3.2
2.1
11.1
7.0
2.9
1.7
t-FDP
1.2
1.72
1.3
2.1
10.2
13.9
0.7
0.3
Laplacian
06.2 0.1
1..9
2.0 0.1
16.0
0.0
6.4000
5.0
1.6
0.8 0.1
7 Benchmarking graph NE
7.1
Graph NE outperforms other node embeddings
We compared graph NE with the popular non-parametric node-embedding algorithms DeepWalk (Perozzi et al. 2014] and node2vec (Grover & Leskovec 2016), as well as with Laplacian Eigenmaps (LE), all optimizing 128- dosecinnsional embeddings. We used node2vec's ess implementation from PyTorch Geometric (Fey & Lenssen, 2019) and the Deep implementation from DGL [Wang el al. 2019). We ran both methods with the default parameters for 100 epochs (as we did for CNE, see Figure S1 for for runtimes). For node2vec, we ran a sweep over the parameters p.g E (0.25, 0.5, 1, 4), as the original paper, and report the results with the highest neighbor recall (for all results, see Figure 52 For LE we used scikit-learn (Pedregosa et al. 2011), with LOBPCG (Knyazev al. 2007) for solving the generalized eigenproblem.
9

---

Published in Transactions on Machine Learning Research (11/2025)
We found that graph NE outperformed the other algorithms in terms of neighbor recall on seven datasets out of eight; the only exception was the PubMed dataset (Figure 3 Table 2 Across all datasets, the average gap in neighbor recall between graph NE and the best other method was 13.4 percentage points, showing a strong improvement over competitors. In terms of the top-k 2-hop neighbor recall (Appendix C), our graph NE outperformed the competitors on all datasels apart from the Photo graph (Table S6
In terms of the classification accuracies, the results on most datasets were very similar across all methods. Graph NE had slightly lower ANN accuracy the two smallest datasets (Citeseer and Cora), and was the best or within 1% of the best all other datasets (Figure Bb, Table S2 In terms of linear accuracy, graph NE yielded competitive results and lagged only slightly behind other methods for some datasets (Figure B Table S3 Curiously, graph NE with T = 0.5 was the best or within 1% of the best on all datasets apart from Cora and MAG, where it was slightly behind (Table S3p: but this temperature led to substantially worse neighbor recall (Table S1 This suggests trade-off between linear classification and neighbor quality.
Graph NE outperformed the other methods on the link prediction task (Table S4 but the performance for many methods was close to saturated 100%. For that reason we prefer the KNN recall metric which is conceptually similar (Appendix By. Graph NE also outperformed all other methods in terms of Spearman correlation between the shortest-path distances and the embedding distances, on all datasets apart from MAG, where Laplacian Eigenmaps showed the best results (Figure S5
In summary, results terms of classification accuracies were all similar, but neighbor recall showed large and pronounced differences with graph NE performing the best by a large margin.
7.2 Graph NE outperforms other graph layouts in terms of local structure
We benchmarked graph NE with the openTSNE backend against five existing graph-layout algorithms: SGtSNEpi Pitsianis et al. (2019), Force Atlas2 (FA2; Jacomy et al. 2014). Laplacian Eigenmaps (L.E; Belkin & Niyogi 2003), DRGraph (Zhu et al., 2020a), and t-FDP (Zhong et al. 2023). We also performed comparisons with Diffusion Maps (Coifiman & Lafon 2006 with L 1 diffusion step), which differ from Laplacian Eigenmaps only by scaling, but found that they produced results very similar to Laplacian Eigenmaps, so we do not report them separately. We did not include tsNET (Kruiger et al. 2017), because it cannot embed large graphs H Unless specified otherwise, we used the original implementation of the algorithms and ran them with the default parameters. For FA2 we used the Barnes-Huu implementation by Chippada (2017) Both t-SNE and t-FDP are implemented in Cython, DRGraph and SGtSNEpi are implemented in C++ and offer wrappers in Python and Julia, respectively. For consistency, we used Diffusion Maps initialization for all algorithms where possible (all except SGiSNEpi and DRGraph).
Graph NE showed outstanding performance on all of our benchmark datasets. The neighbor recall of graph NE was always the highest, with SGtSNEpi only somctimes coming close (Figure 4b, Table 2). In the top-k 2-hop neighbor recall that focuses on pairs of nodes that share many neighbors (Appendix C graph NE also performed on average the best (Table S6). In this metric, SGtSNEpi only marginally (within 1%)
outperformed graph NE on two graphs and was much worse on several other graphs.
In terms of KNN accuracy, graph NE was either the top performing method or within 1% of the top performing
method for all datasets (Figure 4 p, Table S2 In terms of linear accuracy the same was true for six out of the eight datasets (Figure c Table S3
In terms of the Spearman correlation between the shortest-path distances and the embedding distances, consistent winners across datasets were ForceAtlas2 and t-FDP (Table (5).. This is likely related to the fact that in 2D embeddings, there a trade-off between preserving local and global structure (Böhm et al. 2022
Qualitatively, graph NE layouts performed well in terms of separating clusters from each other and bringing out sub-cluster details within individual clusters (Figure 5
While paper in print, Mcidiana al. (025 developed fast implementation of tNT We leave its benchmarking
future work. In parallel, Meldlana & Hong (2025 studied UMAP applied to the pairwise shortest-patt distances between nodes, like IsNE.
10

---

Published in Transactions on Machine Learning Research (11/2025)
graph NE
b
DRGraph
ForceAtlas2
d
[-FDP
ANN acc,+91.08
ANN acc. +7.7%
MNN 77c.477.48%
ANN acc. -7.2%
(ec7/l-41.7%
12.66
11.2%
recall 10.4%
Computer
10
M 1000
10
e
NNN -92.9%
f
ANN
g
88. h
84 9%
17.11
20.7%
recall 20.6%
22.7%
Photo
10
1000
10
Figure 5: Emboddings of the Computer and Photo datasets obtained using graph NE (graph t-SN),, DRGraph, ForceA llas2, and (-FDP. Embeddings were aligned using Procrustes rotation. See Figure 53 for all datasets and methods.
8 Discussion
We suggested graph NE, a novel approach to non-parametric node embeddings, and showed that outperforms existing competitors in terms of preserving local graph structure, both for high-dimensional embeddings and for 2D graph layouts. Graph NE can be efficiently implemented using existing neighbor-embeddding backends, CNE and TSEn Both backends scale linearly in the number of graph edges and achieve competitive runtimes for large graphs. For small graphs, we observed that the open'TSNE backend was slower than some of the competitors (Figure $1).
In this work, we focused on complex real-world graphs and have purposefully not lested our graph NE on simple planar graphs or 3D mesh graphs that are often used for benchmarking graph layout algorithms, Such graphs are arguably not an interesting case for high-dimensional embeddings, and we aimed to use the same graphs for all of our benchmarks.
Our work opens up several directions for future work. First, CNE allows to train parametric embeddings DDamrich et al. 2023), which we have not explored here. How would parametric CNE with a GCN mapping compare to existing GCL methods, in particular augmentation--ree methods? Parametric models need to use node features. Given the ongoing debate about the usefulness of combining node and edge information graph learning (Errica et al.] 2020 Faber al.) |022 [eehler-Speichhe would be interesting to study how much node features can help with node embeddings.
al. [2024] Coupette et al., 2025), it
Second, we only used I-SNE-like losses here, but a similar approach could be implemented using other neighbor- embedding algorithms, e.g. UMAP (Mclnnes et al. 2018). How would graph UMAP (2D and high-D) perform for graph layouts and node embeddings, especially in contrast to DRgraph and Deep SeepWalk/nodeevee which, like UMAP, use negative sampling for optimization?
Third, our results point to a non-trivial effect that the temperature parameter can have on InfoNCE-based embeddings (Figure D Further investigation of this phenomenon and its potential relevance for contrastive learning in computer vision and other domains also remains for future work.
Our graph NE algorithms succeed because of their simplicity, not despite of it. The straightforward loss function enables efficient optimization strategies, which scale linearly with the graph size and preserve nodes' neighbors better than other algorithms.
11

---

Published in Transactions on Machine Learning Research (11/2025)
Code availability
Our code is available at http:://ggthhub.cccm/rrreeell//d
hrapsshllaagghhh-aaaoaa paper
Acknowledgements
The authors would like thank Leland McInnes and Pavlin Poličar for discussions, and Philipp Berens for support and feedback. The authors benefited from the discussions at the Dagstuhl seminar 24122 supported by the Leibniz Center for Informatics.
This work was partially funded by the Gemeinnützige Hertie-Stiftung, the Cyber Valley Research Fund (D.30.28739), the Danmarks Frie Forskningsfond (Sapere Aude 1051-00106B), and the National Institutes of Health (UM1MH130981)). The content solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dmitry Kobak is member of the Germany's Excellence cluster 2064 "Machine Learning New Perspectives for Science" (EXC 390727645). The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-TS) for supporting Jan Niklas Böhm.
References
Nurudin GlnzaezzGonzalez, Andreas Kaltenbrunner, and Vicenç Gómez. Beyond Weisfeiler Lehman with local ego-network encodings. Machine Learning and Knowledge Extraction, 5(4):1234-1120) 2023.
Aleksandr Artemenkov and Maxim Panov. NOVis: noise contrastive approach for scalable visualization. Proceedings of The Web Conference 2020, pp. 2941-2947, 2020.
Maya Bechler-Speicher, Tdo Amos, Ran Gilad-Bachrach, and Amir Globerson. Graph neural networks use graphs when they shouldn't. In International Conference on Machine Learning, 2024.
Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373-1390 2003.
Anna C Belkina, Christopher O Ciccolella, Rina Anno, Richard Halpert, Josef Spidlen, and Jennifer E Snyder-Cappione. Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets. Nature Communications,, 10(1):5415, 2019.
Jan Niklas Böhm, Philipp Berens, and Dmitry Kobak. Attraction-repulsion spectrum neighbor embeddings. The Journal of Machine Learning Research, 23(1):4118-411 2022.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pp. 1597 1607. PMLR, 2020.
Bhargav Chippada. forceatlas2: Fastest Gephi's ForceAtlas2 graph layout algorithm implemented for Python 2017.
and NetworkX. https://github. http:/bbaggaacchppppaaafaeeeeeee
Ronald R Coifman and Stéphane Lafon. Diffusion maps. Applied and Computational Harmonic Analysis, (1):5-30) 30, 2006.
Corinna Coupette, Jeremy Wayland, Emily Simons, and Bastian Rieck, No metric to rule them all: Toward principled evaluations of graph-learning datascts. In International Conference on Machine Learning, 2025.
Sebastian Damrich, Niklas Böhm, Fred A Hamprecht, and Dmitry Kobak. From t-SNE to UMAP with contrastive learning. In International Conference on Learning Representations, 2023.
Aaron Defazio, Francis Bach, and Simon Lacosto-Julien. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in Neural Information Processing Systems, volume 27, 2014.
12

---

Published in Transactions on Machine Learning Research (11/2025)
Andrew Draganov, Jakob Jorgensen, Katrine Scheel, Davide Mottin, Ira Assent, Tyrus Berry, and Cig- dem Aslay. ActUp: analyzing and consolidating iSNE & UMAP. In Proceedings of the Thirby-Second International Joint Conference on Artificial Intelligence, pp. 3651 3658, 2023.
Federico Errica, Marco Podda, Davide Bacciu, Alessio Micheli, al. A fair comparison of graph neural networks for graph classification. In International Conference on Learning Representations, 2020.
Lukas Faber, Yifan Lu, and Roger Wattenhofer. Should graph neural networks use features, edges, or both? arxiu, 2021.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
Thomas MJ Fruchterman and Edward M Reingold. Graph drawing by force-directed placement. Soflwure:
Practice and Experience, 21(11):1129 1164, 1991.
Emden R Gansner, Yifan Hu, and Stephen North. A maxent-stress model for graph layout. IEEE Transactions on Visualization and Computer Graphics, 19(6):927-94) -940, 2012.
Martin Grohe. word2vec, node2vee, graph2vee, x2vec: Towards theory of vector embeddings of structured data. In Proceedings of the 39th ACM SICMOO-SIIAACT-SICAA Symposium on Principles of Database Systems, pp. 1-16, 2020.
Adilya Grover and Jure Leskovec. node2vec: Scalable feature learning for nelworks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855 864, 2016.
Xiaojun Guo, Yifei Wang, Zeming Wei, and Yisen Wang. Architecture matters: Uncovering implicit mechanisms in graph contrastive learning, In Advances in Newral Information Processing Systems, 2023.
Aric Hagberg, Pieter Swart, and Daniel S Chult. Exploring network structure, dynamics, and function using NelworkX. Technical report, Los Alamos National Lab, 2008.
Kaveh Hassaui and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. International Conference on Machine Learning, pp. 4116 4126. PMLR, 2020.
Geoffrey E Hinton and Sam Roweis. Stochastic neighbor embedding. Advances in Newral Information Processing Systems, 15, 2002.
Paul W. Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social Networks, 5(2):109 137, 1983. ISSN 0378-8733.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovee. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems, 33:22118 221132, 2020.
Yifan Hu. Efficient, high-quality force-directed graph drawing. Mathematica Journal, 10(1):37-71, 2005.
Mathieu Jacomy, Tommaso Venturini, Sebastien Heymann, and Mathieu Bastian. ForceAtlas2, a continuous graph layout algorithm for handy network visualization designed for the gephi software. PloS One, 9(6): e98679, 2014.
Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. Exploring the limits of language modeling. arXiv preprint arXi::1602.02. 2016.
Megha Khosla, Vinay Sctty, and Avishck Anand. A comparative study for unsupervised network represcntation learning. IEEE Transactions on Knowledge and Data Engineering, 33(5):1807-1818, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference Learning Representations, 2015.
13

---

Published in Transactions on Machine Learning Research (11/2025)
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. Interna- tional Conference for Learning Regresenlalions, 2017.
A. V. Knyazev, M. E. Argentati, I. Lashuk, and E. E. Ovtchinnikor. Block locally optimal preconditioned cigenvaluc xolvers (blopex) in hypre and petsc. SIAM Journal on Scientific Computing, 29(6))2224-22)) 2007.
Dmitry Kobak and George C Linderman. Initialization is critical for preserving global data structure in both t-SNE and UMAP. Nature Biotechnology, 39(2):156-177, 2021.
Johannes F Kruiger, Paulo E Rauber, Rafael Messias Martins, Andreas Kerren, Stephen Kobourov, and Alexandru C Telca Graph layouts by t-SNE. In Computer Graphics Forum, volume 36, pp. 283-294. Wiley Online Library, 2017.
Yann LeCun, Léon Botton, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11)22278 2324, 1998.
Namkyeong Lee, Junseok Lee, and Chanyoung Park. Augmentation-free self-supervised learning on graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 7372 7380, 2022.
Yao Yang Leow, Thomas Laurent, and Xavier Bresson. GraphTSNE: visualization technique for graph- structured data. Representation Learning on Graphs and Monifold Workshop at the International Conference for Learning Representations, 2019.
Haifeng Li, Jun Cao, Jiawei Zhu, Qinyao Luo, Silu He, and Xuying Wang. Angmentation-frre graph contrastive learning of -ivariantdisscininalve representations. IEEE Transactions on Neural Networks and Learving Systems, pp. 1-11, 2023.
George C Linderman and Stefan Steinerberger. Clustering with t-SNE, provably. STAM Journal Mathematics of Data Science, 1(2):313 332, 2019.
George C Linderman, Manas Rachh, Jeremy G Hoskins, Stefan Steinerberger, and Yuval Kluger. Fast interpolation- I-SNE for improved visualization of single-cell RNA-seq data. Nature Methods, 16(3): 243-245, 2019.
Zhuang Ma and Michael Collins, Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency. In Proceedings of the 2018 Conference on Empirical Methods in Nalural Language Processing, pp. 3698 0700,,77, 2018.
Leland Mclnnes, John Healy, and James Melville. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arxiv:1802.00 08426, 2018.
Amyra Meidiana and Seok-Hee Hong. SS-GUMAP, SL-GUMAP, SSSL-GUMAP: Fast UMAP algorithms for large graph drawing. arXiv preprint Xix:25500 09909099..979,, 2025.
Amyra Meidiana, Seok-Hee Hong, and Lwan-Liu Ma. BH-NNET, FIc-ISEET L-tsEET: Fast tsNET algorithms for large graph drawing. arXiv preprint Xiw:22009.1978. 2025.
Tomas Mikolov, Tlya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 26, 2013.
Jacob Miller, Vahan Huroyan, and Stephen Kobourov. Balancing between the local and global structures (LGS) in graph embodding. In International Symposium on Graph Drawing and Network Visualization, pp- 263 279. Springer, 2023.
Andreas Noack. Energy models for graph clustering. Journal of Graph Algorithms and Applications, 11(2): 453 -8-4 2007.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXi:: 1ett1100770448 2018.
Fabian Pedregosa, Gaäl Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825- 2830, 2011.
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701-710, 2014.
Nikos Pitsianis, Alexandros-Staree Ilooouuls, Dimitris Floros, and Xiaobai Sun. Spaceland embedding of sparse stochastic graphs. In 2019 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1-8, 2019.
Nikos Pitsianis, Alexandros-Stavros Niopoulos, Dimitris Floros, and Xiaobai Sun. Sg-tsne-x. http:://ee) archive. /tt.a///rr./////022222222221//// http:///////tt/////////epp.p.. duke. 2024. Accessed: 2025-01-25.
Pavlin G. Poličar, Martin Stražar, and Blaž Zupan. openTSNE: modular Python library for t-SNE dimensionality reduction and embedding. Journal of Statistical Software, 109(3):1-30, 2024.
Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. A
theoretical analysis of contrastive unsupervised representation learning. In International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 5628-5653 5637. PMLR, 2019.
Peter H. Schönemann. A generalized solution of the orthogonal Procrustes problem. Psychometrikk, 1966.
Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L Dyer, Remi Munos, Petar Veličković, and Michal Valko. Large-scale representation learning on graphs via bootstrapping. In International Conference on Learning Representations, 2022.
Puja Trivedi, Ekdeep Singh Lubana, Yujun Yan, Yaoqing Yang, and Danai Koutra. Augmentations in graph contrastive learning: Current methodological flaws & towards better practices. In Proceedings of the ACM Web Conference 2022, pp. 1538-15499 2022.
Laurens van der Maaten. Accelerating 1-SNE using tree-based algorithms. The Journal of Machine Learving Research, 15(1: 3245, 2014.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(1)), 2008.
Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon, Deep graph infomax. International Conference for Learning Representations, 2(3):4, 2019.
Minjic Wang, Da Zheng, Zihao Ye, Quan Gan, Mufci Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arxiv: 1909.01391, 2019.
Yifei Wang, Qi Zhang, Tianqi Du, Jiansheng Yang, Zhouchen Lin, and Yisen Wang. A message passing per- spective on learning dynamics of contrastive learning. International Conference on Learning Representations, 2023.
Zhirong Yang, Jaakko Peltonen, and Samuel Kaski. Scalable optimization of neighbor embedding for visualization. In International Conference on Machine Learning, pp. 127-135. PMLR, 2013.
Zhirong Yang, Yuwei Chen, Denis Sedov, Samuel Kaski, and Jukka Corander. Stochastic cluster embedding. Statistics and Computing, 33(1):12, 2023.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812- 33:5812-5823, 2020.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf, and Philip Yu. From canonical correlation analysis to self-supervised graph neural networks. Advances in Newral Information Processing Systems, 34:76- 1776-89,40 2021.
Hengrui Zhang, Qitian Wu, Yu Wang, Shaofeng Zhang, Junchi Yan, and Philip S Yu. Localized contrastive learning on graphs. arXiv preprint 3rXio:2212.04600 2022.
Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. In Advances in Newral Information Processing Systems, volume 31, 2018.
Shiqi Zhang, Renchi Yang, Xiaokui Xiao, Xiao Yan, and Bo Tang. Effective and efficient pagerank-based positioning for graph visualization. Proceedings of the ACM on Management of Data, 1(1):1 27, 2023.
Fahai Zhong, Mingliang Xue, Jian Zhang, Fan Zhang, Rui Ban, Oliver Deussen, and Yunhai Wang. Force- directed graph layouts revisited: new force based on the t-distribution. IEEE Transactions on Visualization and Compuler Graphics, 2023.
Minfeng Zhu, Wei Chen, Yuanzhe Hu, Yuxuan Hou, Liangjun Liu, and Kaiyuan Zhang. DRGraph: An efficient graph layout algorithm for large-scale graphs by dimensionality reduction. IEEE Transactions on Visualization and Computer Graphics, 276))1166-1166, 2020a.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. Xiv preprint arXiv:2006. 04 131, 2020b.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive lcarning with adaplive augmentation. In Proceedings of the Web Conference 2021, pp. 2069 2080, 2021.
16

---

Published in Transactions on Machine Learning Research (11/2025)
Appendix
A Proof of Theorem 1
Theorem 4.1.
/adapted from Ma & Collins 2018 Let. be a probabilily distribution over S 1 jiij n), so that for all pairs ij, there path "iki....,Phi. with each step having positive probability. Let w(O) be a family of non-negalive functions S I2z paramelrized by 0 O and syremelric in and meaning (0) (0) Let further be probability distribution over [nx] = [I....,n, with full support. Suppose there is some 0" E E and some > O with w(f")/e p. Then, H" minimizes the loss
Way (0)(0)
CInfoNCE (0) -Riga-p BKy log
(8)
(0)(E) + >a= Wike (0)/Ek.
and for any other minimizer 0 E @ there exists ë >> 0 with w(0)/ē= p. In particular, the minima of CInfaNCE: correspond one-to-one to those of
(p, w(0)/Z()) -Ei-- log
Wij
(9)
Z(0)
where Z(0) Eig Wiy (f)
Proof. The key idea is to rewrite the loss as average over cross-entropy losses over the set {0....,m) For i, ko.-.. him E (1......) ..) define
ali, K(i,ka... ku. ..... "o.....,,,))) Σ piks,
II EKC)
(11)
#=0
pikk Ilw awe Exe
pit EEn Eneo Piku Eke
( i, k.ko..... km)
(12)
ai,i, ko km)
Wik, (0)/EH,
ro k.ko.... .a.....,, km)) = Ev= Wskk, (0)/Ek.
(13)
Note that ali, ku .., )m)) can only be zero Pik, for all p = ,,...,m. 3(u i, ko, km)) for μ O,..., m. II all Wik, (0) are zero for v O,..., m, we define If they are identical to zero, B(- ko...., k....,,,m) and 70(- i, k.ko..... ..., km) are probability distributions over {o....,mi) m).
m. In this case, we define
(h( i, ..,
By the proof of Theorem 4.1 in Ma & Collins 2018), we can rewrite
ali, Ko, ... kim)
m Σ ===
CIntoNCE(
Σ
3(u ko. ..... ., km log ro ho...... kim)
(14)
m +
i.ko.... E(n]
The latter term in parentheses is cross-entropy loss over probability distributions over the set {0, .0....... m).
It becomes minimal if and only if 3(- i, ko...., .ob.....))).... %0(- (- i, ko.oo....... km). Since afi, ko, km) is positive as soon as min, Pix, > 0 for some μ, we conclude that the minima of LInloNCE are those for which B(- .o....,................... km) = 70 i, ko...., km) whenever min, Pik. > O.
Clearly H" is a minimum of CInfoNCE as
B(- i, ko..........................k... k.m) 70. ko ... km)
holds for all ko,..., ki [m].
17

---

Published in Transactions on Machine Learning Research (11/2025)
Conversely, let 0 be minimizer of (InfoNCF(()) Consider ij with Pij 0 and arbitrary i [n]. Taking ko = j and .......he kim k, we have
8(0 i, ko = j.k1 = k...,, km k) = =7 i ko = =j.ks i.kı = ....., km = k)
(15)
Pog/Es
wijl wir(0)/5 (0)/E)
(hence way (0) > 0)
(16)
Pss/Es + mppsh/Ex
Way (0)/E) + mwss mu)s(()/EE
1
1
4
(17)
1 + m Pinki
mig (0)5)
PiyEx
1 1+ m
Wiy (9)5k
Wik (0)
< Pik
(18)
Pij
wiy (0)
Σn Pik
En Wsk
(19)
Pij
wiy
Wiy(0)
Σx Wik (0)
Ci.
(20)
Pij
Σn Pik
which only depends on i, not on j. By symmetry of w, we have for any ij with Pij > 0 that pjs >> 0, too, and
G = wir(0)
Pij wir(Or) w):(0*) = w)r(0) Cj.
wiy(0) = ():(
Pje
Now we use the assumption that one can get from any i to any j via transitions with positive p. This implies that for each i, there is some j with Pij > 0 (which we ensure in our experiments by only considering connected component, Section 5). Morcover, for any i.j we get Cj =: So, we have for any ij with Pij > O that Pij = wiz(0)fe
Finally, let ūj be such that Pij = 0. By the path connectedness assumption, there some k E [n] with Pi > O.
Taking kn = and ki,..., km = j we get as above that
Pik
Wik
(21)
Pik + mp:
wa((0) + mw);(()
from which can conclude that wiy (0) must be zero, whenever Pty is zero. This implies p p= w(0)/c.
It is well-known that the KL divergence between two probability distributions is minimal if and only if they are equal.
Our setup in Theorem 4.1 deviates from that of Theorem 4.1 in Ma & Collins 2018). because they only show
a statement along the lines of Equation (20). In contrast, we show that the InfoNCE loss has same minima as the KL divergence using stronger assumptions (Ma & Collinsts Assumption 2.2 symmetry, and the path connectedness with positive probability).
B Link prediction as evaluation metric
Link prediction one of the standard evaluation criteria in the graph-lcarning community (Kipf & Welling 2017). In link-prediction (ask, the predictive model given pairs of nodes which may may not be connected by an edge and must rank these pairs by the likelihood of predicting an edge correctly (Zhang & Chen 2018). We used 10% of all graph edges as positive test pairs and equally many non-edges as negative test pairs. We scored each of the test pairs based on their embedding points, either by cosine similarity (128D) or by negative Euclidean distance (2D) and computed the area under the ROC curve, similar to the setup from Grover & Leskoved (2016).
The neighbor recall metric is conceptually similar to link prediction. is the recall of a directed link predictor P that predicts the links from node i to the k; nearest nodes to i's, where k; is the degree of node
18

---

Published in Transactions on Machine Learning Research (11/2025)
i. In the case of constant, known node degree k, the recall r is linearly related to the accuracy of P by = 1 (1 r) 2n/(n 1). As the task of link prediction is easier and saturates faster compared to the neighbor recall (Table S1 vs. Table S4 we use neighbor recall as our main metric.
C Top-k 2-hop neighbor recall as evaluation metric
Our neighbor recall metric, defined in Equation 110 treats all neighbors of node equally. In some cases, it may be desirable to locus on node pairs that have many shared neighbors, and make sure that they are embedded close together. We quantified this using an additional metric, which we call top-k 2-hop neighbor recall.
We first compute number of shared neighbors for any pair of nodes i,j:
S(i,)) - Nalll]
(22)
where [i is the set of neighbors of node graph G. Be definition, the S(i,,) is non-zero for all 2-hop neighbors of node i. Note that S(i,i)/ No U Najil gives Jaccard similarity between i and j.
Now let Ns denote the set of k nodes with the highest values of S(i, The top-k 2-hop neighbor recall is the fraction of these nodes contained among the k closest embedding points Ng. of node i:
Top-k 2-hop recall = M
(23)
k
We used & 10 and report the results in Table S6 For nodes that had fewer than other nodes with non-zero S(i, +), the [ contained fewer than elements.
19

---

Published in Transactions on Machine Learning Research (11/2025)
D Supplementary Figures and Tables
Table S1: Neighbor recall for all methods and datasets (in %). All values are mean standard deviation across three training runs. The top performing method for each dimensionality (and all methods within 1%)
is highlighted in bold. Methods in blue are ours. "Graph NEr" means that the temperature was learned as parameter during training, see Section
d
Method
Citeseor
Cora
PubMed
Photo
Computer
MNIST
arXiv
MAG
graph NE
81.0 = 0.1
83.8 + 0.0
44.3 + 0.2
70.3 + 0.1
64.8 +
96.0
0.0
72.3 0.3
89.0
+ U.4
graph NEr
80.3 1 0.0
82.8 0.1.
42.3 1 0.1
69.5
1 0.0
64.0 0.1
96.0
0.0
72.5 0.7
91.0
0.1
CNE,
0.5
56.9 = 0.1
58.1 1.10
0.3
0.1
0.0
30.2
0.2
33.2
0.2
DeepWalk
60.5 = 0.7
0.3
0.5
50.0
4.3
70.8
70.80000 0.2
0.5
00.0
0.6
node2vec
1.0
0.5
4.2
0.2
0.6
Laplacian E.
53.4 1 0.0
56.7 1: 0.0
18.3 0.1
39.7 39.7 0.0
32.4
0.0 38.5
32.1
40.6
0.2
graph NE
71.7 + 1.8
66.7 0.4
0.1
0.1
41.8
40.2 0.2
36.3
32.3
SGrSNEpi
09.1
0.3
0.6
0.2
0.3
0.2
29.1 0.1
0.26
8.1
0.3
DRGraph
0.4
0.9
0.1
0.3
1..0
4.
3.2
ForceAtlas2
38.8
0.3
2.34
3.2
20.6
2.1
11.10.11
7.0
2.9
1.7 0.1
t-FDP
24.4
1.3
10.2
0.7
0.3
Laplacian E.
26.2
1..9
2.1
16.0 0.0
6.4 0.0
0.0
1.6
0.8 0.801.1 0.1
Table S2: kNN classification accuracy. The same setup for each
Table S1
with random training splits used
Method
Citesver
Cora
PubMed
Photo
Computer
MNIST
arXiv
graph NE
72.0 = 0.4
82.7 0.0
84.1 + 0.1
94.3 0.1
92.4
97.2 0.0
71.3 0.1
41.6
graph NEr
72.2
0.4
83.1
83.8 0.3
0.1
0.2
97.1
7..
41.7
ONE, 7 0.5
72.8 0.2
0.2
06.1:000
92.6 0.0
96.9 0.0 91.4000
96.9
71.1
36.5
0.1 0.1
128
Delp
73.6
0.1
97.0
71.2
node2ver
73.1 0.4
82.8 0.5
83.6 1 0.4
1.2
90.5 0.2
96.8 1 0.0
1.01
34.4 0.1
Laplacian
74.5 - 0.0
83.1 0.0
82.6
0.0
93.0
90.3
0.1
graph NE
70.3 1: 0.4
83.1 1: 1.5
82.9 0.5
0.5
91.0
96.8 1. 0.1
6.24
35.3 1 0.0
SG:SNEpi
70.6 0.6
80.5 1.3
8.94
0.4
0.3
0.1
0.3
23.5 0.3
DRGraph
70.3 = 0.7
79.8
8.6
0.2
0.9
95.2 0.3
5..7
23.3 0.1
ForenArlas2
69.3
0.3
0.2
0.2
0.4
0.6
(-FDP
06.0 1.0
0.6
6..2
84.5
0.5
8.0
26.1
7.0 0.6
Laplacian
65.6 0.0
7.8
72.4
0.2
73.2
75.3 0.1
27.8
9.4
Table S3: Linear classification accuracy. The same sctup as in Table S2 applics.
Method
Citeseer
Cora
PubMed
Photo
Computer
MNIST
arXiv
MAG
graph NE
71.5 + 1.2
$1.3 1.0
80.9 + 0.2
93.0
0.2
89.7 + 0.2
95.3 0.0
6.26
26. 1 0.1
graph NEr
0.3
63.6
ONE,
0.5
72.8 1 0.8
1..3
83.6 1 0.3
92.7
1 0.2
0.2
97.1 1 0.0
67.9
32.2 0.0
128
DeepWalk
70.3
0.7
81.9 0.3
0.2
68.1
node2vec
66.4
81.0
77.0
0.5
8..5
96.1 0.1
61.6
29.9
0.2
Laplacian E.
73.6
0.0
86.7 + 0.0
8.0
92.8 0.0
85.5 0.0
97.0
40.0
21.3 0.1
graph NE
63.2 2.7
66.9 1.2
6.08
72.5
4.8
0.7
96.0 0.1
48.4
18.9 0.6 0.6
SGISNEpi
79.2
93.3 0.7
DRGraph
09.111.
64.8
6.2
6.2
1.5
9..2
47.6
18.5 = 0.3
ForceArlas2
62.4 02
0.0
7.06
7.4
6.2
7..1
45.4
18.2
tFDP
50.5
0.2
5.88
0.4
0.5
0.2
26.5
8.0
Laplacian E.
47.6 0.0
4.02
57.1 1.0
60.4 60.4 0.0
47.3
60.4
1..
7.1
20

---

Published in Transactions on Machine Learning Research (11/2025)
Table S4: Area under the link prediction ROC curve. The same setup as in Table S2 See Appendix B
Method
Citeseer
Cora
PubMed
Photo
Computer
MNIST
arXiv
MAG
geaph NE
100.0 0.0
100.0
100.0
0.0
0.0
100.0 =
100.0
graph NEr
100.0 0.0
1.0
100.0 1
9,.
9.0
100.0 0.0
100.0
0.0
100.0 100.0=00 0.0
CNE, 7 U.5
99.7 = 0.0
99.7 9..7
9.0
9.0
9.0
99.8 0.0
98.7 0.0
99.7
128
DeepWalk
99.8 0.0
99.8 1.00
0.0
98.1
9..2
100.0
99.5 0.0
100.0
node2vec
91.6 0.3
8.2
0.3
8.2
7.2
9.0
66.3 0.2
93.7 0.1
Laplacian
graph NE
98.2 0.1
0.2
96.3
96.4
9..1
98.4
95.7
0.2
98.4 0.0
SGtSN Epi
97.6 0.2
95.5
96.0
9..1
93.8
0.12
95.3 0.1
96.4 0.1
DRGmaph
98.9 1: 0.0
97.6
9.0
96.4
94.5
98.7 0.0
97.2 0.1
99.0 0.0
2
Atlas2
0.1
9.07
9..6
96.4
9.0
0.0
97.2 0.1
98.8 0.1
t-FDP
97.8
9..1
9.3
96.1
91.3
98.9
88.8 0.5
92.8 0.6
Laplacian
0.0
0.0
1.4
Table $5: Spearman correlation between the shortest-path distances and the embedding distances (Euclidean for 2D embeddings, cosine for 128D embeddings). The same setup as in Table S1
Method
Citrseer
Cora
PuhMed
Photo
Computer
MNIST
arXiv
MAG
graph NE
67.4 = 1.0
81.6 0.1
71.4 = 0.3
82.4
0.3
75.6 0.0
54.2 0.6
60.7 0.6
7.3 1.3:10 0.8
graph NEr
48.4
59.2
1.2
2.1
ONE,
0.5
33.5 = 0.0
0.11
32.9 0.2
09.0
32.0
5..
7.8
21.1 1 0.3
DeepWalk
31.1 0.3
2.69
17.0=
0..
31
4.9
0.5
9.7 17711
node2vec
27.1 = 1.3
0.5
23.6 0.8
4.3
42.7 =
35.1
3.0
15.3 -
Laplacian
29.6 1 0.0
26.1
1.0.0
37.7
0.0
25.8 0.0
30.0
0.0
26.8 1.00000.00
4..
37.5
0.4
graph NE
56.9 1.5
47.6
35.0 2.6
58.3
47.2
4..6
9.99
34.0
SGtSNEpl
46.5 6.1
40.0
33.81.7
2.7
44.1 1.7
5.3
33.8
29.9
DRGraph
61.8 3.2
0.9
41.3 1.2
1.
49.5
51.3
1.
31.0
PorcuAtlas2
65.3 0.2
71.6100
51.0
0.0
69.7
54.9
0.0
58.2
1.00
4.7
23.4
0.5
1.FDP
65.7 0.1
71.1 1.1
63.6
0.0
0.4
63.8
0.5
0.0
1.79
53.6
Laplacian E.
45.5 0.0
50.7 0.0
21.3 0.0
5.0
36.2 = 0.0
0.0
35.7 1.5
20.7
0.3
Table S6: Top-k (k 10) 2-hop neighbor recall. The same setup as in Table ST
See Appendix C
Method
Citeseer
Cora
PubMed
Photo
Computer
MNIST
arXiv
MAG
graph NE
44.5 =
+ 0.0
+ 0.0
27.2
25.9
57.8
30.3
30.9
graph NEr
44.3 1 0.0
48.1 0.1
35.410..
26.7 1 0.0
25.8 0.0
57.7 1 0.1
30.2 0.1
29.6 1 0.2
ONE,
38.0 +
38.8
30.5
0.0
2.0
22.3
128
DelpWalk
39.7 0.1
0.3
3.3
0.1
0.2
53.1 03.1:00
25.9
30.7 2
pode2vor
41.8 0.1
43.7 0.11
3..4
30.9
0.11
24.4
36.1 0.3 0.3
20.9 0.1
14.2
0.1
Laplacian
36.5 0.0
38.0 0.0
2.0
23.1
0.0
36.5
0.0
27.0
graph
35.0
28.2
21.8
SGISNEpi
34.1
3.4
28.7
23.8
21.
24.6
1.2
5.2 0.1
DRGraph
2.52
0.5
0.2
9.2
6.1
7.1
3.8
2.1
ForceAtlas2
25.3 0.0
0.1
1..
11.
3.8
3.2
1.4 0.1
t-FDP
17.3 0.0
1.2
3.810.1
1..6
5.1
11.5
0.6
0.2
Laplacian
0.0
0.0
3.2
0.2
Table S7: Learned temperature for the graph NET variant in Tables S1 runs reported. Standard deviations were alsways below 0.0005.
S5
Means across three training
Method
Citeseer
Cora
PubMed
Photo
Computer
MNIST
arXiv
128 graph NE.
0.071
0.077
0.070
0.079
0.076
0.057
0.058
0.042
21

---

Published in Transactions on Machine Learning Research (11/2025)
a
128D
b
2D
10,000
10,000 s
graph NE
graph NE
Runtime
SGiSNEpi
DeepWalk
DRGraph
100
node2vec
100 s
ForceAtlas2
Laplacian E.
Laplacian E.
1s
Citeseer
Cora PubMed
Photo
Computer
MNIST KNN
arXiv
MAG
Citeseer
Cora PubMed
Photo
Computer
MNIST KNN
arXiv
MAG
Figure S1: Computation times. All computations were performed on a cluster which isolates the computing resources and removes interference between concurrent computations. All 2D experiments require only CPUs and were ran on 8 cores of an Intel Xeon Gold 6226R. Experiments in 128D) ran on single Nvidia 2080ti GPU card. For node2vec, this shows runtime for p g our actual runtime including hyperparameter tuning was much larger.
ran 25 parameter combinations (Figure S2
Cora KNN acc.
Computer
Photo KNN acc.
recall
lin. acc.
recall
KNN acc.
lin. acc.
recall
lin. acc.
100%
75%
25%
Citeseer
MNIST ANN
PubMed
100%
100%
100%
75%
75%
75%
50%
50%
50%
25%
25%
25%
arXiv
MAG
100%
100%
75%
9
50%
50%
2
0.25
0
p
Figure S2: Evaluations for node2vec Grover Leskoved 2016 with the hyperparameter sweep over P. E (0.25, 0.5, 1, 2, 4). These parameter values were taken from the original node2vec paper. The highest. neighbor recall was always achieved p 0.25, q 4. This corresponds to oversampling unseen nodes.
22

---

Published in Transactions on Machine Learning Research (11/2025)
graph NE
SGtSNEpi
DRGraph
ForceAtlas2
[-HDP
Laplacian E.
70.3% 69.0%
70.8% 59.4%
72.6% 444.7
69.3% 39.4%
63.7% 24.2%
65.6%
Citesea
26.2%
80.6% 6%
79.8%
82.7%
81.5%
77.0%
71.8%
65.7%
55..8
31.6%
24.4%
14.6%
17.9%
Cora
82.4%
81.4%
79.8%
77.2%
62.5%
72.4%
24.9%
23.0%
8.0%
3.3%
1.2%
2.1%
PubMed
93.3%
92.6%
90.1%
88.4%
84.4%
84.8%
47.1%
24.1%
22.7%
20.7%
21.9%
16.0%
Photo
91.1%
90.8%
82.3%
76.8%
72.2%
73.1%
41.9%
33.8%
13.5%
1 11.0%
10.4%
6.5%
Computer
96.7%
97.0%
95.0%
81.1%
84.2%
75.4%
MNIST KNN
-4.33
29.0%
10.3%
0.0%
13.8%
5.0%
69.3%
66.3%
54.6%
50.3%
25.1%
28.6%
35.7%
23.3%
55.4%
2,55%
00.7%
1.6%
arXiv
35.3%
23.2%
23.7%
20.2%
7.4% 0.4%
KNN acc, 9.7% recall: 0.9%
82.1%
8.5%
3.0%
1.8%
MAG
Figure S3: Embeddings of all considered datasets obtained using our graph NE, SGLSNEpi Pitsianis et al. 2019), DRGraph (Zhu et al. 2020aa), Atrc2 (Jacomy al.) 2014p, t-FP (Zhngg et al. 2023), and Laplacian Eigenmaps (Belkin & Niyogi 2003). Embeddings in each row were aligned using orthogona Procrustes rotation (Schönemann) 1966). Numbers in the top-right corner correspond to the ANN accuracy and the neighbor recall, respectively (as indicated in the bottom-right panel).
23

---

Published in Transactions on Machine Learning Research (11/2025)
Default
Random init.
C
Whole-matrix norm.
KNN
41.75%
recall 42
recall 23.3
Computer
Default
Random init.
f
Whole-matrix norm.
ANN aoc 92.9%
ANN acc. 93.3% recall 456.8%
KNN acc. 91.3% -9call-39.68
r40111-47.15.
Photo
Figure S4: The effect of initialization and normalization on 2D graph NE for the Computer and Photo datasets. (a, d) Default graph NE with Diffusion Maps initialization and using por-node normalization of the adjacency matrix. (b, e) Graph NE using random initialization. (c, f) Graph NE with whole-matrix normalization. Embeddings in each row were aligned using Procrustes rotation.
24


Reviews:
[{"review_id": "4XHTt2Cl57", "reviewer": "Reviewer_tCp4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a graph node embedding algorithm that builds on ideas from low-dimensional graph visualization, particularly contrastive neighbor embeddings. A theorem apparently shows an equivalence between the optima of the proposed InfoNCE objective and the well-known tSNE objective for embedding high-dimensional data. Experiments show that Graph NE has high neighbor recall, meaning that neighboring points in the graph are embedded among the nearest neighbors in metric space.", "claims_and_evidence_yn": "Yes", "claims_explanation": "I'm not very familiar with this area but to my understanding the experiments are convincing. \n\nGiven the equivalence of tSNE and infoNCE, would an alternative baseline be to compute embeddings from graph Laplacian and then perform tSNE directly on these embeddings?\n\nRegarding the theory, a major point of confusion was the relationship between the affinities q_{ij} and w_{ij} (introduced without explanation in equation (3)). Are these the same? Similarly, I didn't understand the role of q_theta^* in the statement of theorem 4.1. Please define this term in the theorem statement or elsewhere.", "audience_yn": "Yes", "audience_explanation": "I suspect people working on network embeddings would be interested, but it's not my area so it's hard to say for sure.", "requested_changes": "Please see my questions about about the notation and potential baselines.", "broader_impact_concerns": "no concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Yk2rBh3KE9", "reviewer": "Reviewer_H86V", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "ContributionsThe authors extend the fields of non-parametric node embedding and graph layouts by introducing graph NE. They combine normalization of the adjacency matrix with InfoNCE.\n\nI appreciate the work done by the authors and the extent to which their method has been tested on multiple datasets using different performance metrics, and the many visualisations provided in the Appendix.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors claim to introduce a non-parametric graph representation learning method. They say that it can be used for both high dimensional embeddings as well as visualisations. \nThe authors present an extensive experiment testing both the embedding power (in 128D) and give ample visualisations of the result sin 2D.", "audience_yn": "Yes", "audience_explanation": "This paper is interesting to TMLR’s audience because it presents a novel way of embedding and visualizing networks. This has implications outside this work as well, since qualitative visualisations can support the results and story of other network-based papers as well.", "requested_changes": "•\tI believe the authors should elaborate more on the hyperparameters. They are fixed, but I believe this should be justified clearly. This includes the choice to stick to dimension 128. \n•\tEspecially for visualisations, calculation time is the major bottle neck. I think this should get a more prominent role in the paper. Now, it is put away in appendix. Regarding these tests, the authors should clarify why LE is not present in the results of Figure S1. \n•\tRegarding the graph layouts in 2D in Section 4.3, are the authors applying Equation (7) for graph NE, or SGtSNEpi? Looking at the results from the experiments, it seems like SGtSNEpi is only used as a method to compare to, but including it in Section 4.3 causes some confusion. \n•\tIn Section 4.3, the authors quickly mentioned ‘visually unpleasing embeddings.’ It would be informative if the authors reported why this is unpleasing, and what metrics (if any) they used for this. A visualization being ‘visually pleasing’ could be a qualitative metric the authors could report for Figures S3 and S4 as well.  \n•\tIn the introduction, at the end of the second paragraph, ‘neighbor-embedding approaches’ is mentioned for the first time, but it is not entirely clear to what part of the methods mentioned in paragraph it applies. It should be better to also introduce the term in the same way as the authors did with ‘node-embedding methods’ (start of the same paragraph). \n•\tI believe the last paragraph of the related work part on parametric node embeddings (the part in italics) should be kept for the methodology section.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "S4PJGqQBDQ", "reviewer": "Reviewer_AE4y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Graph Neighbour Embeddings (Graph NE), a non-parametric node embedding method that avoids random walks and instead directly optimises embeddings by pulling adjacent nodes closer together. Empirical results on multiple datasets are given, and compared with those of other non-parametric methods like DeepWalk and node2vec. The method is also applied to the problem of 2D graph layouts, comparing its results with classical layout algorithms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The evidence is mostly convincing, with extensive experiments on multiple datasets and comparisons to several baselines. However, some points would benefit from further clarification and extension:\n\n\nIn particular, I am not convinced by the use of the proposed recall metric to prove the superiority of the approach compared to SoTA solutions.\nAll direct neighbours of a node are treated equally, which does not seem to be what users usually want when, for example, visualising a network in 2D. Usually, nodes which share more common neighbours should be placed in closer proximity than nodes with no common neighbours.\n\nI suggest defining a distance metric based on this notion (or a normalised version of it) and then evaluating, for example, with precision@k (with k being, for example, 1, 3, 10, ...) as well as the Spearman correlation between this new distance metric and the embedding distances. This way, the shares of the k closest nodes in the embedding are actually among the most similar nodes in the original graph, using a finer metric than just direct neighbourhood (which still could be used as a first discriminator for distance)\n\nAnother concern is the performance in terms of computation times in 2D embeddings for many smaller networks (as evidenced in S2b). It seems to be worse than competing algorithms in smaller networks by several orders of magnitude. This may put limits to the algorithms' practical usability in visualisation tools, like for example Gephi, where users tend to expect results within a few seconds.\n\nFurthermore, the authors show that their results may depend on the temperature parameter τ, but only show the effect of two parameter values and leave more detailed experiments to future work. However, in the Appendix, results for NEτ are shown (which seem to optimise for this parameter) but are never explained in the main text, nor are the corresponding values of the temperature parameter τ reported. I suggest that since there are already results reported, the authors should extend the manuscript and at least explain these results, and mention a range of suitable values of the parameter.\n\nFinally, there have been studies (see, for example, [1]  that show how by using also the degrees of the neighbours when encoding a node, performance in link prediction tasks can be improved compared to DeepWalk and node2vec. The authors might want to compare their method against this setting.\n\n[1 ]Alvarez-Gonzalez et al. (2023). Beyond Weisfeiler–Lehman with Local Ego-Network Encodings. Machine Learning and Knowledge Extraction, 5(4), 1234-1265.\n\nAn additional advantage of the method used in the study mentioned above is that it is inductive, meaning that it is able to encode new nodes that are not known when the graph is first encoded. The authors might also want to comment on a limitation of their method in this respect (If I understood correctly, it is transductive).", "audience_yn": "Yes", "audience_explanation": "Although the applicability of the paper may be limited (nearly all real-world graphs do have node features) it provides both novel insights (showing that pulling neighbours directly might be effective) and practical impact (potentially better embeddings and graph layouts), which should be of interest to a segment of TMLR’s audience.", "requested_changes": "- Consider incorporating a common-neighbours-based proximity metric (or a normalised variant) in addition to direct adjacency, and evaluate with measures such as precision@k or Spearman correlation.\n\n- Explain better that for smaller networks, the performance in terms of Computation times in 2D embeddings is much worse than competing algorithms.\n\n- Give more details about the NEτ variant of their method and the impact of other values of their temperature parameter\n\n- Explain the limitations of their study regarding inductive graph embedding techniques\n\n- Add computation times to the visualisations provided in Figure 5 and Figures S3\n\n- Report the area under the ROC curve for the link prediction tasks as stated already in the Appendix.", "broader_impact_concerns": "I do not have any major concerns. The work is mainly methodological, advancing unsupervised graph embedding techniques. It does not directly raise ethical risks or problematic applications beyond the usual caveats for graph representation learning. As with all graph embedding methods, there is a possibility of bias amplification when applied to social or citation networks, but this is not specific to the proposed method", "additional_comments": "The caption of Table S4 speaks about Link prediction accuracy, while in the text in Appendix C it is stated that the area under the ROC curve has been computed. The latter is the standard metric and should be reported.\n\nOn page 4, it is stated that \"the cosine distance and the squared\nEuclidean distance differ only by a constant\" which is slightly imprecise. It would be better to say \"the cosine distance and the squared\nEuclidean distance differs only by a constant factor\"", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)