# Fair Benchmark for Unsupervised Node Representation Learning
###### Abstract  
Most machine-learning algorithms assume that instances are independent of each other. This does not hold for networked data. Node representation learning (NRL) aims to learn low-dimensional vectors to represent nodes in a network, such that all actionable patterns in topological structures and side information can be preserved. The widespread availability of networked data, e.g., social media, biological networks, and traffic networks, along with plentiful applications, facilitate the development of NRL. However, it has become challenging for researchers and practitioners to track the state-of-the-art NRL algorithms, given that they were evaluated using different experimental settings and datasets. To this end, in this paper, we focus on unsupervised NRL and propose a fair and comprehensive evaluation framework to systematically evaluate state-of-the-art unsupervised NRL algorithms. We comprehensively evaluate each algorithm by applying it to three evaluation tasks, i.e., classification fine tuned via a validation set, link prediction fine-tuned in the first run, and classification fine tuned via link prediction. In each task and each dataset, all NRL algorithms were fine-tuned using a random search within a fixed amount of time. Based on the results for three tasks and eight datasets, we evaluate and rank thirteen unsupervised NRL algorithms. network embedding; benchmark; hyperparameter tuning; attributed networks; graph neural networks +
Footnote †: journal:  
0000-0001-6000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-00000-0000-00000-00000-00000-00000-00000-00000-00000-00000-000000-00000-000000  
hidden layers can perform a wide variety of tasks, such as node classification [11; 12], anomaly detection [13], community detection [14], and link prediction [15; 16]. In the early 2000s, as illustrated in Figure 1, NRL was known as graph embedding [17; 18; 19], which was mainly used to perform dimensionality reduction for feature vectors. Its core idea can be summarized as two steps. First, given a set of instances, such as documents or images of faces, associated with feature vectors, we construct a new graph to connect all instances. Its \((i,j)^{\text{th}}\) edge weight is set as the shortest path distance [18], reconstruction coefficient [20; 21], or similarity [17] between nodes \(i\) and \(j\) in the feature vector space. Second, we apply multidimensional scaling [18] or eigen decomposition to the adjacency matrix or its variation [17; 20]. In such a way, the learned node representations can be employed as the latent features of instances. It has been proven that most dimensionality reduction techniques, such as principal component analysis and linear discriminant analysis, could be reformulated as graph-embedding processes with constraints [19]. Meanwhile, efforts have also been devoted to utilizing NRL as an intermediate process to conduct clustering [22; 23; 24], known as spectral clustering. Its key idea is similar to graph embedding [19]. Based on feature vectors, we construct a new network depicting similarities between instances, and then calculate the eigenvectors of the graph Laplacian matrix of this new network. The learned eigenvectors serve as the latent representations of instances. To predict the clusters, a traditional clustering algorithm such as \(k\)-means is applied to the eigenvectors. Later, since mid 2000s, with the rapid development of the web, especially social-networking sites, there has been an increase in the types and numbers of real-world networks available, such as Facebook, Flickr, and Twitter. NRL has been employed as an intermediate step when performing node classification [12; 25; 26] and visualizations [27] in real-world networks, including online social networks such as Flickr [26] and BlogCatalog [28], academic networks [25], molecule structures [27], and linked web pages [12].