# Fair Benchmark for Unsupervised Node Representation Learning
## 3 Unsupervised Node Representation Learning Algorithms and Datasets
### Unsupervised Node Representation Learning Methods Used in FURL
#### 3.1.1. Node Representation Learning for Pure Networks  
We roughly categorize the existing NRL algorithms for pure networks into three classes, i.e., negative-sampling-based, matrix-factorization-based, and spectral-embedding-based. First, typical negative-sampling-based NRL algorithms includes DeepWalk [1], node2 vec [31], and LINE [10]. Second, the typical matrix-factorization-based NRL algorithms include NetMF [38] and ProNE [39]. The key idea is to construct a proximity matrix and use singular value decomposition or other matrix-factorization algorithms to obtain the graph embedding. Third, the key idea of spectral embedding [22] is to compute the eigendecomposition of a normalized Laplacian matrix of the network. Spectral embedding based on NRL algorithms is not scalable, since the time complexity of eigendecomposition is \(\mathbb{O}(N^{2})\), where \(N\) denotes the total number of nodes. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Algorithms** & **Hyperparameters \& Search Space** & **Type** \\ \hline \multirow{3}{*}{DeepWalk} & number\_walks: [5; 80] & int \\  & walk\_length: [5; 80] & int \\  & window\_size: [2; 40] & int \\ \hline \multirow{3}{*}{metapath2 vec} & window\_size: [1, 20] & int \\  & min\_count: [1; 10] & int \\  & walk\_length: [5; 80] & int \\  & walk\_num: [5; 80] & int \\ \hline \multirow{3}{*}{node2 vec} & p: (0.25, 0.5, 1, 2, 4) & discrete \\  & q: (0.25, 0.5, 1, 2, 4) & discrete \\  & walk\_size: [5; 120] & int \\  & length: [5; 120] & int \\  & window: [5; 15] & int \\ \hline NetMF & window: [5; 15] & int \\ \hline \multirow{3}{*}{ProNE} & step: [5; 20] & int \\  & theta: [0,1] & float \\  & mu: [0,1] & float \\ \hline NetSMF & window: [5; 15] & int \\ \hline HOPE & lr: (0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005) & discrete \\ \hline \multirow{3}{*}{SDNE} & nu1: (0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005) & discrete \\  & nu2: (0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.001, 0.00005] & discrete \\  & alpha: (0.5, 0.4, 0.3, 0.2, 0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005) & discrete \\  & beta: (0, 0.1, 0.5, 5, 10, 15, 20, 30] & discrete \\ \hline \multirow{3}{*}{LINE} & walk\_length: [1, 100] & int \\  & walk\_num: [1, 100] & int \\  & alpha: (0.5, 0.4, 0.3, 0.2, 0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005) & discrete \\ \hline \multirow{3}{*}{GAE} & lr: (0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005) & discrete \\  & nb\_epochs: [100, 1500] & int \\  & dropout: [0, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] & discrete \\  & hid\_dim: (64, 128, 256] & discrete \\  & weight\_decay: \([0,\,5\times 10^{-4},10^{-3}]\) & discrete \\ \hline \multirow{3}{*}{CAN} & lr: (0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005) & discrete \\  & nb\_epochs: [100, 1500] & int \\  & dropout: [0, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] & discrete \\ \hline \multirow{3}{*}{DGI} & lr: (0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005) & discrete \\  & nb\_epochs: [100, 1500] & int \\ \hline \multirow{3}{*}{FeatWalk} & alpha: [0, 1] & float \\  & num\_paths: \([10,50]\) & int \\  & path\_length: [5; 50] & int \\  & win\_size: [5; 15] & int \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hyperparameter search space of unsupervised NRL algorithms. * **DeepWalk**[1]: It first performs random walks on the network to convert it into a series of truncated node sequences. Then, it considers the recorded node sequences as a \"corpus\", i.e., each node as a word and each sequence as a sentence, and applies a wording embedding technique to embed this \"corpus\". In this way, we could learn a low-dimensional representation for each word (node), by using efficient wording embedding techniques. * **metapath2 vec**[40]: It formalizes meta-path-based random walks to construct the heterogeneous neighborhood of a node and then leverages a heterogeneous skip-gram model to learn node embedding representations.