# Fair Benchmark for Unsupervised Node Representation Learning
## 3 Unsupervised Node Representation Learning Algorithms and Datasets
### Unsupervised Node Representation Learning Methods Used in FURL
#### 3.1.2 Node Representation Learning for Attributed Networks  
**Definition 1** (Attributed Network).: _We refer the textual, image, numerical, or categorial data associated with each node as node attributes. A network with node attributes is defined as an attributed network. Examples include social networks with user-generated content, paper citation networks with abstracts, and protein-protein interaction networks with property descriptions._  
In this paper, we select four typical unsupervised NRL algorithms for attributed networks, as follows.  
* **GAE**[44]: It learns node embedding representations by reconstructing the network structure using autoencoders. A two-layer graph convolutional network is employed to conduct encoding.
* **CAN**[45]: It learns an embedding representation for each node and for each node attribute category. These two types of representations, i.e., node representations and attribute-category representations, are learned by variational autoencoders, and are in the same space.
* **DGI**[36]: It learns node embeddings by maximizing the mutual information between the local patch representation and the global graph representation.
* **FeatWalk**[46]: It advances random walks by introducing an attribute-enhanced walking strategy. In addition to random walks on the original network, it also allows walking from one node to another node through their shared node attribute categories. The joint truncated node sequences thus become more diverse.  
To handle networks with node attributes, the field of graph neural networks (GNNs) has expanded rapidly in recent years [47; 48; 49; 50]. Existing GNN models belong to the fam ily of message passing frameworks [51; 52], which update information for each node by recursively aggregating messages from its immediate neighbors in the graph. GNNs are generally classified into two categories, as follows.  
The first class is spectral-based approaches [53; 54; 55; 56; 57; 58; 59], which generalize convolution operators on the grid-like data to graphs based on spectral graph theory [60]. GCN [56] employs the first-order Chebyshev polynomial approximation for graph convolution. In a follow-up, [PERSON] et al. [59] analyze the expression power of GNNs according to the Weisfeiler-Lehman test for graph isomorphism. SGC [61] further simplifies GCN by removing the inner transformation matrices and non-linear activation functions. Although effective, they could struggle with modeling long-range dependencies because of a recursive message-passing restriction. APPNP [58] suggests replacing the original graph convolution matrix with a graph diffusion [62]. MixHop [63], N-GCN, [64] and Truncated Krylov [65] attempt to exploit multi-scale information in each layer [66]. The second line of research is spatial-based approaches [9; 34; 67; 68], which stem from the vector domain and focus on using the graph structure directly for message propagation. One characteristic of these is to adopt the learnable aggregation function to aggregate neighbors. GraphSage [9] introduces mean/max/LSTM pooled aggregation functions to integrate neighborhood information. GAT [69] learns to assign different attention weights at each layer based on node features. Attention-based GNN models have achieved state-of-the-art results on several graph learning tasks [68; 70; 71].  
However, most GNN models are dedicated to end-to-end classification. There is an issue when converting GNN models into unsupervised NRL methods. For example, if we directly employ one of the hidden layers as the learned embedding representations, GNN models would perform badly in Tasks 1, 2, and 3.