# A Scalable Deep Network for Graph Clustering
## 2 Related Work  
Graph clustering divides the unlabeled nodes into different clusters with a certain metric. After that, we can mine the relationships between different nodes in a graph. The early graph clustering models perform poorly on real-world datasets due to their shallow architecture and learning capabilities, such as matrix factorization [19] and DeepWalk [20]. In addition, [PERSON] et al [21]. propose two complementary algorithms for graph clustering called K-algorithm and M-algorithm. The combination of these two algorithms can obtain several local optimizations on the graph and they can be used with different cost functions. However, the two algorithms fail to integrate the graph topology information, which limits their final performance. Recently, many more effective models applied to unsupervised learning are proposed, such as auto-encoder [22] and generative adversarial networks (GAN) [23]. On this basis, many graph clustering models combined with deep learning techniques have been proposed and they have achieved good performance. The Graph Auto-encoder (GAE) [5] combines the auto-encoder with graph convolution. It first utilizes the two GCN layers to capture the information between graph topology and node features and then reconstructs an adjacency matrix to be as similar to the original matrix. The ARGA [7] adopts the adversarial training scheme to normalize the embedding process to obtain more robust embeddings. However, none of the above-mentioned models are clustering task-oriented joint optimization training methods. The DAEGC [9] combines the two components to jointly optimize the embedding module and the clustering module, which improves the quality of the embeddings and the clustering effect. Meanwhile, the SDCN [10] integrates the structural information into deep clustering by using a transfer operator to combine the auto-encoder with the GCN module. It can conduct end-to-end clustering training with the dual self-supervision module. Although these methods have superior performance, they still use the GCN module based on the message passing mechanism as the backbone network, limiting the scalability of these models. However, the above models also have several drawbacks: (1) There is no solution to the orthogonal relationship between the model depth and the neighborhood range (2) Too many smoothing iterations or GCN layers stacking lead to over-smoothing issues. To solve the limitations of the traditional GNN models, scalable GNN models are proposed. Early scalable GNNs simplified the model by sampling the graph, the GraphSAGE [24] samples the neighbors around the target node with the same probability, while the Fast-GCN [25] samples the nodes according to the importance of each node. Due to their node-wise sampling or layer-wise sampling method, they fail to learn large-scale sparse graphs effectively. On this basis, the GraphSAINT [26] proposes a subgraph-wise sampling method with high scalability, which decouples sampling from GNNs and further reduces the computational costs. The other direction of scalable GNNs in recent years is to simplify the model structure. The SGC [13] transforms the nonlinear GCN into a simple linear model by repeatedly eliminating the nonlinear function between the GCN layers and folding the final function into a linear function. The PPNP [12] modifies the propagation scheme by adopting the relationship between GCN and pagerank. The AGC [16] advocates the use of high-order graph convolution to capture the global features of the graph and it can adaptively select the appropriate order according to different graphs, while the AGE [27] optimizes the model of GAE by decoupling the GCNs and modifies the GNN models from the perspective of graph signal processing. The S2 GC [15] adopts an improved Markov diffusion kernel to derive a simpler variant of GCN that captures the global and local context of each node. Nevertheless, the SIGN [14] points out that the features among multiple layers should be considered together instead of a certain layer.