Evaluating Network Embeddings : Node 2 Vec vs Spectral Clustering vs GCN

Node classification on popular social network datasets in the graph setting arises in various real world networks. Being able to label a particular entity in a graph based on its neighboring graph structure and predicting relationships between entities plays an important role in analyzing social networks and content on the web. With the the resurgence of machine learning, supervised learning problems have been shown to accomplish a number of different tasks from classifying animals to determining the model of cars [7]. These methods have been further extended by the resurgence of Deep Learning to accomplish the same tasks such as classifying images with higher accuracy. However, it is noted by Leskovec et al. [2] that prediction tasks on graphs require careful feature engineering. In this paper we adapt work by Kipf et al [4][5] in order to leverage Graph Convolutional Neural Networks as a means of evaluating spectral embeddings in comparison to embeddings generated by the Node2Vec algorithm. Problem Introduction With the surge of popularity in various social media services, so too have we experienced a massive influx in the amount of available user data. With so much data floating around, it would be a significant waste for us not to utilize it for the purpose of improving people’s lives. One such avenue through which we may learn about people’s behaviors and preferences is via network analysis on social graphs. The underlying networks over which these social media services are built upon are not only rich in information, but they are also subjects of a vast amount of study and research. By applying what we have learned from work that has been done on similar network problems, we hope to gain insight into people’s social behaviors. For this project, we will explore the effectiveness of graph embedding algorithms such as node2vec and computing the top-k eigenvectors in predicting node label assignments. We strive to determine just how much information we are able to get from the structures of a network alone and evaluate this using a classification problem. The algorithms which we will use are the node2vec random walk algorithm [2], a simple spectral embedding technique, and the graph convolutional networks algorithm [4]. We aim to see how well these are able to do on the aforementioned classification problems and in what way we may improve them. Review of Prior Relevant Work With the resurgence of Machine Learning and specifically Deep Learning, a whole host of classification and prediction problems have been made amenable to being solved with a high degree of accuracy. However, it is noted by Leskovec et al. [2] that prediction tasks on graphs require careful feature engineering. In this paper we adapt work by Kipf et al [4][5] in order to leverage Graph Convolutional Neural Networks as a means of evaluating node embeddings specifically those generated from the Node2Vec and those generated from taking the top-k eigenvectors of the normalized adjacency matrix. Similar work by grover et al. [2] introduces a novel approach to learning continuous feature representations for nodes in networks. In addition to GCN, Deep Feature Learning for Graphs has been illustrated in the work by Rossi et al [9] which introduces a framework, DeepGL, for computing a hierarchy of graph representations. Furthermore, there have been efforts made to learn embeddings in a fully unsupervised manner by instead learning subgraph embeddings [6]. Another issue to note is that node2vec does not perform as well with structural equivalence tasks due to the fact that many networks exhibiting strong homophily. In such cases, approaches which try to learn the latent representation of a network will tend to perform poorly [8]. Furthermore, a long standing technique of graph analysis focuses on studying the eigenvalues and eigenvectors of the normalized adjacency matrix of the graph. For example, recent work by [1] looks at leveraging eigenvectors of the Laplacian of a graph to predict climate variability. Eigenvectors of graphs have also been shown to be effective at spectral clustering as shown by [10]. This leads us