Exploiting Node Content for Multiview Graph Convolutional Network and Adversarial Regularization (2020)

Abstract: Network representation learning (NRL) is crucial in the area of graph learning. Recently, graph autoencoders and its variants have gained much attention and popularity among various types of node embedding approaches. Most existing graph autoencoder-based methods aim to minimize the reconstruction errors of the input network while not explicitly considering the semantic relatedness between nodes. In this paper, we propose a novel network embedding method which models the consistency across different views of networks. More specifically, we create a second view from the input network which captures the relation between nodes based on node content and enforce the latent representations from the two views to be consistent by incorporating a multiview adversarial regularization module. The experimental studies on benchmark datasets prove the effectiveness of this method, and demonstrate that our method compares favorably with the state-of-the-art algorithms on challenging tasks such as link prediction and node clustering. We also evaluate our method on a real-world application, i.e., 30-day unplanned ICU readmission prediction, and achieve promising results compared with several baseline methods.

## introduction over the last few years, network representation learning, or node embedding, has gained increasing interest in the community of machine learning, due to the popularity of the special data form. in reality, datasets from different fields are often in the form of networks, such as social networks, drug-targetinteraction networks, mobile phone networks, citation networks, etc. it is therefore very important to find a way to well represent the networks, which is challenging because there is no direct way to encode the high-dimensional data into low-dimensional feature vectors efficiently @xcite . moreover, network embedding techniques benefit a variety of downstream applications like link prediction, node classification and node clustering. in recent years, researchers have developed different kinds of network embedding approaches, many of which have shown great performance in analytical evaluation and have been quite effective in downstream applications. these studies range from traditional machine learning techniques like matrix factorization to recent deep-learning-based methods like graph autoencoders. traditional models, or shallow models, usually optimize the embeddings of nodes directly. for these shallow models, the mapping from networks to vectors is simply a embedding lookup, i.e., each node corresponds to a unique embedding vector @xcite . factorization-based approaches like grarep @xcite , hope @xcite and random walk-based approaches like deepwalk @xcite , node2vec @xcite all fall into this category. shallow models generally suffer from computational inefficiency and lack of ability to well represent complex networks. more recently, deep models, or autoencoder-based approaches, have been gaining more and more attention, and have shown superior performance in many applications. compared with shallow models which use a simple lookup table as the encoder function, deep models usually use deep neural networks as the encoder. for example, sdne @xcite and dngr @xcite use deep neural networks as the encoder and decoder functions to generate low-dimensional representations. @xcite aggregate neighborhood messages based on convolutional encoders, e.g., graph convolutional networks (gcn) @xcite and its variants, to generate node embeddings. the encoders share parameters across nodes and it leads to better efficiency. note that gcn variants like graphsage @xcite and gat @xcite are not discussed as they mostly focus on message passing which is not the main focus of this work. another successful variant of graph autoencoders incorporates the generative adversarial networks (gan) for representation learning. for example, arga and arvga @xcite enforce the latent node embeddings to match a prior normal distribution based on an adversarial training mechanism. the adversarial training procedure usually provides regularization and results in more robust and meaningful representations @xcite . dbgan @xcite estimates the prior distribution of latent representations by prototype learning and aims to balance both sample-level and distribution-level consistency via a novel bidirectional adversarial learning framework. a common theme among most of the aforementioned approaches is that they do not explicitly consider the semantic relatedness between nodes. for shallow models like deepwalk @xcite , they mostly only focus on preserving the topological structure of the network while neglecting the rich information in node content. for deep models, they implicitly incorporate node content by aggregating neighborhood node features using powerful encoders like graph convolutional networks. in this paper, we propose a novel network embedding method based on multiview graph convolutional network and adversarial regularization. the method aims to preserve the distribution consistency across two views of the network, as well as shape the output representations to match an arbitrary prior distribution, by incorporating a multiview adversarial regularization module. more specifically, we regard the topological structure as the first and main view of the network, and create a second view that captures the relatedness between nodes based on node content. different from dbgan @xcite which tries to reconstruct the node features directly, the proposed method relaxes this requirement and focuses on preserving the semantic relatedness between them. a multiview reconstruction loss function is leveraged to optimize the model jointly. we evaluate the proposed method on three diverse applications. the experimental results on benchmark datasets demonstrate that the method outperforms the state-of-the-art algorithms in link prediction and node clustering. we also evaluate our method on a real-world downstream application, i.e., icu readmission prediction, and the method compares favorably with several baseline methods. our contributions can be summarized as follows: ## experiments in this section, we evaluate our proposed method based on three tasks. first, we conduct the experiment of link prediction on the benchmark dataset of three citation networks. we also report the experiment of node clustering on these networks. finally, we apply the proposed method to a real-world medical application, i.e., 30-day unplanned icu readmission prediction. ## related work recently, researchers use specifically designed encoders to aggregate the local neighborhood information of nodes, to generate low-dimensional embeddings. for example, gae and vgae (kipf and welling, 2016b) are two methods that use graph convolution networks (gcn) (kipf and welling, 2016a) as the encoder. vgae uses the gaussian distribution as a prior and pushes the learned representations close to this prior by incorporating a kl divergence penalty. arga and arvga incorporate an adversarial regularization framework for the same purpose, which is essentially similar in spirit to vgae. actually, incorporating adversarial regularization terms and matching the latent representations to a prior distribution are particularly useful for generating robust and meaningful representations when dealing with real-world complex graph data, which is first proposed by adversarial autoencoder (aae) @xcite . we extend the adversarial regularization framework to a multiview scenario where the distribution consistency across graph space and node content space are to be preserved. but unlike previous methods like dbgan @xcite and dane @xcite which aim to reconstruct the node content directly, we focus on the semantic relatedness among them. ## conclusion in this study, we propose a novel network embedding method, i.e., mrgae, which models the consistency of node representations across two specific views of networks. to achieve this, we incorporate a multiview adversarial regularization module and specifically design a loss function for joint optimization. we conduct extensive and diverse experiments for evaluation, and the results demonstrate the superb performance of the proposed method.