Link Prediction in Hypergraphs using Graph Convolutional Networks

Link prediction in simple graphs is a fundamental problem in which new links between nodes are predicted based on the observed structure of the graph . However , in many real-world applications , there is a need to model relationships among nodes which go beyond pairwise associations . For example , in a chemical reaction , relationship among the reactants and products is inherently higher-order . Additionally , there is need to represent the direction from reactants to products . Hypergraphs provide a natural way to represent such complex higher-order relationships . Even though Graph Convolutional Networks ( GCN ) have recently emerged as a powerful deep learning-based approach for link prediction over simple graphs , their suitability for link prediction in hypergraphs is unexplored – we fill this gap in this paper and propose Neural Hyperlink Predictor ( NHP ) . NHP adapts GCNs for link prediction in hypergraphs . We propose two variants of NHP – NHP-U and NHP-D – for link prediction over undirected and directed hypergraphs , respectively . To the best of our knowledge , NHP-D is the first method for link prediction over directed hypergraphs . Through extensive experiments on multiple real-world datasets , we show NHP ’ s effectiveness . 1 INTRODUCTION . The problem of link prediction in graphs has numerous applications in the fields of social network analysis ( Liben-Nowell & Kleinberg , 2003 ) , knowledge bases ( Nickel et al. , 2016 ) , bioinformatics ( Lü & Zhou , 2011 ) to name a few . However , in many real-world problems relationships go beyond pairwise associations . For example , in chemical reactions data the relationship representing a group of chemical compounds that can react is inherently higher-order and similarly , the co-authorship relationship in a citation network is higher-order etc . Hypergraphs provide a natural way to model such higher-order complex relations . Hyperlink prediction is the problem of predicting such missing higher-order relationships in a hypergraph . Besides the higher-order relationships , modeling the direction information between these relationships is also useful in many practical applications . For example , in the chemical reactions data , in addition to predicting groups of chemical compounds which form reactants and/or products , it is also important to predict the direction between reactants and products , i.e. , a group of reactants react to give a group of products . Directed hypergraphs ( Gallo et al. , 1993 ) provide a way to model the direction information in hypergraphs . Similar to the undirected hypergraphs , predicting the missing hyperlinks in a directed hypergraph is also useful in practical settings . Figure 1 illustrates the difference between modeling the chemical reactions data using undirected and directed hypergraphs . Most of the previous work on hyperlink prediction ( Zhou et al. , 2006 ; Zhang et al. , 2018 ) focus only on undirected hypergraphs . In this work we focus both on undirected and directed hypergraphs . Recently , Graph Convolutional Networks ( GCNs ) ( Kipf & Welling , 2017 ) have emerged as a powerful tool for representation learning on graphs . GCNs have also been successfully applied for link prediction on normal graphs ( Zhang & Chen , 2018 ; van den Berg et al. , 2018 ; Schlichtkrull et al. , 2018 ; Kipf & Welling , 2016 ) . Inspired by the success of GCNs for link prediction in graphs and deep learning in general Wang et al . ( 2017 ) , we propose a GCN-based framework for hyperlink prediction which works for both undirected and directed hypergraphs . We make the following contributions : • We propose a Graph Convolutional Networks ( GCN ) -based framework called Neural Hyperlink Predictor ( NHP ) for the problem of hyperlink prediction . To the best of our knowledge , this is the first ever deep learning based approach for this problem . • We extend the proposed NHP for the problem of hyperlink prediction in directed hypergraphs . To the best of our knowledge , this is the first ever attempt at the problem of link prediction in directed hypergraphs . • Through extensive experiments on multiple real-world datasets , we show the effectiveness of proposed NHP for link prediction in both undirected and directed hypergraphs . We have released NHP ’ s source code at this anonymous location : https : //anonymous.4open . science/repository/7d86231e-f6ba-4795-ae51-ac28d89f1521/ . 2 RELATED WORK . In this section , we briefly review related work in deep learning on graphs and link prediction on hypergraphs . Learning representations on graphs : The key advancements in learning low-dimensional node representations in graphs include matrix factorisation-based methods , random-walk based algorithms , and deep learning on graphs ( Hamilton et al. , 2017 ) . Our work is based on deep learning on graphs . Geometric deep learning ( Bronstein et al. , 2017 ) is an umbrella phrase for emerging techniques attempting to generalise ( structured ) deep neural network models to non-Euclidean domains such as graphs and manifolds . The earliest attempts to generalise neural networks to graphs embed each node in an Euclidean space with a recurrent neural network ( RNN ) and use those embeddings as features for classification or regression of nodes or graphs ( Gori et al. , 2005 ; Scarselli et al. , 2009 ) . A CNN-like deep neural neural network on graphs was later formulated in the spectral domain in a pioneering work ( Bruna et al. , 2014 ) by a mathematically sound definition of convolution on graph employing the analogy between the classical Fourier transforms and projections onto the eigen basis of the graph Laplacian operator ( Hammond et al. , 2011 ) . Initial works proposed to learn smooth spectral multipliers of the graph Laplacian , although at high computational cost ( Bruna et al. , 2014 ; Henaff et al. , 2015 ) . To resolve the computational bottleneck and avoid the expensive computation of eigenvectors , the ChebNet framework ( Defferrard et al. , 2016 ) learns Chebyshev polynomials of the graph Laplacian ( hence the name ChebNet ) . The graph convolutional network ( GCN ) ( Kipf & Welling , 2017 ) is a simplified ChebNet framework that uses simple filters operating on 1-hop local neighborhoods of the graph . A second formulation of convolution on graph is in the spatial domain ( or equivalently in the vertex domain ) where the localisation property is provided by construction . One of the first formulations of a spatial CNN-like neural network on graph generalised standard molecular feature extraction methods based on circular fingerprints ( Duvenaud et al. , 2015 ) . Subsequently , all of the above types ( RNN , spectral CNN , spatial CNN on graph ) were unified into a single message passing neural network ( MPNN ) framework ( Gilmer et al. , 2017 ) and a variant of MPNN has been shown to achieve state-of-the-art results on an important molecular property prediction benchmark . The reader is referred to a comprehensive literature review ( Bronstein et al. , 2017 ) and a survey ( Hamilton et al. , 2017 ) on the topic of deep learning on graphs and learning representation on graphs respectively . Below , we give an overview of related research in link prediction on hypergraphs where relationships go beyond pairwise . Link Prediction on hypergraphs : Machine learning on hypergraphs was introduced in a seminal work ( Zhou et al. , 2006 ) that generalised the powerful methodology of spectral clustering to hypergraphs and further inspired algorithms for hypergraph embedding and semi-supervised classification of hypernodes . Link prediction on hypergraph ( hyperlink prediction ) has been especially popular for social networks to predict higher-order links such as a user releases a tweet containing a hashtag ( Li et al. , 2013 ) and to predict metadata information such as tags , groups , labels , users for entities ( images from Flickr ) ( Arya & Worring , 2018 ) . Techniques for hyperlink prediction on social networks include ranking for link proximity information ( Li et al. , 2013 ) and matrix completion on the ( incomplete ) incidence matrix of the hypergraph ( Arya & Worring , 2018 ; Monti et al. , 2017 ) . Hyperlink prediction has also been helpful to predict multi-actor collaborations ( Sharma et al. , 2014 ) . In other works , a dual hypergraph has been constructed from the intial ( primal ) hypergraph to cast the hyperlink prediction as an instance of vertex classification problem on the dual hypergraph ( Lugo-Martinez & Radivojac , 2017 ) . Coordinated matrix maximisation ( CMM ) predicts hyperlinks in the adjacency space with non-negative matrix factorisation and least square matching performed alternately in the vertex adjacency space ( Zhang et al. , 2018 ) . CMM uses expectation maximisation algorithm for optimisation for hyperlink prediction tasks such as predicting missing reactions of organisms ’ metabolic networks . 3 PRELIMINARIES . Undirected hypergraph is an ordered pair H = ( V , E ) where V = { v1 , · · · , vn } is a set of n hypernodes and E = { e1 , · · · , em } ⊆ 2V is a set of m hyperlinks . The problem of link prediction in an incomplete undirected hypergraphH involves predicting missing hyperlinks from Ē = 2V − E based on the current set of observed hyperlinks E. The number of hypernodes in any given hyperlink e ∈ E can be any integer between 1 and 2n . This variable cardinality of a hyperlink makes traditional graph-based link prediction methods infeasible because they are based on exactly two input features ( those of the two nodes potentially forming a link ) . The variable cardinality problem also results in an exponentially large inference space because the total number of potential hyperlinks is O ( 2n ) . However , in practical cases , there is no need to consider all the hyperlinks in Ē as most of them can be easily filtered out ( Zhang et al. , 2018 ) . For example , for the task of finding missing metabolic reactions , we can restrict hyperlink prediction to all feasible reactions because the infeasible reactions seldom have biological meanings . In other cases such as predicting multi-author collaborations of academic/technical papers , hyperlinks have cardinalities less than a small number , as papers seldom have more than 6 authors . The number of restricted hyperlinks in such practical cases is not exponential and hence hyperlink prediction on the restricted set of hyperlinks becomes a feasible problem . Formally , a hyperlink prediction problem ( Zhang et al. , 2018 ) is a tuple ( H , E ) , where H = ( V , E ) is a given incomplete hypergraph and E is a set of ( restricted ) candidate hyperlinks with E ⊆ E . The problem is to find the most likely hyperlinks missing in H from the set of hyperlinks E − E. Directed hypergraph ( Gallo et al. , 1993 ) is an ordered pair H = ( V , E ) where V = { v1 , · · · , vn } is a set of n hypernodes and E = { ( t1 , h1 ) , · · · , ( tm , hm ) } ⊆ 2V is a set of m directed hyperlinks . Each e ∈ E is denoted by ( t , h ) where t ⊆ V is the tail and h ⊆ V is the head with t 6= Φ , h 6= Φ . As shown in figure 1 , chemical reactions can be modeled by directed hyperlinks with chemical substances forming the set V . Observe that this model captures and is general enough to subsume previous graph models : • an undirected hyperlink is the special case when t = h • a directed simple link ( edge ) is the special case when |t| = |h| = 1 Similar to the undirected case , the directed hyperlink prediction problem is a tuple ( H , E ) , where H = ( V , E ) is a given incomplete directed hypergraph and E is a set of candidate hyperlinks with E ⊆ E . The problem is to find the most likely hyperlinks missing in H from the set of hyperlinks E − E .

Summary: This paper proposed Neural Hyperlink Predictor (NHP) to perform link prediction based on graph convolutional network (GCN). Following prior work, the hyperlink prediction is perform in the dual hypergraph, where each node represents a hyperlink in the primal hypergraph. The original problem is then equivalent to a simple node classification problem. To deal with directed hyperlink, a separate term is added to distinguish heads from tails.