Higher-order Structure Prediction in Evolving Graph Simplicial Complexes

1 INTRODUCTION . Numerous types of networks like social ( Liben-Nowell & Kleinberg , 2007a ) , biological ( Airoldi et al. , 2006 ) , and chemical reaction networks ( Wegscheider , 1911 ) are highly dynamic , as they evolve and grow rapidly via the appearance of new interactions , represented as the introduction of new links / edges between the nodes of a network . Identifying the underlying mechanisms by which such networks evolve over time is a fundamental question that is not yet fully understood . Typically , insight into the temporal evolution of networks has been obtained via a classical inferential problem called link prediction , where given a snapshot of the network at time t along with its linkage pattern , the task is to assess whether a pair of nodes will be linked at a later time t′ > t. While inferring pairwise links is an important problem , it is oftentimes observed that most of the real-world graphs exhibit higher-order group-wise interactions that involve more than two nodes at once . Examples illustrating human group behavior involve a co-author relationship on a single paper and a network of e-mails to multiple recipients . In nature too , one can observe several proteins interacting together in a biological network simultaneously . In spite of their significance , in comparison to single edge inference , relatively fewer works have studied the problem of predicting higher-order group-wise interactions . Benson et al . ( 2018 ) originally introduced a simplex to model group-wise interactions between nodes in a graph . They proposed predicting a simplicial closure event , whereby an open simplex ( with just pairwise interactions between member vertices ) transitions to a closed simplex ( where all member vertices participate in the higher-order relationship simultaneously ) , in the near future . Figure 1 ( Middle ) shows an example of such a transition from an open triangle to a closed one . Recently , several works have proposed modeling higher-order interactions as hyperedges in a hypergraph ( Xu et al. , 2013 ; Zhang et al. , 2018 ; Yoon et al. , 2020 ; Patil et al. , 2020 ) . Given a hyperedge ht at time t , the inference task is to predict the future arrival of a new hyperedge ht′ , which covers a larger set of vertices than ht and contains all the vertices in ht . Figure 1 ( Right ) illustrates this hyperedge prediction task . Although prediction models based on either simplicial closure event prediction or hyperedge arrival , deal with higher-order structures , they both fail to capture the highly complex and non-linear evolution of higher-order structures over time . Both these kinds of models have two major limitations . First , they predict structures from a single static snapshot of the graph , thus not viewing the evolution process of adding new edges as a time process . Second , their feature extraction is mostly based on popular heuristics ( Adamic & Adar , 2003 ; Brin & Page , 2012 ; Jeh & Widom , 2002 ; Zhou et al. , 2009a ; Barabási & Albert , 1999 ; Bhatia et al. , 2019 ) that work well in practice but are not accompanied by strong theoretical guarantees . In addition to the aforementioned shortcomings , hypergraph based methods model higher-order structures as hyperedges , which omit lower-dimensional substructures present within a single hyperedge . As a consequence , they can not distinguish between various substructure relationships . For example , hyperedge [ A , B , C ] in Figure 1 ( Right ) can not distinguish between group relationships like [ [ A , B ] , [ B , C ] , [ A , C ] ] ( a set of pairwise interactions ) versus [ A , B , C ] ( all A , B and C simultaneously in a relationship ) . This problem is remedied by the use of simplices because they naturally model these substructures as a collection of subsets ( i.e. , faces ) of the simplex . We provide real-world examples of where our simplicial complex based approach can play a significant role . ( i ) Organic Chemistry : It is quite common to have the same set of elements interacting with each other in different configurations , which result in very different functioning compounds ( Ma et al. , 2011 ) . Specifically , R-thalidomide and S-thalidomide are two different configurations of thalidomide , where the R-form was meant to help sedate pregnant women , while the S-form unfortunately resulted in birth defects . This is a famous example in stereo chemistry to show the consequences of mistaking two extremely close configurations ( differing by a single bond ) as being the same . Structure prediction to avoid such phenomenon in drug synthesis allows chemists to achieve a much higher yield and avoid wastage of expensive resources . ( ii ) Gene expression networks : Gene networks have nodes that represent genes and edges connect genes which have similar expression patterns ( Zhang & Horvath , 2005 ) . Subgraphs called modules are tightly connected genes in such a gene expression network . Genomics research provides evidence that higher-order gene expression relationships ( like second and third-order ) and their measurements can have very important implications for cancer prognosis . When making structural predictions in these aforementioned examples , our simplicial complex based approach provides much more fine-grained control over competing methods by capturing subtler differences in configurations . To combat these challenges , our approach views the evolving graph1 as a time process under the framework of nonparametric time series prediction , which models the evolution of higher-order structures ( as simplices ) and their local neighborhoods ( spatial dimension ) over a moving time window ( temporal dimension ) . Our inference problem is then modeled as predicting the evolution to a higher-dimensional simplex at time t′ > t , given a simplex at time t. It is important to note that this task is more general and greatly diverges from the task proposed by Benson et al . ( 2018 ) . Our 1We handle the incremental model ( edge insertions only ) as opposed to the harder fully dynamic model ( edge insertions and deletions allowed ) for which most previous methods too can not provide theoretical guarantees . task requires just a single simplex σ in order to predict a higher-dimensional simplex τ , whose face / subset is σ , whereas Benson et al . ( 2018 ) requires the presence of all constituent σ faces in order to predict τ . For example , in Benson et al . ( 2018 ) ( also shown in Figure 1 ( Middle ) ) all faces [ A , B ] , [ B , C ] and [ A , C ] ( open triangle ) need to be present in order to predict a closed triangle [ A , B , C ] . Contrastingly , in our approach , just a single face/edge like [ A , B ] or [ B , C ] or [ A , C ] , suffices to predict its evolution to [ A , B , C ] . Figure 1 ( Left ) illustrates an additional example of our proposal to predict a 3-simplex [ A , B , C , D ] given only one of its faces [ A , B , C ] . To this effect , we succinctly capture the features characterizing the local neighborhood of a simplex as a combination of a face-vector ( Björner & Kalai , 2006 ) , which is a well-established vector signature in combinatorial topology literature and a novel scoring function which infers the affinity of sub-simplices based on the strength of their past interactions . Based on these features , we design a kernel estimator to infer future evolution to higher-dimensional simplices and prove both the consistency and asymptotic normality of our estimator . Our contributions : ( a ) We propose a kernel estimator that predicts higher-order interactions in an evolving network . ( b ) We prove the consistency and asymptotic normality of our kernel estimator . ( c ) We evaluate our method on real-world dynamic networks by proposing higher-order link prediction baselines and observe significant gains in prediction accuracy in comparison to the baselines . 1.1 RELATED STUDIES . Single link prediction : Most literature that predicts a single edge/link can be broadly classified as based on : ( i ) heuristics , ( ii ) random-walks , or ( iii ) graph neural networks ( GNNs ) . ( i ) Heuristic methods comprise of Common neighbors , Adamic-adar ( Adamic & Adar , 2003 ) , PageRank ( Brin & Page , 2012 ) , SimRank ( Jeh & Widom , 2002 ) , resource allocation ( Zhou et al. , 2009a ) , preferential attachment ( Barabási & Albert , 1999 ) , persistence homology based ranking ( Bhatia et al. , 2019 ) , and similarity-based methods ( Liben-Nowell & Kleinberg , 2007b ; Lü & Zhou , 2011 ) . ( ii ) Random walk based methods consist of DeepWalk ( Perozzi et al. , 2014 ) , Node2Vec ( Grover & Leskovec , 2016b ) and SpectralWalk ( Sharma et al. , 2020 ) . ( iii ) Finally , for both link prediction and node classification tasks , recent works are mainly GNN-based methods such as VGAE ( Kipf & Welling , 2016 ) , WYS ( Abu-El-Haija et al. , 2018 ) , and SEAL ( Zhang & Chen , 2018b ) . Higher-order link prediction : Benson et al . ( 2018 ) are the first to introduce a higher-order link prediction problem where they study the likelihoods of future higher-order group interactions as simplicial closure events ( explained earlier ) . Furthermore , there are studies using hypergraphs which also help naturally represent group relations ( Xu et al. , 2013 ; Zhang et al. , 2018 ; Yoon et al. , 2020 ; Patil et al. , 2020 ) . Especially , to represent higher-order relationships , Yoon et al . ( 2020 ) proposed n-projected graphs . For larger n , i.e. , higher-order groups , the enumeration of subsets , and keeping track of node co-occurrences quickly becomes infeasible . In comparison to a hypergraph , our graph simplicial complex is closed under taking subsets , which enables us to better encode more information for improved inference . 2 PRELIMINARY : GRAPH SIMPLICIAL COMPLEX ( GSC ) . We start with a general notion of an abstract simplicial complex ( ASC ) , then define a simplex using ASCs . We specialize this definition to graphs and define a graph simplicial complex ( GSC ) . Definition 1 ( Abstract simplicial complex and simplex ) . An abstract simplicial complex ( ASC ) is a collection A of finite non-empty sets , such that if σ is an element of A , then so is every nonempty subset of σ . The element σ of A is called a simplex of A ; its dimension is one less than the number of its elements . Now , we analyze graphs using the definition of ASCs . Let G = ( V , E ) be a finite graph with vertex set V and edge set E. A graph simpicial complex ( GSC ) G on G is an ASC consisting of subsets of V . In particular , G is a collection of subgraphs of G. With graphs , we denote a ddimensional simplex ( or d-simplex ) of a GSC by σ ( d ) = [ v0 , v1 , . . . , vd ] . Each non-empty subset of σ ( d ) is called a face of σ ( d ) . We define several notions related to GSCs , that are useful for describing the evolution of graphs . Definition 2 ( Filtered GSC ) . For I ⊂ N , a filtered GSC indexed over I is a family ( Gt ) t∈I of GSCs such that for every t ≤ t′ in I , Gt ⊂ Gt′ holds . Obviously , Gt0 ⊂ Gt1 ⊂ . . .Gtn is a discrete filtration induced by the arrival times of the simplices : Gti \ Gti−1 = σti . This depicts a higher-order analogue of an evolving graph ( incremental model ) , which allows attaching new simplices at each time-step to an existing GSC to build a new GSC . A filtered GSC Gt , p for the last p discrete time steps is defined as Gt , p : = ( Gt′ ) tt′=t−p = ( Gt−p , . . . , Gt ) , where Gt′ ⊃ Gt′−1 . We define a notion for dealing with the neighborhood around a given simplex σ ( d ) ∈ G. We introduce a set of all simplices of dimension d′ or less from G , i.e. , G ( d ′ ) − : = { σ ( d ) ∈ G | d ≤ d′ } . G ( 0 ) − is a vertex set and G ( 1 ) − is the set of edges and vertices . We also write i ∼ j whenever vertices i and j are adjacent in G ( 1 ) − , and write i ∼k j to indicate that vertex j is k-reachable from i , i.e. , there exists a path of length at most k , connecting i and j in G ( 1 ) − . Then , we define a ball around σ ( d ) . Definition 3 ( k-ball centered at vertex and simplex ) . At time t , we define a k-ball centered at vertex i by Bt , k ( i ) : = { j : i ∼k j and i , j ∈ G ( 0 ) t− } . We also define a k-ball centered at a simplex σ ( d ) as Bt , k ( σ ( d ) ) : = ⋃ i : Vert ( σ ( d ) ) Bt , k ( i ) , where Vert ( σ ( d ) ) denotes all vertexes in σ ( d ) . Now , we define a sub-complex G′t ( σ ( d ) ) ⊆ Gt as the GSC that contains all the simplices in Gt spanned by the vertices in the k-ball Bt , k ( σ ( d ) ) .

Summary: This paper provide a method for high-order structure prediction problem. Specifically, the paper first defines a high-order structure on graphs named graph simplicial complex (GSC). Then the paper introduces a feature generation method used for the high-order structures. The features are also used in the proposed method for high-order structure prediction. The proposed method is based on a nonparametric kernel which carries the feature similarities of high-order structures. With this kernel the method uses a Bernoulli distribution for the prediction of the existence of the high-order structure in unseen times.