Hypergraph Convolutional Networks via Equivalency  between  Hypergraphs and Undirected Graphs

As a powerful tool for modeling the complex relationships , hypergraphs are gaining popularity from the graph learning community . However , commonly used algorithms in deep hypergraph learning were not specifically designed for hypergraphs with edge-dependent vertex weights ( EDVWs ) . To fill this gap , we build the equivalency condition between EDVW-hypergraphs and undirected simple graphs , which enables utilizing existing undirected graph neural networks as subroutines to learn high-order interactions induced by EDVWs of hypergraphs . Specifically , we define a generalized hypergraph with vertex weights by proposing a unified random walk framework , under which we present the equivalency condition between generalized hypergraphs and undigraphs . Guided by the equivalency results , we propose a Generalized Hypergraph Convolutional Network ( GHCN ) architecture for deep hypergraph learning . Furthermore , to improve the longrange interactions and alleviate the over-smoothing issue , we further propose the Simple Hypergraph Spectral Convolution ( SHSC ) model by constructing the Discounted Markov Diffusion Kernel from our random walk framework . Extensive experiments from various domains including social network analysis , visual objective classification , and protein fold classification demonstrate that the proposed approaches outperform state-of-the-art spectral methods with a large margin . 1 INTRODUCTION . Hypergraphs , whose edges link to the arbitrary number of vertices to model high-order relationships , attract much attention recently from researchers in the graph learning community ( Feng et al. , 2019 ; Jiang et al. , 2019 ; Zhang et al. , 2021 ) . The topology of hypergraph can be considered to be embedded in vertex weights , which represents the connection strength between hyperedges and vertices ( Chitra & Raphael , 2019 ) . The vertex weights can be divided into two categories , edgeindependent and edge-dependent ( Chitra & Raphael , 2019 ; Hayashi et al. , 2020 ) , depending on whether the vertex weights are related to the incident hyperedges or not ( EDVW-hypergraph implies that each vertex v is assigned a weight qe ( v ) ∈ R for each incident hyperedge e ) . Many hypergraph neural networks have been proposed ( Feng et al. , 2019 ; Jiang et al. , 2019 ; Yadati , 2020 ; Huang & Yang , 2021 ) to handle hypergraphs with edge-independent vertex weights ( EIVW-hypergraph ) . However , rigorous study for hypergraphs with edge-dependent vertex weights ( EDVW-hypergraph ) is still lacked , though they enjoy stronger expressive power ( Ding & Yilmaz , 2010 ; Huang et al. , 2010 ; Li et al. , 2018a ; Zhang et al. , 2018 ) . In this work , we are devoted to spectral-based methods , which are considered to be robust and to allow for simple model property analysis ( Wu et al. , 2020 ) . Unfortunately , the spectral theory of EDVW-hypergraph has not been well established . A recent work ( Chitra & Raphael , 2019 ) provides the spectral theory for EDVW-hypergraphs , and indicates that the defined EDVW-hypergraphs is equivalent to digraphs . Though digraph convolutional networks can be directly used for EDVWhypergraphs , most existing convolutional algorithms for digraphs are more or less related to undirected graphs , e.g. , ( Monti et al. , 2018 ; Li et al. , 2020 ; Ma et al. , 2019 ; Tong et al. , 2020 ) . Essentially , they transform digraphs to undigraphs with various techniques due to the challenges of directed graph Laplacian . In addition , key theoretical analysis of digraph convolutional networks properties , such as expressive power ( Xu et al. , 2019 ) or the performance of models in deep layers ( Li et al. , 2018b ) , yet remains open . An arguably more succinct way for EDVW-hypergraph learning is to use undigraph convolutional networks . Compared with digraphs , the undigraph convolutional networks have been extensively studied and there exist various effective strategies for analyzing their properties , to name a few ( Kipf & Welling , 2017 ; Chen et al. , 2020 ; Zhu & Koniusz , 2021 ) . Furthermore , one can take established techniques on undigraphs as a subroutine for hypergraph learning , which would largely ease hypergraph learning in real-world scenarios . It remains unclear , however , whether EDVW-hypergraphs can be made equivalent to undigraphs in a principled way , thereby hindering this promising prospect . In this paper , we focus on constructing the equivalency condition between hypergraphs and undigraphs . This makes it possible to utilize undigraphs as low-order encoders of hypergraphs , and thus enabling hypergraph learning directly from the equivalent undirected graphs by means of undigraph convolutional networks . Along this route , we have made the following contributions : 1 ) We define a generalized hypergraph capturing EDVWs via designing a two-step unified random walk framework . In addition , we construct the equivalency conditions to fill the gap between the generalized hypergraph and corresponding undigraph . 2 ) We present a unified Laplacian from the equivalency conditions and propose two spectral convolution models ( GHCN , SHSC ) for EDVW-hypergraphs learning . We analyze the over-smoothing issues ( Li et al. , 2018b ) based on their spectral properties deduced from equivalency conditions . 3 ) Extensive experiments across various domains ( social network analysis , visual objective classification and protein modeling ) demonstrate the effectiveness of the proposed methods ( GHCN and SHSC ) for both EIVW-hypergraph and EDVW-hypergraph learning tasks . Notably , we are first to adopt EDVW-hypergraphs for protein structures and obtain significant performance boost . Due to space limit , more discussions on related works are deferred to Appendix A . 2 THE THEORY OF EQUIVALENCY . edge-dependent otherwise ( Chitra & Raphael , 2019 ) . If qe ( u ) equals to 1 for all linked u and e , Q would reduce to the binary incident matrix H , which is widely used to represent the structure of hypergraph ( Zhou et al. , 2006 ; Carletti et al. , 2021 ) . The matrix Q can also be viewed as a weighted incident matrix . Note that a graph is a special case of a hypergraph with hyperedge degree |e| = 2 . We say a hypergraph is connected when its unweighted clique graph ( Chitra & Raphael , 2019 ) is connected and we assume hypergraphs are connected . Throughout this paper , equivalency refers to equivalency between hypergraphs and undigraphs if not otherwise specified . 2.1 UNIFIED RANDOM WALK AND GENERALIZED HYPERGRAPHS . Traditionally , hypergraph random walk is defined by a two-step manner ( Zhou et al. , 2006 ; Ducournau & Bretto , 2014 ) ( Fig . 2 ) . Then , Chitra & Raphael ( 2019 ) raise a new random walk involving the edge-dependent vertex weights into the second step and build the equivalency between EDVWhypergraphs and digraphs . However , due to the limitation of their hypergraph definition , they fail to answer whether the EDVW-hypergraphs can be equal to undigraphs . So in this part , we integrate the existing two-step random walk methods ( Chitra & Raphael , 2019 ; Carletti et al. , 2020 ; 2021 ) to obtain a comprehensive unified random walk framework with the vertex weights added into the first step , based on which we further define a generalized hypergraph . Definition 1 ( Unified Random Walk on Hypergraphs ) The unified random walk on a hypergraph H is defined in a two-step manner : Given the current vertex u , Step I : choose an arbitrary hyperedge e incident to u , with the probability p1 = w ( e ) ∑ v∈V Q2 ( v , e ) ρ ( ∑ v∈V Q2 ( v , e ) ) Q1 ( u , e ) ∑ e∈E w ( e ) ∑ v∈V Q2 ( v , e ) ρ ( ∑ v∈V Q2 ( v , e ) ) Q1 ( u , e ) ; ( 1 ) Step II : choose an arbitrary vertex v from e , with the probability p2 = Q2 ( v , e ) ∑ v∈V Q2 ( v , e ) , ( 2 ) where w ( e ) denotes the hyperedge weight ; Q1 ( u , e ) denotes the contribution of vertex u to hyperedge e in the first step while Q2 ( v , e ) represents the contribution of vertex v to hyperedge e in the second step ; and ρ ( · ) is a real-valued function , for example , a power function ( · ) σ . Suppose δ ( e ) : = ∑ v∈V Q2 ( v , e ) is the degree of hyperedge , and d ( v ) : =∑ e∈E w ( e ) δ ( e ) ρ ( δ ( e ) ) Q1 ( v , e ) is the degree of vertex v. The transition probability of our unified random walk on a hypergraph from vertex u to vertex v is : P ( u , v ) = ∑ e∈E w ( e ) Q1 ( u , e ) Q2 ( v , e ) ρ ( δ ( e ) ) d ( u ) . ( 3 ) Here , P ( u , v ) can be written in a |V| × |V| matrix form : P = D−1v Q1Wρ ( De ) Q > 2 , where Dv , De are diagonal matrices with entries Dv ( v , v ) = d ( v ) and De ( e , e ) = δ ( e ) , respectively . ρ ( De ) represents the function ρ acting on each element of De . Notably , the ρ does not always work properly . We present the failure conditions in the Appendix G.4 . More detailed intuition of our purpose to design P ( u , v ) can be found in Appendix B.2 . Under our framework , the existing hypergraph random walk ( Zhou et al. , 2006 ; Chitra & Raphael , 2019 ; Carletti et al. , 2020 ; 2021 ) can be seen as special cases with specific p1 , p2 ( see Appendix B.3 ) . The definition of our framework is lazy since it allows self-loops ( P ( v , v ) > 0 ) . The non-lazy version is provided in Appendix C. Based on the framework , we define the generalized hypergraph for constructing equivalency between EDVW-hypergraphs and undigraphs as follows . Definition 2 ( Generalized hypergraph ) A generalized hypergraph is a hypergraph associated with the unified random walk in Definition 1 , denoted as H ( V , E , W , Q1 , Q2 ) . Here , Q1 , Q2 are vertex weights matrices , each of which can be edge-independent or edge-dependent . 2.2 THE EQUIVALENCY BETWEEN HYPERGRAPHS AND UNDIGRAPHS . In this part , we provide the condition under which the generalized hypergraph is equivalent to weighted undirected clique graph . By establishing the equivalency , we can view graphs as loworder encoders of EDVW-hypergraphs , so any undigraph technique can be used as a subroutine for hypergraph learning . The clique graph of H ( V , E , W , Q1 , Q2 ) is denoted as GC , which is a unweighted graph with vertices V and edge set { ( u , v ) : u , v ∈ e , e ∈ E } . Actually , GC turns all hyperedges into cliques . We first present the definition of the equivalency in connection with Markov chains . Definition 3 ( Chitra & Raphael ( 2019 ) ) Let M1 , M2 be the Markov chains with the same finite state space , and let PM1 and PM2 be their probability transition matrices , respectively . M1 , M2 are equivalent if PM1u , v = P M2 u , v for all states u and v. The equivalency between digraph and hypergraph is straightforward and we defer to Appendix E. On the other hand , it is easy to understand that any simple undigraph can be constructed as a special generalized hypergraph , while the converse is not necessarily true . We provide a counterexample in Appendix G.4 . This result implies that undigraph convolutional networks can not trivially be extended to generalized hypergraphs . So , we give one of the implicit equations for formulating the equivalency problem to explore the explicit equivalency conditions . Lemma 1 Let H ( V , E , W , Q1 , Q2 ) denote the generalized hypergraph in Definition 2 . Let F ( Q1 , Q2 ) ( u , v ) : =∑ e∈E w ( e ) ρ ( δ ( e ) ) Q1 ( u , e ) Q2 ( v , e ) and T ( Q ) ( u ) : =∑ e∈E w ( e ) δ ( e ) ρ ( δ ( e ) ) Q ( u , e ) . When Q1 , Q2 satisfies the following equation T ( Q2 ) ( u ) T ( Q1 ) ( v ) F ( Q1 , Q2 ) ( u , v ) = T ( Q2 ) ( v ) T ( Q1 ) ( u ) F ( Q1 , Q2 ) ( v , u ) , ∀u , v ∈ V ( 4 ) there exists a weighted undirected clique graph GC such that a random walk onH is equivalent to a random walk on GC with edge weights ω ( u , v ) = T ( Q2 ) ( u ) F ( Q1 , Q2 ) ( u , v ) /T ( Q1 ) ( u ) if T ( Q1 ) ( u ) 6= 0 and 0 otherwise . The proof can be found in Appendix G.2 . Next , we derive the explicit condition under which random walks on hypergraphs are equivalent to undigraphs . Theorem 1 ( Equivalency between generalized hypergraph and weighted undigraph ) Let H ( V , E , W , Q1 , Q2 ) denote the generalized hypergraph in Definition 2 . When H satisfies any of the condition bellow : Condition ( 1 ) Q1 and Q2 are both edge-independent ; Condition ( 2 ) Q1 = kQ2 ( k ∈ R ) , there exists a weighted undirected clique graph GC such that a random walk on H is equivalent to a random walk on GC . It is easy to verify that conditions ( 1 ) and ( 2 ) satisfy equation 4 ( proof in Appendix G.3 ) . This theorem brings insights from three aspects : i ) Hypergraph foundation . The equivalency itself is a fundamental problem and remains open before this work ( Agarwal et al. , 2006 ; Chitra & Raphael , 2019 ) . Our conclusion provides more adaptive and explores more essential conditions thanks to the unity of our random walk and the generalization of hypergraph defined ( Figure 3 ) . Notably , Chitra & Raphael ( 2019 ) shows that the equivalency condition is : Q1 = H and Q2 is edge-independent , which can be viewed as a special case of the condition ( 1 ) , while Theorem 1 implies the equivalency is not only related to the dependent relationships between vertex weights and edges , but also to whether the vertex weights used in the first and second step of the unified random walk are proportional . Furthermore , thanks to the introduced Q1 in the generalized hypergraph , we can obtain the equivalency between EDVW-hypergraph and undigraph ( i.e . condition ( 2 ) ) , filling an important gap in the equivalency theory of EDVW-hypergraph . ii ) Providing a theoretical basis for hypergraph applications . The condition ( 2 ) , containing both the edge-independent vertex weights and edge-dependent vertex weights cases which match different hypergraph applications ( Ding & Yilmaz , 2010 ; Zeng et al. , 2016 ; Zhang et al. , 2018 ) , provides those with theoretical explanations . The condition ( 2 ) also reveals that the existing random walks ( Carletti et al. , 2020 ; 2021 ) on hypergraphs are equivalent to undigraph . iii ) Hypergraph learning . The equivalent undigraphs can be viewed as the lower-order encoders of hypergraphs . Thus , one can obtain hypergraph representations directly by exploiting the undigraph learning methods . Especially , the condition ( 2 ) , which implies that an EDVW-hypergraph can be equivalent to an undigraph , gives the theoretical guarantee for learning hypergraphs without losing edge-dependent vertex weights via undigraph neural networks . Next , we provide the spectral theory for the generalized hypergraphs .

Summary: Real-world relational datasets (e.g., academic data, protein data) contain group relationships and can be modelled via hypergraphs.  One way to capture group higher-order relationships is to define a hypergraph with edge-dependent vertex weights (EDVWs) and such a hypergraph has been shown to be equivalent to directed graphs [Chitra et al., ICML'19]. The main contributions of this paper are: 1) Proposal of a generalised hypergraph capturing EDVWs with equivalence to undirected graphs, 2) Proposal of a Laplacian based on the generalised EDVW and exploration of existing spectral convolutional methods, and 3) Empirical evaluation on academic, protein, and visual object datasets.