Hypergraph Convolutional Networks via Equivalency  between  Hypergraphs and Undirected Graphs

As a powerful tool for modeling the complex relationships , hypergraphs are gaining popularity from the graph learning community . However , commonly used algorithms in deep hypergraph learning were not specifically designed for hypergraphs with edge-dependent vertex weights ( EDVWs ) . To fill this gap , we build the equivalency condition between EDVW-hypergraphs and undirected simple graphs , which enables utilizing existing undirected graph neural networks as subroutines to learn high-order interactions induced by EDVWs of hypergraphs . Specifically , we define a generalized hypergraph with vertex weights by proposing a unified random walk framework , under which we present the equivalency condition between generalized hypergraphs and undigraphs . Guided by the equivalency results , we propose a Generalized Hypergraph Convolutional Network ( GHCN ) architecture for deep hypergraph learning . Furthermore , to improve the longrange interactions and alleviate the over-smoothing issue , we further propose the Simple Hypergraph Spectral Convolution ( SHSC ) model by constructing the Discounted Markov Diffusion Kernel from our random walk framework . Extensive experiments from various domains including social network analysis , visual objective classification , and protein fold classification demonstrate that the proposed approaches outperform state-of-the-art spectral methods with a large margin . 1 INTRODUCTION . Hypergraphs , whose edges link to the arbitrary number of vertices to model high-order relationships , attract much attention recently from researchers in the graph learning community ( Feng et al. , 2019 ; Jiang et al. , 2019 ; Zhang et al. , 2021 ) . The topology of hypergraph can be considered to be embedded in vertex weights , which represents the connection strength between hyperedges and vertices ( Chitra & Raphael , 2019 ) . The vertex weights can be divided into two categories , edgeindependent and edge-dependent ( Chitra & Raphael , 2019 ; Hayashi et al. , 2020 ) , depending on whether the vertex weights are related to the incident hyperedges or not ( EDVW-hypergraph implies that each vertex v is assigned a weight qe ( v ) ∈ R for each incident hyperedge e ) . Many hypergraph neural networks have been proposed ( Feng et al. , 2019 ; Jiang et al. , 2019 ; Yadati , 2020 ; Huang & Yang , 2021 ) to handle hypergraphs with edge-independent vertex weights ( EIVW-hypergraph ) . However , rigorous study for hypergraphs with edge-dependent vertex weights ( EDVW-hypergraph ) is still lacked , though they enjoy stronger expressive power ( Ding & Yilmaz , 2010 ; Huang et al. , 2010 ; Li et al. , 2018a ; Zhang et al. , 2018 ) . In this work , we are devoted to spectral-based methods , which are considered to be robust and to allow for simple model property analysis ( Wu et al. , 2020 ) . Unfortunately , the spectral theory of EDVW-hypergraph has not been well established . A recent work ( Chitra & Raphael , 2019 ) provides the spectral theory for EDVW-hypergraphs , and indicates that the defined EDVW-hypergraphs is equivalent to digraphs . Though digraph convolutional networks can be directly used for EDVWhypergraphs , most existing convolutional algorithms for digraphs are more or less related to undirected graphs , e.g. , ( Monti et al. , 2018 ; Li et al. , 2020 ; Ma et al. , 2019 ; Tong et al. , 2020 ) . Essentially , they transform digraphs to undigraphs with various techniques due to the challenges of directed graph Laplacian . In addition , key theoretical analysis of digraph convolutional networks properties , such as expressive power ( Xu et al. , 2019 ) or the performance of models in deep layers ( Li et al. , 2018b ) , yet remains open . An arguably more succinct way for EDVW-hypergraph learning is to use undigraph convolutional networks . Compared with digraphs , the undigraph convolutional networks have been extensively studied and there exist various effective strategies for analyzing their properties , to name a few ( Kipf & Welling , 2017 ; Chen et al. , 2020 ; Zhu & Koniusz , 2021 ) . Furthermore , one can take established techniques on undigraphs as a subroutine for hypergraph learning , which would largely ease hypergraph learning in real-world scenarios . It remains unclear , however , whether EDVW-hypergraphs can be made equivalent to undigraphs in a principled way , thereby hindering this promising prospect . In this paper , we focus on constructing the equivalency condition between hypergraphs and undigraphs . This makes it possible to utilize undigraphs as low-order encoders of hypergraphs , and thus enabling hypergraph learning directly from the equivalent undirected graphs by means of undigraph convolutional networks . Along this route , we have made the following contributions : 1 ) We define a generalized hypergraph capturing EDVWs via designing a two-step unified random walk framework . In addition , we construct the equivalency conditions to fill the gap between the generalized hypergraph and corresponding undigraph . 2 ) We present a unified Laplacian from the equivalency conditions and propose two spectral convolution models ( GHCN , SHSC ) for EDVW-hypergraphs learning . We analyze the over-smoothing issues ( Li et al. , 2018b ) based on their spectral properties deduced from equivalency conditions . 3 ) Extensive experiments across various domains ( social network analysis , visual objective classification and protein modeling ) demonstrate the effectiveness of the proposed methods ( GHCN and SHSC ) for both EIVW-hypergraph and EDVW-hypergraph learning tasks . Notably , we are first to adopt EDVW-hypergraphs for protein structures and obtain significant performance boost . Due to space limit , more discussions on related works are deferred to Appendix A . 2 THE THEORY OF EQUIVALENCY . edge-dependent otherwise ( Chitra & Raphael , 2019 ) . If qe ( u ) equals to 1 for all linked u and e , Q would reduce to the binary incident matrix H , which is widely used to represent the structure of hypergraph ( Zhou et al. , 2006 ; Carletti et al. , 2021 ) . The matrix Q can also be viewed as a weighted incident matrix . Note that a graph is a special case of a hypergraph with hyperedge degree |e| = 2 . We say a hypergraph is connected when its unweighted clique graph ( Chitra & Raphael , 2019 ) is connected and we assume hypergraphs are connected . Throughout this paper , equivalency refers to equivalency between hypergraphs and undigraphs if not otherwise specified . 2.1 UNIFIED RANDOM WALK AND GENERALIZED HYPERGRAPHS . Traditionally , hypergraph random walk is defined by a two-step manner ( Zhou et al. , 2006 ; Ducournau & Bretto , 2014 ) ( Fig . 2 ) . Then , Chitra & Raphael ( 2019 ) raise a new random walk involving the edge-dependent vertex weights into the second step and build the equivalency between EDVWhypergraphs and digraphs . However , due to the limitation of their hypergraph definition , they fail to answer whether the EDVW-hypergraphs can be equal to undigraphs . So in this part , we integrate the existing two-step random walk methods ( Chitra & Raphael , 2019 ; Carletti et al. , 2020 ; 2021 ) to obtain a comprehensive unified random walk framework with the vertex weights added into the first step , based on which we further define a generalized hypergraph . Definition 1 ( Unified Random Walk on Hypergraphs ) The unified random walk on a hypergraph H is defined in a two-step manner : Given the current vertex u , Step I : choose an arbitrary hyperedge e incident to u , with the probability p1 = w ( e ) ∑ v∈V Q2 ( v , e ) ρ ( ∑ v∈V Q2 ( v , e ) ) Q1 ( u , e ) ∑ e∈E w ( e ) ∑ v∈V Q2 ( v , e ) ρ ( ∑ v∈V Q2 ( v , e ) ) Q1 ( u , e ) ; ( 1 ) Step II : choose an arbitrary vertex v from e , with the probability p2 = Q2 ( v , e ) ∑ v∈V Q2 ( v , e ) , ( 2 ) where w ( e ) denotes the hyperedge weight ; Q1 ( u , e ) denotes the contribution of vertex u to hyperedge e in the first step while Q2 ( v , e ) represents the contribution of vertex v to hyperedge e in the second step ; and ρ ( · ) is a real-valued function , for example , a power function ( · ) σ . Suppose δ ( e ) : = ∑ v∈V Q2 ( v , e ) is the degree of hyperedge , and d ( v ) : =∑ e∈E w ( e ) δ ( e ) ρ ( δ ( e ) ) Q1 ( v , e ) is the degree of vertex v. The transition probability of our unified random walk on a hypergraph from vertex u to vertex v is : P ( u , v ) = ∑ e∈E w ( e ) Q1 ( u , e ) Q2 ( v , e ) ρ ( δ ( e ) ) d ( u ) . ( 3 ) Here , P ( u , v ) can be written in a |V| × |V| matrix form : P = D−1v Q1Wρ ( De ) Q > 2 , where Dv , De are diagonal matrices with entries Dv ( v , v ) = d ( v ) and De ( e , e ) = δ ( e ) , respectively . ρ ( De ) represents the function ρ acting on each element of De . Notably , the ρ does not always work properly . We present the failure conditions in the Appendix G.4 . More detailed intuition of our purpose to design P ( u , v ) can be found in Appendix B.2 . Under our framework , the existing hypergraph random walk ( Zhou et al. , 2006 ; Chitra & Raphael , 2019 ; Carletti et al. , 2020 ; 2021 ) can be seen as special cases with specific p1 , p2 ( see Appendix B.3 ) . The definition of our framework is lazy since it allows self-loops ( P ( v , v ) > 0 ) . The non-lazy version is provided in Appendix C. Based on the framework , we define the generalized hypergraph for constructing equivalency between EDVW-hypergraphs and undigraphs as follows . Definition 2 ( Generalized hypergraph ) A generalized hypergraph is a hypergraph associated with the unified random walk in Definition 1 , denoted as H ( V , E , W , Q1 , Q2 ) . Here , Q1 , Q2 are vertex weights matrices , each of which can be edge-independent or edge-dependent . 2.2 THE EQUIVALENCY BETWEEN HYPERGRAPHS AND UNDIGRAPHS . In this part , we provide the condition under which the generalized hypergraph is equivalent to weighted undirected clique graph . By establishing the equivalency , we can view graphs as loworder encoders of EDVW-hypergraphs , so any undigraph technique can be used as a subroutine for hypergraph learning . The clique graph of H ( V , E , W , Q1 , Q2 ) is denoted as GC , which is a unweighted graph with vertices V and edge set { ( u , v ) : u , v ∈ e , e ∈ E } . Actually , GC turns all hyperedges into cliques . We first present the definition of the equivalency in connection with Markov chains . Definition 3 ( Chitra & Raphael ( 2019 ) ) Let M1 , M2 be the Markov chains with the same finite state space , and let PM1 and PM2 be their probability transition matrices , respectively . M1 , M2 are equivalent if PM1u , v = P M2 u , v for all states u and v. The equivalency between digraph and hypergraph is straightforward and we defer to Appendix E. On the other hand , it is easy to understand that any simple undigraph can be constructed as a special generalized hypergraph , while the converse is not necessarily true . We provide a counterexample in Appendix G.4 . This result implies that undigraph convolutional networks can not trivially be extended to generalized hypergraphs . So , we give one of the implicit equations for formulating the equivalency problem to explore the explicit equivalency conditions . Lemma 1 Let H ( V , E , W , Q1 , Q2 ) denote the generalized hypergraph in Definition 2 . Let F ( Q1 , Q2 ) ( u , v ) : =∑ e∈E w ( e ) ρ ( δ ( e ) ) Q1 ( u , e ) Q2 ( v , e ) and T ( Q ) ( u ) : =∑ e∈E w ( e ) δ ( e ) ρ ( δ ( e ) ) Q ( u , e ) . When Q1 , Q2 satisfies the following equation T ( Q2 ) ( u ) T ( Q1 ) ( v ) F ( Q1 , Q2 ) ( u , v ) = T ( Q2 ) ( v ) T ( Q1 ) ( u ) F ( Q1 , Q2 ) ( v , u ) , ∀u , v ∈ V ( 4 ) there exists a weighted undirected clique graph GC such that a random walk onH is equivalent to a random walk on GC with edge weights ω ( u , v ) = T ( Q2 ) ( u ) F ( Q1 , Q2 ) ( u , v ) /T ( Q1 ) ( u ) if T ( Q1 ) ( u ) 6= 0 and 0 otherwise . The proof can be found in Appendix G.2 . Next , we derive the explicit condition under which random walks on hypergraphs are equivalent to undigraphs . Theorem 1 ( Equivalency between generalized hypergraph and weighted undigraph ) Let H ( V , E , W , Q1 , Q2 ) denote the generalized hypergraph in Definition 2 . When H satisfies any of the condition bellow : Condition ( 1 ) Q1 and Q2 are both edge-independent ; Condition ( 2 ) Q1 = kQ2 ( k ∈ R ) , there exists a weighted undirected clique graph GC such that a random walk on H is equivalent to a random walk on GC . It is easy to verify that conditions ( 1 ) and ( 2 ) satisfy equation 4 ( proof in Appendix G.3 ) . This theorem brings insights from three aspects : i ) Hypergraph foundation . The equivalency itself is a fundamental problem and remains open before this work ( Agarwal et al. , 2006 ; Chitra & Raphael , 2019 ) . Our conclusion provides more adaptive and explores more essential conditions thanks to the unity of our random walk and the generalization of hypergraph defined ( Figure 3 ) . Notably , Chitra & Raphael ( 2019 ) shows that the equivalency condition is : Q1 = H and Q2 is edge-independent , which can be viewed as a special case of the condition ( 1 ) , while Theorem 1 implies the equivalency is not only related to the dependent relationships between vertex weights and edges , but also to whether the vertex weights used in the first and second step of the unified random walk are proportional . Furthermore , thanks to the introduced Q1 in the generalized hypergraph , we can obtain the equivalency between EDVW-hypergraph and undigraph ( i.e . condition ( 2 ) ) , filling an important gap in the equivalency theory of EDVW-hypergraph . ii ) Providing a theoretical basis for hypergraph applications . The condition ( 2 ) , containing both the edge-independent vertex weights and edge-dependent vertex weights cases which match different hypergraph applications ( Ding & Yilmaz , 2010 ; Zeng et al. , 2016 ; Zhang et al. , 2018 ) , provides those with theoretical explanations . The condition ( 2 ) also reveals that the existing random walks ( Carletti et al. , 2020 ; 2021 ) on hypergraphs are equivalent to undigraph . iii ) Hypergraph learning . The equivalent undigraphs can be viewed as the lower-order encoders of hypergraphs . Thus , one can obtain hypergraph representations directly by exploiting the undigraph learning methods . Especially , the condition ( 2 ) , which implies that an EDVW-hypergraph can be equivalent to an undigraph , gives the theoretical guarantee for learning hypergraphs without losing edge-dependent vertex weights via undigraph neural networks . Next , we provide the spectral theory for the generalized hypergraphs .

Summary: This paper proves under what conditions the equivalency between a generalized hypergraph (a hypergraph with node weights $Q_1$ and $Q_2$ for two-step random walk) and an undirected graph holds. The theory applies to ordinary hypergraphs as a special case with $Q_1=Q_2=H$. Then, the authors derive the stationary random walk distribution, the hypergraph Laplacian, as well as the generalized spectral hypergraph convolution form leveraging the Laplacian. To alleviate oversmoothing, they further propose to use a diffusion kernel to build a simple hypergraph spectral convolution. Experiments on four tasks show better performance than state-of-the-art methods.