Finding Bipartite Components in Hypergraphs

1 Introduction . Spectral methods study the efficient matrix representation of graphs and datasets , and apply the algebraic properties of these matrices to design efficient algorithms . Over the last three decades , spectral methods have become one of the most powerful techniques in machine learning , and have had comprehensive applications in a wide range of domains , including clustering [ 24 , 31 ] , image and video segmentation [ 26 ] , and network analysis [ 25 ] , among many others . While the success of this line of research is based on our rich understanding of Laplacian operators of graphs , there has been a sequence of very recent work studying non-linear Laplacian operators for more complex objects ( i.e. , hypergraphs ) and employing these non-linear operators to design hypergraph algorithms with better performance . 1.1 Our contribution . In this work , we study the non-linear Laplacian-type operators for hypergraphs , and employ such an operator to design a polynomial-time algorithm for finding bipartite components in hypergraphs . The main contribution of our work is as follows : First of all , we introduce and study a non-linear Laplacian-type operator JH for any hypergraph H . While we ’ ll formally define the operator JH in Section 3 , one can informally think about JH as a variant of the standard non-linear hypergraph Laplacian LH studied in [ 5 , 20 , 27 ] , and this variation is needed to study the other end of the spectrum of LH . We present a polynomial-time algorithm that finds some eigenvalue λ and its associated eigenvector of JH , and our algorithm is based on the following heat diffusion process : starting from an arbitrary vector f0 ∈ Rn that describes the initial heat distribution among the vertices , we use f0 to construct some 2-graph1 G0 , and use the diffusion process in G0 to represent the one in the original hypergraph H and update ft ; this process continues until the time at which G0 can not be used to appropriately simulate the diffusion process in H any more . At this point , we use the currently maintained ft to construct another 2-graph Gt 1Throughout the paper , we refer to non-hyper graphs as 2-graphs . Similarly , we always use LH to refer to the non-linear hypergraph Laplacian operator , and use LG as the standard 2-graph Laplacian . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . to simulate the diffusion process in H , and update ft . This process continues until the vector ft converges ; see Figure 1 for illustration . We theoretically prove that this heat diffusion process is unique , well-defined , and our maintained vector ft converges to some eigenvector of JH . While this result is quite interesting on its own and forms the basis of our second result , our analysis shows that , for certain hypergraphs H , both the operator JH and LH could have ω ( 1 ) eigenvectors . This result answers an open question in [ 5 ] , which asks whether LH could have more than 2 eigenvectors2 . Secondly , we present a polynomial-time algorithm that , given a hypergraph H = ( VH , EH , w ) as input , finds disjoint subsets L , R ⊂ VH that are highly connected with each other . The key to our algorithm is a Cheeger-type inequality for hypergraphs that relates the spectrum of JH and the bipartiteness ratio of H , an analog of βG studied in [ 28 ] for 2-graphs . Both the design and analysis of our algorithm is inspired by [ 28 ] , however our analysis is much more involved because of the non-linear operator JH and hyperedges of different ranks . Our second result alone answers an open question posed by [ 33 ] , which asks whether there is a hypergraph operator which satisfies a Cheeger-type inequality for bipartiteness . The significance of our work is further demonstrated by extensive experimental studies of our algorithms on both synthetic and real-world datasets . In particular , on the well-known Penn Treebank corpus that contains 49 , 208 sentences and over 1 million words , our purely unsupervised algorithm is able to identify a significant fraction of verbs from non-verbs in its two output clusters . Hence , we believe that our work could potentially have many applications in unsupervised learning for hypergraphs . Using the publicly available code of our implementation , we welcome the reader to explore further applications of our work in even more diverse datasets . 1.2 Related work . The spectral theory of hypergraphs using non-linear operators is introduced in [ 5 ] and generalised in [ 33 ] . The operator they describe is applied for hypergraph clustering applications in [ 20 , 27 ] . There are many approaches for finding clusters in hypergraphs by constructing a 2-graph which approximates the hypergraph and using a 2-graph clustering algorithm directly [ 7 , 19 , 35 ] . Another 2We underline that , while the operator LG of a 2-graph G has n eigenvalues , the number of eigenvalues of LH is unknown because of its non-linearity . As answering this open question isn ’ t the main point of our work , we refer the reader to Appendix C for detailed discussion . approach for hypergraph clustering is based on tensor spectral decomposition [ 15 , 18 ] . [ 21 , 23 , 36 ] consider the problem of finding densely connected clusters in 2-graphs . Heat diffusion processes are used for clustering 2-graphs in [ 10 , 17 ] . [ 14 ] studies a different , flow-based diffusion process for finding clusters in 2-graphs , and [ 13 ] generalises this to hypergraphs . We note that all of these methods solve a different problem to ours , and can not be compared directly . Our algorithm is related to the hypergraph max cut problem , and the state-of-the-art approximation algorithm is given by [ 34 ] . [ 28 ] introduces graph bipartiteness and gives an approximation algorithm for the 2-graph max cut problem . To the best of our knowledge , we are the first to generalise this notion of bipartiteness to hypergraphs . Finally , we note that there have been recent improvements in the time complexity for solving linear programs [ 11 , 29 ] although we do not take these into account in our analysis since the goal of this paper is not to obtain the fastest algorithm possible . 2 Notation . 2-graphs . Throughout the paper , we call a non-hyper graph a 2-graph [ 4 , 8 ] . We always use G = ( VG , EG , w ) to express a 2-graph , in which every edge e ∈ EG consists of two vertices in VG and we let n = |VG| . The degree of any vertex u ∈ VG is defined by dG ( u ) , ∑ v∈VG w ( u , v ) , and for any S ⊆ V the volume of S is defined by volG ( S ) , ∑ u∈S dG ( u ) . Following [ 28 ] , the bipartiteness ratio of any disjoint sets L , R ⊂ VG is defined by βG ( L , R ) , 2w ( L , L ) + 2w ( R , R ) + w ( L ∪R , L ∪R ) volG ( L ∪R ) where w ( A , B ) = ∑ ( u , v ) ∈A×B w ( u , v ) , and we further define βG , minS⊂V βG ( S , V \S ) . Notice that a low βG-value means that there is a dense cut between L andR , and there is a sparse cut between L ∪R and V \ ( L ∪R ) . In particular , βG = 0 implies that ( L , R ) forms a bipartite component of G. We use DG to denote the n× n diagonal matrix whose entries are ( DG ) uu = dG ( u ) , for all u ∈ V . Moreover , we use AG to denote the n× n adjacency matrix whose entries are ( AG ) uv = w ( u , v ) , for all u , v ∈ V . The Laplacian matrix is defined by LG , DG − AG . In addition , we define JG , DG +AG , and JG , D−1/2G JGD −1/2 G . For any real and symmetric matrix A , the eigenvalues of A are denoted by λ1 ( A ) ≤ · · · ≤ λn ( A ) , and the eigenvector associated with λi ( A ) is denoted by fi ( A ) for 1 ≤ i ≤ n. Hypergraphs . Let H = ( VH , EH , w ) be a hypergraph with n = |VH | vertices and weight function w : EH 7→ R+ . For any vertex v ∈ VH , the degree of v is defined by dH ( v ) , ∑ e∈EH w ( e ) · I [ v ∈ e ] , where I [ X ] = 1 if event X holds and I [ X ] = 0 otherwise . The rank of edge e ∈ EH is the total number of vertices in e. For any A , B ⊂ VH , the cut value between A and B is defined by w ( A , B ) , ∑ e∈EH w ( e ) · I [ e ∩A 6= ∅ ∧ e ∩B 6= ∅ ] . Sometimes , we are required to analyse the weights of edges that intersect some vertex sets and not others . To this end , we define for any A , B , C ⊆ VH that w ( A , B | C ) , ∑ e∈EH w ( e ) · I [ e ∩A 6= ∅ ∧ e ∩B 6= ∅ ∧ e ∩ C = ∅ ] , and we sometimes write w ( A | C ) , w ( A , A | C ) for simplicity . Generalising the notion of the bipartiteness ratio of a 2-graph , the bipartiteness ratio of sets L , R in a hypergraph H is defined by βH ( L , R ) , 2w ( L|L ) + 2w ( R|R ) + w ( L , L ∪R|R ) + w ( R , L ∪R|L ) vol ( L ∪R ) , and we define βH , minS⊂V βH ( S , V \ S ) . For any hypergraph H and f ∈ Rn , we define the discrepancy of an edge e ∈ EH with respect to f as ∆f ( e ) , max u∈e f ( u ) + min v∈e f ( v ) . For any non-linear operator J : Rn 7→ Rn , we say that ( λ , f ) is an eigen-pair if and only if Jf = λf and note that in general , a non-linear operator can have any number of eigenvalues and eigenvectors . It is important to remember that throughout the paper , we always use the letter H to represent a hypergraph , and G to represent a 2-graph . Clique reduction . The clique reduction of a hypergraph H is a 2-graph G such that VG = VH and for every edge e ∈ EH , G contains a clique on the vertices in e with edge weights 1/ ( re − 1 ) where re is the rank of the edge e. The clique reduction is a common tool for designing hypergraph algorithms [ 1 , 7 , 35 ] , and for this reason we use it as a baseline algorithm in this paper . We note that hypergraph algorithms based on the clique reduction often perform less well when there are edges with large rank in the hypergraph . Specifically , in Appendix C we use two r-uniform hypergraphs as examples to show that no matter how we weight the edges in the clique reduction , some cuts can not be approximated better than a factor of O ( r ) . This is one of the main reasons to develop spectral theory for hypergraphs through heat diffusion processes [ 5 , 27 , 33 ] .

Summary: This paper proposes a polynomial-time algorithm that finds densely connected bipartite components in a hypergraph. The algorithm is based on a heat diffusion process generalized from graphs to hypergraphs. Cheeger-type approximation guarantee is established. Empirical results show that the new method has superior performance for both synthetic and real hypergraphs.