Local Hyper-Flow Diffusion

1 Introduction . Hypergraphs [ 1 ] generalize graphs by allowing a hyperedge to consist of multiple nodes that capture higher-order relationships in complex systems and datasets [ 2 ] . Hypergraphs have been used for music recommendation on Last.fm data [ 3 ] , news recommendation [ 4 ] , sets of product reviews on Amazon [ 5 ] , and sets of co-purchased products at Walmart [ 6 ] . Beyond the internet , hypergraphs are used for analyzing higher-order structure in neuronal , air-traffic and food networks [ 7 , 8 ] . In order to explore and understand higher-order relationships in hypergraphs , recent work has made use of cut-cost functions that are defined by associating each hyperedge with a specific set function . These functions assign specific penalties of separating the nodes within individual hyperedges . They generalize the notion of hypergraph cuts and are crucial for determining small-scale community structure [ 8 , 9 ] . The most popular cut-cost functions with increasing capability to model complex multiway relationships are the unit cut-cost [ 10–12 ] , cardinality-based cut-cost [ 13 , 14 ] and general submodular cut-cost [ 15 , 16 ] . An illustration of a hyperedge and the associated cut-cost function is given in Figure 1 . In the simplest setting , all cut-cost functions take value either 0 or 1 ( e.g. , the case when γ1 = γ2 = 1 in Figure 1b ) , we obtain a unit cut-cost hypergraph . In a slightly more general setting , the cut-costs are determined solely by the number of nodes in either side of the hyperedge cut ( e.g. , the case when γ1 = 1/2 and γ2 = 1 in Figure 1b ) , we obtain a cardinality-based hypergraph . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . We refer to hypergraphs associated with arbitrary submodular cut-cost functions ( e.g. , the case when γ1 = 1/2 and 0 ≤ γ2 ≤ 1 in Figure 1b ) as general submodular hypergraphs . Hypergraphs that arise from data science applications consist of interesting small-scale local structure such as local communities [ 9 , 17 ] . Exploiting this structure is central to the above mentioned applications on hypergraphs and related applications in machine learning and applied mathematics [ 18 ] . We consider local hypergraph clustering as the task of finding a community-like cluster around a given set of seed nodes , where nodes in the cluster are densely connected to each other while relatively isolated to the rest of the graph . One of the most powerful primitives for the local hypergraph clustering task is the graph diffusion . Diffusion on a graph is the process of spreading a given initial mass from some seed node ( s ) to neighbor nodes using the edges of the graph . Graph diffusions have been successfully employed in the industry , for example , both Pinterest and Twitter use diffusion methods for their recommendation systems [ 19–21 ] . Google uses diffusion methods to perform clustering query refinements [ 22 ] . Let us not forget PageRank [ 23 , 24 ] , Google ’ s model for their search engine . Empirical and theoretical performance of local diffusion methods is often measured on the problem of local hypergraph clustering [ 9 , 25 , 26 ] . Existing local diffusion methods only directly apply to hypergraphs with the unit cut-cost [ 17,27 ] . For the slightly more general cardinality-based cut-cost , they rely on graph reduction techniques which result in a rather pessimistic edge-size-dependent approximation error [ 9 , 13 , 27 , 28 ] . Moreover , none of the existing methods is capable of processing general submodular cut-costs . In this work , we are interested in designing a diffusion framework that ( i ) achieves stronger theoretical guarantees for the problem of local hypergraph clustering , ( ii ) is flexible enough to work with general submodular hypergraphs , and ( iii ) permits computationally efficient algorithms . We propose the first local diffusion method that simultaneously accomplishes these goals . In what follows we describe our main contributions and previous work . In Section 2 we provide preliminaries and notations . In Section 3 we introduce our diffusion model from a combinatorial flow perspective . In Section 4 we discuss the local hypergraph clustering problem and Cheeger-type quadratic ap- proximation error . In Section 6 we perform experiments using both synthetic and real datasets . 1.1 Our main contributions . In this work we propose a generic local diffusion model that applies to hypergraphs characterized by a rich class of cut-cost functions that covers many previously studied special cases , e.g. , unit , cardinality-based and submodular cut-costs . We provide the first edge-size-independent Cheeger-type approximation error for the problem of local hypergraph clustering using any of these cut-costs . In particular , assume that there exists a cluster C with conductance Φ ( C ) , and assume that we are given a set of seed nodes that reasonably overlaps with C , then the proposed diffusion model can be used to find a cluster Ĉ with conductance at most O ( √ Φ ( C ) ) ( in the appendix we show that an ` p-norm version of the proposed model can achieve O ( Φ ( C ) ) asymptotically ) . Our hypergraph diffusion model is formulated as a convex optimization problem . It has a natural combinatorial flow interpretation that generalizes the notion of network flows over hyperedges . We show that the optimization problem can be solved efficiently by an alternating minimization method . In addition , we prove that the number of nonzero nodes in the optimal solution is independent of the size of the hypergraph , and it only depends on the size of the initial mass . This key property ensures that our algorithm scales well in practice for large datasets . We evaluate our method using both synthetic and real-world data . We show that our method improves accuracy significantly for hypergraphs with unit , cardinarlity-based and general submodular cut-costs for local clustering . 1.2 Previous work . Recently , clustering methods on hypergraphs received renewed interest . Different methods require different assumptions about the hyperedge cut-cost , which can be roughly categorized into unit cut-cost , cardinality-based ( and submodular ) cut-cost and general submodular cut-cost . Moreover , existing methods can be either global , where the output is not localized around a given set of seed nodes , or local , where the output is a tiny cluster around a set of seed nodes . Local algorithms are the only scalable ones for large hypergraphs , which is our main focus . Many works propose global methods and thus they are not scalable to large hypergraphs [ 7,8,15–17,29–38 ] . Local diffusion-based methods are more relevant to our work [ 9 , 13,27 ] . In particular , iterative hypergraph min-cut methods for the local hypergraph clustering problem can be adopted [ 13 ] . However , these methods require in theory and in practice a large seed set , i.e. , they are not expansive and thus can not work with one seed node . The expansive combinatorial diffusion [ 25 ] is generalized for hypergraphs [ 27 ] , which can detect a target cluster using only one seed node . However , combinatorial methods have a large bias towards low conductance clusters as opposed to finding the target cluster [ 39 ] . The most relevant paper to our work is [ 9 ] . However , the proposed methods in [ 9 ] depend on a reduction from hypergraphs to directed graphs . This results in an approximation error for clustering that is proportional to the size of hyperedges and induces performance degeneration when the hyperedges are large . In fact , none of the above approaches ( including global and local ones ) has an edge-size-independent approximation error bound for even simple cardinality-based hypergraphs . Moreover , existing local approaches do not work for general submodular hypergraphs . 2 Preliminaries and Notations . Submodular function . Given a set S , we denote 2S the power set of S and |S| the cardinality of S. A submodular function F : 2S → R is a set function such that F ( A ) +F ( B ) ≥ F ( A∪B ) +F ( A∩B ) for any A , B ⊆ S. Submodular hypergaph . A hypergraph H = ( V , E ) is defined by a set of nodes V and a set of hyperedgesE ⊆ 2V , i.e. , each hyperedge e ∈ E is a subset of V . A hypergraph is termed submodular if every e ∈ E is associated with a submodular function we : 2e → R≥0 [ 15 ] . The weight we ( S ) indicates the cut-cost of splitting the hyperedge e into two subsets , S and e \ S. This general form allows us to describe the potentially complex higher-order relation among multiple nodes ( Figure 1 ) . A proper hyperedge weight we should satisfy that we ( ∅ ) = we ( e ) = 0 . To ease notation we extend the domain of we to 2V by setting we ( S ) : = we ( S ∩ e ) for any S ⊆ V . We assume without loss of generality that we is normalized by ϑe : = maxS⊆e we ( S ) , so that we ( S ) ∈ [ 0 , 1 ] for any S ⊆ V . For the sake of simplicity in presentation , we assume that ϑe = 1 for all e.1 A submodular hypergraph is written as H = ( V , E , W ) whereW : = { we , ϑe } e∈E . Note that when we ( S ) = 1 for any S ∈ 2e\ { ∅ , e } , the definition reduces to unit cut-cost hypergraphs . When we ( S ) only depends on |S| , it reduces to cardinality-based cut-cost hypergraphs . Vector/Function on V or E. For a set of nodes S ⊆ V , we denote 1S the indicator vector of S , i.e. , [ 1S ] v = 1 if v ∈ S and 0 otherwise . For a vector x ∈ R|V | , we write x ( S ) : = ∑ v∈S xv , where xv in the entry in x that corresponds to v ∈ V . We define the support of x as supp ( x ) : = { v ∈ V |xv 6= 0 } . The support of a vector in R|E| is defined analogously . We refer to a function over nodes x : V → R and its explicit representation as a |V |-dimensional vector interchangeably . Volume , cut , conductance . Given a submodular hypergraph H = ( V , E , W ) , the degree of a node v is defined as dv : = | { e ∈ E : v ∈ e } | . We reserve d for the vector of node degrees and D = diag ( d ) . We refer to vol ( S ) : = d ( S ) as the volume of S ⊆ V . A cut is treated as a proper subset C ⊂ V , or a partition ( C , C̄ ) where C̄ : = V \ C. The cut-set of C is defined as ∂C : = { e ∈ E|e ∩ C 6= ∅ , e∩ C̄ 6= ∅ } ; the cut-size of C is defined as vol ( ∂C ) : = ∑ e∈∂C ϑewe ( C ) = ∑ e∈E ϑewe ( C ) . The conductance of a cut C in H is Φ ( C ) : = vol ( ∂C ) min { vol ( C ) , vol ( V \C ) } . Flow . A flow routing over a hyperedge e is a function re : e→ R where re ( v ) specifies the amount of mass that flows from { v } to e \ { v } over e. To ease notation we extend the domain of re to V by identifying re ( v ) = 0 for v 6∈ e , so re is treated as a function re : V → R or equivalently a |V |-dimensional vector . The net ( out ) flow at a node v is given by ∑ e∈E re ( v ) . Given a routing 1This is without loss of generality . In the appendix we show that our method works with arbitrary ϑe > 0. function re and a set of nodes S ⊆ V , a directional routing on e with direction S → e \ S is represented by re ( S ) , which specifies the net amount of mass that flows from S to e \ S. A routing re is called proper if it obeys flow conservation , i.e. , rTe 1e = 0 . Our flow definition generalizes the notion of network flows to hypergraphs . We provide concrete illustrations in Figure 2 .

Summary: The paper introduces a novel local hypergraph partitioning framework. This utilizes a new notion of diffusion defined through an optimization problem (Section 3), and allows to partition hypergraphs using several different types of cut-cost functions on the hyperedges. Of particular interest is the case of general submodular cut-cost functions, which did not appear in prior work.  A numerical algorithm to solve the optimization problem and partition the hypergraph is provided (Section 5). The algorithm is not proved to be local, however experiments are given, illustrating the performance of the algorithm and how it compares to existing approaches to the same problem (Section 6). These show that the algorithm recovers high quality solutions on both synthetic and real-world data, and does so in a seemingly local fashion, which allows for amenable runtimes. The author(s) also motivate(s) their interest in general submodular cut-cost functions by giving concrete experimental evidence that, in specific situations (with real-world data), these lead to better quality outputs.  The author(s) also give(s) theoretical guarantees on the quality of the solution, based on clearly stated assumptions (Section 4).