Local Hyper-Flow Diffusion

1 Introduction . Hypergraphs [ 1 ] generalize graphs by allowing a hyperedge to consist of multiple nodes that capture higher-order relationships in complex systems and datasets [ 2 ] . Hypergraphs have been used for music recommendation on Last.fm data [ 3 ] , news recommendation [ 4 ] , sets of product reviews on Amazon [ 5 ] , and sets of co-purchased products at Walmart [ 6 ] . Beyond the internet , hypergraphs are used for analyzing higher-order structure in neuronal , air-traffic and food networks [ 7 , 8 ] . In order to explore and understand higher-order relationships in hypergraphs , recent work has made use of cut-cost functions that are defined by associating each hyperedge with a specific set function . These functions assign specific penalties of separating the nodes within individual hyperedges . They generalize the notion of hypergraph cuts and are crucial for determining small-scale community structure [ 8 , 9 ] . The most popular cut-cost functions with increasing capability to model complex multiway relationships are the unit cut-cost [ 10–12 ] , cardinality-based cut-cost [ 13 , 14 ] and general submodular cut-cost [ 15 , 16 ] . An illustration of a hyperedge and the associated cut-cost function is given in Figure 1 . In the simplest setting , all cut-cost functions take value either 0 or 1 ( e.g. , the case when γ1 = γ2 = 1 in Figure 1b ) , we obtain a unit cut-cost hypergraph . In a slightly more general setting , the cut-costs are determined solely by the number of nodes in either side of the hyperedge cut ( e.g. , the case when γ1 = 1/2 and γ2 = 1 in Figure 1b ) , we obtain a cardinality-based hypergraph . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . We refer to hypergraphs associated with arbitrary submodular cut-cost functions ( e.g. , the case when γ1 = 1/2 and 0 ≤ γ2 ≤ 1 in Figure 1b ) as general submodular hypergraphs . Hypergraphs that arise from data science applications consist of interesting small-scale local structure such as local communities [ 9 , 17 ] . Exploiting this structure is central to the above mentioned applications on hypergraphs and related applications in machine learning and applied mathematics [ 18 ] . We consider local hypergraph clustering as the task of finding a community-like cluster around a given set of seed nodes , where nodes in the cluster are densely connected to each other while relatively isolated to the rest of the graph . One of the most powerful primitives for the local hypergraph clustering task is the graph diffusion . Diffusion on a graph is the process of spreading a given initial mass from some seed node ( s ) to neighbor nodes using the edges of the graph . Graph diffusions have been successfully employed in the industry , for example , both Pinterest and Twitter use diffusion methods for their recommendation systems [ 19–21 ] . Google uses diffusion methods to perform clustering query refinements [ 22 ] . Let us not forget PageRank [ 23 , 24 ] , Google ’ s model for their search engine . Empirical and theoretical performance of local diffusion methods is often measured on the problem of local hypergraph clustering [ 9 , 25 , 26 ] . Existing local diffusion methods only directly apply to hypergraphs with the unit cut-cost [ 17,27 ] . For the slightly more general cardinality-based cut-cost , they rely on graph reduction techniques which result in a rather pessimistic edge-size-dependent approximation error [ 9 , 13 , 27 , 28 ] . Moreover , none of the existing methods is capable of processing general submodular cut-costs . In this work , we are interested in designing a diffusion framework that ( i ) achieves stronger theoretical guarantees for the problem of local hypergraph clustering , ( ii ) is flexible enough to work with general submodular hypergraphs , and ( iii ) permits computationally efficient algorithms . We propose the first local diffusion method that simultaneously accomplishes these goals . In what follows we describe our main contributions and previous work . In Section 2 we provide preliminaries and notations . In Section 3 we introduce our diffusion model from a combinatorial flow perspective . In Section 4 we discuss the local hypergraph clustering problem and Cheeger-type quadratic ap- proximation error . In Section 6 we perform experiments using both synthetic and real datasets . 1.1 Our main contributions . In this work we propose a generic local diffusion model that applies to hypergraphs characterized by a rich class of cut-cost functions that covers many previously studied special cases , e.g. , unit , cardinality-based and submodular cut-costs . We provide the first edge-size-independent Cheeger-type approximation error for the problem of local hypergraph clustering using any of these cut-costs . In particular , assume that there exists a cluster C with conductance Φ ( C ) , and assume that we are given a set of seed nodes that reasonably overlaps with C , then the proposed diffusion model can be used to find a cluster Ĉ with conductance at most O ( √ Φ ( C ) ) ( in the appendix we show that an ` p-norm version of the proposed model can achieve O ( Φ ( C ) ) asymptotically ) . Our hypergraph diffusion model is formulated as a convex optimization problem . It has a natural combinatorial flow interpretation that generalizes the notion of network flows over hyperedges . We show that the optimization problem can be solved efficiently by an alternating minimization method . In addition , we prove that the number of nonzero nodes in the optimal solution is independent of the size of the hypergraph , and it only depends on the size of the initial mass . This key property ensures that our algorithm scales well in practice for large datasets . We evaluate our method using both synthetic and real-world data . We show that our method improves accuracy significantly for hypergraphs with unit , cardinarlity-based and general submodular cut-costs for local clustering . 1.2 Previous work . Recently , clustering methods on hypergraphs received renewed interest . Different methods require different assumptions about the hyperedge cut-cost , which can be roughly categorized into unit cut-cost , cardinality-based ( and submodular ) cut-cost and general submodular cut-cost . Moreover , existing methods can be either global , where the output is not localized around a given set of seed nodes , or local , where the output is a tiny cluster around a set of seed nodes . Local algorithms are the only scalable ones for large hypergraphs , which is our main focus . Many works propose global methods and thus they are not scalable to large hypergraphs [ 7,8,15–17,29–38 ] . Local diffusion-based methods are more relevant to our work [ 9 , 13,27 ] . In particular , iterative hypergraph min-cut methods for the local hypergraph clustering problem can be adopted [ 13 ] . However , these methods require in theory and in practice a large seed set , i.e. , they are not expansive and thus can not work with one seed node . The expansive combinatorial diffusion [ 25 ] is generalized for hypergraphs [ 27 ] , which can detect a target cluster using only one seed node . However , combinatorial methods have a large bias towards low conductance clusters as opposed to finding the target cluster [ 39 ] . The most relevant paper to our work is [ 9 ] . However , the proposed methods in [ 9 ] depend on a reduction from hypergraphs to directed graphs . This results in an approximation error for clustering that is proportional to the size of hyperedges and induces performance degeneration when the hyperedges are large . In fact , none of the above approaches ( including global and local ones ) has an edge-size-independent approximation error bound for even simple cardinality-based hypergraphs . Moreover , existing local approaches do not work for general submodular hypergraphs . 2 Preliminaries and Notations . Submodular function . Given a set S , we denote 2S the power set of S and |S| the cardinality of S. A submodular function F : 2S → R is a set function such that F ( A ) +F ( B ) ≥ F ( A∪B ) +F ( A∩B ) for any A , B ⊆ S. Submodular hypergaph . A hypergraph H = ( V , E ) is defined by a set of nodes V and a set of hyperedgesE ⊆ 2V , i.e. , each hyperedge e ∈ E is a subset of V . A hypergraph is termed submodular if every e ∈ E is associated with a submodular function we : 2e → R≥0 [ 15 ] . The weight we ( S ) indicates the cut-cost of splitting the hyperedge e into two subsets , S and e \ S. This general form allows us to describe the potentially complex higher-order relation among multiple nodes ( Figure 1 ) . A proper hyperedge weight we should satisfy that we ( ∅ ) = we ( e ) = 0 . To ease notation we extend the domain of we to 2V by setting we ( S ) : = we ( S ∩ e ) for any S ⊆ V . We assume without loss of generality that we is normalized by ϑe : = maxS⊆e we ( S ) , so that we ( S ) ∈ [ 0 , 1 ] for any S ⊆ V . For the sake of simplicity in presentation , we assume that ϑe = 1 for all e.1 A submodular hypergraph is written as H = ( V , E , W ) whereW : = { we , ϑe } e∈E . Note that when we ( S ) = 1 for any S ∈ 2e\ { ∅ , e } , the definition reduces to unit cut-cost hypergraphs . When we ( S ) only depends on |S| , it reduces to cardinality-based cut-cost hypergraphs . Vector/Function on V or E. For a set of nodes S ⊆ V , we denote 1S the indicator vector of S , i.e. , [ 1S ] v = 1 if v ∈ S and 0 otherwise . For a vector x ∈ R|V | , we write x ( S ) : = ∑ v∈S xv , where xv in the entry in x that corresponds to v ∈ V . We define the support of x as supp ( x ) : = { v ∈ V |xv 6= 0 } . The support of a vector in R|E| is defined analogously . We refer to a function over nodes x : V → R and its explicit representation as a |V |-dimensional vector interchangeably . Volume , cut , conductance . Given a submodular hypergraph H = ( V , E , W ) , the degree of a node v is defined as dv : = | { e ∈ E : v ∈ e } | . We reserve d for the vector of node degrees and D = diag ( d ) . We refer to vol ( S ) : = d ( S ) as the volume of S ⊆ V . A cut is treated as a proper subset C ⊂ V , or a partition ( C , C̄ ) where C̄ : = V \ C. The cut-set of C is defined as ∂C : = { e ∈ E|e ∩ C 6= ∅ , e∩ C̄ 6= ∅ } ; the cut-size of C is defined as vol ( ∂C ) : = ∑ e∈∂C ϑewe ( C ) = ∑ e∈E ϑewe ( C ) . The conductance of a cut C in H is Φ ( C ) : = vol ( ∂C ) min { vol ( C ) , vol ( V \C ) } . Flow . A flow routing over a hyperedge e is a function re : e→ R where re ( v ) specifies the amount of mass that flows from { v } to e \ { v } over e. To ease notation we extend the domain of re to V by identifying re ( v ) = 0 for v 6∈ e , so re is treated as a function re : V → R or equivalently a |V |-dimensional vector . The net ( out ) flow at a node v is given by ∑ e∈E re ( v ) . Given a routing 1This is without loss of generality . In the appendix we show that our method works with arbitrary ϑe > 0. function re and a set of nodes S ⊆ V , a directional routing on e with direction S → e \ S is represented by re ( S ) , which specifies the net amount of mass that flows from S to e \ S. A routing re is called proper if it obeys flow conservation , i.e. , rTe 1e = 0 . Our flow definition generalizes the notion of network flows to hypergraphs . We provide concrete illustrations in Figure 2 .

Summary: Submodular hypergraph is a generalization of hypergraph, where each hyperedge e is associated with a submodular function w_e such that w_e(0) = w_e(e) = 0. We can naturally define the notion of conductance for submodular hypergraphs by generalizing that for graphs. The main contribution of this work is a local clustering algorithm for submodular hypergraphs, where the goal is to find a vertex set with a small conductance that exists near a seed set just by looking around it.  The algorithm is based on a flow problem that transfers some amount of mass from the seed vertices to other vertices, with some regularizer that makes the problem amenable to continuous optimization. Let C be the target cluster. Then, the algorithm outputs a set of conductance O(sqrt(C)) if vol(S \cap C) / vol(C) = \Omega(1) and vol(S \cap C) / vol(S) = \Omega(1) (vol(S) is the volume of S) and some other conditions are satisfied. It is also shown that the support of the optimal solution has a small l1 norm. The authors proposed an alternating-minimization algorithm to solve the flow problem so that the number of vertices considered (= vertices with non-zero potentials) in the intermediate steps are small.  Empirical results are encouraging. The number of vertices with non-zero potentials is just a few thousands for a hypergraph with >2 million vertices. Also, the proposed algorithm outputs a set of a smaller conductance than previous methods on synthetic graphs and real-world hypergraphs when the submodular functions associated with the hyperedges are chosen properly.