On the Power of Edge Independent Graph Models

1 Introduction . Our work centers on edge independent graph models , in which each edge ( i , j ) is added to the graph independently with some probability Pij ∈ [ 0 , 1 ] . Formally , Definition 1 ( Edge Independent Graph Model ) . For any symmetric matrix P ∈ [ 0 , 1 ] n×n let G ( P ) be the distribution over undirected unweighted graphs where G ∼ G ( P ) contains edge ( i , j ) independently , with probability Pij . I.e. , p ( G ) = ∏ ( i , j ) ∈E ( G ) Pij · ∏ ( i , j ) /∈E ( G ) ( 1− Pij ) . Edge independent models encompass many classic random graph models . This includes the ErdösRényi model , where for all i ̸= j , Pij = p for some fixed p ∈ [ 0 , 1 ] [ 10 ] . It also includes the stochastic block model where Pij = p if two nodes are in the same community and Pij = q if two nodes are in different communities for some fixed p , q ∈ [ 0 , 1 ] with q < p [ 29 ] . Other examples include e.g. , the Chung-Lu configuration model [ 5 ] , stochastic Kronecker graphs [ 17 ] . Recently , significant attention has focused on graph generative models , which seek to learn a distribution over graphs that share similar properties to a given training graph , or set of graphs . Many algorithms parameterize this distribution as an edge independent model or closely related distribution . E.g. , NetGAN and the closely related CELL model both produce P ∈ [ 0 , 1 ] n×n and then sample edges independently without replacement with probabilities proportional to its entries , ensuring that at least one edge is sampled adjacent to each node [ 3 , 23 ] . Variational Graph Autoencoders ( VGAE ) , GraphVAE , Graphite , and MolGAN are also all based on edge independent models [ 16 , 28 , 8 , 13 ] . Given their popularity in both classical and modern graph generative models , it is natural to ask : 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , virtual . How suited are edge independent models to modeling real-world networks . Are they able to capture features such as power-law degree distributions , small-world properties , and high clustering coefficients ( triangle densities ) ? 1.1 Impossibility Results for Edge Independent Models . In this work we focus on the ability of edge independent models to generate graphs with high triangle , or other small subgraph densities . High triangle density ( equivalently , a high clustering coefficient ) is a well-known hallmark of real-work networks [ 31 , 25 , 9 ] and has been the focus of recent work exploring the power and limitations of edge-independent graph models [ 27 , 4 ] . It is clear that edge independent models can generate triangle dense graphs . In particular , P ∈ [ 0 , 1 ] n×n in Def . 1 can be set to the binary adjacency matrix of any undirected graph , and G ( P ) will generate that graph with probability 1 , no matter how triangle dense it is . However , this would not be a particularly interesting generative model – ideally G ( P ) should generate a wide range of graphs . To capture this intuitive notion , we define the overlap of an edge-independent model , which is closely related to the overlap stopping criterion for training used in training graph generative models [ 3 , 23 ] . Definition 2 ( Expected Overlap ) . For symmetric P ∈ [ 0 , 1 ] n×n let V ( P ) def= EG∼G ( P ) |E ( G ) | and Ov ( P ) def = EG1 , G2∼G ( P ) |E ( G1 ) ∩ E ( G2 ) | V ( P ) . That is , for any P ∈ [ 0 , 1 ] n×n , Ov ( P ) ∈ [ 0 , 1 ] is the ratio of the expected number of edges shared by two graphs drawn independently from G ( P ) to the expected number of edges in a graph drawn from G ( P ) . In one extreme , when P is a binary adjacency matrix , Ov ( P ) = 1 , and our generative model has simply memorized a single graph . In the other , if Pij = p for all i ̸= j ( i.e. , G ( P ) is Erdös-Rényi ) , Ov ( P ) = p. This is the minimum possible overlap when V ( P ) = p · ( n 2 ) . Our main result is that for any edge independent model with bounded overlap , G ∼ G ( P ) can not have too many triangles in expectation . In particular : Theorem 1 ( Main Result – Expected Triangles ) . For a graph G , let ∆ ( G ) denote the number of triangles in G. Consider symmetric P ∈ [ 0 , 1 ] n×n . EG∼G ( P ) [ ∆ ( G ) ] ≤ √ 2 3 ·Ov ( P ) 3/2 · V ( P ) 3/2 . As an example , consider the setting where the distribution generates sparse graphs , with V ( P ) = Θ ( n ) . Theorem 1 shows that whenever Ov ( P ) = o ( 1/n1/3 ) , EG∼G ( P ) ∆ ( G ) = o ( n ) – i.e . the graph is very triangle sparse with the number of triangles sublinear in the number of nodes . This verifies that an Erdös-Rényi graph can not achieve simultaneously linear number of edges ( i.e. , Ov ( P ) = O ( 1/n ) ) and super-linear number of triangles ( i.e. , Ov ( P ) = Ω ( 1/n1/3 ) ) under our proposed lens of viewing generative models . We extend Theorem 1 to give similar bounds for the density of squares and other k-cycles ( Thm . 4 ) , as well as for the global clustering coefficient ( Thm . 6 ) . In all cases we show that our bounds are tight – e.g. , in the triangle case , there is indeed an edge independent model with EG∼G ( P ) [ ∆ ( G ) ] = Θ ( Ov ( P ) 3/2 · V ( P ) 3/2 ) , matching the lower bound in Theorem 1 . 1.2 Empirical Findings . Our theoretical results help explain why , despite performing well in a variety of other metrics , edge independent graph generative models have been reported to generate graphs with many fewer triangles and squares on average than the real-world graphs that they are trained on . Rendsburg et al . [ 23 ] test a suite of these models , including their own CELL model and the related NetGAN model [ 3 ] . Of all these models , when trained on the CORA-ML graph with 2,802 triangles and 14,268 squares , none is able to generate graphs with more than 1,461 triangles and 6,880 squares on average . Similar gaps are observed for a number of other graphs . Rendsburg et al . also report that the triangle count increases as their notion of overlap ( closely related to Def . 2 ) increases . Theorem 1 demonstrates that this underestimation of triangle count , and its connection to overlap is inherent to all edge independent models , no matter how refined a method used to learn the underlying probability matrix P . While our theoretical results bound the performance of any edge independent model , there may still be variation in how specific models trade-off overlap and realistic graph generation . To better understand this trade-off , we introduce two simple models with easily tunable overlap as baselines . One is based on reproducing the degree sequence of the original graph ; the other , which is even simpler , is based on reproducing the volume . In both models , P is a weighted average of the input graph adjacency matrix and a probability matrix of minimal complexity which matches either the input degrees or the volume . In the latter case , to match just the volume , we simply use an Erdös-Rényi graph . In the former case , to match the degree sequence , we introduce our own model , the odds product model ; this model is similar to the Chung-Lu configuration model [ 5 ] , but , unlike Chung-Lu , is able to match degree sequences of real-world graphs with high maximum degree . We find that these simple baselines are often competitive with more complex models like CELL in terms of matching key graph statistics , like triangle count and clustering coefficient , at similar levels of overlap . 1.3 Related Work . Existing impossibility results . Our work is inspired by that of Seshadhri et al . [ 27 ] , which also proves limitations on the ability of edge independent models to represent triangle dense graphs . They show that if P = max ( 0 , min ( 1 , XXT ) ) where X ∈ Rn×k for k ≪ n and the max and min are applied entrywise , then G ∼ G ( P ) can not have many triangles adjacent to low-degree nodes in expectation . This setting arises commonly when P is generated using low-dimensional node embeddings – represented by the rows of X . Chanpuriya et al . [ 4 ] , show that in a slightly more general model , where P = max ( 0 , min ( 1 , XY T ) ) , this lower bound no longer holds – X , Y ∈ Rn×k can be chosen so that P is the binary adjacency matrix of any graph with maximum degree upper bounded by O ( k ) – no matter how triangle dense that graph is . Thus , even such low-rank edge independent models can represent triangle dense graphs – by memorizing a single one . In the appendix , we prove a similar result when P is generated from the CELL model of [ 23 ] , which simplifies NetGAN [ 3 ] . Our results show that this trade-off between the ability to capture triangle density and memorization is inherent – even without any low-rank constraint , edge independent models with low overlap simply can not represent graphs with high triangle or other small subgraph density . It is well understood that specific edge independent models , e.g. , Erdös-Rényi graphs , the Chung-Lu model , and stochastic Kronecker graphs , do not capture many properties of real-world networks , including high triangle density [ 31 , 22 ] . Our results can be viewed as a generalization of these observations , to all edge independent models with low overlap . Despite the limitations of classic models , edge independent models are still very prevalent in today ’ s literature on graph generative models . Our more general results make clear the limitations of this approach . Non-independent models . While edge independent models are very prevalent in the literature , many important models do not fit into this framework . Classic models include the Barabási–Albert and other preferential attachment models [ 2 ] , Watts–Strogatz small-world graphs [ 31 ] , and random geometric graphs [ 6 ] . Many of these models were introduced directly in response to shortcomings of classic edge independent models , including their inability to produce high triangle densities More recent graph generative models include GraphRNN [ 32 ] and a number of other works [ 19 , 20 ] . Our impossibility results do not apply to such models , and in fact suggest that perhaps they may be preferable to edge independent models , if a distribution over graphs with high triangle density is desired . A very interesting direction for future work would be to prove limitations on broad classes of non-independent models , and perhaps to understand exactly what type of correlation amongst edges is needed to generate graphs with both low overlap 1and hallmark features of real-world networks .

Summary: In this paper, the authors analyze some properties of edge-independent generative graph models: given a fixed matrix of connection probabilities in [0,1], the edges are drawn independently. This is the basis for several methods to generate graphs that match certain statistics of a given input graph (*over the same set of nodes*). It has been observed that such models tend to produce less triangles (or generally k-cycles) than real graphs. In light of this, the authors show that the expected density of k-cycles can be bounded by the model *overlap*, which measures how much the model tends to memorize only one graph, as opposed to generating a large variety of graphs. The proofs are simple and are mostly given in the main paper. They also show that their bounds are tight, up to multiplicative constants, for Erdös-Rényi graphs, which are models with minimal overlap. Numerical experiments are performed to compare several edge-independent models, including a novel one introduced in the paper, and illustrate the (negative) theoretical findings.