On the Power of Edge Independent Graph Models

1 Introduction . Our work centers on edge independent graph models , in which each edge ( i , j ) is added to the graph independently with some probability Pij ∈ [ 0 , 1 ] . Formally , Definition 1 ( Edge Independent Graph Model ) . For any symmetric matrix P ∈ [ 0 , 1 ] n×n let G ( P ) be the distribution over undirected unweighted graphs where G ∼ G ( P ) contains edge ( i , j ) independently , with probability Pij . I.e. , p ( G ) = ∏ ( i , j ) ∈E ( G ) Pij · ∏ ( i , j ) /∈E ( G ) ( 1− Pij ) . Edge independent models encompass many classic random graph models . This includes the ErdösRényi model , where for all i ̸= j , Pij = p for some fixed p ∈ [ 0 , 1 ] [ 10 ] . It also includes the stochastic block model where Pij = p if two nodes are in the same community and Pij = q if two nodes are in different communities for some fixed p , q ∈ [ 0 , 1 ] with q < p [ 29 ] . Other examples include e.g. , the Chung-Lu configuration model [ 5 ] , stochastic Kronecker graphs [ 17 ] . Recently , significant attention has focused on graph generative models , which seek to learn a distribution over graphs that share similar properties to a given training graph , or set of graphs . Many algorithms parameterize this distribution as an edge independent model or closely related distribution . E.g. , NetGAN and the closely related CELL model both produce P ∈ [ 0 , 1 ] n×n and then sample edges independently without replacement with probabilities proportional to its entries , ensuring that at least one edge is sampled adjacent to each node [ 3 , 23 ] . Variational Graph Autoencoders ( VGAE ) , GraphVAE , Graphite , and MolGAN are also all based on edge independent models [ 16 , 28 , 8 , 13 ] . Given their popularity in both classical and modern graph generative models , it is natural to ask : 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , virtual . How suited are edge independent models to modeling real-world networks . Are they able to capture features such as power-law degree distributions , small-world properties , and high clustering coefficients ( triangle densities ) ? 1.1 Impossibility Results for Edge Independent Models . In this work we focus on the ability of edge independent models to generate graphs with high triangle , or other small subgraph densities . High triangle density ( equivalently , a high clustering coefficient ) is a well-known hallmark of real-work networks [ 31 , 25 , 9 ] and has been the focus of recent work exploring the power and limitations of edge-independent graph models [ 27 , 4 ] . It is clear that edge independent models can generate triangle dense graphs . In particular , P ∈ [ 0 , 1 ] n×n in Def . 1 can be set to the binary adjacency matrix of any undirected graph , and G ( P ) will generate that graph with probability 1 , no matter how triangle dense it is . However , this would not be a particularly interesting generative model – ideally G ( P ) should generate a wide range of graphs . To capture this intuitive notion , we define the overlap of an edge-independent model , which is closely related to the overlap stopping criterion for training used in training graph generative models [ 3 , 23 ] . Definition 2 ( Expected Overlap ) . For symmetric P ∈ [ 0 , 1 ] n×n let V ( P ) def= EG∼G ( P ) |E ( G ) | and Ov ( P ) def = EG1 , G2∼G ( P ) |E ( G1 ) ∩ E ( G2 ) | V ( P ) . That is , for any P ∈ [ 0 , 1 ] n×n , Ov ( P ) ∈ [ 0 , 1 ] is the ratio of the expected number of edges shared by two graphs drawn independently from G ( P ) to the expected number of edges in a graph drawn from G ( P ) . In one extreme , when P is a binary adjacency matrix , Ov ( P ) = 1 , and our generative model has simply memorized a single graph . In the other , if Pij = p for all i ̸= j ( i.e. , G ( P ) is Erdös-Rényi ) , Ov ( P ) = p. This is the minimum possible overlap when V ( P ) = p · ( n 2 ) . Our main result is that for any edge independent model with bounded overlap , G ∼ G ( P ) can not have too many triangles in expectation . In particular : Theorem 1 ( Main Result – Expected Triangles ) . For a graph G , let ∆ ( G ) denote the number of triangles in G. Consider symmetric P ∈ [ 0 , 1 ] n×n . EG∼G ( P ) [ ∆ ( G ) ] ≤ √ 2 3 ·Ov ( P ) 3/2 · V ( P ) 3/2 . As an example , consider the setting where the distribution generates sparse graphs , with V ( P ) = Θ ( n ) . Theorem 1 shows that whenever Ov ( P ) = o ( 1/n1/3 ) , EG∼G ( P ) ∆ ( G ) = o ( n ) – i.e . the graph is very triangle sparse with the number of triangles sublinear in the number of nodes . This verifies that an Erdös-Rényi graph can not achieve simultaneously linear number of edges ( i.e. , Ov ( P ) = O ( 1/n ) ) and super-linear number of triangles ( i.e. , Ov ( P ) = Ω ( 1/n1/3 ) ) under our proposed lens of viewing generative models . We extend Theorem 1 to give similar bounds for the density of squares and other k-cycles ( Thm . 4 ) , as well as for the global clustering coefficient ( Thm . 6 ) . In all cases we show that our bounds are tight – e.g. , in the triangle case , there is indeed an edge independent model with EG∼G ( P ) [ ∆ ( G ) ] = Θ ( Ov ( P ) 3/2 · V ( P ) 3/2 ) , matching the lower bound in Theorem 1 . 1.2 Empirical Findings . Our theoretical results help explain why , despite performing well in a variety of other metrics , edge independent graph generative models have been reported to generate graphs with many fewer triangles and squares on average than the real-world graphs that they are trained on . Rendsburg et al . [ 23 ] test a suite of these models , including their own CELL model and the related NetGAN model [ 3 ] . Of all these models , when trained on the CORA-ML graph with 2,802 triangles and 14,268 squares , none is able to generate graphs with more than 1,461 triangles and 6,880 squares on average . Similar gaps are observed for a number of other graphs . Rendsburg et al . also report that the triangle count increases as their notion of overlap ( closely related to Def . 2 ) increases . Theorem 1 demonstrates that this underestimation of triangle count , and its connection to overlap is inherent to all edge independent models , no matter how refined a method used to learn the underlying probability matrix P . While our theoretical results bound the performance of any edge independent model , there may still be variation in how specific models trade-off overlap and realistic graph generation . To better understand this trade-off , we introduce two simple models with easily tunable overlap as baselines . One is based on reproducing the degree sequence of the original graph ; the other , which is even simpler , is based on reproducing the volume . In both models , P is a weighted average of the input graph adjacency matrix and a probability matrix of minimal complexity which matches either the input degrees or the volume . In the latter case , to match just the volume , we simply use an Erdös-Rényi graph . In the former case , to match the degree sequence , we introduce our own model , the odds product model ; this model is similar to the Chung-Lu configuration model [ 5 ] , but , unlike Chung-Lu , is able to match degree sequences of real-world graphs with high maximum degree . We find that these simple baselines are often competitive with more complex models like CELL in terms of matching key graph statistics , like triangle count and clustering coefficient , at similar levels of overlap . 1.3 Related Work . Existing impossibility results . Our work is inspired by that of Seshadhri et al . [ 27 ] , which also proves limitations on the ability of edge independent models to represent triangle dense graphs . They show that if P = max ( 0 , min ( 1 , XXT ) ) where X ∈ Rn×k for k ≪ n and the max and min are applied entrywise , then G ∼ G ( P ) can not have many triangles adjacent to low-degree nodes in expectation . This setting arises commonly when P is generated using low-dimensional node embeddings – represented by the rows of X . Chanpuriya et al . [ 4 ] , show that in a slightly more general model , where P = max ( 0 , min ( 1 , XY T ) ) , this lower bound no longer holds – X , Y ∈ Rn×k can be chosen so that P is the binary adjacency matrix of any graph with maximum degree upper bounded by O ( k ) – no matter how triangle dense that graph is . Thus , even such low-rank edge independent models can represent triangle dense graphs – by memorizing a single one . In the appendix , we prove a similar result when P is generated from the CELL model of [ 23 ] , which simplifies NetGAN [ 3 ] . Our results show that this trade-off between the ability to capture triangle density and memorization is inherent – even without any low-rank constraint , edge independent models with low overlap simply can not represent graphs with high triangle or other small subgraph density . It is well understood that specific edge independent models , e.g. , Erdös-Rényi graphs , the Chung-Lu model , and stochastic Kronecker graphs , do not capture many properties of real-world networks , including high triangle density [ 31 , 22 ] . Our results can be viewed as a generalization of these observations , to all edge independent models with low overlap . Despite the limitations of classic models , edge independent models are still very prevalent in today ’ s literature on graph generative models . Our more general results make clear the limitations of this approach . Non-independent models . While edge independent models are very prevalent in the literature , many important models do not fit into this framework . Classic models include the Barabási–Albert and other preferential attachment models [ 2 ] , Watts–Strogatz small-world graphs [ 31 ] , and random geometric graphs [ 6 ] . Many of these models were introduced directly in response to shortcomings of classic edge independent models , including their inability to produce high triangle densities More recent graph generative models include GraphRNN [ 32 ] and a number of other works [ 19 , 20 ] . Our impossibility results do not apply to such models , and in fact suggest that perhaps they may be preferable to edge independent models , if a distribution over graphs with high triangle density is desired . A very interesting direction for future work would be to prove limitations on broad classes of non-independent models , and perhaps to understand exactly what type of correlation amongst edges is needed to generate graphs with both low overlap 1and hallmark features of real-world networks .

Summary: The main result is that certain structural properties of an edge-independent graph model, e.g. the expected number of triangles, are upper bounded by how strongly the graphs sampled from the model are expected to be overlapping.   Experiments on two datasets empirically confirm that several edge-independent baseline models require a high amount of overlap in their sampled graphs in order to match the number of triangles in the original graph dataset.