Original: Thinned random measures for sparse graphs with overlapping communities
Revised: Thinned random measures for sparse graphs with overlapping communities

Original: Abstract
Revised: Abstract

Original: Network models for exchangeable arrays, including most stochastic block models, generate dense graphs with a limited ability to capture many characteristics of real-world social and biological networks.
Revised: Network models for exchangeable arrays, including most stochastic block models, generate dense graphs with a limited ability to capture many characteristics of real-world social and biological networks.

Original: A class of models based on completely random measures like the generalized gamma process (GGP) have recently addressed some of these limitations.
Revised: A class of models based on completely random measures like the generalized gamma process (GGP) have recently addressed some of these limitations.

Original: We propose a framework for thinning edges from realizations of GGP random graphs that models observed links via nodes’ overall propensity to interact, as well as the similarity of node memberships within a large set of latent communities.
Revised: We propose a framework for thinning edges from realizations of GGP random graphs that models observed links via nodes’ overall propensity to interact, as well as the similarity of node memberships within a large set of latent communities.

Original: Our formulation allows us to learn the number of communities from data, and enables efficient Monte Carlo methods that scale linearly with the number of observed edges, and thus (unlike dense block models) sub-quadratically with the number of entities or nodes.
Revised: Our formulation allows us to learn the number of communities from data, and enables efficient Monte Carlo methods that scale linearly with the number of observed edges, and thus (unlike dense block models) sub-quadratically with the number of entities or nodes.

Original: We compare to alternative models for both dense and sparse networks, and demonstrate effective recovery of latent community structure for real-world networks with thousands of nodes.
Revised: We compare to alternative models for both dense and sparse networks, and demonstrate effective recovery of latent community structure for real-world networks with thousands of nodes.

Original:  Introduction
Revised:  Introduction

Original: Given observations of (often binary) relationships Y ij between pairs of nodes or entities ( i j ) many relational models [1] seek to uncover an underlying set of communities.
Revised: Given observations of (often binary) relationships Y ij between pairs of nodes or entities ( i j ) many relational models [1] seek to uncover an underlying set of communities.

Original: Classic stochastic blockmodels [2] generalize mixture models for clustering non-relational data by assigning each entity to one of K communities (clusters).
Revised: Classic stochastic blockmodels [2] generalize mixture models for clustering non-relational data by assigning each entity to one of K communities (clusters).

Original: The infinite relational model (IRM) [3] instead uses a Dirichlet process prior [4] to partition entities into single communities.
Revised: The infinite relational model (IRM) [3] instead uses a Dirichlet process prior [4] to partition entities into single communities.

Original: While the IRM allows the number of communities to be inferred from data, later work has shown that real-world social networks are better captured by models which allow nodes to participate in multiple communities [5] including applications of the hierarchical Dirichlet process (HDP) [6] to relational data [7].
Revised: While the IRM allows the number of communities to be inferred from data, later work has shown that real-world social networks are better captured by models which allow nodes to participate in multiple communities [5] including applications of the hierarchical Dirichlet process (HDP) [6] to relational data [7].

Original: Node relationships may also be modeled by shared features [8, 9] learned via the Indian Buffet Process [10], by a combination of node and interaction factors [11], or by proximity in a latent space [12, 13].
Revised: Node relationships may also be modeled by shared features [8, 9] learned via the Indian Buffet Process [10], by a combination of node and interaction factors [11], or by proximity in a latent space [12, 13].

Original: There is an extensive literature on descriptive statistics of biological and social networks [14, 15] including degree distributions, path distances and “small world” phenomena [16], community structures and modularity, and notions of centrality and causality.
Revised: There is an extensive literature on descriptive statistics of biological and social networks [14, 15] including degree distributions, path distances and “small world” phenomena [16], community structures and modularity, and notions of centrality and causality.

Original: In particular, sparsity is a ubiquitous phenomenon in real-world networks [14, 15]: as network size grows, the number of edges grows more slowly than the quadratic number of node pairs.
Revised: In particular, sparsity is a ubiquitous phenomenon in real-world networks [14, 15]: as network size grows, the number of edges grows more slowly than the quadratic number of node pairs.

Original: However, the IRM and HDP relational models (and a large literature of related models [1]) generate dense graphs where the number of edges scales quadratically with the number of nodes.
Revised: However, the IRM and HDP relational models (and a large literature of related models [1]) generate dense graphs where the number of edges scales quadratically with the number of nodes.

Original: In fact, a classic representation theorem [17, 18] shows that any generative model that regards graphs as exchangeable adjacency matrices, meaning arrays whose distribution is invariant to permutations of node indices, generates dense graphs.
Revised: In fact, a classic representation theorem [17, 18] shows that any generative model that regards graphs as exchangeable adjacency matrices, meaning arrays whose distribution is invariant to permutations of node indices, generates dense graphs.

Original: Most existing probabilistic network models, which generate an adjacency matrix by sampling edges from independent Bernoulli distributions given latent node-specific parameters, have this limitation.
Revised: Most existing probabilistic network models, which generate an adjacency matrix by sampling edges from independent Bernoulli distributions given latent node-specific parameters, have this limitation.

Original: Another important property of many real-world networks is assortative mixing [19, 20, 21], that is the presence of more popular or sociable nodes to which other nodes are more likely to connect.
Revised: Another important property of many real-world networks is assortative mixing [19, 20, 21], that is the presence of more popular or sociable nodes to which other nodes are more likely to connect.

Original: Models with degree-correction mechanisms [22, 23, 24] account for this phenomenon.
Revised: Models with degree-correction mechanisms [22, 23, 24] account for this phenomenon.

Original: By representing graphs as a latent process (a completely random measure), Caron and Fox [25] showed that it is possible to formulate generative models that capture the sparsity of real-world networks as well as assortative mixing.
Revised: By representing graphs as a latent process (a completely random measure), Caron and Fox [25] showed that it is possible to formulate generative models that capture the sparsity of real-world networks as well as assortative mixing.

Original: Related models, including certain infinite limits of graphs called graphons , have been studied by several authors [26, 27].
Revised: Related models, including certain infinite limits of graphs called graphons , have been studied by several authors [26, 27].

Original: However, these models mostly produce homogeneous graphs with sparsity and heavy-tailed degree distributions, but lacking the community structure of real networks.
Revised: However, these models mostly produce homogeneous graphs with sparsity and heavy-tailed degree distributions, but lacking the community structure of real networks.

Original: Two notable exceptions are work by Herlau et al.
Revised: Two notable exceptions are work by Herlau et al.

Original: [28] and
Revised: [28] and

Original: Todeschini et al.
Revised: Todeschini et al.

Original: [29] that augment random-measure models [25] with latent community structure.
Revised: [29] that augment random-measure models [25] with latent community structure.

Original: Alternative approaches to sparse networks regard edges, rather than nodes, as the core of the generative process [30, 31, 32].
Revised: Alternative approaches to sparse networks regard edges, rather than nodes, as the core of the generative process [30, 31, 32].

Original: Another approach to simultaneously capture local density and global sparsity models graphs as a collection of cliques [33].
Revised: Another approach to simultaneously capture local density and global sparsity models graphs as a collection of cliques [33].

Original: In this paper, we propose a novel random graph model that efficiently thins edges from sparse homogeneous graphs to reveal community structure while maintaining sparsity.
Revised: In this paper, we propose a novel random graph model that efficiently thins edges from sparse homogeneous graphs to reveal community structure while maintaining sparsity.

Original: Unlike [28], we allow entities to have membership in multiple communities.
Revised: Unlike [28], we allow entities to have membership in multiple communities.

Original: Unlike [29], we allow completely flexible specification of the hierarchical Bayesian prior on latent community memberships; importantly, this enables the network-specific learning of the appropriate number of latent communities.
Revised: Unlike [29], we allow completely flexible specification of the hierarchical Bayesian prior on latent community memberships; importantly, this enables the network-specific learning of the appropriate number of latent communities.

Original: We further develop an efficient Monte Carlo inference algorithm that, unlike nearly all dense block models, scales linearly with the number of observed edges and thus sub-quadratically with the number of entities.
Revised: We further develop an efficient Monte Carlo inference algorithm that, unlike nearly all dense block models, scales linearly with the number of observed edges and thus sub-quadratically with the number of entities.

Original: Experiments show recovery of communities for networks with thousands of nodes.
Revised: Experiments show recovery of communities for networks with thousands of nodes.

Original:  Background: Stochastic Blockmodels for Dense and Sparse Networks
Revised:  Background: Stochastic Blockmodels for Dense and Sparse Networks

Original: An undirected binary network with N nodes and E edges may be represented by a symmetric N × N adjacency matrix Y .
Revised: An undirected binary network with N nodes and E edges may be represented by a symmetric N × N adjacency matrix Y .

Original: If there is an edge (link) between nodes i ̸
Revised: If there is an edge (link) between nodes i ̸

Original: =
Revised: =

Original: j then Y ij = 1 , otherwise Y ij = 0 .
Revised: j then Y ij = 1 , otherwise Y ij = 0 .

Original: 2.1 Mixed Membership Stochastic Blockmodels
Revised: 2.1 Mixed Membership Stochastic Blockmodels

Original: Stochastic blockmodels (SBMs) [2] assume that each node belongs to one of K latent communities, and that the probability of an edge depends on how strongly their communities are connected.
Revised: Stochastic blockmodels (SBMs) [2] assume that each node belongs to one of K latent communities, and that the probability of an edge depends on how strongly their communities are connected.

Original: The community c i ∈ { 1 , . . .
Revised: The community c i ∈ { 1 , . . .

Original: , K } of node i follows a categorical distribution
Revised: , K } of node i follows a categorical distribution

Original: c
Revised: c

Original: i ind ∼ Cat ( β ) , where β = ( β 1 , . . .
Revised: i ind ∼ Cat ( β ) , where β = ( β 1 , . . .

Original: , β K ) so that β k is the frequency of community k .
Revised: , β K ) so that β k controls the relative size of community k .

Original: Edges are sampled independently as
Revised: Edges are sampled independently as

Original: Y ij ind ∼ Bernoulli ( η c i c j ) , where η kℓ =
Revised: Y ij ind ∼ Bernoulli ( η c i c j ) , where η kℓ =

Original: η ℓk is the probability of an edge between nodes in communities k and ℓ .
Revised: η ℓk is the probability of an edge between nodes in communities k and ℓ .

Original: These interaction probabilities are often assigned conjugate beta priors, η kℓ ind ∼ Beta ( τ a , τ b ) .
Revised: These interaction probabilities are often assigned conjugate beta priors, η kℓ ind ∼ Beta ( τ a , τ b ) .

Original: Mixed membership stochastic blockmodels (MMSBs) [5] extend SBMs to allow nodes to be members of multiple communities.
Revised: Mixed membership stochastic blockmodels (MMSBs) [5] extend SBMs to allow nodes to be members of multiple communities.

Original: Let π i =
Revised: Let π i =

Original: ( π i 1 , . . . , π iK ) denote a K -dimensional probability vector representing the strength of affiliation of node i to each of K communities.
Revised: ( π i 1 , . . . , π iK ) denote a K -dimensional probability vector representing the strength of affiliation of node i to each of K communities.

Original: For every pair of nodes ( i, j ) , the communities governing their interaction are sampled as c ij ind ∼ Cat ( π i ) , c ji ind ∼ Cat ( π j ) .
Revised: For every pair of nodes ( i, j ) , the communities governing their interaction are sampled as c ij ind ∼ Cat ( π i ) , c ji ind ∼ Cat ( π j ) .

Original: Then, like standard SBMs, edges are sampled independently as Y ij ind ∼ Bernoulli ( η c ij c ji ) .
Revised: Then, like standard SBMs, edges are sampled independently as Y ij ind ∼ Bernoulli ( η c ij c ji ) .

Original: Community memberships are typically assigned a hierarchical prior, such as
Revised: Community memberships are typically assigned a hierarchical prior, such as

Original: [Equation]
Revised: [Equation]

Original: The concentration parameter ζ controls the polarization of π i and its variation around β , with smaller values of ζ inducing more polarized community memberships that place significant probability on only a few communities, and larger values of ζ inducing community memberships that differ very little from β .
Revised: The concentration parameter ζ controls the polarization of π i and its variation around β , with smaller values of ζ inducing more polarized community memberships that place significant probability on only a few communities, and larger values of ζ inducing community memberships that differ very little from β .

Original: When fitting this model to data, we can treat K as an upper bound to the number of communities (that is, larger than the maximum number of communities that we expect to be necessary to model the network), fix γ ≪ K and let ζ be small.
Revised: When fitting this model to data, we can treat K as an upper bound to the number of communities (that is, larger than the maximum number of communities that we expect to be necessary to model the network), fix γ ≪ K and let ζ be small.

Original: In this way, the hierarchical Dirichlet formulation allows network-specific learning of the number of communities by favoring sparse
Revised: In this way, the hierarchical Dirichlet formulation allows network-specific learning of the number of communities by favoring sparse

Original: 1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent)
Revised: 1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent)

Original: GG P t h i nned
Revised: GG P t h i nned

Original: GG P                         01    1      0 

Revised: GG P                         01    1      0

Original: 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple undirected graph observed)           community frequency vectors β
Revised: 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple undirected graph observed)           community frequency vectors β

Original: = ( β 1 , . .
Revised: = ( β 1 , . .

Original: . , β K ) with some elements β h ≈ 0 , h ∈ { 1 , . .
Revised: . , β K ) with some elements β h ≈ 0 , h ∈ { 1 , . .

Original: . , K } .
Revised: , K } [34].

Original: The hierarchical Dirichlet process [6, 7] is the limit of this prior as K approaches infinity [34].
Revised: The hierarchical Dirichlet process [6, 7] is the limit of this prior as K approaches infinity [35].

Original: 2.2 Sparse Network Models via Completely Random Measures
Revised: 2.2 Sparse Network Models via Completely Random Measures

Original: For any fixed community frequencies β and interaction probabilities η , the SBM and MMSB may only generate dense graphs where the number of edges scales quadratically with the number of nodes [18].
Revised: For any fixed community frequencies β and interaction probabilities η , the SBM and MMSB may only generate dense graphs where the number of edges scales quadratically with the number of nodes [18].

Original: In contrast, many real-world networks appear to be sparse [15].
Revised: In contrast, many real-world networks appear to be sparse [15].

Original: Heuristics are often used to fit (mixed membership) SBMs to large but sparse networks, such as fixing (rather than learning)
Revised: Heuristics are often used to fit (mixed membership) SBMs to large but sparse networks, such as fixing (rather than learning)

Original: η
Revised: η

Original: kℓ = ε ≈ 0 for k ̸ = ℓ ; for example Kim et al.
Revised: kℓ = ε ≈ 0 for k ̸ = ℓ ; for example Kim et al.

Original: [7] fix ε = e − 30 .
Revised: [7] fix ε = e − 30 .

Original: We seek to build models that can capture mixed memberships and sparsity simultaneously without needing to rely on such heuristics.
Revised: We seek to build models that can capture mixed memberships and sparsity simultaneously without needing to rely on such heuristics.

Original: Completely Random Measures.
Revised: Completely Random Measures.

Original: By representing the graph as a point process on the plane, Caron and Fox [25] showed that it is possible to generate sparse graphs by appropriately choosing the mean measure of the point process.
Revised: By representing the graph as a point process on the plane, Caron and Fox [25] showed that it is possible to generate sparse graphs by appropriately choosing the mean measure of the point process.

Original: According to their model, for i ̸
Revised: According to their model, for i ̸

Original: = j ,
Revised: = j ,

Original: [Equation]
Revised: [Equation]

Original: where w i > 0 represents the sociability of node i : nodes with higher w i have higher probability to interact with other nodes, and hence greater expected degree.
Revised: where w i > 0 represents the sociability of node i : nodes with higher w i have higher probability to interact with other nodes, and hence greater expected degree.

Original: Differently from the (mixed membership) SBMs, where all parameters governing nodes’ interactions are sampled independently, here nodes’ sociabilities are generated altogether using the jumps of a completely random measure (CRM) [35].
Revised: Differently from the (mixed membership) SBMs, where all parameters governing nodes’ interactions are sampled independently, here nodes’ sociabilities are generated altogether using the jumps of a completely random measure (CRM) [36].

Original: Each node is also independently associated to a real-valued location ℓ i uniformly distributed on the real line, and then a set of potential nodes is defined by restricting to sociabilities with a sampled location in the interval [0 , α ] ,
Revised: Each node is also independently associated to a real-valued location ℓ i uniformly distributed on the real line, and then a set of potential nodes is defined by restricting to sociabilities with a sampled location in the interval [0 , α ] ,

Original: [Equation]
Revised: [Equation]

Original: Here α > 0 controls the (random) number of nodes N α in the network by determining the size of the interval in which jumps are included.
Revised: Here α > 0 controls the (random) number of nodes N α in the network by determining the size of the interval in which jumps are included.

Original: Depending on the distribution of the jumps, the model can capture both sparse and dense graphs.
Revised: Depending on the distribution of the jumps, the model can capture both sparse and dense graphs.

Original: Intuitively, as detailed in Sec.
Revised: Intuitively, as detailed in Sec.

Original: 5.1 of [25], for sparse networks the distribution of the jumps needs to place almost all of its mass near zero.
Revised: 5.1 of [25], for sparse networks the distribution of the jumps needs to place almost all of its mass near zero.

Original: This model requires CRMs for which the sum ¯ 
Revised: This model requires CRMs for which the sum ¯ W α

Original: [Equation] i of the jumps in [ [Equation] α ] is finite.
Revised: = (cid:80) i : ℓ i ∈ [0 ,α ] w i of the jumps in [0 , α ] is finite.

Original: A simple undirected network is then generated via a binary projection of an underlying directed multigraph, where the total number of edges n ij from node i to node j is independently distributed as
Revised: A simple undirected network is then generated via a binary projection of an underlying directed multigraph, where the total number of edges n ij from node i to node j is independently distributed as

Original: [Equation]
Revised: [Equation]

Original: An edge is present in the undirected network if there is at least one directed edge in the latent directed multigraph, that is Y ij = 1 ( n ij + n ji ≥ 1) for nodes i ̸
Revised: An edge is present in the undirected network if there is at least one directed edge in the latent directed multigraph, that is Y ij = 1 ( n ij + n ji ≥ 1) for nodes i ̸

Original: = j .
Revised: = j .

Original: Because the sum of independent Poisson random variables is Poisson, Eq.
Revised: Because the sum of independent Poisson random variables is Poisson, Eq.

Original: (4) implies P ( n ij + n ji = 0)
Revised: (4) implies P ( n ij + n ji = 0)

Original: =
Revised: =

Original: exp {− 2 w i w j } , from which Eq.
Revised: exp {− 2 w i w j } , from which Eq.

Original: (2) follows.
Revised: (2) follows.

Original: The number N of nodes in the observed network then equals the number N α of nodes that have at least one edge in the underlying multigraph.
Revised: The number N of nodes in the observed network then equals the number N α of nodes that have at least one edge in the underlying multigraph.

Original: This construction of the binary matrix from a multigraph [25] is visualized in Fig. 
Revised: This construction of the binary matrix from a multigraph [25] is visualized in Fig. 1(top).

Original: The sum-property of the Poisson distribution also implies that, given ¯ W α , the total number of edges¯
Revised: The sum-property of the Poisson distribution also implies that, given ¯ W α , the total number of edges

Original: D α in the multigraph has a Poisson ( ¯ W 2 α ) distribution.
Revised: D α in the multigraph has a Poisson ( ¯ W 2 α ) distribution.

Original: Since n ij has a high probability of being 0 for most node pairs ( i, j ) , it is more efficient to first sample¯ D α
Revised: Since n ij has a high probability of being 0 for most node pairs ( i, j ) , it is more efficient to first sample D α

Original: and then independently assign each edge to a pair of nodes based on their sociabilities.
Revised: and then independently assign each edge to a pair of nodes based on their sociabilities.

Original: In more detail, Eq.
Revised: In more detail, Eq.

Original: (4) is equivalent to¯ D α | ¯ W α ∼ Poisson ( ¯ W 
Revised: (4) is equivalent to D α | ¯ W α ∼ Poisson ( ¯ W 2 α ) ,

Original: [Equation][Equation]
Revised: [Equation]

Original: where x ev ∈ { 1 , 2 , . . . }
Revised: where x ev ∈ { 1 , 2 , . . . }

Original: for v = 1 , 2 indicates the nodes sampled for edges e ∈ { 1 , . . 
Revised: for v = 1 , 2 indicates the nodes sampled for edges e ∈ { 1 , . . .

Original: ,¯ D α } .
Revised: , D α } .

Original: Caron and Fox [25] propose the generalized gamma process (GGP) [36, 37, 38] as a flexible but tractable CRM for W α , with parameters τ ∈ (0 , ∞ ) , and σ ∈ ( −∞ , 0] for dense graphs or σ ∈ (0 , 1) for sparse graphs.
Revised: Caron and Fox [25] propose the generalized gamma process (GGP) [37, 38, 39] as a flexible but tractable CRM for W α , with parameters τ ∈ (0 , ∞ ) , and σ ∈ ( −∞ , 0] for dense graphs or σ ∈ (0 , 1) for sparse graphs.

Original: Fig.
Revised: Fig.

Original: 2 (black component) provides a visual summary of the generative process underlying the construction of a GGP random graph, using a directed graphical model that highlights the conditional dependencies (and independencies) between its variables.
Revised: 2 (black component) provides a visual summary of the generative process underlying the construction of a GGP random graph, using a directed graphical model that highlights the conditional dependencies (and independencies) between its variables.

Original: Fig.
Revised: Fig.

Original: 3a shows a network sampled from the GGP model by means of a node-edge diagram.
Revised: 3a shows a network sampled from the GGP model by means of a node-edge diagram.

Original: Sparse Block Models.
Revised: Sparse Block Models.

Original: A limitation of the framework described above is that it does not model the community (block) structure of the network – a well-recognized feature of complex networks.
Revised: A limitation of the framework described above is that it does not model the community (block) structure of the network – a well-recognized feature of complex networks.

Original: Herlau et al.
Revised: Herlau et al.

Original: [28] generalized the approach in [25] to accommodateboth sparse and dense networks with community structure.
Revised: [28] generalized the approach in [25] to accommodate networks with community structure.

Original: Specifically, they introduce a latent discrete variable c i ∈ { 1 , . .
Revised: Specifically, they introduce a latent discrete variable c i ∈ { 1 , . .

Original: , K } to indicate the assignment of node i to one of K communities, like in the SBMs.
Revised: , K } to indicate the assignment of node i to one of K communities, like in the SBMs.

Original: A bivariate CRM incorporates both the sociability weights and a set of parameters, denoted by η c i c j , capturing the interaction strength between two communities
Revised: A bivariate CRM incorporates both the sociability weights and a set of parameters, denoted by η c i c j , capturing the interaction strength between two communities

Original: c i and c j in the underlying multigraph.
Revised: c i and c j in the underlying multigraph.

Original: For their formulation, the total number of edges n ij from node i to node j are independently distributed as n ij | c i , c j , w i , w j ind ∼ Poisson ( η c i c j w i w j ) , and the likelihood from Eq.
Revised: For their formulation, the total number of edges n ij from node i to node j are independently distributed as n ij | c i , c j , w i , w j ind ∼ Poisson ( η c i c j w i w j ) , and the likelihood from Eq.

Original: (2) of the observed nodes is modified as Y ij |
Revised: (2) of the observed nodes is modified as Y ij |

Original: c i , c j , w i , w j ind ∼ Bernoulli (1 − e − 2 η cicj w i w j ) .
Revised: c i , c j , w i , w j ind ∼ Bernoulli (1 − e − 2 η cicj w i w j ) .

Original: We refer to this approach as the stochastic block model generalized gamma process (SBM-GGP).
Revised: We refer to this approach as the stochastic block model generalized gamma process (SBM-GGP).

Original: Sparse Mixed Membership.
Revised: Sparse Mixed Membership.

Original: The SBM-GGP models sparse and dense networks with community structure, but does not capture overlapping community structures; it induces networks whose nodes are partitioned into disjoint communities.
Revised: The SBM-GGP models sparse and dense networks with community structure, but does not capture overlapping community structures; it induces networks whose nodes are partitioned into disjoint communities.

Original: In follow-up work, Todeschini et al.
Revised: In follow-up work, Todeschini et al.

Original: [29] extended the
Revised: [29] extended the

Original: CRM-based framework discussed above by associating a vector ( w i 1 , .
Revised: CRM-based framework discussed above by associating a vector ( w i 1 , .

Original: . . , w iK ) of sociabilities to each node
Revised: . . , w iK ) of sociabilities to each node

Original: i , to represent different levels of affiliation of nodes to the K latent communities.
Revised: i , to represent different levels of affiliation of nodes to the K latent communities.

Original: A node may have high levels of affiliation to more than a community, leading to the formation of edges across multiple communities.
Revised: A node may have high levels of affiliation to more than a community, leading to the formation of edges across multiple communities.

Original: The vectors of node sociabilities are distributed according to a compound
Revised: The vectors of node sociabilities are distributed according to a compound

Original: CRM [39]; specifically, their implementation uses a compound generalized gamma process (CGGP).
Revised: CRM [40]; specifically, their implementation uses a compound generalized gamma process (CGGP).

Original: The likelihood function is modified accordingly, by independently sampling undirected edges as
Revised: The likelihood function is modified accordingly, by independently sampling undirected edges as

Original: Y ij | w i , w j ind ∼ Bernoulli (1 − exp {− 2 (cid:80) Kk
Revised: Y ij | w i , w j ind ∼ Bernoulli (1 − exp {− 2 (cid:80) Kk

Original: =

Revised: =

Original: w ik w jk } ) .
Revised: w ik w jk } ) .

Original: The latent community weights are modeled
Revised: The latent community weights are modeled

Original: ∞ ¯ D α
Revised: ∞ ¯ D α

Original: N × N w
Revised: N × N w

Original: i
Revised: i

Original: α σ τ π i ζ β γ
Revised: α σ τ π i ζ β γ

Original: K x ex ec ec e
Revised: K x ex ec ec e

Original: Y ij as w ik = w i 0 ψ
Revised: Y ij as w ik = w i 0 ψ

Original: ik , where w i 0 are sampled from a GGP as in Eq.
Revised: ik , where w i 0 are sampled from a GGP as in Eq.

Original: (3), and ψ ik are gamma-distributed random variables ψ ik ind ∼ Gamma ( a k , b k ) whose shape a k and rate b k may be inferred from data.
Revised: (3), and ψ ik are gamma-distributed random variables ψ ik ind ∼ Gamma ( a k , b k ) whose shape a k and rate b k may be inferred from data.

Original: As [29] point out, the CGGP model exploits an (unconstrained) non-negative matrix factorization to define the Bernoulli probability link.
Revised: As [29] point out, the CGGP model exploits an (unconstrained) non-negative matrix factorization to define the Bernoulli probability link.

Original: In the Appendix, we demonstrate that this lack of constraints hampers the ability of the CGGP model to recover the true community structure of the network.
Revised: Our results in Sec. 5.1 suggest that this lack of constraints affects the ability of the CGGP model to recover the true community structure of the network.

Original: Better regularized approaches to non-negative matrix factorization, like the one we propose, may improve the identifiability of latent community structures and network parameters [40].
Revised: Better regularized approaches to non-negative matrix factorization, like the one we propose, may improve the identifiability of latent community structures and network parameters [41].

Original:  The Thinned CRM Network Model
Revised:  The Thinned GGP Network Model

Original: We now add overlapping community memberships to the GGP network model [25] via vectors of probabilities sampled from a hierarchical Dirichlet distribution.
Revised: We now add overlapping community memberships to the GGP network model [25] via vectors of probabilities sampled from a hierarchical Dirichlet distribution.

Original: Unlike the formulation of Todeschini et al.
Revised: Unlike the formulation of Todeschini et al.

Original: [29], our model enables learning the number of communities from data (see Secs.
Revised: [29], our model enables learning the number of communities from data (see Secs.

Original: 2.1 and 4).
Revised: 2.1 and 4).

Original: By using probability vectors rather than unconstrained non-negative values to model community memberships, our model provides a regularized approach to inference in the GGP model, which in simulations (Sec. 5.1) provides a substantial increase in community detection accuracy.
Revised: By using probability vectors rather than unconstrained non-negative values to model community memberships, our model provides a regularized approach to inference in the GGP model, which in simulations (Sec. 5.1) provides a substantial increase in community detection accuracy.

Original: Let each node i have both a sociability parameter w i from the GGP as in (3), and a vector of probabilities π i =
Revised: Let each node i have both a sociability parameter w i from the GGP as in (3), and a vector of probabilities π i =

Original: ( π i 1 , . . . , π iK ) drawn from a hierarchical Dirichlet distribution as in (1).
Revised: ( π i 1 , . . . , π iK ) drawn from a hierarchical Dirichlet distribution as in (1).

Original: Moreover, let the number of potential edges between nodes i and j in the latent multigraph depend only on their sociabilities, as in (4).
Revised: Moreover, let the number of potential edges between nodes i and j in the latent multigraph depend only on their sociabilities, as in (4).

Original: For each of the n ij potential edges, node i independently samples a community from Cat ( π i ) , and node j samples a community from Cat ( π j ) .
Revised: For each of the n ij potential edges, node i independently samples a community from Cat ( π i ) , and node j samples a community from Cat ( π j ) .

Original: If these community assignments match, the edge is retained; otherwise, it is thinned (i.e., discarded).
Revised: If these community assignments match, the edge is retained; otherwise, it is thinned (i.e., discarded).

Original: See Figs.
Revised: See Figs.

Original: 1 and 3 for examples.
Revised: 1 and 3 for examples.

Original: More formally, let ˙ n ij be the number of multigraph edges between a pair of nodes i, j that is retained (not thinned).
Revised: More formally, let ˙ n ij be the number of multigraph edges between a pair of nodes i, j that is retained (not thinned).

Original: Edges e ∈ { 1 , . . .
Revised: Edges e ∈ { 1 , . . .

Original: ,¯ D α } in the GGP multigraph are stochastically thinned as follows:
Revised: , D α } in the GGP multigraph are stochastically thinned as follows:

Original: [Equation]
Revised: [Equation]

Original: Here x ev ∈ { 1 , 2 , . . .
Revised: Here x ev ∈ { 1 , 2 , . . .

Original: } for v = 1 , 2 indicate the nodes associated with edge e as in Eq.
Revised: } for v = 1 , 2 indicate the nodes associated with edge e as in Eq.

Original: An edge Y ij is then present in the undirected graph if
Revised: An edge Y ij is then present in the undirected graph if

Original: and only if at least one multigraph edge is not thinned.
Revised: and only if at least one multigraph edge is not thinned.

Original: Equivalently, Eq.
Revised: Equivalently, Eq.

Original: (7) implies that the observed undirected graph is a binary projection of the multigraph edges that areselected , i.e. retained after the thinning process.
Revised: (7) implies that the observed undirected graph is a binary projection of the multigraph edges that are retained after the thinning process.

Original: To determine the distribution of Y ij , note that the probability that nodes i and j are assigned to the same community (marginalizing across communities) equals P ( c ei = c ej )
Revised: To determine the distribution of Y ij , note that the probability that nodes i and j are assigned to the same community (marginalizing across communities) equals P ( c ei = c ej )

Original: = (cid:80)
Revised: = (cid:80)

Original: Kk =

Revised: Kk =

Original: π ik π jk .
Revised: π ik π jk .

Original: Therefore, marginalizing over the latent multigraph, ˙ n ij |w i , w j , π [Equation] , π [Equation]
Revised: Therefore, marginalizing over the latent multigraph, ˙ n ij |

Original: ind ∼ Poisson ( w

Original: [Equation]
Revised: [Equation]

Original: w

Original: [Equation]

Original: (cid:80) Kk [Equation] The likelihood of the observed network thus equals
Revised: The likelihood of the observed network thus equals

Original: [Equation]
Revised: [Equation]

Original: Unlike the MMSB and GGP models, our thinned generalized gamma process (TGGP) model favors edges between nodes that have both large sociabilities and similar community memberships.
Revised: Unlike the MMSB and GGP models, our thinned generalized gamma process (TGGP) model favors edges between nodes that have both large sociabilities and similar community memberships.

Original: The
Revised: The

Original: TGGP model is summarized graphically in Fig.
Revised: TGGP model is summarized graphically in Fig.

Original: 2 and illustrated in Figs.
Revised: 2 and illustrated in Figs.

Original: 1 and 3.
Revised: 1 and 3.

Original: [Figure]
Revised: [Figure]

Original:  Monte Carlo Posterior Inference
Revised:  Monte Carlo Posterior Inference

Original: In our TGGP model, the latent (unthinned) multigraph has a distribution (Eq.
Revised: In our TGGP model, the latent (unthinned) multigraph has a distribution (Eq.

Original: that depends only on the node sociability parameters w i .
Revised: that depends only on the node sociability parameters w i .

Original: The thinning process (Eq.
Revised: The thinning process (Eqs.

Original:  (7)) then depends only on the community membership vectors
Revised: (6) and (7)) then depends only on the community membership vectors

Original: π i .
Revised: π i .

Original: As can be deduced from the Markov properties of the directed graphical model in Fig. 2, given these latent multigraphs, the posterior distributions of ( w 1 , w 2 , . . . )
Revised: As can be deduced from the Markov properties of the directed graphical model in Fig. 2, given these latent multigraphs, the posterior distributions of ( w 1 , w 2 , . . . )

Original: and ( π 1 , π 2 , . . . ) are conditionally independent.
Revised: and ( π 1 , π 2 , . . . ) are conditionally independent.

Original: This design simplifies posterior inference, and in particular allows us to apply Theorem 6 of [25] to derive the posterior of nodes’ sociabilities, provided wecan condition on the (latent) multigraph.
Revised: This design simplifies posterior inference, and in particular allows us to apply Theorem 6 of [25] to derive the posterior of nodes’ sociabilities, provided we condition on the (latent) multigraph.

Original: We thus implement a Gibbs sampling strategy for posterior inference where we sample both thinned and retained edges in the latent multigraph, as well as their community assignments, to allow for efficient updates of other variables.
Revised: We thus implement a Gibbs sampling strategy for posterior inference where we sample both thinned and retained edges in the latent multigraph, as well as their community assignments, to allow for efficient updates of other variables 1 .

Original: Some steps in the sampler are straightforwardly derived, considering that we can borrow the approach detailed in [25] to update node sociabilities and GGP hyperparameters (see the Appendix for a detailed outline).
Revised: Some steps in the sampler are straightforwardly derived, considering that we can rely on the approach described in [25] to update node sociabilities and GGP hyperparameters (see the Appendix for a detailed outline).

Original: However, the implementation of the sampler requires some additional careful development – which we detail below – to sample the latent multigraph, due to the unobserved thinning process.
Revised: However, the implementation of the sampler requires some additional careful development – which we detail below – to sample the latent multigraph, due to the unobserved thinning process.

Original: Multigraph sampling is made more computationally efficient via model properties that are unique to sparse, as opposed to dense, stochastic block models.
Revised: Multigraph sampling is made more computationally efficient via model properties that are unique to sparse, as opposed to dense, stochastic block models.

Original: Letting (filled dot) ˙ n ij denote the number of unthinned(i.e. selected) multigraph edges as in (7), and (empty dot) ˚ n ij = n ij − ˙ n ij denote the number of thinned multigraph edges, we sample the latent multigraph as follows.
Revised: Letting (filled dot) ˙ n ij denote the number of unthinned multigraph edges as in (7), and (empty dot) ˚ n ij = n ij − ˙ n ij denote the number of thinned multigraph edges, we sample the latent multigraph as follows.

Original: Sampling of ˙n ij : From Eq.
Revised: Sampling of ˙n ij : From Eq.

Original: (7), there exists an edge in the observed graph if and only if there is at least one unthinned edge between nodes i and j in the latent multigraph.
Revised: (7), there exists an edge in the observed graph if and only if there is at least one unthinned edge between nodes i and j in the latent multigraph.

Original: Thus rather than considering each pair i, j , we only need to sample ˙ n ij when Y ij = 1 , an operation with cost linear in the number of observed (undirected) edges.
Revised: Thus rather than considering each pair i, j , we only need to sample ˙ n ij when Y ij = 1 , an operation with cost linear in the number of observed (undirected) edges.

Original: Conversely, Y ij = 0 implies ˙ n ij = ˙ n ji = 0 .
Revised: Conversely, Y ij = 0 implies ˙ n ij = ˙ n ji = 0 .

Original: Since there must be at least one unthinned edge when Y ij = 1 , the posterior follows a zero-truncated Poisson distribution:
Revised: Since there must be at least one unthinned edge when Y ij = 1 , the posterior follows a zero-truncated Poisson distribution:

Original: [Equation]
Revised: [Equation]

Original: For each unthinned edge we sample a single community assignment, since by definition these are the edges whose nodes have been assigned to the same communities.
Revised: For each unthinned edge we sample a single community assignment, since by definition these are the edges whose nodes have been assigned to the same communities.

Original: We can sample the community assignment of an unthinned edge between i and j easily as a draw from Cat ( π i 1 π j 1 , . . .
Revised: We can sample the community assignment of an unthinned edge between i and j easily as a draw from Cat ( π i 1 π j 1 , . . .

Original: , π iK π jK ) .
Revised: , π iK π jK ) .

Original: Sampling of ˚n ij : By construction, since thinned edges are unobserved, all pairs of nodes ( i, j ) may have edges that are thinned.
Revised: Sampling of ˚n ij : By construction, since thinned edges are unobserved, all pairs of nodes ( i, j ) may have edges that are thinned.

Original: These edges are auxiliary variables necessary to obtain the full conditional posterior distribution of W α .
Revised: These edges are auxiliary variables necessary to obtain the full conditional posterior distribution of W α .

Original: Also, to update community memberships and global frequencies, it is necessary to assign every thinned edge to a pair of discordant (non-matching) communities.Thanks to the properties of Poisson processes, we do not need to sample ˚ n ij for all the pairs of nodes;
Revised: Also, to update community memberships and global frequencies, it is necessary to assign every thinned edge to a pair of discordant (non-matching) communities.

Original: we
Revised: Thanks

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: N a
Revised: N a

Original: N
Revised: N

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: a α −0.
Revised: a α −0.

Original: 20000 iteration s σ 0.
Revised: 20000 iteration s σ 0.

Original: [Figure]
Revised: [Figure]

Original: t
Revised: t

Original: [Figure]
Revised: [Figure]

Original:  can develop a more efficient sampler that exploits the thinned model construction.
Revised: to the properties of Poisson processes, we do not need to sample ˚ n ij for all node pairs; we can develop a more efficient sampler that exploits the thinned model construction.

Original: We first sample a proposed total number of thinned edges ˚ N ∼ Poisson (cid:16) 2 (cid:80) Ni,j =1 w i w j (cid:17) .
Revised: We first sample a proposed total number of thinned edges ˚ D P ∼ Poisson (cid:16) 2 (cid:80) Ni,j =1 w i w j (cid:17) .

Original: We then assign each of these independently to a pair of nodes according to their sociabilities as in Eq.
Revised: We then assign each of these independently to a pair of nodes according to their sociabilities as in Eq.

Original: (5), and subsequently to a pair of commmunities given the memberships of the assigned nodes as in Eq.
Revised: (5), and subsequently to a pair of commmunities given the memberships of the assigned nodes as in Eq.

Original: Finally, we determine ˚ n ij from the number of edges that were assigned to discordant communities.
Revised: Finally, we determine ˚ n ij from the number of edges that were assigned to discordant communities.

Original: Sampling unobserved nodes: As illustrated in Figs.
Revised: Sampling unobserved nodes: As illustrated in Figs.

Original: 1 and 3, the total number of nodes N α with at least one edge in the unthinned multigraph is likely to be different from the total number of nodes with at least one edge after thinning and – as a consequence – from the total number of nodes N in the observed binary graph Y .
Revised: 1 and 3, the total number of nodes N α with at least one edge in the unthinned multigraph is likely to be different from the total number of nodes with at least one edge after thinning and – as a consequence – from the total number of nodes N in the observed binary graph Y .

Original: There is thus some latent number N α − N ≥ 0 of nodes whose edges with the observed nodes have all been thinned.
Revised: There is thus some latent number N α − N ≥ 0 of nodes whose edges with the observed nodes have all been thinned.

Original: In order to learn this number, our MCMC sampler includes an approximate update of N α according to the mean ratio N α /N from graphs simulated from the
Revised: In order to learn this number, our MCMC sampler includes an approximate update of N α according to the mean ratio N α /N from graphs simulated from the

Original: GGP prior given the latest samples of the hyperparameters α, σ, τ .
Revised: GGP prior given the latest samples of the hyperparameters α, σ, τ .

Original: We found that this approximate method leads to convergence of the empirical MCMC-based estimate of N α across all simulated and real data that we considered (see Fig. 4).
Revised: We found that this approximate method leads to convergence of the empirical MCMC-based estimate of N α across all simulated and real data that we considered (see Fig. 4).

Original: Sampling of the thinned edges from these unobserved nodes introduces some additional complications that we efficiently resolve  in the Appendix.
Revised: Sampling of the thinned edges from these unobserved nodes introduces some additional complications that we efficiently resolve as detailed in the Appendix.

Original: Sampling community memberships π i and global frequencies β : From the conjugacy of Dirichlet priors to categorical likelihoods, node community memberships π i have closed-form Dirichlet posteriors
Revised: Sampling community memberships π i and global frequencies β : From the conjugacy of Dirichlet priors to categorical likelihoods, node community memberships π i have closed-form Dirichlet posteriors

Original: π i | M i , β ind ∼ Dirichlet ( ζβ 1 + M i 1 , . .
Revised: π i | M i , β ind ∼ Dirichlet ( ζβ 1 + M i 1 , . .

Original: , ζβ K + M iK ) , where M ik equals the number of edges that node i participated in while being assigned to community k .
Revised: , ζβ K + M iK ) , where M ik equals the number of edges that node i participated in while being assigned to community k .

Original: Given community assignments of nodes in the latent multigraph, the global community frequencies β may be efficiently resampled using auxiliary-variable methods developed for the hierarchical Dirichlet process [6] (see Appendix).
Revised: Given community assignments of nodes in the latent multigraph, the global community frequencies β may be efficiently resampled using auxiliary-variable methods developed for the hierarchical Dirichlet process [6] (see Appendix).

Original: By learning β , node community memberships π i become sparse, placing significant probability mass only on a data-dependent subset of the full set of K communities allocated in Eq.
Revised: By learning β , node community memberships π i become sparse, placing significant probability mass only on a data-dependent subset of the full set of K communities allocated in Eq.

Original: Computational complexity: For a K -community TGGP model of a graph with E observed edges, resampling the latent multigraph requires O ( EK ) operations.
Revised: Computational complexity: For a K -community TGGP model of a graph with E observed edges, resampling the latent multigraph requires O ( EK ) operations.

Original: (The thinned multigraph has fewer edges, and thus its resampling increases costs by a small constant.)
Revised: (The thinned multigraph has fewer edges, and thus its resampling increases costs by a small constant.)

Original: Closed-form resampling of community memberships is faster, requiring only O ( NK ) operations for an N -node graph.
Revised: Closed-form resampling of community memberships is faster, requiring only O ( NK ) operations for an N -node graph.

Original: The
Revised: The

Original: CGGP model [29] uses Hamiltonian Monte Carlo proposals with similar cost, but empirically the
Revised: CGGP model [29] uses Hamiltonian Monte Carlo proposals with similar cost, but empirically the

Original: CGGP sampler mixes slower and has inferior performance (see experiments).
Revised: CGGP sampler mixes slower and has inferior performance (see experiments).

Original: By exploiting sparse matrices and parallelization, both samplers may be scaled to networks with tens of thousands of nodes; very-large networks may require alternative approximate inference algorithms.
Revised: By exploiting sparse matrices and parallelization, both samplers may be scaled to networks with tens of thousands of nodes; very-large networks may require alternative approximate inference algorithms.

Original: MCMC sampler validation: We evaluate the proposed posterior inference method on a graph with 15 communities simulated from the TGGP model.
Revised: MCMC sampler validation: We evaluate the proposed posterior inference method on a graph with 15 communities simulated from the TGGP model.

Original: The simulated graph had 1571 undirected, community-concordant edges, N = 386 observed nodes (i.e., with at least one community-concordant edge), and N α − N =192 unobserved nodes (i.e., with only community-discordant or self edges).
Revised: The simulated graph had 1571 undirected, unthinned edges, N = 386 observed nodes (i.e., with at least one unthinned edge), and N α − N =

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Revised: unobserved nodes (i.e., with only thinned or self edges).

Original: Fig.
Revised: Fig.

Original: 4 (top) shows traceplots from three separate MCMC chains, each of which was run for 20, iterations.
Revised: 4 (top) shows traceplots from three separate MCMC chains, each of which was run for 20,000 iterations.

Original: The traceplot of the total number of nodes (observed + unobserved) N α shows all values sampled since initialization to demonstrate that even very different initializations led the three MCMC chains to quickly concentrate around the same (true) value of N α .
Revised: The traceplot of the total number of nodes (observed + unobserved) N α shows all values sampled since initialization to demonstrate that even very different initializations led the three MCMC chains to quickly concentrate around the same (true) value of N α .

Original: The traceplots of the GGP hyperparameters α, σ , and τ show samples after convergence (which took roughlyiterations) and demonstrate that the posterior distribution effectively concentrated around the true values of these hyperparameters.
Revised: The traceplots of the GGP hyperparameters α, σ , and τ show samples after convergence (which took roughly 1000 iterations) and demonstrate that the posterior distribution effectively concentrated around the true values of these hyperparameters.

Original: Fig.
Revised: Fig.

Original: 4 (bottom) shows summaries of the posterior distributions of nodes’ sociabilities for 40 randomly selected nodes spanning from low to relatively large sociability, suggesting that the TGGP model also effectively recovers nodes’ sociability parameters.
Revised: 4 (bottom) shows summaries of the posterior distributions of nodes’ sociabilities for 40 randomly selected nodes spanning from low to relatively large sociability, suggesting that the TGGP model also effectively recovers nodes’ sociability parameters.

Original:  Experimental Results 5.1 Simulation
Revised:  Experimental Results 5.1 Simulation

Original: We discuss a simulation study where we investigated the performance of our proposed TGGP model, and the CGGP model of [29], based on simulated data generated from either model.
Revised: We discuss a simulation study where we investigated the performance of our proposed TGGP model, and the CGGP model of [29], based on simulated data generated from either model.

Original: First, we present the results from a sparse graph with 15 communities, simulated from our TGGP model by setting α = 250 , σ = 0 .
Revised: First, we present the results from a sparse graph with 15 communities, simulated from our TGGP model by setting α = 250 , σ = 0 .

Original: 1 , τ = 1 for the distribution of node sociabilities and γ = 10 , ζ = 0 .
Revised: 1 , τ = 1 for the distribution of node sociabilities and γ = 10 , ζ = 0 .

Original: 2 for the distribution of node community memberships.
Revised: 2 for the distribution of node community memberships.

Original: The adjacency matrix of the resulting undirected graph is shown in Figure 5(a, bottom).
Revised: The adjacency matrix of the resulting undirected graph is shown in Figure 5(a, bottom).

Original: Figure 5(b, bottom) sorts the simulated nodes into blocks according to their main membership and plots the density of edges in each block, demonstrating that the simulated graph has a clear block structure.
Revised: Figure 5(b, bottom) sorts the simulated nodes into blocks according to their main membership and plots the density of edges in each block, demonstrating that the simulated graph has a clear block structure.

Original: We then run our MCMC sampler for 50, iterations, discarding the first 40,000 samples as burn-in.
Revised: We then run our MCMC sampler for 50, iterations, discarding the first 40,000 samples as burn-in.

Original: For our model fitting, we let 50 be the upper bound to the number of communities and we set γ = 10 and ζ = 0 .
Revised: For our model fitting, we let 50 be the upper bound to the number of communities and we set γ = 10 and ζ = 0 .

Original: 5 to allow for learning the number of communities.
Revised: 5 to allow for learning the number of communities.

Original: From a qualitative comparison of the (b) and (c) bottom panes in Figure 5, we see that the community memberships recovered from our model are close to the underlying truth.The credible intervals indicate that the posterior distributions of all parameters tend to concentrate around the true values used for simulation (see Appendix for plots).
Revised: From a qualitative comparison of the (b) and (c) bottom panes in Figure 5, we see that the community memberships recovered from our model are close to the underlying truth.

Original: In contrast, the CGGP model [29] struggles to recover the true community structure of the network (Fig.
Revised: In contrast, the CGGP model [29] struggles to recover the true community structure of the network (Fig.

Original: 5(d bottom))
Revised: 5(d bottom))

Original: , despite being fitted with the number of communities K = 15 set to match the truth.
Revised: , despite being fitted with the number of communities K = 15 set to match the truth.

Original: We also simulated a graph from the CGGP model of [29] with parameters a k = 1 /K, b k = 1 chosen to induce a community membership prior similar to our TGGP.
Revised: We also simulated a graph from the CGGP model of [29] with parameters a k = 1 /K, b k =chosen to induce a community membership prior similar to our TGGP.

Original: We fit both the CGGP model and the TGGP model to the generated data.
Revised: We fit both the CGGP model and the TGGP model to the generated data.

Original: When fitting with the CGGP we still set the number of communities equal to the truth ( K = 15 ).
Revised: When fitting with the CGGP we still set the number of communities equal to the truth ( K = 15 ).

Original: The results plotted in Fig. 5(d) indicate that the block structures learned by our (misspecified) TGGP model are nevertheless closer to truth than those recovered by the CGGP.
Revised: The results plotted in Fig. 5(d) indicate that the block structures learned by our (misspecified) TGGP model are nevertheless closer to truth than those recovered by the CGGP.

Revised: Additional results (shown in Appendix B) suggest that the parameters that represent the relative size of communities across the network are also more difficult to identify in the

Original:  To quantify the superior community recovery of our TGGP model, recall that for each observed node the CGGP estimates a K -dim.
Revised: CGGP than in our TGGP formulation. To quantify the superior community recovery of our TGGP model, recall that for each observed node the CGGP estimates a K -dim.

Original: positive real vector of community-specific sociabilities ( w i 1 , . . .
Revised: positive real vector of community-specific sociabilities ( w i 1 , . . .

Original: , w iK ) , while our TGGP infers a
Revised: , w iK ) , while our TGGP infers a

Original: K -dim.
Revised: K -dim.

Original: probability vector of community memberships ( π i 1 , . . . , π iK ) .
Revised: probability vector of community memberships ( π i 1 , . . . , π iK ) .

Original: For quantitative comparison, we standardize true and inferred sociability vectors by dividing each w ik by the sum (cid:80)
Revised: For quantitative comparison, we standardize true and inferred sociability vectors by dividing each w ik by the sum (cid:80)

Original: Kh =1 w ih .
Revised: Kh =1 w ih .

Original: Also, since the TGGP model was fitted with K set to an upper-bound ofrather than to the true number of communities, when analyzing TGGP results we pad the true vectors of community memberships (for TGGPsimulated data) and sociabilities (for CGGPsimulated data) with zeros.
Revised: Also, since the TGGP model was fitted with K set to an upper-bound ofrather than to the true number of communities, when analyzing TGGP results we pad the true vectors of community memberships (for TGGPsimulated data) and sociabilities (for CGGPsimulated data) with zeros.

Original: For estimated parameters, community indices are then permuted to maximize similarity with true parameters.
Revised: For estimated parameters, community indices are then permuted to maximize similarity with true parameters.

Original: Letting ψ i and ˆ ψ i be, respectively, true and inferred
Revised: Letting ψ i and ˆ ψ i be, respectively, true and inferred

Original: community memberships/sociabilities, we define similarity as 1 − 1 N (cid:80) Ni =1 d ( ψ i , ˆ ψ i ) where the distance is the L1 total variation d ( ψ i , ˆ ψ i ) =
Revised: community memberships/sociabilities, we define similarity as 1 − 1 N (cid:80) Ni =1 d ( ψ i , ˆ ψ i ) where the distance is the L1 total variation d ( ψ i , ˆ ψ i ) =

Original: 12 (cid:80)
Revised: 12 (cid:80)

Original: Kk =1 | ψ ik − ˆ ψ ik | .
Revised: Kk =1 | ψ ik − ˆ ψ ik | .

Original: Fig.
Revised: Fig.

Original: 6 confirms that the TGGP more accurately recovers community memberships for both networks, and for nodes of both high and low degree.
Revised: 6 confirms that the TGGP more accurately recovers community memberships for both networks, and for nodes of both high and low degree.

Original: 5.2 Real network data
Revised: 5.2 Real network data

Original: We compare the performance of our TGGP framework for adding overlapping communities to the GGP model with the approaches detailed in Sec. 2.2 and 2.1: the CGGP (sparse with mixed memberships) [29], the SBM-GGP model (sparse with single memberships) [28], and the more classical SBM (dense with single membership) [2] and MMSB (dense with mixed memberships) [5].
Revised: We compare the performance of our TGGP framework for adding overlapping communities to the GGP model with the approaches detailed in Sec. 2.2 and 2.1: the CGGP (sparse with mixed memberships) [29], the SBM-GGP model (sparse with single memberships) [28], and the more classical SBM (dense with single membership) [2] and MMSB (dense with mixed memberships) [5].

Original: We run 50,000 iterations of the five models’ MCMC on four real-world networks; see Appendix for data sources and pre-processing.
Revised: We run 50,000 iterations of the five models’ MCMC on four real-world networks; see Appendix for data sources and pre-processing.

Original: Each model was fit to fully observed data to learn node-specific parameters (e.g., sociabilities and community memberships) and community-interaction probabilities, using the values from the last MCMC iteration.
Revised: Each model was fit to fully observed data to learn node-specific parameters (e.g., sociabilities and community memberships) and community-interaction probabilities, using the values from the last MCMC iteration.

Original: Two different measures of posterior predictive accuracy [42] were then used to assess the goodness of model fit.
Revised: Two different measures of posterior predictive accuracy [43] were then used to assess the goodness of model fit.

Original: For the first evaluation measure, we randomly selected 5% of the entries equal to 1 in the adjacency matrix and set them to 0.
Revised: For the first evaluation measure, we randomly selected 5% of the entries equal to 1 in the adjacency matrix and set them to 0.

Original: We then compute the probability of being 1 of all entries that are equal to 0 in the modified adjacency matrix to assess how well each model can distinguish the entries that should in fact be 1.
Revised: We then compute the probability of being 1 of all entries that are equal to 0 in the modified adjacency matrix to assess how well each model can distinguish the entries that should in fact be 1.

Original: This edge-retrieval measure is motivated by common applications of network analysis to recommendation tasks (suggesting which edges that are not observed should be present).
Revised: This edge-retrieval measure is motivated by common applications of network analysis to recommendation tasks (suggesting which edges that are not observed should be present).

Original: Fig.
Revised: Fig.

Original: 7 (top) plots recall (proportion of ones recovered) on the horizontal axis, and F-score (geometric mean of recall and precision, which is the proportion of ones correctly predicted among all entries predicted as ones) on the vertical axis.
Revised: 7 (top) plots recall (proportion of ones recovered) on the horizontal axis, and F-score (geometric mean of recall and precision, which is the proportion of ones correctly predicted among all entries predicted as ones) on the vertical axis.

Original: This retrieval task is very challenging because, for sparse networks, the missing edges are a tiny proportion of all zero entries in the modified adjacency matrix.
Revised: This retrieval task is very challenging because, for sparse networks, the missing edges are a tiny proportion of all zero entries in the modified adjacency matrix.

Original: For the second evaluation measure, we randomly obscure 5% of all entries in the adjacency matrix to assess how well each model can recover whether an entry is equal to 0 or to 1.
Revised: For the second evaluation measure, we randomly obscure 5% of all entries in the adjacency matrix to assess how well each model can recover whether an entry is equal to 0 or to 1.

Original: Fig.
Revised: Fig.

Original: 7 (bottom) plots the receiver operating characteristic (ROC) curves, where false positive rate (number of zeros predicted as ones divided by the number of zeros) is on the horizontal axis, and true positive rate (proportion of ones correctly predicted as ones divided by the number of ones) is on the vertical axis.
Revised: 7 (bottom) plots the receiver operating characteristic (ROC) curves, where false positive rate (number of zeros predicted as ones divided by the number of zeros) is on the horizontal axis, and true positive rate (proportion of ones correctly predicted as ones divided by the number of ones) is on the vertical axis.

Original: This second task has been used in many prior papers but is not as hard as the first one, because for a model to score well it suffices to learn small interaction probabilities, since almost all of the randomly selected entries are 0.
Revised: This second task has been used in many prior papers but is not as hard as the first one, because for a model to score well it suffices to learn small interaction probabilities, since almost all of the randomly selected entries are 0.

Original: Fig.
Revised: Fig.

Original: 7 shows that our TGGP model does consistently better than all baseline methods for all networks considered, according to both evaluation measures.
Revised: 7 shows that our TGGP model does consistently better than all baseline methods for all networks considered, according to both evaluation measures.

Original:  Discussion
Revised:  Discussion

Original: We have proposed a framework for the analysis of binary network data that extends the GGP-based model of Caron and Fox [25] by allowing for overlapping community structures, depending both on the overall sociability of the nodes and the similarity of their community memberships.
Revised: We have proposed a framework for the analysis of binary network data that extends the GGP-based model of Caron and Fox [25] by allowing for overlapping community structures, depending both on the overall sociability of the nodes and the similarity of their community memberships.

Original: Our generative model uses a novel latent multi-graph framework where nodes can connect within the same or across different communities, but the edges formed across different communities are hidden (thinned) in the projection giving rise to the observed network.
Revised: Our generative model uses a novel latent multi-graph framework where nodes can connect within the same or across different communities, but the edges formed across different communities are hidden (thinned) in the projection giving rise to the observed network.

Original: In contrast with alternative extensions of the original GGP network model [28, 29], our thinned generalized gamma process (TGGP) enables mixed memberships and facilitates regularization of the community distributions for each node, resulting in improved inference and reconstruction of the latent community structures.
Revised: In contrast with alternative extensions of the original GGP network model [28, 29], our thinned generalized gamma process (TGGP) enables mixed memberships and facilitates regularization of the community distributions for each node, resulting in improved inference and reconstruction of the latent community structures.

Original: Also, our model allows for learning the number of communities directly from the data, by encouraging recovery of a set of non-empty communities smaller than a specified upper bound.
Revised: Also, our model allows for learning the number of communities directly from the data, by encouraging recovery of a set of non-empty communities smaller than a specified upper bound.

Original: Monte Carlo inference for the TGGP scales linearly with the number of observed edges, and thus subquadratically with the number of nodes in sparse graphs.
Revised: Monte Carlo inference for the TGGP scales linearly with the number of observed edges, and thus subquadratically with the number of nodes in sparse graphs.

Original: This leads to strong edge prediction performance for social and biological networks of moderate size.
Revised: This leads to strong edge prediction performance for social and biological networks of moderate size.

Original: However, further inference innovations may be needed to scale to very-large networks with hundreds of thousands of nodes.
Revised: However, further inference innovations may be needed to scale to very-large networks with hundreds of thousands of nodes.

Original: The proposed TGGP model is amenable to further extensions.
Revised: The proposed TGGP model is amenable to further extensions.

Original: As the adopted mixed membership framework relies on the well-studied Dirichlet-multinomial specification, the vast literature on hierarchical Dirichlet mixtures can be leveraged to capture more complex networks.
Revised: As the adopted mixed membership framework relies on the well-studied Dirichlet-multinomial specification, the vast literature on hierarchical Dirichlet mixtures can be leveraged to capture more complex networks.

Original: It is possible to include additional node covariates or metadata to guide the allocation of nodes to communities.
Revised: It is possible to include additional node covariates or metadata to guide the allocation of nodes to communities.

Original: Finally, distributional and asymptotic properties of our model can be investigated by exploiting results related tofinite mixturesand mixtures of finite mixtures [see, e.g., 43, 44, 45, 34].
Revised: Finally, distributional and asymptotic properties of our model can be investigated by exploiting results related to mixtures

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: This research supported in part by NSF CAREER Award No. IIS1758028, and by the HPI Research
Revised: This research supported in part by NSF CAREER Award No. IIS1758028, and by the HPI Research

Original: Center in Machine Learning and Data Science at UC Irvine.A Monte Carlo Posterior Inference
Revised: Center in Machine Learning and Data Science at UC Irvine.

Original: We review the Gibbs sampling steps, some of which employ Metropolis-Hastings proposals rather than exact updates, for our thinned generalized gamma process (TGGP) model.

Original: We then describe each step in more detail.

Original: Gibbs sampling steps: Overview

Original: Update the edge counts and community assignments in the latent multigraph; 2.

Original: Update nodes’ community memberships; 3.

Original: Update the global expected value of community memberships; 4.

Original: Update nodes’ sociabilities; 5.

Original: Update the hyperparameters of the distribution of sociabilities; 6.

Original: Update the dimension of the latent multigraph.

Original: A.1 Updating edge counts and community assignments in the latent multigraph

Original: We start by recalling that, in our TGGP model, the observed simple undirected graph is a binary projection of an unobserved directed multigraph.

Original: For each edge in which a node participates in the multigraph, the node is assigned to a community.

Original: For a given (non-self) edge, if the communities that the two nodes are assigned to coincide, we say that the edge is community-concordant .

Original: Otherwise, if the communities differ, we say that the edge is community-discordant .

Original: We note there may be self-edges in the latent multigraph.

Original: However, since we assume that self-edges are not observed irrespective of community assignments, no distinction is needed between community-concordant and discordant self-edges and we refer to any of them simply as self-edges.

Original: Two nodes have an edge in the observed undirected graph if there is at least one community-concordant edge between those nodes in the multigraph.

Original: Observed nodes are those that participate in at least one community-concordant edge.

Original: The nodes that in the multigraph are connected only within communitydiscordant or self-edges are unobserved nodes .

Original: We call N the number of observed nodes and we let i = N + 1 , . . .

Original: , N α denote the indices of unobserved nodes.

Original: To easily sample from the posterior of nodes’ sociabilities we need (for both observed and unobserved nodes) each node’s total degree, which equals the total number of edges in the multigraph (both community-concordant and -discordant) in which the node participates.

Original: Also, to easily sample from the posterior of nodes’ community memberships, we need their community assignment for each of the edges that they are connected to in the multigraph (both community-concordant and -discordant).

Original: For both purposes, it suffices to track the entries of a single N α × K matrix of summary statistics M , where M ik equals the number of edges that node i participated in while being assigned to community k .

Original: The total degree of node i may then be obtained as M i · = (cid:80)

Original: Kk =1 M ik .

Original: We denote the counts of the assignments of a node i to each community (that is, row i of M ) by M i =

Original: ( M i 1 , . . . , M iK ) .

Original: We will break down the sampling of the entries of M in three steps: first edge counts and community assignments for community-concordant edges, second community-discordant and self-edges between observed nodes, and finally community-discordant and self-edges involving unobserved nodes.

Original: See

Original: Figure 8 for an illustration.

Original: A.1.1 Community-concordant edges

Original: Recall that we assumed that an edge between two nodes is observed if and only if there is at least one community-concordant edge between those nodes in the latent multigraph.

Original: Then, the presence of an edge between nodes i and j in the observed graph tells us that ˙ n ij + ˙ n ji > 0 , that is the sum of directed, community-concordant edges from i to j and from j to i in the latent multigraph is strictly positive.

Original: Similarly, the absence of an edge in the observed graph tells us that there is no community-concordant edge.

Original: Therefore, the number of community-concordant edges between two nodes only needs to be sampled for those pairs for which Y ij = 1 , ( i ≥ j ).

Original: An illustration is in Figure 8a.

Original: Because a priori , the number of community-concordant edges from i to j and from j to i have the same Poisson (cid:16) w i w j (cid:80)

Original: [-Kk =
-]
Revised: [-Kk =
-]

Original: π ik π jk (cid:17) distribution, a posteriori we have

Original: [Equation]

Original: By definition, the community assignment of the nodes involved in a community-concordant edge coincide.

Original: So we need to sample a single community assignment valid for both nodes i and j for each of the ˙ n ij + ˙ n ji community-concordant edges:

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: Multinomial

Original: [Equation]

Original: (cid:19)(cid:33) , and we increment M i =

Original: ( M i 1 , . . . , M iK ) and M j = ( M j 1 , . . . , M jK ) accordingly.

Original: Computational efficiency.

Original: Note that the computational cost of sampling community-concordant edges is linear in the number of observed edges.

Original: A.1.2 Community-discordant and self-edges between observed nodes

Original: Because community-discordant edges in the latent multigraph don’t affect the observed simple graph, all pairs of observed nodes can have community-discordant edges, both those pairs i and j for which

Original: Y ij = 1 and those pairs i ′ and j ′ for which

Original: Y i ′ j ′

Original: = 0 .

Original: We thus need to sample the total number ˚ n ij + ˚ n ji of community-discordant edges for all pairs i, j ∈ { 1 , . .

Original: , N } with i ≥ j (see Figure 8b).

Original: Because they are independent of the observed graph (conditional on nodes’ sociabilities and memberships), the posterior distribution of communitydiscordant edges is the same as their prior:

Original: [Equation]

Original: However, rather than considering every pair of nodes, we can leverage the properties of independent

Original: Poisson-distributed random variables to draw (˚ n ij + ˚ n ji ) for all pairs of nodes from the correct distribution in a more efficient way.

Original: Specifically, we can first sample an auxiliary variable ˜ N E from the prior distribution on the total number of edges and self-edges involving only observed nodes:

Original: [Equation]

Original: Then, we can proceed by associating each edge e = 1 , . . . , ˜ N E to a pair of nodes based only on nodes’ sociabilities, and assigning each sampled node to a community based only on its own memberships: x ev iid ∼

Original: [Equation]

Original: Finally, restricting to the resulting discordant-community edges and self-edges, we have obtained a sample for them from the correct distribution.

Original: Accordingly, we increment the counts ( M i 1 , . . .

Original: , M iK ) of community assignments for observed nodes i = 1 , . . .

Original: , N , i.e. the first N rows of M .

Original: [Figure]

Original: Computational efficiency.

Original: Note that, using the strategy described in this subsection, the computational cost for sampling community-discordant and self edges depends on the cost of assigning ˜ N E edges to a pair of nodes.

Original: Thanks to fast sampling techniques for sampling from discrete distributions like the alias method [46, 47, 48], this operation can be made to require only O ( N log N ) computational time for pre-processing, after which edge assignments x ev may be generated in constant time.

Original: A.1.3 Community-discordant and self edges involving unobserved nodes

Original: Let N α − N be the number of nodes that are active (have at least one connection) in the latent multigraph, but not in the observed simple graph.

Original: Let the labels of the nodes be ordered so that i = N + 1 , . .

Original: , N α correspond to the labels of these unobserved nodes.

Original: Conditional on these nodes not being observed but having connections in the latent multigraph, the information that we know is that each of them can only have community-discordant or self-edges, but also that they must be associated to at least one community-discordant or self-edge in the latent multigraph.

Original: This constraint implies that some care is required when sampling edges involving unobserved nodes.

Original: Specifically, we must generate the edge counts and associated nodes , that is simulate how many edges each unobserved node participates in, as well as what other nodes are associated to these edges.

Original: Given the nodes associated to an edge involving an unobserved node, community assignments can then be sampled quite easily, considering that they need to be discordant for non-self edges and that no restriction is applied to self-edges.

Original: Edge counts and associated nodes.

Original: To correctly incorporate the constraint that each unobserved node must be associated to at least one community-discordant or self-edge, the strategy that we adopt is to sample the total number of edges and the community assignments of unobserved nodes sequentially, starting from those involving node N + 1 with any other node (that is, the yellow entries in Figure 8c).

Original: Let s i , for i = N + 1 , . . .

Original: [-, N α , be an auxiliary indicator variable that takes value 
-]
Revised: [-, N α , be an auxiliary indicator variable that takes value 
-]

Original: if node i has been associated to an edge already and 1 otherwise.

Original: The variable s i thus indicates if the constraint that node i must be associated to at least one edge is still active ( s i = 1 ) or is inactive ( s i = 0 ).

Original: As we start from node ( N + 1) , all constraints are active and so s i = 1 for all

Original: i = N + 1 , . . . , N α .

Original: We then sample the total number of edges ˚ n ( N +1) between node ( N + 1) and any other node from

Original: [Equation]

Original: ˚ p ( N +1) =

Original: N (cid:88)

Original: [Equation]

Original: =

Original: [Equation]

Original: ( N

Original: [Equation]

Original: N α (cid:88)

Original: j = N + ˚ p ( N +1) j , ˚ p ( N +1)

Original: j = w j (cid:32) 1 −

Original: K (cid:88) k

Original: = π

Original: ( N +1)

Original: k π jk (cid:33) .

Original: Given the total number of edges associated with node ( N + 1) , we set x ( N +1) e= i and associate another node (possibly node N + 1 itself) to each edge e = 1 , . . .

Original: , ˚ n ( N +1) , according to

Original: [Equation]

Original: We then set s ( N +1)

Original: [-= 
-]
Revised: [-= 
-]

Original: and also s i = 0 for every node i ∈ { N +2 , . . .

Original: , N α } that has been associated to some edges with node ( N + 1) .

Original: Next, we can move on to sampling all the remaining edges involving node N + 2 , that is those with any other node except node N + 1 ( the light green entries in Figure 8c).

Original: In general, when we sample edges for node ( N + d ) for d = 2 , . . .

Original: , ( N α − N ) , there are two considerations we must keep into account: (1) the edges between node ( N + d ) and nodes ( N + 1) , . . .

Original: , ( N + d − 1) have already been sampled and so those nodes should not be re-considered; (2) if s ( N + d ) = 0 , the constraint on node ( N + d ) is no longer active and we may possibly sample 0 additional edges.

Original: Putting these considerations together and letting ˚ n ( N + d ) be the number of remaining edges to sample when considering node ( N + d ) , that is those with nodes { 1 , . . .

Original: , N, N + d, . .

Original: . , N α } , we have ˚ n ( N + d ) | ( w 1 , . .

Original: . , w N α ) , ( π 1 , . .

Original: , π N α ) , s ( N + d ) ∼   0-Trunc-Poisson (cid:16) 2 w ( N + d ) ˚ p ( N + d ) + w 2( N + d ) (cid:17) if s ( N + d ) = 1 ,

Original: Poisson (cid:16) 2 w ( N + d ) ˚ p ( N + d ) + w 2( N + d ) (cid:17) if s ( N + d ) = 0 , where ˚

Original: [Equation]

Original: Then for e = 1 , . .

Original: . , ˚ n ( N + d ) , we set x ( N + d ) e= N + d and sample

Original: P ( x ( N + d ) e

Original: [Equation]

Original: ∝

Original: [Equation]

Original: ˚

Original: [Equation]

Original: N + d )

Original: [Equation]

Original: , if j = 1 , . . . , N, N + d + 1 , . .

Original: . , N α , w i if j = N + d.

Original: Community assignments.

Original: For every node i = N + 1 , . . .

Original: , N α with ˚ n i > 0 , and for each e = 1 , . .

Original: ˚ n i , we sample a community assignment c ( i ) e for node i given x ( i ) e 2 from

Original: [Equation]

Original: for k = 1 , . . .

Original: , K .

Original: For non-self-edges, we rescale π ik accounting for the knowledge that the community to which the other node associated to edge e is assigned needs to differ from that to which node i is assigned for this edge.

Original: We increment M ic ( i ) e accordingly, and then given c ( i ) e and x ( i ) e 2 , we can sample a community assignment ¯ c ( i ) e for node x ( i ) e 2 from

Original: P

Original: [Equation]

Original: i

Original: [Equation]

Original: and then increment M x ( i ) e 2 ¯ c ( i ) e .

Original: Notice that for non-self-edges, we have removed the community sampled for node i from the range of available communities for the assignment of node x ( i ) e 2 , and rescaled π x ( i ) e 2 accordingly.

Original: Computational efficiency.

Original: In our TGGP model, unobserved nodes will have low sociability parameters; otherwise, they would be likely to be linked to unthinned edges.

Original: Thus ˚ n ( N + d ) tends to be small for all d = 1 , . .

Original: , N α − N .

Original: For sampling their edges and community assignments, it is just necessary to compute a ( N + 1)

Original: × N α matrix ˚ P whose entries ˚ p ij represent the relative probability of associating node j to an edge sampled for node i .

Original: A.2 Updating nodes’ community memberships

Original: Here we describe the update of π i =

Original: ( π i 1 , . . . , π iK ) , the distribution over community memberships of node i = 1 , . . . , N α .

Original: Recall the prior on π i :

Original: [Equation]

Original: where β = ( β 1 , . .

Original: . , β K ) is a vector of community frequencies that itself has a Dirichlet prior, and equals the expected value of π i .

Original: For our experiments we set ζ = 0 .

Original: 5 , to encourage community memberships of individual nodes to be more polarized than β and typically only use a subset of the available communities.

Original: For each edge in the latent multigraph to which node i is associated, the generative model has node i being assigned to a community sampled independently from (1 , . . .

Original: , K ) with probabilities ( π i 1 , . . .

Original: , π iK ) , like in multinomial sampling.

Original: The Dirichlet and the multinomial are conjugate distributions, so the posterior of π i is easily obtained.

Original: Indeed, conditional on the concentration parameter β

Original: and the N α × K matrix M whose rows record the number of times M i =

Original: ( M i 1 , . . .

Original: M iK ) that an edge i = 1 , . .

Original: , N α was assigned to each community k = 1 , . . .

Original: , K , the posterior distribution of the vectors of community memberships are obtained as:

Original: [Equation]

Original: An advantage of this specification is that it allows sparse community memberships, in that the sample ( π i 1 , . . .

Original: , π iK ) may have elements that are (almost) exactly equal to zero.

Original: A.3 Updating the global expected value of community memberships

Original: The posterior distribution of the expected value β = ( β 1 , . .

Original: . , β K ) of nodes’ community memberships informs about summaries of the latent community structure of a network, such as the likely number of active communities and the relative size of different communities.

Original: Recall the prior on β :

Original: [Equation]

Original: where the value of γ controls the sparsity of the prior on β , with values of γ ≪ K being more sparsity-inducing.

Original: In our experiments, we fix γ = 10 but K = 50 .

Original: This hierarchical prior approaches the hierarchical Dirichlet process [6] in the limit as K → ∞ .

Original: Here we illustrate a strategy for sampling from the posterior of β = ( β 1 , . .

Original: . , β K ) that relies on auxiliary variables drawn according to the so-called “Chinese restaurant franchise” representation of a hierarchical Dirichlet model [6].

Original: We can represent each node i = 1 , . .

Original: . , N α as a restaurant, where every new community assignment for node i can be represented as a new customer entering restaurant i and ordering a dish (community) k ∈ { 1 , . .

Original: , K } .

Original: When a new customer enters, they decide to sit at an existing table t ∈ { 1 , . .

Original: , T i } with probability proportional to the number of customers currently sitting at that table (and eat the dish k t that was ordered by the first customer who sat at the table); or, they can start a new table T i + 1 with probability proportional to ζ , and pick a new dish k T i + from the menu according to β .

Original: Let t ie be the table chosen when customer e enters restaurant i , or equivalently the table chosen for the community assignment of node i for when edge e is associated to it.

Original: Then letting T ( e ) i be the number of tables already started at restaurant i when customer e enters, and letting q ( e )

Original: =

Original: ( q ( e ) i 1 , . .

Original: , q ( e ) iT ( e ) i ) be the number of customers already sitting at each of the tables when customer e enters:

Original: [Equation]

Original: Conditional on t ie and letting ˜ κ t be the dish served at table t , the dish κ ie ∈ { 1 , . .

Original: , K } ordered when customer e enters restaurant i is sampled according to

Original: [Equation]

Original: From the last sampling equation we see that the posterior distribution of β depends on how many tables (rather than customers) are being served each dish across all restaurants in the franchise.

Original: Let

Original: T ik be the number of tables serving dish k at restaurant i and let T • k = (cid:80) N α

Original: i =1 T ik be the total number of tables serving dish k across all restaurants in the franchise.

Original: The summary statistics that we need for the posterior of β is the vector T =

Original: ( T • 1 , . . .

Original: , T • K ) .

Original: We then sample, for every restaurant i = 1 , . . .

Original: , N α (i.e. for every node) and every dish k = 1 , . .

Original: , K (i.e. for every community), the number of tables T ik ≤ M ik occupied by the M ik customers eating dish k at restaurant i .

Original: Note that M ik is the entry in row i and column k of the N α × K matrix

Original: M sampled at step 1 (see Section A.1).

Original: Because the distribution of table arrangement is invariant to the order in which customers enter, we can assign one at a time each of the e = 1 , . . .

Original: , M ik customers eating dish k at restaurant i to either sit at an existing table ( t ike = 0 ) or to start a new table ( t ike = 1 ) according to:

Original: [Equation]

Original: where we have ζβ k since we are conditioning on the customer being served

Original: dish k

Original: and we weight ζβ k with respect to ( e − 1) because, when considering customer e , there are already ( e − 1) customers sitting at some table serving dish k .

Original: Thus, the chances that customer e sits at one of the existing tables is proportional to ( e − 1) .

Original: Finally, we set T ik =

Original: (cid:80) M

Original: ik e =1 t ike .

Original: After sampling the auxiliary variables, we can easily update β according to

Original: [Equation]

Original: A.4 Updating nodes’ sociabilities

Original: Let M i • =

Original: (cid:80) Kk =1 M ik be the total degree of node i (whose sampling was described in Section A.1).

Original: Conditional on the vector of total degrees ( M 1 • , . .

Original: , M N α • ) and on the hyperparameters α, σ and τ , the update of nodes’ sociabilities ( w 1 , . . .

Original: , w N α ) can be done via a Hamiltonian Monte Carlo step [49, 50] as detailed by Caron and Fox in Section 7.2 and in Appendix F.1 of [25].

Original: A.5 Updating hyperparameters of sociabilities’ distribution

Original: We assume the same improper priors on the hyperparameters α, σ , and τ as in [25]:

Original: [Equation]

Original: To update these hyperparameters we follow the approach that Caron and Fox describe in Section 7. and in Appendix F.2 in [25].

Original: A.6 Updating the dimension of the latent multigraph

Original: Even though our interest lies in the sociabilities and community memberships of observed nodes i = 1 , . .

Original: , N , to derive their exact posterior we need to also consider the unobserved nodes N +1 , . .

Original: . , N α , that is those nodes whose edges in the latent multigraph are all thinned, being all either discordantcommunity or self-edges.

Original: The distribution of the number of nodes N α with at least one edge in the latent multigraph is not tractable, and we thus resort to an approximate method for updating N α .

Original: Our update of N α is based on sampling Q multigraphs from the GGP prior evaluated at the current values of α, σ and τ .

Original: For simulated graph q , let N ( q ) be the number of nodes associated to at least one concordant-community edge, and N ( q ) α be the number of nodes associated to at least one edge (of any type).

Original: We then compute the average ratio r Q = 1 Q (cid:80) Qq =1 N ( q ) α N ( q ) and set the new value of

Original: N ( new ) α =

Original: Nr Q .

Original: If the new value N ( new ) α is smaller than the older value N ( old ) α , we keep the N ( new ) α unobserved nodes that currently have the largest sociabilities, along with their current community memberships.

Original: If N ( new ) α > N ( old ) α , we keep all of the current N ( old ) α unobserved nodes, sample the sociabilities for the remaining N ( new ) α − N ( old ) α nodes by resampling from the sociabilities of the current nodes, and assign community memberships from the prior evaluated at the current value of β .

Original: Our experiments on simulated data suggest that, by updating N α every 100 iterations and setting

Original: Q = 10 , this method allows MCMC chains to concentrate around the true value of N α and of the hyperparameters α, σ , and τ (see Figure 4 (top) in the manuscript).

Original: B Additional results on simulated data

Original: In Section 3 of the manuscript we observed that our TGGP specification is more regularized, and thus may be more easily identified, than the compound GGP formulation of [29].

Original: Recall that, in our specification, the community memberships for node i are a probability vector π

Original: i over K communities generated from π i |

Original: [Equation]

Original: iid ∼ Dirichlet (

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: In contrast, for the compound GGP the membership of node i in community k is modeled as

Original: [Equation]

Original: for

Original: [Equation]

Original: [Equation]

Original: N α , k = 1 , . .

Original: [Equation]

Original: [Equation]

Original: , where nodes’ baseline sociabilities ( w 10 , w 20 , . . . ) are drawn according to a generalized gamma process.

Original: In Section 5.1 of the manuscript, we noted how our results suggested that the TGGP model may be better at recovering the underlying community structure of both TGGP and CGGP simulated graphs.

Original: Here we provide additional insights into this comparison.

Original: Figures 9a and 9b show that, for a CGGP-simulated graph with 15 communities, a k = 1 / 15 and b k

Original: = 1 for all k = 1 , . .

Original: , 15 , the samples ( a 1 , . . .

Original: , a 15 ) from the posterior estimated fitting the CGGP model are not concentrated around the true value or 1 / 15 , and several among b 1 τ, . . .

Original: , b K τ (which according to [29] is better identifiable than b 1 , . . .

Original: , b K ) are considerably larger than their true value of 1 .

Original: Conversely, Figure 9c shows that the TGGP model fitted with K = 50 correctly concentrates β k around 0 for most k = 1 , . .

Original: , K and, for the remaining communities, it tends to draw samples around the true value of 1 / 15 (the simulated graph for the TGGP has 15 communities and a true value β k

Original: = 1 / 15 for each of them).

Original: > 0.

Original: < 0.

Original: 0.3 0400 26a k den s i t y (a) a k densities (CGGP [29]) > <    2.

Original: b k t den s i t y (b) b k τ densities (CGGP [29]) 0.

Original: 1 5 10 15 20 25 30 35 40 45 50 b k v a l ue

Original: [Figure]

Original: C Real-data networks: preparation and references

Original: The four real-data networks used in our experiments were pre-processed by extracting the main component for the networks, by removing self-edges and by making the observed network undirected.

Original: Summary information for each of these networks is displayed in Table 1.

Original: [Table]

Original: The networks Reed and Simmons were downloaded from https://archive.org/details/ oxford-2005-facebook-matrix , the network SmaGri was downloaded from https://www. cise.ufl.edu/research/sparse/matrices/Pajek/SmaGri.html , and the network Yeast was obtained from http://vlado.fmf.uni-lj.si/pub/networks/data/bio/Yeast/Yeast.htm .