Original: Thinned random measures for sparse graphs with overlapping communities
Revised: Thinned random measures for sparse graphs with overlapping communities

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Network models for exchangeable arrays, including most stochastic block models,generate dense graphs with a limited ability to capture many characteristics ofreal-world social and biological networks.
Revised: Network models for exchangeable arrays, including most stochastic block models,generate dense graphs with a limited ability to capture many characteristics ofreal-world social and biological networks.

Original: A class of models based on completelyrandom measures like the generalized gamma process (GGP) have recently addressed some of these limitations.
Revised: A class of models based on completelyrandom measures like the generalized gamma process (GGP) have recently addressed some of these limitations.

Original: We propose a framework for thinning edgesfrom realizations of GGP random graphs that models observed links by both nodes’overall propensity to interact, and on the similarity of node memberships withina large set of latent communities.
Revised: We propose a framework for thinning edgesfrom realizations of GGP random graphs that models observed links via nodes’overall propensity to interact, as well as the similarity of node memberships withina large set of latent communities.

Original: Our formulation allows us to learn the numberof communities from data, and enables efficient Monte Carlo methods that scalelinearly with the number of observed edges, and thus (unlike dense block models)sub-quadratically with the number of entities or nodes.
Revised: Our formulation allows us to learn the numberof communities from data, and enables efficient Monte Carlo methods that scalelinearly with the number of observed edges, and thus (unlike dense block models)sub-quadratically with the number of entities or nodes.

Original: We compare to alternativemodels for both dense and sparse networks, and demonstrate effective recovery oflatent community structure for real-world networks with thousands of nodes.
Revised: We compare to alternativemodels for both dense and sparse networks, and demonstrate effective recovery oflatent community structure for real-world networks with thousands of nodes.

Original:  Introduction
Revised:  Introduction

Original: Given observations of (often binary) relationships Y ij between pairs of nodes or entities ( i, j ) ,many relational models [1] seek to uncover an underlying set of communities.
Revised: Given observations of (often binary) relationships Y ij between pairs of nodes or entities ( i, j ) ,many relational models [1] seek to uncover an underlying set of communities.

Original: Classic stochasticblockmodels [2] generalize mixture models for clustering non-relational data by assigning each entityto one of K communities (clusters).
Revised: Classic stochasticblockmodels [2] generalize mixture models for clustering non-relational data by assigning each entityto one of K communities (clusters).

Original: The infinite relational model (IRM) [3] instead uses a Dirichletprocess prior [4] to partition entities into single communities.
Revised: The infinite relational model (IRM) [3] instead uses a Dirichletprocess prior [4] to partition entities into single communities.

Original: While the IRM allows the number ofcommunities to be inferred from data, later work has shown that real-world social networks are bettercaptured by models which allow individuals to participate in multiple communities [5], includingapplications of the hierarchical Dirichlet process (HDP)
Revised: While the IRM allows the number ofcommunities to be inferred from data, later work has shown that real-world social networks are bettercaptured by models which allow individuals to participate in multiple communities [5], includingapplications of the hierarchical Dirichlet process (HDP)

Original: [6] to relational data [7].
Revised: [6] to relational data [7].

Original: There is an extensive literature on descriptive statistics of biological and social networks [8, 9], including degree distributions, path distances and “small world” phenomena [10], community structures andmodularity and notions of centrality and causality.
Revised: There is an extensive literature on descriptive statistics of biological and social networks [8, 9], including degree distributions, path distances and “small world” phenomena [10], community structures andmodularity, and notions of centrality and causality.

Original: In particular, sparsity is a ubiquitous phenomenonin real-world networks [8, 9]: as network size grows, the number of edges grows more slowly than thequadratic number of node pairs.
Revised: In particular, sparsity is a ubiquitous phenomenonin real-world networks [8, 9]: as network size grows, the number of edges grows more slowly than thequadratic number of node pairs.

Original: However, the IRM and HDP relational models (and a large literatureof related models [1]) generate dense graphs where the number of edges scales quadratically withthe number of nodes.
Revised: However, the IRM and HDP relational models (and a large literatureof related models [1]) generate dense graphs where the number of edges scales quadratically withthe number of nodes.

Original: In fact, a classic representation theorem [11] shows that any generative modelwhich samples edges from independent Bernoulli distributions, given latent node-specific parameters,generates dense graphs; most existing probabilistic network models have this form.
Revised: In fact, a classic representation theorem [11, 12] shows that any generativemodel that regards graphs as exchangeable adjacency matrices, meaning arrays whose distributionis invariant to permutations of node indices, generates dense graphs. Most existing probabilisticnetwork models, which generate an adjacency matrix by sampling edges from independent Bernoullidistributions given latent node-specific parameters, have this limitation.

Original: By representing graphs as a latent process (a completely random measure), Caron and Fox [12]showed that it is possible to create generative models that capture the sparsity of real-world networks.
Revised: By representing graphs as a latent process (a completely random measure), Caron and Fox [13]showed that it is possible to formulate generative models that capture the sparsity of real-worldnetworks.

Original: Related models, including certain infinite limits of graphs called graphons , have been studied byseveral authors [13, 14, 15].
Revised: Related models, including certain infinite limits of graphs called graphons , have been studied by several authors [14, 15, 16].

Original: However, these models mostly produce homogeneous graphs withsparsity and heavy-tailed degree distributions, but lacking the community structure of real networks.
Revised: However, these models mostly produce homogeneous graphswith sparsity and heavy-tailed degree distributions, but lacking the community structure of realnetworks.

Original: Two notable exceptions are work by Herlau et al.
Revised: Two notable exceptions are work by Herlau et al.

Original: [16] and Todeschini et al.
Revised: [17] and Todeschini et al.

Original: [17] that augment therandom-measure models of [12] with variables encoding latent community structure.
Revised: [18] thataugment the random-measure models of [13] with variables encoding latent community structure.

Original: In this paper, we propose a novel random graph model that 
Revised: In this paper, we propose a novel random graph model that efficiently thins edges from sparsehomogeneous graphs to reveal community structure while maintaining sparsity. Unlike [17], we allowentities to have membership in multiple communities. Unlike [18], we allow completely flexiblespecification of the hierarchical Bayesian prior on latent community memberships; importantly, thisenables the network-specific learning of the appropriate number of latent communities. We furtherdevelop an efficient Monte Carlo inference algorithm that, unlike nearly all dense block models,scales linearly with the number of observed edges and thus sub-quadratically with the number ofentities.

Revised: Experiments show recovery of communities for networks with thousands of nodes.

Original:  Background: Stochastic Blockmodels for Dense and Sparse Networks
Revised:  Background: Stochastic Blockmodels for Dense and Sparse Networks

Original: An undirected binary network with N nodes and E edges may be represented by an N × N binaryadjacency matrix Y .Y ij = Y ji
Revised: An undirected binary network with N nodes and E edges may be represented by a symmetric N × Nadjacency matrix Y .

Original: = 1 if there is an edge (link) between nodes i ̸
Revised: If there is an edge (link) between nodes i ̸

Original: = j  , otherwise Y ij = 0 .
Revised: = j then Y ij = 1 , otherwise Y ij = 0 .

Original: 2.1 Mixed Membership Stochastic Blockmodels
Revised: 2.1 Mixed Membership Stochastic Blockmodels

Original: Stochastic blockmodels (SBMs) [2] assume that each node belongs to one of K latent communities,where the probability of an edge depends on how strongly their communities are connected.
Revised: Stochastic blockmodels (SBMs) [2] assume that each node belongs to one of K latent communities,and that the probability of an edge depends on how strongly their communities are connected.

Original: Letc i ∈ { 1 , . . .
Revised: The community c i ∈ { 1 , . . .

Original: , K }indicate the community of node i , where c i ind ∼ Cat ( β ) and β k is the frequency ofcommunity k .
Revised: , K } of node i follows a categorical distribution c i ind ∼ Cat ( β ) , whereβ = ( β 1 , . . . , β K ) so that β k is the frequency of community k .

Original: Edges arethen sampled independently as Y ij ind ∼ Bernoulli ( η c i c j ) , where η kℓ =
Revised: Edges are sampled independently as Y ij ind ∼ Bernoulli ( η c i c j ) , where η kℓ =

Original: η ℓkis the probability of an edge between a node in community k anda node in community ℓ .
Revised: η ℓk is the probability of an edge between nodes in communitiesk and ℓ .

Original: Theseinteraction probabilities are often assigned conjugate beta priors, η kℓ  ∼ Beta ( τ a , τ b ) .
Revised: These interaction probabilities are often assigned conjugate beta priors, η kℓ ind ∼ Beta ( τ a , τ b ) .

Original: Mixed membership stochastic blockmodels (MMSBs) [5] extend block models to allow nodes to bemembers of multiple communities.
Revised: Mixed membership stochastic blockmodels (MMSBs) [5] extend SBMs to allow nodes to be membersof multiple communities.

Original: Let π i =
Revised: Let π i =

Original: ( π i 1 , . . . , π iK ) denote a K -dimensional probabilityvector representing the strength of affiliation of node i to each of K communities.
Revised: ( π i 1 , . . . , π iK ) denote a K -dimensional probability vectorrepresenting the strength of affiliation of node i to each of K communities.

Original: For every pair ofnodes ( i, j ) , the communities governing their interaction are sampled as c ij  ∼ Cat ( π i ) , c ji  ∼ Cat ( π j ) .
Revised: For every pair of nodes( i, j ) , the communities governing their interaction are sampled as c ij ind ∼ Cat ( π i ) , c ji ind ∼ Cat ( π j ) .

Original: Then like standard SBMs, edges are sampled independently as Y ij ind ∼ Bernoulli ( η c ij c ji ) .
Revised: Then, like standard SBMs, edges are sampled independently as Y ij ind ∼ Bernoulli ( η c ij c ji ) .

Original: Communitymemberships are typically assigned a hierarchical prior, such as
Revised: Communitymemberships are typically assigned a hierarchical prior, such as

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: When fitting this model to data, we can treat K as an upper bound to the number of communities(that is, larger than the maximum number of communities that we expect to be necessary to modelthe network) and fix γ ≪ K .

Original: By setting K ≫ γ , the model allows network-specific learning of the number of communities byfavoring a sparse community frequency vector β  .
Revised: In this way, the hierarchical Dirichlet formulation allows networkspecific learning of the number of communities by favoring sparse community frequency vectorsβ = ( β 1 , . . . , β K ) with some elements β h ≈ 0 , h ∈ { 1 , . .

Original:  The hierarchical Dirichlet process [6, 7] is the limitof this prior as K approaches infinity [18].
Revised: . , K } . The hierarchical Dirichletprocess [6, 7] is the limit of this prior as K approaches infinity [19].

Original: 2.2 Sparse Network Models via Completely Random Measures
Revised: 2.2 Sparse Network Models via Completely Random Measures

Original: For any fixed community frequencies β and interaction probabilities η , the SBM and MMSB mayonly generate dense graphs where the number of edges scales quadratically with the number ofnodes [19].
Revised: For any fixed community frequencies β and interaction probabilities η , the SBM and MMSB mayonly generate dense graphs where the number of edges scales quadratically with the number ofnodes [12].

Original: In contrast, many real-world networks appear to be sparse [9].
Revised: In contrast, many real-world networks appear to be sparse [9].

Original: Heuristics are often usedto fit (mixed) SBMs to large but sparse networks, such as fixing (rather than learning) η kℓ = ε ≈ 
Revised: Heuristics are often usedto fit (mixed membership) SBMs to large but sparse networks, such as fixing (rather than learning)η kℓ = ε ≈ 0 for k ̸ = ℓ ; for example Kim et al.

Original: 0for k ̸ = ℓ ; Kim et al.

Original: [7] fix ε = e − 30 .
Revised: [7] fix ε = e − 30 .

Original: We seek to avoid such heuristics by building models thatsimultaneously capture sparsity and (mixed) membership in latent communities.
Revised: We seek to build models that cancapture mixed memberships and sparsity simultaneously without needing to rely on such heuristics.

Original: Completely Random Measures.
Revised: Completely Random Measures.

Original: By representing the graph as a point process on the plane, Caronand Fox [12] showed that it is possible to generate sparse graphs by appropriately choosing the meanmeasure of the point process.
Revised: By representing the graph as a point process on the plane, Caronand Fox [13] showed that it is possible to generate sparse graphs by appropriately choosing the meanmeasure of the point process.

Original: According to their model, for i ̸
Revised: According to their model, for i ̸

Original: = j ,
Revised: = j ,

Original: [Equation]
Revised: [Equation]

Revised: 1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent)

Original: {+GG P t h i nned GG P                         01    1      0 
 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple undirected graph observed)+}           where w i > 0 represents the sociability of node i : nodes with higher w i have higher probability tointeract with other nodes, and hence greater expected degree.
Revised: {+GG P t h i nned GG P                         01    1      0 
 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple undirected graph observed)+}           where w i > 0 represents the sociability of node i : nodes with higher w i have higher probabilityto interact with other nodes, and hence greater expected degree.

Original: Node sociabilities are generated as thejumps of a completely random measure (CRM) [20] at real-valued locations ℓ i uniformly distributedon an interval [0 , α ] ,
Revised: Differently from the (mixedmembership) SBMs where all parameters governing nodes’ interactions are sampled independently,here nodes’ sociabilities are generated altogether using the jumps of a completely random measure(CRM) Each node is also independently associated to a real-valued location ℓ i uniformlydistributed on the real line and then a set of potential nodes is defined by restricting to sociabilitieswith a sampled location in the interval [0 , α ] ,

Original: [Equation]
Revised: [Equation]

Original: Here α > 0 controls the (random) number of nodes N α in the network by determining the size ofthe interval in which jumps are included.
Revised: Here α > 0 controls the (random) number of nodes N α in the network by determining the size ofthe interval in which jumps are included.

Original: Depending on the distribution of the jumps, the model cancapture both sparse and dense graphs.
Revised: Depending on the distribution of the jumps, the model cancapture both sparse and dense graphs.

Original: Intuitively, as detailed in Sec.
Revised: Intuitively, as detailed in Sec.

Original: 5.1 of [12], for sparse networksthe distribution of the jumps needs to place almost all of its mass near zero.
Revised: 5.1 of [13], for sparse networksthe distribution of the jumps needs to place almost all of its mass near zero.

Original: This model requires CRMs for which the sum ¯ W α
Revised: This model requires CRMs for which the sum ¯ W α

Original: = (cid:80) i : ℓ i ∈ [0 ,α ] w i of the jumps in [0 , α ] is finite.
Revised: = (cid:80) i : ℓ i ∈ [0 ,α ] w i of the jumps in [0 , α ] is finite.

Original: The observed network is then generated via a binary projection of an underlying (directed) multigraph, 1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent) GG P t h i nned
Revised: A simple undirected network is then generated via a binary projection of an underlying directedmultigraph, where the total number of edges n ij from node i to node j is independently distributed asn ij | w i , w j ind ∼ Poisson ( w i w j ) .

Original: [-GG P                         01    1      0 
 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple-] undirected graph observed)           where the total number of edges n ij from node i to node j isindependently distributed as [Equation] We then set Y ij = 1 ( n ij + n ji ≥ 1) for nodes i ̸
Revised: [-GG P                         01    1      0 
 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple-]An edge is present in the undirected network if there is at least one directed edge in the latent directedmultigraph, that is Y ij = 1 ( n ij + n ji ≥ 1) for nodes i ̸

Original: = j .
Revised: = j .

Original: Because the sum of independent Poissonrandom variables is Poisson, Eq.
Revised: Because the sum of independent Poissonrandom variables is Poisson, Eq.

Original: (4) implies P ( n ij + n ji = 0)
Revised: (4) implies P ( n ij + n ji = 0)

Original: =
Revised: =

Original: exp {− 2 w i w j } , from which Eq.
Revised: exp {− 2 w i w j } , from which Eq.

Original: (2)follows.
Revised: (2) follows.

Original: The number N of nodes in the observed network then equals the number N α of nodes thathave at least one edge in the underlying multigraph.
Revised: The number N of nodes in the observed network then equals the number N α of nodes thathave at least one edge in the underlying multigraph.

Original: This construction of the binary matrix from amultigraph [12] is visualized in Fig.
Revised: This construction of the binary matrix from amultigraph [13] is visualized in Fig.

Original: The sum-property of the Poisson distribution also implies that, given ¯ W α , the total number of edges¯ D α in the multigraph has a Poisson ( ¯ W 2 α ) distribution.
Revised: The sum-property of the Poisson distribution also implies that, given ¯ W α , the total number of edges¯ D α in the multigraph has a Poisson ( ¯ W 2 α ) distribution.

Original: Since n ij has a high probability of being 0for most node pairs ( i, j ) , it is more efficient to first sample ¯ D α, and then independently assign eachedge to a pair of nodes based on their sociabilities.
Revised: Since n ij has a high probability of being 0 formost node pairs ( i, j ) , it is more efficient to first sample ¯ D α and then independently assign each edgeto a pair of nodes based on their sociabilities.

Original: In more detail, Eq.
Revised: In more detail, Eq.

Original: (4) is equivalent 
Revised: (4) is equivalent to

Original: to¯ D α | ¯ W α ∼ Poisson( ¯ W 2 α ) ,
Revised: [Equation] Poisson

Original: [Equation]
Revised: [Equation]

Original:  for v = 1 , 2 and e = 1 , . .
Revised: where x ev ∈ { 1 , 2 , . . . } for v = 1 , 2 indicates the nodes sampled for edges e ∈ { 1 , . .

Original: , ¯ D α  .
Revised: , ¯ D α } .

Original: Caron and Fox [12] propose the generalized gamma process(GGP)
Revised: Caronand Fox [13] propose the generalized gamma process (GGP)

Original: [21, 22, 23] as a flexible but tractable CRM for W α , with parameters τ ∈ (0 , ∞ ) , andσ ∈ ( −∞ , 0] for densely graphs or σ ∈ (0 , 1) for sparse graphs.
Revised: [21, 22, 23] as a flexible but tractable CRM for W α , with parameters τ ∈ (0 , ∞ ) , and σ ∈ ( −∞ , 0] for dense graphs or σ ∈ (0 , 1) forsparse graphs.

Original: We summarize this generativeprocess via a directed graphical model in Fig. 2, and a simulated network is visualized in Fig. [Figure]
Revised: Fig. 2 (black component) provides a visual summary of the generative processunderlying the construction of a GGP random graph, using a directed graphical model that highlightsthe conditional dependencies (and independencies) between its variables.

Original: c [Figure]
Revised: Figure 3a shows a networksampled from the GGP model by means of a node-edge diagram.

Original: Sparse Block Models.
Revised: Sparse Block Models.

Original: A limitation of the framework described above is that it does not modelthe community (block) structure of the network – a well-recognized feature of complex networks.
Revised: A limitation of the framework described above is that it does not modelthe community (block) structure of the network – a well-recognized feature of complex networks.

Original: Herlau et al.
Revised: Herlau et al.

Original: [16] generalized the approach in [12] to accommodate both sparse and dense networkswith community structure.
Revised: [17] generalized the approach in [13] to accommodate both sparse and dense networkswith community structure.

Original: More specifically, they introduce a latent assignment indicator, c i ∈{ 1 , . .
Revised: Specifically, they introduce a latent discrete variable c i ∈ { 1 , . .

Original: , K } , to indicate the assignment of node i to one of K communities.
Revised: , K }to indicate the assignment of node i to one of K communities, like in the SBMs.

Original: A bivariate CRMincorporates both the sociability weights and a set of parameters, say η c i c j , capturing the interactionstrength between two communities c i and c j in the underlying multigraph.
Revised: A bivariate CRMincorporates both the sociability weights and a set of parameters, denoted by η c i c j , capturing theinteraction strength between two communities c i and c j in the underlying multigraph.

Original: As a result of theirformulation, the total number of edges n ij from node i to node j are independently distributed asn ij | c
Revised: For theirformulation, the total number of edges n ij from node i to node j are independently distributed asn ij | c

Original: i , c j , w i , w j ind ∼ Poisson ( η c i c j w i w j ) , and the likelihood  (2) of the observed nodes is modifiedas Y ij | c i , c j , w i , w j ind ∼ Bernoulli (1 − e − 2 η cicj w i w j ) . 
Revised: i , c j , w i , w j ind ∼ Poisson ( η c i c j w i w j ) , and the likelihood from Eq. (2) of the observed nodesis modified as Y ij | c i , c j , w i , w j ind ∼ Bernoulli (1 − e − 2 η cicj w i w j ) . We refer to this approach as thestochastic block model generalized gamma process (SBM-GGP).

Original: Sparse Mixed Membership.
Revised: Sparse Mixed Membership.

Original: The model proposed by Herlau et al.

Original: [16] accommodates sparse anddense networks with community structure but does not model overlapping community structures,i.e. their formulation results in a network partitioned into disjoint communities.
Revised: The SBM-GGP models sparse and dense networks with communitystructure, but does not capture overlapping community structures; it induces networks whose nodesare partitioned into disjoint communities.

Original: In follow-upwork, Todeschini et al.
Revised: In follow-up work, Todeschini et al.

Original: [17] extended the CRM-based framework discussed above by associatinga vector ( w i 1 , .. .
Revised: [18] extended the CRM-based framework discussed above by associating a vector ( w i 1 , .

Original:  , w iK ) of sociabilities to each node i , to represent different levels of affiliationof a node to thelatent communities k = 1 , . . , K .
Revised: . . , w iK ) of sociabilities toeach node i , to represent different levels of affiliation of nodes to the K latent communities.

Original: A node may have high levels of affiliationto more than a community, leading to the formation of edges across multiple communities.
Revised: A nodemay have high levels of affiliation to more than a community, leading to the formation of edges acrossmultiple communities.

Original: Thevectors of nodes’ sociabilities are distributed according to a compound CRM [24], specifically in theirimplementation they use a compound generalized gamma process (CGGP).
Revised: The vectors of node sociabilities are distributed according to a compound CRM [24]; specifically, their implementation uses a compound generalized gamma process (CGGP).

Original: The likelihood functionis modified accordingly by specifying Y ij | w i  w j ind ∼ Bernoulli (1 − exp {− 2 (cid:80) 
Revised: The likelihood function is modified accordingly, by independently sampling undirected edges as Y ij | w i , w j ind ∼ Bernoulli (1 − exp {− 2 (cid:80) Kk

Original: Kk =1{+=
+} w ik w jk } ) .
Revised: {+=
+} w ik w jk } ) .

Original: The latent community weights are further modeled as w ik = w i 0 β{+
 ψ+} ik , where w i 0 is defined via ageneralized gamma process and β ik ind ∼ Gamma ( a k , b k ) .
Revised: The latent community weights are modeledas w ik = w i {+
 ψ+} ik , where w i 0 are sampled from a GGP as in Eq. (3), and ψ ik are gamma-distributedrandom variables ψ ik ind ∼ Gamma ( a k , b k ) whose shape a k and rate b k may be inferred from data.

Original: Thus, as [17] point out, the  model exploitsan (unconstrained) non-negative matrix factorization to define the Bernoulli probability link.
Revised: As [18] point out, the CGGP model exploits an (unconstrained) non-negative matrix factorization todefine the Bernoulli probability link.

Original: In the Appendix, we show how this lack of constraints could hamper the ability of the  model to recover thetrue community structure of the network.
Revised: In the Appendix, we demonstrate that this lack of constraintshampers the ability of the CGGP model to recover the true community structure of the network.

Original: On the contrary, the regularization approaches to non-matrixfactorization should improve identification of underlying structures and identifiability of the networkparameters [25].
Revised: Better regularized approaches to non-negative matrix factorization, like the one we propose, mayimprove the identifiability of latent community structures and network parameters [25].

Original:  The thinned CRM model
Revised:  The Thinned CRM Network Model

Original: In this work, we explore introducing overlapping community memberships in the innovative GGPmodel by Caron and Fox [12] as vectors of probabilities sampled from a hierarchical Dirichletdistribution.
Revised: We now add overlapping community memberships to the GGP network model [13] via vectors ofprobabilities sampled from a hierarchical Dirichlet distribution.

Original: Compared to the formulation in Todeschini et al.
Revised: Unlike the formulation of Todeschiniet al.

Original: [17], our model enables learning thenumber of communities from the data.
Revised: [18], our model enables learning the number of communities from data (see Secs. 2.1 and 4).

Original: Moreover, using probability vectors rather than unconstrainednon-negative values to model community memberships, our model provides a regularized approach toinference in the GGP model, which appears to provide an increased accuracy in community detectionin simulations.
Revised: By using probability vectors rather than unconstrained non-negative values to model communitymemberships, our model provides a regularized approach to inference in the GGP model, which insimulations (Sec. 5.1) provides a substantial increase in community detection accuracy. ∞ ¯ D α N × N w i

Original:  Let a node i have both a sociability parameter w i from the GGP as in (3) and a vector of probabilitiesπ i =
Revised: α σ τ π i ζ β γ K x ex ec ec e Y ij Let each node i have both a sociability parameter w i from the GGP as in (3), and a vector ofprobabilities π i =

Original: ( π i 1 , . . . , π iK ) drawn from a hierarchical Dirichlet distribution as in (1).
Revised: ( π i 1 , . . . , π iK ) drawn from a hierarchical Dirichlet distribution as in (1).

Original: Moreover, let thenumber of potential edges between nodes i and j in the latent multigraph depend only on theirsociabilities, as in (4).
Revised: Moreover,let the number of potential edges between nodes i and j in the latent multigraph depend only on theirsociabilities, as in (4).

Original: For each of their n ij potential edges, nodes i  and  j are each assigned [Figure]
Revised: For each of the n ij potential edges, node i independently samples a communityfrom Cat ( π i ) , and node j samples a community from Cat ( π j ) .

Original: to a community according to their respective membership probabilities: only if the assignmentscorrespond to the same community, the edge is retained, otherwise it is thinned (i.e., discarded).
Revised: If these community assignmentsmatch, the edge is retained; otherwise, it is thinned (i.e., discarded).

Original: See 
Revised: See Figs.

Original: Figures 1 and 3 for an illustration.
Revised: 1 and 3 for examples.

Original: In formulas, for every edge e = 1 , . . , ¯ D α in the multigraph, thenumber of  edges between a pair of nodes i, j that is retained in the  multigraph ˙ n ij and the existenceof an observed edge Y ij in the undirected graph is determined by
Revised: More formally, let ˙ n ij be the number of multigraph edges between a pair of nodes i, j that is retained(not thinned). Edges e ∈ { 1 , . . . , ¯ D α } in the GGP multigraph are stochastically thinned as follows:

Original: [Equation]
Revised: [Equation]

Original: 7) where ˙ n ij (full dot) indicates the number of (directed) edges from i to j in the latent multigraphthat are retained, and x ev are indicators in { 1 , 2 , . . . 
Revised: Here x ev ∈ { 1 , 2 , . . . } for v = 1 , 2 indicate the nodes associated with edge e as in Eq. Anedge Y ij is then present in the undirected graph if and only if at least one multigraph edge is notthinned.

Original: } as in (5), v = 1 , 2 .
Revised: Equivalently, Eq.

Original: According to (7), weinterpret an observed simple, undirected, graph as obtained from a binary projection of the possiblymultiple retained edges generated in the latent multigraph.
Revised: (7) implies that the observed undirected graph is a binary projection of themultigraph edges that are selected , i.e. retained after the thinning process.

Original: The probability of two nodes beingassigned to the same community is P ( c ei = c ej )=
Revised: To determine the distribution of Y ij , note that the probability that nodes i and j are assigned to thesame community (marginalizing across communities) equals P ( c ei = c ej )

Original:  (cid:80)
Revised: = (cid:80)

Original: Kk =

Revised: Kk =

Original: π ik π jk after marginalizing acrosscommunities.
Revised: π ik π jk .

Original: Therefore,  ˙ n ij | w i , w j , π i , π j ind ∼ Poisson ( w i w j (cid:80) Kk =1 π ik π jk ) , marginalizing overthe latent multigraph.
Revised: Therefore, marginalizing over the latent multigraph, ˙ n ij | w i , w j , π i , π j ind ∼ Poisson ( w i w j (cid:80) Kk =1 π ik π jk ) .

Original: Thus, the likelihood of the observed network is obtained obtained as
Revised: The likelihood of the observed network thus equals

Original: [Equation]
Revised: [Equation]

Original: where we see that, differently than in the MMSB or in the GGP framework, our  model favors edgesbetween nodes that have both large sociabilities and similar community memberships.
Revised: Unlike the MMSB and GGP models, our thinned generalized gamma process (TGGP) model favorsedges between nodes that have both large sociabilities and similar community memberships.

Original: The  model issummarized graphically in Figure 2 andis illustrated in Figures 4c, 4d and 1.
Revised: The TGGP model is summarized graphically in Fig. 2 and illustrated in Figs. 1, 3, 4a.

Original:   Posterior Inference
Revised:  Monte Carlo Posterior Inference

Original: The formulation of the model as thinning of a multigraph sampled only according to the sociabilityparameters of the nodes (eq.
Revised: In our TGGP model, the latent (unthinned) multigraph has a distribution (Eq. that depends onlyon the node sociability parameters w i .

Original: (5) and (7)) leads to posterior distributions of ( w 1 , w 2 , . . . )
Revised: The thinning process (Eq. (7)) then depends only on thecommunity membership vectors

Original: and( π 1 , π 2 , . . . )that are conditionally independent given the total number of proposed nodes n ij betweenany two edges i and j in the multigraph.
Revised: π i . As can be deduced from the Markov properties of the directedgraphical model in Fig. 2, given these latent multigraphs, the posterior distributions of ( w 1 , w 2 , . . . ) and ( π 1 , π 2 , . . . ) are conditionally independent.

Original: This characteristic of our model allows us to apply Theorem6 in [12] to derive the posterior of nodes’ sociabilities, provided we can condition on the (latent)multigraph.
Revised: This design simplifies posterior inference, andin particular allows us to apply Theorem 6 of [13] to derive the posterior of nodes’ sociabilities,provided we can condition on the (latent) multigraph.

Original: Therefore, we implement a Gibbs sampling strategy for posterior inference, wherewe sample both thinned and retained edges in the latent multigraph as well as their communityassignments to allow for efficient updates of the variables of interest, ( w 1 , w 2 , . . . ) and ( π 1 , π 2 , . . . ) .
Revised: We thus implement a Gibbs sampling strategyfor posterior inference where we sample both thinned and retained edges in the latent multigraph, aswell as their community assignments, to allow for efficient updates of other variables.

Original: Some steps in the sampler are straightforwardly derived, considering that we can borrow the approachdetailed in [12] to update nodes’ sociabilities and GGP hyperparameters (see the Appendix foran outline of the sampling scheme).
Revised: Some steps in the sampler are straightforwardly derived, considering that we can borrow the approachdetailed in [13] to update node sociabilities and GGP hyperparameters (see the Appendix for a [Figure]

Original:  However, the implementation of the sampler requires someadditional careful development – which we detail below – in sampling the latent multigraph, dueto the latent thinning process.
Revised: detailed outline). However, the implementation of the sampler requires some additional carefuldevelopment – which we detail below – to sample the latent multigraph, due to the unobservedthinning process.

Original: These aspects exemplify some of the computational advantages thatare specific tothe sparse, as opposed to the dense block model approach.
Revised: Multigraph sampling is made more computationally efficient via model propertiesthat are unique to sparse, as opposed to dense, stochastic block models.

Original: In the multigraph, let˚ n ij = n ij − ˙ n ij (empty dot) indicate the number ofedges from i to j that are thinned due tomismatched community assignments.
Revised: Letting (filled dot) ˙ n ij denotethe number of unthinned (i.e. selected) multigraph edges as in (7), and (empty dot) ˚ n ij = n ij − ˙ n ijdenote the number of thinned multigraph edges, we sample the latent multigraph as follows.

Original: Here is the strategy adopted to sample the latent multigraph:
Revised: Sampling of ˙n ij : From Eq.

Original: Sampling of ˙n ij : The existence of retained edges between nodes i and j in the latent multigraph implies an edge in the observed graph according to (7).
Revised: (7), there exists an edge in the observed graph if and only if there is at least one unthinned edge between nodes i and j in the latent multigraph.

Original: Thus, rather than considering each pair i, j , we only need to sample ˙ n ij when Y ij = 1 , and so only as many as there are observed edges in the graph, and we can set the remaining ones to zero.
Revised: Thus rather than considering each pair i, j , we only need to sample ˙ n ij when Y ij = 1 , an operation with cost linear in the number of observed (undirected) edges. Conversely, Y ij = 0 implies ˙ n ij = ˙ n ji = 0 .

Original: Since there must be at least one latent edge when Y ij = 1 , 
Revised: Since there must be at least one unthinned edge when Y ij = 1 , the posterior follows a zero-truncated Poisson distribution:

Original: [Equation]
Revised: [Equation]

Original:  For retained edges weneed to sample a single community assignment, since by definition these arethe edges whose nodes have been assigned to the same communities.
Revised: Zero-Trunc-Poisson [Equation] For each unthinned edge we sample a single community assignment, since by definition these are theedges whose nodes have been assigned to the same communities.

Original: We can sample the communityassignment of a retained edge between i and j easily as a draw from Cat ( π i 1 π j 1 , . . .
Revised: We can sample the communityassignment of an unthinned edge between i and j easily as a draw from Cat ( π i 1 π j 1 , . . .

Original: , π iK π jK ) .
Revised: , π iK π jK ) .

Original: Sampling of ˚n ij : By construction, since thinned edges are unobserved, all pairs of nodes ( i, j )may have edges that are thinned.
Revised: Sampling of ˚n ij : By construction, since thinned edges are unobserved, all pairs of nodes ( i, j ) mayhave edges that are thinned.

Original: These edges are auxiliary variables necessary to obtain the fullconditional posterior distribution of W α .
Revised: These edges are auxiliary variables necessary to obtain the full conditionalposterior distribution of W α .

Original: Also, to update community memberships, it is necessary toassign every thinned edge to a pair of discordant  communities.
Revised: Also, to update community memberships and global frequencies, it isnecessary to assign every thinned edge to a pair of discordant (non-matching) communities.

Original: Thanks to the properties of Poissonprocesses, we do not need to sample ˚ n ij for all the pairs of nodes. Instead, we can resort to anefficient sampling strategy, that exploits the construction based on thinning.
Revised: Thanksto the properties of Poisson processes, we do not need to sample ˚ n ij for all the pairs of nodes; wecan develop a more efficient sampler that exploits the thinned model construction.

Original: More specifically, we canfirst sample a proposed total number of thinned edges ˚ N from a Poisson (cid:16) 2 (cid:80) Ni,j =1 w i w j (cid:17) .
Revised: We first samplea proposed total number of thinned edges ˚ N ∼ Poisson (cid:16) 2 (cid:80) Ni,j =1 w i w j (cid:17) .

Original: Then,we can assign each of these independently to a pair of nodes according to their sociabilities, and thenindependently to a pair of commmunities given the memberships of the assigned nodes (as in 7).
Revised: We then assign each ofthese independently to a pair of nodes according to their sociabilities as in Eq. (5), and subsequentlyto a pair of commmunities given the memberships of the assigned nodes as in Eq.

Original: Wewill then accept those edges that were assigned to discordant communities.
Revised: Finally, wedetermine ˚ n ij from the number of edges that were assigned to discordant communities.

Original: Finally, as illustrated in Figures 1 and 3, the total number of nodes  with at least one edge in themultigraph pre- thinning N α is likely to be different fromboth the total number of nodes with atleast one edge after thinning and – as a consequence – from the total number of nodes N in theobserved binary graph Y .
Revised: Sampling unobserved nodes: As illustrated in Figs. 1 and 3, the total number of nodes N α with atleast one edge in the unthinned multigraph is likely to be different from the total number of nodeswith at least one edge after thinning and – as a consequence – from the total number of nodes N in theobserved binary graph Y .

Original: Thus, there typically ought to be some latent number N α − N ≥ 0 ofnodes whose edges with the observed nodes have all been thinned.
Revised: There is thus some latent number N α − N ≥ 0 of nodes whose edges withthe observed nodes have all been thinned.

Original: In order to learn such number, inour MCMC, we make an approximate update of N α according to the mean ratio N α /N from graphssimulated from the GGP prior given the latest samples of the hyperparameters α, σ and τ .
Revised: In order to learn this number, our MCMC sampler includesan approximate update of N α according to the mean ratio N α /N from graphs simulated from the GGP prior given the latest samples of the hyperparameters α, σ, τ .

Original: We foundthat this heuristic method leads to convergence of the empirical MCMC-based estiamte of N α acrossall simulated and  real data that we considered (see Appendix).
Revised: We found that this approximatemethod leads to convergence of the empirical MCMC-based estimate of N α across all simulated and [Figure] [Figure] real data that we considered (see Appendix).

Original: Results5.1 Simulation
Revised: Sampling of the thinned edges from these unobservednodes introduces some additional complications that we efficiently resolve in the Appendix. Sampling community memberships π i : Conditioned on the discrete community assignments foredges in the latent multigraph, the node community memberships π i have closed-form Dirichletposteriors (see Appendix). Given π , the global community frequencies β may be efficiently resampledusing auxiliary-variable methods developed for the hierarchical Dirichlet process [6]. By learning β ,the node community memberships π i become sparse, placing significant probability mass only on adata-dependent subset of the full set of K communities allocated in Eq.

Revised: Computational complexity: For a K -community TGGP model of a graph with E observed edges,resampling the latent multigraph requires O ( EK ) operations.

Revised: (The thinned multigraph has feweredges, and thus its resampling increases costs by a small constant.)

Revised: Closed-form resampling ofcommunity memberships is faster, requiring only O ( NK ) operations for an N -node graph.

Revised: The

Revised: CGGP model [18] uses Hamiltonian Monte Carlo proposals with similar cost, but empirically the

Revised: CGGP sampler mixes slower and has inferior performance (see experiments).

Revised: By exploiting sparsematrices and parallelization, both samplers may be scaled to networks with tens of thousands ofnodes; very-large networks may require alternative approximate inference algorithms.

Original:  We discuss a simulation study where we investigate the performances of our proposed method andthe CGGP model of [17], based on simulated data generated from either model.
Revised: 5.1 Simulation We discuss a simulation study where we investigated the performance of our proposed TGGP model,and the CGGP model of [18], based on simulated data generated from either model.

Original: First, we present the results from a sparse graph with 15 communities, simulated from our TGGPmodel, by setting α = 250 , σ = 0 .
Revised: First, we present the results from a sparse graph with 15 communities, simulated from our TGGPmodel by setting α = 250 , σ = 0 .

Original: 1 , τ = 1 for the distribution of the nodes’ sociabilities and γ = 10and ζ = 0 .
Revised: 1 , τ = 1 for the distribution of node sociabilities andγ = 10 , ζ = 0 .

Original: 2 for the distribution of nodes’ membership.
Revised: 2 for the distribution of node community memberships.

Original: The adjacency matrix of the resulting simple,undirected, graph is shown in Figure 4 (a, bottom).
Revised: The adjacency matrix of the resulting simple, undirected graph is shown in Figure 4(a, bottom).

Original: Figure 4 (b, bottom) sorts the simulated nodesinto blocks according to their main membership and plots the density of edges in each block.
Revised: Figure 4(b, bottom) sorts the simulated nodes into blocks according to their main membership and plots thedensity of edges in each block.

Original: Wesee that the simulated graph has a clear block structure.
Revised: We see that the simulated graph has a clear block structure.

Original: We then run our MCMC sampler for 50,000iterations, discarding the first 40,000 samples as burn-in.
Revised: We then run our MCMC sampler for 50,000 iterations, discarding the first 40,000 samples asburn-in.

Original: For our model fitting, we set the maximumnumber of communities to be 50 and we set γ = 10 and ζ = 0 .
Revised: For our model fitting, we let 50 be theupper bound to the number of communities andwe set γ = 10 and ζ = 0 .

Original: 5 to allow for learning the number ofcommunities.
Revised: 5 to allow for learningthe number of communities.

Original: From a qualitative comparison of the (b) and (c) bottom panes in Figure 4, we see that [Figure] [Figure] [Figure]
Revised: From a qualitative comparison of the (b) and (c) bottom panesin Figure 4, we see that the community memberships recovered from our model are closeto the underlying truth.

Original: [Figure]

Original: [Figure]

Original: [Figure] w i the community memberships recovered from our model are close to the underlying truth. Figure 5shows summaries of the posterior distributions of nodes’ sociabilities for 40 randomly selected nodesspanning from low to relatively large sociability. We can see that the posterior distributions of allparameters tend to concentrate around the true values used for simulation (see, e.g., Figure 5 95%credible intervals for the sociability parameters).
Revised: The credible intervalsindicate that the posterior distributions of allparameters tend to concentrate around the truevalues used for simulation (see Appendix forplots).

Original: On the contrary, the CGGP model by [17] appearsto struggle recovering the true community structure of the network (Figure 4, d bottom), despitesetting the number of communities equal to  the truth.
Revised: In contrast, the CGGP model [18] struggles to recover the true community structure ofthe network (Fig. 4(d bottom)), despite beingfitted with the number of communities K = 15set to match the truth.

Original: We also simulated a graph from the CGGP model of [17] with parameters set similarly as thoseused in the simulation above. In addition, we set a k = 1 /K and b k = 1  to make the prior oncommunity memberships in [17] close to ours.
Revised: We also simulated a graphfrom the CGGP model of [18] with parametersa k = 1 /K, b k = 1 chosen to induce a community membership prior similar to our TGGP.

Original: We fit both the CGGP model and the TGGP model tothe generated data.
Revised: We fit boththe CGGP model and the TGGP model to the generated data.

Original: When fitting with the CGGP we still set the number of communities equal to thetruth, i.e. 15.
Revised: When fitting with the CGGP we stillset the number of communities equal to the truth ( K = 15 ).

Original: The results plotted in  4(d) suggests that the block structures learned by our  model arecloser to  the true ones also under this mis-specification of the generative model.
Revised: The results plotted in Fig. 4(d) indicatethat the block structures learned by our (misspecified) TGGP model are nevertheless closer to truththan those recovered by the CGGP. To quantify the superior community recovery of our TGGP model, recall that for each observed nodethe CGGP estimates a K -dim. positive real vector of community-specific sociabilities ( w i 1 , . . . , w iK ) ,while our TGGP infers a K -dim.

Revised: probability vector of community memberships ( π i 1 , . . . , π iK ) .

Revised: Forquantitative comparison, we standardize true and inferred sociability vectors by dividing each w ikby the sum (cid:80)

Original: 5.2 Real network data
Revised: Kh =1 w ih . Also, since the TGGP model was fitted with K set to an upper-bound of 50rather than to the true number of communities, when analyzing TGGP results we pad the true vectorsof community memberships (for TGGP-simulated data) and sociabilities (for CGGP-simulateddata) with zeros. For estimated parameters, community indices are then permuted to maximizesimilarity with true parameters. Letting ψ i and ˆ ψ i be, respectively, true and inferred communitymemberships/sociabilities, we define similarity as 1 − 1 N (cid:80) Ni =1 d ( ψ i , ˆ ψ i ) where the distance is the

Revised: L1 total variation d ( ψ i , ˆ ψ i ) =

Revised: 12 (cid:80)

Revised: Kk =1 | ψ ik − ˆ ψ ik | .

Revised: Fig.

Revised: 5 confirms that the TGGP more accuratelyrecovers community memberships for both networks, and for nodes of both high and low degree.

Revised: 5.2 Real network data

Original: To assess the performance of our method for introducing overlapping communitiesin the GGPmodel for network as compared to the  CGGP approach by [17], we evaluate the two methods on
Revised: We compare the performance of our TGGP framework for adding overlapping communities to the GGP model with the approaches detailed in Sec. 2.2: the CGGP (sparse with mixed memberships)of [18] and the SBM-GGP model (sparse with single memberships) of [17].

Original: [Figure]
Revised: We also compare tomore classical models from Sec. 2.1: the SBM (dense with single membership) of [2] and the MMSB(dense with mixed memberships) of [5]. We run 50,000 iterations of the five models’ MCMC on fourreal-world networks; see Appendix for data sources and pre-processing. Each model was fit to fullyobserved data to learn node-specific parameters (e.g., sociabilities and community memberships) andcommunity-interaction probabilities, using the values from the final MCMC iteration.

Original: two different forms of posterior predictive accuracy on 4 real world networks (a description of howthe data was obtained and pre-processed can be found in the appendix).
Revised: Two differentmeasures of posterior predictive accuracy were then used to assess the goodness of model fit. For the first evaluation measure, we randomly selected 5% of the entries equal to 1 in the adjacencymatrix and set them to 0. We then compute the probability of being 1 of all entries that are equal to0 in the modified adjacency matrix to assess how well each model can distinguish the entries thatshould in fact be 1. This edge-retrieval measure is motivated by common applications of networkanalysis to recommendation tasks (suggesting which edges that are not observed should be present).

Original: We run 50,000 iterations ofthe two models’ MCMC on the fully observed data, and we used the model parameters estimatedat the last iteration to infer (1) what entries of  the  adjacency matrix incorrectly classified as 0 aremost likely to be edges and (2) the presence or absence of an edge if we consider 5% of  entries of theadjacency matrix as missing.
Revised: Fig. 6 (top) plots recall (proportion of ones recovered) on the horizontal axis, and F-score (geometricmean of recall and precision, which is the proportion of ones correctly predicted among all entriespredicted as ones) on the vertical axis. This retrieval task is very challenging because, for sparsenetworks, the missing edges are a tiny proportion of all zero entries in the modified adjacency matrix. For the second evaluation measure, we randomly obscure 5% of all entries in the adjacency matrixto assess how well each model can recover whether an entry is equal to 0 or to 1.

Original: Note that the first task is more difficult than the  second one, becausemost entries in the adjacency matrix are truly zeros (and so it is difficult to find the ones, while it isnot as difficult to predict if an entry is a 0 or a 1).
Revised: Fig. 6 (bottom)plots the receiver operating characteristic (ROC) curves, where false positive rate (number of zerospredicted as ones divided by the number of zeros) is on the horizontal axis, and true positive rate(proportion of ones correctly predicted as ones divided by the number of ones) is on the vertical axis. This second task has been used in many prior papers but is not as hard as the first one, because for amodel to score well it suffices to learn small interaction probabilities, since almost all of the randomlyselected entries are 0.

Original: Figure 6 shows that our method does consistentlybetter than the CGGP.
Revised: Fig. 6 shows that our TGGP model does consistently better than all baselinemethods for all networks considered, according to both predictive-performance measures.

Original:  Discussion
Revised:  Discussion

Original: We have proposed a framework for the analysis of binary network data that extends the GGP-basedmodel of Caron and Fox [12] by allowing for overlapping community structures, depending bothon the overall sociability of the nodes and the similarity of their community memberships.
Revised: We have proposed a framework for the analysis of binary network data that extends the GGP-basedmodel of Caron and Fox [13] by allowing for overlapping community structures, depending bothon the overall sociability of the nodes and the similarity of their community memberships.

Original: Theassumed generative model proposes a novel latent multi-graph framework where nodes are allowedto connect between different communities, but thecorresponding edges  are hidden (thinned) in theprojection giving rise to the observed network.
Revised: Theassumed generative model proposes a novel latent multi-graph framework where nodes are allowedto connect within the same or across different communities, but the edges formed across differentcommunities are hidden (thinned) in the projection giving rise to the observed network.

Original: With respect to alternative extensions of the original GGP-based model, e.g. the one in Todeschini et al. [17], the framework we propose allows for aregularized approach in the modeling of the latentstrength of community memberships for eachnode, which results in improved inference and reconstruction of the true community structures bothin simulation and data analysis.
Revised: With respectto alternative extensions of the original GGP network model [17, 18], our TGGP enables mixedcommunity memberships and facilitates regularization of the latent community distributions for eachnode, resulting in improved inference and reconstruction of the latent community structures for bothsimulated and real data.

Original: Furthermore, our model formulation allows for learning the numberof communities directly from the data, as it does not require fixing this number  a priori .
Revised: Furthermore, our model formulation allows for learning the number ofcommunities directly from the data, by encouraging inferring a number of non-empty communitiespotentially smaller than a set upper bound.

Original: The proposed  model is amenable to further extensions.
Revised: The proposed TGGP model is amenable to further extensions.

Original: Since the proposed mixed membershipallocation framework relies on the well-studied Dirichlet-multinomial specification, it is possible toleverage the vast literature on (finite and infinite) hierarchical Dirichlet mixture models to modelcomplex heterogeneous network data.
Revised: Since our mixed membership allocationframework relies on the well-studied dirichlet-multinomial specification, it is possible to leverage thevast literature on (finite and infinite) hierarchical dirichlet mixture models to capture more complexand heterogeneous network data.

Original: It is also possible to include additional available information oncovariates to guide the allocation of nodes tothe communities.
Revised: It is also possible to include additional node covariates or metadatato guide the allocation of nodes to communities.

Original: Finally, distributional and asymptoticproperties of our model formulation can be investigated by exploiting recent results in the literaturerelated to finite mixtures and mixtures of finite mixtures [see, e.g., 27, 28, 29, 18].
Revised: Finally, distributional and asymptotic propertiesof our model formulation can be investigated by exploiting recent results in the literature related tofinite mixtures and mixtures of finite mixtures [see, e.g., 27, 28, 29, 19].

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] See Section 3, 4,  and
Revised: [Yes] See Sections 1, 3, 4, 5, and

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section  5 and
Revised: [Yes] See Section 1, 3, 4, 5 and

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A](d)
Revised: [N/A]

Original:  Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] We providedthe assumptions and references, given the limitation of pages. 
Revised: [No] We provided the assumptions, results and references, given the limitation of pages. Most results we

Revised: employ follow from analogous results in [13], with similar assumptions.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [No] Most results we employ
Revised: [No] Most results we employ

Original: follow from analogous results in [12]
Revised: follow from analogous results in [13]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main ex-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main ex-

Original: perimental results (either in the supplemental material or as a URL)?
Revised: perimental results (either in the supplemental material or as a URL)?

Original: [No] The codewill be made publicly available on Github upon acceptance.
Revised: [No] The code will be made publicly available on Github upon acceptance.

Original: The datasets are publiclyavailable.(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: The datasets are publicly available. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See Sections 4a nd5(c)
Revised: [Yes] See Sections 4 nd

Original:  Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?

Original: [Yes] See Figure 5, for example (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [Yes] See Figure 5, for example(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [No] We will do in the resubmission4.
Revised: [No] We will do in the resubmission4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [TODO]
Revised: [Yes] See Section 5(b)

Original: (b) Did you mention the license of the assets?
Revised: Did you mention the license of the assets?

Original: [TODO]
Revised: [N/A](c)

Original: (c) Did you include any new assets either in the supplemental material or as a URL?[TODO](d)
Revised: Did you include any new assets either in the supplemental material or as a URL? [N/A] (d) Did you discuss whether

Original: Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
Revised: and how consent was obtained from people whose data you’reusing/curating?

Original: [TODO]( e)
Revised: [N/A](e)

Original: Did you

Original: discuss whether

Original:  the data you are using/ curating contains personally identifiableinformation 
Revised: Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?

Original: or

Original: offensive content? [TODO If
Revised: [N/A]5.

Original:  you used crowdsourcing or conducted research with human subjects.
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots , if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [TODO
Revised: [N/A](b)

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [TODO
Revised: [N/A](c)

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [TODO]
Revised: [N/A]