Original: Thinned random measures for sparse graphs with overlapping communities
Revised: Thinned random measures for sparse graphs with overlapping communities

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Network models for exchangeable arrays, including most stochastic block models,generate dense graphs with a limited ability to capture many characteristics ofreal-world social and biological networks.
Revised: Network models for exchangeable arrays, including most stochastic block models, generate dense graphs with a limited ability to capture many characteristics of real-world social and biological networks.

Original: A class of models based on completelyrandom measures like the generalized gamma process (GGP) have recently addressed some of these limitations.
Revised: A class of models based on completely random measures like the generalized gamma process (GGP) have recently addressed some of these limitations.

Original: We propose a framework for thinning edgesfrom realizations of GGP random graphs that models observed links via nodes’overall propensity to interact, as well as the similarity of node memberships withina large set of latent communities.
Revised: We propose a framework for thinning edges from realizations of GGP random graphs that models observed links via nodes’ overall propensity to interact, as well as the similarity of node memberships within a large set of latent communities.

Original: Our formulation allows us to learn the numberof communities from data, and enables efficient Monte Carlo methods that scalelinearly with the number of observed edges, and thus (unlike dense block models)sub-quadratically with the number of entities or nodes.
Revised: Our formulation allows us to learn the number of communities from data, and enables efficient Monte Carlo methods that scale linearly with the number of observed edges, and thus (unlike dense block models) sub-quadratically with the number of entities or nodes.

Original: We compare to alternativemodels for both dense and sparse networks, and demonstrate effective recovery oflatent community structure for real-world networks with thousands of nodes.
Revised: We compare to alternative models for both dense and sparse networks, and demonstrate effective recovery of latent community structure for real-world networks with thousands of nodes.

Original:  Introduction
Revised:  Introduction

Original: Given observations of (often binary) relationships Y ij between pairs of nodes or entities ( i, j ) ,many relational models [1] seek to uncover an underlying set of communities.
Revised: Given observations of (often binary) relationships Y ij between pairs of nodes or entities ( i j ) many relational models [1] seek to uncover an underlying set of communities.

Original: Classic stochasticblockmodels [2] generalize mixture models for clustering non-relational data by assigning each entityto one of K communities (clusters).
Revised: Classic stochastic blockmodels [2] generalize mixture models for clustering non-relational data by assigning each entity to one of K communities (clusters).

Original: The infinite relational model (IRM) [3] instead uses a Dirichletprocess prior [4] to partition entities into single communities.
Revised: The infinite relational model (IRM) [3] instead uses a Dirichlet process prior [4] to partition entities into single communities.

Original: While the IRM allows the number ofcommunities to be inferred from data, later work has shown that real-world social networks are bettercaptured by models which allow individuals to participate in multiple communities [5], includingapplications of the hierarchical Dirichlet process (HDP) [6] to relational data [7]. 
Revised: While the IRM allows the number of communities to be inferred from data, later work has shown that real-world social networks are better captured by models which allow nodes to participate in multiple communities [5] including applications of the hierarchical Dirichlet process (HDP) [6] to relational data [7]. Node relationships may also be modeled by shared features [8, 9] learned via the Indian Buffet Process [10], by a combination of node and interaction factors [11], or by proximity in a latent space [12, 13].

Original: There is an extensive literature on descriptive statistics of biological and social networks [8, 9], including degree distributions, path distances and “small world” phenomena [10], community structures andmodularity, and notions of centrality and causality.
Revised: There is an extensive literature on descriptive statistics of biological and social networks [14, 15] including degree distributions, path distances and “small world” phenomena [16], community structures and modularity, and notions of centrality and causality.

Original: In particular, sparsity is a ubiquitous phenomenonin real-world networks [8, 9]: as network size grows, the number of edges grows more slowly than thequadratic number of node pairs.
Revised: In particular, sparsity is a ubiquitous phenomenon in real-world networks [14, 15]: as network size grows, the number of edges grows more slowly than the quadratic number of node pairs.

Original: However, the IRM and HDP relational models (and a large literatureof related models [1]) generate dense graphs where the number of edges scales quadratically withthe number of nodes.
Revised: However, the IRM and HDP relational models (and a large literature of related models [1]) generate dense graphs where the number of edges scales quadratically with the number of nodes.

Original: In fact, a classic representation theorem [11, 12] shows that any generativemodel that regards graphs as exchangeable adjacency matrices, meaning arrays whose distributionis invariant to permutations of node indices, generates dense graphs.
Revised: In fact, a classic representation theorem [17, 18] shows that any generative model that regards graphs as exchangeable adjacency matrices, meaning arrays whose distribution is invariant to permutations of node indices, generates dense graphs.

Original: Most existing probabilisticnetwork models, which generate an adjacency matrix by sampling edges from independent Bernoullidistributions given latent node-specific parameters, have this limitation. 
Revised: Most existing probabilistic network models, which generate an adjacency matrix by sampling edges from independent Bernoulli distributions given latent node-specific parameters, have this limitation. Another important property of many real-world networks is assortative mixing [19, 20, 21], that is the presence of more popular or sociable nodes to which other nodes are more likely to connect.

Original:  By representing graphs as a latent process (a completely random measure), Caron and Fox [13]showed that it is possible to formulate generative models that capture the sparsity of real-worldnetworks.
Revised: Models with degree-correction mechanisms [22, 23, 24] account for this phenomenon. By representing graphs as a latent process (a completely random measure), Caron and Fox [25] showed that it is possible to formulate generative models that capture the sparsity of real-world networks as well as assortative mixing.

Original: Related models, including certain infinite limits of graphs called graphons , have been studied by several authors [14, 15, 16].
Revised: Related models, including certain infinite limits of graphs called graphons , have been studied by several authors [26, 27].

Original: However, these models mostly produce homogeneous graphswith sparsity and heavy-tailed degree distributions, but lacking the community structure of realnetworks.
Revised: However, these models mostly produce homogeneous graphs with sparsity and heavy-tailed degree distributions, but lacking the community structure of real networks.

Original: Two notable exceptions are work by Herlau et al.
Revised: Two notable exceptions are work by Herlau et al.

Original: [17] and Todeschini et al.
Revised: [28] and Todeschini et al.

Original: [18] thataugment the random-measure models of [13] withvariables encoding latent community structure. 
Revised: [29] that augment random-measure models [25] with latent community structure. Alternative approaches to sparse networks regard edges, rather than nodes, as the core of the generative process [30, 31, 32].

Original:  In this paper, we propose a novel random graph model that efficiently thins edges from sparsehomogeneous graphs to reveal community structure while maintaining sparsity.
Revised: Another approach to simultaneously capture local density and global sparsity models graphs as a collection of cliques [33]. In this paper, we propose a novel random graph model that efficiently thins edges from sparse homogeneous graphs to reveal community structure while maintaining sparsity.

Original: Unlike [17], we allowentities to have membership in multiple communities.
Revised: Unlike [28], we allow entities to have membership in multiple communities.

Original: Unlike [18], we allow completely flexiblespecification of the hierarchical Bayesian prior on latent community memberships; importantly, thisenables the network-specific learning of the appropriate number of latent communities.
Revised: Unlike [29], we allow completely flexible specification of the hierarchical Bayesian prior on latent community memberships; importantly, this enables the network-specific learning of the appropriate number of latent communities.

Original: We furtherdevelop an efficient Monte Carlo inference algorithm that, unlike nearly all dense block models,scales linearly with the number of observed edges and thus sub-quadratically with the number ofentities.
Revised: We further develop an efficient Monte Carlo inference algorithm that, unlike nearly all dense block models, scales linearly with the number of observed edges and thus sub-quadratically with the number of entities.

Original: Experiments show recovery of communities for networks with thousands of nodes.
Revised: Experiments show recovery of communities for networks with thousands of nodes.

Original:  Background: Stochastic Blockmodels for Dense and Sparse Networks
Revised:  Background: Stochastic Blockmodels for Dense and Sparse Networks

Original: An undirected binary network with N nodes and E edges may be represented by a symmetric N × Nadjacency matrix Y .
Revised: An undirected binary network with N nodes and E edges may be represented by a symmetric N × N adjacency matrix Y .

Original: If there is an edge (link) between nodes i ̸
Revised: If there is an edge (link) between nodes i ̸

Original: =
Revised: =

Original: j then Y ij = 1 , otherwise Y ij = 0 .
Revised: j then Y ij = 1 , otherwise Y ij = 0 .

Original: 2.1 Mixed Membership Stochastic Blockmodels
Revised: 2.1 Mixed Membership Stochastic Blockmodels

Original: Stochastic blockmodels (SBMs) [2] assume that each node belongs to one of K latent communities,and that the probability of an edge depends on how strongly their communities are connected.
Revised: Stochastic blockmodels (SBMs) [2] assume that each node belongs to one of K latent communities, and that the probability of an edge depends on how strongly their communities are connected.

Original: The community c i ∈ { 1 , . . .
Revised: The community c i ∈ { 1 , . . .

Original: , K } of node i follows a categorical distribution
Revised: , K } of node i follows a categorical distribution

Original: c i ind ∼ Cat ( β ) , whereβ = ( β 1 , . . 
Revised: c i ind ∼ Cat ( β ) , where β = ( β 1 , . . .

Original: . , β K ) so that β k is the frequency of community k .
Revised: , β K ) so that β k is the frequency of community k .

Original: Edges are sampled independently as
Revised: Edges are sampled independently as

Original: Y ij ind ∼ Bernoulli ( η c i c j ) , where η kℓ =
Revised: Y ij ind ∼ Bernoulli ( η c i c j ) , where η kℓ =

Original: η ℓk is the probability of an edge between nodes in communitiesk and ℓ .
Revised: η ℓk is the probability of an edge between nodes in communities k and ℓ .

Original: These interaction probabilities are often assigned conjugate beta priors, η kℓ ind ∼ Beta ( τ a , τ b ) .
Revised: These interaction probabilities are often assigned conjugate beta priors, η kℓ ind ∼ Beta ( τ a , τ b ) .

Original: Mixed membership stochastic blockmodels (MMSBs) [5] extend SBMs to allow nodes to be membersof multiple communities.
Revised: Mixed membership stochastic blockmodels (MMSBs) [5] extend SBMs to allow nodes to be members of multiple communities.

Original: Let π i =
Revised: Let π i =

Original: ( π i 1 , . . . , π iK ) denote a K -dimensional probability vectorrepresenting the strength of affiliation of node i to each of K communities.
Revised: ( π i 1 , . . . , π iK ) denote a K -dimensional probability vector representing the strength of affiliation of node i to each of K communities.

Original: For every pair of nodes( i, j ) , the communities governing their interaction are sampled as c ij ind ∼ Cat ( π i ) , c ji ind ∼ Cat ( π j ) .
Revised: For every pair of nodes ( i, j ) , the communities governing their interaction are sampled as c ij ind ∼ Cat ( π i ) , c ji ind ∼ Cat ( π j ) .

Original: Then, like standard SBMs, edges are sampled independently as Y ij ind ∼ Bernoulli ( η c ij c ji ) .
Revised: Then, like standard SBMs, edges are sampled independently as Y ij ind ∼ Bernoulli ( η c ij c ji ) .

Original: Communitymemberships are typically assigned a hierarchical prior, such as
Revised: Community memberships are typically assigned a hierarchical prior, such as

Original: [Equation]
Revised: [Equation]

Revised: The concentration parameter ζ controls the polarization of π i and its variation around β , with smaller values of ζ inducing more polarized community memberships that place significant probability on only a few communities, and larger values of ζ inducing community memberships that differ very little from β .

Original: When fitting this model to data, we can treat K as an upper bound to the number of communities(that is, larger than the maximum number of communities that we expect to be necessary to modelthe network) and fix γ ≪ K .
Revised: When fitting this model to data, we can treat K as an upper bound to the number of communities (that is, larger than the maximum number of communities that we expect to be necessary to model the network), fix γ ≪ K and let ζ be small.

Original: In this way, the hierarchical Dirichlet formulation allows networkspecific learning of the number of communities by favoring sparse community frequency vectorsβ ={+1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent) GG P t h i nned GG P                         01    1      0 
+}
Revised: In this way, the hierarchical Dirichlet formulation allows network-specific learning of the number of communities by favoring sparse {+1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent) GG P t h i nned GG P                         01    1      0 
+}

Revised: 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple undirected graph observed)           community frequency vectors β

Original:  ( β 1 , . .
Revised: = ( β 1 , . .

Original: . , β K ) with some elements β h ≈ 0 , h ∈ { 1 , . .
Revised: . , β K ) with some elements β h ≈ 0 , h ∈ { 1 , . .

Original: . , K } .
Revised: . , K } .

Original: The hierarchical Dirichletprocess [6, 7] is the limit of this prior as K approaches infinity [19].
Revised: The hierarchical Dirichlet process [6, 7] is the limit of this prior as K approaches infinity [34].

Original: 2.2 Sparse Network Models via Completely Random Measures
Revised: 2.2 Sparse Network Models via Completely Random Measures

Original: For any fixed community frequencies β and interaction probabilities η , the SBM and MMSB mayonly generate dense graphs where the number of edges scales quadratically with the number ofnodes [12].
Revised: For any fixed community frequencies β and interaction probabilities η , the SBM and MMSB may only generate dense graphs where the number of edges scales quadratically with the number of nodes [18].

Original: In contrast, many real-world networks appear to be sparse [9].
Revised: In contrast, many real-world networks appear to be sparse [15].

Original: Heuristics are often usedto fit (mixed membership) SBMs to large but sparse networks, such as fixing (rather than learning)η kℓ = ε ≈ 0 for k ̸ = ℓ ; for example Kim et al.
Revised: Heuristics are often used to fit (mixed membership) SBMs to large but sparse networks, such as fixing (rather than learning) η kℓ = ε ≈ 0 for k ̸ = ℓ ; for example Kim et al.

Original: [7] fix ε = e − 30 .
Revised: [7] fix ε = e − 30 .

Original: We seek to build models that cancapture mixed memberships and sparsity simultaneously without needing to rely on such heuristics.
Revised: We seek to build models that can capture mixed memberships and sparsity simultaneously without needing to rely on such heuristics.

Original: Completely Random Measures.
Revised: Completely Random Measures.

Original: By representing the graph as a point process on the plane, Caronand Fox [13] showed that it is possible to generate sparse graphs by appropriately choosing the meanmeasure of the point process.
Revised: By representing the graph as a point process on the plane, Caron and Fox [25] showed that it is possible to generate sparse graphs by appropriately choosing the mean measure of the point process.

Original: According to their model, for i ̸
Revised: According to their model, for i ̸

Original: = j ,
Revised: = j ,

Original: [Equation]
Revised: [Equation]

Original: [-1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent) GG P t h i nned GG P                         01    1      0 
 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple undirected graph observed)-] where w i > 0 represents the sociability of node i : nodes with higher w i have higher probabilityto interact with other nodes, and hence greater expected degree.
Revised: [-1    1 1   1  1 1  1     11   1    0 1   0  1 1  1     00   directed multigraph (latent) GG P t h i nned GG P                         01    1      0 
 10  0 0  00 1 0 0 01  0 0  00 0 0 1 00  0 0  00 0 0 0 10  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 0  00 0 0 0 00  0 1  00 0 00 0 00  0 0  00 0simple undirected graph observed)-] where w i > 0 represents the sociability of node i : nodes with higher w i have higher probability to interact with other nodes, and hence greater expected degree.

Original: Differently from the (mixedmembership) SBMs where all parameters governing nodes’ interactions are sampled independently,here nodes’ sociabilities are generated altogether using the jumps of a completely random measure(CRM)
Revised: Differently from the (mixed membership) SBMs, where all parameters governing nodes’ interactions are sampled independently, here nodes’ sociabilities are generated altogether using the jumps of a completely random measure (CRM) [35].

Original: Each node is also independently associated to a real-valued location ℓ i uniformlydistributed on the real line and then a set of potential nodes is defined by restricting to sociabilitieswith a sampled location in the interval [0 , α ] ,
Revised: Each node is also independently associated to a real-valued location ℓ i uniformly distributed on the real line, and then a set of potential nodes is defined by restricting to sociabilities with a sampled location in the interval [0 , α ] ,

Original: [Equation]
Revised: [Equation]

Original: Here α > 0 controls the (random) number of nodes N α in the network by determining the size ofthe interval in which jumps are included.
Revised: Here α > 0 controls the (random) number of nodes N α in the network by determining the size of the interval in which jumps are included.

Original: Depending on the distribution of the jumps, the model cancapture both sparse and dense graphs.
Revised: Depending on the distribution of the jumps, the model can capture both sparse and dense graphs.

Original: Intuitively, as detailed in Sec.
Revised: Intuitively, as detailed in Sec.

Original: 5.1 of [13], for sparse networksthe distribution of the jumps needs to place almost all of its mass near zero.
Revised: 5.1 of [25], for sparse networks the distribution of the jumps needs to place almost all of its mass near zero.

Original: This model requires CRMs for which the sum ¯W α
Revised: This model requires CRMs for which the sum ¯

Original: = (cid:80) i : ℓ i ∈ [0 ,α ] w i of the jumps in [0 , α ] is finite.
Revised: [Equation] i of the jumps in [ [Equation] α ] is finite.

Original: A simple undirected network is then generated via a binary projection of an underlying directedmultigraph, where the total number of edges n ij from node i to node j is independently distributed asn ij | w i , w j ind ∼ Poisson ( w i w j ) .
Revised: A simple undirected network is then generated via a binary projection of an underlying directed multigraph, where the total number of edges n ij from node i to node j is independently distributed as [Equation]

Original: An edge is present in the undirected network if there is at least one directed edge in the latent directedmultigraph, that is Y ij = 1 ( n ij + n ji ≥ 1) for nodes i ̸
Revised: An edge is present in the undirected network if there is at least one directed edge in the latent directed multigraph, that is Y ij = 1 ( n ij + n ji ≥ 1) for nodes i ̸

Original: = j .
Revised: = j .

Original: Because the sum of independent Poissonrandom variables is Poisson, Eq.
Revised: Because the sum of independent Poisson random variables is Poisson, Eq.

Original: (4) implies P ( n ij + n ji = 0)
Revised: (4) implies P ( n ij + n ji = 0)

Original: =
Revised: =

Original: exp {− 2 w i w j } , from which Eq.
Revised: exp {− 2 w i w j } , from which Eq.

Original: (2) follows.
Revised: (2) follows.

Original: The number N of nodes in the observed network then equals the number N α of nodes thathave at least one edge in the underlying multigraph.
Revised: The number N of nodes in the observed network then equals the number N α of nodes that have at least one edge in the underlying multigraph.

Original: This construction of the binary matrix from amultigraph [13] is visualized in Fig.
Revised: This construction of the binary matrix from a multigraph [25] is visualized in Fig.

Original: The sum-property of the Poisson distribution also implies that, given ¯ W α , the total number of edges¯ D α in the multigraph has a Poisson ( ¯ W 2 α ) distribution.
Revised: The sum-property of the Poisson distribution also implies that, given ¯ W α , the total number of edges ¯ D α in the multigraph has a Poisson ( ¯ W 2 α ) distribution.

Original: Since n ij has a high probability of being 0 formost node pairs ( i, j ) , it is more efficient to first sample
Revised: Since n ij has a high probability of being 0 for most node pairs ( i, j ) , it is more efficient to first sample

Original: ¯ D α
Revised: ¯ D α

Original: and then independently assign each edgeto a pair of nodes based on their sociabilities.
Revised: and then independently assign each edge to a pair of nodes based on their sociabilities.

Original: In more detail, Eq.
Revised: In more detail, Eq.

Original: (4) is equivalent to [Equation] Poisson 
Revised: (4) is equivalent to ¯ D α | ¯ W α ∼ Poisson ( ¯ W

Original: [Equation] 
Revised: [Equation] [Equation]

Original: where x ev ∈ { 1 , 2 , . . . }
Revised: where x ev ∈ { 1 , 2 , . . . }

Original: for v = 1 , 2 indicates the nodes sampled for edges e ∈ { 1 , . .
Revised: for v = 1 , 2 indicates the nodes sampled for edges e ∈ { 1 , . .

Original: , ¯ D α } .
Revised: , ¯ D α } .

Original: Caronand Fox [13] propose the generalized gamma process (GGP) [21, 22, 23] as a flexible but tractable CRM for W α , with parameters τ ∈ (0 , ∞ ) , and σ ∈ ( −∞ , 0] for dense graphs or σ ∈ (0 , 1) forsparse graphs.
Revised: Caron and Fox [25] propose the generalized gamma process (GGP) [36, 37, 38] as a flexible but tractable CRM for W α , with parameters τ ∈ (0 , ∞ ) , and σ ∈ ( −∞ , 0] for dense graphs or σ ∈ (0 , 1) for sparse graphs.

Original: Fig.
Revised: Fig.

Original: 2 (black component) provides a visual summary of the generative processunderlying the construction of a GGP random graph, using a directed graphical model that highlightsthe conditional dependencies (and independencies) between its variables.
Revised: 2 (black component) provides a visual summary of the generative process underlying the construction of a GGP random graph, using a directed graphical model that highlights the conditional dependencies (and independencies) between its variables.

Original: Figure 3a shows a networksampled from the GGP model by means of a node-edge diagram.
Revised: Fig. 3a shows a network sampled from the GGP model by means of a node-edge diagram.

Original: Sparse Block Models.
Revised: Sparse Block Models.

Original: A limitation of the framework described above is that it does not modelthe community (block) structure of the network – a well-recognized feature of complex networks.
Revised: A limitation of the framework described above is that it does not model the community (block) structure of the network – a well-recognized feature of complex networks.

Original: Herlau et al.
Revised: Herlau et al.

Original: [17] generalized the approach in [13] to accommodate both sparse and dense networkswith community structure.
Revised: [28] generalized the approach in [25] to accommodate both sparse and dense networks with community structure.

Original: Specifically, they introduce a latent discrete variable c i ∈ { 1 , . .
Revised: Specifically, they introduce a latent discrete variable c i ∈ { 1 , . .

Original: , K }to indicate the assignment of node i to one of K communities, like in the SBMs.
Revised: , K } to indicate the assignment of node i to one of K communities, like in the SBMs.

Original: A bivariate CRMincorporates both the sociability weights and a set of parameters, denoted by η c i c j , capturing theinteraction strength between two communities c i and c j in the underlying multigraph.
Revised: A bivariate CRM incorporates both the sociability weights and a set of parameters, denoted by η c i c j , capturing the interaction strength between two communities c i and c j in the underlying multigraph.

Original: For theirformulation, the total number of edges n ij from node i to node j are independently distributed asn ij | c i , c j , w i , w j ind ∼ Poisson ( η c i c j w i w j ) , and the likelihood from Eq.
Revised: For their formulation, the total number of edges n ij from node i to node j are independently distributed as n ij | c i , c j , w i , w j ind ∼ Poisson ( η c i c j w i w j ) , and the likelihood from Eq.

Original: (2) of the observed nodesis modified as Y ij | c i , c j , w i , w j ind ∼ Bernoulli (1 − e − 2 η cicj w i w j ) .
Revised: (2) of the observed nodes is modified as Y ij | c i , c j , w i , w j ind ∼ Bernoulli (1 − e − 2 η cicj w i w j ) .

Original: We refer to this approach as thestochastic block model generalized gamma process (SBM-GGP).
Revised: We refer to this approach as the stochastic block model generalized gamma process (SBM-GGP).

Original: Sparse Mixed Membership.
Revised: Sparse Mixed Membership.

Original: The SBM-GGP models sparse and dense networks with communitystructure, but does not capture overlapping community structures; it induces networks whose nodesare partitioned into disjoint communities.
Revised: The SBM-GGP models sparse and dense networks with community structure, but does not capture overlapping community structures; it induces networks whose nodes are partitioned into disjoint communities.

Original: In follow-up work, Todeschini et al.
Revised: In follow-up work, Todeschini et al.

Original: [18] extended the
Revised: [29] extended the

Original: CRM-based framework discussed above by associating a vector ( w i 1 , .
Revised: CRM-based framework discussed above by associating a vector ( w i 1 , .

Original: . . , w iK ) of sociabilities toeach node i , to represent different levels of affiliation of nodes to the K latent communities.
Revised: . . , w iK ) of sociabilities to each node i , to represent different levels of affiliation of nodes to the K latent communities.

Original: A nodemay have high levels of affiliation to more than a community, leading to the formation of edges acrossmultiple communities.
Revised: A node may have high levels of affiliation to more than a community, leading to the formation of edges across multiple communities.

Original: The vectors of node sociabilities are distributed according to a compound
Revised: The vectors of node sociabilities are distributed according to a compound

Original: CRM [24]; specifically, their implementation uses a compound generalized gamma process (CGGP).
Revised: CRM [39]; specifically, their implementation uses a compound generalized gamma process (CGGP).

Original: The likelihood function is modified accordingly, by independently sampling undirected edges as
Revised: The likelihood function is modified accordingly, by independently sampling undirected edges as

Original: Y ij | w i , w j ind ∼ Bernoulli (1 − exp {− 2 (cid:80) Kk
Revised: Y ij | w i , w j ind ∼ Bernoulli (1 − exp {− 2 (cid:80) Kk

Original: =

Revised: =

Original: w ik w jk } ) .
Revised: w ik w jk } ) .

Original: The latent community weights are modeledas w ik = w i[-
-]
Revised: The latent community weights are modeled ∞ ¯ D α N × N w i[-
-]

Original: ψ
Revised: α σ τ π i ζ β γ

Original:  ik , where w i 0 are sampled from a GGP as in Eq.
Revised: K x ex ec ec e Y ij as w ik = w i 0 ψ ik , where w i 0 are sampled from a GGP as in Eq.

Original: (3), and ψ ik are gamma-distributedrandom variables ψ ik ind ∼ Gamma ( a k , b k ) whose shape a k and rate b k may be inferred from data.
Revised: (3), and ψ ik are gamma-distributed random variables ψ ik ind ∼ Gamma ( a k , b k ) whose shape a k and rate b k may be inferred from data.

Original: As [18] point out, the CGGP model exploits an (unconstrained) non-negative matrix factorization todefine the Bernoulli probability link.
Revised: As [29] point out, the CGGP model exploits an (unconstrained) non-negative matrix factorization to define the Bernoulli probability link.

Original: In the Appendix, we demonstrate that this lack of constraintshampers the ability of the CGGP model to recover the true community structure of the network.
Revised: In the Appendix, we demonstrate that this lack of constraints hampers the ability of the CGGP model to recover the true community structure of the network.

Original: Better regularized approaches to non-negative matrix factorization, like the one we propose, mayimprove the identifiability of latent community structures and network parameters [25].
Revised: Better regularized approaches to non-negative matrix factorization, like the one we propose, may improve the identifiability of latent community structures and network parameters [40].

Original:  The Thinned CRM Network Model
Revised:  The Thinned CRM Network Model

Original: We now add overlapping community memberships to the GGP network model [13] via vectors ofprobabilities sampled from a hierarchical Dirichlet distribution.
Revised: We now add overlapping community memberships to the GGP network model [25] via vectors of probabilities sampled from a hierarchical Dirichlet distribution.

Original: Unlike the formulation of Todeschiniet al.
Revised: Unlike the formulation of Todeschini et al.

Original: [18], our model enables learning the number of communities from data (see Secs.
Revised: [29], our model enables learning the number of communities from data (see Secs.

Original: 2.1 and 4).
Revised: 2.1 and 4).

Original: By using probability vectors rather than unconstrained non-negative values to model communitymemberships, our model provides a regularized approach to inference in the GGP model, which insimulations (Sec. 5.1) provides a substantial increase in community detection accuracy.∞ ¯ D α N × N w i
Revised: By using probability vectors rather than unconstrained non-negative values to model community memberships, our model provides a regularized approach to inference in the GGP model, which in simulations (Sec. 5.1) provides a substantial increase in community detection accuracy.

Original: α σ τ π i ζ β γ K x ex ec ec e Y ij Let each node i have both a sociability parameter w i from the GGP as in (3), and a vector ofprobabilities π i =
Revised: Let each node i have both a sociability parameter w i from the GGP as in (3), and a vector of probabilities π i =

Original: ( π i 1 , . . . , π iK ) drawn from a hierarchical Dirichlet distribution as in (1).
Revised: ( π i 1 , . . . , π iK ) drawn from a hierarchical Dirichlet distribution as in (1).

Original: Moreover,let the number of potential edges between nodes i and j in the latent multigraph depend only on theirsociabilities, as in (4).
Revised: Moreover, let the number of potential edges between nodes i and j in the latent multigraph depend only on their sociabilities, as in (4).

Original: For each of the n ij potential edges, node i independently samples a communityfrom Cat ( π i ) , and node j samples a community from Cat ( π j ) .
Revised: For each of the n ij potential edges, node i independently samples a community from Cat ( π i ) , and node j samples a community from Cat ( π j ) .

Original: If these community assignmentsmatch, the edge is retained; otherwise, it is thinned (i.e., discarded).
Revised: If these community assignments match, the edge is retained; otherwise, it is thinned (i.e., discarded).

Original: See Figs.
Revised: See Figs.

Original: 1 and 3 for examples.
Revised: 1 and 3 for examples.

Original: More formally, let ˙ n ij be the number of multigraph edges between a pair of nodes i, j that is retained(not thinned).
Revised: More formally, let ˙ n ij be the number of multigraph edges between a pair of nodes i, j that is retained (not thinned).

Original: Edges e ∈ { 1 , . . .
Revised: Edges e ∈ { 1 , . . .

Original: , ¯ D α } in the GGP multigraph are stochastically thinned as follows:
Revised: , ¯ D α } in the GGP multigraph are stochastically thinned as follows:

Original: [Equation]
Revised: [Equation]

Original: Here x ev ∈ { 1 , 2 , . . .
Revised: Here x ev ∈ { 1 , 2 , . . .

Original: } for v = 1 , 2 indicate the nodes associated with edge e as in Eq.
Revised: } for v = 1 , 2 indicate the nodes associated with edge e as in Eq.

Original: Anedge Y ij is then present in the undirected graph if and only if at least one multigraph edge is notthinned.
Revised: An edge Y ij is then present in the undirected graph if and only if at least one multigraph edge is not thinned.

Original: Equivalently, Eq.
Revised: Equivalently, Eq.

Original: (7) implies that the observed undirected graph is a binary projection of themultigraph edges that are selected , i.e. retained after the thinning process.
Revised: (7) implies that the observed undirected graph is a binary projection of the multigraph edges that are selected , i.e. retained after the thinning process.

Original: To determine the distribution of Y ij , note that the probability that nodes i and j are assigned to thesame community (marginalizing across communities) equals P ( c ei = c ej )
Revised: To determine the distribution of Y ij , note that the probability that nodes i and j are assigned to the same community (marginalizing across communities) equals P ( c ei = c ej )

Original: = (cid:80)
Revised: = (cid:80)

Original: Kk =

Revised: Kk =

Original: π ik π jk .
Revised: π ik π jk .

Original: Therefore, marginalizing over the latent multigraph, ˙ n ij | w i , w j , π i , π j ind ∼ Poisson ( w i w j (cid:80) Kk
Revised: Therefore, marginalizing over the latent multigraph, ˙ n ij | w i , w j , π [Equation] , π [Equation]

Original: =1 π ik π jk ) .
Revised: ind ∼ Poisson ( w [Equation] w [Equation]

Original:  The likelihood of the observed network thus equals
Revised: (cid:80) Kk [Equation] The likelihood of the observed network thus equals

Original: [Equation]
Revised: [Equation]

Original: Unlike the MMSB and GGP models, our thinned generalized gamma process (TGGP) model favorsedges between nodes that have both large sociabilities and similar community memberships.
Revised: Unlike the MMSB and GGP models, our thinned generalized gamma process (TGGP) model favors edges between nodes that have both large sociabilities and similar community memberships.

Original: The
Revised: The

Original: TGGP model is summarized graphically in Fig.
Revised: TGGP model is summarized graphically in Fig.

Original: 2 and illustrated in Figs.
Revised: 2 and illustrated in Figs.

Original: 1, 3, 4a.
Revised: 1 and 3. [Figure]

Original:  Monte Carlo Posterior Inference
Revised:  Monte Carlo Posterior Inference

Original: In our TGGP model, the latent (unthinned) multigraph has a distribution (Eq.
Revised: In our TGGP model, the latent (unthinned) multigraph has a distribution (Eq.

Original: that depends onlyon the node sociability parameters w i .
Revised: that depends only on the node sociability parameters w i .

Original: The thinning process (Eq.
Revised: The thinning process (Eq.

Original: (7)) then depends only on thecommunity membership vectors
Revised: (7)) then depends only on the community membership vectors

Original: π i .
Revised: π i .

Original: As can be deduced from the Markov properties of the directedgraphical model in Fig. 2, given these latent multigraphs, the posterior distributions of ( w 1 , w 2 , . . . )
Revised: As can be deduced from the Markov properties of the directed graphical model in Fig. 2, given these latent multigraphs, the posterior distributions of ( w 1 , w 2 , . . . )

Original: and ( π 1 , π 2 , . . . ) are conditionally independent.
Revised: and ( π 1 , π 2 , . . . ) are conditionally independent.

Original: This design simplifies posterior inference, andin particular allows us to apply Theorem 6 of [13] to derive the posterior of nodes’ sociabilities,provided we can condition on the (latent) multigraph.
Revised: This design simplifies posterior inference, and in particular allows us to apply Theorem 6 of [25] to derive the posterior of nodes’ sociabilities, provided we can condition on the (latent) multigraph.

Original: We thus implement a Gibbs sampling strategyfor posterior inference where we sample both thinned and retained edges in the latent multigraph, aswell as their community assignments, to allow for efficient updates of other variables.
Revised: We thus implement a Gibbs sampling strategy for posterior inference where we sample both thinned and retained edges in the latent multigraph, as well as their community assignments, to allow for efficient updates of other variables.

Original: Some steps in the sampler are straightforwardly derived, considering that we can borrow the approachdetailed in [13] to update node sociabilities and GGP hyperparameters (see the Appendix for a [Figure]
Revised: Some steps in the sampler are straightforwardly derived, considering that we can borrow the approach detailed in [25] to update node sociabilities and GGP hyperparameters (see the Appendix for a detailed outline).

Original: detailed outline). However, the implementation of the sampler requires some additional carefuldevelopment – which we detail below – to sample the latent multigraph, due to the unobservedthinning process.
Revised: However, the implementation of the sampler requires some additional careful development – which we detail below – to sample the latent multigraph, due to the unobserved thinning process.

Original: Multigraph sampling is made more computationally efficient via model propertiesthat are unique to sparse, as opposed to dense, stochastic block models.
Revised: Multigraph sampling is made more computationally efficient via model properties that are unique to sparse, as opposed to dense, stochastic block models.

Original: Letting (filled dot) ˙ n ij denotethe number of unthinned (i.e. selected) multigraph edges as in (7), and (empty dot) ˚ n ij = n ij − ˙ n ijdenote the number of thinned multigraph edges, we sample the latent multigraph as follows.
Revised: Letting (filled dot) ˙ n ij denote the number of unthinned (i.e. selected) multigraph edges as in (7), and (empty dot) ˚ n ij = n ij − ˙ n ij denote the number of thinned multigraph edges, we sample the latent multigraph as follows.

Original: Sampling of ˙n ij : From Eq.
Revised: Sampling of ˙n ij : From Eq.

Original: (7), there exists an edge in the observed graph if and only if there is at least one unthinned edge between nodes i and j in the latent multigraph.
Revised: (7), there exists an edge in the observed graph if and only if there is at least one unthinned edge between nodes i and j in the latent multigraph.

Original: Thus rather than considering each pair i, j , we only need to sample ˙ n ij when Y ij = 1 , an operation with cost linear in the number of observed (undirected) edges.
Revised: Thus rather than considering each pair i, j , we only need to sample ˙ n ij when Y ij = 1 , an operation with cost linear in the number of observed (undirected) edges.

Original: Conversely, Y ij = 0 implies ˙ n ij = ˙ n ji = 0 .
Revised: Conversely, Y ij = 0 implies ˙ n ij = ˙ n ji = 0 .

Original: Since there must be at least one unthinned edge when Y ij = 1 , the posterior follows a zero-truncated Poisson distribution:[Equation] Zero-Trunc-Poisson
Revised: Since there must be at least one unthinned edge when Y ij = 1 , the posterior follows a zero-truncated Poisson distribution:

Original: [Equation]
Revised: [Equation]

Original: For each unthinned edge we sample a single community assignment, since by definition these are theedges whose nodes have been assigned to the same communities.
Revised: For each unthinned edge we sample a single community assignment, since by definition these are the edges whose nodes have been assigned to the same communities.

Original: We can sample the communityassignment of an unthinned edge between i and j easily as a draw from Cat ( π i 1 π j 1 , . . .
Revised: We can sample the community assignment of an unthinned edge between i and j easily as a draw from Cat ( π i 1 π j 1 , . . .

Original: , π iK π jK ) .
Revised: , π iK π jK ) .

Original: Sampling of ˚n ij : By construction, since thinned edges are unobserved, all pairs of nodes ( i, j ) mayhave edges that are thinned.
Revised: Sampling of ˚n ij : By construction, since thinned edges are unobserved, all pairs of nodes ( i, j ) may have edges that are thinned.

Original: These edges are auxiliary variables necessary to obtain the full conditionalposterior distribution of W α .
Revised: These edges are auxiliary variables necessary to obtain the full conditional posterior distribution of W α .

Original: Also, to update community memberships and global frequencies, it isnecessary to assign every thinned edge to a pair of discordant (non-matching) communities.
Revised: Also, to update community memberships and global frequencies, it is necessary to assign every thinned edge to a pair of discordant (non-matching) communities.

Original: Thanksto the properties of Poisson processes, we do not need to sample ˚ n ij for all the pairs of nodes; wecan develop a more efficient sampler that exploits the thinned model construction.
Revised: Thanks to the properties of Poisson processes, we do not need to sample ˚ n ij for all the pairs of nodes; we [Figure] [Figure]

Revised: N a

Revised: N

Original: We first samplea proposed total number of thinned edges ˚ N ∼ Poisson (cid:16) 2 (cid:80) Ni,j =1 w i w j (cid:17) .
Revised: [Figure] [Figure] a α −0. 20000 iteration s σ 0.

Revised: [Figure]

Revised: t

Revised: [Figure]

Revised: can develop a more efficient sampler that exploits the thinned model construction.

Revised: We first sample a proposed total number of thinned edges ˚ N ∼ Poisson (cid:16) 2 (cid:80) Ni,j =1 w i w j (cid:17) .

Original: We then assign each ofthese independently to a pair of nodes according to their sociabilities as in Eq.
Revised: We then assign each of these independently to a pair of nodes according to their sociabilities as in Eq.

Original: (5), and subsequentlyto a pair of commmunities given the memberships of the assigned nodes as in Eq.
Revised: (5), and subsequently to a pair of commmunities given the memberships of the assigned nodes as in Eq.

Original: Finally, wedetermine ˚ n ij from the number of edges that were assigned to discordant communities.
Revised: Finally, we determine ˚ n ij from the number of edges that were assigned to discordant communities.

Original: Sampling unobserved nodes: As illustrated in Figs.
Revised: Sampling unobserved nodes: As illustrated in Figs.

Original: 1 and 3, the total number of nodes N α with atleast one edge in the unthinned multigraph is likely to be different from the total number of nodeswith at least one edge after thinning and – as a consequence – from the total number of nodes N in theobserved binary graph Y .
Revised: 1 and 3, the total number of nodes N α with at least one edge in the unthinned multigraph is likely to be different from the total number of nodes with at least one edge after thinning and – as a consequence – from the total number of nodes N in the observed binary graph Y .

Original: There is thus some latent number N α − N ≥ 0 of nodes whose edges withthe observed nodes have all been thinned.
Revised: There is thus some latent number N α − N ≥ 0 of nodes whose edges with the observed nodes have all been thinned.

Original: In order to learn this number, our MCMC sampler includesan approximate update of N α according to the mean ratio N α /N from graphs simulated from the
Revised: In order to learn this number, our MCMC sampler includes an approximate update of N α according to the mean ratio N α /N from graphs simulated from the

Original: GGP prior given the latest samples of the hyperparameters α, σ, τ .
Revised: GGP prior given the latest samples of the hyperparameters α, σ, τ .

Original: We found that this approximatemethod leads to convergence of the empirical MCMC-based estimate of N α across all simulated and[Figure] [Figure] real data that we considered (see Appendix).
Revised: We found that this approximate method leads to convergence of the empirical MCMC-based estimate of N α across all simulated and real data that we considered (see Fig. 4).

Original: Sampling of the thinned edges from these unobservednodes introduces some additional complications that we efficiently resolve in the Appendix.
Revised: Sampling of the thinned edges from these unobserved nodes introduces some additional complications that we efficiently resolve in the Appendix.

Original: Sampling community memberships π i  : Conditioned on the discrete community assignments foredges in the latent multigraph, the  node community memberships π i have closed-form Dirichletposteriors (see Appendix).
Revised: Sampling community memberships π i and global frequencies β : From the conjugacy of Dirichlet priors to categorical likelihoods, node community memberships π i have closed-form Dirichlet posteriors π i | M i , β ind ∼ Dirichlet ( ζβ 1 + M i 1 , . . , ζβ K + M iK ) , where M ik equals the number of edges that node i participated in while being assigned to community k .

Original: Given π , the global community frequencies β may be efficiently resampledusing auxiliary-variable methods developed for the hierarchical Dirichlet process [6].
Revised: Given community assignments of nodes in the latent multigraph, the global community frequencies β may be efficiently resampled using auxiliary-variable methods developed for the hierarchical Dirichlet process [6] (see Appendix).

Original: By learning β ,the node community memberships π i become sparse, placing significant probability mass only on adata-dependent subset of the full set of K communities allocated in Eq.
Revised: By learning β , node community memberships π i become sparse, placing significant probability mass only on a data-dependent subset of the full set of K communities allocated in Eq.

Original: Computational complexity: For a K -community TGGP model of a graph with E observed edges,resampling the latent multigraph requires O ( EK ) operations.
Revised: Computational complexity: For a K -community TGGP model of a graph with E observed edges, resampling the latent multigraph requires O ( EK ) operations.

Original: (The thinned multigraph has feweredges, and thus its resampling increases costs by a small constant.)
Revised: (The thinned multigraph has fewer edges, and thus its resampling increases costs by a small constant.)

Original: Closed-form resampling ofcommunity memberships is faster, requiring only O ( NK ) operations for an N -node graph.
Revised: Closed-form resampling of community memberships is faster, requiring only O ( NK ) operations for an N -node graph.

Original: The
Revised: The

Original: CGGP model [18] uses Hamiltonian Monte Carlo proposals with similar cost, but empirically the
Revised: CGGP model [29] uses Hamiltonian Monte Carlo proposals with similar cost, but empirically the

Original: CGGP sampler mixes slower and has inferior performance (see experiments).
Revised: CGGP sampler mixes slower and has inferior performance (see experiments).

Original: By exploiting sparsematrices and parallelization, both samplers may be scaled to networks with tens of thousands ofnodes; very-large networks may require alternative approximate inference algorithms. 
Revised: By exploiting sparse matrices and parallelization, both samplers may be scaled to networks with tens of thousands of nodes; very-large networks may require alternative approximate inference algorithms. MCMC sampler validation: We evaluate the proposed posterior inference method on a graph with 15 communities simulated from the TGGP model.

Revised: The simulated graph had 1571 undirected, community-concordant edges, N = 386 observed nodes (i.e., with at least one community-concordant edge), and N α − N = 192 unobserved nodes (i.e., with only community-discordant or self edges).

Revised: [Figure]

Original: Experimental Results
Revised: [Figure] [Figure] [Figure] [Figure]

Original: 5.1 Simulation
Revised: Fig. 4 (top) shows traceplots from three separate MCMC chains, each of which was run for 20, iterations. The traceplot of the total number of nodes (observed + unobserved) N α shows all values sampled since initialization to demonstrate that even very different initializations led the three MCMC chains to quickly concentrate around the same (true) value of N α . The traceplots of the GGP hyperparameters α, σ , and τ show samples after convergence (which took roughlyiterations) and demonstrate that the posterior distribution effectively concentrated around the true values of these hyperparameters.

Original:  We discuss a simulation study where we investigated the performance of our proposed TGGP model,and the CGGP model of [18], based on simulated data generated from either model.
Revised: Fig. 4 (bottom) shows summaries of the posterior distributions of nodes’ sociabilities for 40 randomly selected nodes spanning from low to relatively large sociability, suggesting that the TGGP model also effectively recovers nodes’ sociability parameters.  Experimental Results 5.1 Simulation We discuss a simulation study where we investigated the performance of our proposed TGGP model, and the CGGP model of [29], based on simulated data generated from either model.

Original: First, we present the results from a sparse graph with 15 communities, simulated from our TGGPmodel by setting α = 250 , σ = 0 .
Revised: First, we present the results from a sparse graph with 15 communities, simulated from our TGGP model by setting α = 250 , σ = 0 .

Original: 1 , τ = 1 for the distribution of node sociabilities andγ = 10 , ζ = 0 .
Revised: 1 , τ = 1 for the distribution of node sociabilities and γ = 10 , ζ = 0 .

Original: 2 for the distribution of node community memberships.
Revised: 2 for the distribution of node community memberships.

Original: The adjacency matrix of the resultingsimple, undirected graph is shown in Figure 4(a, bottom).
Revised: The adjacency matrix of the resulting undirected graph is shown in Figure 5(a, bottom).

Original: Figure 4(b, bottom) sorts the simulated nodes into blocks according to their main membership and plots thedensity of edges in each block. We see that the simulated graph has a clear block structure.
Revised: Figure 5(b, bottom) sorts the simulated nodes into blocks according to their main membership and plots the density of edges in each block, demonstrating that the simulated graph has a clear block structure.

Original: We then run our MCMC sampler for 50,000 iterations, discarding the first 40,000 samples asburn-in.
Revised: We then run our MCMC sampler for 50, iterations, discarding the first 40,000 samples as burn-in.

Original: For our model fitting, we let 50 be theupper bound to the number of communities andwe set γ = 10 and ζ = 0 .
Revised: For our model fitting, we let 50 be the upper bound to the number of communities and we set γ = 10 and ζ = 0 .

Original: 5 to allow for learningthe number of communities.
Revised: 5 to allow for learning the number of communities.

Original: From a qualitative comparison of the (b) and (c) bottom panesin Figure 4, we see that the community memberships recovered from our model are closeto the underlying truth.
Revised: From a qualitative comparison of the (b) and (c) bottom panes in Figure 5, we see that the community memberships recovered from our model are close to the underlying truth.

Original: The credible intervalsindicate that the posterior distributions of allparameters tend to concentrate around the truevalues used for simulation (see Appendix forplots).
Revised: The credible intervals indicate that the posterior distributions of all parameters tend to concentrate around the true values used for simulation (see Appendix for plots).

Original: In contrast, the CGGP model [18] struggles to recover the true community structure ofthe network (Fig. 4(d bottom)), despite beingfitted with the number of communities K = 15set to match the truth.
Revised: In contrast, the CGGP model [29] struggles to recover the true community structure of the network (Fig. 5(d bottom)) , despite being fitted with the number of communities K = 15 set to match the truth.

Original: We also simulated a graphfrom the CGGP model of [18] with parametersa k = 1 /K, b k = 1 chosen to induce a community membership prior similar to our TGGP.
Revised: We also simulated a graph from the CGGP model of [29] with parameters a k = 1 /K, b k = 1 chosen to induce a community membership prior similar to our TGGP.

Original: We fit boththe CGGP model and the TGGP model to the generated data.
Revised: We fit both the CGGP model and the TGGP model to the generated data.

Original: When fitting with the CGGP we stillset the number of communities equal to the truth ( K = 15 ).
Revised: When fitting with the CGGP we still set the number of communities equal to the truth ( K = 15 ).

Original: The results plotted in Fig. 4(d) indicatethat the block structures learned by our (misspecified) TGGP model are nevertheless closer to truththan those recovered by the CGGP.
Revised: The results plotted in Fig. 5(d) indicate that the block structures learned by our (misspecified) TGGP model are nevertheless closer to truth than those recovered by the CGGP.

Original: To quantify the superior community recovery of our TGGP model, recall that for each observed nodethe CGGP estimates a K -dim.
Revised: To quantify the superior community recovery of our TGGP model, recall that for each observed node the CGGP estimates a K -dim.

Original: positive real vector of community-specific sociabilities ( w i 1 , . . .
Revised: positive real vector of community-specific sociabilities ( w i 1 , . . .

Original: , w iK ) ,while our TGGP infers a K -dim.
Revised: , w iK ) , while our TGGP infers a K -dim.

Original: probability vector of community memberships ( π i 1 , . . . , π iK ) .
Revised: probability vector of community memberships ( π i 1 , . . . , π iK ) .

Original: Forquantitative comparison, we standardize true and inferred sociability vectors by dividing each w ikby the sum (cid:80)
Revised: For quantitative comparison, we standardize true and inferred sociability vectors by dividing each w ik by the sum (cid:80)

Original: Kh =1 w ih .
Revised: Kh =1 w ih .

Original: Also, since the TGGP model was fitted with K set to an upper-bound of 50rather than to the true number of communities, when analyzing TGGP results we pad the true vectorsof community memberships (for TGGP-simulated data) and sociabilities (for CGGP-simulateddata) with zeros.
Revised: Also, since the TGGP model was fitted with K set to an upper-bound ofrather than to the true number of communities, when analyzing TGGP results we pad the true vectors of community memberships (for TGGPsimulated data) and sociabilities (for CGGPsimulated data) with zeros.

Original: For estimated parameters, community indices are then permuted to maximizesimilarity with true parameters.
Revised: For estimated parameters, community indices are then permuted to maximize similarity with true parameters.

Original: Letting ψ i and ˆ ψ i be, respectively, true and inferred communitymemberships/sociabilities, we define similarity as 1 − 1 N (cid:80) Ni =1 d ( ψ i , ˆ ψ i ) where the distance is the L1 total variation d ( ψ i , ˆ ψ i ) =
Revised: Letting ψ i and ˆ ψ i be, respectively, true and inferred community memberships/sociabilities, we define similarity as 1 − 1 N (cid:80) Ni =1 d ( ψ i , ˆ ψ i ) where the distance is the L1 total variation d ( ψ i , ˆ ψ i ) =

Original: 12 (cid:80)
Revised: 12 (cid:80)

Original: Kk =1 | ψ ik − ˆ ψ ik | .
Revised: Kk =1 | ψ ik − ˆ ψ ik | .

Original: Fig.
Revised: Fig.

Original: 5 confirms that the TGGP more accuratelyrecovers community memberships for both networks, and for nodes of both high and low degree.
Revised: 6 confirms that the TGGP more accurately recovers community memberships for both networks, and for nodes of both high and low degree.

Original: 5.2 Real network data
Revised: 5.2 Real network data

Original: We compare the performance of our TGGP framework for adding overlapping communities to the GGP model with the approaches detailed in Sec. 2.2: the CGGP (sparse with mixed memberships)of [18] and the SBM-GGP model (sparse with single memberships) of [17].
Revised: We compare the performance of our TGGP framework for adding overlapping communities to the GGP model with the approaches detailed in Sec. 2.2 and 2.1: the CGGP (sparse with mixed memberships) [29], the SBM-GGP model (sparse with single memberships) [28], and the more classical SBM (dense with single membership) [2] and MMSB (dense with mixed memberships) [5].

Original: We also compare tomore classical models from Sec. 2.1: the SBM (dense with single membership) of [2] and the MMSB(dense with mixed memberships) of [5].

Original: We run 50,000 iterations of the five models’ MCMC on fourreal-world networks; see Appendix for data sources and pre-processing.
Revised: We run 50,000 iterations of the five models’ MCMC on four real-world networks; see Appendix for data sources and pre-processing.

Original: Each model was fit to fullyobserved data to learn node-specific parameters (e.g., sociabilities and community memberships) andcommunity-interaction probabilities, using the values from the final MCMC iteration.
Revised: Each model was fit to fully observed data to learn node-specific parameters (e.g., sociabilities and community memberships) and community-interaction probabilities, using the values from the last MCMC iteration.

Original: Two differentmeasures of posterior predictive accuracy  were then used to assess the goodness of model fit.
Revised: Two different measures of posterior predictive accuracy [42] were then used to assess the goodness of model fit.

Original: For the first evaluation measure, we randomly selected 5% of the entries equal to 1 in the adjacencymatrix and set them to 0.
Revised: For the first evaluation measure, we randomly selected 5% of the entries equal to 1 in the adjacency matrix and set them to 0.

Original: We then compute the probability of being 1 of all entries that are equal to0 in the modified adjacency matrix to assess how well each model can distinguish the entries thatshould in fact be 1.
Revised: We then compute the probability of being 1 of all entries that are equal to 0 in the modified adjacency matrix to assess how well each model can distinguish the entries that should in fact be 1.

Original: This edge-retrieval measure is motivated by common applications of networkanalysis to recommendation tasks (suggesting which edges that are not observed should be present).
Revised: This edge-retrieval measure is motivated by common applications of network analysis to recommendation tasks (suggesting which edges that are not observed should be present).

Original: Fig.
Revised: Fig.

Original: 6 (top) plots recall (proportion of ones recovered) on the horizontal axis, and F-score (geometricmean of recall and precision, which is the proportion of ones correctly predicted among all entriespredicted as ones) on the vertical axis.
Revised: 7 (top) plots recall (proportion of ones recovered) on the horizontal axis, and F-score (geometric mean of recall and precision, which is the proportion of ones correctly predicted among all entries predicted as ones) on the vertical axis.

Original: This retrieval task is very challenging because, for sparsenetworks, the missing edges are a tiny proportion of all zero entries in the modified adjacency matrix.
Revised: This retrieval task is very challenging because, for sparse networks, the missing edges are a tiny proportion of all zero entries in the modified adjacency matrix.

Original: For the second evaluation measure, we randomly obscure 5% of all entries in the adjacency matrixto assess how well each model can recover whether an entry is equal to 0 or to 1.
Revised: For the second evaluation measure, we randomly obscure 5% of all entries in the adjacency matrix to assess how well each model can recover whether an entry is equal to 0 or to 1.

Original: Fig.
Revised: Fig.

Original: 6 (bottom)plots the receiver operating characteristic (ROC) curves, where false positive rate (number of zerospredicted as ones divided by the number of zeros) is on the horizontal axis, and true positive rate(proportion of ones correctly predicted as ones divided by the number of ones) is on the vertical axis.
Revised: 7 (bottom) plots the receiver operating characteristic (ROC) curves, where false positive rate (number of zeros predicted as ones divided by the number of zeros) is on the horizontal axis, and true positive rate (proportion of ones correctly predicted as ones divided by the number of ones) is on the vertical axis.

Original: This second task has been used in many prior papers but is not as hard as the first one, because for amodel to score well it suffices to learn small interaction probabilities, since almost all of the randomlyselected entries are 0.
Revised: This second task has been used in many prior papers but is not as hard as the first one, because for a model to score well it suffices to learn small interaction probabilities, since almost all of the randomly selected entries are 0.

Original: Fig.
Revised: Fig.

Original: 6 shows that our TGGP model does consistently better than all baselinemethods for all networks considered, according to both predictive-performance measures.
Revised: 7 shows that our TGGP model does consistently better than all baseline methods for all networks considered, according to both evaluation measures.

Original:  Discussion
Revised:  Discussion

Original: We have proposed a framework for the analysis of binary network data that extends the GGP-basedmodel of Caron and Fox [13] by allowing for overlapping community structures, depending bothon the overall sociability of the nodes and the similarity of their community memberships.
Revised: We have proposed a framework for the analysis of binary network data that extends the GGP-based model of Caron and Fox [25] by allowing for overlapping community structures, depending both on the overall sociability of the nodes and the similarity of their community memberships.

Original: Theassumed generative model proposes a novel latent multi-graph framework where nodes are allowedto connect within the same or across different communities, but the edges formed across differentcommunities are hidden (thinned) in the projection giving rise to the observed network.
Revised: Our generative model uses a novel latent multi-graph framework where nodes can connect within the same or across different communities, but the edges formed across different communities are hidden (thinned) in the projection giving rise to the observed network.

Original: With respectto alternative extensions of the original GGP network model [17, 18], our TGGP enables mixedcommunity memberships and facilitates regularization of thelatent community distributions for eachnode, resulting in improved inference and reconstruction of the latent community structures for bothsimulated and real data.
Revised: In contrast with alternative extensions of the original GGP network model [28, 29], our thinned generalized gamma process (TGGP) enables mixed memberships and facilitates regularization of the community distributions for each node, resulting in improved inference and reconstruction of the latent community structures.

Original: Furthermore, our modelformulation allows for learning the number ofcommunities directly from the data, by encouraging inferring a number of non-empty communitiespotentially smaller than a set upper bound. 
Revised: Also, our model allows for learning the number of communities directly from the data, by encouraging recovery of a set of non-empty communities smaller than a specified upper bound. Monte Carlo inference for the TGGP scales linearly with the number of observed edges, and thus subquadratically with the number of nodes in sparse graphs. This leads to strong edge prediction performance for social and biological networks of moderate size. However, further inference innovations may be needed to scale to very-large networks with hundreds of thousands of nodes.

Original: The proposed TGGP model is amenable to further extensions.
Revised: The proposed TGGP model is amenable to further extensions.

Original: Since our mixed membership allocationframework relies on the well-studied dirichlet-multinomial specification, it is possible to leverage thevast literature on(finite and infinite) hierarchical dirichlet mixture models to capture more complexand heterogeneous network data.
Revised: As the adopted mixed membership framework relies on the well-studied Dirichlet-multinomial specification, the vast literature on hierarchical Dirichlet mixtures can be leveraged to capture more complex networks.

Original: It isalso possible to include additional node covariates or metadatato guide the allocation of nodes to communities.
Revised: It is possible to include additional node covariates or metadata to guide the allocation of nodes to communities.

Original: Finally, distributional and asymptotic propertiesof our modelformulation can be investigated by exploitingrecent resultsin the literature related tofinite mixtures and mixtures of finite mixtures [see, e.g., 27, 28, 29, 19].
Revised: Finally, distributional and asymptotic properties of our model can be investigated by exploiting results related to finite mixtures and mixtures of finite mixtures [see, e.g., 43, 44, 45, 34]. Acknowledgments and Disclosure of Funding This research supported in part by NSF CAREER Award No. IIS1758028, and by the HPI Research Center in Machine Learning and Data Science at UC Irvine.

Original: Checklist
Revised: A Monte Carlo Posterior Inference

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: We review the Gibbs sampling steps, some of which employ Metropolis-Hastings proposals rather than exact updates, for our thinned generalized gamma process (TGGP) model. We then describe each step in more detail.

Original: [Yes] See Sections 1, 3, 4, 5, and 
Revised: Update the edge counts and community assignments in the latent multigraph; 2. Update nodes’ community memberships; 3. Update the global expected value of community memberships; 4.

Original: (b) Did you describe the limitations of your work?
Revised: Update nodes’ sociabilities; 5. Update the hyperparameters of the distribution of sociabilities; 6. Update the dimension of the latent multigraph. A.1 Updating edge counts and community assignments in the latent multigraph

Revised: We start by recalling that, in our TGGP model, the observed simple undirected graph is a binary projection of an unobserved directed multigraph.

Revised: For each edge in which a node participates in the multigraph, the node is assigned to a community.

Revised: For a given (non-self) edge, if the communities that the two nodes are assigned to coincide, we say that the edge is community-concordant .

Revised: Otherwise, if the communities differ, we say that the edge is community-discordant .

Revised: We note there may be self-edges in the latent multigraph.

Revised: However, since we assume that self-edges are not observed irrespective of community assignments, no distinction is needed between community-concordant and discordant self-edges and we refer to any of them simply as self-edges.

Revised: Two nodes have an edge in the observed undirected graph if there is at least one community-concordant edge between those nodes in the multigraph.

Revised: Observed nodes are those that participate in at least one community-concordant edge.

Revised: The nodes that in the multigraph are connected only within communitydiscordant or self-edges are unobserved nodes .

Original: [Yes] See Section 1, 3, 4, 5 and 
Revised: We call N the number of observed nodes and we let i = N + 1 , . . . , N α denote the indices of unobserved nodes. To easily sample from the posterior of nodes’ sociabilities we need (for both observed and unobserved nodes) each node’s total degree, which equals the total number of edges in the multigraph (both community-concordant and -discordant) in which the node participates. Also, to easily sample from the posterior of nodes’ community memberships, we need their community assignment for each of the edges that they are connected to in the multigraph (both community-concordant and -discordant).

Revised: For both purposes, it suffices to track the entries of a single N α × K matrix of summary statistics M , where M ik equals the number of edges that node i participated in while being assigned to community k .

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: The total degree of node i may then be obtained as M i · = (cid:80) Kk =1 M ik . We denote the counts of the assignments of a node i to each community (that is, row i of M ) by M i = ( M i 1 , . . . , M iK ) .

Revised: We will break down the sampling of the entries of M in three steps: first edge counts and community assignments for community-concordant edges, second community-discordant and self-edges between observed nodes, and finally community-discordant and self-edges involving unobserved nodes.

Revised: See

Revised: Figure 8 for an illustration.

Revised: A.1.1 Community-concordant edges

Revised: Recall that we assumed that an edge between two nodes is observed if and only if there is at least one community-concordant edge between those nodes in the latent multigraph.

Revised: Then, the presence of an edge between nodes i and j in the observed graph tells us that ˙ n ij + ˙ n ji > 0 , that is the sum of directed, community-concordant edges from i to j and from j to i in the latent multigraph is strictly positive.

Revised: Similarly, the absence of an edge in the observed graph tells us that there is no community-concordant edge.

Revised: Therefore, the number of community-concordant edges between two nodes only needs to be sampled for those pairs for which Y ij = 1 , ( i ≥ j ).

Revised: An illustration is in Figure 8a.

Revised: Because a priori , the number of community-concordant edges from i to j and from j to i have the same Poisson (cid:16) w i w j (cid:80)

Original: {+Kk =
+}
Revised: {+Kk =
+}

Revised: π ik π jk (cid:17) distribution, a posteriori we have

Original: [N/A]
Revised: [Equation]

Revised: By definition, the community assignment of the nodes involved in a community-concordant edge coincide.

Revised: So we need to sample a single community assignment valid for both nodes i and j for each of the ˙ n ij + ˙ n ji community-concordant edges:

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: Multinomial

Revised: [Equation]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (cid:19)(cid:33) , and we increment M i = ( M i 1 , . . . , M iK ) and M j = ( M j 1 , . . . , M jK ) accordingly. Computational efficiency. Note that the computational cost of sampling community-concordant edges is linear in the number of observed edges.

Revised: A.1.2 Community-discordant and self-edges between observed nodes

Original: them?
Revised: Because community-discordant edges in the latent multigraph don’t affect the observed simple graph, all pairs of observed nodes can have community-discordant edges, both those pairs i and j for which Y ij = 1 and those pairs i ′ and j ′ for which Y i ′ j ′ = 0 .

Revised: We thus need to sample the total number ˚ n ij + ˚ n ji of community-discordant edges for all pairs i, j ∈ { 1 , . .

Revised: , N } with i ≥ j (see Figure 8b).

Revised: Because they are independent of the observed graph (conditional on nodes’ sociabilities and memberships), the posterior distribution of communitydiscordant edges is the same as their prior:

Original: [Yes]
Revised: [Equation]

Revised: However, rather than considering every pair of nodes, we can leverage the properties of independent

Revised: Poisson-distributed random variables to draw (˚ n ij + ˚ n ji ) for all pairs of nodes from the correct distribution in a more efficient way.

Revised: Specifically, we can first sample an auxiliary variable ˜ N E from the prior distribution on the total number of edges and self-edges involving only observed nodes:

Revised: [Equation]

Revised: Then, we can proceed by associating each edge e = 1 , . . . , ˜ N E to a pair of nodes based only on nodes’ sociabilities, and assigning each sampled node to a community based only on its own memberships: x ev iid ∼

Revised: [Equation]

Revised: Finally, restricting to the resulting discordant-community edges and self-edges, we have obtained a sample for them from the correct distribution.

Revised: Accordingly, we increment the counts ( M i 1 , . . .

Revised: , M iK ) of community assignments for observed nodes i = 1 , . . .

Revised: , N , i.e. the first N rows of M .

Revised: [Figure]

Revised: Computational efficiency.

Revised: Note that, using the strategy described in this subsection, the computational cost for sampling community-discordant and self edges depends on the cost of assigning ˜ N E edges to a pair of nodes.

Revised: Thanks to fast sampling techniques for sampling from discrete distributions like the alias method [46, 47, 48], this operation can be made to require only O ( N log N ) computational time for pre-processing, after which edge assignments x ev may be generated in constant time.

Revised: A.1.3 Community-discordant and self edges involving unobserved nodes

Revised: Let N α − N be the number of nodes that are active (have at least one connection) in the latent multigraph, but not in the observed simple graph.

Original: If you are including theoretical results...
Revised: Let the labels of the nodes be ordered so that i = N + 1 , . . , N α correspond to the labels of these unobserved nodes. Conditional on these nodes not being observed but having connections in the latent multigraph, the information that we know is that each of them can only have community-discordant or self-edges, but also that they must be associated to at least one community-discordant or self-edge in the latent multigraph. This constraint implies that some care is required when sampling edges involving unobserved nodes.

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: Specifically, we must generate the edge counts and associated nodes , that is simulate how many edges each unobserved node participates in, as well as what other nodes are associated to these edges. Given the nodes associated to an edge involving an unobserved node, community assignments can then be sampled quite easily, considering that they need to be discordant for non-self edges and that no restriction is applied to self-edges. Edge counts and associated nodes. To correctly incorporate the constraint that each unobserved node must be associated to at least one community-discordant or self-edge, the strategy that we adopt is to sample the total number of edges and the community assignments of unobserved nodes sequentially, starting from those involving node N + 1 with any other node (that is, the yellow entries in Figure 8c).

Original: [No] We provided{+Let s i , for i = N + 1 , . . . , N α , be an auxiliary indicator variable that takes value 
 if node i has been associated to an edge already and 1 otherwise. The variable s i thus indicates if the constraint that node i must be associated to at least one edge is still active ( s i = 1 ) or is inactive ( s i = 0 ).+}
Revised: {+Let s i , for i = N + 1 , . . . , N α , be an auxiliary indicator variable that takes value 
 if node i has been associated to an edge already and 1 otherwise. The variable s i thus indicates if the constraint that node i must be associated to at least one edge is still active ( s i = 1 ) or is inactive ( s i = 0 ).+}

Revised: As we start from node ( N + 1) , all constraints are active and so s i = 1 for all

Original:  the assumptions, results and references, given the limitation of pages.
Revised: i = N + 1 , . . . , N α . We then sample the total number of edges ˚ n ( N +1) between node ( N + 1) and any other node from [Equation] ˚ p ( N +1) =

Revised: N (cid:88)

Original: Most results we
Revised: [Equation] = [Equation] ( N

Original: employ follow from analogous results in [13], with similar assumptions.
Revised: [Equation] N α (cid:88) j = N + ˚ p ( N +1) j , ˚ p ( N +1) j = w j (cid:32) 1 −

Original: (b) Did you include complete proofs of all theoretical results?
Revised: K (cid:88) k = π ( N +1) k π jk (cid:33) .

Original: [No] Most results we employ
Revised: Given the total number of edges associated with node ( N + 1) , we set x ( N +1) e= i and associate another node (possibly node N + 1 itself) to each edge e = 1 , . . . , ˚ n ( N +1) , according to [Equation] We then set s ( N +1)

Original: follow from analogous results{+= 
 and also s i = 0 for every node i ∈ { N +2 , . . . , N α } that has been associated to some edges with node ( N + 1) . Next, we can move on to sampling all the remaining edges involving node N + 2 , that is those with any other node except node N + 1 ( the light green entries+} in [13]
Revised: {+= 
 and also s i = 0 for every node i ∈ { N +2 , . . . , N α } that has been associated to some edges with node ( N + 1) . Next, we can move on to sampling all the remaining edges involving node N + 2 , that is those with any other node except node N + 1 ( the light green entries+} in Figure 8c).

Revised: In general, when we sample edges for node ( N + d ) for d = 2 , . . .

Revised: , ( N α − N ) , there are two considerations we must keep into account: (1) the edges between node ( N + d ) and nodes ( N + 1) , . . .

Revised: , ( N + d − 1) have already been sampled and so those nodes should not be re-considered; (2) if s ( N + d ) = 0 , the constraint on node ( N + d ) is no longer active and we may possibly sample 0 additional edges.

Revised: Putting these considerations together and letting ˚ n ( N + d ) be the number of remaining edges to sample when considering node ( N + d ) , that is those with nodes { 1 , . . .

Revised: , N, N + d, . .

Revised: . , N α } , we have ˚ n ( N + d ) | ( w 1 , . .

Revised: . , w N α ) , ( π 1 , . .

Revised: , π N α ) , s ( N + d ) ∼   0-Trunc-Poisson (cid:16) 2 w ( N + d ) ˚ p ( N + d ) + w 2( N + d ) (cid:17) if s ( N + d ) = 1 ,

Original: If you ran experiments...
Revised: Poisson (cid:16) 2 w ( N + d ) ˚ p ( N + d ) + w 2( N + d ) (cid:17) if s ( N + d ) = 0 , where ˚ [Equation] Then for e = 1 , . . . , ˚ n ( N + d ) , we set x ( N + d ) e= N + d and sample

Original: (a) Did you include the code, data, and instructions needed to reproduce the main ex-
Revised: P ( x ( N + d ) e [Equation] ∝ [Equation]

Original: perimental results (either in the supplemental material or as a URL)?
Revised: ˚

Original: [No] The code
Revised: [Equation] N + d )

Original: will be made publicly available on Github upon acceptance.
Revised: [Equation]

Original: The datasets are publicly
Revised: , if j = 1 , . . . , N, N + d + 1 , . .

Original: available.
Revised: . , N α , w i if j = N + d. Community assignments.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: For every node i = N + 1 , . . . , N α with ˚ n i > 0 , and for each e = 1 , . . ˚ n i , we sample a community assignment c ( i ) e for node i given x ( i ) e 2 from

Original: [Yes] See Sections 4 nd
Revised: [Equation] for k = 1 , . . . , K .

Original: (c) Did you report error bars (e.g., with respect to  the random seed after running experi-
Revised: For non-self-edges, we rescale π ik accounting for the knowledge that the community to which the other node associated to edge e is assigned needs to differ from that to which node i is assigned for this edge. We increment M ic ( i ) e accordingly, and then given c ( i ) e and x ( i ) e 2 , we can sample a community assignment ¯ c ( i ) e for node x ( i ) e 2 from

Original: ments multiple times)?
Revised: P [Equation] i [Equation]

Revised: and then increment M x ( i ) e 2 ¯ c ( i ) e .

Revised: Notice that for non-self-edges, we have removed the community sampled for node i from the range of available communities for the assignment of node x ( i ) e 2 , and rescaled π x ( i ) e 2 accordingly.

Revised: Computational efficiency.

Revised: In our TGGP model, unobserved nodes will have low sociability parameters; otherwise, they would be likely to be linked to unthinned edges.

Revised: Thus ˚ n ( N + d ) tends to be small for all d = 1 , . .

Revised: , N α − N .

Revised: For sampling their edges and community assignments, it is just necessary to compute a ( N + 1)

Revised: × N α matrix ˚ P whose entries ˚ p ij represent the relative probability of associating node j to an edge sampled for node i .

Revised: A.2 Updating nodes’ community memberships

Revised: Here we describe the update of π i =

Revised: ( π i 1 , . . . , π iK ) , the distribution over community memberships of node i = 1 , . . . , N α .

Revised: Recall the prior on π i :

Revised: [Equation]

Original: [Yes] See Figure 5, for example(d) Did you include the total amount of compute and  the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: where β = ( β 1 , . . . , β K ) is a vector of community frequencies that itself has a Dirichlet prior, and equals the expected value of π i . For our experiments we set ζ = 0 . 5 , to encourage community memberships of individual nodes to be more polarized than β and typically only use a subset of the available communities.

Revised: For each edge in the latent multigraph to which node i is associated, the generative model has node i being assigned to a community sampled independently from (1 , . . .

Original: [No] We will do in  the resubmission4.
Revised: , K ) with probabilities ( π i 1 , . . . , π iK ) , like in multinomial sampling. The Dirichlet and the multinomial are conjugate distributions, so the posterior of π i is easily obtained. Indeed, conditional on the concentration parameter β

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: and the N α × K matrix M whose rows record the number of times M i = ( M i 1 , . . . M iK ) that an edge i = 1 , . . , N α was assigned to each community k = 1 , . . .

Revised: , K , the posterior distribution of the vectors of community memberships are obtained as:

Revised: [Equation]

Revised: An advantage of this specification is that it allows sparse community memberships, in that the sample ( π i 1 , . . .

Revised: , π iK ) may have elements that are (almost) exactly equal to zero.

Revised: A.3 Updating the global expected value of community memberships

Revised: The posterior distribution of the expected value β = ( β 1 , . .

Revised: . , β K ) of nodes’ community memberships informs about summaries of the latent community structure of a network, such as the likely number of active communities and the relative size of different communities.

Original: (a) If your work uses
Revised: Recall the prior on β : [Equation] where the value of γ controls the sparsity of the prior on β , with values of γ ≪ K being more sparsity-inducing. In our experiments, we fix γ = 10 but K = 50 .

Revised: This hierarchical prior approaches the hierarchical Dirichlet process [6] in the limit as K → ∞ .

Revised: Here we illustrate a strategy for sampling from the posterior of β = ( β 1 , . .

Revised: . , β K ) that relies on auxiliary variables drawn according to the so-called “Chinese restaurant franchise” representation of a hierarchical Dirichlet model [6].

Original:  existing assets, did you cite the creators?
Revised: We can represent each node i = 1 , . . . , N α as a restaurant, where every new community assignment for node i can be represented as a new customer entering restaurant i and ordering a dish (community) k ∈ { 1 , . . , K } . When a new customer enters, they decide to sit at an existing table t ∈ { 1 , . .

Original: [Yes] See Section 5(b)
Revised: , T i } with probability proportional to the number of customers currently sitting at that table (and eat the dish k t that was ordered by the first customer who sat at the table); or, they can start a new table T i + 1 with probability proportional to ζ , and pick a new dish k T i + from the menu according to β . Let t ie be the table chosen when customer e enters restaurant i , or equivalently the table chosen for the community assignment of node i for when edge e is associated to it. Then letting T ( e ) i be the number of tables already started at restaurant i when customer e enters, and letting q ( e ) =

Original: Did you mention the license of the assets?
Revised: ( q ( e ) i 1 , . . , q ( e ) iT ( e ) i ) be the number of customers already sitting at each of the tables when customer e enters: [Equation] Conditional on t ie and letting ˜ κ t be the dish served at table t , the dish κ ie ∈ { 1 , . .

Revised: , K } ordered when customer e enters restaurant i is sampled according to

Original: [N/A](c)
Revised: [Equation]

Revised: From the last sampling equation we see that the posterior distribution of β depends on how many tables (rather than customers) are being served each dish across all restaurants in the franchise.

Original: Did you include any new assets either in the supplemental material or as a URL?
Revised: Let T ik be the number of tables serving dish k at restaurant i and let T • k = (cid:80) N α i =1 T ik be the total number of tables serving dish k across all restaurants in the franchise. The summary statistics that we need for the posterior of β is the vector T =

Original: [N/A] (d) Did you discuss whether
Revised: ( T • 1 , . . . , T • K ) . We then sample, for every restaurant i = 1 , . . . , N α (i.e. for every node) and every dish k = 1 , . .

Original:  and how consent was obtained from people whose data you’reusing/curating?
Revised: , K (i.e. for every community), the number of tables T ik ≤ M ik occupied by the M ik customers eating dish k at restaurant i . Note that M ik is the entry in row i and column k of the N α × K matrix M sampled at step 1 (see Section A.1). Because the distribution of table arrangement is invariant to the order in which customers enter, we can assign one at a time each of the e = 1 , . . .

Revised: , M ik customers eating dish k at restaurant i to either sit at an existing table ( t ike = 0 ) or to start a new table ( t ike = 1 ) according to:

Original: [N/A](e)
Revised: [Equation]

Revised: where we have ζβ k since we are conditioning on the customer being served

Revised: dish k

Revised: and we weight ζβ k with respect to ( e − 1) because, when considering customer e , there are already ( e − 1) customers sitting at some table serving dish k .

Original: Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
Revised: Thus, the chances that customer e sits at one of the existing tables is proportional to ( e − 1) . Finally, we set T ik = (cid:80) M ik e =1 t ike .

Revised: After sampling the auxiliary variables, we can easily update β according to

Original: [N/A]5.
Revised: [Equation]

Revised: A.4 Updating nodes’ sociabilities

Revised: Let M i • =

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: (cid:80) Kk =1 M ik be the total degree of node i (whose sampling was described in Section A.1). Conditional on the vector of total degrees ( M 1 • , . . , M N α • ) and on the hyperparameters α, σ and τ , the update of nodes’ sociabilities ( w 1 , . . . , w N α ) can be done via a Hamiltonian Monte Carlo step [49, 50] as detailed by Caron and Fox in Section 7.2 and in Appendix F.1 of [25].

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: We assume the same improper priors on the hyperparameters α, σ , and τ as in [25]: [Equation] To update these hyperparameters we follow the approach that Caron and Fox describe in Section 7. and in Appendix F.2 in [25].

Original: [N/A](b)
Revised: Even though our interest lies in the sociabilities and community memberships of observed nodes i = 1 , . . , N , to derive their exact posterior we need to also consider the unobserved nodes N +1 , . . . , N α , that is those nodes whose edges in the latent multigraph are all thinned, being all either discordantcommunity or self-edges.

Revised: The distribution of the number of nodes N α with at least one edge in the latent multigraph is not tractable, and we thus resort to an approximate method for updating N α .

Original: Did you describe any potential participant risks, with links to Institutional Review
Revised: Our update of N α is based on sampling Q multigraphs from the GGP prior evaluated at the current values of α, σ and τ . For simulated graph q , let N ( q ) be the number of nodes associated to at least one concordant-community edge, and N ( q ) α be the number of nodes associated to at least one edge (of any type). We then compute the average ratio r Q = 1 Q (cid:80) Qq =1 N ( q ) α N ( q ) and set the new value of N ( new ) α =

Original: Board (IRB) approvals, if applicable?
Revised: Nr Q . If the new value N ( new ) α is smaller than the older value N ( old ) α , we keep the N ( new ) α unobserved nodes that currently have the largest sociabilities, along with their current community memberships. If N ( new ) α > N ( old ) α , we keep all of the current N ( old ) α unobserved nodes, sample the sociabilities for the remaining N ( new ) α − N ( old ) α nodes by resampling from the sociabilities of the current nodes, and assign community memberships from the prior evaluated at the current value of β . Our experiments on simulated data suggest that, by updating N α every 100 iterations and setting

Revised: Q = 10 , this method allows MCMC chains to concentrate around the true value of N α and of the hyperparameters α, σ , and τ (see Figure 4 (top) in the manuscript).

Revised: B Additional results on simulated data

Revised: In Section 3 of the manuscript we observed that our TGGP specification is more regularized, and thus may be more easily identified, than the compound GGP formulation of [29].

Revised: Recall that, in our specification, the community memberships for node i are a probability vector π

Revised: i over K communities generated from π i |

Revised: [Equation]

Revised: iid ∼ Dirichlet (

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: In contrast, for the compound GGP the membership of node i in community k is modeled as

Original: [N/A](c)
Revised: [Equation] for

Revised: [Equation]

Revised: [Equation]

Revised: N α , k = 1 , . .

Revised: [Equation]

Revised: [Equation]

Revised: , where nodes’ baseline sociabilities ( w 10 , w 20 , . . . ) are drawn according to a generalized gamma process.

Revised: In Section 5.1 of the manuscript, we noted how our results suggested that the TGGP model may be better at recovering the underlying community structure of both TGGP and CGGP simulated graphs.

Revised: Here we provide additional insights into this comparison.

Revised: Figures 9a and 9b show that, for a CGGP-simulated graph with 15 communities, a k = 1 / 15 and b k

Revised: = 1 for all k = 1 , . .

Revised: , 15 , the samples ( a 1 , . . .

Revised: , a 15 ) from the posterior estimated fitting the CGGP model are not concentrated around the true value or 1 / 15 , and several among b 1 τ, . . .

Revised: , b K τ (which according to [29] is better identifiable than b 1 , . . .

Revised: , b K ) are considerably larger than their true value of 1 .

Revised: Conversely, Figure 9c shows that the TGGP model fitted with K = 50 correctly concentrates β k around 0 for most k = 1 , . .

Original: Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
Revised: , K and, for the remaining communities, it tends to draw samples around the true value of 1 / 15 (the simulated graph for the TGGP has 15 communities and a true value β k = 1 / 15 for each of them). > 0. < 0.

Revised: 0.3 0400 26a k den s i t y (a) a k densities (CGGP [29]) > <    2.

Revised: b k t den s i t y (b) b k τ densities (CGGP [29]) 0.

Revised: 1 5 10 15 20 25 30 35 40 45 50 b k v a l ue

Original: [N/A]
Revised: [Figure]

Revised: C Real-data networks: preparation and references

Revised: The four real-data networks used in our experiments were pre-processed by extracting the main component for the networks, by removing self-edges and by making the observed network undirected.

Revised: Summary information for each of these networks is displayed in Table 1.

Revised: [Table]

Revised: The networks Reed and Simmons were downloaded from https://archive.org/details/ oxford-2005-facebook-matrix , the network SmaGri was downloaded from https://www. cise.ufl.edu/research/sparse/matrices/Pajek/SmaGri.html , and the network Yeast was obtained from http://vlado.fmf.uni-lj.si/pub/networks/data/bio/Yeast/Yeast.htm .