SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lee M. Gunderson
Gatsby Unit
Gecia Bravo-Hermsdorff
Peter Orbanz
In this work, we describe a method that determines an exact map from a finite set of subgraph densities to the parameters of a stochastic block model (SBM) matching these densities. Given a number  $K$  of blocks, the subgraph densities of a finite number of stars and bistars uniquely determines a single element of the class of all degree-separated stochastic block models with  $K$  blocks. Our method makes it possible to translate estimates of these subgraph densities into model parameters, and hence to use subgraph densities directly for inference. The computational overhead is negligible; computing the translation map is polynomial in  $K$ , but independent of the graph size once the subgraph densities are given.
The class of infinitely exchangeable random graphs is a large class of network models that contains all graphon models [6, 21] and stochastic block models (SBMs) [1, 2, 12]. The key statistics of such random graphs are subgraph densities: each fixed, finite graph defines a subgraph density, and it is well known that every exchangeable graph distribution is completely determined by the collection of all its densities of finite subgraphs [12, 21].
In general, an infinite number of densities is required to uniquely determine an infinitely exchangeable graph distribution. For a subclass of models, a finite number of densities suffice. In combinatorics, these random graph distributions are called finitely forcible [21]. In particular, every stochastic block model is completely determined by a finite number of subgraph densities [21]. Such a finite number of subgraph densities can be consistently estimated at a guaranteed asymptotic rate from a single graph-data as this graph grows large [3, 17, 30].
It has long been recognized that the role subgraph densities play for exchangeable network models is analogous to that played by moments for estimation problems on Euclidean sample spaces [5, 7, 12, 22]. On Euclidean space, the method of moments [24, 26] is a powerful inference tool, in particular for mixture models [4, 14]. This suggests that subgraph densities should be well-suited for inference in stochastic block models in particular, since these models are a network analogue of mixture models and only a finite number of densities must be estimated.
Nonetheless, most inference algorithms used in practice (e.g., [8, 9, 13, 18, 25]) do no rely on subgraph densities, but rather on some combination of maximum likelihood estimation, MCMC sampling, variational inference, clustering of the nodes, or other heuristics. The reason is that estimating subgraph densities is only a first step in the inference process — once the densities are known, they must be translated into an estimate of a graphon or stochastic block model.
This paper introduces a new method that does so by generalizing a classical strategy for inferring latent sources (sometimes known as Prony's method [26]) to graph data. The classical Prony's
method is an example of an algorithm that recovers a sparse signal from noisy data, and is related to compressed sensing [28] and the notion of a matrix pencil [23].
Our method proceeds in two steps. The first step (section 3.2) is essentially an application of Prony's method to estimate properties of individual latent blocks, such as their normalized degrees and their relative sizes. The second step (section 4.2) is entirely new; by leveraging the properties inferred in the first step, it uses a generalization of Prony's method to infer properties of pairs of the latent blocks, such as the connection properties of a stochastic block model.
The proposed method solves several problems simultaneously: it estimates parameters without laborious and delicate numerical fitting, and makes it possible to efficiently sample from the model without the frequently-encountered problem of degeneracy that plagues many intuitively attractive network models [15, 16, 19]. In contrast to other algorithms for fitting stochastic block models (e.g., [8, 10, 11, 25]) and graphons (e.g., [2, 20, 29]), our method requires essentially zero computational overhead once the subgraph densities have been estimated.
2 Notation and Definitions
We use lower-case bold symbols to denote vectors, and upper-case bold symbols to denote matrices and higher-order tensors. Non-bold versions of these symbols refer to particular entries of the bold versions, indicated by their subscripts. We use  $\circ$  to denote Hadamard product (i.e., element-wise multiplication),  $\bullet$  for inner product, and angled brackets  $\langle \rangle$  to denote expectation.
A graph  $G$  is defined by a set of vertices/nodes  $V(G)$  and a set of edges  $E(G) \subseteq V(G) \times V(G)$  denoting pairwise connections between nodes. For simplicity, here we focus on undirected, unweighted graphs, with no self-loops or parallel edges. Indexing the  $N = |V(G)|$  nodes by the positive integers  $[N]$ , we can represent  $G$  by its  $N$ -by-  $N$  adjacency matrix  $\mathbf{A}$ , where  $A_{ij} = A_{ji} = 1$  if there is an edge between nodes  $i$  and  $j$ , and 0 otherwise.
A stochastic block model  $\mathrm{SBM}(\pi, \mathbf{B})$  is defined by:  $\pi$ , a probability vector of length  $K$ , which assigns each node to one of  $K$  (unobserved) latent blocks/communities; and  $\mathbf{B}$ , a  $K$ -by- $K$  connectivity matrix, whose entries  $B_{kk'}$  give the probability of an edge connecting a node in block  $k$  to a node in block  $k'$ . As we are considering undirected graphs,  $\mathbf{B}$  is symmetric.
Sampling a graph from a  $\mathrm{SBM}(\pi, \mathbf{B})$  proceeds in two steps. First, for each node  $n \in [N]$ , sample its latent block  $k(n) \in [K]$  independently from  $\pi$ . Then, for each pair of nodes  $(n, n')$ , include an edge between them independently with probability  $B_{k(n)k(n')}$ . Thus, a stochastic block model defines a distribution over graphs with  $N$  nodes for all choices of  $N$ , and can be identified with the limit of this sequence of distributions as  $N \to \infty$ .
As mentioned in the introduction, such infinitely exchangeable graph distributions can be characterized by their homomorphism densities  $\mu(g)$ . Indexed by (some family of) subgraphs  $g$ , homomorphism densities are the graphical analogue of the classical moments of a distribution [5, 12, 22]. For a stochastic block model, one can compute  $\mu(g)$  by summing over all possible assignments of the vertices of  $g$  to the blocks of  $\mathrm{SBM}(\pi, \mathbf{B})$  [21]:
$$
\underbrace {\mu (g) \Big | _ {\mathrm {S B M} (\pi , \mathbf {B})}} _ {\substack {\text {homomorphism density}\\\mu \text {of a subgraph } g \text { in an}\\\text {S B M given by } \pi \text { and } \mathbf {B}\\\text {with } K \text { blocks}}} = \sum_ {\substack {\varphi : V (g) \rightarrow [ K ]\\\text {sum over all maps } \varphi\\\text {from vertices in } g \text { to the } K \text { blocks}}} ^ {K ^ {| V (g) |}} \left[ \underbrace {\left(\prod_ {i \in V (g)} \pi_ {\varphi (i)}\right)} _ {\text {probability of assigning} \text {the vertices to those blocks}} \times \underbrace {\left(\prod_ {(i , j) \in E (g)} B _ {\varphi (i) \varphi (j)}\right)} _ {\text {probability of the corresponding edges}} \right] \tag{1}
$$
In the following two sections (sections 3 and 4), we describe how to recover the parameters of an SBM from its subgraph densities. Later, in section 5, we describe how to infer these subgraph densities using a graph sampled from such an SBM.
<sup>1</sup>Note that the notion of homomorphism only enforces that the edges in  $g$  be present; it does not enforce the absence of edges between pairs of nodes in  $g$  that do not share an edge. For example, when  $g$  has no edges, the homomorphism density is always 1. This is in contrast to the notion of induced subgraph density, which does enforce the absence of those edges.
3 Mapping Star Densities to Block Degrees
As the first step of our method, we obtain the normalized degrees of the blocks  $\mathbf{d}$  (equation (4)), as well as their relative sizes  $\pi$ . This can be seen as an application of the classical matrix pencil method to the degree distribution, for which the density of the star subgraphs serve as sufficient statistics.
3.1 Classical Coin Collecting
Before we explain our graph pencil method, let us consider the simpler case of a mixture model for infinitely exchangeable sequences of binary variables. Also known as the Bernoulli mixture model [14, 27], such a distribution can be thought of as the outcomes of (some number of) flips of (some number of) biased coins, where each coin is sampled i.i.d. from a (possibly unequal) mixture of  $K$  different biases.
To recover the parameters of this model, one must infer the  $K$  (unobserved) latent biases  $b_{k}$ , as well as the fraction  $\pi_{k}$  of coins with each bias. Note that one does not know which of the coins have the same latent bias (otherwise the inference problem would be trivial).
The moments of this distribution are the expectation of powers of these latent biases, which are indexed by the exponents  $r \in \mathbb{N}$ :
$$
\langle b ^ {r} \rangle = \sum_ {k} \pi_ {k} b _ {k} ^ {r} \tag {2}
$$
Note that these moments can be consistently estimated from the observed data, as we are averaging over the latent variables. From these moments, one can systematically infer the mixture proportion  $\pi$  and the biases  $\mathbf{b}$  using the matrix pencil method.
In short, construct two matrices  $\mathbf{C}$  and  $\mathbf{C}'$ , with entries  $C_{ij} = \langle b^{i + j}\rangle$  and  $C_{ij}' = \langle b^{i + j + 1}\rangle$ :
$$
\mathbf {C} = \left[ \begin{array}{c c c c} \langle b ^ {0} \rangle & \langle b ^ {1} \rangle & \dots & \langle b ^ {K - 1} \rangle \\ \langle b ^ {1} \rangle & \langle b ^ {2} \rangle & \dots & \langle b ^ {K} \rangle \\ \vdots & \vdots & \ddots & \vdots \\ \langle b ^ {K - 1} \rangle & \langle b ^ {K} \rangle & \dots & \langle b ^ {2 K - 2} \rangle \end{array} \right] \quad \mathbf {C} ^ {\prime} = \left[ \begin{array}{c c c c} \langle b ^ {1} \rangle & \langle b ^ {2} \rangle & \dots & \langle b ^ {K} \rangle \\ \langle b ^ {2} \rangle & \langle b ^ {3} \rangle & \dots & \langle b ^ {K + 1} \rangle \\ \vdots & \vdots & \ddots & \vdots \\ \langle b ^ {K} \rangle & \langle b ^ {K + 1} \rangle & \dots & \langle b ^ {2 K - 1} \rangle \end{array} \right] (3)
$$
While not immediately obvious, it is easy to show that the eigenvalues of  $\mathbf{C}'\mathbf{C}^{-1}$  are the entries of  $\mathbf{b}$ . From these, it is straightforward to obtain the associated entries of  $\pi$ . In the next section, we show why this is the case, while superficially replacing the biases  $\mathbf{b}$  of the latent coin types with the normalized average degrees  $\mathbf{d}$  of the latent node blocks.
3.2 Distilling the Degree Distribution
The first step of our graph pencil method is to apply the standard matrix pencil method to the obtain the average normalized degree of each of the  $K$  latent blocks (that is, the likelihood that a random node shares an edge with a node in block  $k$ ). Denoted by  $\mathbf{d}$ , the entries of this vector are:
$$
d _ {k} = \sum_ {j} \pi_ {j} B _ {j k} \tag {4}
$$
These  $d_{k}$  are latent parameters of the blocks, so we can solve for them in exactly the same way as we did for  $\hat{b}_{k}$  in the previous coin example. To this end, we consider the analogous moments:
$$
\langle d ^ {r} \rangle = \sum_ {k} \pi_ {k} d _ {k} ^ {r} \tag {5}
$$
As before, while  $\pi_{k}$  and  $d_{k}$  are unobserved, the resulting moments can be consistently estimated from observations. In particular, they are precisely the homomorphism densities of the star subgraphs:
$$
\langle d ^ {0} \rangle = \mu (\bullet) = 1 \quad \langle d ^ {1} \rangle = \mu (\updownarrow) \quad \langle d ^ {2} \rangle = \mu (\updownarrow) \quad \langle d ^ {3} \rangle = \mu (\updownarrow) \quad \dots . \tag {6}
$$
We then define the two matrices  $\mathbf{C}^{\circ}$  and  $\mathbf{C}^{\mathrm{d}}$ , with entries  $C_{ij}^{\circ} = \langle d^{i + j}\rangle$  and  $C_{ij}^{\mathrm{d}} = \langle d^{i + j + 1}\rangle$ , and the eigenvalues of  $\mathbf{C}^{\mathrm{d}}(\mathbf{C}^{\circ})^{-1}$  are the normalized degrees of the  $K$  blocks. To see why, notice that  $\mathbf{C}^{\circ}$  can be written as a sum of  $K$  rank-1 matrices (weighted by  $\pi$ ):
$$
\begin{array}{l} \mathbf {C} ^ {\circ} = \sum_ {k} \pi_ {k} \left[ \begin{array}{c c c c} d _ {k} ^ {0} & d _ {k} ^ {1} & \dots & d _ {k} ^ {K - 1} \\ d _ {k} ^ {1} & d _ {k} ^ {2} & \dots & d _ {k} ^ {K} \\ \vdots & \vdots & \ddots & \vdots \\ d _ {k} ^ {K - 1} & d _ {k} ^ {K} & \dots & d _ {k} ^ {2 K - 2} \end{array} \right] = \sum_ {k} \pi_ {k} \left[ \begin{array}{c} d _ {k} ^ {0} \\ d _ {k} ^ {1} \\ \vdots \\ d _ {k} ^ {K - 1} \end{array} \right] \left[ \begin{array}{c c c c} d _ {k} ^ {0} & d _ {k} ^ {1} & \dots & d _ {k} ^ {K - 1} \end{array} \right] \\ = \left[ \begin{array}{c c c} d _ {1} ^ {0} & \dots & d _ {K} ^ {0} \\ \vdots & \ddots & \vdots \\ d _ {1} ^ {K - 1} & \dots & d _ {K} ^ {K - 1} \end{array} \right] \left[ \begin{array}{c c c} \pi_ {1} & \dots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \dots & \pi_ {K} \end{array} \right] \left[ \begin{array}{c c c} d _ {1} ^ {0} & \dots & d _ {K} ^ {K - 1} \\ \vdots & \ddots & \vdots \\ d _ {K} ^ {0} & \dots & d _ {K} ^ {K - 1} \end{array} \right] \\ = \mathbf {V} \operatorname {d i a g} (\boldsymbol {\pi}) \mathbf {V} ^ {\top} \tag {7} \\ \end{array}
$$
where we have defined  $\mathbf{V}$  as the matrix with entries  $V_{jk} = d_k^{j - 1}$ , i.e.:
$$
\mathbf {V} = \left[ \begin{array}{c c c c} d _ {1} ^ {0} & d _ {2} ^ {0} & \dots & d _ {K} ^ {0} \\ d _ {1} ^ {1} & d _ {2} ^ {1} & \dots & d _ {K} ^ {1} \\ \vdots & \vdots & \ddots & \vdots \\ d _ {1} ^ {K - 1} & d _ {2} ^ {K - 1} & \dots & d _ {K} ^ {K - 1} \end{array} \right] \tag {8}
$$
By decomposing  $\mathbf{C}^{\mathrm{d}}$  in the same manner, we get:
$$
\mathbf {C} ^ {\mathbf {d}} = \mathbf {V} \operatorname {d i a g} (\pi \mathbf {d}) \mathbf {V} ^ {\top} \tag {9}
$$
where  $\mathrm{diag}(\pi \mathbf{d})$  is the diagonal matrix with entries  $\pi_k d_k$ .
We now see that  $\mathbf{C}^{\mathrm{d}}(\mathbf{C}^{\circ})^{-1}$  can be diagonalized as follows:
$$
\begin{array}{l} \mathbf {C} ^ {\mathbf {d}} \left(\mathbf {C} ^ {\circ}\right) ^ {- 1} = \left(\mathbf {V} \operatorname {d i a g} (\pi \mathbf {d}) \mathbf {V} ^ {\top}\right) \left(\mathbf {V} \operatorname {d i a g} (\pi) \mathbf {V} ^ {\top}\right) ^ {- 1} \\ = \mathbf {V} \operatorname {d i a g} (\boldsymbol {\pi} \mathbf {d}) \mathbf {V} ^ {\top} \left(\mathbf {V} ^ {\top}\right) ^ {- 1} \operatorname {d i a g} (\boldsymbol {\pi} ^ {- 1}) \mathbf {V} ^ {- 1} \\ = \mathbf {V} \operatorname {d i a g} (\boldsymbol {\pi} \mathbf {d}) \operatorname {d i a g} (\boldsymbol {\pi} ^ {- 1}) \mathbf {V} ^ {- 1} \\ = \mathbf {V} \operatorname {d i a g} (\mathbf {d}) \mathbf {V} ^ {- 1} \tag {10} \\ \end{array}
$$
Hence, the eigenvalues of  $\mathbf{C}^{\mathbf{d}}(\mathbf{C}^{\circ})^{-1}$  are indeed the normalized degrees  $\mathbf{d}$  of the  $K$  latent blocks:
$$
\operatorname {e i g v a l} \left(\mathbf {C} ^ {\mathbf {d}} \left(\mathbf {C} ^ {\circ}\right) ^ {- 1}\right) = \left\{d _ {k} \right\} _ {k \in [ K ]} \tag {11}
$$
Remark 1 (Degree separated assumption). This is where the assumption of degree-separated blocks is used; if the normalized degrees of the blocks are not unique, then  $\mathbf{V}$  will not be invertible.
From the entries of  $\mathbf{d}$ , we know the entries of  $\mathbf{V}$ , and can solve a linear system of equations for the corresponding entries of  $\pi$ :
$$
\sum_ {k} V _ {j k} \pi_ {k} = \left\langle d ^ {j - 1} \right\rangle \tag {12}
$$
4 From Bistar Densities to a Stochastic Block Model
This second step of our graph pencil method is the main insight of our paper. It uses the results from the previous step to reconstruct the connection probabilities  $B_{k,k'}$ .
4.1 Gluing Graphs: An Algebra
We now describe a natural algebra over rooted subgraphs (see Chapter 6 of [21]). We find that using this notation makes our method easier to understand, and facilitates reasoning about its generalizations. For a longer introduction, see appendix A.
Recall the definition of homomorphism subgraph densities of an SBM in equation (1). For a given distribution  $\mathrm{SBM}(\pi, \mathbf{B})$ , each subgraph  $g$  corresponds to a scalar density  $\mu(g)$ . Rooted homomorphism subgraph densities can be seen as a generalization of this notion. In addition to the subgraph  $g$ , (singly-)rooted densities also require a designated "rooted" vertex  $v \in V(g)$ . And instead of a single scalar, they correspond to (a vector of)  $K$  scalars  $\mu(g, v)$ , indexed by the block  $k \in [K]$  to which the rooted vertex  $v$  is mapped:
$$
\underbrace {\mu_ {k} (g , v) \Big | _ {\mathrm {S B M} (\boldsymbol {\pi} , \mathbf {B})}} _ {\substack {\text {homomorphism density}\\\mu \text {of a subgraph } g \text { with}\\\text {rooted vertex } v \text { in block } k}} = \underbrace {\sum_ {\substack {\varphi : V (g) \rightarrow [ K ]\\\text {s.t. } \varphi (v) = k}} ^ {K ^ {| V (g) | - 1}} \left[ \underbrace {\left(\prod_ {i \in V (g) \backslash v} \pi_ {\varphi (i)}\right)} _ {\text {probability of assigning the remaining vertices to those blocks}} \times \underbrace {\left(\prod_ {(i , j) \in E (g)} B _ {\varphi (i) \varphi (j)}\right)} _ {\text {probability of the corresponding edges}} \right]} _ {(1 3)}
$$
The sum is now over maps  $\varphi$  that send the rooted vertex  $v$  to a particular block  $k$ , and the probability of assigning the vertices  $V(g)$  to blocks includes only the remaining "unrooted" vertices. All the edges  $E(g)$  contribute to the product exactly as in equation (1).
A rooted vertex with a single edge corresponds to the normalized degrees of the  $K$  blocks:
$$
\text {(r o o t e d v e r t e x w i t h a n e d g e)} \quad \int \quad \longleftrightarrow \quad \mathbf {d} \quad (\text {n o r m a l i z e d d e g r e e s o f t h e b l o c k s}) \tag {14}
$$
Rooted subgraphs may be combined via the gluing product, denoted by  $\circ$ . The gluing product corresponds to taking their disjoint union, then merging (i.e., "gluing") the rooted vertices:
$$
\text {(g l u i n g r o o t e d v e r t i c e s)} \quad \mathbf {\Uparrow} \circ \mathbf {\Uparrow} = \mathbf {\Uparrow} \quad \longleftrightarrow \quad \mathbf {d} \circ \mathbf {d} = \mathbf {d} ^ {2} \quad \text {(e n t r y w i s e m u l t i p l i c a t i o n)}
$$
$$
\mathbf {\Omega} \circ \mathbf {\Psi} = \mathbf {\Psi} \quad \longleftrightarrow \quad \mathbf {d} \circ \mathbf {d} ^ {2} = \mathbf {d} ^ {3}
$$
$$
\mathbb {Y} \circ \mathbb {Y} = \mathbb {Y} \quad \longleftrightarrow \quad \mathrm {d} ^ {2} \circ \mathrm {d} ^ {3} = \mathrm {d} ^ {5} \tag {15}
$$
Algebraically, the gluing product corresponds to entrywise multiplication of the vectors, also known as the Hadamard product.
Finally, to convert a rooted subgraph into an observable subgraph density, the rooted vertex may be "unrooted" by taking the dot product with  $\pi$ :
$$
(\text {u n r o o t i n g v e r t i c e s}) \quad \bullet \mathbb {V} \quad \longleftrightarrow \quad \mathrm {d} ^ {3} \bullet \pi = \langle d ^ {4} \rangle \quad (\text {d o t p r o d u c t w i t h} \pi) \tag {16}
$$
Table 1: Operations using homomorphism densities of singly-rooted subgraphs.
glyph	symbol	meaning
♀	d = B • π	vector of the K normalized degrees
♂	d² = d ∘ d	entrywise multiplication
♀♀	d³ • π	homomorphism density of subgraph
Before describing how to recover the entries of  $\mathbf{B}$ , let us summarize the method for recovering the normalized degrees from the previous section using the language of rooted graphs.
For an SBM with  $K$  blocks, define a vector  $\mathbf{v}$  of (at least)  $K$  rooted subgraphs, and take the outer product of this vector with itself where entries are combined via the gluing product. For example:
$$
\mathbf {v} = \left[ \begin{array}{c} \circ \\ \bullet \\ \text {个} \end{array} \right] \quad \mathbf {v} \circ \mathbf {v} ^ {\top} = \left[ \begin{array}{c} \circ \\ \bullet \\ \text {个} \end{array} \right] \circ \left[ \begin{array}{c} \circ \\ \bullet \\ \text {个} \end{array} \right] ^ {\top} = \left[ \begin{array}{c c c} \circ & \bullet & \text {个} \\ \bullet & \text {个} & \text {个} \\ \text {个} & \text {个} & \text {个} \end{array} \right] \tag {17}
$$
While the rooted subgraph densities are not directly observable, their unrooted counterparts are, and the entries of  $\mathbf{C}^{\bullet}$  (equation (3), left) are given by the densities of these unrooted versions:
$$
\mathbf {C} ^ {\circ} = \left(\mathbf {v} \circ \mathbf {v} ^ {\top}\right) \bullet \pi = \left[ \begin{array}{c c c} \bullet & \overset {\bullet} {\mathbb {V}} & \overset {\bullet} {\mathbb {V}} \\ \overset {\bullet} {\mathbb {V}} & \overset {\bullet} {\mathbb {V}} & \overset {\bullet} {\mathbb {V}} \end{array} \right] \tag {18}
$$
(The glyph containing an unrooted vertex and no edges evaluates to 1.)
To obtain the second matrix  $\mathbf{C}^{\mathrm{d}}$  (equation (3), right), glue the rooted subgraph  $\updownarrow$  to all entries before unrooting:
$$
\mathbf {C} ^ {\mathrm {d}} = \left(\left(\mathbf {v} \circ \mathbf {v} ^ {\top}\right) \circ \mathbf {0}\right) \bullet \pi = \left[ \begin{array}{c c c} \updownarrow & \Downarrow & \Downarrow \\ \Downarrow & \Downarrow & \Downarrow \\ \Downarrow & \Downarrow & \Downarrow \end{array} \right] \tag {19}
$$
Indeed, the reason that the spectrum of  $\mathbf{C}^{\circ}(\mathbf{C}^{\mathbf{d}})^{-1}$  gives the normalized degrees  $\mathbf{d}$  of the latent blocks is precisely because we glued an extra copy of the corresponding rooted subgraph  $\mathbf{\Phi}$  in the construction of  $\mathbf{C}^{\mathrm{d}}$ . The next section follows a similar recipe.
4.2 Extracting the Edge Expectations
The previous matrices  $\mathbf{C}^{\circ}$  and  $\mathbf{C}^{\mathrm{d}}$  were obtained by taking a dot product that sums over the  $K$  blocks. This allowed us to obtain properties of a single block (e.g., its degree  $d_{k}$ ). The first main insight of this paper is that the matrix pencil method can also be used to obtain properties of pairs of blocks (i.e., their connection probability  $B_{kk'}$ ) by summing over such pairs.
The extension of the gluing algebra to birooted subgraphs is relatively straightforward; a birooted subgraph  $(g; u, v)$  has two (distinct) rooted vertices, and their gluing product is given by taking the disjoint union then separately merging the vertices rooted  $u$  and the vertices rooted  $v$ .
Table 2: Operations using homomorphism densities of doubly-rooted subgraphs.
glyph	symbol	meaning
○○○	d1T = L	row matrix of (left) normalized degrees
1dT = R	column matrix of (right) normalized degrees
B	matrix of connection probabilities
○○○	L2○ B○R	entrywise multiplication
○○○	π • (L2○ B○R) • π = μ(○○○)	observable subgraph density
For an SBM with  $K$  blocks, there are  $K + \binom{K}{2}$  degrees of freedom for the entries of  $\mathbf{B}$ . So we define a vector  $\mathbf{v}$  of (at least)  $K + \binom{K}{2}$  birooted subgraphs. To this end, we use the symmetric polynomials, with exponent at most  $K$ , in the two variables  $\updownarrow$  and  $\updownarrow$ , corresponding to the normalized degrees of the blocks corresponding to the two roots.
$$
\mathbf {v} = \left[ \begin{array}{c c c} \circ & \circ \\ \bigstar & + & \bigstar \\ \bigstar & \bigstar \end{array} \right] \quad \mathbf {v} \circ \mathbf {v} ^ {\top} = \left[ \begin{array}{c c c} \circ & \circ & \bigstar \\ \bigstar & + & \bigstar \\ \bigstar & \bigstar \end{array} \right] \tag {20}
$$
In this example for  $K = 2$ , the second row of  $\mathbf{v}$  has been symmetrized with respect to the two rooted vertices, resulting in a formal linear combination of elements in the gluing algebra.
As before,  $\mathbf{C}^{\infty}$  is obtained by unrooting the entries, while for the construction of  $\mathbf{C}^{\mathbf{B}}$ , we glue the rooted subgraph  $\circ \circ$  corresponding to entries of  $\mathbf{B}$ .