SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[38] R. A. Rossi and N. K. Ahmed. Role discovery in networks. IEEE Transactions on Knowledge and Data Engineering, 27(4):1112-1131, 2014.
[39] R. A. Rossi, R. Zhou, and N. K. Ahmed. Estimation of graphlet counts in massive networks. IEEE Transactions on Neural Networks and Learning Systems, 30(1):44-57, 2018.
[40] R. A. Rossi, D. Jin, S. Kim, N. K. Ahmed, D. Koutra, and J. B. Lee. On proximity and structural role-based embeddings in networks: Misconceptions, techniques, and applications. ACM Transactions on Knowledge Discovery from Data (TKDD), 14(5):1-37, 2020.
[41] R. J. Sánchez-García. Exploiting symmetry in network analysis. Communications Physics, 3 (1):1-15, 2020.
[42] M. T. Schaub, N. O'Clery, Y. N. Billeh, J.-C. Delvenne, R. Lambiotte, and M. Barahona. Graph partitions and cluster synchronization in networks of oscillators. *Chaos: An Interdisciplinary Journal of Nonlinear Science*, 26(9):094821, 2016.
[43] I. Schomburg, A. Chang, C. Ebeling, M. Gremse, C. Heldt, G. Huhn, and D. Schomburg. Brenda, the enzyme database: updates and major new developments. *Nucleic acids research*, 32(suppl_1):D431–D433, 2004.
[44] F. I. Stamm, M. Scholkemper, M. Strohmaier, and M. T. Schaub. Neighborhood structure configuration models. Proceedings of the ACM Web Conference 2023, 2022. URL https://api-semanticscholar.org/CorpusID:252873344.
[45] S. H. Strogatz. Exploring complex networks. Nature, 410(6825):268-276, 2001.
[46] N. Wale, I. A. Watson, and G. Karypis. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems, 14(3):347-375, 2008.
[47] B. Weisfeiler and A. Leman. The reduction of a graph to canonical form and the algebra which appears therein. NTI, Series, 2(9):12-16, 1968.
[48] D. R. White and K. P. Reitz. Graph and semigroup homomorphisms on networks of relations. Social Networks, 5(2):193-234, 1983.
[49] X. Wu. Optimal quantization by matrix searching. Journal of algorithms, 12(4):663-673, 1991.
[50] K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
[51] Y. Yuan, G.-B. Stan, L. Shi, M. Barahona, and J. Goncalves. Decentralised minimum-time consensus. Automatica, 49(5):1227-1235, 2013.
A Example of Communities vs. Roles
Figure A1: Toy example showing two networks and their role/community structure. The left has two exact roles but how many communities it has is not clear. The right has 3 perfect communities, but only a single role.
B Example of Isomorphic Nodes that receive a different role
Figure A2: Example graphs showing undesirable properties of the optimizer of eq. (6). Nodes 1 and 2 are in the same cEP class; they are even isomorphic. However, the minimizer of eq. (6) puts them into different classes. The partition minimizing eq. (6) as given by the right yields a score of 0.707, whereas the best partition that respects the cEP yields 0.816.
C Proof of theorem 4.1
Theorem 4.1: Let  $\mathcal{H}$  be the set of indicator matrices  $H\in \{0,1\}^{n\times k}$  s.t.  $H\mathbb{1}_k = \mathbb{1}_n$ . Let  $A\in \mathbb{R}^{n\times n}$  be an adjacency matrix. Assume the dominant eigenvector to the eigenvalue  $\rho (A)$  of  $A$  is unique. Using the  $\ell_1$  norm in eq. (6), the optimizer
$$
O P T = \arg \min _ {H \in \mathcal {H}} \lim _ {d \to \infty} \Gamma_ {d - E P} (A, H)
$$
Proof. Consider the long-term cost function (eq. 6,7):
$$
\begin{array}{l} c _ {\mathrm {d - E P}} (A, H) = \sum_ {t} ^ {d} \frac {1}{\rho (A) ^ {t}} | | A ^ {t} H - H D ^ {- 1} H ^ {\top} A ^ {t} H | | \\ = \sum_ {t} ^ {d} | | (\mathbb {I} _ {n} - H D ^ {- 1} H ^ {\top}) \frac {1}{\rho (A) ^ {t}} A ^ {t} H | | \\ \stackrel {\lim  d \rightarrow \infty} {=} | | (\mathbb {I} _ {n} - H D ^ {- 1} H ^ {\top}) w v ^ {\top} H | | \\ \end{array}
$$
We arrive at a formulation akin to the  $k$ -means cost function. However,  $w v^{\top} H$  is in general not independent of the clustering, as would be the case in the usual formulation of  $k$ -means. This can be used advantageously by rewriting the above matrix equation element-wise:
$$
\begin{array}{l} = \sum_ {i} ^ {n} \sum_ {j} ^ {k} \left| w _ {i} \left(v ^ {\top} H _ {-, j}\right) - \frac {1}{\left| C \left(w _ {i}\right) \right|} \sum_ {l \in C \left(w _ {i}\right)} w _ {l} \left(v ^ {\top} H _ {-, j}\right) \right| \\ = \sum_ {i} ^ {n} \sum_ {j} ^ {k} | (w _ {i} - \frac {1}{| C (w _ {i}) |} \sum_ {l \in C (w _ {i})} w _ {l}) (v ^ {\top} H _ {-, j}) | \\ \end{array}
$$
It is possible to completely draw out the constant factor of  $\sum_{j} v^{\top} H_{-j} = \sum_{i} v_{i}$  since the row sums of  $H$  are 1 and the components  $v_{i} \geq 0$  are non-negative.
$$
\begin{array}{l} = \sum_ {i} ^ {n} \sum_ {j} ^ {k} (v ^ {\top} H _ {-, j}) | (w _ {i} - \frac {1}{| C (w _ {i}) |} \sum_ {l \in C (w _ {i})} w _ {l}) | \\ = \operatorname {c o n s t} \sum_ {i} ^ {n} \left| \left(w _ {i} - \frac {1}{\left| C \left(w _ {i}\right) \right|} \sum_ {l \in C \left(w _ {i}\right)} w _ {l}\right) \right| \\ \end{array}
$$
We end up at a formulation equivalent to clustering  $w$  into  $k$  clusters using  $k$ -means. We can now notice that  $w$  is only 1-dimensional and as such the  $k$ -means objective can be optimized in  $\mathcal{O}(n\log (n) + nk)$  [49, 15].
D Proof of theorem 4.2
Proof. We reduce from the PLANAR-K-MEANS problem, which is shown to be NP-hard in [27]. In PLANAR-K-MEANS, we are given a set  $\{(x_1, y_1), \ldots, (x_n, y_n)\}$  of  $n$  points in the plane and a number  $k$  and a cost  $c$ . The problem is to find a partition of the points into  $k$  clusters such that the cost of the partition is at most  $c$ , where the cost of a partition is the sum of the squared distances of each point to the center of its cluster. We now formulate the decision variant of optimizing the short-term cost which we show is NP-hard.
Definition D.1 (K-AEP). Let  $G = (V, E)$  be a graph,  $k \in \mathbb{N}$  and  $c \in \mathbb{R}$ . K-AEP is then the problem of deciding whether there exists a partition of the nodes in  $V$  into  $k$  clusters such that the short-term cost  $\Gamma_{1 - \mathrm{EP}}$  (eq. (7)) using the squared L2 norm is at most  $c$ .
Let  $W(X,Y)$  be the sum of the weights of all edges between  $X, Y \subseteq V$ . Additionally, for a given partition indicator matrix  $H$ , let  $C_i$  be the set of nodes  $v$  s.t.  $H_{v,i} = 1$ . For the following proof, the equivalent definition of the short-term cost function (eq. 6) using the squared L2 norm is more convenient:
$$
\Gamma_ {\mathrm {E P}} (A, H) = \sum_ {i} \sum_ {j} \sum_ {v \in C _ {i}} \left(W (\{v \}, C _ {j}) - \frac {1}{| C _ {i} |} W (C _ {i}, C _ {j})\right) ^ {2}
$$
We now show that K-AEP is NP-hard by reduction from PLANAR-K-MEANS.
Construction. Given  $\{(x_1, y_1), \ldots, (x_n, y_n)\}$  of  $n$  points in the plane and a number  $k'$  and a cost  $c'$  construct the following graph: We shift the given points by  $-\min_{i \in [n]} x_i$  in their  $x$ -coordinate and by  $-\min_{i \in [n]} y_i$  in their  $y$ -coordinate. This makes them non-negative, but does not change the problem. Let  $D = 1 + \sum_{i=1}^{n} x_i^2 + y_i^2$ . Notice that  $D$  is an upper bound on the cost of the  $k$ -means partition. To start, let  $V = \{a, b\}$ . Add self-loops of weight  $3D$  to  $a$  and of weight  $6D$  to  $b$ . For each point  $(x_i, y_i)$ , add a node  $m_i$  to  $V$  and add edges  $m_i a$  of weight  $x_i$  and  $m_i b$  of weight  $y_i$  to  $E$ .
$G = (V,E),k = k^{\prime} + 2,c = c^{\prime}$  are now the inputs to K-AEP.
Correctness. We now prove that the PLANAR-K-MEANS instance has a solution if and only if K-AEP has a solution. Assume that PLANAR-K-MEANS has a solution  $S' = (S_1', \dots, S_{k'}')$  that has cost  $c^* \leq c'$ . Then the solution we construct for K-AEP is  $S = (S_1, \dots, S_{k'}, \{a\}, \{b\})$ , where  $m_i \in S_j \iff (x_i, y_i) \in S_j'$ . The cost of this solution is:
$$
e ^ {+} = \sum_ {i = 1} ^ {k} \sum_ {j = 1} ^ {k} \sum_ {v \in S _ {i}} (W (\{v \}, S _ {j}) - \frac {1}{| S _ {i} |} W (S _ {i}, S _ {j})) ^ {2}
$$
Since  $a$  and  $b$  are in singleton clusters, their outgoing edges do not differ from the cluster average and so incur no cost. The remaining edges either go from  $V \setminus \{a, b\}$  to  $a$  or from  $V \setminus \{a, b\}$  to  $b$ . So, the sum reduces to:
$$
c ^ {+} = \sum_ {i} \sum_ {v \in S _ {i}} \left(\left(w (v, a) - \frac {1}{| S _ {i} |} W (S _ {i}, \{a \})\right) ^ {2} + \left(w (v, b) - \frac {1}{| S _ {i} |} W (S _ {i}, \{b \})\right) ^ {2}\right)
$$
Since  $\mu_x(S_i) := \frac{1}{|S_i|} W(S_i, \{a\})$  is the average weight of the edges from  $S_i$  to  $a$ , and these edges have weight according to the  $x$  coordinate of the point they were constructed from,  $\mu_x(S_i)$  is equal to the mean  $x$  coordinate within the cluster  $S_i'$ . This concludes the proof of this direction, as:
$$
c ^ {+} = \sum_ {S _ {i} ^ {\prime} \in S ^ {\prime}} \sum_ {(x _ {l}, y _ {l}) \in S _ {i} ^ {\prime}} (x _ {l} - \mu_ {x} (S _ {i} ^ {\prime})) ^ {2} + (y _ {l} - \mu_ {y} (S _ {i} ^ {\prime})) ^ {2} = c ^ {*} \leq c ^ {\prime}
$$
For the other direction, assume we are given a solution  $S = (S_{1},\dots,S_{k + 2})$  to K-AEP with cost  $c^{+}\leq c$ . We distinguish two cases:
Case 1:  $\exists i\in \mathbb{N}$  s.t.  $S_{i}\supseteq \{a\}$  or  $S_{i}\supseteq \{b\}$ . Assume that  $S_{i}\supseteq \{a\}$ , if also  $b\in S_i$  then the cost is at least the difference of the two self-loops:
$$
\begin{array}{l} c ^ {+} \geq \sum_ {v \in S _ {i}} \left(W (\{v \}, S _ {i}) - \frac {1}{| S _ {i} |} W (S _ {i}, S _ {i})\right) ^ {2} \\ \geq \left(\frac {1}{2} \max  _ {u, v \in S _ {i}} W (\{v \}, S _ {i}) - W (\{u \}, S _ {i})\right) ^ {2} \\ \geq \left(\frac {1}{2} (w (b, b) - W (\{a \}, S _ {i}))\right) ^ {2} \\ \geq \left(\frac {1}{2} (6 D - 4 D)\right) ^ {2} = D ^ {2} \geq D \\ \end{array}
$$
If instead, some  $m_j \in S_i$ , then the cost is at least the difference of the self-loop to  $a$  and the edge from  $m_j$  to  $a$ :
$$
\begin{array}{l} c ^ {+} \geq \sum_ {v \in S _ {i}} \left(W (\{v \}, S _ {i}) - \frac {1}{| S _ {i} |} W (S _ {i}, S _ {i})\right) ^ {2} \\ \geq \left(\frac {1}{2} (w (a, a) - W (\{m _ {j} \}, S _ {i}))\right) ^ {2} \\ \geq \left(\frac {1}{2} (3 D - D)\right) ^ {2} = D ^ {2} \geq D \\ \end{array}
$$
Thus  $c^+ \geq D$  is so large that any clustering of the points has at most cost  $c \geq D$  thus a solution to the PLANAR-K-MEANS instance exists. The case where  $S_i \supseteq \{b\}$  is analogous.
Case 2: Case 1 doesn't hold. In this case, we have  $S = (S_{1}, \dots, S_{k}, \{a\}, \{b\})$  which yields a clustering  $S' = (S_1', \dots, S_k')$  for the PLANAR-K-MEANS, where  $m_i \in S_j \iff (x_i, y_i) \in S_j'$ . This instance has cost  $c^+ = c^* \leq c$ .
E Proof of theorem 4.3
Proof. Consider the adjacency matrix  $A_B$  of a simple binomial random graph of size  $n$  - i.e. a single block of the SBM. Let  $\delta^* < \frac{\delta}{3}$ . Using the Chernoff bound for binomial random variables, we have that the degree of a single node  $i$  is within the ball of size  $\delta^*$  with probability:
$$
\operatorname * {P r} \left(| (A _ {B} \mathbb {1}) _ {i} - (\mathbb {E} [ (A _ {B} \mathbb {1}) _ {i} ] | \geq \delta^ {*} \cdot n\right) \leq 2 e ^ {- 2 n (\delta^ {*}) ^ {2}}
$$
The probability that all nodes fall in close proximity to the expectation, is then simply:
$$
\operatorname * {P r} \left(\| A _ {B} \mathbb {1} - \mathbb {E} [ A _ {B} \mathbb {1} ] \| _ {\infty} \geq \delta^ {*} \cdot n\right) \leq \left(1 - 2 e ^ {- 2 n (\delta^ {*}) ^ {2}}\right) ^ {n}
$$
Finally, in the SBM setting, we have  $k^2$  such blocks and the probability that none of the nodes are far away from the expectation in any of these blocks is:
$$
\Pr \left(\frac {\left\| A H H _ {\text {r o l e}} ^ {(T)} - \mathbb {E} [ A H H _ {\text {r o l e}} ^ {(T)} ] \right\|}{n} \geq \delta^ {*}\right) \leq \left(1 - 2 e ^ {- 2 n (\delta^ {*}) ^ {2}}\right) ^ {n k ^ {2}}
$$
We can upper bound this by its first-order Taylor approximation:
$$
\begin{array}{l} \left(1 - 2 e ^ {- 2 n \left(\delta^ {*}\right) ^ {2}}\right) ^ {n k ^ {2}} \leq p \leq 1 - 2 n k ^ {2} e ^ {- 2 n \left(\delta^ {*}\right) ^ {2}} \\ \Leftrightarrow \quad \frac {(p - 1) (\delta^ {*}) ^ {2}}{k ^ {2}} \leq - 2 n (\delta^ {*}) ^ {2} e ^ {- 2 n (\delta^ {*}) ^ {2}} \\ \Leftrightarrow \qquad \mathcal {W} _ {- 1} \left(\frac {(p - 1) (\delta^ {*}) ^ {2}}{k ^ {2}}\right) \geq - 2 n (\delta^ {*}) ^ {2} \\ \Leftrightarrow - \frac {9 \mathcal {W} _ {- 1} ((p - 1) \delta^ {2} / 9 k ^ {2})}{2 \delta^ {2}} \leq n \\ \end{array}
$$
Thus with probability at least  $p$ , the maximum deviation from the expected mean is  $\delta^{*}$ , which is why we simply assume this to be the case going forward, i.e.:
$$
\frac {1}{n} \max _ {i, j} \left(\left| \left(A H H _ {\mathrm {r o l e}} ^ {(T)} - \mathbb {E} [ A H H _ {\mathrm {r o l e}} ^ {(T)} ]\right) _ {i, j} \right|\right) <   \frac {\delta}{3}
$$
Consider the L1 distance of nodes inside the same cluster: This is at most  $k\frac{\delta}{3}$ . For nodes that belong to different clusters, this will be at least  $k(\delta - 2\delta^{*}) > k\frac{\delta}{3}$ . Therefore, the average linkage will combine all nodes belonging to the same role before it links nodes that belong to different roles.
Corollary E.1. Let  $A^{(1)}, \ldots, A^{(s)}$  be independent samples of the RIP model with the same role assignment ( $\Omega_{\text{role}}$  must not necessarily be the same). Assuming the prerequisites of theorem 4.3 for  $A = \frac{1}{s} \sum_{i=1}^{s} A^{(i)}$  except eq. 8. If
$$
s > - \frac {9 \mathcal {W} _ {- 1} ((q - 1) \delta^ {2} / 9 k ^ {2})}{2 n \delta^ {2}}
$$
Then with probability at least  $q$ : Algorithm 1 finds the correct role assignment using average linkage for clustering.