SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The following fact shows that every property of low-level pseudo-distributions can be derived by low-degree sum-of-squares proofs.
Fact A.18 (Completeness). Suppose  $d \geqslant r' \geqslant r$  and  $\mathcal{A}$  is a collection of polynomial constraints with degree at most  $r$ , and  $\mathcal{A} \vdash \{\sum_{i=1}^{n} w_i^2 \leqslant B\}$  for some finite  $B$ .
Let  $\{g\geqslant 0\}$  be a polynomial constraint. If every degree-  $^d$  pseudo-distribution that satisfies  $D\mid_{r}\mathcal{A}$  also satisfies  $D\mid_{\overline{r^{\prime}}}\{g\geqslant 0\}$ , then for every  $\varepsilon >0$ , there is a sum-of-squares proof  $\mathcal{A}\big|_{\overline{d}}\{g\geqslant -\varepsilon \}$ .
A.2.3 Explicitly bounded distributions
We will consider a subset of subgaussian distributions denoted as certifiably subgaussians. Many subgaussians distributions are known to be certifiably subgaussian (see [36]).
Definition A.19 (Explicitly bounded distribution). Let  $t \in \mathbb{N}$ . A distribution  $D$  over  $\mathbb{R}^d$  with mean  $\mu$  is called 2t-explicitly  $\sigma$ -bounded if for each even integer  $s$  such that  $1 \leqslant s \leqslant t$  the following equation has a degree  $s$  sum-of-squares proof in the vector variable  $u$
$$
\left| \frac {2 s}{u} \left\{\underset {\mathbf {x} \sim D} {\mathbb {E}} \langle \mathbf {x} - \mu , u \rangle^ {2 s} \leqslant (\sigma s) ^ {s} \cdot \| u \| _ {2} ^ {2 s} \right\} \right.
$$
Furthermore, we say that  $D$  is explicitly bounded if it is  $2t$ -explicitly  $\sigma$ -bounded for every  $t \in \mathbb{N}$ . A finite set  $X \subseteq \mathbb{R}^d$  is said to be  $2t$ -explicitly  $\sigma$ -bounded if the uniform distribution on  $X$  is  $2t$ -explicitly  $\sigma$ -bounded.
Sets that are  $2t$ -explicitly  $\sigma$ -bounded with large intersection satisfy certain key properties. Before introducing them we conveniently present the following definition.
Definition A.20 (Weight vector inducing distribution). Let  $Y$  be a set of size  $n$  and let  $p \in [0,1]^n$  be a vector satisfying  $\| p \|_1 = 1$ . We say that  $p$  induces the distribution  $D$  with support  $Y$  if
$$
\mathbb {P} _ {\mathbf {y} \sim D} [ \mathbf {y} = y _ {i} ] = p _ {i}.
$$
Theorem A.21 ([36, 30]). Let  $Y \subseteq \mathbb{R}^d$  be a set of cardinality  $n$ . Let  $p, p' \in [0,1]^n$  be weight vectors satisfying  $\| p \|_1 = \| p' \|_1 = 1$  and  $\| p - p' \|_1 \leqslant \beta$ . Suppose that  $p$  (respectively  $p'$ ) induces a 2t-explicitly  $\sigma_1$ -bounded (resp.  $\sigma_2$ ) distribution over  $Y$  with mean  $\mu_{(p)}$  (resp.  $\mu_{(p')}$ ). There exists an absolute constant  $\beta^*$  such that, if  $\beta \leqslant \beta^*$ , then for  $\sigma = \sigma_1 + \sigma_2$ :
$$
\left\| \mu_ {(p)} - \mu_ {(p ^ {\prime})} \right\| \leqslant \beta^ {1 - 1 / 2 t} \cdot O \left(\sqrt {\sigma t}\right).
$$
In the context of learning Gaussian mixtures, we will make heavy use of the statement below.
Theorem A.22 ([36, 30]). Let  $Y$  be a  $2t$ -explicitly  $\sigma$ -bounded set of size  $n$ . Let  $p \in \mathbb{R}^n$  be the weight vector inducing the uniform distribution over  $Y$ . Let  $p' \in \mathbb{R}^n$  be a unit vector satisfying  $\| p - p' \|_1 \leqslant \beta$  for some  $\beta \leqslant \beta^*$  where  $\beta^*$  is a small constant. Then  $p'$  induces a  $2t$ -explicitly  $(\sigma + O(\beta^{1 - 1/2t}))$ -bounded distribution over  $Y$ .
B Stability of strongly-convex optimization
In this section, we prove  $\ell_2$  sensitivity bounds for the minimizers of a general class of (strongly) convex optimization problems. In particular, we show how to translate a uniform point-wise sensitivity bound for the objective functions into a  $\ell_2$  sensitivity bound for the minimizers.
Lemma B.1 (Stability of strongly-convex optimization). Let  $\mathcal{V}$  be a set of datasets. Let  $\mathcal{K}(\mathcal{V})$  be a family of closed convex subsets of  $\mathbb{R}^m$  parametrized by  $Y\in \mathcal{V}$  and let  $\mathcal{F}(\mathcal{V})$  be a family of functions  $f_{Y}:\mathcal{K}(Y)\to \mathbb{R}$ , parametrized by  $Y\in \mathcal{V}$ , such that:
(i) for adjacent datasets  $Y, Y' \in \mathcal{V}$  and  $X \in \mathcal{K}(Y)$  there exist  $Z \in \mathcal{K}(Y) \cap \mathcal{K}(Y')$  satisfying  $|f_{Y}(X) - f_{Y'}(Z)| \leqslant \alpha$  and  $|f_{Y}(Z) - f_{Y'}(Z)| \leqslant \alpha$ .
(ii)  $f_{Y}$  is  $\kappa$ -strongly convex in  $X \in \mathcal{K}(Y)$ .
Then for  $Y, Y' \in \mathcal{Y}$ ,  $\hat{X} \coloneqq \arg \min_{X \in \mathcal{K}(Y)} f_{Y}(X)$  and  $\hat{X}' \coloneqq \arg \min_{X' \in \mathcal{K}(Y')} f_{Y'}(X')$ , it holds
$$
\left\| \hat {X} - \hat {X} ^ {\prime} \right\| _ {2} ^ {2} \leqslant \frac {1 2 \alpha}{\kappa}.
$$
Proof. Let  $Z \in \mathcal{K}(Y) \cap \mathcal{K}(Y')$  be a point such that  $\left|f_{Y}(\hat{X}) - f_{Y'}(Z)\right| \leqslant \alpha$  and  $|f_{Y}(Z) - f_{Y'}(Z)| \leqslant \alpha$ . By  $\kappa$ -strong convexity of  $f_{Y}$  and  $f_{Y'}$  (Proposition G.2) it holds
$$
\begin{array}{l} \left\| \hat {X} - \hat {X} ^ {\prime} \right\| _ {2} ^ {2} \leqslant 2 \left\| \hat {X} - Z \right\| _ {2} ^ {2} + 2 \left\| Z - \hat {X} ^ {\prime} \right\| _ {2} ^ {2} \\ \leqslant \frac {4}{\kappa} \left(f _ {Y} (Z) - f _ {Y} (\hat {X}) + f _ {Y ^ {\prime}} (Z) - f _ {Y ^ {\prime}} (\hat {X} ^ {\prime})\right). \\ \end{array}
$$
Suppose w.l.o.g.  $f_{Y}(\hat{X}) \leqslant f_{Y^{\prime}}(\hat{X}^{\prime})$ , for a symmetric argument works in the other case. Then
$$
f _ {Y} (Z) \leqslant f _ {Y ^ {\prime}} (Z) + \alpha \leqslant f _ {Y} (\hat {X}) + 2 \alpha
$$
$$
f _ {Y} (\hat {X}) \leqslant f _ {Y ^ {\prime}} (\hat {X} ^ {\prime}) \leqslant f _ {Y ^ {\prime}} (Z) \leqslant f _ {Y} (\hat {X}) + \alpha .
$$
It follows as desired
$$
f _ {Y} (Z) - f _ {Y} (\hat {X}) + f _ {Y ^ {\prime}} (Z) - f _ {Y ^ {\prime}} (\hat {X} ^ {\prime}) \leqslant 3 \alpha .
$$
C Private recovery for stochastic block models
In this section, we present how to achieve exact recovery in stochastic block models privately and thus prove Theorem 1.3. To this end, we first use the stability of strongly convex optimization (Lemma B.1) to obtain a private weak recovery algorithm in Appendix C.1. Then we show how to privately boost the weak recovery algorithm to achieve exact recovery in Appendix C.2. In Appendix C.4, we complement our algorithmic results by providing an almost tight lower bound on the privacy parameters. We start by defining the relevant notion of adjacent datasets.
Definition C.1 (Adjacent graphs). Let  $G, G'$  be graphs with vertex set  $[n]$ . We say that  $G, G'$  are adjacent if  $|E(G) \triangle E(G')| = 1$ .
Remark C.2 (Parameters as public information). We remark that we assume the parameters  $n, \gamma, d$  to be public information given in input to the algorithm.
C.1 Private weak recovery for stochastic block models
In this section, we show how to achieve weak recovery privately via stability of strongly convex optimization (Lemma B.1). We first introduce one convenient notation. The error rate of an estimate  $\hat{x} \in \{\pm 1\}^n$  of the true partition  $x \in \{\pm 1\}^n$  is defined as  $\mathrm{err}(\hat{x}, x) := \frac{1}{n}\cdot \min \{\mathrm{Ham}(\hat{x}, x), \mathrm{Ham}(\hat{x}, -x)\}$ .<sup>24</sup> Our main result is the following theorem.
Theorem C.3. Suppose  $\gamma \sqrt{d} \geqslant 12800, \varepsilon, \delta \geqslant 0$ . There exists an (Algorithm C.4) such that, for any  $x \in \{\pm 1\}^n$ , on input  $\mathbf{G} \sim \mathrm{SBM}_n(\gamma, d, x)$ , outputs  $\hat{x}(\mathbf{G}) \in \{\pm 1\}^n$  satisfying
$$
\operatorname {e r r} \left(\hat {x} (\mathbf {G}), x\right) \leqslant O \left(\frac {1}{\gamma \sqrt {d}} + \frac {1}{\gamma d} \cdot \frac {\log (2 / \delta)}{\varepsilon^ {2}}\right)
$$
with probability  $1 - \exp(-\Omega(n))$ . Moreover, the algorithm is  $(\varepsilon, \delta)$ -differentially private for any input graph and runs in polynomial time.
Before presenting the algorithm we introduce some notation. Given a graph  $G$ , let  $Y(G) := \frac{1}{\gamma d} (A(G) - \frac{d}{n} J)$  where  $A(G)$  is the adjacency matrix of  $G$  and  $J$  denotes all-one matrices. Define  $\mathcal{K} := \{X \in \mathbb{R}^{n \times n} \mid X \succeq 0, X_{ii} = \frac{1}{n} \forall i\}$ . The algorithm starts with projecting matrix  $Y(G)$  to set  $\mathcal{K}$ . To ensure privacy, then it adds Gaussian noise to the projection  $X_{1}$  and obtains a private matrix  $X_{2}$ . The last step applies a standard rounding method.
Algorithm C.4 (Private weak recovery for SBM).
Input: Graph  $G$
Operations:
1. Projection:  $X_{1}\gets \operatorname{argmin}_{X\in \mathcal{K}}\| Y(G) - X\|_{F}^{2}$ .
2. Noise addition:  $\mathbf{X}_2\gets X_1 + \mathbf{W}$  where  $\mathbf{W}\sim \mathcal{N}\left(0,\frac{24}{n\gamma d}\frac{\log(2 / \delta)}{\varepsilon^2}\right)^{n\times n}$ .
3. Rounding: Compute the leading eigenvector  $\mathbf{v}$  of  $\mathbf{X}_2$  and return  $\mathrm{sign}(\mathbf{v})$ .
In the rest of this section, we will show Algorithm C.4 is private in Lemma C.7 and its utility guarantee in Lemma C.8. Then Theorem C.3 follows directly from Lemma C.7 and Lemma C.8.
Privacy analysis Let  $\mathcal{V}$  be the set of all matrices  $Y(G) = \frac{1}{\gamma d} (A(G) - \frac{d}{n} J)$  where  $G$  is a graph on  $n$  vertices. We further define  $q:\mathcal{V}\to \mathcal{K}$  to be the function
$$
q (Y) := \operatorname {a r g m i n} _ {X \in \mathcal {K}} \| Y - X \| _ {\mathrm {F}} ^ {2}. \tag {C.1}
$$
We first use Lemma B.1 to prove that function  $q$  is stable.
Lemma C.5 (Stability). The function  $q$  as defined in Eq. (C.1) has  $\ell_2$ -sensitivity  $\Delta_{q,2} \leqslant \sqrt{\frac{24}{n\gamma d}}$ .
$^{24}$ Note  $|\langle \hat{x},x\rangle | = (1 - 2\operatorname {err}(\hat{x},x))\cdot n$  for any  $\hat{x},x\in \{\pm 1\} ^n$
Proof. Let  $g: \mathcal{V} \times \mathcal{K} \to \mathbb{R}$  be the function  $g(Y, X) \coloneqq \|X\|_{\mathrm{F}}^{2} - 2\langle Y, X \rangle$ . Applying Lemma B.1 with  $f_{Y}(\cdot) = g(Y, \cdot)$ , it suffices to prove that  $g$  has  $\ell_{1}$ -sensitivity  $\frac{4}{n\gamma d}$  with respect to  $Y$  and that it is 2-strongly convex with respect to  $X$ . The  $\ell_{1}$ -sensitivity bound follows by observing that adjacent  $Y, Y'$  satisfy  $\|Y - Y'\|_{1} \leqslant \frac{2}{\gamma d}$  and that any  $X \in \mathcal{K}$  satisfies  $\|X\|_{\infty} \leqslant \frac{1}{n}$ . Thus it remains to prove strong convexity with respect to  $X \in \mathcal{K}$ . Let  $X, X' \in \mathcal{K}$  then
$$
\begin{array}{l} \left\| X ^ {\prime} \right\| _ {\mathrm {F}} ^ {2} = \left\| X \right\| _ {\mathrm {F}} ^ {2} + 2 \langle X ^ {\prime} - X, X \rangle + \left\| X - X ^ {\prime} \right\| _ {\mathrm {F}} ^ {2} \\ = \| X \| _ {F} ^ {2} + 2 \langle X ^ {\prime} - X, X + Y - Y \rangle + \| X - X ^ {\prime} \| _ {F} ^ {2} \\ = g (Y, X) + \langle X ^ {\prime} - X, \nabla g (X, Y) \rangle + 2 \langle X ^ {\prime}, Y \rangle + \| X - X ^ {\prime} \| _ {\mathrm {F}} ^ {2}. \\ \end{array}
$$
That is  $g(Y, X)$  is 2-strongly convex with respect to  $X$ . Note any  $X \in \mathcal{K}$  is symmetric. Then the result follows by Lemma B.1.
Remark C.6. In the special case where the constraint set  $\mathcal{K}$  does not depend on input dataset  $Y$  (e.g. stochastic block models), the proof can be cleaner as follows. Let  $f_{Y}(X)\coloneqq \| X\|_{F}^{2} - 2\langle X,Y\rangle$ . Let  $\hat{X}\coloneqq \operatorname *{argmin}_{X\in \mathcal{K}}f_{Y}(X)$  and  $\hat{X^{\prime}}\coloneqq \operatorname *{argmin}_{X\in \mathcal{K}}f_{Y^{\prime}}(X)$ . Suppose without loss of generality  $f_{Y}(\hat{X})\leqslant f_{Y^{\prime}}(\hat{X}^{\prime})$ , for a symmetric argument works in the other case. Then
$$
f _ {Y} (\hat {X}) \leqslant f _ {Y ^ {\prime}} (\hat {X} ^ {\prime}) \leqslant f _ {Y ^ {\prime}} (\hat {X}) \leqslant f _ {Y} (\hat {X}) + \alpha .
$$
Then it is easy to show the algorithm is private.
Lemma C.7 (Privacy). The weak recovery algorithm (Algorithm C.4) is  $(\varepsilon, \delta)$ -DP.
Proof. Since any  $X \in \mathcal{K}$  is symmetric, we only need to add a symmetric noise matrix to obtain privacy. Combining Lemma C.5 with Lemma A.12, we immediately get that the algorithm is  $(\varepsilon, \delta)$ -private.
Utility analysis Now we show the utility guarantee of our priavte weak recovery algorithm.
Lemma C.8 (Utility). For any  $x \in \{\pm 1\}^n$ , on input  $\mathbf{G} \sim \mathsf{SBM}_n(\gamma, d, x)$ , Algorithm C.4 efficiently outputs  $\hat{x}(\mathbf{G}) \in \{\pm 1\}^n$  satisfying
$$
\operatorname {e r r} \left(\hat {x} (\mathbf {G}), x\right) \leqslant \frac {6 4 0 0}{\gamma \sqrt {d}} + \frac {7 0 0 0}{\gamma d} \cdot \frac {\log (2 / \delta)}{\varepsilon^ {2}},
$$
with probability  $1 - \exp (-\Omega (n))$
To prove Lemma C.8, we need the following lemma which is an adaptation of a well-known result in SBM [28, Theorem 1.1]. Its proof is deferred to Appendix H.
Lemma C.9. Consider the settings of Lemma C.8. With probability  $1 - \exp (-\Omega (n))$
$$
\left\| X _ {1} (\mathbf {G}) - \frac {1}{n} x x ^ {\top} \right\| _ {F} ^ {2} \leqslant \frac {8 0 0}{\gamma \sqrt {d}}.
$$
Proof of Lemma C.8. By Lemma C.9, we have
$$
\left\| X _ {1} (\mathbf {G}) - \frac {1}{n} x x ^ {\top} \right\| \leqslant \left\| X _ {1} (\mathbf {G}) - \frac {1}{n} x x ^ {\top} \right\| _ {F} \leqslant \sqrt {\frac {8 0 0}{\gamma \sqrt {d}}} =: r (\gamma , d)
$$
with probability  $1 - \exp(-\Omega(n))$ . We condition our following analysis on this event happening.
Let  $\mathbf{u}$  be the leading eigenvector of  $X_{1}(\mathbf{G})$ . Let  $\lambda_{1}$  and  $\lambda_{2}$  be the largest and second largest eigenvalues of  $X_{1}(\mathbf{G})$ . By Weyl's inequality (Lemma F.1) and the assumption  $\gamma \sqrt{d} \geqslant 12800$ , we have
$$
\lambda_ {1} - \lambda_ {2} \geqslant 1 - 2 r (\gamma , d) \geqslant \frac {1}{2}.
$$
Let  $\mathbf{v}$  be the leading eigenvector of  $X_{1}(\mathbf{G}) + \mathbf{W}$ . By Davis-Kahan's theorem (Lemma F.2), we have
$$
\left\| \mathbf {u} - \mathbf {v} \right\| \leqslant \frac {2 \left\| \mathbf {W} \right\|}{\lambda_ {1} - \lambda_ {2}} \leqslant 4 \left\| \mathbf {W} \right\|,
$$
$$
\left\| \mathbf {u} - x / \sqrt {n} \right\| \leqslant 2 \left\| X _ {1} (\mathbf {G}) - \frac {1}{n} x x ^ {\top} \right\| \leqslant 2 r (\gamma , d).
$$
Putting things together and using Fact E.1, we have
$$
\left\| \mathbf {v} - x / \sqrt {n} \right\| \leqslant \| \mathbf {u} - \mathbf {v} \| + \left\| \mathbf {u} - x / \sqrt {n} \right\| \leqslant \frac {2 4 \sqrt {6}}{\sqrt {\gamma d}} \frac {\sqrt {\log (2 / \delta)}}{\varepsilon} + 2 r (\gamma , d)
$$
Observe  $\mathrm{Ham}(\mathrm{sign}(y),x)\leqslant \| y - x\| ^2$  for any  $y\in \mathbb{R}^n$  and any  $x\in \{\pm 1\} ^n$ . Then with probability  $1 - \exp (-\Omega (n))$ ,
$$
\frac {1}{n} \cdot \operatorname {H a m} (\operatorname {s i g n} (\mathbf {v}), x) \leqslant \left\| \mathbf {v} - x / \sqrt {n} \right\| ^ {2} \leqslant \frac {6 4 0 0}{\gamma \sqrt {d}} + \frac {7 0 0 0}{\gamma d} \cdot \frac {\log (2 / \delta)}{\varepsilon^ {2}}.
$$
Proof of Theorem C.3. By Lemma C.7 and Lemma C.8.
C.2 Private exact recovery for stochastic block models
In this section, we prove Theorem 1.3. We show how to achieve exact recovery in stochastic block models privately by combining the private weak recovery algorithm we obtained in the previous section and a private majority voting scheme.
Since exact recovery is only possible with logarithmic average degree (just to avoid isolated vertices), it is more convenient to work with the following standard parameterization of stochastic block models. Let  $\alpha >\beta >0$  be fixed constants. The intra-community edge probability is  $\alpha \cdot \frac{\log n}{n}$ , and the intercommunity edge probability is  $\beta \cdot \frac{\log n}{n}$ . In the language of Model 1.1, it is  $\mathrm{SBM}_n(\frac{\alpha + \beta}{2}\cdot \log n,\frac{\alpha - \beta}{\alpha + \beta},x)$ . Our main result is the following theorem.
Theorem C.10 (Private exact recovery of SBM, restatement of Theorem 1.3). Let  $\varepsilon, \delta \geqslant 0$ . Suppose  $\alpha, \beta$  are fixed constants satisfying<sup>25</sup>
$$
\sqrt {\alpha} - \sqrt {\beta} \geqslant 1 6 \quad a n d \quad \alpha - \beta \geqslant \Omega \left(\frac {1}{\varepsilon^ {2}} \cdot \frac {\log (2 / \delta)}{\log n} + \frac {1}{\varepsilon}\right), \tag {C.2}
$$
Then there exists an algorithm (Algorithm C.12) such that, for any balanced $^{26}$ $x \in \{\pm 1\}^n$ , on input  $\mathbf{G} \sim \mathsf{SBM}_n\left(\frac{\alpha + \beta}{2} \cdot \log n, \frac{\alpha - \beta}{\alpha + \beta}, x\right)$ , outputs  $\hat{x}(\mathbf{G}) \in \{x, -x\}$  with probability  $1 - o(1)$ . Moreover, the algorithm is  $(\varepsilon, \delta)$ -differentially private for any input graph and runs in polynomial time.
Remark C.11. In a standard regime of privacy parameters where  $\varepsilon \leqslant O(1)$  and  $\delta = 1 / \mathrm{poly}(n)$ , the private exact recovery threshold Eq. (C.2) reads
$$
\sqrt {\alpha} - \sqrt {\beta} \geqslant 1 6 \quad a n d \quad \alpha - \beta \geqslant \Omega \left(\varepsilon^ {- 2} + \varepsilon^ {- 1}\right),
$$
Recall the non-private exact recovery threshold is  $\sqrt{\alpha} -\sqrt{\beta} >\sqrt{2}$ . Thus the non-private part in Eq. (C.2), i.e. 16, is close to optimal.
Algorithm C.12 starts with randomly splitting the input graph  $G$  into two subgraphs  $\mathbf{G}_1$  and  $\mathbf{G}_2$ . Setting the graph-splitting probability to  $1/2$ , each subgraph will contain about half of the edges of  $G$ . Then we run an  $(\varepsilon, \delta)$ -DP weak recovery algorithm (Algorithm C.4) on  $\mathbf{G}_1$  to get a rough estimate  $\tilde{x}(\mathbf{G}_1)$  of accuracy around  $90\%$ . Finally, we boost the accuracy to  $100\%$  by doing majority voting (Algorithm C.13) on  $\mathbf{G}_2$  based on the rough estimate  $\tilde{x}(\mathbf{G}_1)$ . That is, if a vertex has more neighbors from the opposite community (according to  $\tilde{x}(\mathbf{G}_1)$ ) in  $\mathbf{G}_2$ , then we assign this vertex to the opposite community. To make the majority voting step private, we add some noise to the vote.
25 In the language of Model 1.1, for any  $t$  we have  $\sqrt{\alpha} - \sqrt{\beta} \geqslant t$  if and only if  $\frac{d}{\log n} (1 - \sqrt{1 - \gamma^2}) \geqslant \frac{t^2}{2}$ . Recall a vector  $x \in \{\pm 1\}^n$  is said to be balanced if  $\sum_{i=1}^{n} x_i = 0$ .
Algorithm C.12 (Private exact recovery for SBM).
1. Graph-splitting: Initialize  $\mathbf{G}_1$  to be an empty graph on vertex set  $V(G)$ . Independently put each edge of  $G$  in  $\mathbf{G}_1$  with probability  $1/2$ . Let  $\mathbf{G}_2 = G \setminus \mathbf{G}_1$ .
2. Rough estimation on  $\mathbf{G}_1$ : Run the  $(\varepsilon, \delta)$ -DP partial recovery algorithm (Algorithm C.4) on  $\mathbf{G}_1$  to get a rough estimate  $\tilde{x}(\mathbf{G}_1)$ .
3. Majority voting on  $\mathbf{G}_2$ : Run the  $(\varepsilon, 0)$ -DP majority voting algorithm (Algorithm C.13) with input  $(\mathbf{G}_2, \tilde{x}(\mathbf{G}_1))$  and get output  $\hat{\mathbf{x}}$ .
4. Return  $\hat{\mathbf{x}}$ .
Algorithm C.13 (Private majority voting).
Input: Graph  $G$ , rough estimate  $\tilde{x} \in \{\pm 1\}^n$
1. For each vertex  $v \in V(G)$ , let  $\mathbf{Z}_v = \mathbf{S}_v - \mathbf{D}_v$  where
$$
\begin{array}{l} \bullet \mathbf {D} _ {v} = \sum_ {\{u, v \} \in E (G)} \mathbb {1} _ {[ \tilde {x} _ {u} \neq \tilde {x} _ {v} ]}, \\ \bullet \mathbf {S} _ {v} = \sum_ {\{u, v \} \in E (G)} \mathbb {1} _ {[ \tilde {x} _ {u} = \tilde {x} _ {v} ]}. \\ \end{array}
$$
$$
S e t \hat {\mathbf {x}} _ {v} = \operatorname {s i g n} \left(\mathbf {Z} _ {v} + \mathbf {W} _ {v}\right) \cdot \tilde {x} \left(\mathbf {G} _ {1}\right) _ {v} w h e r e \mathbf {W} _ {v} \sim \mathrm {L a p} (2 / \varepsilon).
$$
2. Return  $\hat{\mathbf{x}}$ .
In the rest of this section, we will show Algorithm C.12 is private in Lemma C.15 and it recovers the hidden communities exactly with high probability in Lemma C.17. Then Theorem C.10 follows directly from Lemma C.15 and Lemma C.17.
Privacy analysis. We first show the differential privacy of the majority voting algorithm (Algorithm C.13) with respect to input graph  $G$  (i.e. assuming fixed the input rough estimate).
Lemma C.14. Algorithm C.13 is  $(\varepsilon, 0)$ -DP with respect to input  $G$ .
Proof. Observing the  $\ell_1$ -sensitivity of the degree count function  $Z$  in step is 2, the  $(\varepsilon, 0)$ -DP follows directly from Laplace mechanism (Lemma A.12) and post-processing (Lemma A.2).
Then the privacy of the private exact recovery algorithm (Algorithm C.12) is a consequence of composition.
Lemma C.15 (Privacy). Algorithm C.12 is  $(\varepsilon, \delta)$ -DP.
Proof. Let  $\mathcal{A}_1: \mathcal{G}_n \to \{\pm 1\}^n$  denote the  $(\varepsilon, \delta)$ -DP recovery algorithm in step 2. Let  $\mathcal{A}_2: \mathcal{G}_n \times \{\pm 1\}^n \to \{\pm 1\}^n$  denote the  $(\varepsilon, 0)$ -DP majority voting algorithm in step 3. Let  $\mathcal{A}$  be the composition of  $\mathcal{A}_1$  and  $\mathcal{A}_2$ .
We first make several notations. Given a graph  $H$  and an edge  $e$ ,  $H_{e}$  is a graph obtained by adding  $e$  to  $H$ . Given a graph  $H$ ,  $\mathbf{G}_1(H)$  is a random subgraph of  $H$  by keeping each edge of  $H$  with probability 1/2 independently.
Now, fix two adjacent graphs  $G$  and  $G_e$  where edge  $e$  appears in  $G_e$  but not in  $G$ . Also, fix two arbitrary possible outputs  $x_1, x_2 \in \{\pm 1\}^n$  of algorithm  $\mathcal{A}$ .<sup>27</sup> It is direct to see,
$$
\mathbb {P} \left(\mathcal {A} (G) = \left(x _ {1}, x _ {2}\right)\right) = \sum_ {H \subseteq G} \mathbb {P} \left(\mathcal {A} _ {1} (H) = x _ {1}\right) \mathbb {P} \left(\mathcal {A} _ {2} (G \setminus H, x _ {1}) = x _ {2}\right) \mathbb {P} \left(\mathbf {G} _ {1} (G) = H\right). \tag {C.3}
$$
27We can imagine that algorithm  $\mathcal{A}$  first outputs  $(x_{1},x_{2})$  and then outputs  $x_{2}$  as a post-processing step.
Since  $\mathbb{P}(\mathbf{G}_1(G) = H) = \mathbb{P}(\mathbf{G}_1(G_e) = H) + \mathbb{P}(\mathbf{G}_1(G_e) = H_e)$  for any  $H\subseteq G$  , we have
$$
\begin{array}{l} \mathbb {P} \left(\mathcal {A} \left(G _ {e}\right) = \left(x _ {1}, x _ {2}\right)\right) = \sum_ {H \subseteq G} \mathbb {P} \left(\mathcal {A} _ {1} (H) = x _ {1}\right) \mathbb {P} \left(\mathcal {A} _ {2} \left(G _ {e} \backslash H, x _ {1}\right) = x _ {2}\right) \mathbb {P} \left(\mathbf {G} _ {1} \left(G _ {e}\right) = H\right) \\ + \mathbb {P} \left(\mathcal {A} _ {1} \left(H _ {e}\right) = x _ {1}\right) \mathbb {P} \left(\mathcal {A} _ {2} \left(G _ {e} \backslash H _ {e}, x _ {1}\right) = x _ {2}\right) \mathbb {P} \left(\mathbf {G} _ {1} \left(G _ {e}\right) = H _ {e}\right) \tag {C.4} \\ \end{array}
$$
Since both  $\mathcal{A}_1$  and  $\mathcal{A}_2$  are  $(\varepsilon, \delta)$ -DP, we have for each  $H \subseteq G$ ,
$$
\mathbb {P} \left(\mathcal {A} _ {1} \left(H _ {e}\right) = x _ {1}\right) \leqslant e ^ {\varepsilon} \mathbb {P} \left(\mathcal {A} _ {1} (H) = x _ {1}\right) + \delta , \tag {C.5}
$$
$$
\mathbb {P} \left(\mathcal {A} _ {2} \left(G _ {e} \backslash H, x _ {1}\right) = x _ {2}\right) \leqslant e ^ {\varepsilon} \mathbb {P} \left(\mathcal {A} _ {2} (G \backslash H, x _ {1}) = x _ {2}\right) + \delta . \tag {C.6}
$$
Plugging Eq. (C.5) and Eq. (C.6) into Eq. (C.4), we obtain
$$
\begin{array}{l} \mathbb {P} \left(\mathcal {A} \left(G _ {e}\right) = \left(x _ {1}, x _ {2}\right)\right) \leqslant \sum_ {H \subseteq G} \left[ e ^ {\varepsilon} \mathbb {P} \left(\mathcal {A} _ {1} (H) = x _ {1}\right) \mathbb {P} \left(\mathcal {A} _ {2} (G \setminus H, x _ {1}) = x _ {2}\right) + \delta \right] \mathbb {P} \left(\mathbf {G} _ {1} (G) = H\right) \\ = e ^ {\varepsilon} \mathbb {P} (\mathcal {A} (G) = (x _ {1}, x _ {2})) + \delta . \\ \end{array}
$$
Similarly, we can show
$$
\mathbb {P} \left(\mathcal {A} (G) = \left(x _ {1}, x _ {2}\right)\right) \leqslant e ^ {\varepsilon} \mathbb {P} \left(\mathcal {A} \left(G _ {e}\right) = \left(x _ {1}, x _ {2}\right)\right) + \delta . \tag {C.7}
$$
Utility analysis. We first show the utility guarantee of the priavte majority voting algorithm.