SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Badih Ghazi
Google Research Mountain View, CA, US badihghazi@gmail.com
Pritish Kamath
Google Research Mountain View, CA, US pritish@alum.mit.edu
Ravi Kumar  
Google Research  
Mountain View, CA, US  
ravi.k53@gmail.com
Pasin Manurangsi
Google Research Bangkok, Thailand pasin@google.com
Adam Sealfon  
Google Research  
New York, NY, US  
adamsealfon@google.com
We study the problem of computing pairwise statistics, i.e., ones of the form  $\binom{n}{2}^{-1}\sum_{i\neq j}f(x_i,x_j)$ , where  $x_{i}$  denotes the input to the  $i$ th user, with differential privacy (DP) in the local model. This formulation captures important metrics such as Kendall's  $\tau$  coefficient, Area Under Curve, Gini's mean difference, Gini's entropy, etc. We give several novel and generic algorithms for the problem, leveraging techniques from DP algorithms for linear queries.
Differential privacy (DP) [DMNS06] is a widely studied and used notion for quantifying the privacy protections of algorithms in data analytics and machine learning. It has witnessed many practical deployments [EPK14, Sha14, Gre16, App17, DKY17, Abo18, RE19, TM20, KT18, RSP $^{+}$ 21]. On a high level, DP dictates that the output of a (randomized) algorithm remains statistically indistinguishable if the input of any single user is modified; the degree of statistical indistinguishability is quantified by the privacy parameter  $\varepsilon > 0$ .
An important setting of DP is the local model [EGS03,  $\mathrm{KLN^{+}11}$ ], where each of  $n$  users holds an input that they wish to keep private. An analyst wishes to compute some known function of the users inputs. The analyst and the users engage in an interactive protocol at the end of which the analyst is supposed to compute an estimate to the value of the function on the user inputs. When the aforementioned statistical indistinguishability property is enforced on the algorithm's transcript, the algorithm is said to be  $\varepsilon$ -local DP (see Section 2 for a formal definition). Non-interactive algorithms refer to the setting where each user sends a single (DP) message to the analyst, who is then supposed to output an estimate of the desired value without any further interaction with the users. As usual in the interactive local DP setting, we assume a broadcast model where every communication is visible to all users and to the analyst (and subject to the DP constraint). The number of rounds of an interactive protocol refers to the number of back-and-forths between the set of users and the analyst.
While the local DP setting offers a compelling trust model compared to the central DP model (where an analyst is assumed to have access to the raw user data and the privacy guarantee is only enforced at its output), the local setting is often limited by lower bounds on the error incurred by private protocols (e.g., [BNO08, CSS12]). In this work, we study some basic tasks in analytics and learning, and give local DP protocols with significantly smaller error than what was previously known.
Quadratic Form Computation. Throughout the paper, we consider the following task:
Definition 1 (Quadratic Form Computation). Given a matrix  $W \in \mathbb{R}^{k \times k}$ . Each user  $i$  receives  $x_{i} \in [k]$ . The goal is to compute the quadratic form  $h_{\mathbf{x}}^T W h_{\mathbf{x}}$  where  $h_{\mathbf{x}} \in \mathbb{Z}_{\geq 0}^k$  denotes the normalized histogram of the input, i.e.,  $(h_{\mathbf{x}})_b \coloneqq \frac{1}{n} \cdot |\{i \in [n] \mid x_i = b\}|$ .
An algorithm computing quadratic forms immediately implies an algorithm for computing pairwise statistics (also known as  $U$ -statistics of degree 2) defined as  $\mathcal{F}(X) = \frac{1}{\binom{n}{2}} \sum_{1 \leq i < j \leq n} f(x_i, x_j)$ , where  $f: \mathcal{X}^2 \to \mathbb{R}$  is a symmetric function (called kernel), and  $x_i$  is the input to the  $i$ th user. The family of pairwise statistics is notable in that it contains several statistical quantities that are widely used in different areas, including the Area Under the Curve (AUC), Kendall's  $\tau$  coefficient, the Gini's mean difference, and the Gini's diversity index (aka Gini-Simpson index or Gini's impurity). Computing quadratic forms has been studied in the context of local DP by Bell et al. [BBGK20] who gave algorithms for functions  $f$  that are Lipschitz and for estimating the AUC of a classifier; the latter was improved recently [CM23]. While these problems are simple in central DP (as we can directly add noise to the function value), we note that a clear challenge for computing pairwise statistics (and hence quadratic forms) in local DP is that each summand depends on data points held by two different users.
Linear Queries. Computation of quadratic forms is a natural "degree-2" variant of the so-called linear queries (defined below), a well-studied problem in the DP literature.
Definition 2 (Linear Queries). Given a workload matrix  $W \in \mathbb{R}^{k \times k}$ , the goal is to compute  $Wh_{\mathbf{x}}$  where  $h_{\mathbf{x}} \in \mathbb{Z}_{\geq 0}^{k}$  denotes the normalized histogram of the input.
There is a long line of work on privately answering linear queries. In both the central and local models, the optimal errors achievable by DP algorithms are (mostly) well-understood for various types of errors, such as the  $\ell_2^2$ -error or the  $\ell_{\infty}$ -error [HT10, BDKT12, LMH  $^{+}15$ , NTZ16, BUV18, BBNS19, ENU20, Nik23]. Recall that the MSE of an estimate  $\hat{z}$  of a real value  $z$  is defined as  $\mathrm{MSE}(\hat{z}, z) \coloneqq \mathbb{E}_{\hat{z}}[(\hat{z} - z)^2]$ . For our purpose, we consider the following natural measure of accuracy:
Definition 3 (mMSE). Given a mechanism  $\mathcal{M}$  that answers linear queries for a workload matrix  $W\in \mathbb{R}^{k\times k}$ , its maximum mean square error (mMSE) is defined as  $\mathrm{mMSE}(\mathcal{M};W)\coloneqq \max_{\mathbf{x}\in [k]^n}\max_{j\in [k]}\mathrm{MSE}(\mathcal{M}(\mathbf{x})_j,(Wh_{\mathbf{x}})_j)$ .
In a recent seminal work, Edmonds et al. [ENU20] provide a nearly optimal characterization of the error achievable by non-interactive  $\varepsilon$ -local DP algorithms. To describe their result, we need some additional definitions of norms on matrices and related quantities:
$\triangleright 1 \rightarrow 2$  norm: For any matrix  $A$ ,  $\| A \|_{1 \rightarrow 2}$  is the maximum  $(\ell_2)$ -norm of its columns.
$\triangleright \ell_{\infty}$ -norm: For any matrix  $A$ ,  $\| A \|_{\infty}$  is the maximum absolute value among its entries.
$\triangleright$  Factorization norm: For  $W \in \mathbb{R}^{k \times k}$ , let  $\gamma_2(W) \coloneqq \min_{L^T R = W} \| L \|_{1 \to 2} \| R \|_{1 \to 2}$ .
$\triangleright$  Approximate-Factorization norm: For  $W \in \mathbb{R}^{k \times k}$  and  $\alpha \geq 0$ , let  $\gamma_2(W; \alpha) := \min_{\| \tilde{W} - W \|_\infty \leq \alpha} \gamma_2(\tilde{W})$ .
Finally, we define  $\zeta(W,n) \coloneqq \min_{\alpha \geq 0} (\gamma_2(W;\alpha) + \alpha \varepsilon \sqrt{n})$ . (Note that  $\zeta(W,n) \leq \gamma_2(W)$  as we can pick  $\alpha = 0$ .) The result of Edmonds et al. [ENU20] states that this quantity, up to polylogarithmic factors, governs the best error achievable for linear queries in the non-interactive local DP model<sup>2</sup>:
Theorem 4 ([ENU20]). For any workload matrix  $W$ , there is a non-interactive  $\varepsilon$ -local DP mechanism for linear queries with mMSE at most  $O\left(\frac{\zeta(W,n)^2}{\varepsilon^2n}\right)$ . Furthermore, any non-interactive  $\varepsilon$ -local DP mechanism must incur mMSE at least  $\tilde{\Omega}\left(\frac{\zeta(W,n)^2}{\varepsilon^2n}\right)$ .
Given such a tight and generic characterization for linear queries, it is natural to ask:
Can we characterize the error of computing the quadratic form for any matrix  $W$  with local DP?
<sup>1</sup>Note that, unlike the three quantities above, this is not actually a norm.
2We remark that [ENU20] did not state their bounds in the form we present in Theorem 4 (or even for mMSE); we provide more detail on how to interpret their lower bound in this form in Appendix B. The upper bound is via the matrix mechanism, introduced in central DP earlier in  $\mathrm{[LMH^{+}15}$  NTZ16].
In this work, we answer this question by establishing connections between the problems of computing quadratic forms and linear queries. Leveraging the wealth of knowledge on the latter, we obtain several new (and general) upper and lower bounds for the former.
Non-Interactive Local DP. First, in the non-interactive setting, we give the following upper and lower bounds. Together, they show that the error one can expect for computing quadratic forms is essentially the same as that for computing linear queries (as provided in Theorem 4).
Theorem 5 (Non-interactive Algorithm). For any  $W \in \mathbb{R}^{k \times k}$ , there is a non-interactive  $\varepsilon$ -local DP mechanism for estimating quadratic forms on  $W$  with MSE at most  $O\left(\frac{\zeta(W, n)^2 (\log k)}{\varepsilon^2 n}\right)$ .
Theorem 6 (Non-interactive Lower Bound). For any symmetric  $W \in \mathbb{R}^{k \times k}$ , any non-interactive  $\varepsilon$ -local DP mechanism for estimating quadratic forms on  $W$  must incur MSE at least  $\tilde{\Omega}\left(\frac{\zeta(W,n)^2}{\varepsilon^2n}\right)$ .
Our results above are generic and can be applied to any pairwise statistics. To demonstrate the power of the algorithm, we now state implications for several classes of statistics that have been studied in the privacy literature. Due to space constraints, we defer their definitions to Section 5.
Corollary 7. There is a non-interactive  $\varepsilon$ -local DP mechanism for computing pairwise statistics for:
$\triangleright$  Any  $G$ -Lipschitz function  $f:[0,1]\to \mathbb{R}$ , with MSE  $O\left(\frac{G^2\log n}{\varepsilon^2n}\right)$
$\triangleright$  Kendall's  $\tau$  coefficient, with MSE  $O\left(\frac{(\log k)^5}{\varepsilon^2n}\right)$
$\triangleright$  AUC under the balancedness assumption<sup>3</sup>, with MSE  $O\left(\frac{(\log k)^3}{\varepsilon^2n}\right)$ ,
Gini's diversity index, with MSE  $O\left(\frac{\log k}{\varepsilon^2n}\right)$ .
For  $O(1)$ -Lipschitz functions, which includes several well-known metrics such as Gini mean difference, we improve upon the algorithm of Bell et al. [BBGK20] whose MSE is  $O\left(\frac{1}{\varepsilon\sqrt{n}}\right)$ . We are not aware of any previous bounds on Kendall's  $\tau$  coefficient before; the metric was mentioned in [BBGK20] without any error guarantee given. For AUC, our bound is the same as in [BBGK20] but worse than a follow-up work [CM23] by a log  $k$  factor; nonetheless, we stress that our result is derived as a corollary of a generic algorithm without relying too deeply on AUC (except for the  $\gamma_{2}$ -norm of its matrix). For Gini's diversity index, our bound is again worse than that from Bravo et al.  $\mathrm{[BHBFG^{+}22]}$  by log  $k$  factor, but their algorithm requires the use of public randomness; we do not use any public randomness. (Throughout this work, when we refer to the local model without further specification, we assume no public randomness, aka private-coin model.)
Remark. Since our non-interactive algorithm only requires a vector-summation primitive, we can also apply protocols for vector summation in the shuffle model (e.g., [CSU+19, BBGN20, GMPV20]) to obtain an  $(\varepsilon, \delta)$ -DP protocol in the shuffle model, reducing the MSE by a factor of  $n$  for each setting in Theorem 5 and Corollary 7.
Interactive Local DP. Finally, we also provide an interactive algorithm whose MSE does not depend on  $\gamma_{2}(W)$ , as long as  $n$  is sufficiently large:
Theorem 8 (Interactive Algorithm). For any  $W \in \mathbb{R}^{k \times k}$ , there is a three-round  $\varepsilon$ -DP algorithm for estimating quadratic forms on  $W$  such that, for  $n \geq \left(\frac{\gamma_2(W) \cdot \log k}{\|W\|_{\infty} \cdot \varepsilon}\right)^{O(1)}$ , the MSE is  $O\left(\frac{\|W\|_{\infty}^2}{\varepsilon^2 n}\right)$ .
We note that the dependence  $O\left(\frac{\|W\|_{\infty}^{2}}{\varepsilon^{2}n}\right)$  is the best possible: even when the  $k = 2$  and  $W$  is binary, the problem is as hard as binary summation, which is known to require MSE at least  $\Omega \left(\frac{1}{\varepsilon^2n}\right)$  even for an arbitrary number of rounds of interactions [CSS12], and we can rescale this hard instance to get any desired  $\ell_{\infty}$ -norm.
<sup>3</sup>The balancedness assumption for AUC states that there are  $\Omega(n)$  examples with each label 0, 1. This is a required assumption to achieve an error in the form presented, as otherwise when e.g., there is a single 0-labeled example, it is impossible to achieve any non-trivial guarantee.
Combining the non-interactive lower bound in Theorem 6 and the interactive upper bound in Theorem 8, our results imply that there exists a matrix  $W \in \mathbb{R}^{k \times k}$ ,  $\varepsilon > 0$ ,  $n \in \mathbb{N}$  such that interactive algorithms are provably more accurate than non-interactive ones for privately computing the quadratic form on  $W$ . This places computing pairwise statistics as one of the few problems (and perhaps the most natural) that are known to separate interactive and non-interactive local DP. Due to space constraints, we defer further discussion of this to the Appendix.
We now give a brief technical overview of our proofs.
Black-Box Reduction from Linear Queries (Lower Bound). As mentioned earlier, our results are shown through connections between computing quadratic forms and linear queries. We will start with black-box reductions between the two. Suppose that we have a non-interactive  $\varepsilon$ -local DP algorithm  $\mathbb{A}$  for computing the quadratic form on  $W$ . We will construct a non-interactive  $\varepsilon$ -local DP algorithm  $\mathbb{A}'$  for computing linear queries of  $W$ . This will allow us to establish our lower bound (Theorem 6) as a consequence of the lower bound for linear queries (Theorem 4).
For simplicity of this overview, instead of considering the full quadratic form, suppose that  $\mathbb{A}$  works on  $2n$  users with inputs  $x_{1},\ldots ,x_{n},y_{1},\ldots ,y_{n}$  and computes an estimate of  $h_{\mathbf{y}}^{T}Wh_{\mathbf{x}}$ . On input  $x_{1},\ldots ,x_{n}$ , algorithm  $\mathbb{A}$  works as follows:
Each user  $i$  runs the randomizer of  $\mathbb{A}$  on  $x_{i}$  to get a response  $o_i$  and sends it to the analyst.
For each  $j \in [k]$ , the analyst simulates running the randomizer of  $\mathbb{A}$  on  $y_{1} = \dots = y_{n} = j$  to get responses  $o_1', \ldots, o_n'$ . Then, the analyst lets  $\hat{z}_j$  be the estimator of  $\mathbb{A}$  based on the responses  $o_1, \ldots, o_n, o_1', \ldots, o_n'$ .
$\triangleright$  The analyst then outputs  $(\hat{z}_1,\dots ,\hat{z}_k)$  as its estimate.
In other words, the randomized responses from  $\mathbb{A}$  are used as an "oracle" for  $\mathbb{A}'$  to compute the different linear queries. The key observation here is that, when we set  $y_{1} = \dots = y_{n} = j$ ,  $\mathbb{A}$  produces an estimate for  $h_{\mathbf{y}}^{T}Wh_{\mathbf{x}} = \mathbf{1}_{j}^{T}Wh_{\mathbf{x}} = (Wh_{\mathbf{x}})_{j}$  as desired. Note that this reduction only works in the non-interactive setting: if we were in the interactive setting, the responses on  $x_{1},\ldots ,x_{n}$  (of protocol  $\mathbb{A}$ ) would have been dependent on those of  $y_{1},\ldots ,y_{n}$ . Therefore, the first step of the reduction would have been impossible.
While this encapsulates the high-level ideas of our proof, there are some details that needs to be handled. E.g., if  $\mathbb{A}$  outputs the quadratic form  $h_{\mathbf{x} \cup \mathbf{y}}^T Wh_{\mathbf{x} \cup \mathbf{y}}$ , there are "cross terms" of the form  $h_{\mathbf{x}}^T Wh_{\mathbf{x}}$  that need to be removed. We formally describe and analyze the full reduction in Section 4.
Black-Box Reduction to Linear Queries (Algorithm). We can also give a reduction in the reverse direction, although this results in an additional round of communication. Specifically, let  $\mathbb{A}'$  be a non-interactive  $\varepsilon$ -local DP algorithm for computing linear queries of  $W$ . We can construct a two-round  $(2\varepsilon)$ -local DP algorithm  $\mathbb{A}$  for computing the quadratic form on  $W$  as follows.
$\triangleright$  First Round: Run  $\mathbb{A}^{\prime}$  on all users to compute an estimate  $(\hat{z}_1,\dots ,\hat{z}_k)$  for  $Wh_{\mathbf{x}}$
$\triangleright$  Second Round: Each user  $j \in [n]$ , sends  $o_j = \hat{z}_{x_j} + \kappa_j$  to the analyst where  $\kappa_j$  is (appropriately calibrated) Laplace noise. The analyst then outputs  $\frac{1}{n} (o_1 + \dots + o_n)$ .
Again, we omit some details for simplicity, such as the fact that each  $\hat{z}_j$  may not be bounded a priori, which may make the second step violate DP. However, these are relatively straightforward to handle.
If there were no noise, we would have  $o_{j} = \mathbf{1}_{x_{j}}^{T}Wh_{\mathbf{x}}$  and thus  $\frac{1}{n}\left(o_1 + \dots +o_n\right) = h_\mathbf{x}^T Wh_\mathbf{x}$  as desired. Furthermore, it is not hard to see that the error from  $\kappa_{1},\ldots ,\kappa_{n}$  is dominated by the error from  $\mathbb{A}'$  in the first step. In other words, we get an error similar to the one in Theorem 5 here, but the protocol is interactive (two-round). Making the protocol non-interactive requires us to step away from the black-box approach and open up the linear query algorithm (Theorem 4).
White-Box Algorithms. For simplicity, in this section we describe protocols with accuracy that depends on  $\gamma_{2}(W)$ , which can be larger than  $\zeta(W, n)$ . The desired error dependence on  $\zeta(W, n)$  can be obtained from this bound via a reduction, as shown in Lemma 10.
To understand our algorithm, we first describe the matrix mechanism for linear queries (cf. [ENU20]). That algorithm works as follows: factorize  $W = L^T R$ . Then, user  $i$  sends  $o_i^R = R\mathbf{1}_{x_i} + z_i^R$
where  $z_{i}^{R}$  is a appropriately selected random noise. In other words, each user privatizes  $W\mathbf{1}_{x_i}$  and sends it to the analyst. Finally, the analyst outputs  $L^T\left(\frac{1}{n}(o_1^R + \dots + o_n^R)\right)$ . This is exactly equal to  $Wh_{\mathbf{x}} + L^T Z^R$  where  $Z^R \coloneqq \frac{1}{n}(z_1^R + \dots + z_n^R)$ . It is possible to select the noise in such a way that  $Z^R$  is  $\left(\frac{\|R\|_{1 \to 2}}{\varepsilon \sqrt{n}}\right)$ -sub-Gaussian. This leads to an mMSE of  $O\left(\frac{\gamma_2(W)^2}{\varepsilon^2 n}\right)$ .
This suggests a natural approach for quadratic forms: in addition to sending (a privatized version of)  $R\mathbf{1}_{x_i}$ , the user sends a privatized version of  $L\mathbf{1}_{x_i}$ , i.e.,  $o_i^L = L\mathbf{1}_{x_i} + z_i^L$  where  $z_i^L$  is a random noise, to the analyst. The analyst then outputs  $\left\langle \frac{1}{n}(o_1^L + \dots + o_n^L), \frac{1}{n}(o_1^R + \dots + o_n^R) \right\rangle$ . Letting  $Z^L := \frac{1}{n}(z_1^L + \dots + z_n^L)$ , the output can be written as  $h_{\mathbf{x}}^T W h_{\mathbf{x}} + \langle L h_{\mathbf{x}}, Z^R \rangle + \langle R h_{\mathbf{x}}, Z^L \rangle + \langle Z^L, Z^R \rangle$ . There are three error terms:  $\langle L h_{\mathbf{x}}, Z^R \rangle$ ,  $\langle R h_{\mathbf{x}}, Z^L \rangle$ , and  $\langle Z^L, Z^R \rangle$ . Similar to linear queries analysis, it is not hard to see that the first two terms contribute  $O\left(\frac{\gamma_2(W)^2}{\varepsilon^2 n}\right)$  to the MSE. Unfortunately, the last term is problematic for us: a simple calculation shows that it contributes  $O\left(\frac{\ell}{\varepsilon^{4} n^2}\right)$  to the MSE, where  $\ell$  denotes the number of rows of  $L, R$ . A priori, this term can be quite large as there is no obvious bound on  $\ell$ . In fact, if  $W$  is full rank (which is the case for most popular pairwise statistics), then we know that  $\ell$  must be at least  $k$ . This leads to an undesired error term  $O\left(\frac{k}{\varepsilon^{4} n^2}\right)$ , which dominates the first term for small-to-moderate values of  $n$ , i.e., when  $n \leq \frac{k}{\varepsilon^{2} \gamma_{2}(W)^{2}}$ .
To overcome this, we observe that, if we only look for approximate factorization (in the same sense as  $\gamma_{2}(W;\alpha)$  defined above), then it is always possible to reduce  $\ell$  via dimensionality reduction techniques (e.g., [DG03]). Namely, we may pick a (e.g., random Gaussian) matrix  $A$  and replace  $L,R$  with  $AL,AR$  respectively. Selecting the number of rows of  $A$  appropriately then yields Theorem 5.
Additional Interactive Algorithms. To describe our algorithm, let us start by instantiating the above black-box two-round reduction using the matrix mechanism. In this context, the reduction yields the following algorithm:
$\triangleright$  First Round: Each user  $i$  sends  $o_i^L = L1_{x_i} + z_i^L$  to the analyst, where  $z_i^L$  is appropriately selected random noise. The analyst then broadcasts  $O^L = \frac{1}{n} (o_1^L +\dots +o_n^L)$  to each user.
$\triangleright$  Second Round: Each user  $j \in [n]$  sends  $o_j = \langle O^L, R\mathbf{1}_{x_j} \rangle + \kappa_j$  to the analyst, where  $\kappa_j$  is appropriately calibrated Laplace noise. The analyst then outputs  $\frac{1}{n} (o_1 + \dots + o_n)$ .
It is not hard to see that the  $\kappa_{j}$  noise terms together contribute at most  $O(1 / \varepsilon^{2}n)$  to the MSE. Therefore, the main noise comes from the first step. Similar to the previous discussion, this noise can be written as  $\langle Rh_{\mathbf{x}},Z^L\rangle$  where  $Z^{L}\coloneqq \frac{1}{n} (z_{1}^{L} + \dots +z_{n}^{L})$ . The contribution of this noise to the MSE is then  $O\left(\frac{\gamma_2(W)^2}{\varepsilon^2n}\right)$ . The  $\gamma_2^2 (W)$  term shows up in the error because  $\| Rh_{\mathbf{x}}\| _2$  can be as large as  $\| R\|_{1\to 2}$ . The idea motivating our improvement is simple: Can we replace the  $Rh_{\mathbf{x}}$  term with a term that is much smaller?
This brings us to the following strategy. We will use an additional round at the beginning to compute a rough estimate  $\mu$  of  $Rh_{\mathbf{x}}$ . Using the so-called projection mechanism [BBNS19], it is possible to compute  $\mu$  such that  $\| Rh_{\mathbf{x}} - \mu \| \leq \frac{(\gamma_2(W) \cdot (\log k) / \varepsilon)^{O(1)}}{n^{\Omega(1)}}$ . The subspace orthogonal to  $\mu$  can be processed in a similar manner as before, but now the error term will just be  $\langle Rh_{\mathbf{x}} - \mu, Z^L \rangle$ . Since  $\| Rh_{\mathbf{x}} - \mu \|$  is now much smaller (approaches 0 as  $n \to \infty$ ), this gives us the improved error bound. We note that the direction of  $\mu$  can be handled by having each user  $i$  directly send  $\langle o_i^L, \mu \rangle$  plus an appropriately calibrated noise. This summarizes the high-level idea of our approach.
Due to space constraints, we focus on the non-interactive algorithms in the main body and defer the proof of the interactive algorithm to the Appendix.
Differential Privacy. For  $\varepsilon \geq 0$ , an algorithm  $\mathcal{M}$  is  $\epsilon$ -DP if for every pair  $X, X'$  of inputs that differ on one user's input and for every possible output  $o$ ,  $\operatorname*{Pr}[\mathcal{M}(X) = o] \leq e^{\varepsilon} \cdot \operatorname*{Pr}[\mathcal{M}(X') = o]$ .
See also [NTZ16] for the original projection mechanism that was proposed for the central model.
An algorithm  $\mathbb{A}$  in the local DP model consists of a randomizer  $\mathcal{R}$  and an analyst, computed as  $\mathbb{A}(X) = \mathrm{analyst}(\mathcal{R}(x_1),\ldots ,\mathcal{R}(x_n))$  for input  $X = \{x_{1},\dots,x_{n}\}$ .  $\mathbb{A}$  is said to be (non-interactive)  $\varepsilon$ -local DP if  $\mathcal{R}$  is  $\varepsilon$ -DP.
A real-valued random variable  $Z$  is  $\sigma$ -sub-Gaussian iff  $\mathbb{E}[\exp (Z^2 /\sigma^2)]\leq 2$ . A  $\mathbb{R}^d$ -valued random variable  $Z$  is  $\sigma$ -sub-Gaussian iff  $\langle \theta ,Z\rangle$  is  $\sigma$ -sub-Gaussian for all unit vectors  $\theta \in \mathbb{R}^d$ .