SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Our main result is a unified framework to bound the information revealed about the unknown  $\theta$  by the transcript of the messages obtained via the constraints defined by the channel family  $\mathcal{W}$ . The framework is versatile and provides tight bounds for several families of continuous and discrete distributions, several families of information constraints such as communication and local differential privacy, and for the family of  $\ell_p$  loss functions for  $p \geq 1$ .
Our approach at a high-level proceeds as below: We first consider the "perturbation space"  $\mathcal{Z} := \{-1, +1\}^k$ , for some suitable  $k$ . We associate with each  $z \in \mathcal{Z}$  a parameter  $\theta_z \in \Theta$ , and refer to  $\mathbf{p}_{\theta_z}$  simply as  $\mathbf{p}_z$ . These distributions are designed in a way that the distance between  $\theta_z$  and  $\theta_{z'}$  is large when the Hamming distance between  $z$  and  $z'$  is large. With this, the difficulty of estimating  $\theta$  will be captured in the difficulty of estimating the associated  $z$ . This will make our approach compatible with the standard Assouad's method for deriving lower bounds (cf. [33]).
Then, we let  $Z = (Z_{1},\ldots ,Z_{k})$  be a random variable over  $\mathcal{Z}$ . Under some assumptions on the distribution of  $Z$ , we will bound the information between the individual  $Z_{i}$ s and the transcript  $(Y^{n},U)$  induced by a family of channels  $\mathcal{W}$ . Combining the two steps above provides us with the desired lower bounds. Formally, let  $\mathcal{Z}\coloneqq \{-1, + 1\} ^k$  for some  $k$  and  $\{\mathbf{p}_z\}_{z\in \mathcal{Z}}$  (where  $\mathbf{p}_z = \mathbf{p}_{\theta_z}$ ) be a collection of distributions over  $\mathcal{X}$ , indexed by  $z\in \mathcal{Z}$ . For  $z\in \mathcal{Z}$ , denote by  $z^{\oplus i}\in \mathcal{Z}$  the vector obtained by flipping the sign of the  $i$ th coordinate of  $z$ . To bound the information that can be obtained about the underlying  $z$  from the observations, we make the following assumptions:
Assumption 1 (Densities Exist). For every  $z \in \mathcal{Z}$  and  $i \in [k]$  it holds that  $\mathbf{p}_{z \oplus i} \ll \mathbf{p}_z$ , and there exist measurable functions  $\phi_{z,i} : \mathcal{X} \to \mathbb{R}$  such that  $\frac{\mathrm{d}\mathbf{p}_{z \oplus i}}{\mathrm{d}\mathbf{p}_z} = 1 + \phi_{z,i}$ .
The functions  $\phi_{z,i}$  capture the change in density when the coordinate  $i$  is flipped. In our applications below, we will have discrete distributions or continuous densities, and the Radon-Nikodym derivatives above can be replaced with the corresponding ratios between the pmfs and pdfs, respectively.
Assumption 2 (Orthogonality). There exists some  $\alpha^2 \geq 0$  such that, for all  $z \in \mathcal{Z}$  and distinct  $i,j \in [k]$ ,  $\mathbb{E}_{\mathbf{p}_z}[\phi_{z,i}\phi_{z,j}] = 0$  and  $\mathbb{E}_{\mathbf{p}_z}[\phi_{z,i}^2] \leq \alpha^2$ .
Note that from Assumption 1 we have that  $\mathbb{E}_{\mathbf{p}_z}[\phi_{z,i}] = 0$  for each  $i$ . In conjunction with Assumption 2 this implies that for any fixed  $z \in \mathcal{Z}$ , the family  $(1, \phi_{z,1}, \ldots, \phi_{z,k})$  is orthogonal and uniformly bounded in  $L^2(\mathcal{X}, \mathbf{p}_z)$ . Taken together, Assumption 1 and Assumption 2 roughly say that the densities can be decomposed into uncorrelated "perturbations" across coordinates of  $\mathcal{Z}$ . In later sections, we will show that for several families, such as discrete distributions, product Bernoulli distributions, and spherical Gaussians, well-known constructions for lower bounds satisfy these assumptions.
Our first bound given in (3) only requires Assumption 1; by imposing the additional structure of Assumption 2, we obtain the more specialized bound given in (4). Interestingly, (3) can be strengthened further when the following subgaussianity assumption holds.
<sup>3</sup>By Hölder's inequality, we also have  $\ell_{\infty}(u,v) \leq \ell_p(u,v) \leq d^{1/p} \ell_{\infty}(u,v)$  for all  $p \geq 1$  and  $u,v \in \mathbb{R}^d$ , which implies that for  $p := \log d$  we have  $\ell_{\infty}(u,v) \leq \ell_p(u,v) \leq 2 \ell_{\infty}(u,v)$ : i.e.,  $\ell_{\log d}(u,v)$  gives a factor-2 approximation of the  $\ell_{\infty}$  loss. This further extends to  $s$ -sparse vectors, with a factor  $s^{1/p}$  instead of  $d^{1/p}$ .
Assumption 3 (Subgaussianity). There exists some  $\sigma \geq 0$  such that, for all  $z\in \mathcal{Z}$ , the random vector  $\phi_z(X)\coloneqq (\phi_{z,i}(X))_{i\in [k]}\in \mathbb{R}^k$  is  $\sigma^2$ -subgaussian for  $X\sim \mathbf{p}_z$ .<sup>4</sup>
Let  $Z = (Z_{1},\ldots ,Z_{k})$  be a random variable over  $\mathcal{Z}$  such that  $\operatorname*{Pr}[Z_i = 1] = \tau$  for all  $i\in [k]$  and the  $Z_{i}$ 's are all independent; we denote this distribution by  $\mathrm{Rad}(\tau)^{\otimes k}$ . Our main result is an upper bound on the average amount of information that can be obtained about a coordinate of  $Z$  from the transcript  $(Y^{n},U)$  of a sequentially interactive protocol, as a function of the information constraint channels and  $\phi_{Z,i}$ 's. This result only requires Assumption 1, and is presented below in its most general form, suited to applications beyond those discussed in the current paper.
Theorem 1 (Information contraction bound: Technical form). Fix  $\tau \in (0,1 / 2]$ . Let  $\Pi$  be a sequentially interactive protocol using  $\mathcal{W}^n$ , and let  $Z$  be a random variable on  $\mathcal{Z}$  with distribution  $\mathrm{Rad}(\tau)^{\otimes k}$ . Let  $(Y^n,U)$  be the transcript of  $\Pi$  when the input  $X_{1},\ldots ,X_{n}$  is i.i.d. with common distribution  $\mathbf{p}_Z$ , with density function  $\mathbf{p}_Z^{Y^n}$ . Then, under Assumption 1,
$$
\left(\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \left(\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}}\right)\right) ^ {2} \leq \frac {7}{k} \sum_ {t = 1} ^ {n} \max  _ {z \in \mathcal {Z}} \max  _ {W \in \mathcal {W} _ {t}} \sum_ {i = 1} ^ {k} \int_ {\mathcal {Y}} \frac {\mathbb {E} _ {\mathbf {p} _ {z}} \left[ \phi_ {z , i} (X) W (y \mid X) \right] ^ {2}}{\mathbb {E} _ {\mathbf {p} _ {z}} [ W (y \mid X) ]} \mathrm {d} \mu , \tag {3}
$$
where  $\mathbf{p}_{+i}^{Y^n} := \mathbb{E}\left[\mathbf{p}_Z^{Y^n} \mid Z_i = +1\right]$ ,  $\mathbf{p}_{-i}^{Y^n} := \mathbb{E}\left[\mathbf{p}_Z^{Y^n} \mid Z_i = -1\right]$ .
We now instantiate this result, invoking Assumptions 2 and 3, to give simple "plug-and-play" bounds which can be applied readily to several inference problems and information constraints.
Theorem 2. Fix  $\tau \in (0,1 / 2]$ . Let  $\Pi$  be a sequentially interactive protocol using  $\mathcal{W}^n$ , and let  $Z$  be a random variable on  $\mathcal{Z}$  with distribution  $\mathrm{Rad}(\tau)^{\otimes k}$ . Let  $(Y^n,U)$  be the transcript of  $\Pi$  when the input  $X_{1},\ldots ,X_{n}$  is i.i.d. with common distribution  $\mathbf{p}_Z$ . Then, under Assumptions 1 and 2, we have
$$
\left(\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \left(\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}}\right)\right) ^ {2} \leq \frac {7}{k} \alpha^ {2} \sum_ {t = 1} ^ {n} \max  _ {z \in \mathcal {Z}} \max  _ {W \in \mathcal {W} _ {t}} \int_ {\mathcal {Y}} \frac {\operatorname {V a r} _ {\mathbf {p} _ {z}} [ W (y \mid X) ]}{\mathbb {E} _ {\mathbf {p} _ {z}} [ W (y \mid X) ]} \mathrm {d} \mu . \tag {4}
$$
Moreover, if Assumption 3 holds as well, we have
$$
\left(\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \left(\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}}\right)\right) ^ {2} \leq \frac {1 4}{k} \sigma^ {2} \sum_ {t = 1} ^ {n} \max  _ {z \in \mathcal {Z}} \max  _ {W \in \mathcal {W} _ {t}} I (\mathbf {p} _ {z}; W), \tag {5}
$$
where  $I(\mathbf{p}_z;W)$  denotes the mutual information  $I(X;Y)$  between the input  $X \sim \mathbf{p}_z$  and the output  $Y$  of the channel  $W$  with  $X$  as input.
As an illustrative and important corollary, we now derive the implications of this theorem for communication and privacy constraints. For both constraints our tight (or nearly tight) bounds in Section 5 follow directly from these corollaries.
Corollary 1 (Local privacy constraints). For  $\mathcal{W} = \mathcal{W}^{\mathrm{priv},\varepsilon}$  and any family of distributions  $\{\mathbf{p}_z,z\in$ $\{-1, + 1\} ^k\}$  satisfying Assumptions 1 and 2, with the notation of Theorem 2, we have
$$
\left(\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \left(\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}}\right)\right) ^ {2} \leq \frac {7}{k} n \alpha^ {2} \left((e ^ {\varepsilon} - 1) ^ {2} \wedge e ^ {\varepsilon}\right). \tag {6}
$$
$$
\left(\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \left(\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}}\right)\right) ^ {2} \leq \frac {1 4}{k} n \sigma^ {2} \varepsilon . \tag {7}
$$
Corollary 2 (Communication constraints). For any family of channels  $\mathcal{W}$  with finite output space  $\mathcal{V}$  and any family of distributions  $\{\mathbf{p}_z, z \in \{-1, +1\}^k\}$  satisfying Assumptions 1 and 2, with the notation of Theorem 2, we have
$$
\left(\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \left(\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}}\right)\right) ^ {2} \leq \frac {7}{k} n \alpha^ {2} | \mathcal {Y} |. \tag {8}
$$
<sup>4</sup>Recall that a r.v.  $Y$  is  $\sigma^2$ -subgaussian if  $\mathbb{E}[Y] = 0$  and  $\mathbb{E}[e^{\lambda Y}] \leq e^{\sigma^2 \lambda^2 / 2}$  for all  $\lambda \in \mathbb{R}$ ; and that a vector-valued r.v.  $Y$  is  $\sigma^2$ -subgaussian if its projection  $\langle Y, v \rangle$  is  $\sigma^2$ -subgaussian for every unit vector  $v$ .
$$
\left(\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \left(\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}}\right)\right) ^ {2} \leq \frac {1 4}{k} n \sigma^ {2} \log | \mathcal {Y} |. \tag {9}
$$
4 An Assouad-type bound
In the previous section we provided an upper bound on  $\frac{1}{k}\sum_{i = 1}^{k}\mathrm{d}_{\mathrm{TV}}\bigl (\mathbf{p}_{+i}^{Y^n},\mathbf{p}_{-i}^{Y^n}\bigr)$ . We now prove a lower bound for this quantity in terms of the parameter estimation task we set out to solve. This is an "Assouad's lemma-type" bound, which when combined with Theorem 2 will establish the bounds for  $n$ ; and, reorganizing, the minimax rate lower bounds. To state the result, we require the following assumption, which relates the  $\ell_p$  distance between parameters  $\theta_z$ s to the distance between  $zs$ .
Assumption 4 (Additive loss). Fix  $p \in [1, \infty)$ . For every  $z, z' \in \mathcal{Z} \subset \{-1, +1\}^k$ ,
$$
\ell_ {p} (\theta_ {z}, \theta_ {z ^ {\prime}}) = 4 \gamma \left(\frac {\mathrm {d} _ {\mathrm {H a m}} (z , z ^ {\prime})}{\tau k}\right) ^ {1 / p},
$$
where  $\mathrm{d}_{\mathrm{Ham}}(z,z') \coloneqq \sum_{i=1}^{k} \mathbb{1}\{z_i \neq z_i'\}$  denotes the Hamming distance.
Lemma 1 (Assouad-type bound). Let  $p \geq 1$  and assume that  $\{\mathbf{p}_z, z \in \mathcal{Z}\}$ ,  $\tau \in [0,1/2]$  satisfy Assumption 4. Let  $Z$  be a random variable on  $\mathcal{Z} = \{-1, +1\}^k$  with distribution  $\mathrm{Rad}(\tau)^{\otimes k}$ . Suppose that  $(\Pi, \hat{\theta})$  constitutes an  $(n,\gamma)$ -estimator of  $\mathcal{P}_{\Theta}$  using  $\mathcal{W}^n$  under  $\ell_p$  loss (see Definition 2) and  $\operatorname{Pr}_Z[\mathbf{p}_Z \in \mathcal{P}_{\Theta}] \geq 1 - \tau/4$ . Then the transcript  $(Y^n, U)$  of  $\Pi$  satisfies
$$
\frac {1}{k} \sum_ {i = 1} ^ {k} \mathrm {d} _ {\mathrm {T V}} \Big (\mathbf {p} _ {+ i} ^ {Y ^ {n}}, \mathbf {p} _ {- i} ^ {Y ^ {n}} \Big) \geq \frac {1}{4},
$$
We now consider three distribution families: product Bernoulli distributions and Gaussian distributions with identity covariance matrix (and  $s$ -sparse mean vectors), and discrete distributions (multinomials), to illustrate the generality and efficacy of our bounds. We describe these three families below, before addressing each of them in their respective subsection. Due to space constraints, we present the proof of the upper bounds in Appendix C, and proof of the lower bounds in Appendix G.
Sparse Product Bernoulli  $(\mathcal{B}_{d,s})$ . Let  $1 \leq s \leq d$ ,  $\Theta = \{\theta \in [-1,1]^d : \| \theta \|_0 \leq s\}$ , and  $\mathcal{X} = \{-1,1\}^d$ . Let  $\mathcal{P}_{\Theta} := \mathcal{B}_{d,s}$  be the family of  $d$ -dimensional  $s$ -sparse product Bernoulli distributions over  $\mathcal{X}$ . Namely, for  $\theta = (\theta_1, \ldots, \theta_d) \in \Theta$ , the distribution  $\mathbf{p}_{\theta}$  is equal to  $\otimes_{j=1}^{d} \operatorname{Rad}(\frac{1}{2} (\theta_j + 1))$ : a distribution on  $\{-1, +1\}^d$  such that the marginal distributions are independent, and for which the mean of the  $j$ th marginal is  $\theta_j$ .
Sparse Gaussian  $(\mathcal{G}_{d,s})$ . Let  $1 \leq s \leq d$ ,  $\Theta = \{\theta \in [-1,1]^d : \| \theta \|_0 \leq s\}$ , and  $\mathcal{X} = \mathbb{R}^d$ . Let  $\mathcal{P}_{\Theta} := \mathcal{G}_{d,s}$  be the family of  $d$ -dimensional spherical Gaussian distributions with bounded  $s$ -sparse mean. That is, for  $\theta \in \Theta$ ,  $\mathbf{p}_{\theta} = \mathcal{G}(\theta, \mathbb{I})$  with mean  $\theta$  and covariance matrix  $\mathbb{I}$ . We note that this general formulation assumes  $\| \theta \|_{\infty} \leq 1$  (from the choice of  $\Theta$ ).<sup>5</sup>
Discrete distributions  $(\Delta_d)$ . Let  $\Theta = \left\{\theta \in [0,1]^d : \sum_{i=1}^d \theta_i = 1\right\} \subseteq \mathbb{R}^d$  and  $\mathcal{X} = \{1,2,\ldots,d\}$ . Let  $\mathcal{P}_{\Theta} := \Delta_d$ , where  $\Delta_d$  is the standard  $(d-1)$ -simplex of all probability mass functions over  $\mathcal{X}$ . Namely, the distribution  $\mathbf{p}_{\theta}$  is a distribution on  $\mathcal{X}$ , where, for  $j \in [d]$ , the probability assigned to the element  $j$  is  $\mathbf{p}_{\theta}(j) = \theta_j$ . For a unified presentation, we view  $\theta$  as the mean vector of the "categorical distribution," namely the distribution of vector  $(\mathbb{1}\{X = x\}, x \in \mathcal{X})$  for  $X$  with distribution  $\mathbf{p}_{\theta}$ .
This assumption that the mean is bounded, in our case in  $\ell_{\infty}(\mathbf{0},1)$ , is standard, and necessary in order to obtain finite upper bounds: indeed, a packing argument shows that if the mean is assumed to be in a ball of radius  $R$ , then a  $\log^{\Omega(1)} R$  dependence in the sample complexity is necessary in both the communication-constrained and LDP settings. Our choice of radius 1 is arbitrary, and our upper bounds can be generalized to any  $R \geq 1$ .
We now define our measure of interest, the minimax error rate of mean estimation.
Definition 3 (Minimax rate of mean estimation). Let  $\mathcal{P}$  be a family of distributions parameterized by  $\Theta \subseteq \mathbb{R}^d$ . For  $p \in [1, \infty]$ ,  $n \in \mathbb{N}$ , and a family of channels  $\mathcal{W}$ , the minimax error rate of mean estimation for  $\mathcal{P}$  using  $\mathcal{W}^n$  under  $\ell_p$  loss, denoted  $\mathcal{E}_p(\mathcal{P}, \mathcal{W}, n)$ , is the least  $\gamma \in (0, 1]$  such that there exists an  $(n, \gamma)$ -estimator for  $\mathcal{P}$  using  $\mathcal{W}$  under  $\ell_p$  loss (see Definition 2).
We obtain lower bounds on the minimax rate of mean estimation for the different families above by specializing our general bound. Importantly, our methodology is not specific to  $\ell_p$  losses, and can be used for arbitrary additive losses such as (squared) Hellinger or, indeed, for any loss function for which an analogue of Lemma 1 can be derived.
Product Bernoulli family. We first establish the following bounds for  $\mathcal{B}_{d,s}$  under privacy and communication constraints.
Theorem 3 (Product Bernoulli). Fix  $p \in [1, \infty)$ . For  $4\log d \leq s \leq d$ ,  $\varepsilon \in (0, \infty)$ , and  $\ell \geq 1$ ,
$$
\sqrt {\frac {d s ^ {2 / p}}{n \left(\varepsilon^ {2} \wedge \varepsilon\right)}} \wedge 1 \lesssim \mathcal {E} _ {p} \left(\mathcal {B} _ {d, s}, \mathcal {W} ^ {\text {p r i v}, \varepsilon}, n\right) \lesssim \sqrt {\frac {d s ^ {2 / p}}{n \left(\varepsilon^ {2} \wedge 1\right)}} \tag {10}
$$
$$
\sqrt {\frac {d s ^ {2 / p}}{n \ell} \vee \frac {s ^ {2 / p} \log \frac {2 d}{s}}{n}} \wedge 1 \lesssim \mathcal {E} _ {p} \left(\mathcal {B} _ {d, s}, \mathcal {W} ^ {\operatorname {c o m m}, \ell}, n\right) \lesssim \sqrt {\frac {d s ^ {2 / p}}{n \ell} \vee \frac {s ^ {2 / p} \log \frac {2 d}{s}}{n}} \tag {11}
$$
For  $p = \infty$ , we have the upper bounds
$$
\mathcal {E} _ {\infty} (\mathcal {B} _ {d, s}, \mathcal {W} ^ {\mathrm {p r i v}, \varepsilon}, n) = O \left(\sqrt {\frac {d \log s}{n \varepsilon^ {2}}}\right) \quad a n d \quad \mathcal {E} _ {\infty} (\mathcal {B} _ {d, s}, \mathcal {W} ^ {\mathrm {c o m m}, \ell}, n) = O \left(\sqrt {\frac {d \log s}{n \ell} \vee \frac {\log d}{n}}\right),
$$
while the lower bounds given in Eqs. (10) and (11) hold for  $p = \infty$ , too. $^6$
Remark 1. Previous work had shown, in the simpler noninteractive model, a rate lower bound scaling as  $\sqrt{ds / (n\ell)\log(2d / s)}$  for the specific case of  $\ell_2$  loss (see, for instance, [21, Theorem 7] for the sparse Gaussian case, which implies the Bernoulli one). An analogous phenomenon was observed for local privacy (e.g., [14]). Thus, by removing this logarithmic factor from the upper bound, our result establishes the first (to the best of our knowledge) separation between interactive and noninteractive protocols for sparse mean estimation under communication or local privacy constraints.
Remark 2. Although we stated for simplicity the lower bounds of Theorem 3 in the case where all  $n$  players have a similar local constraints (i.e., same privacy parameter  $\varepsilon$ , or same bandwidth constraint  $\ell$ ), it is immediate to check from the application of Theorem 2 that the result extends to different constraints for each player; replacing  $n(\varepsilon^2 \wedge \varepsilon)$  and  $n\ell$  in the statement by  $\sum_{t=1}^{n} \varepsilon_t^2 \wedge \varepsilon_t$  and  $\sum_{t=1}^{n} \ell_t$ , respectively. A similar remark applies to the Gaussian and discrete families.
Gaussian family. We derive a lower bound for  $\mathcal{E}_p(\mathcal{G}_{d,s},\mathcal{W},n)$  under local privacy (captured by  $\mathcal{W} = \mathcal{W}^{\mathrm{priv},\varepsilon}$ ) and communication (captured by  $\mathcal{W} = \mathcal{W}^{\mathrm{comm},\ell}$ ) constraints. Recall that for product Bernoulli mean estimation we had optimal bounds for both privacy and communication constraints for all finite  $p$ . For Gaussians, we will obtain tight bounds for privacy constraints for  $\varepsilon \in (0,1]$ . However, for communication constraints and privacy constraints when  $\varepsilon \geq 1$ , our bounds for Gaussian distributions lose a (single) logarithmic factor in some parameter regimes.
Theorem 4 (Gaussian distributions). Fix  $p \in [1, \infty)$ . For  $4\log d \leq s \leq d$ , under LDP constraints, when  $\varepsilon \in (0, 1]$ ,
$$
\sqrt {\frac {d s ^ {2 / p}}{n \varepsilon^ {2}}} \wedge 1 \lesssim \mathcal {E} _ {p} \left(\mathcal {G} _ {d, s}, \mathcal {W} ^ {\text {p r i v}, \varepsilon}, n\right) \lesssim \sqrt {\frac {d s ^ {2 / p}}{n \varepsilon^ {2}}} \tag {12}
$$
and when  $\varepsilon > 1$ ,
$$
\sqrt {\frac {d s ^ {2 / p}}{n \varepsilon \log (n d)}} \wedge 1 \lesssim \mathcal {E} _ {p} \left(\mathcal {G} _ {d, s}, \mathcal {W} ^ {\text {p r i v}, \varepsilon}, n\right) \lesssim \sqrt {\frac {d s ^ {2 / p}}{n}} \tag {13}
$$
That is, the upper and lower bounds only differ by a log  $s$  factor for  $p = \infty$ .
<sup>7</sup>As in the Bernoulli case, we here focus for simplicity on the case where the communication (resp., privacy) parameters are the same for all players, but our lower bounds easily extend.
Under communication constraints,
$$
\sqrt {\frac {d s ^ {2 / p}}{n \ell \log (d n)} \vee \frac {s ^ {2 / p} \log \frac {2 d}{s}}{n}} \wedge 1 \lesssim \mathcal {E} _ {p} \left(\mathcal {G} _ {d, s}, \mathcal {W} ^ {\text {c o m m}, \ell}, n\right) \lesssim \sqrt {\frac {d s ^ {2 / p}}{n \ell} \vee \frac {s ^ {2 / p} \log \frac {2 d}{s}}{n}} \tag {14}
$$
The same bound as in Theorem 3 hold for  $p = \infty$ .
We emphasize that, as discussed in Sections 1.1 and 1.2, to the best of our knowledge our results provide the first lower bounds for interactive Gaussian mean estimation under these constraints.
Discrete distribution estimation. We are able to derive a lower bound for  $\mathcal{E}_p(\Delta_d, \mathcal{W}, n)$ , the minimax rate for discrete density estimation, under local privacy and communication constraints. In the interest of space, we focus here on two important corollaries; first, for the case of total variation distance  $(\ell_1)$ , where combining it with known upper bounds we obtain optimal bounds for all  $\varepsilon > 0$ . In particular, for  $\varepsilon \in (0, 1]$  (high-privacy regime) we retrieve the lower bound established in [5], which matches the upper bound from [4]. For  $\varepsilon > 1$  (low-privacy regime), our bound matches the upper bound for the noninteractive case, established in [32, 1], showing that even in this low-privacy regime interactivity cannot lead to better rates, except maybe up to constant factors.
Corollary 3 (Total variation distance). For  $\varepsilon >0$ , we have
$$
\mathcal {E} _ {1} \left(\Delta_ {d}, \mathcal {W} ^ {\text {p r i v}, \varepsilon}, n\right) \asymp \sqrt {\frac {d ^ {2}}{n \left((e ^ {\varepsilon} - 1) ^ {2} \wedge e ^ {\varepsilon}\right)}} \wedge 1. \tag {15}
$$
For  $\ell \geq 1$
$$
\mathcal {E} _ {1} \left(\Delta_ {d}, \mathcal {W} ^ {\operatorname {c o m m}, \ell}, n\right) \asymp \sqrt {\frac {d ^ {2}}{n \left(2 ^ {\ell} \wedge d\right)}} \wedge 1. \tag {16}
$$
For the case of  $\ell_2$  estimation, we also obtain order-optimal bounds:
Corollary 4 ( $\ell_2$  density estimation). For  $\varepsilon > 0$ , we have
$$
\mathcal {E} _ {2} \left(\Delta_ {d}, \mathcal {W} ^ {\text {p r i v}, \varepsilon}, n\right) \asymp \sqrt {\frac {d}{n \left(e ^ {\varepsilon} - 1\right) ^ {2} \wedge e ^ {\varepsilon}}} \wedge \sqrt [ 4 ]{\frac {1}{n \left(e ^ {\varepsilon} - 1\right) ^ {2} \wedge e ^ {\varepsilon}}} \wedge 1. \tag {17}
$$
$$
\mathcal {E} _ {2} \left(\Delta_ {d}, \mathcal {W} ^ {\operatorname {c o m m}, \ell}, n\right) \asymp \sqrt {\frac {d}{n \left(2 ^ {\ell} \wedge d\right)}} \wedge \sqrt [ 4 ]{\frac {1}{n \left(2 ^ {\ell} \wedge d\right)}} \wedge 1. \tag {18}
$$
The authors would like to thank Yanjun Han, and the anonymous reviewers for helpful comments on an earlier version of this paper.
[1] Jayadev Acharya, Ziteng Sun, and Huanyu Zhang. Hadamard response: Estimating distributions privately, efficiently, and with little communication. In Kamalika Chaudhuri and Masashi Sugiyama, editors, Proceedings of Machine Learning Research, volume 89 of Proceedings of Machine Learning Research, pages 1120-1129. PMLR, 16-18 Apr 2019. URL http://proceedings.mlr.press/v89/acharya19a.html.
[2] Jayadev Acharya, Clément L. Canonne, and Himanshu Tyagi. Distributed signal detection under communication constraints. volume 125 of Proceedings of Machine Learning Research, pages 41-63. PMLR, July 2020. URL http://proceedings.mlr.press/v125/acharya20b.html.
[3] Jayadev Acharya, Clément L. Canonne, and Himanshu Tyagi. Inference under information constraints I: Lower bounds from chi-square contraction. IEEE Trans. Inform. Theory, 66 (12):7835-7855, 2020. ISSN 0018-9448. doi: 10.1109/TIT.2020.3028440. URL https://doi.org/10.1109/TIT.2020.3028440. Preprint available at arXiv:abs/1812.11476.
A Fully interactive model
In this appendix, we describe how to extend our results, presented in the sequentially interactive model, to the more general interactive setting. We first formally define this setting and the corresponding notion of protocols. Hereafter, we use \* for the Kleene star operation, i.e.,  $V^{*} = \bigcup_{n = 0}^{\infty}V^{n}$