SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[AD20a] Hilal Asi and John C Duchi. "Instance-optimality in differential privacy via approximate inverse sensitivity mechanisms". In: Advances in neural information processing systems 33 (2020), pp. 14106–14117.
[AD20b] Hilal Asi and John C Duchi. "Near instance-optimality in differential privacy". In: arXiv preprint arXiv:2005.10630 (2020).
[AMS99] Noga Alon, Yossi Matias, and Mario Szegedy. "The space complexity of approximating the frequency moments". In: J. Comput. Syst. Sci 58.1 (1999), pp. 137-147.
[BCL13] Sébastien Bubeck, Nicolo Cesa-Bianchi, and Gábor Lugosi. "Bandits with heavy tail". In: IEEE Trans. Inf. Theory 59.11 (2013), pp. 7711-7717.
[Cat12] Olivier Catoni. "Challenging the empirical mean and empirical variance: a deviation study". In: Ann. I. H. Poincaré -PR 48.4 (2012), pp. 1148-1185.
[DLLO16] Luc Devroye, Matthieu Lerasle, Gábor Lugosi, and Roberto I. Oliveira. "Sub-Gaussian mean estimators". In: Ann. Stat 44.6 (2016), pp. 2695-2725.
[FLN01] Ronald Fagin, Amnon Lotem, and Moni Naor. "Optimal aggregation algorithms for middleware". In: Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems. 2001, pp. 102-113.
[GLP23] Shivam Gupta, Jasper C.H. Lee, and Eric Price. "Finite-Sample Symmetric Mean Estimation with Fisher Information Rate". In: To appear in Proc. COLT'23. 2023.
[HLY21] Ziyue Huang, Yuting Liang, and Ke Yi. "Instance-optimal Mean Estimation Under Differential Privacy". In: Advances in Neural Information Processing Systems 34 (2021), pp. 25993-26004.
[JVV86] Mark R. Jerrum, Leslie G. Valiant, and Vijay V. Vazirani. “Random generation of combinatorial structures from a uniform distribution”. In: Theor. Comput. Sci 43 (1986), pp. 169–188.
[Kee10] R.W. Keener. Theoretical Statistics: Topics for a Core Course. Springer Texts in Statistics. Springer New York, 2010.
[LM19] Gabor Lugosi and Shahar Mendelson. Mean estimation and regression under heavy-tailed distributions—a survey. 2019.
[LV22] Jasper C.H. Lee and Paul Valiant. "Optimal Sub-Gaussian Mean Estimation in  $\mathbb{R}$ ". In: 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS). IEEE. 2022, pp. 672-683.
[Min19] Stanislav Minsker. "Distributed statistical estimation and rates of convergence in normal approximation". In: *Electron. J. Statist.* 13.2 (2019), pp. 5213-5252.
[NY83] A.S. Nemirovsky and D.B. Yudin. Problem Complexity and Method Efficiency in Optimization. Wiley, 1983.
[Rou21] Tim Roughgarden. Beyond the worst-case analysis of algorithms. Cambridge University Press, 2021.
[Sta97] Ross M. Starr. General Equilibrium Theory: An Introduction. 1997.
[VV16] Gregory Valiant and Paul Valiant. "Instance optimal learning of discrete distributions". In: Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, 2016, pp. 142-155.
[VV17] Gregory Valiant and Paul Valiant. "An automatic inequality prover and instance optimal identity testing". In: SIAM Journal on Computing 46.1 (2017), pp. 429-455.
A Interpreting Neighborhood Optimality
In this section, we derive basic properties and intuitions about the definitions of neighborhood Pareto bounds and neighborhood optimality.
In particular, we show that neighborhood optimality is a notion that smoothly interpolates between instance optimality and admissibility, depending on what neighborhood function is used to instantiate the definition. We also show a sufficient condition for proving that an error function  $\epsilon$  is a neighborhood Pareto bound (Proposition 13). Lastly, we discuss and compare neighborhood optimality and the notion of local minimax optimality that has appeared in the literature.
A.1 As an interpolation between instance optimality and admissibility
Recall that Definitions 11 and 12 both require specifying the neighborhood function. Our first observation is the monotonicity in the definition of neighborhood Pareto bounds, in the sense of the following straightforward proposition which we state without proof.
Proposition 18 Suppose the neighborhood functions  $N_{n,\delta}$  and  $N_{n,\delta}^{\prime}$  are such that  $N_{n,\delta}(p)\subseteq$ $N_{n,\delta}^{\prime}(p)$  for all  $p$ . Then, if a given error function  $\epsilon_{n,\delta}$  is a neighborhood Pareto bound with respect to  $N_{n,\delta}$ , then  $\epsilon_{n,\delta}$  is also a neighborhood Pareto bound with respect to  $N_{n,\delta}^{\prime}$ .
We can extend this monotonicity observation from neighborhood Pareto bounds to neighborhood optimal estimators.
Proposition 19 Suppose the neighborhood functions  $N_{n,\delta}$  and  $N_{n,\delta}'$  are such that  $N_{n,\delta}(p) \subseteq N_{n,\delta}'(p)$  for all  $p$ . Then, if  $\hat{\mu}$  is a  $\kappa$ -neighborhood optimal estimator with respect to  $N_{n,\delta}$ , then  $\hat{\mu}$  is also  $\kappa$ -neighborhood optimal with respect to  $N_{n,\delta}'$ .
To understand how the set of neighborhood optimal estimators vary as we change the neighborhood function, we examine the two extreme examples as special cases, where  $N_{n,\delta} \equiv \emptyset$  and  $N_{n,\delta} \equiv \mathcal{P}_1$ .
Let us first consider the case when the neighborhoods are all empty. In this case, Definition 11 simplifies to requiring a neighborhood Pareto bound  $\epsilon_{n,\delta}$  to be such that, for every distribution  $p\in \mathcal{P}_1$ , no estimator  $\hat{\mu}$  (specialized to  $p$ ) can have error strictly less than  $\epsilon_{n,\delta}(p)$  with probability  $1 - \delta$  over  $n$  samples from  $p$ . It is straightforward to check that any estimator that is  $\kappa$ -neighborhood optimal with respect to empty neighborhoods is equivalent to being instance optimal (Definition 27 in Appendix D) up to a  $\kappa$  factor, which as explained before, is impossible to achieve. The only possible neighborhood Pareto bound with respect to empty neighborhoods is  $\epsilon_{n,\delta}\equiv 0$ , since every distribution has a trivial estimator that outputs its mean hardcoded.
At the other extreme, consider the case where all the neighborhoods contain all the distributions in  $\mathcal{P}_1$ . Here, Definition 11 simplifies (after taking a contrapositive) to requiring a neighborhood Pareto bound  $\epsilon$  to be such that, for every distribution  $p \in \mathcal{P}_1$ , if an estimator has error at most  $\epsilon_{n,\delta}(q)$  with probability  $1 - \delta$  for all distributions  $q \neq p \in \mathcal{P}_1$ , then it must have error at least  $\epsilon_{n,\delta}(p)$  with probability  $1 - \delta$  for distribution  $p$ . It is again straightforward to check that, by definition, an estimator is 1-neighborhood optimal with respect to the constant- $\mathcal{P}_1$  neighborhood function if and only if the estimator is admissible in  $\mathcal{P}_1$  (Definition 28 in Appendix D). As explained before, admissibility is a somewhat weak notion on estimators, and includes trivial hardcoded estimators.
Combining these two extreme cases and the monotonicity propositions, we have shown that neighborhood optimality is a notion which interpolates between instance optimality and admissibility. More technically, neighborhood optimality can be viewed as a homomorphism mapping the partial ordering of neighborhood functions (the ordering induced by set inclusion) to the partial ordering of sets of estimators (also ordered by set inclusion).
We remark that, while it may be tempting to view neighborhood Pareto bounds as lower bounds, and neighborhood optimality as exhibiting an estimator whose error function matches a lower bound up to a constant factor, such an interpretation is actually not valid. The reason is that, depending on the choice of the neighborhood function, given an estimator with error function  $\epsilon_{n,\delta}^{\mathrm{upper}}$  and a neighborhood Pareto bound  $\epsilon_{n,\delta}^{\mathrm{pareto}}$ , it can be the case that on some distributions  $p \in \mathcal{P}_1$ , we have  $\epsilon_{n,\delta}^{\mathrm{pareto}}(p) > \epsilon_{n,\delta}^{\mathrm{upper}}(p)$ ; namely, these Pareto bounds should not be viewed as lower bounds. A simple
example is the previous case where the neighborhood function is  $\mathcal{P}_1$ , in which case it is straightforward to check that for any (hardcoded) constant  $\hat{\mu} \in \mathbb{R}$ , the error function  $\epsilon_{n,\delta}^{\mathrm{pareto}}(p) = |\hat{\mu} - \mu_p|$  is a neighborhood Pareto bound. For any distribution  $p$  whose mean is extremely far from the constant  $\hat{\mu}$ , the median-of-means algorithm will have accuracy better than  $|\hat{\mu} - \mu_p|$ . Because of such examples, we take care not to refer to our bounds as lower bounds, but instead call them Pareto bounds in this paper.
A.2 Indistinguishability implies a neighborhood Pareto bound
We restate Proposition 13 from Section 3.2 here for clarity, and give the proof of it below.
Proof. Suppose the "if" condition in the proposition is true, yet, for the sake of contradiction, that  $\epsilon_{n,\delta}$  is not a neighborhood Pareto bound. Then, there exists a distribution  $p\in \mathcal{P}_1$  and an estimator  $\hat{\mu}$  taking  $n$  i.i.d. samples such that
- For all distributions  $q \in N_{n,\delta}(p)$ , with probability  $1 - \delta$  over the  $n$  i.i.d. samples from  $q$ ,  $|\hat{\mu} - \mu_q| \leq \epsilon_{n,\delta}(q)$ .
- With probability  $1 - \delta$  over the  $n$  i.i.d. samples from  $p$ ,  $|\hat{\mu} - \mu_p| < \epsilon_{n,\delta}(p)$ .
By the proposition condition, there must exist some distribution  $q \in N_{n,\delta}(p)$  such that  $|\mu_p - \mu_q| \geq \epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q)$  and it is information-theoretically impossible to distinguish  $p$  and  $q$  with probability  $1 - \delta$  using  $n$  samples. However, we can construct the following distinguisher: compute a mean estimate  $\hat{\mu}$ , return  $p$  if  $\hat{\mu}$  is within  $\epsilon_{n,\delta}(p)$  of  $\mu_p$  and return  $q$  otherwise. By our assumption on  $\hat{\mu}$ , this distinguisher will succeed with probability at least  $1 - \delta$ , thus contradicting the proposition statement.
We remark that the proof of Proposition 13 actually establishes a stronger result, that the error function  $\epsilon_{n,\delta}$  is a neighborhood Pareto bound for the singleton neighborhood  $N_{n,\delta}^{*}(p) = \{q(p)\}$  where  $q(p)$  is the  $q$  constructed from  $p$  according to Proposition 13. Recall that the monotonicity property of Proposition 18 says that neighborhood Pareto bounds are stronger for smaller neighborhoods; thus this bound for singleton neighborhoods implies the corresponding bound for any larger neighborhoods  $N_{n,\delta} \supset N_{n,\delta}^{*}$ .
A.3 Comparing with local minimax optimality
Here, we compare our definition of neighborhood optimality with the notion of local minimax optimality from prior literature. At a high level, neighborhood optimality imposes admissibility within each local neighborhood, whereas local minimax optimality imposes minimax optimality within each local neighborhood. We argue that local minimax is too sensitive to the choice of neighborhood structure, although the two definitions are different in a rather subtle way and perhaps difficult to see at first glance. We concretely illustrate their differences via 1) a proposition showing that for practical purposes, local minimax bounds are a weaker notion than neighborhood Pareto bounds and 2) a simple example choice of an "inappropriate" neighborhood structure, such that an intuitively absurd local minimax bound holds for this neighborhood structure, but the same bound fails to satisfy the definition of a neighborhood Pareto bound. Together, these results show that neighborhood optimality is a stronger and more robust notion than local minimax optimality.
For ease of comparison, we first phrase the local minimax definition in the same form as our definition of neighborhood optimality.
Definition 20 (Local minimax bounds with respect to  $N_{n,\delta}$ ) Let  $n$  be the number of samples and  $\delta$  be the failure probability. Given a neighborhood function  $N_{n,\delta}:\mathcal{P}_1\to 2^{\mathcal{P}_1}$ , we say that the error function  $\epsilon_{n,\delta}(p):\mathcal{P}_1\to \mathbb{R}_0^+$  is a local minimax bound for  $\mathcal{P}_1$  with respect to  $N_{n,\delta}$  if for all
distributions  $p \in \mathcal{P}_1$ , there is no estimator  $\hat{\mu}$  such that for all distributions  $q \in N_{n,\delta}(p) \cup \{p\}$ , when given  $n$  i.i.d. samples from  $q$ , the estimator  $\hat{\mu}$  achieves  $|\hat{\mu} - \mu_q| < \epsilon_{n,\delta}(p)$  with probability  $1 - \delta$ .
Definition 21 ( $\kappa$ -locally minimax estimators) For a parameter  $\kappa > 1$ , sample complexity  $n$ , failure probability  $\delta$  and neighborhood function  $N_{n,\delta}$ , a mean estimator  $\hat{\mu}$  is  $\kappa$ -locally minimax with respect to  $N_{n,\delta}$  if there exists an error function  $\epsilon_{n,\delta}(p)$  such that  $\epsilon_{n,\delta}(p)$  is a local minimax bound, and  $\hat{\mu}$  gives estimation error at most  $\kappa \cdot \epsilon_{n,\delta}(p)$  with probability at least  $1 - \delta$  when taking  $n$  i.i.d. samples from any distribution  $p \in \mathcal{P}$ .
The notions of local minimax bounds and locally minimax estimators also have the same monotonicity properties as neighborhood Pareto bounds and neighborhood optimal estimators, analogous to Propositions 18 and 19. When the neighborhood function  $N_{n,\delta}$  is constantly equal to the empty set, local minimax optimality again coincides with instance optimality. Furthermore, local minimax bounds can be proved using indistinguishability arguments, via a reduction analogous to Proposition 13 up to minor changes in parameters.
However, one key difference between the two styles of definition arises when we consider proving neighborhood Pareto bounds or local minimax bounds using such indistinguishability arguments. Recall from Section 3.2 that, by relying on Proposition 13, the main technical result of this paper is to show that for every distribution  $p$ , there exists a "neighbor"  $q(p)$  whose mean is far from  $p$ , with  $q(p)$  satisfying suitable structural properties (so that  $q(p) \in N_{n,\delta}(p)$  in our eventual choice of  $N_{n,\delta}$  in Definition 15), such that  $p$  and  $q$  cannot be distinguished with probability  $1 - \delta$  using  $n$  samples. As explained in Section 3.2, this proof strategy effectively proves a neighborhood Pareto bound over the singleton neighborhood function  $N_{n,\delta}^{*}(p) = \{q(p)\}$ . Thus, it is meaningful to compare neighborhood Pareto bounds and local minimax bounds when the neighborhoods are singletons. The following proposition shows that, up to a constant factor of 2 in the error, a neighborhood Pareto bound on singleton neighborhoods implies a potentially much larger local minimax bound.
Proposition 22 (For singleton neighborhoods, local minimax bounds are weaker than neighborhood Pareto bounds) Let  $n$  be the number of samples and  $\delta$  be the failure probability. Consider a neighborhood function  $N_{n,\delta}^{*}$  such that for all  $p \in \mathcal{P}_1$ ,  $N_{n,\delta}^{*}(p)$  is a singleton set containing a distribution  $q(p) \neq p$ . Suppose  $\epsilon_{n,\delta}$  is a neighborhood Pareto bound with respect to  $N_{n,\delta}^{*}$ , and that  $\epsilon_{n,\delta}(p) > 0$  for all  $p \in \mathcal{P}_1$ . Then, the function  $\frac{1}{2} (\epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q(p))$  is a local minimax bound with respect to  $N_{n,\delta}^{*}$ . Note that the above function is lower bounded by  $\Omega(\max(\epsilon_{n,\delta}(p), \epsilon_{n,\delta}(q(p)))$  and can be much larger than  $\epsilon_{n,\delta}(p)$ .
Proof. Suppose for the sake of contradiction that  $\frac{1}{2} (\epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q(p))$  is not a local minimax bound with respect to  $N_{n,\delta}^{*}$  but  $\epsilon_{n,\delta}$  is a neighborhood Pareto bound.
We observe that, since  $\epsilon_{n,\delta}$  is a neighborhood Pareto bound, it must be the case that for every distribution  $p\in \mathcal{P}_1$ , we have  $|\mu_p - \mu_q|\geq \epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q(p))$ . Otherwise, for any distribution  $p$  not satisfying the above, there is a trivial hardcoded estimator that outputs a number  $\hat{\mu}$  such that  $|\hat{\mu} -\mu_q| < \epsilon_{n,\delta}(q(p))$  and  $|\hat{\mu} -\mu_p| < \epsilon_{n,\delta}(p)$ .
Since  $\frac{1}{2} (\epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q(p))$  is not a local minimax bound, there exists some distribution  $p$  and some estimator  $\hat{\mu}$  such that 1) with probability at least  $1 - \delta$  over  $n$  samples from  $p$ ,  $|\hat{\mu} - \mu_p| < \frac{1}{2} (\epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q(p)))$  and 2) the same for  $q$ . However, we already know that  $|\mu_p - \mu_q| \geq \epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q(p))$ , which implies that we can use the mean estimator  $\hat{\mu}$  to distinguish  $p$  and  $q$  with probability  $1 - \delta$  over  $n$  samples. Using this distinguisher, we construct a new estimator  $\hat{\mu}'$  which outputs  $\mu_p$  if the distinguisher thinks the distribution is  $p$ , and  $\mu_q$  otherwise. This new estimator  $\hat{\mu}'$  has 0 error with probability  $1 - \delta$  over  $n$  samples, which contradicts the assumption that  $\epsilon_{n,\delta}$  is a neighborhood Pareto bound and that  $\epsilon_{n,\delta}(p) > 0$ .
The notion of local minimax bounds is therefore (potentially much) weaker than the notion of a neighborhood Pareto bound we introduce in this work. We now show that this can actually happen, if we do not choose the neighborhood structure carefully. We give a concrete example of a neighborhood structure in which an absurdly large local minimax bounds holds, but this bad bound is (rightfully) rejected by the definition of neighborhood Pareto bounds.
As a representative simple example, consider  $p = \mathcal{N}(0,1)$  and define its neighborhood as a singleton set  $N(p) = \{q = \mathcal{N}(\eta,1)\}$  where  $\eta \ll 1$  is small enough that  $p$  and  $q$  are indistinguishable with
$n$  samples. Now define the neighborhood of  $q$  to also be a singleton set  $N(q) = \{q'\}$ , where  $q'$  is constructed by moving a tiny bit of mass of  $q$  such that  $\mu_{q'} = \eta + 10^6$ , but  $q$  and  $q'$  are indistinguishable. Consider an absurd error function  $\epsilon(p) = \eta/2$ ,  $\epsilon(q) = 10^6/2$ , which is far too large for  $q$  since we expect  $O\left(\sqrt{\log \frac{1}{\delta} / n}\right) \ll 1$  estimation error for  $q$  (e.g. by using a standard sub-Gaussian mean estimator). Yet,  $\epsilon$  is a local minimax bound (c.f. Definition 21) under the neighborhood function  $N$ , since, given two indistinguishable distributions  $p$  and  $q$ , no estimator can get error less  $|\mu_p - \mu_q|/2$ . On the other hand, we can also check that Definition 12 rejects this absurd  $\epsilon$  error function from being a neighborhood Pareto bound. To see this, consider the neighborhood of  $p$ , consisting only of  $q$ . Consider the hardcoded estimator  $\hat{\mu}$  always outputting 0:  $\hat{\mu}$  violates the condition of neighborhood Pareto bounds since its error for  $p$  is  $|\hat{\mu} - \mu_p| = 0 < \eta/2$  and for  $q$  is  $|\hat{\mu} - \mu_q| = \eta \ll 10^6/2$ .
This example, together with Proposition 22, show that neighborhood optimality is a more robust notion than local minimax optimality when being applied to inappropriately chosen neighborhood structures. While we believe our paper uses an "appropriate" neighborhood structure, we still emphasize the importance of introducing definitions that are properly resilient to absurd instantiations. For this reason, we have chosen to present our results in this paper as a neighborhood Pareto bound and optimality.
B Remaining proofs for Theorem 2
In Definition 4, we claimed that there exists a parameter  $a$  satisfying certain conditions in Case 2 of the construction. We formally show that this parameter exists in the following lemma.
Lemma 23 Let  $n$  be the number of samples and  $\delta$  be the failure probability, and assume that  $\frac{\log\frac{1}{\delta}}{n}$  is bounded by some sufficiently small absolute constant. Let  $p$  be any distribution such that  $|\mu_p - \mu_{p_n^*}| \leq \sigma_{p_n^*}\sqrt{c\frac{\log\frac{1}{\delta}}{n}}$ , and we assume that  $\mu_p = 0$  without loss of generality. Then, the equation
always has solution  $a \in \left(0, \frac{1}{\sigma_{p_n^*}} \sqrt{\frac{\log \frac{1}{\delta}}{n}}\right]$ .
Proof. We first point out that, for the (potentially) non-unit measure  $q^{+}$  defined by  $\frac{\mathrm{d}q^{+}}{\mathrm{d}p}(x) = 1 + \min(1, \max(-1, ax))$ , the left hand side of the equation equals the difference between the first moments of  $q^{+}$  and  $p$ . Namely,
$$
\int_ {- \infty} ^ {\infty} x (\mathrm {d} q ^ {+} - \mathrm {d} p) = \int_ {- \infty} ^ {\infty} x \min (1, \max (- 1, a x)) \mathrm {d} p = \int_ {- \infty} ^ {- \frac {1}{a}} (- x) \mathrm {d} p + a \cdot \int_ {- \frac {1}{a}} ^ {\frac {1}{a}} x ^ {2} \mathrm {d} p + \int_ {\frac {1}{a}} ^ {\infty} x \mathrm {d} p
$$
Our goal is to find  $a \in \left(0, \frac{1}{\sigma_{p_n^*}} \sqrt{\frac{\log \frac{1}{\delta}}{n}} \right]$  such that this first moment shift equals  $\frac{1}{8} \sigma_{p_n^*} \sqrt{\frac{\log \frac{1}{\delta}}{n}}$ .
Because  $\min(1, \max(-1, ax))$  is increasing in  $a$  and continuous in  $a$ , the first moment shift is also increasing and continuous in  $a$ . When  $a = 0$  then  $q^+ = p$ , and the shift clearly equals 0. Thus it suffices for us to show that the first moment shift is at least  $\frac{1}{8} \sigma_{p_n^*} \sqrt{\frac{\log \frac{1}{8}}{n}}$  when  $a = \frac{1}{\sigma_{p_n^*}} \sqrt{\frac{\log \frac{1}{8}}{n}}$ . The lemma then follows from the intermediate value theorem.
Consider the construction of  $p_n^*$  in Definition 1, and let  $r$  be the trimming radius of  $p$ . We do a case analysis, either  $\frac{1}{\sigma_{p_n^*}} \sqrt{\frac{\log \frac{1}{\delta}}{n}} \leq 1 / r$  or  $\frac{1}{\sigma_{p_n^*}} \sqrt{\frac{\log \frac{1}{\delta}}{n}} \geq 1 / r$ .
Case  $\frac{1}{\sigma_{p_n^*}}\sqrt{\frac{\log\frac{1}{\delta}}{n}}\leq 1 / r$  At  $a = \frac{1}{\sigma_{p_n^*}}\sqrt{\frac{\log\frac{1}{\delta}}{n}}$  we have
$$
\begin{array}{l} a \cdot \int_ {- \frac {1}{a}} ^ {\frac {1}{a}} x ^ {2} \mathrm {d} p \geq a \cdot \int_ {- r} ^ {r} x ^ {2} \mathrm {d} p \quad \text {s i n c e} a \leq 1 / r \\ \geq a \cdot \int_ {- r} ^ {r} \left(x - \mu_ {p _ {n} ^ {*}}\right) ^ {2} \mathrm {d} p \quad \text {s i n c e t h e m e a n s q u a r e d e r r o r i s m i n i m i z e d a t t h e m e a n} \\ \geq \frac {1}{2} a \cdot \sigma_ {p _ {n} ^ {*}} ^ {2} \quad \text {s i n c e o v e r} [ - r, r ] \text {w e h a v e} \frac {\mathrm {d} p _ {n} ^ {*}}{\mathrm {d} p} = \frac {1}{1 - \frac {0 . 4 5}{n} \log \frac {1}{\delta}} \leq 2 \text {f o r s u f f . s m a l l} \frac {\log \frac {1}{\delta}}{n} \\ = \frac {1}{2} \frac {1}{\sigma_ {p _ {n} ^ {*}}} \sqrt {\frac {\log \frac {1}{\delta}}{n}} \cdot \sigma_ {p _ {n} ^ {*}} ^ {2} \quad \text {b y} \\ = \frac {1}{2} \sigma_ {p _ {n} ^ {*}} \sqrt {\frac {\log \frac {1}{\delta}}{n}} \\ \end{array}
$$
Case  $\frac{1}{\sigma_{p_n^*}}\sqrt{\frac{\log\frac{1}{\delta}}{n}}\geq 1 / r$  At  $a = \frac{1}{\sigma_{p_n^*}}\sqrt{\frac{\log\frac{1}{\delta}}{n}}$  we have
$$
\begin{array}{l} \int_ {- \infty} ^ {- \frac {1}{a}} (- x) d p + \int_ {\frac {1}{a}} ^ {\infty} x d p \geq \int_ {- \infty} ^ {- r} (- x) d p + \int_ {r} ^ {\infty} x d p \quad \text {s i n c e} a \geq \frac {1}{r} \\ \geq r \cdot \int_ {\mathbb {R} \backslash [ - r, r ]} 1 \mathrm {d} p \\ = r \cdot \frac {0 . 4 5 \log \frac {1}{\delta}}{n} \quad \text {b y} p _ {n} ^ {*} \\ = 0. 4 5 r \cdot a \cdot \sigma_ {p _ {n} ^ {*}} \sqrt {\frac {\log \frac {1}{\delta}}{n}} \\ \geq 0. 4 5 \sigma_ {p _ {n} ^ {*}} \sqrt {\frac {\log \frac {1}{\delta}}{n}} \quad \text {s i n c e} a \geq \frac {1}{r} \\ \end{array}
$$
Summarizing, in either case, at  $a = \frac{1}{\sigma_{p_n^*}}\sqrt{\frac{\log\frac{1}{\delta}}{n}}$ , we have that the first moment shift between  $q^{+}$  and  $p$  is