SlowGuess/NeurIPS_2023_OCR huggingface dataset:
First, consider the case when the first term is larger, namely,  $|\mu_p - \mu_{p_n^*}| > \sigma_{p_n^*}\sqrt{4.5\log\frac{1}{\delta} / n}$ . Our goal in constructing  $q$  is to maximize  $|\mu_p - \mu_q|$  subject to  $q$  being indistinguishable from  $p$ . Given that 1) the mean of  $p_n^*$  is already far from the mean of  $p$  by assumption in the case analysis, and 2)  $p$  and  $p_n^*$  are by construction hard to distinguish since only a small amount of probability mass was trimmed from  $p$  to make  $p_n^*$ , we simply need to construct  $q$  as a carefully chosen convex combination of  $p$  and  $p_n^*$ , and show that this  $q$  indeed satisfies all the properties in the definition of  $N_{n,\delta}(p)$ .
Next, for the remaining case when the variance term  $\sigma_{p_n^*}\sqrt{4.5\log\frac{1}{\delta}/n}$  is larger than the remaining term  $|\mu_p - \mu_{p_n^*}|$  in  $\epsilon_{n,\delta}(p)$ , we now want to construct  $q$  such that the mean shift  $|\mu_p - \mu_q|$  is large compared to the variance term  $\sigma_{p_n^*}\sqrt{4.5\log\frac{1}{\delta}/n}$ , while ensuring that  $q$  is indistinguishable from  $p$ . To achieve this, we create  $q$  that is a "skewed" version of  $p$ , scaling the probability density by a linear function  $1 + ax$ , where larger  $a$  means more mean shift between  $q$  and  $p$ , but also means that it is easier to distinguish  $q$  from  $p$ . Technically, we truncate the linear scaling factor  $1 + ax$  to lie between 0 and 2, so that  $q$  will satisfy Condition 3 of the requirements for  $q$  presented in our main theorem, Theorem 2; the probability mass might not be 1 after this skewing, so we might need to normalize; also, we point out that for the purposes of Theorem 2 we do not care whether the mean of  $q$  is shifted to the left or right of  $p$  (corresponding to choosing a positive or negative parameter  $a$ ), and the construction makes use of this choice.
Definition 4 (Construction of indistinguishable pair) Given a distribution  $p$ , we construct a distribution  $q$  in a shift-and-scale invariant manner as follows.
Case 1:  $|\mu_p - \mu_{p_n^*}| > \sigma_{p_n^*}\sqrt{4.5\frac{\log\frac{1}{\delta}}{n}}$ . Define  $\lambda = \frac{3}{4}$  and construct  $q$  to be the weighted average  $q = \lambda p + (1 - \lambda)p_n^*$ .
Case 2:  $|\mu_p - \mu_{p_n^*}| \leq \sigma_{p_n^*} \sqrt{4.5 \frac{\log \frac{1}{\delta}}{n}}$ . Without loss of generality, assume that  $\mu_p = 0$ . Let parameter  $a$  be the solution in the interval  $\left(0, \frac{1}{\sigma_{p_n^*}} \sqrt{\frac{\log \frac{1}{\delta}}{n}} \right]$  to the below equation characterizing the mean shift, as guaranteed by Lemma 23 in Appendix B:
$$
\int_ {- \infty} ^ {- \frac {1}{a}} (- x) \mathrm {d} p + a \cdot \int_ {- \frac {1}{a}} ^ {\frac {1}{a}} x ^ {2} \mathrm {d} p + \int_ {\frac {1}{a}} ^ {\infty} x \mathrm {d} p = \frac {1}{8} \sigma_ {p _ {n} ^ {*}} \sqrt {\frac {\log \frac {1}{\delta}}{n}}
$$
We construct two non-unit measures  $q^{+}, q^{-}$ , defined as scaled versions of  $p$ , as  $\frac{\mathrm{d}q^{\pm}}{\mathrm{d}p}(x) = 1 + \min(1, \max(-1, \pm ax))$ . By symmetry the masses of  $q^{+}$  and  $q^{-}$  sum to 2; thus one of  $q^{+}, q^{-}$  has mass at least 1. Construct  $q$  by choosing that one of  $q^{+}, q^{-}$ , and downscaling it by a factor  $b \in [\frac{1}{2}, 1]$  such that the total probability mass is indeed 1.
In the rest of the subsections, we will prove all the properties of the construction needed by Theorem 2. We observe that the non-unit measure  $q^{-}$  is "symmetric" to  $q^{+}$ , in the sense that its first moment shift
from  $p$  is identical but of the opposite sign. Therefore, most of the analysis below will assume the  $q^{+}$  case without loss of generality.
2.1 Checking that  $\frac{\mathrm{d}q}{\mathrm{d}p}\leq 2$
It is straightforward to check that  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$  by construction, for both cases of Definition 4.
Lemma 5 Suppose there is a sufficiently small absolute constant that upper bounds  $\frac{\log\frac{1}{\delta}}{n}$ . Given a distribution  $p$ , if we construct  $q$  as in Case 1 (the large  $|\mu_{p_n^*} - \mu_p|$  case) in Definition 4, then  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ .
Proof. Let the support of  $p_n^*$  be  $[x_{\mathrm{left}}, x_{\mathrm{right}}]$ . At  $x \notin [x_{\mathrm{left}}, x_{\mathrm{right}}]$ , we have  $\frac{\mathrm{d}q}{\mathrm{d}p} = \frac{\lambda \, \mathrm{d}p}{\mathrm{d}p} = \frac{3/4 \, \mathrm{d}p}{\mathrm{d}p} \leq 2$ . Otherwise, at  $x \in [x_{\mathrm{left}}, x_{\mathrm{right}}]$ , we have
$$
\frac {\mathrm {d} q}{\mathrm {d} p} = \frac {\lambda \mathrm {d} p + (1 - \lambda) \mathrm {d} p _ {n} ^ {*}}{\mathrm {d} p} = \frac {\mathrm {d}}{\mathrm {d} p} \left(\lambda p + \frac {1 - \lambda}{1 - \frac {0 . 4 5 \log \frac {1}{\delta}}{n}} p\right) \quad (\text {b y})
$$
where the last line follows from the fact that  $\lambda$  is a constant in  $(0,1)$  and  $\frac{\log\frac{1}{\delta}}{n}$  is assumed to be bounded by some sufficiently small absolute constant.
Lemma 6 Suppose there is a sufficiently small absolute constant that upper bounds  $\frac{\log\frac{1}{\delta}}{n}$ . Given a distribution  $p$ , if we construct  $q$  as in Case 2 (the small  $|\mu_{p_n^*} - \mu_p|$  case) in Definition 4, then  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ .
Proof. For  $q^+$ , we have  $\frac{\mathrm{d}q^+}{\mathrm{d}p}(x) = 1 + \min(1, \max(-1, a^+ x))$  for some value of  $a^+$ , and the right hand side is always between 0 and 2; proven similarly, we have  $0 \leq \frac{\mathrm{d}q^-}{\mathrm{d}p} \leq 2$ . Noting that  $q$  is constructed by scaling down one of  $q^-$  and  $q^+$ , we also have  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ .
2.2 Bounding the squared Hellinger distance
We consider each case of the construction in Definition 4 separately. We first bound the Hellinger distance between  $p$  and the  $q$  constructed in Case 1 of Definition 4 (when  $|\mu_p - \mu_{p_n^*}|$  is large).
Lemma 7 Suppose there is a sufficiently small constant that upper bounds both  $\frac{\log\frac{1}{\delta}}{n}$  and  $\delta$ . Given a distribution  $p$ , if we construct  $q$  as in Case 1 (the large  $|\mu_{p_n^*} - \mu_p|$  case) in Definition 4, then  $\log(1 - d_{\mathrm{H}}^2(p, q)) \geq \frac{1}{2n} \log 4\delta$ .
Since Case 1 of the construction of  $q$  in Definition 4 linearly interpolates between  $p$  and—a very slightly trimmed version of  $p$ , namely  $-p_n^*$ , the resulting distribution  $q$  remains close to  $p$ ; the calculation is in Appendix B.1. The next lemma bounds the squared Hellinger distance of  $p$  and  $q$  in Case 2 of Definition 4 (when  $|\mu_p - \mu_{p_n^*}|$  is small).
Lemma 8 Suppose there is a sufficiently small constant that upper bounds both  $\frac{\log\frac{1}{\delta}}{n}$  and  $\delta$ . Given a distribution  $p$ , if we construct  $q$  as in Case 2 (the small  $|\mu_{p_n^*} - \mu_p|$  case) in Definition 4, then  $\log(1 - d_{\mathrm{H}}^2(p, q)) \geq \frac{1}{2n} \log 4\delta$ .
The proof (see Appendix B.1) uses a technical lemma to relate  $d_H(p, q)$  to  $d_H(p, q^+)$  or  $d_H(p, q^-)$ , and then uses a linearization of the definition of Hellinger distance to relate it to the mean shift between  $p$  and  $q^+$  or  $q^-$ , which is bounded by Definition 4.
2.3 Lower bounding  $|\mu_q - \mu_p|$
We show the lower bound separately for the two cases in the construction of  $q$  in Definition 4. The small  $|\mu_{p_n^*} - \mu_p|$  case is a direct corollary of Lemma 23, which bounds the mean shift in this case.
Lemma 9 Suppose there is a sufficiently small constant that upper bounds both  $\frac{\log\frac{1}{\delta}}{n}$  and  $\delta$ . Given a distribution  $p$ , if we construct  $q$  as in Case 1 (the large  $|\mu_{p_n^*} - \mu_p|$  case) in Definition 4, then  $|\mu_q - \mu_p| \geq \frac{1}{8}\epsilon_{n,\delta}(p)$ .
Proof. Without loss of generality, assume that  $\mu_p = 0$ . Recall that  $q = \lambda p + (1 - \lambda)p_n^*$  and  $\lambda = \frac{3}{4}$  from Case 1 of Definition 4. Thus,  $|\mu_q - \mu_p| = \frac{1}{4} |\mu_{p_n^*} - \mu_p|$ . Furthermore, we have  $\epsilon_{n,\delta}(p) = |\mu_{p_n^*} - \mu_p| + \sigma_{p_n^*}\sqrt{4.5\frac{\log\frac{1}{\delta}}{n}}$  from the definition of  $\epsilon_{n,\delta}(p)$  and  $|\mu_{p_n^*} - \mu_p| > \sigma_{p_n^*}\sqrt{4.5\frac{\log\frac{1}{\delta}}{n}}$  from the lemma assumption and Case 1 of Definition 4, which then imply that  $\epsilon_{n,\delta}(p) \leq 2|\mu_{p_n^*} - \mu_p|$ . Combining the two inequalities yields  $|\mu_q - \mu_p| \geq \frac{1}{8}\epsilon_{n,\delta}(p)$  as desired.
Lemma 10 Suppose there is a sufficiently small constant that upper bounds both  $\frac{\log\frac{1}{\delta}}{n}$  and  $\delta$ . Given a distribution  $p$ , if we construct  $q$  as in Case 2 (the small  $|\mu_{p_n^*} - \mu_p|$  case) in Definition 4, then  $|\mu_q - \mu_p| \geq \frac{1}{32}\epsilon_{n,\delta}(p)$ .
Proof. Without loss of generality, let  $q^+$  be the non-unit measure used to construct  $q$ , that is  $q = bq^+$  for some  $b \in [\frac{1}{2}, 1]$ . By the bound on  $b$  as well as the construction of  $q^+$  in Definition 4, we have  $|\mu_q - \mu_p| = b|\int x\mathrm{d}q^+ - \int x\mathrm{d}p| \geq \frac{1}{16}\sigma_{p_n^*}\sqrt{\frac{\log\frac{1}{\delta}}{n}}$ . Additionally, we both have  $\epsilon_{n,\delta}(p) = |\mu_{p_n^*} - \mu_p| + \sigma_{p_n^*}\sqrt{4.5\frac{\log\frac{1}{\delta}}{n}}$  from the definition of  $\epsilon_{n,\delta}(p)$  and  $|\mu_{p_n^*} - \mu_p| \leq \sigma_{p_n^*}\sqrt{4.5\frac{\log\frac{1}{\delta}}{n}}$  from the lemma assumption and Case 2 of Definition 4, both of which imply  $\epsilon_{n,\delta}(p) \leq 2\sigma_{p_n^*}\sqrt{4.5\frac{\log\frac{1}{\delta}}{n}}$ . Combining the two inequalities above, we have that  $|\mu_q - \mu_p| \geq \frac{1}{32}\epsilon_{n,\delta}(p)$ .
3 Neighborhood Optimality: A New Definition Framework
Our main result is a specific and technical indistinguishability result. This section aims to clarify, through a new definition framework, the optimality notion that our technical result implies.
3.1 Neighborhood Optimality
Usual notions of "beyond worst-case optimality" include "instance optimality" which is unattainably strong, and "admissibility"/"Pareto efficiency" from the statistics and economics literature, which is too weak. In particular, the latter notion is too weak in the sense that a trivial estimator that always outputs the same hardcoded mean estimate is actually admissible, despite being algorithmically "vacuous". We define and explain these notions formally in Appendix D.
Given that neither of the usual definitions are suitable for mean estimation, in this work we give a new optimality definition, which we call neighborhood optimality. We state its definition in this section, and explore its basic properties and intuition in Appendix A, including how the definition smoothly interpolates between instance optimality and admissibility. Our definition is also related to the notion of local minimax optimality, which we compare with in Appendix A.3. The differences are subtle, yet, as we show in Appendix A.3, local minimax is too weak a notion and, when instantiated inappropriately, allows for absurd bounds to be proven. We thus advocate for this new optimality definition, which correctly rejects such absurd bounds. As an application of our new framework, we prove in Section 4 that the median-of-means estimator is neighborhood optimal up to constant factors. It is an open question to find a neighborhood optimal estimator without the constant factor slackness.
Let  $\mathcal{P}_1$  be the entire set of all distributions with a finite first moment over  $\mathbb{R}$ . We say that  $N$  is a neighborhood function (defined over  $\mathcal{P}_1$ ) if  $N$  maps a distribution  $p \in \mathcal{P}_1$  to a set of distributions  $N(p) \subseteq \mathcal{P}_1$ . For the purposes of the rest of the definitions, it will not matter whether  $p \in N(p)$ . Similarly, an error function  $\epsilon$  maps distributions to non-negative numbers, like  $\epsilon_{n,\delta}$  in our main result, Theorem 2. In the later definitions, we use the notations  $N_{n,\delta}$  and  $\epsilon_{n,\delta}$  to denote their dependence on the sample complexity  $n$  and failure probability  $\delta$ .
Given these two notions, we can now define neighborhood Pareto bounds with respect to  $N_{n,\delta}$ , which imposes admissibility structure within the local neighborhood  $N_{n,\delta}(p)$  of every distribution  $p \in \mathcal{P}_1$ .
Definition 11 (Neighborhood Pareto bounds with respect to  $N_{n,\delta}$ ) Let  $n$  be the number of samples and  $\delta$  be the failure probability. Given a neighborhood function  $N_{n,\delta}:\mathcal{P}_1\rightarrow 2^{\mathcal{P}_1}$ , we say that the error function  $\epsilon_{n,\delta}(p):\mathcal{P}_1\to \mathbb{R}_0^+$  is a neighborhood Pareto bound for  $\mathcal{P}_1$  with respect to  $N_{n,\delta}$  if for all distributions  $p\in \mathcal{P}_1$ , no estimator  $\hat{\mu}$  taking  $n$  i.i.d. samples can simultaneously achieve the following two conditions:
- For all  $q \in N_{n,\delta}(p)$ , with probability  $1 - \delta$  over the  $n$  i.i.d. samples from  $q$ ,  $|\hat{\mu} - \mu_q| \leq \epsilon_{n,\delta}(q)$ .
- With probability  $1 - \delta$  over the  $n$  i.i.d. samples from  $p$ ,  $|\hat{\mu} - \mu_p| < \epsilon_{n,\delta}(p)$ .
Note the strict inequality in the second bullet: namely, it is impossible to "beat" the error function over an entire neighborhood, where "beating" is defined as attaining the error function over the neighborhood, and performing strictly better than the error function for  $p$ . The above two bullet points essentially capture admissibility within the local neighborhood  $N_{n,\delta}(p) \cup \{p\}$  — compare with Definition 28—and the definition requires admissibility within every such local neighborhood, over every possible  $p$ .
The neighborhood  $N_{n,\delta}(p)$  in a neighborhood Pareto bound can be interpreted as the set of distributions "near  $p$ " which, if an estimator performs well on distribution  $p$ , then we should reasonably expect or want it to perform well also on all the distributions in the local neighborhood  $N_{n,\delta}(p)$ .
Using the notion of neighborhood Pareto bounds, we can now define  $\kappa$ -neighborhood optimal estimators, which are estimators whose performances are matched by neighborhood Pareto bounds.
Definition 12  $((\kappa, \tau)$ -Neighborhood optimal estimators) Let  $\kappa > 1$  be a multiplicative loss factor in estimation error, and  $\tau > 1$  be a multiplicative loss factor in sample complexity. Given the parameters  $\kappa, \tau > 1$ , sample complexity  $n$ , failure probability  $\delta$  and neighborhood function  $N_{n,\delta}$ , a mean estimator  $\hat{\mu}$  is  $(\kappa, \tau)$ -neighborhood optimal with respect to  $N_{n,\delta}$  if there exists an error function  $\epsilon_{n,\delta}(p)$  such that  $\min(\epsilon_{n/\tau,\delta}(p), \epsilon_{n,\delta}(p))$  is a neighborhood Pareto bound $^2$ , and  $\hat{\mu}$  gives estimation error at most  $\kappa \cdot \epsilon_{n,\delta}(p)$  with probability at least  $1 - \delta$  when taking  $n$  i.i.d. samples from any distribution  $p \in \mathcal{P}_1$ .
As a basic example and sanity check, in Appendix E, we show that any trivial estimator that outputs a hardcoded mean estimate cannot be  $\kappa$ -neighborhood optimal with respect to our chosen neighborhood function (Definition 15 in Section 4) for any  $\kappa$ .
3.2 Indistinguishability implies a neighborhood Pareto bound
Even though it might not look obvious how we can prove a neighborhood Pareto bound from its definition, we show that our main indistinguishability result essentially implies such a bound. The proof essentially follows the straightforward estimation-to-testing reduction intuition, and we give it formally in Appendix A.2.
Proposition 13 ("Local" indistinguishability bounds imply neighborhood Pareto bounds) The error function  $\epsilon_{n,\delta}$  is a neighborhood Pareto bound with respect to the neighborhood function  $N_{n,\delta}$  if for every distribution  $p\in \mathcal{P}_1$ , there exists a distribution  $q\in N_{n,\delta}(p)$ , with  $q\neq p$ , such that  $|\mu_p - \mu_q|\geq \epsilon_{n,\delta}(p) + \epsilon_{n,\delta}(q)$  and it is information-theoretically impossible to distinguish  $p$  and  $q$  with probability  $1 - \delta$  using  $n$  samples.
4 Median-of-Means is Neighborhood Optimal
To apply our new definitional framework, we choose a reasonable neighborhood function  $N_{n,\delta}$  and show that the median-of-means algorithm is neighborhood optimal with respect to this choice.
In Appendix C, we give the following (straightforward) re-analysis of median-of-means, which will form the upper bound part for neighborhood optimality.
2While it is intuitive to expect that an error function decreases in  $n$ , it might not be true in general. Indeed, the definition of  $\epsilon_{n,\delta}(p)$  we use in the main result is not necessarily monotonic. This is why we use a min in the neighborhood Pareto bound requirement.
Proposition 14 Consider a distribution  $p$  with mean  $\mu_p$ , a sample size  $n$ , and a median-of-means group count  $4.5\log \frac{1}{\delta}$ . Let  $p_n^*$  be the  $\frac{0.45}{n}\log \frac{1}{\delta}$ -trimmed distribution from Definition 1, and  $\mu_{p_n^*}$  and  $\sigma_{p_n^*}$  be the mean and standard deviation of  $p_n^*$  respectively. Then, the median-of-means estimator has error  $|\mu_p - \mu_{p_n^*}| + 3\sigma_{p_n^*}\sqrt{\frac{4.5\log\frac{1}{\delta}}{n}}$  except with probability at most  $\delta$ .
We can now discuss the neighborhood choice for the corresponding neighborhood Pareto bound. Recall that, intuitively, neighborhood optimality is asking "how well can our algorithm do on  $p$  given that we also want our algorithm to do similarly well on a neighborhood of  $p$ "; and thus, the smaller we choose the neighborhood, the stronger the resulting theorem. We thus define the neighborhood of  $p$  to consist of distributions that are similar or similarly nice to  $p$  in 4 different ways:
Definition 15 (Choice of neighborhood function  $N_{n,\delta}$  in Theorem 16) Define  $N_{n,\delta}(p)$  to be the set of distributions  $q \in \mathcal{P}_1$  such that
1.  $\epsilon_{n / 3,\delta}(q)\leq 100\epsilon_{n,\delta}(p)$
2.  $\log (1 - d_{\mathrm{H}}^2 (p,q))\geq \frac{1}{2n}\log 4\delta$
3.  $|\mu_q - \mu_p| \leq \epsilon_{n,\delta}(p)$
4. For all  $x\in \mathbb{R}$ $\frac{\mathrm{d}q}{\mathrm{d}p} (x)\leq 2$
As a basic sanity check, we show in Appendix E that a trivial, hardcoded estimator cannot be neighborhood optimal—this is mostly a consequence of Property 3 above. See Appendix E for a formal statement and proof. We then show:
Theorem 16 Let  $n$  be the number of samples and  $\delta$  be the failure probability. Assume that there is a sufficiently small constant which upper bounds both  $\frac{\log\frac{1}{\delta}}{n}$  and  $\delta$ .
Consider the neighborhood function  $N_{n,\delta}$  of Definition 15. Recall the error function defined in Definition 1 as  $\epsilon_{n,\delta}(p) = |\mu_p - \mu_{p_n^*}| + \sigma_{p_n^*}\sqrt{\frac{4.5\log\frac{1}{\delta}}{n}}$ . Then, for some sufficiently large constant  $\kappa$ , the error function  $\frac{1}{\kappa}\min (\epsilon_{n / 3,\delta},\epsilon_{n,\delta})$  is a neighborhood Pareto bound with respect to  $N_{n,\delta}$ .
Combined with Proposition 14 stating that the median-of-means estimator has error function  $O(\epsilon_{n,\delta})$ , this implies the median-of-means estimator is  $(\kappa, 3)$ -neighborhood optimal with respect to  $N_{n,\delta}$ .
The main component of the proof is our construction of  $q$ , and the accompanying analysis of Theorem 2 showing that  $q$  is well behaved in several senses. We specifically show Lemma 17, a slight extension of Theorem 2:
Lemma 17 Let  $n$  be the sample complexity and  $\delta$  be the failure probability, and recall the definition of  $\epsilon_{n,\delta}$  from Definition 1. Assume that there is a sufficiently small constant which upper bounds both  $\frac{\log\frac{1}{\delta}}{n}$  and  $\delta$ . Then for any distribution  $p$ , there exists a distribution  $q \neq p$  such that the mean of  $q$  is  $\frac{1}{32}\epsilon_{n,\delta}(p)$  different from the mean of  $p$ , and  $\log(1 - d_{\mathrm{H}}^2(p,q)) \geq \frac{1}{2n} \log 4\delta$ , and  $q \in N_{n,\delta}(p)$ .
See Appendix F for the proof of Lemma 17. We will now use Lemma 17 to prove Theorem 16.
Proof of Theorem 16. By Proposition 13, it suffices to show that, for every distribution  $p \in \mathcal{P}_1$ , there exists a distribution  $q \in N_{n,\delta}(p)$  with  $q \neq p$  such that  $|\mu_p - \mu_q| \geq \frac{1}{\kappa} (\min(\epsilon_{n/3,\delta}, \epsilon_{n,\delta})(p) + \min(\epsilon_{n/3,\delta}, \epsilon_{n,\delta})(q))$  for some large constant  $\kappa$ , and no tester can distinguish  $p$  and  $q$  with probability  $1 - \delta$  using  $n$  samples.
Given a distribution  $p \in \mathcal{P}_1$ , consider the distribution  $q \in N_{n,\delta}(p)$  guaranteed by Lemma 17. Since  $q$  satisfies  $\log (1 - d_{\mathrm{H}}^2(p, q)) \geq \frac{1}{2n} \log 4\delta$ , by Fact 1 we know that  $p$  and  $q$  are indistinguishable with probability  $1 - \delta$  using  $n$  samples.
It remains to check that  $|\mu_p - \mu_q| \geq \frac{1}{\kappa} (\min(\epsilon_{n/3,\delta}, \epsilon_{n,\delta})(p) + \min(\epsilon_{n/3,\delta}, \epsilon_{n,\delta})(q))$  for some sufficiently large constant  $\kappa$ . By Lemma 17, we have
$$
\begin{array}{l} \left| \mu_ {p} - \mu_ {q} \right| \geq \Omega \left(\epsilon_ {n, \delta} (p)\right) \geq \Omega \left(\epsilon_ {n, \delta} (p) + \epsilon_ {n, \delta} (p)\right) \geq \Omega \left(\epsilon_ {n / 3, \delta} (q) + \epsilon_ {n, \delta} (p)\right) \quad \text {b y} \\ \geq \Omega (\min  (\epsilon_ {n / 3, \delta}, \epsilon_ {n, \delta}) (p) + \min  (\epsilon_ {n / 3, \delta}, \epsilon_ {n, \delta}) (q)) \\ \end{array}
$$
which completes the proof of Theorem 16.
We thank the anonymous reviewers for insightful comments and suggestions on this work. Jasper C.H. Lee is supported in part by the generous funding of a Croucher Fellowship for Postdoctoral Research, NSF award DMS-2023239, NSF Medium Award CCF-2107079 and NSF AiTF Award CCF-2006206. Maoyuan Song is supported in part by NSF award CCF-1910411, NSF award CCF-2228814, and NSF award CCF-2127806. Paul Valiant is supported by NSF award CCF-2127806.