SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of the first bullet. Define  $S = [-1, 1] \setminus [\mu_q - r_q, \mu_q + r_q]$ . Firstly, due to triangle inequality, we have
$$
\left| \int_ {S} (x - \mu_ {q}) \mathrm {d} q \right| \leq \left| \int_ {S} (\mu_ {p} - \mu_ {q}) \mathrm {d} q \right| + \left| \int_ {S} (x - \mu_ {p}) \mathrm {d} q \right| \leq | \mu_ {p} - \mu_ {q} | + \left| \int_ {S} (x - \mu_ {p}) \mathrm {d} q \right|
$$
The Cauchy-Schwarz inequality then says that
$$
\left| \int_ {S} (x - \mu_ {p}) \mathrm {d} q \right| \leq \sqrt {\int_ {S} (x - \mu_ {p}) ^ {2} \mathrm {d} q} \sqrt {\int_ {S} \mathrm {d} q}
$$
Note that the second integral in the right hand side is bounded by the amount of trimmed probability mass, which is at most  $\frac{1.35\log\frac{1}{\delta}}{n}$ . The first integral, since  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ , is bounded by twice the variance of  $p$  in  $[-1,1]$ , namely  $2\sigma_{p_n^*}^2$ . Thus the product of the square roots of the two integrals on the right hand side is bounded by  $\sigma_{p_n^*}\sqrt{2\frac{1.35\log\frac{1}{\delta}}{n}}$ , yielding the desired bound.
Proof of the second bullet. Without loss of generality, we assume  $\mu_q \geq \mu_p = 0$ . Thus by the assumption of this case, that  $[\mu_q - r_q, \mu_q + r_q]$  extends outside  $[-1,1]$ , we have that  $\mu_q + r_q > 1$ , because of the asymmetry that  $\mu_q \geq 0$ . Since  $\mu_q \leq \frac{1}{4}$  by the lemma assumption, we also have  $r_q \geq \frac{3}{4}$ .
Recall that  $p$  has a total of  $\frac{0.45\log\frac{1}{\delta}}{n}$  mass outside  $[-1,1]$  by the lemma assumption, and that  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ . This implies that  $q$  has at most  $\frac{0.9\log\frac{1}{\delta}}{n}$  mass outside  $[-1,1]$ . As  $\frac{1.35\log\frac{1}{\delta}}{n}$  mass is trimmed from  $q$  to construct  $q_{n/3}^*$ , there is thus at least  $\frac{0.45\log\frac{1}{\delta}}{n}$  mass trimmed from  $q$  inside the interval  $[-1,1]$ . Furthermore, since  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ , we conclude that there is thus at least  $\frac{0.225\log\frac{1}{\delta}}{n}$  mass trimmed from  $p$  inside the interval  $[-1,1]$ . Now denote the set on which  $q$  is trimmed, restricted to  $[-1,1]$  by  $S = [-1,1] \setminus [\mu_q - r_q, \mu_q + r_q]$ . Since  $p$  has at least  $\frac{0.225\log\frac{1}{\delta}}{n}$  mass in  $S$ , meaning that  $S \neq \emptyset$ , and  $\mu_q + r_q > 1$  from earlier, it must thus be the case that  $\mu_q - r_q > -1$  and therefore  $r_q < \frac{5}{4}$ .
Since  $\min_{x_i\in S}|x_i| > \frac{1}{2}$ , we conclude that  $(\sigma_{p_n^*})^2 +(\mu_{p_n^*})^2 = \int_{-1}^1 x^2\mathrm{d}p_n^*\geq \int_{-1}^1 x^2\mathrm{d}p\geq$ $\int_S x^2\mathrm{d}p\geq \frac{c}{2^2\cdot 20}\frac{\log\frac{1}{\delta}}{n}$ . Using the standard inequality that the  $\ell_1$  norm is at least the  $\ell_2$  norm, we have that  $\sigma_{p_n^*} + \mu_{p_n^*}\geq \sqrt{(\sigma_{p_n^*})^2 + (\mu_{p_n^*})^2}\geq \sqrt{\frac{c}{2^2\cdot 20}\frac{\log\frac{1}{\delta}}{n}}$ . Thus, since  $\sqrt{\frac{4.5\log\frac{1}{\delta}}{n}}\leq \frac{1}{3}$  by the lemma assumption, we have that  $\epsilon_{n,\delta}(p) = \sigma_{p_n^*}\sqrt{\frac{4.5\log\frac{1}{\delta}}{n}} +\mu_{p_n^*}\geq \sqrt{\frac{9}{80}}\frac{4.5\log\frac{1}{\delta}}{n}$ . This yields the second inequality in the second bullet point.
We now prove the first inequality in the second bullet, which bounds the difference between two integrals; we bound this by bounding the probability mass in each integral and multiplying this by a bound on the integrand of each integral. Explicitly, the amount of probability mass of  $q$  that is in  $[-1, 1]$  but outside of  $[\mu_q - r_q, \mu_q + r_q]$  is at most  $\frac{1.35\log\frac{1}{\delta}}{n}$ , and furthermore, for every  $x \in [-1, 1] \setminus [\mu_q - r_q, \mu_q + r_q]$ , we have  $|x - \mu_q| \leq \frac{5}{4}$  due to  $|\mu_q| \leq \frac{1}{4}$ . In parallel, the amount of probability mass of  $p$  that is outside of  $[-1, 1]$  but in  $[\mu_q - r_q, \mu_q + r_q]$  is at most  $\frac{0.45\log\frac{1}{\delta}}{n}$ , and since  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ , we conclude that  $q$  has at most  $\frac{0.9\log\frac{1}{\delta}}{n}$  mass outside of  $[-1, 1]$  but in  $[\mu_q - r_q, \mu_q + r_q]$ . Also, for every  $x \in [\mu_q - r_q, \mu_q + r_q] \setminus [-1, 1]$ , we have  $|x - \mu_q| \leq \frac{5}{4}$  because  $r_q \leq \frac{5}{4}$ . Thus, the left hand side of the second bullet is at most  $(2 + 3)\frac{5}{4} 0.45\frac{\log\frac{1}{\delta}}{n} = 2.8125\frac{\log\frac{1}{\delta}}{n}$ , as claimed.
Lemma 33 Given a distribution  $p$  with  $\mu_p = 0$  and whose trimming radius for constructing  $p_n^*$  is equal to 1, let  $q$  be the distribution constructed from  $p$  as in Case 2 (the small  $|\mu_{p_n^*} - \mu_p|$  case) of Definition 4. Then  $\left|\int_{\mathbb{R} \setminus [-1,1]} (x - \mu_q) \, \mathrm{d}q\right| \leq 5 \sigma_{p_n^*} \sqrt{\frac{\log \frac{1}{\delta}}{n}}$ .
$$
\left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} (x - \mu_ {q}) \mathrm {d} q \right| \leq \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \mathrm {d} q \right| + \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} - \mu_ {q} \mathrm {d} q \right| \leq \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \mathrm {d} q \right| + | \mu_ {q} |
$$
and since  $|\mu_q| \leq \frac{1}{8} \sigma_{p_n^*} \sqrt{\frac{\log \frac{1}{\delta}}{n}}$  by Lemma 31, we only have to bound the first term by some constant multiple of  $\sigma_{p_n^*} \sqrt{\frac{\log \frac{1}{\delta}}{n}}$ .
Recall the construction of  $q$  in Definition 4, as a downscaled version of either  $q^{+}$  or  $q^{-}$ . Without loss of generality (by reflecting  $p$  and  $q$  as appropriate), we assume we use  $q^{+}$ . Specifically,  $q^{+}$  is the non-unit measure such that  $\frac{\mathrm{d}q^{+}}{\mathrm{d}p}(x) = 1 + \min(1, \max(-1, ax))$  for some value  $a$  specified in Definition 4. Since  $q$  is a downscaled version of  $q^{+}$ , we will bound  $\left| \int_{\mathbb{R} \setminus [-1,1]} x \, \mathrm{d}q \right| \leq \left| \int_{\mathbb{R} \setminus [-1,1]} x \, \mathrm{d}q^{+} \right|$  by a constant multiple of  $\sigma_{p_n^*} \sqrt{\frac{\log \frac{1}{\delta}}{n}}$ .
Observe that for any  $a > 0$ , we have that  $x$  and  $\min(1, \max(-1, ax))$  have the same sign everywhere, which means
$$
\left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \min  (1, \max  (- 1, a x))) \mathrm {d} p \right| \leq \left| \int_ {\mathbb {R}} x \min  (1, \max  (- 1, a x))) \mathrm {d} p \right|
$$
Additionally, because  $\frac{\log\frac{1}{\delta}}{n}$  is upper bounded by a sufficiently small absolute constant and  $|\mu_{p_n^*}|\leq \sigma_{p_n^*}\sqrt{\frac{4.5}{n}\log\frac{1}{\delta}}$  from the lemma assumption, we get  $|\frac{\mu_{p_n^*}}{1 - \frac{0.45}{n}\log\frac{1}{\delta}} |\leq 2\sigma_{p_n^*}\sqrt{\frac{0.45}{n}\log\frac{1}{\delta}}\leq$ $4.5\sigma_{p_n^*}\sqrt{\frac{\log\frac{1}{\delta}}{n}}.$
We can now upper bound  $\left|\int_{\mathbb{R}\setminus [-1,1]}x\mathrm{d}q^{+}\right|$  as follows.
$$
\begin{array}{l} \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \mathrm {d} q ^ {+} \right| = \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x (1 + \min  (1, \max  (- 1, a x))) \mathrm {d} p \right| \\ \leq \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \mathrm {d} p \right| + \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \min  (1, \max  (- 1, a x))\right) \mathrm {d} p \\ \leq \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \mathrm {d} p \right| + \left| \int_ {\mathbb {R}} x \min  (1, \max  (- 1, a x))\right) \mathrm {d} p \\ = \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} x \mathrm {d} p \right| + \left| \int_ {\mathbb {R}} x (\mathrm {d} q ^ {+} - \mathrm {d} p) \right| \\ = \left| \frac {\mu_ {p _ {n} ^ {*}}}{1 - \frac {0 . 4 5}{n} \log \frac {1}{\delta}} \right| + \left| \int_ {\mathbb {R}} x \mathrm {d} q ^ {+} \right| \quad \text {s i n c e} \mu_ {p} = 0 \\ \leq \left| \frac {\mu_ {p _ {n} ^ {*}}}{1 - \frac {0 . 4 5}{n} \log \frac {1}{\delta}} \right| + 2 | \mu_ {q} | \quad \text {s i n c e q i s a d o w n s c a l e o f q ^ {+} b y f a c t o r a t m o s t 2} \\ \leq 5 \sigma_ {p _ {n} ^ {*}} \sqrt {\frac {\log \frac {1}{\delta}}{n}} \quad \text {u s i n g t h e p r i o r b o u n d s o n t h e t w o t e r m s a n d t h a t 4 . 5 + \frac {2}{8} \leq 5} \\ \end{array}
$$
Lemma 34 Given a distribution  $p$  with  $\mu_p = 0$  and whose trimming radius for constructing  $p_n^*$  is equal to 1, let  $q$  be the distribution constructed from  $p$  as in Case 1 (the large  $|\mu_{p_n^*} - \mu_p|$  case) of Definition 4. Then  $\left|\int_{\mathbb{R} \setminus [-1,1]} (x - \mu_q) \, \mathrm{d}q\right| \leq 5|\mu_{p_n^*} - \mu_p|$
Proof. From the construction,  $q$  outside  $[-1, 1]$  just consists of the corresponding portion of  $p$  multiplied by  $\lambda \in (0, 1)$ . Thus
$$
\begin{array}{l} \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} (x - \mu_ {q}) \mathrm {d} q \right| = \lambda \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} (x - \mu_ {q}) \mathrm {d} p \right| \\ = \lambda \left| \left(\mu_ {q} - \mu_ {p}\right) \cdot \frac {0 . 4 5 \log \frac {1}{\delta}}{n} + \int_ {\mathbb {R} \backslash [ - 1, 1 ]} \left(x - \mu_ {p}\right) \mathrm {d} p \right| \\ \leq | \mu_ {p} - \mu_ {q} | + \frac {| \mu_ {p _ {n} ^ {*}} - \mu_ {p} |}{1 - \frac {0 . 4 5 \log {\frac {1}{\delta}}}{n}} \\ \end{array}
$$
Since  $q$  is an interpolation between  $p$  and  $p_n^*$ , the mean of  $q$  is in between the mean of  $p$  and the mean of  $p_n^*$ , and thus  $|\mu_p - \mu_q| \leq |\mu_{p_n^*} - \mu_p|$ . Furthermore, since  $\frac{1}{n}\log \frac{1}{\delta}$  is upper bounded by some sufficiently small absolute constant, the entire bound above can be finally bounded by  $5|\mu_{p_n^*} - \mu_p|$ , as desired.
Lemma 35 Given a distribution  $p$  with  $\mu_p = 0$  and whose trimming radius for constructing  $p_n^*$  is equal to 1, consider constructing  $q$  according to Definition 4. Recall also the notation for the distribution  $q_{n/3}^*$  which is the  $\left(\frac{1.35}{n}\log \frac{1}{\delta}\right)$ -trimmed version of  $q$  as in Definition 1, and suppose  $q_{n/3}^*$  is formed by trimming  $q$  to some interval  $[\mu_q - r_q, \mu_q + r_q]$ . Assuming that  $\frac{\log \frac{1}{\delta}}{n}$  is upper bounded by some sufficiently small absolute constant, then  $|\mu_q - \mu_{q_{n/3}^*}| \leq 50\epsilon_{n,\delta}(p)$ .
Proof. By the definition of  $q_{n / 3}^{*}$
$$
\left(1 - \frac {1 . 3 5 \log \frac {1}{\delta}}{n}\right) \mu_ {q} + \frac {1 . 3 5 \log \frac {1}{\delta}}{n} \mu_ {q} = \mu_ {q} = \left(1 - \frac {1 . 3 5 \log \frac {1}{\delta}}{n}\right) \mu_ {q _ {n / 3} ^ {*}} + \int_ {\mathbb {R} \backslash [ \mu_ {q} - r _ {q}, \mu_ {q} + r _ {q} ]} x   \mathrm {d} q
$$
Further noting that  $\int_{\mathbb{R}\setminus [\mu_q - r_q,\mu_q + r_q]}\mu_q\mathrm{d}q = \frac{1.35\log\frac{1}{\delta}}{n}\mu_q$ , since we trim exactly  $\frac{1.35\log\frac{1}{\delta}}{n}$  probability mass outside of  $[\mu_q - r_q,\mu_q + r_q]$ , we can rearrange the above to get
$$
\left(1 - \frac {1 . 3 5 \log \frac {1}{\delta}}{n}\right) \left(\mu_ {q} - \mu_ {q _ {n / 3} ^ {*}}\right) = \int_ {\mathbb {R} \backslash [ \mu_ {q} - r _ {q}, \mu_ {q} + r _ {q} ]} \left(x - \mu_ {q}\right) d q
$$
As a result, to bound  $|\mu_q - \mu_{q_{n / 3}^*}|$ , it suffices to bound  $\left|\int_{\mathbb{R}\setminus [\mu_q - r_q,\mu_q + r_q]}(x - \mu_q)\mathrm{d}q\right|$ .
By the triangle inequality, we have that
$$
\begin{array}{l} \left| \int_ {\mathbb {R} \backslash [ \mu_ {q} - r _ {q}, \mu_ {q} + r _ {q} ]} (x - \mu_ {q}) \mathrm {d} q \right| = \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} (x - \mu_ {q}) \mathrm {d} q + \int_ {- 1} ^ {1} (x - \mu_ {q}) \mathrm {d} q - \int_ {\mu_ {q} - r _ {q}} ^ {\mu_ {q} + r _ {q}} (x - \mu_ {q}) \mathrm {d} q \right| \\ \leq \left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} (x - \mu_ {q}) \mathrm {d} q \right| + \left| \int_ {- 1} ^ {1} (x - \mu_ {q}) \mathrm {d} q - \int_ {\mu_ {q} - r _ {q}} ^ {\mu_ {q} + r _ {q}} (x - \mu_ {q}) \mathrm {d} q \right| \\ \end{array}
$$
and we bound the two terms separately.
By Lemmas 33 and 34, we have  $\left|\int_{\mathbb{R}\setminus [-1,1]}(x - \mu_q)\mathrm{d}q\right|\leq 5|\mu_p - \mu_{p_n^*}| + 5\sigma_{p_n^*}\sqrt{\frac{\log\frac{1}{\delta}}{n}}$  in either case of the construction of  $q$ .
To apply Lemma 32, we need to verify that the construction of  $q$  is such that  $|\mu_q - \mu_p| \leq 1/4$  when the trimming radius for constructing  $p_n^*$  from  $p$  is 1. This was shown in Lemmas 30 and 31. Therefore, by the two cases of Lemma 32, we have  $\left|\int_{-1}^{1}(x - \mu_q)\mathrm{d}q - \int_{\mu_q - r_q}^{\mu_q + r_q}(x - \mu_q)\mathrm{d}q\right| \leq \max \left(|\mu_p - \mu_q| + \sqrt{\frac{3}{5}}\sigma_{p_n^*}\sqrt{\frac{4.5\log\frac{1}{\delta}}{n}}, 2\epsilon_{n,\delta}(p)\right)$ . From Lemmas 30 and 31 again, we have that  $|\mu_p - \mu_q| \leq \epsilon_{n,\delta}(p)$ .
Combining all these bounds, we have
$$
\begin{array}{l} \left| \mu_ {q} - \mu_ {q _ {n / 3} ^ {*}} \right| \leq \frac {1}{1 - \frac {1 . 3 5 \log \frac {1}{\delta}}{n}} \left(\left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} (x - \mu_ {q})   \mathrm {d} q \right| + \left| \int_ {- 1} ^ {1} (x - \mu_ {q})   \mathrm {d} q - \int_ {\mu_ {q} - r _ {q}} ^ {\mu_ {q} + r _ {q}} (x - \mu_ {q})   \mathrm {d} q \right|\right) \\ \leq 2 \left(\left| \int_ {\mathbb {R} \backslash [ - 1, 1 ]} (x - \mu_ {q}) \mathrm {d} q \right| + \left| \int_ {- 1} ^ {1} (x - \mu_ {q}) \mathrm {d} q - \int_ {\mu_ {q} - r _ {q}} ^ {\mu_ {q} + r _ {q}} (x - \mu_ {q}) \mathrm {d} q \right|\right) \\ \leq 1 0 \left| \mu_ {p} - \mu_ {p _ {n} ^ {*}} \right| + 1 0 \sigma_ {p _ {n} ^ {*}} \sqrt {\frac {\log \frac {1}{\delta}}{n}} + 2 \max  \left(\epsilon_ {n, \delta} (p) + \sqrt {\frac {3}{5}} \sigma_ {p _ {n} ^ {*}} \sqrt {\frac {4 . 5 \log \frac {1}{\delta}}{n}}, 2 \epsilon_ {n, \delta} (p)\right) \\ \leq 5 0 \epsilon_ {n, \delta} (p) \\ \end{array}
$$
where the last inequality uses the definition of  $\epsilon_{n,\delta}(p) = |\mu_p - \mu_{p_n^*}| + \sigma_{p_n^*}\sqrt{\frac{4.5\log\frac{1}{\delta}}{n}}.$
Lemma 36 Given a distribution  $p$  with  $\mu_p = 0$  and whose trimming radius for constructing  $p_n^*$  is equal to 1, consider any distribution  $q$  such that  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$  and  $|\mu_q| \leq \frac{1}{4}$ . Recall the notation for the distribution  $q_{n/3}^*$  which is the  $\left(\frac{1.35}{n}\log \frac{1}{\delta}\right)$ -trimmed version of  $q$  as in Definition 1, and suppose  $q_{n/3}^*$  is formed by trimming  $q$  to some interval  $[\mu_q - r_q, \mu_q + r_q]$ . Assuming that  $\frac{\log \frac{1}{\delta}}{n}$  is upper bounded by some sufficiently small absolute constant, then  $\sigma_{q_{n/3}^*} \leq 50(\sigma_{p_n^*} + |\mu_p - \mu_{p_n^*}|)$ .
Proof. Without loss of generality, assume that  $\mu_q \geq 0$ . In the following proof, we will aim to upper bound  $\mathbb{E}_{X \leftarrow q^*}[X^2]$  by  $57((\sigma_{p_n^*})^2 + (\mu_{p_n^*})^2)$ , which implies  $\sigma_{q_{n/3}^*} \leq \sqrt{\mathbb{E}_{X \leftarrow q^*}[X^2]} \leq \sqrt{57}\sqrt{(\sigma_{p_n^*})^2 + (\mu_{p_n^*})^2} \leq 8(\sigma_{p_n^*} + |\mu_{p_n^*}|)$ , yielding the lemma statement.
Before we bound  $\mathbb{E}_{X\leftarrow q^{*}}[X^{2}]$ , we first show that by the assumption of  $\mu_q\geq 0$ , it must be the case that  $\mu_{q} - r_{q}\geq -1$
Recall that  $p$  has a total of  $\frac{0.45\log\frac{1}{\delta}}{n}$  mass outside  $[-1,1]$  by the lemma assumption, and that  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$ . This implies that  $q$  has at most  $\frac{0.9\log\frac{1}{\delta}}{n}$  outside  $[-1,1]$ . Since  $\frac{1.35\log\frac{1}{\delta}}{n}$  mass is trimmed from  $q$  to construct  $q_{n/3}^*$ , there is thus at least  $\frac{0.45\log\frac{1}{\delta}}{n}$  mass trimmed from  $q$  inside the interval  $[-1,1]$ .
Since  $\mu_q \geq \mu_p = 0$ , it must be the case that  $\mu_q - r_q \geq -1$ ; otherwise  $r_q \geq 1$  and hence  $\mu_q + r_q \geq 1$ , which would mean that  $[\mu_q - r_q, \mu_q + r_q]$  completely contains  $[-1, 1]$ , contradicting the fact that  $q$  trims non-zero mass from the interval  $[-1, 1]$ .
We will now start bounding  $\mathbb{E}_{X\leftarrow q^{*}}[X^{2}]$
$$
\begin{array}{l} \underset {X \leftarrow q _ {n / 3} ^ {*}} {\mathbb {E}} \left[ X ^ {2} \right] = \int_ {\mu_ {q} - r _ {q}} ^ {\mu_ {q} + r _ {q}} x ^ {2} \mathrm {d} q _ {n / 3} ^ {*} \\ = \frac {1}{1 - \frac {1 . 3 5 \log {\frac {1}{\delta}}}{n}} \int_ {\mu_ {q} - r _ {q}} ^ {\mu_ {q} + r _ {q}} x ^ {2} \mathrm {d} q \\ \leq \frac {2}{1 - \frac {1 . 3 5 \log \frac {1}{\delta}}{n}} \int_ {\mu_ {q} - r _ {q}} ^ {\mu_ {q} + r _ {q}} x ^ {2} d p \quad \text {s i n c e} \frac {d q}{d p} \leq 2 \\ \leq \frac {2}{1 - \frac {1 . 3 5 \log {\frac {1}{\delta}}}{n}} \int_ {- 1} ^ {1} x ^ {2} \mathrm {d} p + \frac {2}{1 - \frac {1 . 3 5 \log {\frac {1}{\delta}}}{n}} \int_ {1} ^ {\max (1, \mu_ {q} + r _ {q})} x ^ {2} \mathrm {d} p \\ \end{array}
$$
We can bound the two integrals separately. First:
$$
\begin{array}{l} \int_ {- 1} ^ {1} x ^ {2} \mathrm {d} p \leq \int_ {- 1} ^ {1} x ^ {2} \mathrm {d} p _ {n} ^ {*} \quad \text {s i n c e} p _ {n} ^ {*} \text {i s s c a l e d u p a f t e r t r i m m i n g} p \text {t o r e n o r m a l i z e} \\ = \left(\sigma_ {p _ {n} ^ {*}}\right) ^ {2} + \left(\mu_ {p _ {n} ^ {*}}\right) ^ {2} \\ \end{array}
$$
Second, we will bound  $\int_{1}^{\max(1, \mu_q + r_q)} x^2 \, \mathrm{d}p$ . If  $\mu_q + r_q \leq 1$  then the integral is 0. Otherwise, we have  $\mu_q + r_q > 1$ , and we have to bound the above integral, using the following bounds on  $r_q$ ,  $\mu_q$  and the second moment of  $p_n^*$ .
Consider the mass of  $q$  that was trimmed from within  $[-1, 1]$ . Since  $\mu_q + r_q > 1$ , the (at least  $\frac{0.45\log\frac{1}{\delta}}{n}$ ) mass that was trimmed from  $q$  must be within  $[-1, \mu_q - r_q]$ . Further recall that  $\frac{\mathrm{d}q}{\mathrm{d}p} \leq 2$  which implies that  $p$  has at least  $\frac{0.225\log\frac{1}{\delta}}{n}$  mass within  $[-1, \mu_q - r_q]$ . Since the trimming interval for constructing  $p_n^*$  from  $p$  is  $[-1, 1]$ , and  $p_n^*$  is constructed from scaling up the trimmed version of  $p$ , we conclude that  $p_n^*$  also has at least  $\frac{0.225\log\frac{1}{\delta}}{n}$  mass within  $[-1, \mu_q - r_q]$ .
As we assumed that  $\mu_q \leq \frac{1}{4}$  in the lemma statement, we have from  $\mu_q + r_q > 1$  that  $r_q \geq \frac{3}{4}$ . This furthermore implies that  $|\mu_q - r_q| \geq \frac{1}{2}$ . Thus the  $\geq \frac{0.225\log\frac{1}{\delta}}{n}$  probability mass of  $p_n^*$  in  $[-1, \mu_q - r_q]$  contributes at least  $\frac{1}{2^2} \frac{0.225\log\frac{1}{\delta}}{n}$  to the second moment of  $p_n^*$ ; namely  $\mathbb{E}_{X \leftarrow p_n^*}[X^2] \geq \frac{0.225}{4} \frac{\log\frac{1}{\delta}}{n}$ .
We can also bound  $r_q \leq \frac{5}{4}$ , since  $\mu_q - r_q > -1$  and  $\mu_q \leq \frac{1}{4}$ ; thus  $\mu_q + r_q \leq \frac{3}{2}$ .
$$
\begin{array}{l} \int_ {1} ^ {\mu_ {q} + r _ {q}} x ^ {2} \mathrm {d} p \leq (\mu_ {q} + r _ {q}) ^ {2} \int_ {1} ^ {\mu_ {q} + r _ {q}} \mathrm {d} p \\ \leq \frac {9}{4} \int_ {1} ^ {\mu_ {q} + r _ {q}} \mathrm {d} p \\ \leq \frac {9}{4} \cdot \frac {0 . 4 5 \log \frac {1}{\delta}}{n} \quad \text {s i n c e} \int_ {- 1} ^ {1} d p = 1 - \frac {0 . 4 5 \log \frac {1}{\delta}}{n} \\ \leq 1 8 \underset {X \leftarrow p _ {n} ^ {*}} {\mathbb {E}} \left[ X ^ {2} \right] \quad \text {u s i n g t h e p r i o r b o u n d o n t h e s e c o n d m o m e n t} \\ = 1 8 \left(\left(\sigma_ {p _ {n} ^ {*}}\right) ^ {2} + \left(\mu_ {p _ {n} ^ {*}}\right) ^ {2}\right) \\ \end{array}
$$
Summarizing, when  $\frac{\log\frac{1}{\delta}}{n}$  is bounded by some sufficiently small absolute constant, we have
$$
\begin{array}{l} \underset {X \leftarrow q _ {n / 3} ^ {*}} {\mathbb {E}} \left[ X ^ {2} \right] \leq \frac {2}{1 - \frac {1 . 3 5 \log \frac {1}{\delta}}{n}} \int_ {- 1} ^ {1} x ^ {2} \mathrm {d} p + \frac {2}{1 - \frac {1 . 3 5 \log \frac {1}{\delta}}{n}} \int_ {1} ^ {\max  (1, \mu_ {q} + r _ {q})} x ^ {2} \mathrm {d} p \\ \leq 3 \cdot (1 + 1 8) \cdot \left(\left(\sigma_ {p _ {n} ^ {*}} ^ {*}\right) ^ {2} + \left(\mu_ {p _ {n} ^ {*}} ^ {*}\right) ^ {2}\right) \\ = 5 7 \left(\left(\sigma_ {p _ {n} ^ {*}}\right) ^ {2} + \left(\mu_ {p _ {n} ^ {*}}\right) ^ {2}\right) \\ \end{array}
$$
which yields the lemma, by the argument at the very beginning of the proof.
Lemma 37 Consider constructing distribution  $q$  from  $p$  according to Definition 4. Assuming that  $\frac{\log\frac{1}{\delta}}{n}$  is upper bounded by some sufficiently small absolute constant, then  $\epsilon_{n/3,\delta}(q) \leq 100\epsilon_{n,\delta}(p)$ .
Proof. Since the construction of  $q$  in Definition 4 satisfies the assumptions of Lemma 36, this lemma follows directly from summing up the bounds of Lemmas 35 and 36, that
$$
\epsilon_ {n / 3, \delta} (q) = | \mu_ {q} - \mu_ {q _ {n / 3} ^ {*}} | + \sigma_ {q _ {n / 3} ^ {*}} \sqrt {\frac {4 . 5 \log \frac {1}{\delta}}{n}} \leq 5 0 \epsilon_ {n, \delta} (p) + 5 0 (\sigma_ {p _ {n} ^ {*}} + | \mu_ {p} - \mu_ {p _ {n} ^ {*}} |) \sqrt {\frac {4 . 5 \log \frac {1}{\delta}}{n}} \leq 1 0 0 \epsilon_ {n, \delta} (p)
$$
where the last inequality uses the definition of  $\epsilon_{n,\delta}(p) = |\mu_p - \mu_{p_n^*}| + \sigma_{p_n^*}\sqrt{\frac{4.5\log\frac{1}{\delta}}{n}}$  and that  $\frac{\log\frac{1}{\delta}}{n}$  is bounded by a sufficiently small constant.