SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
| \mu_ {1} | = \frac {\int_ {| a |} ^ {b} x q (x) d x}{1 - \int_ {a} ^ {b} q (x) d x} \leq \frac {b \int_ {| a |} ^ {b} q (x) d x}{\int_ {- \infty} ^ {a} q (x) d x + \int_ {b} ^ {\infty} q (x) d x} \leq \frac {b (b + a)}{2 \epsilon}
$$
By the tail bound of logconcave distributions (Lemma 11),
$$
| b | <   1 + \ln {\frac {1}{\epsilon}}
$$
$$
| \mu_ {1} | <   \frac {(1 + \ln \frac {1}{\epsilon}) \epsilon^ {s + 1}}{2 \epsilon} <   \epsilon^ {s} \ln (\frac {1}{\epsilon})
$$
Next for  $a \geq 0$ , we can bound  $b$  by  $\epsilon^{s + 1} / 2$  because  $|b| \geq |a|$ . This implies that  $b - a \leq \epsilon^{s + 1} / 2$  which leads to a contradiction that  $\int_{a}^{b} q(x) dx \geq \epsilon$ . Therefore,  $a$  can only be negative in this scenario.
Lemma 32. For  $|\mu_1| \leq r$  with  $r < 1/6$ , we can bound the variance as follows.
$$
\frac {\sigma_ {2} ^ {2}}{1 + 2 e r} \leq \sigma_ {1} ^ {2} \leq \sigma_ {2} ^ {2}
$$
Proof. By Lemma 31, we know  $a < 0$ . We can calculate the variance as
$$
\sigma_ {2} ^ {2} = \frac {\int_ {b} ^ {\infty} x ^ {2} q (x) d x}{\int_ {b} ^ {\infty} q (x) d x}, \quad \sigma_ {1} ^ {2} = \frac {\int_ {b} ^ {\infty} x ^ {2} q (x) d x + \frac {1}{2} \int_ {- a} ^ {b} x ^ {2} q (x) d x}{\int_ {b} ^ {\infty} q (x) d x + \frac {1}{2} \int_ {- a} ^ {b} q (x) d x}
$$
On one hand,
$$
\frac {\int_ {- a} ^ {b} x ^ {2} q (x) d x}{\int_ {b} ^ {\infty} x ^ {2} q (x) d x} \leq \frac {b ^ {2} \int_ {- a} ^ {b} q (x) d x}{b ^ {2} \int_ {b} ^ {\infty} q (x) d x} = \frac {\int_ {- a} ^ {b} q (x) d x}{\int_ {b} ^ {\infty} q (x) d x}
$$
$$
\frac {\sigma_ {1} ^ {2}}{\sigma_ {2} ^ {2}} = \frac {1 + \frac {\int_ {- a} ^ {b} x ^ {2} q (x) d x}{2 \int_ {b} ^ {\infty} x ^ {2} q (x) d x}}{1 + \frac {\int_ {- a} ^ {b} q (x) d x}{2 \int_ {b} ^ {\infty} q (x) d x}} \leq 1
$$
On the other hand, since  $|\mu_1| \leq r$ , by Lemma 8,
$$
r \geq | \mu_ {1} | = \frac {\int_ {- a} ^ {b} x q (x) d x}{\int_ {- a} ^ {b} q (x) d x + 2 \int_ {b} ^ {\infty} q (x) d x} > \frac {\frac {1}{e} \int_ {- a} ^ {b} q (x) d x}{\int_ {- a} ^ {b} q (x) d x + 2 \int_ {b} ^ {\infty} q (x) d x}
$$
$$
\frac {\sigma_ {2} ^ {2}}{\sigma_ {1} ^ {2}} = \frac {1 + \frac {\int_ {- a} ^ {b} q (x) d x}{2 \int_ {b} ^ {\infty} q (x) d x}}{1 + \frac {\int_ {- a} ^ {b} x ^ {2} q (x) d x}{2 \int_ {b} ^ {\infty} x ^ {2} q (x) d x}} \leq 1 + \frac {\int_ {- a} ^ {b} q (x) d x}{2 \int_ {b} ^ {\infty} q (x) d x} \leq 1 + \frac {e r}{1 - e r} <   1 + 2 e r
$$
Lemma 33. The variance  $\sigma_2^2$  is monotonically increasing with respect to  $b$ . Furthermore,  $\sigma_2^2 \geq 1$ .
Proof. By taking the derivative,
$$
\frac {\partial \sigma_ {2} ^ {2}}{\partial b} = - \frac {q (b)}{(\int_ {b} ^ {\infty} q (x) d x) ^ {2}} (b ^ {2} \int_ {b} ^ {\infty} q (x) d x - \int_ {b} ^ {\infty} x ^ {2} q (x) d x) > 0
$$
So for  $b > 0$
$$
\sigma_ {2} ^ {2} \geq \frac {\int_ {0} ^ {\infty} x ^ {2} q (x) d x}{\int_ {0} ^ {\infty} q (x) d x} = 1
$$
Lemma 34 (Approximation for Re-weighted Second Moment). For  $|a + b| \leq \epsilon^5$ , by choosing  $\alpha = -c\epsilon^2$ , then for some constant  $C' > 0$ , we have the following inequalities.
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - C ^ {\prime} \epsilon^ {5} \leq \int_ {b} ^ {\infty} e ^ {\alpha \left(y - \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y \tag {C.4}
$$
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - C ^ {\prime} \epsilon^ {5} \leq \int_ {- a} ^ {\infty} e ^ {\alpha \left(y + \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y + C ^ {\prime} \epsilon^ {5} \tag {C.5}
$$
Proof. By Lemma 31, we can bound  $|\mu_1|$  as
$$
| \mu_ {1} | <   \epsilon^ {4} \ln {\frac {1}{\epsilon}} <   \epsilon^ {3}
$$
We begin by showing that  $\int_{b}^{\infty}e^{\alpha y^{2} / \sigma_{2}^{2}}y^{2}q(y)dy$  is close to  $\int_{b}^{\infty}e^{\alpha y^{2} / \sigma_{1}^{2}}y^{2}q(y)dy$ . By Lemma 32,
$$
\frac {\sigma_ {2} ^ {2}}{1 + 2 e \epsilon^ {3}} \leq \sigma_ {1} ^ {2} \leq \sigma_ {2} ^ {2}
$$
Since  $\alpha < 0$ ,  $\alpha y^2 / \sigma_1^2 < \alpha y^2 / \sigma_2^2$  for  $y > 0$ . This implies
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y.
$$
$$
\begin{array}{l} \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \\ \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} (1 + 2 e \epsilon^ {3}) / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y \\ = \int_ {b} ^ {\infty} \left(1 - e ^ {2 e \alpha y ^ {2} \epsilon^ {3} / \sigma_ {2} ^ {2}}\right) e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y \\ \leq \int_ {b} ^ {\infty} \frac {2 e | \alpha | y ^ {2} \epsilon^ {3}}{\sigma_ {2} ^ {2}} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y \\ = \frac {2 e | \alpha | \epsilon^ {3}}{\sigma_ {2} ^ {2}} \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {4} q (y) d y \\ \leq \frac {2 e | \alpha | \epsilon^ {3}}{\sigma_ {2} ^ {2}} \int_ {0} ^ {\infty} y ^ {4} q (y) d y \\ \end{array}
$$
$$
\begin{array}{l} \leq \frac {2 e | \alpha | \epsilon^ {3}}{\sigma_ {2} ^ {2}} * 8 ^ {4} / 2 \\ = \frac {8 ^ {4} e c \epsilon^ {5}}{\sigma_ {2} ^ {2}} \\ \end{array}
$$
The last inequality is implied by Lemma 12. Furthermore, by Lemma 33,  $\sigma_2^2 \geq 1$ . So there exists a constant  $c_1 > 0$  such that
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - c _ {1} \epsilon^ {5} \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y. \tag {C.6}
$$
This also applies for the integral from  $-a$  to  $\infty$ . To be specific,
$$
\int_ {- a} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - c _ {1} \epsilon^ {5} \leq \int_ {- a} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {- a} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y. \tag {C.7}
$$
Next we will show that  $\int_{b}^{\infty}e^{\alpha y^{2} / \sigma_{1}^{2}}y^{2}q(y)dy$  and  $\int_{b}^{\infty}e^{\alpha (y - \mu_1)^2 /\sigma_1^2}y^2 q(y)dy$  are close to each other. Since  $\mu_{1} < 0, \alpha < 0$ , we derive that  $\alpha y^{2} > \alpha (y - \mu_{1})^{2}$ . This implies
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y > \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y
$$
$$
\begin{array}{l} \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y - \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \\ \leq \int_ {b} ^ {\infty} \left(1 - e ^ {- \left(| \alpha | \mu_ {1} ^ {2} + | \alpha \mu_ {1} | y\right) / \sigma_ {1} ^ {2}}\right) e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \\ \leq \frac {| \alpha \mu_ {1} |}{\sigma_ {1} ^ {2}} \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} (y + | \mu_ {1} |) y ^ {2} q (y) d y \\ \leq \frac {| \alpha \mu_ {1} |}{\sigma_ {1} ^ {2}} \int_ {0} ^ {\infty} (y + | \mu_ {1} |) y ^ {2} q (y) d y \\ \leq c _ {2} \epsilon^ {5} \quad \text {f o r s o m e c o n s t a n t} c _ {2} > 0 \\ \end{array}
$$
The last inequality is implied by Lemma 12 and Lemma 33. Combining two inequalities, we get
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y - c _ {2} \epsilon^ {5} \leq \int_ {b} ^ {\infty} e ^ {\alpha \left(y - \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \tag {C.8}
$$
Similarly, we will show the approximation inequality for  $\int_{-a}^{\infty}e^{\alpha (y + \mu_1)^2 /\sigma_1^2}y^2 q(y)dy$ . We will decompose the integral by the summation of the integral on  $[-a,6\ln (1 / \epsilon)]$  and  $(6\ln (1 / \epsilon),\infty)$  respectively. For the first part of the integral,
$$
\begin{array}{l} \left| \int_ {- a} ^ {6 \ln (1 / \epsilon)} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y - \int_ {- a} ^ {6 \ln (1 / \epsilon)} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \right| \\ = \left| \int_ {- a} ^ {6 \ln (1 / \epsilon)} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} \left(e ^ {\frac {\alpha \mu_ {1}}{\sigma_ {1} ^ {2}} (2 y + \mu_ {1})} - 1\right) y ^ {2} q (y) d y \right| \\ \leq \int_ {- a} ^ {- \mu_ {1} / 2} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} \left(1 - e ^ {\frac {\alpha \mu_ {1}}{\sigma_ {1} ^ {2}} (2 y + \mu_ {1})}\right) y ^ {2} q (y) d y + \int_ {- \mu_ {1} / 2} ^ {6 \ln (1 / \epsilon)} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} \left(e ^ {\frac {\alpha \mu_ {1}}{\sigma_ {1} ^ {2}} (2 y + \mu_ {1})} - 1\right) y ^ {2} q (y) d y \\ \end{array}
$$
The first term can be bounded using  $e^{-t} \geq 1 - t$ .
$$
\begin{array}{l} \int_ {- a} ^ {- \mu_ {1} / 2} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} \left(1 - e ^ {\frac {\alpha \mu_ {1}}{\sigma_ {1} ^ {2}} (2 y + \mu_ {1})}\right) y ^ {2} q (y) d y \leq \int_ {- a} ^ {- \mu_ {1} / 2} \frac {\alpha \mu_ {1}}{\sigma_ {1} ^ {2}} (- 2 y - \mu_ {1}) y ^ {2} q (y) d y \\ \leq \int_ {- a} ^ {- \mu_ {1} / 2} | \alpha | | \mu_ {1} | ^ {2} y ^ {2} q (y) d y \leq \epsilon^ {8} \\ \end{array}
$$
The second term can be bounded using the upper limit of the integral. For  $y \leq 6\ln(1/\epsilon)$ ,
$$
e ^ {\frac {\alpha \mu_ {1}}{\sigma_ {1} ^ {2}} (2 y + \mu_ {1})} - 1 \leq e ^ {2 4 \epsilon^ {6} \ln^ {2} (1 / \epsilon)} - 1 \leq 4 0 \epsilon^ {5}
$$
Substituting into the second term, we get
$$
\int_ {- \mu_ {1} / 2} ^ {6 \ln (1 / \epsilon)} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} \left(e ^ {\frac {\alpha \mu_ {1}}{\sigma_ {1} ^ {2}} (2 y + \mu_ {1})} - 1\right) y ^ {2} q (y) d y \leq 4 0 \epsilon^ {5} \int_ {0} ^ {\infty} y ^ {2} q (y) d y = 2 0 \epsilon^ {5}
$$
Combining both terms,
$$
\left| \int_ {- a} ^ {6 \ln (1 / \epsilon)} e ^ {\alpha \left(y + \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y - \int_ {- a} ^ {6 \ln (1 / \epsilon)} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \right| \leq 2 1 \epsilon^ {5} \tag {C.9}
$$
For the remaining part of the integral, we can bound using logconcave distribution's upper bound as in Lemma 13. For any  $t \geq 3$ ,
$$
q (t) \leq q (0) \cdot 2 ^ {- t / 3} <   e ^ {- t / 5}
$$
Then the following holds with some constant  $c_3^\prime >0$
$$
\int_ {6 \ln (1 / \epsilon)} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {6 \ln (1 / \epsilon)} ^ {\infty} y ^ {2} e ^ {- y / 5} d y \leq \left(1 + 6 \ln \left(\frac {1}{\epsilon}\right)\right) e ^ {- 6 \ln \left(\frac {1}{\epsilon}\right)} \leq c _ {3} ^ {\prime} \epsilon^ {5} \tag {C.10}
$$
Similarly, we get
$$
\int_ {6 \ln (1 / \epsilon)} ^ {\infty} e ^ {\alpha (y + \mu_ {2}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq c _ {3} ^ {\prime} \epsilon^ {5} \tag {C.11}
$$
With Equations (C.9), (C.10), (C.11), we get
$$
\left| \int_ {- a} ^ {\infty} e ^ {\alpha \left(y + \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y - \int_ {- a} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \right| \leq c _ {3} \epsilon^ {5} \text {f o r c o n s t a n t} c _ {3} > 0 \tag {C.12}
$$
Combining Equations (C.6), (C.7), (C.8), (C.12), we have
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - (c _ {1} + c _ {2}) \epsilon^ {5} \leq \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y
$$
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y - (c _ {1} + c _ {3}) \epsilon^ {5} \leq \int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y + c _ {3} \epsilon^ {5}
$$
By choosing  $C' = c_1 + c_2 + c_3$ , we prove the lemma.
Lemma 35 (Approximated for Re-weighted Zeroth Moment). By choosing  $\alpha = -c\epsilon^2$ , for some constant  $C' > 0$ , we have the following inequalities.
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} q (y) d y - C ^ {\prime} \epsilon^ {5} \leq \int_ {b} ^ {\infty} e ^ {\alpha \left(y - \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} q (y) d y \tag {C.13}
$$
$$
\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} q (y) d y - C ^ {\prime} \epsilon^ {5} \leq \int_ {- a} ^ {\infty} e ^ {\alpha \left(y + \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} q (y) d y \leq \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} q (y) d y + C ^ {\prime} \epsilon^ {5} \tag {C.14}
$$
The proof follows exactly from the proof of Lemma 34 by replacing  $y^2$  with 1.
Now we are ready to prove Lemma 5.
$$
\underset {x \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x ^ {2}} x ^ {2} = \frac {\int_ {b / \sigma_ {2}} ^ {\infty} e ^ {\alpha x ^ {2}} x ^ {2} \sigma_ {2} q (x \sigma_ {2}) d x}{\int_ {b} ^ {\infty} q (x) d x} = \frac {\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y}{\sigma_ {2} ^ {2} \int_ {b} ^ {\infty} q (x) d x}
$$
$$
\begin{array}{l} \underset {x \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x ^ {2}} x ^ {2} = \frac {\int_ {- \infty} ^ {(a - \mu_ {1}) / \sigma_ {1}} e ^ {\alpha x ^ {2}} x ^ {2} \sigma_ {1} q (x \sigma_ {1} + \mu_ {1}) d x + \int_ {(b - \mu_ {1}) / \sigma_ {1}} ^ {\infty} e ^ {\alpha x ^ {2}} x ^ {2} \sigma_ {1} q (x \sigma_ {1} + \mu_ {1}) d x}{\int_ {- a} ^ {\infty} q (x) d x + \int_ {b} ^ {\infty} q (x) d x} \\ = \frac {\int_ {- \infty} ^ {a} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} (y - \mu_ {1}) ^ {2} q (y) d y + \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} (y - \mu_ {1}) ^ {2} q (y) d y}{\sigma_ {1} ^ {2} \left(\int_ {- a} ^ {\infty} q (x) d x + \int_ {b} ^ {\infty} q (x) d x\right)} \\ = \frac {\int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} (y + \mu_ {1}) ^ {2} q (y) d y + \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} (y - \mu_ {1}) ^ {2} q (y) d y}{\sigma_ {1} ^ {2} \left(\int_ {- a} ^ {\infty} q (x) d x + \int_ {b} ^ {\infty} q (x) d x\right)} \\ = \frac {\int_ {- a} ^ {\infty} e ^ {\alpha \left(y + \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y + \int_ {b} ^ {\infty} e ^ {\alpha \left(y - \mu_ {1}\right) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y}{\sigma_ {1} ^ {2} \left(\int_ {- a} ^ {\infty} q (x) d x + \int_ {b} ^ {\infty} q (x) d x\right)} \\ + \frac {2 \mu_ {1}}{\sigma_ {1} ^ {2}} \frac {\int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y q (y) d y - \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y q (y) d y}{\int_ {- a} ^ {\infty} q (x) d x + \int_ {b} ^ {\infty} q (x) d x} \\ + \frac {\mu_ {1} ^ {2}}{\sigma_ {1} ^ {2}} \frac {\int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} q (y) d y + \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} q (y) d y}{\int_ {- a} ^ {\infty} q (x) d x + \int_ {b} ^ {\infty} q (x) d x} \\ \end{array}
$$
The first term is close to  $\underset {x\sim \hat{r}}{\mathbb{E}}e^{\alpha x^2}x^2$  while the second and third terms are close to zero. We first give the bound on the absolute values of last two terms. Since  $\alpha < 0$
$$
\begin{array}{l} \left| \int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y q (y) d y - \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y q (y) d y \right| \\ \leq \int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y q (y) d y + \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y q (y) d y \\ \leq \int_ {- a} ^ {\infty} y q (y) d y + \int_ {b} ^ {\infty} y q (y) d y <   1 \\ \end{array}
$$
$$
\int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} q (y) d y + \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} q (y) d y <   1
$$
By Lemma 32, Lemma 34 and Lemma 35, we have
$$
\begin{array}{l} \frac {\int_ {- a} ^ {\infty} e ^ {\alpha (y + \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y + \int_ {b} ^ {\infty} e ^ {\alpha (y - \mu_ {1}) ^ {2} / \sigma_ {1} ^ {2}} y ^ {2} q (y) d y}{\sigma_ {1} ^ {2} \left(\int_ {- a} ^ {\infty} q (x) d x + \int_ {b} ^ {\infty} q (x) d x\right)} \\ \leq \frac {2 \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y + C ^ {\prime} \epsilon^ {5}}{2 \int_ {b} ^ {\infty} q (y) d y} \cdot \frac {1 + 2 e \epsilon^ {3}}{\sigma_ {2} ^ {2}} \\ = \frac {\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y}{\sigma_ {2} ^ {2} \int_ {b} ^ {\infty} q (y) d y} + \frac {C ^ {\prime} \epsilon^ {5}}{\sigma_ {2} ^ {2} 2 \int_ {b} ^ {\infty} q (y) d y} + \frac {2 e \epsilon^ {3}}{\sigma_ {2} ^ {2}} \frac {2 \int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y + C ^ {\prime} \epsilon^ {5}}{2 \int_ {b} ^ {\infty} q (y) d y} \\ <   \frac {\int_ {b} ^ {\infty} e ^ {\alpha y ^ {2} / \sigma_ {2} ^ {2}} y ^ {2} q (y) d y}{\sigma_ {2} ^ {2} \int_ {b} ^ {\infty} q (y) d y} + c _ {1} \epsilon^ {4} \quad \text {f o r c o n s t a n t} c _ {1} > 0 \\ \end{array}
$$
By combining with the second and third terms, we conclude that for constants  $c_{3}, c_{4} > 0$
$$
\underset {x \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x ^ {2}} x ^ {2} - c _ {3} \epsilon^ {4} <   \underset {x \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x ^ {2}} x ^ {2} <   \underset {x \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x ^ {2}} x ^ {2} + c _ {4} \epsilon^ {4} \tag {C.15}
$$
$$
\underset {x \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x ^ {2}} - c _ {3} \epsilon^ {4} <   \underset {x \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x ^ {2}} <   \underset {x \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x ^ {2}} + c _ {4} \epsilon^ {4} \tag {C.16}
$$
Then, we would like to compute the gap between first and second eigenvalues of the re-weighted second moment of  $\hat{q}$ . We denote  $Q$  as the product of  $\hat{q}$  and  $d - 1$  fold of  $q$ . For  $2 \leq j \leq d$ ,
$$
\underset {x \sim Q} {\mathbb {E}} e ^ {\alpha \| x \| ^ {2}} x _ {1} ^ {2} - \underset {x \sim Q} {\mathbb {E}} e ^ {\alpha \| x \| ^ {2}} x _ {j} ^ {2} = \left(\underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} x _ {1} ^ {2} \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} - \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} x _ {2} ^ {2} \underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}}\right) \left(\underset {x \sim q} {\mathbb {E}} e ^ {\alpha x ^ {2}}\right) ^ {d - 2}
$$
We define  $T(\alpha)$  as follows.
$$
T (\alpha) = \underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} x _ {1} ^ {2} \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} - \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} x _ {2} ^ {2} \underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}}
$$
Recall that  $S(\alpha)$  is defined in Lemma 30.
$$
S (\alpha) = \underset {x _ {1} \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} x _ {1} ^ {2} \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} - \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} x _ {2} ^ {2} \underset {x _ {1} \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}}
$$
We calculate the difference between  $T(\alpha)$  and  $S(\alpha)$  using Equation (C.15) and Equation (C.16).
$$
\begin{array}{l} S (\alpha) - T (\alpha) \\ = \left(\underset {x _ {1} \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} x _ {1} ^ {2} - \underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} x _ {1} ^ {2}\right) \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} - \left(\underset {x _ {1} \sim \hat {r}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} - \underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}}\right) \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} x _ {2} ^ {2} \\ \le c _ {3} \epsilon^ {4} + c _ {4} \epsilon^ {4} \\ \end{array}
$$
By Lemma 30,  $S(\alpha) > C' \epsilon^3$  for constant  $C' > 0$ . So we know  $T(\alpha) > C \epsilon^3$  for  $C > 0$ . Then all proof follows as same as the case when  $a + b = 0$ . We write out the proof for completeness.
For  $2\leq j\leq d$
$$
\begin{array}{l} \underset {x \sim P} {\mathbb {E}} e ^ {\alpha \| x \| ^ {2}} x _ {1} ^ {2} - \underset {x \sim P} {\mathbb {E}} e ^ {\alpha \| x \| ^ {2}} x _ {j} ^ {2} \\ = \left(\underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} x _ {1} ^ {2} \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} - \underset {x _ {2} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {2} ^ {2}} x _ {1} ^ {2} \underset {x _ {1} \sim q} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}}\right) \left(\underset {x \sim q} {\mathbb {E}} e ^ {\alpha x ^ {2}}\right) ^ {d - 2} \\ = T(\alpha)\frac{\underset{x\sim P}{\mathbb{E}}e^{\alpha\|x\|^{2}}x_{1}^{2}}{\underset{x_{1}\sim\hat{q}}{\mathbb{E}}e^{\alpha x_{1}^{2}}x_{1}^{2}\underset{x_{2}\sim q}{\mathbb{E}}e^{\alpha x_{2}^{2}}} \\ \end{array}
$$
$$
\underset {x _ {1} \sim \hat {q}} {\mathbb {E}} e ^ {\alpha_ {3} x _ {1} ^ {2}} x _ {1} ^ {2} \leq \underset {x _ {1} \sim \hat {q}} {\mathbb {E}} x _ {1} ^ {2} = 1, \underset {x _ {2} \sim q} {\mathbb {E}} e ^ {\alpha_ {3} x _ {2} ^ {2}} \leq 1.
$$
Also we have shown that  $T(\alpha_3) > C\epsilon^3$ . So we have
$$
\phi (v) := \underset {x \sim P} {\mathbb {E}} \frac {e ^ {\alpha_ {3} \| x \| ^ {2} v ^ {\top} x x ^ {\top} v}}{v ^ {\top} v}
$$
$$
\phi (e _ {1}) - \phi (e _ {j}) > C \epsilon^ {3} \phi (e _ {1})
$$
$$
\phi (v) = \frac {1}{\sum_ {i = 1} ^ {d} \gamma_ {i} ^ {2}} \mathbb {E} e ^ {\alpha_ {3} \| x \| ^ {2}} \left(\sum_ {i = 1} ^ {d} \gamma_ {i} x _ {i}\right) ^ {2}
$$
$$
\begin{array}{l} = \frac {1}{\sum_ {i = 1} ^ {d} \gamma_ {i} ^ {2}} \left(\sum_ {i = 1} ^ {d} \gamma_ {i} ^ {2} \phi \left(e _ {i}\right) + 2 \mathbb {E} e ^ {\alpha \| x \| ^ {2}} \sum_ {i \neq j} \gamma_ {i} \gamma_ {j} x _ {i} x _ {j}\right) \\ = \frac {1}{\sum_ {i = 1} ^ {d} \gamma_ {i} ^ {2}} \left(\sum_ {i = 1} ^ {d} \gamma_ {i} ^ {2} \phi \left(e _ {i}\right) + 2 \sum_ {i \neq j} \gamma_ {i} \gamma_ {j} \mathbb {E} e ^ {\alpha_ {3} \sum_ {k \neq i, j} x _ {k} ^ {2}} \mathbb {E} e ^ {\alpha_ {3} \langle x, e _ {i} \rangle^ {2}} x _ {1} \mathbb {E} e ^ {\alpha \langle x, e _ {j} \rangle^ {2}} x _ {j}\right) \\ = \frac {\sum_ {i = 1} ^ {d} \gamma_ {i} ^ {2} \phi (e _ {i})}{\sum_ {i = 1} ^ {d} \gamma_ {i} ^ {2}} \\ \leq \phi (e _ {1}) \\ \end{array}
$$
This shows that the top eigenvalue of  $\tilde{\Sigma}$  is  $\lambda_1 = \max_v g(v) = g(e_1)$ . In other words, the top eigenvector is  $e_1$ . Similarly the second eigenvalue of  $\tilde{\Sigma}$  is  $\lambda_2 = \max_{v:v\perp e_1}\phi (v) = \phi (e_j), 2\leq j\leq d$ . So we get