SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\lesssim \frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} |}{d} \left| w _ {t} ^ {\prime} - w _ {t} \right|
$$
(B.c.  $\hat{\sigma}_{4,d}^2\lesssim \hat{\sigma}_{2,d}^2$  and  $|D_4|\lesssim \gamma_4\lesssim \gamma_2^2\lesssim |D_2|$  for  $t\leq T_{\mathbb{R}}$  (Lemma D.8))
Using the same argument with Lemma I.16, we can bound the fourth term in Eq. (D.107):
$$
\left| \left(w _ {t} ^ {\prime}\right) ^ {2} Q _ {t} \left(w _ {t} ^ {\prime}\right) - w _ {t} ^ {2} Q _ {t} \left(w _ {t}\right) \right| \lesssim \frac {\hat {\sigma} _ {2 , d} ^ {2} \left| D _ {2 , t} \right|}{d} \left| w _ {t} ^ {\prime} - w _ {t} \right| \tag {D.114}
$$
Combining the bounds we obtained for all the terms of Eq. (D.107), we obtain
$$
\begin{array}{l} \frac {d}{d t} \left(w _ {t} ^ {\prime} - w _ {t}\right) \geq 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \left(w _ {t} ^ {\prime} - w _ {t}\right) - O \left(\hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \left(w _ {R, t} ^ {2} + 1 / d\right) \left(w _ {t} ^ {\prime} - w _ {t}\right)\right) (D.115) \\ = \left(2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | - O \left(\hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \left(w _ {R, t} ^ {2} + 1 / d\right)\right)\right) \left(w _ {t} ^ {\prime} - w _ {t}\right) (D.116) \\ \end{array}
$$
$$
\frac {\frac {d}{d t} \left(w _ {t} ^ {\prime} - w _ {t}\right)}{w _ {t} ^ {\prime} - w _ {t}} \geq 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| - O \left(\hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \left(w _ {R, t} ^ {2} + 1 / d\right)\right) \tag {D.117}
$$
First, let us consider how much  $w_{t}^{\prime} - w_{t}$  grows during Phase 1. At this point, we apply Lemma D.4 with  $\iota_{2} = \iota_{\mathrm{U}}$  and  $0 < \iota_{1} < \iota_{\mathrm{U}}$ , with  $\delta = \frac{1}{\log d}$ . Let us first verify that the assumptions of Lemma D.4 are satisfied. By Lemma D.5,  $F_{2} \asymp \frac{\sqrt{d}}{(\log d)^{2}}$ , using the notation of Lemma D.4. Thus,  $\log F_{2} \leq \log d$ , meaning  $\delta \leq \frac{1}{\log F_{2}}$ . By Assumption 3.2,  $\delta \leq \frac{\gamma_{2}^{2}}{16}$ , and by Proposition I.5,  $\mathbb{P}(|\iota| > \iota_{\mathrm{U}}) \leq \frac{1}{\log d} \leq \frac{\gamma_{2}^{2}}{16}$  clearly holds. Thus, the assumptions of Lemma D.4 are satisfied. As a consequence of Lemma D.4 (specifically Eq. (D.47)) we obtain
$$
\exp \left(\int_ {0} ^ {T _ {1}} 2 \hat {\sigma} _ {2, d} ^ {2} \mid D _ {2, t} \mid d t\right) \gtrsim F _ {2} \gtrsim \frac {\sqrt {d}}{(\log d) ^ {2}} \tag {D.118}
$$
For  $t \leq T_1$ , we have  $w_{R,t}^2 \leq \delta^2 \leq O\left(\frac{1}{\log d}\right)$  by Definition 4.5 (and because  $w_t(\iota_{\mathrm{R}}) \leq w_t(\iota_{\mathrm{U}})$  by Proposition D.37). Thus,
$$
\begin{array}{l} \frac {w _ {T _ {1}} ^ {\prime} - w _ {T _ {1}}}{w _ {0} ^ {\prime} - w _ {0}} \gtrsim \exp \left(\int_ {0} ^ {T _ {1}} 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \cdot \left(1 - O \left(\frac {1}{\log d}\right)\right) d t\right) \tag {ByEq.(D.117)} \\ \gtrsim \left(\frac {\sqrt {d}}{(\log d) ^ {2}}\right) ^ {1 - O (1 / \log d)} \quad (\text {B y E q .} (D. 1 1 8)) \\ \gtrsim \frac {\sqrt {d}}{(\log d) ^ {2}} \quad (\mathrm {B . c .} d ^ {1 / \log d} \leq O (1)) \\ \end{array}
$$
Additionally, we apply Lemma D.4 with  $\iota_{2} = \iota_{\mathrm{R}}$  (and an arbitrary  $\iota_{1}\in [\iota_{\mathrm{L}},\iota_{\mathrm{R}}]$ ) with  $\delta = \xi = \frac{1}{2\log\log d}$ , and  $t_0 = T_1$  and  $t_1$  being the time  $T_{\mathrm{R}}$  such that  $w_{T_{\mathrm{R}}}(\iota_{\mathrm{R}}) = \xi$ . We verify that the conditions of Lemma D.4 hold. Using the notation of Lemma D.4,  $F_{2} = \frac{w_{T_{\mathrm{R}}}(\iota_{\mathrm{R}})}{w_{T_{1}}(\iota_{\mathrm{R}})} = \xi \kappa (\log d)^{2}$  by Lemma D.7, and thus  $\log F_2\leq \frac{1}{\delta}$ . Again by Assumption 3.2,  $\delta \leq \frac{\gamma_2^2}{16}$ , and by Proposition D.6,  $\mathbb{P}(|\iota | > \iota_{\mathrm{R}})\leq O(\kappa)\leq \frac{\gamma_2^2}{16}$ . Thus, we apply Lemma D.4 to obtain
$$
\exp \left(\int_ {T _ {1}} ^ {T} 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | d t\right) \gtrsim F _ {2} \gtrsim \xi \kappa (\log d) ^ {2} \tag {D.119}
$$
For  $t \leq T_{\mathbb{R}}$ ,  $w_{R,t}^{2} \leq \xi^{2} \leq \delta^{2}$ , meaning that
$$
\begin{array}{l} \frac {w _ {T _ {\mathrm {R}}} ^ {\prime} - w _ {T _ {\mathrm {R}}}}{w _ {T _ {1}} ^ {\prime} - w _ {T _ {1}}} \gtrsim \exp \left(\int_ {T _ {1}} ^ {T _ {\mathrm {R}}} 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \cdot (1 - O (\delta^ {2})) d t\right) \quad \text {(B y E q . (D . 1 1 7))} \\ \gtrsim \left(\xi \kappa (\log d) ^ {2}\right) ^ {1 - O \left(\delta^ {2}\right)} \quad \text {(B y E q . (D . 1 1 9))} \\ \gtrsim \xi \kappa (\log d) ^ {2} \quad \left(\mathrm {B . c .} (\log d) ^ {O (\delta)} = (\log d) ^ {O \left(\frac {1}{\log \log d}\right)} \leq O (1)\right) \\ \end{array}
$$
In summary, since  $w_{t}^{\prime} - w_{t}$  grows by  $\frac{\sqrt{d}}{(\log d)^2}$  (up to a constant factor) from time 0 to time  $T_{1}$ , and by  $\xi \kappa (\log d)^2$  (up to a constant factor) from time  $T_{1}$  to time  $T_{\mathrm{R}}$ , this means
$$
\frac {w _ {T _ {\mathrm {R}}} ^ {\prime} - w _ {T _ {\mathrm {R}}}}{w _ {0} ^ {\prime} - w _ {0}} \gtrsim \xi \kappa \sqrt {d} \tag {D.120}
$$
from time 0 to time  $T_{\mathrm{R}}$ , as desired.
We now use the previous lemma to obtain a lower bound on  $|\Phi(w) - \Phi(w')|$  by using the bi-Lipschitzness of  $\Phi$ .
Lemma D.16 (Initial Large Distance Between  $w$  and  $w'$ ). Suppose we are in the setting of Theorem 3.3. Let  $T_{\mathbb{R}}$  be as in the statement of Lemma D.7. If  $\iota, \iota'$  are sampled independently from  $\rho_0$ , then with probability at least  $1 - O(\epsilon)$ ,
$$
\left| \Phi \left(w _ {T _ {\mathrm {R}}} (\iota)\right) - \Phi \left(w _ {T _ {\mathrm {R}}} \left(\iota^ {\prime}\right)\right) \right| \gtrsim \xi \kappa^ {3} \tag {D.121}
$$
Proof of Lemma D.16. By Eqs. (1.16) and (1.18) of Atkinson and Han [12], the probability density function of  $\iota$  is  $p(\iota) = \frac{\Gamma(d / 2)}{\sqrt{\pi}\Gamma((d - 1) / 2)} (1 - \iota^2)^{\frac{d - 3}{2}}$ . Thus, for  $\iota \in [\iota_{\mathrm{L}},\iota_{\mathrm{R}}]$ , the conditional density function  $p(\iota \mid \iota \in [\iota_{\mathrm{L}},\iota_{\mathrm{R}}])$  can be upper bounded as
$$
\begin{array}{l} p \left(\iota \mid \iota \in \left[ \iota_ {\mathrm {L}}, \iota_ {\mathrm {R}} \right]\right) \lesssim \frac {p (\iota)}{p \left(\iota \in \left[ \iota_ {\mathrm {L}} , \iota_ {\mathrm {R}} \right]\right)} (D.122) \\ \lesssim \frac {\Gamma (d / 2)}{\sqrt {\pi} \Gamma ((d - 1) / 2)} \cdot \frac {\left(1 - \iota^ {2}\right) ^ {\frac {d - 3}{2}}}{p \left(\iota \in \left[ \iota_ {\mathrm {L}}, \iota_ {\mathrm {R}} \right]\right)} (D.123) \\ \lesssim \frac {\Gamma (d / 2)}{\sqrt {\pi} \Gamma ((d - 1) / 2)} \cdot \frac {(1 - \iota^ {2}) ^ {\frac {d - 3}{2}}}{1 - O (\kappa)} \quad \text {(B y} \\ \lesssim \sqrt {d} \quad (\text {B y S t i r l i n g ' s F o r m u l a}) \\ \end{array}
$$
In particular, for any interval  $I$  of length at most  $\frac{\kappa^2}{\sqrt{d}}$ ,
$$
\mathbb {P} \left(\iota \in I \mid \iota \in \left[ \iota_ {\mathrm {L}}, \iota_ {\mathrm {R}} \right]\right) \lesssim \sqrt {d} \cdot | I | \lesssim \kappa^ {2} \tag {D.124}
$$
and from this it follows that
$$
\mathbb {P} _ {\iota , \iota^ {\prime} \sim \rho_ {0}} \left(| \iota - \iota^ {\prime} | \leq \frac {\kappa^ {2}}{\sqrt {d}} \mid \iota , \iota^ {\prime} \in [ \iota_ {\mathrm {L}}, \iota_ {\mathrm {R}} ]\right) \lesssim \kappa^ {2} \tag {D.125}
$$
Next, suppose  $\iota, \iota' \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$  with  $\iota' > \iota$ , and  $|\iota - \iota'| \geq \frac{\kappa^2}{\sqrt{d}}$ . Then, by Lemma D.15, if  $T_{\mathrm{R}}$  is the time  $T$  mentioned in the statement of Lemma D.7, then
$$
w _ {T _ {\mathrm {R}}} \left(\iota^ {\prime}\right) - w _ {T _ {\mathrm {R}}} (\iota) \gtrsim \xi \kappa^ {3} \tag {D.126}
$$
Suppose we sample  $\iota, \iota'$  independently from  $\rho_0$ . By Proposition D.6, with probability at least  $1 - O(\kappa)$ ,  $\iota, \iota' \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$ . Thus, by Eq. (D.125), the probability that  $\iota, \iota' \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$  and  $|\iota - \iota'| \geq \frac{\kappa^2}{\sqrt{d}}$  is at least  $1 - O(\kappa)$ . In summary, if we sample  $\iota, \iota'$  independently from  $\rho_0$ , then with probability  $1 - O(\kappa)$ ,
$$
\left| \Phi \left(w _ {T _ {\mathrm {R}}} (\iota)\right) - \Phi \left(w _ {T _ {\mathrm {R}}} \left(\iota^ {\prime}\right)\right) \right| \gtrsim \xi \kappa^ {3} \tag {D.127}
$$
by Lemma D.14, as desired.
Next, we show that during Phase 3, Case 1, at any time, there is a significant fraction of particles whose distance from 0 or 1 can be bounded below.
Lemma D.17. In the setting of Theorem 3.3, suppose  $D_{4,t} \leq 0$  and  $D_{2,t} \geq 0$ . Then,
$$
\mathbb {P} _ {w \sim \rho_ {t}} \left(\left| w \right| \in \left[ \gamma_ {2} ^ {3 / 4}, 1 - \gamma_ {2} ^ {1 / 2} \right]\right) \geq \frac {\gamma / 2}{2} \tag {D.128}
$$
Proof of Lemma D.17. For convenience, we define  $\tau = \sqrt{\gamma_2}$  and  $\beta \geq 1.1$  such that  $\gamma_4 = \beta \tau^4$ . (Here,  $\beta$  and  $\tau$  are well-defined by Assumption 3.2.) First, since  $D_{2,t} \geq 0$  during Phase 3 Case 1, we know that
$$
\underset {w _ {t} \sim \rho_ {t}} {\mathbb {E}} \left[ P _ {2, d} (w) \right] \geq \tau^ {2} \quad (\text {B y})
$$
and since  $D_{4,t} \leq 0$  during Phase 3 Case 1, we know that
$$
\underset {w _ {t} \sim \rho_ {t}} {\mathbb {E}} \left[ P _ {4, d} (w) \right] \leq \beta \tau^ {4} \quad (\text {B y})
$$
Rearranging using  $P_{2,d}(t) = \frac{d}{d - 1} t^2 -\frac{1}{d - 1}$ , we obtain
$$
\frac {d}{d - 1} \underset {w _ {t} \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} ] - \frac {1}{d - 1} \geq \tau^ {2} \tag {D.129}
$$
$$
\underset {w _ {t} \sim \rho_ {t}} {\mathbb {E}} \left[ w ^ {2} \right] \geq \frac {d - 1}{d} \tau^ {2} + \frac {1}{d} \tag {D.130}
$$
Similarly, rearranging using  $P_{4,d}(t) = \frac{d^2 + 6d + 8}{d^2 - 1} t^4 - \frac{6d + 12}{d^2 - 1} t^2 + \frac{3}{d^2 - 1}$ , we obtain
$$
\frac {d ^ {2} + 6 d + 8}{d ^ {2} - 1} \underset {w _ {t} \sim \rho_ {t}} {\mathbb {E}} [ w ^ {4} ] - \frac {6 d + 1 2}{d ^ {2} - 1} \underset {w _ {t} \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} ] + \frac {3}{d ^ {2} - 1} \leq \beta \tau^ {4} \tag {D.131}
$$
and rearranging gives
$$
\underset {w _ {t} \sim \rho_ {t}} {\mathbb {E}} \left[ w ^ {4} \right] \leq \beta \tau^ {4} + O \left(\frac {1}{d}\right) \tag {D.132}
$$
because  $w \leq 1$  and we can assume  $\beta \tau^4 \leq 1$  due to Assumption 3.2. Suppose that with probability more than  $1 - \frac{\tau^2}{2}$  under  $\rho_t$ ,  $w \notin [\tau^{3/2}, 1 - \tau]$ . Then,
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} ] = \mathbb {P} _ {w \sim \rho_ {t}} (w \geq 1 - \tau) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} \mid w \geq 1 - \tau ] \tag {D.133}
$$
$$
+ \mathbb {P} _ {w \sim \rho_ {t}} (w \in [ \tau^ {3 / 2}, 1 - \tau ]) \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} \mid w \in [ \tau^ {3 / 2}, 1 - \tau ] ] \tag {D.134}
$$
$$
+ \mathbb {P} _ {w \sim \rho_ {t}} (w \leq \tau^ {3 / 2}) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} \mid w \leq \tau^ {3 / 2} ] \tag {D.135}
$$
$$
\leq \mathbb {P} _ {w \sim \rho_ {t}} (w \geq 1 - \tau) + \frac {\tau^ {2}}{2} + \tau^ {3}
$$
(By assumption that  $\mathbb{P}_{w\sim \rho_t}(w\in [\tau^{3 / 2},1 - \tau ])\leq \frac{\tau^2}{2})$
$$
\leq \mathbb {P} _ {w \sim \rho_ {t}} (w \geq 1 - \tau) + \frac {3 \tau^ {2}}{4}
$$
(B.c.  $\tau \leq 1 / 4$  (by Assumption 3.2,  $\tau$  is sufficiently small))
which by Eq. (D.130) implies that
$$
\mathbb {P} _ {w \sim \rho_ {t}} (w \geq 1 - \tau) \geq \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} ] - \frac {3 \tau^ {2}}{4} \geq \frac {d - 1}{d} \tau^ {2} + \frac {1}{d} - \frac {3 \tau^ {2}}{4} = \frac {\tau^ {2}}{4} - \frac {\tau^ {2}}{d} + \frac {1}{d} \geq \frac {\tau^ {2}}{4} \quad (\mathrm {D}. 1 3 6)
$$
where the last inequality is because  $\tau \leq 1$ . On the other hand,
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ w ^ {4} \right] = \mathbb {P} _ {w \sim \rho_ {t}} (w \geq 1 - \tau) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ w ^ {4} \mid w \geq 1 - \tau \right] \tag {D.137}
$$
$$
+ \mathbb {P} _ {w \sim \rho_ {t}} (w \in [ \tau^ {3 / 2}, 1 - \tau ]) \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {4} \mid w \in [ \tau^ {3 / 2}, 1 - \tau ] ] \tag {D.138}
$$
$$
+ \mathbb {P} _ {w \sim \rho_ {t}} (w \leq \tau^ {3 / 2}) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {4} \mid w \leq \tau^ {3 / 2} ] \tag {D.139}
$$
$$
\geq \mathbb {P} _ {w \sim \rho_ {t}} (w \geq 1 - \tau) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {4} \mid w \geq 1 - \tau ] \tag {D.140}
$$
$$
\geq (1 - \tau) ^ {4} \cdot \mathbb {P} _ {w \sim \rho_ {t}} (w \geq 1 - \tau) \tag {D.141}
$$
$$
\geq (1 - \tau) ^ {4} \cdot \frac {\tau^ {2}}{4} \quad (\text {B y E q .} (D. 1 3 6))
$$
$$
\geq \frac {\tau^ {2}}{6 4} \quad (\mathrm {B . c .} \tau \leq 1 / 2 (\text {s e e A s s u m p t i o n 3 . 2}))
$$
By Eq. (D.132), this is a contradiction, since it implies that
$$
\frac {\tau^ {2}}{6 4} \leq \beta \tau^ {4} + O \left(\frac {1}{d}\right) \leq 2 \beta \tau^ {4} \tag {D.142}
$$
where the second inequality is because  $\beta \geq 1.1$  and  $d$  is sufficiently large (see Assumption 3.2). This implies that  $128\beta \tau^2 \geq 1$ , which contradicts Assumption 3.2 (since  $\gamma_2$  is chosen to be sufficiently small). Thus, our original assumption that  $\mathbb{P}_{w \sim \rho_t}(w \in [\tau^{3/2}, 1 - \tau]) \leq \frac{\tau^2}{2}$  is incorrect.
The purpose of the next three lemmas is to show that if Phase 3, Case 1 occurs, i.e. if  $D_{2,T_2} = 0$  and  $D_{4,T_2} < 0$ , then for all  $t \geq T_2$ , we have  $D_{2,t} \geq 0$  and  $D_{4,t} \leq 0$ . First, the following lemma states that if  $D_{4,t}$  is much smaller than  $D_{2,t}$  in absolute value, and  $D_{4,t} < 0$  and  $D_{2,t} > 0$ , then the velocity of the particles  $w \geq 0$  will be significantly negative.
Lemma D.18. Suppose we are in the setting of Theorem 3.3, and suppose  $D_{4,T_2} = 0$  and  $t \geq T_2$  such that  $D_{4,t} \leq 0$  and  $D_{2,t} \geq 0$ . If  $|D_{4,t}| \leq \xi |D_{2,t}|$  where  $\xi = \frac{1}{\log\log d}$ , then for all  $w \in [0,1]$ ,
$$
v _ {t} (w) = - \left(1 - w ^ {2}\right) \cdot \left(1 \pm O (\xi)\right) \cdot 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w \tag {D.143}
$$
In particular, for  $w \in [\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$ ,
$$
v _ {t} (w) \lesssim - \gamma_ {2} ^ {5 / 4} \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \tag {D.144}
$$
Proof. Suppose  $|D_{4,t}| \leq \xi |D_{2,t}|$  where  $\xi = \frac{1}{\log\log d}$ . Let  $P_t$  and  $Q_t$  be as in Lemma 4.4. Then, for any  $w \in [0,1]$ ,
$$
\begin{array}{l} P _ {t} (w) = \left(2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {3}\right) (D.145) \\ = \left(2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w - 4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | w ^ {3}\right) (D.146) \\ = \left(2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w \pm O \left(\xi \hat {\sigma} _ {4, d} ^ {2} | D _ {2, t} | w ^ {3}\right)\right) (D.147) \\ = (1 \pm O (\xi)) \cdot 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w \quad \left(\text {B y A s s u m p t i o n 3 . 2 ,} \hat {\sigma} _ {4, d} ^ {2} \lesssim \hat {\sigma} _ {2, d} ^ {2}\right) \\ \end{array}
$$
Thus, for  $w\in [0,1]$
$$
\begin{array}{l} v _ {t} (w) = - \left(1 - w ^ {2}\right) \left(P _ {t} (w) + Q _ {t} (w)\right) \tag {By Lemma 4.4} \\ = - (1 - w ^ {2}) \left(| P _ {t} (w) | \pm O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} |}{d} w\right)\right) \\ \left(\mathrm {B . c .} \left| D _ {2, t} \right| \geq \xi \left| D _ {4, t} \right|\right. \text {a n d} \hat {\sigma} _ {4, d} ^ {2} \lesssim \hat {\sigma} _ {2, d} ^ {2} \text {b y} \\ = - \left(1 - w ^ {2}\right) \left(\left(1 \pm O (\xi)\right) \cdot 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w \pm O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} |}{d} w\right)\right) \quad \text {(B y E q . (D . 1 4 5))} \\ = - \left(1 - w ^ {2}\right) \cdot \left(1 \pm O (\xi)\right) \cdot 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w \quad \left(\mathrm {B . c .} \xi \geq \frac {1}{d}\right) \\ \end{array}
$$
$$
\begin{array}{l} v _ {t} (w) \lesssim - \left(1 - \left(1 - \gamma_ {2} ^ {1 / 2}\right) ^ {2}\right) \cdot \left(1 - O (\xi)\right) \cdot 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \gamma_ {2} ^ {3 / 4} (D.148) \\ \lesssim - \left(1 - \left(1 - \gamma_ {2} ^ {1 / 2}\right) ^ {2}\right) \cdot \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \gamma_ {2} ^ {3 / 4} \quad (\mathrm {B . c .} \xi = \frac {1}{\log \log d}) \\ \lesssim - \left(2 \gamma_ {2} ^ {1 / 2} - \gamma_ {2}\right) \cdot \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \gamma_ {2} ^ {3 / 4} (D.149) \\ \lesssim - \gamma_ {2} ^ {5 / 4} \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \quad (\mathrm {B . c .} \gamma_ {2} ^ {1 / 2} \geq \gamma_ {2} \text {s i n c e} \gamma_ {2} \leq 1) \\ \end{array}
$$
The next lemma essentially implies that if at some point  $D_{4,t} \leq 0$  and  $D_{2,t} > 0$ , then it will never be the case that  $D_{4,t} > 0$  and  $D_{2,t} > 0$ .
Lemma D.19. Suppose we are in the setting of Theorem 3.3, and suppose  $D_{4,t} = 0$  and  $D_{2,t} \geq 0$  for some  $t \geq 0$ . Then,  $\left.\frac{d}{ds} D_{4,s}\right|_{s=t} \leq 0$ .
Proof. We can calculate that
$$
\frac {d}{d t} D _ {4, t} = \frac {d}{d t} \underset {w \sim \rho_ {t}} {\mathbb {E}} [ P _ {4, d} (w) ] = \underset {w \sim \rho_ {t}} {\mathbb {E}} [ P _ {4, d} ^ {\prime} (w) \cdot v _ {t} (w) ] \tag {D.150}
$$
Recall from Eq. (C.4) that
$$
P _ {4, d} (t) = \frac {d ^ {2} + 6 d + 8}{d ^ {2} - 1} t ^ {4} - \frac {6 d + 1 2}{d ^ {2} - 1} t ^ {2} + \frac {3}{d ^ {2} - 1} \tag {D.151}
$$
meaning that
$$
P _ {4, d} ^ {\prime} (t) = 4 t ^ {3} \pm O \left(\frac {1}{d}\right) \cdot \left(| t ^ {3} | + | t |\right) = 4 t ^ {3} \pm O \left(\frac {1}{d}\right) | t | \tag {D.152}
$$
Observe that for  $t \gtrsim \frac{1}{\sqrt{d}}$ ,  $P_{4,d}'(t) > 0$  and for  $t \lesssim \frac{1}{\sqrt{d}}$ , it may be the case that  $P_{4,d}'(t) < 0$ . In particular,
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ P _ {4, d} ^ {\prime} (w) \cdot v _ {t} (w) \right] = \mathbb {P} \left(\left| w \right| \lesssim 1 / \sqrt {d}\right) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ P _ {4, d} ^ {\prime} (w) \cdot v _ {t} (w) \mid | w | \lesssim 1 / \sqrt {d} \right] \tag {D.153}
$$
$$
+ \mathbb {P} (| w | \gtrsim 1 / \sqrt {d}) _ {w \sim \rho_ {t}} \mathbb {E} [ P _ {4, d} ^ {\prime} (w) \cdot v _ {t} (w) \mid | w | \gtrsim 1 / \sqrt {d} ] \tag {D.154}
$$
$$
\leq O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} |}{d ^ {2}}\right) + \mathbb {P} (| w | \gtrsim 1 / \sqrt {d}) \underset {w \sim \rho_ {t}} {\mathbb {E}} [ P _ {4, d} ^ {\prime} (w) \cdot v _ {t} (w) \mid | w | \gtrsim 1 / \sqrt {d} ]
$$
$$
(\text {B . c .} | w | \lesssim \frac {1}{\sqrt {d}} \text {a n d b y L e m m a D . 1 8})
$$
$$
\leq O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} \left| D _ {2 , t} \right|}{d ^ {2}}\right) + \mathbb {P} \left(\left| w \right| \in \left[ \gamma_ {2} ^ {3 / 4}, 1 - \gamma_ {2} ^ {1 / 2} \right]\right) \tag {D.155}
$$
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ P _ {4, d} ^ {\prime} (w) \cdot v _ {t} (w) \mid | w | \in \left[ \gamma_ {2} ^ {3 / 4}, 1 - \gamma_ {2} ^ {1 / 2} \right] \right] \tag {D.156}
$$
$$
\leq O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} |}{d ^ {2}}\right) - \mathbb {P} (| w | \in [ \gamma_ {2} ^ {3 / 4}, 1 - \gamma_ {2} ^ {1 / 2} ]) \gamma_ {2} ^ {9 / 4} \cdot \gamma_ {2} ^ {5 / 4} \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} |
$$
$$
\left(\mathrm {B . c .} P _ {4, d} ^ {\prime} (w) \gtrsim \gamma_ {2} ^ {9 / 4}, \text {a n d b y L e m m a D . 1 8}\right)
$$
$$
\leq O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} \left| D _ {2 , t} \right|}{d ^ {2}}\right) - \frac {\gamma_ {2}}{2} \cdot \gamma_ {2} ^ {1 4 / 4} \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \tag {By Lemma D.17}
$$
$$
<   0 \quad (\text {B y A s s u m p t i o n} 3. 2 \text {s i n c e} d \text {i s s u f f i c i e n t l y l a r g e})
$$
meaning that  $\frac{d}{dt} D_{4,t} < 0$ , as desired.
The next lemma essentially implies that if at some point  $D_{4,t} \leq 0$  and  $D_{2,t} > 0$ , it will never be the case that  $D_{4,t} \leq 0$  and  $D_{2,t} < 0$ .
Lemma D.20. Suppose we are in the setting of Theorem 3.3, and suppose  $D_{4,t} \leq 0$  and  $D_{2,t} = 0$ , for some  $t \geq 0$ . Then,  $\left.\frac{d}{ds} D_{2,s}\right|_{s=t} \geq 0$ .
Proof. Recall from Eq. (C.4) that  $P_{2,d}(t) = \frac{d}{d - 1} t^2 - \frac{1}{d - 1}$  meaning that  $P_{2,d}'(t) = \frac{2d}{d - 1} t$ . Thus,
$$
\frac {d}{d t} D _ {2, t} = \frac {d}{d t} \underset {w \sim \rho_ {t}} {\mathbb {E}} [ w ^ {2} ] = \frac {d}{d - 1} \underset {w \sim \rho_ {t}} {\mathbb {E}} [ 2 w \cdot v _ {t} (w) ] \tag {D.157}
$$
If  $D_{4,t} \leq 0$  and  $D_{2,t} = 0$ , then for any particle  $w \in [0,1]$ , we can write
$$
\begin{array}{l} v _ {t} (w) = - \left(1 - w ^ {2}\right) \left(P _ {t} (w) + Q _ {t} (w)\right) (By Lemma 4.4) \\ = - \left(1 - w ^ {2}\right) \left(4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {3} \pm O \left(\frac {\hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d}\right) | w |\right) (By Lemma 4.4) \\ = \left(1 - w ^ {2}\right) \left(4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | w ^ {3} \pm O \left(\frac {\hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d}\right) | w |\right) (D.158) \\ \end{array}
$$
where the last equality is because  $D_{4,t} \leq 0$ . Observe that  $v_{t}(w) \geq 0$  for  $w \gtrsim \frac{1}{\sqrt{d}}$ , while it may be the case that  $v_{t}(w) \leq 0$  for  $w \lesssim \frac{1}{\sqrt{d}}$ . Thus,
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} [ 2 w \cdot v _ {t} (w) ] = \mathbb {P} (| w | \lesssim 1 / \sqrt {d}) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} [ 2 w \cdot v _ {t} (w) \mid | w | \lesssim 1 / \sqrt {d} ] \tag {D.159}
$$
$$
+ \mathbb {P} (| w | \gtrsim 1 / \sqrt {d}) \underset {w \sim \rho_ {t}} {\mathbb {E}} [ 2 w \cdot v _ {t} (w) \mid | w | \lesssim 1 / \sqrt {d} ] \tag {D.160}
$$
$$
\geq - O \left(\frac {\hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d ^ {2}}\right) + \mathbb {P} (| w | \gtrsim 1 / \sqrt {d}) \underset {w \sim \rho_ {t}} {\mathbb {E}} [ 2 w \cdot v _ {t} (w) \mid | w | \gtrsim 1 / \sqrt {d} ]
$$
(By bounding Eq. (D.158) when  $|w|\lesssim \frac{1}{\sqrt{d}}$
$$
\geq - O \left(\frac {\hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d ^ {2}}\right) + \mathbb {P} (| w | \in \left[ \gamma_ {2} ^ {3 / 4}, 1 - \gamma_ {2} ^ {1 / 2} \right]) \tag {D.161}
$$
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} [ 2 w \cdot v _ {t} (w) | | w | \in \left[ \gamma_ {2} ^ {3 / 4}, 1 - \gamma_ {2} ^ {1 / 2} \right] ] \tag {D.162}
$$
$$
\gtrsim - O \left(\frac {\hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d ^ {2}}\right) + \mathbb {P} (| w | \in \left[ \gamma_ {2} ^ {3 / 4}, 1 - \gamma_ {2} ^ {1 / 2} \right]) \cdot \gamma_ {2} ^ {9 / 4} \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | \tag {ByEq.(D.158)}
$$
$$
\gtrsim - O \left(\frac {\hat {\sigma} _ {4 , d} ^ {2} \left| D _ {4 , t} \right|}{d ^ {2}}\right) - \frac {\gamma_ {2}}{2} \cdot \gamma_ {2} ^ {9 / 4} \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| \quad (\text {B y})
$$
$\geq 0$  (By Assumption 3.2 since  $d$  is sufficiently large)
Putting all of the previous lemmas together, we obtain the following invariant: a lower bound for  $|\Phi(w) - \Phi(w')|$ , for a large fraction of  $w, w'$ , which holds throughout Phase 3, Case 1. Note that in the proof of this invariant, we need to make use of the fact that once we enter the case where  $D_{2,t} > 0$  and  $D_{4,t} < 0$ , we cannot leave this case. If this is not true, then we cannot use the fact that  $|\Phi(w_t) - \Phi(w'_t)|$  is increasing as a function of  $t$  (Lemma D.13) since if  $D_{4,t} > 0$  at any point, then  $|\Phi(w_t) - \Phi(w'_t)|$  could decrease, and we would not have control over this decrease.