SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Now, we combine Lemma D.25, Lemma D.26 and Lemma D.27 to inductively show that for all  $t \geq T_2$ , we have  $D_{2,t} \leq 0$ ,  $D_{4,t} \geq 0$  and most importantly,  $w_t(\iota)$  can be bounded away from 0 and 1 for  $\iota \in [\iota_{\mathrm{L}}, \iota_{\mathrm{M}}]$ .
Lemma D.28 (Phase 3 Case 2 Invariants). Suppose we are in the setting of Theorem 3.3. Assume  $D_{2,T_2} < 0$  and  $D_{4,T_2} = 0$ , i.e. Phase 3 Case 2 holds. Then, for all  $t \geq T_2$ , the following will hold:
1.  $D_{2,t} \leq 0$  and  $D_{4,t} \geq 0$
2. Let  $\iota_{\mathrm{M}}$  be defined as in Lemma D.25. Then,  $w_{t}(\iota)\geq \xi^{2}\kappa^{2}$  for all  $\iota \in [\iota_{\mathrm{L}},\iota_{\mathrm{M}}]$  and  $w_{t}(\iota_{\mathrm{M}})\leq 1 - \kappa^{4}\xi^{3}$ .
Proof of Lemma D.28. For the purpose of this proof, define
$$
I = \{s \geq T _ {2} \mid D _ {2, s} \leq 0 \text {a n d} D _ {4, s} \geq 0 \} \tag {D.252}
$$
and let  $t_0 = \sup I$ , meaning that for  $t \in [T_2, t_0]$ ,  $D_{2,t} \leq 0$  and  $D_{4,t} \geq 0$ . Assume for the sake of contradiction that  $t_0 < \infty$ . Then, by the continuity of  $D_{2,t}$  and  $D_{4,t}$  as functions of  $t$ ,  $D_{2,t} \leq 0$  and  $D_{4,t} \geq 0$ . Since  $t_0 = \sup I$ , either  $D_{2,t_0} = 0$  or  $D_{4,t_0} = 0$  (and one of these has to be nonzero since otherwise  $t_0 = \infty$ , which is a contradiction). By Lemma D.24 and Lemma D.23, for all  $t \in [T_2, t_0]$  for all  $\iota, \iota' \in [\iota_{\mathrm{L}}, \iota_{\mathrm{M}}]$ ,
$$
\left| \Phi \left(w _ {t} (\iota)\right) - \Phi \left(w _ {t} \left(\iota^ {\prime}\right)\right) \right| \leq 2 \log \frac {1}{\kappa} + \log \frac {1}{\xi} + C \tag {D.253}
$$
where  $C$  is a universal constant. Thus, applying Lemma D.25 with  $s = t_0$ , we find that for all  $t \in [T_2, t_0]$ , for all  $\iota \geq \iota_{\mathrm{L}}$ ,  $w_t(\iota) \geq \xi^2 \kappa^2$ , and furthermore, for all  $t \in [T_2, t_0]$ ,  $w_t(\iota_{\mathbf{M}}) \leq 1 - \kappa^4 \xi^3$ .
To finish the proof, there are two cases to address: either  $D_{2,t_0} < 0$  and  $D_{4,t_0} = 0$ , or  $D_{2,t_0} = 0$  and  $D_{4,t_0} > 0$ . In the first case, by Lemma D.26,  $\left.\frac{d}{dt}D_{4,t}\right|_{t=t_0} > 0$ , meaning that for a sufficiently small open interval of time  $J$  containing  $t_0$ ,  $D_{4,t}$  is increasing on this interval. This gives us a contradiction, since it implies that for some  $t_0' > t_0$  (with  $t_0' - t_0$  sufficiently small)  $D_{4,t}$  is nonnegative on  $[t_0, t_0']$ , and since  $D_{2,t_0} < 0$ , by continuity,  $D_{2,t} < 0$  for  $t \in [t_0, t_0']$  as long as  $t_0' - t_0 > 0$  is sufficiently small. Thus,  $\sup I > t_0$ , which is a contradiction. In the second case, we can apply the same argument using Lemma D.27 in place of Lemma D.26.
In summary, if  $D_{4,T_2} = 0$  and  $D_{2,T_2} < 0$ , then for all  $t \geq T_2$ ,  $D_{2,t} \leq 0$  and  $D_{4,t} \geq 0$ . As a consequence of Lemma D.23, Lemma D.24, and Lemma D.25, for all  $\iota \in [\iota_{\mathrm{L}},\iota_{\mathrm{M}}]$ ,  $w_{t}(\iota) \geq \xi^{2}\kappa^{2}$  and furthermore,  $w_{t}(\iota_{\mathrm{M}}) \leq 1 - \kappa^{4}\xi^{3}$ , for all  $t \geq T_2$ .
We now proceed to bound the running time of Phase 3, Case 2, by dividing it into a few subcases, and showing that the running time that each of these subcases contribute is within our desired bound. The first lemma below bounds the running time spent in the subcase where the positive root  $r$  of  $P_{t}$  is larger than  $\frac{1}{2}$  (and where  $D_{4,t}$  is large relative to  $D_{2,t}$ ).
Lemma D.29. Suppose we are in the setting of Theorem 3.3, and assume that  $D_{4,T_2} = 0$  and  $D_{2,T_2} < 0$ , i.e. we are in Phase 3, Case 2. Then, at most  $\frac{1}{\hat{\sigma}_{2,d}^2\xi^4\kappa^6}\log \left(\frac{\gamma_2}{\epsilon}\right)$  time  $t\geq T_{2}$  elapses such that the following hold:
1. If  $r$  is the positive root of  $P_t$  (as defined in Lemma 4.4) then  $r \geq \frac{1}{2}$ .
2.  $|D_{4,t}| \geq \xi |D_{2,t}|$
3.  $L(\rho_t)\geq (\hat{\sigma}_{2,d}^2 +\hat{\sigma}_{4,d}^2)\epsilon^2$
Proof of Lemma D.29. Let  $t \geq T_2$  such that  $|D_{4,t}| \geq \xi |D_{2,t}|$ . We can write
$$
P _ {t} (w) = 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {3} = 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w (w - r) (w + r) \tag {D.254}
$$
where  $r$  is the positive root of  $P_{t}(w)$ . We are now going to lower bound the expected velocity while  $r \geq \frac{1}{2}$  by lower bounding  $P_{t}(w)$  for a large fraction of  $w$ .
For all  $\iota \geq \iota_{\mathrm{L}}$ , and  $t \geq T_2$ , we have  $w_t(\iota) \geq \xi^2 \kappa^2$ , by Lemma D.28. Thus,  $\mathbb{P}_{w \sim \rho_t}(w \geq \xi^2 \kappa^2) \leq 1 - O(\kappa)$  by Proposition D.6. Additionally, because  $D_{2,t} \leq 0$ , we have  $\mathbb{E}_{w \sim \rho_t}[w^2] \leq \gamma_2 + O(1 / d)$  by Proposition D.38, meaning that by Markov's inequality,  $\mathbb{P}_{w \sim \rho_t}(|w| \geq \frac{1}{3}) \lesssim \gamma_2 + 1 / d \leq \gamma_2$  where the last inequality is because  $d \geq c_3$  by Assumption 3.2. By a union bound,
$$
\mathbb {P} _ {w \sim \rho_ {t}} \left(\left| w \right| \in \left[ \xi^ {2} \kappa^ {2}, 1 / 3 \right]\right) \geq 1 - O \left(\gamma_ {2} + \kappa\right) \tag {D.255}
$$
For particles  $w$  such that  $|w|\in [\xi^2\kappa^2,1 / 3]$ , we can lower bound the velocity if  $r\geq \frac{1}{2}$ :
$$
\begin{array}{l} \left| v _ {t} (w) \right| \gtrsim (1 - w ^ {2}) \left| P _ {t} (w) + Q _ {t} (w) \right| (By Lemma 4.4) \\ \gtrsim | P _ {t} (w) + Q _ {t} (w) | \quad (\mathrm {B . c .} w \leq 1 / 3) \\ \gtrsim | P _ {t} (w) | - | Q _ {t} (w) | (D.256) \\ \gtrsim \left| 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {3} \right| - \left| \lambda_ {d} ^ {(1)} w + \lambda_ {d} ^ {(3)} w ^ {3} \right| (By Lemma 4.4) \\ \gtrsim \left. 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {3} \right| - O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} \left| D _ {2 , t} \right| + \hat {\sigma} _ {4 , d} ^ {2} \left| D _ {4 , t} \right|}{d}\right) | w | \quad (\text {B y}) \\ \gtrsim \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} | w | | w - r | | w + r | - O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d}\right) \quad (\text {B y}) \\ \gtrsim \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \cdot \xi^ {2} \kappa^ {2} \cdot \frac {1}{6} \cdot \frac {1}{2} - O \Big (\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d} \Big) \\ (\mathrm {B . c .} w \in [ \xi^ {2} \kappa^ {2}, 1 / 3 ] \text {a n d} r \geq 1 / 2) \\ \gtrsim \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \xi^ {2} \kappa^ {2} (D.257) \\ \end{array}
$$
where the last inequality is because  $|D_{4,t}| \geq \xi |D_{2,t}|$  and  $d$  is sufficiently large by Assumption 3.2. Thus,
$$
\begin{array}{l} \underset {w \sim \rho_ {t}} {\mathbb {E}} | v _ {t} (w) | ^ {2} \gtrsim \mathbb {P} _ {w \sim \rho_ {t}} \left(| w | \in \left[ \xi^ {2} \kappa^ {2}, 1 / 3 \right]\right) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ | v _ {t} (w) | ^ {2} \mid | w | \in \left[ \xi^ {2} \kappa^ {2}, 1 / 3 \right] \right] \tag {D.258} \\ \gtrsim \left(1 - O \left(\gamma_ {2} + \kappa\right)\right) \cdot \hat {\sigma} _ {4, d} ^ {4} D _ {4, t} ^ {2} \xi^ {4} \kappa^ {4} \quad (\text {B y E q . (D . 2 5 5)}) \\ \stackrel {\sim} {>} \hat {\sigma} _ {4, d} ^ {4} D _ {4, t} ^ {2} \xi^ {4} \kappa^ {4} \quad (\text {B y A s s o m p t i o n 3 . 2 s i n c e} \gamma_ {2}, d \text {s u f f i c i e n t l y s m a l l , l a r g e r e s p .}) \\ \end{array}
$$
$$
\begin{array}{l} \frac {d L \left(\rho_ {t}\right)}{d t} \lesssim - \hat {\sigma} _ {4, d} ^ {4} D _ {4, t} ^ {2} \xi^ {4} \kappa^ {4} (By Lemma D.39) \\ \lesssim - \hat {\sigma} _ {4, d} ^ {4} \left(D _ {4, t} ^ {2} + \xi^ {2} D _ {2, t} ^ {2}\right) \xi^ {4} \kappa^ {4} \quad (\mathrm {B . c .} | D _ {4, t} | \geq \xi | D _ {2, t} |) \\ \lesssim - \hat {\sigma} _ {4, d} ^ {4} \left(D _ {4, t} ^ {2} + D _ {2, t} ^ {2}\right) \xi^ {4} \kappa^ {6} (D.259) \\ \lesssim - \hat {\sigma} _ {4, d} ^ {2} \xi^ {4} \kappa^ {6} L (\rho_ {t}) \quad \left(\mathrm {B . c .} \hat {\sigma} _ {2, d} ^ {2} \asymp \hat {\sigma} _ {4, d} ^ {2} \text {b y A s s u m p t i o n 3 . 2}\right) \\ \end{array}
$$
Since  $L(\rho_0) \asymp \hat{\sigma}_{4,d}^2\gamma_2^2$  by Lemma D.8 and Assumption 3.2, the amount of time spent in the case where  $r(t) \geq \frac{1}{2}, |D_{4,t}| \geq \xi |D_{4,t}|$  and  $L(\rho_t) \geq (\hat{\sigma}_{2,d}^2 + \hat{\sigma}_{4,d}^2)\epsilon^2$  after  $t \geq T_2$  is at most  $\frac{1}{\hat{\sigma}_{2,d}^2\xi^4\kappa^6}\log \left(\frac{\gamma_2}{\epsilon}\right)$  by Gronwall's inequality (Fact I.13).
Next, we deal with the subcase where  $D_{4,t}$  is small relative to  $D_{2,t}$ .
Lemma D.30. Suppose we are in the setting of Theorem 3.3, and  $D_{4,T_2} = 0$  and  $D_{2,T_2} < 0$ , i.e. we are in Phase 3, Case 2. Then, at most  $\frac{1}{\hat{\sigma}_{2,d}^2\xi^4\kappa^4}\log \left(\frac{\gamma_2}{\epsilon}\right)$  time  $t\geq T_2$  elapses such that the following hold:
1.  $|D_{4,t}| \leq \xi |D_{2,t}|$
2.  $L(\rho_t)\geq (\hat{\sigma}_{2,d}^2 +\hat{\sigma}_{4,d}^2)\epsilon^2$
Proof of Lemma D.30. Suppose  $|D_{4,t}| \leq \xi |D_{2,t}|$  at some time  $t \geq T_2$ . Then, we can lower bound the velocity for a large fraction of  $w$ 's, as follows. First, we provide a bound on  $P_t(w)$  (defined in Lemma 4.4):
$$
\begin{array}{l} P _ {t} (w) = 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {3} \quad (\text {B y}) \\ = - 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| w + 4 \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| w ^ {3} \quad (\text {B . c .} D _ {2, t} <   0 \text {a n d} D _ {4, t} > 0 \text {b y L e m m a D . 2 8}) \\ = - 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w + O (\xi \hat {\sigma} _ {4, d} ^ {2} | D _ {2, t} | w ^ {3}) \quad \left(\mathrm {B . c .} | D _ {4, t} | \leq \xi | D _ {2, t} |\right) \\ = - 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w + O \left(\xi \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w ^ {3}\right) \quad \text {(B . c .} \hat {\sigma} _ {2, d} ^ {2} \lesssim \hat {\sigma} _ {4, d} ^ {2} \text {b y A s s u m p t i o n 3 . 2)} \\ = - 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w \cdot (1 - O (\xi)) \tag {D.260} \\ \end{array}
$$
Next, we provide a bound on  $Q_{t}(w)$ :
$$
\begin{array}{l} \left| Q _ {t} (w) \right| = \left| \lambda_ {d} ^ {(1)} w + \lambda_ {d} ^ {(3)} w \right| \quad (\text {B y}) \\ = O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d}\right) | w | \tag {ByLemma4.4} \\ = O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} |}{d}\right) | w | \quad (\mathrm {B . c .} | D _ {4, t} | \leq \xi | D _ {2, t} | \text {a n d} \hat {\sigma} _ {4, d} ^ {2} \lesssim \hat {\sigma} _ {2, d} ^ {2}) \\ = O \left(\xi \hat {\sigma} _ {2, d} ^ {2} \mid D _ {2, t} |\right) \mid w \mid \quad \text {(B . c .} 1 / d \leq \frac {1}{\log \log d} = \xi \text {b y A s s u m p t i o n 3 . 2)} \\ \end{array}
$$
$$
\left| P _ {t} (w) + Q _ {t} (w) \right| \geq 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| w \cdot (1 - O (\xi)) \tag {D.261}
$$
By Lemma D.28, we have  $D_{2,t} \leq 0$ , meaning that by Proposition D.38,
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ w ^ {2} \right] \leq \gamma_ {2} + O (1 / d) \tag {D.262}
$$
Thus, by Markov's inequality and Assumption 3.2,
$$
\mathbb {P} _ {w \sim \rho_ {t}} (| w | \geq 1 / 2) \lesssim \gamma_ {2} \tag {D.263}
$$
Furthermore, by Lemma D.28, for all  $\iota \geq \iota_{\mathrm{L}}$ ,  $w_{t}(\iota) \geq \xi^{2}\kappa^{2}$  for all times  $t \geq T_{2}$ , meaning that  $w_{t} \geq \xi^{2}\kappa^{2}$  with probability at least  $1 - O(\kappa)$  under  $\rho_{t}$ , by Proposition D.6. Thus, by a union bound,
$$
\mathbb {P} _ {w \sim \rho_ {t}} \left(\left| w \right| \in \left[ \xi^ {2} \kappa^ {2}, 1 / 2 \right]\right) \geq 1 - O \left(\gamma_ {2}\right) - O (\kappa) \geq \frac {1}{2} \tag {D.264}
$$
where the last inequality is by Assumption 3.2 since  $\gamma_{2}$  and  $d$  are sufficiently small and large respectively. Thus the average velocity is at least
$$
\underset {w \sim \rho_ {t}} {\mathbb {E}} | v _ {t} (w) | ^ {2} \gtrsim \mathbb {P} _ {w \sim \rho_ {t}} \left(\xi^ {2} \kappa^ {2} \leq | w | \leq 1 / 2\right) \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} [ | v _ {t} (w) | ^ {2} \mid \xi^ {2} \kappa^ {2} \leq | w | \leq 1 / 2 ] \tag {D.265}
$$
$$
\gtrsim \frac {1}{2} \cdot \underset {w \sim \rho_ {t}} {\mathbb {E}} [ | v _ {t} (w) | ^ {2} \mid \xi^ {2} \kappa^ {2} \leq | w | \leq 1 / 2 ] \tag {D.266}
$$
$$
\gtrsim \underset {w \sim \rho_ {t}} {\mathbb {E}} \left[ \left| v _ {t} (w) \right| ^ {2} \mid \xi^ {2} \kappa^ {2} \leq | w | \leq 1 / 2 \right] \tag {D.267}
$$
$$
\gtrsim \mathbb {E} _ {w \sim \rho_ {t}} [ (1 - w ^ {2}) ^ {2} | P _ {t} (w) + Q _ {t} (w) | ^ {2} \mid \xi^ {2} \kappa^ {2} \leq | w | \leq 1 / 2 ] \tag {D.268}
$$
$$
\gtrsim \mathbb {E} _ {w \sim \rho_ {t}} \left[ | P _ {t} (w) + Q _ {t} (w) | ^ {2} \mid \xi^ {2} \kappa^ {2} \leq | w | \leq 1 / 2 \right] \quad (\text {B . c .} | w | \leq 1 / 2)
$$
$$
\gtrsim \mathbb {E} _ {w \sim \rho_ {t}} \left[ \hat {\sigma} _ {2, d} ^ {4} \left| D _ {2, t} \right| ^ {2} w ^ {2} \mid \xi^ {2} \kappa^ {2} \leq | w | \leq 1 / 2 \right] \quad (\text {B y E q . (D . 2 6 1)})
$$
$$
\stackrel {>} {_ \sim} \hat {\sigma} _ {2, d} ^ {4} \left| D _ {2, t} \right| ^ {2} \xi^ {4} \kappa^ {4} \tag {D.269}
$$
Thus, at any time  $t$  where  $|D_{4,t}| \leq \xi |D_{2,t}|$ , we have
$$
\frac {d L \left(\rho_ {t}\right)}{d t} \lesssim - \hat {\sigma} _ {2, d} ^ {4} \left| D _ {2, t} \right| ^ {2} \xi^ {4} \kappa^ {4} \tag {D.270}
$$
$$
\lesssim - \hat {\sigma} _ {2, d} ^ {2} (\hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | ^ {2} + \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | ^ {2}) \xi^ {4} \kappa^ {4}
$$
$$
\left. \right.\left. \right.\left. \right.\left. \right.\left. \right.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\left.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right.\right\rangle_ {d} ^ {2} d _ {1} ^ {2} d _ {2} ^ {2} d _ {3} ^ {2} d _ {4} ^ {2} d _ {5} ^ {2} d _ {6} ^ {2} d _ {7} ^ {2} d _ {8} ^ {2} d _ {9} ^ {2} d _ {1 0} ^ {2} d _ {1 1} ^ {2} d _ {1 2} ^ {2} d _ {1 3} ^ {2} d _ {1 4} ^ {2} d _ {1 5} ^ {2} d _ {1 6} ^ {2} d _ {1 7} ^ {2} d _ {1 8} ^ {2} d _ {1 9} ^ {2} d _ {2 0} ^ {2} d _ {2 1} ^ {2} d _ {2 2} ^ {2} d _ {2 3} ^ {2} d _ {2 4} ^ {2} d _ {2 5} ^ {2} d _ {2 6} ^ {2} d _ {2 7} ^ {2} d _ {2 8} ^ {2} d _ {2 9} ^ {2} d _ {3 0} ^ {2} d _ {3 1} ^ {2} d _ {3 2} ^ {2} d _ {3 3} ^ {2} d _ {\dots}\right)\right\rangle_ {\text {(B . c .)}} \right\}.
$$
$$
\lesssim - \hat {\sigma} _ {2, d} ^ {2} \xi^ {4} \kappa^ {4} L (\rho_ {t}) \tag {D.271}
$$
Since the initial loss is  $L(\rho_0) \lesssim \hat{\sigma}_{2,d}^2\gamma_2^2$  by Lemma D.8, by an argument similar to that used in Lemma D.29, using Gronwall's inequality (Fact I.13), we find that
$$
\int_ {T _ {2}} ^ {\infty} 1 \left(\left| D _ {4, t} \right| \leq \xi \left| D _ {2, t} \right|\right) 1 \left(L \left(\rho_ {t}\right) \geq \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) \epsilon^ {2}\right) d t \lesssim \frac {1}{\hat {\sigma} _ {2 , d} ^ {2} \xi^ {4} \kappa^ {4}} \log \left(\frac {\gamma_ {2}}{\epsilon}\right) \tag {D.272}
$$
i.e. the time spent in the subcase in the statement of this lemma is at most  $\frac{1}{\hat{\sigma}_{2,d}^2\xi^4\kappa^4}\log \left(\frac{\gamma_2}{\epsilon}\right)$ , as desired.
We have already considered the case where  $|D_{4,t}| \geq \xi |D_{2,t}|$  and  $r \geq \frac{1}{2}$ . Next, we must deal with the case where  $|D_{4,t}| \geq \xi |D_{2,t}|$ , but where  $r \leq \frac{1}{2}$ . We further split this case into two subcases. In the next lemma, we deal with the subcase where  $w_t(\iota_{\mathbb{R}}) \geq 1 - \xi$ , meaning a non-negligible portion of the particles are close to 1.
Lemma D.31. Suppose we are in the setting of Theorem 3.3, and  $D_{4,T_2} = 0$  and  $D_{2,T_2} < 0$ , i.e. we are in Phase 3, Case 2. Then, the amount of time  $t \geq T_2$  that elapses while the following conditions simultaneously hold:
1.  $|D_{4,t}| \geq \xi |D_{2,t}|$
2.  $r \leq \frac{1}{2}$  where  $r$  is the positive root of  $P_{t}$
3.  $w_{t}(\iota_{\mathbb{R}})\geq 1 - \xi$
4.  $L(\rho_t)\geq (\hat{\sigma}_{2,d}^2 +\hat{\sigma}_{4,d}^2)\epsilon^2$
is at most  $\frac{1}{\hat{\sigma}_{4,d}^2\epsilon\xi^7\kappa^{12}}\log \left(\frac{\gamma_2}{\epsilon}\right)$
Proof of Lemma D.31. If  $w(\iota_{\mathbb{R}}) \geq 1 - \xi$  at any time, then the subsequent times when it could possibly be moving to towards 1 are:
1. During phase 2 — note that before phase 2,  $w_{t}(\iota_{\mathrm{R}}) \leq \frac{1}{\log d}$ , so it is not possible for  $w_{t}(\iota_{\mathrm{R}})$  to be larger than  $1 - \xi$  since, by Lemma D.9,  $w_{t}(\iota_{\mathrm{R}})$  is increasing during Phase 1.
2. The times  $t \geq T_2$  when  $|D_{4,t}| \leq \xi |D_{2,t}|$
3. The times  $t \geq T_2$  when  $|D_{4,t}| \geq \xi |D_{2,t}|$  and  $r \geq \frac{1}{2}$ , where  $r$  is the positive root of  $P_t(w)$ .
The only case not mentioned above consists of the times  $t \geq T_2$  when  $|D_{4,t}| \geq \xi |D_{2,t}|$  and the positive root  $r$  of  $P_t(w)$  is less than  $\frac{1}{2}$ . In this case,  $w_t(\iota_{\mathbb{R}})$  will be moving away from 1. For this reason, we also do not need to consider the case discussed in Lemma D.32.
Thus, we will use upper bounds on the velocity and time during the first 3 cases listed above, and use a lower bound on the velocity in the last. First note that the velocity can always be upper bounded as follows:
$$
\begin{array}{l} v _ {t} (w) \lesssim (1 - w) (1 + w) \cdot \left| P _ {t} (w) + Q _ {t} (w) \right| (By Lemma 4.4) \\ \lesssim (1 - w) (1 + w) \cdot \left| 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | w + 4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | w ^ {3} + O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d}\right) | w | \right| (By Lemma 4.4) \\ \lesssim (1 - w) \cdot \left(\hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | + \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} |\right) \quad (\mathrm {B . c .} | w | \leq 1) \\ \end{array}
$$
$$
\frac {d}{d t} (1 - w) \gtrsim - \left(\hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| + \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right|\right) (1 - w) \gtrsim - \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) (1 - w) \tag {D.273}
$$
During Phase 2. By Lemma D.10, the running time during Phase 2 is  $T_{2} - T_{1} \lesssim \frac{1}{(\hat{\sigma}_{2,d}^{2} + \hat{\sigma}_{4,d}^{2})\xi^{6}\kappa^{12}} \log \left(\frac{\gamma_{2}}{\epsilon}\right)$ . At the beginning of Phase 2,  $w_{t}(\iota_{\mathbb{R}}) \leq \frac{1}{\log d}$ , by the definition of Phase 1. Thus, by Gronwall's inequality (Fact I.13), at time  $T_{2}$
$$
\begin{array}{l} 1 - w _ {T _ {2}} \left(\iota_ {\mathrm {R}}\right) \geq \left(1 - \frac {1}{\log d}\right) \cdot \exp \left(- \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) \cdot \frac {1}{\left(\hat {\sigma} _ {2 , d} ^ {2} + \hat {\sigma} _ {4 , d} ^ {2}\right) \xi^ {6} \kappa^ {1 2}} \log \left(\frac {\gamma_ {2}}{\epsilon}\right)\right) \\ \text {(B y F a c t I . 1 3 , E q . (D . 2 7 3) a n d b o u n d o n T _ {2} - T _ {1})} \\ \geq \left(1 - \frac {1}{\log d}\right) \cdot \exp \left(- \frac {1}{\xi^ {6} \kappa^ {1 2}} \log \left(\frac {\gamma_ {2}}{\epsilon}\right)\right) \tag {D.274} \\ \end{array}
$$
Times  $t \geq T_2$  when  $|D_{4,t}| \leq \xi |D_{2,t}|$  and  $L(\rho_t) \geq (\hat{\sigma}_{2,d}^2 + \hat{\sigma}_{4,d}^2)\epsilon^2$ . By Lemma D.30, this case contributes at most  $\frac{1}{\hat{\sigma}_{2,d}^2\xi^4\kappa^4}\log \left(\frac{\gamma_2}{\epsilon}\right)$  time to the overall running time. Thus, by Gronwall's inequality (Fact I.13), the additional factor that this case contributes to  $1 - w_{t}(\iota_{\mathrm{R}})$  is larger than or equal to
$$
\exp \left(- \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) \cdot \frac {1}{\hat {\sigma} _ {2 , d} ^ {2} \xi^ {4} \kappa^ {4}} \log \left(\frac {\gamma_ {2}}{\epsilon}\right)\right) \geq \exp \left(- \frac {C}{\xi^ {4} \kappa^ {4}} \log \left(\frac {\gamma_ {2}}{\epsilon}\right)\right) \tag {D.275}
$$
Times  $t \geq T_2$  when  $|D_{4,t}| \geq \xi |D_{2,t}|$ ,  $r(t) \geq \frac{1}{2}$  and  $L(\rho_t) \geq (\hat{\sigma}_{2,d}^2 + \hat{\sigma}_{4,d}^2)\epsilon^2$ . By Lemma D.29, this case contributes at most  $\frac{1}{\hat{\sigma}_{2,d}^2\xi^4\kappa^6}\log \left(\frac{\gamma_2}{\epsilon}\right)$  to the overall running time. Thus, by Gronwall's inequality (Fact I.13), the additional factor that this case contributes to  $1 - w_{t}(\iota_{\mathrm{R}})$  is larger than or equal to
$$
\exp \left(- \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) \cdot \frac {1}{\hat {\sigma} _ {2 , d} ^ {2} \xi^ {4} \kappa^ {6}} \log \left(\frac {\gamma_ {2}}{\epsilon}\right)\right) \geq \exp \left(- \frac {C}{\xi^ {4} \kappa^ {6}} \log \left(\frac {\gamma_ {2}}{\epsilon}\right)\right) \tag {D.276}
$$
Times  $t \geq T_2$  when  $|D_{4,t}| \geq \xi |D_{2,t}|$ ,  $r(t) \leq \frac{1}{2}$  and  $w_{\mathbf{R}} \geq 1 - \xi$ . In this case, we obtain a lower bound on  $\frac{d}{dt}(1 - w_{\mathbb{R}})$ , where we have written  $w_{\mathrm{R}} = w_t(\iota_{\mathrm{R}})$  for convenience. First let us obtain a lower bound on  $P_t(w_{\mathbb{R}})$ :
$$
\begin{array}{l} P _ {t} \left(w _ {\mathrm {R}}\right) \gtrsim 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w _ {\mathrm {R}} + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w _ {\mathrm {R}} ^ {3} \tag {D.277} \\ \gtrsim 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w _ {R} \left(w _ {R} - r\right) \left(w _ {R} + r\right) \quad (\text {B . c .} r \text {i s} P _ {t}) \\ \gtrsim \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} (1 - \xi - r) \quad (\mathrm {B . c .} D _ {4, t} \geq 0 \text {a n d} w _ {\mathrm {R}} \geq 1 - \xi) \\ \gtrsim \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \quad (\mathrm {B . c .} r \leq \frac {1}{2} \text {a n d} \xi \leq \frac {1}{\log \log d}) \\ \end{array}
$$
Next let us obtain an upper bound on  $Q_{t}(w_{\mathbb{R}})$ :
$$
\begin{array}{l} \left| Q _ {t} \left(w _ {\mathrm {R}}\right) \right| \lesssim \left| \lambda_ {d} ^ {(1)} \right| + \left| \lambda_ {d} ^ {(3)} \right| \tag {By Lemma 4.4} \\ \lesssim \frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d} \quad (\text {B y}) \\ \lesssim \frac {\hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{\xi d} \quad (\mathrm {B . c .} | D _ {2, t} | \leq | D _ {4, t} | / \xi \text {a n d} \hat {\sigma} _ {2, d} ^ {2} \lesssim \hat {\sigma} _ {4, d} ^ {2} \text {b y A s s u m p t i o n 3 . 2}) \\ \end{array}
$$
Thus, because  $\frac{1}{\xi d} \asymp \frac{\log \log d}{d}$ ,
$$
P _ {t} \left(w _ {\mathrm {R}}\right) + Q _ {t} \left(w _ {\mathrm {R}}\right) \gtrsim \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \tag {D.278}
$$
and when  $w_{\mathbb{R}} \geq 1 - \xi$
$$
\begin{array}{l} \frac {d}{d t} \left(1 - w _ {\mathrm {R}}\right) \gtrsim - v _ {t} \left(w _ {\mathrm {R}}\right) (D.279) \\ \gtrsim (1 - w _ {\mathrm {R}}) (1 + w _ {\mathrm {R}}) \left(P _ {t} \left(w _ {\mathrm {R}}\right) + Q _ {t} \left(w _ {\mathrm {R}}\right)\right) (By Lemma 4.4) \\ \gtrsim (1 - w _ {\mathrm {R}}) \cdot \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \quad (\text {B y E q . (D . 2 7 8) a n d b . c .} w _ {\mathrm {R}} \geq 1 - \xi) \\ \end{array}
$$