SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Finally, we show Lemma D.10 which concludes the analysis of Phase 2. For times  $t \geq T_{\mathrm{R}}$  within Phase 2, the particles with  $|\iota| \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$  satisfy  $|w_t(\iota)| \gtrsim \xi \kappa^2$ . Using this observation, and the fact that  $D_{2,t}$  and  $D_{4,t}$  have the same sign (meaning that the two terms of  $P_t(w) = 2\hat{\sigma}_{2,d}^2 D_{2,t}w + 4\hat{\sigma}_{4,d}^2 D_{4,t}w^3$  do not cancel) we show that the velocity  $v(w)$  is large on average. By Lemma D.39, this allows us to show that the loss decreases quickly at any time  $t \geq T_{\mathrm{R}}$  in Phase 2, meaning that either the loss goes below  $(\hat{\sigma}_{2,d}^2 + \hat{\sigma}_{4,d}^2)\epsilon^2$  quickly, or we exit Phase 2 quickly.
Lemma D.10 (Phase 2 Summary). Suppose we are in the setting of Theorem 3.3. Let  $T_{2}$  be the minimum time such that either  $D_{2,T_2} = 0$  or  $D_{4,T_2} = 0$ , i.e.  $T_{2}$  is the end of phase 2. Then, either  $T_{2} - T_{1} \lesssim \frac{(\log \log d)^{18}}{(\hat{\sigma}_{2,d}^{2} + \hat{\sigma}_{4,d}^{2})} \log \left(\frac{\gamma_{2}}{\epsilon}\right)$  or  $T_{*,\epsilon} - T_{1} \lesssim \frac{(\log \log d)^{18}}{(\hat{\sigma}_{2,d}^{2} + \hat{\sigma}_{4,d}^{2})} \log \left(\frac{\gamma_{2}}{\epsilon}\right)$ .
Proof of Lemma D.10. First, let  $T_{\mathrm{R}}$  be as defined in Lemma D.7. By Lemma D.7,  $T_{\mathrm{R}} - T_{1} \lesssim \frac{1}{\hat{\sigma}_{2,d}^{2}\gamma_{2}} \log \log d$ . Furthermore, at time  $T_{\mathrm{R}}$ , for all  $\iota$  such that  $|\iota| \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$ ,  $|w_{t}(\iota)| \gtrsim \xi \kappa^{2}$ .
Our proof strategy is now to show that  $v(w)$  is large on average, and then apply Lemma D.39. At any time  $t \in [T_{\mathbb{R}}, T_2]$ , since  $D_{2,t} \leq 0$ , Proposition D.38 implies that  $\mathbb{E}_{w \sim \rho_t}[w^2] \leq \gamma_2 + O\left(\frac{1}{d}\right)$ . Thus, by Markov's inequality, and because  $d \geq c_3$  (where  $c_3$  is defined in Assumption 3.2),
$$
\mathbb {P} _ {w \sim \rho_ {t}} (| w | \geq 1 / 2) \leq 4 \cdot \left(\gamma_ {2} + \frac {1}{d}\right) \leq 5 \tau^ {2} \tag {D.77}
$$
For all  $\iota$  with  $|\iota| \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$ ,  $|w_t(\iota)|$  is increasing as a function of  $t$  on  $[0, T_2]$ , by Lemma D.9. Thus, by Proposition D.6 and a union bound,  $\mathbb{P}_{w \sim \rho_t}(\xi \kappa^2 \leq |w_t(\iota)| \leq 1/2) \geq 1 - O(\kappa) - 5\gamma_2 \geq \frac{1}{2}$  for all  $t \geq T_{\mathrm{R}}$  and all  $\iota$  with  $|\iota| \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$ . (Note that  $1 - O(\kappa) - 5\gamma_2 \geq \frac{1}{2}$  holds because  $\gamma_2$  is less than a
sufficiently small constant by Assumption 3.2.) Finally, for all  $w$  satisfying  $\xi \kappa^2 \leq |w| \leq \frac{1}{2}$ , using Lemma 4.4 we can compute that
$$
\begin{array}{l} \left| v _ {t} (w) \right| \geq \left(1 - w ^ {2}\right) \cdot \left(\left| P _ {t} (w) \right| - \left| Q _ {t} (w) \right|\right) (By Lemma 4.4) \\ \geq \left(1 - w ^ {2}\right) \cdot \left| 2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | | w | + 4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | | w ^ {3} | \right| - O \left(\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d}\right) | w | (By Lemma 4.4) \\ \geq \frac {3}{4} \cdot \left(2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \xi \kappa^ {2} + 4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | \cdot \xi^ {3} \kappa^ {6}\right) - O \Big (\frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d} \Big) | w | \quad (\mathrm {B . c .}   \xi \kappa^ {2} \leq | w | \leq \frac {1}{2}) \\ \geq \frac {1}{2} \cdot \left(2 \hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | \xi \kappa^ {2} + 4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | \cdot \xi^ {3} \kappa^ {6}\right) \\ \end{array}
$$
(B.c.  $d$  is sufficiently large (Assumption 3.2))
Thus, by Lemma D.39,
$$
\frac {d L \left(\rho_ {t}\right)}{d t} \lesssim - \left(\hat {\sigma} _ {2, d} ^ {2} \mid D _ {2, t} \mid \xi \kappa^ {2} + \hat {\sigma} _ {4, d} ^ {2} \mid D _ {4, t} \mid \xi^ {3} \kappa^ {6}\right) ^ {2} \tag {D.78}
$$
$$
\lesssim - \xi^ {6} \kappa^ {1 2} \left(\hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| + \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right|\right) ^ {2} \tag {D.79}
$$
$$
\lesssim - \xi^ {6} \kappa^ {1 2} \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) \left(\hat {\sigma} _ {2, d} ^ {2} | D _ {2, t} | + \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} |\right) \cdot \left(| D _ {2, t} | + | D _ {4, t} |\right) \tag {D.80}
$$
$$
\lesssim - \xi^ {6} \kappa^ {1 2} \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) \left(\hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| ^ {2} + \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| ^ {2}\right) \tag {D.81}
$$
$$
\lesssim - \xi^ {6} \kappa^ {1 2} \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) L \tag {By Lemma 4.2}
$$
Thus, since  $L(\rho_{T_{\mathbb{R}}}) \lesssim (\hat{\sigma}_{2,d}^{2} + \hat{\sigma}_{4,d}^{2})\gamma_{2}$  (by Lemma D.8), by Gronwall's inequality (Fact I.13), we know that for  $t \in [T_{\mathbb{R}}, T_2]$ ,
$$
\begin{array}{l} L \left(\rho_ {t}\right) \leq L \left(\rho_ {T _ {\mathrm {R}}}\right) \cdot \exp \left(\int_ {T _ {\mathrm {R}}} ^ {t} - \xi^ {6} \kappa^ {1 2} \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right) d s\right) (D.82) \\ = L \left(\rho_ {T _ {\mathrm {R}}}\right) e ^ {- (t - T _ {\mathrm {R}}) \xi^ {6} \kappa^ {1 2} \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right)} (D.83) \\ \end{array}
$$
Thus, if  $L(\rho_t) \gtrsim (\hat{\sigma}_{2,d}^2 + \hat{\sigma}_{4,d}^2)\epsilon^2$ , this implies that
$$
e ^ {- \left(t - T _ {\mathrm {R}}\right) \xi^ {6} \kappa^ {1 2} \left(\hat {\sigma} _ {2, d} ^ {2} + \hat {\sigma} _ {4, d} ^ {2}\right)} \gtrsim \frac {\epsilon^ {2}}{\gamma_ {2}} \tag {D.84}
$$
and rearranging gives  $t - T_{\mathbb{R}} \leq \frac{1}{(\hat{\sigma}_{2,d}^{2} + \hat{\sigma}_{4,d}^{2})\xi^{6}\kappa^{12}}\log \left(\frac{\gamma_{2}}{\epsilon^{2}}\right)$ . Thus, either  $T_{2} - T_{\mathbb{R}} \leq \frac{1}{(\hat{\sigma}_{2,d}^{2} + \hat{\sigma}_{4,d}^{2})\xi^{6}\kappa^{12}}\log \left(\frac{\gamma_{2}}{\epsilon^{2}}\right)$ , or there exists a time  $t \in [T_{\mathbb{R}}, T_{2}]$  such that  $L(\rho_t) \lesssim (\hat{\sigma}_{2,d}^2 + \hat{\sigma}_{4,d}^2)\epsilon^2$  and such that  $t - T_{\mathbb{R}} \leq \frac{1}{(\hat{\sigma}_{2,d}^{2} + \hat{\sigma}_{4,d}^{2})\xi^{6}\kappa^{12}}\log \left(\frac{\gamma_{2}}{\epsilon}\right)$ . This completes the proof of the lemma.
D.5 Phase 3, Case 1
Phase 3, Case 1 is the case where  $D_{2,T_2} = 0$  and  $D_{4,T_2} < 0$ , i.e. the case where  $D_{2,t}$  reaches 0 first. As we later show in Lemma D.19 and Lemma D.20, if this occurs, then for all  $t \geq T_2$ , it will be the case that  $D_{2,t} \geq 0$  and  $D_{4,t} \leq 0$ . We now outline our analysis of this case — the goal of our analysis is to show that either  $|D_{2,t}| \leq \epsilon$  or  $|D_{4,t}| \leq \epsilon$  holds within  $\frac{1}{\hat{\sigma}_{2,d}^2 + \hat{\sigma}_{4,d}^2} \cdot \mathrm{poly}\left(\frac{\log\log d}{\epsilon}\right)$  time after the start of Phase 3, Case 1.
Proof Strategy for Phase 3, Case 1 Our overall proof strategy in this section is to show that  $|v(w)|$  is large for a large portion of particles  $w$ , and then to apply Lemma D.39 to show that the loss decreases quickly. Recall that
$$
v _ {t} (w) = - \left(1 - w ^ {2}\right) \left(P _ {t} (w) + Q _ {t} (w)\right) \tag {D.85}
$$
and intuitively, we can ignore  $Q_{t}(w)$  due to the factor of  $\frac{1}{d}$  in its coefficients. Thus, we can write
$$
\begin{array}{l} v _ {t} (w) \approx - \left(1 - w ^ {2}\right) P _ {t} (w) (D.86) \\ \approx - \left(1 - w ^ {2}\right) \cdot w \cdot \left(2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {2}\right). (D.87) \\ \end{array}
$$
Since  $D_{4,t} < 0$  and  $D_{2,t} > 0$ , we can further rewrite the above as
$$
v _ {t} (w) \approx - (1 - w ^ {2}) \cdot w \cdot 4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | (w - r) (w + r) \tag {D.88}
$$
where  $r = \sqrt{-\frac{\hat{\sigma}_{2,d}^2D_{2,t}}{2\hat{\sigma}_{4,d}^2D_{4,t}}}$  is the positive root of the last factor  $(2\hat{\sigma}_{2,d}^{2}D_{2,t} + 4\hat{\sigma}_{4,d}^{2}D_{4,t}w^{2})$ . Thus, to show that the velocity  $v_{T}(w)$  is large, we want to show that there is a large fraction of particles  $w\in [0,1]$  simultaneously satisfying the following: (1)  $w$  is far from 1, (2)  $w$  is far from 0, and (3)  $w$  is far from  $r$ . Indeed, in Lemma D.17, we show that out of the particles  $w\in [0,1]$ , at least a  $\frac{\gamma_2}{2}$  fraction of these particles is in  $[\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$ , enabling us to bound the distance of these particles  $w$  from 0 and 1.
Thus, to show that  $v_{t}(w)$  is large for these particles, it suffices to show that  $|w - r|$  is large on average for the particles  $w \in [\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$ , i.e. the conditional expectation  $\mathbb{E}_{w \sim \rho_t}[(w - r)^2 \mid w \in [\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$  is large. In turn, this is at least the conditional variance of  $w$  conditioned on  $w$  being in  $[\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$ , so it suffices to show that  $\mathrm{Var}(w \mid w \in [\gamma_2^{3/4}, 1 - \gamma_2^{1/2}])$  is large, and by Proposition D.40 it suffices to show that  $\mathbb{E}[(w - w')^2 \mid w, w' \in [\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$  is large.
To show that the quantity  $\mathbb{E}[(w - w')^2 \mid w, w' \in [\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$  is large, intuitively we would like to show that during Phase 3, Case 1,  $w$  and  $w'$  are getting farther apart. However, this is not true. If we write the velocity as  $v_t(w) = (1 - w^2) \cdot w \cdot (4\hat{\sigma}_{4,d}^2 |D_{4,t}|w^2 - 2\hat{\sigma}_{2,d}^2 |D_{2,t}|)$ , then the last factor is indeed increasing as a function of  $w \in [0,1]$ , and thus "pushes" the different particles  $w, w'$  further apart. This does not work as a formal argument, because as two particles  $w, w'$  become closer to 0 or 1, their distance  $|w - w'|$  will decrease again due to the factors  $w$  and  $(1 - w^2)$  in the velocity. However, we are only concerned with particles  $w \in [\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$  — we must make precise the intuition that the behavior of the particles very close to 0 or 1 does not matter.
To make this proof strategy precise, we define the potential function  $\Phi(w) = \log\left(\frac{w}{\sqrt{1 - w^2}}\right)$  in Definition D.11 and show that  $|\Phi(w) - \Phi(w')|$  is increasing in Phase 3, Case 1. As shown in Lemma D.12, we have  $\frac{d}{dt}\Phi(w) \approx -2\hat{\sigma}_{2,d}^2D_{2,t} - 4\hat{\sigma}_{4,d}^2D_{4,t}w^2$  — this is simply because  $\frac{d\Phi(w)}{dw} = \frac{1}{w(1 - w^2)}$ . In other words,  $\frac{d}{dt}\Phi(w)$  is exactly  $v(w)$ , but without the factors  $w$  and  $(1 - w^2)$  — we have selected this potential function  $\Phi(w)$  specifically to eliminate these factors. Using this observation, it is easy to show that  $|\Phi(w) - \Phi(w')|$  is increasing and thus obtain a lower bound on  $|\Phi(w) - \Phi(w')|$  for a large fraction of pairs  $w, w'$ . Finally, to convert this to a lower bound on  $|w - w'|$ , we use the fact that  $\Phi$  is Lipschitz on the interval  $[\gamma_2^{3/4}, 1 - \gamma_2^{1/2}]$  (Lemma D.14), which gives us  $|w - w'| \geq \frac{1}{\gamma_2^{O(1)}} |\Phi(w) - \Phi(w')|$ .
We now repeat the definition of the potential function:
Definition D.11 (Potential Function  $\Phi$ ). We define  $\Phi : [0, 1] \to (-\infty, \infty)$  by  $\Phi(w) = \log \left(\frac{w}{\sqrt{1 - w^2}}\right)$ .
We note that  $\frac{d\Phi(w)}{dw} = \frac{1}{w(1 - w^2)}$ . Next, we note the following useful fact about the potential function (we note that this does not require any assumption that we are in Phase 3, Case 1, and just follows from algebraic manipulations):
Lemma D.12. In the setting of Theorem 3.3,
$$
\frac {d}{d t} \Phi (w _ {t}) = - 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} - 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {2} - \lambda_ {d} ^ {(1)} - \lambda_ {d} ^ {(3)} w ^ {2} \tag {D.89}
$$
where  $\lambda_{d}^{(1)}$  and  $\lambda_{d}^{(3)}$  are as in the statement of Lemma 4.4.
Proof of Lemma D.12. For any  $w_{t}$
$$
\frac {d}{d t} \Phi \left(w _ {t}\right) = \frac {d \Phi}{d w} \cdot \frac {d w _ {t}}{d t} = \frac {1}{w \left(1 - w ^ {2}\right)} \cdot \frac {d w _ {t}}{d t} \tag {D.90}
$$
Thus, writing  $\frac{dw_t}{dt} = -(1 - w^2) \cdot (P_t(w) + Q_t(w))$  where  $P_t$  and  $Q_t$  are defined in Lemma 4.4, we obtain
$$
\begin{array}{l} \frac {d}{d t} \Phi \left(w _ {t}\right) = \frac {1}{w \left(1 - w ^ {2}\right)} \cdot \frac {d w _ {t}}{d t} (D.91) \\ = - \frac {1}{w} \cdot \left(P _ {t} (w) + Q _ {t} (w)\right) (D.92) \\ = - \frac {P _ {t} (w)}{w} - \frac {Q _ {t} (w)}{w} (D.93) \\ = - 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} - 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {2} - \frac {Q _ {t} (w)}{w} (D.94) \\ = - 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} - 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w ^ {2} - \lambda_ {d} ^ {(1)} - \lambda_ {d} ^ {(3)} w ^ {2} (D.95) \\ \end{array}
$$
Intuitively, when  $D_{2} > 0$  and  $D_{4} < 0$ , the main part of the velocity is  $P_{t}(w) = -2\hat{\sigma}_{2,d}^{2}D_{2,t}w - 4\hat{\sigma}_{4,d}^{2}D_{4,t}w^{3}$  which is an upward sloping cubic polynomial — thus, the particles  $w$  move away from the positive root of this polynomial, and the distance between any pair  $w, w'$  of particles is increasing provided that  $w$  and  $w'$  are not too close to 0 or 1. In the following few lemmas, we make this intuition formal using the potential function  $\Phi$ . We first show that for two particles  $w, w'$ , the distance between  $\Phi(w)$  and  $\Phi(w')$  is increasing as long as  $D_{4,t} \leq 0$ . Note that this condition holds not only during Phase 3, Case 1, but also during Phases 1 and 2.
Lemma D.13 ( $\Phi(w)$  and  $\Phi(w')$  Moving Apart). In the setting of Theorem 3.3, suppose  $D_{4,t} \leq 0$  for all  $t$  in some time interval  $[t_0, t_1]$ , and let  $\iota_1, \iota_2 > 0$ . Then,
$$
\left| \Phi \left(w _ {t} \left(\iota_ {1}\right)\right) - \Phi \left(w _ {t} \left(\iota_ {2}\right)\right) \right| \tag {D.96}
$$
is increasing on the interval  $[t_0, t_1]$ .
Proof of Lemma D.13. Suppose  $\iota_1, \iota_2 \in [0,1]$  such that  $\iota_1 < \iota_2$  — by Proposition D.37, we will always have  $w_t(\iota_1) \leq w_t(\iota_2)$ . For convenience, in the rest of this proof, we will write  $w_{1,t} = w_t(\iota_1)$  and  $w_{2,t} = w_t(\iota_2)$ . Since  $\Phi$  is an increasing function of  $w$ , it will also always be the case that  $\Phi(w_{2,t}) \geq \Phi(w_{1,t})$ . Thus, to show that  $|\Phi(w_{2,t}) - \Phi(w_{1,t})|$  is increasing, it suffices to show that  $\Phi(w_{2,t}) - \Phi(w_{1,t})$  is increasing. By Lemma D.12,
$$
\begin{array}{l} \frac {d}{d t} \left(\Phi \left(w _ {2, t}\right) - \Phi \left(w _ {1, t}\right)\right) = \left(- 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} - 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w _ {2, t} ^ {2} - \lambda_ {d} ^ {(1)} - \lambda_ {d} ^ {(3)} w _ {2, t} ^ {2}\right) (D.97) \\ - \left(- 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} - 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w _ {1, t} ^ {2} - \lambda_ {d} ^ {(1)} - \lambda_ {d} ^ {(3)} w _ {1, t} ^ {2}\right) (D.98) \\ = - 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \left(w _ {2, t} ^ {2} - w _ {1, t} ^ {2}\right) - \lambda_ {d} ^ {(3)} \left(w _ {2, t} ^ {2} - w _ {1, t} ^ {2}\right) (D.99) \\ = 4 \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| \left(w _ {2, t} ^ {2} - w _ {1, t} ^ {2}\right) - \lambda_ {d} ^ {(3)} \left(w _ {2, t} ^ {2} - w _ {1, t} ^ {2}\right) (D.100) \\ \end{array}
$$
where  $\lambda_d^{(1)}$  and  $\lambda_d^{(3)}$  are as in Lemma 4.4. Recall from the statement of Lemma 4.4 that  $|\lambda_d^{(3)}| \lesssim \hat{\sigma}_{4,d}^2 |D_{4,t}| \cdot \frac{1}{d}$ . Therefore,
$$
\begin{array}{l} \frac {d}{d t} \left(\Phi \left(w _ {2, t}\right) - \Phi \left(w _ {1, t}\right)\right) = 4 \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| \left(w _ {2, t} ^ {2} - w _ {1, t} ^ {2}\right) - \lambda_ {d} ^ {(3)} \left(w _ {2, t} ^ {2} - w _ {1, t} ^ {2}\right) (D.101) \\ \geq 3 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | \left(w _ {2, t} ^ {2} - w _ {1, t} ^ {2}\right) (D.102) \\ \end{array}
$$
and the right hand side is nonnegative since  $w_{2,t} \geq w_{1,t} \geq 0$ . Thus,  $\Phi(w_{2,t}) - \Phi(w_{1,t})$  is increasing, as desired.
Next, we prove a helpful lemma showing that the potential function is bi-Lipschitz. This is useful both when we show that  $|\Phi(w) - \Phi(w')|$  is initially large (as it allows us to leverage an initial lower bound on  $|w - w'|$ ) and when we show that  $|w - w'|$  is large later during Phase 3, Case 1 (as it allows us to leverage the lower bound we obtain on  $|\Phi(w) - \Phi(w')|$ ).
Lemma D.14. For all  $w, w' \in [0,1]$ ,  $|\Phi(w) - \Phi(w')| \geq |w - w'|$ . Additionally, for  $\eta < \frac{1}{2}$  and  $w, w' \in [\eta, 1 - \eta]$ ,  $|\Phi(w) - \Phi(w')| \leq \frac{1}{\eta^2} |w - w'|$ .
Proof of Lemma D.14. First, for all  $w \in [0, 1]$ ,  $\frac{d\Phi(w)}{dw} = \frac{1}{w(1 - w^2)} > 1$ , and the first statement of the lemma follows. For the second statement of the lemma, observe that for  $w \in [\eta, 1 - \eta]$ ,
$$
\left| \frac {d \Phi (w)}{d w} \right| \leq \frac {1}{w \left(1 - w ^ {2}\right)} \leq \frac {1}{\eta \left(1 - (1 - \eta) ^ {2}\right)} = \frac {1}{\eta \left(1 - (1 - 2 \eta + \eta^ {2})\right)} = \frac {1}{\eta \left(2 \eta - \eta^ {2}\right)} <   \frac {1}{\eta^ {2}} \tag {D.103}
$$
where the last equality is because  $\eta >\eta^2$  (so  $2\eta -\eta^{2} > \eta$ ). Thus,  $\Phi$  is  $\frac{1}{\eta^2}$ -Lipschitz on the interval  $[\eta ,1 - \eta ]$
We next need to show that the potential initially has a large value, at the beginning of Phase 3, Case 1. We show this in the next two lemmas, by showing that for any two particles  $w$ ,  $w'$ , the distance between them grows by a large factor during Phase 2 (specifically, by time  $T_{\mathbb{R}}$ ) and then using this to obtain a lower bound on  $|\Phi(w) - \Phi(w')|$  by the bi-Lipschitzness of  $\Phi$ .
Lemma D.15. In the setting of Theorem 3.3, let  $T_{\mathbb{R}}$  be as defined in Lemma D.7. Then, for  $\iota, \iota' \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$ ,
$$
\frac {w _ {T _ {\mathrm {R}}} \left(\iota^ {\prime}\right) - w _ {T _ {\mathrm {R}}} (\iota)}{\iota^ {\prime} - \iota} \gtrsim \xi \kappa \sqrt {d} \tag {D.104}
$$
Proof of Lemma D.15. Let  $\iota, \iota' \in [\iota_{\mathrm{L}}, \iota_{\mathrm{R}}]$ , with  $\iota < \iota'$ . For convenience, we will write  $w_t = w_t(\iota)$  and  $w_t' = w_t(\iota')$ . Our goal is to obtain a lower bound on the factor by which  $w_t' - w_t$  grows by time  $T_{\mathrm{R}}$ . First, define  $P_t$  and  $Q_t$  as in Lemma 4.4, and observe that
$$
\begin{array}{l} \frac {d}{d t} \left(w _ {t} ^ {\prime} - w _ {t}\right) = - \left(1 - \left(w _ {t} ^ {\prime}\right) ^ {2}\right) \left(P _ {t} \left(w _ {t} ^ {\prime}\right) + Q _ {t} \left(w _ {t} ^ {\prime}\right)\right) - \left(1 - w _ {t} ^ {2}\right) \left(P _ {t} \left(w _ {t}\right) + Q _ {t} \left(w _ {t}\right)\right) (D.105) \\ = - \left(P _ {t} \left(w _ {t} ^ {\prime}\right) - P _ {t} \left(w _ {t}\right)\right) + \left(\left(w _ {t} ^ {\prime}\right) ^ {2} P _ {t} \left(w _ {t} ^ {\prime}\right) - w _ {t} ^ {2} P _ {t} \left(w _ {t}\right)\right) - \left(Q _ {t} \left(w _ {t} ^ {\prime}\right) - Q _ {t} \left(w _ {t}\right)\right) (D.106) \\ + \left(\left(w _ {t} ^ {\prime}\right) ^ {2} Q _ {t} \left(w _ {t} ^ {\prime}\right) - w _ {t} ^ {2} Q _ {t} \left(w _ {t}\right)\right) (D.107) \\ \end{array}
$$
We first obtain a lower bound on the first term  $-\left(P_t(w_t') - P_t(w_t)\right)$ , and then show that the other terms are lower-order terms as long as  $t \leq T_{\mathrm{R}}$ . For  $t \in [0, T_{\mathrm{R}}]$ ,
$$
\begin{array}{l} - \left(P _ {t} \left(w _ {t} ^ {\prime}\right) - P _ {t} \left(w _ {t}\right)\right) = - \left(2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w _ {t} ^ {\prime} + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \left(w _ {t} ^ {\prime}\right) ^ {3}\right) + \left(2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} w _ {t} + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} w _ {t} ^ {3}\right) (Lemma 4.4) \\ = - 2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} \left(w _ {t} ^ {\prime} - w _ {t}\right) - 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \left(\left(w _ {t} ^ {\prime}\right) ^ {3} - w _ {t} ^ {3}\right) (D.108) \\ = 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \left(w _ {t} ^ {\prime} - w _ {t}\right) + 4 \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| \left(\left(w _ {t} ^ {\prime}\right) ^ {3} - w _ {t} ^ {3}\right) \quad (\mathrm {B . c .} t \leq T _ {\mathrm {R}}, D _ {2, t}, D _ {4, t} <   0 \text {b y L e m m a D . 8}) \\ = 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \left(w _ {t} ^ {\prime} - w _ {t}\right) + 4 \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| \left(\left(w _ {t} ^ {\prime}\right) ^ {2} + w _ {t} w _ {t} ^ {\prime} + w _ {t} ^ {2}\right) \left(w _ {t} ^ {\prime} - w _ {t}\right) (D.109) \\ \geq 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \left(w _ {t} ^ {\prime} - w _ {t}\right) \quad (\mathrm {B . c . o m i t t e d t e r m i s n o n n e g a t i v e} - w _ {t} ^ {\prime} > w _ {t} \text {b y P r o p o s i t i o n D . 3 7}) \\ \end{array}
$$
Next, let us show that the remaining terms in Eq. (D.107) are lower-order terms which will not decrease the growth rate of  $w_{t}^{\prime} - w_{t}$  too much. First let us deal with the absolute value of the second
term in Eq. (D.107). For convenience, let  $w_{R,t} = w_t(\iota_{\mathbb{R}})$ . Then,
$$
\begin{array}{l} \left| \left(w _ {t} ^ {\prime}\right) ^ {2} P _ {t} \left(w _ {t} ^ {\prime}\right) - w _ {t} ^ {2} P _ {t} \left(w _ {t}\right) \right| \lesssim \left| \left(2 \hat {\sigma} _ {2, d} ^ {2} D _ {2, t} \left(w _ {t} ^ {\prime}\right) ^ {3} + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \left(w _ {t} ^ {\prime}\right) ^ {5}\right) \right. (D.110) \\ \left. - \left(2 \hat {\sigma} _ {4, d} ^ {2} D _ {2, t} \left(w _ {t}\right) ^ {3} + 4 \hat {\sigma} _ {4, d} ^ {2} D _ {4, t} \left(w _ {t}\right) ^ {5}\right) \right| (D.111) \\ \lesssim 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \left| \left(w _ {t} ^ {\prime}\right) ^ {3} - w _ {t} ^ {3} \right| + 4 \hat {\sigma} _ {4, d} ^ {2} \left| D _ {4, t} \right| \left| \left(w _ {t} ^ {\prime}\right) ^ {5} - w _ {t} ^ {5} \right| (By Lemma 4.4) \\ \lesssim 2 \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| \left(\left. w _ {t} ^ {\prime}\right) ^ {2} + w _ {t} ^ {\prime} w _ {t} + w _ {t} ^ {2} \right| \left| w _ {t} ^ {\prime} - w _ {t} \right| (D.112) \\ + 4 \hat {\sigma} _ {4, d} ^ {2} | D _ {4, t} | | \left(w _ {t} ^ {\prime}\right) ^ {4} + \left(w _ {t} ^ {\prime}\right) ^ {3} w _ {t} + \left(w _ {t} ^ {\prime}\right) ^ {2} w _ {t} ^ {2} + w _ {t} ^ {\prime} w _ {t} ^ {3} + w _ {t} ^ {4} | | w _ {t} ^ {\prime} - w _ {t} | (D.113) \\ \lesssim \hat {\sigma} _ {2, d} ^ {2} \left| D _ {2, t} \right| w _ {R, t} ^ {2} \left| w _ {t} ^ {\prime} - w _ {t} \right| \quad (\mathrm {B . c .} | w _ {t} |, | w _ {t} ^ {\prime} | \leq w _ {R, t} \text {b y} \\ \end{array}
$$
where the last inequality is also because  $\hat{\sigma}_{4,d}^2\lesssim \hat{\sigma}_{2,d}^2$  , and  $\gamma_4\lesssim \gamma_2^2$  (Assumption 3.2) and  $|D_{2}|\geq \frac{\gamma_{2}}{2}$  and  $|D_4|\leq \frac{3\gamma_4}{2}$  for  $t\le T_{\mathrm{R}}$  (Lemma D.8), meaning that the first term in Eq. (D.113) dominates up to universal constant factors. Next, let us deal with the absolute value of the third term in Eq. (D.107). By Lemma I.16,
$$
| Q _ {t} (w _ {t}) ^ {\prime} - Q _ {t} (w _ {t}) | \lesssim \frac {\hat {\sigma} _ {2 , d} ^ {2} | D _ {2 , t} | + \hat {\sigma} _ {4 , d} ^ {2} | D _ {4 , t} |}{d} \cdot | w _ {t} ^ {\prime} - w _ {t} |
$$
(By definition of  $\lambda_{d}^{(1)},\lambda_{d}^{(3)}$  , Lemma 4.4)