SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Recalling the definition of  $T_{\mathrm{II},k}^{\mathrm{PT*}}$  and  $T_{\mathrm{II}}^{\mathrm{PT*}}, T_{\mathrm{II},k}^{\mathrm{PT*}} < T_{\mathrm{II}}^{\mathrm{PT*}}$  implies that  $\left\langle \boldsymbol{w}_k(T_{\mathrm{II},k}^{\mathrm{PT*}}), \boldsymbol{x}_{-}\right\rangle = 0$ .
Then recalling our proof of (S2)(P1), we obtain  $\left\langle \pmb{b}_k(T_{\mathrm{II},k}^{\mathrm{PT*}}),\pmb{x}_+\right\rangle >0$
Hence, we obtain  $\pmb{w}(T_{\Pi,k}^{\mathrm{PT}_*}) \in \mathcal{M}_+^+ \cap \mathcal{M}_-^0$ .
Step II. Dynamics after  $t = T_{\mathrm{II},k}^{\mathrm{PT*}}$ .
In this step, we will analyze the training dynamics after  $\pmb{w}_k(T_{\mathrm{II},k}^{\mathrm{PT*}}) \in \mathcal{M}_+^+ \cap \mathcal{M}_-^0$ , i.e.  $\pmb{b}_k(T_{\mathrm{II},k}^{\mathrm{PT*}}) \in \mathcal{P}_+^+ \cap \mathcal{P}_-^0$ . We first analysis the vector field around the manifold  $\mathcal{P}_+^+ \cap \mathcal{P}_-^0$ . For any  $\tilde{\pmb{b}} \in \mathcal{P}_+^+ \cap \mathcal{P}_-^0$  and  $0 < \delta_0 \ll 1$ , we know that  $\mathcal{P}_+^0 \cap \mathcal{P}_-^+$  separates its neighborhood  $\mathcal{B}(\tilde{\pmb{b}}, \delta_0)$  into two domains  $\mathcal{G}_- = \{\pmb{b} \in \mathcal{B}(\tilde{\pmb{b}}, \delta_0) : \langle \pmb{b}, \pmb{x}_- \rangle < 0\}$  and  $\mathcal{G}_+ = \{\pmb{b} \in \mathcal{B}(\tilde{\pmb{b}}, \delta_0) : \langle \pmb{b}, \pmb{x}_- \rangle > 0\}$ . Following Definition H.1, we calculate the limited vector field on  $\tilde{\pmb{b}}$  from  $\mathcal{G}_-$  and  $\mathcal{G}_+$ .
$$
\frac {\mathrm {d} \boldsymbol {b}}{\mathrm {d} t} = \boldsymbol {F} ^ {-}, \text {w h e r e} \boldsymbol {F} ^ {-} = \frac {\kappa_ {2}}{\sqrt {m}} \frac {p}{1 + p} e ^ {- f _ {+} (t)} \boldsymbol {x} _ {+}.
$$
(ii) The limited vector field  $F^{+}$  on  $\tilde{\pmb{b}}$  (from  $\mathcal{G}_{+}$ ):
$$
F _ {N} ^ {-} = \left\langle \boldsymbol {F} ^ {-}, \boldsymbol {x} _ {-} \right\rangle = \frac {\kappa_ {2} p e ^ {- f _ {+} (t)}}{\sqrt {m} (1 + p)} \cos \Delta ,
$$
$$
F _ {N} ^ {+} = \left\langle \boldsymbol {F} ^ {+}, \boldsymbol {x} _ {-} \right\rangle = \frac {\kappa_ {2} e ^ {f _ {-} (t)}}{\sqrt {m} (1 + p)} \cos \Delta - \frac {\kappa_ {2} p e ^ {- f _ {+} (t)}}{\sqrt {m} (1 + p)}.
$$
We further define the hitting time to check whether  $\pmb{w}_k(t) \in \mathcal{M}_+^+ \cap \mathcal{M}_-^0$  for  $T_{\mathrm{II},k}^{\mathrm{PT}*} \leq t \leq T_{\mathrm{II}}^{\mathrm{PT}*}$ .
$$
\tau_ {+, k} ^ {+} := \inf  \left\{t \in [ T _ {\mathrm {I I}, k} ^ {\mathrm {P T} *}, T _ {\mathrm {I I}} ^ {\mathrm {P T} *} ]: \langle \boldsymbol {w} _ {k} (t), \boldsymbol {x} _ {+} \rangle \leq 0 \right\}.
$$
From the definition of  $T_{\mathrm{II}}^{\mathrm{PT*}}$ , we know that  $\langle \pmb{F}_{+}(t),\pmb{x}_{-}\rangle = \frac{p}{1 + p} e^{-f_{+}(t)}\cos \Delta -\frac{1}{1 + p} e^{f_{-}(t)} < 0$  for any  $t\in [T_{\mathrm{II}},T_{\mathrm{II}}^{\mathrm{PT*}}]$ , which means  $F_N^+ < 0$ . And it is clear that  $F_N^- >0$ . Hence, the dynamics
corresponds to Case (I) in Definition H.1 ( $F_N^- > 0$  and  $F_N^+ < 0$ ), which means that  $\pmb{b}_k(t)$  can not leave  $\mathcal{P}_{-}^{0}$  for any  $t \in [T_{\mathrm{II},k}^{\mathrm{PT*}}, \tau_{+,k}^{+}]$ , and the dynamics of  $\pmb{b}_k$  for  $t \in [T_{\mathrm{II},k}^{\mathrm{PT*}}, \tau_{+,k}^{+}]$  satisfies:
$$
\frac {\mathrm {d} \boldsymbol {b} _ {k} (t)}{\mathrm {d} t} = \frac {\kappa_ {2} p e ^ {- f _ {+} (t)}}{\sqrt {m} (1 + p)} \left(\boldsymbol {x} _ {+} - \boldsymbol {x} _ {-} \cos \Delta\right), t \in \left[ T _ {\mathrm {I I}, k} ^ {\mathrm {P T} *}, \tau_ {+, k} ^ {+} \right].
$$
By Lemma C.1, we know that the dynamics of  $\pmb{w}_k(t)$  on  $\mathcal{M}_{+}^{+}\cap \mathcal{M}_{-}^{0}$  and the dynamics of  $\rho_{k}(t)$  are:
$$
\begin{array}{l} \frac {\mathrm {d} \boldsymbol {w} _ {k} (t)}{\mathrm {d} t} = \frac {\kappa_ {2} p e ^ {- f _ {+} (t)}}{\rho_ {k} (t) \sqrt {m} (1 + p)} \left(\boldsymbol {x} _ {+} - \langle \boldsymbol {w} _ {k}, \boldsymbol {x} _ {+} \rangle \boldsymbol {w} _ {k} - \boldsymbol {x} _ {-} \cos \Delta\right). \\ \frac {\mathrm {d} \rho_ {k} (t)}{\mathrm {d} t} = \frac {\kappa_ {2} p e ^ {- f _ {+} (t)}}{\sqrt {m} (1 + p)} \left\langle \boldsymbol {w} _ {k} (t), \boldsymbol {x} _ {+} \right\rangle . \\ \end{array}
$$
Moreover, The dynamics above also ensures that:
$$
\begin{array}{l} \left\langle \boldsymbol {b} _ {k} (t), \boldsymbol {x} _ {+} \right\rangle = \left\langle \boldsymbol {b} _ {k} \left(T _ {\mathrm {I I}, k} ^ {\mathrm {P T} *}\right), \boldsymbol {x} _ {+} \right\rangle + \int_ {T _ {\mathrm {I I}, k} ^ {\mathrm {P T} *}} ^ {t} \frac {\kappa_ {2} p e ^ {- f _ {+} (s)}}{\sqrt {m} (1 + p)} \sin^ {2} \Delta \mathrm {d} s \\ > \left\langle \pmb {b} _ {k} (T _ {\mathrm {I I}, k} ^ {\mathrm {P T} *}), \pmb {x} _ {+} \right\rangle > 0, \quad \forall t \in [ T _ {\mathrm {I I}, k} ^ {\mathrm {P T} *}, \tau_ {+, k} ^ {+} ]. \\ \end{array}
$$
which means  $\tau_{+,k}^{+} = T_{\mathrm{II}}^{\mathrm{PT*}}$ . Hence, we have proved (S2)(P2).
Proof of (S2)(P3). Our proof for (S2)(P1) and (S2)(P2) imply this result directly.
Lemma E.2 (Evolution of the prediction in Phase III*).
For any  $t\in [T_{\mathrm{II}},T_{\mathrm{II}}^{\mathrm{PT*}}]$  , we have
$$
\begin{array}{l} \frac {e ^ {- C _ {1}}}{1 + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} (t - T _ {\mathrm {I I}})} \leq e ^ {f _ {-} (t) - f _ {-} (T _ {\mathrm {I I}})} \leq \frac {1}{1 + C _ {0} e ^ {(f _ {-} (T _ {\mathrm {I I}}) - C _ {1})} (t - T _ {\mathrm {I I}})}, \\ \exp \left(- C _ {2} (t - T _ {\mathrm {I I}})\right) \leq e ^ {f _ {+} \left(T _ {\mathrm {I I}}\right) - f _ {+} (t)} \leq 1, \\ \end{array}
$$
$$
C _ {0} = \Theta \left(\frac {\kappa_ {2} ^ {2} \Delta^ {2}}{p}\right), C _ {1} = \mathcal {O} \left(\kappa_ {2} \sqrt {\kappa_ {1} \kappa_ {2}}\right), e ^ {f _ {-} (T _ {\mathrm {I I}})} = \Theta \left(p ^ {- \frac {\alpha \cos \Delta}{1 - \alpha \cos \Delta}}\right), C _ {2} = \Theta \left(\kappa_ {2} ^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}}\right).
$$
Step I. Preparation. With the help of Lemma E.1(S1) and (S2)(P3), we know that
(i) For  $k\in \mathcal{K}_{-}$ , we have
$$
\langle \boldsymbol {w} _ {k} (t), \boldsymbol {x} _ {+} \rangle = 0, \quad \langle \boldsymbol {w} _ {k} (t), \boldsymbol {x} _ {-} \rangle > 0, \quad \forall t \in \left[ T _ {\mathrm {I I}}, T _ {\mathrm {I I}} ^ {\mathrm {P T} *} \right].
$$
(ii) For  $k\in \mathcal{K}_{+}$ , we have
$$
\langle \boldsymbol {w} _ {k} (t), \boldsymbol {x} _ {+} \rangle > 0, \quad \langle \boldsymbol {w} _ {k} (t), \boldsymbol {x} _ {-} \rangle \geq 0, \quad \forall t \in \left[ T _ {\mathrm {I I}}, T _ {\mathrm {I I}} ^ {\mathrm {P T} *} \right];
$$
So  $f_{+}(t)$  and  $f_{-}(t)$  have the following representation for any  $t \in [T_{\mathrm{II}}, T_{\mathrm{II}}^{\mathrm{PT*}}]$ :
$$
f _ {+} (t) = \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {+},
$$
$$
f _ {-} (t) = \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {-} - \sum_ {k \in \mathcal {K} _ {-}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {-}.
$$
Step II. Evolution of  $f_{-}(t)$ .
To begin with, we need to do a rough estimate of  $\sum_{k\in \mathcal{K}_{+}}\frac{\kappa_2}{\sqrt{m}}\pmb{b}_k^\top (t)\pmb{x}_{-}$ . Let  $k\in \mathcal{K}_{+}$ . For any  $t\in [T_{\Pi ,k}^{\mathrm{PT*}},T_{\Pi}^{\mathrm{PT*}}]$ , we have  $\langle \pmb {b}_k(t),\pmb {x}_-\rangle = 0$ . And for any  $t\in [T_{\mathrm{II}},T_{\mathrm{II},k}^{\mathrm{PT*}})$ , we have:
$$
\frac {\mathrm {d}}{\mathrm {d} t} \left\langle \boldsymbol {b} _ {k} (t), \boldsymbol {x} _ {-} \right\rangle \stackrel {\text {L e m m a}} {=} ^ {\text {E . 1}} \frac {\kappa_ {2}}{\sqrt {m}} \left\langle \boldsymbol {F} _ {k} (t), \boldsymbol {x} _ {-} \right\rangle <   0.
$$
Therefore, for any  $t\in [T_{\mathrm{II}},T_{\mathrm{II}}^{\mathrm{PT*}}]$  we have  $0\leq \langle \pmb {b}_k(t),\pmb {x}_-\rangle \leq \langle \pmb {b}_k(T_{\mathrm{II}}),\pmb {x}_-\rangle$  so
$$
\begin{array}{l} 0 \leq \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {-} \leq \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \boldsymbol {x} _ {-}, \\ - \sum_ {k \in \mathcal {K} _ {-}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {-} \leq f _ {-} (t) \leq \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \boldsymbol {x} _ {-} - \sum_ {k \in \mathcal {K} _ {-}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {-}. \\ \end{array}
$$
According to Lemma E.1, it follows that for any  $k \in \mathcal{K}_{-}$ , its dynamics is  $\frac{\mathrm{d}\pmb{b}_k(t)}{\mathrm{d}t} = \frac{\kappa_2 e^{f_{-}(t)}}{\sqrt{m}(1 + p)}\left(\pmb{x}_{-} - \pmb{x}_{+}\cos \Delta\right)$ , thus
$$
\begin{array}{l} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {-} = \boldsymbol {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \boldsymbol {x} _ {-} + \int_ {T _ {\mathrm {I I}}} ^ {t} \left\langle \frac {\mathrm {d} \boldsymbol {b} _ {k} (s)}{\mathrm {d} s}, \boldsymbol {x} _ {-} \right\rangle \mathrm {d} s = \boldsymbol {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \boldsymbol {x} _ {-} + \frac {\kappa_ {2} \sin^ {2} \Delta}{\sqrt {m} (1 + p)} \int_ {T _ {\mathrm {I I}}} ^ {t} e ^ {f _ {-} (s)} \mathrm {d} s, \\ \sum_ {k \in \mathcal {K} _ {-}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {-} = \sum_ {k \in \mathcal {K} _ {-}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \boldsymbol {x} _ {-} + \frac {m _ {-} \kappa_ {2} ^ {2} \sin^ {2} \Delta}{m (1 + p)} \int_ {T _ {\mathrm {I I}}} ^ {t} e ^ {f _ {-} (s)} \mathrm {d} s. \\ \end{array}
$$
Therefore, we have two-side bounds of  $f_{-}(t)$ :
$$
f _ {-} (t) \leq - \frac {m _ {-} \kappa_ {2} ^ {2} \sin^ {2} \Delta}{m (1 + p)} \int_ {T _ {\mathrm {I I}}} ^ {t} e ^ {f _ {-} (s)} \mathrm {d} s - \sum_ {k \in \mathcal {K} _ {-}} \frac {\kappa_ {2}}{\sqrt {m}} \pmb {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \pmb {x} _ {-} + \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \pmb {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \pmb {x} _ {-},
$$
$$
f _ {-} (t) \geq - \frac {m _ {-} \kappa_ {2} ^ {2} \sin^ {2} \Delta}{m (1 + p)} \int_ {T _ {\mathrm {I I}}} ^ {t} e ^ {f _ {-} (s)} \mathrm {d} s - \sum_ {k \in \mathcal {K} _ {-}} \frac {\kappa_ {2}}{\sqrt {m}} \pmb {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \pmb {x} _ {-}.
$$
For simplicity, we denote  $C_0 \coloneqq \frac{m - \kappa_2^2\sin^2\Delta}{m(1 + p)}$ ,  $C_-^- \coloneqq \sum_{k\in \mathcal{K}_-}\frac{\kappa_2}{\sqrt{m}}\pmb{b}_k^\top (T_{\mathrm{II}})\pmb{x}_-$  and  $C_-^+ \coloneqq \sum_{k\in \mathcal{K}_+}\frac{\kappa_2}{\sqrt{m}}\pmb{b}_k^\top (T_{\mathrm{II}})\pmb{x}_-$ . Then we have:
$$
- C _ {0} \int_ {T _ {\mathrm {I I}}} ^ {t} e ^ {f _ {-} (s)} \mathrm {d} s - C _ {-} ^ {-} \leq f _ {-} (t) \leq - C _ {0} \int_ {T _ {\mathrm {I I}}} ^ {t} e ^ {f _ {-} (s)} \mathrm {d} s - C _ {-} ^ {-} + C _ {-} ^ {+}.
$$
Let  $\Psi(t) \coloneqq \int_{T_{\mathrm{II}}}^{t} e^{f_{-}(s)} \, \mathrm{d}s$ , then  $\frac{\mathrm{d}\Psi(t)}{\mathrm{d}t} = e^{f_{-}(t)}$ . So  $\Psi(T_{\mathrm{II}}) = 0$  and
$$
- C _ {0} \Psi (t) - C _ {-} ^ {-} \leq \log \left(\frac {\mathrm {d} \Psi (t)}{\mathrm {d} t}\right) \leq - C _ {0} \Psi (t) - C _ {-} ^ {-} + C _ {-} ^ {+},
$$
$$
e ^ {- C _ {-} ^ {-}} e ^ {- C _ {0} \Psi (t)} \leq \frac {\mathrm {d} \Psi (t)}{\mathrm {d} t} \leq e ^ {- C _ {-} ^ {-} + C _ {-} ^ {+}} e ^ {- C _ {0} \Psi (t)}
$$
For the right hand, for any  $\epsilon \in (0,1)$ , we consider the auxiliary ODE:
$$
\left\{ \begin{array}{l} \frac {\mathrm {d} \mathcal {P} (t)}{\mathrm {d} t} = e ^ {- C _ {-} ^ {-} + (1 + \epsilon) C _ {-} ^ {+}} e ^ {- C _ {0} \mathcal {P} (t)}, \\ \mathcal {P} (T _ {\mathrm {I I}}) = 0. \end{array} \right.
$$
The solution of this ODE is  $\mathcal{P}(t) = \frac{1}{C_0}\log \left(1 + C_0e^{-C_-^+ +(1 + \epsilon)C_-^+}(t - T_{\mathrm{II}})\right)$ . From the Comparison Principle of ODEs, we have the upper bound for  $\Psi (t)$ :
$$
\Psi (t) \leq \mathcal {P} (t) = \frac {1}{C _ {0}} \log \left(1 + C _ {0} e ^ {- C _ {-} ^ {-} + (1 + \epsilon) C _ {-} ^ {+}} (t - T _ {\mathrm {H I}})\right).
$$
Taking  $\epsilon \to 0$ , we obtain
$$
\Psi (t) \leq \frac {1}{C _ {0}} \log \left(1 + C _ {0} e ^ {- C _ {-} ^ {-} + C _ {-} ^ {+}} (t - T _ {\mathrm {I I}})\right).
$$
In the similar way, we can derive the lower bound for  $\Psi (t)$
$$
\Psi (t) \geq \frac {1}{C _ {0}} \log \left(1 + C _ {0} e ^ {- C _ {-} ^ {-}} (t - T _ {\mathrm {I I}})\right).
$$
Consequently, we infer that
$$
f _ {-} (t) \leq - C _ {0} \Psi (t) - C _ {-} ^ {-} + C _ {-} ^ {+} \leq - \log \left(1 + C _ {0} e ^ {- C _ {-} ^ {-}} (t - T _ {\mathrm {I I}})\right) - C _ {-} ^ {-} + C _ {-} ^ {+},
$$
$$
f _ {-} (t) \geq - C _ {0} \Psi (t) - C _ {-} ^ {-} \geq - \log \left(1 + C _ {0} e ^ {- C _ {-} ^ {-} + C _ {-} ^ {+}} (t - T _ {\mathrm {I I}})\right) - C _ {-} ^ {-}.
$$
Noticing  $f_{-}(T_{\mathrm{II}}) = C_{-}^{+} - C_{-}^{-}$ , we obtain
$$
- \log \left(1 + C _ {0} e ^ {- C _ {-} ^ {-} + C _ {-} ^ {+}} (t - T _ {\mathrm {I I}})\right) - C _ {-} ^ {+} \leq f _ {-} (t) - f _ {-} (T _ {\mathrm {I I}}) \leq - \log \left(1 + C _ {0} e ^ {- C _ {-} ^ {-}} (t - T _ {\mathrm {I I}})\right).
$$
Noticing  $f_{-}(T_{\mathrm{II}}) = C_{-}^{+} - C_{-}^{-}$ , this inequality means
$$
\frac {e ^ {- C _ {-} ^ {+}}}{1 + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} (t - T _ {\mathrm {I I}})} \leq e ^ {f _ {-} (t) - f _ {-} (T _ {\mathrm {I I}})} \leq \frac {1}{1 + C _ {0} e ^ {(f _ {-} (T _ {\mathrm {I I}}) - C _ {-} ^ {+})} (t - T _ {\mathrm {I I}})}.
$$
where  $C_0 = \frac{m_{-}\kappa_2^2\sin^2\Delta}{m(1 + p)} = \Theta \left(\frac{\kappa_2^2\Delta^2}{p}\right)$ . Moreover, according to Lemma D.8 (S1)(S2), we have
$$
e ^ {f _ {-} (T _ {\mathrm {I I}})} = \Theta \left(p ^ {- \frac {\alpha \cos \Delta}{1 - \alpha \cos \Delta}}\right),
$$
$$
C _ {-} ^ {+} = \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (T _ {\mathrm {I I}}) \boldsymbol {x} _ {-} = \mathcal {O} \left(m _ {-} \frac {\kappa_ {2}}{\sqrt {m}} \frac {\sqrt {\kappa_ {1} \kappa_ {2}}}{\sqrt {m}}\right) = \mathcal {O} \left(\kappa_ {2} \sqrt {\kappa_ {1} \kappa_ {2}}\right).
$$
Step III. Evolution of  $f_{+}(t)$ .
Let  $k \in \mathcal{K}_{+}$ . According to Lemma E.1 (S2)(P2) and (S2)(P3), it follows that for any  $k \in \mathcal{K}_{+}$ , its dynamics during  $t \in [T_{\mathrm{II}}, T_{\mathrm{II}}^{\mathrm{PT*}}]$  is
$$
\frac {\mathrm {d} \boldsymbol {b} _ {k} (t)}{\mathrm {d} t} = \frac {\kappa_ {2}}{\sqrt {m}} \left\langle \boldsymbol {F} _ {+} (t), \boldsymbol {x} _ {+} \right\rangle ;
$$
$$
\mathrm {o r} \frac {\mathrm {d} \boldsymbol {b} _ {k} (t)}{\mathrm {d} t} = \frac {\kappa_ {2} p e ^ {- f _ {+} (t)}}{\sqrt {m} (1 + p)} \Big (\boldsymbol {x} _ {+} - \boldsymbol {x} _ {-} \cos \Delta \Big).
$$
$$
\begin{array}{l} f _ {+} (t) = \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} (t) \boldsymbol {x} _ {+} = \sum_ {k \in \mathcal {K} _ {+}} \frac {\kappa_ {2}}{\sqrt {m}} \boldsymbol {b} _ {k} ^ {\top} \left(T _ {\mathrm {I I}}\right) \boldsymbol {x} _ {+} + \sum_ {k \in \mathcal {K} _ {+}} \int_ {T _ {\mathrm {I I}}} ^ {t} \frac {\kappa_ {2}}{\sqrt {m}} \left\langle \frac {\mathrm {d} \boldsymbol {b} _ {k} (t)}{\mathrm {d} s}, \boldsymbol {x} _ {+} \right\rangle \mathrm {d} s \\ = f _ {+} \left(T _ {\mathrm {I I}}\right) + \frac {\kappa_ {2}}{\sqrt {m}} \sum_ {k \in \mathcal {K} _ {+}} \int_ {T _ {\mathrm {I I}}} ^ {t} \left\langle \frac {\mathrm {d} \boldsymbol {b} _ {k} (t)}{\mathrm {d} s}, \boldsymbol {x} _ {+} \right\rangle \mathrm {d} s. \\ \end{array}
$$
On the one hand, for any  $t\in [T_{\mathrm{II}},T_{\mathrm{II}}^{\mathrm{PT*}}]$  , we have the lower bound:
$$
f _ {+} (t) \geq f _ {+} \left(T _ {\mathrm {I I}}\right) + \frac {\kappa_ {2} ^ {2}}{m} \sum_ {k \in \mathcal {K} _ {+}} \int_ {T _ {\mathrm {I I}}} ^ {t} \min  \left\{\langle \boldsymbol {F} _ {+} (s), \boldsymbol {x} _ {+} \rangle , \frac {p e ^ {- f _ {+} (s)}}{1 + p} \sin^ {2} \Delta \right\} \mathrm {d} s \geq f _ {+} \left(T _ {\mathrm {I I}}\right).
$$
On the other hand, for any  $t \in [T_{\mathrm{II}}, T_{\mathrm{II}}^{\mathrm{PT*}}]$ , we can derive an upper bound:
$$
\begin{array}{l} f _ {+} (t) \leq f _ {+} \left(T _ {\mathrm {I I}}\right) + \frac {\kappa_ {2} ^ {2}}{m} \sum_ {k \in \mathcal {K} _ {+}} \int_ {T _ {\mathrm {I I}}} ^ {t} \max  \left\{\langle \boldsymbol {F} _ {+} (s), \boldsymbol {x} _ {+} \rangle , \frac {p e ^ {- f _ {+} (s)}}{1 + p} \sin^ {2} \Delta \right\} d s \\ \leq f _ {+} (T _ {\mathrm {I I}}) + \frac {\kappa_ {2} ^ {2}}{m} \sum_ {k \in \mathcal {K} _ {+}} \int_ {T _ {\mathrm {I I}}} ^ {t} \max  \left\{\frac {p e ^ {- f _ {+} (s)}}{1 + p}, \frac {p e ^ {- f _ {+} (s)}}{1 + p} \sin^ {2} \Delta \right\} \mathrm {d} s \\ \leq f _ {+} (T _ {\mathrm {I I}}) + \frac {\kappa_ {2} ^ {2}}{m} \sum_ {k \in \mathcal {K} _ {+}} \int_ {T _ {\mathrm {I I}}} ^ {t} \frac {p e ^ {- f _ {+} (s)}}{1 + p} \mathrm {d} s \leq f _ {+} (T _ {\mathrm {I I}}) + \frac {\kappa_ {2} ^ {2} m _ {+}}{m} \frac {p e ^ {- f _ {+} (T _ {\mathrm {I I}})}}{1 + p} (t - T _ {\mathrm {I I}}). \\ \end{array}
$$
$$
\exp \left(- \frac {\kappa_ {2} ^ {2} m _ {+}}{m} \frac {p e ^ {- f _ {+} (T _ {\mathrm {I I}})}}{1 + p} (t - T _ {\mathrm {I I}})\right) \leq e ^ {f _ {+} (T _ {\mathrm {I I}}) - f _ {+} (t)} \leq 1,
$$
$$
\frac {\kappa_ {2} ^ {2} m _ {+}}{m} \frac {p e ^ {- f _ {+} (T _ {\mathrm {I I}})}}{1 + p} \stackrel {\mathrm {L e m m a D . 8}} {=} \Theta \left(\kappa_ {2} ^ {2} e ^ {- f _ {+} (T _ {\mathrm {I I}})}\right) = \Theta \left(\kappa_ {2} ^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}}\right).
$$
Lemma E.3 (Nearly fixed vector filed in Phase III*).
There exist absolute constants  $Q_{1}, Q_{2} > 0$ , such that: For any time  $T_{\mathrm{fix}} \in [T_{\mathrm{II}}, +\infty)$ , if we choose  $\kappa_{1}, \kappa_{2}$  s.t.
$$
\kappa_ {2} ^ {2} \Big (T _ {\mathrm {f i x}} \wedge T _ {\mathrm {I I}} ^ {\mathrm {P T} *} - T _ {\mathrm {I I}} \Big) p ^ {- \frac {1}{1 - \alpha \cos \Delta}} = \mathcal {O} (\Delta^ {2}), \quad \kappa_ {2} ^ {2} \sqrt {\frac {\kappa_ {1}}{\kappa_ {2}}} = \mathcal {O} (\Delta^ {2}),
$$
then for any  $t \in [T_{\mathrm{II}}, T_{\mathrm{fix}} \wedge T_{\mathrm{II}}^{\mathrm{PT*}}]$ , we have
$$
\langle \boldsymbol {F} _ {+} (t), \boldsymbol {x} _ {+} \rangle \leq \frac {Q _ {1}}{2} \Delta^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}}, \quad \langle \boldsymbol {F} _ {+} (t), \boldsymbol {x} _ {-} \rangle \geq - \frac {Q _ {2}}{2} \Delta^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}}.
$$
Proof of Lemma E.3.
For simplicity, we denote  $\delta_T\coloneqq T_{\mathrm{fix}}\wedge T_{\mathrm{II}}^{\mathrm{PT}*} - T_{\mathrm{II}}$  From Lemma E.2, for any  $t\in [T_{\mathrm{II}},T_{\mathrm{fix}}\wedge T_{\mathrm{II}}^{\mathrm{PT*}}]$  we have
$$
e ^ {f _ {-} (t) - f _ {-} (T _ {\mathrm {I I}})} - 1 \leq \frac {1}{1 + C _ {0} e ^ {(f _ {-} (T _ {\mathrm {I I}}) - C _ {1})} \delta_ {T}} - 1 \leq 0,
$$
$$
e ^ {f _ {-} (t) - f _ {-} (T _ {\mathrm {I I}})} - 1 \geq \frac {e ^ {- C _ {1}}}{1 + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}} - 1 = \frac {e ^ {- C _ {1}} - 1 - C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}}{1 + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}} \geq - \frac {C _ {1} + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}}{1 + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}}
$$
$$
e ^ {f _ {+} (T _ {\mathrm {I I}}) - f _ {+} (t)} - 1 \leq 0,
$$
$$
e ^ {f _ {+} (T _ {\mathrm {I I}}) - f _ {+} (t)} - 1 \geq e ^ {- C _ {2} \delta_ {T}} - 1 \geq - C _ {2} \delta_ {T}.
$$
Recalling Lemma D.8 (S1), there exists absolute constants  $Q_{1}, Q_{2} > 0$  such that
$$
\langle \pmb {F} _ {+} (T _ {\mathrm {I I}}), \pmb {x} _ {+} \rangle = \frac {p e ^ {- f _ {+} (T _ {\mathrm {I I}})}}{1 + p} - \frac {e ^ {f _ {-} (T _ {\mathrm {I I}})}}{1 + p} \cos \Delta \geq Q _ {1} \Delta^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}},
$$
$$
\langle \boldsymbol {F} _ {+} (T _ {\mathrm {I I}}), \boldsymbol {x} _ {-} \rangle = \frac {p e ^ {- f _ {+} (T _ {\mathrm {I I}})}}{1 + p} \cos \Delta - \frac {e ^ {f _ {-} (T _ {\mathrm {I I}})}}{1 + p} \leq - Q _ {2} \Delta^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}}.
$$
Step I. Bounding the term  $\langle \pmb{F}_+(t), \pmb{x}_+\rangle$ .
$$
\begin{array}{l} | \langle \boldsymbol {F} _ {+} (t), \boldsymbol {x} _ {+} \rangle - \langle \boldsymbol {F} _ {+} (T _ {\mathrm {I I}}), \boldsymbol {x} _ {+} \rangle | \\ = \left| \frac {p}{1 + p} e ^ {- f _ {+} (t)} - \frac {p}{1 + p} e ^ {- f _ {+} \left(T _ {\mathrm {I I}}\right)} - \frac {e ^ {f _ {-} (t)}}{1 + p} \cos \Delta + \frac {e ^ {f _ {-} \left(T _ {\mathrm {I I}}\right)}}{1 + p} \cos \Delta \right| \\ \leq \left| \frac {p}{1 + p} e ^ {- f _ {+} (t)} - \frac {p}{1 + p} e ^ {- f _ {+} \left(T _ {\mathrm {I I}}\right)} \right| + \left| \frac {e ^ {f _ {-} (t)}}{1 + p} \cos \Delta - \frac {e ^ {f _ {-} \left(T _ {\mathrm {I I}}\right)}}{1 + p} \cos \Delta \right| \\ \leq \frac {p}{1 + p} e ^ {- f _ {+} (T _ {\mathrm {I I}})} \left| e ^ {f _ {+} (T _ {\mathrm {I I}}) - f _ {+} (t)} - 1 \right| + \frac {e ^ {f _ {-} (T _ {\mathrm {I I}})}}{1 + p} \left| e ^ {f _ {-} (t) - f _ {-} (T _ {\mathrm {I I}})} - 1 \right| \\ \leq \frac {p}{1 + p} e ^ {- f _ {+} (T _ {\mathrm {I I}})} C _ {2} \delta_ {T} + \frac {e ^ {f _ {-} (T _ {\mathrm {I I}})}}{1 + p} \frac {C _ {1} + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}}{1 + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}} \\ \end{array}
$$
To ensure  $|\langle \pmb{F}_{+}(t),\pmb{x}_{+}\rangle -\langle \pmb{F}_{+}(T_{\mathrm{II}}),\pmb{x}_{+}\rangle |\leq \frac{1}{2} Q_{1}\Delta^{2}p^{-\frac{1}{1 - \alpha\cos\Delta}}$  , we need only select parameters such that
$$
\frac {p}{1 + p} e ^ {- f _ {+} (T _ {\mathrm {I I}})} C _ {2} \delta_ {T} + \frac {e ^ {f _ {-} (T _ {\mathrm {I I}})}}{1 + p} \frac {C _ {1} + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}}{1 + C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T}} \leq \frac {1}{2} Q _ {1} \Delta^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}}.
$$
From Lemma E.2 and Lemma D.8, we have:
$$
C _ {0} = \Theta \left(\frac {\kappa_ {2} ^ {2} \Delta^ {2}}{p}\right), C _ {1} = \mathcal {O} \left(\kappa_ {2} \sqrt {\kappa_ {1} \kappa_ {2}}\right), C _ {2} = \Theta \left(\kappa_ {2} ^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}}\right),
$$
$$
e ^ {- f _ {+} (T _ {\mathrm {I I}})} = \Theta \left(p ^ {- \frac {1}{1 - \alpha \cos \Delta}}\right), e ^ {f _ {-} (T _ {\mathrm {I I}})} = \Theta \left(p ^ {- \frac {\alpha \cos \Delta}{1 - \alpha \cos \Delta}}\right).
$$
Therefore, if we take
$$
C _ {0} e ^ {f _ {-} (T _ {\mathrm {I I}})} \delta_ {T} = \Theta \left(\kappa_ {2} ^ {2} \Delta^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}} \delta_ {T}\right) = \mathcal {O} (1),
$$
$$
\begin{array}{l} \frac {p}{1 + p} e ^ {- f _ {+} \left(T _ {\mathrm {I I}}\right)} C _ {2} \delta_ {T} + \frac {e ^ {f _ {-} \left(T _ {\mathrm {I I}}\right)}}{1 + p} \frac {C _ {1} + C _ {0} e ^ {f _ {-} \left(T _ {\mathrm {I I}}\right)} \delta_ {T}}{1 + C _ {0} e ^ {f _ {-} \left(T _ {\mathrm {I I}}\right)} \delta_ {T}} \\ = \Theta \left(\kappa_ {2} ^ {2} p ^ {- \frac {2}{1 - \alpha \cos \Delta}} \delta_ {T}\right) + \Theta \left(p ^ {- \frac {1}{1 - \alpha \cos \Delta}} \left(\mathcal {O} \left(\kappa_ {2} \sqrt {\kappa_ {1} \kappa_ {2}}\right) + \kappa_ {2} ^ {2} \Delta^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}} \delta_ {T}\right)\right) \\ = \Theta \left(\kappa_ {2} ^ {2} p ^ {- \frac {1}{1 - \alpha \cos \Delta}} \left(\delta_ {T} p ^ {- \frac {1}{1 - \alpha \cos \Delta}} + \mathcal {O} (\sqrt {\frac {\kappa_ {1}}{\kappa_ {2}}})\right)\right). \\ \end{array}
$$
If we can take
$$
\kappa_ {2} ^ {2} \delta_ {T} p ^ {- \frac {1}{1 - \alpha \cos \Delta}} = \mathcal {O} (\Delta^ {2}), \quad \kappa_ {2} ^ {2} \sqrt {\frac {\kappa_ {1}}{\kappa_ {2}}} = \mathcal {O} (\Delta^ {2}),
$$