SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Intuition on Theorem 4.2. Before we dive in to the rigorous proofs, we provide an intuition on Theorem 4.2. Lemma B.2 establishes that  $p_t$  is bounded and  $q_t$  monotonically increases slowly, with an increment of  $\mathcal{O}(\eta^2)$  per step. Lemma 2.1 shows that the map  $f_{q_t}(p) = \left(1 - \frac{2r(p_t)}{q_t}\right)p_t$  has a stable 2-period orbit  $\{\pm \hat{r}(q_t)\}$  when  $q_t \in (0,1)$ . Consequently, when  $q_t$  is treated as a fixed value,  $p_t$  converges to the orbit  $\{\pm \hat{r}(q_t)\}$ , leading to  $s_t$  converging to 1. In the (early) short-term dynamics,  $q_t$  is nearly fixed for small step size  $\eta$ , and hence  $s_t$  converges to 1. In long-term dynamics perspective,  $q_t$  gradually increases and at the same time,  $s_t$  stays near the value 1. In Theorem 4.2, we prove that it takes only  $t_a = \mathcal{O}_{\delta,\ell}(\log(\eta^{-1}))$  time steps for  $s_t$  to converge close to 1 (Phase I,  $t \leq t_a$ ), and after that,  $s_t$  stay close to 1 for the remaining iterations (Phase II,  $t > t_a$ ).
We informally summarize the lemmas used in the proof of Theorem 4.2. Lemma B.3 states that in the early phase of training, there exists a time step  $t_0$  where  $s_{t_0}$  becomes smaller or equal to  $\frac{2}{2 - r(1)}$ , which is smaller than  $2(1 + \eta^2)^{-1}$ . Lemma B.5 demonstrates that if  $s_t$  is smaller than  $2(1 + \eta^2)^{-1}$  and  $|p_t|\geq \hat{r} (1 - \frac{\delta}{4})$ , then  $|s_t - 1|$  decreases exponentially. For the case where  $|p_t| < \hat{r} (1 - \frac{\delta}{4})$ , Lemma B.6 proves that  $|p_t|$  increases at an exponential rate. Moreover, Lemma B.4 shows that if  $s_t < 1$  at some time step, then  $s_{t + 1}$  is upper bounded by  $1 + \mathcal{O}(\eta^{2})$ . Combining these findings, Proposition B.7 establishes that in the early phase of training, there exists a time step  $t_a^*$  such that  $s_{t_a^*} = 1 + \mathcal{O}_{\delta ,\ell}(\eta^2)$ . Lastly, Lemma B.8 demonstrates that if  $s_t = 1 + \mathcal{O}_{\delta ,\ell}(\eta^2)$ , then  $|s_t - 1 - h(p_t)\eta^2|$  decreases exponentially.
Now we prove Theorem 4.2, starting with the lemma below.
Lemma B.3. There exists a time step  $t_0 = \mathcal{O}_{\delta, \ell}(1)$  such that  $s_{t_0} \leq \frac{2}{2 - r(1)}$ .
Proof. We start by proving the following statement: for any  $0 \leq t \leq T$ , if  $\frac{2}{2 - r(1)} < s_t < 2r(1)^{-1}$ , then  $s_{t + 1} < 2r(1)^{-1}$  and  $|p_{t + 1}| \leq (1 - \frac{r(1)}{2})|p_t|$ . Suppose that  $\frac{2}{2 - r(1)} < s_t < 2r(1)^{-1}$ . Then from Eq. (6), it holds that
$$
\left| \frac {p _ {t + 1}}{p _ {t}} \right| = \left| 1 - \frac {2}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r (p _ {t}) ^ {2} \right| \leq \left| 1 - \frac {2}{s _ {t}} \right| + \eta^ {2} \leq 1 - r (1) + \eta^ {2} \leq 1 - \frac {r (1)}{2},
$$
for small step size  $\eta$ . Hence,  $|p_{t + 1}| \leq (1 - \frac{r(1)}{2}) |p_t|$ . Now we prove  $s_{t + 1} < 2r(1)^{-1}$ . Assume the contrary that  $s_{t + 1} \geq 2r(1)^{-1}$ . Then,  $r(p_{t + 1}) = \frac{q_{t + 1}}{s_{t + 1}} < q_{t + 1} < 1 - \frac{\delta}{2}$  so that  $|p_{t + 1}| \geq \hat{r}(1 - \frac{\delta}{2})$ . By Mean Value Theorem, there exists  $p_t^* \in (|p_{t + 1}, |p_t|)$  such that (recall that  $\frac{r'(p)}{r(p)^2} < 0$  for  $p > 0$ )
$$
\begin{array}{l} \frac {1}{r \left(p _ {t + 1}\right)} = \frac {1}{r \left(\left| p _ {t} \right| - \left(\left| p _ {t} \right| - \left| p _ {t + 1} \right|\right)\right)} = \frac {1}{r \left(p _ {t}\right)} + \frac {r ^ {\prime} \left(p _ {t} ^ {*}\right)}{r \left(p _ {t} ^ {*}\right) ^ {2}} \left(\left| p _ {t} \right| - \left| p _ {t + 1} \right|\right) \\ \leq \frac {1}{r (p _ {t})} + \frac {r ^ {\prime} (| p _ {t + 1} |)}{r (| p _ {t + 1} |) ^ {2}} \left(\frac {r (1) | p _ {t} |}{2}\right) \\ \leq \frac {1}{r (p _ {t})} - \frac {\left| r ^ {\prime} (\hat {r} (1 - \frac {\delta}{2})) \right|}{(1 - \frac {\delta}{2}) ^ {2}} \left(\frac {r (1) \hat {r} (1 - \frac {\delta}{2})}{2}\right) \\ = \frac {1}{r \left(p _ {t}\right)} - \Omega_ {\delta , \ell} (1), \\ \end{array}
$$
where we used Assumption 4.3 (i) and  $\hat{r}\big(1 - \frac{\delta}{2}\big)\leq |p_{t + 1}|\leq \big(1 - \frac{r(1)}{2}\big)|p_t|\leq |p_t|$ . Consequently,
$$
s _ {t + 1} = \frac {q _ {t + 1}}{r (p _ {t + 1})} = (1 + \mathcal {O} (\eta^ {2})) q _ {t} \left(\frac {1}{r (p _ {t})} - \Omega_ {\delta , \ell} (1)\right) \leq \frac {q _ {t}}{r (p _ {t})} = s _ {t} <   2 r (1) ^ {- 1},
$$
for small step size  $\eta$ . This gives a contradiction to our assumption that  $s_{t+1} \geq 2r(1)^{-1}$ . Hence, we can conclude that  $s_{t+1} < 2r(1)^{-1}$ , as desired.
We proved that for any  $0 \leq t \leq T$ , if  $\frac{2}{2 - r(1)} < s_t < 2r(1)^{-1}$ , it holds that  $s_{t + 1} < 2r(1)^{-1}$  and  $|p_{t + 1}| \leq (1 - \frac{r(1)}{2})|p_t|$ . At initialization,  $|p_0| \leq 1$  and  $q_0 < 1$ , so that  $s_0 < r(1)^{-1}$ . If  $s_0 \leq \frac{2}{2 - r(1)}$ , then  $t_0 = 0$  is the desired time step. Suppose that  $s_0 > \frac{2}{2 - r(1)}$ . Then, we have  $s_1 < 2r(1)^{-1}$  and  $|p_1| \leq (1 - \frac{r(1)}{2})|p_0| \leq 1 - \frac{r(1)}{2}$ . Then we have either  $s_1 \leq \frac{2}{2 - r(1)}$ , or  $\frac{2}{2 - r(1)} < s_1 < 2r(1)^{-1}$ . In the previous case,  $t_0 = 1$  is the desired time step. In the latter case, we can repeat the same argument and obtain  $s_2 < 2r(1)^{-1}$  and  $|p_2| \leq (1 - \frac{r(1)}{2})^2$ . By inductively repeating the same argument, we can obtain a time step  $t_0 \leq \log (\hat{r}(1 - \frac{\delta}{2})) / \log (1 - \frac{r(1)}{2}) = \mathcal{O}_{\delta,\ell}(1)$  such that either  $s_{t_0} \leq \frac{2}{2 - r(1)}$ , or  $|p_{t_0}| \leq \hat{r}(1 - \frac{\delta}{2})$ . In the latter case,  $r(p_{t_0}) \geq 1 - \frac{\delta}{2} > q_{t_0}$ , and hence  $s_{t_0} < 1 < \frac{2}{2 - r(1)}$ . Therefore,  $t_0 = \mathcal{O}_{\delta,\ell}(1)$  is the desired time step satisfying  $s_{t_0} \leq \frac{2}{2 - r(1)}$ .
According to Lemma B.3, there exists a time step  $t_0 = \mathcal{O}_{\delta,\ell}(1)$  such that  $s_{t_0} \leq \frac{2}{2-r(1)} < 2(1+\eta^2)^{-1}$  for small step size  $\eta$ . Now we prove the lemma below.
Lemma B.4. Suppose that  $s_t \leq 1$ . Then, it holds that  $s_{t+1} \leq 1 + \mathcal{O}(\eta^2)$ .
Proof. For any  $p \in (0, \hat{r}(\frac{q_t}{2}))$ , we have  $r(p) \geq \frac{q_t}{2}$  so that  $|f_{q_t}(p)| = (-1 + \frac{2r(p)}{q_t})p$ . Hence,
$$
\frac {\partial}{\partial p} | f _ {q _ {t}} (p) | = \frac {2 r (p)}{q _ {t}} \left(1 + \frac {p r ^ {\prime} (p)}{r (p)}\right) - 1,
$$
for any  $p \in (0, \hat{r}\left(\frac{q_t}{2}\right))$ . By Assumption 4.3 (ii) and convexity of  $\ell$ , both  $r(p)$  and  $1 + \frac{pr'(p)}{r(p)} = \frac{\ell''(p)}{r(p)}$  are positive, decreasing function on  $(0, \hat{r}\left(\frac{q_t}{2}\right))$ . Consequently,  $\frac{\partial}{\partial p} |f_{q_t}(p)|$  is a decreasing function on  $(0, \hat{r}\left(\frac{q_t}{2}\right))$ .
Now note that  $\frac{q_t}{2} < q_t < 1$ , which means  $\hat{r}(1) = 0 < \hat{r}(q_t) < \hat{r}\left(\frac{q_t}{2}\right)$  by the definition of  $\hat{r}$ . Note that  $\frac{\partial}{\partial p} |f_{q_t}(p)|$  at  $p = \hat{r}(q_t)$  evaluates to
$$
\frac {\partial}{\partial p} | f _ {q _ {t}} (\hat {r} (q _ {t})) | = 1 + \frac {2 \hat {r} (q _ {t}) r ^ {\prime} (\hat {r} (q _ {t}))}{r (\hat {r} (q _ {t}))} \geq 1 + \frac {2 \hat {r} (c _ {0}) r ^ {\prime} (\hat {r} (c _ {0}))}{r (\hat {r} (c _ {0}))} \geq 0,
$$
where the first inequality used Assumption 4.3 (ii) and  $\hat{r}(q_t) < \hat{r}(c_0)$ , which comes from  $q_t > c_0 \coloneqq \max\{r(z_0), \frac{1}{2}\}$ . The second inequality holds because  $q_t > c_0 \geq r(z_0)$  where  $z_0 \coloneqq \sup_{z} \left\{\frac{z r'(z)}{r(z)} \geq -\frac{1}{2}\right\}$ , from the statement of Theorem 4.2.
Therefore, since  $\frac{\partial}{\partial p} |f_{q_t}(p)|$  is decreasing on  $(0,\hat{r} (\frac{q_t}{2}))$  and is nonnegative at  $\hat{r} (q_t)$ , for any  $p\in (0,\hat{r} (q_t))$ , it holds that  $\frac{\partial}{\partial p} |f_{q_t}(p)|\geq 0$ . In other words,  $|f_{q_t}(p)|$  is an increasing function on  $(0,\hat{r} (q_t))$ . Since  $0\leq s_t\leq 1$ , we have  $|p_t|\leq \hat{r} (q_t)$  and it holds that
$$
| p _ {t + 1} | = \left(- 1 + \frac {2}{s _ {t}} - \eta^ {2} p _ {t} ^ {2} r (p _ {t}) ^ {2}\right) | p _ {t} | \leq \left(- 1 + \frac {2}{s _ {t}}\right) | p _ {t} | = | f _ {q _ {t}} (p _ {t}) | \leq | f _ {q _ {t}} (\hat {r} (q _ {t})) | = \hat {r} (q _ {t}).
$$
Therefore, with this inequality and Lemma B.2, we can conclude that
$$
s _ {t + 1} = \frac {q _ {t + 1}}{r (p _ {t + 1})} = \frac {q _ {t}}{r (p _ {t + 1})} (1 + \mathcal {O} (\eta^ {2})) \leq \frac {q _ {t}}{r (\hat {r} (q _ {t}))} (1 + \mathcal {O} (\eta^ {2})) = 1 + \mathcal {O} (\eta^ {2}).
$$
Using Lemma B.4, we prove the following lemma.
Lemma B.5. For any  $0 \leq t \leq T$ , if  $s_t < 2(1 + \eta^2)^{-1}$ ,  $|s_t - 1| > \frac{\eta^2}{2}$ , and  $r(p_t) \leq 1 - \frac{\delta}{4}$ , then
$$
\left| s _ {t + 1} - 1 \right| \leq (1 - d) \left| s _ {t} - 1 \right| + \mathcal {O} (\eta^ {2}),
$$
where  $d\in (0,\frac{1}{2} ]$  is a constant which depends on  $\delta$  and  $\ell$
Proof. By Eq. (6) and 1-Lipschitzness of  $\ell$ ,
$$
\frac {p _ {t + 1}}{p _ {t}} = 1 - \frac {2}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r (p _ {t}) ^ {2} <   1 - (1 + \eta^ {2}) + \eta^ {2} = 0,
$$
so that  $p_t$  and  $p_{t+1}$  have opposite signs. By Mean Value Theorem, there exists  $\theta_t$  between  $-1$  and  $(1 - \frac{2}{s_t} + \eta^2 p_t^2 r(p_t)^2)$  satisfying
$$
\begin{array}{l} \frac {1}{r \left(p _ {t + 1}\right)} = \frac {1}{r \left(- p _ {t} + \left(\frac {2 \left(s _ {t} - 1\right)}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r \left(p _ {t} ^ {2}\right)\right) p _ {t}\right)} \\ = \frac {1}{r (- p _ {t})} - \frac {r ^ {\prime} (\theta_ {t} p _ {t})}{r (\theta_ {t} p _ {t}) ^ {2}} \left(\frac {2 (s _ {t} - 1)}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r (p _ {t}) ^ {2}\right) p _ {t} \\ = \frac {1}{r \left(p _ {t}\right)} - \frac {\left| r ^ {\prime} \left(\theta_ {t} p _ {t}\right) \right|}{r \left(\theta_ {t} p _ {t}\right) ^ {2}} \left(\frac {2 \left(s _ {t} - 1\right)}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r \left(p _ {t}\right) ^ {2}\right) \left| p _ {t} \right|, \tag {8} \\ \end{array}
$$
where the last equality used the fact that  $p_t$  and  $\theta_t p_t$  have opposite signs and  $r'(z)$  and  $z$  have opposite signs. Note that  $|\theta_t p_t|$  is between  $|p_t|$  and  $|p_{t+1}|$ . Consequently, the value  $\frac{|r'(\theta_t p_t)|}{r(\theta_t p_t)^2}$  is between  $\frac{|r'(p_t)|}{r(p_t)^2}$  and  $\frac{|r'(p_{t+1})|}{r(p_{t+1})^2}$  by Assumption 4.3 (i). We will prove the current lemma based on Eq. (8). We divide into following three cases: (1)  $s_t \geq 1$  and  $s_{t+1} \geq 1$ , (2)  $s_t \geq 1$  and  $s_t < 1$ , and (3)  $s_t < 1$ .
Case 1. Suppose that  $s_t \geq 1$  and  $s_{t+1} \geq 1$ . Here, we have  $|p_t| \geq \hat{r}(q_t) \geq \hat{r}(1 - \frac{\delta}{2})$  and similarly  $|p_{t+1}| \geq \hat{r}(1 - \frac{\delta}{2})$ . By Assumption 4.3 (i),  $\frac{|r'(\theta_t p_t)|}{r(\theta_t p_t)^2} \geq \frac{|r'(\hat{r}(1 - \frac{\delta}{2}))|}{(1 - \frac{\delta}{2})^2}$ . Hence, Eq. (8) gives
$$
\frac {1}{r (p _ {t + 1})} \leq \frac {1}{r (p _ {t})} - \frac {| r ^ {\prime} (\hat {r} (1 - \frac {\delta}{2})) |}{(1 - \frac {\delta}{2}) ^ {2}} \left(\frac {2 (s _ {t} - 1)}{s _ {t}}\right) \hat {r} \left(1 - \frac {\delta}{2}\right).
$$
Consequently, by Lemma B.2,
$$
\begin{array}{l} s _ {t + 1} = \frac {q _ {t} (1 + \mathcal {O} (\eta^ {2}))}{r (p _ {t + 1})} = \frac {q _ {t}}{r (p _ {t + 1})} + \mathcal {O} (\eta^ {2}) \leq s _ {t} - \frac {| r ^ {\prime} (\hat {r} (1 - \frac {\delta}{2})) |}{(1 - \frac {\delta}{2}) ^ {2}} \left(\frac {2 (s _ {t} - 1)}{s _ {t}}\right) \hat {r} \left(1 - \frac {\delta}{2}\right) q _ {t} + \mathcal {O} (\eta^ {2}) \\ \leq s _ {t} - \frac {| r ^ {\prime} (\hat {r} (1 - \frac {\delta}{2})) |}{(1 - \frac {\delta}{2}) ^ {2}} (s _ {t} - 1) \hat {r} \left(1 - \frac {\delta}{2}\right) \frac {1}{2} + \mathcal {O} (\eta^ {2}) \\ \leq s _ {t} - \frac {\hat {r} (1 - \frac {\delta}{2}) | r ^ {\prime} (\hat {r} (1 - \frac {\delta}{2})) |}{2 (1 - \frac {\delta}{2}) ^ {2}} (s _ {t} - 1) + \mathcal {O} (\eta^ {2}), \\ \end{array}
$$
where we used  $q_{t} > c_{0} \geq \frac{1}{2}$  and  $s_t < 2(1 + \eta^2)^{-1} < 2$ . Therefore, we can obtain the following inequality:
$$
0 \leq s _ {t + 1} - 1 \leq \left(1 - \frac {\hat {r} (1 - \frac {\delta}{2}) | r ^ {\prime} (\hat {r} (1 - \frac {\delta}{2})) |}{2 (1 - \frac {\delta}{2}) ^ {2}}\right) (s _ {t} - 1) + \mathcal {O} (\eta^ {2}).
$$
Case 2. Suppose that  $s_t \geq 1$  and  $s_{t+1} < 1$ . Here, we have  $r(p_{t+1}) > q_{t+1} \geq q_t \geq r(p_t)$ , so that  $|p_{t+1}| < |p_t|$ . Consequently,  $\frac{|r'(\theta_t p_t)|}{r(\theta_t p_t)^2} \leq \frac{|r'(p_t)|}{r(p_t)^2}$  by Assumption 4.3 (i). Hence, we can deduce from Eq. (8) that
$$
\begin{array}{l} \frac {1}{r \left(p _ {t + 1}\right)} \geq \frac {1}{r \left(p _ {t}\right)} - \frac {\left| r ^ {\prime} \left(p _ {t}\right) \right|}{r \left(p _ {t}\right) ^ {2}} \left(\frac {2 \left(s _ {t} - 1\right)}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r \left(p _ {t}\right) ^ {2}\right) | p _ {t} | \\ = \frac {1}{r \left(p _ {t}\right)} - \frac {2 \left| p _ {t} r ^ {\prime} \left(p _ {t}\right) \right|}{r \left(p _ {t}\right) q _ {t}} \left(s _ {t} - 1\right) - \eta^ {2} \left| p _ {t} ^ {3} r ^ {\prime} \left(p _ {t}\right) \right| \\ \geq \frac {1}{r \left(p _ {t}\right)} - \frac {2 \left| p _ {t} r ^ {\prime} \left(p _ {t}\right) \right|}{r \left(p _ {t}\right) q _ {t}} \left(s _ {t} - 1\right) - \eta^ {2} p _ {t} ^ {2} r \left(p _ {t}\right) \\ = \frac {1}{r \left(p _ {t}\right)} + \frac {2 p _ {t} r ^ {\prime} \left(p _ {t}\right)}{r \left(p _ {t}\right) q _ {t}} \left(s _ {t} - 1\right) - \mathcal {O} \left(\eta^ {2}\right), \\ \end{array}
$$
where we used  $|p_t r'(p_t)| \leq r(p_t)$  since  $1 + \frac{p_t r'(p_t)}{r(p_t)} = \frac{\ell''(p_t)}{r(p_t)} > 0$  and  $|p_t| \leq 4$  by Lemma B.2. Consequently, by Lemma B.2 ( $q_t \leq q_{t+1}$ ) and Assumption 4.3 (ii),
$$
s _ {t + 1} \geq \frac {q _ {t}}{r \left(p _ {t + 1}\right)} \geq s _ {t} + \frac {2 p _ {t} r ^ {\prime} \left(p _ {t}\right)}{r \left(p _ {t}\right)} \left(s _ {t} - 1\right) - \mathcal {O} \left(\eta^ {2}\right) \geq s _ {t} + \frac {8 r ^ {\prime} (4)}{r (4)} \left(s _ {t} - 1\right) - \mathcal {O} \left(\eta^ {2}\right).
$$
Note that  $1 > 1 + \frac{4r'(4)}{r(4)} = \frac{\ell''(4)}{r(4)} > 0$  holds by convexity of  $\ell$ . Therefore, we can obtain the following inequality:
$$
0 \leq 1 - s _ {t + 1} \leq - \left(1 + \frac {8 r ^ {\prime} (4)}{r (4)}\right) \left(s _ {t} - 1\right) + \mathcal {O} (\eta^ {2}),
$$
where  $-1 < 1 + \frac{8r'(4)}{r(4)} < 1$ .
Case 3. Suppose that  $s_t < 1$ . By Lemma B.4, it holds that  $s_{t+1} \leq 1 + \mathcal{O}(\eta^2)$ . Moreover, we assumed  $r(p_t) \leq 1 - \frac{\delta}{4}$ , so that  $|p_t| \geq \hat{r}(1 - \frac{\delta}{4})$ . We also have
$$
| p _ {t + 1} | = \left(- 1 + \frac {2}{s _ {t}} - \eta^ {2} p _ {t} ^ {2} r (p _ {t} ^ {2})\right) | p _ {t} | \geq \left(- 1 + \frac {2}{1 - \frac {\eta^ {2}}{2}} - \eta^ {2}\right) | p _ {t} | > | p _ {t} | \geq \hat {r} \left(1 - \frac {\delta}{4}\right),
$$
where we used the assumption  $|s_t - 1| > \frac{\eta^2}{2}$ , and  $|pr(p)| = |\ell'(p)| \leq 1$  due to 1-Lipschitzness of  $\ell$ . Consequently, by Assumption 4.3 (i), it holds that  $\frac{|r'(\theta_t p_t)|}{r(\theta_t p_t)^2} \geq \frac{|r'(\hat{r}(1 - \frac{\delta}{4}))|}{(1 - \frac{\delta}{4})^2}$ . Hence, by Eq. (8), we have
$$
\begin{array}{l} \frac {1}{r \left(p _ {t + 1}\right)} \geq \frac {1}{r \left(p _ {t}\right)} + \frac {\left| r ^ {\prime} \left(\hat {r} \left(1 - \frac {\delta}{4}\right)\right) \right|}{\left(1 - \frac {\delta}{4}\right) ^ {2}} \left(\frac {2 \left(1 - s _ {t}\right)}{s _ {t}}\right) \hat {r} \left(1 - \frac {\delta}{4}\right) \\ \geq \frac {1}{r (p _ {t})} + \frac {\left| r ^ {\prime} (\hat {r} (1 - \frac {\delta}{4})) \right|}{(1 - \frac {\delta}{4}) ^ {2}} 2 (1 - s _ {t}) \hat {r} \left(1 - \frac {\delta}{4}\right) \\ = \frac {1}{r (p _ {t})} + \frac {2 \hat {r} (1 - \frac {\delta}{4}) | r ^ {\prime} (\hat {r} (1 - \frac {\delta}{4})) |}{(1 - \frac {\delta}{4}) ^ {2}} (1 - s _ {t}), \\ \end{array}
$$
and hence, by Lemma B.2 ( $q_{t} \leq q_{t+1}$ ) and  $q_{t} > c_{0} \geq \frac{1}{2}$ , we get
$$
s _ {t + 1} \geq \frac {q _ {t}}{r (p _ {t + 1})} \geq s _ {t} + \frac {\hat {r} (1 - \frac {\delta}{4}) | r ^ {\prime} (\hat {r} (1 - \frac {\delta}{4})) |}{(1 - \frac {\delta}{4}) ^ {2}} (1 - s _ {t}).
$$
Therefore, we can obtain the following inequality:
$$
- \mathcal {O} (\eta^ {2}) \leq 1 - s _ {t + 1} \leq \left(1 - \frac {\hat {r} (1 - \frac {\delta}{4}) | r ^ {\prime} (\hat {r} (1 - \frac {\delta}{4})) |}{(1 - \frac {\delta}{4}) ^ {2}}\right) (1 - s _ {t}),
$$
where we used Lemma B.4 to obtain the first inequality.
Combining the three cases, we can finally conclude that if we choose
$$
d := \min  \left\{\frac {1}{2}, \frac {\hat {r} (1 - \frac {\delta}{2}) | r ^ {\prime} (\hat {r} (1 - \frac {\delta}{2})) |}{2 (1 - \frac {\delta}{2}) ^ {2}}, 2 \left(1 + \frac {4 r ^ {\prime} (4)}{r (4)}\right), \frac {\hat {r} (1 - \frac {\delta}{4}) | r ^ {\prime} (\hat {r} (1 - \frac {\delta}{4})) |}{(1 - \frac {\delta}{4}) ^ {2}} \right\} \in \left(0, \frac {1}{2} \right],
$$
then  $|s_{t + 1} - 1| \leq (1 - d)|s_t - 1| + \mathcal{O}(\eta^2)$ .
Lemma B.5 implies that if  $s_t < 2(1 + \eta^2)^{-1}$  and  $|p_t| \geq \hat{r}(1 - \frac{\delta}{4})$ , then  $|s_t - 1|$  exponentially decreases. We prove Lemma B.6 to handle the regime  $|p_t| < \hat{r}(1 - \frac{\delta}{4})$ , which is stated below.
Lemma B.6. For any  $0 \leq t \leq T$ , if  $r(p_t) \geq 1 - \frac{\delta}{4}$ , it holds that
$$
\left| \frac {p _ {t + 1}}{p _ {t}} \right| \geq \frac {4}{4 - \delta}.
$$
Proof. If  $r(p_t) \geq 1 - \frac{\delta}{4}$ , then  $s_t = \frac{q_t}{r(p_t)} < \frac{1 - \frac{\delta}{2}}{1 - \frac{\delta}{4}} = \frac{4 - 2\delta}{4 - \delta}$ , where we used  $q_t < 1 - \frac{\delta}{2}$  for any  $0 \leq t \leq T$ . Consequently,
$$
\left| \frac {p _ {t + 1}}{p _ {t}} \right| = \frac {2}{s _ {t}} - 1 - \eta^ {2} p _ {t} ^ {2} r (p _ {t} ^ {2}) \geq \frac {2 (4 - \delta)}{4 - 2 \delta} - 1 - \eta^ {2} = \frac {2}{2 - \delta} - \eta^ {2} \geq \frac {4}{4 - \delta},
$$
for small step size  $\eta$ .
Now we prove Proposition B.7, which proves that  $s_t$  reaches close to 1 with error bound of  $\mathcal{O}(\eta^2)$ . Proposition B.7. There exists a time step  $t_a^* = \mathcal{O}_{\delta, \ell}(\log(\eta^{-1}))$  satisfying
$$
s _ {t _ {a} ^ {*}} = 1 + \mathcal {O} _ {\delta , \ell} (\eta^ {2}). \tag {9}
$$
Proof. By Lemma B.3, there exists a time step  $t_0 = \mathcal{O}_{\delta,\ell}(1)$  such that  $s_{t_0} \leq \frac{2}{2-r(1)}$ . Here, we divide into two possible cases: (1)  $s_{t_0} < 1$ , and (2)  $1 \leq s_{t_0} \leq \frac{2}{2-r(1)}$ .
Case 1. Suppose that  $s_{t_0} < 1$ . By Lemma B.6, if  $r(p_{t_0}) \geq 1 - \frac{\delta}{4}$  (or equivalently,  $|p_{t_0}| \leq \hat{r}(1 - \frac{\delta}{4})$ ), then there exists a time step  $t_1 \leq t_0 + \log \left( \frac{\hat{r}(1 - \frac{\delta}{4})}{|p_{t_0}|} \right) / \log \left( \frac{4}{4 - \delta} \right) = \mathcal{O}_{\delta, \ell}(1)$  such that  $|p_{t_1}| \geq \hat{r}(1 - \frac{\delta}{4})$ . We denote the first time step satisfying  $|p_{t_1}| \geq \hat{r}(1 - \frac{\delta}{4})$  and  $t_1 \geq t_0$  by  $t_1 = \mathcal{O}_{\delta, \ell}(1)$ . By Lemma B.4, it holds that  $s_{t_1} \leq 1 + \mathcal{O}(\eta^2)$  since  $s_{t_1 - 1} < 1$ . Consequently, if  $s_{t_1} \geq 1 - \frac{\eta^2}{2}$ , then  $|s_{t_1} - 1| \leq \mathcal{O}(\eta^2)$  so that  $t_a^* = t_1$  is the desired time step. Hence, it suffices to consider the case when  $s_{t_1} < 1 - \frac{\eta^2}{2}$ . Here, we can apply Lemma B.5 which implies that
$$
\left| s _ {t _ {1} + 1} - 1 \right| \leq (1 - d) \left| s _ {t _ {1}} - 1 \right| + \mathcal {O} \left(\eta^ {2}\right),
$$
where  $d$  is a constant which depends on  $\delta$  and  $\ell$ . Then, there are two possible cases: either  $|s_{t_1} - 1| \leq \mathcal{O}(\eta^2 d^{-1})$ , or  $|s_{t_1 + 1} - 1| \leq (1 - \frac{d}{2}) |s_{t_1} - 1|$ . It suffices to consider the latter case, suppose that  $|s_{t_1 + 1} - 1| \leq (1 - \frac{d}{2}) |s_{t_1} - 1|$ . Since we are considering the case  $s_{t_1} < 1 - \frac{\eta^2}{2}$ , again by Lemma B.4, we have  $s_{t_1 + 1} \leq 1 + \mathcal{O}(\eta^2)$ . Since  $|\frac{p_{t_1 + 1}}{p_{t_1}}| = \frac{2}{s_{t_1}} - 1 - \mathcal{O}(\eta^2)$ ,  $|p_{t_1 + 1}| \geq |p_{t_1}| \geq \hat{r}(1 - \frac{\delta}{4})$  must be satisfied unless  $s_{t_1} = 1 + \mathcal{O}(\eta^2)$  already holds. If  $s_{t_1 + 1} \geq 1 - \frac{\eta^2}{2}$ , then  $|s_{t_1 + 1} - 1| \leq \mathcal{O}(\eta^2)$  so that  $t_a^* = t_1 + 1$  is the desired time step; if not, we can again apply Lemma B.5 and repeat the analogous argument. Hence, there exists a time step  $t_2 \leq t_1 + \log \left(\frac{\eta^2}{1 - s_{t_1}}\right) / \log(1 - \frac{d}{2}) = \mathcal{O}_{\delta, \ell}(\log(\eta^{-1}))$ , such that  $|s_{t_2} - 1| \leq \mathcal{O}(\eta^2 d^{-1}) = \mathcal{O}_{\delta, \ell}(\eta^2)$ .
Case 2. Suppose that  $1 \leq s_{t_0} \leq \frac{2}{2 - r(1)}$ . Then,  $r(p_{t_0}) \leq q_{t_0} \leq 1 - \frac{\delta}{2}$ , so we can apply Lemma B.5. There are two possible cases: either  $|s_{t_0 + 1} - 1| \leq \mathcal{O}(\eta^2 d^{-1}) = \mathcal{O}_{\delta,\ell}(\eta^2)$ , or  $|s_{t_0 + 1} - 1| \leq (1 - \frac{d}{2}) |s_{t_0} - 1|$ . It suffices to consider the latter case. If  $s_{t_0 + 1} \geq 1$ , we can again apply Lemma B.5 and repeat the analogous argument. Hence, we can obtain a time step  $t_0' \leq t_0 + \log \left(\frac{\eta^2}{1 - s_{t_0}}\right) / \log \left(1 - \frac{d}{2}\right) = \mathcal{O}_{\delta,\ell}(\log (\eta^{-1}))$  such that either  $s_{t_0'} < 1$  or  $|s_{t_0'} - 1| = \mathcal{O}_{\delta,\ell}(\eta^2)$  is satisfied. If  $s_{t_0'} < 1$ , we proved in Case 1 that there exists a time step  $t_2' = t_0' + \mathcal{O}_{\delta,\ell}(\log (\eta^{-1}))$  such that  $|s_{t_2'} - 1| \leq \mathcal{O}_{\delta,\ell}(\eta^2)$ , and this is the desired bound.
Now we carefully handle the error term  $\mathcal{O}(\eta^2)$  obtained in Proposition B.7 and a provide tighter bound on  $s_t$  by proving Lemma B.8 stated below.
Lemma B.8. If  $|s_t - 1| = \mathcal{O}_{\delta, \ell}(\eta^2)$ , then it holds that
$$
\left| s _ {t + 1} - 1 - h \left(p _ {t + 1}\right) \eta^ {2} \right| \leq \left(1 + \frac {2 p _ {t} r ^ {\prime} \left(p _ {t}\right)}{r \left(p _ {t}\right)}\right) \left| s _ {t} - 1 - h \left(p _ {t}\right) \eta^ {2} \right| + \mathcal {O} _ {\delta , \ell} \left(\eta^ {4} p _ {t} ^ {2}\right),
$$
Proof. Suppose that  $s_t = 1 + \mathcal{O}_{\delta, \ell}(\eta^2)$ . Then,  $|p_{t+1}| = \left|1 - \frac{2}{s_t} + \eta^2 p_t^2 r(p_t)^2\right| \cdot |p_t| = (1 + \mathcal{O}_{\delta, \ell}(\eta^2)) |p_t|$ . By Eq. (8) proved in Lemma B.5, there exists  $\epsilon_t = \mathcal{O}_{\delta, \ell}(\eta^2)$  which satisfies the following:
$$
\begin{array}{l} \frac {1}{r \left(p _ {t + 1}\right)} = \frac {1}{r \left(p _ {t}\right)} + \frac {r ^ {\prime} \left(\left(1 + \epsilon_ {t}\right) p _ {t}\right)}{r \left(\left(1 + \epsilon_ {t}\right) p _ {t}\right) ^ {2}} \left(\frac {2 \left(s _ {t} - 1\right)}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r \left(p _ {t}\right) ^ {2}\right) p _ {t} \\ = \frac {1}{r (p _ {t})} + \left(\frac {r ^ {\prime} (p _ {t})}{r (p _ {t}) ^ {2}} + \mathcal {O} _ {\delta , \ell} (\eta^ {2} p _ {t})\right) \left(\frac {2 (s _ {t} - 1)}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r (p _ {t}) ^ {2}\right) p _ {t} \\ = \frac {1}{r (p _ {t})} + \frac {r ^ {\prime} (p _ {t})}{r (p _ {t}) ^ {2}} \left(\frac {2 (s _ {t} - 1)}{s _ {t}} + \eta^ {2} p _ {t} ^ {2} r (p _ {t}) ^ {2}\right) p _ {t} + \mathcal {O} _ {\delta , \ell} (\eta^ {4} p _ {t} ^ {2}), \\ \end{array}
$$
where we used the Taylor expansion on  $\frac{r'(p)}{r(p)^2}$  with the fact that  $\frac{d}{dp}\left(\frac{r'(p)}{r(p)^2}\right)$  is bounded on  $[-4, 4]$  and that  $|p_t| \leq 4$  to obtain the second equality. Note that  $q_{t+1} = (1 - \eta^2 p_t^2 r(p_t)(2q_t - r(p_t)))^{-1} q_t$  by Eq. (6). Consequently,