SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Then we have  $f(x) = w(x) + \sum_{i=1}^{m} \sum_{r=1}^{j_i} p_{ir}(x) + \sum_{i=1}^{n} \sum_{r=1}^{k_i} q_{ir}(x)$ . We know that  $\frac{r+2}{r+1} \leq 1.5$  for any  $r \geq 1$ . Then we can show that
$$
\lim  _ {x \rightarrow a _ {i}} \frac {\left| f ^ {\prime} (x) \right| ^ {1 . 5}}{\left| f ^ {\prime \prime} (x) \right|} = \lim  _ {x \rightarrow a _ {i}} \frac {\left| p _ {i j _ {i}} ^ {\prime} (x) \right| ^ {1 . 5}}{\left| p _ {i j _ {i}} ^ {\prime \prime} (x) \right|} \geq \frac {1}{j _ {i} + 1}, \tag {7}
$$
where the first equality is because one can easily verify that the first and second order derivatives of  $p_{ij_i}$  dominate those of all other terms when  $x$  goes to  $a_i$ , and the second equality is by standard calculations noting that  $\frac{j_i + 2}{j_i + 1} \leq 1.5$ . Note that (7) implies that, for any  $L_{\rho} > j_i + 1$ , there exists  $\delta_i > 0$  such that
$$
\left| f ^ {\prime \prime} (x) \right| \leq L _ {\rho} \left| f ^ {\prime} (x) \right| ^ {1. 5}, \quad \text {i f} | x - a _ {i} | <   \delta_ {i}. \tag {8}
$$
Similarly, one can show  $\lim_{x\to \infty}\frac{\left|f'(x)\right|^{1.5}}{\left|f''(x)\right|} = \infty$ , which implies there exists  $x_0 > 0$  such that
$$
\left| f ^ {\prime \prime} (x) \right| \leq L _ {\rho} \left| f ^ {\prime} (x) \right| ^ {1. 5}, \quad \text {i f} | x | > x _ {0}. \tag {9}
$$
$$
\mathcal {B} := \left\{x \in \mathbb {R} \mid | x | \leq x _ {0} \text {a n d} | x - a _ {i} | \geq \delta_ {i}, \forall i \right\}.
$$
$$
\left| f ^ {\prime \prime} (x) \right| \leq L _ {0}, \quad \text {i f} x \in \mathcal {B}. \tag {10}
$$
Combining (8), (9), and (10), we have shown
$$
\left| f ^ {\prime \prime} (x) \right| \leq L _ {0} + L _ {\rho} \left| f ^ {\prime} (x) \right| ^ {1. 5}, \quad \forall x \in \operatorname {d o m} (f),
$$
For the second part, consider the ration function  $f(x) = 1 / x$ . Then we know that  $f'(x) = -1 / x^2$  and  $f''(x) = 2 / x^3$ . Note that for any  $\rho < 1.5$  and  $0 < x \leq \min \{(L_0 + 1)^{-1/3}, (L_\rho + 1)^{-1/(3-2\rho)}\}$ , we have
$$
| f ^ {\prime \prime} (x) | = \frac {1}{x ^ {3}} + \frac {1}{x ^ {3 - 2 \rho}} \cdot | f ^ {\prime} (x) | ^ {\rho} > L _ {0} + L _ {\rho} | f ^ {\prime} (x) | ^ {\rho},
$$
which shows  $f$  is not  $(\rho, L_0, L_\rho)$  smooth for any  $\rho < 1.5$  and  $L_0, L_\rho \geq 0$ .
Finally, we complete this section with the proof of Proposition 3.7, which shows self-concordant functions are  $(2,L_0,L_2)$ -smooth for some  $L_0,L_\rho \geq 0$ .
Proof of Proposition 3.7. Let  $h: \mathbb{R} \to \mathbb{R}$  be a self-concordant function. We have  $h'''(x) \leq 2h''(x)^{3/2}$ . Then, for  $x \in (a, b)$ , we can obtain
$$
\frac {1}{2} h ^ {\prime \prime} (x) ^ {- 1 / 2} h ^ {\prime \prime \prime} (x) \leq h ^ {\prime \prime} (x).
$$
Integrating both sides from  $x_0$  to  $y$  for  $x_0, y \in (a,b)$ , we have
$$
h ^ {\prime \prime} (y) ^ {1 / 2} - h ^ {\prime \prime} (x _ {0}) ^ {1 / 2} \leq h ^ {\prime} (y) - h ^ {\prime} (x _ {0}).
$$
$$
h ^ {\prime \prime} (y) \leq \left(h ^ {\prime \prime} \left(x _ {0}\right) ^ {1 / 2} - h ^ {\prime} \left(x _ {0}\right) + h ^ {\prime} (y)\right) ^ {2} \leq 2 \left(h ^ {\prime \prime} \left(x _ {0}\right) ^ {1 / 2} - h ^ {\prime} \left(x _ {0}\right)\right) ^ {2} + 2 h ^ {\prime} (y) ^ {2}.
$$
Since  $h''(y) > 0$ , we have  $|h''(y)| = h''(y)$ . Therefore, the above inequality shows that  $h$  is  $(2, L_0, L_2)$ -smooth with  $L_0 = 2(h''(x_0)^{1/2} - h'(x_0))^2$  and  $L_2 = 2$ .
A.2 Proof of Proposition 3.2
In order to prove Proposition 3.2, we need the following several lemmas. First, the lemma below partially generalizes Gronwall's inequality.
Lemma A.3. Let  $\alpha : [a, b] \to [0, \infty)$  and  $\beta : [0, \infty) \to (0, \infty)$  be two continuous functions. Suppose  $\alpha'(t) \leq \beta(\alpha(t))$  almost everywhere over  $(a, b)$ . Denote function  $\phi(u) := \int \frac{1}{\beta(u)} du$ . We have for all  $t \in [a, b]$ ,
$$
\phi (\alpha (t)) \leq \phi (\alpha (a)) - a + t.
$$
Proof of Lemma A.3. First, by definition, we know that  $\phi$  is increasing since  $\phi' = \frac{1}{\beta} > 0$ . Let function  $\gamma : [a, b] \to \mathbb{R}$  be the solution of the following differential equation
$$
\gamma^ {\prime} (t) = \beta (\gamma (t)) \forall t \in (a, b), \quad \gamma (a) = \alpha (a). \tag {11}
$$
$$
d \phi (\gamma (t)) = \frac {d \gamma (t)}{\beta (\gamma (t))} = d t.
$$
Integrating both sides, noting that  $\gamma (a) = \alpha (a)$  by (11), we obtain
$$
\phi (\gamma (t)) - \phi (\alpha (a)) = t - a.
$$
Then it suffices to show  $\phi(\alpha(t)) \leq \phi(\gamma(t))$ ,  $\forall t \in [a, b]$ . Note that the following inequality holds almost everywhere.
$$
(\phi (\alpha (t)) - \phi (\gamma (t))) ^ {\prime} = \phi^ {\prime} (\alpha (t)) \alpha^ {\prime} (t) - \phi^ {\prime} (\gamma (t)) \gamma^ {\prime} (t) = \frac {\alpha^ {\prime} (t)}{\beta (\alpha (t))} - \frac {\gamma^ {\prime} (t)}{\beta (\gamma (t))} \leq 0,
$$
where the inequality is because  $\alpha'(t) \leq \beta(\alpha(t))$  by the assumption of this lemma and  $\gamma'(t) = \beta(\gamma(t))$  by (11). Since  $\phi(\alpha(a)) - \phi(\gamma(a)) = 0$ , we know for all  $t \in [a, b]$ ,  $\phi(\alpha(t)) \leq \phi(\gamma(t))$ , which completes the proof.
With Lemma A.3, one can bound the gradient norm within a small enough neighborhood of a given point as in the following lemma.
Lemma A.4. If the objective function  $f$  is  $\ell$ -smooth, for any two points  $x, y \in \mathbb{R}^d$  such that the closed line segment between  $x$  and  $y$  is contained in  $\mathcal{X}$ , if  $\| y - x \| \leq \frac{a}{\ell(\|\nabla f(x)\| + a)}$  for any  $a > 0$ , we have
Proof of Lemma A.4. Denote  $z(t) \coloneqq (1 - t)x + ty$  for  $0 \leq t \leq 1$ . Then we know  $z(t) \in \mathcal{X}$  for all  $0 \leq t \leq 1$  by the assumption made in this lemma. Then we can also define  $\alpha(t) \coloneqq \| \nabla f(z(t)) \|$  for  $0 \leq t \leq 1$ . Note that for any  $0 \leq t \leq s \leq 1$ , by triangle inequality,
$$
\alpha (s) - \alpha (t) \leq \| \nabla f (z (s)) - \nabla f (z (t)) \|. \tag {12}
$$
We know that  $\alpha(t) = \|\nabla f(z(t))\|$  is differentiable almost everywhere since  $f$  is second order differentiable almost everywhere (Here we assume  $\alpha(t) \neq 0$  for  $0 < t < 1$  without loss of generality. Otherwise, one can define  $t_m = \sup \{0 < t < 1 \mid \alpha(t) = 0\}$  and consider the interval  $[t_m, 1]$  instead). Then the following equality holds almost everywhere
$$
\begin{array}{l} \alpha^ {\prime} (t) = \lim _ {s \downarrow t} \frac {\alpha (s) - \alpha (t)}{s - t} \leq \lim _ {s \downarrow t} \frac {\| \nabla f (z (s)) - \nabla f (z (t)) \|}{s - t} = \left\| \lim _ {s \downarrow t} \frac {\nabla f (z (s)) - \nabla f (z (t))}{s - t} \right\| \\ = \left\| \nabla^ {2} f (z (t)) (y - x) \right\| \leq \left\| \nabla^ {2} f (z (t)) \right\| \| y - x \| \leq \ell (\alpha (t)) \| y - x \|, \\ \end{array}
$$
where the first inequality is due to (12) and the last inequality is by Definition 1. Let  $\beta(u) \coloneqq \ell(u) \cdot \|y - x\|$  and  $\phi(u) \coloneqq \int_0^u \frac{1}{\beta(v)} dv$ . By Lemma A.3, we know that
Denote  $\psi (u)\coloneqq \int_0^u\frac{1}{\ell(v)} dv = \phi (u)\cdot \| y - x\|$ . We have
$$
\begin{array}{l} \psi \left(\| \nabla f (y) \|\right) \leq \psi \left(\| \nabla f (x) \|\right) + \| y - x \| \\ \leq \psi (\| \nabla f (x) \|) + \frac {a}{\ell (\| \nabla f (x) \| + a)} \\ \leq \int_ {0} ^ {\| \nabla f (x) \|} \frac {1}{\ell (v)} d v + \int_ {\| \nabla f (x) \|} ^ {\| \nabla f (x) \| + a} \frac {1}{\ell (v)} d v \\ = \psi (\| \nabla f (x) \| + a). \\ \end{array}
$$
With Lemma A.4, we are ready to prove Proposition 3.2.
Proof of Proposition 3.2. We prove the two directions in this proposition separately.
1. An  $(r,\ell)$ -smooth function is  $\ell$ -smooth.
For each fixed  $x \in \mathcal{X}$  where  $\nabla^2 f(x)$  exists and any unit-norm vector  $w$ , by Definition 2, we know that for any  $t \leq r(\|\nabla f(x)\|)$ ,
$$
\| \nabla f (x + t w) - \nabla f (x) \| \leq t \cdot \ell (\| \nabla f (x) \)).
$$
Then we know that
$$
\begin{array}{l} \left\| \nabla^ {2} f (x) w \right\| = \left\| \lim  _ {t \downarrow 0} \frac {1}{t} (\nabla f (x + t w) - \nabla f (x)) \right\| \\ = \lim  _ {t \downarrow 0} \frac {1}{t} \| (\nabla f (x + t w) - \nabla f (x)) \| \leq \ell (\| \nabla f (x) \|), \\ \end{array}
$$
which implies  $\| \nabla^2 f(x)\| \leq \ell (\| \nabla f(x)\|)$  for any point  $x$  if  $\nabla^2 f(x)$  exists.
Then it suffices to show that  $\nabla^2 f(x)$  exists almost everywhere. Note that for each  $x\in \mathcal{X}$ , Definition 2 states that the gradient function is  $\ell (\| \nabla f(x)\|)$  Lipschitz within the ball  $\mathcal{B}(x,r(\| \nabla f(x)\|))$ . Then by Rademacher's Theorem,  $f$  is twice differentiable almost everywhere within this ball. Then we can show it is also twice differentiable almost everywhere within the entire domain  $\mathcal{X}$  as long as we can cover  $\mathcal{X}$  with countably many such balls. Define  $\mathcal{S}_n\coloneqq \{x\in \mathcal{X}\mid n\leq \| \nabla f(x)\| \leq n + 1\}$  for integer  $n\geq 0$ . We have  $\mathcal{X} = \cup_{n\geq 0}\mathcal{S}_n$ . One can easily find an internal covering of  $\mathcal{S}_n$  with balls of size  $r(n + 1)^3$ , i.e., there exist  $\{x_{n,i}\}_{i\geq 0}$ , where  $x_{n,i}\in S_n$ , such that  $\mathcal{S}_n\subseteq \cup_{i\geq 0}\mathcal{B}(x_{n,i},r(n + 1))\subseteq \cup_{i\geq 0}\mathcal{B}(x_{n,i},r(\| \nabla f(x_{n,i})\|))$ . Therefore we have  $\mathcal{X}\subseteq \cup_{n,i\geq 0}\mathcal{B}(x_{n,i},r(\| \nabla f(x_{n,i})\|))$  which completes the proof.
2. An  $\ell$ -smooth function satisfying Assumption 1 is  $(r, m)$ -smooth where  $m(u) := \ell(u + a)$  and  $r(u) := a / m(u)$  for any  $a > 0$ .
For any  $y \in \mathbb{R}^d$  satisfying  $\| y - x\| \leq r(\|\nabla f(x)\|) = \frac{a}{\ell(\|\nabla f(x)\| + a)}$ , denote  $z(t) \coloneqq (1 - t)x + ty$  for  $0 \leq t \leq 1$ . We first show  $y \in \mathcal{X}$  by contradiction. Suppose  $y \notin \mathcal{X}$ , let us define  $t_{\mathrm{b}} \coloneqq \inf \{0 \leq t \leq 1 \mid z(t) \notin \mathcal{X}\}$  and  $z_{\mathrm{b}} \coloneqq z(t_{\mathrm{b}})$ . Then we know  $z_{\mathrm{b}}$  is a boundary point of  $\mathcal{X}$ . Since  $f$  is a closed function with an open domain, we have
$$
\lim  _ {t \uparrow t _ {\mathrm {b}}} f (z (t)) = \infty . \tag {13}
$$
On the other hand, by the definition of  $t_{\mathrm{b}}$ , we know  $z(t) \in \mathcal{X}$  for every  $0 \leq t < t_{\mathrm{b}}$ . Then by Lemma A.4, for all  $0 \leq t < t_{\mathrm{b}}$ , we have  $\| \nabla f(z(t)) \| \leq \| \nabla f(x) \| + a$ . Therefore for all  $0 \leq t < t_{\mathrm{b}}$ ,
<sup>3</sup>We can find an internal covering in the following way. We first cover  $S_{n}$  with countably many hyper-cubes of length  $r(n + 1) / \sqrt{d}$ , which is obviously doable. Then for each hyper-cube that intersects with  $S_{n}$ , we pick one point from the intersection. Then the ball centered at the picked point with radius  $r(n + 1)$  covers this hyper-cube. Therefore, the union of all such balls can cover  $S_{n}$ .
which contradicts (13). Therefore we have shown  $y \in \mathcal{X}$ . Since  $y$  is chosen arbitrarily with the ball  $\mathcal{B}(x, r(\| \nabla f(x) \|))$ , we have  $\mathcal{B}(x, r(\| \nabla f(x) \|)) \subseteq \mathcal{X}$ . Then for any  $x_1, x_2 \in \mathcal{B}(x, r(\| \nabla f(x) \|))$ , we denote  $w(t) := tx_1 + (1 - t)x_2$ . Then we know  $w(t) \in \mathcal{B}(x, r(\| \nabla f(x) \|))$  for all  $0 \leq t \leq 1$  and can obtain
$$
\begin{array}{l} \left\| \nabla f (x _ {1}) - \nabla f (x _ {2}) \right\| = \left\| \int_ {0} ^ {1} \nabla^ {2} f (w (t)) \cdot (x _ {1} - x _ {2}) d t \right\| \\ \leq \| x _ {1} - x _ {2} \| \cdot \int_ {0} ^ {1} \ell (\| \nabla f (x) \| + a) d t \\ = m \left(\left\| \nabla f (x) \right\|\right) \cdot \left\| x _ {1} - x _ {2} \right\|, \\ \end{array}
$$
where the last inequality is due to Lemma A.4.
A.3 Proofs of lemmas implied by generalized smoothness
In this part, we provide the proofs of the useful properties stated in Section 3.1.2, including Lemma 3.3, Lemma 3.5, and Corollary 3.6.
Proof of Lemma 3.3. First, note that since  $\ell$  is non-decreasing and  $r$  is non-increasing, we have  $\ell (\| \nabla f(x)\|)\leq \ell (G) = L$  and  $r(G)\leq r(\| \nabla f(x)\|)$ . Then by Definition 2, we directly have that  $\mathcal{B}(x,r(G))\subseteq \mathcal{B}(x,r(\| \nabla f(x)\|))\subseteq \mathcal{X}$ , and that for any  $x_{1},x_{2}\in \mathcal{B}(x,r(G))$ , we have
$$
\left\| \nabla f \left(x _ {1}\right) - \nabla f \left(x _ {2}\right) \right\| \leq \ell \left(\left\| \nabla f (x) \right\|\right) \| x _ {1} - x _ {2} \| \leq L \| x _ {1} - x _ {2} \|.
$$
Next, for the second inequality in (2), define  $z(t) \coloneqq (1 - t)x_2 + tx_1$  for  $0 \leq t \leq 1$ . We know  $z(t) \in \mathcal{B}(x,r(G))$ . Note that we have shown
$$
\left\| \nabla f (z (t)) - \nabla f (x _ {2}) \right\| \leq L \| z (t) - x _ {2} \| = t L \| x _ {1} - x _ {2} \|. \tag {14}
$$
$$
\begin{array}{l} f (x _ {1}) - f (x _ {2}) = \int_ {0} ^ {1} \left\langle \nabla f (z (t), x _ {1} - x _ {2} \right\rangle d t \\ = \int_ {0} ^ {1} \left\langle \nabla f (x _ {2}), x _ {1} - x _ {2} \right\rangle + \left\langle \nabla f (z (t)) - \nabla f (x _ {2}), x _ {1} - x _ {2} \right\rangle d t \\ \leq \left\langle \nabla f (x _ {2}), x _ {1} - x _ {2} \right\rangle + L \| x _ {1} - x _ {2} \| ^ {2} \int_ {0} ^ {1} t d t \\ = \left\langle \nabla f (x _ {2}), x _ {1} - x _ {2} \right\rangle + \frac {L}{2} \| x _ {1} - x _ {2} \| ^ {2}, \\ \end{array}
$$
where the inequality is due to (14).
Proof of Lemma 3.5. If  $f$  is  $\ell$ -smooth, by Proposition 3.2,  $f$  is also  $(r,m)$ -smooth where  $m(u) = \ell(2u)$  and  $r(u) = u / \ell(2u)$ . Then by Lemma 3.3 where we choose  $G = \| \nabla f(x) \|$ , we have that  $\mathcal{B}\left(x, \frac{\|\nabla f(x)\|}{\ell(2\|\nabla f(x)\|)}\right) \subseteq \mathcal{X}$ , and that for any  $x_1, x_2 \in \mathcal{B}\left(x, \frac{\|\nabla f(x)\|}{\ell(2\|\nabla f(x)\|)}\right)$ , we have
$$
f (x _ {1}) \leq f (x _ {2}) + \left\langle \nabla f (x _ {2}), x _ {1} - x _ {2} \right\rangle + \frac {\ell (2 \| \nabla f (x) \|)}{2} \| x _ {1} - x _ {2} \|.
$$
Choosing  $x_{2} = x$  and  $x_{1} = x - \frac{\nabla f(x)}{\ell(2\|\nabla f(x)\|)}$ , it is easy to verify that  $x_{1}, x_{2} \in \mathcal{B}\left(x, \frac{\|\nabla f(x)\|}{\ell(2\|\nabla f(x)\|)}\right)$ . Therefore, we have
$$
f ^ {*} \leq f \left(x - \frac {\nabla f (x)}{\ell (2 \| \nabla f (x) \|)}\right) \leq f (x) - \frac {\| \nabla f (x) \| ^ {2}}{2 \ell (2 \| \nabla f (x) \|)},
$$
Proof of Corollary 3.6. We first show  $G < \infty$ . Note that since  $\ell$  is sub-quadratic, we know  $\lim_{u\to \infty}2\ell (2u) / u^2 = 0$ . Therefore, for any  $F > 0$ , there exists some  $M > 0$  such that  $2\ell (2u) / u^2 < 1 / F$  for every  $u > M$ . In other words, for any  $u$  satisfying  $u^2\leq 2\ell (2u)\cdot F$  we must have  $u\leq M$ . Therefore, by definition of  $G$ , we have  $G\leq M < \infty$  if  $F > 0$ . If  $F = 0$ , we trivially get  $G = 0 < \infty$ . Also, since the set  $\{u\geq 0\mid u^2\leq 2\ell (2u)\cdot F\}$  is closed and bounded, we know its supremum  $G$  is in this set and it is also straightforward to show  $G^{2} = 2\ell (2G)\cdot F$ .
Next, by Lemma 3.5, we know
$$
\left\| \nabla f (x) \right\| ^ {2} \leq 2 \ell \left(2 \left\| \nabla f (x) \right\|\right) \cdot \left(f (x) - f ^ {*}\right) \leq 2 \ell \left(2 \left\| \nabla f (x) \right\|\right) \cdot F.
$$
Then based on the definition of  $G$ , we have  $\| \nabla f(x) \| \leq G$ .
B Analysis of GD for convex functions
In this section, we provide the detailed convergence analysis of gradient descent in the convex setting, including the proofs of Lemma 4.1 and Theorem 4.2, for which the following lemma will be helpful.
Lemma B.1 (Co-coercivity). If  $f$  is convex and  $(r, \ell)$ -smooth, for any  $x \in \mathcal{X}$  and  $y \in \mathcal{B}(x, r(\| \nabla f(x) \|)/2)$ , we have  $y \in \mathcal{X}$  and
$$
\langle \nabla f (x) - \nabla f (y), x - y \rangle \geq \frac {1}{L} \| \nabla f (x) - \nabla f (y) \| ^ {2},
$$
where  $L = \ell (\| \nabla f(x)\|$
Proof of Lemma B.1. Define the Bregman divergences  $\phi_x(w) \coloneqq f(w) - \langle \nabla f(x), w \rangle$  and  $\phi_y(w) \coloneqq f(w) - \langle \nabla f(y), w \rangle$ , which are both convex functions. Since  $\nabla \phi_x(w) = \nabla f(w) - \nabla f(x)$ , we have  $\nabla \phi_x(x) = 0$  which implies  $\min_w \phi_x(w) = \phi_x(x)$  as  $\phi_x$  is convex. Similarly we have  $\min_w \phi_y(w) = \phi_y(y)$ .
Denote  $r_x \coloneqq r(\| \nabla f(x) \|)$ . Since  $f$  is  $(r, \ell)$ -smooth, we know its gradient  $\nabla f$  is  $L$ -Lipschitz locally in  $\mathcal{B}(x, r_x)$ . Since  $\nabla \phi_x(w) - \nabla f(w) = \nabla f(x)$  is a constant, we know  $\nabla \phi_x$  is also  $L$ -Lipschitz locally in  $\mathcal{B}(x, r_x)$ . Then similar to the proof of Lemma 3.3, one can easily show that for any  $x_1, x_2 \in \mathcal{B}(x, r_x)$ , we have
$$
\phi_ {x} \left(x _ {1}\right) \leq \phi_ {x} \left(x _ {2}\right) + \left\langle \nabla \phi_ {x} \left(x _ {2}\right), x _ {1} - x _ {2} \right\rangle + \frac {L}{2} \| x _ {1} - x _ {2} \| ^ {2}. \tag {15}
$$
Note that for any  $y \in \mathcal{B}(x, r(\| \nabla f(x) \|)/2)$  as in the lemma statement,
$$
\left\| y - \frac {1}{L} \nabla \phi_ {x} (y) - x \right\| \leq \| y - x \| + \frac {1}{L} \| \nabla f (y) - \nabla f (x) \| \leq 2 \| y - x \| \leq r _ {x},
$$
where the first inequality uses triangle inequality and  $\nabla \phi_x(y) = \nabla f(y) - \nabla f(x)$ ; and the second inequality uses Definition 2. It implies that  $y - \frac{1}{L}\nabla \phi_x(y) \in \mathcal{B}(x, r_x)$ . Then we can obtain
$$
\phi_ {x} (x) = \min  _ {w} \phi_ {x} (w) \leq \phi_ {x} \left(y - \frac {1}{L} \nabla \phi_ {x} (y)\right) \leq \phi_ {x} (y) - \frac {1}{2 L} \| \nabla \phi_ {x} (y) \| ^ {2},
$$
where the last inequality uses (15) where we choose  $x_{1} = y - \frac{1}{L}\nabla \phi_{x}(y)$  and  $x_{2} = y$ . By the definition of  $\phi_{x}$ , the above inequality is equivalent to
$$
\frac {1}{2 L} \left\| \nabla f (y) - \nabla f (x) \right\| ^ {2} \leq f (y) - f (x) - \langle \nabla f (x), x - y \rangle .
$$
Similar argument can be made for  $\phi_y(\cdot)$  to obtain
$$
\frac {1}{2 L} \left\| \nabla f (y) - \nabla f (x) \right\| ^ {2} \leq f (x) - f (y) - \langle \nabla f (y), y - x \rangle .
$$
Summing up the two inequalities, we can obtain the desired result.
With Lemma B.1, we prove Lemma 4.1 as follows.
Proof of Lemma 4.1. Let  $L = \ell(G)$ . We first verify that  $x^{+} \in \mathcal{B}(x, r(G)/2)$ . Note that
$$
\left\| x ^ {+} - x \right\| = \| \eta \nabla f (x) \| \leq \eta G \leq r (G) / 2,
$$
where we choose  $\eta \leq r(G) / (2G)$ . Thus by Lemma B.1, we have
$$
\begin{array}{l} \left\| \nabla f (x ^ {+}) \right\| ^ {2} = \| \nabla f (x) \| ^ {2} + 2 \langle \nabla f (x ^ {+}) - \nabla f (x), \nabla f (x) \rangle + \left\| \nabla f (x ^ {+}) - \nabla f (x) \right\| ^ {2} \\ = \| \nabla f (x) \| ^ {2} - \frac {2}{\eta} \langle \nabla f (x ^ {+}) - \nabla f (x), x ^ {+} - x \rangle + \left\| \nabla f (x ^ {+}) - \nabla f (x) \right\| ^ {2} \\ \leq \| \nabla f (x) \| ^ {2} + \left(1 - \frac {2}{\eta L}\right) \left\| \nabla f (x ^ {+}) - \nabla f (x) \right\| ^ {2} \\ \leq \left\| \nabla f (x) \right\| ^ {2}, \\ \end{array}
$$
where the first inequality uses Lemma B.1 and the last inequality chooses  $\eta \leq 2 / L$ .
With Lemma 4.1, we are ready to prove both Theorem 4.2 and Theorem 4.3.
Proof of Theorem 4.2. Denote  $G \coloneqq \| \nabla f(x_0) \|$ . Then we trivially have  $\| \nabla f(x_0) \| \leq G$ . Lemma 4.1 states that if  $\| \nabla f(x_t) \| \leq G$  for any  $t \geq 0$ , then we also have  $\| \nabla f(x_{t+1}) \| \leq \| \nabla f(x_t) \| \leq G$ . By induction, we can show that  $\| \nabla f(x_t) \| \leq G$  for all  $t \geq 0$ . Then the rest of the proof basically follows the standard textbook analysis. We still provide the detailed proof below for completeness.
Note that  $\| x_{t + 1} - x_t\| = \eta \| \nabla f(x_t)\| \leq \eta G\leq r(G)$ , where we choose  $\eta \leq r(G) / (2G)$ . Thus we can apply Lemma 3.3 to obtain
$$
\begin{array}{l} 0 \geq f (x _ {t + 1}) - f (x _ {t}) - \langle \nabla f (x _ {t}), x _ {t + 1} - x _ {t} \rangle - \frac {L}{2} \| x _ {t + 1} - x _ {t} \| ^ {2} \\ \geq f \left(x _ {t + 1}\right) - f \left(x _ {t}\right) - \left\langle \nabla f \left(x _ {t}\right), x _ {t + 1} - x _ {t} \right\rangle - \frac {1}{2 \eta} \left\| x _ {t + 1} - x _ {t} \right\| ^ {2}, \tag {16} \\ \end{array}
$$
where the last inequality chooses  $\eta \leq 1 / L$ . Meanwhile, by convexity between  $x_{t}$  and  $x^{*}$ , we have
$$
0 \geq f \left(x _ {t}\right) - f ^ {*} + \langle \nabla f \left(x _ {t}\right), x ^ {*} - x _ {t} \rangle . \tag {17}
$$
Note that  $(t + 1)\times (16) + (17)$  gives
$$
\begin{array}{l} 0 \geq f (x _ {t}) - f ^ {*} + \langle \nabla f (x _ {t}), x ^ {*} - x _ {t} \rangle \\ + (1 + t) \left(f \left(x _ {t + 1}\right) - f \left(x _ {t}\right) - \left\langle \nabla f \left(x _ {t}\right), x _ {t + 1} - x _ {t} \right\rangle - \frac {1}{2 \eta} \left\| x _ {t + 1} - x _ {t} \right\| ^ {2}\right). \\ \end{array}
$$
Then reorganizing the terms of the above inequality, noting that
$$
\begin{array}{l} \left\| x _ {t + 1} - x ^ {*} \right\| ^ {2} - \left\| x _ {t} - x ^ {*} \right\| ^ {2} = \left\| x _ {t + 1} - x _ {t} \right\| ^ {2} + 2 \langle x _ {t + 1} - x _ {t}, x _ {t} - x ^ {*} \rangle \\ = \left\| x _ {t + 1} - x _ {t} \right\| ^ {2} + 2 \eta \langle \nabla f (x _ {t}), x ^ {*} - x _ {t} \rangle , \\ \end{array}
$$
we can obtain
$$
(t + 1) \left(f \left(x _ {t + 1}\right) - f ^ {*}\right) + \frac {1}{2 \eta} \left\| x _ {t + 1} - x ^ {*} \right\| ^ {2} \leq t \left(f \left(x _ {t}\right) - f ^ {*}\right) + \frac {1}{2 \eta} \left\| x _ {t} - x ^ {*} \right\| ^ {2}.
$$
The above inequality implies  $t(f(x_{t}) - f^{*}) + \frac{1}{2\eta}\| x_{t} - x^{*}\|^{2}$  is a non-increasing potential function, which directly implies the desired result.
Proof of Theorem 4.3. Since strongly convex functions are also convex, by the same argument as in the proof of Theorem 4.2, we have  $\| \nabla f(x_{t})\| \leq G \coloneqq \| \nabla f(x_{0})\|$  for all  $t \geq 0$ . Moreover, (16) still holds. For  $\mu$ -strongly-convex function, we can obtain a tighter version of (17) as follows.
$$
0 \geq f (x _ {t}) - f ^ {*} + \langle \nabla f (x _ {t}), x ^ {*} - x _ {t} \rangle + \frac {\mu}{2} \| x ^ {*} - x _ {t} \| ^ {2}. \tag {18}
$$
Let  $A_0 = 0$  and  $A_{t + 1} = (1 + A_t) / (1 - \eta \mu)$  for all  $t \geq 0$ . Combining (16) and (18), we have
$$
\begin{array}{l} 0 \geq \left(A _ {t + 1} - A _ {t}\right) \left(f \left(x _ {t}\right) - f ^ {*} + \langle \nabla f (x _ {t}), x ^ {*} - x _ {t} \rangle\right) \\ + A _ {t + 1} \left(f (x _ {t + 1}) - f (x _ {t}) - \langle \nabla f (x _ {t}), x _ {t + 1} - x _ {t} \rangle - \frac {1}{2 \eta} \| x _ {t + 1} - x _ {t} \| ^ {2}\right). \\ \end{array}
$$
$$
A _ {t + 1} (f (x _ {t + 1}) - f ^ {*}) + \frac {1 + \eta \mu A _ {t + 1}}{2 \eta} \| x _ {t + 1} - x ^ {*} \| ^ {2} \leq A _ {t} (f (x _ {t}) - f ^ {*}) + \frac {1 + \eta \mu A _ {t}}{2 \eta} \| x _ {t} - x ^ {*} \| ^ {2}.
$$
The above inequality means  $A_{t}(f(x_{t}) - f^{*}) + \frac{1 + \eta\mu A_{t}}{2\eta}\| x_{t} - x^{*}\|^{2}$  is a non-increasing potential function. Thus by telescoping we have
C Analysis of NAG for convex functions
In this section, we provide the detailed analysis of Nesterov's accelerated gradient method in the convex setting. As we discussed in Section 4.2, the stepsize size choice in Theorem 4.4 is smaller than the classical one. Therefore, we provide a more fine-grained version of the theorem, which allows the stepsize to depend on the degree of  $\ell$ .
Theorem C.1. Suppose  $f$  is convex and  $\ell$ -smooth. For  $\alpha \in (0,2]$ , if  $\ell(u) = o(u^{\alpha})$ , i.e.,  $\lim_{u \to \infty} \ell(u) / u^{\alpha} = 0$ , then there must exist a constant  $G$  such that for  $L := \ell(2G)$ , we have