SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ta Duy Nguyen
taduy@bu.edu
Alina Ene
aene@bu.edu
Thien Hang Nguyen
nguyen.thien@northeastern.edu
Huy Le Nguyen
In this work, we study the convergence in high probability of clipped gradient methods when the noise distribution has heavy tails, i.e., with bounded  $p$ th moments, for some  $1 < p \leq 2$ . Prior works in this setting follow the same recipe of using concentration inequalities and an inductive argument with union bound to bound the iterates across all iterations. This method results in an increase in the failure probability by a factor of  $T$ , where  $T$  is the number of iterations. We instead propose a new analysis approach based on bounding the moment generating function of a well chosen supermartingale sequence. We improve the dependency on  $T$  in the convergence guarantee for a wide range of algorithms with clipped gradients, including stochastic (accelerated) mirror descent for convex objectives and stochastic gradient descent for nonconvex objectives. Our high probability bounds achieve the optimal convergence rates and match the best currently known in expectation bounds. Our approach naturally allows the algorithms to use time-varying step sizes and clipping parameters when the time horizon is unknown, which appears difficult or even impossible using existing techniques from prior works. Furthermore, we show that in the case of clipped stochastic mirror descent, several problem constants, including the initial distance to the optimum, are not required when setting step sizes and clipping parameters.
Stochastic optimization is a well-studied area with many applications ranging from machine learning, to operation research, numerical linear algebra and beyond. In contrast to deterministic algorithms, stochastic algorithms might fail, and a pertinent question is how often does failure happen and how to increase the success rate. These questions are especially important in critical applications where failure is not tolerable, or when a single run is costly in time and resources. Fortunately, the standard stochastic gradient descent (SGD) algorithm has been shown to converge with high probability under a light-tailed noise distribution such as sub-Gaussian distributions [23, 12, 27, 14, 11, 10, 18], which gives strong guarantee on the success of single runs. However, recent observations in popular deep learning applications, such as training attention models [33] and convolutional networks [30], reveal a more challenging optimization landscape: the gradient noises follow heavy-tailed distributions, where the variance may be infinite [29, 33, 9], whereas
the standard light-tailed setting assumes that all the moments are bounded. Heavy-tailed gradient noises can cause algorithms like SGD to fail, and this mismatch between theory and practice has been suggested to be one of the reasons for the strong preference of adaptive methods like Adam over SGD in modern settings [33].
In this work, we consider the setting of heavy-tailed noise proposed by Zhang et al., (2020) [33], where the (unbiased) gradient noise only has bounded  $p$ th moments, for some  $p \in (1, 2]$ . While standard SGD can fail to converge when the variance is unbounded, i.e. when  $p < 2$ , [33] show that SGD with appropriate clipping (or Clipped-SGD) converges in expectation under heavy-tailed noise, where the convergence rate depends on  $O\left(\frac{1}{\delta}\right)$  if  $\delta$  is the targeted maximum failure probability. It is more desirable, however, to obtain convergence results in high probability, where the convergence rate depends instead on  $O\left(\log \frac{1}{\delta}\right)$ , which gives better guarantees for single runs.
Recent follow-up works [2, 28, 19] show that variants of Clipped-SGD in fact converge with high probability. This is a pleasing result, extending the earlier work by [7] for  $p = 2$ . However, there are several shortcomings of these results when compared with the corresponding bounds in the light-tailed setting. First, the clipped algorithm uses a fixed step size and a fixed clipping parameter depending on the number of iterations, which precludes results with unknown time horizons. Secondly, the convergence guarantees are worse than the light-tailed bounds by a log  $T$  factor, even for fixed step sizes and clipping parameters. These issues beg a qualitative question:
Is heavy-tailed noise inherently harder than light-tailed noise?
In this work, we answer the above question for Clipped-SGD and the general clipped (accelerated) stochastic mirror descent (Clipped-SMD) algorithm. We give an improved analysis framework that not only gives tighter bounds matching the light-tailed noise setting, but also allows for step sizes and clipping parameters for unknown time horizons. Furthermore, we show that this framework is applicable to various settings, from finding minimizers of convex functions with arbitrarily large domains using (accelerated) mirror descent, to finding stationary points for non-convex functions using gradient descent.
1.1 Contributions and Techniques
Our work addresses several open questions posed by previous works including handling general domains and dealing with an unknown time horizon under heavy-tailed noise. Qualitatively, we close the logarithmic suboptimality gap and achieve the optimal rate in several settings. More specifically:
- We demonstrate a novel approach to analyze clipped gradient methods in high probability that is general and applies to various standard settings. In the convex setting, we analyze Clipped-SMD and clipped stochastic accelerated mirror descent. In the non-convex setting, we analyze Clipped-SGD. Using our new analysis, we show that clipped methods attain time-optimal convergence in high probability for both convex and nonconvex objectives under heavy-tailed gradient noise. In the convex setting, we obtain an  $O\left(T^{\frac{1 - p}{p}}\right)$  convergence rate for arbitrary (not necessarily compact) convex domains for Clipped-SMD and  $O\left(T^{\frac{1 - p}{p}}\sigma + T^{-2}\right)$  for accelerated Clipped-SMD, where  $\sigma$  is the noise parameter. These rates are time-optimal and match the lower bounds in [26, 31]. In the nonconvex setting, we obtain the optimal convergence rate of  $O\left(T^{\frac{2 - 2p}{3p - 2}}\right)$  for clipped-SGD. This bound is also time-optimal and matches the lower bound in [33]; it also complements the in-expectation convergence of clipped-SGD provided by [33].
- Previous works for heavy-tailed noises follow the recipe of using Freedman-type inequalities [4, 3] as a blackbox and bound the iterates inductively for all iterations. This process incurs an additional  $\log T$  dependency in the final convergence rate; in other words, the success probability goes from  $1 - \delta$  to  $1 - T\delta$ . The step sizes and clipping parameters of this approach depend on the time horizon  $T$  to enable the union bound and induction across all iterations in the analysis, excluding the important case when the time horizon is unknown. Our whitebox approach forgoes the aforementioned induction, not only circumventing the  $\log T$  loss but also allowing for an unknown time horizon. We further show that our analysis allows for a choice of step size and clipping parameters that do not depend on generally unknown parameters like the noise-parameter  $\sigma$ , the
failure probability  $\delta$ , and the initial distance to the optimum, all of which appear impossible using only the techniques from prior works.
- Our whitebox approach analyzes the moment generating function of a well chosen martingale difference sequence to obtain tight rates for stochastic gradient methods. This approach is closest to the work of [18], which only works in the light-tailed noise setting. In contrast to the light-tailed noise setting where all the moments are well controlled, the heavy-tailed setting often requires algorithms to incorporate gradient clipping for controlling the possibly infinite moments. However, this makes the gradient estimate biased and requires more careful attention to control the bias propagating through the algorithm. Naively applying the technique in [18] is not enough to handle heavy-tailed noise. Rather, as will be shown in our analysis, we introduce a novel history-dependent weights for the martingale sequence that is able to cope with the propagating bias term of clipped methods for heavy-tailed noise across various settings.
High probability convergence for light-tailed noises. Convergence in high probability of stochastic gradient algorithms has been established for sub-Gaussian noises in a number of prior works, including [23, 12, 27, 14, 11, 10] for convex problems with bounded domain (or bounded Bregman diameter) or with strong convexity. Other works [17, 20, 16] study convergence of variants of SGD for nonconvex objectives, where they consider sub-Gaussian and sub-Weibull noises. The most relevant to ours in this line of work is the one by [18], where a whitebox approach is employed to obtain tight rates for stochastic gradient methods in the light-tailed noise setting. However, their technique is not directly applicable in the heavy-tailed noise setting, where we need to introduce new ideas to handle the biases introduced by gradient clipping.
High probability convergence for noises with bounded variance and heavy tails. The design of new gradient algorithms and their analysis in the presence of heavy-tailed noises has drawn significant recent interest. Starting from the work [25] which propose Clipped-SGD to handle exploding gradients in recurrent neural networks, the recent works [30, 29, 33, 9] give new motivation for clipped methods in the context of convolutional networks and attention deep networks that attempts to explain the dominance of adaptive methods over SGD in practical modern scenarios.
While the convergence in expectation of vanilla SGD has been extensively studied [5, 23, 13, 18], only recently has the convergence of Clipped-SGD with heavy tailed noises been closely examined. There, [33] first show the convergence in expectation of Clipped-SGD for nonconvex functions and provide a matching lower bound. In the convex regime, several works with different clipping strategies for the case of  $p = 2$  have shown high probability convergence for smooth problems with bounded domain [22, 24], smooth unconstrained problems [7], and non-smooth problems [8]. A variant of Clipped-SGD that utilizes momentum [2] has also been shown to converge with high probability for bounded  $p$ th moments gradient noise. However, the analysis in [2] requires a strong assumption which implies that the true gradients are bounded, a restrictive assumption that excludes objectives like quadratic functions.
More recently, [28, 19, 34] give nearly-optimal convergence rates for several Clipped-SGD variants. These works follow the recipe of using Freedman-type inequalities [4, 3] as a blackbox and bound the iterates inductively for all iterations, which incur an additional  $\log T$  dependency in the final convergence rate. We show in our work that existing convergence rates can be tightened up and improved. Tight lower bounds for the optimal convergence rate have been shown by [26, 31] for convex objectives and by [33] for nonconvex settings. In both cases, our paper provides optimal convergence guarantees.
In a related but different line of work, [32] show that vanilla SGD can converge with heavy tailed noise for a special type of strongly convex functions, and [31] show that stochastic mirror descent converges in expectation for a special choice of mirror maps, although only for strongly convex objectives with bounded domains.
2 Preliminaries: Assumptions and Notations
We study the problem  $\min_{x\in \mathcal{X}}f(x)$  where  $f:\mathbb{R}^d\to \mathbb{R}$  and  $\mathcal{X}$  is the domain of the problem. In the convex setting, we assume that  $\mathcal{X}$  is a convex set but not necessarily compact. We let  $\| \cdot \|$  be an
Table 1: Previous and new results for high-probability convergence (with failure probability  $\delta$ ) of clipped SMD and SGD under heavy tailed noise:  $\mathbb{E}[\| \widehat{\nabla} f(x) - \nabla f(x)\|_{*}^{p}\mid x]\leq \sigma^{p}$  for some  $p\in (1,2]$ , where  $\widehat{\nabla} f(x)$  denotes the stochastic gradient at  $x$  for the objective  $f$ . For the convex setting, the error bounds are for the optimality gaps  $\frac{1}{T}\sum_{t = 1}^{T}f(x_{t}) - f^{*}$ . For the nonconvex setting, we bound the gradient norm  $\frac{1}{T}\sum_{t = 1}^{T}\| \nabla f(x_{t})\|^{2}$ . Here,  $\tilde{O} (\cdot)$  hides polylog  $T$  factors. Note that, for simplicity, we do not compare against results in more specialized settings such as bounded domain or bounded gradients, as well as other variants of clipped SGD.
Assumptions	Convex Setting (Clipped-SMD)	Non-convex Setting (Clipped-SGD)
Lower bound	p ∈ (1, 2]	Ω(T1/p)[31]	Ω(T-(2(p-1)/3p-2)[33]
Previous high-probability results	Known T	O(T1/p)[28]	O(T1/p)[28]
Our results	Known T	O(T1/p)(Thm 4.1)	O(2-2p/T3p-2)(Thm 3.1)
Unknown T	O(T1/p)(Thm 4.4)	O(2-2p/T3p-2)(Thm B.2)
arbitrary norm and  $\| \cdot \| _*$  be its dual norm. In the nonconvex setting, we take  $\mathcal{X}$  to be  $\mathbb{R}^d$  and consider only the  $\ell_2$  norm.
Our paper works with the following assumptions:
(1) Existence of a minimizer: In the convex setting, we assume that there exists  $x^{*} \in \arg \min_{x \in \mathcal{X}} f(x)$ . We let  $f^{*} = f(x^{*})$ .
(1') Existence of a finite lower bound: In the nonconvex setting, we assume that  $f$  admits a finite lower bound, i.e.,  $f^{*} := \inf_{x \in \mathbb{R}^{d}} f(x) > -\infty$ .
(2) Unbiased estimator: We assume that our algorithm is allowed to query a stochastic first-order oracle that returns a history-independent, unbiased gradient estimator  $\widehat{\nabla} f(x)$  of  $\nabla f(x)$  for any  $x\in \mathcal{X}$ . That is, conditioned on the history and the queried point  $x$ , we have  $\mathbb{E}[\widehat{\nabla} f(x)\mid x] = \nabla f(x)$ .
(3) Bounded  $p$ th moment noise: We assume that there exists  $\sigma > 0$  such that for some  $1 < p \leq 2$  and for any  $x \in \mathcal{X}$ ,  $\widehat{\nabla} f(x)$  satisfies  $\mathbb{E}[\| \widehat{\nabla} f(x) - \nabla f(x)\|_*^p \mid x] \leq \sigma^p$ .
(4)  $L$ -smoothness: We consider the class of  $L$ -smooth functions: for all  $x, y \in \mathbb{R}^d$ ,  $\| \nabla f(x) - \nabla f(y) \|_* \leq L \| x - y \|$ .
2.2 Gradient Clipping Operator and Notations
We introduce the gradient clipping operator and its general properties used in Clipped-SMD (Algorithm 2) and Clipped-SGD (Algorithm 1). Let  $x_{t}$  be the output at iteration  $t$  of an algorithm of interest. We denote by  $\widehat{\nabla} f(x_{t})$  the stochastic gradient obtained by querying the gradient oracle. The clipped gradient estimate  $\widetilde{\nabla} f(x_{t})$  is taken as
$$
\widetilde {\nabla} f (x _ {t}) = \min  \left\{1, \frac {\lambda_ {t}}{\left\| \widehat {\nabla} f (x _ {t}) \right\| _ {*}} \right\} \widehat {\nabla} f (x _ {t}), \tag {1}
$$
where  $\lambda_{t}$  is the clipping parameter used in iteration  $t$ . In subsequent sections, we let  $\Delta_t \coloneqq f(x_t) - f^*$  denote the optimal function value gap at  $x_{t}$ . We let  $\mathcal{F}_t = \sigma \left(\widehat{\nabla} f(x_1),\dots ,\widehat{\nabla} f(x_t)\right)$  be the natural filtration at time  $t$  and define the following notations for the stochastic error, the deviation, and the bias of the clipped gradient estimate at time  $t$ :
$$
\theta_ {t} = \widetilde {\nabla} f (x _ {t}) - \nabla f (x _ {t}); \quad \theta_ {t} ^ {u} = \widetilde {\nabla} f (x _ {t}) - \mathbb {E} \left[ \widetilde {\nabla} f (x _ {t}) \mid \mathcal {F} _ {t - 1} \right]; \quad \theta_ {t} ^ {b} = \mathbb {E} \left[ \widetilde {\nabla} f (x _ {t}) \mid \mathcal {F} _ {t - 1} \right] - \nabla f (x _ {t}).
$$
Note that  $\theta_t^u +\theta_t^b = \theta_t$  . Regardless of the convexity of the function  $f$  , the following lemma provides upper bounds for these quantities. These bounds can be found in prior works [7, 33, 19, 28] for the
Algorithm 1 Clipped-SGD
Parameters: initial point  $x_{1}$ , step sizes  $\{\eta_t\}$ , clipping parameters  $\{\lambda_t\}$  for  $t = 1$  to  $T$  do
$$
\widetilde {\nabla} f (x _ {t}) = \min  \left\{1, \frac {\lambda_ {t}}{\| \widehat {\widetilde {\nabla}} f (x _ {t}) \|} \right\} \widehat {\nabla} f (x _ {t})
$$
$$
x _ {t + 1} = x _ {t} - \eta_ {t} \nabla f (x _ {t})
$$
special case of  $\ell_2$  norm. The extension to the general norm follows in the same manner, which we omit in this work.
Lemma 2.1. For stochastic gradients  $\widehat{\nabla} f(x_{t})$  with bounded pth moment noise, the clipped gradients  $\widetilde{\nabla} f(x_{t})$  satisfy the following properties:
$$
\left\| \theta_ {t} ^ {u} \right\| _ {*} = \left\| \widetilde {\nabla} f (x _ {t}) - \mathbb {E} \left[ \widetilde {\nabla} f (x _ {t}) \mid \mathcal {F} _ {t - 1} \right] \right\| _ {*} \leq 2 \lambda_ {t}. \tag {2}
$$
Furthermore, if  $\| \nabla f(x_{t})\|_{*}\leq \frac{\lambda_{t}}{2}$  then
$$
\left\| \theta_ {t} ^ {b} \right\| _ {*} = \left\| \mathbb {E} \left[ \widetilde {\nabla} f (x _ {t}) \mid \mathcal {F} _ {t - 1} \right] - \nabla f (x _ {t}) \right\| _ {*} \leq 4 \sigma^ {p} \lambda_ {t} ^ {1 - p}; \tag {3}
$$
$$
\mathbb {E} \left[ \| \theta_ {t} ^ {u} \| _ {*} ^ {2} \right] = \mathbb {E} \left[ \left\| \widetilde {\nabla} f (x _ {t}) - \mathbb {E} _ {t} \left[ \widetilde {\nabla} f (x _ {t}) \right] \right\| _ {*} ^ {2} \mid \mathcal {F} _ {t - 1} \right] \leq 4 0 \sigma^ {p} \lambda_ {t} ^ {2 - p}. \tag {4}
$$
Finally, we state a simple but important lemma that bounds the moment generating function of a zero-mean bounded random variable. The proof can be found in, for example, equation (3) of [1].
Lemma 2.2. Let  $X$  be a random variable such that  $\mathbb{E}[X] = 0$  and  $|X| \leq R$  almost surely. Then for  $0 \leq \lambda \leq \frac{1}{R}$
$$
\mathbb {E} \left[ \exp \left(\lambda X\right) \right] \leq \exp \left(\frac {3}{4} \lambda^ {2} \mathbb {E} \left[ X ^ {2} \right]\right).
$$
3 Clipped Stochastic Gradient Descent for Nonconvex Functions
In this section, we study the convergence of Clipped-SGD for nonconvex functions. Here, we consider the domain to be  $\mathbb{R}^d$  equipped with the standard  $\ell_2$  norm. We first outline a blackbox concentration argument to show convergence in high probability of Algorithm 1 and then follow up with a more powerful whitebox approach that allows for a tight high probability convergence analysis.
Comparison to previous works. In the simple setting of known time horizon and without momentum for Clipped-SGD, the  $\widetilde{O}(T^{\frac{2 - 2p}{3p - 2}})$  convergence rate has not been shown before to the best of our knowledge. The recent work by [28] study this case and only give a suboptimal rate of  $\widetilde{O}(T^{\frac{1 - p}{p}})$ . Note that [2, 19] study other variants of Clipped-SGD with momentum incorporated. Although [2, 19] achieve the nearly-optimal time dependency of  $\widetilde{O}(T^{\frac{2 - 2p}{3p - 2}})$  in the non-convex settings, they rely on using blackbox concentration inequalities which result in a suboptimal convergence rate that also requires a known time horizon.
We first present the guarantee for known time horizon  $T$  via our whitebox approach in Theorem 3.1 and defer the statement for unknown  $T$  in Theorem B.2 to the Appendix.
Theorem 3.1. Assume that  $f$  satisfies Assumption (1'), (2), (3), (4). Let  $\gamma \coloneqq \max \left\{\log \frac{1}{\delta}; 1\right\}$  and  $\Delta_1 \coloneqq f(x_1) - f^*$ . For known time horizon  $T$ , we choose  $\lambda_t$  and  $\eta_t$  such that
$$
\lambda_ {t} := \lambda := \max  \left\{\left(\frac {8 \gamma}{\sqrt {L \Delta_ {1}}}\right) ^ {\frac {1}{p - 1}} T ^ {\frac {1}{3 p - 2}} \sigma^ {\frac {p}{p - 1}}; 2 \sqrt {9 0 L \Delta_ {1}}; 3 2 ^ {\frac {1}{p}} \sigma T ^ {\frac {1}{3 p - 2}} \right\}
$$
$$
\eta_ {t} := \eta := \frac {\sqrt {\Delta_ {1}} T ^ {\frac {1 - p}{3 p - 2}}}{8 \lambda \sqrt {L} \gamma} = \frac {\sqrt {\Delta_ {1}}}{8 \sqrt {L} \gamma} \min  \left\{\left(\frac {8 \gamma}{\sqrt {L \Delta_ {1}}}\right) ^ {\frac {- 1}{p - 1}} T ^ {\frac {- p}{3 p - 2}} \sigma^ {\frac {- p}{p - 1}}; \frac {T ^ {\frac {1 - p}{3 p - 2}}}{2 \sqrt {9 0 L \Delta_ {1}}}; \frac {T ^ {\frac {- p}{3 p - 2}}}{3 2 ^ {1 / p} \sigma} \right\}.
$$
Then with probability at least  $1 - \delta$
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 1} ^ {T} \| \nabla f (x _ {t}) \| ^ {2} \leq 7 2 0 \sqrt {\Delta_ {1} L} \gamma \max  \left\{\left(\frac {8 \gamma}{\sqrt {L \Delta_ {1}}}\right) ^ {\frac {1}{p - 1}} T ^ {\frac {2 - 2 p}{3 p - 2}} \sigma^ {\frac {p}{p - 1}}; \right. \\ \left. 2 \sqrt {9 0 L \Delta_ {1}} T ^ {\frac {1 - 2 p}{3 p - 2}}; 3 2 ^ {1 / p} \sigma T ^ {\frac {2 - 2 p}{3 p - 2}} \right\} = O \left(T ^ {\frac {2 - 2 p}{3 p - 2}}\right). \\ \end{array}
$$
Remark 3.2. In comparison to the corresponding results in [28] (Theorem E.2), while our result achieves a poly  $T$  factor better rate when  $p < 2$ , the dependency on  $\log \frac{1}{\delta}$  in our result contains a dependency on  $p$  while the result in [28] does not. That term can dominate the convergence rate in the regime when  $\delta$  is very small and  $p$  is very close to 1. Hence, an open question is to remove such dependency on  $p$  for the  $\log \frac{1}{\delta}$  term while still maintain the optimal rate on  $T$ .
Now, we turn to the analysis, starting with the key Lemma 3.3 (proof in the Appendix).
Lemma 3.3. Assume that  $f$  satisfies Assumption (1'), (2), (3), (4) and  $\eta_t \leq \frac{1}{L}$  then for all  $t \geq 1$
$$
\begin{array}{l} \frac {\eta_ {t}}{2} \left\| \nabla f (x _ {t}) \right\| ^ {2} \leq \Delta_ {t} - \Delta_ {t + 1} + \left(L \eta_ {t} ^ {2} - \eta_ {t}\right) \left\langle \nabla f (x _ {t}), \theta_ {t} ^ {u} \right\rangle + \frac {3 \eta_ {t}}{2} \left\| \theta_ {t} ^ {b} \right\| ^ {2} \\ + L \eta_ {t} ^ {2} \left(\| \theta_ {t} ^ {u} \| ^ {2} - \mathbb {E} \left[ \| \theta_ {t} ^ {u} \| ^ {2} \mid \mathcal {F} _ {t - 1} \right]\right) + L \eta_ {t} ^ {2} \mathbb {E} \left[ \| \theta_ {t} ^ {u} \| ^ {2} \mid \mathcal {F} _ {t - 1} \right]. \tag {5} \\ \end{array}
$$
Remark 3.4. In Lemma 3.3, we decompose the RHS into appropriate terms that allow us to define a martingale. This lemma helps us understand why we can achieve a better convergence rate  $O(T^{\frac{2 - 2p}{3p - 2}})$  (for minimizing the norm squared of the gradient) in comparison to the best rate of  $O(T^{\frac{1 - p}{p}})$  in the convex setting. We focus on the error term  $\langle \nabla f(x_t),\theta_t\rangle = \langle \nabla f(x_t),\theta_t^u\rangle + \langle \nabla f(x_t),\theta_t^b\rangle$  on the RHS of (5). Since this error contains the gradient  $\nabla f(x_{t})$ , we leverage some of the gain  $\| \nabla f(x_{t})\|^{2}$  on the LHS of 5: we use Cauchy-Schwarz to bound  $\langle \nabla f(x_t),\theta_t^b\rangle \leq \frac{1}{2}\| \nabla f(x_t)\| ^2 +\frac{1}{2}\| \theta_t^b\| ^2$  and use the some of the gain to absorb the first term. Then setting our parameters  $\lambda_{t},\eta_{t}$  appropriately to balance the remaining terms helps us achieve the  $O(T^{\frac{2 - 2p}{3p - 2}})$  rate. Contrast this to the convex setting in the next section: the mismatch between the error term that contains the distance term  $\| x^{*} - x_{t}\|$  and the gain term that contains the function value gap  $f(x_{t}) - f^{*}$  prevents us from using the gain to absorb some of the error. Thus, this explains the convergence rate discrepancy between the convex case and the non-convex setting (see also Remark 4.6).
Before giving a sketch of our whitebox approach, we present a sketch of a blackbox argument that gives a nearly time-optimal convergence rate. This approach has an additional  $\log T$  factor in the final rate but will serve as a point of comparison for our new techniques, which will close the logarithmic gap.