SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Aiming towards the minimizers: fast convergence of SGD for overparameterized problems
Chaoyue Liu*, Dmitriy Drusvyatskiy**, Yian Ma*, Damek Davis***, and Mikhail Belkin*
* Halicioglu Data Science Institute, University of California San Diego
** Mathematics Department, University of Washington
*** School of Operations Research and Information Engineering, Cornell University
Modern machine learning paradigms, such as deep learning, occur in or close to the interpolation regime, wherein the number of model parameters is much larger than the number of data samples. In this work, we propose a regularity condition within the interpolation regime which endows the stochastic gradient method with the same worst-case iteration complexity as the deterministic gradient method, while using only a single sampled gradient (or a minibatch) in each iteration. In contrast, all existing guarantees require the stochastic gradient method to take small steps, thereby resulting in a much slower linear rate of convergence. Finally, we demonstrate that our condition holds when training sufficiently wide feedforward neural networks with a linear output layer.
Recent advances in machine learning and artificial intelligence have relied on fitting highly overparameterized models, notably deep neural networks, to observed data; e.g. [39, 37, 16, 23]. In such settings, the number of parameters of the model is much greater than the number of data samples, thereby resulting in models that achieve near-zero training error. Although classical learning paradigms caution against overfitting, recent work suggests ubiquity of the "double descent" phenomenon [3], wherein significant overparameterization actually improves generalization. The stochastic gradient method is the workhorse algorithm for fitting overparametrized models to observed data and understanding its performance is an active area of research. The goal of this paper is to obtain improved convergence guarantees for SGD in the interpolation regime that better align with its performance in practice.
Classical optimization literature emphasizes conditions akin to strong convexity as the phenomena underlying rapid convergence of numerical methods. In contrast, interpolation problems are almost never convex, even locally, around their solutions [24]. Furthermore, common optimization problems have complex symmetries, resulting in nonconvex sets of minimizers. Case in point, standard formulations for low-rank matrix recovery [4] are invariant under orthogonal transformations while ReLU neural networks are invariant under rebalancing of adjacent weight matrices [9]. The Polyak-Lojasiewicz (PL) inequality, introduced independently in [26] and [34], serves as an alternative to strong convexity that holds often in applications and underlies rapid convergence of numerical algorithms. Namely, it has been known since [34] that gradient descent converges under the PL condition at a linear rate  $O(\exp(-t / \kappa))$ , where  $\kappa$  is the condition number of the function. In contrast, convergence guarantees for the stochastic gradient method under PL—the predominant algorithm in practice—are much less satisfactory. Indeed, all known results require SGD to take
<sup>1</sup>In particular, for a smooth function with  $\beta$ -Lipschitz gradient satisfying a PL inequality with constant  $\alpha$ , the condition number is  $\kappa = \beta / \alpha$ .
Figure 1: Convergence plot of SGD when training a fully connected neural network with 3 hidden layers and 1000 neurons in each on MNIST (left) and a ResNet-28 on CIFAR-10 (right). MNIST has 60k images and the convergence behavior stabilizes with small batchsize of  $m = 64$ ; CIFAR-10 has 60k images and the convergence behavior stabilizes with batchsize of  $m = 512$ , although early on in training (first 20k iterations) convergence behavior looks identical for all batchesizes.
shorter steps than gradient descent to converge at all [2, 12], with the disparity between the two depending on the condition number  $\kappa$ . The use of the small stepsize directly translates into a slow rate of convergence. This requirement is in direct contrast to practice, where large step-sizes are routinely used. As a concrete illustration of the disparity between theory and practice, Figure 1 depicts the convergence behavior of SGD for training a neural network on the MNIST data set. As is evident from the Figure, even for small batch sizes, the linear rate of convergence of SGD is comparable to that of GD with an identical stepsize  $\eta = 0.1$ . Moreover, experimentally, we have verified that the interval of stepsizes leading to convergence for SGD is comparable to that of GD; indeed, the two are off only by a factor of 20. Using large stepsizes also has important consequences for generalization. Namely, recent works [19, 5] suggest that large step sizes bias the iterates towards solutions that generalize better to unseen data. Therefore understanding the dynamics of SGD with large step sizes is an important research direction. The contribution of our work is as follows.
In this work, we highlight regularity conditions that endow SGD with a fast linear rate of convergence  $\exp(-t/\kappa)$  both in expectation and with high probability, even when the conditions hold only locally. Moreover, we argue that the conditions we develop are reasonable because they provably hold on any compact region when training sufficiently wide feedforward neural networks with a linear output layer.
1.1 Outline of main results.
We will focus on the problem of minimizing a loss function  $\mathcal{L}$  under the following two assumptions. First, we assume that  $\mathcal{L}$  grows quadratically away from its set of global minimizers  $S$ :
$$
\mathcal {L} (w) \geq \frac {\alpha}{2} \cdot \operatorname {d i s t} ^ {2} (w, S) \quad \forall w \in B _ {r} \left(w _ {0}\right), \tag {QG}
$$
where  $B_r(w_0)$  is a ball of radius  $r$  around the initial point  $w_0$ . This condition is standard in the optimization literature and is implied for example by the PL-inequality holding on the ball  $B_r(w_0)$ ; see Section A.2. Secondly, and most importantly, we assume there are exist constants  $\theta, \rho > 0$  satisfying the aiming condition:
$$
\langle \nabla \mathcal {L} (w), w - \operatorname {p r o j} _ {S} (w) \rangle \geq \theta \cdot \mathcal {L} (w) \quad \forall w \in B _ {r} \left(w _ {0}\right). \tag {Aiming}
$$
Here,  $\operatorname{proj}_S(w)$  denotes a nearest point in  $S$  to  $w$  and  $\mathrm{dist}(w,S)$  denotes the distance from  $w$  to  $S$ . The aiming condition ensures that the negative gradient  $-\nabla \mathcal{L}(w)$  points towards  $S$  in the sense that  $-\nabla \mathcal{L}(w)$  correlated nontrivially with the direction  $\operatorname{proj}_S(w) - w$ . At first sight, the aiming condition appears similar to quasar-convexity, introduced in [14] and further studied in [15, 22, 20]. Namely a function  $\mathcal{L}$  is quasar-convex relative to a fixed point  $\bar{w} \in S$  if the estimate (Aiming) holds with  $\operatorname{proj}_S(w)$  replaced by  $\bar{w}$ . Although the distinction between aiming and quasar-convexity may
$^2$ If  $\mathrm{proj}_S(w)$  is not a singleton,  $\mathrm{proj}_S(w)$  in the expression should be replaced with any element  $\bar{w}$  from  $\mathrm{proj}_S(w)$ .
appear mild, it is significant. As a concrete example, consider the function  $\mathcal{L}(x,y) = \frac{1}{2} (y - ax^2)^2$  for any  $a > 0$ . It is straightforward to see that  $\mathcal{L}$  satisfies the aiming condition on some neighborhood of the origin. However, for any neighborhood  $U$  of the origin, the function  $\mathcal{L}$  is not quasar-convex on  $U$  relative to any point  $(x,ax^2)\in U$ ; see Section C. More generally, we show that (Aiming) holds automatically for any  $C^3$  smooth function  $\mathcal{L}$  satisfying (QG) locally around the solution set. Indeed, we may shrink the neighborhood to ensure that  $\theta$  is arbitrarily close to 2. Secondly, we show that (Aiming) holds for sufficiently wide feedforward neural networks with a linear output layer.
Our first main result can be summarized as follows. Roughly speaking, as long as the SGD iterates remain in  $B_r(w_0)$ , they converge to  $S$  at a fast linear rate  $O(\exp(-t\alpha\theta^2/\beta))$  with high probability.
Theorem 1.1 (Informal). Consider minimizing the function  $\mathcal{L}(w) = \mathbb{E}\ell(w,z)$ , where the losses  $\ell(\cdot, z)$  are nonnegative and have  $\beta$ -Lipschitz gradients. Suppose that the minimal value of  $\mathcal{L}$  is zero and both regularity conditions (QG) and (Aiming) hold. Then as long as the SGD iterates  $w_t$  remain in  $B_r(w_0)$ , they converge to  $S$  at a linear rate  $O(\exp(-t\alpha\theta^2/\beta))$  with high probability.
The proof of the theorem is short and elementary. The downside is that the conclusion of the theorem is conditional on the iterates remaining in  $B_r(w_0)$ . Ideally, one would like to estimate this probability as a function of the problem parameters. With this in mind, we show that for a special class of nonlinear least squares problems, including those arising when fitting wide neural networks, this probability may be estimated explicitly. The end result is the following unconditional theorem.
Theorem 1.2 (Informal). Consider the loss  $\mathcal{L}(w) = \frac{1}{n}\sum_{i=1}^{n}(f(w,x_i) - y_i)^2$ , where  $f(w,\cdot)$  is a fully connected neural network with  $l$  hidden layers and a linear output layer. Let  $\lambda_{\infty}$  be the minimal eigenvalue of the Neural Tangent Kernel of an infinitely wide neural network. Then with high probability both conditions (QG) and (Aiming) hold on a ball of radius  $r$  around the initial point  $w_0 \sim N(0,I)$  with  $\theta = 1$  and  $\alpha = \lambda_{\infty}/2$ , as long as the network width  $m$  satisfies  $m = \tilde{\Omega}(nr^{6l+2}/\lambda_{\infty}^2)$ . If in addition  $r = \Omega(1/\delta\sqrt{\lambda_{\infty}})$ , with probability  $1-\delta$ , SGD with stepsize  $\eta = \Theta(1)$  converges to a zero-loss solution at the fast rate  $O(\exp(-t\lambda_{\infty}))$ . This parameter regime is identical as for gradient descent to converge in [24], with the only exception of the inflation of  $r$  by  $1/\delta$ .
A key part of the argument is to estimate the probability that the iterates remain in a ball  $B_r(w_0)$ . A naive approach is to bound the length of the iterate trajectory in expectation, but this would then require the radius  $r$  to expand by an additional factor of  $1 / \lambda_{\infty}$ , which in turn would increase  $m$  multiplicatively by  $\lambda_{\infty}^{-6l - 2}$ . We avoid this exponential blowup by a careful stopping time argument and the transition to linearity phenomenon that has been shown to hold for sufficiently wide neural networks [24]. While Theorem 1.2 is stated with a constant failure probability, there are standard ways to remove the dependence. One option is to simply set  $\delta_1 = \Omega(1)$  and rerun SGD logarithmically many times from the same initialization  $w_0$  and return the final iterate with smallest function value. Section 4 outlines a more nuanced strategy based on a small ball assumption, which entirely avoids computation of the function values of the full objective.
1.2 Comparison to existing work.
We next discuss how our results fit within the existing literature, summarized in Table 1. Setting the stage, consider the problem of minimizing a smooth function  $\mathcal{L}(w) = \mathbb{E}\ell (w,z)$  and suppose for simplicity that its minimal value is zero. We say that  $\mathcal{L}$  satisfies the Polyak-Lojasiewicz (PL) inequality if there exists  $\alpha >0$  satisfying
$$
\| \nabla \mathcal {L} (w) \| ^ {2} \geq 2 \alpha \cdot \mathcal {L} (w), \tag {PL}
$$
for all  $w \in \mathbf{R}^d$ . In words, the gradient  $\nabla \mathcal{L}(w)$  dominates the function value  $\mathcal{L}(w)$ , up to a power. Geometrically, such functions have the distinctive property that the gradients of the rescaled function  $\sqrt{\mathcal{L}(w)}$  are uniformly bounded away from zero outside the solution set. See Figures 2a and 2b for an illustration. Using the PL inequality, we may associate to  $\mathcal{L}$  two condition numbers, corresponding to the full objective and its samples, respectively. Namely, we define  $\bar{\kappa} \triangleq \bar{\beta}/\alpha$  and  $\kappa \triangleq \beta/\alpha$ , where  $\bar{\beta}$  is a Lipschitz constant of the full gradient  $\nabla \mathcal{L}$  and  $\beta$  is a Lipschitz constant of the sampled gradients  $\nabla \ell(\cdot, z)$  for all  $z$ . Clearly, the inequality  $\bar{\kappa} \leq \kappa$  holds and we will primarily be interested in settings where the two are comparable.
The primary reason why the PL condition is useful for optimization is that it ensures linear convergence of gradient-type algorithms. Namely, it has been known since Polyak's seminal work
(a)  $\mathcal{L}(w)$  satisfying the PL condition
(b) gradients of  $\sqrt{\mathcal{L}(w)}$  are far from zero
[34] that the full-batch gradient descent iterates  $w_{t + 1} = w_t - \frac{1}{\beta}\nabla \mathcal{L}(w_t)$  converge at the linear rate  $O(\exp (-t / \bar{\kappa}))$ . More recent papers have extended results of this type to a wide variety of algorithms both for smooth and nonsmooth optimization [27, 7, 21, 30, 1] and to settings when the PL inequality holds only locally on a ball [24, 32].
For stochastic optimization problems under the PL condition, the story is more subtle, since the rates achieved depend on moment bounds on the gradient estimator, such as:
$$
\mathbb {E} \left[ \| \nabla \ell (w, z) \| ^ {2} \right] \leq A \mathcal {L} (w) + B \| \nabla \mathcal {L} (w) \| ^ {2} + C, \tag {1.1}
$$
for  $A, B, C \geq 0$ . In the setting where  $C > 0$ —the classical regime—stochastic gradient methods converge sublinearly at best, due to well-known lower complexity bounds in stochastic optimization [31]. On the other hand, in the setting where  $C = 0$ —interpolation problems—stochastic gradient methods converge linearly when equipped with an appropriate stepsize, as shown in [2, Theorem 1], [22, Corollary 2], [38], and [13, Theorem 4.6]. Although linear convergence is assured, the rate of linear convergence under the PL condition and interpolation is an order of magnitude worse than in the deterministic setting. Namely, the three papers [2, Theorem 1], [22, Corollary 2] and [13, Theorem 4.6] obtain linear rates on the order of  $\exp(-t/\bar{\kappa}\kappa)$ . On the other hand, in the case  $A = C = 0$ , which is called the strong growth property, the paper [38, Theorem 4] yields the seemingly better rate  $\exp(-t/B\bar{\kappa})$ . The issue, however, is that  $B$  can be extremely large. As an illustrative example, consider the loss functions  $\ell(w,z) = \frac{1}{2}\mathrm{dist}^2(w,Q_z)$  where  $Q_z$  are smooth manifolds. A quick computation shows that equality  $\|\nabla\ell(w,z)\|^2 = 2\ell(w,z)$  holds. Therefore, locally around the intersection of  $\cap_{z}Q_{z}$ , the estimate (1.1) with  $A = C = 0$  is exactly equivalent to the PL-condition with  $B = 1/\alpha$ . As a further illustration, Figure 3 shows the possible large value of the constant  $B$  along the SGD iterates for training a neural network on MNIST. Another related paper is [36]: assuming so-called small gradient confusion and requiring a stronger version of PL condition (i.e., each individual loss  $l_i$  is  $\mu$ -PL), the paper [36] showed a slow rate of convergence  $\exp(-t/n^2\kappa)$ , where  $n$  is the dataset size.
Figure 3: We train a fully-connected neural network on the MNIST dataset. The network has 4 hidden layers, each with 1024 neurons. We optimize the MSE loss using SGD with a batch size 512 and a learning rate 0.5. The training was run over 1k epochs, and the ratio  $\mathbb{E}[\|\nabla\ell(w,z)\|^2]/\|\nabla\mathcal{L}(w)\|^2$  is evaluated every 100 epochs. The ratio grows almost linearly during training, suggesting that strong growth is practically not satisfied with a constant coefficient  $B$ .
Reference	Bound on E[||∇l(w,z)||2]	Quasar Convex?	Rate
[2, Theorem 1]	2β(L(w) - L*)	No	exp(-t/κk)
[38, Theorem 4]	B||∇L(w)||2	No	exp(-t/Bk)
[22, Corollary 2]	AL(w) + B||∇L(w)||2	No	exp(-t/κmax{B,A/α})
[13, Theorem 4.6]	2β(L(w) - L*) + ||∇L(w)||2	No	exp(-t/κk)
[11, Corollary 3.3]	σ2+ 2α||w - w*||2	Yes	Sublinear
[20, Theorem 4.4]	σ2+ ||∇L(w)||2	Yes	Sublinear
This work	2β(L(w) - L*)	(Aiming)	exp(-tθ2/κ)
Table 1: Comparison to recent work on nonconvex stochastic gradient methods under the  $\alpha$ -PL and smoothness conditions. We define  $\bar{\kappa} = \bar{\beta}/\alpha$  and  $\kappa = \beta/\alpha$ , where  $\bar{\beta}$  is a Lipschitz constant of the full gradient  $\nabla \mathcal{L}$  and  $\beta$  is a Lipschitz constant of the sampled gradients  $\nabla \ell(\cdot, z)$  for all  $z$ .
The purpose of this work is to understand whether we can improve stepsize selection and the convergence rate of SGD for nonconvex problems under the (local) PL condition and interpolation. Unfortunately, the PL condition alone appears too weak to yield improved rates. Instead, we take inspiration from recent work on accelerated deterministic nonconvex optimization, where the recently introduced quasar-convexity condition has led to improved rates [15]. We note that quasar-convexity is a very restrictive assumption in the interpolation regime because it requires the solution set to be an affine subspace; see the discussion in Appendix E. Recent work has also shown that quasar convexity can lead to accelerated sublinear rates of convergence for certain stochastic optimization problems [20, Theorem 4.4] (and the concurrent work [11, Corollary 3.3]), but to the best of our knowledge, there are no works that analyze improved linear rates of convergence. Thus, in this work, we fill the gap in the literature, by providing a rate that matches that of deterministic gradient descent and allows for a large stepsize. Moreover, in contrast to most available results, we only assume that regularity conditions hold on a ball—the common setting in applications. The local nature of the assumptions requires us to bound the probability of the iterates escaping.
2 Main results
Throughout the paper, we will consider the stochastic optimization problem
$$
\min_{w}\mathcal{L}(w)\triangleq \underset {z\sim \mathcal{P}}{\mathbb{E}}\ell (w,z),
$$
where  $\mathcal{P}$  is a probability distribution that is accessible only through sampling and  $\ell(\cdot, z)$  is a differentiable function on  $\mathbf{R}^d$ . We let  $S$  denote the set of minimizers of  $\mathcal{L}$ . We impose that  $\mathcal{L}$  satisfies the following assumptions on a set  $\mathcal{W}$ . The two main examples are when  $\mathcal{W}$  is a ball  $B_r(w_0)$  and when  $\mathcal{W}$  is a tube around the solution set:
$$
S _ {r} \triangleq \{w \in \mathbf {R} ^ {d}: \operatorname {d i s t} (w, S) \leq r \}.
$$
Assumption 1 (Running assumptions). Suppose that there exist constants  $\alpha, \beta, \theta \geq 0$  and a set  $\mathcal{W} \subset \mathbf{R}^d$  satisfying the following.
1. (Interpolation) The losses  $\ell(w, z)$  are nonnegative, the minimal value of  $\mathcal{L}$  is zero, and the set of minimizers  $S \triangleq \operatorname{argmin} \mathcal{L}$  is nonempty.
2. (Smoothness) For almost every  $z \sim \mathcal{P}$ , the loss  $\ell(\cdot, z)$  is differentiable and the gradient  $\nabla \ell(\cdot, z)$  is  $\beta$ -Lipschitz continuous on  $\mathcal{W}$ .
3. (Quadratic growth) The estimate holds:
$$
\mathcal {L} (w) \geq \frac {\alpha}{2} \cdot \operatorname {d i s t} ^ {2} (w, S) \quad \forall w \in \mathcal {W}. \tag {2.1}
$$
4. (Aiming) For all  $w \in \mathcal{W}$  there exists a point  $\bar{w} \in \mathrm{proj}(w, S)$  such that
$$
\langle \nabla \mathcal {L} (w), w - \bar {w} \rangle \geq \theta \cdot \mathcal {L} (w). \tag {2.2}
$$
We define the condition number  $\kappa \triangleq \beta /\alpha$
As explained in the introduction, the first three conditions (1)-(3) are classical in the literature. In particular, both quadratic growth (3) on a ball  $\mathcal{W} = B_r(w_0)$  and existence of solutions in  $\mathcal{W}$  follow from a local PL-inequality. In order to emphasize the local nature of the condition, following [24] we say that  $\mathcal{L}$  is  $\alpha$ - $PL^*$  on  $\mathcal{W}$  if the inequality  $(\mathsf{PL})$  holds for all  $w \in \mathcal{W}$ . We recall the proof of the following lemma in Section A.2.
Lemma 2.1 (PL* condition implies quadratic growth). Suppose that  $\mathcal{L}$  is differentiable and is  $\alpha$ -PL* on a ball  $B_{2r}(w_0)$ . Then as long as  $\mathcal{L}(w_0) < \frac{1}{2}\alpha r^2$ , the intersection  $S \cap B_r(w_0)$  is nonempty and
$$
\mathcal {L} (w) \geq \frac {\alpha}{8} \operatorname {d i s t} ^ {2} (w, S) \quad \forall w \in B _ {r} (w _ {0}).
$$
The aiming condition (4) is very closely related to quasar-convexity, which requires (2.2) to hold for all  $w \in \mathbf{R}^d$  and a distinguished point  $\bar{w} \in S$  that is independent of  $w$ . This distinction may seem mild, but is in fact important because aiming holds for a much wider class of problems. As a concrete example, consider the function  $\mathcal{L}(x,y) = \frac{1}{2} (y - ax^2)^2$  for any  $a > 0$ . It is straightforward to see that  $\mathcal{L}$  satisfies the aiming condition on some neighborhood of the origin. However, for any neighborhood  $U$  of the origin, the function  $\mathcal{L}$  is not quasar-convex on  $U$  relative to any point  $(x,ax^2) \in U$ ; see Section C. We now show that (4) is valid locally for any  $C^3$ -smooth function satisfying quadratic growth, and we may take  $\theta$  arbitrarily close to 2 by shrinking  $r$ . Later, we will also show that problems of learning wide neural networks also satisfy the aiming condition.
Theorem 2.2 (Local aiming). Suppose that  $\mathcal{L}$  is  $C^2$ -smooth and  $\nabla^2\mathcal{L}$  is  $L$ -Lipschitz continuous on the tube  $S_r$ . Suppose moreover that  $\mathcal{L}$  satisfies the quadratic growth condition (2.1) and  $r < \frac{6\alpha}{5L}$ . Then the aiming condition (2.2) holds with parameter  $\theta = 2 - \frac{5Lr}{3\alpha}$ . An analogous statement holds if  $S_r$  is replaced by a ball  $B_r(\bar{w}_0)$  for some  $\bar{w}_0 \in S$ .
Algorithm 1 SGD  $(w_0,\eta ,T)$
Initialize: Initial  $w_0 \in \mathbf{R}^d$ , learning rate  $\eta > 0$ , iteration counter  $T \in \mathbb{N}$ . For  $t = 1, \dots, T - 1$  do:
$$
\mathrm {S a m p l e} z _ {t} \sim \mathcal {P}
$$
$$
\operatorname {S e t} w _ {t + 1} = w _ {t} - \eta \nabla \ell \left(w _ {t}, z _ {t}\right).
$$
Return:  $w_{T}$ .
2.1 SGD under regularity on a tube  $S_{r}$
Convergence analysis for SGD (Algorithm 1) is short and elementary in the case  $\mathcal{W} = S_r$  and therefore this is where we begin. We note, however, that the setting  $\mathcal{W} = B_r(w_0)$  is much more realistic, as we will see, but also more challenging.
The converge analysis of SGD proceeds by a familiar one-step contraction argument.
Lemma 2.3 (One-step contraction on a tube). Suppose that Assumption 1 holds on a tube  $\mathcal{W} = S_{2r}$  and fix a point  $w\in S_r$ . Define the updated point  $w^{+} = w - \eta \nabla f(w,z)$  where  $z\sim \mathcal{P}$ . Then for any stepsize  $\eta < \frac{\theta}{\beta}$ , the estimate holds:
$$
\underset {z \sim \mathcal {P}} {\mathbb {E}} \operatorname {d i s t} ^ {2} \left(w ^ {+}, S\right) \leq \left(1 - \alpha \eta \left(\theta - \beta \eta\right)\right) \operatorname {d i s t} ^ {2} (w, S). \tag {2.3}
$$
Using the one step guarantee of Lemma 2.3, we can show that SGD iterates converge linearly to  $S$  if Assumption 1 holds on a tube  $\mathcal{W} = S_{2r}$ . The only complication is to argue that the iterates are unlikely to leave the tube if we start in a slightly smaller tube  $S_{r'}$  for some  $r' < r$ . We do so with a simple stopping time argument.
Theorem 2.4 (Convergence on a tube). Suppose that Assumption 1 holds relative to a tube  $\mathcal{W} = S_{2r}$  for some constant  $r > 0$ . Fix a stepsize  $\eta > 0$  satisfying  $\eta < \frac{\theta}{\beta}$ . Fix a constant  $\delta_1 > 0$  and a point