SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
T \geq \frac {\log 2}{\log (1 + B A ^ {2} / T)}.
$$
Now note that  $\log (1 + x) \geq \frac{x}{1 + x}$  for  $x \geq 0$ , which implies that  $\frac{1}{\log(1 + x)} \leq 1 + \frac{1}{x}$  for  $x \geq 0$ . Applying this gives that
$$
\frac {1}{\log (1 + B A ^ {2} / T)} \leq 1 + \frac {T}{B A ^ {2}}.
$$
So to guarantee  $\frac{1}{1 - \theta^T} \leq 2$ , from (26) it is sufficient to enforce that
$$
T \geq 1 + \frac {T}{B A ^ {2}}
$$
$$
\geq \log (2) \left(1 + \frac {T}{B A ^ {2}}\right).
$$
Assuming  $A \geq \sqrt{2 / B}$ , this last condition holds if
$$
T \geq 1 + \frac {1}{B A ^ {2} - 1}. \tag {27}
$$
Since we also impose that  $BA^2 \geq 2$  we have that  $\frac{1}{BA^2 - 1} \leq 1$  thus for (27) to hold it suffices that  $T \geq 2$ . Substituting  $\gamma = \frac{A}{\sqrt{T}}$  and the relaxation  $\frac{1}{1 - \theta^T} \leq 2$  into the original bound gives the claimed result.
9.3 Proximal gradient descent with strong convexity and smoothness
We start by recalling some standard properties of the proximal operator.
Lemma 33. Let  $\gamma >0$  and  $w^{*}\in \operatorname{argmin}_{w}l(w) + h(w)$ . Assume  $h(w)$  is convex and that  $l(w)$  is continuously differentiable at  $w^{*}$ . Then
1. For all  $w, w'$  and  $\gamma > 0$ ,  $\left\| \mathrm{prox}_{\gamma h}(w) - \mathrm{prox}_{\gamma h}(w') \right\|_2 \leq \| w - w' \|_2$ .
2.  $w^{*} = \mathrm{prox}_{\gamma h}(w^{*} - \gamma \nabla l(w^{*}))$
3.  $v = \operatorname{prox}_{\gamma h}(w) \Leftrightarrow \frac{w - v}{\gamma} \in \partial h(v)$
Alternatively, if we use the decaying stepsize  $\gamma_t = \min \left\{\frac{\mu}{2a}, \frac{1}{\mu} \frac{2t + 1}{(t + 1)^2}\right\}$ , then
$$
\mathbb {E} \left\| w ^ {T} - w ^ {*} \right\| ^ {2} \leq \frac {1 6 \left\lfloor a / \mu^ {2} \right\rfloor^ {2}}{T ^ {2}} \left\| w ^ {0} - w ^ {*} \right\| ^ {2} + \frac {8}{\mu^ {2} T} (b + M ^ {2} \left\| w ^ {*} - \bar {w} \right\| _ {2} ^ {2}). \tag {12}
$$
Proof. We start by using the non-expansiveness and fixed-point properties of the proximal operator to write
$$
\begin{array}{l} \left\| w ^ {t + 1} - w ^ {*} \right\| _ {2} ^ {2} = \left\| \operatorname {p r o x} _ {\gamma h} \left(w ^ {t} - \gamma g ^ {t}\right) - \operatorname {p r o x} _ {\gamma h} \left(w ^ {*} - \gamma \nabla l \left(w ^ {*}\right)\right) \right\| _ {2} ^ {2} \\ \leq \left\| w ^ {t} - w ^ {*} + \gamma (\nabla l (w ^ {*}) - g ^ {t}) \right\| _ {2} ^ {2} \\ = \left\| w ^ {t} - w ^ {*} \right\| _ {2} ^ {2} + \gamma^ {2} \left\| \nabla l \left(w ^ {*}\right) - g ^ {t} \right\| _ {2} ^ {2} + 2 \gamma \left\langle w ^ {t} - w ^ {*}, \nabla l \left(w ^ {*}\right) - g ^ {t} \right\rangle . \\ \end{array}
$$
Above the first line uses the fixed point property of the proximal operator that  $w^{*} = \mathrm{prox}_{\gamma h}(w^{*} - \gamma \nabla l(w))$ , while the second line uses the fact that the proximal operator is non-expansive.
Now, we apply our assumption on the gradient estimator to write
$$
\begin{array}{l} \mathbb {E} \left[ \left\| \nabla l \left(w ^ {*}\right) - g ^ {t} \right\| _ {2} ^ {2} \mid w ^ {t} \right] \leq 2 \mathbb {E} \left[ \left\| \nabla l \left(w ^ {*}\right) \right\| _ {2} ^ {2} \mid w ^ {t} \right] + 2 \mathbb {E} \left[ \left\| g ^ {t} \right\| _ {2} ^ {2} \mid w ^ {t} \right] \\ \leq 2 \| \nabla l (w ^ {*}) \| _ {2} ^ {2} + 2 \left(a \| w ^ {t} - w ^ {*} \| _ {2} ^ {2} + b\right) \\ \leq 2 M ^ {2} \left\| w ^ {*} - \bar {w} \right\| _ {2} ^ {2} + 2 \left(a \left\| w ^ {t} - w ^ {*} \right\| _ {2} ^ {2} + b\right). \\ \end{array}
$$
For the last term, we have by strong convexity and the fact that  $g^{t}$  is an unbiased estimator that
$$
\begin{array}{l} \mathbb {E} \left[ \left\langle w ^ {t} - w ^ {*}, \nabla l \left(w ^ {*}\right) - g ^ {t} \right\rangle \mid w _ {t} \right] = - \left\langle w ^ {*} - w ^ {t}, \nabla l \left(w ^ {*}\right) - \nabla l \left(w ^ {t}\right) \right\rangle \\ \leq - \mu \left\| w ^ {t} - w ^ {*} \right\| _ {2} ^ {2}, \\ \end{array}
$$
where the second line follows from the fact that  $l$  is  $\mu$  strongly convex.
Putting the pieces together, we get that
$$
\begin{array}{l} \mathbb {E} \left[ \left\| w ^ {t + 1} - w ^ {*} \right\| _ {2} ^ {2} \right] \leq \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| _ {2} ^ {2} + 2 \gamma^ {2} M ^ {2} \left\| w ^ {*} - \bar {w} \right\| _ {2} ^ {2} + 2 \gamma^ {2} \mathbb {E} \left(a \left\| w ^ {t} - w ^ {*} \right\| _ {2} ^ {2} + b\right) - 2 \gamma \mu \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| \\ = \left(1 - 2 \gamma \mu + 2 \gamma^ {2} a\right) \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| _ {2} ^ {2} + 2 \gamma^ {2} \left(b + M ^ {2} \| w ^ {*} - \bar {w} \| _ {2} ^ {2}\right) \\ \end{array}
$$
Now the following conditions are equivalent:
$$
1 - 2 \gamma \mu + 2 \gamma^ {2} a \leq 1 - \gamma \mu
$$
$$
2 \gamma \mu - 2 \gamma^ {2} a \geq \gamma \mu
$$
$$
2 \mu - 2 \gamma a \geq \mu
$$
$$
2 \mu - \mu \geq 2 \gamma a
$$
$$
\mu \geq 2 \gamma a
$$
$$
\gamma \leq \frac {\mu}{2 a}
$$
$$
\mathbb {E} \left[ \left\| w ^ {t + 1} - w ^ {*} \right\| _ {2} ^ {2} \right] \leq (1 - \gamma \mu) \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| _ {2} ^ {2} + 2 \gamma^ {2} \left(b + M ^ {2} \| w ^ {*} - \bar {w} \| _ {2} ^ {2}\right).
$$
Now since  $\gamma \leq \frac{1}{\mu} \Leftrightarrow 1 - \gamma \mu \geq 0$  we can apply the above recursively, which gives that
$$
\mathbb {E} \left[ \left\| w ^ {t + 1} - w ^ {*} \right\| _ {2} ^ {2} \right] \leq (1 - \gamma \mu) ^ {t} \left\| w ^ {0} - w ^ {*} \right\| _ {2} ^ {2} + 2 \gamma^ {2} \left(b + M ^ {2} \left\| w ^ {*} - \bar {w} \right\| _ {2} ^ {2}\right) \sum_ {k = 1} ^ {t - 1} (1 - \gamma \mu) ^ {k}.
$$
But we can bound this geometric sum by
$$
\sum_ {k = 1} ^ {t - 1} \left(1 - \gamma \mu\right) ^ {k} = \frac {1 - \left(1 - \gamma \mu\right) ^ {t}}{\gamma \mu} \leq \frac {1}{\gamma \mu},
$$
leading to the result that
$$
\mathbb {E} \left[ \left\| w ^ {t + 1} - w ^ {*} \right\| _ {2} ^ {2} \right] \leq (1 - \gamma \mu) ^ {t} \left\| w ^ {0} - w ^ {*} \right\| _ {2} ^ {2} + \frac {2 \gamma}{\mu} \left(b + M ^ {2} \left\| w ^ {*} - \bar {w} \right\| _ {2} ^ {2}\right).
$$
To prove the anytime result, we can now apply Theorem 31. To do so, we need to map the notation we use here to the notation used in Theorem 31. The mapping we need is  $\mathcal{L} = \frac{a}{\mu}$ , and  $\sigma^2 = \left(b + M^2\| w^* - \bar{w}\|_2^2\right)$  which when inserting into Theorem 31 gives the result.
Corollary 34. Under the conditions of Theorem 7, if  $\gamma = \frac{A\log T}{T}$  for  $\frac{1}{\mu} \leq A$ , and  $T$  is large enough that  $\frac{T}{\log T} \geq \frac{2aA}{\mu}$ ,
$$
\mathbb {E} \left\| w ^ {T + 1} - w ^ {*} \right\| _ {2} ^ {2} \leq \frac {1}{T} \left\| w ^ {0} - w ^ {*} \right\| _ {2} ^ {2} + \frac {2 A \log T}{\mu T} \left(b + M ^ {2} \| w ^ {*} - \bar {w} \| _ {2} ^ {2}\right) = \Omega \left(\frac {\log T}{T}\right).
$$
Proof. The previous theorem can only be applied when  $\gamma \leq \frac{\mu}{2a}$ , i.e. when  $\frac{A\log T}{T} \leq \frac{\mu}{2a}$  or  $\frac{2aA}{\mu} \leq \frac{T}{\log T}$ . Supposing that is true, observe that  $(1 - x)^T \leq \exp(-xT)$ , from which it follows that  $(1 - \gamma \mu)^T \leq \exp(-\mu A\log T) = \frac{1}{T^{\mu A}} \leq \frac{1}{T}$ .
9.4 Proximal gradient descent with convexity and smoothness
$$
\mathbb {E} \left[ f (\bar {w} ^ {T}) - \inf  f \right] \leq \gamma \left(a \frac {\| w ^ {0} - w ^ {*} \| ^ {2}}{(1 - \theta^ {T})} + b\right),
$$
Proof. Let us start by looking at  $\| w^{t + 1} - w^* \|^2 - \| w^t - w^* \|^2$ . Expanding the squares, we have that
$$
\frac {1}{2 \gamma_ {t}} \| w ^ {t + 1} - w ^ {*} \| ^ {2} - \frac {1}{2 \gamma_ {t}} \| w ^ {t} - w ^ {*} \| ^ {2} = \frac {- 1}{2 \gamma_ {t}} \| w ^ {t + 1} - w ^ {t} \| ^ {2} - \langle \frac {w ^ {t} - w ^ {t + 1}}{\gamma_ {t}}, w ^ {t + 1} - w ^ {*} \rangle .
$$
Since  $w^{t + 1} = \mathrm{prox}_{\gamma_t h}(w^t -\gamma_t g^t)$ , we know from Lemma 33 that
$$
\frac {\left(w ^ {t} - \gamma_ {t} g ^ {t}\right) - w ^ {t + 1}}{\gamma_ {t}} \in \partial h \left(w ^ {t + 1}\right)
$$
and therefore that
$$
\frac {w ^ {t} - w ^ {t + 1}}{\gamma_ {t}} \in g ^ {t} + \partial h (w ^ {t + 1}),
$$
where  $\partial h(w)$  is the subdifferential of  $h$ . Consequently there exists a subgradient  $b^{t + 1}\in \partial h(w^{t + 1})$  such that
$$
\begin{array}{l} \frac {1}{2 \gamma_ {t}} \| w ^ {t + 1} - w ^ {*} \| ^ {2} - \frac {1}{2 \gamma_ {t}} \| w ^ {t} - w ^ {*} \| ^ {2} \tag {28} \\ = \frac {- 1}{2 \gamma_ {t}} \| w ^ {t + 1} - w ^ {t} \| ^ {2} - \langle g ^ {t} + b ^ {t + 1}, w ^ {t + 1} - w ^ {*} \rangle \\ = \frac {- 1}{2 \gamma_ {t}} \| w ^ {t + 1} - w ^ {t} \| ^ {2} - \langle g ^ {t} - \nabla \ell (w ^ {t}), w ^ {t + 1} - w ^ {*} \rangle - \langle \nabla \ell (w ^ {t}) + b ^ {t + 1}, w ^ {t + 1} - w ^ {*} \rangle . \\ \end{array}
$$
We decompose the last term of (28) as
$$
- \left\langle \nabla \ell \left(w ^ {t}\right) + b ^ {t + 1}, w ^ {t + 1} - w ^ {*} \right\rangle = - \left\langle b ^ {t + 1}, w ^ {t + 1} - w ^ {*} \right\rangle - \left\langle \nabla \ell \left(w ^ {t}\right), w ^ {t + 1} - w ^ {t} \right\rangle + \left\langle \nabla \ell \left(w ^ {t}\right), w ^ {*} - w ^ {t} \right\rangle . \tag {29}
$$
For the first term in the above we can use that  $b^{t + 1} \in \partial h(w^{t + 1})$  is a subgradient together with the defining property of subgradient to write
$$
- \left\langle b ^ {t + 1}, w ^ {t + 1} - w ^ {*} \right\rangle = \left\langle b ^ {t + 1}, w ^ {*} - w ^ {t + 1} \right\rangle \leq h \left(w ^ {*}\right) - h \left(w ^ {t + 1}\right). \tag {30}
$$
On the second term we can use the fact that  $l$  is  $M$ -smooth to write
$$
- \left\langle \nabla \ell \left(w ^ {t}\right), w ^ {t + 1} - w ^ {t} \right\rangle \leq \frac {M}{2} \left\| w ^ {t + 1} - w ^ {t} \right\| ^ {2} + \ell \left(w ^ {t}\right) - \ell \left(w ^ {t + 1}\right). \tag {31}
$$
On the last term we can use the convexity of  $f$  to write
$$
\langle \nabla \ell \left(w ^ {t}\right), w ^ {*} - w ^ {t} \rangle \leq \ell \left(w ^ {*}\right) - \ell \left(w ^ {t}\right). \tag {32}
$$
By inserting (30), (31), (32) into (29) gives
$$
\begin{array}{l} - \left\langle \nabla \ell \left(w ^ {t}\right) + g ^ {t + 1}, w ^ {t + 1} - w ^ {*} \right\rangle \leq h \left(w ^ {*}\right) + \ell \left(w ^ {*}\right) - h \left(w ^ {t + 1}\right) - \ell \left(w ^ {t + 1}\right) + \frac {M}{2} \| w ^ {t + 1} - w ^ {t} \| ^ {2} \\ = f \left(w ^ {*}\right) - f \left(w ^ {t + 1}\right) + \frac {M}{2} \| w ^ {t + 1} - w ^ {t} \| ^ {2}. \\ \end{array}
$$
Now using the above in (28), and our assumption that  $\gamma_t M \leq 1$ , we obtain
$$
\begin{array}{l} \frac {1}{2 \gamma_ {t}} \| w ^ {t + 1} - w ^ {*} \| ^ {2} - \frac {1}{2 \gamma_ {t}} \| w ^ {t} - w ^ {*} \| ^ {2} \\ \leq \frac {- 1}{2 \gamma_ {t}} \| w ^ {t + 1} - w ^ {t} \| ^ {2} + \frac {L}{2} \| w ^ {t + 1} - w ^ {t} \| ^ {2} - (f (w ^ {t + 1}) - \inf  f) - \langle g ^ {t} - \nabla \ell (w ^ {t}), w ^ {t + 1} - w ^ {*} \rangle \\ \leq - \left(f \left(w ^ {t + 1}\right) - \inf  f\right) - \langle g ^ {t} - \nabla \ell \left(w ^ {t}\right), w ^ {t + 1} - w ^ {*} \rangle . \tag {33} \\ \end{array}
$$
We now have to control the last term of (33) in expectation. To shorten our notation we temporarily introduce the operators
$$
T (w) \stackrel {\text {d e f}} {=} w - \gamma_ {t} \nabla \ell (w),
$$
$$
\hat {T} (w) \stackrel {\text {d e f}} {=} w - \gamma_ {t} g ^ {t}. \tag {34}
$$
Notice in particular that  $w^{t + 1} = \mathrm{prox}_{\gamma_{th}}(\hat{T}(w^t))$ . So the last term of (33) can be decomposed as
$$
\begin{array}{l} {- \langle g ^ {t} - \nabla \ell (w ^ {t}), w ^ {t + 1} - w ^ {*} \rangle} = {- \langle g ^ {t} - \nabla \ell (w ^ {t}), \operatorname * {p r o x} _ {\gamma_ {t} h} (\hat {T} (w ^ {t})) - \operatorname * {p r o x} _ {\gamma_ {t} h} (T (w ^ {t})) \rangle} \\ - \left\langle g ^ {t} - \nabla \ell \left(w ^ {t}\right), \operatorname {p r o x} _ {\gamma_ {t} h} \left(T \left(w ^ {t}\right)\right) - w ^ {*} \right\rangle . \tag {35} \\ \end{array}
$$
We observe that the last term is, in expectation, is equal to zero. This is because  $\mathrm{prox}_{\gamma_t h}(T(w^t)) - w^*$  is deterministic when conditioned on  $w^t$ . Since we will later on take expectations, we drop this term now and keep on going. As for the first term, using the nonexpansiveness of the proximal operator (Lemma 33), we have that
$$
\begin{array}{l} - \langle g ^ {t} - \nabla \ell (w ^ {t}), \operatorname {p r o x} _ {\gamma_ {t} h} (\hat {T} (w ^ {t})) - \operatorname {p r o x} _ {\gamma_ {t} h} (T (w ^ {t})) \rangle \leq \| g ^ {t} - \nabla \ell (w ^ {t}) \| \| \hat {T} (w ^ {t}) - T (w ^ {t}) \| \\ = \gamma_ {t} \| g ^ {t} - \nabla \ell (w ^ {t}) \| ^ {2}. \\ \end{array}
$$
Using the above two bounds in (35) we have proved that (after taking expectation)
$$
- \mathbb {E} \left\langle g ^ {t} - \nabla \ell \left(w ^ {t}\right), w ^ {t + 1} - w ^ {*} \right\rangle \leq \gamma_ {t} \mathbb {E} \| g ^ {t} - \nabla \ell \left(w ^ {t}\right) \| ^ {2} = \gamma_ {t} \mathbb {V} \left[ g ^ {t} \right].
$$
Injecting the above inequality into (33) and multiplying through by  $\gamma_{t}$ , we obtain
$$
\frac {1}{2} \mathbb {E} \| w ^ {t + 1} - w ^ {*} \| ^ {2} - \frac {1}{2} \mathbb {E} \left[ \| w ^ {t} - w ^ {*} \| ^ {2} \right] \leq - \gamma_ {t} \mathbb {E} \left[ f \left(w ^ {t + 1}\right) - \inf  f \right] + \gamma_ {t} ^ {2} \mathbb {V} \left[ g ^ {t} \right]. \tag {36}
$$
From now on we assume that the stepsize sequence is constant  $\gamma_t \equiv \gamma$ . Now, applying our assumption on the gradient estimator, we have
$$
\mathbb {V} \left[ g ^ {t} \right] \leq \mathbb {E} \| g ^ {t} \| ^ {2} \leq a \mathbb {E} \left| \left| w ^ {t} - w ^ {*} \right| \right| ^ {2} + b
$$
Inject this inequality into (36), and reordering the terms gives
$$
\begin{array}{l} \gamma \mathbb {E} [ f (w ^ {t + 1}) - \inf  f ] \leq \frac {1}{2} \mathbb {E} [ \| w ^ {t} - w ^ {*} \| ^ {2} ] - \frac {1}{2} \mathbb {E} [ \| w ^ {t + 1} - w ^ {*} \| ^ {2} ] + \gamma^ {2} (a \mathbb {E} [ \| w ^ {t} - w ^ {*} \| ^ {2} ] + b) \\ = \frac {1}{2} \left(1 + 2 \gamma^ {2} a\right) \mathbb {E} \left[ \| w ^ {t} - w ^ {*} \| ^ {2} \right] - \frac {1}{2} \mathbb {E} \left[ \| w ^ {t + 1} - w ^ {*} \| ^ {2} \right] + \gamma^ {2} b \\ = \frac {1}{2} C \mathbb {E} \left[ \| w ^ {t} - w ^ {*} \| ^ {2} \right] - \frac {1}{2} \mathbb {E} \left[ \| w ^ {t + 1} - w ^ {*} \| ^ {2} \right] + \gamma^ {2} b \tag {37} \\ \end{array}
$$
where we introduced the constant  $C \stackrel{\mathrm{def}}{=} 1 + 2a\gamma^2$ . We will now do a weighted telescoping by introducing a sequence  $\alpha_{t+1} > 0$  of weights. Multiplying the above inequality by some  $\alpha_{t+1} > 0$ , and summing over  $0, \ldots, T-1$  gives
$$
\gamma \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1} \mathbb {E} [ f (w ^ {t + 1}) - \inf  f ] \leq \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1} \left(C \frac {1}{2} \mathbb {E} [ \| w ^ {t} - w ^ {*} \| ^ {2} ] - \frac {1}{2} \mathbb {E} [ \| w ^ {t + 1} - w ^ {*} \| ^ {2} ]\right) + \gamma^ {2} b \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}.
$$
To be able to use a telescopic sum in the right-hand term, we need that  $\alpha_{t + 1}C = \alpha_t$  (an idea we borrowed from [54]). This holds by choosing  $\alpha_{t}\stackrel {\mathrm{def}}{=}\frac{1}{C^{t}}$ , which allows us to write
$$
\begin{array}{l} \gamma \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1} \mathbb {E} [ f (w ^ {t + 1}) - \inf  f ] \leq \sum_ {t = 0} ^ {T - 1} \left(\frac {\alpha_ {t}}{2} \mathbb {E} [ \| w ^ {t} - w ^ {*} \| ^ {2} ] - \frac {\alpha_ {t + 1}}{2} \mathbb {E} [ \| w ^ {t + 1} - w ^ {*} \| ^ {2} ]\right) + \gamma^ {2} b \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1} \\ = \frac {\alpha_ {0}}{2} \| w ^ {0} - w ^ {*} \| ^ {2} - \frac {\alpha_ {T}}{2} \mathbb {E} \left[ \| w ^ {T} - w ^ {*} \| ^ {2} \right] + \gamma^ {2} b \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1} \\ \leq \frac {\alpha_ {0}}{2} \| w ^ {0} - w ^ {*} \| ^ {2} + \gamma^ {2} b \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}. \\ \end{array}
$$
Let us now introduce  $\bar{w}^T \stackrel{\mathrm{def}}{=} \frac{\sum_{t=0}^{T-1} \alpha_{t+1} w^{t+1}}{\sum_{t=0}^{T-1} \alpha_{t+1}}$ , using that  $\alpha_0 = 1$ , and by using the convexity of  $f$  with Jensen's inequality, we obtain
$$
\begin{array}{l} \mathbb {E} \left[ f (\bar {w} ^ {T}) \right] - \inf  f = \mathbb {E} \left[ \frac {\sum_ {t = 0} ^ {T - 1} \alpha_ {t + t} f (w ^ {t + 1})}{\sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} \right] - \inf  f \\ \leq \frac {\sum_ {t = 0} ^ {T - 1} \alpha_ {t + t} \mathbb {E} [ f (w ^ {t + 1}) ]}{\sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} - \inf  f \\ = \frac {\sum_ {t = 0} ^ {T - 1} \alpha_ {t + t} \mathbb {E} [ f (w ^ {t + 1}) - \inf  f ]}{\sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} \\ = \frac {\sum_ {t = 0} ^ {T - 1} \alpha_ {t + t} \mathbb {E} [ f (w ^ {t + 1}) - \inf  f ]}{\sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} \\ \leq \frac {\frac {1}{2 \gamma} \| w ^ {0} - w ^ {*} \| ^ {2} + \gamma b \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}}{\sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} \\ = \frac {1}{2 \gamma \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} \| w ^ {0} - w ^ {*} \| ^ {2} + \gamma b \\ = \frac {1}{2 \gamma \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} \| w ^ {0} - w ^ {*} \| ^ {2} + \gamma b \\ = \frac {1}{2 \gamma \sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1}} \| w ^ {0} - w ^ {*} \| ^ {2} + \gamma b. \\ \end{array}
$$
To finish the proof, it remains to compute the geometric series  $\sum_{t=0}^{T-1} \alpha_{t+1}$  which is given by
$$
\sum_ {t = 0} ^ {T - 1} \alpha_ {t + 1} = \frac {1}{C} \sum_ {t = 0} ^ {T - 1} \frac {1}{C ^ {t}} = \frac {1}{C} \frac {1 - 1 / C ^ {T}}{1 - 1 / C} = \frac {1 - 1 / C ^ {T}}{C - 1}.
$$
Using the above and substituting for  $C = 1 + 2a\gamma^2$  and  $\theta = 1 / C$  gives
$$
\begin{array}{l} \mathbb {E} \left[ f (\bar {w} ^ {T}) \right] - \inf  f \leq \frac {C - 1}{2 \gamma (1 - 1 / C ^ {T})} \| w ^ {0} - w ^ {*} \| ^ {2} + \gamma b \\ = \frac {a \gamma}{\left(1 - \theta^ {T}\right)} \| w ^ {0} - w ^ {*} \| ^ {2} + \gamma b \\ \end{array}
$$
Finally by applying Lemma 32 with where  $B = 2a$ ,  $A = \sqrt{\frac{2}{B}} = \frac{1}{\sqrt{a}}$  and thus  $\gamma = \frac{A}{\sqrt{T}} = \frac{1}{\sqrt{aT}}$ . Note that with this choice of  $\gamma$  we have that
$$
\gamma = \frac {1}{\sqrt {a T}} \leq \frac {1}{M} \Leftrightarrow T \geq \frac {M ^ {2}}{a}.
$$
Thus for  $T \geq 2$  the result of Lemma 32 gives
$$
\mathbb {E} \left[ f (\bar {w} ^ {T}) \right] - \inf  f \leq \frac {1}{\sqrt {a} \sqrt {T}} \left(2 a \| w ^ {0} - w ^ {*} \| ^ {2} + b\right) = \Omega \left(\frac {1}{\sqrt {T}}\right).
$$
9.5 Projected gradient descent with strong convexity
Proof. Plugging in one step of Proj-SGD we have that
$$
\begin{array}{l} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} = \left\| \operatorname {p r o j} _ {\mathcal {W}} \left(w ^ {t} - \gamma_ {t} g ^ {t}\right) - \operatorname {p r o j} _ {\mathcal {W}} w ^ {*} \right\| ^ {2} \\ \leq \left\| w ^ {t} - w ^ {*} - \gamma_ {t} g ^ {t} \right\| ^ {2}, \\ \end{array}
$$
where we used that projections are non-expansive. Taking expectation conditioned on  $w^t$ , and expanding squares we have that
$$
\begin{array}{l} \mathbb {E} \left[ \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} \mid w ^ {t} \right] \leq \left\| w ^ {t} - w ^ {*} \right\| ^ {2} - 2 \gamma_ {t} \left\langle \nabla f (w ^ {t}), w ^ {t} - w ^ {*} \right\rangle + \gamma_ {t} ^ {2} \mathbb {E} \left[ \left\| g ^ {t} \right\| ^ {2} \mid w ^ {t} \right] \\ \leq \left(1 - \mu \gamma_ {t}\right) \left\| w ^ {t} - w ^ {*} \right\| ^ {2} - 2 \gamma_ {t} \left(f \left(w ^ {t}\right) - f \left(w ^ {*}\right)\right) + a \gamma_ {t} ^ {2} \left\| w ^ {t} - w ^ {*} \right\| ^ {2} + b \gamma_ {t} ^ {2}, \\ \end{array}
$$
where we used  $\mathbb{E}[g^t] = \nabla f(w^t)$  and definition 5, and that  $f$  is  $\mu$ -strongly convex. Taking full expectation and re-arranging gives
$$
2 \gamma_ {t} \mathbb {E} \left[ f \left(w ^ {t}\right) - f \left(w ^ {*}\right) \right] \leq \left(1 - \gamma_ {t} \left(\mu - a \gamma_ {t}\right)\right) \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| ^ {2} - \mathbb {E} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} + b \gamma_ {t} ^ {2}.
$$
Now using that  $\gamma_t \leq \frac{\mu}{2a}$  we have that
$$
1 - \gamma_ {t} (\mu - a \gamma_ {t}) \leq 1 - \frac {\gamma_ {t} \mu}{2}
$$
$$
2 \gamma_ {t} \mathbb {E} \left[ f \left(w ^ {t}\right) - f \left(w ^ {*}\right) \right] \leq \left(1 - \frac {\mu \gamma_ {t}}{2}\right) \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| ^ {2} - \mathbb {E} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} + b \gamma_ {t} ^ {2}. \tag {38}
$$
Re-arranging we have that
$$
\begin{array}{l} \mathbb {E} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} \leq \left(1 - \frac {\mu \gamma_ {t}}{2}\right) \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| ^ {2} - 2 \gamma_ {t} \mathbb {E} [ f (w ^ {t}) - f (w ^ {*}) ] + b \gamma_ {t} ^ {2} \\ \leq \left(1 - \frac {\mu \gamma_ {t}}{2}\right) \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| ^ {2} + b \gamma_ {t} ^ {2}. \tag {39} \\ \end{array}
$$
Using  $\gamma_t \equiv \gamma$  constant, and since  $\gamma \leq \frac{2}{\mu} \Leftrightarrow 1 - \frac{\mu\gamma}{2} \geq 0$  we have by unrolling the recurrence that
$$
\mathbb {E} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} \leq \left(1 - \frac {\mu \gamma}{2}\right) ^ {t + 1} \left\| w ^ {0} - w ^ {*} \right\| ^ {2} + \sum_ {k = 0} ^ {t} \left(1 - \frac {\mu \gamma}{2}\right) ^ {k} b \gamma^ {2}.
$$
$$
\sum_ {k = 0} ^ {t - 1} \left(1 - \frac {\gamma \mu}{2}\right) ^ {t} \gamma^ {2} = 2 \gamma^ {2} \frac {1 - (1 - \gamma \mu / 2) ^ {t}}{\gamma \mu} \leq \frac {2 \gamma^ {2}}{\gamma \mu} = \frac {2 \gamma}{\mu},
$$
$$
\mathbb {E} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} \leq \left(1 - \frac {\mu \gamma}{2}\right) ^ {t + 1} \left\| w ^ {0} - w ^ {*} \right\| ^ {2} + \frac {2 \gamma b}{\mu},
$$
which concludes the proof of (13).
To prove (14) we will apply Theorem 31, for which we will switch our notation to match that of Theorem 31. That is, substituting  $\mathcal{L} = \frac{a}{\mu}$ ,  $\sigma^2 = \frac{b}{2}$  and  $\hat{\mu} = \frac{\mu}{2}$  into (39) gives
$$
\mathbb {E} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} \leq (1 - \hat {\mu} \gamma_ {t}) \mathbb {E} \left\| w ^ {t} - w ^ {*} \right\| ^ {2} + 2 \sigma^ {2} \gamma_ {t} ^ {2}, \tag {40}
$$
which now holds for  $\gamma_t \leq \frac{1}{2\mathcal{L}}$  and fits exactly the format of (23), excluding the additional hat on  $\mu$ . Thus we know from following verbatim the proof that continues after (23) that for a stepsize schedule of
$$
\gamma_ {t} = \left\{ \begin{array}{l l} \frac {1}{2 \mathcal {L}} & \text {f o r} t <   t ^ {*} \\ \frac {1}{\hat {\mu}} \frac {2 t + 1}{(t + 1) ^ {2}} & \text {f o r} t \geq t ^ {*} \end{array} \right.
$$
for  $t^* = 4\lfloor \mathcal{L} / \hat{\mu}\rfloor$  we have that
$$
\mathbb {E} \left[ \| w ^ {T + 1} - w ^ {*} \| ^ {2} \right] \leq \frac {1 6 \lfloor \mathcal {L} / \hat {\mu} \rfloor^ {2}}{(T + 1) ^ {2}} \| w ^ {0} - w ^ {*} \| ^ {2} + \frac {\sigma^ {2}}{\hat {\mu} ^ {2}} \frac {8}{T + 1}.
$$
After switching back notation  $\mathcal{L} = \frac{a}{\mu}$ ,  $\sigma^2 = \frac{b}{2}$  and  $\hat{\mu} = \frac{\mu}{2}$  gives (14).
9.6 Projected gradient descent with convexity
Proof. Plugging in one step of the algorithm, we have that
$$
\begin{array}{l} \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} = \left\| \operatorname {p r o j} _ {\mathcal {W}} \left(w ^ {t} - \gamma_ {t} g ^ {t}\right) - \operatorname {p r o j} _ {\mathcal {W}} \left(w ^ {*}\right) \right\| ^ {2} \\ \leq \left\| w ^ {t} - w ^ {*} - \gamma_ {t} g ^ {t} \right\| ^ {2}, \\ \end{array}
$$
$$
\begin{array}{l} \mathbb {E} \left[ \left\| w ^ {t + 1} - w ^ {*} \right\| ^ {2} \mid w ^ {t} \right] \leq \left\| w ^ {t} - w ^ {*} \right\| ^ {2} - 2 \gamma_ {t} \left\langle \nabla f (w ^ {t}), w ^ {t} - w ^ {*} \right\rangle + \gamma_ {t} ^ {2} \mathbb {E} \left[ \left\| g ^ {t} \right\| ^ {2} \mid w ^ {t} \right] \\ \leq \left\| w ^ {t} - w ^ {*} \right\| ^ {2} - 2 \gamma_ {t} \left(f \left(w ^ {t}\right) - f \left(w ^ {*}\right)\right) + a \gamma_ {t} ^ {2} \left\| w ^ {t} - w ^ {*} \right\| ^ {2} + b \gamma_ {t} ^ {2}, \\ \end{array}
$$
where we used that  $g^{t}$  is an unbiased estimator with bounded squared norm. Taking full expectation and re-arranging gives