SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} _ {G} [ \| g (x) \| ^ {2} ] = \frac {d ^ {2}}{4 \ell^ {2} h ^ {2}} \sum_ {i = 1} ^ {\ell} \mathbb {E} _ {v ^ {(i)}} [ (f (x + h v ^ {(i)}) - f (x - h v ^ {(i)}) - f (x) + f (x)) ^ {2} \| v ^ {(i)} \| ^ {2} ] \\ \leq \frac {d ^ {2}}{2 \ell^ {2} h ^ {2}} \sum_ {i = 1} ^ {\ell} \left[ \mathbb {E} _ {v ^ {(i)}} [ (f (x + h v ^ {(i)}) - f (x)) ^ {2} \| v ^ {(i)} \| ^ {2} ] \right. \\ \left. + \mathbb {E} _ {v ^ {(i)}} \big [ (f (x - h v ^ {(i)}) - f (x)) ^ {2} \| v ^ {(i)} \| ^ {2} \big ] \right] \\ = \frac {d ^ {2}}{\ell^ {2} h ^ {2}} \sum_ {i = 1} ^ {\ell} \mathbb {E} _ {v ^ {(i)}} [ (f (x + h v ^ {(i)}) - f (x)) ^ {2} ], \\ \end{array}
$$
where the last equation follows by symmetry. Adding and subtracting  $\langle \nabla f(x), h v^{(i)} \rangle$  we derive
$$
\begin{array}{l} \mathbb {E} _ {G} [ \| g (x) \| ^ {2} ] \leq \frac {d ^ {2}}{\ell^ {2} h ^ {2}} \sum_ {i = 1} ^ {\ell} \mathbb {E} _ {v ^ {(i)}} \left[ \left(f (x + h v ^ {(i)}) - f (x) - \left\langle \nabla f (x), h v ^ {(i)} \right\rangle + \left\langle \nabla f (x), h v ^ {(i)} \right\rangle\right) ^ {2} \right] \\ \leq \frac {2 d ^ {2}}{\ell^ {2} h ^ {2}} \sum_ {i = 1} ^ {\ell} \left(\mathbb {E} _ {v ^ {(i)}} \left[ \left(f (x + h v ^ {(i)}) - f (x) - \left\langle \nabla f (x), h v ^ {(i)} \right\rangle\right) ^ {2} \right] \right. \\ \left. + \mathbb {E} _ {v ^ {(i)}} \left[ \left(\left\langle \nabla f (x), h v ^ {(i)} \right\rangle\right) ^ {2} \right]\right). \\ \end{array}
$$
Denote by  $\beta (\mathbb{S}^{d - 1})$  the surface area of  $\mathbb{S}^{d - 1}$ . The Descent Lemma [41] implies
$$
\begin{array}{l} \mathbb {E} _ {G} [ \| g (x) \| ^ {2} ] \leq \frac {2 d ^ {2}}{\ell^ {2} h ^ {2}} \sum_ {i = 1} ^ {\ell} \left[ \left(\frac {L _ {1} ^ {2}}{4} h ^ {4}\right) + \mathbb {E} \left[ \left(\left\langle \nabla f (x), h v ^ {(i)} \right\rangle\right) ^ {2} \right] \right] \\ = \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h ^ {2} + \frac {2 d ^ {2}}{\ell^ {2} h ^ {2}} \sum_ {i = 1} ^ {\ell} \mathbb {E} \left[ \left(\left\langle \nabla f (x), h v ^ {(i)} \right\rangle\right) ^ {2} \right] \\ = \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h ^ {2} + \frac {2 d ^ {2}}{\ell^ {2} \beta (\mathbb {S} ^ {d - 1})} \sum_ {i = 1} ^ {\ell} \int_ {\mathbb {S} ^ {d - 1}} \nabla f (x) ^ {\intercal} v ^ {(i)} v ^ {(i) \intercal} \nabla f (x) d \sigma (v). \\ \end{array}
$$
By Lemma 2, we get the claim. Indeed,
$$
\begin{array}{l} \mathbb {E} _ {G} [ \| g (x) \| ^ {2} ] \leq \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h ^ {2} + \frac {2 d ^ {2}}{\ell^ {2} \beta (\mathbb {S} ^ {d - 1})} \sum_ {i = 1} ^ {\ell} \left(\frac {\beta (\mathbb {S} ^ {d - 1})}{d} \| \nabla f (x) \| ^ {2}\right) \\ = \frac {2 d}{\ell} \| \nabla f (x) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h ^ {2}. \\ \end{array}
$$
A.2 Auxiliary results and proofs for the nonsmooth setting, convex, and nonconvex.
In this subsection, for every  $k$ , we will denote by  $\mathcal{F}_k$  the  $\sigma$ -algebra  $\sigma(G_0, \ldots, G_{k-1})$ .
Lemma 5. Let  $f: \mathbb{R}^d \to \mathbb{R}$  be a lower semi-continuous function and denote with  $S = \arg \min f$  and  $f^* = \min f$ . Then,
$$
\left\{ \begin{array}{l l} (A) & \forall x ^ {*} \in S,   \exists \lim  _ {k} \| x _ {k} - x ^ {*} \| \\ (B) & \lim  _ {k} \inf  f (x _ {k}) = f ^ {*} \end{array} \right. \Rightarrow \exists x _ {\infty} \in S \quad s. t. \quad x _ {k} \to x _ {\infty}.
$$
Proof. Since (B) holds, we have that exists  $(x_{k_j})_{j\in \mathbb{N}}$  subsequence of  $(x_{k})_{k\in \mathbb{N}}$  such that  $f(x_{k_j})\to f^*$ . Since  $S\neq \emptyset$  and (A) we have that
$$
\exists x ^ {*} \in S \quad \text {a n d} \quad \exists \lim  _ {k} \| x _ {k} - x ^ {*} \|
$$
Thus, the sequence  $(x_{k})_{k\in \mathbb{N}}$  is bounded and, therefore, also  $(x_{k_j})_{j\in \mathbb{N}}$  is bounded. Taking a convergent subsequence  $(x_{k_{j_n}})_{n\in \mathbb{N}}$  of  $(x_{k_j})_{j\in \mathbb{N}}$ , we have that exists  $x_{\infty}$  s.t.
$$
x _ {k _ {j _ {n}}} \to x _ {\infty}.
$$
Since  $f$  is assumed to be lower semi-continuous, we have that
$$
f (x _ {\infty}) \leq \lim  _ {n} \inf  f \left(x _ {k _ {j _ {n}}}\right) = f ^ {*} = \lim  _ {j} f \left(x _ {k _ {j}}\right).
$$
Thus, we have that  $x_{\infty}\in S$  which implies, by (A), that
$$
\exists \lim  _ {k} \| x _ {k} - x _ {\infty} \| \quad \text {a n d} \quad \lim  _ {n} \| x _ {k _ {j _ {n}}} - x _ {\infty} \| = 0.
$$
Hence, since  $x_{k_{j_n}}$  is a subsequence of  $x_{k}$
$$
\lim  _ {k} \| x _ {k} - x _ {\infty} \| = 0,
$$
and, therefore,  $x_{k}\rightarrow x_{\infty}\in S$
Lemma 6 (Convergence: convex non-smooth). Assume that  $f$  is convex and  $L_0$  Lipschitz continuous. Let  $(x_k)_{k \in \mathbb{N}}$  be the sequence generated by Algorithm 1 and let  $x^* \in \arg \min f$ . Then, for every  $k \in \mathbb{N}$ , the following inequality holds:
$$
\mathbb {E} [ \| x _ {k + 1} - x ^ {*} \| ^ {2} | \mathcal {F} _ {k} ] - \| x _ {k} - x ^ {*} \| ^ {2} + 2 \alpha_ {k} (f (x _ {k}) - f (x ^ {*})) \leq 2 c \frac {L _ {0} ^ {2} d}{\ell} \alpha_ {k} ^ {2} + 2 L _ {0} \alpha_ {k} h _ {k},
$$
where  $c$  is some non-negative constant independent from the dimension. Moreover, if the stepsizes satisfy Assumption 2, we have
$$
\lim  _ {k \rightarrow + \infty} f (x _ {k}) = f (x ^ {*}) \quad a. s,
$$
and there exists a random variable  $\hat{x}$  taking values in  $n$  arg min  $f$  such that  $x_{k}\rightarrow \hat{x}$  a.s.
Proof. Let  $k \in \mathbb{N}$ . By Algorithm 1,
$$
\left\| x _ {k + 1} - x ^ {*} \right\| ^ {2} - \left\| x _ {k} - x ^ {*} \right\| ^ {2} = \alpha_ {k} ^ {2} \| g _ {k} \| ^ {2} - 2 \alpha_ {k} \left\langle g _ {k}, x _ {k} - x ^ {*} \right\rangle . \tag {6}
$$
Since  $f_{h_k}$  is convex by Proposition 1 and  $\mathbb{E}[g_k|\mathcal{F}_k] = \nabla f_{h_k}(x_k)$  (see Lemma 1), we have
$$
- \left\langle \nabla f _ {h _ {k}} (x _ {k}), x _ {k} - x ^ {*} \right\rangle \leq f _ {h _ {k}} \left(x ^ {*}\right) - f _ {h _ {k}} (x _ {k}).
$$
Thus, taking the conditional expectation with respect to  $\mathcal{F}_k$ , by Lemma 4, we get,
$$
\mathbb{E}[\| x_{k + 1} - x^{*}\|^{2}|\mathcal{F}_{k}] - \| x_{k} - x^{*}\|^{2}\leq \underbrace{2c\frac{L_{0}^{2}d}{\ell}\alpha_{k}^{2}}_{=: C_{k}} - 2\alpha_{k}(f_{h_{k}}(x_{k}) - f_{h_{k}}(x^{*})).
$$
Then, by Proposition 1,
$$
\mathbb {E} [ \| x _ {k + 1} - x ^ {*} \| ^ {2} | \mathcal {F} _ {k} ] - \| x _ {k} - x ^ {*} \| ^ {2} \leq C _ {k} - 2 \alpha_ {k} (f (x _ {k}) - f (x ^ {*})) + 2 L _ {0} \alpha_ {k} h _ {k}.
$$
Next suppose that Assumption 2 holds. Rearranging the terms,
$$
\mathbb {E} \left[ \| x _ {k + 1} - x ^ {*} \| ^ {2} \mid \mathcal {F} _ {k} \right] - \| x _ {k} - x ^ {*} \| ^ {2} + 2 \alpha_ {k} \left(f (x _ {k}) - f (x ^ {*})\right) \leq C _ {k} + 2 L _ {0} \alpha_ {k} h _ {k},
$$
with  $C_k \in \ell^1$  and  $\alpha_k h_k \in \ell^1$ . Therefore, Robbins-Siegmund Theorem [43] implies that  $(\| x_k - x^*\|)_{k \in \mathbb{N}}$  is a.s. convergent,  $\alpha_k (f(x_k) - f(x^*)) \in \ell^1$  a.s. and thus, since  $\alpha_k \notin \ell^1$ ,
$$
\lim  _ {k \rightarrow \infty} \inf  f (x _ {k}) = f \left(x ^ {*}\right) \quad \text {a . s .} \tag {7}
$$
We derive from [32, Lemma 9.9] and Lemma 5 that there exists a random variable  $\hat{x}$  taking values in  $\arg \min f$  such that  $x_{k} \to \hat{x}$  a.s. Finally, continuity of  $f$  yields that  $\lim_{k} f(x_{k}) = f(x_{*})$  a.s.
In the next Lemma, to derive bounds on function values, we study the sequence  $(f_{h_k}(x_{k + 1}) - f_{h_k}(x_k))_{k\in \mathbb{N}}$ . It is the difference between the smoothed function at iteration  $k$  evaluated at  $x_{k}$  and at  $x_{k + 1}$ . It corresponds to the function value decrease between the iterations  $k + 1$  and  $k$  if  $h_k$  is constant.
Lemma 7 (Function Value decrease: nonconvex non-smooth setting). Under Assumption 1, let  $(x_{k})_{k\in \mathbb{N}}$  be the sequence generated by Algorithm 1. Then,
$$
\mathbb {E} \left[ f _ {h _ {k}} \left(x _ {k + 1}\right) \mid \mathcal {F} _ {k} \right] - f _ {h _ {k}} \left(x _ {k}\right) \leq - \alpha_ {k} \| \nabla f _ {h _ {k}} \left(x _ {k}\right) \| ^ {2} + c \frac {L _ {0} ^ {3} d \sqrt {d}}{\ell} \frac {\alpha_ {k} ^ {2}}{h _ {k}},
$$
Proof. By Lemma 1, we have that  $f_{h_k}$  is  $L_0\sqrt{d} / h_k$ -smooth. Thus, by the Descent Lemma [41],
$$
f _ {h _ {k}} (x _ {k + 1}) - f _ {h _ {k}} (x _ {k}) \leq - \alpha_ {k} \left\langle \nabla f _ {h _ {k}} (x _ {k}), g _ {k} \right\rangle + \frac {L _ {0} \sqrt {d}}{2 h _ {k}} \alpha_ {k} ^ {2} \| g _ {k} \| ^ {2}.
$$
Taking the conditional expectation with respect to  $\mathcal{F}_k$
$$
\mathbb {E} \left[ f _ {h _ {k}} \left(x _ {k + 1}\right) \mid \mathcal {F} _ {k} \right] - f _ {h _ {k}} \left(x _ {k}\right) \leq - \alpha_ {k} \| \nabla f _ {h _ {k}} \left(x _ {k}\right) \| ^ {2} + \frac {L _ {0} \sqrt {d}}{2 h _ {k}} \alpha_ {k} ^ {2} \mathbb {E} [ \| g _ {k} \| ^ {2} \mid \mathcal {F} _ {k} ]. \tag {8}
$$
The claim follows from Lemma 4.
A.3 Auxiliary results for smooth setting.
Lemma 8 (Function value decrease: convex smooth setting). Under Assumption 3, let  $(x_{k})_{k\in \mathbb{N}}$  be the sequence generated by Algorithm 1. Then the following holds:
$$
\mathbb {E} [ f (x _ {k + 1}) | \mathcal {F} _ {k} ] - f (x _ {k}) \leq - \alpha_ {k} \left(\frac {1}{2} - \frac {L _ {1} d}{\ell} \alpha_ {k}\right) \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2} \alpha_ {k} h _ {k} ^ {2}}{8} + \frac {L _ {1} ^ {3} d ^ {2}}{4 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2}.
$$
Proof. By the Descent Lemma [41] and Algorithm 1,
$$
f (x _ {k + 1}) - f (x _ {k}) \leq - \alpha_ {k} \left\langle \nabla f (x _ {k}), g _ {k} \right\rangle + \frac {L _ {1}}{2} \alpha_ {k} ^ {2} \| g _ {k} \| ^ {2}.
$$
Taking the conditional expectation and by Lemma 4,
$$
\mathbb {E} [ f (x _ {k + 1}) | \mathcal {F} _ {k} ] - f (x _ {k}) \leq - \alpha_ {k} \left\langle \nabla f (x _ {k}), \nabla f _ {h _ {k}} (x _ {k}) \right\rangle + \frac {L _ {1}}{2} \alpha_ {k} ^ {2} \left[ \frac {2 d}{\ell} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h _ {k} ^ {2} \right].
$$
Adding and subtracting  $\nabla f(x_{k})$
$$
\begin{array}{l} \mathbb {E} [ f (x _ {k + 1}) | \mathcal {F} _ {k} ] - f (x _ {k}) \leq - \alpha_ {k} \left\langle \nabla f (x _ {k}), \nabla f _ {h _ {k}} (x _ {k}) - \nabla f (x _ {k}) \right\rangle - \alpha_ {k} \| \nabla f (x _ {k}) \| ^ {2} \\ + \frac {L _ {1}}{2} \alpha_ {k} ^ {2} \left[ \frac {2 d}{\ell} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h _ {k} ^ {2} \right]. \\ \end{array}
$$
By Cauchy-Schwarz inequality and Proposition 1,
$$
\begin{array}{l} \mathbb {E} [ f (x _ {k + 1}) | \mathcal {F} _ {k} ] - f (x _ {k}) \leq \alpha_ {k} \left(\frac {L _ {1} d}{2} h _ {k}\right) \| \nabla f (x _ {k}) \| - \alpha_ {k} \| \nabla f (x _ {k}) \| ^ {2} \\ + \frac {L _ {1}}{2} \alpha_ {k} ^ {2} \left[ \frac {2 d}{\ell} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h _ {k} ^ {2} \right]. \\ \end{array}
$$
By Young's inequality,
$$
\begin{array}{l} \mathbb {E} [ f (x _ {k + 1}) | \mathcal {F} _ {k} ] - f (x _ {k}) \leq \frac {L _ {1} ^ {2} d ^ {2} \alpha_ {k} h _ {k} ^ {2}}{8} + \frac {\alpha_ {k}}{2} \| \nabla f (x _ {k}) \| ^ {2} - \alpha_ {k} \| \nabla f (x _ {k}) \| ^ {2} \\ + \frac {L _ {1}}{2} \alpha_ {k} ^ {2} \left[ \frac {2 d}{\ell} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} h _ {k} ^ {2} \right]. \\ = - \alpha_ {k} \left(\frac {1}{2} - \frac {L _ {1} d}{\ell} \alpha_ {k}\right) \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2} \alpha_ {k} h _ {k} ^ {2}}{8} + \frac {L _ {1} ^ {3} d ^ {2}}{4 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2}. \\ \end{array}
$$
Lemma 9 (Convergence in smooth setting). Let  $(x_{k})_{k\in \mathbb{N}}$  be the sequence generated by Algorithm 1 and let  $x^{*}\in \operatorname *{argmin}_{x\in \mathbb{R}^{d}}f(x)$  . Then, under Assumption 3, the following inequality holds
$$
\begin{array}{l} \mathbb {E} [ \| x _ {k + 1} - x ^ {*} \| ^ {2} | \mathcal {F} _ {k} ] - \| x _ {k} - x ^ {*} \| ^ {2} \leq \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} \\ + L _ {1} d \alpha_ {k} h _ {k} \| x _ {k} - x ^ {*} \| - 2 \alpha_ {k} \left\langle \nabla f (x _ {k}), x _ {k} - x ^ {*} \right\rangle . \\ \end{array}
$$
Moreover, if  $f$  is convex, Assumption 4 holds and  $\alpha_{k} \leq \bar{\alpha} < \ell / (2dL_{1})$ . Then
-  $(\alpha_{k}\| \nabla f(x_{k})\|^{2})_{k\in \mathbb{N}}\in \ell^{1}a.s.$
-  $\left(\| x_{k} - x^{*}\|\right)_{k\in \mathbb{N}}$  is a.s. convergent.
-  $(\alpha_{k}(f(x_{k}) - f(x^{*})))_{k\in \mathbb{N}}\in \ell^{1}a.s.$
- there exists a random variable  $\hat{x}$  taking values in  $\arg \min f$  such that  $x_{k} \to \hat{x}$  a.s. and  $\lim_{k \to \infty} f(x_{k}) = \min f$ .
$$
\left\| x _ {k + 1} - x ^ {*} \right\| ^ {2} - \left\| x _ {k} - x ^ {*} \right\| ^ {2} = \alpha_ {k} ^ {2} \left\| g _ {k} \right\| ^ {2} - 2 \alpha_ {k} \left\langle g _ {k}, x _ {k} - x ^ {*} \right\rangle .
$$
Taking the conditional expectation,
$$
\mathbb {E} [ \| x _ {k + 1} - x ^ {*} \| ^ {2} | \mathcal {F} _ {k} ] - \| x _ {k} - x ^ {*} \| ^ {2} = \alpha_ {k} ^ {2} \mathbb {E} [ \| g _ {k} \| ^ {2} | \mathcal {F} _ {k} ] - 2 \alpha_ {k} \left\langle \nabla f _ {h _ {k}} (x _ {k}), x _ {k} - x ^ {*} \right\rangle .
$$
For every  $k$ , set  $u_{k} = \| x_{k} - x^{*}\|$ . By Lemma 4,
$$
\mathbb {E} \left[ u _ {k + 1} ^ {2} \mid \mathcal {F} _ {k} \right] - u _ {k} ^ {2} \leq \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} - 2 \alpha_ {k} \left\langle \nabla f _ {h _ {k}} (x _ {k}), x _ {k} - x ^ {*} \right\rangle .
$$
$$
- 2 \alpha_ {k} \left\langle \nabla f _ {h _ {k}} (x _ {k}), x _ {k} - x ^ {*} \right\rangle = 2 \alpha_ {k} \left\langle \nabla f _ {h _ {k}} (x _ {k}) - \nabla f (x _ {k}), x ^ {*} - x _ {k} \right\rangle - 2 \alpha_ {k} \left\langle \nabla f (x _ {k}), x _ {k} - x ^ {*} \right\rangle .
$$
$$
\begin{array}{l} \mathbb {E} \left[ u _ {k + 1} ^ {2} \mid \mathcal {F} _ {k} \right] - u _ {k} ^ {2} \leq \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} \\ + 2 \alpha_ {k} \left\langle \nabla f _ {h _ {k}} (x _ {k}) - \nabla f (x _ {k}), x ^ {*} - x _ {k} \right\rangle - 2 \alpha_ {k} \left\langle \nabla f (x _ {k}), x _ {k} - x ^ {*} \right\rangle . \\ \end{array}
$$
By the Cauchy-Schwarz inequality,
$$
\begin{array}{l} \mathbb {E} \left[ u _ {k + 1} ^ {2} \mid \mathcal {F} _ {k} \right] - u _ {k} ^ {2} \leq \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} \\ + 2 \alpha_ {k} \| \nabla f _ {h _ {k}} (x _ {k}) - \nabla f (x _ {k}) \| u _ {k} - 2 \alpha_ {k} \left\langle \nabla f (x _ {k}), x _ {k} - x ^ {*} \right\rangle . \\ \end{array}
$$
The first claim follows from Proposition 1. By Proposition 1 and Young's inequality with parameter  $\tau_{k} = \alpha_{k}h_{k}$ , we get
$$
\begin{array}{l} \mathbb {E} \left[ u _ {k + 1} ^ {2} \mid \mathcal {F} _ {k} \right] - u _ {k} ^ {2} \leq \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} \\ + \frac {L _ {1} d}{2 \tau_ {k}} \alpha_ {k} ^ {2} h _ {k} ^ {2} + \frac {L _ {1} d \tau_ {k}}{2} u _ {k} ^ {2} - 2 \alpha_ {k} \left\langle \nabla f (x _ {k}), x _ {k} - x ^ {*} \right\rangle . \\ = \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} \tag {9} \\ + \frac {L _ {1} d}{2} \alpha_ {k} h _ {k} + \frac {L _ {1} d}{2} \alpha_ {k} h _ {k} u _ {k} ^ {2} \\ - 2 \alpha_ {k} \left\langle \nabla f (x _ {k}), x _ {k} - x ^ {*} \right\rangle . \\ \end{array}
$$
Since  $f$  is convex, by Baillon-Haddad Theorem [3], we derive that
$$
\begin{array}{l} \mathbb {E} \left[ u _ {k + 1} ^ {2} \mid \mathcal {F} _ {k} \right] - u _ {k} ^ {2} \leq - 2 \left(\frac {1}{L _ {1}} - \frac {d}{\ell} \alpha_ {k}\right) \alpha_ {k} \| \nabla f (x _ {k}) \| ^ {2} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} \\ + \frac {L _ {1} d}{2} \alpha_ {k} h _ {k} + \frac {L _ {1} d}{2} \alpha_ {k} h _ {k} u _ {k} ^ {2}. \\ \end{array}
$$
By Assumption 4,
$$
\begin{array}{l} \mathbb{E}[u_{k + 1}^{2}|\mathcal{F}_{k}] - u_{k}^{2}\leq -2\underbrace{\left(\frac{1}{L_{1}} - \frac{d}{\ell}\bar{\alpha}\right)}_{=: \Delta}\alpha_{k}\| \nabla f(x_{k})\|^{2} + \underbrace{\frac{L_{1}d}{2}\alpha_{k}h_{k}}_{=: \rho_{k}}u_{k}^{2} \\ + \underbrace {\frac {L _ {1} d}{2} \alpha_ {k} h _ {k} + \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2}} _ {=: C _ {k}}. \\ \end{array}
$$
Note that  $\Delta > 0$ . Thus, rearranging the terms
$$
\mathbb {E} \left[ u _ {k + 1} ^ {2} \mid \mathcal {F} _ {k} \right] - \left(1 + \rho_ {k}\right) u _ {k} ^ {2} + 2 \Delta \alpha_ {k} \| \nabla f (x _ {k}) \| ^ {2} \leq C _ {k}.
$$
Since  $\rho_k, C_k \in \ell^1$  by Assumption 4, Robbins-Siegmund Theorem [43] ensures that  $(u_k^2)_{k \in \mathbb{N}}$  is convergent and  $(\alpha_k \| \nabla f(x_k) \|^2)_{k \in \mathbb{N}} \in \ell^1$  a.s. Since  $f$  is convex, it follows from (9) that
$$
\mathbb {E} \left[ u _ {k + 1} ^ {2} \mid \mathcal {F} _ {k} \right] - (1 + \rho_ {k}) u _ {k} ^ {2} \leq \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} - 2 \alpha_ {k} (f (x _ {k}) - f (x ^ {*})) + C _ {k}.
$$
Robbins-Siegmund Theorem [43] implies that  $(\alpha_{k}(f(x_{k}) - f(x^{*})))_{k\in \mathbb{N}}\in \ell^{1}$  a.s. Assumption 4 implies that  $\alpha_{k}\not\in\ell^{1}$  therefore
$$
\lim  _ {k} \inf  f \left(x _ {k}\right) - f \left(x ^ {*}\right) = 0 \text {a . s .} \tag {10}
$$
By Lemma 8 and Assumption 4, we have that the sequence  $\mathbb{E}[f(x_{k + 1}) - f(x^{*})|\mathcal{F}_{k}] - (f(x_{k}) - f(x^{*}))$  is upper-bounded by a sequence in  $\ell^1$ . Thus, by Robbins-Siegmund Theorem [43],  $\lim_k(f(x_k) - f(x^*))$  exists a.s. Then, it follows from (10) that
$$
\lim  _ {k \to \infty} f (x _ {k}) = f (x ^ {*}) \quad a. s.
$$
Moreover, as we saw before,  $(\| x_k - x^*\|)_{k\in \mathbb{N}}$  is convergent a.s. for every  $x^{*}\in \arg \min f$ . Then, by Opial's Lemma [40], there exists a random variable  $\hat{x}$  taking values in  $\arg \min f$  such that  $x_{k}\to \hat{x}$  a.s.
Lemma 10 (Gradient bound: convex smooth setting). Suppose that Assumptions 3 and 4 hold, and assume  $f$  to be convex. Let  $(x_{k})_{k\in \mathbb{N}}$  be the sequence generated by Algorithm 1. Then, for every  $k\in \mathbb{N}$  and every  $x^{*}\in \arg \min f$ ,
$$
\sum_ {i = 0} ^ {k} \alpha_ {i} \mathbb {E} [ \| \nabla f (x _ {i}) \| ^ {2} ] \leq \frac {1}{2 \Delta} \Big (S _ {k} + \sum_ {i = 0} ^ {k} \rho_ {i} \sqrt {\mathbb {E} [ \| x _ {i} - x ^ {*} \| ^ {2} ]} \Big),
$$
$$
\sqrt {\mathbb {E} [ \| x _ {k} - x ^ {*} \| ^ {2} ]} \leq \sqrt {S _ {k - 1}} + \sum_ {i = 0} ^ {k} \rho_ {i},
$$
$$
\Delta := \left(\frac {1}{L _ {1}} - \frac {d}{\ell} \bar {\alpha}\right), \quad S _ {k} := \| x _ {0} - x ^ {*} \| + \sum_ {i = 0} ^ {k} C _ {i}
$$
$$
C _ {k} := \frac {L _ {1} ^ {2} d ^ {2}}{2 \ell} \alpha_ {k} ^ {2} h _ {k} ^ {2} \quad a n d \quad \rho_ {k} := L _ {1} d \alpha_ {k} h _ {k}.
$$
Proof. By Lemma 9 we derive
$$
\begin{array}{l} \mathbb {E} \left[ \| x _ {k + 1} - x ^ {*} \| ^ {2} | \mathcal {F} _ {k} \right] - \| x _ {k} - x ^ {*} \| ^ {2} \leq \frac {2 d}{\ell} \alpha_ {k} ^ {2} \| \nabla f (x _ {k}) \| ^ {2} + C _ {k} \\ + \rho_ {k} \| x _ {k} - x ^ {*} \| - 2 \alpha_ {k} \left\langle \nabla f (x _ {k}), x _ {k} - x ^ {*} \right\rangle . \\ \end{array}
$$
By Baillon-Haddad Theorem and Assumption 4,
$$
\mathbb {E} [ \| x _ {k + 1} - x ^ {*} \| ^ {2} | \mathcal {F} _ {k} ] - \| x _ {k} - x ^ {*} \| ^ {2} \leq - 2 \Delta \alpha_ {k} \| \nabla f (x _ {k}) \| ^ {2} + C _ {k} + \rho_ {k} \| x _ {k} - x ^ {*} \|.
$$
Let  $u_{k} \coloneqq \sqrt{\mathbb{E}\left[\|x_{k} - x^{*}\|^{2}\right]}$ . Taking the full expectation, by Jensen inequality we have
$$
u _ {k + 1} ^ {2} - u _ {k} ^ {2} \leq - 2 \Delta \alpha_ {k} \mathbb {E} [ \| \nabla f (x _ {k}) \| ^ {2} ] + \rho_ {k} u _ {k} + C _ {k}.
$$
Summing the previous inequality from  $i = 0,\dots ,k$  , we get
$$
u _ {k + 1} ^ {2} + 2 \Delta \sum_ {i = 0} ^ {k} \alpha_ {i} \mathbb {E} [ \| \nabla f (x _ {i}) \| ^ {2} ] \leq \underbrace {u _ {0} ^ {2} + \sum_ {i = 0} ^ {k} C _ {i}} _ {=: S _ {k}} + \sum_ {i = 0} ^ {k} \rho_ {i} u _ {i}. \tag {11}
$$
Since  $u_{k}$  is non-negative, the first claim of the lemma follows. Since  $\Delta > 0$ ,  $\rho_{k} \geq 0$ ,  $S_{k}$  is non-decreasing, and  $S_{k} \geq u_{0}^{2}$  in (11), then
$$
u _ {k + 1} ^ {2} \leq S _ {k} + \sum_ {i = 0} ^ {k} \rho_ {i} u _ {i}.
$$
Thus, the (discrete) Bihari's Lemma [32, Lemma 9.8] yields
$$
u _ {k + 1} \leq \frac {1}{2} \sum_ {i = 0} ^ {k} \rho_ {i} + \left[ S _ {k} + \left(\frac {1}{2} \sum_ {i = 0} ^ {k} \rho_ {i}\right) ^ {2} \right] ^ {1 / 2} \leq \sqrt {S _ {k}} + \sum_ {i = 0} ^ {k} \rho_ {i},
$$
concluding the proof.
B Proofs of Main Results
By Lemma 6,
$$
\mathbb {E} [ \| x _ {k + 1} - x ^ {*} \| ^ {2} | \mathcal {F} _ {k} ] - \| x _ {k} - x ^ {*} \| ^ {2} + 2 \alpha_ {k} (f (x _ {k}) - f (x ^ {*})) \leq 2 c \frac {L _ {0} ^ {2} d}{\ell} \alpha_ {k} ^ {2} + 2 L _ {0} \alpha_ {k} h _ {k}.
$$
Rearranging the terms, taking the full expectation, and summing the first  $k$  iterations
$$
\sum_ {i = 0} ^ {k} \alpha_ {i} \mathbb {E} [ (f (x _ {i}) - f (x ^ {*})) ] \leq \frac {\| x _ {0} - x ^ {*} \| ^ {2}}{2} + c \frac {d L _ {0} ^ {2}}{\ell} \sum_ {i = 0} ^ {k} \alpha_ {i} ^ {2} + L _ {0} \sum_ {i = 0} ^ {k} \alpha_ {i} h _ {i}.
$$
Let  $\bar{x}_k \coloneqq \sum_{i=0}^k \alpha_i x_i / (\sum_{i=0}^k \alpha_i)$ . Dividing by  $\sum_{i=0}^k \alpha_i$  and observing that by convexity we have
$$
\mathbb {E} [ f (\bar {x} _ {k}) - \min  f ] \leq \frac {\sum_ {i = 0} ^ {k} \alpha_ {i} \mathbb {E} [ (f (x _ {i}) - f (x ^ {*})) ]}{\sum_ {i = 0} ^ {k} \alpha_ {i}},
$$
we get the first claim. Under Assumption 2, the second claim holds by Lemma 6.
B.2 Proof of Corollary 1
By Theorem 1,
$$
\mathbb {E} [ f (\bar {x} _ {k}) - \min f ] \leq \frac {1}{\sum_ {i = 0} ^ {k} \alpha_ {i}} \bigg (\frac {\| x _ {0} - x ^ {*} \| ^ {2}}{2} + c \frac {d L _ {0} ^ {2}}{\ell} \sum_ {i = 0} ^ {k} \alpha_ {i} ^ {2} + L _ {0} \sum_ {i = 0} ^ {k} \alpha_ {i} h _ {i} \bigg).
$$
Replacing  $\alpha_{k}$  and  $h_k$  with the sequences in the statement,
$$
\mathbb {E} [ f (\bar {x} _ {k}) - f (x ^ {*}) ] \leq \frac {C _ {1}}{\alpha k ^ {1 - \theta}} + \frac {C _ {2}}{k ^ {\rho}} h + \frac {d}{\ell} \frac {C _ {3}}{k ^ {\theta}} \alpha ,
$$
$$
C _ {1} := \frac {(1 - \theta) \| x _ {0} - x ^ {*} \| ^ {2}}{2}, \qquad C _ {2} := \frac {L _ {0} (1 - \theta)}{(1 - \theta - \rho)} \quad \text {a n d} \quad C _ {3} := \frac {c L _ {0} ^ {2} (1 - \theta)}{(1 - 2 \theta)}.
$$
The second point of the corollary can be proved replacing  $\alpha_{k} = \alpha$  and  $h_k = h$ . Now, to prove the third point, fix  $\varepsilon \in (0,1)$ . Since we want  $\mathbb{E}[f(\bar{x}_k) - f(x^*)] \leq \varepsilon$ , we impose
$$
\frac {\| x _ {0} - x ^ {*} \| ^ {2}}{2 \alpha k} + \frac {c d L _ {0} ^ {2}}{\ell} \alpha + L _ {0} h \leq \varepsilon .
$$
Choosing  $h_k = h \leq \frac{\varepsilon}{2L_0}$ , to get the previous inequality it is sufficient to impose
$$
\frac {\| x _ {0} - x ^ {*} \| ^ {2}}{2 \alpha k} + \frac {c d L _ {0} ^ {2}}{\ell} \alpha \leq \frac {\varepsilon}{2}.
$$
We fix a priori a number of iterations  $K$  and we minimize the left hand side with respect to  $\alpha$ , obtaining
$$
\alpha = \sqrt {\frac {\ell}{d}} \frac {\| x _ {0} - x ^ {*} \|}{\sqrt {2 c K} L _ {0}}.
$$
Thus, for  $h_k = h \leq \frac{\varepsilon}{2L_0}$ ,  $\alpha$  as above and
$$
K \geq \frac {8 \| x _ {0} - x ^ {*} \| ^ {2} L _ {0} ^ {2} c d}{\ell \varepsilon^ {2}},
$$
we have  $\mathbb{E}[f(\bar{x}_k) - f(x^*)] \leq \varepsilon$ . Note that, since the computation of the surrogate requires  $2\ell$  function evaluations, to ensure an error of  $\varepsilon$  we need to perform a number of function evaluations of the order
$$
\mathcal {O} (d \varepsilon^ {- 2}).
$$
By Lemma 7,