SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \| \nabla F (\pmb {x} ^ {t}) \| _ {2} ^ {2} \right] + \frac {1}{2} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \| \mathbb {E} ^ {t + 1} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} \right] \\ \leq \frac {2 (\mathbb {E} [ F (\pmb {x} ^ {T}) ] - F ^ {\star})}{\gamma T} + L _ {F} \gamma \big (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2} \big) + \frac {\sigma_ {\mathrm {b i a s}} ^ {2}}{m} \\ \end{array}
$$
Then we can minimize the right-hand size by optimizing  $\gamma$  to
$$
\gamma = \sqrt {\frac {2 (F (\pmb {x} ^ {0}) - F ^ {\star})}{L _ {F} (\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2}) T}}
$$
which is smaller than the bound of step size  $\gamma \leq \frac{1}{2L_F}$  if  $T$  is greater than the following constant which does not rely on the target precision  $\varepsilon$
$$
T \geq \frac {8 L _ {F} (F (\pmb {x} ^ {0}) - F ^ {\star})}{\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2}}.
$$
Then the upper bound of gradient becomes
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} \right] \leq \sqrt {\frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star}) L _ {F} \left(\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2}\right)}{T}} + \frac {\sigma_ {\mathrm {b i a s}} ^ {2}}{m}.
$$
By taking inner batch size of at least
$$
m \geq \frac {\sigma_ {\mathrm {b i a s}} ^ {2}}{\varepsilon^ {2}},
$$
and iteration  $T$  greater than
$$
T \geq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star}) L _ {F} (\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2})}{\varepsilon^ {4}},
$$
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} \right] \leq 2 \varepsilon^ {2}.
$$
By picking  $\boldsymbol{x}^s$  uniformly at random among  $\{\boldsymbol{x}^t\}_{t=0}^{T-1}$ , we get the desired guarantee.
The resulting sample complexity of BSGD to get to an  $\varepsilon$ -stationary point is  $\mathcal{O}(\varepsilon^{-6})$ .
D.3 Convergence of E-BSGD
In this section, we analyze the sample complexity of Algorithm 2 (E-BSGD) for the CSO problem.
Lemma 6 (Bias and Variance of E-BSGD) The bias and variance of  $E$ -BSGD are
$$
\mathcal {E} _ {b i a s} ^ {t + 1} \leq \frac {\tilde {\sigma} _ {b i a s} ^ {2}}{m ^ {4}}, \quad \mathcal {E} _ {v a r} ^ {t + 1} \leq 1 4 \left(\frac {\sigma_ {i n} ^ {2}}{m} + \sigma_ {o u t} ^ {2}\right)
$$
where  $\sigma_{in}^{2} = \zeta_{g}^{2}C_{f}^{2} + \sigma_{g}^{2}C_{g}^{2}L_{f}^{2}$ ,  $\sigma_{out}^{2} = C_{F}^{2}$ , and  $\tilde{\sigma}_{bias}^{2} = C_{g}^{2}C_{e}^{2}$  with  $C_e^2$  defined in Corollary 1.
Proof: Denote  $G^{t + 1} = G_{\mathrm{E - BSGD}}^{t + 1}(7)$ . Like previously (Lemma 5), let  $\mathbb{E}[\cdot ]$  denote the conditional expectation  $\mathbb{E}^{t + 1}[\cdot |t]$  which conditions on all past randomness until time  $t$ . In E-BSGD, we apply extrapolation to  $\nabla f_{\xi}(\cdot)$ . The bias can be estimated with the help of Corollary 1 as
$$
\begin{array}{l} \mathcal {E} _ {\text {b i a s}} ^ {t + 1} = \left\| \nabla F (\boldsymbol {x} ^ {t + 1}) - \mathbb {E} [ G ^ {t + 1} ] \right\| _ {2} ^ {2} \\ \leq C _ {g} ^ {2} \mathbb {E} _ {\xi} \left[ \left\| \nabla f _ {\xi} \left(\mathbb {E} _ {\eta | \xi} \left[ g _ {\eta} \left(\boldsymbol {x} ^ {t}\right) \right]\right) - \mathbb {E} \left[ \mathcal {L} _ {\mathcal {D} _ {\boldsymbol {g}, \xi} ^ {t + 1}} ^ {(2)} \nabla f _ {\xi} (0) \right] \right\| _ {2} ^ {2} \right] \\ \leq \frac {C _ {g} ^ {2} C _ {e} ^ {2}}{m ^ {4}}. \\ \end{array}
$$
Since the variance of BSGD in Lemma 5 is upper bounded by  $\frac{\sigma_{\mathrm{in}}^2}{m} +\sigma_{\mathrm{out}}^2$  , then Lemma 2 gives
$$
\mathcal {E} _ {\text {v a r}} ^ {t + 1} \leq 1 4 \left(\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\text {o u t}} ^ {2}\right).
$$
This proves the claimed bounds.
$$
T = \Omega \left(L _ {F} \left(F \left(\boldsymbol {x} ^ {0}\right) - F ^ {\star}\right) \left(\tilde {L} _ {F} ^ {2} / m + C _ {F} ^ {2}\right) \varepsilon^ {- 4}\right).
$$
Proof: The proof is very similar to Theorem 6. Denote  $G^{t + 1} = G_{\mathrm{E - BSGD}}^{t + 1}$  (7). Using descent lemma (Lemma 4) and bias-variance bounds of E-BSGD (Lemma 6)
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} \right] + \frac {1}{2} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \left\| \mathbb {E} ^ {t + 1} [ G ^ {t + 1} | t ] \right\| _ {2} ^ {2} \right] \\ \leq \frac {2 (\mathbb {E} [ F (\pmb {x} ^ {T}) ] - F ^ {\star})}{\gamma T} + 1 4 L _ {F} \gamma \big (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2} \big) + \frac {C _ {g} ^ {2} C _ {e} ^ {2}}{m ^ {4}}. \\ \end{array}
$$
Then we optimize  $\gamma$  to
$$
\gamma = \sqrt {\frac {(F (\pmb {x} ^ {0}) - F ^ {\star})}{7 L _ {F} (\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2}) T}}
$$
$$
T \geq \frac {4 L _ {F} (F (\pmb {x} ^ {0}) - F ^ {\star})}{7 (\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2})}.
$$
Then the gradient norm has the following upper bound.
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \| \nabla F (\pmb {x} ^ {t}) \| _ {2} ^ {2} \leq 4 \sqrt {\frac {7 (F (\pmb {x} ^ {0}) - F ^ {\star}) L _ {F} (\sigma_ {\mathrm {i n}} ^ {2} + \sigma_ {\mathrm {o u t}} ^ {2})}{T}} + \frac {\bar {\sigma} _ {\mathrm {b i a s}} ^ {2}}{m ^ {4}}.
$$
In order to reach  $\varepsilon$ -stationary point, i.e.
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \| \nabla F (\pmb {x} ^ {t}) \| _ {2} ^ {2} \leq \varepsilon^ {2},
$$
we can enforce
$$
4 \sqrt {\frac {7 (F (\pmb {x} ^ {0}) - F ^ {\star}) L _ {F} (\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2})}{T}} \leq \varepsilon^ {2}, \quad \frac {C _ {g} ^ {2} C _ {e} ^ {2}}{m ^ {4}} \leq \varepsilon^ {2}.
$$
$$
m = \Omega (\tilde {\sigma} _ {\mathrm {b i a s}} ^ {1 / 2} \varepsilon^ {- 1 / 2}),
$$
$$
T \geq \frac {1 1 2 (F (\pmb {x} ^ {0}) - F ^ {\star}) L _ {F} (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2})}{\varepsilon^ {4}},
$$
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} \leq 3 \varepsilon^ {2}.
$$
D.4 Convergence of BSpiderBoost
In this section, we reanalyze the BSpiderBoost algorithm of [16] to obtain bounds on bias and variance of its gradient estimates. Theorem 6 shows that BSpiderBoost achieves an  $\mathcal{O}(\varepsilon^{-5})$  sample complexity.
Let  $G_{\mathrm{BSB}}^{t + 1}$  as the BSpiderBoost gradient estimate
$$
G _ {\mathrm {B S B}} ^ {t + 1} = \left\{ \begin{array}{l l} G _ {\mathrm {B S B}} ^ {t} + \frac {1}{B _ {2}} \sum_ {\xi \in \mathcal {B} _ {2}} (G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t}) & \text {w i t h p r o b . 1 - p _ {\mathrm {o u t}}} \\ \frac {1}{B _ {1}} \sum_ {\xi \in \mathcal {B} _ {1}} G _ {\mathrm {B S G D}} ^ {t + 1} & \text {w i t h p r o b . p _ {\mathrm {o u t}}}. \end{array} \right.
$$
Lemma 7 (Bias and Variance of BSpiderBoost) If  $\gamma \leq \min \left\{\frac{1}{2L_F},\frac{\sqrt{B_2}}{6L_F}\right\}$ , then the bias and variance of BSpiderBoost are
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {b i a s} ^ {t + 1} ] \leq \frac {2 \sigma_ {b i a s} ^ {2}}{m} + \frac {(1 - p _ {o u t}) ^ {3}}{p _ {o u t} B _ {2}} \frac {5 6 L _ {F} ^ {2} \gamma^ {2}}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t + 1} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T p _ {o u t}} + 1) \frac {4 (1 - p _ {o u t}) ^ {2}}{B _ {1}} (\frac {\sigma_ {i n} ^ {2}}{m} + \sigma_ {o u t} ^ {2})
$$
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {v a r} ^ {t + 1} ] \leq \frac {2 8 (1 - p _ {o u t}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t + 1} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T} + p _ {o u t}) \frac {2}{B _ {1}} (\frac {\sigma_ {i n} ^ {2}}{m} + \sigma_ {o u t} ^ {2}),
$$
$$
w h e r e \sigma_ {i n} ^ {2} = \zeta_ {g} ^ {2} C _ {f} ^ {2} + \sigma_ {g} ^ {2} C _ {g} ^ {2} L _ {f} ^ {2}, \sigma_ {o u t} = C _ {F} ^ {2}, a n d \sigma_ {b i a s} ^ {2} = \sigma_ {g} ^ {2} C _ {g} ^ {2} L _ {f} ^ {2}.
$$
Proof: Denote  $G^{t + 1} = G_{\mathrm{BSB}}^{t + 1}$  (8). Like previously (Lemma 5), let  $\mathbb{E}[\cdot ]$  denote the conditional expectation  $\mathbb{E}^{t + 1}[\cdot |t]$  which conditions on all past randomness until time  $t$ . Denote  $G_L^{t + 1}$  and  $G_S^{t + 1}$  as the large batch and small batch in BSpiderBoost separately, i.e.,
$$
\left\{ \begin{array}{l l} G _ {L} ^ {t + 1} = \frac {1}{B _ {1}} \sum_ {\xi \in \mathcal {B} _ {1}} G _ {\text {B S G D}} ^ {t + 1} & \text {w i t h p r o b .} p _ {\text {o u t}} \\ G _ {S} ^ {t + 1} = G ^ {t} + \frac {1}{B _ {2}} \sum_ {\xi \in \mathcal {B} _ {2}} (G _ {\text {B S G D}} ^ {t + 1} - G _ {\text {B S G D}} ^ {t}) & \text {w i t h p r o b .} 1 - p _ {\text {o u t}}. \end{array} \right.
$$
The bias of BSpiderBoost can be decomposed to its distance to BSGD and the distance from BSGD to the full gradient, i.e.,
$$
\begin{array}{l} \mathcal {E} _ {\text {b i a s}} ^ {t + 1} = \left\| \nabla F (\boldsymbol {x} ^ {t + 1}) - \mathbb {E} [ G ^ {t + 1} ] \right\| _ {2} ^ {2} \\ \leq 2 \left\| \nabla F \left(\boldsymbol {x} ^ {t + 1}\right) - \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t + 1} \right] \right\| _ {2} ^ {2} + 2 \left\| \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t + 1} \right] - \mathbb {E} \left[ G ^ {t + 1} \right] \right\| _ {2} ^ {2} \tag {14} \\ \leq \frac {2 \sigma_ {\mathrm {b i a s}} ^ {2}}{m} + 2 \left\| \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t + 1} \right] - \mathbb {E} \left[ G ^ {t + 1} \right] \right\| _ {2} ^ {2}. \\ \end{array}
$$
where the last inequality uses the bias of BSGD from Lemma 5. Then the second term can be bounded as follows
$$
\begin{array}{l} \left\| \mathbb {E} [ G _ {\mathrm {B S G D}} ^ {t + 1} ] - \mathbb {E} [ G ^ {t + 1} ] \right\| _ {2} ^ {2} = (1 - p _ {\mathrm {o u t}}) ^ {2} \left\| \mathbb {E} [ G _ {\mathrm {B S G D}} ^ {t + 1} ] - \mathbb {E} [ G _ {S} ^ {t + 1} ] \right\| _ {2} ^ {2} \\ = \left(1 - p _ {\text {o u t}}\right) ^ {2} \left\| \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t} \right] - G ^ {t} \right\| _ {2} ^ {2}. \\ \end{array}
$$
By taking the expectation of randomness of  $G^{t}$
$$
\begin{array}{l} \left\| \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t + 1} \right] - \mathbb {E} \left[ G ^ {t + 1} \right] \right\| _ {2} ^ {2} = (1 - p _ {\text {o u t}}) ^ {2} \left(\left\| \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t} \right] - \mathbb {E} \left[ G ^ {t} \right] \right\| _ {2} ^ {2} + \mathbb {E} \left\| G ^ {t} - \mathbb {E} \left[ G ^ {t} \right] \right\| _ {2} ^ {2}\right) \\ = \left(1 - p _ {\text {o u t}}\right) ^ {2} \left(\left| \left| \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t} \right] - \mathbb {E} [ G ^ {t} ] \right| \right| _ {2} ^ {2} + \mathcal {E} _ {\mathrm {v a r}} ^ {t}\right) \\ \end{array}
$$
Note that  $\left\| \mathbb{E}[G_{\mathrm{BSGD}}^1] - \mathbb{E}[G^1]\right\|_2^2 = 0$  as the first iteration always chooses the large batch. Then as we always use large batch at  $t = 0$  we know that
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left\| \mathbb {E} \left[ G _ {\text {B S G D}} ^ {t + 1} \right] - \mathbb {E} \left[ G ^ {t + 1} \right] \right\| _ {2} ^ {2} \leq \frac {(1 - p _ {\text {o u t}}) ^ {2}}{p _ {\text {o u t}}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathcal {E} _ {\text {v a r}} ^ {t + 1}. \tag {15}
$$
Therefore combine (14) and (15) we can upper bound the bias
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathcal {E} _ {\text {b i a s}} ^ {t + 1} \leq \frac {2 \sigma_ {\text {b i a s}} ^ {2}}{m} + \frac {2 (1 - p _ {\text {o u t}}) ^ {2}}{p _ {\text {o u t}}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathcal {E} _ {\text {v a r}} ^ {t + 1}. \tag {16}
$$
Variance. Now we consider the variance,
$$
\begin{array}{l} \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} = \mathbb {E} \left[ \left\| G ^ {t + 1} - \mathbb {E} [ G ^ {t + 1} ] \right\| _ {2} ^ {2} \right] \\ \leq \left(1 - p _ {\mathrm {o u t}}\right) \mathbb {E} \left[ \left\| G _ {S} ^ {t + 1} - \mathbb {E} [ G _ {S} ^ {t + 1} ] \right\| _ {2} ^ {2} \right] + p _ {\mathrm {o u t}} \mathbb {E} \left[ \left\| G _ {L} ^ {t + 1} - \mathbb {E} [ G _ {L} ^ {t + 1} ] \right\| _ {2} ^ {2} \right] \\ = \frac {(1 - p _ {\mathrm {o u t}})}{B _ {2}} \mathbb {E} \left[ \left\| G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} - \mathbb {E} [ G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} ] \right\| _ {2} ^ {2} \right] + \frac {p _ {\mathrm {o u t}}}{B _ {1}} \mathbb {E} \left[ \left\| G _ {\mathrm {B S G D}} ^ {t + 1} - \mathbb {E} [ G _ {\mathrm {B S G D}} ^ {t + 1} ] \right\| _ {2} ^ {2} \right] \\ \leq \frac {1 - p _ {\text {o u t}}}{B _ {2}} \mathbb {E} \left[ \left\| G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} - \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} \right] \right\| _ {2} ^ {2} \right] + \frac {p _ {\text {o u t}}}{B _ {1}} \left(\frac {\sigma_ {\text {i n}} ^ {2}}{m} + \sigma_ {\text {o u t}} ^ {2}\right) \tag {17} \\ \end{array}
$$
where the last equality is because the large batch in BSpiderBoost is similar to BSGD.
$$
\mathcal {E} _ {\text {v a r}} ^ {1} = \mathbb {E} \left[ \left\| G ^ {1} - \mathbb {E} [ G ^ {1} ] \right\| _ {2} ^ {2} \right] = \mathbb {E} \left[ \left\| G _ {L} ^ {1} - \mathbb {E} [ G _ {L} ^ {1} ] \right\| _ {2} ^ {2} \right] = \frac {1}{B _ {1}} \mathbb {E} \left[ \left\| G _ {\mathrm {B S G D}} ^ {1} - \mathbb {E} [ G _ {\mathrm {B S G D}} ^ {1} ] \right\| _ {2} ^ {2} \right] \leq \frac {1}{B _ {1}} \left(\frac {\sigma_ {\mathrm {m}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}\right). \tag {18}
$$
Finally, we expand the variance at small batch size epoch
$$
\begin{array}{l} \mathbb {E} \left[ \left\| G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} - \mathbb {E} \left[ G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} \right] \right\| _ {2} ^ {2} \right] \\ = \underbrace {\mathbb {E} \left[ \left\| G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} - \mathbb {E} _ {\eta | \xi , \tilde {\eta} | \xi} \left[ G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} \right] \right\| _ {2} ^ {2} \right]} _ {\text {I n n e r v a r i a n c e} \mathcal {T} _ {\mathrm {i n}}} \\ + \underbrace {\mathbb {E} _ {\xi} \left[ \left\| \mathbb {E} _ {\eta | \xi , \tilde {\eta} | \xi} [ G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} ] - \mathbb {E} [ G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} ] \right\| _ {2} ^ {2} \right]} _ {\text {O u t e r v a r i a n c e} \mathcal {T} _ {\text {o u t}}} \\ \end{array}
$$
The outer variance  $\mathcal{T}_{\mathrm{out}}$  can be upper bounded as
$$
\begin{array}{l} \mathcal {T} _ {\mathrm {o u t}} \leq \mathbb {E} _ {\xi} \left[ \left| \left| \mathbb {E} _ {\eta | \xi , \tilde {\eta} | \xi} [ G _ {\mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {B S G D}} ^ {t} ] \right| \right| _ {2} ^ {2} \right] \\ = \mathbb {E} _ {\xi} \left[ \left\| \left(\mathbb {E} _ {\tilde {\eta} | \xi} [ \nabla g _ {\tilde {\eta}} (\boldsymbol {x} ^ {t}) ]\right) ^ {\top} \mathbb {E} _ {\eta | \xi} [ \nabla f _ {\xi} (\frac {1}{m} \sum_ {\eta \in H _ {\xi}} g _ {\eta} (\boldsymbol {x} ^ {t})) ] - \left(\mathbb {E} _ {\tilde {\eta} | \xi} [ \nabla g _ {\tilde {\eta}} (\boldsymbol {x} ^ {t - 1}) ]\right) ^ {\top} \mathbb {E} _ {\eta | \xi} [ \nabla f _ {\xi} (\frac {1}{m} \sum_ {\eta \in H _ {\xi}} g _ {\eta} (\boldsymbol {x} ^ {t - 1})) ] \right\| _ {2} ^ {2} \right] \\ \leq 2 \mathbb {E} _ {\boldsymbol {\xi}} \left[ \left\| (\mathbb {E} _ {\tilde {\eta} | \boldsymbol {\xi}} [ \nabla g _ {\tilde {\eta}} (\boldsymbol {x} ^ {t}) ] - \mathbb {E} _ {\tilde {\eta} | \boldsymbol {\xi}} [ \nabla g _ {\tilde {\eta}} (\boldsymbol {x} ^ {t - 1}) ]) ^ {\top} \mathbb {E} _ {\boldsymbol {\eta} | \boldsymbol {\xi}} [ \nabla f _ {\boldsymbol {\xi}} (\frac {1}{m} \sum_ {\boldsymbol {\eta} \in H _ {\boldsymbol {\xi}}} g _ {\boldsymbol {\eta}} (\boldsymbol {x} ^ {t})) ] \right\| _ {2} ^ {2} \right] \\ + 2 \mathbb {E} _ {\boldsymbol {\xi}} \left[ \left\| (\mathbb {E} _ {\tilde {\eta} | \boldsymbol {\xi}} [ \nabla g _ {\tilde {\eta}} (\pmb {x} ^ {t - 1}) ]) ^ {\top} \mathbb {E} _ {\boldsymbol {\eta} | \boldsymbol {\xi}} [ \nabla f _ {\boldsymbol {\xi}} (\frac {1}{m} \sum_ {\boldsymbol {\eta} \in H _ {\boldsymbol {\xi}}} g _ {\boldsymbol {\eta}} (\pmb {x} ^ {t})) - \nabla f _ {\boldsymbol {\xi}} (\frac {1}{m} \sum_ {\boldsymbol {\eta} \in H _ {\boldsymbol {\xi}}} g _ {\boldsymbol {\eta}} (\pmb {x} ^ {t - 1})) ] \right\| _ {2} ^ {2} \right] \\ \leq 2 L _ {g} ^ {2} C _ {f} ^ {2} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} + 2 C _ {g} ^ {4} L _ {f} ^ {2} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} \\ = 2 L _ {F} ^ {2} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} \\ = 2 L _ {F} ^ {2} \gamma^ {2} \left\| G ^ {t} \right\| _ {2} ^ {2}. \\ \end{array}
$$
The inner variance can be bounded by
Then we have that
$$
\begin{array}{l} \mathcal {T} _ {\mathrm {i n}} \leq \frac {4 L _ {g} ^ {2} C _ {f} ^ {2}}{m} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} + \frac {4 L _ {g} ^ {2} C _ {f} ^ {2}}{m} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} \\ + 4 C _ {g} ^ {2} \mathbb {E} \left[ \left\| \nabla f _ {\xi} \left(\frac {1}{m} \sum_ {\eta \in H _ {\xi}} g _ {\eta} (\boldsymbol {x} ^ {t})\right) - \nabla f _ {\xi} \left(\frac {1}{m} \sum_ {\eta \in H _ {\xi}} g _ {\eta} (\boldsymbol {x} ^ {t - 1})\right) - \left(\nabla f _ {\xi} \left(\mathbb {E} _ {\eta | \xi} [ g _ {\eta} (\boldsymbol {x} ^ {t}) ]\right) - \nabla f _ {\xi} \left(\mathbb {E} _ {\eta | \xi} [ g _ {\eta} (\boldsymbol {x} ^ {t - 1}) ]\right)\right) \right\| _ {2} ^ {2} \right] \\ + 4 C _ {g} ^ {2} \mathbb {E} \left[ \left\| \left(\nabla f _ {\xi} \left(\mathbb {E} _ {\eta | \xi} \left[ g _ {\eta} \left(\boldsymbol {x} ^ {t}\right) \right]\right) - \nabla f _ {\xi} \left(\mathbb {E} _ {\eta | \xi} \left[ g _ {\eta} \left(\boldsymbol {x} ^ {t - 1}\right) \right]\right)\right) \right. \right. \\ \left. \left. - \mathbb {E} _ {\eta | \xi} \left[ \nabla f _ {\xi} \left(\frac {1}{m} \sum_ {\eta \in H _ {\xi}} g _ {\eta} (\boldsymbol {x} ^ {t})\right) - \nabla f _ {\xi} \left(\frac {1}{m} \sum_ {\eta \in H _ {\xi}} g _ {\eta} (\boldsymbol {x} ^ {t - 1})\right) \right] \right\| _ {2} ^ {2} \right] \\ + \frac {4 C _ {g} ^ {4} L _ {f} ^ {2}}{m} \| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \| _ {2} ^ {2} \\ \leq \frac {8 L _ {g} ^ {2} C _ {f} ^ {2}}{m} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} + \frac {8 C _ {g} ^ {4} L _ {f} ^ {2}}{m} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} + \frac {4 C _ {g} ^ {4} L _ {f} ^ {2}}{m} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} \\ \leq \frac {1 2 L _ {F} ^ {2}}{m} \left\| \boldsymbol {x} ^ {t} - \boldsymbol {x} ^ {t - 1} \right\| _ {2} ^ {2} \\ = \frac {1 2 L _ {F} ^ {2} \gamma^ {2}}{m} \left\| G ^ {t} \right\| _ {2} ^ {2}. \\ \end{array}
$$
To sum up, the variance is bounded by
$$
\begin{array}{l} \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} \leq \frac {2 (1 - p _ {\mathrm {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \left(1 + \frac {6}{m}\right) \left\| G ^ {t} \right\| _ {2} ^ {2} + \frac {p _ {\mathrm {o u t}}}{B _ {1}} \left(\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}\right) \\ \leq \frac {1 4 (1 - p _ {\mathrm {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \left\| G ^ {t} \right\| _ {2} ^ {2} + \frac {p _ {\mathrm {o u t}}}{B _ {1}} \left(\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}\right) \\ = \frac {1 4 (1 - p _ {\mathrm {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \mathbb {E} ^ {t} [ \| G ^ {t} \| _ {2} ^ {2} | t - 1 ] + \frac {p _ {\mathrm {o u t}}}{B _ {1}} (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}). \\ \end{array}
$$
Then averaging over time and now we redefine  $\mathbb{E}[\cdot ] = \mathbb{E}^T\ldots [\mathbb{E}^0 [\cdot ]]$
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \mathcal {E} _ {\text {v a r}} ^ {t + 1} \right] \leq \frac {1 4 (1 - p _ {\text {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \left\| G ^ {t + 1} \right\| _ {2} ^ {2} \right] + \frac {\mathcal {E} _ {\text {v a r}} ^ {1}}{T} + \frac {p _ {\text {o u t}}}{B _ {1}} \left(\frac {\sigma_ {\text {i n}} ^ {2}}{m} + \sigma_ {\text {o u t}} ^ {2}\right) \\ = \frac {1 4 (1 - p _ {\mathrm {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \big (\mathbb {E} [ \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} ] + \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] \big) + \frac {\mathcal {E} _ {\mathrm {v a r}} ^ {1}}{T} + \frac {p _ {\mathrm {o u t}}}{B _ {1}} (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}). \\ \end{array}
$$
If we take  $\gamma \leq \frac{\sqrt{B_2}}{6L_F}$ , then  $\frac{14(1 - p_{\mathrm{out}})L_F^2\gamma^2}{B_2} \leq \frac{1}{2}$ , therefore
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} ] \leq \frac {2 8 (1 - p _ {\mathrm {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + \frac {\mathcal {E} _ {\mathrm {v a r}} ^ {1}}{T} + \frac {2 p _ {\mathrm {o u t}}}{B _ {1}} (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}) \\ \stackrel {(1 8)} {\leq} \frac {2 8 (1 - p _ {\mathrm {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T} + p _ {\mathrm {o u t}}) \frac {2}{B _ {1}} (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}) \\ \end{array}
$$
Then with (16), we can bound the bias by
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\text {b i a s}} ^ {t + 1} ] \leq \frac {2 \sigma_ {\text {b i a s}} ^ {2}}{m} + \frac {(1 - p _ {\text {o u t}}) ^ {3}}{p _ {\text {o u t}} B _ {2}} \frac {5 6 L _ {F} ^ {2} \gamma^ {2}}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T p _ {\text {o u t}}} + 1) \frac {4 (1 - p _ {\text {o u t}}) ^ {2}}{B _ {1}} (\frac {\sigma_ {\text {i n}} ^ {2}}{m} + \sigma_ {\text {o u t}} ^ {2})
$$
Theorem 7 (BSpiderBoost Convergence) Consider the (CSO) problem. Suppose Assumptions 3, 4, 5 hold true. Let step size  $\gamma \leq 1 / (13L_F)$ . Then for BSpiderBoost,  $\mathbf{x}^s$  picked uniformly at random among  $\{\mathbf{x}^t\}_{t=0}^{T-1}$  satisfies:  $\mathbb{E}[\|\nabla F(\mathbf{x}^s)\|_2^2] \leq \varepsilon^2$ , for nonconvex  $F$ , if the inner batch size  $m = \Omega(\sigma_{bias}^2\varepsilon^{-2})$ , the hyperparameters of the outer loop of BSpiderBoost are  $B_1 = (\sigma_{in}^2/m + \sigma_{out}^2)\varepsilon^{-2}$ ,  $B_2 = \mathcal{O}(\varepsilon^{-1})$ ,  $p_{out} = 1/B_2$ , and the number of iterations  $T = \Omega(L_F(F(\mathbf{x}^0) - F^\star)\varepsilon^{-2})$ , where  $\sigma_{in}^2 = \zeta_g^2C_f^2 + \sigma_g^2C_g^2L_f^2$ ,  $\sigma_{out}^2 = C_F^2$ , and  $\sigma_{bias}^2 = \sigma_g^2C_g^2L_f^2$ .
Proof: Denote  $G^{t + 1} = G_{\mathrm{BSB}}^{t + 1}$  (8). Using descent lemma (Lemma 4) and bias-variance bounds of BSpiderBoost (Lemma 7)