SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} ] + \frac {1}{2 T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {*})}{\gamma T} + L _ {F} \gamma \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \mathcal {E} _ {\text {v a r}} ^ {t + 1} \right] + \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \mathcal {E} _ {\text {b i a s}} ^ {t + 1} \right] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + L _ {F} \gamma \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \mathcal {E} _ {\text {v a r}} ^ {t + 1} \right] + \frac {2 \sigma_ {\text {b i a s}} ^ {2}}{m} + \frac {2}{p _ {\text {o u t}}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \mathcal {E} _ {\text {v a r}} ^ {t + 1} \right] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + \frac {2 \sigma_ {\text {b i a s}} ^ {2}}{m} + \frac {3}{p _ {\text {o u t}}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\text {v a r}} ^ {t + 1} ] \\ \end{array}
$$
where the last inequality use  $\gamma \leq \frac{1}{2L_F}$ . Use the variance estimation of  $G^{t + 1}$  and choose  $B_{2}p_{\mathrm{out}} = 1$
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} ] + \frac {1}{2 T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + \frac {2 \sigma_ {\text {b i a s}} ^ {2}}{m} + 8 4 L _ {F} ^ {2} \gamma^ {2} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T p _ {\text {o u t}}} + 1) \frac {6}{B _ {1}} (\frac {\sigma_ {\text {i n}} ^ {2}}{m} + \sigma_ {\text {o u t}} ^ {2}). \\ \end{array}
$$
Now we can let  $\gamma \leq \frac{1}{13L_F}$  such that  $84L_F^2\gamma^2 \leq \frac{1}{2}$ .
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla F (\pmb {x} ^ {t}) \| _ {2} ^ {2} ] \leq \frac {2 (F (\pmb {x} ^ {0}) - F ^ {\star})}{\gamma T} + \frac {2 \sigma_ {\mathrm {b i a s}} ^ {2}}{m} + \left(\frac {1}{T p _ {\mathrm {o u t}}} + 1\right) \frac {6}{B _ {1}} \left(\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}\right).
$$
In order for the right-hand side to be  $\varepsilon^2$ , the inner batch size
$$
m \geq \frac {2 \sigma_ {\mathrm {p i a s}} ^ {2}}{\varepsilon^ {2}},
$$
and the outer batch size
$$
B _ {1} = \frac {\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2}}{\varepsilon^ {2}}, \quad B _ {2} = \sqrt {B _ {1}}, \quad p _ {\mathrm {o u t}} = \frac {1}{B _ {2}}.
$$
The step size  $\gamma$  is upper bounded by  $\min \left\{\frac{1}{2L_F},\frac{\sqrt{B_2}}{6L_F},\frac{1}{13L_F}\right\}$ . As  $B_{2}\geq 1$ , we can take  $\gamma = \frac{1}{13L_F}$ . So we need iteration  $T$  greater than
$$
T \geq \frac {2 6 L _ {F} (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\varepsilon^ {2}}.
$$
By picking  $\pmb{x}^s$  uniformly at random among  $\{\pmb{x}^t\}_{t=0}^{T-1}$ , we get the desired guarantee.
The resulting sample complexity of BSpiderBoost to get to an  $\varepsilon$ -stationary point is  $\mathcal{O}(\varepsilon^{-5})$ .
D.5 Convergence of E-BSpiderBoost
In this section, we analyze the sample complexity of Algorithm 3 (E-BSpiderBoost) for the CSO problem.
Lemma 8 (Bias and Variance of E-BSpiderBoost) The bias and variance of E-BSpiderBoost are
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {v a r} ^ {t + 1} ] \leq \frac {2 8 (1 - p _ {o u t}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T p _ {o u t}} + 1) \frac {2 8 p _ {o u t}}{B _ {1}} (\frac {\sigma_ {i n} ^ {2}}{m} + \sigma_ {o u t} ^ {2})
$$
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {b i a s} ^ {t + 1} ] \leq \frac {2 \tilde {\sigma} _ {b i a s} ^ {2}}{m ^ {4}} + \frac {2}{p _ {o u t}} \frac {(1 - p _ {o u t}) ^ {2}}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {v a r} ^ {t + 1} ],
$$
where  $\sigma_{in}^{2} \coloneqq \zeta_{g}^{2}C_{f}^{2} + \sigma_{g}^{2}C_{g}^{2}L_{f}^{2}$ ,  $\sigma_{out} = C_{F}^{2}$ , and  $\tilde{\sigma}_{bias}^{2} = C_{g}^{2}C_{e}^{2}$  with  $C_e^2$  defined in Corollary 1.
Proof: Denote  $G^{t + 1} = G_{\mathrm{E - BSB}}^{t + 1}$  (9). Like previously (Lemma 5), let  $\mathbb{E}[\cdot ]$  denote the conditional expectation  $\mathbb{E}^t [\cdot |t]$  which conditions on all past randomness until time  $t$ . Let  $G^{t + 1} = G_{\mathrm{E - BSB}}^{t + 1}$  be the E-BSpiderBoost update. We expand the bias as follows
$$
\begin{array}{l} \mathcal {E} _ {\text {b i a s}} ^ {t + 1} = \left\| \nabla F \left(\boldsymbol {x} ^ {t + 1}\right) - \mathbb {E} \left[ G ^ {t + 1} \right] \right\| _ {2} ^ {2} \\ \leq 2 \left\| \nabla F (\boldsymbol {x} ^ {t + 1}) - \mathbb {E} [ G _ {\mathrm {E - B S G D}} ^ {t + 1} ] \right\| _ {2} ^ {2} + 2 \left\| \mathbb {E} [ G _ {\mathrm {E - B S G D}} ^ {t + 1} ] - \mathbb {E} [ G ^ {t + 1} ] \right\| _ {2} ^ {2}. \\ \end{array}
$$
From Lemma 6, we know that
$$
\left\| \nabla F (\boldsymbol {x} ^ {t + 1}) - \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t + 1} \right] \right\| _ {2} ^ {2} \leq \frac {\tilde {\sigma} _ {\mathrm {b i a s}} ^ {2}}{m ^ {4}}.
$$
The distance between  $\mathbb{E}[G_{\mathrm{E - BSGD}}^{t + 1}]$  and  $\mathbb{E}[G^{t + 1}]$  can be bounded as follows.
$$
\begin{array}{l} \left\| \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t + 1} \right] - \mathbb {E} \left[ G ^ {t + 1} \right] \right\| _ {2} ^ {2} = (1 - p _ {\text {o u t}}) ^ {2} \left\| \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t + 1} \right] - \left(G ^ {t} + \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t + 1} - G _ {\mathrm {E - B S G D}} ^ {t} \right]\right) \right\| _ {2} ^ {2} \\ = \left(1 - p _ {\text {o u t}}\right) ^ {2} \left\| \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t} \right] - G ^ {t} \right\| _ {2} ^ {2} \\ \end{array}
$$
Taking expectation with respect to  $G^{t}$
$$
\left\| \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t + 1} \right] - \mathbb {E} \left[ G ^ {t + 1} \right] \right\| _ {2} ^ {2} \leq (1 - p _ {\text {o u t}}) ^ {2} \left(\left\| \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t} \right] - \mathbb {E} \left[ G ^ {t} \right] \right\| _ {2} ^ {2} + \left\| G ^ {t} - \mathbb {E} \left[ G ^ {t} \right] \right\| _ {2} ^ {2}\right).
$$
where  $\left\| \mathbb{E}[G_{\mathrm{E - BSGD}}^1 ] - \mathbb{E}[G^1 ]\right\| _2^2 = 0$  . By averaging over time we have
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left\| \mathbb {E} [ G _ {\mathrm {E - B S G D}} ^ {t + 1} ] - \mathbb {E} [ G ^ {t + 1} ] \right\| _ {2} ^ {2} \leq \frac {1}{p _ {\mathrm {o u t}}} \frac {(1 - p _ {\mathrm {o u t}}) ^ {2}}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} ].
$$
Then the bias is bounded by
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\mathrm {b i a s}} ^ {t + 1} ] \leq \frac {2 \tilde {\sigma} _ {\mathrm {b i a s}} ^ {2}}{m ^ {4}} + \frac {2}{p _ {\mathrm {o u t}}} \frac {(1 - p _ {\mathrm {o u t}}) ^ {2}}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} ].
$$
Variance. Since the extrapolation only gives a constant overhead given Lemma 2
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \left\| G _ {\mathrm {B S B}} ^ {t + 1} - \mathbb {E} ^ {t} [ G _ {\mathrm {B S B}} ^ {t + 1} | t ] \right\| _ {2} ^ {2} \right] \leq \frac {2 8 (1 - p _ {\mathrm {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \left\| \mathbb {E} ^ {t} [ G _ {\mathrm {B S B}} ^ {t + 1} | t ] \right\| _ {2} ^ {2} ] + \left(\frac {1}{T} + p _ {\mathrm {o u t}}\right) \frac {2 8}{B _ {1}} (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{m} + \sigma_ {\mathrm {o u t}} ^ {2}).
$$
Then the variance is bounded by
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\text {v a r}} ^ {t + 1} ] \leq \frac {2 8 (1 - p _ {\text {o u t}}) L _ {F} ^ {2} \gamma^ {2}}{B _ {2}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T p _ {\text {o u t}}} + 1) \frac {2 8 p _ {\text {o u t}}}{B _ {1}} (\frac {\sigma_ {\text {i n}} ^ {2}}{m} + \sigma_ {\text {o u t}} ^ {2}).
$$
Theorem 4 [E-BSpiderBoost Convergence] Consider the (CSO) problem under the same assumptions as Theorem 3. Let step size  $\gamma \leq 1 / (13L_F)$ . Then the output  $\pmb{x}^s$  of E-BSpiderBoost (Algorithm 3) satisfies:  $\mathbb{E}[\| \nabla F(\pmb{x}^s)\|_2^2 ]\leq \varepsilon^2$ , for nonconvex  $F$ , if the inner batch size  $m = \mathcal{O}(C_eC_g\varepsilon^{-0.5})$ , the hyperparameters of the outer loop of E-BSpiderBoost  $B_{1} = (\tilde{L}_{F}^{2} / m + C_{F}^{2})\varepsilon^{-2}$ ,  $B_{2} = \sqrt{B_{1}}$ ,  $p_{out} = 1 / B_{2}$ , and the number of iterations
Proof: Denote  $G^{t + 1} = G_{\mathrm{E - BSB}}^{t + 1}$  (9). Using descent lemma (Lemma 4) and bias-variance bounds of E-BSpiderBoost (Lemma 8)
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} ] + \frac {1}{2 T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + L _ {F} \gamma \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\text {v a r}} ^ {t + 1} ] + \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\text {b i a s}} ^ {t + 1} ] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + L _ {F} \gamma \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} ] + \frac {2 \tilde {\sigma} _ {\mathrm {b i a s}} ^ {2}}{m ^ {4}} + \frac {2}{p _ {\mathrm {o u t}}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} ] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + \frac {2 \tilde {\sigma} _ {\mathrm {b i a s}} ^ {2}}{m ^ {4}} + \frac {3}{p _ {\mathrm {o u t}}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \mathcal {E} _ {\mathrm {v a r}} ^ {t + 1} ] \\ \end{array}
$$
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} ] + \frac {1}{2 T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] \\ \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + \frac {2 \tilde {\sigma} _ {\text {b i a s}} ^ {2}}{m ^ {4}} + 8 4 L _ {F} ^ {2} \gamma^ {2} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \mathbb {E} ^ {t} [ G ^ {t + 1} | t ] \| _ {2} ^ {2} ] + (\frac {1}{T p _ {\text {o u t}}} + 1) \frac {8 4}{B _ {1}} (\frac {\sigma_ {\text {i n}} ^ {2}}{m} + \sigma_ {\text {o u t}} ^ {2}). \\ \end{array}
$$
Now we can let  $\gamma \leq \frac{1}{13L_F}$  such that  $84L_{F}^{2}\gamma^{2}\leq \frac{1}{2}$
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \| \nabla F \left(\boldsymbol {x} ^ {t}\right) \| _ {2} ^ {2} \right] \leq \frac {2 \left(F \left(\boldsymbol {x} ^ {0}\right) - F ^ {\star}\right)}{\gamma T} + \frac {2 \tilde {\sigma} _ {\text {b i a s}} ^ {2}}{m ^ {4}} + \left(\frac {1}{T p _ {\text {o u t}}} + 1\right) \frac {8 4}{B _ {1}} \left(\frac {\sigma_ {\text {i n}} ^ {2}}{m} + \sigma_ {\text {o u t}} ^ {2}\right). \tag {19}
$$
In order to make the right-hand side  $\varepsilon^2$ , the inner batch size
$$
m = \Omega (\tilde {\sigma} _ {\mathrm {b i a s}} ^ {2} \varepsilon^ {- 0. 5}),
$$
$$
B _ {1} = \frac {\left(\sigma_ {\mathrm {i n}} ^ {2} / m + \sigma_ {\mathrm {o u t}} ^ {2}\right)}{\varepsilon^ {2}}, \quad B _ {2} = \sqrt {B _ {1}}, \quad p _ {\mathrm {o u t}} = \frac {1}{B _ {2}}.
$$
The step size  $\gamma$  is upper bounded by  $\min \left\{\frac{1}{2L_F},\frac{\sqrt{B_2}}{6L_F},\frac{1}{13L_F}\right\}$ . As  $B_{2} \geq 1$ , we can take  $\gamma = \frac{1}{13L_F}$ . So we need iteration  $T$  greater than
E Stationary Point Convergence Proofs from Section 4 (FCCO)
In this section, we provide the convergence proofs for the FCCO problem. We start by analyzing a variant of BSpiderBoost (Algorithm 3) for this case in Appendix E.1. In Appendix E.2, we present a multi-level variance reduction approach (called NestedVR) that applies variance reduction in both outer (over the random variable  $i$ ) and inner (over the random variable  $\eta | i$ ) loops. In Appendix E.3, we analyze E-NestedVR. As in the case of CSO analyses, our proofs go via bounds on bias and variance terms of these algorithms.
E.1 E-BSpiderBoost for FCCO problem
Theorem 8 Consider the (FCCO) problem. Suppose Assumptions 3, 4, 5, 6 hold true. Let step size  $\gamma = \mathcal{O}(1 / L_F)$ . Then the output of  $E$ -BSpiderBoost (Algorithm 3) satisfies:  $\mathbb{E}[\| \nabla F(\boldsymbol{x}^s)\|_2^2] \leq \varepsilon^2$ , for nonconvex  $F$ , if the inner batch size  $m = \Omega (\max \{C_e C_g \varepsilon^{-1/2}, \sigma_{in}^2 n^{-1} \varepsilon^{-2}\})$ , the hyperparameters of the outer loop of  $E$ -BSpiderBoost  $B_1 = n$ ,  $B_2 = \sqrt{n}$ ,  $p_{out} = 1 / B_2$ , and the number of iterations  $T = \Omega (L_F(F(\boldsymbol{x}^0) - F^\star) \varepsilon^{-2})$ . The resulting sample complexity is
$$
\mathcal {O} \left(L _ {F} (F (\boldsymbol {x} ^ {0}) - F ^ {\star}) \max  \left\{\frac {\sqrt {n} C _ {e} C _ {g}}{\varepsilon^ {2 . 5}}, \frac {\sigma_ {i n} ^ {2}}{\sqrt {n} \varepsilon^ {4}} \right\}\right).
$$
Remark 9 The sample complexity depends on the relation between  $n$  and  $\varepsilon$
- When  $n = \mathcal{O}(1)$ , we have a complexity of  $\mathcal{O}(\varepsilon^{-4})$ . This happens because we did not apply variance reduction for the inner loop.
- When  $n = \Theta(\varepsilon^{-2/3})$ , E-BSpiderBoost has same performance as MSVR-V2 [19] of  $\mathcal{O}(n\varepsilon^{-3}) = \mathcal{O}(\varepsilon^{-11/3})$ .
- When  $n = \Theta(\varepsilon^{-1.5})$ , E-BSpiderBoost achieves a better sample complexity of  $\mathcal{O}(\varepsilon^{-3.25})$  than  $\mathcal{O}(\varepsilon^{-4.5})$  from MSVR-V2 [19].
- When  $n = \Theta(\varepsilon^{-2})$ , we recover  $\mathcal{O}(\varepsilon^{-3.5})$  sample complexity as in Theorem 4.
Proof: Denote  $G^{t + 1} = G_{\mathrm{E - BSB}}^{t + 1}$  (9). As we are using the finite-sum variant of SpiderBoost for the outer loop of the CSO problem, we only need to change the (17) and (18) to reflect that the outer variance is 0 now instead of  $\frac{\sigma_{\mathrm{out}}^2}{B_1}$  in the general CSO case. More concretely, we update (17) to
$$
\begin{array}{l} \mathcal {E} _ {\text {v a r}} ^ {t + 1} = \mathbb {E} [ \| G ^ {t + 1} - \mathbb {E} [ G ^ {t + 1} ] \| _ {2} ^ {2} ] \\ \leq \left(1 - p _ {\text {o u t}}\right) \mathbb {E} \left[ \left\| G _ {S} ^ {t + 1} - \mathbb {E} \left[ G _ {S} ^ {t + 1} \right] \right\| _ {2} ^ {2} \right] + p _ {\text {o u t}} \mathbb {E} \left[ \left\| G _ {L} ^ {t + 1} - \mathbb {E} \left[ G _ {L} ^ {t + 1} \right] \right\| _ {2} ^ {2} \right] \\ = \frac {\left(1 - p _ {\text {o u t}}\right)}{B _ {2}} \mathbb {E} \left[ \left\| G _ {\mathrm {E - B S G D}} ^ {t + 1} - G _ {\mathrm {E - B S G D}} ^ {t} - \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t + 1} - G _ {\mathrm {E - B S G D}} ^ {t} \right] \right\| _ {2} ^ {2} \right] + \frac {p _ {\text {o u t}}}{B _ {1}} \mathbb {E} \left[ \left\| G _ {\mathrm {E - B S G D}} ^ {t + 1} - \mathbb {E} \left[ G _ {\mathrm {E - B S G D}} ^ {t + 1} \right] \right\| _ {2} ^ {2} \right] \\ \leq \frac {1 - p _ {\text {o u t}}}{B _ {2}} \mathbb {E} \left[ \left\| G _ {\mathrm {E} - \mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {E} - \mathrm {B S G D}} ^ {t} - \mathbb {E} \left[ G _ {\mathrm {E} - \mathrm {B S G D}} ^ {t + 1} - G _ {\mathrm {E} - \mathrm {B S G D}} ^ {t} \right] \right\| _ {2} ^ {2} \right] + \frac {p _ {\text {o u t}}}{B _ {1}} \frac {\sigma_ {\mathrm {i n}} ^ {2}}{m}. \tag {20} \\ \end{array}
$$
and change (18) to
$$
\mathcal {E} _ {\text {v a r}} ^ {1} = \mathbb {E} [ \| G ^ {1} - \mathbb {E} [ G ^ {1} ] \| _ {2} ^ {2} ] = \mathbb {E} [ \| G _ {L} ^ {1} - \mathbb {E} [ G _ {L} ^ {1} ] \| _ {2} ^ {2} ] = \frac {1}{B _ {1}} \mathbb {E} [ \| G _ {\mathrm {E - B S G D}} ^ {1} - \mathbb {E} [ G _ {\mathrm {E - B S G D}} ^ {1} ] \| _ {2} ^ {2} ] \leq \frac {1}{B _ {1}} \frac {\sigma_ {\mathrm {i n}} ^ {2}}{m}. \tag {21}
$$
Then our analysis only has to start from the updated version of (19)
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla F (\boldsymbol {x} ^ {t}) \| _ {2} ^ {2} ] \leq \frac {2 (F (\boldsymbol {x} ^ {0}) - F ^ {\star})}{\gamma T} + \frac {2 \tilde {\sigma} _ {\text {b i a s}} ^ {2}}{m ^ {4}} + \left(\frac {1}{T p _ {\text {o u t}}} + 1\right) \frac {8 4}{B _ {1}} \frac {\sigma_ {\text {i n}} ^ {2}}{m}.
$$
We would like all terms on the right-hand side to be bounded by  $\varepsilon^2$ . From  $\frac{2\tilde{\sigma}_{\mathrm{bias}}^2}{m^4} \leq \varepsilon^2$  we know that
$$
m = \Omega (\frac {\tilde {\sigma} _ {\mathrm {b i a s}} ^ {1 / 2}}{\varepsilon^ {1 / 2}}).
$$
From  $\left(\frac{1}{Tp_{\mathrm{out}}} + 1\right)\frac{84}{B_1}\frac{\sigma_{\mathrm{in}}^2}{m} \leq \varepsilon^2$ , we know that
$$
m = \Omega (\frac {\sigma_ {\mathrm {i n}} ^ {2}}{n \varepsilon^ {2}}).
$$
From  $\frac{2(F(\pmb{x}^0) - F^\star)}{\gamma T}\leq \varepsilon^2$  , we can choose that
$$
\gamma = \mathcal {O} \left(\frac {1}{L _ {F}}\right), \quad T = \Omega \left(\frac {L _ {F} \left(F \left(\boldsymbol {x} ^ {0}\right) - F ^ {\star}\right)}{\varepsilon^ {2}}\right).
$$
Now the total sample complexity for E-BSpiderBoost for the FCCO problem becomes
$$
B _ {2} m T = \mathcal {O} \left(L _ {F} ^ {2} (F (\pmb {x} ^ {0}) - F ^ {\star}) \max \left\{\frac {\sqrt {n} \tilde {\sigma} _ {\mathrm {b i a s}} ^ {1 / 2}}{\varepsilon^ {2 . 5}}, \frac {\sigma_ {\mathrm {i n}} ^ {2}}{\sqrt {n} \varepsilon^ {4}} \right\}\right).
$$
E.2 Convergence of NestedVR
NestedVR Algorithm. We start by describing the NestedVR construction. We maintain states  $\mathbf{y}_i^{t+1}$  and  $\mathbf{z}_i^{t+1}$  to approximate
$$
\boldsymbol {y} _ {i} ^ {t + 1} \approx \mathbb {E} _ {\eta | i} \left[ g _ {\eta} \left(\boldsymbol {x} ^ {t}\right) \right], \quad \boldsymbol {z} _ {i} ^ {t + 1} \approx \mathbb {E} _ {\tilde {\eta} | i} \left[ \nabla g _ {\tilde {\eta}} \left(\boldsymbol {x} ^ {t}\right) \right].
$$
In iteration  $t + 1$ , if  $i$  is selected, then the state  $\pmb{y}_i^{t + 1}$  is updated as follows
$$
\boldsymbol {y} _ {i} ^ {t + 1} = \left\{ \begin{array}{l l} \frac {1}{S _ {1}} \sum_ {\eta \in H _ {i}} g _ {\eta} (\boldsymbol {x} ^ {t}) & \text {w i t h p r o b .} p _ {\text {i n}} \\ \boldsymbol {y} _ {i} ^ {t} + \frac {1}{S _ {2}} \sum_ {\eta \in H _ {i}} (g _ {\eta} (\boldsymbol {x} ^ {t}) - g _ {\eta} (\boldsymbol {\phi} _ {i} ^ {t})) & \text {w i t h p r o b .} 1 - p _ {\text {i n}}, \end{array} \right.
$$
where  $\phi_i^t$  is the last time node  $i$  is visited. If  $i$  is not selected, then
$$
\boldsymbol {y} _ {i} ^ {t + 1} = \boldsymbol {y} _ {i} ^ {t}.
$$
In this case,  $\pmb{y}_i^{t + 1}$  was never used to compute  $\nabla f_{i}(\pmb{y}_{i}^{t + 1})$  because  $i$  is not selected at the time  $t + 1$ . We use the following quantities
$$
\hat {\boldsymbol {z}} _ {i} ^ {t + 1} = \mathbb {E} _ {\bar {\eta} | i} [ \nabla g _ {\bar {\eta}} (\boldsymbol {x} ^ {t}) ], \quad \boldsymbol {z} _ {i} ^ {t + 1} = \frac {1}{m} \sum_ {\bar {\eta} \in \hat {H} _ {i}} \nabla g _ {\bar {\eta}} (\boldsymbol {x} ^ {t}). \tag {22}
$$
We use  $G_{\mathrm{NVR}}^{t + 1}$  as the actual updates,
$$
G _ {\mathrm {N V R}} ^ {t + 1} = \left\{ \begin{array}{l l} \frac {1}{B _ {1}} \sum_ {i \in \mathcal {B} _ {1}} (\boldsymbol {z} _ {i} ^ {t + 1}) ^ {\top} \nabla f _ {i} (\boldsymbol {y} _ {i} ^ {t + 1}) & \text {w i t h p r o b .} p _ {\text {o u t}} \\ G _ {\mathrm {N V R}} ^ {t} + \frac {1}{B _ {2}} \sum_ {i \in \mathcal {B} _ {2}} ((\boldsymbol {z} _ {i} ^ {t + 1}) ^ {\top} \nabla f _ {i} (\boldsymbol {y} _ {i} ^ {t + 1}) - (\boldsymbol {z} _ {i} ^ {t}) ^ {\top} \nabla f _ {i} (\tilde {\boldsymbol {y}} _ {i} ^ {t})) & \text {w i t h p r o b .} 1 - p _ {\text {o u t}}. \end{array} \right.
$$
We can also use the following quantity  $\hat{G}_{\mathrm{NVR}}^{t + 1}$  as an auxiliary
$$
\hat {G} _ {\mathrm {N V R}} ^ {t + 1} = \left\{ \begin{array}{l l} \frac {1}{B _ {1}} \sum_ {i \in \mathcal {B} _ {1}} (\hat {\boldsymbol {z}} _ {i} ^ {t + 1}) ^ {\top} \nabla f _ {i} (\boldsymbol {y} _ {i} ^ {t + 1}) & \text {w i t h p r o b . p _ {\mathrm {o u t}}} \\ \hat {G} _ {\mathrm {N V R}} ^ {t} + \frac {1}{B _ {2}} \sum_ {i \in \mathcal {B} _ {2}} ((\hat {\boldsymbol {z}} _ {i} ^ {t + 1}) ^ {\top} \nabla f _ {i} (\boldsymbol {y} _ {i} ^ {t + 1}) - (\hat {\boldsymbol {z}} _ {i} ^ {t}) ^ {\top} \nabla f _ {i} (\tilde {\boldsymbol {y}} _ {i} ^ {t})) & \text {w i t h p r o b . 1 - p _ {\mathrm {o u t}}}. \end{array} \right.
$$
Here we use  $\tilde{\pmb{y}}_i^t$  to represent an i.i.d. copy of  $\pmb{y}_i^t$  where  $i$  is selected at time  $t$ .
The iterate  $x^{t + 1}$  is therefore updated
$$
\boldsymbol {x} ^ {t + 1} = \boldsymbol {x} ^ {t} - \gamma G _ {\mathrm {N V R}} ^ {t + 1}.
$$
Lemma 9 The error between  $G_{NVR}^{t + 1}$  and  $\hat{G}_{NVR}^{t + 1}$  can be upper bounded as follows
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \left\| G _ {\mathrm {N V R}} ^ {t + 1} - \hat {G} _ {\mathrm {N V R}} ^ {t + 1} \right\| _ {2} ^ {2} \right] \leq \frac {1}{B _ {1}} \frac {C _ {f} ^ {2} \sigma_ {g} ^ {2}}{m} + \frac {4 (1 - p _ {o u t})}{B _ {2} m p _ {o u t}} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left(\mathbb {E} [ \| G _ {i} ^ {t + 1} - G _ {i} ^ {t} \| _ {2} ^ {2} ]\right).
$$
Proof: In this proof, we ignore the subscript in  $G_{\mathrm{NVR}}^{t + 1}$  and  $\hat{G}_{\mathrm{NVR}}^{t + 1}$ , we bound the error between  $G^{t + 1}$  and associated  $\hat{G}^{t + 1}$  where
$$
G _ {i} ^ {t + 1} = \left(\frac {1}{m} \sum_ {\tilde {\eta} \in \tilde {H} _ {i}} \nabla g _ {\tilde {\eta}} (\boldsymbol {x})\right) ^ {\top} \nabla f _ {i} (\boldsymbol {y} _ {i} ^ {t + 1}),
$$
$$
\hat {G} _ {i} ^ {t + 1} = \left(\mathbb {E} _ {\tilde {\eta} | i} [ \nabla g _ {\tilde {\eta}} (\boldsymbol {x}) ]\right) ^ {\top} \nabla f _ {i} \left(\boldsymbol {y} _ {i} ^ {t + 1}\right).
$$
Let's only consider the expectation over the randomness of  $\nabla g_{\tilde{\eta}}$
$$
\begin{array}{l} \mathbb {E} _ {H _ {i}} \left[\left\| G _ {i} ^ {t + 1} - \hat {G} _ {i} ^ {t + 1} \right\| _ {2} ^ {2} \right] \leq \mathbb {E} _ {\tilde {\eta} | i} \left[ \right.\left\| \right.\left( \right.\frac {1}{m} \sum_ {\tilde {\eta} \in \tilde {H} _ {i}} \nabla g _ {\tilde {\eta}} (\boldsymbol {x}) - \mathbb {E} _ {\tilde {\eta} | i} [ \nabla g _ {\tilde {\eta}} (\boldsymbol {x})) ] \left. \right\| _ {2} ^ {2} \left. \right] \mathbb {E} [ \| \nabla f _ {i} (\boldsymbol {y} _ {i} ^ {t + 1}) \| _ {2} ^ {2} ] \\ \leq \frac {C _ {f} ^ {2}}{m} \mathbb {E} _ {\tilde {\eta} | i} \left[ \left\| \nabla g _ {\tilde {\eta}} (\boldsymbol {x}) - \mathbb {E} _ {\tilde {\eta} | i} [ \nabla g _ {\tilde {\eta}} (\boldsymbol {x})) ] \right\| _ {2} ^ {2} \right] \\ \leq \frac {C _ {f} ^ {2} \sigma_ {g} ^ {2}}{m}. \\ \end{array}
$$
Then we can bound the error as follows