SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yossi Arjevani, Yair Carmon, John C Duchi, Dylan J Foster, Nathan Srebro, and Blake Woodworth. Lower bounds for non-convex stochastic optimization. Mathematical Programming, 199(1-2): 165-214, 2023.
Hilal Asi, Yair Carmon, Arun Jambulapati, Yujia Jin, and Aaron Sidford. Stochastic bias-reduced gradient methods. In Advances in Neural Information Processing Systems, 2021.
Dimitris Bertsimas and Nathan Kallus. From predictive to prescriptive analytics. Management Science, 66(3):1025-1044, 2020.
Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM Review, 60(2):223-311, 2018.
Tianyi Chen, Yuejiao Sun, and Wotao Yin. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. In Advances in Neural Information Processing Systems, 2021.
Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin. A single-timescale method for stochastic bilevel optimization. In International Conference on Artificial Intelligence and Statistics, 2022a.
Xin Chen, Niao He, Yifan Hu, and Zikun Ye. Efficient algorithms for minimizing compositions of convex functions and random functions and its applications in network revenue management. arXiv preprint arXiv:2205.01774, 2022b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, 2009.
Priya Donti, Brandon Amos, and J Zico Kolter. Task-based end-to-end model learning in stochastic optimization. In Advances in Nneural Information Processing Systems, 2017.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, 2017.
Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246, 2018.
Saeed Ghadimi, Guanghui Lan, and Hongchao Zhang. Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization. Mathematical Programming, 155(1-2):267-305, 2016.
Michael B Giles. Multilevel Monte Carlo methods. Acta Numerica, 24:259-328, 2015.
Adam N Glynn and Kevin M Quinn. An introduction to the augmented inverse propensity weighted estimator. Political Analysis, 18(1):36-56, 2010.
Takashi Goda and Wataru Kitade. Constructing unbiased gradient estimators with finite variance for conditional stochastic optimization. Mathematics and Computers in Simulation, 204:743-763, 2023.
Paul Grigas, Meng Qi, and Zuo-Jun Max Shen. Integrated conditional estimation-optimization. arXiv preprint arXiv:2110.12351, 2021.
Zhishuai Guo and Tianbao Yang. Randomized stochastic variance-reduced methods for stochastic bilevel optimization. arXiv preprint arXiv:2105.02266, 2021.
Aman Sinha, Hongseok Namkoong, and John Duchi. Certifiable distributional robustness with principled adversarial training. In International Conference on Learning Representations, 2018.
Bokun Wang, Zhuoning Yuan, Yiming Ying, and Tianbao Yang. Memory-based optimization methods for model-agnostic meta-learning and personalized federated learning. Journal of Machine Learning Research, 24:1-46, 2023a.
Jie Wang, Rui Gao, and Yao Xie. Sinkhorn distributionally robust optimization. arXiv preprint arXiv:2109.11926, 2023b.
Pengwei Xing, Songtao Lu, Lingfei Wu, and Han Yu. Big-fed: Bilevel optimization enhanced graph-aided federated learning. IEEE Transactions on Big Data, 2022.
Jincheng Yang, Luhao Zhang, Ningyuan Chen, Rui Gao, and Ming Hu. Decision-making with side information: A causal transport robust approach. Optimization Online, 2022.
Junjie Yang, Kaiyi Ji, and Yingbin Liang. Provably faster algorithms for bilevel optimization. In Advances in Neural Information Processing Systems, 2021.
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hierarchically structured meta-learning. In International Conference on Machine Learning, 2019.
Luhao Zhang, Jincheng Yang, and Rui Gao. Optimal robust policy for feature-based newsvendor. Management Science, 2023.
A Proofs of Technical Results
Proof of Lemma 1. Since the function  $\psi$  is  $S_{\psi}$ -Lipschitz smooth, we have
$$
\begin{array}{l} \mathbb {E} \psi \left(x _ {t + 1}\right) - \psi \left(x _ {t}\right) \\ \leq \mathbb {E} \nabla \psi (x _ {t}) ^ {\top} (x _ {t + 1} - x _ {t}) + \frac {S _ {\psi}}{2} \mathbb {E} \| x _ {t + 1} - x _ {t} \| ^ {2} \\ = - \alpha_ {t} \mathbb {E} \nabla \psi (x _ {t}) ^ {\top} v (x _ {t}) + \frac {S _ {\psi} \alpha_ {t} ^ {2}}{2} \mathbb {E} \| v (x _ {t}) \| ^ {2} \\ \leq - \alpha_ {t} \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} + \alpha_ {t} \mathbb {E} \nabla \psi (x _ {t}) ^ {\top} (\nabla \psi (x _ {t}) - v (x _ {t})) \\ + S _ {\psi} \alpha_ {t} ^ {2} \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2} + S _ {\psi} \alpha_ {t} ^ {2} \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} \\ = - \left(\alpha_ {t} - S _ {\psi} \alpha_ {t} ^ {2}\right) \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} + \alpha_ {t} \mathbb {E} \nabla \psi (x _ {t}) ^ {\top} (\nabla \psi (x _ {t}) - v (x _ {t})) + S _ {\psi} \alpha_ {t} ^ {2} \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2} \\ = - \left(\alpha_ {t} - S _ {\psi} \alpha_ {t} ^ {2}\right) \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} + \alpha_ {t} \mathbb {E} \left[ \nabla \psi (x _ {t}) ^ {\top} \mathbb {E} \left[ \nabla \psi (x _ {t}) - v (x _ {t}) \mid x _ {t} \right] \right] \\ + S _ {\psi} \alpha_ {t} ^ {2} \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2} \\ \leq - \alpha_ {t} / 2 \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} + \alpha_ {t} \mathbb {E} [ \nabla \psi (x _ {t}) ^ {\top} \mathbb {E} [ \nabla \psi (x _ {t}) - v (x _ {t}) \mid x _ {t} ] ] + S _ {\psi} \alpha_ {t} ^ {2} \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2} \\ \leq - \alpha_ {t} / 2 \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} + \alpha_ {t} \mathbb {E} [ \| \nabla \psi (x _ {t}) \| \| \mathbb {E} [ \nabla \psi (x _ {t}) - v (x _ {t}) | x _ {t} ] \| ] + S _ {\psi} \alpha_ {t} ^ {2} \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2}, \\ \end{array}
$$
where the first inequality uses Lipschitz smoothness of  $\psi$ , the first equality uses the updates of the SGD algorithm, the second inequality uses the Cauchy-Schwarz inequality, the third equality uses the conditional expectation and the tower property, the third inequality uses the fact that  $\alpha_{t} \leq 1 / (2S_{\psi})$  and the fourth inequality uses the Cauchy-Schwarz inequality.
Rearranging terms and setting  $\alpha_{t} = \alpha$ , we have
$$
\begin{array}{l} \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} \leq \frac {2 (\mathbb {E} \psi (x _ {t}) - \mathbb {E} \psi (x _ {t + 1}))}{\alpha} \\ + 2 \mathbb {E} \| \nabla \psi (x _ {t}) \| \| \mathbb {E} [ \nabla \psi (x _ {t}) - v (x _ {t}) \mid x _ {t} ] \| + 2 S _ {\psi} \alpha \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2}. \\ \end{array}
$$
Averaging from  $t = 1$  to  $t = T$ , we have
$$
\begin{array}{l} \mathbb {E} \| \nabla \psi (\widehat {x} _ {T}) \| ^ {2} = \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} \| \nabla \psi (x _ {t}) \| ^ {2} \leq \frac {2 (\psi (x _ {1}) - \min _ {x} \psi (x))}{\alpha T} \\ + \frac {2}{T} \sum_ {t = 1} ^ {T} \mathbb {E} \| \nabla \psi (x _ {t}) \| \| \mathbb {E} [ \nabla \psi (x _ {t}) - v (x _ {t}) | x _ {t} ] \| + \frac {2 S _ {\psi} \alpha}{T} \sum_ {t = 1} ^ {T} \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2} \\ \leq \frac {2 (\psi (x _ {1}) - \min _ {x} \psi (x))}{\alpha T} + \frac {2}{T} \sum_ {t = 1} ^ {T} \left[ L _ {\psi} \| \mathbb {E} [ \nabla \psi (x _ {t}) - v (x _ {t}) ] \| + S _ {\psi} \alpha \mathbb {E} \| v (x _ {t}) - \nabla \psi (x _ {t}) \| ^ {2} \right], \\ \end{array}
$$
where the inequality holds as  $\psi$  is  $L_{\psi}$ -Lipschitz continuous and thus  $\| \nabla \psi (x)\| \leq L_{\psi}$  for all  $x$ .
To demonstrate the bias, variance as well as sampling and computational costs for building DL-SGD and RT-MLMC gradient estimators, we first show the iterate convergence of EpochsGD on the lower-level minimization problem with the side information. The analysis follows similarly as [Hazan and Kale, 2014, Asi et al., 2021].
Lemma 3 (Error of EpochsSGD). For given  $x$  and  $\xi$ , the iterates  $y_{K + 1}^{0}$  of Algorithm 1 with  $\beta_0 = (4\mu_g)^{-1}$  satisfies
$$
\mathbb {E} \left\| y _ {K + 1} ^ {0} - y ^ {*} (x; \xi) \right\| ^ {2} \leq 2 L _ {g, 0} ^ {2} \mu_ {g} ^ {- 2} 2 ^ {- (K + 1)}.
$$
It is important to note that the initial stepsize, denoted as  $\beta_0$ , doesn't necessarily have to be equal to  $(4\mu_g)^{-1}$ . Indeed, equivalent results can be achieved with a constant  $\beta_0 > 0$ . The choice of this
specific  $\beta_0$  value is primarily to streamline the analysis. In reality, there are numerous instances where the value of  $\mu_g$  is unknown beforehand. Under such circumstances, one practical approach could be to set  $\beta_0$  to a standard value, such as 0.4.
Proof of Lemma 3. Denote  $G(x,y;\xi) \coloneqq \mathbb{E}_{\eta|\xi} g(x,y;\eta,\xi)$ . For the ease of notation, throughout the proof,  $\mathbb{E}$  denotes taking full expectation conditioned on a given  $x$  and  $\xi$ . Utilizing the update of  $y$  in the  $k$ -th epoch of EpochsSGD algorithm, it holds for any  $y$  and any  $j = 0,\dots,2^k - 1$  that
$$
\begin{array}{l} \mathbb {E} \| y _ {k} ^ {j + 1} - y \| ^ {2} \\ = \mathbb {E} \| y _ {k} ^ {j} - \beta_ {k} \nabla_ {2} g (x, y _ {k} ^ {j}; \eta_ {k} ^ {j}, \xi) - y \| ^ {2} \\ = \mathbb {E} \| y _ {k} ^ {j} - y \| ^ {2} + \beta_ {k} ^ {2} \mathbb {E} \| \nabla_ {2} g (x, y _ {k} ^ {j}; \eta_ {k} ^ {j}, \xi) \| ^ {2} - 2 \beta_ {k} \mathbb {E} \nabla_ {2} g (x, y _ {k} ^ {j}; \eta_ {k} ^ {j}, \xi) ^ {\top} (y _ {k} ^ {j} - y) \\ \leq \mathbb {E} \| y _ {k} ^ {j} - y \| ^ {2} + \beta_ {k} ^ {2} L _ {g, 0} ^ {2} - 2 \beta_ {k} (\mathbb {E} G (x, y _ {k} ^ {j}; \xi) - G (x, y; \xi)), \\ \end{array}
$$
where the inequality utilizes the convexity of  $g$  and  $G$  in  $y$ . Rearranging terms, the above inequality yields
$$
\mathbb {E} G (x, y _ {k} ^ {j}; \xi) - G (x, y; \xi) \leq \frac {\mathbb {E} \| y _ {k} ^ {j} - y \| ^ {2} - \mathbb {E} \| y _ {k} ^ {j + 1} - y \| ^ {2}}{2 \beta_ {k}} + \frac {\beta_ {k} L _ {g , 0} ^ {2}}{2}.
$$
Summing up from  $j = 0$  to  $j = 2^k - 1$  and dividing  $2^k$  on both sides, we obtain the relation
$$
\begin{array}{l} \mathbb {E} G (x, y _ {k + 1} ^ {0}; \xi) - G (x, y; \xi) \\ \leq \frac {1}{2 ^ {k}} \sum_ {j = 0} ^ {2 ^ {k} - 1} \mathbb {E} G \left(x, y _ {k} ^ {j}; \xi\right) - G (x, y; \xi) \leq \frac {\mathbb {E} \| y _ {k} ^ {0} - y \| ^ {2}}{2 \beta_ {k} 2 ^ {k}} + \frac {\beta_ {k} L _ {g , 0} ^ {2}}{2}, \tag {12} \\ \end{array}
$$
where the inequality uses the convexity of  $G$  in  $y$  and Jensen's inequality.
Since  $G(x, y; \xi)$  is  $\mu_g$ -strongly convex in  $y$  for any given  $x$  and  $\xi$ , it holds that
$$
\begin{array}{l} G (x, y _ {1} ^ {0}; \xi) - G (x, y ^ {*} (x; \xi); \xi) \\ \leq - \nabla_ {2} G (x, y _ {1} ^ {0}; \xi) ^ {\top} \left(y ^ {*} (x; \xi) - y _ {1} ^ {0}\right) - \frac {\mu_ {g}}{2} \| y ^ {*} (x; \xi) - y _ {1} ^ {0} \| ^ {2} \\ \leq \max  _ {y} \left\{- \nabla_ {2} G (x, y _ {1} ^ {0}; \xi) ^ {\top} (y - y _ {1} ^ {0}) - \frac {\mu_ {g}}{2} \| y - y _ {1} ^ {0} \| ^ {2} \right\} \\ = \frac {\| \nabla_ {2} G (x , y _ {1} ^ {0} ; \xi) \| ^ {2}}{2 \mu_ {g}} \leq \frac {L _ {g , 0} ^ {2}}{2 \mu_ {g}}. \\ \end{array}
$$
Next, we use induction on  $k$  to show
$$
\mathbb {E} G (x, y _ {k} ^ {0}; \xi) - G (x, y ^ {*} (x; \xi); \xi) \leq \frac {L _ {g , 0} ^ {2}}{2 ^ {k} \mu_ {g}}.
$$
The base step for  $k = 1$  follows from the inequality established above. As for the induction step, suppose that
$$
\mathbb {E} G (x, y _ {k} ^ {0}; \xi) - G (x, y ^ {*} (x; \xi); \xi) \leq \frac {L _ {g , 0} ^ {2}}{2 ^ {k} \mu_ {g}}
$$
holds for some  $k \geq n$ . Plugging  $y = y^{*}(x;\xi)$  into (12), we thus find
$$
\begin{array}{l} \mathbb {E} G (x, y _ {k + 1} ^ {0}; \xi) - G (x, y ^ {*} (x; \xi); \xi) \leq \frac {\mathbb {E} \| y _ {k} ^ {0} - y ^ {*} (x ; \xi) \| ^ {2}}{2 \beta_ {k} 2 ^ {k}} + \frac {\beta_ {k} L _ {g , 0} ^ {2}}{2} \\ \leq \frac {\mathbb {E} G (x , y _ {k} ^ {0} ; \xi) - G (x , y ^ {*} (x ; \xi) ; \xi)}{\mu_ {g} \beta_ {k} 2 ^ {k}} + \frac {\beta_ {0} L _ {g , 0} ^ {2}}{2 ^ {K + 1}} \\ = \frac {\mathbb {E} G (x , y _ {k} ^ {0} ; \xi) - G (x , y ^ {*} (x ; \xi) ; \xi)}{\mu_ {g} \beta_ {0}} + \frac {\beta_ {0} L _ {g , 0} ^ {2}}{2 ^ {K + 1}} \\ \leq \frac {L _ {g , 0} ^ {2}}{\mu_ {g} ^ {2} \beta_ {0} 2 ^ {k}} + \frac {\beta_ {0} L _ {g , 0} ^ {2}}{2 ^ {k + 1}} \\ = \frac {L _ {g , 0} ^ {2}}{\mu_ {g} 2 ^ {k + 2}} + \frac {L _ {g , 0} ^ {2}}{\mu_ {g} 2 ^ {k + 3}} \\ \leq \frac {L _ {g , 0} ^ {2}}{\mu_ {g} 2 ^ {k + 1}}, \\ \end{array}
$$
where the second inequality uses the strong convexity of  $G$  in  $y$  and the fact that  $y^{*}(x;\xi)$  minimizes  $G$ , the first equality uses our assumption that  $\beta_{k} = \beta_{0} / 2^{k}$ , the third inequality uses the induction condition, and the second equality inequality uses the definition of  $\beta_0 = (4\mu_g)^{-1}$ . It concludes the induction. Therefore, we have
$$
\mathbb {E} G (x, y _ {K + 1} ^ {0}; \xi) - G (x, y ^ {*} (x; \xi); \xi) \leq \frac {L _ {g , 0} ^ {2}}{\mu_ {g} 2 ^ {K + 1}}.
$$
By the  $\mu_g$ -strong convexity of  $G(x,y;;\xi)$  and the fact that  $y^{*}(x;\xi)$  is the minimizer, we thus have
$$
\mathbb {E} \| y _ {K + 1} ^ {0} - y ^ {*} (x; \xi) \| ^ {2} \leq \frac {2}{\mu_ {g}} \mathbb {E} G (x, y _ {K + 1} ^ {0}; \xi) - G (x, y ^ {*} (x; \xi); \xi) \leq \frac {L _ {g , 0} ^ {2}}{\mu_ {g} ^ {2} 2 K}.
$$
Proof of Lemma 2. We first demonstrate the properties of the RT-MLMC gradient estimator and then show that of the DL-SGD gradient estimator. To facilitate the analysis, we define
$$
\begin{array}{l} V (x) = \mathbb {E} _ {\mathbb {P} _ {\xi}, \mathbb {P} _ {\eta | \xi}} \left[ \nabla_ {1} f (x, y ^ {*} (x; \xi); \eta , \xi) \right. \\ - \left(\mathbb {E} _ {\mathbb {P} _ {\eta^ {\prime} | \xi}} \nabla_ {1 2} ^ {2} g (x, y ^ {*} (x; \xi); \eta^ {\prime}, \xi)\right) \mathbb {E} _ {\{\eta_ {n} \} _ {n = 1} ^ {\widehat {N}} \sim \mathbb {P} _ {\eta | \xi}} [ \widehat {\Lambda} (x, y ^ {*} (x; \xi); \xi) ] \nabla_ {2} f (x, y ^ {*} (x; \xi); \eta , \xi) ]. \\ \end{array}
$$
RT-MLMC gradient estimator By the triangle inequality, we have
$$
\| \mathbb {E} \widehat {v} (x) - \nabla F (x) \| \leq \| \mathbb {E} \widehat {v} (x) - V (x) \| + \| V (x) - \nabla F (x) \|.
$$
From Chen et al. [2022b, Lemma 2.2], we know for given  $x, y$ , and  $\xi$  that
$$
\left\| \mathbb {E} _ {\left\{\eta_ {n} \right\} _ {n = 1} ^ {\widehat {N}} \sim \mathbb {P} _ {\eta | \xi}} \left[ \widehat {\Lambda} (x, y; \xi) \right] - \Lambda (x, y; \xi) \right\| \leq \frac {1}{\mu_ {g}} \left(1 - \frac {\mu_ {g}}{2 L _ {g , 1}}\right) ^ {N}
$$
Utilizing Lipschitz continuity of  $\nabla g$  and  $f$ , we know that
$$
\left\| V (x) - \nabla F (x) \right\| \leq \frac {L _ {g , 1} L _ {f , 0}}{\mu_ {g}} \left(1 - \frac {\mu_ {g}}{2 L _ {g , 1}}\right) ^ {N}.
$$
On the other hand, by the definition of  $\widehat{v} (x)$  , we have
$$
\begin{array}{l} \mathbb {E} \widehat {v} (x) - V (x) \\ = \mathbb {E} \nabla_ {1} f (x, y _ {K + 1} ^ {0}; \eta , \xi) - \mathbb {E} \nabla_ {1} f (x, y ^ {*} (x; \xi); \eta , \xi) \\ + \mathbb {E} \nabla_ {1 2} ^ {2} g (x, y _ {K + 1} ^ {0}; \eta , \xi) \left[ \widehat {\Lambda} (x, y _ {K + 1} ^ {0}; \xi) \right] \nabla_ {2} f (x, y _ {K + 1} ^ {0}; \eta , \xi) \\ - \mathbb {E} \nabla_ {1 2} ^ {2} g (x, y ^ {*} (x; \xi); \eta^ {\prime}, \xi) \left[ \widehat {\Lambda} (x, y ^ {*} (x; \xi); \xi) \right] \nabla_ {2} f (x, y ^ {*} (x; \xi); \eta^ {\prime}, \xi). \\ \end{array}
$$
By the Lipschitz continuity of  $\nabla f$  and Lemma 3, we have
$$
\begin{array}{l} \left\| \mathbb {E} \nabla_ {1} f (x, y _ {K + 1} ^ {0}; \eta , \xi) - \mathbb {E} \nabla_ {1} f (x, y ^ {*} (x; \xi); \eta , \xi) \right\| \\ \leq L _ {f, 1} \mathbb {E} \| y _ {K + 1} ^ {0} - y ^ {*} (x; \xi) \| \\ \leq \frac {L _ {f , 1} L _ {g , 0}}{\mu_ {g} 2 ^ {K / 2}}. \\ \end{array}
$$
Utilizing the Lipschitz continuity of  $\nabla f$ ,  $\nabla g$ , and  $\nabla^2 g$  in  $y$ , we have
$$
\begin{array}{l} \| \mathbb {E} \widehat {v} (x) - V (x) \| \\ \leq L _ {f, 1} \mathbb {E} \| y _ {K + 1} ^ {0} - y ^ {*} (x; \xi) \| + \frac {L _ {g , 1} L _ {f , 1} N}{L _ {g , 1}} \mathbb {E} \| y _ {K + 1} ^ {0} - y ^ {*} (x; \xi) \| + \frac {L _ {f , 0} L _ {g , 2} N}{L _ {g , 1}} \mathbb {E} \| y _ {K + 1} ^ {0} - y ^ {*} (x; \xi) \| \\ + L _ {f, 0} L _ {g, 1} \frac {N}{L _ {g , 1}} \sum_ {N ^ {\prime} = 1} ^ {N} \frac {1}{N ^ {\prime}} N ^ {\prime} \mathbb {E} \| y _ {K + 1} ^ {0} - y ^ {*} (x; \xi) \| \\ \leq \frac {L _ {f , 1} L _ {g , 0}}{\mu_ {g} 2 ^ {K / 2}} + \left(\frac {L _ {g , 1} L _ {f , 1} N}{L _ {g , 1}} + \frac {L _ {f , 0} L _ {g , 2} N}{L _ {g , 1}} + L _ {f, 0} L _ {g, 1} \frac {N ^ {2}}{L _ {g , 1}}\right) \frac {L _ {g , 0}}{\mu_ {g} 2 ^ {K / 2}} \\ = \mathcal {O} \left(\frac {N ^ {2}}{2 ^ {K / 2}}\right). \\ \end{array}
$$
Next, we show the sampling and computational costs. To build up the RT-MLMC gradient estimator  $\widehat{v} (x)$ , we need one sample of  $\xi$ , and the number of samples of  $\eta$  from  $\mathbb{P}_{\eta |\xi}$  is
$$
\sum_ {N ^ {\prime} = 1} ^ {N} \frac {1}{N ^ {\prime}} N ^ {\prime} + \sum_ {k = 1} ^ {K} (2 ^ {k + 1} - 1) \frac {2 ^ {- k}}{1 - 2 ^ {- K - 1}} <   N + 3 K.
$$
On average, the iteration needed for EpochSGD is
$$
\sum_ {k = 1} ^ {K} \left(2 ^ {k + 1} - 1\right) \frac {2 ^ {- k}}{1 - 2 ^ {- K - 1}} <   3 K.
$$
Next, we demonstrate the variance of  $\widehat{v} (x)$ . Denote
$$
\begin{array}{l} H _ {K} (1) := \nabla_ {1} f (x, y _ {K} ^ {0}; \eta^ {\prime \prime}, \xi), \\ H _ {K} (2) := \nabla_ {1 2} ^ {2} g (x, y _ {K} ^ {0}; \eta^ {\prime}, \xi) \left[ \widehat {\Lambda} (x, y _ {K} ^ {0}; \xi) \right] \nabla_ {2} f (x, y _ {K} ^ {0}; \eta^ {\prime \prime}, \xi), \\ H ^ {*} (1) := \nabla_ {1} f (x, y ^ {*} (x; \xi); \eta^ {\prime \prime}, \xi), \\ H ^ {*} (2) := \nabla_ {1 2} ^ {2} g (x, y ^ {*} (x; \xi); \eta^ {\prime}, \xi) \left[ \widehat {\Lambda} (x, y ^ {*} (x; \xi); \xi) \right] \nabla_ {2} f (x, y ^ {*} (x; \xi); \eta^ {\prime \prime}, \xi). \\ \end{array}
$$
Thus one may rewrite  $\widehat{v} (x)$  and  $V(x)$  as
$$
\begin{array}{l} \widehat {v} (x) = \frac {H _ {k + 1} (1) - H _ {k} (1) - H _ {k + 1} (2) + H _ {k} (2)}{p _ {k}} + H _ {1} (1) - H _ {1} (2), \\ V (x) = \mathbb {E} \left[ H ^ {*} (1) - H ^ {*} (2) \right]. \\ \end{array}
$$
It holds that
$$
\begin{array}{l} \mathbb {E} \| \widehat {v} (x) - \mathbb {E} \widehat {v} (x) \| ^ {2} \\ \leq \mathbb {E} \| \widehat {v} (x) - V (x) \| ^ {2} \\ \leq 2 \mathbb {E} \| \widehat {v} (x) - H _ {1} (1) + H _ {1} (2) \| ^ {2} + 2 \mathbb {E} \| H _ {1} (1) - H _ {1} (2) - V (x) \| ^ {2} \\ \leq 4 \mathbb {E} \left\| \frac {1}{p _ {k}} \left(H _ {k + 1} (1) - H _ {k} (1)\right) \right\| ^ {2} + 4 \mathbb {E} \left\| \frac {1}{p _ {k}} \left(H _ {k + 1} (2) - H _ {k} (2)\right) \right\| ^ {2} + 2 \mathbb {E} \| H _ {1} (1) - H _ {1} (2) - V (x) \| ^ {2}, \\ \end{array}
$$
where the first inequality holds by the definition of variance, the second inequality holds by the Cauchy-Schwarz inequality, the third inequality uses the Cauchy-Schwarz inequality and the definition of  $\widehat{v}(x)$ . It remains to analyze  $\mathbb{E}\left\| \frac{1}{p_k} (H_{k+1}(1) - H_k(1)) \right\|^2$ ,  $\mathbb{E}\left\| \frac{1}{p_k} (H_{k+1}(2) - H_k(2)) \right\|^2$ , and  $\mathbb{E}\| H_1(1) - H_1(2) - V(x)\|^2$ .
- For the first term, we have
$$
\begin{array}{l} \mathbb {E} \left\| \frac {1}{p _ {k}} \left(H _ {k + 1} (1) - H _ {k} (1)\right) \right\| ^ {2} \\ = \sum_ {k = 1} ^ {K} p _ {k} ^ {- 1} \mathbb {E} \| H _ {k + 1} (1) - H _ {k} (1) \| ^ {2} \\ \leq \sum_ {k = 1} ^ {K} p _ {k} ^ {- 1} L _ {f, 1} ^ {2} \mathbb {E} \| y _ {k + 1} ^ {0} - y _ {k} ^ {0} \| ^ {2} \\ \leq \sum_ {k = 1} ^ {K} p _ {k} ^ {- 1} L _ {f, 1} ^ {2} 2 \left(\mathbb {E} \left\| y _ {k + 1} ^ {0} - y ^ {*} (x; \xi) \right\| + \mathbb {E} \left\| y ^ {*} (x; \xi) - y _ {k} ^ {0} \right\| ^ {2}\right) \\ \leq \sum_ {k = 1} ^ {K} p _ {k} ^ {- 1} L _ {f, 1} ^ {2} \frac {6 L _ {g , 0} ^ {2}}{\mu_ {g} ^ {2} 2 ^ {k}} \\ \leq K L _ {f, 1} ^ {2} \frac {6 L _ {g , 0} ^ {2}}{\mu_ {g} ^ {2}}, \\ \end{array}
$$
where the first inequality uses the Lipschitz continuity of  $\nabla f$ , the second inequality uses the Cauchy-Schwarz inequality, the third inequality uses Lemma 3, and the last inequality uses the definition of  $p_k$ .
- For the second term, we may conduct a similar analysis.
$$
\begin{array}{l} \mathbb {E} \left\| \frac {1}{p _ {k}} \left(H _ {k + 1} (2) - H _ {k} (2)\right) \right\| ^ {2} \\ = \sum_ {k = 1} ^ {K} p _ {k} ^ {- 1} \mathbb {E} \| H _ {k + 1} (2) - H _ {K} (2) \| ^ {2} \\ \leq \sum_ {k = 1} ^ {K} p _ {k} ^ {- 1} 6 \left(L _ {f, 1} ^ {2} N ^ {2} + \frac {L _ {f , 0} ^ {2} L _ {g , 2} ^ {2} N ^ {2}}{L _ {g , 1} ^ {2}} + L _ {f, 0} ^ {2} N ^ {4}\right) \mathbb {E} \| y _ {K + 1} ^ {0} - y _ {K} ^ {0} \| ^ {2} \\ \leq \sum_ {k = 1} ^ {K} p _ {k} ^ {- 1} 2 \left(\frac {L _ {g , 0} ^ {2}}{\mu_ {g} ^ {2} 2 ^ {K}} + \frac {L _ {g , 0} ^ {2}}{\mu_ {g} ^ {2} 2 ^ {K - 1}}\right) 6 \left(L _ {f, 1} ^ {2} N ^ {2} + \frac {L _ {f , 0} ^ {2} L _ {g , 2} ^ {2} N ^ {2}}{L _ {g , 1} ^ {2}} + L _ {f, 0} ^ {2} N ^ {4}\right) \\ = \frac {3 6 K L _ {g , 0} ^ {2}}{\mu_ {g} ^ {2}} \left(L _ {f, 1} ^ {2} N ^ {2} + \frac {L _ {f , 0} ^ {2} L _ {g , 2} ^ {2} N ^ {2}}{L _ {g , 1} ^ {2}} + L _ {f, 0} ^ {2} N ^ {4}\right), \\ \end{array}
$$
where the first inequality follows utilizing Lipschitz continuity of  $f$ ,  $\nabla f$ ,  $\nabla g$ ,  $\nabla^2 g$  in  $y$ , the second inequality uses Lemma 3, and the last inequality uses the definition of  $p_k$ .
- For the third term, we have
$$
\begin{array}{l} \mathbb {E} \| H _ {1} (1) - H _ {1} (2) - V (x) \| ^ {2} \\ \leq 2 \mathbb {E} \| H _ {1} (1) - \mathbb {E} H ^ {*} (1) \| ^ {2} + 2 \mathbb {E} \| H _ {1} (2) - \mathbb {E} H ^ {*} (2) \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} \mathbb {E} \| H _ {1} (1) - \mathbb {E} H ^ {*} (1) \| ^ {2} \\ = \mathbb {E} \| \nabla_ {1} f (x, y _ {1} ^ {0}; \eta , \xi) - \mathbb {E} \nabla_ {1} f (x, y _ {1} ^ {0}; \eta , \xi) \| ^ {2} \\ + \mathbb {E} \| \mathbb {E} \nabla_ {1} f (x, y _ {1} ^ {0}; \eta , \xi) - \mathbb {E} \nabla_ {1} f (x, y ^ {*} (x; \xi); \eta , \xi) \| ^ {2} \\ \leq \sigma_ {f} ^ {2} + \frac {L _ {f , 1} ^ {2} L _ {g , 0} ^ {2}}{2 \mu_ {g} ^ {2}}, \\ \end{array}
$$
where the first equality holds as  $\mathbb{E}\|a + b\|^2 = \mathbb{E}\|a\|^2 + \mathbb{E}\|b\|^2 + 2\mathbb{E}a^\top b$ , and last inequality holds by Lemma 3 and the Lipschitz continuity of  $\nabla f$ . On the other hand, we have
$$
\begin{array}{l} \mathbb {E} \left\| H _ {1} (2) - \mathbb {E} H ^ {*} (2) \right\| ^ {2} \\ = \mathbb {E} \| H _ {1} (2) - \mathbb {E} H _ {1} (2) \| ^ {2} + \mathbb {E} \| \mathbb {E} H _ {1} (2) - \mathbb {E} H ^ {*} (2) \| ^ {2} \\ \leq \mathbb {E} \| H _ {1} (2) \| ^ {2} + \mathbb {E} \| \mathbb {E} H _ {1} (2) - \mathbb {E} H ^ {*} (2) \| ^ {2} \\ \leq L _ {g, 1} ^ {2} L _ {f, 0} ^ {2} \sum_ {\widehat {N} = 1} ^ {N} \frac {1}{\widehat {N}} \frac {N ^ {2}}{4 L _ {g , 1}} + 6 \left(L _ {f, 1} ^ {2} N ^ {2} + \frac {L _ {f , 0} ^ {2} L _ {g , 2} ^ {2} N ^ {2}}{L _ {g , 1} ^ {2}} + L _ {f, 0} ^ {2} N ^ {4}\right) \frac {L _ {g , 0} ^ {2}}{2 \mu_ {g} ^ {2}} \\ = L _ {g, 1} ^ {2} L _ {f, 0} ^ {2} \frac {N ^ {2}}{4 L _ {g , 1}} + 6 \left(L _ {f, 1} ^ {2} N ^ {2} + \frac {L _ {f , 0} ^ {2} L _ {g , 2} ^ {2} N ^ {2}}{L _ {g , 1} ^ {2}} + L _ {f, 0} ^ {2} N ^ {4}\right) \frac {L _ {g , 0} ^ {2}}{2 \mu_ {g} ^ {2}}, \\ \end{array}
$$
where the first equality uses  $\mathbb{E}\|a + b\|^2 = \mathbb{E}\|a\|^2 + \mathbb{E}\|b\|^2 + 2\mathbb{E}a^\top b$ , and last inequality holds by Lemma 3 and the Lipschitz continuity.
As a result, we conclude that the variance satisfies that
$$
\mathbb {E} \| \widehat {v} (x) - \mathbb {E} \widehat {v} (x) \| ^ {2} \leq \mathbb {E} \| \widehat {v} (x) - V (x) \| ^ {2} = \mathcal {O} (K N ^ {4}).
$$
DL-SGD gradient estimators Note that  $\mathbb{E}\widehat{v} (x) = \mathbb{E}\widehat{v}^{K}(x)$ . Thus the bias follows directly from the analysis of RT-MLMC.