SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[18] N. Golowich, S. Pattathil, and C. Daskalakis. Tight last-iterate convergence rates for no-regret learning in multi-player games. NeurIPS, 2020.
[19] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NeurIPS, 2014.
[20] E. Gorbunov, H. Berard, G. Gidel, and N. Loizou. Stochastic extragradient: General analysis and improved rates. In AISTATS, 2022.
[21] E. Gorbunov, N. Loizou, and G. Gidel. Extragraphient method: O (1/k) last-iterate convergence for monotone variational inequalities and connections with cocoercivity. In AISTATS, 2022.
[22] E. Gorbunov, S. Horváth, P. Richtárik, and G. Gidel. Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top. *ICLR*, 2023.
[23] E. Gorbunov, A. Taylor, S. Horvath, and G. Gidel. Convergence of proximal point and extragradient-based methods beyond monotonicity: the case of negative comonotonicity. ICML, 2023.
[24] R. Gower, O. Sebbouh, and N. Loizou. Sgd for structured nonconvex functions: Learning rates, minibatching and interpolation. In AISTATS, 2021.
[25] R. M. Gower, N. Loizou, X. Qian, A. Sailanbayev, E. Shulgin, and P. Richtárik. Sgd: General analysis and improved rates. In ICML, 2019.
[26] F. Hanzely and P. Richtárik. Accelerated coordinate descent with arbitrary sampling and best rates for minibatches. In AISTATS, 2019.
[27] S. Horváth and P. Richtárik. Nonconvex variance reduced optimization with arbitrary sampling. In ICML, 2019.
[28] Y.-G. Hsieh, F. Iutzeler, J. Malick, and P. Mertikopoulos. On the convergence of single-call stochastic extra-gradient methods. NeurIPS, 2019.
[29] Y.-G. Hsieh, F. Iutzeler, J. Malick, and P. Mertikopoulos. Explore aggressively, update conservatively: Stochastic extragrain methods with variable stepsize scaling. NeurIPS, 2020.
[30] A. Juditsky, A. Nemirovski, and C. Tauvel. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1(1):17-58, 2011.
[31] A. Khaled, O. Sebbouh, N. Loizou, R. M. Gower, and P. Richtárik. Unified analysis of stochastic gradient methods for composite convex and smooth optimization. Journal of Optimization Theory and Applications, 2020.
[32] G. M. Korpelevich. The extragradient method for finding saddle points and other problems. Matecon, 12:747-756, 1976.
[33] S. Lee and D. Kim. Fast extra gradient methods for smooth structured nonconvex-nonconcave minimax problems. NeurIPS, 2021.
[34] C. J. Li, Y. Yu, N. Loizou, G. Gidel, Y. Ma, N. Le Roux, and M. Jordan. On the convergence of stochastic extragradient for bilinear games using restarted iteration averaging. In AISTATS, 2022.
[35] T. Lin, C. Jin, and M. Jordan. On gradient descent ascent for nonconvex-concave minimax problems. In ICML, 2020.
[36] T. Lin, Z. Zhou, P. Mertikopoulos, and M. Jordan. Finite-time last-iterate convergence for multi-agent learning in games. In ICML, 2020.
[37] M. Liu, Y. Mroueh, J. Ross, W. Zhang, X. Cui, P. Das, and T. Yang. Towards better understanding of adaptive gradient algorithms in generative adversarial nets. *ICLR*, 2020.
[58] Z. Qu and P. Richtárik. Coordinate descent with arbitrary sampling i: Algorithms and complexity. Optimization Methods and Software, 31(5):829-857, 2016.
[59] P. Richtárik and M. Takáč. On optimal probabilities in stochastic coordinate descent methods. arXiv preprint arXiv:1310.3438, 2013.
[60] P. Richtárik and M. Takác. Stochastic reformulations of linear systems: algorithms and convergence theory. SIAM Journal on Matrix Analysis and Applications, 41(2):487-524, 2020.
[61] M. Rolinek and G. Martius. L4: Practical loss-based stepsize adaptation for deep learning. NeurIPS, 2018.
[62] E. K. Ryu, K. Yuan, and W. Yin. Ode analysis of stochastic gradient methods with optimism and anchoring for minimax problems. arXiv preprint arXiv:1905.10899, 2019.
[63] O. Sebbouh, N. Gazagnadou, S. Jelassi, F. Bach, and R. Gower. Towards closing the gap between the theory and practice of svrg. NeurIPS, 2019.
[64] S. Sokota, R. D'Orazio, J. Z. Kolter, N. Loizou, M. Lanctot, I. Mitliagkas, N. Brown, and C. Kroer. A unified approach to reinforcement learning, quantal response equilibria, and two-player zero-sum games. *ICLR*, 2023.
[65] M. V. Solodov and B. F. Svaiter. A hybrid approximate extragradients-proximal point algorithm using the enlargement of a maximal monotone operator. Set-Valued Analysis, 7(4):323-345, 1999.
[66] C. Song, Z. Zhou, Y. Zhou, Y. Jiang, and Y. Ma. Optimistic dual extrapolation for coherent non-monotone variational inequalities. In NeurIPS, 2020.
[67] S. U. Stich. Unified optimal analysis of the (stochastic) gradient method. arXiv preprint arXiv:1907.04232, 2019.
[68] R. Szlendak, A. Tyurin, and P. Richtárik. Permutation compressors for provably faster distributed nonconvex optimization. *ICLR*, 2022.
[69] Q. Tran Dinh, D. Liu, and L. Nguyen. Hybrid variance-reduced sgd algorithms for minimax problems with nonconvex-linear function. NeurIPS, 2020.
[70] S. Vaswani, F. Bach, and M. Schmidt. Fast and faster convergence of sgd for over-parameterized models and an accelerated perceptron. In AISTATS, 2019.
[71] S. Vaswani, A. Mishkin, I. Laradji, M. Schmidt, G. Gidel, and S. Lacoste-Julien. Painless stochastic gradient: Interpolation, line-search, and convergence rates. NeurIPS, 2019.
[72] J. Wang, T. Zhang, S. Liu, P.-Y. Chen, J. Xu, M. Fardad, and B. Li. Adversarial attack generation empowered by min-max optimization. NeurIPS, 2021.
[73] Y. Yu, T. Lin, E. V. Mazumdar, and M. Jordan. Fast distributionally robust learning with variance-reduced min-max optimization. In AISTATS, 2022.
We organize the Supplementary Material as follows: Section A discusses the existing literature related to our work. In Section B, we present some technical lemmas required for our analysis, while in Section C, we provide a simple problem where the bounded variance assumption does not hold. Then, in Section D, we provide the proofs of propositions related to Expected Residual. Next, Section E presents the proofs of the main theorems, while a proposition related to arbitrary sampling is proved in Section F. Finally, additional numerical experiments are presented in Section G.
1 Introduction 1
1.1 Main Contributions 2
2 Stochastic Reformulation of VIPs & Single-Call Extragradient Methods 3
3 Expected Residual 4
4 Convergence Analysis 6
4.1 Quasi-Strongly Monotone Problems 6
4.2 Weak Minty Variational Inequality Problems 8
5 Beyond Uniform Sampling 8
6 Numerical Experiments 9
6.1 Strongly Monotone Problems 9
6.2 Weak Minty Variational Inequality Problems 10
A Further Related Work 17
B Technical Preliminaries 19
C Example: A Problem where the Bounded Variance Condition not Hold 20
D Proofs of Results on Expected Residual 21
D.1 Proof of Lemma 3.2 21
D.2 Proof of Proposition 3.3 22
D.3 Proof of Proposition 3.4 23
E Main Convergence Analysis Results 24
E.1 Proof of Theorem 4.1 25
E.2 Proof of Theorem 4.3 26
E.3 Proof of Theorem 4.4 27
E.4 Proof of Theorem E.4 27
E.5 Proof of Theorem 4.5 33
F Further Results on Arbitrary Sampling 35
F.1 Proof of Proposition 5.1 35
G Numerical Experiments 37
G.1 More Details on the Numerical Experiments of Section 6 37
G.2 Additional Experiments 37
G.2.1 Strongly Monotone Quadratic Game: 38
G.2.2 Weak Minty VIPs Continued 38
The references necessary to motivate our work and connect it to the most relevant literature are included in the appropriate sections of the main body of the paper. In this section, we present a broader view of the literature, including more details on closely related work and more references to papers that are not directly related to our main results.
- Classes of Structured Non-monotone Operators. With an increasing interest in improved computational speed, first-order methods are the primary choice for solving VIPs. However, computation of an approximate first-order locally optimal solution of a general non-monotone VIP is intractable [13, 33]. It motivates us to exploit the additional structures prevalent in large classes of non-monotone VIPs. Recently [20, 28] provide convergence guarantees of stochastic methods for solving quasi-strongly monotone VIPs, while [29] for problems satisfying error-bound conditions. [14] defined the notion of a weak MVI (4) covering classes of non-monotone VIPs.
- Assumptions on Operator Noise. The standard analysis of stochastic methods for solving VIPs relies on bounded variance assumption. [8, 14, 28, 17] use bounded variance assumption (i.e.  $\mathbb{E}\| F_i(x) - F(x)\|^2\leq \sigma^2$  for all  $x$ ) while [52, 1] assume bounded operators for their analysis. However, there are examples of simple quadratic games that do not satisfy these conditions. It has motivated researchers to look for alternative/relaxed assumptions on distributions. [44] provides convergence of Stochastic Gradient Descent Ascent Method under Expected Cocoercivity. [29, 49] considered alternative assumptions for analyzing Stochastic Extragradient Methods that do not imply boundedness of the variance. However, there is no analysis of single-call extragradient methods without bounded variance assumption.
- Weak Minty Variational Inequalities. Numerous contemporary studies look to identify first-order methods for efficiently solving min-max optimization problems. It varies from simple convex-concave to nontrivial nonconvex nonconcave objectives. Though there has been a significant development in the convex-concave setting, [13] demonstrates that even finding local solutions are intractable for general nonconvex nonconcave objectives. Therefore, researchers seek to identify the structure of objective functions for which it is possible to resolve the intractability issues. [14] proposes the notion of non-monotonicity, which generalizes the existence of a Minty solution (i.e.,  $\rho = 0$  in (4)). This problem is known as weak Minty variational inequality in the literature. [14, 54] provides convergence guarantees of the Extragradient Method for weak Minty variational inequality. They establish a convergence rate of  $\mathcal{O}(1 / k)$  for the squared operator norm. [33] shows that it is possible to have an accelerated extragradient method even for non-monotone problems. Furthermore, [8] provides a convergence guarantee for the SOG with a complexity bound of  $\mathcal{O}(\varepsilon^{-2})$ . However, all papers exploring stochastic extragradient methods for solving weak Minty variational inequality consider bounded variance assumption [8, 14]. Moreover, all algorithms solving Weak Minty variational inequality require increasing batchsize. Recently, [55] introduced BCSEG+ which can solve weak minty variational inequality without increasing batchsize. BCSEG+ involves three oracle calls per iteration and addition of a bias-corrected term in the extrapolation step.
- Arbitrary Sampling Paradigm. As we mentioned in the main paper, the stochastic reformulation (6) of the original problem (1) allows us to analyze single-call extragradient methods under the arbitrary sampling paradigm. That is, provide a unified analysis for SPEG that captures multiple sampling strategies, including  $\tau$ -minibatch and importance samplings. An arbitrary sampling analysis of a stochastic optimization method was first proposed in the context of the randomized coordinate descent method for solving strongly convex functions in [59]. Since then, several other stochastic methods were studied in this regime, including accelerated coordinate descent algorithms [58, 26], randomized iterative methods for solving consistent linear systems [60, 41, 40], randomized gossip algorithms [39, 42], stochastic gradient descent (SGD) [25, 24], and variance reduced methods [57, 27, 31]. The first analysis of stochastic algorithms under the arbitrary sampling paradigm for solving variational inequality problems was proposed in [43, 44]. In [43, 44], the authors focus on algorithms like the stochastic Hamiltonian method, the stochastic gradient descent ascent, and the stochastic consensus optimization. These ideas were later extended to the case of Stochastic Extragradient by [20]. To the best of our knowledge, our work is the first that provides an analysis of single-call extragradient methods under the arbitrary sampling paradigm.
- Overparameterized Models and Interpolation. For a function  $f(x) \coloneqq \frac{1}{n} \sum_{i=1}^{n} f_i(x)$  we say that interpolation condition holds if there exists  $x^*$  such that  $\min_x f_i(x) = f_i(x^*)$  for all  $i \in [n]$  (or equivalently  $\nabla f_i(x^*) = 0$  for smooth convex functions) [24]. The interpolation condition is satisfied when the underlying models are sufficiently overparameterized [70]. Some known examples include deep matrix factorization and classification using neural networks [3, 61, 70]. The interpolated model structure enables SGD and other optimization algorithms to have faster convergence [24, 45, 15]. Inspired by this, one can extend the notion of the interpolation condition to operators. In this scenario, we say that the VIP (1) is interpolated if there exists solution  $x^*$  of (1) such that  $F_i(x^*) = 0$  for all  $i \in [n]$ . This concept has been explored for analyzing the stochastic extragradient method in [71, 34]. We highlight that our proposed theorems show fast convergence of SPEG in this interpolated regime (when  $\sigma_*^2 = 0$ ). To the best of our knowledge, our work is the first that proves such convergence for SPEG. In Fig. 2a, we experimentally verify the fast convergence for solving a strongly monotone interpolated problem.
- Deterministic Extragradient Methods. The Extragradient method (EG) [32] and its single-call variant, Optimistic Gradient (OG) [56], were proposed to overcome the convergence issues of gradient descent-ascent method for solving monotone problems. Since their introduction, these methods have been revisited and explored in various ways. [50] analyzed EG and OG as an approximation of the Proximal Point method to solve bilinear and strongly convex-strongly concave min-max problems. [65] and [62] provide the best-iterate convergence guarantees of EG and OG with a rate of  $\mathcal{O}(1 / \kappa)$  for solving monotone problems. However, providing a last-iterate convergence rate of EG and OG for monotone VIPs has been a long-lasting open problem that was only recently resolved. The works of [18, 21, 10] prove a last-iterate  $\mathcal{O}(1 / \kappa)$  convergence rate for these methods. Finally, in the deterministic setting, some recent works provide convergence analysis of EG and OG for solving weak MVI (4) [14, 54, 8, 23].
Throughout our work, we assume
Assumption B.1. Operator  $F$  in (1) is L Lipschitz, i.e.,  $\forall x,y\in \mathbb{R}^d$  operator  $F$  satisfies
$$
\| F (x) - F (y) \| \leq L \| x - y \|. \tag {18}
$$
Operators  $F_{i}:\mathbb{R}^{d}\to \mathbb{R}^{d}$  of problem (1) are  $L_{i}$ -Lipschitz, i.e.,  $\forall x,y\in \mathbb{R}^d$  operator  $F_{i}$  satisfies
$$
\left\| F _ {i} (x) - F _ {i} (y) \right\| \leq L _ {i} \| x - y \|. \tag {19}
$$
In our proofs, we often use the following simple inequalities.
Lemma B.2. For all  $a, b, a_1, a_2, \dots, a_n \in \mathbb{R}^d$ ,  $n \geq 1, \alpha > 0$ , we have the following inequalities:
$$
\langle a, b \rangle \leq \| a \| \| b \|, \tag {20}
$$
$$
\langle a, b \rangle \leq \frac {1}{2 \alpha} \| a \| ^ {2} + \frac {\alpha}{2} \| b \| ^ {2}, \tag {21}
$$
$$
\left\| a + b \right\| ^ {2} \leq 2 \left\| a \right\| ^ {2} + 2 \left\| b \right\| ^ {2}, \tag {22}
$$
$$
\left\| a \right\| ^ {2} \geq \frac {1}{2} \left\| a + b \right\| ^ {2} - \left\| b \right\| ^ {2}, \tag {23}
$$
$$
\left\| \sum_ {i = 1} ^ {n} a _ {i} \right\| ^ {2} \leq n \sum_ {i = 1} ^ {n} \| a _ {i} \| ^ {2}. \tag {24}
$$
Inequality (22) is well known as Young's Inequality. Now, we present a simple property of unbiased estimators.
Lemma B.3. For an unbiased estimator  $g$  of operator  $F$  i.e.  $\mathbb{E}[g(x)] = F(x)$  we have
$$
\mathbb {E} \| g (x) - F (x) \| ^ {2} = \mathbb {E} \| g (x) \| ^ {2} - \| F (x) \| ^ {2}. \tag {25}
$$
Next, we present the following lemma from [67], which plays a vital role in proving the convergence guarantee of Theorem 4.4.
Lemma B.4. (Simplified Verison of Lemma 3 from [67]) Let the non-negative sequence  $\{r_k\}_{k\geq 0}$  satisfy the relation  $r_{k + 1}\leq (1 - a\gamma_k)r_k + c\gamma_k^2$  for all  $k\geq 0$ , parameters  $a,c\geq 0$  and any nonnegative sequence  $\{\gamma_k\}_{k\geq 0}$  such that  $\gamma_{k}\leq \frac{1}{h}$  for some  $h\geq a,h > 0$ . Then for any  $K\geq 0$  one can choose  $\{\gamma_k\}_{k\geq 0}$  as follows:
$$
\text {i f} K \leq \frac {h}{a}, \quad \gamma_ {k} = \frac {1}{h},
$$
$$
\text {i f} K > \frac {h}{a} \text {a n d} k <   k _ {0}, \quad \gamma_ {k} = \frac {1}{h},
$$
$$
\text {i f} K > \frac {h}{a} \text {a n d} k \geq k _ {0}, \quad \gamma_ {k} = \frac {2}{a (\kappa + k - k _ {0})},
$$
where  $\kappa = \frac{2h}{a}$  and  $k_{0} = \left\lceil \frac{K}{2}\right\rceil$ . For this choice of  $\gamma_{k}$  the following inequality holds:
$$
r _ {K} \leq \frac {3 2 h r _ {0}}{a} \exp \left(- \frac {a K}{2 h}\right) + \frac {3 6 c}{a ^ {2} K}.
$$
We use the next lemma to bound the trace of matrix products.
Lemma B.5. For positive semidefinite matrices  $A, B \in \mathbb{R}^{d \times d}$  we have
$$
\operatorname {t r} (A B) \leq \lambda_ {\max } (B) \operatorname {t r} (A), \tag {26}
$$
where  $\lambda_{\max}(B)$  denotes the maximum eigenvalue of  $B$
Next lemma proves equivalence of SPEG and SOG:
Proposition B.6 (Equivalence of SPEG and SOG). Consider the iterates of SPEG  $\{x_{k},\hat{x}_{k}\}_{k = 1}^{\infty}$  with constant step-sizes  $\omega_{k} = \omega ,\gamma_{k} = \gamma$  in (7). Then  $\hat{x}_k$  follows the iteration rule of SOG i.e.
$$
\hat {x} _ {k + 1} = \hat {x} _ {k} - \omega_ {k} F _ {v _ {k}} (\hat {x} _ {k}) - \gamma_ {k} \left[ F _ {v _ {k}} (\hat {x} _ {k}) - F _ {v _ {k - 1}} (x _ {k - 1}) \right] \tag {27}
$$
Proof. From the update rule of SPEG (7) we get
$$
\begin{array}{l} \hat {x} _ {k + 1} = x _ {k + 1} - \gamma F _ {v _ {k}} (\hat {x} _ {k}) \\ = x _ {k} - \omega F _ {v _ {k}} (\hat {x} _ {k}) - \gamma F _ {v _ {k}} (\hat {x} _ {k}) \\ = x _ {k} - (\omega + \gamma) F _ {v _ {k}} \left(\hat {x} _ {k}\right) \\ = \hat {x} _ {k} + \gamma F _ {v _ {k - 1}} (\hat {x} _ {k - 1}) - (\omega + \gamma) F _ {v _ {k}} (\hat {x} _ {k}) \\ = \hat {x} _ {k} - \omega F _ {v _ {k}} (\hat {x} _ {k}) - \gamma \left(F _ {v _ {k}} (\hat {x} _ {k}) - F _ {v _ {k - 1}} (\hat {x} _ {k - 1})\right). \\ \end{array}
$$
This shows that SPEG iterations are equivalent to SOG, with  $\hat{x}_k$  being the  $k$ -th iterate of SOG.
C Example: A Problem where the Bounded Variance Condition not Hold
Here, we provide a simple problem that does not satisfy the bounded variance assumption. Consider the linear regression problem
$$
\min  _ {x \in \mathbb {R}} f (x) := \frac {1}{2} (a _ {1} x - b _ {1}) ^ {2} + \frac {1}{2} (a _ {2} x - b _ {2}) ^ {2}
$$
where  $x \in \mathbb{R}$ . Here  $f_{1}(x) = (a_{1}x - b_{1})^{2}$  and  $f_{2}(x) = (a_{2}x - b_{2})^{2}$ . Now consider the estimator  $g(x)$  of  $\nabla f(x)$  under uniform sampling i.e.  $g(x)$  takes the value  $\nabla f_{1}(x)$  with probability  $\frac{1}{2}$  and  $\nabla f_{2}(x)$  with probability  $\frac{1}{2}$ . Then we have
$$
\begin{array}{l} \mathbb {E} \| g (x) - \nabla f (x) \| ^ {2} = \frac {1}{2} \| \nabla f _ {1} (x) - \nabla f (x) \| ^ {2} + \frac {1}{2} \| \nabla f _ {2} (x) - \nabla f (x) \| ^ {2} \\ = \frac {1}{2} \cdot \frac {1}{4} \| \nabla f _ {1} (x) - \nabla f _ {2} (x) \| ^ {2} + \frac {1}{2} \cdot \frac {1}{4} \| \nabla f _ {2} (x) - \nabla f _ {1} (x) \| ^ {2} \\ = \frac {1}{4} \| \nabla f _ {1} (x) - \nabla f _ {2} (x) \| ^ {2} \\ { = } { \frac { 1 } { 4 } \left( 2 ( a _ { 1 } x - b _ { 1 } ) a _ { 1 } - 2 ( a _ { 2 } x - b _ { 2 } ) a _ { 2 } \right) ^ { 2 } } \\ { = } { \left( ( a _ { 1 } ^ { 2 } - a _ { 2 } ^ { 2 } ) x - ( a _ { 1 } b _ { 1 } - a _ { 2 } b _ { 2 } ) \right) ^ { 2 } } \\ \end{array}
$$
Therefore,  $\mathbb{E}\| g(x) - \nabla f(x)\| ^2$  is a quadratic function of  $x$  with the coefficient of  $x$  being positive. Hence, as  $x\to \infty$  , we have  $\mathbb{E}\| g(x) - \nabla f(x)\| ^2\to \infty$  , which means that a constant can not bound the variance.
D Proofs of Results on Expected Residual
D.1 Proof of Lemma 3.2
Proof. Using Young's Inequality (22), we get
$$
\begin{array}{l} \mathbb {E} \| g (x) - F (x) \| ^ {2} \stackrel {(2 2)} {\leq} 2 \mathbb {E} \| g (x) - F (x) - g \left(x ^ {*}\right) \| ^ {2} + 2 \mathbb {E} \| g \left(x ^ {*}\right) \| ^ {2} \\ \stackrel {(\text {E R})} {\leq} \delta \| x - x ^ {*} \| ^ {2} + 2 \mathbb {E} \| g (x ^ {*}) \| ^ {2}. \\ \end{array}
$$
Then breaking down the RHS, we obtain
$$
\mathbb {E} \| g (x) \| ^ {2} - \| F (x) \| ^ {2} \stackrel {(2 5)} {\leq} \delta \| x - x ^ {*} \| ^ {2} + 2 \mathbb {E} \| g (x ^ {*}) \| ^ {2}.
$$
Now we rearrange the terms and set  $\sigma_{*}^{2} = \mathbb{E}\| g(x^{*})\|^{2}$  to complete the proof of this Lemma.
Proposition D.1. If  $F_{i}$  are  $L_{i}$ -lipschitz then Expected Residual condition (ER) holds. In that case
$$
\delta = \frac {2}{n} \sum_ {i = 1} ^ {n} L _ {i} ^ {2} \mathbb {E} (v _ {i} ^ {2}).
$$
In addition, if  $F$  is  $\mu$ -quasi strongly monotone (3) then we have
$$
\delta = \frac {2}{n} \sum_ {i = 1} ^ {n} L _ {i} ^ {2} \mathbb {E} \left(v _ {i} ^ {2}\right) - 2 \mu^ {2}.
$$
$$
\begin{array}{l} \mathbb {E} \| \left(F _ {v} (x) - F _ {v} \left(x ^ {*}\right)\right) - \left(F (x) - F \left(x ^ {*}\right)\right) \| ^ {2} = \mathbb {E} \| F _ {v} (x) - F _ {v} \left(x ^ {*}\right) \| ^ {2} + \| F (x) - F \left(x ^ {*}\right) \| ^ {2} \\ - 2 \mathbb {E} \left\langle F _ {v} (x) - F _ {v} \left(x ^ {*}\right), F (x) - F \left(x ^ {*}\right) \right\rangle \\ = \mathbb {E} \| F _ {v} (x) - F _ {v} \left(x ^ {*}\right) \| ^ {2} - \| F (x) - F \left(x ^ {*}\right) \| ^ {2} \\ = \mathbb {E} \| F _ {v} (x) - F _ {v} (x ^ {*}) \| ^ {2} - \| F (x) \| ^ {2} \\ { = } { \mathbb { E } \left\| \frac { 1 } { n } \sum _ { i = 1 } ^ { n } v _ { i } ( F _ { i } ( x ) - F _ { i } ( x ^ { * } ) ) \right\| ^ { 2 } - \| F ( x ) \| ^ { 2 } } \\ = \frac {1}{n ^ {2}} \mathbb {E} \left\| \sum_ {i = 1} ^ {n} v _ {i} \left(F _ {i} (x) - F _ {i} \left(x ^ {*}\right)\right) \right\| ^ {2} - \| F (x) \| ^ {2} \\ \stackrel {(2 4)} {\leq} \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbb {E} \left(v _ {i} ^ {2}\right) \| F _ {i} (x) - F _ {i} \left(x ^ {*}\right) \| ^ {2} - \| F (x) \| ^ {2} \\ \stackrel {(1 9)} {\leq} \frac {\| x - x ^ {*} \| ^ {2}}{n} \sum_ {i = 1} ^ {n} \mathbb {E} \left(v _ {i} ^ {2}\right) L _ {i} ^ {2} - \| F (x) \| ^ {2}. \tag {28} \\ \end{array}
$$
The first part of the lemma follows by ignoring the positive term  $\| F(x) \|^2$ . For the second part we assume  $F$  is  $\mu$ -quasi strongly monotone. Then we have
$$
\mu \| x - x ^ {*} \| ^ {2} \stackrel {(3)} {\leq} \langle F (x), x - x ^ {*} \rangle \stackrel {(2 0)} {\leq} \| F (x) \| \| x - x ^ {*} \|.
$$
Cancelling  $\| x - x^{*}\|$  from both sides we get
$$
\mu \| x - x ^ {*} \| \leq \| F (x) \|. \tag {29}
$$
Therefore we have the following bound for  $\mu$ -quasi strongly monotone operator  $F$ :
$$
\mathbb {E} \| \left(F _ {v} (x) - F _ {v} \left(x ^ {*}\right)\right) - \left(F (x) - F \left(x ^ {*}\right)\right) \| ^ {2} \stackrel {(2 8), (2 9)} {\leq} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \mathbb {E} \left(v _ {i} ^ {2}\right) L _ {i} ^ {2} - \mu^ {2}\right) \| x - x ^ {*} \| ^ {2}.
$$
This proves the second part of the lemma. This lemma ensures that the Lipschitz property is sufficient to guarantee Expected Residual (ER) condition.
D.2 Proof of Proposition 3.3
Proof. Proposition D.1 implies that Lipschitzness of all operators  $F_{i}$  is enough to ensure that ER holds. For  $\tau$ - minibatch sampling, denote the matrix  $\mathbf{R} = \left(F_1(x) - F_1(x^*),\dots ,F_n(x) - F_n(x^*)\right)\in \mathbb{R}^{d\times n}$ . Then we obtain the following bound: