Optimal Extragradient-Based Algorithms for Stochastic Variational Inequalities with Separable Structure
Authors: Angela Yuan
Chris Junchi Li
Gauthier Gidel
Michael Jordan
Quanquan Gu
Simon Shaolei Du
Venue: NeurIPS 2023 poster (2023)
Keywords: Stochastic variational inequalities
convex-concave separable saddle-point optimization
extragradient-based algorithm
Nesterov's acceleration
scheduled restarting
scaling reduction

Abstract: We consider the problem of solving stochastic monotone variational inequalities with a separable structure using a stochastic first-order oracle. Building on standard extragradient for variational inequalities we propose a novel algorithm---stochastic \emph{accelerated gradient-extragradient} (AG-EG)---for strongly monotone variational inequalities (VIs). Our approach combines the strengths of extragradient and Nesterov acceleration. By showing that its iterates remain in a bounded domain and applying scheduled restarting, we prove that AG-EG has an optimal convergence rate for strongly monotone VIs. Furthermore, when specializing to the particular case of bilinearly coupled strongly-convex-strongly-concave saddle-point problems, including bilinear games, our algorithm achieves fine-grained convergence rates that match the respective lower bounds, with the stochasticity being characterized by an additive statistical error term that is optimal up to a constant prefactor.

Optimal Extragradient-Based Algorithms for Stochastic Variational Inequalities with Separable Structure
Huizhuo Yuan
Chris Junchi Li
Gauthier Gidel
Michael I. Jordan 1.c Quanquan Gu Simon S. Du
Department of Computer Science, University of California, Los Angeles {hzyuan, qgu)@ca. ucla.edu
Department of Electrical Engineering and Computer Sciences, University of California, Berkeley (junchili, jordan) @erddnn)ess.. θcs berkeley edu DIRO, Université de Montréal and Mila gauthier.gize www.-gidellloooee Sandee@@mmntreel.coomm 4 Department Statistics, University of California, Berkeley Paul G. Allen School of Computer Science and Engineering, University of Washington ssdu0cs. washington. edu
Abstract
We consider the problem of solving stochastic monotone variational inequalities with a separable structure using a stochastic first-order oracle. Building on standard extragradient for variational inequalities we propose novel algorithm- -tsochatm-ttcchattii accelerated
(AG-EG)- strongly monotone variational inequalities (VIs). Our approach combines the strengths of extragradient and Nesterov acceleration. By showing that its iterates remain in a bounded domain and applying scheduled restarting. we prove that AG-EG has an optimal convergence rate for strongly monotone VIs. Furthermore, when specializing to the particular case of bilinearly coupled coongly-covvex-strone -wroosty-cooooaaanne saddle- point problems, including bilinear games, our algorithm achieves fine-grained convergence rates that match the respective lower bounds, with the stochasticity being characterized by additive statistical error term that is optimal up to a constant prefactor.
Introduction
The variational inequality (VI) problem plays central role in wide range of optimization problems with convex structure, including convex minimization, saddle-point problems, and games [Facchinei
and Pang 2003 Nemirovski, 2004 Nemirovski al. 2009 Juditsky al. 2011 Jordan 2023
A general VI problem aims to find solution x" E 2 that satisfies:
(W(="), x" - z) < 0, Vz Z,
(I)
where 2 is finitedimmeniinal closed and convex feasible and W(-) is monotone operator in the following form:
W(z) + +V+
(2)
where F is continuously differentiable with L-Lipschitz continuous gradient and is j-strongly convex, 7 an M-Lipschitz monotone operator, J' E O. is the subgradient simple and convex function, & and ( are drawn from distributions De and Dç. respectively. This formulation captures a separable structure in which H usually models the competing forces in a system, and J models
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

nonsmooth factor. In addition, we consider the stochastic setting where we can only access V.F and H. through their unbiased estimators (=:: and H(( () respectively.
notable instance of the VI problem | with separable structure is the widely studied bilinearly concaave saddle-poing problem:
coupled
min max #ER" yERm
$ (x. y) = F(x) + ((, y) - G(y) = Eg (f(=; E) + E5 [1( y) - Eg (g E)
(3)
where (x, y) x By - x T u + u y is the bilinear coupling function with the coupling
matrix B E Raxm. Note that 3 is a special instance of (I) when taking F(z) = F(z) + G(y).
H(() (x: y): vyH H y)] and J = O. In addition to wide range of applications in
economics, problems of form 3 are becoming increasingly important in machine learning. For instance, appears in reinforcement learning, differentiable games, regularized empirical risk minimization, and robust optimization formulations. It can also be seen as a local approximation of the nonconvex-nonconeee
minimax games-e.g. the generative adversarial network (GAN)
Goodfellow et al. 2020 -around local Nash equilibrium [Mescheder et al.) 2017
Nagarajan and
Kolter 2017
In this paper, we aim to improve the efficiency of solving ! by utilizzig the structural information of the monotone operator in 2 More specifically, consider the case when F is strongly monotone, or zero. Although optimal convergence results have been obtained for the monotone VI problem ① CCen al., 2017 as well as the special case of convex-concave saddle-point problem with bilinear coupling [Chn et al. 2014). it remains open how to design an optimal algorithm for the strongly monotone VI problem. Notably, for the special case when F and/or G' are strongly convex, several concurrent works have independently obtained the optimal convergence
rates (Kovaeev al. 2022 [hekuuparampii
al., 2022, Tin al. 2022 Metelev et al. 2022 Li
et al. 2022b]. On the other hand, when both F and G are zero, optimal convergence results have been obtained by Li al. [2022a] and the accelerated gradient optimistic gradient approach (Li et al.
20226 We defer more complete overview of related work to the appendix.
1.1
Main Contributions
We start with the strongly monotone VI problem in an unbounded feasible set, extending the scope of recent work such as Jin et al. 2022] and going beyond earlier studies that focus on nonstrongly monotone VIs in bounded feasible set (Juditsky et al. 2011 Chen al. 2017 | We propose class of algorithms named stochastic accelerated gradient-extree -exeeet--exrrrraiien (AG-EG), which combine Nesterov acceleration with the extragradient method. By employing either strong-convexity shifting technique or scheduled restarting scheme, our algorithm achieves convergence rates that match the lower bounds for the general strongly monotone VI problem ( the special SC-SC saddle-point problem (3. and bilinear games, in both deterministic and stochastic settings, thus providing a unified optimal solution. In sharp contrast to the accelerated mirror-prox (AMP) algorithm proposed by Chen et al. 2017]. Jordan et al. 2202)] our analysis does not rely on the boundedness of the feasible set 2, which makes our algorithm projection-free free. We also extend our algorithm to VIs with bounded feasible set and/or nondifferentiable convex regularization through proximal mapping. We summarize our contributions as follows:
(1) We present direct approach for separable strongly monotone VIs, where the iteration complexity
lower bound due to Zhang et al. [2022] is matched O I + _ log (44). which
admits sharp near-unity coefficient 2.. Theorem 2.3 Here o² is the weighted, uniform variance bound on the stochastic gradient and stochastic operator.
(2) We also present a stochastic AG-EG algorithm equipped with scheduled restarting, which achieves the sharpest possible iteration complexity of O ( ((VE+ + log (2) + x=62 for finding an e-optimal point. The deterministic part matches the complexity lower bound in Zhang al. 2022), while the stochastic part matches the optimal statistical error.
+
'IIs in an unbounded Feasible set is more difficult to solve because existing algorithms and analyses crucially rely on the boundedness of the feasible set.
2

---

When specializing the VI problem to bilinearly coupled SC-SC saddle-poing problems, our results have the following implications:
(SC-SC) Saddle-Point Problem.
For the class of SC-SC equipped with scaling
saddle-point problems, the stochastic AG-EG descent-ascent Algorithm reduction, achieves an iteration complexity of
o o(lv
!
LE
Lc
Amaux
log
+ P
(4)
where F R -> R is Lp-smooth and Hrr-tlyngly convex, R - R is L-g-smooth and HGGsstonggyy convex. When the optimization problem deterministic, the complexity upper bound matches the lower bound of Zhang et al. 2022 3.1 Corollary 2.8
Bilinear Games. For bilinear games (V (x: 0 and Vg 0 almost surely), Algorithm equipped with scheduled restarting achieves an iteration complexity of
Ameri (B B) log OVV
v Amto
(BB Amar
B)
Aminn (BBT) 5
Arnin BB Je
where Ti is the variance of the stochastic gradient on the bilinear coupling term. When there no randomness, this complexity result reduces to O Amace B)
log (1) for bilinear games,
Xmin BB
matching the lower bound of Ibrahim et al. 2020
[ 3.2 Corollary 3.3
Organization.
The rest of this paper is organized as follows. Section proposes the Accelerated Descent-Ascent algorithm for strongly monotone Vls. showing that it achieves an accelerated convergence rate, extending to VIs with bounded domains with proximal operator. Section 3 discusses two specific instances of saddle-poin problems, where our proposed AG-EG algorithm has a convergence rate that matches the corresponding lower bounds. Finally, Section summarizes our results and suggests future directions.
Notation. Let Amax (M) (resp. Amin (M) be the largest (resp. smallest) eigenvalue a real symmet- ric matrix M. Let V max(a, b) (resp. AA min(a, b)) denote the maximum (resp. minimum) value of two reals a, b. For two nonnegative real sequences (a,) and (b,). we write a, = (b,)) On S 6n (resp. an l(bb,) an Z b,) to denote an I C'b,, (resp. an 2 Cb,) for all 2 1 for a positive, numerical constant C, and let an x bn if both an S bn and a, Z bn hold. We also let a, = Ô (b,,) denote a, < Cb, where C' hides polylogarithmic factor in problem-dependent constants. We let [x; y] E R concatenate two vectors x R and y R" Finally for two real symmetric matrices A and B, we denote A < B (resp. A Z B) when v T(A B)v s 0 (resp. v (A - B) 2 0) holds for all vectors v.
2 Accelerated Gradient-Extragradien Descent- Algorithm
In this section, we focus on accelerating the extragradient algorithm for the strongly monotone VI problem in (I with separable structure 2 Our algorithm design draws inspiration from the work
Chen et al. [2017] on the stochastic Accelerated MirrorProx (AMP) algorithm for nonstrongly monotone VIs. The AMP algorithm applies Nesterov-type acceleration top of the mirror- prox method Korpelevich, 1976 Nemirovski 2004 and attains the optimal iteration complexity of
o (V M theory for the AMP algorithm can only deal with VIs with bounded domain. Our algorithm not only achieves the optimal convergence rates for the strongly monotone VI problem with separable structure but we also remove the dependency on the diameter of the feasible Therefore, our algorithm can deal with VIs with unbounded domains.
However, the big-O notation hides the diameter the feasible set, and the existing
For the function class of bilinear games, we assume that m where B is a nonsingular square matrix, that Amin (BB and the complexity makes sense. See 3.2 for more on this.

---

Throughout 2 we maintain conceptual simplicity by presenting all our algorithm designs in the deterministic setting, while presenting the convergence results in the more general stochastic setting. These results can be easily reduced to the deterministic setting when the stochastic noise vanishes.
2.1 Setting and Assumptions
In this section, we formally introduce our assumptions. We first state the smoothness and monotonicity assumptions that we impose on F and H.
Assumption 2.1 (Monotonicity, strong convexity and smoothness) We assume that function F()) contimuously differentiable with L-Lipschitz continuous gradient and is p-strongly convex. That is, for any z, z' 2,
$ - 2112 S -(
Furthermore, operator H(--) is monotone and M-Liipschizz in the sense thatfor any 2, z' E 2,
(H(2) - H(2²), = 2 0, [[H(z)) - H(=')l s M||= - =
Second, we impose assumptions on the noise variance.
Assumption 2.2 (Unbiased gradients and variance bounds) We assume that 2 E Z, samples E Dg and ( De are drawn from given distrihutions such that the following conditions hold: Eg [v( E)] = FF(x), E. [M((; 5)1 = H(z), and
Eg [V:((z))) {= Str.
Ee -(izz)))HH()))) S Bil
(6)
For all results in this work, we suppose that Assumptions 2.1 and 2.2 hold with appropriate parameter settings. Given desired accuracy E > 0, our goal to find an e-optmmal point defined as:
Definition 2.1 (s-Optimal point) A point z E Z is called an e-optimal point for the VI problem in if l E.
2.2 The ExtraGradient (EG) Algorithm
We first consider the case where Z is the entire space R" and the objective is smooth (J 0). The extragradient (EG) algorithm, introduced by Korpelevich (1976). is designed to address cyclic behavior in saddle- point problems by introducing an extrapolated point for gradient evaluation. In the context of VI problems (I) let 2, represents the t-th iterate of the EG algorithm. The update rule of EG is as follows:
xt+ = 21 - - n - (x)()))
where 7j > 0 the step size. For a L-smooth and ju-strongly monotone operator W. Isng (1955, Mokhtari et al. 2020 Gidel et al. [019a]] have shown that the EG algorithm achieves an iteration complexity of (O log L/=)), where x L/u denotes the condition number of the problem.
2.3 Accelerating the ExtraGradient Algorithm, Direct Approach
The convergence rate of the EG algorithm is far from optimal for the strongly monotone VI problem in ( with separable structure 2 Firstly, the update rule in takes W whole without utilizing the separable structure. This prevents us from exploiting the properties of VJ. Secondly, in the case of bilinear games, the established lower bound for EG is 2 D(vR lolloo//))) rather than Q(x log This discrepancy highlights the potential for accelerating the EG algorithm in various directions. We first rewrite the EG update rule 7 as follows:
2 = 2t-1 (> 21-1 17 (H((zz-1) (((x)))),
2, 2 1) - 1
21-1
(z }))
(8)
To accelerate the process based on V.F. we consider Nesterov's 's second acceleration scheme on
minimizing a single convex function F [Tseng] 2008 Lan and Zhou 2018 Lin et al. 2020c
md -
(1 az)2 + 24 zt-1
VF md
86 1),
2,
(1 az) 88 + 04
(9)
a

---

where ar is the extrapolation step size in the standard three-line Nesterov scheme. Here we adopt the notation 2n and 2 indicate the middle point and the aggregated point (Chen al. 2017]. respectively. Next, to achieve acceleration, we replace the gradient of VJ evaluated at both 2;-1 and - in by the gradient evaluated at the extrapolated point z in O Furthermore, we shift the index of 23d by 1/2 to indicate the use of 2 instead of in the THd update in 2 In addition, take into account the j4-strong convexity of JF and shift the gradient of the strongly convex part V= V= (H(z) u(z - zu]), we obtain the following update rule for a direct version of an accelerated EG
zall?] = p²z zo) from F=)) to H(z) as W(z) (VF((z) - ju(z - zo)) +
algorithm (different step size schemes for 77 are required for different algorithmic designs):
md t = (1 a,)= + a4
2 2:-1 77 (H((z--1) VF(2t11 p1 z(-1)),
(10)
21 = 1 (H(Z,- ;/ V= md [1x - x
28 -- = - -1-aa) r0 + 1.
We call the algorithm in (T)) the accelerated prudient-extragradde direct approach AGGEG-DDirect and postpone its full description to Algorithm 2 in $C.T The final output of the direct approach is 2T after T iterates. The following theorem records the convergence rate and iteration complexity of AG-EG (direct approach).
Theorem 2.3 (Convergence stochastic AG-EG, direct approach) Suppose Assumptions
2.
and 2.2 hold. Fix any r (0, E (0, xo), let Ki 4 a and set the step size upper bound "+v1+TMg" For any sequence of step sizes an (0, a] and mp. at the iterates of
stochastic AG-EG (direct approach) satisfy that for all t 1,.... T, we have
E
2" 2²11
2|20
2²1 (+ + +1+1
II a.) +
30 112
0
2
II ar)) r=st1
(11)
where we define σ
v3 - + (2 4)
In the rest of the paper, we use the same definition a as in Theorem 2.3 The proof of Theorem
provided in $D.4. We further note that one possible choice of step size is to let a, such that reduces
E || 2"11 S = 2=11 ( + +1] e aat +
30²
For any given T 2 1, by choosing the optimal a $ (1 (1+ log ā. implies
1TT ( +1) || -211())
30
E| B| xz 211 < 2|2 - 2²1 2112((t +1) &T
+
30² NºT
(1+ (1+log
2T 302
(+ +1) 11 - x))))
Prescribing the desired accuracy E > 0, the iteration complexity output e-optimal minimax point is
((V M m2:z) log ((# +11+1 ||zo -=0 ==)
We conjecture that the logarithmic factor in the optimal statistical rate p²aa is removable using a proper diminishing step size, a possibility that we reserve for future study. In the setting of deterministic optimization, setting 0 and r - 1-, 3 -> 0+ in Theorem 2.3
x²
we obtain the
optimal iteration complexity bound as follows:
1 +
1
2 n
log ((+ + 11)
(12)
'Throughout this work, we focus on the iteration complexity whereas the required number of queries to the stochastic gradient oracle three times the iteration complexity.
5

---

Algorithm Stochastic AccelrrieeGGraiieeeEEEtaaCaaiat (AG-BG) Descent-Ascent Algorithm, with Scheduled Restarting
Require: Initialization 20 total number of epochs y 2 1, total number of per-epoch iterates (T' :
stepsizes (at, Tet ..... for s = 1,2, _ do
Set 20 t 20
8-
[--1] 20 + 20 20 < =0=0 [--1)
for = 1,,,...
To do
Draw samples = De from oracle, and also 4 Se ~ D5 independently from oracle
= t x
1: + (
ag + (1 - at x=)2²=1 + a
=
t 11-1 ?t| M( (((zz-);s)) + VF md 1i
mnd f (1 ac+1 ) + a4+121
end for
Set 20
+ +=T,-t T
\/Warm-start using the output of the previous epoch)
end for |
Output: 20
Remark 2.4 Our complexity bounds fundamentally differs from the previous analysis [Chen et al. 2017 Jordan et al. 2023) for separable smooth (strongly) monotone VIs. The convergence results in previous studies are dependent on the diameter of the domain, whereas our convergence rate is independent of the domain parameters and eliminates the need for projection onto bounded domain. Moreover, our contributions go beyond those of Chen et al.) [2017] by extending the analysis to the strongly monotone case. In comparison with Jordan et al. [2023]. we design an algorithm where V.F strongly monotone and resolve the open problem of extending the analysis to the stochastic case. Additionally, our complexity bound in (12) indicates a near-unity coefficient on the condition-number exponent, improving the corresponding coefficient in Chen al. 2017 Theorem 15] by an asymptotic factor of 4.
The direct approach, which reduces to EG when VF and μ = 0, falls short of attaining optimality within the specific regime of bilinear games. In the next subsection, will introduce new algorithm that can overcome this limitation.
2.4 Accelerating the ExtraGradient Algorithm with Scheduled Restarting
In this subsection, solve problem ①I by further accelerating the stochastic EG algorithm. Rather than directly relying on the strong monotonicity of V.F, the inner updates our new algorithm identical to the updates in 10 with μ 0. Due to the domain-indeeendeen nature of
analysis, we can apply the scheduled restarting technique (O donoghue and Candes 2013] Roulet
and d' Aspremont] 2017 Renegar and Grimmer 2022 to the outer loop, accelerating the algorithm from sublinear convergence to linear convergence. In addition, the output of our algorithm is the aggregated point = after T' iterates. We present the full algorithm in Algorithm T--
We first present the convergence rate of a single epoch (i.e., the inner loop) of Algorithm | in Theorem 2.5 To accommodate more flexibility in the choice of parameters, introduce three constants r, B, and C' in the theorem statement.
Theorem 2.5 (Convergence of stochastic AG-EG, one epoch) Suppose Assumptions 2.7 and 2.2 hold. For any fixed epoch length T 2 1. any constant r E (0, 1), 8 E (0, C (0,00) choose step sizes a₂ प. and qh such that
1+8 Mt, 1 = = B+
(13)

---

where B oVT(T+1)
The output -1- - a single epoch of Algorithm I satisfies
E
2
AAV 1+3 p(T+1)
x M) B B| 2²11²
M
2 uVT
E E
2=11?
T-
(14)
where the prefactor = 1++ S + C² reduces to when o 0.
The proof of Theorem 2.5 is provided in D.3 We make few remarks Theorem 2.5 follows:
Remark 2.6 In the setting of deterministic optimization, by taking = 0, + 17, + 0+ in our
analysis, with step size choice Tt 21.+M1' we obtain that
ag
211
2 MTT+T (7
M) 11)l =211;
(15)
In this setting, the algorithm is independent of B and requires knowledge of ||0 2 _ In the face of stochasticity, we choose C. = 1 when the initial distance to the optimal point is known. Alternatively, when only an over-estimate To of vTl22z - 2²2 is available, we can set (large enough) C
2- Do 2 1 to obtain
2L
101 p(T+1)
rT + 2V
1+B M
r +
uVT
4a 1'o
(16)
Remark 2.7 When the constants are concern, the coarse-grained choices of and 3 = would suffice. Nevertheless, to optimize the constants, the tradeoff between the deviation of from and B from 0 is crucial, it determines balance between the stochastic gradient noise variance and the convergence rate coefficients.
To prepare for our multi-epoch result with the help of scheduled restarting, we perform an induction based (66] as follows. Supposing that E|126--1 |s-1| 1, we have
18e--s hold, by taking and 201
E E
[x]
11
13 NT!
+
M -gee-
PVT, Toe
Setting the right-hand side of the above inequality to satisfy S Ie and solving for T2, need the epoch length satisfies T, as
L M
p Thus, we can obtain the total iteration complexity
p
M
(V
M S
8-1
V
where S log
This yields the following multi-epoch iteration complexity bound:
Corollary 2.8 (Iteration complexity of stochastic AG-EG with scheduled restarting) Under the same condition of Theorem
2.5
the stochustic AG-EG with scheduled restarting in Algorithm 7 with
epoch length T. x
M
pa has a total iteration complexity of
((V
+ (V+++))
log (t)
(17)
Note that the hard instance constructed by Zhang et al. 2022 can be modified in a straightforward way to establish a lower bound of 5 ((V + log for our monotone VI demonstrating the optimality of Corollary 2.8 in the deterministic separable setting. An alternative optimality argument proceeds follows: the first term / matches the lower bound for the minimization of
strongly convex function Nesterov 2004]. and the second term M matches the lower bound for VI for non-strongly monotone operator when V.F =0 [Ouyang and Xu 202T]. This together gives
lower bound for solving monotone VI via similar argument by Thekumparampil et al. 2022
7

---

3 match It is worth noting that while both complexity bounds in Corollary 2.8 and Theorem
the lower bound Zhang al. [2022 for strongly monotone VIs with separable structure, the direct approach in 22. reduces to the last-iterate independent-sampll stochastic extragradient (SEG)
algorithm in bilinear games. Consequently, the deterministic part (a = 0) fails to match the lower bound in Ibrahim et al.
[220]. In the stochastic case with noise variance bounded away from zero,
the direct approach in 2.3 can exhibit nonconvergence behavior [Hsich et all 2020, $3J. The AG-EG algorithm in 2.44 this issue by restarting the average-iterate SEG, matching the lower bound
results (see $3.2 for more details). In addition, the complexity bound in [7 also eliminates the log
prefactor of the statistical error term nجم² compared to Theorem 2.3 The optimality of our algorithm
lies in not only the optimization complexity but also the statistical error rate
pr Here the e-optimal
point & is defined as ||= - x=1 < S
2.5 Extension of AG-EG to Proximal Algorithms
In previous subsections, we have focused on the case where the feasible set 2 represents the entire space and the nondifferentiable convex function J is dropped. We now extend the AG-EG algorithm and its analysis to the more general setting that has a bounded feasible set (via Euclidean projection onto the feasible set) as well as nondifferentiable convex regularization term (via a proximal operator). These settings are useful in various applications, such as the variational inequality on
the Lorentz cone where projection onto 2 = {ax, t) E (m+11) : || S {t} is required [Chen et al. 2017], and the two-player game that involves projection onto the probability simplex, among others. To deal with bounded feasible set 2, we adopt a variant of the EG algorithm, where we project the
extrapolated point and the main iterates back onto the feasible set Z of W:
2 - = Pz [=1-1 - r))((z = arg min (z 2t-1 21-1, 17 + 2 ||z - -===111??
ZE2
2, = Pz z1-1 m)( = arg min #E2 = 2 +))
+ 2 || z1-1 11. (18)
where P2(=)- = arg min ||> - =11 the Euclidean projection operator. To handle the nondif-
ferentiable simple convex function J, we replace the projection operator in (88) by the following proximal mapping defined via a Bregman divergence B(-,)):
prox = argmin (v, u- z) + B(z, u) + Juu). nEZ
(19)
In fact, 18 can be seen as a special case of 19 when choosing the Bregman divergence B(z, u)
b||z u and J(u) as the set indicator function of the feasible set Z. Therefore, by substituting the prox- prox-mapping 199 into the AG-EG updates introduced in $2.4 we obtain the more general proximal AG-EG algorithm in Algorithm 3 (See in $C.2). which reduces to Algorithm D when J= B(z, u) I - ull2 and 2 = R". Moreover, we assume that B(.,-) is Pgg-stronlyy convex. Without loss of generality, in contrast to the previous assumption of μ-strong convexity for F, we instead assume that F is /--troongyy convex with respect to the Bregman divergence B(.,.) (See, for example, Hazan and Kale [0144, Xu et al. (0018). Similar to Corollary 2.8
we have the following
iteration complexity result, whose proof is deferred to D.5
Corollary 2.9 (Iteration complexity of stochastic proximal AG-EG with scheduled restarting)
Under the same condition of Theorem 2.3 the stochastic proximal AG-EG with scheduled restarting +
in Algorithm with epoch length Tg = PPB of
o² B(z.x=)) has a total iteration complexity p² H'llggl--e
((V
L
M
log (!)
=²B(zn=²")
pps
For the deterministic case, proximal AG-EG with scheduled restarting has a total iteration complexity M log (t) to output an s-optimal point of I
of O ((V
L.
+
AMB
"The optimal statistical error rate 5or has been achieved by multistage algorithm in Fallah et al. 22020) where the s-optimal point is defined by || =12 < s. In our paper, the s-optimal point is defined by d² ||* - 1 S e. Therefore, our statistical error rate can be translated into , using their definition, which matches their result.
8

---

Implications for Specific Instances
In this section, we discuss the implications of our AG-EG algorithm and its convergence rates when applying to two instances of saddle-point problems.
3.1 Strongl-ComeexStrongy-CCoore
Saddle-Point Problem
For the stochastic bilinearly-coupled SC-SC saddle-point problem e we note that the smoothness and strong convexity parameters LF, LG- HFF, and μaG of F and G may differ. To accommodate these variations in curvature information, we employ scaling reduction technique. This technique enables us convert the SC-SC with equal strong convexity parameters for F and by reparametrizing the objective function. The same argument is also applicable to the direct approach.
In lieu of consider
min max y
F ÿ) F(x) H( v) G(D).
where f i) 9 (æ, y) with the symbolic reparamettiization a
il 9)
PF
H(x 9). Gig) G(y) and also their derivatives Vg v)
p2
V ( y), VŌ(Ú)
V RE VGy (the stochastic oracles h,g follow the same rule). is straightforward to verify that $ ÿ) is saatoongly-oonvexx--stttoo summarized by the rescaled updates on y::
The essence of our update rules can be
Ú. -1-1 The Vg 1. De + Vg (1(
y: = y 1-1 17t
BE
(v 3. 9. 3 (e) + (o(0))
Therefore, it suffices analyze Algorithm for f ú) and due this scaling reduction, we only need to prove all results for the case of =
μ.. It is also straightforward to justify corresponding scaling changes as: LF oF LG, M = pr Amax (
(B BB. and μ = μp. The following corollary is recovered by reverting the scaling reduction from $(2 9) to (x.
Corollary 3.1 (Iteration complexity of stochastic AG-EG on SC-SC saddle-point problem)
For solving G Algorithm with an epoch length T, v
Le
La or pa +
v
Amaz (B B
4PPG
M 1ge
has total iteration complexity of
((V LE
o
La +
Amaz (B PFPC
log (
In the deterministic case, the iteration complexity in Theorem 2.8 matches the lower bound established by Zhang et al. (2022), i.e., Ω
((v Le La
+
Amas (B
log (1)
Moreover, our algorithm
Prpc
achieves the optimal statistical rate of upea
q² up a constant prefactor.
Remark 3.2 A well-known finding regarding the second scheme of Nesterov acceleration is its connection to the primal-dual method [Lan and Zhou 2018 Lin et al. 2020cy. This finding has been incorporated into the design of the LPD algorithm [Thekumprampiil et al.) [222), where a Chambolle- Pock-l-yle instead of the extragradient used in this paper. The LPD algorithm [ also achieves the optimal complexity for the deterministic ciinearly-coupller saddle-point problem.
arimal-dual method is utilized
approximation of proximal point methods,
[Thekumprammil et al. (022]
3.2 Bilinear Games
In this subsection, we consider the particular case of bilinear games. We assume m such that B
is a nonsingular square matrix, V fxx [ E) = 0 and Vgly: {) = 0 a.s., so 3 reduces to
min max F (x. y) E& [h(x, y: ()] H(,, v) = x By x U + u y,
(20)

---

and Algorithm B reduces to the independent-sampee extragradient descent-ascent algorithm for 20 The saddle point [z" w in this case is the unique solution to the linear equation
Ug uvy
which has closed-form solution
B
-B
Uu
Our results imply the following iteration complexity for solving stochastic bilinear games.
Corollary 3.3 (Iteration complexity of stochastic AG-EG, bilinear games) For solving choose the step sizes Oi = -FI
20
2 and Tjs =
which case Algorithm 7 with an epoch
Vemax (B B)
length T3 = V
Amaz Amnin
has the total iteration complexity of
BBT
o
Amazz [B B) log (BBT)
V
(BB Amax (B B)
din Arnin (BB )e²
(21)
Amin
(BBB
Note that our choice of the step size is maximal and is independent of the noise. In the deterministic setting, letting Bil x E Ammin (BB >mmax B), the complexity bound in Corollary 3.3 reduces
O Amar (B B
log
which matches the lower bound in Ibrahim al. [2020] Notably,
Amin BB
Azizian et al. 20 proposed an algorithm achieving an upper bound that matches the lower bound in Ibrahim et al. 2020
Li et [222aa] also proposed a lower-bound matching SEG algorithm that uses shared sample in both steps under an unbounded noise assumption. In contrast, our algorithm in the msappeddent-sample setting with bounded noise variance.
Remark 3.4 Standard acceleration techniques do not attain the optimal nonasymptotic convergence rate for bilinear games [Gidel et al. 2019b]. This limitation applies various algorithms, including the direct approach 22.3%. as well as several other acceleration techniques (Thekumparampi et al. 2022 Kovalev al. 2022 Jin et al. 2022], all of which fall short of achieving optimal acceleration for bilinear games. Therefore, matching both lower bounds in a single algorithu in the general stochastic setting has been open problem. While Li et al.) (2022b) present algorithm that achieves both lower bounds a single algoritlmm, it relies on the use of optimistic gradients rather than extragradients on the bilinear coupling function. Farthermore, our algorithm and analysis is more general than those in Li et al. (2022b) as we can handle the general variational inequality with proximal operators.
4 Conclusions
We have presented a stochastic extragradient-basee acceleration algorithm, AG-EG, for solving stochastic monotone variational inequalities with separable structure. The iteration complexity of our algorithm matches the lower bound and is independent of the size of the feasible set. When specialized solving the bilinearly coupled saddle-point problem (3) our AG-EG algorithm simultaneously matches lower bounds due to Zhang et al. [2022] and Ibrahim et al. (2200) for strongly-convex strongly-concavy and bilinear games, respectively. To the best of our knowledge, this the first time that all three lower bounds have been met by a single algorithm. There are some remaining issues to be addressed, however, including the case of one-sided nonstrong convexity, the setting of unbounded noise variance, and the characterization of the full parameter regime dependency on Amin (BB) These are left as important directions for future research.
Acknowledgments and Disclosure of Funding
This work is supported in part by Canada CIFAR AI Chair to GG, by the Mathematical Data Science program of the Office of Naval Research under grant number N00014-18-1-2764 and also the Vannevar Bush Faculty Fellowship program under grant number N00014-21-1-294 and National Science Foundation (NSF) grant I1S-1901252 MIJ. This work is also supported in part by the NSF CAREER Award I 1906169 QG and by NSF Award's 10171 and DMS-2134100 SSD.
10

---

References
Waiss Azizian, loannis Mitliagkas, Simon Lacoste-Julien, and Gauthier Gidel. A tight and unified analysis of gradient-based based methods for whole spectrum of differentiable games. In International Conference on Artificial Intelligence and Statistics, pages 2863-2873, PMLR, 2020a.
Waiss Azizian, Damien Scieur, Ioannis Mitliagkas, Simon Lacoste-Julien, and Gauthier Gidel. Accelerating smooth games by manipulating spectral shapes, In International Conference on Artificial Intelligence and Statistics, pages 1705-1715. PMLR, 2020b.
Yunmei Chen, Guanghui Lan, and Yuyuan Ouyang. Optimal primal-dual methods for class of saddle point problems. SIAM Journal Optimization, 1774):1779-1814, 2014.
Yunmei Chen, Guanghui Lan, and Yuyuan Ouyang. Accelerated schemes for class variational inequalities. Mathematical Programming. 165(1):113-149, 2017.
Ziyi Chen, Qunwei Li, and Yi Zhou. Finding local minimax points via (stochastic) cubic-regularized gda: Global convergence and complexity. arXiv preprint arXiv:2110.07098, 2021.
Michael B Cohen, Aaron Sidford, and Kevin Tian. Relative Lipschitzness in extragradient meth- ods and direct recipe for acceleration. In 12th Innovations in Theoretical Computer Science Conference (IT'S 202/), volume 185, page 62, 2021.
Christoph Dann, Gerhard Neumann, Jan Peters, et al. Policy evaluation with temporal differences: survey and comparison. Journal Machine Learning Research, 15:809-883. 2014.
Constantinos Daskalakis, Andrew Ilyas, Vasilis Syrgkanis, and Haoyang Zeng. Training GANs with optimism. In International Conference on Learning Representations, 2018.
Simon S Du, Jianshu Chen, Lihong Li, Lin Xiao, and Dengyong Zhou. Stochastic variance reduction methods for policy evaluation. In International Conference on Machine Learning, pages 1049- 1058. PMLR, 2017.
Francisco Facchinei and Jong-Shi Pang. Finite-Dimersional Variational Inequalities and Comple- mentarity Problems. Springer, 2003.
Alireza Fallah, Asuman Ozdaglar, and Sarath Pattathil. An optimal multistage stochastic gradient method for minimax problems. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 3573-3579. IEEE, 2020.
Saeed Ghadimi and Guanghui Lan. Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization i: generic algorithmic framework. SIAM Journal on Optimization, 22((): 169-1192, 2012.
Gauthier Gidel. Hugo Berard, Gaëtan Vignoud. Pascal Vincent, and Simon Lacoste-Julies variational inequality perspective on generative adversarial networks. In International Conference on Learning Representations, 2019a.
Gauthier Gidel, Reyhane Askari Hemmat, Mohammad Pezeshki, Rémi Le Priol, Gabriel Huang, Simon Lacoste-Julien, and Ioannis Mitliagkas. Negative momentum for improved game dynamics. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1802-1811. PMLR. 2019b.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications the ACM, (3(11): (3(11):1399144, 2020.
Zhishuai Guo, Zhuoning Yuan, Yan Yan, and Tianbao Yang. Fast objective & duality gap convergence for nonconvex-sttronllyccom
min-max problems. arXiv preprint arXiv:2006.0668 2020.
Elad Hazan and Satyen Kale. Beyond the regret minimization barrier: optimal algorithms for stochastic strongly-convex optimization. The Journal of Machine Learning Research, 15(1): 2489-2512 2014.
11

---

Yu-Guan Hsieh, Franck Iutzeler, Jéröme Malick, and Panayotis Mertikopoulos. Explore aggressively, update conservatively: Stochastic extragradient methods with variable stepsize scaling. In Advances in Neural Information Processing Systems, volume 33, pages 16223-16234 2020.
Adam Ibrahim, Waiss Azizian, Gauthier Gidel, and Ioannis Mitliagkas. Linear lower bounds and conditioning of differentiable games. In International Conference on Machine Learning, pages PMLR, 2020.
Alfredo N lusem, Alejandro Jofré, Roberto Imbuzeiro Oliveira, and Philip Thompson. Extragra- dient method with variance reduction for stochastic variational inequalities. SIAM Journal Optimization, 27(2):686-724, 2017.
Samy Jelassi, Carles Domingo-Enrich, Damien Scieur, Arthur Mensch, and Joan Bruna. Extra- gradient with player sampling for faster convergence in n-player games. International Conference on Machine Learning, pages 4736-4745, PMLR, 2020.
Yujia Jin, Aaron Sidford, and Kevin Tian. Sharper rates for separable minimax and finite sum optimization via primal-dual extragradient methods. In Conference on Learning Theory, pages 4362-4415 PMLR, 2022.
Michael Jordan, Tianyi Lin, and Manolis Zampetakis. First-order algorithms for nonlinear general- ized Nash equilibrium problems. Journal of Machine Learning Research, 24(38):1-46, 2023.
Anatoli Juditsky, Arkadi Nemirovski, and Claire Tauvel. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1(1):17-58, 2011.
Galina M Korpelevich. The extragradient method for finding saddle points and other problems.
Ekonomika Matematicheskie Metody, 12:747-756, 1976.
Georgios Kotsalis, Guanghui Lan, and Tianjiao Li. Simple and optimal methods for stochastic variational inequalities, Operator extrapolation. arXiv preprint urXiv:2011.02900 2020.
Dmitry Kovalev, Alexander Gasnikov, and Peter Richtárik. Accelerated primal-dual gradient method for smooth and convex-concave saddle-point problems with bilinear coupling. Advances in Neural Information Processing Systems, 35:21725 21737, 2022.
Guanghui Lan and Yuyuan Ouyang. Mirror-prov sliding methods for solving a class of monotone variational inequalities. arXiv preprint arXiv:2111.0099) 2021.
Guanghui Lan and Yi Zhou, An optimal randomized incremental gradient method. Mathernttiaal Programming, 171:167-215, 2018.
Chris Junchi Li, Yaodong Yu, Nicolas Loizou, Gauthier Gidel, Yi Ma, Nicolas Le Roux, and Michael Jordan. On the convergence of stochastic extragradient for bilinear games using restarted iteration averaging. In International Conference Artificial Intelligence and Statistics, pages 9793-9826. PMLR, 2022a.
Chris Junchi Li, Angela Yuan, Gauthier Gidel, and Michael I Jordan. Nesterov meets optimism: Rate- optimal optimistic- gradient-basee method for stochastic bilinearly-couplet minimax optimization. arXiv preprint 10. 17550, 2022b.
Tengyuan Liang and James Stokes. Interaction matters: A note on non-asymptotic local convergence of generative adversarial networks. In International Conference on Artificial Intelligence and Statistics, pages 907-915, PMLR, 2019.
Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-concat minimax problems. In International Conference on Machine Learning, pages 6083-6093. PMLR, 2020a.
Tianyi Lin, Chi Jin, and Michael Jordan. Near-optimal algorithms for minimax optimization. In Conference Learning Theory, pages 2738-2779. PMLR, 2020b.
Zhouchen Lin, Huan Li, and Cong Fang. Accelerated optimization for machine learning. Nature Singapore: Springer, 2020c.
12

---

Nicolas Loizou, Hugo Berard, Alexia Ioannis Mitliagkas. Stochastic hamiltonian gradient methods for smooth games. In International Conference on Machine Learning, pages 6370-6381. PMLR, 2020.
Pascal Vincent, Simon Lacoste-Julien, and
Luo Luo, Yujun Li, and Cheng Chen. Finding second-order stationary points nonconvex-strongly- concave minimax optimization. Advances in Neural Information Processing Systems, 35:36667- 36679, 2022.
Panayotis Mertikopoulos, Bruno Lecouat, Houssam Zenati, Chuan-Sheng Foo, Vijay Chandrasckhar, and Georgios Piliouras. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile. In International Conference on Learning Representations, 2018.
Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of GANs. Advances in Neural Information Processing Systems, 30, 2017.
Dmitriy Metelev, Alexander Rogozin, Alexander Gasnikov, and Dmitry Kovalev. Decentralized saddle-point problems with different constants of strong convexity and strong concavity, arXiv preprint XXiv:2206.0009, 2022.
Konstantin Mishchenko, Dmitry Kovalev, Egor Shulgin, Peter Richtárik, and Yura Malitsky. Revisit- ing stochastic extragradient. In International Conference on Artificial Intelligence and Statistics, pages 4573-4582. PMLR, 2020.
Aryan Mokhtari, Asuman Ozdaglar, and Sarath Pattathil. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. International Conference on Artificial Intelligence and Statistics, pages 1497-1507. PMLR, 2020.
Vaishnavh Nagarajan and Zico Kolter. Gradient descent GAN optimization is locally stable. Advances in Neural Information Processing Systems, 30, 2017.
Arkadi Nemirovski. Prox-method with rate of convergence (1/t) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization, 15(222229, 2004.
Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization, 19:1574-1609 2009.
Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2004.
Yurii Nesterov and Laura Scrimali. Solving strongly monotone variational and quasi-variational inequalities. Discrete & Continuous Dynamical Systems, (14): 2011. 2011.
Yuyuan Ouyang and Yangyang Xu. Lower complexity bounds of first-order methods for convex- concave bilinear saddle-point problems. Mathematical Programming, 185(1-2):1-35 2021.
Brendan O' o'donoghue and Emmanuel Candes. Adaptive restart for accelerated gradient schemes.
Foundations computational mathematics, 15()):755-732, 2015.
Hassan Rafique, Mingrui Liu, Qihang Lin, and Tianbao Yang. catpy--ccvxex-ccccneee min-max optimization: provable algorithms and applications in machine learning. Optimization Methods and Software, pages 1-35, 2021.
James Renegar and Benjamin Grimmer. A simple nearly optimal restart scheme for speeding up first-order methods. Foundations of Computational Mathematics, 22(1):211 2022.
Vincent Roulet and Alexandre Aspremont. Sharpness, restart and acceleration. Advances Neural Information Processing Systems, 30, 2017.
Ernest Ryu, Kun Yuan, and Wotao Yin. Ode analysis of stochastic gradient methods with optimism and anchoring for minimax problems. arXiv preprint arXiv:1905. 10899, 2019.
13

---

Othmane Sebbouh, Marco Cuturi, and Gabriel Peyré. Randomized stochastic gradient descent ascent. In International Conference on Artificial Intelligence and Statistics, pages 2941-2969. PMLR. 2022.
Kiran K Thekumparampil. Niao He, and Sewoong Oh. Lifted primal-dual method for bilinearly coupled smooth minimax optimization. In International Conference on Artificial Intelligence and Statistics, pages 4281 -4008 PMLR, 2022.
Paul Tseng. On linear convergence of iterative methods for the variational inequality problem.
Journal of Compulational and Applied Mathematics, 60(1-2):237-252, 1995.
Paul Tseng, On accelerated proximal gradient methods for convex-concave optimization. submitted to SIAM Journal on Optimization, 2(3), 2008.
Jialei Wang and Lin Xiao. Exploiting strong convexity from data with primal-dual first-order algorithms. In International Conference on Machine Learning, pages 3694-3702, 2017.
Yuanhao Wang and Jian Li. Improved algorithms for convex-coneawe minimax optimization. Ad- vances Neural Information Processing Systems, 33:4800-4433 2020.
Lin Xiao, Adams Wei Yu, Qihang Lin, and Weizhu Chen. Dscovr: Randomized primal-dual block
coordinate algorithms for asynchronous distributed optimization. Journal of Machine Learning Research, 20(1): 1634-169,, 2019.
Guangzeng Xie, Yuze Han, and Zhihua Zhang. Dippa: An improved method for bilinear saddle point problems. arXiv preprint arXiv:2103.088 082770, 2021.
Pan Xu, Tianhao Wang, and Quanquan Gu. Continuous and discrete-time accelerated stochastic mirror descent for strongly convex functions. In International Conference on Machine Learning. pages 5492-5501. PMLR, 2018
Yan Yan, Yi Xu, Qihang Lin, Lijun Zhang, and Tianbao Yang. Stochastic primal-dual algorithms with faster convergence than o(1/vT) for problems without bilinear structure. arXiv preprint arXiv: 1904. 10/12, 2019.
Yan Yan, Yi Xu, Qihang Lin, Wei Liu, and Tianbao Yang. Optimal epoch stochastic gradient descent ascent methods for min-max optimization. Advances in Neural Information Processing Systems, 33:5789-5800, 2020.
Junchi Yang, Antonio Orvieto, Aurelien Lucchi, and Niao He. Faster single-loop algorithms for mini- max optimization without strong concavity. In International Conference on Artificial Intelligence and Statistics, pages 5517- PMLR, 2022.
Junyu Zhang, Mingyi Hong, and Shuzhong Zhang. On lower iteration complexity bounds for the convex concave saddle point problems. Mathematical Programming, 194(1-2):9019935 2022.
Yuchen Zhang and Lin Xiao. Stochastic primal-dual coordinate method for regularized empirical risk minimization. Journal of Machine Learning Research, 18:1-42, 2017.
14


Reviews:
[{"review_id": "KdyIu3x7gE", "reviewer": "Reviewer_iB8j", "summary": "The work presents a stochastic accelerated gradient-extra gradient (AG-EG) algorithm for strongly monotone variational inequalities (VI) that is designed by combining extra gradient and Nesterov acceleration. The major of the work is extending the formulation to the case when the constraint set is convex but can be unbounded and still match the best-known convergence result in the literature. Two variants of the AG-EG algorithm are presented by the authors, a direct approach and another with scheduled restarting. The convergence rates also match the lower bounds for the special case of VI's in bilinearly coupled strongly convex strongly concave saddle point problems and bilinear games with AG-EG Scheduled Restarting.", "questions": "-  In section 2 a simplified deterministic setting is shown although the final results are presented for the stochastic case. This included the Assumption 1 as well. It leads to confusion about the assumptions, is it in expectation or holds for each random realization of function?\n\n- The authors may add a table comparing the complexity results with the SOTA algorithms for solving stochastic VI problems for better readability and presentation. ", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The optimal convergence rate is established for strongly monotone VI's.\n\nThe paper is written well for most parts except for a few issues which are listed in the weakness section.\n\n", "weaknesses": "\n- The notation in algorithm 1 is confusing for the update equations. \n\n- The work does not demonstrate the efficacy of the approach through empirical studies on an application eg. Reinforcemt Learning, Regularized empirical risk minimization, quadratic games, etc.  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback. We address your comments and questions as follows.\n\n---\n\n**Q1**: The notation in algorithm 1 is confusing for the update equations\n\n**A1**: In Algorithm 1, we introduce notation to distinguish different points in the AGEG method. Specifically, $z_{t-\\frac{1}{2}}$ refers to the extrapolated point, while $z^{\\text{md}}$ and $z^{\\text{ag}}$ denote the \"middle\" and \"aggregated\" points, respectively. In addition, we use notations $\\zeta_{t-\\frac{1}{2}}$, $\\zeta_{t}$, and $\\xi_{t - \\frac{1}{2}}$ to account for noise during various steps. With our notation, $z_x$ depends on $\\zeta_{y\\leq x}$ and $\\xi_{y \\leq x}$ but remains independent of $\\zeta_{y > x}$ and $\\xi_{y > x}$. We provide definitions of $f(x; \\xi), h(x, y, \\zeta)$, and $g(y; \\xi)$ in Sec. 1 as stochastic oracles of $F(x), H(x, y)$, and $G(y)$, respectively.\n\n---\n\n**Q2**: The work does not demonstrate the efficacy of the approach through empirical studies on an application\n\n**A2**: During the rebuttal period, we conducted a comparison between our stochastic AGEG algorithms (with restarting) and stochastic extragradient (SEG) algorithms (with restarting) on quadratic games. Our experiments focused on synthetic quadratic games with varying parameter settings and noise scales. Detailed results have been provided in our uploaded pdf file. We can see that stochastic AGEG with restarting outperforms SEG with restarting by a large margin. Due to the time limit, we will add more empirical results in our final version.\n\n---\n\n**Q3**: Section 2 presents a simplified deterministic setting, although the final results are given for the stochastic case. This included the Assumption 1 as well. It leads to confusion about the assumptions, is it in expectation or holds for each random realization of function?\n\n**A3**: Throughout this paper, our notation $\\mathcal{F}$ and $\\mathcal{H}$ represents functions in their expected forms. Consequently, Assumption 2.1 holds in expectation."}, {"review_id": "uzXGVhkU31", "reviewer": "Reviewer_qR4K", "summary": "The paper studies variational inequalities with separable structures (sum of the gradient of a strongly convex function and a monotone operator). This class subsumes bilinear coupling SC-SC minimax optimization and bilinear games. The authors propose an extragradient algorithm with acceleration by shifting the convexity part from the gradient to the operator. This algorithm matches the lower bounds for strongly monotone VIs (up to some log factors). When specialized in bilinear problems, the algorithm can be coupled with scheduled restarting to match the existing lower bounds for bilinear SC-SC and bilinear games.", "questions": "1. In Jordan et al (2023) the separable structure means sum of the gradient of a general convex function and a strongly monotone operator. Could the authors discuss the difference between this and the one considered in the paper? How would shifting the convexity technique change?\n2. Some existing papers talk about optimal bounds with the attention to the log factors. I am not sure to what extent the authors pointed this out in the related work and the paper. Could the authors discuss?", "limitations": "The authors discussed some limitations of the proposed algorithm.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The proposed algorithm matches several lower bounds at the same time.\n- The algorithms do not require the bounded domain assumption as in some of the previous works.\n- The technical details of the paper are excellent and easy to follow. I checked some of the proofs in the appendix and did not find any issues.\n- The discussion provided in the paper is generally well written.", "weaknesses": "- Scheduled restart may not always be desirable, although this is not necessarily a weak point. For bilinear problems, the algorithm does need scheduled restart to achieve the optimality. The claim in the conclusion that all three lower bounds are matched in one algorithm is therefore not really accurate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback. We address your comments and questions as follows.\n\n---\n\n**Q1**: For bilinear problems, the algorithm does need scheduled restart to achieve optimality. The claim in the conclusion that all three lower bounds are matched in one algorithm is, therefore not really accurate.\n\n**A1**: We appreciate your feedback and the opportunity to clarify our claim. First, our algorithm with scheduled restarting can achieve optimal results for general variational inequality, bilinear games, and bilinearly coupled strongly-convex-strongly concave (SC-SC) settings. Please refer to the table in the uploaded pdf file for the corresponding upper and lower bounds. So our claim that all three lower bounds are matched in one algorithm in the conclusion is valid.\n  \nIn addition, we agree that the scheduled restarting technique may not be ideal. For bilinear game problems, techniques such as scheduled restarting is essential in matching the lower bound.  Alternative approaches exist, such as the work by Azizian et al. [2020b], which uses EG with momentum. However, the accelerated rate in their work is limited to scenarios with large condition numbers. It remains an open question whether a technique can simultaneously accelerate the bilinear part and the minimization part without scheduled restarting. This question is left for future work.\n\n---\n\n**Q2**: In Jordan et al. [2023] the separable structure means sum of the gradient of a general convex function and a strongly monotone operator.\nCould the authors discuss the difference between this and the one considered in the paper?\n\n**A2**: Thanks for your question. Jordan et al. [2023] uses separable structure to refer to the sum of the gradients of a general convex function and a strongly monotone operator, while we use separable structure to refer to the sum of the gradients of a strongly convex function and a general monotone operator.  These two settings are equivalent because we can shift the “strong convexity'' component from the strongly convex function to the monotone operator.\n\n---\n\n**Q3**: Some existing papers talk about optimal bounds with the attention to the log factors. \n\n**A3**: Our upper bounds and existing lower bounds all contain log factors. For example, in Corollary 2.8, Eq. (17), Corollary 3.1, and Corollary 3.3, the complexity is presented with log factors being explicitly considered. In Table 1 in our original supplementary material, and the table in our uploaded pdf file, we have omitted dependency on log factors due to layout reasons. We have added discussions on the log factors in the caption of our table in the uploaded pdf file.\n\n"}, {"review_id": "abURRwyyvg", "reviewer": "Reviewer_oFAu", "summary": "The author(s) studied the extragradient algorithm for the separable strongly monotone VI problems. It was shown that the new analysis can achieve optimal error bounds in various settings.", "questions": "- I suggest the author(s) to include a table of error bounds and assumptions to compare with existing works.", "limitations": "I do not find any negative societal impact of this paper.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The new analysis gives optimal error bounds in various settings, which is a decent contribution to the field.\n- The paper is well-written and easy to follow in general.", "weaknesses": "- I encourage the author(s) to include a table to compare the error bounds and assumptions used with prior works. It seems that the error bound obtained by the author(s) is not entirely new, existing analysis can also achieve the same error bound under other conditions.\n- There is no related work section, which makes it hard for readers to understand the position of this work in the literature.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback. We address your comments and questions as follows.\n\n---\n\n**Q1**: I suggest the author(s) to include a table of error bounds and assumptions to compare with existing works.\n\n**A1**: Thank you for your suggestion. We have added a revised table comparing error bounds and assumptions with existing works in our uploaded pdf file.\n\n---\n\n**Q2**: There is no related work section, which makes it hard for readers to understand the position of this work in the literature.\n\n**A2**: Due to the space limit in the submission, we have deferred the related work section to the supplementary material. We will restructure our content to accommodate a comprehensive related work section in the main content.\n"}, {"review_id": "EPPcbvmoeI", "reviewer": "Reviewer_KTFh", "summary": "This paper studies the stochastic monotone variational inequality problem. They propose the Accelerated Gradient - Extragradient (AG-EG) algorithm which is shown to have the optimal convergence rates for the strongly monotone VI problem.", "questions": "See above", "limitations": "See above", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "This papers studies the important problem of Stochastic VIs, the algorithm, especially the idea of restarting is interesting.", "weaknesses": "Dependence on the Diameter of the Set: A major differentiating point of this paper, as claimed by the authors in Remark 2.4, is that the convergence rate bounds have no dependence on the constraint set. However, there is a dependence on the initial distance of the iterate to the solution. Showing that the iterates are bounded is essentially the only requirement to convert the results of papers like [Chen et a. 2017] to the unconstrained setting. Can the authors describe the main difficulties and technical novelties involved in showing this?\n\nRate dependence on the noise term: As stated in the appendix, the optimal dependence on the noise term has been achieved by a multistage algorithm in [Fallah et al. 2020], where the  dependence on the noise term is $\\mathcal{O}(\\sigma^2/n)$, whereas this paper seems to have a dependence of the form  $\\mathcal{O}(\\sigma^2/\\sqrt{n})$. Am I missing something? Is the optimality of the proposed algorithm only in terms of the first term, where the dependence on the problem parameters $L, \\mu$ etc. are optimal?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback. We address your comments and questions as follows.\n\n---\n\n**Q1**: Showing that the iterates are bounded is essentially the only requirement to convert the results of papers like Chen et al. [2017] to the unconstrained setting. Can the authors describe the main difficulties and technical novelties involved in showing this?\n\n**A1**:  We appreciate your question. While the optimal rate for monotone variational inequalities (VIs) has been achieved by Chen et al. [2017], achieving the optimal rate for strongly-monotone VIs remains largely unsolved. The boundedness of iterates has been one of the main barriers to proving the optimal rate for strongly-monotone VIs. \n\nThe standard gradient descent-ascent method is known to *diverge* for many convex-concave problems [1]. Moreover, in the noisy setting, *even extragradient may diverge* [2]. Thus it is critical to show that the iterates are bounded. It is a challenge as many seminal works on stochastic VIs only consider bounded domains [3,4]. In the VI literature, showing the boundedness of the iterates is often the crux of the proof (see for instance, [5]).\n\nThe main proof technique used in our work is the “bootstrapping” argument shown in step 3 of Sec. D.3. By summing up Eq. (33), a rearrangement of the terms implies a relation of $\\mathbb{E}||z_t - z^*||^2$ with a summation of itself, the result follows by the bootstrapping argument. In sharp contrast, Chen et al. [2017] adopt \"the enlargement of a maximal monotone operator\" technique to deal with unbounded domains, which seems complicated and unnecessary for our purpose. Our bootstrapping method leads to a simpler and more concise analysis. Moreover, Chen et al. [2017] only considered the monotone VI setting rather than the strongly-monotone VI setting.\n\nIn addition, we also consider specific instances of VI problems (i.e., bilinear games, bilinearly coupled SC-SC) with lower-bound matching results by utilizing scaling reduction techniques. These results were not obtained by Chen et al. [2017] either.\n\n[1] Mescheder, Lars, Andreas Geiger, and Sebastian Nowozin. \"Which training methods for GANs do actually converge?.\" International conference on machine learning. PMLR, 2018.\n\n[2] Chavdarova, Tatjana, et al. \"Reducing noise in GAN training with variance reduced extragradient.\" Advances in Neural Information Processing Systems 32 (2019).\n\n[3] Nemirovski, Arkadi, et al. \"Robust stochastic approximation approach to stochastic programming.\" SIAM Journal on optimization 19.4 (2009): 1574-1609.\n\n[4] Juditsky, Anatoli, Arkadi Nemirovski, and Claire Tauvel. \"Solving variational inequalities with stochastic mirror-prox algorithm.\" Stochastic Systems 1.1 (2011): 17-58.\n\n[5] Gorbunov, Eduard, et al. \"Clipped stochastic methods for variational inequalities with heavy-tailed noise.\" NeurIPS 2022.\n\n---\n\n**Q2**: The optimal dependence on the noise term has been achieved by a multistage algorithm in Fallah et al. [2020], where the dependence on the noise term is $\\mathcal{O}\\left(\\sigma^2 / n\\right)$, whereas this paper seems to have a dependence of the form $\\mathcal{O}\\left(\\sigma^2 / \\sqrt{n}\\right)$. Is the optimality of the proposed algorithm only in terms of the first term, where the dependence on the problem parameters \n$L, \\mu$ are optimal?\n\n**A2**: We believe there is a misunderstanding by the reviewer. This is due to the different definitions of the $\\varepsilon$-optimal point. In our paper, the $\\varepsilon$-optimal point is defined by $||z - z^*|| \\leq \\varepsilon$. In contrast, Fallah et al. [2020] define the optimal point by $||z - z^*||^2 \\leq \\varepsilon$. Thus, our $\\varepsilon^2$ is equivalent to their $\\varepsilon$. Therefore, the complexity term that depends on the noise variance $\\sigma^2$ in Corollary 2.8, i.e., $\\frac{\\sigma^2}{\\mu^2 \\varepsilon^2}$ should be translated into $\\frac{\\sigma^2}{\\mu^2 n}$ rather than $\\frac{\\sigma^2}{\\mu^2 \\sqrt{n}}$. This also applies to Corollaries 2.9, 3.1, and 3.3. \nTo sum up, the optimality of our proposed algorithm is not only in terms of the first term but also in the second noise term. We apologize for the confusion caused by the different definition of $\\varepsilon$-optimal point, and will make it clear in the revision. \n\n"}]

Author rebuttal:
Dear reviewers,

Thank you for your time and constructive comments. Based on the feedback of Reviewer iB8j and Reviewer oFAu, we have added a table comparing the complexity results and assumptions with the prior works. In addition, to address the feedback of Reviewer iB8j on empirical studies, we conducted an experiment to examine the performance of stochastic AGEG vs stochastic EG. Please find the table and figure in the uploaded PDF file. For all other comments and feedbacks, we have provided point-by-point responses in the rebuttal separately. 

Meta-review: The author response addressed the reviewers' main concerns. The reviewers all agree that the paper makes solid contributions to this line of work, and it provides optimal results in several settings that remained open.
Decision: Accept (poster)