Enhanced First and Zeroth Order Variance Reduced Algorithms for Min-Max Optimization
Authors: Tengyu Xu
Zhe Wang
Yingbin Liang
H. Vincent Poor
Venue:  (2021)
Keywords: minimax optimization
nonconvex
variance reduction

Abstract: Min-max optimization captures many important machine learning problems such as robust adversarial learning and inverse reinforcement learning, and nonconvex-strongly-concave min-max optimization has been an active line of research. Specifically, a novel variance reduction algorithm SREDA was proposed recently by (Luo et al. 2020) to solve such a problem, and was shown to achieve the optimal complexity dependence on the required accuracy level $\epsilon$. Despite the superior theoretical performance, the convergence guarantee of SREDA requires stringent initialization accuracy and an $\epsilon$-dependent stepsize for controlling the per-iteration progress, so that SREDA can run very slowly in practice. This paper develops a novel analytical framework that guarantees the SREDA's optimal complexity performance for a much enhanced algorithm SREDA-Boost, which has less restrictive initialization requirement and an accuracy-independent (and much bigger) stepsize. Hence, SREDA-Boost runs substantially faster in experiments than SREDA. We further apply SREDA-Boost to propose a zeroth-order variance reduction algorithm named ZO-SREDA-Boost for the scenario that has access only to the information about function values not gradients, and show that ZO-SREDA-Boost outperforms the best known complexity dependence on $\epsilon$. This is the first study that applies the variance reduction technique to zeroth-order algorithm for min-max optimization problems.

Under review as a conference paper at ICLR 2021
ENHANCED FIRST AND ZEROTH ORDER VARIANCE REDUCED ALGORITHMS FOR MIN-MAX OPTIMIZA- TION
Anonymous authors Paper under double-blind review
ABSTRACT
Min-max optimization captures many important machine learning problems such as robust adversarial learning and inverse reinforcement learning, and nonconvex- strongly-concave min-max optimization has been active line of research. Specif- ically, a novel variance reduction algorithm SREDA was proposed recently by (Luo al. 2020) to solve such problem, and was shown achieve the optimal complexity dependence on the required accuracy level e. Despite the superior theoretical performance, the convergence guarantee of SREDA requires stringent initialization accuracy and an e-dependent stepsize for controlling the per-iteration progress, so that SREDA can run very slowly in practice. This paper develops a novel analytical framework that guarantees the SREDA's optimal complexity per- formance for a much enhanced algorithm SREDA-Bood which has less restrictive initialization requirement and accuracy-independemt (and much bigger) stepsize. Hence, SREDA Boost runs substantially faster in experiments than SREDA. We further apply SREDA-BoOST named ZO-SREDA-BBE -oostt for the scenario that has access only to the information about function values not gradients, and show that ZO-SREDA-Bon Boost outperforms the best known complexity dependence on e. This is the first study that applies the variance reduction technique zeroth-order algorithm for min-max optimization problems.
propose zeroth-order variance reduction algorithm
INTRODUCTION
Min-max optimization has attracted sigmificant growth of attention in machine learning as it captures several important machine learning models and problems including generative adversarial networks (GANs) Goodfellow et al. (2014), robust adversarial machine learning Madry et al. (2018), imitation learning Ho & Ermon (2016), etc. Min-max optimization typically takes the following form
E F (x,v: v: 1 Σ'= F(x, E.)
(online case)
min max f(x, v), where f(x, ZERd1 yER*2
A
(1)
(finite-sum case)
where [xx y) takes the expectation form if data samples & are taken in an online fashion, and f(x v) takes the finite-sum form if a dataset of training samples & for i 1...., given in advance.
This paper focuses on the nonconvex wwwroonce-sstongyy-ooonee nonconvex with respect to for all y Rd, and f(x, y) is p-strongly concave with respect to y for all x E Rd The problem then takes the following equivalent form:
min-max problem, in which f(r, y) is
min ZERd1
((x))
max f(x, f(r v) vERd2
(2)
The objective function ((-) in (2) is nonconvex in general, and hence algorithms for solving eq. (2) are expected to attain an approximate (i.e., e-accurate) first-order stationary point. The convergence of deterministic algorithms for solving eq. (2) has been established in Jin et al. (2019); Nouiched et al. (2019); Thekumparampil et al. (2019); ; Lu et al. (2020). SGD-type of stochastic algorithms have also been proposed to solve such problem more efficiently, including SGDmax Jin et al. (2019),

---

Under review as a conference paper at ICLR 2021
PGSMD Rafique et al. (2018), and SGDA Lin et al. (2019), which respectively achieve the overall complexity of Oxx log(L/))) O
and (
Furthermore, several variance reduction methods have been proposed for solving eq. (2) for the sunconnex-strooggyyoooe concave case. PGSVRG Rafique et al. (2018) adopts proximally guided SVRG method and achieves the overall complexity of (1) for the online case and ((k'ne-)) for the finite-sum case. Wai et al. (2019) converted the value function evaluation problem to a specific min- max problem and applied SAGA to achieve the overall complexity of O 2) for the finite-sum case. More recently, Luo et al. (2020) proposed novel nested-loop algorithm named Stochastic Recursive Gradient Descent Ascent (SREDA), which adopts SARAA/SPPDERR-yPD Nguyen et al. (2017a); Fang et al. (2018) of recursive variance reduction method (originally designed for solving the minimization problem) for designing gradient estimators to update both and y.. Specifically, x takes the normalized gradient update in the outer-loop and each update of is followed by an entire inner- loop updates of y. Luo et al. (2020) showed that SREDA achieves an overall complexity of (m=( for the online case in eq. (1), which attains the optimal dependence on e Arjevani et al. (2019), For the finite-sum case, SREDA achieves the complexity of O(n² Vnee -2 + n + (n + k) log(k)/)) for 2 x², and O(kx² =n))- for S k
Although SREDA achieves the optimal complexity performance in theory, two issues may sub- stantially degrade its practice performance. (1) SREDA has stringent requirement the ini- tialization accuracy Ç =
<=x=2 x=22², which hence requires O(n2 log(k/<)) gradient estimations in the initialization, and is rather costly in the high accuracy regime (i.e., for small c). (2) The convergence of SREDA requires the stepsize to be substantially small, i.e., at the e-level with ar = (min(e// (xl (2) 1/(xE))), which restricts each iteration to make only e-level progress with || - Ze 2 = OEe/ (e/r))). Consequently, SREDA can run very slowly in practice.
Thus, a vital question arising here is whether we can guarantee the same optimal complexity performance of SREDA even if we considerably relax its initialization (i.e., much bigger than O(E))) and enlarge its stepsize (i.c., much bigger than (e)). The answer is highly nontrivial because the original analysis framework for SREDA in Luo et al. (2020) critically relies on these restrictions. The first focus of this paper is on developing a novel analytical framework to guarantee that such enhanced SREDA continues to hold the SREDA's 's optimal complexity performance.
Furthermore, in many machine learning scenarios, min-max optimization problems need to be solved without the access of the gradient information, but only the function values, e.g., in multi-agent reinforcement learning with bandit feedback Wei et al. (2017); Zhang et al. (2019) and robotics Wang & Jegelka (2017); Bogunovic et al. (2018). This motivates the design of zeroth-order (i.e., gradient-free) algorithms. For nonconvex-stroonlyy--e conccave min-max optimization, Liu et al. (2019) studied constrained problem and proposed Z0-min-max algorithm that achieves the computational complexity of O((d, + dz)c-O). Wang et al. (2020) designed ZO-SGDA and ZO-SGDMSA, where ZO-SGDMA achieves the best known query complexity of O(di + d2 x -4 log(1/e)) among the reroth-order algorithms for this problem. All of the above studies are of SGD-type, and no efforts have been made on developing variance reduction zeroth-order algorithms for nonconvex- mnnconve--troonglycoomec. min-max optimization to further improve the query complexity.
The second focus of this paper is on applying the aforementioned enhanced SREDA algorithm to design zeroth-order variance reduced algorithm for nonconvex-strowe strongly-concave min-max problems, and further characterizing its complexity guarantee which we anticipate to orderwisely better than that of the existing stochastic algorithms.
1.1 MAIN CONTRIBUTIONS
This paper first studies an enhanced SREDA algorithm, called SREDA-Boost which improves SREDA from two aspects. (I) For the initialization, SREDA-Boos requires only an accuracy of <- which is much less stringent than that of Ç x= 22² required by SREDA. (2) SREDA- Boost allows an aecuracy-independent stepsize a = (1) (I))()))), which is much larger than the c-level stepsize Ommnnf/((x) [|uell)), 1/(xe))) adopted by SREDA. Hence, SREDA-Boos can run much faster than SREDA.
'The constant K. f/ju, where is the strong concavity parameter of f(z, .). and is the Lipschitz constant of the gradient of fxr y) as defined in Assumption 2. Typically, is much larger than onc.
2

---

Under review as a conference paper at ICLR 2021
Table Comparison of stochastic algorithms for nonconvex-sttongly-ooneew min-max problems
Type
Algorithm
Stepsize
Initialization Complexity
Complexity V.
SGDmmx
N/A
O log(
SGDA
O 4)=
N/A
O
FO
PGSMD
(N ")
N/A
O(n'
PGSVRG
O
N/A
O(n'c
SREDA
θ (min| ke m)
OlN'c O(n'c=log(7))
Olw'c
SREDA-BooK.
((x -(-))
O log())))
O(N'E "f
Z0-min-max
O(x=te" te-)
N/A
O of
ZO-SGDA
O(n 18-
N/A
Oldn'e c
70
ZO-SGDMSA
O 1
N/A
Oldr'c log(
ZO-SREDA
( 1
(og(((()
O (d
We clarify that SREDA-Boow should not he expected to improve the complexity order of SREDA, because SREDA already achieves the optimal complexity. Rather, SREDA-Bood improves upon SREDA by much
more relaxed requirements on initialization and stepsize to achieve such optimal performance.
"FO" stands for "First-Onder,, and "ZO" stands for "Zeroth-Order".
We include only the stepsize for updating for comparison.
The complexity for first-order algorithms refer the total gradient computations to attain e-stationary point, and for zeroth-order algorithms refers the total function value queries.
We include only the complexity in the online case in the table, because many studics did not cover the finite-sum case. We comment on the finite-sum case in Section 4 and Section 5.2.
: We define d == d1 + d2.
The first contribution of this paper lies in developing new analysis technique to provide the computational complexity guarantee for SREDA-Boos, establishing that even with considerably relaxed conditions on the initialization and stepsize, SREDA-Boue achieves the same optimal complexity performance as SREDA. The analysis technique of SREDA in Luo et al. (2020) does not handle such a case, because the proof highly relies on the stringent initialization and stepsize requirements. Central to our new analysis framework the novel approach for bounding two inter- connected stochastic error processes: tracking error and gradient estimation error (see Section 4 for their formal definitions), which take three steps: bounding the two error processes accumulatively the entire algorithm execution, decoupling these inter-related stochastic error processes, and establishing each of their relationships with the accumulative gradient estimators.
The second contribution of this paper lies in proposing the zeroth-order variance reduced al- min-max optimization when the
gorithm ZO-SREDA-Boost for smmconvex-strongly-conooe gradient information not accessible. For the online case, we show that ZO-SREDA-Bond achieves an overall query complexity of ((d + d2)m=²=3)) which outperforms the best known complexity (achieved by ZO-SGDMSA Wang et al. (2020)) in the case with S x-1 For the finite-sum case, we show that ZO-SREDA-Boos achieves an overall query complexity of ((d, (162 -
d2(n² 2 + kn) log(k)) when n x², and O(di + d2) (x² + kn) "R -2) when < 22. This is the first study that applies the variance reduction method to zeroth-order min-max optimization.
conconvex-stronlgy.
1.2 RELATED WORK
Due to the vast amount of studies on min-mat max optimization and the variance reduced algorithms, we include below only the studies that are highly relevant to this work.
Variance reduction methods for min- max optimization are highly inspired by those for conventional minimization problems, including SAGA Defazio et al. (2014); Reddi et al. (2016), SVRG Johnson Zhang (2013); Allen-Zhu & Hazan (2016); Allen-Zhu (2017), SARAH Nguyen et (2017a;b; 2018), SPIDER Fang et al. (2018), SpiderBoost Wang al. (2019), etc. But the convergence analysis for min-max optimization is much more challenging, and is typically quite different from their counterparts in minimization problems.
For strongly-comee wwwpegly--ooox---eee www-sssttoooooocccccec SVRG and SAGA to the finite-sum min-max optimization, Palaniappan & Bach (2016) applied and established linear convergence rate, and Chavdarova et al. (2019) proposed SVRE later to obtain a better bound. When the condition number of the problem is very large, Luo et al. (2019) proposed proximal point iteration algorithm to improve the
3

---

Under review as a conference paper at ICLR 2021
performance of SAGA. For some special cases, Du et al. (2017); Du & Hu (2019) showed that the linear convergence rate of SVRG can be maintained without the strongly-conve or strongly concave assumption. Yang et al. (2020) applied SVRG to study the min-max optimization under the two-sided Polyak-L.ojasiwicz condition.
min-max optimization is the focus of this paper. As discuss the beginning of the introduction, the SGD-type algorithms have been developed and studied, including SGDmax Jín et al. (2019), PGSMD Rafique al. (2018), and SGDA Lin et al. (2019). Several variance reduction methods have also been proposed to further improve the performance, including PGSVRG Rafique al. (2018), the SAGA-type algorithm for min-max optimization Wai et al. (2019), and SREDA Luo et al. (2020). Particularly. SREDA has been shown in Luo et al. (2020) to achieve the optimal complexity dependence on e. This paper further provides the convergence guarantee for SREDA-Boost (which enhances SREDA with relaxed initialization and much larger stepsize) by developing a new analysis technique.
Noncommer-strronggyyycoom
While SGD-type zeroth-order algorithms have been studied for min-max optimization, such as Menickelly Wild (2020); Roy et al. (2019) for convex-concave min-max problems and Liu et al. (2019); Wang et al. (2020) for sanconvex-ttrongly-coneee algorithms have not been developed for veroth-order min-max optimization so far. This paper proposes the first such algorithm named ZO-SREDA-Booo for noncomvex-sttongly-comm min-max optimization, and established its complexity performance that outperforms the existing comparable algorithms (see Table 1).
min-max problems, variance reduced
NOTATION AND PRELIMINARIES
In this paper. we use I1-Il, denote the Euclidean norm of vectors. For a finite set S, we denote its cardinality as |S]. For a positive integer n. we denote [n] {1..... nn.. We assume that the min-max problem eq. (2) satisfies the following assumptions, which have also been adopted by Luo et al. (2020) for SREDA. We slightly abuse the notation below to represent the random index in
both the online and finite-sum cases, where in the finite-sum case, EE [-] is with respect to the uniform distribution over ".....)
Assumption 1. The function D(-) is lower bounded, i.e., we have :h inf ((x) > -00.
Assumption 2. The component function F has an averaged E- L-Lipschitz gradient, i.e., for all (z, y), ([,,, [ F [) S e - x'112 2
(x',3/) E dR: x Rd, we huve E&
+ ll - y l[:).
Assumption 3. The function f is p-strongly-concuve in for any x E RR,, and the component function F concave in y. i.e., for any x Rd,, y.y E Rdd and &, we have f(z,y) S (2,8')+ (V,f(z, v'), y/) Ellyy-'' - and F(x, S F(z, v/:E) + F(r,
Assumption 4. The gradient of each component function F(x. E: has bounded vari- ance, Le., there exists a constant o > O such that for any (x.y) E Rdixd xd2, we have [IVF - ff(z, v)]] S a² <C.
Since & is nonconvex general, it is NP-hard to find global minimum. The goal here to develop stochastic gradient algorithms that output an e-stationary point as defined below.
Definition 1. The point & is called an e-stationary point of the differentiable function &b if |V where is a positive constant.
3 SREDA AND SREDA-BOOST ALGORIHHMS
We first introduce the SREDA algorithm proposed in I. Luo et al. (2020), and then describe an enhanced algorithm SREDA-Boo -Bost that we study in this paper.
SREDA (see Option I in Algorithm 1) utilizes the variance reduction techniques proposed in SARAH Nguyen et al. (2017a) and SPIDER Fang et al. (2018) for minimization problems to construct the gradient estimator recursively for min-max optimization. Specifically, the parameters are updated in a nested loop fashion: each update of x; in the outer-loop is followed by (m + 1) updates of y over entire inner loop. Furthermore, the outer-loop updates of x; is divided into epochs for variance reduction. Consider a certain outer-loop epoch {(n, 1)q. nz. - 1} < n [T/q] is positive integer). At the beginning of such epoch, the gradients are evaluated
and y
4

---

Under review as a conference paper at ICLR 2021
with large batch size S1 (see line 6 in Algorithm 1). Then, for each subsequent outer-loop iteration, an inner loop of ConcaveMuximiier (see Algorithm 2) recursively updates the gradient estimators for v.,f(z,)) y) and V, v,,f(z,) y) with small batch size Sz. Note that although the inner loop does not update v; the gradient estimator vaJ (x. y) is updated in the inner loop. With such a variance reduction technique, SREDA outperforms all previous algorithms for conconvex-strongly-cone min-max problems (see Table I), and was shown to achieve the optimal dependency on e in complexity Luo al. (2020).
Algorithm SREDA and SREDA-Bovd -oEDtt
1: Input: zo, initial accuracy Ç, learning rate a., 8 = O+),, batch size Si, Sa and periods q, m.
2: Option (SREDA):
22?; Option II (SREDA-Boost: Ç -=1
3:
Initialization: yo
(AAAHH(-((an)) +) (see Appendix B.2 for iSARAH(-))
4: for t= O, D.., T
1 do
5:
if mod(L, = 0 then draw S₁ samples (6,,... Ess}
6:
Ut 3 Eiel Fas, B,,C.). Et).
Ui 3i Di Vy 36.
else
8: Ut = T 11 Ut 1,
9:
end
10: Option (SREDA): = min कल # 10():: Option II (SREDA-Boosh) o, ((
11:
I(+1 = auti
12:
Mc+1 ConcaveMaximizer(t. m, S2)
13:
end for
14: Output: é chosen uniformly at random from ) T-11
Algorithm 2 ConcaveMaximizzrr/ m,
I: Initialization: 0t,-1 2t. Dt,-1 V/. 02,00
Ie $t.0 yt. ūt,-1 us. ūz
Draw S₂ samples [EI,... ]
3: or,0 6,,-1 +
E. V=
4: ū,,o = utt-1 + 52
Sq
v, 3r.0.
1=1
82
5: I Σ..0. 2t,1
01.0
Būg,
for 1,2,.... m do
7: draw S2 samples (E..... Fs,
d = 01 + 5,
1 82 Va M.
5,
1 5/ V. 1.
9: i = ūc,k-1 + 1 52
L v, Fi Dr.
Lie F.r,
10: TeA++I Tt,ks
14,++1 = U + Būt,
11: end for
12:
Output: y141 = Dn with mi chosen uniformly at random from {0, m
Although SREDA achieves the optimal complexity performance in theory, two issues can substantially slow down its practical performance. (a) Its initialization yo needs satisfy stringent ee-lelel requirement on the accuracy [[[] (xro ) S 222 (see line 2 in Algorithm 1), which requires as large as O(x2 log(x//)) stochastic gradient computations Luo et al. (2020). This is quite costly. (b) SREDA uses an e-dependent stepsize and applies normalized gradient descent, so that each outer-loop update makes only c-level progress given by || xz æzll2 O (re)() This substantially slows down SREDA. By following the analysis of SREDA, it appears that such choices for initialization and stepsize are necessary to obtain the guaranteed convergence rate.
In this paper, we study SREDA-Boost (see Option II in Algorithm 1) that enhances SREDA over the above two issues. (a) SREDA-Boost relaxes the initialization requirement to be V (2) vo) 112] < --1, which requires only OK log k) gradient computations. This improves the computational cost upon SREDA by a factor of (ke--)) (b) SREDA- -Bost adopts c-independent stepsize 0g = a = ((1((s()) for 24 so that each outer-loop update can make much bigger progress than SREDA. As experiments in Section 6 demonstrate, SREDA-Boost runs much faster than SREDA.
To provide the convergence guarantee for CREDA-Boost the analysis of SREDA in Luo et al. (2020) does not apply, because the proof highly depends the stringent requirements the
5

---

Under review as a conference paper at ICLR 2021
initialization and the stepsize. Thus, this paper provides analysis technique for establishing the complexity performance guarantee for SREDA- Boost and further applies it to the gradient- min-max problems.
4 CONVERGENCE ANALYSIS OF SREDA-BOOST
The following theorem provides the computational complexity of SREDA-Boost for finding first- order stationary point of (-) with e accuracy.
Theorem 1. Apply BREDA-Boost solve the online case of the problem eq. (1). Suppose Assumptions 1-4 hold. Let Ç = x= and S2 (KE -1). Then for T' to be least at the order of O (2),,)), Algorithm outputs that satisfies B V lg S with stochastic gradient complexity E
Oxx 3 O
q= 3
()=-), m ((k), S OOo²R²² x=
Furthermore, SREDA Boost also applicable to the finite-sum case of the problem eq. (1) by replacing the large batch S1 of samples used in line 6 of Algorithm with the full set of samples.
Corollary 1. Apply SREDA-Boos described above to solve the finite-sum case of the problem eq. (1). Suppose Assumption 1-4 hold. Under appropriate parameter settings given in Appendix B.4, the overall gradient complexity to attain an e-stationary point O /nee
(n log(k)) for
x², and O ((x2 kn)e- - for n 2
To compare with SREDA, as shown in Luo et al. (2020), SREDA requires stringent initialization and stepsize selection to achieve the optimal complexity performance. In constrast, Theorem and Corollary 1 show that those strict requirements are not necessary for achieving the optimal performance, and establish that SREDA-Bous achieves the same optimal complexity SREDA under more relaxed initialization and much bigger and accurayy-independdnt stepsize a.
The convergence analysis of SREDA-Boost in Theorem 1 very different from the proof of SREDA in Luo et al. (2020). At high level, such analysis mainly focuses on bounding two inter-related errors: tracking error 6, B[lly J (I2, Mt 112 that captures how well the output Ye of the in- ner loop approximates the optimal point y (Ixe for a given Tt, and gradient estimation error A. = [B|l 0y - V (x 112 | [/, V fxr, I1) that captures how well the stochastic gradient estimators approximate the true gradients. In the analysis of SREDA in Luo et al. (2020), the strin- gent requirements for initialization and stepsize and the e-level normalized gradient descent update substantially help bound both errors òN and A₁ separately at the e level for each iteration so that the convergence bound follows. In contrast, this is not applicable to CREDA-BOOST which has relaxed and accuracy-in independet initialization and stepsize. Hence, we develop novel analysis framework to bound the accumulative errors Ets 5g and E/= A₁ over the entire algorithm execution, and then decouple these two inter-related stochastic error processes and establish their relationships with the accumulative gradient estimators ΣT-1 E|[l, 11 The following proof sketch of Theorem I further illustrates our ideas.
The analysis of SREDA- for min-man problems inspired by that for SpiderBoost in Wang et al. (2019) for minimization problems, but the analysis here is much more challenging due to the complicated mathematical nature of min-max optimization. Specifically, SpiderBoost needs to handle only one type of the gradient estimation error, whereas SREDA- -oost requires to handle two strongly coupled errors in min-max problems. Hence, the novelty for analyzing SREDA PREDA-Boost mainly lies in bounding and decoupling the two errors in order characterize their impact on the convergence bound.
ZO-SREDA-BOOST AND CONVERGENCE ANALYSIS
In this section, we study the min-me max problem when the gradient information is not available, but only function values can be used for designing algorithms. Based on the first-order SREDA-Boost algorithm, we first propose the zeroth-one order variance reduced algorithm called TO-SREDA-BaR -Bostt and then provide the convergence analysis for such algorithm.

---

Under review as a conference paper at ICLR 2021
5.1 CO--SREA--BOR BOOST ALGORITHM
The DO-SREDA-Booss algorithm (see Algorithm 4 in Appendix C.I) shares the same update scheme as SREDA-BOR -oosst, but makes the following changes.
(I) In line 3 of SREDA-Boost, instead of using iSARAH, ZO-SREDA-Boos utilizes a zeroth-order algorithm ZO-iSARAH (Algorithm 6 in Appendix C.4) to search an initialization yo-
(2) At the beginning of each epoch in the outer loop (line 6 of SREDA-Boom Boost), ZO- SREDA-Boost utilizes coordinate-win wiie gradient estimators with large batch S given by + dej, Me, Et) - F(x, - deg, ye: Et))e; and ut = (1/S)) Σ5th (F + de F(x,, yt - de,, (e), where e, denotes the j-th canonical unit basis vector. Note that the coordinate-wiss gradient estimator is commonly taken in the zeroth-order variance reduce algorithms such as in Ji et al. (2019) for minimization problems.
" = (1/5)) E
(3) 7O0-SREDA-BoRk replaces ConcaveMaximizer (line 12 of SREDA) by ZO-CConaveeaxximizer (see Algorithm 5), in which the zeroth-order gradient estimators are recursively updated with small batches $2,5 (for update of X) and 2,., (for update of y) based on the Gaussian estima- tors given by Gyu (Ir, y: MM... S.M)) (1/S2,z) " M1 V, E.) F(x, y, {:]]ss/u and H (Zz. y, M.s EMM,) (1/S2,y Eie[sa.)l
H22,,, (,) F(z, y. E:)]or/ p2, where vs N (O, La,), wi ~ [0, 1d,) with 1a denoting the identity matrices with sizes d x d.
[F [F
5.2 CONVERGENCE ANALYSIS OF ZO-SREDA-BOOST
The following theorem provides the query complexity of ZO-sREDA-Boost for finding a first-order stationary point of (() with accuracy.
Theorem 2. Apply ZO-SREDA-Boon in Algorithm 4 to solve the online case of the problem eq. (1). Suppose Assumptions 1-4 hold. Let (= = K a = Ox 3
m = (K)). S1 = 0(o2222= S2.z O(dike S2,, Odgke ) 8 = ((d) d.)0 (-lc) N1 (0 O Algorithm 4 outputs &t that satisfies V[I S with the overall function query complexity ((d, + d2))
O((-() = = O
and μl2 Odt - Then for T to be least at the order of
Furthermore, DO-SREDA-BooDS is also applicable to the finite-sum case of the problem eq. (1), by replacing the large batch sample S used in line 6 of Algorithm 4 with the full set of samples.
Corollary 2. Apply Z0- -D-SRDDA- BREDA-Boost described above to solve the finite-sum case of the problem eq. (1). Suppose Assumptions 1-4 hold. Under appropriate parameter settings given in Appendix C.6. the function query complexity to ollain an e-stationary point is ((ds, + +dz) (2))(nnne -2 + + d2 d2(122 2
kn) log(n)) for n 2 x², and ((d + d2) + kn)e- 2) for n.
x?
Theorem and Corollary 2 provide the first convergence analysis and the query complexity for the variance-reduced zeroth-order algorithms for min-max optimization. These two results indicate that the query complexity of ZO- BREDA-BOos matches the optimal dependence on c of the first-order algorithm SREDA -Boost in Theorem and Corollary 1. The dependence on d1 and d2 typically arises in zeroth-order algorithms due to the estimation of gradients with dimensions d, and d2. Furthermore, in the online case, ZO-SREDA-Bood on e among the existing zeroth- algorithms by a factor of (1/e). Including the conditional number k into consideration, SREDA-Boud outperforms the best known query complexity achieved by 2O-SGDMA in the case with e S K (see Table 1). Furthermore, Corollary 2 provides the first query complexity for the finite-sum zeroth-order min-max problems.
outperforms the best known query complexity dependence
As a by-product, our analysis of ZO-SREDA SREDA-Boost also yields the convergence rate and the query complexity (see Lemma 21) for ZO-iSARAH for the conventional minimization problem, which provides the first complexity result for the zeroth-order recursive variance reduced algorithm SARAH/SPIDER for strongly convex optimization (see Appendix C.4 for detail).
EXPERIMENTS
Our experiments focus on two types of comparisons. First, we compare SREDA BREDA-Boost with SREDA to demonstrate the practical advantage of SREDA-Boost. Second, we compare our proposed zeroth-
7

---

Under review as a conference paper at ICLR 2021
order variance reduction algorithm ZO-SREDA --BsD with the other existing zeroth-order stochastic algorithms and demonstrate the superior performance of DEIZO-SREDEE CREDA-BoosS,
Our experiments solve distributionally robust optimization problem, which is commonly used for studying min-max optimization Lin et al. (2019); Rafique et al. (2018). We conduct the experiments on three datasets from LIBSVM Chang & Lin (2011). The details of the problem and the datasets provided in Appendix A.
We set 0.001 for both algorithms. For SREDA, we set Ox, min(f/ ||eell2 0.005) as specified by the algorithm, and for SREDA-Boost, we set 0+ = 0.005 the algorithm allows. It can be seen in Figure I that SREDA-Boost enjoys much faster convergence speed than SREDA due to the allowance of a large stepsize.
Comparison between SREDA-Boost and SREDA:
SREDA
SREDA
1.5
SREDA
SREDA Boost
SREDA-BBost
SREDA
20.
lioss
04
1050
1003
2000
3000
2003
<05e 6000
number stochastic gradients/m
number of stochastic gradientsin
number of stochastic gradientsin
(a) Dataset: a9a
(h) Dataset: w8a
(c) Dataset: mushrooms
Figure 1: Comparison of the convergence rate between SREDA-Boosi and SREDA.
We compare the performance of our proposed ZO- SREDA-Bou -Bosst with that of two existing stochastic algorithms ZO-SGDA Wang et al. (2020) and ZO-SGDMSA Wang et al. (2020) designed for nonconvex sarcoovee--ttoonlyyy strongly-concave min-max problems. For ZO-SGDA and ZO-SGDMSA, as suggested by the theorem, we set the mini-batch size B = Cd//² and B = Cdz/e² for updating the variables x and y, respectively, For ZO-SREDA-Boost. based on our theory, we set the mini-batch size B = Cdi/e /e and B = Cdz/e for updating the variables x and y. and set $ = n for the large batch, where n is the number of data samples in the dataset. We set C' = 0.1 and = 0.1 for all algorithms. We further set the stepsize 17 0.01 for 2O-SREDA-Boost and ZO-SGDMSA. Since ZO-SGDA is a two time-scale algorithm, we set 17 = 0.01 as the stepsize for the fast time scale, and n/ 1/113 as the stepsize for slow time scale (based on the theory) where R3 = 10. It can be seen in Figure 2 that ZO-SREDA-BOOS substantially outperforms the other two algorithms in terms of the function query complexity (i.e., the running time).
Comparison among zeroth-order Algorithms:
13
Z0-SGDA
ZO-SGDA
ZO-SGDA
ZO.SGDMSA
ZO-SGDMSA ZO-SREDA-Boost
ZO-SGDMSA
3.-
ZO-SREDA-Boost
ZO-SREDA-Boost
0.55
loos
loss
0.8
0.50
0.5
9:30
200020
number of function queriesin
number of function queriesm
number of function queriesi
(a) Dataset: a9a
(b) Dataset: w8a
(c) Dataset: mushrooms
Figure 2: Comparison of function query complexity among three algorithms.
7 CONCLUSION
In this work, we have proposed enhanced variance reduction algorithms, which we call SREDA- -Bostt and DO-SREDA-Booss for solving mmmonnex-stronnlyyooo conccve min-max problems. In specific, BREDA-Boose requires less initialization effort and allows a large stepsize. Moreover, The complexity of SREDA-Boost and CO-SREDA-BooR achieves the best complexity dependence on the targeted accuracy among their same classes of algorithms. We have also developed a novel analysis framework to characterize the convergence and computational complexity for the variance reduction algorithms. We expect such a framework will be useful for studying various other stochastic min-max problems such proximal, momentum, and manifold optimization.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Zeyuan Allen-Zhu Natasha: faster non non-convex stochastic optimization via strongly non-convex parameter. In Proc. International Conference on Machine Learning (ICML), pp. 89-97, 2017.
Zeyuan Allen-Zhu and Elad Hazan. Variance reduction for faster non-convex optimization. In Proc. International Conference Machine Learning (ICML), pp. 699-707, 2016.
Yossi Arjevani, Yair Carmon, John C Duchi, Dylan J Foster, Nathan Srebro, and Blake Woodworth. Lower bounds for non-convex stochastic optimization. arXiv preprint arXiv:192..02 1912.0239,, 2019.
Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, and Volkan Cevher. Adversarially robust optimization with gaussian processes. Proc. Advances in Neural Information Processing Systems (NeurIPS), pp. 5760-5770, 2018.
C. Chang and C. Lin. LIBSVM: A library for support vector machines. ACM Transactions Intelligent Systems and Technology, 2(3):1-27, 2011.
Tatjana Chavdarova, Gauthier Gidel, François Fleuret, and Simon Lacoste-Julien. Reducing noise in GAN training with variance reduced extragradient. In Proc. Advances in Neural Information Processing Systems (NIPS), pp. 391-401, 2019.
Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. In Proc. Advances in Neural Information Processing Systems (NIPS), pp. 1646-1654, 2014.
Simon S Du and Wei Hu. Linear convergence of the primal- duilll-dull gradient method for convex-coneave saddle point problems without strong convexity. In Proc. International Conference on Artificial Intelligence and Statisties (AISTATS), pp. 196-202 205, 2019.
Simon S Du, Jianshu Chen, Lihong Li, Lin Xiao, and Dengyong Zhou. Stochastic variance reduction methods for policy evaluation. In Proc. International Conference Machine Learning (ICML), pp. 1049-1058, 2017.
Cong Fang, Chris Junchi Li. Zhouchen Lin, and Tong Zhang. SPIDER: Near-optimal non-convex op- timization via stochastic path-integrated differential estimator. In Advances Neural Information Processing Systems (NeurIPS), pp. 689-699, 2018.
Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM Journal on Optimization, 23(4)):33412366) 2013.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proc. Advances in Neural Information Processing Systems (NIPS), pp. 2672-2680, 2014.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In Proc. Advances in Neural Information Processing Systems (NIPS), pp. 4565-45733 2016.
Kaiyi Ji, Zhe Wang, Yi Zhou, and Yingbin Liang. Improved zeroth-order variance reduced algorithms and analysis for nonconvex optimization. In Proc. International Conference on Machine Learning, pp. 3109-3109, 2019.
Chi Jin, Praneeth Netrapalli, and Michael Jordan. Minmax optimization: stable limit points of gradient descent ascent are locally optimal. arXiv preprint arXiv: 10XXvv:900.000 1902.0068, 2019.
Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Proc, Advances in Neural Information Processing Systems (NIPS), pp. 315-323, 2013.
Yuan-Chuan Li and Cheh-Chih Yeh. Some equivalent forms of Bernoulli's inequality: A survey. Applied Mathematics, 1007) 2013.
Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-concan minimax problems. arXiv preprint 1rXiv:1906.003: 2019.

---

Under review as a conference paper at ICLR 2021
Sijia Liu, Songtao Lu, Xiangyi Chen, Yao Feng, Kaidi Xu, Abdullah Al-Dujaili, Minyi Hong, and Una-May Obelilly. Min-max optimization without gradients: convergence and applications to adversarial ML. arXiv preprint 1tXiv:1909.13800 2019.
Songtao Lu, Ioannis Tsaknakis, Mingyi Hong. and Yongxin Chen. Hybrid block successive ap- proximation for one-sided non-convex min-max problems: algorithms and applications. IEEE Transactions on Signal Processing, PP:1-1, 04 2020.
Luo Luo, Cheng Chen, Yujun Li, Guangzeng Xie, and Zhihua Zhang. stochastic proximal point algorithm for saddle-point problems. arXiv preprint 195.:::999009 2019.
Luo Luo, Haishan Ye, and Tong Zhang. Stochastic recursive gradient descent ascent for stochastic nonconneex--tronngly-ooooee minimax problems. arXiv preprint arXiv:2001.037244 2020.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In Proc. International Conference on Learning Representations (ICLR), 2018
Matt Menickelly and Stefan M Wild. Derivative-free robust optimization by outer approximations. Mathematical Programming, 179(1-2)157-119, 2020.
Yurii Nesterov. Introductory Lectures on Conver Optimization: A Basic Course, volume 87. Springer Science & Business Media, 2013.
Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17(2):57--56, 2017.
Lam M Nguyen, Jie Liu, Katya Scheinberg, and Martin Takáč. SARAH: A novel method for machine learning problems using stochastic recursive gradient. In Proc. International Conference on Machine Learning (ICML), pp. 2613-2621, 2017a.
Lam M Nguyen, Jie Liu, Katya Scheinberg, and Martin Takáč, Stochastic recursive gradient algorithm for nonconvex optimization. arXiv preprint arXiv: 1705.0766,, 2017b.
Lam M Nguyen, Katya Scheinberg, and Martin Takáč. Inexact SARAH algorithm for stochastic optimization. arXiv preprint arXiv:1811.10111 2018.
Maher Nouiehed, Maziar Sanjabi, Tianjian Huang, Jason D Lee, and Meisam Razaviyayn. Solving class of non-convex min-max games using iterative first order methods. In Proc. Advances in Neural Information Processing Systems (NeurIPS), pp. 14905-14916, 2019.
Balamurugan Palaniappan and Francis Bach. Stochastic variance reduction methods for saddle-point problems. In Proc. Advances in Neural Information Processing Systems (NIPS), pp. 1416-1424, 2016.
Boris Teodorovich Polyak. Gradient methods for minimizing functionals. Zhurnal Vychislitel noi Matematiki Matematicheskoi Fiziki, 3(4):643-655 653, 1963.
Hassan Rafique, Mingrui Liu, Qihang Lin, and Tianbao Yang. Non-conver min-max optimization: provable algorithms and applications machine learning. arXiv preprint arXiv:1810.02066 2018
Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabas Poczos, and Alex Smola. Stochastic variance reduction for nonconvex optimization. In Proc. International Conference on Machine Learning (ICML), 2016.
Abhishek Roy, Yifang Chen, Krishnakumar Balasubramaniann, and Prasant Mohapatra. Online and bandit algorithms for nonstationary stochastic saddle-point optimization. arXiv preprint arXiv:19/2.01699, 2019.
Kiran K Thekumparampil. Prateek Jain, Pranceth Netrapalli, and Sewoong Oh. Efficient algorithms for smooth minimax optimization. Proc. Advances in Neural Information Processing Systems (NeurIPS), 2019.
10

---

Under review as a conference paper at ICLR 2021
Hoi-To Wai, Mingyi Hong. Zhuoran Yang, Zhaoran Wang. and Kexin Tang. Variance reduced policy evaluation with smooth function approximation. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, and Vahid Tarokh. SpiderBoost: A class of faster variance-reduced algorithms for nonconvex optimization. In Proc. Advances Neural Information Processing Systems (NeurIPS), 2019.
Zhongruo Wang, Krishnakumar Balasubramanian, Shiqian Ma, and Meisam Razaviyayn. Zeroth- order algorithms for nonconvex minimax problems with improved complexities. arXiv preprint arXiv:2001.07819 2020.
Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient bayesian optimization. In Proc. International Conference Machine Learning (ICML), 2017.
Chen-Y Yu Wei, Yi-Te Hong, and Chi-Jen Lu. Online reinforcement learning in stochastic games, In Proc. Advances in Neural Information Processing Systems (NIPS), pp. 4987-4997, 2017.
Junchi Yang, Negar Kiyavash, and Niao He. Global convergence and variance-reducce optimizaaiinnn for a class of conconvex-nonconee
minimax problems. arXiv preprint arXiv:2002.09621, 2020.
Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent remtorrmmmeett learning: selective overview of theories and algorithms. arXiv preprint
2019.


Reviews:
[{"review_id": "Qm-l7b31qv3", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a SREDA-Boost, which builds upon SREDA for nonconvex-strongly-concave minimax problem. The SREDA-Boost algorithm is less restrictive to initialization and has an accuracy-independent and larger step size. Thus it can run substantially faster than SREDA. The main contribution to the first-order optimization story is a new analytical framework that builds upon the previous analysis in SREDA and overcomes the dependence of highly accurate initialization via bounding the tracking error and gradient estimation error separately. It also proposes a zeroth-order variance reduction algorithm for the same optimization problem, which has the largest possible step size so far and also improves the complexity of the state-of-the-art in some cases. Various experiments have validated the superiority. The theoretical analysis and empirical results look good to me.\n\n\nMy main concern is the significance of the contribution. It seems that SREDA-Boost is basically the same as SREDA except for stepsize and initialization complexity, which makes the contribution marginal. Though the analysis is definitely new. For the zero-th order optimization problem, it seems that the previous algorithm ZO-SGDMSA does not require good initialization and has the same step size. The complexity of it is also comparable to ZO-SREDA-Boost when kappa is approximately 1/eps. It would be better if the authors could discuss a bit more the initialization requirement of both algorithms and the query complexity, along with the novelty in both first order and zero-th order story.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "IpQDbo0YV-T", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an enhanced variant of the SREDA algorithm (Lou et al 2020), called SREDA-Boost, that improves SREDA on two aspects: the initial complexity and the step-size. The algorithm achieves the same complexity as the original SREDA scheme.\n\nThe main contribution of this paper is perhaps the following aspects:\n-- C1: Improving the initial complexity for finding a starting point y0 of the SREDA.\n-- C2: Proposing a larger stepsize for SREDA compared to the original one.\n-- C3: Injecting a zero-order approximation step for stochastic gradients.\nThe authors also claimed that their analysis is new and different from SREDA. However, in my opinion, this seems to be minor since their proof also relies on the bounds of gradient errors (like variance) as well as the delta_t quantity. There are of course some technical details and steps, but those are not the major contribution.\n\nIn my opinion, the theoretical contribution of this paper is incremental. Indeed, the idea of using SARAH to improve oracle complexity has been widely studied in the literature, including Spider, SpiderBoost, ProxSARAH, etc. Since model (1) is nonconvex-strongly concave, it can be reformulated into (2) as a stochastic optimization problem. Several methods can be used to solve (2). The idea of using multiple loops is also widely used. \n\nThe first contribution (C1) is not really new. The authors simply replace the step of computing y0 by iSARAH to reduce the computational cost. Because the problem is strongly convex, this step can be done by several methods, including accelerated variance-reduced schemes to further improve its complexity. Since this step is not essential, many previous methods just simply skip it.\n\nThe second contribution (C2) is also minor since the idea of using a large batch-size to obtain large step-size has been used in the literature such as Spider-Boost (Wang et al 2019) or ProxSarah (Pham et al 2020). Of course, this enhancement helps SREDA have better practical performance. However, given the previous work, this contribution is very incremental.\nThe use of zero-order oracle (C3) is not new as well, since it is another way of approximating the gradients, and it has been widely used in the literature based on Nesterov's idea. Hence, this step seems to be unnecessary if we assume that the underlying function is L-smooth. In most applications, we can directly compute the stochastic gradient components without using finite difference approximation. Unless the authors can provide compelling examples showing that this is an important step, otherwise, it is not really convinced. \n\nIn terms of algorithm, SREDA/SREDA-Boost is also a multiple-loop algorithm, with at least three loops, making it challenging to implement in practice and it requires a lot of proper tuning and choosing parameters. There are some recent algorithms that can solve the same problems but with a single loop. The authors may want to compare with them, though these are very recent work (see, e.g. https://arxiv.org/abs/2008.08170 and the references therein).\n\nIn addition to the above major comments, the following are some concrete comments:\n--- In Table 1, why NA is put in the \"Initial Complexity\" column of other methods. I believe that some paper may not describe this clearly, but if they use the exact solution (or up to a given accuracy approximation) of the strongly-concave max-problem, then the complexity is the best one since we can use the best algorithm to find it.\n-- The relation between the gradient of $\\Phi$ and the KKT point of (1) should be clarified. \n-- I do not see where v_t is used in Algorithm 2. It seems that the output of Algorithm 2 should include v_t to use in Algorithm 1.\n-- It is not clear what is the problem solved in the numerical experiments?\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "HSaTxilmEDl", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the nonconvex strongly-concave min-max optimization problem. It improves the analysis of an existing method SREDA to make it allow larger step size and less initialization computation. Besides, it extends the algorithm to the case where the objective function is non-differentiable. The authors claimed it is the first zeroth-order variance-reduced method for the min-max problem. Experiments are conducted to demonstrate the improved algorithm is better than existing methods.\n\nThe results of this paper are interesting. However, I don't think the initialization complexities really matter, since they are always dominated by the complexity of the later optimization process. Hence, improving initialization complexity does not make any change to the overall complexity of the whole algorithm. I think the key to making SREDA-Boost performs better than SREDA is the larger step size.\n\nThe presentation of this paper, especially for the algorithms, need to be improved. In line 3 of Algorithm 1, function iSARAH is called. However, the meaning of its two parameters is not defined. It is unclear which variables these two parameters correspond to in iSARAH (Algorithm 3 in appendix). Besides, the \\tilde{v}_{t-1,\\tilde{m}_{t-1}} and \\tilde{u}_{t-1,\\tilde{m}_{t-1}} in line 8 are not defined there. Though these two variables are introduced in Algorithm 2, algorithms need to be self-contained. Furthermore, when $t=0$, line 8 seems wrong.", "extra_scores": {}, "extra_text": {}}, {"review_id": "BrYJ5b4xwse", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a variant SREDA-Boost of the variance reduction method SEDRA for solving nonconvex-strongly-concave min-max problem. The first contribution of the paper is to relax the conditions on the initialization  of SEDRA and moreover enable larger stepsizes ($\\epsilon$-independent stepsizes). As SEDRA is already optimal, such modification does not improve the theoretical convergence rate, but it is beneficial from the practical perspective. The second contribution is to adapt the method to zero order oracle, achieving the state-of-the-art convergence rate. \n\nThe result are presented in a clear and technical manner. My major concern is the lack of high level explanation on why a larger stepsize can be applied. I understand that the paper introduces a novel way for the complexity analysis, however the proof is very long and not easy to check. Hence, it would be helpful to explain in words the key aspects that allows an $\\epsilon$-independent stepsize (in a non-technical manner).  \n\nAnother question I would like to ask is whether the Boost version is necessary to deduce the zeroth order variant. In other words, what would be the complexity of ZO-SREDA. I would expect that it shares the same complexity as  ZO-SREDA-BOOST, even though the stepsize are smaller. Please clarify this point.\n\nThe experiments are rather limited but it is fine for a theoretical paper.  I would expect including more comparisons on competitive methods in Figure 1, as in the current version the only baseline is ZO-SREDA. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a new variant (SREDA-Boost) of a variance-reduced method SEDRA for nonconvex-strongly-concave min-max optimization. Given that SEDRA is already optimal in the worst case, the proposed modification is intended to improve practical performance of the method, by relaxing conditions needed at initialization and allowing larger step sizes. While the reviewers appreciated the main ideas of the paper, they shared concerns about the significance of the paper's technical contributions, which were ultimately not addressed by the authors in the rebuttal phase. 
Decision: Reject