Title: Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimization
Venue: NeurIPS 2024
Accepted: yes
Keywords: Shuffling gradient method; nonconvex-concave minimax problem; oracle complexity; sample without replacement
Abstract: This paper aims at developing novel shuffling gradient-based methods for tackling two classes of minimax problems: nonconvex-linear and nonconvex-strongly concave settings. The first algorithm addresses the nonconvex-linear minimax model and achieves the state-of-the-art oracle complexity typically observed in nonconvex optimization.  It also employs a new shuffling estimator for the ``hyper-gradient'', departing from standard shuffling techniques in optimization. The second method consists of two variants: semi-shuffling and full-shuffling schemes.  These variants tackle the nonconvex-strongly concave minimax setting.  We establish their oracle complexity bounds under standard assumptions, which, to our best knowledge, are the best-known for this specific setting. Numerical examples demonstrate the performance of our algorithms and compare them with two other methods.  Our results show that the new methods achieve comparable performance with SGD, supporting the potential of incorporating shuffling strategies into minimax algorithms.

Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimization

Abstract

This paper aims at developing novel shuffling gradient-based methods for tackling two classes of minimax problems: nonconvex-linear and nonconvex-strongly concave settings. The first algorithm addresses the nonconvex-linear minimax model and achieves the state-of-the-art oracle complexity typically observed in nonconvex optimization. It also employs a new shuffling estimator for the 'hyper-gradient', departing from standard shuffling techniques in optimization. The second method consists of two variants: semi-shuffling and full-shuffling schemes. These variants tackle the nonconvex-strongly concave minimax setting. We establish their oracle complexity bounds under standard assumptions, which, to our best knowledge, are the best-known for this specific setting. Numerical examples demonstrate the performance of our algorithms and compare them with two other methods. Our results show that the new methods achieve comparable performance with SGD, supporting the potential of incorporating shuffling strategies into minimax algorithms.

1 Introduction

Minimax problems arise in various applications across generative machine learning, game theory, robust optimization, online learning, and reinforcement learning (e.g., [1, 2, 3, 5, 12, 13, 17, 19, 21, 25, 35, 40]). These models often involve stochastic settings or large finite-sum objective functions. To tackle these problems, existing methods frequently adapt stochastic gradient descent (SGD) principles to develop algorithms for solving the underlying minimax problems [4, 13]. For instance, in generative adversarial networks (GANs), early algorithms employed stochastic gradient descentascent methods where two routines, each using an SGD loop, ran iteratively [13]. However, practical implementations of SGD often incorporate shuffling strategies, as seen in popular deep learning libraries like TensorFlow and PyTorch. This has motivated recent research on developing shuffling techniques specifically for optimization algorithms [4, 5, 8, 16, 26, 32, 38]. Our work builds upon this trend by developing shuffling methods for two specific classes of minimax problems.

Problem statement. In this paper, we study the following minimax optimization problem:

where f : R p → R ∪ { + ∞ } is a proper, closed, and convex function, H i : R p × R q → R are smooth for all i ∈ [ n ] := { 1 , 2 , · · · , n } , and h : R q → R ∪ { + ∞ } is also a proper, closed, and convex function. In this paper, we will focus on two classes of problems in (1), overlapped to each other.

(NL) H i is nonconvex in w and linear in u as H i ( w,u ) := 〈 F i ( w ) , Ku 〉 for a given function F i : R p → R m and a matrix K ∈ R q × m for all i ∈ [ n ] and ( w,u ) ∈ dom( L ) .

(NC) H i is nonconvex in w and H i ( w, · ) -h ( · ) is strongly concave in u for all ( w,u ) ∈ dom( L ) .

Although (NC) looks more general than (NL), both cases can be overlapped, but one is not a special case of the other. Under these two settings, our approach will rely on a bilevel optimization approach, where the lower-level problem is to solve max u L ( w,u ) , while the upper-level one is min w L ( w,u ) .

Challenges. The setting (NL) is a special case of stochastic nonconvex-concave minimax problems because the objective term H ( w,u ) := 〈 F ( w ) , Ku 〉 is linear in u . It is equivalent to the compositional model (CO) described below. However, if h is only merely convex and not strongly convex (e.g., the indicator of a standard simplex), then Φ 0 in (CO) becomes nonsmooth regardless of F 's properties. This presents our first challenge. A natural approach to address this issue, as discussed in Section 2, is to smooth Φ 0 . The second challenge arises from the composition between the outer function h ∗ and the finite sum F ( · ) in (CO). Unlike standard finite-sum optimization, this composition prevents any direct use of existing techniques, requiring a novel approach for algorithmic development and analysis. The third challenge involves unbiased estimators for gradients or 'hyper-gradients' in minimax problems. Most existing methods rely on unbiased estimators for objective gradients, with limited work exploring biased estimators. While biased estimators can be used, they require variance reduction properties (see, e.g., [10]). The setting (NC) faces the same second and third challenges as the setting (NL). Additionally, when reformulating it as a minimization problem using a bilevel optimization approach (3), constructing a shuffling estimator for the 'hyper-gradient' ∇ Φ 0 becomes unclear. This requires solving the lower-level maximization problem (2). Therefore, it remains an open question whether shuffling gradient-type methods can be extended to this bilevel optimization approach to address (1). In this paper, we address the following research question:

Can we efficiently develop shuffling gradient methods to solve (1) for both (NL) and (NC) settings?

Our attempt to tackle this question leads to a novel way of constructing shuffling estimators for the hyper-gradient ∇ Φ 0 or its smoothed counterpart. This allows us to develop two shuffling gradientbased algorithms with rigorous theoretical guarantees on oracle complexity, matching state-of-the-art complexity results in shuffling-type algorithms for nonconvex optimization.

Related work. Shuffling optimization algorithms have gained significant attention in optimization and machine communities, demonstrating advantages over standard SGDs, see, e.g., [4, 5, 8, 16, 26, 32, 38]. Nevertheless, applying these techniques to minimax problems like (1) remains challenging, with limited existing literature (e.g., [3, 8, 11]). Das et al. in [8] explored a specific case of (1) without nonsmooth terms f and h , assuming strong monotonicity and L -Lipschitz continuity of the gradient ∇ H := [ ∇ w H , -∇ u H ] of the joint objective H . Their algorithm simplifies to a shuffling variant of fixed-point iteration or a gradient descent-ascent scheme, not applicable to our settings. Cho and Yun in [3] built upon [8] by relaxing the strong monotonicity to PolyakŁ ojasiewicz (P Ł ) conditions. This work is perhaps the most closely related one to our algorithm, Algorithm 2, for the (NC) setting. Note that the method in [3] exploits Nash's equilibrium perspective with a simultaneous update, which is different from our alternative update. Moreover, [3] only considers the noncomposite case with f = 0 and h = 0 . Though we only focus on a nonconvex-strongly-concave setting (NC), our results here can be extended to the P Ł condition as in [3]. Very recently, Konstantinos et al. in [11] introduced shuffling extragradient methods for variational inequalities, which encompass convex-concave minimax problems as a special case. However, this also falls outside the scope of our work due to the nonconvexity of (1) in w . Again, all the existing works in [3, 8, 11] utilize a Nash's equilibrium perspective, while ours leverages a bilevel optimization technique. Besides, in contrast to our sampling-without-replacement approach, stochastic and randomized methods (i.e. using i.i.d. sampling strategies) have been extensively studied for minimax problems, see, e.g., [9, 14, 15, 18, 22, 23, 31, 37, 42]. A comprehensive comparison can be found, e.g., in [3].

Contribution. Our main contribution can be summarized as follows.

(a) For setting (NL), we suggest to reformulate (1) into a compositional minimization and exploit a smoothing technique to treat this reformulation. We propose a new way of constructing shuffling estimators for the 'hyper-gradient' ∇ Φ γ (cf. (10)) and establish their properties.

(b) We propose a novel shuffling gradient-based algorithm ( cf. Algorithm 1) to approximate an /epsilon1 -KKT point of (1) for the setting (NL). Our method requires O ( n /epsilon1 -3 ) evaluations of F i and ∇ F i under the strong convexity of h , and O ( n /epsilon1 -7 / 2 ) evaluations of F i and ∇ F i without the strong convexity of h , for a desired accuracy /epsilon1 > 0 .

(c) For setting (NC), we develop two variants of the shuffling gradient method: semi-shuffling and full-shuffling schemes ( cf. Algorithm 2). The semi-shuffling variant combines both gradient ascent and shuffling gradient methods to construct a new algorithm, which requires O ( n /epsilon1 -3 ) evaluations of both ∇ w H i and ∇ u H i . The full-shuffling scheme allows to perform both shuffling schemes on the maximization and the minimization alternatively, requiring either O ( n /epsilon1 -3 ) or O ( n /epsilon1 -4 ) evaluations of ∇ u H i depending on our assumptions, while maintaining O ( n /epsilon1 -3 ) evaluations of ∇ w H i for a given desired accuracy /epsilon1 > 0 .

If a random shuffling strategy is used in our algorithms, then the oracle complexity in all the cases presented above is improved by a factor of √ n . Our settings (NL) and (NC) of (1) are different from existing works [3, 8, 11], as we work with general nonconvexity in w , and linearity or [strong] concavity in u , and both f and h are possibly nonsmooth. Our algorithms are not reduced or similar to existing shuffling methods for optimization, but we use shuffling strategies to form estimators for the hyper-gradient ∇ Φ 0 in (5). The oracle complexity in both settings (NL) and (NC) is similar to the ones in nonconvex optimization and in a special case of (1) from [3] (up to a constant factor).

Paper outline. The rest of this paper is organized as follows. Section 2 presents our bilevel optimization approach to (1) and recalls necessary preliminary results. Section 3 develops our shuffling algorithm to solve the setting (NL) of (1) and establishes its convergence. Section 4 proposes new shuffling methods to solve the setting (NC) and investigates their convergence. Section 5 presents numerical experiments, while technical proofs and supporting results are deferred to Supp. Docs.

Notations. For a function f , we use dom( f ) to denote its effective domain, and ∇ f for its gradient or Jacobian. If f is convex, then ∇ f denotes a subgradient, ∂ f is its subdifferential, and prox f is its proximal operator. We use F t to denote σ ( w 0 , w 1 , · · · , w t ) , a σ -algebra generated by random vectors w 0 , w 1 , · · · , w t , E t [ · ] = E [ ·|F t ] is a conditional expectation, and E [ · ] is the full expectation. As usual, O ( · ) denotes Big-O notation in the theory of algorithm complexity.

2 Bilevel Optimization Approach and Preliminary Results

Our approach relies on a bilevel optimization technique [9] in contrast to Nash's game viewpoint [24], which treats the maximization as a lower level and the minimization as an upper level problem.

2.1 Bilevel optimization approach

The minimax model (1) is split into a lower-level ( i.e. a follower ) maximization problem of the form:

For Φ 0 defined by (2), then the upper-level ( i.e. the leader ) minimization problem can be written as

Clearly, this approach is sequential, and only works if Φ 0 is well-defined, i.e. (2) is globally solvable. Hence, the concavity of H ( w, · ) -h ( · ) w.r.t. to u is crucial for this approach as stated below. However, this assumption can be relaxed to a global solvability of (2) combined with a P Ł condition as in [3].

Assumption 1 (Basic) . Problems (1) and (3) satisfy the following assumptions for all i ∈ [ n ] :

(a) Ψ /star 0 := inf w Ψ 0 ( w ) > -∞ .

(c) Both f : R p → R ∪ { + ∞ } and h : R q → R ∪ { + ∞ } are proper, closed, and convex.

(b) H i is differentiable w.r.t. ( w,u ) ∈ dom( L ) and H i ( w, · ) is concave in u for any w .

This assumption remains preliminary. To develop our algorithms, we will need more conditions on H i and possibly on f and h , which will be stated later. In addition, we can work with a sublevel set

of Ψ 0 for a given initial point w 0 from our methods. If u ∗ 0 ( w ) is uniquely well-defined for given w ∈ L Ψ 0 ( w 0 ) , then by the well-known Danskin's theorem, Φ 0 is differential at w and its gradient is

We adopt the term 'hyper-gradient' from bilevel optimization to name ∇ Φ 0 in this paper.

2.2 Technical assumptions and properties of Φ 0 for nonconvex-linear setting (NL)

(a) Compositional minimization formulation . If H i ( w,u ) := 〈 F i ( w ) , Ku 〉 as in setting (NL), then (1) is equivalently reformulated into the following nonconvex compositional minimization problem:

where h ∗ ( v ) := sup u { 〈 v, u 〉 -h ( u ) } , the Fenchel conjugate of h , and Φ 0 ( w ) = h ∗ ( K T F ( w )) . If h is not strongly convex, then h ∗ is convex but possibly nonsmooth.

(b) Technical assumptions . To develop our algorithms, we also need the following assumptions.

Assumption 2. h is µ h -strongly convex with µ h ≥ 0 , and dom( h ) is bounded by M h < + ∞ . Assumption 3 (For F i ) . For setting (NL) with H i ( w,u ) := 〈 F i ( w ) , Ku 〉 ( i ∈ [ n ]) , assume that

( a) F i is continuously differentiable, and its Jacobian ∇ F i is L F i -Lipschitz continuous.

(c) There exists a positive constant σ J ∈ (0 , + ∞ ) such that

(b) F i is also M F i -Lipschitz continuous or equivalently, its Jacobian ∇ F i is M F i -bounded.

Assumption 2 allows µ h = 0 that also covers the non-strong convexity of h . Assumption 3 is rather standard to develop gradient-based methods for solving (1). Under Assumption 3, the finite-sum F is also M F -Lipschitz continuous and the Jacobian ∇ F of F is also L F -Lipschitz continuous with

Condition (6) can be relaxed to the form 1 n ∑ n i =1 ‖∇ F i ( w ) -∇ F ( w ) ‖ 2 ≤ σ 2 J + Θ J ‖∇ Φ 0 ( w ) ‖ 2 for some Θ J ≥ 0 , where ∇ Φ 0 is a [sub]gradient of Φ 0 or Φ γ (its smoothed approximation). Moreover, under Assumption 3, if µ h > 0 , then ∇ h ∗ is L h ∗ -Lipschitz continuous with L h ∗ := 1 µ h . Thus it is possible (see [9]) to prove that Φ 0 is differentiable, and ∇ Φ 0 is also L Φ 0 -Lipschitz continuous with L Φ 0 := M h ‖ K ‖ L F + M 2 F ‖ K ‖ 2 µ h as a consequence of Lemma 4 when γ ↓ 0 + in Supp. Doc. A.

(c) Smoothing technique for lower-level maximization problem (2) . If h is only merely convex (i.e. µ h = 0 ), then (2) may not be uniquely solvable, leading to the possible non-differentiability of Φ 0 . Let us define the following convex function:

Then, Φ 0 in (2) or (CO) can be written as Φ 0 ( w ) = φ 0 ( F ( w )) = φ 0 ( 1 n ∑ n i =1 F i ( w ) ) . Our goal is to smooth φ 0 if h is not strongly convex, leading to where γ > 0 is a given smoothness parameter and b : R q → R is a proper, closed, and 1 -strongly convex function such that dom( h ) ⊆ dom( b ) . We also denote D b := sup { ‖∇ b ( u ) ‖ : u ∈ dom( h ) } . In particular, if we choose b ( u ) := 1 2 ‖ u -¯ u ‖ 2 for a fixed ¯ u , then u ∗ γ ( v ) = prox h/ γ (¯ u -K T v ) .

Using φ γ , problem (CO) can be approximated by its smoothed formulation:

To develop our method, one key step is to approximate the hyper-gradient of Φ γ in (10), where

Then, ∇ Φ γ is L Φ γ -Lipschitz continuous with L Φ γ := M h ‖ K ‖ L F + M 2 F ‖ K ‖ 2 µ h + γ (see Lemma 4).

2.3 Technical assumptions and properties of Φ 0 for the nonconvex-strongly-concave setting

To develop our shuffling gradient-based algorithms for solving (1) under the nonconvex-stronglyconcave setting (NC), we impose the following assumptions.

Assumption 4 (For H i ) . H i for all i ∈ [ n ] in (1) satisfies the following conditions:

(a) For any given w such that ( w,u ) ∈ dom( H ) , H i ( w, · ) is µ H -strongly concave w.r.t. u .

(b) ∇ H i is ( L w , L u ) -Lipschitz continuous, i.e. for all ( w,u ) , ( ˆ w, ˆ u ) ∈ dom( H ) :

(c) There exist two constants Θ w ≥ 0 and σ w ≥ 0 such that for ( w,u ) ∈ dom( H ) , we have

There exist two constants Θ u ≥ 0 and σ u ≥ 0 such that for all ( w,u ) ∈ dom( H ) , we have

Assumption 4(a) makes sure that our lower-level maximization of (1) is well-defined. Assumption 4(b) and (c) are standard in shuffling gradient-type methods as often seen in nonconvex optimization [9]. Lemma 1 (Smoothness of Φ 0 ) . Under Assumptions 2 and 4, u ∗ 0 ( · ) in (2) is κ -Lipschitz continuous with κ := L u µ H + µ h . Moreover, ∇ Φ 0 in (5) is L Φ 0 -Lipschitz continuous with L Φ 0 := (1 + κ ) L w .

2.4 Approximate KKT points and approximate stationary points

(a) Exact and approximate KKT points and stationary points . A pair ( w /star , u /star ) ∈ dom( L ) is called a KKT (Karush-Kuhn-Tucker) point of (1) if

Given a tolerance /epsilon1 > 0 , our goal is to find an /epsilon1 -approximate KKT point ( ̂ w, ̂ u ) of (1) defined as r w ∈ ∇ w H ( ̂ w, ̂ u ) + ∂ f ( ̂ w ) , r u ∈ -∇ u H ( ̂ w, ̂ u ) + ∂ h ( ̂ u ) , and E [ ‖ [ r w , r u ] ‖ 2 ] ≤ /epsilon1 2 . (16) A vector w /star ∈ dom( Ψ 0 ) is said to be a stationary point of (3) if

Since f is possibly nonsmooth, we can define a stationary point of (3) via a gradient mapping as:

where η > 0 is given. It is well-known that G η ( w /star ) = 0 iff w /star is a stationary point of (3). Again, since we cannot exactly compute w /star , we expect to find an /epsilon1 -stationary point ̂ w T of (3) such that E [ ‖ G η ( w T ) ‖ 2 ] ≤ /epsilon1 2 for a given tolerance /epsilon1 > 0 .

̂ (b) Constructing an approximate stationary point and KKT point from algorithms . Our algorithms below generate a sequence { ˜ w t } T t ≥ 0 such that 1 T +1 ∑ T t =0 E [ ‖ G η ( ˜ w t ) ‖ 2 ] ≤ /epsilon1 2 . Hence, we construct an /epsilon1 -stationary point w T using one of the following two options:

Clearly, we have E [ ‖ G η ( ̂ w T ) ‖ 2 ] ≤ 1 T +1 ∑ T t =0 E [ ‖ G η ( ˜ w t ) ‖ 2 ] ≤ /epsilon1 2 . We need the following result. Lemma 2. (a) If ( w /star , u /star ) is a KKT point of (1) , then w /star is a stationary point of (3) . Conversely, if w /star is a stationary point of (3) , then ( w /star , u ∗ 0 ( w /star )) is a KKT point of (1) .

(b) If ̂ w T is an /epsilon1 -stationary point of (3) and ∇ Φ 0 is L Φ 0 -Lipschitz continuous, then ( w T , u T ) is an ˆ /epsilon1 -KKT point of (1) , where w T := prox η f ( w T -η ∇ Φ 0 ( w T )) , u T := u ∗ 0 ( w T ) , and ˆ /epsilon1 := (1+ L Φ 0 η ) /epsilon1 .

̂ ̂ Lemma 2 allows us to construct an ˆ /epsilon1 -approximate KKT point ( w T , u T ) of (1) from an /epsilon1 -stationary point ̂ w T of either (3) or its smoothed problem (10), where ˆ /epsilon1 = O (max { /epsilon1 , γ } ) . 5

̂ ̂ (c) If ̂ w T is an /epsilon1 -stationary point of (10) , then ( w T , u T ) is an ˆ /epsilon1 -KKT point of (1) , where w T := prox η f ( w T -η ∇ Φ γ ( w T )) , u T := u ∗ γ ( F ( w T )) , and ˆ /epsilon1 := max { (1 + L Φ γ η ) /epsilon1 , γ D b } .

2.5 Technical condition to handle the possible nonsmooth term f

To handle the nonsmooth term f of (1) in our algorithms we require one more condition as in [5].

Assumption 5.

defined by

↓

G

(18)

Let

Φ

γ

be defined by

(10)

, which reduces to

. Assume that there exist two constants

Λ

0

Φ

1

0

given by and

Λ

1

0

(2)

as

γ

such that:

If f = 0 , then G η ( w ) ≡ ∇ Φ γ ( w ) , and Assumption 5 automatically holds with Λ 0 = 1 and Λ 1 = 0 . If f = 0 , then it is crucial to have Λ 0 ≥ 1 in (20). Let us consider two examples to see why?

/negationslash

(i) If f is M f -Lipschitz continuous (e.g., /lscript 1 -norm), then (20) also holds with Λ 0 := 1+ ν > 1 and Λ 1 := 1+ ν M f for a given ν > 0 .

ν (ii ) If f = δ W , the indicator of a nonempty, closed, convex, and bounded set W , then Assumption 5 also holds by the same reason as in Example (i) (see Supp. Doc. A).

3 Shuffling Gradient Method for Nonconvex-Linear Minimax Problems

We first propose a new construction using shuffling techniques to approximate the true gradient ∇ Φ γ in (11) for any γ ≥ 0 . Next, we propose our algorithm and analyze its convergence.

3.1 The shuffling gradient estimators for ∇ Φ γ

Challenges. To evaluate ∇ Φ γ ( w ) in (11), we need to evaluate both ∇ F ( w ) and F ( w ) at each w . However, in SGD or shuffling gradient methods, we want to approximate both quantities at each iteration. Note that this gradient can be written in a finite-sum 1 n ∑ n i =1 ∇ F i ( w ) T ∇ φ γ ( F ( w )) (see (11)), but every summand requires ∇ φ γ ( F ( w )) , which involves the full evaluation of F .

Our estimators. Let F π ( t ) ( i ) ( w ( t ) i -1 ) and ∇ F ˆ π ( t ) ( i ) ( w ( t ) i -1 ) be the function value and the Jacobian component evaluated at w ( t ) i -1 respectively for i ∈ [ n ] , where π ( t ) = ( π ( t ) (1) , π ( t ) (2) , · · · , π ( t ) ( n )) and ˆ π ( t ) = (ˆ π ( t ) (1) , ˆ π ( t ) (2) , · · · , ˆ π ( t ) ( n )) are two permutations of [ n ] := { 1 , 2 , · · · , n } . We want to use these quantities to approximate the function value F ( w ( t ) 0 ) and its Jacobian ∇ F ( w ( t ) 0 ) of F at w ( t ) 0 , respectively, where w ( t ) 0 the iterate vector at the beginning of each epoch t .

For function value F ( w ( t ) 0 ) , we suggest the following approximation at each inner iteration i ∈ [ n ] :

Alternative to (21), for all i ∈ [ n ] , we can simply choose another option:

For Jacobian ∇ F ( w ( t ) 0 ) , we suggest to use the following standard shuffling estimator for all i ∈ [ n ] :

For F ( t ) i from (21) (or (22)) and for ∇ F ( t ) i from (23), we form an approximation of ∇ Φ γ ( w ( t ) 0 ) as

Discussion. The estimator F ( t ) i for F requires n -i more function evaluations F π ( t ) ( j ) ( w ( t ) 0 ) at each epoch t . The first option (21) for F uses 2 n function evaluations F i , while the second one in (22) only needs n function evaluations at each epoch t ≥ 0 . However, (21) uses the most updated information up to the inner iteration i compared to (22), which is expected to perform better. The Jacobian estimator ∇ F ( t ) i is standard and only uses one sample or a mini-batch at each iteration i .

3.2 The shuffling gradient-type algorithm for nonconvex-linear setting (NL)

We propose Algorithm 1, a shuffling gradient-type method, to approximate a stationary point of (10).

Discussion. First, the cost per epoch of Algorithm 1 consists of either 2 n or n function evaluations F i , and n Jacobian evaluations ∇ F i . Compare to standard shuffling gradient-type methods, e.g., in [8], Algorithm 1 has either n more evaluations of F i or the same cost. Second, when implementing

0

+

, and be

η

Algorithm 1 (Shuffling Proximal Gradient-Based Algorithm for Solving (10))

1: Initialization: Choose an initial point ˜ w 0 ∈ dom( Φ 0 ) and a smoothness parameter γ > 0 . 2: for t = 1 , 2 , · · · , T do 3: Set w ( t ) 0 := ˜ w t -1 ; 4: Generate two permutations π ( t ) and ˆ π ( t ) of [ n ] (identically or randomly and independently) 5: for i = 1 , · · · , n do 6: Evaluate F ( t ) i by either (21) or (22) using π ( t ) , and ∇ F ( t ) i by (23) using ˆ π ( t ) . 7: Solve (9) to get u ∗ γ ( F ( t ) i ) and form ˜ ∇ Φ γ ( w ( t ) i -1 ) := ( ∇ F ( t ) i ) T Ku ∗ γ ( F ( t ) i ) . 8: Update w ( t ) i := w ( t ) i -1 -η t n ˜ ∇ Φ γ ( w ( t ) i -1 ) ; 9: end for 10: Compute ˜ w t := prox η t f ( w ( t ) n ) ; 11: end for

Algorithm 1, we do not need to evaluate the full Jacobian ∇ F ( t ) i , but rather the product of matrix ( ∇ F ( t ) i ) T and vector ∇ Φ γ ( F ( t ) i ) as ˜ ∇ Φ γ ( w ( t ) i -1 ) := ( ∇ F ( t ) i ) T ∇ Φ γ ( F ( t ) i ) . Evaluating this matrixvector multiplication is much more efficient than evaluating the full Jacobian ∇ F ( t ) i and ∇ Φ γ ( F ( t ) i ) individually. Third, thanks to Assumption 5, the proximal step ˜ w t := prox η t f ( w ( t ) n ) is only required at the end of each epoch t . This significantly reduces the computational cost if prox η t f is expensive.

3.3 Convergence Analysis of Algorithm 1 for Nonconvex-Linear Setting (NL)

Now, we are ready to state the convergence result of Algorithm 1 in a short version: Theorem 1. The full version of this theorem is Theorem 6, which can be found in Supp. Doc. B.

Theorem 1. Suppose that Assumptions 1, 2, 3, and 5 holds for the setting (NL) of (1) and /epsilon1 > 0 is a sufficiently small tolerance. Let { ˜ w t } be generated by Algorithm 1 after T = O ( /epsilon1 -3 ) epochs using arbitrarily permutations π ( t ) and ˆ π ( t ) and a learning rate η t = η := O ( /epsilon1 ) ( see Theorem 6 in Supp. Doc. B for the exact formulas of T and η ) . Then, we have 1 T +1 ∑ T t =0 ‖ G η t ( w t ) ‖ 2 ≤ /epsilon1 2 .

˜ Our first goal is to approximate a stationary point w /star of (CO) as E [ ‖ G η ( ̂ w ) ‖ 2 ] ≤ /epsilon1 2 , while Algorithm 1 only provides an /epsilon1 -stationary of (10). For a proper choice of γ , it is also an /epsilon1 -stationary point of (3). Corollary 1. Let w T defined by (19) be generated from { w t } of Algorithm 1. Under the conditions of Theorem 1 and any permutations π ( t ) and ˆ π ( t ) , the following statements hold.

˜ Alternatively, if { ˜ w t } is generated by Algorithm 1 after T := O ( n -1 / 2 /epsilon1 -3 ) epochs using two random and independent permutations π ( t ) and ˆ π ( t ) and a learning rate η t = η := O ( n 1 / 2 /epsilon1 ) ( see Theorem 6 in Supp. Doc. B for the exact formulas ) . Then, we have 1 T +1 ∑ T t =0 E [ ‖ G η t ( w t ) ‖ 2 ] ≤ /epsilon1 2 .

̂ ˜

̂ (b) If h is only convex ( i.e. µ h = 0) , then we can set γ := O ( /epsilon1 ) , and Algorithm 1 needs O ( n /epsilon1 -7 / 2 ) evaluations of F i and ∇ F i to achieve an /epsilon1 -stationary w T of (3) .

(a) If h is µ h -strongly convex with µ h > 0 , then we can set γ = 0 , and Algorithm 1 requires O ( n /epsilon1 -3 ) evaluations of F i and ∇ F i to achieve an /epsilon1 -stationary w T of (3) .

̂ If, in addition, π ( t ) and ˆ π ( t ) are sampled uniformly at random without replacement and independently, and Λ 1 = O ( n -1 ) , then the numbers of evaluations of F i and ∇ F i are reduced by a factor of √ n .

4 Shuffling Method for Nonconvex-Strongly Concave Minimax Problems

In this section, we develop shuffling gradient-based methods to solve (1) under the nonconvexstrongly concave setting (NC). Since this setting does not cover the nonconvex-linear setting (NL) in Section 3 as a special case, we need to treat it separately using different ideas and proof techniques.

4.1 The construction of algorithm

Unlike the linear case with H i ( w,u ) = 〈 F i ( w ) , Ku 〉 in Section 3, we cannot generally compute the solution u ∗ 0 ( ˜ w t -1 ) in (2) exactly for a given ˜ w t -1 . We can only approximate u ∗ 0 ( ˜ w t -1 ) by some ˜ u t . This leads to another level of inexactness in an approximate 'hyper-gradient' ∇ Φ 0 ( w ( t ) i -1 ) defined by

There are different options to approximate u ∗ 0 ( ˜ w t -1 ) . We propose two options below, but other choices are possible, including accelerated gradient ascent methods and stochastic algorithms [6, 20].

̂ ̂ ˜ ̂ for a given learning rate ˆ η t > 0 . Then, we finally output u t := u ( t ) S to approximate u ∗ 0 ( w t -1 ) .

(a 1 ) Gradient ascent scheme for the lower-level problem. We apply a standard gradient ascent scheme to update ˜ u t : Starting from s = 0 with u ( t ) 0 := u t -1 , at each epoch s = 1 , · · · , S , we update

˜ ̂ ˜ To make our method more flexible, we allow to perform either only one iteration (i.e. S = 1 ) or multiple iterations (i.e. S > 1 ) of (26). Each iteration s requires n evaluations of ∇ u H i .

(a 2 ) Shuffling gradient ascent scheme for the lower-level problem. We can also construct ˜ u t by a shuffling gradient ascent scheme . Again, we allow to run either only one epoch (i.e. S = 1 ) or multiple epochs (i.e. S > 1 ) of the shuffling algorithm to update ˜ u t , leading to the following scheme: Starting from s := 1 with ̂ u ( t ) 0 := ˜ u t -1 , at each epoch s = 1 , 2 , · · · , S , having ̂ u ( t ) s -1 , we generate a permutation π ( s,t ) of [ n ] and run a shuffling gradient ascent scheme as

̂ At the end of the S -th epoch, we output ˜ u t := ̂ u ( t ) S as an approximation to u ∗ 0 ( ˜ w t -1 ) . Here, we use the same learning rate ˆ η t for all epochs s ∈ [ S ] . Each epoch s requires n evaluations of ∇ u H i .

(b) Shuffling gradient descent scheme for the upper-level minimization problem. Having ˜ u t from either (26) or (27), we run a shuffling gradient descent epoch to update w t from w t -1 as

These two steps (26) (or (27)) in u and (28) in w are implemented alternatively for t = 1 , · · · , T .

(c) The full algorithm. Combining both steps (26) (or (27)) and (28), we can present an alternating shuffling proximal gradient algorithm for solving (1) as in Algorithm 2.

Algorithm 2 (Alternating Shuffling Proximal Gradient Algorithm for Solving (1) under setting (NC))

1: Initialization: Choose an initial point ( ˜ w 0 , ˜ u 0 ) ∈ dom( L ) . 2: for t = 1 , 2 , · · · , T do 3: Compute ˜ u t using either (26) or (27). 4: Set w ( t ) 0 := w t -1 and generate a permutation ˆ π ( t ) of [ n ] .

˜ 7: Update w ( t ) i := w ( t ) i -1 -η t n ˜ ∇ Φ 0 ( w ( t ) i -1 ) . 8: end for

˜ 5: for i = 1 , · · · , n do 6: Evaluate ∇ Φ 0 ( w ( t ) i -1 ) := ∇ w H ˆ π ( t ) ( i ) ( w ( t ) i -1 , ˜ u t ) .

9: Compute ˜ w t := prox η t f ( w ( t ) n ) . 10: end for

Discussion. Algorithm 2 has a similar form as Algorithm 1, where u ∗ 0 ( ˜ w t -1 ) is approximated by ˜ u t . In Algorithm 1, u ∗ 0 ( ˜ w t -1 ) is approximated by u ∗ γ ( F ( t ) i ) . Moreover, Algorithm 1 solves the smoothed problem (10) of (3), while Algorithm 2 directly solves (3). Depending on the choice of method to approximate u ∗ 0 ( ˜ w t -1 ) , we obtain different variants of Algorithm 2. We have proposed two variants:

Semi-shuffling variant: We use (26) for computing ˜ u t to approximate u ∗ 0 ( ˜ w t -1 ) . · Full-shuffling variant: We use (27) for computing u t to approximate u ∗ 0 ( w t -1 ) .

4.2 Convergence analysis

˜ ˜ Note that Algorithm 2 works in an alternative manner, where it approximates u ∗ 0 ( ˜ w t -1 ) up to a certain accuracy before updating ˜ w t . This alternating update is very natural and has been widely applied to solve minimax optimization as well as bilevel optimization problems, see, e.g., [1, 9, 13].

Now, we state the convergence of both variants of Algorithm 2: semi-shuffling and full-shuffling variants. The full proof of the following theorems can be found in Supp. Doc. C.

(a) Convergence of the semi-shuffling variant . Our first result is as follows.

Theorem 2. Suppose that Assumptions 1, 2, 4, and 5 hold for (1) , and G η is defined by (18) .

˜ Consequently, Algorithm 2 requires O ( n /epsilon1 -3 ) evaluations of both ∇ w H i and ∇ u H i to achieve an /epsilon1 -stationary point w T of (3) computed by (19) .

Let { ( ˜ w t , ˜ u t ) } be generated by Algorithm 2 using the gradient ascent scheme (26) with η := O ( /epsilon1 ) explicitly given in Theorem 8 of Supp. Doc. C, ˆ η ∈ (0 , 2 L u + µ h ] , S := O ( 1 ˆ η ( µ h + 4 L u µ H L u + µ H ) -1 ) = O (1) , and T := O ( /epsilon1 -3 ) explicitly given in Theorem 8. Then, we have 1 T +1 ∑ T t =0 ‖ G η ( w t ) ‖ 2 ≤ /epsilon1 2 .

̂ Note that Theorem 2 holds for both S > 1 and S = 1 (i.e. we perform only one iteration of (26)).

(b) Convergence of the full-shuffling variant - The case S > 1 with multiple epochs . We state our results for two separated cases: only H i is µ H -strongly convex, and only h is µ h -strongly convex.

Let { ( ˜ w t , ˜ u t ) } be generated by Algorithm 2 using S epochs of the shuffling routine (27) and fixed learning rates η t = η := O ( /epsilon1 ) as given in Theorem 8 of Supp. Doc. C for a given /epsilon1 > 0 , ˆ η t := ˆ η = O ( /epsilon1 ) , S := ⌊ ln(7 / 2) µ H ˆ η ⌋ , and T := O ( /epsilon1 -3 ) . Then, we have 1 T +1 ∑ T t =0 ‖ G η ( w t ) ‖ 2 ≤ /epsilon1 2 .

Theorem 3 (Strong convexity of H i ) . Suppose that Assumptions 1, 2, 4, and 5 hold, and H i is µ H -strongly concave with µ H > 0 for i ∈ [ n ] , but h is only merely convex.

˜ Consequently, Algorithm 2 requires O ( n /epsilon1 -3 ) evaluations of ∇ w H i and O ( n /epsilon1 -4 ) evaluations of ∇ u H i to achieve an /epsilon1 -stationary point w T of (3) computed by (19) .

(c) Convergence of the full-shuffling variant - The case S = 1 with one epoch . Both Theorems 3 and 4 require O ( n /epsilon1 -4 ) evaluations of ∇ u H i . To improve this complexity, we need two additional assumptions but can perform only one epoch of (27), i.e. S = 1 .

̂ Theorem 4 (Strong convexity of h ) . Suppose that Assumptions 1, 2, 4, and 5 hold for (1) , and h is µ h -strongly convex with µ h > 0 , but H i is only merely concave for all i ∈ [ n ] . Then, under the same settings as in Theorem 3, but with S := ⌊ ln(7 / 2) µ h ˆ η ⌋ , the conclusions of Theorem 3 still hold.

Assumption 6. Let ˆ G η ( u ) := η -1 ( u -prox η h ( u + η ∇ u H ( w,u ))) be the gradient mapping of ψ ( w, · ) := -H ( w, · ) + h ( · ) . Assume that there exist ˆ Λ 0 ≥ 1 and ˆ Λ 1 ≥ 0 such that

Clearly, if h = 0 , then ˆ G η ( u ) = -∇ u H ( w,u ) and (20) automatically holds for ˆ Λ 0 = 1 and ˆ Λ 1 = 0 . Assumption 6 is similar to Assumption 5, and it is required to handle the prox operator of h in (27). Assumption 7. For f in (1) , there exists L f ≥ 0 such that

Clearly, if f is L f -smooth, then (30) holds. If f is also convex, then (30) implies that f is L f -smooth. Under these additional assumptions, we have the following result.

Theorem 5. Suppose that Assumptions 1, 2, 4, 5, 6, and 7 hold and G η is defined by (18) .

Let { ( ˜ w t , ˜ u t ) } be generated by Algorithm 2 using one epoch ( S = 1) of the shuffling routine (27) , and fixed learning rates η t = η := O ( /epsilon1 ) as in Theorem 9 of Supp. Doc. C for a given /epsilon1 > 0 , ˆ η t := ˆ η = 30 κ 2 η , and T := O ( /epsilon1 -3 ) , where κ := L u µ H + µ h . Then, we have 1 T +1 ∑ T t =0 ‖ G η ( ˜ w t ) ‖ 2 ≤ /epsilon1 2 .

Consequently, Algorithm 2 requires O ( n /epsilon1 -3 ) evaluations of both ∇ w H i and of ∇ u H i to achieve an /epsilon1 -stationary point w T of (3) computed by (19) .

̂ Similar to Algorithm 1, if π ( s,t ) and ˆ π ( t ) are generated randomly and independently, Λ 1 = O (1 /n ) , and ˆ Λ 1 = O (1 /n ) , then our complexity stated above can be improved by a factor of √ n . Nevertheless, we omit this analysis. Finally, we can combine each Theorem 2, 3, 4 or 5 and Lemma 2 to construct an ˆ /epsilon1 -KKT point of (1). Theorem 5 has a better complexity than Theorems 3 and 4, but requires stronger assumptions. Algorithm 2 is also different from the one in [3] both in terms of algorithmic form and the underlying problem to be solved, while achieving the same oracle complexity.

5 Numerical Experiments

We perform some experiments to illustrate Algorithm 1 and compare it with two existing and related algorithms. Further details and additional experiments can be found in Supp. Doc. D.

We consider the following regularized stochastic minimax problem studied, e.g., in [9, 33]:

where F i,j : R p × Ω → R + can be viewed as the loss of the j -th model for data point i ∈ [ n ] . If we define φ 0 ( v ) := max 1 ≤ j ≤ m { v j } and f ( w ) := λ 2 ‖ w ‖ 2 , then (31) can be reformulated into (3). Since v j ≥ 0 , we have φ 0 ( v ) := max 1 ≤ j ≤ m { v j } = ‖ v ‖ ∞ = max ‖ u ‖ 1 ≤ 1 〈 v, u 〉 , which is nonsmooth. Thus we can smooth φ 0 as φ γ ( v ) := max ‖ u ‖ 1 ≤ 1 { 〈 v, u 〉 -( γ / 2) ‖ u ‖ 2 } using b ( u ) := 1 2 ‖ u ‖ 2 .

Here, we apply our problem (31) to solve a model selection problem in binary classification with nonnegative nonconvex losses, see, e.g., [41]. Each function F i,j belongs to 4 different nonconvex losses ( m = 4) : F i, 1 ( w, ξ ) := 1 -tanh( b i 〈 a i , w 〉 ) , F i, 2 ( w, ξ ) := log(1 + exp( -b i 〈 a i , w 〉 )) -log(1 + exp( -b i 〈 a i , w 〉 -1)) , F i, 3 ( w, ξ ) := (1 -1 / (exp( -b i 〈 a i , w 〉 ) + 1)) 2 , and F i, 4 ( w, ξ ) := log(1 + exp( -b i 〈 a i , w 〉 )) (see [41] for more details), where ( a i , b i ) represents data samples.

We implement 4 algorithms: our SGM with 2 options, SGD from [10], and Prox-Linear from [11]. We test these algorithms on two datasets from LIBSVM [6]. We set λ := 10 -4 and update the smooothing parameter γ t as γ t := 1 2( t +1) 1 / 3 . The learning rate η for all algorithms is finely tuned from { 100 , 50 , 10 , 5 , 1 , 0 . 5 , 0 . 1 , 0 . 05 , 0 . 01 , 0 . 001 , 0 . 0001 } , and the results are shown in Figure 1 for w8a and rcv1 datasets using k b = 32 blocks. The details of this experiment is given in Supp. Doc. D.

SVM-w8a:n =49749,p =300

SVM -rcv1: n =20242, p =47236

100

SGM-Option1

100

SGM-Option1

SGM-Option 2

SGM-Option2

9 × 10-1

SGD

SGD

Prox Linear

8 × 10-1

Prox Linear

Objective Value

Value

6 × 10-1

Objective

7 × 10-1

6 × 10-1.

4 × 101

3 × 10-1

5 × 10-1

1

0

25

50

75

100

125

150

175

200

0

25

50

75

100

125

150

175

200

Epochs

Epochs

As shown in Figure 1, the two variants of our SGM have a comparable performance with SGD and Prox-Linear , providing supportive evidence for using shuffling strategies in minimax algorithms.

6 Conclusions

This work explores a bilevel optimization approach to address two prevalent classes of nonconvexconcave minimax problems. These problems find numerous applications in practice, including robust learning and generative AIs. Motivated by the widespread use of shuffling strategies in implementing gradient-based methods within the machine learning community, we develop novel shuffling-based algorithms for solving these problems under standard assumptions. The first algorithm uses a nonstandard shuffling strategy and achieves the state-of-the-art oracle complexity typically observed in nonconvex optimization. The second algorithm is also new, flexible, and offers a promising possibility for further exploration. Our results are expected to provide theoretical justification for incorporating shuffling strategies into minimax optimization algorithms, especially in nonconvex settings.

Acknowledgments and Disclosure of Funding

This work was partly supported by the National Science Foundation (NSF): NSF-RTG grant No. NSF DMS-2134107 and the Office of Naval Research (ONR), grant No. N00014-23-1-2588.

References

M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning , pages 214-223, 2017.

M. G. Azar, I. Osband, and R. Munos. Minimax regret bounds for reinforcement learning. In International Conference on Machine Learning , pages 263-272. PMLR, 2017.

A. Ben-Tal, L. El Ghaoui, and A. Nemirovski. Robust optimization . Princeton University Press, 2009.

A. Beznosikov, E. Gorbunov, H. Berard, and N. Loizou. Stochastic gradient descent-ascent: Unified theory and new efficient methods. In International Conference on Artificial Intelligence and Statistics , pages 172-235. PMLR, 2023.

K. Bhatia and K. Sridharan. Online learning with dynamics: A minimax perspective. Advances in Neural Information Processing Systems , 33:15020-15030, 2020.

C.-C. Chang and C.-J. Lin. LIBSVM: A library for Support Vector Machines. ACMTransactions on Intelligent Systems and Technology , 2:27:1-27:27, 2011.

H. Cho and C. Yun. SGDA with shuffling: faster convergence for nonconvex-P Ł minimax optimization. The 11th International Conference on Learning Representations , pp. 1-10, 2022.

A. Das, B. Schölkopf, and M. Muehlebach. Sampling without replacement leads to faster rates in finite-sum minimax optimization. Advances in Neural Information Processing Systems , 35:6749-6762, 2022.

S. Dempe. Foundations of Bilevel Programming . Springer Science & Business Media, 2002.

D. Driggs, J. Liang, and C.-B. Schönlieb. On biased stochastic gradient estimation. Journal of Machine Learning Research , vol. 23, no. 24, pp. 1-43, 2022.

K. Emmanouilidis, R. Vidal, and N. Loizou. Stochastic extragradient with random reshuffling: Improved convergence for variational inequalities. In International Conference on Artificial Intelligence and Statistics , pages 3682-3690. PMLR, 2024.

G. Gidel, H. Berard, G. Vignoud, P. Vincent, and S. Lacoste-Julien. A variational inequality perspective on generative adversarial networks. International Conference on Learning Representations , pp. 1-10, 2019.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems , pages 2672-2680, 2014.

E. Gorbunov, H. Berard, G. Gidel, and N. Loizou. Stochastic extragradient: General analysis and improved rates. In International Conference on Artificial Intelligence and Statistics , pages 7865-7901. PMLR, 2022.

E. Y. Hamedani, A. Jalilzadeh, N. S. Aybat, and U. V. Shanbhag. Iteration complexity of randomized primal-dual methods for convex-concave saddle point problems. arXiv preprint arXiv:1806.04118 , 2018.

J. Z. HaoChen and S. Sra. Random shuffling beats SGD after finite epochs. International Conference on Machine Learning , pp. 2624-2633, 2019.

E. Ho, A. Rajagopalan, A. Skvortsov, S. Arulampalam, and M. Piraveenan. Game theory in defence applications: A review. Sensors , 22(3):1032, 2022.

Y. Hsieh, F. Iutzeler, J. Malick, and P. Mertikopoulos. Explore aggressively, update conservatively: Stochastic extragradient methods with variable stepsize scaling. Advances in Neural Information Processing Systems , 33:16223-16234, 2020.

Abdul Jabbar, Xi Li, and Bourahla Omar. A survey on generative adversarial networks: Variants, applications, and training. ACM Computing Surveys (CSUR) , 54(8):1-49, 2021.

G. Lan. First-order and Stochastic Optimization Methods for Machine Learning . Springer, 2020.

F. Lin, X. Fang, and Z. Gao. Distributionally robust optimization: A review on theory and applications. Numerical Algebra, Control & Optimization , 12(1):159, 2022.

N. Loizou, H. Berard, G. Gidel, I. Mitliagkas, and S. Lacoste-Julien. Stochastic gradient descent-ascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity. Advances in Neural Information Processing Systems , 34:19095-19108, 2021.

L. Luo, H. Ye, and T. Zhang. Stochastic recursive gradient descent ascent for stochastic nonconvex-strongly-concave minimax problems. Advances in Neural Information Processing Systems , vol. 33, pp. 20566-20577, 2020.

Z. Luo, J. Pang, and D. Ralph. Mathematical Programs with Equilibrium Constraints . Cambridge University Press, Cambridge, 1996.

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018.

Q. Meng, W. Chen, Y. Wang, Z.-M. Ma, and T.-Y. Liu. Convergence analysis of distributed stochastic gradient descent with shuffling. Neurocomputing , 337:46-57, 2019.

K. Mishchenko, A. Khaled, and P. Richtárik. Random reshuffling: Simple analysis with vast improvements. Advances in Neural Information Processing Systems , 33:17309-17320, 2020.

K. Mishchenko, A. Khaled, and P. Richtárik. Proximal and federated random reshuffling. In International Conference on Machine Learning , pages 15718-15749. PMLR, 2022.

Y. Nesterov. Introductory lectures on convex optimization: A basic course , volume 87 of Applied Optimization . Kluwer Academic Publishers, 2004.

L. M. Nguyen, Q. Tran-Dinh, D. T. Phan, P. H. Nguyen, and M. van Dijk. A unified convergence analysis for shuffling-type gradient methods. Journal of Machine Learning Research , 22(207):144, 2021.

B. Palaniappan and F. Bach. Stochastic variance reduction methods for saddle-point problems. In Advances in Neural Information Processing Systems , pages 1416-1424, 2016.

I. Safran and O. Shamir. How good is SGD with random shuffling? Conference on Learning Theory , pp. 3250-3284, 2020.

A. Shapiro and A. Kleywegt. Minimax analysis of stochastic problems. Optim. Methods Softw. , 17(3):523-542, 2002.

Q. Tran-Dinh, D. Liu, and L. M. Nguyen. Hybrid variance-reduced SGD algorithms for nonconvex-concave minimax problems. The 34th Conference on Neural Information Processing Systems (NeurIPs 2020) , 2020.

J. Wang, T. Zhang, S. Liu, P.-Y. Chen, J. Xu, M. Fardad, and B. Li. Adversarial attack generation empowered by min-max optimization. Advances in Neural Information Processing Systems , 34:16020-16033, 2021.

M. Wang, E. Fang, and L. Liu. Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions. Math. Program. , 161(1-2):419-449, 2017.

J. Yang, N. Kiyavash, and N. He. Global convergence and variance-reduced optimization for a class of nonconvex-nonconcave minimax problems. arXiv preprint arXiv:2002.09621 , 2020.

B. Ying, K. Yuan, and A. H. Sayed. Convergence of variance-reduced stochastic learning under random reshuffling. arXiv preprint arXiv:1708.01383 , 2(3):6, 2017.

J. Zhang and L. Xiao. Stochastic variance-reduced prox-linear algorithms for nonconvex composite optimization. Mathematical Programming , pp. 1-43, 2022.

K. Zhang, Z. Yang, and T. Ba¸ sar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of reinforcement learning and control , pages 321-384, 2021.

L. Zhao, M. Mammadov, and J. Yearwood. From convex to nonconvex: a loss function analysis for binary classification. In IEEE International Conference on Data Mining Workshops (ICDMW) , pages 1281-1288. IEEE, 2010.

R. Zhao. Optimal stochastic algorithms for convex-concave saddle-point problems. arXiv preprint arXiv:1903.01687 , 2019.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our claims made in the abstract reflects our contribution stated in the introduction, see the 'Contribution" paragraph in the introduction section. Our contribution consists of two algorithms, Algorithm 1 and Algorithm 2, and their theoretical convergence guarantees stated in the subsequent theorems.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: This paper has limitation as it only focuses on two classes of minimax problems defined in (1). Yes, we only consider two classes of minimax problems: nonconvex-linear (NL) and convex-strongly concave (NC), covered by our assumption, Assumptions 1 to 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We state all required assumptions in Assumptions 1 to 5. Our theoretical results stated in each theorem also refers to these assumptions when required. Our full proofs are given in Supp. Docs. due to space limit, and we believe that our technical proofs are correct.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide the details of our experiments, including mathematical models, the detailed implementation of algorithms, the choice of parameters, and datasets. We also upload the code with examples to run and verify.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Our data is available online from LIBSVM. The code is implemented in Python. The code for all experiments is also provided with instruction.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Supp. Doc. D provides all the details of our experiments, including how to select parameters, and how to report our results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: The paper does not have such a result to report.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Our experiments were run on a MacBook Pro. 2.8GHz Quad-Core Intel Core I7, 16Gb Memory specified at the beginning of Supp. Doc. D.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our data is publicly available online from LIBSVM.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: We do not yet know if our paper has an immediate broader impact. However, since our problems and our algorithms are sufficiently general, we hope they will create broader impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not have our own real data or specific model that has a high risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: Our code is open-source and will be made available online under a standard public license.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: It does not have new asset.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: It does not relate to crowdsourcing experiments and research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: It does not require any approval.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work propose shuffling gradient-based methods for nonconvex-linear and nonconvex-strongly convex minimax optimization and obtain complexity results.\n\nSTRENGTHS:\nThe presentation is clear. Based on my knowledge on both shuffling and minimax optimization, I found that the algorithms and complexity results are reasonable. The experiments provide sufficient details for reproducibility.\n\nWEAKNESSES:\nThe contribution is in general combinational, with slightly more techniques such as smoothing and shuffling-based function evaluations. Also, comparison with previous stochastic algorithms for minimax optimization and shuffling for nonconvex optimization is lacking, as shown in question 1 below.\n\nQUESTIONS:\n(1) You could compare your complexity results (Theorem 2) with previous stochastic gradient-based minimax optimization works (e.g. [a]) and shuffling for nonconvex optimization (e.g. [b]). \n\n[a] Lin, T., Jin, C., \\& Jordan, M. (2020, November). On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning (pp. 6083-6093). PMLR.\n\n[b] Nguyen, L. M., Tran-Dinh, Q., Phan, D. T., Nguyen, P. H., \\& Van Dijk, M. (2021). A unified convergence analysis for shuffling-type gradient methods. Journal of Machine Learning Research, 22(207), 1-44.\n\n(2) At the bottom of page 1, for problem (NL), it looks possible to remove matrix $K$ and use $F_i:\\mathbb{R}^p\\to \\mathbb{R}^q$, since $\\langle F_i(w),Ku\\rangle=\\langle K^{\\top}F_i(w),u\\rangle$ and we could replace $K^{\\top}F_i(w)\\in \\mathbb{R}^q$ with $F_i(w)\\in \\mathbb{R}^q$. \n\n(3) In Assumption 2, do you mean for any $v\\in {\\rm dom}(h)$, $||v||_2\\le M_h$? \n\n(4) In \"Condition (6) can be relaxed to the form $\\frac{1}{n} \\sum_{i=1}^n||\\nabla F_i(w)-\\nabla F(w)||^2 \\leq \\sigma_J^2+\\Theta_J||\\nabla \\Phi_0(w)||^2$\", should $\\Phi_0$ be $F$? \n\n(5) After Assumption 5, ``If $f=0$, then $\\mathcal{G} _ {\\eta}(w)\\equiv\\nabla\\Phi_{\\gamma}(w)$''. Should it be $\\nabla\\Phi_0(w)$ instead of $\\nabla\\Phi_{\\gamma}(w)$, since the definition of $\\mathcal{G} _ {\\eta}(w)$ in eq. (18) involves $\\nabla\\Phi_0(w)$ not $\\nabla\\Phi _ {\\gamma}(w)$? Or you might consider defining $\\mathcal{G} _ {\\eta,\\gamma}(w)$ with $\\nabla\\Phi_0(w)$ replaced by $\\nabla\\Phi _ {\\gamma}(w)$, such that $\\epsilon$-stationary point of (3) and that of (10) in Lemma 2 correspond to $\\mathcal{G} _ {\\eta,0}(w)$ and $\\mathcal{G} _ {\\eta,\\gamma}(w)$ respectively. \n\n(6) I found that Eq. (21) (Option 1) needs to save {$F_{\\pi^{(t)}(j)(w_0^{(t)})}$}$_{j=1}^n$ to avoid repeated evaluation of these $n$ values. Yes? \n\n(7) Line 255: \"fix number\" $\\rightarrow$ \"fixed number\". \n\n(8) At the end of Theorem 2, do you mean to change one of the $\\nabla_w\\mathcal{H}_i$ into $\\nabla_u\\mathcal{H}_i$? It seems that the complexity of semi-shuffling is bettter than full-shuffling. Then what's the advantage of full-shuffling?\n\nLIMITATIONS:\nThe checklist mentions the limitation that this work only focuses on nonconvex-linear and nonconvex-strongly convex minimax optimization, and says \"We do not yet know if our paper has an immediate broader impact. However, since our problems and our algorithms are sufficiently general, we hope they will create broader impacts.\" I agree with both of them.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on nonconvex-concave, finite-sum (stochastic) minimax problems with possibly nonsmooth regularization. Aiming to find $\\epsilon$-stationary points, the paper proposes shuffling-based proximal gradient descent-ascent algorithms and verifies gradient computation complexity upper bounds via a bilevel optimization perspective. The results consider various settings, such as (1) the nonconvex-linear (NL) case with either strongly convex or convex $h$, each requiring $\\mathcal{O}(n \\epsilon^{-3})$ and  $\\mathcal{O}(n \\epsilon^{-7/2})$ gradient evaluations, respectively, and (2) the nonconvex-strongly-concave (NS) case with smoothness assumptions, where semi-shuffling requires $\\mathcal{O}(n \\epsilon^{-3})$ while full-shuffling requires (naively) $\\mathcal{O}(n \\epsilon^{-4})$ gradient evaluations. The paper contains proofs of the convergence theorems and supporting numerical experiments.\n\nSTRENGTHS:\n* The theoretical results and the problem-specific proof techniques could be a solid contribution. To the best of my knowledge, this is one of the very few works that discuss shuffling-type algorithms for minimax problems apart from Das et al., 2022 [1] and Cho & Yun, 2023 [2]. I think the results under the relaxed setting, especially the linear, non-strongly-convex case of (NL), and for the bilevel-type shuffling algorithms are quite novel. The paper also includes experimental results that align with the theoretical results.\n\nWEAKNESSES:\n* Despite that the novelty of this work would be in the fact that the results apply for broader settings than previous literature rather than obtaining *faster* convergence under similar settings, the paper still should have included a fair, more detailed comparison with previous work and illustrated how the gradient evaluation complexity becomes different from settings with stronger assumptions. The current draft does not contain any meaningful quantitative comparison with previous work.\n* In particular, I don’t think that the settings in [2] are different from the results for the (NS) setting. Apart from the fact that the objective function does not contain $f$ and $h$, the assumptions used in Theorem 1 of [1] (Assumptions 1-4) and the NS setting seem to be nearly identical, and the goal of finding $\\epsilon$-stationary points also seem to be equivalent. (In fact, for the variable $u$ the PŁ assumption is weaker than strong concavity.) In terms of the convergence rates, Theorem 1 of [1] requires only $\\mathcal{O} (n \\epsilon^{-2} + n^{1/2} \\epsilon^{-3})$ gradient evaluations, which is smaller than $\\mathcal{O} (n \\epsilon^{-3})$. The authors should have at least explained how introducing $f, h$ changes the difficulty of the problem, or given any other reasons why one could prefer the bilevel optimization framework over the simple SGDA algorithm as in [1].\n\nQUESTIONS:\n- Why is it necessary to use bilevel optimization, i.e., update $n$ steps of $w$ and then $n$ steps of $v$, instead of iteration-wise updates? Also, am I right that the results for the (NS) settings are weaker than that of [2] at least when $f, h = 0$? Is there a particular reason why the weaker convergence rates are inevitable if we consider $f, h$?\n- Some technical parts of the algorithms are brought from previous work, such as the use of $\\Phi_{\\gamma}$ inspired by Yang et al., 2020 [3]. Some other parts look new, such as the placement of the proximal steps considering $f$ and $h$ and the variants of algorithms like choosing between (21), (22) in Algorithm 1 and (26), (27) in Algorithm 2. I am curious about the motivations of the new components of the algorithms, especially considering the different types of shuffling techniques I mentioned last—whether these have clear motivations or are more like artifacts of the proof techniques, if there were similar approaches in minimization problems, etc.\n\n- Light question: Would it be able to relax the strongly concave case to weaker PŁ-type assumptions, or consider alternating type updates as in [2]? Moreover, is it possible to compare the presented results for minimax problems with nonconvex, nonsmooth *minimization* as in Cutkosky et al., 2023 [4]?\n- Light question: Are there examples of when the nonconvex-linear (NL) setting could be of interest in practice?\n- Minor: Why are there two bibliographies, and why are the contents different? It seems that the one at the end of the appendix has the correct numbers.\n\nLIMITATIONS:\nThere seems to be no potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper proposes shuffling gradient-based methods for addressing two classes of minimax optimization problems: nonconvex-linear and nonconvex-strongly concave settings. The first algorithm focuses on the nonconvex-linear minimax setting and the second algorithm, consisting of semi-shuffling and full-shuffling schemes, focuses on the nonconvex-strongly concave minimax setting. The authors establish oracle complexity bounds for these algorithms under some standard assumptions. Numerical experiments are conducted.

STRENGTHS:
The authors establish oracle complexity bounds for their algorithms under standard assumptions. These bounds provide theoretical guarantees on the performance of the proposed methods.

WEAKNESSES:
The proposed Algorithm 1 requires solving a maximization problem at each iteration, which can be computationally expensive.

The numerical experiment section is quite weak. First, only Algorithm 1 is tested, with no results related to Algorithm 2. Second, only one minimax model is tested. More complex and popular minimax models should be included, and state-of-the-art competitors should be used to evaluate the performance of the proposed methods.

Typo in Line 187 and Line 459: $M_f \rightarrow M_f^2$

QUESTIONS:
What are the advantages of the proposed methods in this paper, given the introduction of the semi-shuffling and full-shuffling schemes? Do they have a better convergence rate or sample complexity compared to existing methods?

LIMITATIONS:
yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents new shuffling gradient-based methods for solving two classes of nonconvex-concave minimax optimization problems: nonconvex-linear and nonconvex-strongly concave settings. \n\nThe first algorithm is designed for the nonconvex-linear setting and achieves state-of-the-art oracle complexity, employing a new shuffling estimator for the hyper-gradient. \n\nThe second method introduces semi-shuffling and full-shuffling schemes for the nonconvex-strongly concave setting, establishing their oracle complexity bounds for the first time. \n\nNumerical examples are provided to demonstrate the performance of the proposed algorithms.\n\nSTRENGTHS:\n1. The paper is well-written, offering a clear exposition of the current state of the art, along with explicit assumptions and conditions for the proposed methods.\n\n2. The authors introduce novel smoothing techniques similar to Nesterov's smoothing approach for lower-level maximization problems, and establish oracle complexity bounds for both nonconvex-linear and nonconvex-strongly concave settings. This introduces a novel perspective on addressing nonconvex optimization problems.\n\n3. The paper presents two distinct algorithms, each carefully tailored to different problem settings, showcasing a profound comprehension of diverse problem landscapes.\n\nWEAKNESSES:\n1. **Complexity Comparison with Existing Works**. The authors are encouraged to present a comparative table that delineates the oracle complexity of their proposed methods alongside that of existing works, including standard nonshuffling methods. This side-by-side comparison should aim to explicitly demonstrate the theoretical advantages of their methods.\n\n2. **Shuffling Methods vs. Nonshuffling Methods**. The paper should discuss the distinctions between shuffling methods and traditional nonshuffling approaches. Specically, it is better to clarify whether shuffling methods consistently yield superior performance over nonshuffling methods theoretically.\n\n3. **Numerical Experiments Expansion**. While the experiments have been conducted on synthetic datasets and small-scale datasets, which is customary for theoretical papers, the scope of empirical validation could be enhanced. It would be much better if the authors could extend their experiments to include large-scale datasets.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}