A Faster Decentralized Algorithm for Nonconvex Minimax Problems

1 Introduction . Minimax optimization has enormous applications in machine learning tasks such as Generative Adversarial Net ( GAN ) [ 8 ] , adversarial training [ 26 ] and multi-agent reinforcement learning [ 43 ] . Specifically , in minimax optimization , variable x aims to minimize a payoff loss function f ( x , y ) : Rd1 × Rd2 → R while variable y tries to maximize the loss , which can be formulated as min x∈X max y∈Y f ( x , y ) , ( 1 ) where X ⊆ Rd1 and Y ⊆ Rd2 . In the past a few decades , there are plenty of works to study minimax optimization problem in a variety of research fields and many methods have been developed . The most intuitive solution is Gradient Descent Ascent ( GDA ) algorithm [ 6 , 29 ] with equal stepsize ηx = ηy . Asymptotic and nonasymptotic convergence analysis has been provided when f is convex in x and concave in y . Recently , many deterministic and stochastic gradient algorithms for nonconvexstrongly-concave and nonconvex-concave problems were proposed . Some algorithms improve the performance of vanilla GDA method by adopting different stepsize on x and y , such as [ 10 , 19 ] , where the stepsize of y is typically larger than the stepsize of x . Some algorithms update x and y at different frequency , such as [ 14 , 25 , 32 ] . These kind of algorithms usually involve a nested loop structure that updates y more frequently than x to make f ( x , y ) close to function Φ ( x ) , which is defined by Φ ( x ) = max y∈Y f ( x , y ) . ( 2 ) 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . As more large-scale machine learning problems are arising , distributed training becomes a popular and crucial framework because of its ability and efficiency to deal with large data . It is desired to generalize minimax optimization to distributed training to solve large-scale minimax problems . In distributed optimization , the original centralized optimization suffers from a bottleneck communication problem , i.e . the communication traffic on the busiest central node , especially when the network is large [ 18 , 51 ] . To tackle this communication issue , decentralized optimization was proposed and has emerged as a promising technique . It is a kind of distributed machine learning training paradigm that does not rely on the centralized network topology . Different worker nodes collaboratively utilize their own local data to implement large-scale training tasks and at each iteration they only have to communicate with their neighbors . Decentralized algorithms have been shown to enhance the communication efficiency by avoiding the communication overhead problem . Decentralized methods are also advantageous when the network suffers from communication restriction or has low bandwidth between some nodes and the central node . Besides , it is also an essential method in some situations where data are geographically distributed and centralized data processing is not available or there are concerns to preserve data privacy [ 48 ] . Recently many works were proposed to improve the performance of decentralized training . D-PSGD [ 18 ] theoretically justifies the potential advantage of decentralized algorithm . D2 [ 38 ] improves the convergence rate to outperform D-PSGD by eliminating the influence of data variance among different workers . D-SPIDER-SFO [ 33 ] incorporates D2 and SPIDER [ 7 , 44 ] , which is a kind of variance reduction technique [ 15 ] , to further reduce the gradient complexity . DQSFW [ 45 ] studies decentralized constrained problem with Frank-Wolfe method . GT-HSGD [ 46 ] extends hybrid stochastic gradient descent to decentralized setting , which is a variance-reduced approach that does not compute mega batch periodically . However , the decentralized minimax optimization is still very limited and existing methods suffer from very high gradient complexity [ 21 , 41 ] . Thus , we are motivated to design an accelerated decentralized algorithm for minimax problems . In this paper , thus , we propose a faster Decentralized Minimax Hybrid Stochastic Gradient Descent ( DM-HSGD ) algorithm to solve the following decentralized stochastic minimax optimization problem : min x∈Rd1 max y∈Y f ( x , y ) = 1 n n∑ i=1 fi ( x , y ) , fi ( x , y ) : = Eξ ( i ) ∼DiFi ( x , y ; ξ ( i ) ) ( 3 ) where n is the number of worker nodes , Y is a convex set . Here the local component objective function Fi ( x , y ; ξ ( i ) ) is L-smooth , nonconvex in x , and strongly-concave in y . Di is the data distribution on the i-th node . In this paper , the data distribution can be non-identical . Random variable ξ ( i ) is an index sampled from the local data . We summarize our contributions as follows : ( 1 ) In this paper , we propose a new accelerated decentralized stochastic first-order algorithm , named as DM-HSGD , to solve the decentralized nonconvex-strongly-concave minimax optimization problems . Our algorithm is the first stochastic gradient algorithm to solve general decentralized minimax problem on non-identical distributed data with theoretical guarantees . Besides , our algorithm does not require large batch size or nested loop which makes it more practical and efficient to implement . ( 2 ) We provide a completed proof to guarantee the convergence of our algorithm to solve decentralized stochastic minimax optimization . Under nonconvex-strongly-concave condition , our algorithm obtains SFO complexity of O ( κ3 −3 ) to search an -stationary point of function Φ ( x ) = maxy∈Y f ( x , y ) . This result is faster than the complexity of previous decentralized minimax algorithms [ 21 , 41 ] . Moreover , we also prove that our method achieves linear speedup as the number of workers n increases , which verifies its ability to solve large-scale problems . The rest of this paper will be organized as follows . In Section 2 , we will introduce related works . In Section 3 , we will introduce our new DM-HSGD algorithm . In Section 4 , we will show the main theorems of convergence and complexity analysis . In Section 5 , we will discuss our experimental results , and Section 6 will conclude the paper . 2 Related Works . 2.1 Centralized Minimax Optimization . In recent years , many algorithms for solving minimax optimization were proposed , and the majority of them were studied under the nonconvex-strongly-concave condition . SGDmax [ 14 ] is a double loop algorithm that achieves SFO complexity of O ( κ3 −4log ( 1/ ) ) where κ = L/µ is the condition number . Proximally Guided Stochastic Mirror Descent and Variance Reduction ( PGSMD/PGSVRG ) [ 34 ] are double loop algorithms that achieve SFO complexity of O ( κ3 −4 ) for stochastic problem and O ( κ2N −2 ) for finite-sum problem whereN is the number of samples . Multistep GDA ( MGDA ) [ 32 ] is a double loop algorithm and HiBSA [ 23 ] is a single loop algorithm . Both MGDA and HiBSA are deterministic hence they can only solve finite-sum problems . Both of them achieve SFO complexity of O ( κ4N −2 ) . Proximal Dual Implicit Accelerated Gradient ( ProxDIAG ) is a deterministic triple loop algorithm whose SFO complexity for finite-sum problem is O ( κ1/2N −2 ) . SGDA [ 19 ] , Stochastic Recursive gradiEnt Descent Ascent ( SREDA ) [ 25 ] , and Hybrid VarianceReduced SGD [ 40 ] are more related to our work . SGDA is a single loop algorithm to solve nonconvexstrongly-concave and nonconvex-concave minimax problems . For nonconvex-strongly-concave problem , it requires O ( κ3 −4 ) SFO complexity to find an -stationary point of Φ ( x ) . In this paper , we will prove that our method achieves a better SFO complexity . SREDA [ 25 ] is a double loop algorithm that achieves O ( κ3 −3 ) SFO complexity . It accelerates SGDA by using SPIDER , which is a variance reduction technique and utilizes the newest gradient information [ 7 , 30 ] . SREDA also involves a separated initialization algorithm called PiSARAH [ 31 ] to ensure the convergence . More recently , [ 13 ] proposed an efficient mirror descent ascent algorithm for nonconvex-strongly-concave minimax optimization with nonsmooth regularization based on Bregman distance and variance reduced technique of SPIDER . In our paper , we use another variance-reduced technique named STORM or hybrid stochastic gradient descent [ 3 ] to accelerate the algorithm . We will discuss the challenges of using SPIDER on decentralized settings in Section 3 . Different from SREDA , our method only requires a large batch at the first iteration . Except the first iteration , we can use either a single sample or a mini-batch to calculate the stochastic gradient . However , SREDA loads a mega-batch with size O ( −2 ) periodically ( every q iterations ) and needs O ( −1 ) gradient oracles at each iteration , which is not practical for large-scale problems . Besides , the maximizer in SREDA is a nested loop to update variable y and if we count the loop of SPIDER then SREDA is actually a triple algorithm . On the contrary , there is no nested loop in our DM-HSGD , which makes our method more efficient and convenient to implement . Moreover , unlike SREDA , our method does not require a separated initialization algorithm to calculate a precise initial value for y . Hybrid Variance-Reduced SGD algorithm also takes advantage of hybrid stochastic gradient descent to accelerate minimax optimization . For example , [ 40 , 11 ] applied the Hybrid Variance-Reduced SGD to minimax problems . More recently , [ 9 , 12 ] proposed some efficient adaptive gradient descent ascent methods for nonconvex-strongly-concave minimax optimization based on momentum techniques including Hybrid Variance-Reduced SGD . 2.2 Decentralized Minimax Optimization . At decentralized setting , most minimax algorithms were proposed for convex-concave problem [ 17 , 28 ] . In [ 22 ] a nonconvex-nonconcave algorithm DPPSP was proposed . However , it is not gradient-based and the closed-form solution to the subproblem is not ensured in our problem . Hence we will not discuss it in this paper . Decentralized Parallel Optimistic Stochastic Gradient ( DPOSG ) [ 21 ] is the first algorithm applicable to a general decentralized minimax problem with theoretical guarantees . It is a single loop minimax algorithm that generalizes Optimistic Stochastic Gradient ( OSG ) [ 2 ] to decentralized training . However , DPOSG has some obvious drawbacks . The first one is that the gradient complexity O ( −12 ) is too high and we are motivated to design a faster algorithm . The second one is that DPOSG only works in the case where the data distribution is identical . When the data distribution is non-identical , the Lemma 3 in [ 21 ] is not satisfied . Actually the assumption of identical data distribution is not satisfied at most decentralized training tasks . Thus , in this paper , we do not use this assumption . More recently , [ 41 ] studied decentralized nonconvex-strongly-concave minimax problems and proposed a double loop deterministic Gradient Tracking/Descent-Ascent algorithm which extends the vanilla GDA to decentralized setting and combines it with gradient tracking . It achieves a gradient complexity of O ( −2 ) . However , in large-scale machine learning tasks such as deep neural network , generally the full gradient is unavailable and the application of deterministic algorithms is very restricted . If we convert Gradient Tracking/Descent-Ascent to stochastic gradient version , the SFO complexity should be at least O ( −4 ) , which is the same result as SGD in nonconvex optimization . Under the same conditions , our new algorithm achieves a better SFO complexity of O ( −3 ) . [ 24 ] studied decentralized reinforcement learning problem based on distributed constrained Markov decision process model and proposed a decentralized policy gradient optimization method named Safe Dec-PG , which achieves SFO complexity of O ( −4 ) . However , the problem studied in [ 24 ] has a special form that is linear in y . In this paper , we focus on general minimax problem . [ 1 ] is a simultaneous work of our work that studies a more general decentralized variational inequality problem with higher complexity . We summarize the comparison of related algorithms for general minimax optimization in Table 1 . For decentralized algorithms DPOSG , GT/DA , and DM-HSGD , we also discuss whether they can converge on non-identical distributed data .

Summary: The paper stuides the nonconvex-strongly-concave min-max optimizaton problem on decentralized setting. In particular, the decentalized min-max problems were seldom studied in the literature and the existing methods suffer from very high gradient complexity. To address this challenge, this paper proposes a new faster decentralized algorithm, namely DM-HSGD, for nonconvex min-max problems by using the variance reduced technique of hybrid stochastic gradient descent. The algorithm achieves SFO complexity of $O(\kappa^3 \epsilon^{-3})$ for decentalized stochastic nonconvex-strongly-concave problems, improving the best existing theoretical results. Numerical results on decentalized settings show the superior performance of DM-HSGD.