Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees

Variational inequalities in general and saddle point problems in particular are increasingly relevant in machine learning applications , including adversarial learning , GANs , transport and robust optimization . With increasing data and problem sizes necessary to train high performing models across various applications , we need to rely on parallel and distributed computing . However , in distributed training , communication among the compute nodes is a key bottleneck during training , and this problem is exacerbated for high dimensional and over-parameterized models . Due to these considerations , it is important to equip existing methods with strategies that would allow to reduce the volume of transmitted information during training while obtaining a model of comparable quality . In this paper , we present the first theoretically grounded distributed methods for solving variational inequalities and saddle point problems using compressed communication : MASHA1 and MASHA2 . Our theory and methods allow for the use of both unbiased ( such as Randk ; MASHA1 ) and contractive ( such as Topk ; MASHA2 ) compressors . We empirically validate our conclusions using two experimental setups : a standard bilinear min-max problem , and large-scale distributed adversarial training of transformers . 1 INTRODUCTION . 1.1 THE EXPRESSIVE POWER OF VARIATIONAL INEQUALITIES . Due to their abstract mathematical nature and the associated flexibility they offer in modeling various practical problems of interests , variational inequalities ( VI ) have been an active area of research in applied mathematics for more than half a century ( Korpelevich , 1976 ; Harker & Pang. , 1990 ; Facchinei & Pang , 2003 ) . It is well known that VIs can be used to formulate and study convex optimization problems , convex-concave saddle point problems and games , for example , in an elegant unifying mathematical framework ( Korpelevich , 1976 ; Bauschke & Combettes , 2017 ) . Recently , Gidel et al . ( 2019 ) pointed out that multi-player games can be cast as VIs , and proposed to study mini-max or non-zero-sum games formulations of GANs ( Goodfellow et al. , 2014 ) in this fashion . This allowed them to successfully transfer established insights and well-known techniques from the vast literature on VIs to the study of GANs . In particular , oscillatory behavior of optimization methods ( such as SGD ) not originally designed to solve VI problems is well understood in the VI literature , and established tools , such as averaging and extrapolation , can be successfully applied to the training of GANs . Besides their usefulness in studying GANs and alternative adversarial learning models ( Madry et al. , 2018 ) , VIs have recently attracted considerable attention of the machine learning community due to their ability to model other situations where the minimization of a single loss function does not suffice , such as auction theory ( Syrgkanis et al. , 2015 ) and robust and multi-agent reinforcement learning ( Pinto et al. , 2017 ) . In summary , VIs have recently become a potent tool enabling new advances in practical machine learning situations reaching beyond supervised learning where optimization problems and techniques , which can be seen as special instances of VIs and methods for solving them , reign supreme . 1.2 TRAINING OF SUPERVISED MODELS VIA DISTRIBUTED OPTIMIZATION . On the other hand , in the domain of classical , and hence also much better understood , supervised machine learning characterized by the fact that standard optimization techniques apply and work well , researchers and practitioners face other challenges that are currently beyond the reach of existing VI methods . Indeed , the training of modern supervised machine learning models in general , and deep neural networks in particular , is still extremely challenging . Due to their desire to improve the generalization of deployed models , machine learning engineers need to rely on training datasets of ever increasing sizes and on elaborate over-parametrized models ( Arora et al. , 2018 ) . Supporting workloads of such unprecedented magnitudes would be impossible without combining the latest advances in hardware acceleration , distributed systems and distributed algorithm design ( Verbraeken et al. , 2019 ) . When training such modern supervised models in a distributed fashion , communication cost is often the bottleneck of the training system , and for this reason , a lot of effort was recently targeted at the design of communication efficient distributed optimization methods ( Konečný et al. , 2016 ; Smith et al. , 2018 ; Ghosh et al. , 2020 ; Gorbunov et al. , 2021 ) . A particularly successful technique for improving the communication efficiency of distributed first order optimization methods is communication compression . The idea behind this technique is rooted in the observation that in practical implementations it often advantageous to communicate messages compressed via ( often randomized ) lossy compression techniques instead of communicating the full messages ( Seide et al. , 2014 ; Alistarh et al. , 2017 ) . If the number of parallel workers is large enough , the noise introduced by compression is reduced , and training with compressed communication will often lead to the comparable test error while reducing the amount of communicated bits , which results in faster training , both in theory and practice ( Mishchenko et al. , 2019 ; Gorbunov et al. , 2021 ) . 1.3 TWO CLASSES OF COMPRESSION OPERATORS . We say that a ( possibly ) stochastic mapping Q : Rd ! Rd is an unbiased compression operator if there exists a constant q 1 such that EQ ( z ) = z , EkQ ( z ) k2 qkzk2 , 8z 2 Rd . ( 1 ) Further , we say that a stochastic mapping C : Rd ! Rd is a contractive compression operator if there exists a constant 1 such that EkC ( z ) zk2 ( 1 1/ ) kzk2 , 8z 2 Rd . ( 2 ) If b is the number of bits needed to represent a single float ( e.g. , b = 32 or b = 64 ) , then the number of bits needed to represent a generic vector z 2 Rd is kzkbits : = bd . To describe how much a compression operator reduces its input vector on average , we define the notion of expected density , defined via : = 1bdEkQ ( z ) kbits , where kQ ( z ) kbits denotes the number of bits needed to represent the quantized vector Q ( z ) . Note that 1 . For the Randk operator ( Alistarh et al. , 2018 ; Beznosikov et al. , 2020 ) we have q = d/k and = k/d . 1.4 TOWARDS COMMUNICATION-EFFICIENT DISTRIBUTED METHODS FOR VIS . While classical VI algorithms , such as the extragradient method originally proposed by Korpelevich ( 1976 ) and later studied by many authors ( Nemirovski , 2004 ; Juditsky et al. , 2008 ) , were not designed to work in a distributed environment , virtually all methods that were ( Yuan et al. , 2014 ; Hou et al. , 2021 ; Deng & Mahdavi , 2021 ; Beznosikov et al. , 2021b ; c ) do not consider the general VI problem , but tackle the special case of saddle point problems only . Moreover , none of these distributed methods support communication compression , with the exception of the work of Yuan et al . ( 2014 ) , which relies on rounding to the nearest integer multiple of a certain quantity . This compression mechanism does not offer theoretical benefits and does not even lead to convergence to the solution due to the errors introduced through rounding persist and prevent the method from solving the problem . 2 SUMMARY OF CONTRIBUTIONS . In this paper , we investigate whether it is possible to design communication-efficient algorithms for solving distributed VI problems by borrowing generic communication compression techniques ( 1 ) and ( 2 ) from the optimization literature ( Seide et al. , 2014 ; Alistarh et al. , 2017 ; Mishchenko et al. , 2019 ; Gorbunov et al. , 2021 ; Richtárik et al. , 2021 ) and adapting and embedding them into established and efficient methods for solving VIs ( Korpelevich , 1976 ; Nemirovski , 2004 ; Juditsky et al. , 2008 ; Alacaoglu & Malitsky , 2021 ) . Whether or not this is possible is an open problem . In summary , we design the first provably communication-efficient algorithms for solving general distributed VI problems ( see Section 3 , Equation 3 ) in the deterministic ( see ( 4 ) ) and stochastic ( see ( 5 ) ) regimes , supporting both unbiased ( MASHA1 = Algorithm 1 ) and contractive ( MASHA2 = Algorithm 2 ) compressors . Our methods are explicitly designed to be variance reduced to achieve better theoretical properties and better practical performance . In Table 1 we give a high level overview of existing methods for VIs , and contrast them with our methods and results . We now elaborate a bit more : 2.1 TWO DISTRIBUTED PROBLEMS : DETERMINISTIC AND STOCHASTIC . We study two distributed VI problems : i ) deterministic , where the monotone operator F : Rd ! Rd featured in the VI is the average of M operators { Fm } Mm=1 , where M is the number of devices/machines , which can be evaluated in each communication round , and ii ) stochastic , where each monotone operator Fm : Rd ! Rd has a finite-sum structure on its own , and only a single operator in the sum can be evaluated in each iteration . In contrast to previous works , we study general constrained VIs in the distributed setup ( see Section 3 ) , and not merely saddle point problems . 2.2 TWO NEW METHODS WITH COMPRESSED COMMUNICATION : MASHA1 AND MASHA2 We develop two extensions of the extragradient / extrastep method of Korpelevich ( 1976 ) to distributed VIs depending on whether we use unbiased ( 1 ) or contractive ( 2 ) compressors , since each type of compressor demands a different algorithmic design and a different analysis . In particular , contractive compressors are notoriously hard to analyze even for optimization problems ( Karimireddy et al. , 2019 ; Richtárik et al. , 2021 ) . Our method based on unbiased compressors is called MASHA1 ( Algorithm 1 ) , and our method based on contraction compressors is called MASHA2 ( Algorithm 2 ) . Both are designed to handle the deterministic and also the stochastic setting , and both are enhanced with bespoke variance-reduction techniques for better theoretical and practical performance . Due to space restrictions , we only describe MASHA1 in the main body of the paper , and relegate MASHA2 and the associated theory to Appendix B . 2.3 THEORETICAL COMPLEXITY RESULTS . We establish a number of theoretical complexity results for our methods , which we summarize in Table 2 . We consider the strongly convex - strongly concave regime as well as the more general convex - concave regime . In the first case we obtain linear convergence results ( O ( log 1/✏ ) ) in terms of the distance to solution , and in the latter we obtain fast sublinear convergence results ( O ( 1/✏ ) ) in terms of the gap function . To get an estimate for the number of information transmitted , one need to multiply the estimates from Table 1 by 1/q and 1/ , respectively . Then we get that from the point of view of the transmitted information , MASHA1 is better by a factor p 1/q + 1/M in comparison with the original extragradient . 3 PROBLEM FORMULATION AND ASSUMPTIONS . Let us first introduce basic notation . We write hx , yi : = Pd i=1 xiyi to denote the standard Euclidean inner product of vectors x , y 2 Rd . This induces ` 2-norm in Rd as usual : kxk : = p hx , xi . We also introduce the proximal operator , defined as proxg ( z ) : = argminu2Z { g ( u ) + 1 2ku zk 2 } , which is well defined for proper lower semicontinuous convex functions g : Rd ! R [ { +1 } .

Summary: The paper studies the communication needed in order for a group of distributed players to collectively solve a variational inequality problem. The paper provides two algorithms MASHA1, MASHA2 to do this that solve both the deterministic and stochastic cases.  They also provide experimental results on applying their techniques to Bilinear saddle point problem and adversarial training of transformers.