Title: Byzantine-Tolerant Methods for Distributed Variational Inequalities
Venue: NeurIPS 2023
Accepted: yes
Keywords: byzantine robustness

variational inequalities

min-max problems
Abstract: Robustness to Byzantine attacks is a necessity for various distributed training scenarios. When the training reduces to the process of solving a minimization problem, Byzantine robustness is relatively well-understood. However, other problem formulations, such as min-max problems or, more generally, variational inequalities, arise in many modern machine learning and, in particular, distributed learning tasks. These problems significantly differ from the standard minimization ones and, therefore, require separate consideration. Nevertheless, only one work [Abidi et al., 2022] addresses this important question in the context of Byzantine robustness. Our work makes a further step in this direction by providing several (provably) Byzantine-robust methods for distributed variational inequality, thoroughly studying their theoretical convergence, removing the limitations of the previous work, and providing numerical comparisons supporting the theoretical findings.

Byzantine-Tolerant Methods for Distributed Variational Inequalities

Abstract

Robustness to Byzantine attacks is a necessity for various distributed training scenarios. When the training reduces to the process of solving a minimization problem, Byzantine robustness is relatively well-understood. However, other problem formulations, such as min-max problems or, more generally, variational inequalities, arise in many modern machine learning and, in particular, distributed learning tasks. These problems significantly differ from the standard minimization ones and, therefore, require separate consideration. Nevertheless, only one work [Adibi et al., 2022] addresses this important question in the context of Byzantine robustness. Our work makes a further step in this direction by providing several (provably) Byzantine-robust methods for distributed variational inequality, thoroughly studying their theoretical convergence, removing the limitations of the previous work, and providing numerical comparisons supporting the theoretical findings.

1 Introduction

Modern machine learning tasks require to train large models with billions of parameters on huge datasets to achieve reasonable quality. Training of such models is usually done in a distributed manner since otherwise it can take a prohibitively long time [Li, 2020]. Despite the attractiveness of distributed training, it is associated with multiple difficulties not existing in standard training.

In this work, we focus on one particular aspect of distributed learning Byzantine tolerance/robustness - the robustness of distributed methods to the presence of Byzantine workers 2 , i.e., such workers that can send incorrect information (maliciously or due to some computation errors/faults) and are assumed to be omniscient. For example, this situation can appear in collaborative training, when several participants (companies, universities, individuals) that do not necessarily know each other train some model together [Kijsipongse et al., 2018, Diskin et al., 2021] or when the devices used in training are faulty [Ryabinin et al., 2021]. When the training reduces to the distributed minimization problem, the question of Byzantine robustness is studied relatively well both in theory and practice [Karimireddy et al., 2022, Lyu et al., 2020].

However, there are a lot of problems that cannot be reduced to minimization, e.g., adversarial training [Goodfellow et al., 2015, Madry et al., 2018], generative adversarial networks (GANs) [Goodfellow et al., 2014], hierarchical reinforcement learning [Wayne and Abbott, 2014, Vezhnevets et al., 2017], adversarial examples games [Bose et al., 2020], and other problems arising in game theory, control theory, and differential equations [Facchinei and Pang, 2003]. Such problems lead to min-max

or, more generally, variational inequality (VI) problems [Gidel et al., 2018] that have significant differences from minimization ones and require special consideration [Harker and Pang, 1990, Ryu and Yin, 2022]. Such problems can also be huge scale, meaning that, in some cases, one has to solve them distributedly. Therefore, similarly to the case of minimization, the necessity in Byzantine-robust methods for distributed VIs arises.

The only existing work addressing this problem is [Adibi et al., 2022], where the authors propose the first Byzantine-tolerant distributed method for min-max and VI problems called Robust Distributed Extragradient ( RDEG ). However, several interesting directions such as application of ( δ, c ) -robust aggregation rules, client momentum [Karimireddy et al., 2021], and checks of computations [Gorbunov et al., 2022b] studied for minimization problems are left unexplored in the case of VIs. Moreover, [Adibi et al., 2022] prove the convergence to the solution's neighborhood that can be reduced only via increasing the batchsize and rely on the assumption that the number of workers is sufficiently large and the fraction of Byzantine workers is smaller than 1 / 16 , which is much smaller than for SOTA results in minimization case. Our work closes these gaps in the literature and resolves the limitations of the results from [Adibi et al., 2022].

1.1 Setting

To make the further presentation precise, we need to introduce the problem and assumptions we make. We consider the distributed unconstrained variational inequality (non-linear equation) problem 3 :

where G denotes the set of regular/good workers and operators F i have an expectation form F i ( x ) := E ξ i [ g i ( x ; ξ i )] . We assume that n workers connected with a server take part in the learning/optimization process and [ n ] = G ⊔ B , where B is the set of Byzantine workers - the subset B of workers [ n ] that can deviate from the prescribed protocol (send incorrect information, e.g., arbitrary vectors instead of stochastic estimators) either intentionally or not and are omniscient 4 , i.e., Byzantine workers can know the results of computations on regular workers and the aggregation rule used by the server. The number of Byzantine workers B = | B | is assumed to satisfy B ≤ δn , where δ < 1 / 2 (otherwise Byzantines form a majority and the problem becomes impossible to solve). The number of regular workers is denoted as G = | G | .

Assumptions. Here, we formulate the assumptions related to the stochasticity and properties of operators { F i } i ∈ G .

Assumption 1. For all i ∈ G the stochastic estimator g i ( x , ξ i ) is an unbiased estimator of F i ( x ) with bounded variance, i.e., E ξ i [ g i ( x , ξ i )] = F i ( x ) and for some σ ≥ 0

The above assumption is known as the bounded variance assumption. It is classical for the analysis of stochastic optimization methods [Nemirovski et al., 2009, Juditsky et al., 2011] and is used in the majority of existing works on Byzantine robustness with theoretical convergence guarantees.

Further, we assume that the data heterogeneity across the workers is bounded.

Assumption 2. There exists ζ ≥ 0 such that for all x ∈ R d

Condition (3) is a standard notion of data heterogeneity in Byzantine-robust distributed optimization [Wu et al., 2020, Zhu and Ling, 2021, Karimireddy et al., 2022, Gorbunov et al., 2023a]. It is worth

mentioning that without any kind of bound on the heterogeneity of { F i } i ∈ G , it is impossible to tolerate Byzantine workers. In addition, homogeneous case ( ζ = 0 ) is also very important and arises in collaborative learning, see [Kijsipongse et al., 2018, Diskin et al., 2021].

Finally, we formulate here several assumptions on operator F . Each particular result in this work relies only on a subset of listed assumptions.

Assumption 3. Operator F : R d → R d is L -Lipschitz, i.e.,

Assumption 4. Operator F : R d → R d is µ -quasi strongly monotone, i.e., for µ ≥ 0

Assumption 5. Operator F : R d → R d is monotone, i.e.,

Assumption 6. Operator F : R d → R d is ℓ -star-cocoercive , i.e., for ℓ ≥ 0

Assumptions 3 and 5 are quite standard for the literature on VIs. Assumptions 4 and 6 can be seen as structured non-monotonicity assumptions. Indeed, there exist examples of non-monotone (and even non-Lipschitz) operators such that Assumptions 4 and 6 holds [Loizou et al., 2021]. However, Assumptions 3 and 5 imply neither (QSM) nor (SC). It is worth mentioning that Assumption 4 is also known under different names, i.e., strong stability [Mertikopoulos and Zhou, 2019] and strong coherent [Song et al., 2020] conditions.

Robust aggregation. We use the formalism proposed by Karimireddy et al. [2021, 2022].

Definition 1.1 ( ( δ, c ) -RAGG [Karimireddy et al., 2021, 2022]) . Let there exist a subset G of random vectors { y 1 , . . . , y n } such that G ≥ (1 -δ ) n for some δ < 1 / 2 and E ∥ y i -y j ∥ 2 ≤ ρ 2 for any fixed pair i, j ∈ G and some ρ ≥ 0 . Then, ̂ y = RAGG ( y 1 , . . . , y n ) is called ( δ, c ) -robust aggregator if for some constant c ≥ 0

where y = 1 G ∑ i ∈ G y i . Further, if the value of ρ is not used to compute ̂ y , then ̂ y is called agnostic ( δ, c ) -robust aggregator and denoted as ̂ y = ARAGG ( y 1 , . . . , y n ) .

The above definition is tight in the sense that for any estimate ̂ y the best bound one can guarantee is E [ ∥ ̂ y -y ∥ 2 ] = Ω( δρ 2 ) [Karimireddy et al., 2021]. Moreover, there are several examples of ( δ, c ) -robust aggregation rules that work well in practice; see Appendix B.

Another important concept for Byzantine-robust learning is the notion of permutation inveriance.

Definition 1.2 (Permutation invariant algorithm) . Define the set of stochastic gradients computed by each of the n workers at some round t to be [˜ g 1 ,t , . . . , ˜ g n,t ] . For a good worker i ∈ G , these represent the true stochastic gradients whereas for a bad worker j ∈ B , these represent arbitrary vectors. The output of any optimization algorithm ALG is a function of these gradients. A permutation-invariant algorithm is one which for any set of permutations over t rounds { π 1 , . . . , π t } , its output remains unchanged if we permute the gradients.

As Karimireddy et al. [2021] prove, any permutation-invariant algorithm fails to converge to any predefined accuracy of the solution (under Assumption 1) even if all regular workers have the same operators/functions, i.e., even when ζ = 0 .

(1) consider only homogeneous case ( ζ = 0 ) .

1.2 Our Contributions

Now we are ready to describe the main contributions of this work.

· Methods with provably robust aggregation. We propose new methods called Stochastic Gradient Descent-Ascent and Stochastic Extragradient with Robust Aggregation ( SGDA-RA and SEG-RA ) - variants of popular SGDA [Dem'yanov and Pevnyi, 1972, Nemirovski et al., 2009] and SEG [Korpelevich, 1976, Juditsky et al., 2011]. We prove that SGDA-RA and SEG-RA work with any ( δ, c ) -robust aggregation rule and converge to the desired accuracy if the batchsize is large enough . In the experiments, we observe that SGDA-RA and SEG-RA outperform RDEG in several cases.

· Client momentum. As the next step, we add client momentum to SGDA-RA and propose Momentum SGDA-RA ( M-SGDA-RA ). As it is shown by [Karimireddy et al., 2021, 2022], client momentum helps to break the permutation invariance of the method and ensures convergence to any predefined accuracy with any batchsize for non-convex minimization problems . In the case of star-cocoercive quasi-strongly monotone VIs, we prove the convergence to the neighborhood of the solution; the size of the neighborhood can be reduced via increasing batchsize only - similarly to the results for RDEG , SGDA-RA , and SEG-RA . We discuss this limitation in detail and point out the non-triviality of this issue. Nevertheless, we show in the experiments that client momentum does help to achieve better accuracy of the solution.

· Methods with random checks of computations. Finally, for homogeneous data case ( ζ = 0 ), we propose a version of SGDA and SEG with random checks of computations ( SGDA-CC , SEG-CC and their restarted versions R-SGDA-CC and R-SEG-CC ). We prove that the proposed methods converge to any accuracy of the solution without any assumptions on the batchsize . This is the first result of this type on Byzantine robustness for distributed VIs. Moreover, when the target accuracy of the solution is small enough, the obtained convergence rates for R-SGDA-CC and R-SEG-CC are not worse than the ones for distributed SGDA and SEG derived in the case of δ = 0 (no Byzantine workers); see the comparison of the convergence rates in Table 1. In the numerical experiments, we consistently observe the superiority of the methods with checks of computations to the previously proposed methods.

1.3 Related Work

Byzantine-robust methods for minimization problems. Classical distributed methods like Parallel SGD [Zinkevich et al., 2010] cannot tolerate even one Byzantine worker. The most evident vulnerability of such methods is an aggregation rule (averaging). Therefore, many works focus on designing and application of different aggregation rules to Parallel SGD -like methods [Blanchard et al., 2017, Yin et al., 2018, Damaskinos et al., 2019, Guerraoui et al., 2018, Pillutla et al., 2022]. However, this is not sufficient for Byzantine robustness: there exist particular attacks [Baruch et al., 2019, Xie et al., 2019] that can bypass popular defenses. [Karimireddy et al., 2021] formalize the definition of robust aggregation (see Definition 1.1), show that many standard aggregation rules are non-robust according to that definition, and prove that any permutation-invariant algorithm with a fixed batchsize can converge only to the ball around the solution with algorithm-independent radius. Therefore, more in-depth algorithmic changes are required that also explain why RDEG , SGDA-RA , and SEG-RA are not converging to any accuracy without increasing batchsize.

One possible way to resolve this issue is to use client momentum [Karimireddy et al., 2021, 2022] that breaks permutation-invariance and allows for convergence to any accuracy. It is also worth mentioning a recent approach by [Allouah et al., 2023], who propose an alternative definition of robust aggregation to the one considered in this paper, though to achieve the convergence to any accuracy in the homogeneous case [Allouah et al., 2023] apply client momentum like in [Karimireddy et al., 2021, 2022]. Another line of work achieves Byzantine robustness through the variance reduction mechanism [Wu et al., 2020, Zhu and Ling, 2021, Gorbunov et al., 2023a]. Finally, for the homogeneous data case, one can apply validation test [Alistarh et al., 2018, Allen-Zhu et al., 2021] or checks of computations [Gorbunov et al., 2022b]. For the summary of other advances, we refer to [Lyu et al., 2020].

Methods for min-max and variational inequalities problems. As mentioned before, minmax/variational inequalities (VIs) problems have noticeable differences with standard minimization. In particular, it becomes evident from the differences in the algorithms' behavior. For example, a direct analog of Gradient Descent for min-max/VIs - Gradient Descent-Ascent ( GDA ) [Krasnosel'skii, 1955, Mann, 1953, Dem'yanov and Pevnyi, 1972, Browder, 1966] - fails to converge for a simple bilinear game. Although GDA converges for a different class of problems (cocoercive/star-cocoercive ones) and its version with alternating steps works well in practice and even provably converges locally [Zhang et al., 2022], many works focus on Extragradient ( EG ) type methods [Korpelevich, 1976, Popov, 1980] due to their provable convergence for monotone Lipschitz problems and beyond [Tran-Dinh, 2023]. Stochastic versions of GDA and EG ( SGDA and SEG ) are studied relatively well, e.g., see [Hsieh et al., 2020, Loizou et al., 2021, Mishchenko et al., 2020, Pethick et al., 2023] for the recent advances.

On the results from [Adibi et al., 2022]. In the context of Byzantine robustness for distributed minmax/VIs, the only existing work is [Adibi et al., 2022]. The authors propose a method called Robust Distributed Extragradient ( RDEG ) - a distributed version of EG that uses a univariate trimmed-mean estimator from [Lugosi and Mendelson, 2021] for aggregation. This estimator satisfies a similar property to (4) that is shown for δ < 1 / 16 and large enough n (see the discussion in Appendix B). In contrast, the known ( δ, c ) -robust aggregation rules allow larger δ , and do not require large n . Despite these evident theoretical benefits, such aggregation rules were not considered in prior works on Byzantine robustness for distributed variational inequalities/min-max problems.

2 Main Results

In this section, we describe three approaches proposed in this work and formulate our main results.

2.1 Methods with Robust Aggregation

We start with the Stochastic Gradient Descent-Accent with ( δ, c ) -robust aggregation ( SGDA-RA ):

where { g t i } i ∈ G are sampled independently. The main result for SGDA-RA is given below.

Theorem 1. Let Assumptions 1, 2, 4 and 6 hold. Then after T iterations SGDA-RA (Algorithm 1) with ( δ, c ) -RAGG and γ ≤ 1 2 ℓ outputs x T such that

The first two terms in the derived upper bound are standard for the results on SGDA under Assumptions 1, 4, and 6, e.g., see [Beznosikov et al., 2023]. The third and the fourth terms come from the presence of Byzantine workers and robust aggregation since the existing ( δ, c ) -robust aggregation rules explicitly depend on δ . The fourth term cannot be reduced without increasing batchsize even when ζ = 0 (homogeneous data case). This is expected since SGDA-RA is permutation invariant. When σ = 0 (regular workers compute full operators), then SGDA-RA converges linearly to the ball centered at the solution with radius O ( √ cδζ / µ ) that matches the lower bound from [Karimireddy et al., 2022]. In contrast, the known results for RDEG are derived for homogeneous data case ( ζ = 0 ). The proof of Theorem 1 is deferred to Appendix D.1.

Using a similar approach we also propose a version of Stochastic Extragradient method with ( δ, c ) -robust aggregation called SEG-RA :

where { g t η i } i ∈ G and { g t η i } i ∈ G are sampled independently. Our main convergence result for SEG-RA is presented in the following theorem; see Appendix D.2 for the proof.

Theorem 2. Let Assumptions 5 1, 2, 3 and 4 hold. Then after T iterations SEG-RA (Algorithm 2) with ( δ, c ) -RAGG , γ 1 ≤ 1 2 µ +2 L and β = γ 2 / γ 1 ≤ 1 / 4 outputs x T such that

Similar to the case of SGDA-RA , the bound for SEG-RA has the term that cannot be reduced without increasing batchsize even in the homogeneous data case. RDEG , which is also a modification of SEG , has the same linearly convergent term, but SEG-RA has a better dependence on the batchsize, needed to obtain the convergence to any predefined accuracy, that is O ( ε -1 ) versus O ( ε -2 ) for RDEG ; see Cor. 3.

In heterogeneous case when σ = 0 , SEG-RA also converges linearly to the ball centered at the solution with radius O ( √ cδζ / µ ) that matches the lower bound.

2.2 Client Momentum

Next, we focus on the version of SGDA-RA that utilizes worker momentum m t i , i.e.,

where g t i = g i ( x t , ξ t i ) , ∀ i ∈ G and g t i = ∗ ∀ i ∈ B and { g t ξ i } i ∈ G are sampled independently. Our main convergence result for this version called M-SGDA-RA is summarized in the following theorem.

Theorem 3. Let Assumptions 1, 2, 4, and 6 hold. Then after T iterations M-SGDA-RA (Algorithm 3) with ( δ, c ) -RAGG outputs x T such that

Despite the fact that M-SGDA-RA is the first algorithm (for VIs) non-invariant to permutations, it also requires large batches to achieve convergence to any accuracy. Even in the context of minimization, which is much easier than VI, the known SOTA analysis of Momentum-SGD relies in the convex case on the unbiasedness of the estimator that is not available due to a robust aggregation. Nevertheless, we prove 6 the convergence to the ball centered at the solution with radius O ( √ cδ ( ζ + σ ) / αµ ) ; see Appendix D.3. Moreover, we show that M-SGDA-RA outperforms in the experiments other methods that require large batches.

2.3 Random Checks of Computations

We start with the Stochastic Gradient Descent-Accent with Checks of Computations ( SGDA-CC ). At each iteration of SGDA-CC , the server selects m workers (uniformly at random) and requests them to check the computations of other m workers from the previous iteration. Let V t be the set of workers that verify/check computations, A t are active workers at iteration t , and V t ∩ A t = ∅ . Then, the update of SGDA-CC can be written as

where { g i ( x t , ξ t i ) } i ∈ G are sampled independently.

The acceptance (of the update) event occurs when the condition ∥ ∥ g t -g i ( x t , ξ t i ) ∥ ∥ ≤ Cσ holds for the majority of workers. If g t is rejected, then all workers re-sample g i ( x t , ξ t i ) until acceptance is achieved. The rejection probability is bounded, as per [Gorbunov et al., 2022b], and can be adjusted by choosing a constant C = O (1) . We assume that the server knows the seeds for generating randomness on workers, and thus, verification of computations is possible. Following each aggregation of g i ( x t , ξ t i ) i ∈ G , the server selects uniformly at random 2 m workers: m workers check the computations at the previous step of the other m workers. For instance, at the ( t +1) -th iteration, the server asks a checking peer i to compute g j ( x t , ξ t j ) , where j is a peer being checked. This is possible if all seeds are broadcasted at the start of the training. Workers assigned to checking do not participate in the training while they check and do not contribute to g t . Therefore, each Byzantine peer is checked at each iteration with a probability of ∼ m / n by some good worker (see the proof of Theorem 4). If the results are mismatched, then both the checking and checked peers are removed from training.

This design ensures that every such mismatch, whether it is caused by honest or Byzantine peers, eliminates at least one Byzantine peer and at most one honest peer (see details in Appendix E.1). It's worth noting that we assume any information is accessible to Byzantines except when each of them will be checked. As such, Byzantine peers can only reduce their relative numbers, which leads us to the main result for SGDA-CC , which is presented below.

Theorem 4. Let Assumptions 1, 4 and 6 hold. Then after T iterations SGDA-CC (Algorithm 5) with γ ≤ 1 2 ℓ outputs x T such that

The above theorem (see Appendix E.1 for the proof) provides the first result that does not require large batchsizes to converge to any predefined accuracy. The first and the second terms in the convergence bound correspond to the SOTA results for SGDA [Loizou et al., 2021]. Similarly to the vanilla SGDA , the convergence can be obtained by decreasing stepsize, however, such an approach does not benefit from collaboration, since the dominating term γσ 2 nB µm (coming from the presence of Byzantine workers) is not inversely dependent on n . Moreover, the result is even worse than for single node SGDA in terms of dependence on n .

To overcome this issue we consider the restart technique for SGDA-CC and propose the next algorithm called R-SGDA-CC . This method consists of r stages. On the t -th stage R-SGDA-CC runs SGDA-CC with γ t for K t iterations from the starting point ̂ x t , which is the output from the previous stage, and defines the obtained point as ̂ x t +1 (see details in Appendix E.2). The main result for R-SGDA-CC is given below.

Theorem 5. Let Assumptions 1, 4 and 6 hold. Then, after r = ⌈ log 2 R 2 ε ⌉ -1 restarts R-SGDA-CC (Algorithm 6) with γ t = min { 1 2 ℓ , √ ( n -2 B -m ) R 2 6 σ 2 2 t K t , √ m 2 R 2 72 qσ 2 2 t B 2 n 2 } and √

K t = ⌈ max { 8 ℓ µ , 96 σ 2 2 t ( n -2 B -m ) µ 2 R 2 , 34 nσB q 2 t mµR }⌉ , where R ≥ ∥ x 0 -x ∗ ∥ , outputs ̂ x r such that E ∥ ̂ x r -x ∗ ∥ 2 ≤ ε . Moreover, the total number of executed iterations of SGDA-CC is

The above result implies that R-SGDA-CC also converges to any accuracy without large batchsizes (see Appendix E.2 for details). However, as the accuracy tends to zero, the dominant term σ 2 ( n -2 B -m ) µε inversely depends on the number of workers. This makes R-SGDA-CC benefit from collaboration, as the algorithm becomes more efficient with an increasing number of workers. Moreover, when B and m are small the derived complexity result for R-SGDA-CC matches the one for parallel SGDA [Loizou et al., 2021], which is obtained for the case of no Byzantine workers.

Next, we present a modification of Stochastic Extragradient with Checks of Computations ( SEG-CC ):

where { g i ( x t , ξ t i ) } i ∈ G and { g i ( ˜ x t , η t i ) } i ∈ G are sampled independently. The events of acceptance g t η ( or g t ξ ) happens if

holds for the majority of workers. An iteration of SEG-CC actually represents two subsequent iteration of SGDA-CC , so we refer to the beginning of the section for more details. Our main convergence results for SEG-CC are summarized in the following theorem; see Appendix E.3 for the proof.

Theorem 6. Let Assumptions 1, 3 and 4 hold. Then after T iterations SEG-CC (Algorithm 7) with γ 1 ≤ 1 2 µ +2 L and β = γ 2 / γ 1 ≤ 1 / 4 outputs x T such that

Similarly to SGDA-CC , SEG-CC does not require large batchsizes to converge to any predefined accuracy and does not benefit of collaboration, though the first two terms correspond to the SOTA convergence results for SEG under bounded variance assumption [Juditsky et al., 2011]. The last term appears due to the presence of the Byzantine workers. The restart technique can also be applied; see Appendix E.4 for the proof.

Theorem 7. Let Assumptions 1, 3, 4 hold. Then, after r = ⌈ log 2 R 2 ε ⌉ -1 restarts R-SEG-CC (Algotithm 8) with γ 1 t = min { 1 2 L , √ ( G -B -m ) R 2 16 σ 2 2 t K t , √ mR 2 8 qσ 2 2 t Bn } , γ 2 t = min { 1 4 L , √ m 2 R 2 64 qσ 2 2 t B 2 n 2 , √ ( G -B -m ) R 2 64 σ 2 K t } and K t = ⌈ max { 8 L µ , 16 nσB √ q 2 t mµR , 256 σ 2 2 t ( G -B -m ) µ 2 R 2 }⌉ , where R ≥ ∥ x 0 -x ∗ ∥ outputs ̂ x r such that E ∥ ̂ x r -x ∗ ∥ 2 ≤ ε . Moreover, the total number of executed iterations of SEG-CC is

The above result states that R-SEG-CC also converges to any accuracy without large batchsizes; see Appendix E.4. But with accuracy tending to zero ( ε → 0 ) the dominating term σ 2 ( n -2 B -m ) µε inversely depends on the number of workers, hence R-SEG-CC benefits from collaboration. Moreover, when B and m are small the derived complexity result for R-SEG-CC matches the one for parallel/minibatched SEG [Juditsky et al., 2011], which is obtained for the case of no Byzantine workers.

3 Numerical Experiments

Quadratic game. To illustrate our theoretical results, we conduct numerical experiments on a quadratic game

The above problem can be re-formulated as a special case of (1) with F defined as follows:

with symmetric matrices A j,i s.t. µ I ≼ A j,i ≼ ℓ I , A i ∈ R d × d and b i ∈ R d ; see Appendix F for the detailed description.

We set ℓ = 100 , µ = 0 . 1 , s = 1000 and d = 50 . Only one peer checked the computations on each iteration ( m = 1 ). We used RFA (geometric median) with bucketing as an aggregator since it showed the best performance. For approximating the median we used Weiszfeld's method with 10 iterations and parameter ν = 0 . 1 [Pillutla et al., 2022]. RDEG [Adibi et al., 2022] provably works only if n ≥ 100 , so here we provide experiments with n = 150 , B = 20 , γ = 2 e -5 . We set the parameter α = 0 . 1 for M-SGDA-RA , and the following parameters for RDEG : α RDEG = 0 . 06 , δ RDEG = 0 . 9 and theoretical value of ϵ ; see Appendix F for more experiments. We tested the algorithms under the following attacks: bit flipping (BF), random noise (RN), inner product manipulation (IPM) Xie et al. [2019] and 'a little is enough' (ALIE) Baruch et al. [2019].

Robust Neural Networks training. Let f ( u ; x, y ) be the loss function of a neural network with parameters u ∈ R d given input x ∈ R m and label y . For example, in our experiments, we let f be the cross entropy loss, and { ( x i , y i ) } N 1 is the MNIST dataset. Now consider the following objective:

This min-max objective adds an extra adversarial noise variable to the input data such that it maximizes the loss, so the neural network should become robust to such noise as it minimizes the loss. We can reformulate this objective as a variational inequality with

We let n = 20 , B = 4 , λ 1 = 0 , and λ 2 = 100 . We fix the learning rate to 0.01 and use a batch size of 32. We run the algorithm for 50 epochs and average our results across 3 runs. We test the algorithms under the following attacks: i) bit flipping (BF), ii) label flipping (LF), iii) inner product manipulation (IPM) Xie et al. [2019], and iv) a little is enough (ALIE) Baruch et al. [2019]. We compare our algorithm SGDA-CC against the following algorithms: i) SGDA-RA , ii) M-SGDA-RA , and iii) RDEG Adibi et al. [2022]. We use RFA with bucket size 2 as the robust aggregator. The results are shown in Figure 2. Specifically, we show the validation error on MNIST after each epoch. We can see that SGDA-CC performs the best, followed closely by M-SGDA-RA .

4 Conclusion

This paper proposes several new algorithms for Byzantine-robust distributed variational inequalities and provides rigorous theoretical convergence analysis for the developed methods. In particular, we

0

2

4

6

8

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=1

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

0

2

4

6

8

Number of gradients

1e5

10

4

10

3

10

2

10

1

Distance to the solution

BF, bs=1

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

0

2

4

6

8

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

IPM (1e+01), bs=1

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

0

2

4

6

8

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

RN (1e+01), bs=1

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

0

2

4

6

8

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=1

SEGCC (2e-05)

SGDACC (2e-05)

MSGDARA (2e-05)

0

2

4

6

8

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

BF, bs=1

SEGCC (2e-05)

SGDACC (2e-05)

MSGDARA (2e-05)

0

2

4

6

8

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+01), bs=1

SEGCC (2e-05)

SGDACC (2e-05)

MSGDARA (2e-05)

0

2

4

6

8

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+01), bs=1

SEGCC (2e-05)

SGDACC (2e-05)

MSGDARA (2e-05)

0

10

20

30

40

50

Epochs

10

1

Error

Adversarial MNIST Error, attack = BF

strategy

SGDA-CC

SGDA-RA

M-SGDA-RA

RDEG

0

10

20

30

40

50

Epochs

10

1

10

0

Error

Adversarial MNIST Error, attack = LF

strategy

SGDA-CC

SGDA-RA

M-SGDA-RA

RDEG

0

10

20

30

40

50

Epochs

10

1

Error

Adversarial MNIST Error, attack = IPM

strategy

SGDA-CC

SGDA-RA

M-SGDA-RA

RDEG

0

10

20

30

40

50

Epochs

10

1

Error

Adversarial MNIST Error, attack = ALIE

strategy

SGDA-CC

SGDA-RA

M-SGDA-RA

RDEG

propose the first methods in this setting that provably converge to any predefined accuracy in the case of homogeneous data. We believe this is an important step towards building a strong theory of Byzantine robustness in the case of distributed VIs.

However, our work has several limitations. First of all, one can consider different/more general assumptions about operators [Beznosikov et al., 2023, Gorbunov et al., 2022a, 2023b] in the analysis of the proposed methods. Next, as we mention in the discussion after Theorem 3, our result for M-SGDA-RA requires large batchsizes, and it remains unclear to us whether this requirement can be removed. Finally, the only results that do not require large batchsizes are derived using the checks of computations that create (although small) computation overhead. Obtaining similar results without checks of computations remains an open problem. Addressing these limitations is a prominent direction for future research.

Acknowledgments and Disclosure of Funding

This work of N. Tupitsa was supported by a grant for research centers in the field of artificial intelligence, provided by the Analytical Center for the Government of the Russian Federation in accordance with the subsidy agreement (agreement identifier 000000D730321P5Q0002) and the agreement with the Moscow Institute of Physics and Technology dated November 1, 2021 No. 70-2021-00138.

References

Adibi, A. Mitra, G. J. Pappas, and H. Hassani. Distributed statistical min-max learning in the presence of byzantine agents. In 2022 IEEE 61st Conference on Decision and Control (CDC) , pages 4179-4184. IEEE, 2022.

Alistarh, Z. Allen-Zhu, and J. Li. Byzantine stochastic gradient descent. Advances in Neural Information Processing Systems , 31, 2018.

Allen-Zhu, F. Ebrahimianghazani, J. Li, and D. Alistarh. Byzantine-resilient non-convex stochastic gradient descent. In International Conference on Learning Representations , 2021.

Allouah, S. Farhadkhani, R. Guerraoui, N. Gupta, R. Pinot, and J. Stephan. Fixing by mixing: A recipe for optimal byzantine ml under heterogeneity. In International Conference on Artificial Intelligence and Statistics , pages 1232-1300. PMLR, 2023.

Baruch, G. Baruch, and Y. Goldberg. A little is enough: Circumventing defenses for distributed learning, 2019.

Beznosikov, E. Gorbunov, H. Berard, and N. Loizou. Stochastic gradient descent-ascent: Unified theory and new efficient methods. In International Conference on Artificial Intelligence and Statistics , pages 172-235. PMLR, 2023.

Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in Neural Information Processing Systems , 30, 2017.

Bose, G. Gidel, H. Berard, A. Cianflone, P. Vincent, S. Lacoste-Julien, and W. Hamilton. Adversarial example games. Advances in neural information processing systems , 33:8921-8934, 2020.

E. Browder. Existence and approximation of solutions of nonlinear variational inequalities. Proceedings of the National Academy of Sciences , 56(4):1080-1086, 1966.

Chen, L. Su, and J. Xu. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems , 1 (2):1-25, 2017.

Damaskinos, E.-M. El-Mhamdi, R. Guerraoui, A. Guirguis, and S. Rouault. Aggregathor: Byzantine machine learning via robust gradient aggregation. Proceedings of Machine Learning and Systems , 1:81-106, 2019.

F. Dem'yanov and A. B. Pevnyi. Numerical methods for finding saddle points. USSR Computational Mathematics and Mathematical Physics , 12(5):11-52, 1972.

Diskin, A. Bukhtiyarov, M. Ryabinin, L. Saulnier, A. Sinitsin, D. Popov, D. V. Pyrkin, M. Kashirin, A. Borzunov, A. Villanova del Moral, et al. Distributed deep learning in open collaborations. Advances in Neural Information Processing Systems , 34:7879-7897, 2021.

Facchinei and J.-S. Pang. Finite-dimensional variational inequalities and complementarity problems . Springer, 2003.

Gidel, H. Berard, G. Vignoud, P. Vincent, and S. Lacoste-Julien. A variational inequality perspective on generative adversarial networks. In International Conference on Learning Representations , 2018. URL https://openreview.net/pdf?id=r1laEnA5Ym .

Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems , 27, 2014.

J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations , 2015.

Gorbunov, D. Kovalev, D. Makarenko, and P. Richtárik. Linearly converging error compensated sgd. Advances in Neural Information Processing Systems , 33:20889-20900, 2020.

Gorbunov, H. Berard, G. Gidel, and N. Loizou. Stochastic extragradient: General analysis and improved rates. In International Conference on Artificial Intelligence and Statistics , pages 7865-7901. PMLR, 2022a.

Gorbunov, A. Borzunov, M. Diskin, and M. Ryabinin. Secure distributed training at scale. In International Conference on Machine Learning , pages 7679-7739. PMLR, 2022b. URL https://proceedings.mlr.press/v162/gorbunov22a/gorbunov22a.pdf .

Gorbunov, S. Horváth, P. Richtárik, and G. Gidel. Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top. International Conference on Learning Representations , 2023a.

Gorbunov, A. Taylor, S. Horváth, and G. Gidel. Convergence of proximal point and extragradientbased methods beyond monotonicity: the case of negative comonotonicity. In International Conference on Machine Learning , pages 11614-11641. PMLR, 2023b.

Guerraoui, S. Rouault, et al. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning , pages 3521-3530. PMLR, 2018.

Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville. Improved training of wasserstein gans. Advances in neural information processing systems , 30, 2017.

T. Harker and J.-S. Pang. Finite-dimensional variational inequality and nonlinear complementarity problems: a survey of theory, algorithms and applications. Mathematical programming , 48(1-3): 161-220, 1990.

Y.-G. Hsieh, F. Iutzeler, J. Malick, and P. Mertikopoulos. Explore aggressively, update conservatively: Stochastic extragradient methods with variable stepsize scaling. Advances in Neural Information Processing Systems , 33:16223-16234, 2020.

Juditsky, A. Nemirovski, and C. Tauvel. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems , 1(1):17-58, 2011.

P. Karimireddy, L. He, and M. Jaggi. Learning from history for byzantine robust optimization. In International Conference on Machine Learning , pages 5311-5319. PMLR, 2021.

P. Karimireddy, L. He, and M. Jaggi. Byzantine-robust learning on heterogeneous datasets via bucketing. In International Conference on Learning Representations , 2022. URL https: //arxiv.org/pdf/2006.09365.pdf .

Kijsipongse, A. Piyatumrong, et al. A hybrid gpu cluster and volunteer computing platform for scalable deep learning. The Journal of Supercomputing , 74(7):3236-3263, 2018.

P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

M. Korpelevich. The extragradient method for finding saddle points and other problems. Matecon , 12:747-756, 1976.

A. Krasnosel'skii. Two remarks on the method of successive approximations. Uspekhi matematicheskikh nauk , 10(1):123-127, 1955.

Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Lamport, R. Shostak, and M. Pease. The byzantine generals problem. ACM Transactions on Programming Languages and Systems , 4(3):382-401, 1982.

Li. Demystifying gpt-3 language model: A technical overview, 2020. " https://lambdalabs. com/blog/demystifying-gpt-3 ".

Loizou, H. Berard, G. Gidel, I. Mitliagkas, and S. Lacoste-Julien. Stochastic gradient descentascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity. Advances in Neural Information Processing Systems , 34:19095-19108, 2021.

Lugosi and S. Mendelson. Robust multivariate mean estimation: the optimality of trimmed mean. 2021.

Lyu, H. Yu, X. Ma, L. Sun, J. Zhao, Q. Yang, and P. S. Yu. Privacy and robustness in federated learning: Attacks and defenses. arXiv preprint arXiv:2012.06337 , 2020.

Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018.

R. Mann. Mean value methods in iteration. Proceedings of the American Mathematical Society , 4 (3):506-510, 1953.

Mertikopoulos and Z. Zhou. Learning in games with continuous action sets and unknown payoff functions. Mathematical Programming , 173(1):465-507, 2019.

Mishchenko, D. Kovalev, E. Shulgin, P. Richtárik, and Y. Malitsky. Revisiting stochastic extragradient. In International Conference on Artificial Intelligence and Statistics , pages 4573-4582. PMLR, 2020.

Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. URL https://openreview.net/forum?id=B1QRgziT-.

Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization , 19(4):1574-1609, 2009.

Pethick, O. Fercoq, P. Latafat, P. Patrinos, and V . Cevher. Solving stochastic weak minty variational inequalities without increasing batch size. In The Eleventh International Conference on Learning Representations , 2023.

Pillutla, S. M. Kakade, and Z. Harchaoui. Robust aggregation for federated learning. IEEE Transactions on Signal Processing , 70:1142-1154, 2022.

D. Popov. A modification of the arrow-hurwicz method for search of saddle points. Mathematical notes of the Academy of Sciences of the USSR , 28:845-848, 1980.

Ryabinin, E. Gorbunov, V. Plokhotnyuk, and G. Pekhimenko. Moshpit sgd: Communicationefficient decentralized training on heterogeneous unreliable devices. Advances in Neural Information Processing Systems , 34:18195-18211, 2021.

K. Ryu and W. Yin. Large-Scale Convex Optimization: Algorithms & Analyses via Monotone Operators . Cambridge University Press, 2022.

Song, Z. Zhou, Y. Zhou, Y. Jiang, and Y. Ma. Optimistic dual extrapolation for coherent nonmonotone variational inequalities. Advances in Neural Information Processing Systems , 33: 14303-14314, 2020.

U. Stich. Unified optimal analysis of the (stochastic) gradient method. arXiv preprint arXiv:1907.04232 , 2019.

Tran-Dinh. Sublinear convergence rates of extragradient-type methods: A survey on classical and recent developments. arXiv preprint arXiv:2303.17192 , 2023.

S. Vezhnevets, S. Osindero, T. Schaul, N. Heess, M. Jaderberg, D. Silver, and K. Kavukcuoglu. Feudal networks for hierarchical reinforcement learning. In International Conference on Machine Learning , pages 3540-3549. PMLR, 2017.

Wayne and L. Abbott. Hierarchical control using networks trained with higher-level forward models. Neural computation , 26(10):2163-2193, 2014.

Wu, Q. Ling, T. Chen, and G. B. Giannakis. Federated variance-reduced stochastic gradient descent with robustness to byzantine attacks. IEEE Transactions on Signal Processing , 68:4583-4596, 2020.

Xie, S. Koyejo, and I. Gupta. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation, 2019.

Y.-R. Yang and W.-J. Li. Basgd: Buffered asynchronous sgd for byzantine learning. In International Conference on Machine Learning , pages 11751-11761. PMLR, 2021.

Yin, Y. Chen, R. Kannan, and P. Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning , pages 5650-5659. PMLR, 2018.

Zhang, Y. Wang, L. Lessard, and R. B. Grosse. Near-optimal local convergence of alternating gradient descent-ascent for minimax optimization. In International Conference on Artificial Intelligence and Statistics , pages 7659-7679. PMLR, 2022.

Zhu and Q. Ling. Broadcast: Reducing both stochastic and compression noise to robustify communication-efficient federated learning. arXiv preprint arXiv:2104.06685 , 2021.

Zinkevich, M. Weimer, L. Li, and A. Smola. Parallelized stochastic gradient descent. Advances in neural information processing systems , 23, 2010.

Contents

A Examples of ( δ, c ) -Robust Aggregation Rules

This section is about how to construct an aggregator satisfying 1.1.

A.1 Aggregators

This subsection examines various aggregators that lack robustness. It means that new attacks can be easily designed to exploit the aggregation scheme, causing its failure. We analyze three commonly employed defenses that are representative.

̸

Krum. For i = j , let i → j denote that x j belongs to the n -q -2 closest vectors to x i . Then,

Krum is computationally expensive, requiring O ( n 2 ) work by the server Blanchard et al. [2017]. CM. Coordinate-wise median computes for the k -th coordinate:

Coordinate-wise median is fast to implement requiring only O ( n ) time Chen et al. [2017].

RFA. Robust federated averaging (RFA) computes the geometric median

Although there is no closed form solution for the geometric median, an approximation technique presented by Pillutla et al. [2022] involves performing several iterations of the smoothed Weiszfeld algorithm, with each iteration requiring a computation of complexity O ( n ) .

A.2 Bucketing algorithm

We use the process of s -bucketing , propose by [Yang and Li, 2021, Karimireddy et al., 2022] to randomly divide n inputs, x 1 to x n , into ⌈ n/s ⌉ buckets, each containing no more than s elements. After averaging the contents of each bucket to create y 1 , . . . , y ⌈ n/s ⌉ , we input them into the aggregator AGGR. The Bucketing Algorithm outlines the procedure. Our approach's main feature is that the resulting set of averaged y 1 , . . . , y ⌈ n/s ⌉ are more homogeneous (with lower variance) than the original inputs.

Algorithm Bucketing Algorithm

1: input { x 1 , . . . , x n } , s ∈ N , aggregation rule AGGR

2: pick random permutation π of [ n ]

3: compute y i ← 1 s ∑ min( n,i · s ) k =( i -1) · s +1 x π ( k ) for i = { 1 , . . . , ⌈ n/s ⌉}

4: output x ← AGGR ( y , . . . , y

̂ 1 ⌈ n/s ⌉ ) // aggregate after bucketing

A.3 Robust Aggregation examples

Next we recall the result from [Karimireddy et al., 2022], that shows that aggregators which we saw, can be made to satisfy 1.1 by combining with bucketing.

Theorem 8. Suppose we are given n inputs { x 1 , . . . , x n } such that E ∥ x i -x j ∥ 2 ≤ ρ 2 for any fixed pair i, j ∈ G and some ρ ≥ 0 for some δ ≤ δ max , with δ max to be defined. Then, running Bucketing Algorithm with s = ⌊ δ max δ ⌋ yields the following:

Krum: E ∥ KRUM ◦ BUCKETING ( x 1 , . . . , x n ) -¯ x ∥ 2 ≤ O ( δρ 2 ) with δ max < 1 4 .

Geometric median: E ∥ RFA ◦ BUCKETING ( x 1 , . . . , x n ) -¯ x ∥ 2 ≤ O ( δρ 2 ) with δ max < 1 2 .

Coordinate-wise median: E ∥ CM ◦ BUCKETING ( x 1 , . . . , x n ) -¯ x ∥ 2 ≤ O ( dδρ 2 ) with δ max < 1 2 .

Note that all these methods satisfy the notion of an agnostic Byzantine robust aggregator (Definition 1.1).

B Further Details on RDEG

Originally RDEG was proposed for min-max problems and represents a variation of SEG with Univariate Trimmed-Mean Estimator aggregation rule. For convenience we give here RDEG pseudocode we used in experiments.

In this section we use the notation π ∈ (0 , 1) for a confidence level.

Robust Distributed Extra-Gradient ( RDEG )

Performance of Univariate Trimmed-Mean Estimator. The TRIM operator takes as input n vectors, and applies coordinatewisely the univariate trimmed mean estimator from Lugosi and Mendelson [2021], described bellow here as Univariate Trimmed-Mean Estimator Algorithm.

Univariate Trimmed-Mean Estimator Algorithm Lugosi and Mendelson [2021]

Input: Corrupted data set Z 1 , . . . , Z n/ 2 , ˜ Z 1 , . . . ˜ Z n/ 2 , corruption fraction δ , and confidence level π .

1: Set ϵ = 8 δ +24 log(4 /π ) n .

2: Let Z ∗ 1 ≤ Z ∗ 2 ≤ · · · ≤ Z ∗ n/ 2 represent a non-decreasing arrangement of { Z i } i ∈ [ n/ 2] . Compute quantiles: γ = Z ∗ ϵn/ 2 and β = Z ∗ (1 -ϵ ) n/ 2 .

3: Compute robust mean estimate ̂ µ Z as follows:

The following result on the performance of Univariate Trimmed-Mean Estimator plays a key role in the analysis of RDEG.

Theorem. [Adibi et al., 2022, Theorem 1] Consider the trimmed mean estimator. Suppose δ ∈ [0 , 1 / 16) , and let π ∈ (0 , 1) be such that π ≥ 4 e -n/ 2 . Then, there exists an universal constant c , such that with probability at least 1 -π ,

Using the latter componentwise result the authors states that

In fact this result is very similar to the Definition 1.1. The main difference is that for Univariate Trimmed-Mean Estimator we have a bound with some probability. The other difference that using

the following representation of the result with ρ 2 = c 2 σ 2

Univariate Trimmed-Mean Estimator has the additional term inversely depending on the number of workers.

Moreover, the result requires δ ∈ [0 , 1 / 16) in contrast to the aggregators we used, that work for wider range of corruption level δ ∈ [0 , 1 / 5] .

Performance guarantees for RDEG . The authors of [Adibi et al., 2022] consider only homogeneous case.

Theorem. [Adibi et al., 2022, Theorem 3] Suppose Assumptions 3 and 4 hold in conjunction with the assumptions on δ and n : the fraction δ of corrupted devices satisfies δ ∈ [0 , 1 / 16) , and the number of agents n is sufficiently large: n ≥ 48 log(16 dT 2 ) . Then, with π = 1 / (4 dT 2 ) and step-size η ≤ 1 / (4 L ) , RDEG guarantees the following with probability at least 1 -1 /T :

where κ = µ/L .

The result implies that RDEG benefits of collaboration only when the corruption level is small. In fact, the term log(4 dT 2 ) n ≤ log(4 dT 2 ) 48 log(16 dT 2 ) ≤ 1 / 48 , so the corruption level should be less than 1 / 48 to make RDEG significantly benefit of collaboration in contrast to our SEG-CC that requires corruption level only less than 1 / 5 . Moreover, in case of larger corruption level, RDEG converges to a ball centered at the solution with radius ˜ O (√ √ δσRκ L ) in contrast to our methods SGDA-RA , SEG-RA and M-SGDA-RA converge to a ball centered at the solution with radius ˜ O (√ cδσ 2 µ 2 ) , that has a better dependence on σ . It is crucial with increasing batchsize ( b = batchsize), since σ 2 depends on a batchsize as 1 .

b

C Auxilary results

C.1 Basic Inequalities

For all a, b ∈ R n and λ > 0 , q ∈ (0 , 1]

C.2 Usefull Lemmas

We write g t i or simply g i instead of g i ( x t , ξ t i ) when there is no ambiguity.

Lemma C.1. Suppose that the operator F is given in the form (1) and Assumptions 1 and 6 hold. Then

where E ξ := Π i E ξ i and g ( x , ξ ) = 1 G ∑ i ∈ G g i ( x ; ξ i ) .

Proof of Lemma C.1. First of one can decomposeda squared norm of a difference and obtain

Since g ( x , ξ ) = 1 G ∑ i ∈ G g i ( x ; ξ i ) , by the definition (1) of F and by Assumption 1 one has and consequently

One can bound E ξ ∥ g ( x , ξ ) -F ( x ) ∥ 2 as

where the last inequality of the above chain follows from (SC). The above chain together with (20) and (SC) implies the statement of the theorem.

Lemma C.2. Let K > 0 be a positive integer and η 1 , η 2 , . . . , η K be random vectors such that E k [ η k ] def = E [ η k | η 1 , . . . , η k -1 ] = 0 for k = 2 , . . . , K . Then

Proof. We start with the following derivation:

Applying similar steps to E [ ∥ ∥ ∥ ∑ K -1 k =1 η k ∥ ∥ ∥ 2 ] , E [ ∥ ∥ ∥ ∑ K -2 k =1 η k ∥ ∥ ∥ 2 ] , . . . , E [ ∥ ∥ ∥ ∑ 2 k =1 η k ∥ ∥ ∥ 2 ] , we get the result.

Lemma C.3. Suppose

holds for γ ≤ γ 0 . Then the choise of and

implies that r K ≤ ε for

Proof. Since b ≥ 3 c 0 ε then c 0 b ≤ ε 3 and c 1 γ b ≤ c 1 γε 3 c 0 . The choise of γ ≤ min ( γ 0 , c 0 c 1 ) implies that c 1 γ b ≤ ε 3 .

The choice of K ≥ 1 a max ( c 1 c 0 , 1 γ 0 ) ln 3 r 0 ε implies that r 0 (1 -aγ ) K ≤ ε 3 and finishes the proof.

Lemma C.4 (see also Lemma 2 from Stich [2019] and Lemma D.2 from Gorbunov et al. [2020]) . Let { r k } k ≥ 0 satisfy

for all K ≥ 0 with some constants a, c 2 > 0 , c 1 ≥ 0 , γ ≤ γ 0 .

Then for we have that

Moreover r K ≤ ε after iterations.

Proof. We have

Next we consider two possible situations.

If γ 0 ≥ ln ( max { 2 , min { ar 0 K / c 1 , a 2 r 0 K 2 / c 2 }} ) a ( K +1) then we choose γ = ln ( max { 2 , min { ar 0 K / c 1 , a 2 r 0 K 2 / c 2 }} ) a ( K +1) and get that

If γ 0 ≤ ln ( max { 2 , min { aK / c 1 , a 2 r 0 K 2 / c 2 }} ) a ( K +1) then we choose γ = γ 0 which implies that

Combining the obtained bounds we get the result.

D Methods that use robust aggregators

First of we provide the result of Karimireddy et al. [2022] that describes error of RAGG, where m t = α g t +(1 -α ) m t -1 .

Lemma D.1 (Aggregation error Karimireddy et al. [2022]) . Given that RAGG satisfies 1.1 holds, the error between the ideal average momentum m t and the output of the robust aggregation rule m t for any t ≥ 1 can be bounded as

where we define for t ≥ 1

For t = 0 we can simplify the bound as ( ρ 0 ) 2 := 24 σ 2 +12 ζ 2 .

Moreover, one can state a uniform bound for ( ρ t ) 2

D.1 Proofs for SGDA-RA

Algorithm 1 SGDA-RA

Input: RAGG, γ

1: for t = 0 , ... do

2: for worker i ∈ [ n ] in parallel

3: g t i ← g i ( x t , ξ i )

4: send g t i if i ∈ G , else send ∗ if Byzantine

5: ̂ g t = RAGG ( g t 1 , . . . , g t n ) and x t +1 ← x t -γ ̂ g t . // update params using robust aggregate

D.1.1 Quasi-Strongly Monotone Case

Theorem (Theorem 1 duplicate) . Let Assumptions 1, 2, 4 and 6 hold. Then after T iterations SGDA-RA (Algorithm 1) with ( δ, c ) -RAGG and γ ≤ 1 2 ℓ outputs x T such that

where ρ 2 = 24 σ 2 +12 ζ 2 by Lemma D.1 with α = 1 .

Proof of Theorem 1. We start the proof with

Since ̂ g t = ̂ g t -F t + F t one has

Applying (11) for ⟨ ̂ g t -g t , x t -x ∗ ⟩ with λ = γµ 2 and (12) for ∥ ̂ g t ∥ 2 = ∥ ∥ ̂ g t -g t + g t ∥ ∥ 2 we derive

Next by taking an expectation E ξ of both sides of the above inequality and rearranging terms obtain

Next we use Lemmas C.1 and D.1 to derive

that together with the choice of γ ≤ 1 2 ℓ and Assumption (QSM) allows to obtain

Next we take full expectation of both sides and obtain

The latter implies

where ρ is bounded by Lemma D.1 with α = 1 .

Corollary 1. Let assumptions of Theorem 1 hold. Then E ∥ ∥ x T -x ∗ ∥ ∥ 2 ≤ ε holds after

iterations of SGDA-RA with γ = min ( 1 2 ℓ , 1 2 µ + µ 6 cδG ) and b ≥ 72 cδσ 2 µ 2 ε .

Proof. If ζ = 0 , ρ 2 = 24 σ 2 the result of Theorem 1 can be simplified as

Applying Lemma C.3 to the last bound we get the result of the corollary.

D.2 Proofs for SEG-RA

To analyze the convergence of SEG introduce the following notation

D.2.1 Auxilary results

Lemma D.2. Let Assumptions 2, 3, 4 and Corollary 2 hold. If

for SEG-RA (Algorithm 2), then g η k ( ˜ x k ) = g η k ( x k -γ 1 ̂ g ξ k ( x k ) ) satisfies the following inequality

where ̂ P k = γ 1 E ξ k , η k [〈 g η k ( ˜ x k ) , x k -x ∗ 〉] and ρ 2 = 24 σ 2 +12 ζ 2 by Lemma D.1 with α = 1 .

Proof. Using the auxiliary iterate ̂ x k +1 = x k -γ 1 g η k ( ˜ x k ) , we get

Taking the expectation E ξ k , η k [ · ] = E [ · | x k ] conditioned on x k from the above identity, using tower property E ξ k , η k [ · ] = E ξ k [ E η k [ · ]] , and µ -quasi strong monotonicity of F ( x ) , we derive

where ξ k i , i ∈ G and η k j , j ∈ G are i.i.d. samples satisfying Assumption 1, i.e., due to the independence we have

Corollary 2. Suppose that the operator F is given in the form (1) and Assumption 1 holds. Then

To upper bound the last term we use simple inequality (16), and apply L -Lipschitzness of F ( x ) :

Finally, we use the above inequality together with (31):

where ̂ P k = γ 1 E ξ k , η k [〈 g η k ( ˜ x k ) , x k -x ∗ 〉] . Rearranging the terms, we obtain (30).

Lemma D.3. Consider SEG-RA (Algorithm 2). Let Assumptions 2, 3, 4 and Corollary 2 hold. If

then g η k ( ˜ x k ) = g η k ( x k -γ 1 ̂ g ξ k ( x k ) ) satisfies the following inequality

or simply

where ̂ P k = γ 1 E ξ k , η k [〈 g η k ( ˜ x k ) , x k -x ∗ 〉] and ρ 2 = 24 σ 2 +12 ζ 2 by Lemma D.1 with α = 1 .

Proof. Since E ξ k , η k [ · ] = E [ · | x k ] and g η k ( ˜ x k ) = g η k ( x k -γ 1 ̂ g ξ k ( x k ) ) , we have

So one have

or simply

that concludes the proof.

D.2.2 Quasi-Strongly Monotone Case

Combining Lemmas D.2 and D.3, we get the following result.

Theorem (Theorem 2 duplicate) . Let Assumptions 1, 2, 3 and 4 hold. Then after T iterations SEG-RA (Algorithm 2) with ( δ, c ) -RAGG , γ 1 ≤ 1 2 µ +2 L and β = γ 2 / γ 1 ≤ 1 / 4 outputs x T such that

where ρ 2 = 24 σ 2 +12 ζ 2 by Lemma D.1 with α = 1 .

Proof of Theorem 2. Since x k +1 = x k -γ 2 ̂ g η k ( ˜ x k ) , we have

Taking the expectation, conditioned on x k ,

using the definition of P = γ E k k [〈 g ( x k ) , x k -x ∗ 〉] , we continue our derivation:

Next, we take the full expectation from the both sides

Unrolling the recurrence, together with the bound on ρ given by Lemma D.1 we derive the result of the theorem:

Corollary 3. Let assumptions of Theorem 2 hold. Then E ∥ ∥ x T -x ∗ ∥ ∥ 2 ≤ ε holds after

iterations of SEG-RA with γ 1 = min ( 1 2 µ +2 L , 1 2 µ + µ 12 cδG ) and b ≥ 288 cδσ 2 βµ 2 ε .

Proof. Next, we plug γ 2 = βγ 1 ≤ γ 1 / 4 into the result of Theorem 2 and obtain

If ζ = 0 , ρ 2 = 24 σ 2 the last reccurence can be unrolled as

Applying Lemma C.3 to the last bound we get the result of the corollary.

D.3 Proofs for M-SGDA-RA

Algorithm 3 M-SGDA-RA

Input:

RAGG,

γ

,

α

∈

[0

,

1]

1: for t = 0 , ... do

2: for worker i ∈ [ n ] in parallel

3:

g

i i i i 4: send m t i if i ∈ G , else send ∗ if Byzantine

g

(

x

,

ξ

i

)

and

m

(1

-

α

)

m

+

α

g

t

i

5: ̂ m t = RAGG ( m t 1 , . . . , m t n ) and x t +1 ← x t -γ ̂ m t aggregate

D.3.1 Quasi-Strongly Monotone Case

Theorem (Theorem 3 duplicate) . Let Assumptions 1, 2, 4, and 6 hold. Then after T iterations M-SGDA-RA (Algorithm 3) with ( δ, c ) -RAGG outputs x T such that

// worker momentum

. // update params using robust

t

t

t

t

-

1

Next, unrolling the following recursion

one obtains

Applying the latter and (11) for ⟨ ̂ m t -m t , x t -x ∗ ⟩ with λ = µγα 2 we obtain

Next by taking an expectation E ξ of both sides of the above inequality and rearranging terms obtain

Next we use Lemma C.1 to bound E ξ ∥ ∥ g j ∥ ∥ 2 and Assumption (QSM) to obtain the following bound

Gathering the above results we have

Next we take full expectation of both sides and obtain

Introducing the following notation

and using that E ∥ ∥ ̂ m t -m t ∥ ∥ 2 ≤ cδρ 2 , where ρ is given by (26) we have

Next we sum the above inequality

If γ ≤ 1 µ we have

So we have

If γ ≤ α 12 ℓ then the following is true

Using the notations for Z t and (QSM) we have

and consequently by Jensen's inequality

With the definition ̂ x t = α 1 -(1 -α ) t +1 ∑ t j =0 (1 -α ) t -j x j then the above implies that Z t ≥ µ E ∥ ∥ ̂ x t -x ∗ ∥ ∥ , that together with (40) gives

Applying the Jensen inequality again with x T = 1 W T ∑ T t =0 w t ̂ x t we derive the final result

together with the bound on ρ given by Lemma D.1.

Corollary 4. Let assumptions of Theorem 3 hold. Then E ∥ ∥ x T -x ∗ ∥ ∥ 2 ≤ ε holds after

iterations of M-SGDA-RA with γ = min ( α 12 ℓ , 1 2 µ + µ 16 cδG ) .

Proof. If ζ = 0 , ρ 2 = 24 σ 2 the result of Theorem 3 can be simplified as

Since 2 µγαW T ≤ ( 1 -µγα 2 ) T +1 we can apply Lemma C.3 and get the result of the corollary.

E Methods with random check of computations

We replace ( δ max , c ) -RAGG with the simple mean, but introduce additional verification that have to be passed to accept the mean. The advantage of such aggregation that it coincides with "good" mean if there are no peers violating the protocol But if there is at least one peer violating the protocol at iteration t we can bound the variance similar to Lemma D.1.

Algorithm 4 CheckComputations

Input: t , G ∪ B , C , Banned =

1: C t +1 = { c t +1 1 , . . . , c t +1 m } , C t +1 ⊂ ( G t ∪ B t ) \ C t and U t +1 = { u t +1 1 , . . . , u t +1 m } , U t +1 ⊂ ( G t ∪ B t ) \ C t , where 2 m workers c t +1 1 , . . . , c t +1 m , u t +1 1 , . . . , u t +1 m are choisen uniformly at

2: for i = 1 , ..., m in parallel c checks computations of u during the next iteration

̸

t t t t ∅ random without replacement. t +1 i t +1 i 3: c t +1 i receives a query to recalculate g ( x t , ξ t u t +1 i ) 4: c t +1 i sends the recalculated g ( x t , ξ t u t +1 i ) 5: for i = 1 , ..., m do 6: if g ( x t , ξ t u t i ) = g t u t i then 7: Banned t = Banned t ∪ { u t i , c t i } . 8: end if Output: C t +1 , G t ∪ B t \ Banned t

Lemma E.1. Let Assumption 2 is satisfied with ζ = 0 . Then the error between the ideal average g t and the average with the recomputation rule g t can be bounded as

Proof of Lemma E.1. Denote the set ˜ G in the following way ˜ G = { i ∈ G t \ C t : ∥ ̂ g t -g t i ∥ ≤ Cσ } .

So that we have

If δ ≤ 1 / 4 then an acceptance of ̂ g t implies that | ˜ G | > n / 4 and | ˜ G | > | G t \ C t | / 3 .

Bringing the above results together gives that

Since checks of computations are only possible in homogeneous case ( ζ = 0 ) then E ∥ g t i -F t ∥ 2 = σ 2 and

Since | G t \ C t | > n -2 B -m

E.1 Proofs for SGDA-CC

Algorithm 5 SGDA-CC

E.1.1 Star Co-coercieve Case

Next we provide convergence guarantees for SGDA-CC

(Algorithm 5) under Assumption 6.

Theorem 9. Let Assumptions 1 and 6 hold. Next, assume that

where ρ 2 = qσ 2 with q = 2 C 2 +12+ 12 n -2 B -m and C = O (1) by Lemma E.1 and R ≥ ∥ x 0 -x ∗ ∥ . Then after K iterations of SGDA-CC (Algorithm 5) it outputs x T such that

Proof. Since | G t \ C t | ≥ n -2 B -m one can derive using the results of Lemmas C.1 and E.1 that

where ✶ k is an indicator function of the event that at least 1 Byzantine peer violates the protocol at iteration k .

To estimate the inner product in the right-hand side we apply Cauchy-Schwarz inequality and then Lemma E.1

Since γ ≤ 1 2 ℓ the above results imlies

Taking the full expectation from the both sides of the above inequality and summing up the results for k = 0 , 1 , . . . , K -1 we derive

Since F satisfies Assumption 6, K -1 ∑ k =0 E [ ⟨ F ( x k ) , x k -x ∗ ⟩ ] ≥ 0 . Using this and new notation R k = ∥ x k -x ∗ ∥ , k > 0 , R 0 ≥ ∥ x 0 -x ∗ ∥ we get implying (after changing the indices) that

for all k = 0 , . . . , K . For k = 0 this inequality trivially holds. Next, assume that it holds for all k = 0 , 1 , . . . , T -1 , T ≤ K -1 . Let us show that it holds for k = T as well. From (46) and (47) we have that E [ R 2 k ] ≤ 2 R 2 0 for all k = 0 , 1 , . . . , T -1 . Therefore,

If a Byzantine peer deviates from the protocol at iteration k , it will be detected with some probability p k during the next iteration. One can lower bound this probability as

Therefore, each individual Byzantine worker can violate the protocol no more than 1 / p times on average implying that

Taking

we ensure that

and, as a result, we get

. Therefore, (47) holds for all k = 0 , 1 , . . . , K . Together with (45) it implies

The last inequality together with Assumption 6 implies

Corollary 5. Let assumptions of Theorem 9 hold. Then 1 K K -1 ∑ k =0 E ∥ ∥ F ( x k ) ∥ ∥ ≤ ε holds after

iterations of SGDA-CC .

Proof.

Let us chose K such that each of the last three terms less or equal ε / 3 , then

where ρ 2 = qσ 2 with q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) by Lemma E.1. The latter implies that

Using the definition of ρ from Lemma E.1 and if B ≤ n 4 , m << n the bound for K can be easily derived.

E.1.2 Quasi-Strongly Monotone Case

Theorem (Theorem 4 duplicate) . Let Assumptions 1, 4 and 6 hold. Then after T iterations SGDA-CC (Algorithm 5) with γ ≤ 1 2 ℓ outputs x T such that

where ρ 2 = qσ 2 with q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) by Lemma E.1.

Proof of Theorem 4. The proof is similar to the proof of Theorem 1

Since ̂ g t = ̂ g t -F t + F t one has

Applying (11) for ⟨ g t , x t -x ∗ ⟩ with λ = γµ 2 and (12) for ∥ ̂ g t ∥ 2 = ∥ ∥ ̂ g t -g t + g t ∥ ∥ 2 we derive

Next by taking an expectation E ξ of both sides of the above inequality and rearranging terms obtain

The difference with the proof of Theorem 1 is that we suppose that the number of peer violating the protocol at an iteration t is known to any "good" peer. So the result of Lemma E.1 writes as follows

✶ where ✶ t is an indicator function of the event that at least 1 Byzantine peer violates the protocol at iteration t . Together with Lemma C.1 we can proceed as follows m

Since γ ≤ 1 2 ℓ and Assumption (QSM) holds we derive

Next we take full expectation of both sides and obtain

The latter implies

If a Byzantine peer deviates from the protocol at iteration t , it will be detected with some probability p t during the next iteration. One can lower bound this probability as

Therefore, each individual Byzantine worker can violate the protocol no more than 1 / p times on average implying that that implies

The latter together with the above bound on E ∥ ∥ x T +1 -x ∗ ∥ ∥ 2 implies the result of the theorem.

Corollary 6. Let assumptions of Theorem 4 hold. Then E ∥ ∥ x T -x ∗ ∥ ∥ 2 ≤ ε holds after

iterations of SGDA-CC (Alg. 5) with

Proof. Using the definition of ρ ( ρ 2 = qσ 2 = O ( σ 2 ) ) from Lemma E.1 and if B ≤ n 4 , m<<n the result of Theorem 4 can be simplified as

Applying Lemma C.4 to the last bound we get the result of the corollary.

E.1.3 Monotone Case

Theorem 10. Suppose the assumptions of Theorem 9 and Assumption 5 hold. Next, assume that

where ρ 2 = qσ 2 with q = 2 C 2 +12+ 12 n -2 B -m and C = O (1) by Lemma E.1 and R ≥ ∥ x 0 -x ∗ ∥ . Then after K iterations of SGDA-CC (Algorithm 5)

where Gap B R ( x ∗ ) ( x K ) = max u ∈ B R ( x ∗ ) 〈 F ( u ) , x K -u 〉 , x K = 1 K K -1 ∑ k =0 x k and R ≥ ∥ x 0 -x ∗ ∥ .

Proof. Combining (16), (14) one can derive

Assumption 5 implies that

and consequently by Jensen inequality

Then maximization in u gives

Taking the full expectation from the both sides of the previous inequality gives

Firstly obtain the bound for the terms that do not depend on u using Assumption 1, Lemma E.1 and Theorem 9

Since E [∥ ∥ x k -u ∥ ∥ ] ≤ E [∥ ∥ x k -x ∗ ∥ ∥ ] + ∥ x ∗ -u ∥ ≤ E [∥ ∥ x k -x ∗ ∥ ∥ ] + max u ∈ B R ( x ∗ ) ∥ x ∗ -u ∥ (48) ≤ 3 R one can derive that

Following Beznosikov et al. [2023] one can derive he bound for the next term:

we have

We notice that E [ F k -g k | F 0 -g 0 , . . . , F k -1 -g k -1 ] = 0 for all k ≥ 1 , i.e., conditions of Lemma C.2 are satisfied. Therefore, applying Lemma C.2, we get

Assembling the above results together gives

Corollary 7. Let assumptions of Theorem 10 hold. Then E [ Gap B R ( x ∗ ) ( x K ) ] ≤ ε holds after

iterations of SGDA-CC .

Proof.

Let us chose K such that each of the last three terms less or equal ε / 3 , then

guarantees that

Using the definition of ρ from Lemma E.1 and if B ≤ n 4 , m << n the bound for K can be easily derived.

E.2 Proofs for R-SGDA-CC

E.2.1 Quasi-Strongly Monotone Case

Algorithm 6 R-SGDA-CC

Input: x 0 - starting point, r - number of restarts, { γ t } r t =1 - stepsizes for SGDA-CC (Alg. 5), { K t } r t =1 - number of iterations for SGDA-CC (Alg. 5) 1: x 0 = x 0

̂ 2: for t = 1 , 2 , . . . , r do

3: Run SGDA-CC (Alg. 5) for K t iterations with stepsize γ t , starting point ̂ x t -1 .

4: Define ̂ x t as ̂ x t = 1 K t K t ∑ k =0 x k,t , where x 0 ,t , x 1 ,t , . . . , x K t ,t are the iterates produced by SGDA-CC (Alg. 5).

5: end for

Output:

̂ x r

Theorem (Theorem 5 duplicate) . Let Assumptions 1, 4 and 6 hold. Then, after r = ⌈ log 2 R 2 ε ⌉ -1 restarts R-SGDA-CC (Algorithm 6) with γ t = min { 1 2 ℓ , √ ( n -2 B -m ) R 2 6 σ 2 2 t K t , √ m 2 R 2 72 qσ 2 2 t B 2 n 2 } and K t = ⌈ max { 8 ℓ µ , 96 σ 2 2 t ( n -2 B -m ) µ 2 R 2 , 34 nσB √ q 2 t mµR }⌉ , where R ≥ ∥ x 0 -x ∗ ∥ , outputs ̂ x r such that E ∥ ̂ x r -x ∗ ∥ 2 ≤ ε . Moreover, the total number of executed iterations of SGDA-CC is

With q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) by Lemma E.1.

Proof of Theorem 5. x K = 1 K ∑ K -1 k =0 x k

Theorem 9 implies that SGDA-CC with

after K iterations.

After the first restart we have guarantees

iterations of SGDA-CC .

Proof. Using the definition of ρ from Lemma E.1 and if B ≤ n 4 , m<<n the bound for r ∑ t =1 K t can be easily derived.

Next, assume that we have E [ ∥ ̂ x t -x ∗ ∥ 2 ] ≤ R 2 0 2 t for some t ≤ r -1 . Then, Theorem 9 implies that

Taking the full expectation from the both sides of previous inequality we get

Therefore, by mathematical induction we have that for all t = 1 , . . . , r

Then, after r = ⌈ log 2 R 2 0 ε ⌉ -1 restarts of SGDA-CC we have E [ ∥ ̂ x r -x ∗ ∥ 2 ] ≤ ε . The total number of iterations executed by SGDA-CC is

Corollary 8. Let assumptions of 5 hold. Then E [ ∥ ̂ x r -x ∗ ∥ 2 ] ≤ ε holds after

E.3 Proofs for SEG-CC

Algorithm 7 SEG-CC

Input:

RAGG,

γ

1: for t = 1 , ... do

2: for worker i ∈ [ n ] in parallel

3: g t ξ i ← g i ( x t , ξ i )

4: send g t ξ i if i ∈ G t , else send ∗ if Byzantine

8: else

˜ ̂

9: recompute

10: end if

12: for worker i ∈ [ n ] in parallel

13: g t η i ← g i ( ˜ x t , η i )

14:

send

g

η

t

i

if

i

∈

G

, else send

∗

if Byzantine

18: else

19: recompute

20: end if

21: C t +1 , G t +1 ∪ B t +1 = CheckComputations ( C t + 1 2 , G t + 1 2 ∪ B t + 1 2 )

E.3.1 Auxilary results

Similarly to Section E.1 we state the following. If a Byzantine peer deviates from the protocol at iteration k , it will be detected with some probability p k during the next iteration. One can lower bound this probability as

Therefore, each individual Byzantine worker can violate the protocol no more than 1 / p times on average implying that

Lemma E.2. Let Assumption 3 holds. Let Algorithm 7 is run with γ 1 ≤ 1 / 2 L and β = γ 2 / γ 1 ≤ 1 / 2 . Then its iterations satisfy

Proof. Since x k +1 = x k -γ 2 ̂ g η k ( ˜ x k ) , we have

Rearranging the terms gives that

Next we use (14)

and obtain the following

If β = γ 2 / γ 1 ≤ 1 / 2

Combining the latter with the result of the following chain

we obtain if γ 1 ≤ 1 / 2 L

E.3.2 Lipschitz Case

Theorem 11. Let Assumptions 1, 3, 5 hold. And let

where ρ 2 = qσ 2 with q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) by Lemma E.1. Then iterations of SEG-CC (Algorithm 7) satisfy for k ≥ 1

where R k = ∥ x k -x ∗ ∥ and R ≥ ∥ x 0 -x ∗ ∥ .

Proof. Substituting u = x ∗ into the result of Lemma E.2 and taking expectation over η k one obtains

To estimate the inner product in the right-hand side we apply Cauchy-Schwarz inequality:

Then Assumption 6 implies

and

Taking expectation over ξ k one obtains that

Assumption 5 implies that 0 ≤ 〈 F ( x ∗ ) , ˜ x k -x ∗ 〉 ≤ 〈 F ( ˜ x k ) , ˜ x k -x ∗ 〉 . Using this and a the notation R k = ∥ x k -x ∗ ∥ , k > 0 , R 0 ≥ ∥ x 0 -x ∗ ∥ we get

0 ≤ R 2 0 -E [ R 2 K ] K + 8 γ 1 γ 2 σ 2 n -2 B -m + 2 γ 2 ρ K K -1 ∑ k =0 √ E [ ∥ x k -x ∗ ∥ 2 ] E [ ✶ k ] + 2 γ 2 2 ρ 2 K K -1 ∑ k =0 E [ ✶ k ] + 4 γ 1 γ 2 ρ 2 K K -1 ∑ k =0 E [ ✶ k -1 2 ] . (62) implying (after changing the indices) that

holds for all k ≥ 0 . In the remaining part of the proof we derive by induction that

for all k = 0 , . . . , K . For k = 0 this inequality trivially holds. Next, assume that it holds for all k = 0 , 1 , . . . , T -1 , T ≤ K -1 . Let us show that it holds for k = T as well. From (64) and (66) we have that E [ R 2 k ] ≤ 2 R 2 0 for all k = 0 , 1 , . . . , T -1 . Therefore,

The latter together with the expected number of at least one peer violations (57) implies

Taking we satisfy conditions of Lemma E.2 and ensure that

8 γ 1 γ 2 σ 2 k n -2 B -m +2 γ 2 ρR 0 nB m +2 γ 2 2 ρ 2 nB m +4 γ 1 γ 2 ρ 2 nB m ≤ R 2 0 4 + R 2 0 4 + R 2 0 4 + R 2 0 4 = R 2 0 , and, as a result, we get E [ R 2 T ] ≤ 2 R 2 0 ≡ 2 R. (67)

Therefore, (66) holds for all k = 0 , 1 , . . . , K . Together with (62) it implies

E.3.3 Quasi-Strongly Monotone Case

Lemma E.3. Let Assumptions 3, 4 and Corollary 2 hold. If

then g η k ( ˜ x k ) = g η k ( x k -γ 1 ̂ g ξ k ( x k ) ) satisfies the following inequality

where ̂ P k = γ 1 E ξ k , η k [〈 g η k ( ˜ x k ) , x k -x ∗ 〉] and ρ 2 = qσ 2 with q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) by Lemma E.1.

Proof. Using the auxiliary iterate ̂ x k +1 = x k -γ 1 g η k ( ˜ x k ) , we get

Taking the expectation E ξ k , η k [ · ] = E [ · | x k ] conditioned on x k from the above identity, using tower property E ξ k , η k [ · ] = E ξ k [ E η k [ · ]] , and µ -quasi strong monotonicity of F ( x ) , we derive

To upper bound the last term we use simple inequality (16), and apply L -Lipschitzness of F ( x ) :

Finally, we use the above inequality together with (70):

where ̂ P k = γ 1 E ξ k , η k [〈 g η k ( ˜ x k ) , x k -x ∗ 〉] . Rearranging the terms, we obtain (69).

Lemma E.4. Let Assumptions 3, 4 and Corollary 2 hold. If

then g η k ( ˜ x k ) = g η k ( x k -γ 1 ̂ g ξ k ( x k ) ) satisfies the following inequality

or simply where ̂ P k = γ 1 E ξ k , η k [〈 g η k ( ˜ x k ) , x k -x ∗ 〉] , where ρ 2 = qσ 2 with q = 2 C 2 +12+ 12 n -2 B -m and C = O (1) by Lemma E.1.

or simply

that concludes the proof.

Combining Lemmas E.3 and E.4, we get the following result.

Theorem (Theorem 6 duplicate) . Let Assumptions 1, 3 and 4 hold. Then after T iterations SEG-CC (Algorithm 7) with γ 1 ≤ 1 2 µ +2 L and β = γ 2 / γ 1 ≤ 1 / 4 outputs x T such that

where q = 2 C 2 +12 + 12 n -2 B -m ; q = O (1) since C = O (1) .

Proof of Theorem 6. Since x k +1 = x k -γ 2 ̂ g η k ( ˜ x k ) , we have

Taking the expectation, conditioned on x k ,

using the definition of ̂ P k = γ 1 E ξ k , η k [〈 g η k ( ˜ x k ) , x k -x ∗ 〉] , we continue our derivation:

.

Next, taking the full expectation from the both sides and obtain

Since γ 2 ≤ 4 µ and that implies using the expected number of at least one peer violations (57) we derive

that together with ρ 2 = qσ 2 with q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) by Lemma E.1 give result of the theorem.

Corollary 9. Let assumptions of Theorem 6 hold. Then E ∥ ∥ x T -x ∗ ∥ ∥ 2 ≤ ε holds after

iterations of SEG-CC with

Proof. Next, we plug γ 2 = βγ 1 ≤ γ 1 / 4 into the result of Theorem 6 and obtain

Using the definition of ρ ( ρ 2 = qσ 2 = O ( σ 2 ) ) from Lemma E.1 and if B ≤ n 4 , m<<n the result of Theorem 6 can be simplified as

Applying Lemma C.4 to the last bound we get the result of the corollary.

E.3.4 Lipschitz Monotone Case

Theorem 12. Suppose the assumptions of Theorem 11 and Assumption 5 hold. Then after K iterations of SEG-CC (Algorithm 7)

Proof. We start the proof with the result of Lemma E.2

that leads to the following inequality

Assumption 5 implies that

and consequently by Jensen inequality

Then maximization in u gives

and

Now the last term

Following Beznosikov et al. [2023] one can derive the bound for the next term:

we have

We notice that E [ F ( ˜ x k ) -g η k ( ˜ x k ) | F ( ˜ x 0 ) -g η 0 ( ˜ x 0 ) , . . . , F ( ˜ x k -1 ) -g η k -1 ( ˜ x k -1 ) ] = 0 for all k ≥ 1 , i.e., conditions of Lemma C.2 are satisfied. Therefore, applying Lemma C.2, we get

Assembling the above results together gives

Corollary 10. Let assumptions of Theorem 12 hold. Then E [ Gap B R ( x ∗ ) ( x K ) ] ≤ ε holds after

iterations of SEG-CC .

Proof.

Let us chose K such that each of the last three terms less or equal ε / 3 , then

where ρ 2 = qσ 2 with q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) by Lemma E.1. The latter implies that

Using the definition of ρ from Lemma E.1 and if B ≤ n 4 , m << n the bound for K can be easily derived.

E.4 Proofs for R-SEG-CC

E.4.1 Quasi Strongly Monotone Case

Algorithm 8 R-SEG-CC

Input: x 0 - starting point, r - number of restarts, { γ t } r t =1 - stepsizes for SEG-CC (Alg. 7), { K t } r t =1 - number of iterations for SEG-CC (Alg. 7), 1: x 0 = x 0

̂ 2: for t = 1 , 2 , . . . , r do

3: Run SEG-CC (Alg. 7) for K t iterations with stepsize γ t , starting point ̂ x t -1 ,

4: Define ̂ x t as ̂ x t = 1 K t K t ∑ k =0 x k,t , where x 0 ,t , x 1 ,t , . . . , x K t ,t are the iterates produced by SEG-CC .

5: end for

Output:

̂ x

r

Theorem (Theorem 7 duplicate) . Let Assumptions 1, 3, 4 hold. Then, after r = ⌈ log 2 R 2 ε ⌉ -1 restarts R-SEG-CC (Algotithm 8) with γ 1 t = min { 1 2 L , √ ( G -B -m ) R 2 16 σ 2 2 t K t , √ mR 2 8 qσ 2 2 t Bn } , γ 2 t = min { 1 4 L , √ m 2 R 2 64 qσ 2 2 t B 2 n 2 , √ ( G -B -m ) R 2 64 σ 2 K t } and K t = ⌈ max { 8 L µ , 16 nσB √ q 2 t mµR , 256 σ 2 2 t ( G -B -m ) µ 2 R 2 }⌉ , where R ≥ ∥ x 0 -x ∗ ∥ outputs ̂ x r such that E ∥ ̂ x r -x ∗ ∥ 2 ≤ ε . Moreover, the total number of executed iterations of SEG-CC is

Proof of Theorem 7. x K = 1 K K -1 ∑ k =0 ˜ x k

Theorem 11 implies that SEG-CC with

guarantees iterations of SEG-CC .

Proof. Using the definition of ρ from Lemma E.1 and if B ≤ n 4 , m<<n the bound for r ∑ t =1 K t can be easily derived.

after K iterations.

After the first restart we have since µγ 2 1 K 1 ≥ 2 .

Next, assume that we have E [ ∥ ̂ x t -x ∗ ∥ 2 ] ≤ R 2 0 2 t for some t ≤ r -1 . Then, Theorem 11 implies that

.

Taking the full expectation from the both sides of previous inequality we get

Therefore, by mathematical induction we have that for all t = 1 , . . . , r

Then, after r = ⌈ log 2 R 2 0 ε ⌉ -1 restarts of SEG-CC we have E [ ∥ ̂ x r -x ∗ ∥ 2 ] ≤ ε . The total number of iterations executed by SEG-CC is

that together with ρ 2 = qσ 2 with q = 2 C 2 +12 + 12 n -2 B -m and C = O (1) given by Lemma E.1 implies the result of the theorem.

Corollary 11. Let assumptions of 7 hold. Then E [ ∥ ̂ x r -x ∗ ∥ 2 ] ≤ ε holds after

F Extra Experiments and Experimental details

F.1 Quadratic games

Firstly, let us clarify notations made in (7):

with symmetric matrices A j,i s.t. µ I ≼ A j,i ≼ ℓ I , A i ∈ R d × d and b i ∈ R d .

Data generation. For each j we sample real random matrix B i with elements sampled from a normal distribution. Then we compute the eigendecomposition and obtain the following B i = U T i D i U i with diagonal D i . Next, we scale D i and obtain ̂ D i with elements lying in [ µ, ℓ ] . Finally we compose A j,i as A j,i = U T i ̂ D i U i . This process ensures that eigenvalues of A j,i all lie between µ and ℓ , and thus F ( x ) is strongly monotone and cocoercive. Vectors b i ∈ R d are sampled from a normal distribution with variance 10 / d .

Experimental setup. For all the experiments we choose ℓ = 100 , µ = 0 . 1 , s = 1000 and d = 50 .

For the experiments presented in the Appendix we simulate n = 20 nodes on a single machine and B = 4 . For methods with checks of computations the only one peer attacks per iteration.

We used RFA with 5 buckets bucketing as an aggregator since it showed the best performance. For approximating the median we used Weiszfeld's method with 10 iterations and parameter ν = 0 . 1 Pillutla et al. [2022].

RDEG [Adibi et al., 2022] provably works only if n ≥ 100 we manually selected parameter ϵ = 0 . 5 using a grid-search and picking the best performing value.

We present experiments with different attack (bit flipping (BF), random noise (RN), inner product manipulation (IPM) Xie et al. [2019] and 'a little is enough' (ALIE) Baruch et al. [2019]) and different batchsizes (bs) 1, 10 and 100. If an attack has a parameter it is specified in the brackets on each plot.

No checks. Firstly we provide a detailed comparison between methods that do not check computation with fixed learning rate value γ = 3 . 3 e -5 . Code for quadratic games is available at https://github.com/nazya/sgda-ra 7 .

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

ALIE (1e-01), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

ALIE (1e-01), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e-01), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

2

10

1

Distance to the solution

ALIE (1e+00), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

ALIE (1e+00), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+00), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

ALIE (1e+02), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

ALIE (1e+02), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

3

10

2

10

1

Distance to the solution

ALIE (1e+02), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

BF, bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

BF, bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

BF, bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

IPM (1e-01), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e-01), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e-01), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

IPM (1e+00), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+00), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+00), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

2

10

1

Distance to the solution

IPM (1e+01), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

IPM (1e+01), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

3

10

2

10

1

Distance to the solution

IPM (1e+01), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

2

10

1

Distance to the solution

IPM (1e+02), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

IPM (1e+02), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

3

10

2

10

1

Distance to the solution

IPM (1e+02), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

RN (1e+01), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

SEGRA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

RN (1e+01), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+01), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

RN (1e+02), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

RN (1e+02), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+02), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

RN (1e+03), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

RN (1e+03), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+03), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

3

10

2

10

1

Distance to the solution

RN (1e+04), bs=1

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

RN (1e+04), bs=10

RDEG

SEGRA

MSGDARA

SGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+04), bs=100

RDEG

SEGRA

MSGDARA

SGDARA

Checks of Computations. Next, using the same setup, we compare M-SGDA-RA , which showed the best performance in the above experiments, with methods that check computations. With checks of computation the best strategy for attackers is that at each iteration only one peer attacks, since it maximizes the expected number of rounds with the presence of actively malicious peers. So the comparison in this paragraph is performed in this setup.

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e-01), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e-01), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

ALIE (1e-01), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+00), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+00), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

ALIE (1e+00), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=1

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

ALIE (1e+01), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+02), bs=1

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

ALIE (1e+02), bs=10

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

ALIE (1e+02), bs=100

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

BF, bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

BF, bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

BF, bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e-01), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e-01), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

IPM (1e-01), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+00), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

5

10

3

10

1

Distance to the solution

IPM (1e+00), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

IPM (1e+00), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+01), bs=1

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+01), bs=10

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

IPM (1e+01), bs=100

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+02), bs=1

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

7

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

IPM (1e+02), bs=10

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

IPM (1e+02), bs=100

SEGCC

SGDACC

MSGDARA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+01), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+01), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

RN (1e+01), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+02), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+02), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

RN (1e+02), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+03), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+03), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

RN (1e+03), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e5

10

6

10

5

10

4

10

3

10

2

10

1

10 0

Distance to the solution

RN (1e+04), bs=1

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e6

10

6

10

5

10

4

10

3

10

2

10

1

10 0

Distance to the solution

RN (1e+04), bs=10

SEGCC

SGDACC

MSGDARA

SGDA

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Number of gradients

1e7

10

7

10

5

10

3

10

1

Distance to the solution

RN (1e+04), bs=100

SEGCC

SGDACC

MSGDARA

SGDA

.

Learning rates. We conducted extra experiments to show the dependence on different learning rate values 1 e -5 , 2 e -5 , 5 e -6 .

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

Number of gradients

1e6

10

3

10

2

10

1

Distance to the solution

ALIE (1e+01), bs=10

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

RDEG (1e-05)

SEGRA (1e-05)

MSGDARA (1e-05)

SGDARA (1e-05)

RDEG (5e-06)

SEGRA (5e-06)

MSGDARA (5e-06)

SGDARA (5e-06)

0

1

2

3

4

5

6

7

Number of gradients

1e7

10

3

10

2

10

1

Distance to the solution

IPM (1e+02), bs=100

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

RDEG (1e-05)

SEGRA (1e-05)

MSGDARA (1e-05)

SGDARA (1e-05)

RDEG (5e-06)

SEGRA (5e-06)

MSGDARA (5e-06)

SGDARA (5e-06)

0

50000

100000

150000

200000

250000

300000

Number of gradients

10

3

10

2

10

1

Distance to the solution

RN (1e+03), bs=1

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

RDEG (1e-05)

SEGRA (1e-05)

MSGDARA (1e-05)

SGDARA (1e-05)

RDEG (5e-06)

SEGRA (5e-06)

MSGDARA (5e-06)

SGDARA (5e-06)

0.0

0.2

0.4

0.6

0.8

1.0

Number of gradients

1e7

10

4

10

3

10

2

10

1

Distance to the solution

BF, bs=10

RDEG (2e-05)

SEGRA (2e-05)

MSGDARA (2e-05)

SGDARA (2e-05)

RDEG (1e-05)

SEGRA (1e-05)

MSGDARA (1e-05)

SGDARA (1e-05)

RDEG (5e-06)

SEGRA (5e-06)

MSGDARA (5e-06)

SGDARA (5e-06)

0.0

0.2

0.4

0.6

0.8

1.0

Number of gradients

1e7

10

5

10

4

10

3

10

2

10

1

Distance to the solution

Scaled BF, bs=10

SEGCC (2e-05)

SGDACC (2e-05)

MSGDARA (2e-05)

SEGCC (1e-05)

SGDACC (1e-05)

MSGDARA (1e-05)

SEGCC (5e-06)

SGDACC (5e-06)

MSGDARA (5e-06)

0

1

2

3

4

5

10

5

10

4

10

3

10

2

10

1

Distance to the solution

Smoothed ALIE (1e+01), bs=10

SEGCC (2e-05)

SGDACC (2e-05)

MSGDARA (2e-05)

SGDA (2e-05)

SEGCC (1e-05)

SGDACC (1e-05)

MSGDARA (1e-05)

SGDA (1e-05)

SEGCC (5e-06)

SGDACC (5e-06)

MSGDARA (5e-06)

SGDA (5e-06)

4.0

4.5

5.0

5.5

6.0

6.5

7.0

7.5

8.0

10

6

10

5

Distance to the solution

Scaled ALIE (1e+01), bs=10

SEGCC (2e-05)

SGDACC (2e-05)

SEGCC (1e-05)

SGDACC (1e-05)

SEGCC (5e-06)

SGDACC (5e-06)

0

1

2

3

4

5

10

6

10

5

10

4

10

3

10

2

10

1

Distance to the solution

Smoothed IPM (1e+02), bs=100

SEGCC (2e-05)

SGDACC (2e-05)

SEGCC (1e-05)

SGDACC (1e-05)

SEGCC (5e-06)

SGDACC (5e-06)

MSGDARA (2e-05)

MSGDARA (1e-05)

MSGDARA (5e-06)

4.0

4.5

5.0

5.5

6.0

6.5

7.0

7.5

8.0

10

5

Distance to the solution

Scaled IPM (1e+02), bs=100

SEGCC (2e-05)

SGDACC (2e-05)

SEGCC (1e-05)

SGDACC (1e-05)

SEGCC (5e-06)

SGDACC (5e-06)

0

100000

200000

300000

400000

500000

600000

10

5

10

4

10

3

10

2

10

1

Distance to the solution

RN (1e+03), bs=1

SEGCC (2e-05)

SGDACC (2e-05)

MSGDARA (2e-05)

SEGCC (1e-05)

SGDACC (1e-05)

MSGDARA (1e-05)

SEGCC (5e-06)

SGDACC (5e-06)

MSGDARA (5e-06)

Number of gradients

1e6

Number of gradients

1e6

Number of gradients

1e7

Number of gradients

1e7

Number of gradients

F.2 Generative Adversarial Networks

One of the most well-known frameworks in which the objective function is a variational inequality is generative adversarial networks (GAN) Goodfellow et al. [2014]. In the simplest case of this setting, we have a generator G : R z → R d and a discriminator D : R d → R , where z denotes the dimension of the latent space. The objective function can be written as

Here, it is understood that D and G are modeled as neural nets and can be optimized in the distributed setting with gradient descent ascent algorithms. However, due to the complexity of the GANs framework, tricks and adjustments are being employed to ensure good results, such as the Wasserstein GAN formulation [Gulrajani et al., 2017] with Lipschitz constraint on D and the spectral normalization [Miyato et al., 2018] trick to ensure the Lipschitzness of D . The discriminator can thus benefit in practice from multiple gradient ascent steps per gradient descent step on the generator. In addition, Adam [Kingma and Ba, 2014] is often used for GANs as they can be very slow to converge and not perform as well with vanilla SGD.

Therefore, in our implementation of GANs in the distributed setting, we employ all of these techniques and show improvements when we add checks of gradient computations to the server. As for the gradients in our implementation, we can think of the accumulated Adam steps within the clients as 'generalized gradients' and aggregate them in the server with checks of computations (by rewinding model and optimizer state). We tried aggregation after each descent or ascent step, after full descentascent step, and after multiple descent-ascent steps. For the first case, we found that GANs converge much more slowly. For the third case, the performance is better but checks of computations take more time. Thus, we choose to report the performance for the second case: aggregations of a full descent-ascent step. Though, we note that experiments for the other cases suggest similar improvements.

The dataset we chose for this experiment is CIFAR-10 [Krizhevsky et al., 2009] because it is more realistic than MNIST yet is still tractable to simulate in the distributed setting. We let n = 10 , B = 2 , and choose a learning rate of 0.001, β 1 = 0 . 5 , and β 2 = 0 . 9 with a batch size of 64. We run the algorithms for 4600 epochs. We could not average across runs as the simulation is very compute intensive and the benefits are obvious. We compare SGDA-RA (RFA with bucket size 2) and SGDA-CC under the following byzantine attacks: i) no attack (NA), ii) label flipping (LF), iii) inner product manipulation (IPM) [Xie et al., 2019], and iv) a little is enough (ALIE) [Baruch et al., 2019]. The architecture of the GAN follows Miyato et al. [2018].

We show the results in Figure 12. The improvements are most significant for the ALIE attack. Even when no attacks are present, checks of computations only slightly affects convergence speed. This experiment should further justify our proposed algorithm and its real-world benefits even for a setting as complex as distributed GANs. Code for GANs is available at https://github.com/zeligism/ vi-robust-agg .

FIDofCIFAR-10,attack=NA

FID of CIFAR-10, attack = LF

55

SGDA-CC

55

SGDA-CC

SGDA-RA

SGDA-RA

50

50

45

45

FID

FID

40

40

35

35

30

30

1000

100n

FIDofCIFAR-10,attack=IPM

FIDofCIFAR-10,attack=ALIE

60

SGDA-CC

SGDA-CC

55

SGDA-RA

55

SGDA-RA

50

50

FID

45

FID

45

40

40

35

35

30

30

1000

2000

3000

4000

1000

2000

3000

4000

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n**Summary:** The authors propose robust optimization methods for solving distributed variational inequalities (VI). The authors provide theoretical guarantees for numerous popular algorithms for solving min-max (more generally VI) problems. The authors corroborate their theoretical findings with experiments on quadratic games and robust neural network training.\n\nSTRENGTHS:\n**Strengths:**\n\n-\tThe authors provide guarantees for a set of algorithms for solving VIs with application to min-max optimization. Specifically, the authors propose a robust version of SGDA and SEG and provide theoretical convergence guarantees for the proposed algorithms. The authors show that if the batch size at each client is large enough the algorithms can converge to desired accuracy. \n-\tThe authors propose momentum versions of the vanilla algorithms and establish improved convergence performance. The authors establish convergence to a neighborhood whose size can be reduced with increasing batch sizes. \n-\tThe authors also propose random checks of computations versions of the methods for the homogeneous setting and provide a theoretical analysis of the proposed methods. \n-\tThe authors evaluate the convergence performance of the proposed algorithms on a quadratic game and for robust neural network training.\n\nWEAKNESSES:\n**Weaknesses:**\n\n-\tA major weakness of the proposed results is the requirement of large batch sizes to ensure convergence to the desired accuracy. Is it feasible to circumvent this requirement?\n-\tAnother drawback of the paper is that it is written in a very dense manner. The authors have attempted to include a host of results in the main paper which has left no space for stating the algorithms along with their discussions in the main paper. It is advisable to make the paper more accessible to the readers by including detailed algorithms in the main paper. \n-\tThe results are presented in terms of iterate convergence but in the contribution section, the authors have mentioned that their result implies exact convergence for the non-convex case which requires convergence in terms of gradient norm. Please clarify why iterate convergence implies convergence for the non-convex case.\n-\tSome of the discussion from the contribution section is missing in the respective sections. For example, the discussion on the convergence of non-convex functions is not included in the section where momentum versions of the algorithms are presented. \n\n---\n\nUpdated the score.\n\nQUESTIONS:\nPlease see weaknesses section above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies convergence guarantees of Byzantine tolerant methods in the context of distributed variational inequalities.\n\nSTRENGTHS:\nThe paper seems to contain a lot of work.\n\nWEAKNESSES:\nIn my opinion, the main weakness of the paper is its writing. I acknowledge that I have not read the papers most related to this one, [Karimireddy et al., 2021], [Gorbunov et al., 2022], and [Adibi et al., 2022]. Therefore, even though I work on related problems, I struggled to understand the theoretical results. For example,\n- Permutation invariance seems like a crucial concept but is not explained anywhere in the main paper.\n- Theorem 1: all I could infer from the bound is that if $\\delta=0$, the last two terms disappear, and the bound makes sense (since it generalizes smooth, strongly convex minimization). But, intuitively, why does the non-vanishing last term appear? \n- Similar comments about Theorem 2 and 3. Adding momentum seems to require using an averaged iterate rather than using the last iterate. Can you comment on if (and why) is this necessary?\n- Theorems 4-7 have little to no accompanying discussion or intuition for the stated bounds.\n- More questions below.\n\nI do not mean to belittle the authors' contribution. Given my judgment from reading the draft of the main paper, and apart from some questions I have below, I think this is a reasonable contribution. And I admit that many ML conference papers follow this style. But, I think this is not very helpful for the majority of the readers, except for the ones who are already too familiar with the field. Given the page limit in conference papers, maybe a journal would be a better venue for this paper.\n\nQUESTIONS:\n- Assumption 1/2 are for uniformly bounded variance/heterogeneity. But this can be unrealistic in the following case: looking at the bound in Theorem 1, with $\\mu$ in the denominator in the last two terms, we can potentially kill these terms by scaling the loss function be a large constant, consequently increasing $\\mu$. So, can the current results be modified to adapt to the case when the variance/heterogeneity is affine increasing with gradient norm squared?\n- Aren't you also using assumption 2 in Theorem 1-3?\n- Why are SGDA-RA and SEG-RA analyzed for separate assumptions? Doesn't SGDA-RA hold under assumptions of Theorem 2?\n- Why are SEG-CC and R-SEG-CC stated under different assumptions?\n\nMajor confusion about random checks of computation:\n- if the server knows random seeds, why does it need the additional $m$ clients to verify computations? Why can't it do that itself? This would be more privacy friendly also, since the clients can still trust the server, but not other clients. \n- And how can a client check other clients, without having access to their datasets?\n- Won't clients need to communicate their datasets for this to happen? Isn't that a major privacy violation?\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper studies the problem of Byzantine-robust distributed learning with variation inequalities, e.g., max-min form. Three different robust methods for this problem are developed from the concepts of robust aggregation, client momentum, and checks of computations, respectively. For each of the proposed methods, this paper proves its convergence via strict theoretical analysis. Simulation experiments on quadratic game as well as robust NN training further demonstrate the superiority of the proposed methods on the predictive performance over the previous method.

STRENGTHS:
1. The studied problem of Byzantine-robust distributed learning with variational inequalities is important. I think this paper do a good job clarifying different scenarios (or conditions) of this problem and develop an effective method for each of the scenario. The proposed various methods and the corresponding analysis improve the frontier of the relevant research areas. 

2. This paper is technically sound with solid theoretical results. Comparison with other relevant work is clear.

3. This paper is well written and easy to follow.

WEAKNESSES:
1. The motivation of tackling variation inequalities problems is not so sufficient. Provide more examples on VIs besides the max-min form may help better understand the significance of this work.

QUESTIONS:
Minor issue:

The font size in Figure 1 is too small, which makes the legends almost unreadable.

LIMITATIONS:
Limitations have been properly discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-08

comments: SUMMARY:
In this paper, the authors provide robust variants of some popular optimization algorithms in the presence of Byzantine workers. They also use client momentum and random checks of computations to enhance the performance further. They provide theoretical guarantees of converges and empirically test their methods against Robust Distributed Extragradient (RDEG) by [Adibi et al., 2022].

STRENGTHS:
The paper is well-written and ideas are presented clearly. The problem description is interesting and the proposed approach combines existing methods with theoretical guarantees. At first glance, proofs of convergence rates seem correct. As their main contribution, the dependency on large batch size is reduced to a lesser extent.

WEAKNESSES:
The novelty of this paper lies in combining existing ideas from the field of distributed optimization and applying them to the problem at hand. In my opinion, these are not original ideas but rather an extension of ideas to a different setting. This is definitely not a trivial task but I believe it's not entirely original either.

QUESTIONS:
It would strengthen the paper if authors could list down novel ideas/challenges of their paper which are unique to this problem - either in the problem formulation or in proof techniques.

LIMITATIONS:
Limitations are adequately addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09