Title: Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm
Venue: NeurIPS 2023
Accepted: yes
Keywords: decentralized algorithm

minimax problem

algorithmic stability

generalization analysis
Abstract: The growing size of available data has attracted increasing interest in solving minimax problems in a decentralized manner for various machine learning tasks. Previous theoretical research has primarily focused on the convergence rate and communication complexity of decentralized minimax algorithms, with little attention given to their generalization. In this paper, we investigate the primal-dual generalization bound of the decentralized stochastic gradient descent ascent (D-SGDA) algorithm using the approach of algorithmic stability under both convex-concave and nonconvex-nonconcave settings. Our theory refines the algorithmic stability in a decentralized manner and demonstrates that the decentralized structure does not destroy the stability and generalization of D-SGDA, implying that it can generalize as well as the vanilla SGDA in certain situations. Our results analyze the impact of different topologies on the generalization bound of the D-SGDA algorithm beyond trivial factors such as sample sizes, learning rates, and iterations. We also evaluate the optimization error and balance it with the generalization gap to obtain the optimal population risk of D-SGDA in the convex-concave setting. Additionally, we perform several numerical experiments which validate our theoretical findings.

Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm

Abstract

The growing size of available data has attracted increasing interest in solving minimax problems in a decentralized manner for various machine learning tasks. Previous theoretical research has primarily focused on the convergence rate and communication complexity of decentralized minimax algorithms, with little attention given to their generalization. In this paper, we investigate the primal-dual generalization bound of the decentralized stochastic gradient descent ascent (D-SGDA) algorithm using the approach of algorithmic stability under both convex-concave and nonconvex-nonconcave settings. Our theory refines the algorithmic stability in a decentralized manner and demonstrates that the decentralized structure does not destroy the stability and generalization of D-SGDA, implying that it can generalize as well as the vanilla SGDA in certain situations. Our results analyze the impact of different topologies on the generalization bound of the D-SGDA algorithm beyond trivial factors such as sample sizes, learning rates, and iterations. We also evaluate the optimization error and balance it with the generalization gap to obtain the optimal population risk of D-SGDA in the convex-concave setting. Additionally, we perform several numerical experiments which validate our theoretical findings.

1 Introduction

Minimax problems have shown extensive applications in machine learning, such as adversarial robustness [26, 16], GAN [11], the zero-sum game [27], multi-agent reinforcement learning [33], AUC maximization [41]. Alongside this, as the use of large-scale models has become widespread, distributed learning algorithms have emerged as a noteworthy approach for handling massive amounts of data and model parameters [5, 1]. Without a parameter server [20] aggregating all data from each local agent, decentralized algorithms that do not rely on the central structure can be advantageous when network bandwidth is low or latency is high, and they can also protect data privacy[21]. In this work, we consider the following decentralized minimax stochastic optimization problems:

where m denotes the number of agents, F i is the local loss function, ξ i represents local data stored on agent i , and X ⊆ R d x , Y ⊆ R d y . Note that the data distributions D i may differ across the agents.

The most straightforward algorithm for solving the above stochastic minimax optimization problem is to apply Stochastic Gradient Descent Ascent (SGDA) [14, 22] in a decentralized manner, named

D-SGDA. Many algorithms [36, 7, 10, 43, 35, 38, 25, 12, 24, 34, 2] have been proposed to solve Problem (1). As for the theoretical part, they mainly focus on analyzing the convergence behavior and communication complexity of their proposed algorithms. Due to the inaccessibility of the data distribution D i , they approximate the expectation value by averaged sum on the training dataset S = {S 1 , ..., S m } with local samples ξ i,l i stored in local dataset S i = { ξ i,l i } 1 ≤ l i ≤ n :

However, it is insufficient to evaluate the stochastic algorithm not to consider the generalization performance, which is roughly the gap between Eq. (1) and Eq. (2). Generally speaking, saddle point of F S ( x , y ) may not be the optimal solution of min x max y F ( x , y ) . As a result, the model learned by Eq. (2) may not perform well on the test dataset. In fact, the generalization gap is a crucial criterion for us to foresee the performance of the trained model on the unknown dataset. Furthermore, it is quite necessary for us to make a trade-off between the optimization error and the generalization gap to obtain models with optimal population risk (see Eq. (1)).

Concerning the stability and generalization of the minimax problem, several works [19, 30, 9, 42] have studied the generalization gap and population risk of some algorithms, including SGDA, SGDmax, PPM, and AGDA. However, these results cannot be directly extended to the decentralized case due to the additional communication step during the training process. Intuitively, the number of nodes and communication topology in decentralized training may exert a potential influence on the model's generalizability. Note that even for the decentralized minimization problem, the generalization and stability of decentralized SGD are adversely affected by an extra non-vanishing term [32], and the stability usually suffers from a constant term λ 2 [44], compared to vanilla SGD. Building upon these findings, we argue that it is worthy to investigate the generalization and stability of D-SGDA for decentralized minimax problems, where there do exist more newly unveiled problems.

To mitigate this theoretical deficiency, we present the first comprehensive analysis of the stability and generalization of D-SGDA for the decentralized minimax problem in this paper. Specifically, we develop a refined stability analysis in a decentralized manner and derive the generalization gap and population risk for D-SGDA under different settings. The main theoretical results are summarized in Table 1. And our main contributions are summarized as follows:

First work on the stability and generalization of D-SGDA for decentralized minimax problem. We extend the concepts of algorithmic stability, which includes argument stability and weak stability, to the decentralized setting. And we establish a universal connection between argument stability and different measures of generalization gap in the framework of decentralization. We propose a subtle technique to distribute the "different" samples in the neighboring datasets among agents by methods of permutation and combination.

New theoretical results. Our theoretical results reveal that decentralized structure does not hurt the stability and generalizability of D-SGDA compared with SGDA and explain how topology of the communication network influences the performance in stongly-convex-strongly-concave,

convex-concave, and nonconvex-nonconcave conditions (see Table 1,2). We also evaluate the optimization error and leverage it with generalization gap to obtain the optimal population risk.

Experiments. We provide several numerical experiments on AUC maximization (C-C) and adversarial learning (NC-NC) in which we vary different factors to support our theoretical findings. The preliminary experimental results align with our theoretical insights.

2 Related Work

Decentralized minimax problem. Existing works mainly focus on improving the convergence rate and communication complexity. Liu et al. [23] propose DPOSG, which is firstly applied in case of nonconvex-nonconcave, i.e., GAN training, and they prove O ( ϵ -12 ) computational complexity and O ( log (1 /ϵ )) communication complexity on the busiest node. Xian et al. [36] propose DM-HSGD with convergence rate of O ( κ 3 ϵ -3 ) and Chen et al. [7] propose DREAM with communication rounds of O ( κ 2 ϵ -2 / √ 1 -λ 2 ) in nonconvex-strongly-concave condition. Chen et al. [6] propose SPIDERGDA and achieve stochastic first-order oracle of O (( n + √ nκ x κ 2 y ) log (1 /ϵ )) under two-sided PL condition. Rogozin et al. [31] propose a Mirror-prox based algorithm with O ( ϵ -1 ) communication complexity in C-C setting. Huang [15], Luo and Ye [25] accelerates by variance reduction. Beznosikov et al. [3] considers time-varying networks with heterogeneous data, Kovalev et al. [18] provides a rigorous complexity for decentralized variational inequalities.

Stability and generalization. There are mainly two approaches to investigating the generalization: algorithm-independent generalization, which is also called uniform convergence generalization, and algorithm-dependent generalization respectively. Where the former may degrade to a vacuous conclusion in [28] and we adopt the latter method in our paper which can better explain the generalization behavior of a detailed algorithm. Bousquet and Elisseeff [4] come up with algorithmic stability, Elisseeff et al. [8] extend the concept to randomized algorithms. Hardt et al. [13] further develop the framework by connecting algorithmic stability with the generalization gap. Sun et al. [32] and Zhu et al. [44] extend the generalization and stability analysis to D-SGD. In the minimax problem, Zhang et al. [42] focus on argument stability and prove O (1 /n ) weak and strong generalization bounds for the SC-SC condition; Farnia and Ozdaglar [9] analyze the uniform stability and generalization gap of GDA, GDmax, and PPM (proximal point method) in the case of NC-NC and Lei et al. [19] summarize the connection between different measures of stability and generalization gap and further develop the corresponding high-probability results. Xing et al. [37] specify the generalization gap for adversarial training and Yang et al. [39] investigate the stability-based generalization of SGDA with differential privacy constraints. Ozdaglar et al. [30] propose a new metric to better evaluate the generalization performance even in the case when the existing metric fails.

3 Problem Formulation

In this section, we provide the necessary assumptions, notations, terminologies of population risk, generalization gap, and algorithmic stability in decentralized minimax problems.

3.1 Basic Assumptions

Notations. We use bold lower case to denote vectors and bold upper case to denote matrices. ∥ · ∥ 2 means ℓ 2 norm for vectors and ∥ · ∥ F means Frobinius norm for matrices, and we will omit the subscript when the type of norm is clear from the context. 1 n ∈ R n denotes the all-one vector and λ i ( · ) represents the i -th largest eigenvalue of a matrix. [ n ] := { 1 , 2 , ..., n } .

Assumption 1 ( Lipschitz continuous ) . Each local function f i is differentiable and there exists G > 0 that f i is G -Lipschitz continuous with respect to both x and y on any given sample ξ i , i.e.,

Assumption 2 ( Lipschitz smooth ) . Each local function f i is differentiable and there exists L > 0 that f i is L -Lipschitz smooth with respect to both x and y on any given sample ξ i , i.e.,

Definition 1 ( Convexity-Concavity ) . For each local loss function f i ( x , y ; ξ i ) , we say that f i is µ x -strongly convex on x if for any given y and on any given sample ξ i , there holds:

we say that f i is µ y -strongly concave on y if for any given x and on any given sample ξ i , there holds:

We can call it is convex w.r.t. x when µ x = 0 and concave w.r.t. y when µ y = 0 .

Remark 1. Assumptions about the Lipschitz continuity and smoothness are commonly used in the context of decentralized minimax optimization problems [42, 9, 32].

3.2 Decentralized Stochastic Gradient Descent Ascent (D-SGDA)

In decentralized setting, each node will exchange information alternatively and we represent the communication network between nodes as G = ( V , E ) , which is a connected graph with node set V = { 1 , 2 , ..., m } and edge set E ⊆ V × V . Specifically, ( i, l ) ∈ E indicates that agent l can receive information from agent i and therefore we symbolize the in and out neighbors as N in ( i ) ≜ { l ∈ V , ( l, i ) ∈ E} and N out ( i ) ≜ { l ∈ V , ( i, l ) ∈ E} respectively. In an undirected graph, there is no consideration about the order, thus ( i, l ) ∈ E implies ( l, i ) ∈ E and the in and out neighbors are identical which we will abbreviate as N for brief. In our work, we focus on undirected graphs. The communication graph is associated with an adjacency matrix, which is also called a mixing matrix, W = [ ω ij ] ∈ R m × m . It implies the connection between m agents that ω ij > 0 if and only if ( j, i ) ∈ E , otherwise ω ij = 0 . And there are some basic assumptions about the mixing matrix which is commonly used in decentralized settings [17, 21, 23, 32].

Assumption 3 ( Mixing matrix ) . We assume the mixing matrix W = [ ω ik ] ∈ [0 , 1] m × m defined on the graph G = ( V , E ) is a symmetric doubly stochastic matrix, which holds the property that W T = W and W 1 m = 1 m , 1 T m W = 1 T m . Besides, we assume λ := max {| λ 2 | , | λ m ( W ) |} ∈ (0 , 1) .

For a symmetric doubly stochastic matrix, W holds the property that: λ 1 = 1 . For different topologies, λ → 1 implies the sparsity while λ → 0 implies the complete connection. Nedi´ c et al. [29] and Ying et al. [40] list upper bounds for the spectral gap 1 -λ over the commonly communication network. More knowledge on decentralized optimization is placed in Appendix A.

In this paper, we study the decentralized minimax problem solved via D-SGDA (see Algorithm 1). We use the superscript to denote the t -th iteration and the subscript to denote the i -th local agent. During iteration, each client first computes its local gradient approximation by ∇ x f i ( x t i , y t i ; ξ i,j t ( i ) ) and ∇ y f i ( x t i , y t i ; ξ i,j t ( i ) ) respectively where j t ( i ) is randomly chosen from [ n ] . Then each client communicates with its neighbor N ( i ) and updates by SGDA.

3.3 Generalization Gap

In a sense, we obtain the result by minimaxing the empirical one F S in Eq. (2), which differs from the population one F in Eq. (1). So we can not guarantee the same performance on the unknown distribution as on the training dataset. And therefore the gap between the empirical one and the population one reflects the ability of generalization. Unlike the standard learning theory which only contains a single variable that can directly define the population risk and empirical risk by the objective function[4]. Owing to the structure of minimax, there are different methods to define the population and empirical risk as concluded in [19], where primal-dual measure starts from the idea of duality gap in optimization. And we first introduce two types of population risks as follows.

Definition 2 ( Population risk ) . For a randomized model ( x , y ) , we define the population risk as:

Algorithm 1 D-SGDA

Strong primal-dual population risk : ∆ s ( x , y ) = E [sup y ′ ∈Y F ( x , y ′ ) -inf x ′ ∈X F ( x ′ , y )] .

Here the expectation is taken over the randomness of the model. By replacing function F with function F S (see Eq. (2)) in Def. 2 when considering the empirical risk, we obtain the corresponding weak primal-dual empirical risk ∆ w S ( x , y ) = sup y ′ ∈Y E [ F S ( x , y ′ )] -inf x ′ ∈X E [ F S ( x ′ , y )] and strong primal-dual empirical risk ∆ s S ( x , y ) = E [sup y ′ ∈Y F S ( x , y ′ ) -inf x ′ ∈X F S ( x ′ , y )] respectively. Subtracting the empirical risk from population risk, we can define the generalization gap as follows. Definition 3 ( Generalization gap ) . For a randomized model ( x , y ) , we define the corresponding generalization gap as:

Weak primal-dual generalization gap : ϵ w gen ( x , y ) = ∆ w ( x , y ) -∆ w S ( x , y ) .

Strong primal-dual generalization gap : ϵ s gen ( x , y ) = ∆ s ( x , y ) -∆ s S ( x , y ) .

Remark 2. Notice that we revise the name as generalization gap to avoid misunderstanding since "generalization error" usually refers to the empirical risk in learning theory (see [4]). Our primal target (see Problem (1)) is to obtain small population risk (Def. 2) which can be considered as a summation like ∆ w ( x , y ) = ϵ w gen ( x , y ) + ∆ w S ( x , y ) , where generalization gap reflects how well the model generalizes and empirical risk reflects the optimization performance. The strong primal-dual risk is stronger than the weak one due to ∆ w ( x , y ) ≤ ∆ s ( x , y ) according to Jensen's inequality. But in some cases, it is sufficient to bound weak primal-dual population risk such as the MDP [42].

3.4 Algorithmic Stability

Inspired by [13] that the generalization gap of an ϵ -stable algorithm can be bounded by ϵ . And this connection between stability and generalization for minimax problem is furthermore established in [19]. For a randomized algorithm A solving the problem (2), we use A ( S ) = ( A x ( S ) , A y ( S )) to denote the output of applying algorithm A on dataset S = {S 1 , ..., S m } . Let E S and E A denote taking expectation on the randomness of the algorithm A and the dataset S respectively. Sometimes we omit the subscript as E [ · ] when it is clear from the context. Next, we first refine the definitions of algorithmic stability in a decentralized manner and then provide a connection between algorithmic stability and generalization gap in the framework of decentralization.

Definition 4 ( Decentralized neighboring dataset ) . We call S , S ′ the decentralized neighboring datasets when there are at most one different sample in each local dataset, where S = {S 1 , S 2 , ..., S m } , S ′ = {S ′ 1 , S ′ 2 , ..., S ′ m } and each S i and S ′ i differs by at most one sample.

Definition 5 ( Decentralized algorithmic stability ) . For a randomized algorithm A , we say:

A is ϵ -argument stable if there holds for any neighboring datasets S , S ′ :

A is ϵ -weakly stable if there holds for any neighboring datasets S , S ′ :

where ξ ≜ { ξ 1 , ..., ξ m } denotes sample index with ξ i ∈ D i and f ( x , y ; ξ ) ≜ 1 m ∑ m i =1 f i ( x , y ; ξ i ) .

And we further specify the stability error as ϵ arg sta ( A ) and ϵ w sta ( A ) respectively.

Remark 3. The definition of neighboring datasets in the decentralized setting can degenerate to the traditional neighboring datasets where there is at most a single different sample between S and S ′ . And the refined concepts of algorithmic stability are also fit for classic stability without decentralization in [19]. These facts validate that our definitions above are well-defined. Notice that the argument stability can imply weak stability because of the property of Lipschitz continuity (see Assumption 1). Specifically speaking, when algorithm A is ϵ -argument stable, then it is √ 2 Gϵ -weakly stable. So we will mainly focus on the argument stability in the rest part.

Theorem 1 ( Connection ) . For an ϵ -argument stable decentralized algorithm A , under Assumption 1, we have the following different measures of generalization gap: √

Weak primal-dual generalization gap: ϵ w gen ( A x ( S ) , A y ( S )) ≤ 2 Gϵ .

Strong primal-dual generalization gap holds under extra Assumption 2 when f is µ SCµ SC:

i x y .

Remark 4. The complete proof is provided in Appendix D. We establish the connection between argument stability and a different measure of generalization gap under different constraints, where weak primal-dual generalization gap does not require any convexity or concavity in part a. and strong primal-dual generalization gap requires both strong convexity and strong concavity in part b.. And this connection is not limited to the single decentralized algorithm D-SGDA, but a universal connection for decentralized minimax algorithms. Actually weak stability is sufficient to prove the weak primal-dual generalization gap in part a. that when algorithm A is ϵ -weakly stable, we have E A , S [ ϵ w gen ( A x ( S ) , A y ( S ))] ≤ ϵ . And the theorem implies that once we have access to the stability error, we can derive the generalization gap as an accompanying result.

4 Theoretical Results on D-SGDA

In this section, we study algorithmic stability and generalization bound in SC-SC, C-C, and NC-NC settings in Sec 4.1, Sec 4.2 and Sec 4.3, respectively. Due to the space limitation, the proofs are placed in the Appendix E, F, G respectively.

4.1 Results on Strongly-Convex-Strongly-Concave Case

Below, we first characterize the argument stability with fixed and decaying learning rates, respectively. Theorem 2 ( Argument Stability ) . Under Assumption 1,2,3 when each f i is µ x -strongly convex and µ y -strongly concave, we have the argument stability bound for D-SGDA (denoted as A ):

where η max t ≜ max { η x ,t , η y ,t } , η min t ≜ min { η x ,t , η y ,t } , µ = min { µ x , µ y } . Furthermore, a. for fixed learning rates, ϵ arg sta ( A ) ≤ 2 G L + µ η min Lµ ( 2( η max ) 2 L 1 -λ + η max n ) . b. for decaying learning rates with η min t = 1 µ ( t +1) and η max t = 1 µ ( t +1) c , c ≤ 1 , we have:

Remark 5. (i) Bound analysis and comparison. For case a.with fixed learning rates, the argument stability is bounded by O ( η 1 -λ + 1 n ) , which can reach O ( 1 (1 -λ ) T + 1 n ) when η ∼ 1 T . For case b. with decaying learning rates, we should require 2 c ≥ L L + µ +1 otherwise the bound can tend to infinity, then we have ϵ arg sta ( A ) ≤ 2 G µ (1 -c + L L + µ ) T 1 -c n + 4 GLC λ µ 2 T L L + µ ( 1 2 c>L/ ( L + µ )+1 2 c -L L + µ -1 +ln T · 1 2 c = L/ ( L + µ )+1 ) . Both can be bounded by ˜ O ( T 1 -c n + C λ T L L + µ ) with ˜ O containing the logarithmic function into consideration, which matches the corresponding results for SGDA (see Thm.2.(e) in [19]) except an extra multiplication factor C λ : ˜ O ( 1 √ T + 1 N ) with decaying learning rates and N the total sample size. (ii) Factor influence. It is apparent that a smaller stability bound is achieved with larger sample size n , and smaller learning rates η (which is associated with larger T ). And notice that the decaying learning rates may slightly underperform than the fixed one, it is easy to explain that at the beginning decaying learning rates are too large and therefore result in weaker stability. The influence of these factors is consistent with vanilla SGDA. (iii) Effect of topology. So the major difference lies in C λ and the number of nodes m . For C λ = ( k/e ) c λ (ln 1 λ ) c + 2 e -1 λ ln 1 λ + 2 c λ ln 1 λ , when λ → 1 , it is bounded by O ( 1 λ ln 1 λ ) and when λ → 0 , C λ is bounded by O ( 1 λ (ln 1 λ ) c ) . We list some common topology with estimated value of λ along with the upper bound of C λ in Table 2. We can conclude that topology with a denser connection (larger spectral gap 1 -λ ) will have a smaller stability error, i.e., be more stable. In extreme conditions, the fully connected network will behave as well as vanilla SGDA, and the stability error of the disconnected network will diverge. Furthermore, under the same topology, fewer nodes will result in better stability.

Generalization gap. According to Thm. 1, we can directly derive the weak and strong primal-dual generalization gap of D-SGDA as √ 2 Gϵ arg sta and G √ 2 + 2 L 2 µ 2 ϵ arg sta respectively. Therefore, we hold the same analysis as stability in above Remark 5.

Next, we will first derive the optimization error and then provide the population risk by decomposition ∆ s ( x , y ) = ϵ s gen ( x , y ) + ∆ s S ( x , y ) . Population risk is an important evaluation for the performance of a stochastic learning algorithm, which will evaluate how our model obtained by training dataset behave over the whole distribution. Notice that we will use the average output instead of the last iterate in analyzing the optimization errors. We denote that:

Theorem 3 ( Strong primal-dual population risk ) . Under Assumption 1,2,3, when each f i is µ x SCµ y SC, we have the strong primal-dual population risk as follows, where η max t ≜ max { η x ,t , η y ,t } , η min t ≜ min { η x ,t , η y ,t } , µ = min { µ x , µ y } , and ( x T ave , y T ave ) is defined in Eq. (3) :

a. for fixed learning rates,

b. for decaying learning rates that η min t = 1 µ ( t +1) and η max t = 1 µ ( t +1) c with c ≤ 1 and 2 c ≥ L L + µ +1 ,

̸

̸

̸

Remark 6. By Jensen's inequality (see Remark 2), we can conclude that weak primal-dual population risk also satisfies the conclusions above. (i) Bound analysis and comparison. For fixed learning rates, it is interesting to see that we should choose η ∼ 1 / √ T to get the optimal population risk of O ( 1 n + 1 (1 -λ ) √ T ) , although when η ∼ 1 /T we can get optimal generalization performance but the optimization error will not converge. While for the decaying learning rates, the population risk bound is ˜ O ( T 1 -c /n + C λ /T min { 1 2 , L L + µ } ) , which matches the corresponding results for SGDA (see Thm.3.(c) in [19]) of O (ln N/Nµ ) when n ∼ T min { 1 2 , L L + µ } . (ii) Topology influence. We omit the trivial factor influence analysis here (or see Remark 5). And the effect of topology is captured quantitatively by C λ and 1 1 -λ which have been discussed in Remark 5 and Table 2.

4.2 Results on Convex-Concave Case

In this section, we provide the argument stability and weak primal-dual population risk of D-SGDA algorithm for the NC-NC condition in the following theorems with proof in Appendix F.

Theorem 4 ( Argument Stability ) . Under Assumption 1,2,3, when each f i is convex-concave, we have the argument stability bound for D-SGDA (denoted as A ):

Remark 7. (i) Bound analysis and comparison. When there is no strong convexity or strong concavity, we can no longer choose the decaying learning rates, otherwise the argument stability error may not converge. For fixed learning rates, ϵ arg sta is upper bounded by O ( ηT n + η 2 T 1 -λ ) , which is slightly looser than SC-SC condition. While we can still choose η ∼ 1 /T to obtain optimal result O ( 1 n + 1 (1 -λ ) T ) . Besides, compared with the corresponding result for SGDA (see Thm.2.(b) in [19]) of O ( √ T N + 1 √ N ) , we can approach it when η ∼ 1 /T 3 / 4 and n ∼ T 3 / 4 . (ii) Topology influence. In C-C condition, the effect of topology on the stability is quantified by 1 1 -λ which has been discussed in Table 2. And we can conclude that denser topology is more stable and fewer nodes will increase stability under the same topology.

Generalization gap. Thm. 1 implies ϵ w gen ( A x ( S ) , A y ( S )) ≤ √ 2 Gϵ arg sta ( A ) . So the generalization gap holds with the same quantitative analysis as stability above. Analogously we can present the weak primal-dual population risk in the following theorem.

Theorem 5 ( Weak primal-dual population risk ) . Under Assumption 1,2,3, when each f i CC, we have the weak primal-dual population risk as follows, where η max t ≜ max { η x ,t , η y ,t } , η min t ≜ min { η x ,t , η y ,t } , µ = min { µ x , µ y } , and ( x T ave , y T ave ) is defined in Eq. (3) :

Remark 8. The weak primal-dual population risk attains optimal of O ( T 1 / 3 n + 1 (1 -λ ) T 1 / 3 ) when we choose η max = η min ∼ 1 /T 2 3 . Note that we select η ∼ 1 /T to obtain optimal generalization performance (see Remark 7), but the optimization error will diverge in that case. Compared with the result of SGDA: O ( N -1 / 2 ) (see Thm.3.(b) in [19]), our result can approach it by n 1 / 2 ∼ T 1 / 3 . Then the effect of topology and number of nodes on the population risk is reflected by 1 1 -λ , which has been discussed in Remark 5 and Table 2.

4.3 Results on Nonconvex-Nonconcave Case

In this section, we present the weak stability and weak primal-dual generalization gap of D-SGDA algorithm for the NC-NC problem in the following theorem with proof in Appendix G.

Theorem 6 ( Weak stability ) . Under Assumption 1,2,3, denoting D-SGDA algorithm as A , we have the following weak stability bound when η max t ≜ max { η x ,t , η y ,t } , and η min t ≜ min { η x ,t , η y ,t } :

a. for fixed learning rates that η max t = η max , and η min t = η min ,

b. for decaying learning rates that η min t = 1 t +1 , and η max t = 1 ( t +1) c , c ≤ 1 ,

Weak primal-dual generalization gap. According to Thm. 1, we can derive the weak primal-dual generalization gap as ϵ w gen ( A x ( S ) , A y ( S )) ≤ √ 2 Gϵ w sta ( A ) for D-SGDA in NC-NC condition.

Remark 9. For case a. with fixed learning rates, the weak stability and weak primal-dual generalization gap is bounded by O ( ηT n + η 2 T 1 -λ ) , which can reach O ( 1 n + 1 (1 -λ ) T ) when η ∼ 1 T . For case b. with decaying learning rates, the stability and generalization gap is bounded by O (( C λ ) 1 c + L T L c + L ( m n ) 1 -1 c + L ) . It is evident to analyze the influence of factors that, larger sample size and fewer nodes will result in a smaller stability error and generalization gap, which coincides with results in (S)C-(S)C conditions (see Remark 5). Approaching to the weak primal-dual generalization bound of O ( n -2 cρ +1 2 cρ +3 T 2 cρ 2 cρ +3 ) provided ρ -weakly-convex-weakly-concave (see Thm.5 in [19]). C λ reflects the effect of topology on the stability and generalization gap and its value has been discussed in Reamrk 5 and Table 2.

0

2000

4000

6000

8000

10000

12000

14000

16000

Number of iterations

0.000

0.002

0.004

0.006

0.008

0.010

Euclidean distance

=0.1

=0.3

=1.0

=3.0

0

2000

4000

6000

8000

10000

12000

14000

16000

Number of iterations

0.000

0.001

0.002

0.003

0.004

0.005

Euclidean distance

meshgrid

all

exponential

ring

star

0

2000

4000

6000

8000

10000

12000

14000

16000

Number of iterations

0.00

0.02

0.04

0.06

0.08

Euclidean distance between models

Size=100

Size=200

Size=300

Size=400

Size=500

0

500

1000

1500

2000

2500

3000

3500

4000

Number of iterations

0.000

0.005

0.010

0.015

0.020

0.025

Euclidean distance between models

=0.1

=0.3

=1.0

=3.0

0

2000

4000

6000

8000

10000

12000

14000

16000

Number of iterations

0.030

0.025

0.020

0.015

0.010

0.005

0.000

0.005

ACC distance

=0.1

=0.3

=1.0

=3.0

0

2000

4000

6000

8000

10000

12000

14000

16000

Number of iterations

0.06

0.04

0.02

0.00

0.02

0.04

ACC distance

meshgrid

all

exponential

ring

star

0

2000

4000

6000

8000

10000

12000

14000

16000

Number of iterations

0.08

0.06

0.04

0.02

0.00

0.02

0.04

0.06

ACC distance between models

Size=100

Size=200

Size=300

Size=400

Size=500

0

500

1000

1500

2000

2500

3000

3500

4000

Number of iterations

0.04

0.02

0.00

0.02

0.04

ACC distance between models

=0.1

=0.3

=1.0

=3.0

0.05

2

4

6

8

10

Number of Passes

0.0000

0.0005

0.0010

0.0015

0.0020

0.0025

0.0030

0.0035

Euclidean Distance

lr=0.0002

lr=0.0001

2

4

6

8

10

Number of Passes

0.00

0.01

0.02

0.03

0.04

Euclidean Distance

node=2

node=4

node=6

node=8

node=10

0

20

40

60

80

100

Number of Iterations

0.0000

0.0002

0.0004

0.0006

0.0008

0.0010

0.0012

Euclidean Distance

Size400

Size600

Size800

Size1000

2

4

6

8

10

Number of Passes

0.000

0.001

0.002

0.003

0.004

0.005

Euclidean Distance

mode=single

mode=star

mode=all

mode=meshgrid

mode=exponential

mode=ring

2

4

6

8

10

Number of Passes

0.0000

0.0005

0.0010

0.0015

0.0020

Euclidean Distance

lr=0.0002

lr=0.0001

2

4

6

8

10

Number of Passes

0.005

0.000

0.005

0.010

0.015

0.020

0.025

0.030

Euclidean Distance

node=2

node=4

node=6

node=8

node=10

0

20

40

60

80

100

Number of Iterations

0.0000

0.0005

0.0010

0.0015

0.0020

0.0025

Euclidean Distance

Size400

Size600

Size800

Size1000

2

4

6

8

10

Number of Passes

0.0000

0.0005

0.0010

0.0015

0.0020

Euclidean Distance

mode=single

mode=star

mode=all

mode=meshgrid

mode=exponential

mode=ring

5 Experiments

Experiments Setup. We evaluate our theoretical results of the C-C case by adopting the SOLAM method [41] to solve the AUC problem on two datasets svmguide and w5a , and the NC-NC case by solving the generative adversarial network on MNIST . We extend the methods for both cases to a decentralized implementation. Our experimental setting follows the way conducted in [13, 19] to study how the stability and generalizability of D-SGDA would behave along the learning process with different factors, including learning rates, typologies, nodes, and sample sizes. We employ the same randomized method to generate two model sequences, one for the original data and another for a one-observation perturbing data, and subsequently calculate the Euclidean distance ∆ between their respective parameter sets. Additional implementation details can be found in the Appendix .

Results analysis: From Fig. 1 and Fig. 2, we can observe that: (i) faster learning rates, fewer number of nodes and smaller sample size can result in a smaller Euclidean distance between weights and a smaller difference between training dataset and validation dataset; (ii) the performance of different topology on stability: fully connected(all) > exponential > grid ≈ ring ≥ star > disconnected(single).

These validate our theoretical results of the algorithmic stability for D-SGDA (see Remark 5 below Thm. 2). And the impact of different topologies also coincides with our discussion in Table 2.

6 Conclusion

In this paper, we provide the first comprehensive analysis for the stability and generalization of D-SGDA for decentralized minimax problems. Our theoretical results show that a decentralized structure does not destroy the stability and generalization of D-SGDA, instead we can leverage between the iterations and the number of nodes, as well as sample size to achieve better population performance. Numerical experiments also validate our theory. Our analysis technique has the potential used for studying the ability and generalization of other decentralized minimax algorithms.

Limitation&Broader Impacts. In our analysis, we require the Lipschitz smoothness, which may be further relaxed in future work. In addition, D-SGDA can converge with the heterogeneous data distribution, whose stability and generalization are still unexplored in this work. Since our work focuses on the theoretical understanding of D-SGDA, it does not suffer from negative impacts.

References

Ron Bekkerman, Mikhail Bilenko, and John Langford. Scaling up machine learning: Parallel and distributed approaches . Cambridge University Press, 2011.

Aleksandr Beznosikov, Gesualdo Scutari, Alexander Rogozin, and Alexander Gasnikov. Distributed saddle-point problems under data similarity. Advances in Neural Information Processing Systems , 34:8172-8184, 2021.

Aleksandr Beznosikov, Pavel Dvurechenskii, Anastasiia Koloskova, Valentin Samokhin, Sebastian U Stich, and Alexander Gasnikov. Decentralized local stochastic extra-gradient for variational inequalities. Advances in Neural Information Processing Systems , 35:38116-38133, 2022.

Olivier Bousquet and Andr 'e Elisseeff. Stability and generalization. The Journal of Machine Learning Research , 2:499-526, 2002.

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning , 3(1):1-122, 2011.

Lesi Chen, Boyuan Yao, and Luo Luo. Faster stochastic algorithms for minimax optimization under polyak-{\ L } ojasiewicz condition. Advances in Neural Information Processing Systems , 35:13921-13932, 2022.

Lesi Chen, Haishan Ye, and Luo Luo. A simple and efficient stochastic algorithm for decentralized nonconvex-strongly-concave minimax optimization. arXiv preprint arXiv:2212.02387 , 2022.

Andre Elisseeff, Theodoros Evgeniou, Massimiliano Pontil, and Leslie Pack Kaelbing. Stability of randomized learning algorithms. Journal of Machine Learning Research , 6(1), 2005.

Farzan Farnia and Asuman Ozdaglar. Train simultaneously, generalize better: Stability of gradient-based minimax learners. In International Conference on Machine Learning , pages 3174-3185. PMLR, 2021.

Hongchang Gao. Decentralized stochastic gradient descent ascent for finite-sum minimax problems. arXiv preprint arXiv:2212.02724 , 2022.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM , 63(11):139-144, 2020.

Hongyi Guo, Zuyue Fu, Zhuoran Yang, and Zhaoran Wang. Decentralized single-timescale actor-critic on zero-sum two-player stochastic games. In International Conference on Machine Learning , pages 3899-3909. PMLR, 2021.

Moritz Hardt, Ben Recht, and Yoram Singer. Train faster, generalize better: Stability of stochastic gradient descent. In International conference on machine learning , pages 1225-1234. PMLR, 2016.

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems , 30, 2017.

Feihu Huang. Adaptive federated minimax optimization with lower complexities. arXiv preprint arXiv:2211.07303 , 2022.

Zhuo Huang, Miaoxi Zhu, Xiaobo Xia, Li Shen, Jun Yu, Chen Gong, Bo Han, Bo Du, and Tongliang Liu. Robust generalization against photon-limited corruptions via worst-case sharpness minimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16175-16185, 2023.

Anastasia Koloskova, Tao Lin, Sebastian U Stich, and Martin Jaggi. Decentralized deep learning with arbitrary communication compression. In International Conference on Learning Representations .

Dmitry Kovalev, Aleksandr Beznosikov, Abdurakhmon Sadiev, Michael Persiianov, Peter Richtárik, and Alexander Gasnikov. Optimal algorithms for decentralized stochastic variational inequalities. arXiv preprint arXiv:2202.02771 , 2022.

Yunwen Lei, Zhenhuan Yang, Tianbao Yang, and Yiming Ying. Stability and generalization of stochastic gradient methods for minimax problems. In International Conference on Machine Learning , pages 6175-6186. PMLR, 2021.

Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. Scaling distributed machine learning with the parameter server. In 11th { USENIX } Symposium on Operating Systems Design and Implementation ( { OSDI } 14) , pages 583-598, 2014.

Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. Advances in neural information processing systems , 30, 2017.

Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning , pages 6083-6093. PMLR, 2020.

Mingrui Liu, Wei Zhang, Youssef Mroueh, Xiaodong Cui, Jarret Ross, Tianbao Yang, and Payel Das. A decentralized parallel algorithm for training generative adversarial nets. Advances in Neural Information Processing Systems , 33:11056-11070, 2020.

Songtao Lu, Kaiqing Zhang, Tianyi Chen, Tamer Ba¸ sar, and Lior Horesh. Decentralized policy gradient descent ascent for safe multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 8767-8775, 2021.

Luo Luo and Haishan Ye. Decentralized stochastic variance reduced extragradient method. arXiv preprint arXiv:2202.00509 , 2022.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations .

Eric V Mazumdar, Michael I Jordan, and S Shankar Sastry. On finding local nash equilibria (and only local nash equilibria) in zero-sum games. arXiv preprint arXiv:1901.00838 , 2019.

Vaishnavh Nagarajan and J Zico Kolter. Uniform convergence may be unable to explain generalization in deep learning. Advances in Neural Information Processing Systems , 32, 2019.

Angelia Nedi´ c, Alex Olshevsky, and Michael G Rabbat. Network topology and communicationcomputation tradeoffs in decentralized optimization. Proceedings of the IEEE , 106(5):953-976, 2018.

Asuman E Ozdaglar, Sarath Pattathil, Jiawei Zhang, and Kaiqing Zhang. What is a good metric to study generalization of minimax learners? In Advances in Neural Information Processing Systems .

Alexander Rogozin, Aleksandr Beznosikov, Darina Dvinskikh, Dmitry Kovalev, Pavel Dvurechensky, and Alexander Gasnikov. Decentralized distributed optimization for saddle point problems. arXiv preprint arXiv:2102.07758 , 2021.

Tao Sun, Dongsheng Li, and Bao Wang. Stability and generalization of decentralized stochastic gradient descent. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 9756-9764, 2021.

Hoi-To Wai, Zhuoran Yang, Zhaoran Wang, and Mingyi Hong. Multi-agent reinforcement learning via double averaging primal-dual optimization. Advances in Neural Information Processing Systems , 31, 2018.

Chen-Yu Wei, Chung-Wei Lee, Mengxiao Zhang, and Haipeng Luo. Last-iterate convergence of decentralized optimistic gradient descent/ascent in infinite-horizon competitive markov games. In Conference on learning theory , pages 4259-4299. PMLR, 2021.

Xidong Wu, Zhengmian Hu, and Heng Huang. Decentralized riemannian algorithm for nonconvex minimax problems. arXiv preprint arXiv:2302.03825 , 2023.

Wenhan Xian, Feihu Huang, Yanfu Zhang, and Heng Huang. A faster decentralized algorithm for nonconvex minimax problems. Advances in Neural Information Processing Systems , 34: 25865-25877, 2021.

Yue Xing, Qifan Song, and Guang Cheng. On the algorithmic stability of adversarial training. Advances in neural information processing systems , 34:26523-26535, 2021.

Yangyang Xu. Decentralized gradient descent maximization method for composite nonconvex strongly-concave minimax problems. arXiv preprint arXiv:2304.02441 , 2023.

Zhenhuan Yang, Shu Hu, Yunwen Lei, Kush R Vashney, Siwei Lyu, and Yiming Ying. Differentially private sgda for minimax problems. In Uncertainty in Artificial Intelligence , pages 2192-2202. PMLR, 2022.

Bicheng Ying, Kun Yuan, Yiming Chen, Hanbin Hu, Pan Pan, and Wotao Yin. Exponential graph is provably efficient for decentralized deep training. Advances in Neural Information Processing Systems , 34:13975-13987, 2021.

Yiming Ying, Longyin Wen, and Siwei Lyu. Stochastic online auc maximization. Advances in neural information processing systems , 29, 2016.

Junyu Zhang, Mingyi Hong, Mengdi Wang, and Shuzhong Zhang. Generalization bounds for stochastic saddle point problems. In International Conference on Artificial Intelligence and Statistics , pages 568-576. PMLR, 2021.

Xin Zhang, Zhuqing Liu, Jia Liu, Zhengyuan Zhu, and Songtao Lu. Taming communication and sample complexities in decentralized policy evaluation for cooperative multi-agent reinforcement learning. Advances in Neural Information Processing Systems , 34:18825-18838, 2021.

Tongtian Zhu, Fengxiang He, Lan Zhang, Zhengyang Niu, Mingli Song, and Dacheng Tao. Topology-aware generalization of decentralized sgd. In International Conference on Machine Learning , pages 27479-27503. PMLR, 2022.

Appendix

Table of Contents

A Decentralized Optimization

fully connected

grid

ring

star

exponential

B Notation

X t ≜ ( x t 1 , ..., x t m ) T ∈ R m × d x ; Y t ≜ ( y t 1 , ..., y t m ) T ∈ R m × d y ; and we use [ X , Y ] t ∈ R m × ( d x + d y ) to denote the concatenation [( x t 1 y t 1 ) , ( x t 2 y t 2 ) , ..., ( x t m y t m )] T .

We use ξ t to denote the samples collected in the t -th iteration, i.e., { ξ 1 ,j t (1) , ..., ξ m,j t ( m ) } .

We denote the gradients in the t -th iteration as follows:

We use P ∈ R m × m to denote the matrix whose elements are all 1 m , I to denote the identity matrix, and 1 m ∈ R m to denote the vector with all elements equal to 1 .

C Important Lemmas

In this section, we provide some important lemmas as fundamentals of the following proof.

Lemma 1. We define G g,η as

with η max ≜ max { η x , η y } and η min ≜ min { η x , η y } .Under the assumption that g is L -Lipschitz smooth, then we have:

G g,η is (1 + η max L ) -expansive.

When g is µ x -strongly-convex w.r.t. x and µ y -strongly-concave w.r.t. y , letting L + µ 2 ( η max ) 2 ≤ η min ≤ L + µ 2 1 Lµ where µ ≜ min { µ x , µ y } , then G g,η is (1 -η min Lµ L + µ ) -expansive.

Proof. For case a. where we do not require strong convexity or strong concavity, we have:

When function g is further µ x -strongly convex and µ y -strongly concave in case b., thus we have:

Recalling the co-coercivity of a L-Lipschitz smooth and µ -strongly convex function f ( x ) that [13]:

So we can get:

Combining above inequalities (4) and (5), we can get:

2 2 Lµ

Lemma 2. Letting P ∈ R m × m denote the matrix whose elements are all 1 m , following the update rule of D-SGDA (see Algorithm 1), we have:

where η max s = max { η x ,s , η y ,s } .

Proof. The Lipschitz continuity (see Assumption 1) implies that for any x ∈ X and y ∈ Y on a given sample ξ , the gradient value is bounded by G , i.e., ∥ ∥ ∥ ∥ ( η x ,t ∇ x f ( X t , Y t ; ξ t ) -η y ,t ∇ y f ( X t , Y t ; ξ t ) )∥ ∥ ∥ ∥ ≤ max { η x ,t , η y ,t } √ mG . Following the proof of Lemma 8 in [32], we can get the result, which can be expressed as [∑ m i =1 ∥ x t i -x t ∥ 2 + ∥ y t i -y t ∥ 2 ] 1 / 2 ≤ 2 √ mG ∑ t -1 s =0 η max s λ t -1 -s

Lemma 3. When 0 < λ < 1 ,

Proof.

whereas for the integral we have:

Therefore we have:

When 0 < k < 1 , ∑ t -1 j =0 λ t -1 -j ( j +1) k ≤ 1 t k ( λ t -1 t k + λ t 2 -1 t + 2 k λ ln 1 λ ) , where λ t -1 t k ≤ ( k/e ) k λ (ln 1 λ ) k , λ t 2 -1 t ≤ 2 e -1 λ ln 1 λ . So we can define C λ = ( k/e ) k λ (ln 1 λ ) k + 2 e -1 λ ln 1 λ + 2 k λ ln 1 λ that ∑ t -1 j =0 λ t -1 -j ( j +1) k ≤ C λ t k . And for the case k = 1 we can roughly consider the logarithm as constants so our discussion can be put together into the above category as k = 1 .

Remark 10. Considering function h ( λ ) = 1 λ ln 1 λ , it monotonically decrease in interval (0 , 1 e ) and monotonically increase in interval ( 1 e , 1) with minimal value of e . Analogously for the function g ( λ ) = 1 λ (ln 1 λ ) k , which monotonically decrease in interval (0 , 1 e k ) and monotonically increase in interval ( 1 e k , 1) with minimal value of ( e k ) k .Therefore, when k = 1 , C λ is bounded by O ( 1 λ ln 1 λ ) ; otherwise, when λ → 1 , C λ is bounded by O ( 1 λ ln 1 λ ) ; when λ → 0 , C λ is bounded by O ( 1 λ (ln 1 λ ) k ) .

D Proof of the Connection

In this section, we will first figure out a fundamental lemma that illustrates the structure of dataset in decentralized setting. Then we will provide proof for the connection between argument stability and primal-dual generalization gap.

Lemma 4. Denoting dataset as S = {S 1 , ..., S m } , and denoting each local samples stored in S i as S i = { ξ i,l i } l i =1 ,...,n . Then we have the decomposition equation of the empirical function:

Proof.

Proof of Theorem 1. We let S = {S 1 , ..., S m } where S i = { ξ i,l } 1 ≤ l ≤ n and S ′ = {S ′ 1 , ..., S ′ m } where S ′ i = { ξ ′ i,l } 1 ≤ l ≤ n be two different datasets while S i and S ′ i are drawn from the same distribution D i . Further we define S ( l ) = {S ( l 1 ) 1 , S ( l 2 ) 2 , ..., S ( l m ) m } where l = ( l 1 , l 2 , ..., l m ) and S ( l i ) i = { ξ i, 1 , ..., ξ i,l i -1 , ξ ′ i,l i , ξ i,l i +1 , ..., ξ i,n } . So we can say that there is at most one different sample in each local dataset between S and S ( l ) . For function F : X × Y ↦→ R , we denote y ∗ ( x ) = arg max y ∈Y F ( x , y ) .

For case a. the weak primal-dual generalization gap ϵ w gen ( A x ( S ) , A y ( S )) = ∆ w ( A x ( S ) , A y ( S )) -∆ w S ( A x ( S ) , A y ( S )) . We first make some adjustments to the definition:

where the expectation can be detailed into E A , S in this situation. For the first term, we have

where the first equation is due to the symmetric distribution between S i and S ′ i and there are overall n m permutations of l . And the second equation is due to the independence of sample ξ i,l i from the training process upon dataset S l and Lemma 4.

And we can analyze the second term in a similar way:

Therefore we can get the weak primal-dual generalization gap that:

For case b. the strong primal-dual generalization gap ϵ s gen ( A x ( S ) , A y ( S )) = ∆ s ( A x ( S ) , A y ( S )) -∆ s S ( A x ( S ) , A y ( S )) . Observing the structure that:

where the expectation can be detailed into E S , A in this situation.

For the first term,

Since S i and S ′ i are drawn from the same distribution, and there are n m permutations of l = { l 1 , ..., l m } so we can get equality ( i ) . In equality ( ii ) , we denote y ∗ S ( l ) as arg max y ∈Y F ( A x ( S ) , y ) .

While for equality ( iii ) , ξ i,l i is independent from the training process under dataset S l for each local agent respectively.

Using the property of Lipschitz continuous (see Assumption 1), we can further get:

where inequality ( a ) is a conclusion of Lemma 4.3 in [22] that ∥ y ∗ S ( l ) -y ∗ S ∥ ≤ L µ y ∥A x ( S ( l ) ) -A x ( S ) ∥ .

Combining above two inequalities, we can get:

where we get the last but one inequality from Lemma 4.

We can do a similar operation on the second counterpart since f i is also µ x -strongly convex on x .

So the overall strong primal-dual generalization gap satisfies:

where µ = min { µ x , µ y } .

E Proof in the Strongly-Convex-Strongly-Concave Case

In this section, we will prove the stability and generalization gap of our D-SGDA in the case of µ x -strongly convex and µ y -strongly concave.

E.1 Proof of Stability

Proof of Theorem 2. We use ( x t , y t ) and ( ˙ x t , ˙ y t ) to represent the output in the t -th iteration when applying D-SGDA on any arbitrary neighbouring dataset S and S ′ respectively. Since the DSGDA algorithm is symmetric [4] with respect to the dataset S = {S 1 , ..., S m } , we can assume the different samples appear in the last location in each S i without loss of generalization, i.e., S ′ = {S ′ 1 , ..., S ′ m -1 , S ′ m } where S ′ i = { ξ i, 1 , ξ i, 2 , ..., ξ i,n -1 , ξ ′ i,n } differs from S i = { ξ i, 1 , ..., ξ i,n } in the n -th data.

First concentrating on the iteration, we have:

where in the first equality we use the definition that x t = 1 m ∑ m i =1 x t i , y t = 1 m ∑ m i =1 y t i , and the last equality is due to the fact that W 1 m m = 1 m m . We use the notation P ( X , Y ) [( x y )] ∈ R d x + d y as an abbreviation of ( P X [ x ] P Y [ y ] ) ∈ R d x + d y , and P ( X , Y ) m [[ X , Y ] T ] means every column should obey P ( X , Y ) [( x y )] .

In the t -th iteration, there is a probability of C m 0 m (1 -1 n ) m -m 0 ( 1 n ) m 0 that there are m -m 0 agents not selecting the last different samples while the rest m 0 agents selecting exactly the last different samples. Without loss of generalization, we assume the first m -m 0 not selecting and the rest m 0 agents selecting the different samples respectively. So we can get:

where equality ( i ) is due to the setting that agents from 1 to m -m 0 select ξ i,j t ( i ) which is not the last different one while the rest agents from m -m 0 +1 to m select the last different samples, i.e., ξ i,n and ξ ′ i,n respectively.

Then we can decompose the term inside the projection into several parts as following:

According to Lemma 1 and the preoperty of Lipschitz continuity (see Assumption 1), we can bound the term I 1 as:

where η min t ≜ min { η x ,t , η y ,t } , η max t ≜ max { η x ,t , η y ,t } , and µ ≜ min { µ x , µ y } . For the term I 2 , it is bounded by:

For the term II 1 and II ′ 1 ,

And it is the same as the term II 2 + II ′ 2 that ∥ II 2 + II ′ 2 ∥ ≤ 2 η max t LG ∑ t -1 s =0 η max s λ t -1 -s . Taking expectation on the choice of each local sample, we can get:

where the last inequality is due to the property of binomial coefficient that C m 0 m · m 0 m = C m 0 -1 m -1 . Recursively applying the above inequality, we can get:

where the initial difference is zero that X 0 = 0 and Y 0 = 0 . And we use the last iterate as the output of Algorithm A , i.e., ( A x ( S ) , A y ( S )) = ( x T , y T ) .

For case a. when η x ,t and η y ,t are fixed, we further get:

Further when η max = η min ≜ η , the argument stability turns out to be

For case b. when η max t = 1 µ ( t +1) c and η min t = 1 µ ( t +1) , c ≤ 1 . Since 1 ± a ≤ exp { a } , we have:

Then back to inequality (9), we can simplify it as:

E.2 Proof of the Empirical Risk

Then we are going to present the optimization error of D-SGDA in the SC-SC condition.

Theorem 7 ( Optimization error ) . Under assumption 1,2,3, each local function is f i is µ x SCµ y SC. We further bound the restriction set by sup x ∈X ∥ x ∥ ≤ C x and sup y ∈Y ∥ y ∥ ≤ C y , then we have the strong primal-dual empirical risk over the dataset S on the average output in T iterations as following:

a. When learning rates η x ,t and η y ,t are fixed,

b. When learning rates are varying that η x ,t = 1 µ x ( t +1) c x and η y ,t = 1 µ y ( t +1) c y

∆ s S ( x T ave , y T ave ) ≤ 2 G ( C x + C y ) √ T + T x + T y + T max . where α ∈ { x , y } , k min = min { c x , c y } and µ = min { µ x , µ y } : T α = { G 2 2 µ α 1+ln T T c α = 1 G 2 2 µ α (1 -c α ) T cα 0 < c α < 1 ; T max = { 4 GLC λ ( C x + C y ) ln T µT k min = 1 4 GLC λ ( C x + C y ) µ (1 -k min ) T k min 0 < k min < 1 .

Proof of Theorem 7. First, we should notice that when each f i owns the property of µ x -strong convexity and µ y -strong concavity, then the linear summation should inherit the properties, i.e., F S ( x , y )= 1 m ∑ m i =1 1 n ∑ n l =1 f i ( x , y ; ξ i,l ) is µ x -strongly convex on x and µ y -strongly concave on y . First observe the update rule of x (see Algorithm 1), for any given x ∈ X :

where for the first inequality, we can recall how we transform in the proof for stability (see Appendix E.1) that x t +1 = ( X t +1 ) T 1 m m = ( WX t -η x ,t ∇ x f ( X t , Y t ; ξ t )) T 1 m m = x t -η x ,t ∇ x f ( X t , Y t ; ξ t ) T 1 m m regardless of the projection.

For the second term, we have:

̸

We decompose the third term in association with the empirical function that:

where the last inequality is due to the strong convexity of the empirical function F S on parameter x . Then we combine the inequalities (10) (11) (12) and we can get:

For case a. when learning rates η x ,t and η y ,t are fixed and we write them as η x and η y respectively. Taking summatation over the above inequality from t = 0 to t = T and using the concavity of the empirical function on y :

Then we take expectation on the randomness of the algorithm (notice that we do not take expectation on the randomness of dataset) on both sides of above inequality and choose the infinity of x on the left side since the above inequality holds for any x ∈ X :

For term I , it is decomposed as:

where inequality ( i ) is because of the fact that taking expectation on the randomness of j t is equal to the empirical result on the dataset S and inequality ( ii ) is due to Cauchy-Schwarz inequality and Lemma 2.

Then for the term II , we decompose its corresponding quadratic one:

̸

̸

where equality ( a ) owes to that taking expectation on the randomness of j t or j t ′ equals to the empirical risk and inequality ( b ) is due to the Lipschitz smoothness of each f i and Cauchy-Schwarz inequality.

̸

̸

̸

̸

Combining above inequalities of term I and II into the inequality (14) and we can summarize as:

̸

Dividing both sides by T and we can get:

And we can get the other-hand result in the same symmetric way:

Combining above two inequalities we can get the result:

Then we come to the case b. when η x ,t = 1 µ x · ( t +1) c x and η y ,t = 1 µ y · ( t +1) c y . Back to the inequality (13), we simplify it into:

Taking summation on the above inequality from t = 0 to t = T -1 and making use of the convexity of F S on the second parameter that:

Next we will take expectation on the randomness of the algorithm on both sides and proceed in the same way as we do in case a. that:

̸

Without loss of generalization, we assume η max t = η x ,t = 1 µ x ( t +1) c x . Therefore the summation ∑ t -1 s =0 η max s λ t -1 -s comes out to be ∑ t -1 s =0 λ t -1 -s µ x ( t +1) c x ≤ C λ µ x t c x . Then we have to analyse in different categories for the value of c x .

First when c x = 1 , the summation result turns out to be ∑ T -1 t =0 1 t +1 ≤ 1 + ln T ,

When 0 < c x < 1 , ∑ T -1 t =0 1 ( t +1) c x ≤ T 1 -c x 1 -c x , then we have:

It is in a similar way to get the symmetric result on the other hand and combining both results we can obtain the strong primal-dual empirical risk:

where and

E.3 Proof of Strong/Weak Primal-Dual Population Risk

In this part, we are going to prove the strong and weak primal-dual population risk of the algorithm D-SGDA. Actually this is an obvious result as a summary of above lemmas and theorems.

Proof of Theorem 3. As we introduced the population risk (see Def. 2), we decompose the population risk into generalization gap and empirical risk that:

Notice that we use the average iterate ( x T ave , y T ave ) instead of the last iterate to denote the output of D-SGDA. The first term is the averaged version of the strong primal-dual generalization gap and we will make some adjustments.

First we have the argument stability bound of D-SGDA that E A ∥ ∥ ∥ ∥ ( x T -˙ x T y T -˙ y T )∥ ∥ ∥ ∥ ≤ ϵ arg sta ( A ) , where ( x T , y T ) and ( ˙ x T , ˙ y T ) denote the T -th output when D-SGDA is executed on the neighboring dataset

respectively. So we can make use of the convexity of the norm and obtain the argument stability of averaged-version:

Then the strong primal-dual generalization gap should be G √ 2 + 2 L 2 µ 2 ϵ arg sta ( A ) following Thm. 1 when each f i is strongly convex w.r.t. x and strongly concave w.r.t. y .

At last, the strong primal-dual empirical risk E A [∆ s S ( x T ave , y T ave )] is studied in Thm. 7. So we can combine above two bounds to analyze the strong primal-dual population risk in different categories when learning rates are fixed:

And when learning rates are varying that η min t = 1 µ ( t +1) and η max t = 1 µ ( t +1) c , c = 1 , requiring 2 c > L L + µ +1 :

When c < 1 and requiring 2 c > L L + µ +1 :

F Proof in the Convex-Concave Case

In this section, we will provide corresponding proof for argument stability, optimization error and weak primal-dual population risk in the C-C condition.

F.1 Proof of Stability

Proof of Theorem 4. Analogous to Eq. (9) in the proof for SC-SC (see Appendix E.1), considering C-C a special case for µ x SCµ y SC when µ x = 0 , µ y = 0 .

Thus we can get the result:

F.2 Proof of the Empirical Risk

Similar to Thm. 7, considering C-C as a special case of SC-SC for µ x = µ y = 0 , we can get the weak primal-dual empirical risk in the following corollary, where we use the Jensen's inequality that ∆ w S ( x , y ) ≤ ∆ s S ( x , y ) .

Corollary 1. Under assumption 1,2,3 and the restriction that sup x ∈X ∥ x ∥ ≤ C x and sup y ∈Y ∥ y ∥ ≤ C y , each local function is f i is C-C. We have the weak primal-dual empirical risk over the dataset S on the average output in T iterations as following for fixed learning rates:

F.3 Proof of Weak Primal-Dual Population Risk

Proof of Theorem 5. When each local function f i is not strongly convex or strongly concave, we can not get access to the strong primal-dual generalization gap but weak primal-dual generalization gap. Following the same step in above proof that:

Analogously we have the weak primal-dual generalization gap according to Thm. 1 that:

In the case without strong convexity or strong concavity, we select the fixed learning rates and following the Thm. 7, we can bound the weak primal-dual empirical risk as:

Finally we combine above patterns and we can get the weak primal-dual population risk:

G Proof in Nonconvex-Nonconcave Case

In this section, we will provide proof for weak stability and therefore we can derive the weak primal-dual generalization gap following Thm. 1 in the NC-NC condition.

G.1 Important Lemmas

Before we present the proof for the stability bound in nonconvex-nonconcave case, we should first introduce an important lemma which describes the fact that D-SGDA will run several iterations before encountering the different samples. We extend the Lemma 3.11 in [13] and make adjustments on Lemma F.1 in [19] to fit our decentralized setting.

Lemma 5. Let S = {S 1 , ..., S m } and S ′ = {S ′ 1 , ..., S ′ m } be any arbitrary neighboring datasets, ( x t , y t ) and ( ˙ x t , ˙ y t ) represent output in t -th iteration under dataset S and S ′ respectively. We further require each local function is bounded that | f i ( x , y ; ξ ) | ≤ B, ∀ x ∈ X , y ∈ Y . Denoting

Proof. First according to the property of Lipschitz continuity (see Assumption 1):

Then we decompose the expectation by the law of total expectation:

̸

̸

̸

̸

While the event that δ t 0 = 0 means the training process has already encountered the different samples before t 0 :

̸

Combining above inequalities and we can prove the Lemma.

G.2 Proof of Stability

Proof of Theorem 6. We are under the same setting as in the proof for SC-SC (see Appendix E.1) that ( x t , y t ) and ( ˙ x t , ˙ y t ) representing the t -th output over neighboring dataset S and S ′ respectively. And we assume each local dataset S ′ i in S ′ = {S ′ 1 , ..., S ′ m } differs from S by the last sample without loss of generalization, i.e., S i = { ξ i, 1 , ..., ξ i,n } while S ′ i = { ξ i, 1 , ..., ξ i,n -1 , ξ ′ i,n } . Analogous to the decomposition equality (7), we will bound the term I 1 and I 2 without convexity or concavity. Referring to Lemma 1, we have:

where we use the same denotation that η min t = min { η x ,t , η y ,t } and η max t = max { η x ,t , η y ,t } . So we can get the similar result as the inequality (8):

Recursively applying above inequalities from t = t 0 to t = T -1 :

∥

∥

0

∥

∥

x

y

t

t

-

-

˙

x

˙

y

t

t

)∥

∥

∥

∥

When learning rates are fixed that η x ,t = η x and η y ,t = η y , then we have:

Then combining with Lemma 5, we have the following result:

where it can obtain the optimal of 2 √ 2 G 2 ( η max T n + 2 Lη max 2 T 1 -λ ) when t 0 = 0 .

(

where we use the fact that

δ

t

=

= 0

.

When learning rates are varying that η min t = 1 t +1 and η max t = 1 ( t +1) c , c ≤ 1 , then we can simplify the production by 1 ± a ≤ exp { a } :

Then back to the inequality (16), we can obtain:

Requiring c + L > 1 for convergence and combining with Lemma 5, we have the following inequality:

where for the last inequality, when t 0 = ( ( c + L -1)( 2 √ 2 G 2 T L ( c + L -1) n + 4 √ 2 G 2 LC λ T L 2 c + L -1 ) Bm n ) 1 c + L , it can obtain minimal.

Taking supremum over parameters x , y separately and over the random sample, we can get the weak stability for D-SGDA in the NC-NC condition.

H Primal Metric

In addition to the primal-dual population risk (see Def. 2) as well as the corresponding generalization gap (see Def. 3), primal measure extends the relative concept in single variable minimization problem.

The excess primal population risk is defined as ∆ ex ( x )=sup y ′ ∈Y F ( x , y ′ ) -inf x ′ ∈X sup y ′ ∈Y F ( x ′ , y ′ ) ; and the excess primal empirical risk is ∆ ex S ( x ) = sup y ′ ∈Y F S ( x , y ′ ) -inf x ′ ∈X sup y ′ ∈Y F S ( x ′ , y ′ ) , for a randomized model ( x , y ) .

There are two ways to define the corresponding generalization gap. One is to directly subtract the excess primal empirical risk from the excess primal population risk, defined as the excess primal generalization gap: ϵ ex gen ( x ) = ∆ ex ( x ) -∆ ex S ( x ) (called primal gap in [30]). Another one is to neglect the difference between the saddle point of F and F S , defined as the primal generalization gap: ϵ pr gen ( x ) = sup y ′ ∈Y F ( x , y ′ ) -sup y ′ ∈Y F S ( x , y ′ ) .

Remark 11. Our definition of strong primal-dual population risk (see Def. 2) has included the expectation inside for assistance with the definition of weak primal-dual population risk (see Def. 2). Although the definitions in [19] hold different forms, which do not contain the expectation inside, while our theoretical result aims at the same value.

Ozdaglar et al. [30] points out that the excess primal generalization gap ϵ ex gen can act as a better metric to characterize the generalizability in the nonconvex condition. It is our limitation that we do not

calculate the corresponding excess primal generalization gap and population risk for nonconvex case in our paper.

While we omit the primal generalization gap ϵ pr gen and excess primal population risk ∆ ex under C-C condition in the main text, for it can be derived from the corresponding proof of strong primal-dual risk. And we will illustrate them in the following as a corollary.

Corollary 2. For an ϵ -argument stable decentralized algorithm A , under Assumption 1, 2, when each f i is µ y -strongly concave on the second parameter, we have the primal generalization gap: E A , S [ ϵ pr gen ( A x ( S ))] ≤ G √ 1 + L 2 µ 2 y ϵ .

Proof of Corollary 2. ϵ pr gen ( A x ( S )) is exactly the first counterpart of the strong primal-dual generalization gap ϵ s gen ( A x ( S ) , A y ( S )) in Eq. (6). And referring to the proof for case b. in Thm. 1 (see Appendix D), we can get the result as above.

Corollary 3. Under Assumption 1,2,3, when each f i is µ x SCµ y SC, we have the excess primal population risk as follows, where η max t ≜ max { η x ,t , η y ,t } , η min t ≜ min { η x ,t , η y ,t } , µ = min { µ x , µ y } , and ( x T ave , y T ave ) is defined in Eq. (3) :

a. for fixed learning rates,

b. for decaying learning rates that η min t = 1 µ ( t +1) and η max t = 1 µ ( t +1) c with c ≤ 1 and 2 c ≥ L L + µ +1 ,

̸

̸

̸

Proof of Corollary 3. Firstly we already know the argument stability bound for D-SGDA (denoted as A ) on the last iterate. Then the averaged output follows due to the convexity of the norm:

Then we have the primal generalization gap according to Corollary 2 that E A , S [ ϵ pr gen ( x T ave )] ≤ G √ 1 + L 2 µ 2 y ϵ arg sta ( A ) .

We decompose the primal population risk for the averaged output ( x T ave , y T ave ) as follows:

where the inequality is due to: inf x ′ ∈X F ( x ′ , A y ( S )) ≤ inf x ′ ∈X sup y ′ ∈Y F ( x ′ , y ′ ) .

The first term is the primal generalization gap. And notice that the third term is analogous to the contrast side of the primal generalization gap. So both of them can be bounded by ϵ pr gen ( x T ave ) .

While the second term is the strong primal-dual empirical risk referring to the proof of Thm. 7 (see Appendix E.2).

Overall we can get the excess primal population risk, almost the same with the strong primal-dual population risk for SC-SC case (see Appendix E.3) except for a √ 2 -times factor in the argument stability error. And our bound analysis for the excess primal population risk is consistent with strong primal-dual population risk (see Remark 6 below Thm. 3).

I Additional Experiments

In this paper, we include two experiments including solving the AUC problem on svmguide and w5a by the decentralized SGD to verify the conclusions for the Convex-Concave case and the generative adversarial network training for the Nonconvex-Nonconcave Case on MNIST .

I.1 General setup.

Different from the stability and generalization analysis of the way in [19], we need to deal with learning in a decentralized manner. In our experiments, we denote the total number of clients as N c and S = { S 1 , S 2 , · · · , S N c } as the set of samples, where S i represents the observations stored in the i -th client. And we let N S i denote the size of S i . We follow the same experimental setting as outlined in [13, 19] to build a neighboring/perturbing dataset S ′ , which is constructed by individually changing one observation on each node. That is to say, for each S i , S ′ i is constructed by randomly changing one element in S i . Then, we deploy the totally same sub-model on each client and initialize them to the same starting point. Then, each sub-model is trained on its local data. After each iteration, each sub-model is communicated with some other clients as per a predefined communication topology. To evaluate the distance between two models trained on S and S ′ , after finishing training, we obtain an ensemble model by averaging all sub-models collected from all clients.

I.2 Detailed implementations.

For the AUC problem, we get two model squences { ( w,v ) } and { ( w ′ , v ′ ) } . Then, we calculate the Euclidean distance ∆ = ( || w -w ′ || 2 2 + || v -v ′ || 2 2 ) 1 / 2 . For training each model on each client, the algorithm we used in our experiment is SOLAM [41], which is the SGDA designed for the minimax AUCproblem. We repeat the experiments 10 times to report the average results as well as the standard deviation.

For the generative adversarial learning problem, we just take the vanilla GAN structure, of which the generator and the discriminator comprise 4 fully connected layers, respectively. The leaky ReLU is taken before the output layer. Following [19], we ignore all forms of regularization such as the weight decay or dropout, and data augmentation tricks. We take 3 different seeds and 3 different ways to construct S ′ , which means changing different observations (total 9 runs). To evaluate the model distance, we also take the Euclidean distance between the generator and discriminator separately.

Our implementation is highly based on the two source codes 23 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper establishes the algorithmic-stability analysis of the decentralized stochastic gradient descent-ascent (D-SGDA) algorithm which is commonly used for min-max problems in distributed settings. This is turn leads to generalization bounds of D-SGDA for various assumptions on convexity-concavity of the objective. In particular, the paper focuses on two notions of generalization for min-max problems, namely weak and strong primal-dual generalization error. The bounds are expressed in terms of sample-size, training horizon and certain information about the mixing matrix such as the spectral gap. The flavor of results are closely-related to the recent analyses in literature that obtain algorithmic-stability bounds for decentralized gradient descent, however this work extend these to min-max scenarios.  The bounds (as expressed in the theorems) take complicated forms in term of the problem parameters, however the authors provide simplified interpretations of them in the remarks. In particular, the derived bounds strongly depend on the learning rate and spectral gap of mixing matrix in order to be meaningful. In particular, for strongly convex-strongly concave objectives, the generalization bounds are of order $O(\\frac{\\eta}{1-\\lambda} + \\frac{1}{n})$ and for the convex-concave case the bounds are $O(\\frac{\\eta T}{n} + \\frac{\\eta^2 T}{1-\\lambda})$ for learning-rate $\\eta$, sample size $n$, training time $T$ and the spectral gap $1-\\lambda.$\n\nSTRENGTHS:\nThe studied problem is new as it is the first work to obtain generalization bounds for D-SGDA. Although the generalization error of decentralized gradient descent and the gradient-descent algorithms were studied separately in existing works, this paper gives a unifying analysis. The method is largely based on the work of Hardt et al and relevant works on the gradient descent-ascent literature. \n\nThe derived bounds are in rather simple forms and might be informative on the role of sample-size or some other parameters such as the strong-convexity parameter on the generalization. \n\nThe paper is generally clear and easy to read although some aspects of the presentation can be improved as I will discuss in the next section.\n\nSeveral experiments on different topologies presented in the paper seem to back the general claim on the role of spectral-gap and connectivity on generalization performance. I will discuss this more in the next section.\n\nWEAKNESSES:\nWeaknesses and some questions:\n\n1 - One of my main concerns is that the generalization bounds derived in this paper are essentially vacuous for a large set of commonly used learning-rates or training time choices. Even for strongly convex-strongly concave setup, the generalization bounds are meaningful only when considering time-decaying learning rates. The authors do not compare their results to the state-of-the-art results for the generalization of ordinary gradient descent-ascent with fixed learning rate, therefore it is impossible to discuss the optimality of the obtained rates. This is especially important due to the role of step-size on the optimization and convergence of D-SGDA. The derived bounds also appear to be very sub-optimal for convex-concave objectives: the bounds are increasing in $T$ (because of the $\\eta^2 T/(1-\\lambda)$ term), therefore common early stopping choices such as $T\\approx \\sqrt{n}$ lead to a vacuous generalization bound unless the learning rate is decaying.      \n\n2 - The results in Table 1 are not very informative: there is no comparison with Decentralized SGD (D-SGD) and SGDA methods and the choice of step-size for each row is also not specified. It will be helpful if these comparisons are added.      \n\n3 - In line 47-49 the authors assert that *\"Note that even for the decentralized minimization problem, the generalization and\n stability of decentralized SGD are adversely affected by an extra non-vanishing term [31], and the stability usually suffers from a constant term λ^2\"*. While the current results on the stability of D-SGD indeed suffer from this additional term, these are just upper bounds therefore the conclusions on the role of topology/mixing matrix are not accurate until the optimality of these bounds are proved. \n\nIn the conclusions section the authors state *\"Our theoretical results show that a decentralized\nstructure does not destroy the stability and generalization of D-SGDA\"* . This is wrong as explained above and even in contradiction with the generalization rates obtained throughout the paper, as the generalization error rates is severely affected by decentralization.  \n\n4- A precise discussion on the optimization guarantees of DSGDA with the given learning rate selections seems lacking in the paper. This will be beneficial for understanding final test error rates. \n\n5 - Some parts of the paper can be rephrased: \n\n- The descriptions in the \"Related Works\" section are wrong in some places: The work of Bousquete and Elisseeff  established the connection between generalization and stability. Hardt et al. just showed that the output of SGD satisfies the stability notion. I think the lines 87-89 need to be edited. \n\n- The sentence in line 80 \"Huang [15]....\" is also not clear and I think it needs to be rephrased. \n\n- The authors refer to (Sun et al. 2021) and (Zhu et al. 2022) for previous works on generalization of decentralized methods. Some missing and relevant works that study generalization of DGD are the following:\n\n  *\"Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent\"* by Richards et al. 2020.\n\n  *\"On Generalization of Decentralized Learning with Separable Data\"* by Taheri et al. 2023.\n\n  *\"High-dimensional inference over networks: Linear convergence and statistical guarantees.\"* by Sun et al 2022.\n\n\n- In the abstract (lines 11-13), the purpose of the sentence is not clear, since the bounds essentially are in terms of sample-size, learning rate and iterations. \n\n- The sentence in line 35 is not clear. Can the authors please rephrase this sentence?\n\n- Line 150: the sentence starting with \".And therefore\" needs to be rephrased. Also see Line 235 and the sentence \".So...\".\n\nQUESTIONS:\nPlease see the last section for questions and suggestions.\n\nLIMITATIONS:\nThe limitations of the work are clear and I see no potential negative societal impact related to this work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
this paper provides an analysis of the stability and generalization of decentralized GDA algorithms in SCSC, CC, and NCNC settings. the main results indicate that a decentralized setting GDA has similar error bounds as centralized settings. numerical experiments on AUC and GAN problems are reported.

STRENGTHS:
1. the analysis looks solid to me and the results build the bridge between GDA and DGDA for generalization bound.
2. the numerical experiments are strong and convinsing.

WEAKNESSES:
1. the figures are too small, need to zoom in to read the content.

QUESTIONS:
1. what is the main technical difficulty for DGDA compared with the centralized case?

LIMITATIONS:
see weakness
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the primal-dual generalization gap bound of the decentralized stochastic gradient descent ascent (D-SGDA) algorithm. The authors start with a general decentralized minimax stochastic optimization problem, and its empirical counterpart calculated by the training dataset consisting of local samples. They model the network as an undirected graph with symmetric double stochastic matrix W. They assume Lipschitz-continuity and Lipschitz-smoothness on the local functions for the algorithm. They define weak and strong population risks, weak and strong generalization gaps -between the training dataset and original unknown distribution-, and algorithmic stability. The main purpose of the paper is the connection between and e-argument stable algorithm to its generalization gaps, both weak and strong. The authors show that the weak and strong -for strong convexity- strong concavity satisfied- generalization gaps are bounded by the argument stability error of an e-argument stable algorithm. The authors further show the boundedness of the argument and weak stability errors for strongly convex-strongly concave, convex-concave, and nonconvex-nonconcave settings. They analyze the generalization gaps for various parameters, including network topology, learning rates, and the number of nodes to show that the results are bounded as proposed.\n\nSTRENGTHS:\nThe paper aims to investigate the connection between argument and weak stability with strong and weak generalization gaps for a training dataset for D-SGDA algorithm. The authors claim to propose this connection not only for D-SGDA algorithm, but for decentralized minimax algorithms in general. Their generalization and stability bounds for decentralized SGDA extent the stability and generalization for vanilla SGDA, so it can be said that the study is original. The overall quality of the paper is well, the problem is well-defined in a clear way and the approach is shown clearly. The results are significant at an acceptable level, since the study proposes the decentralized structure does not violate stability and generalization of SGDA algorithms, and decentralized minimax algorithms in general, as the bounds are shown by the authors.\n\nWEAKNESSES:\nIn general, the paper is well-structured, but I believe a few things could be more elaborate. For example:\n\n1. The mixing matrix and its purpose, how to select λ, what the W corresponds for a network topology could be explained better.\n\n2. Proof of Theorem 1 in Appendix could show some intermediate steps where 𝑆(𝑙) is included.\n\n3. Typos: Theorem 2, part b. wording, Ref. 6, sizes of parentheses in equations\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nAs the authors state, the analysis requires Lipschitz continuity and smoothness, and the stability and generalization analysis of D-SGDA for heterogeneous data distribution is missing. How would the analysis on stability and generalization vary for a directed network? In this case, W wouldn’t be necessarily symmetric, how would that affect the analysis?", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studied the generalization analysis of the decentralized stochastic gradient descent ascent algorithm through the lens of argument stability for solving minimax problems. Strong/weak primal-dual population risks are established for both convex-concave, strongly convex-strongly concave and nonconvex-nonconcave cases.\n\nSTRENGTHS:\n1. The paper provided a comprehensive stability and generalization analysis of D-SGDA for solving the minimax problem. Their results imply that the decentralized structure does not destroy the stability and generalization of SGDA.\n\n2. The impact of different topologies of the decentralized structure on the generalization bound is observed, which is very interesting.\n\n3. Experiments validate the theoretical results.\n\nWEAKNESSES:\n1. The results in Theorem 1 might be improved. Specifically, [1] established the connection between on-average argument stability and the weak primal-dual generalization gap for Markov chain SGDA (Here, SGDA is a special case of Markov chain SGDA) only with the Lipschitz assumption. Also, [2] provided this connection for Lipschitz losses. However, Theorem 1 requires the loss to satisfy both Lipschitz continuous and smooth conditions. For strong primal-dual generalization gap, [2] established the connection under the assumption $f$ is $\\mu_y$ strongly-concave, while Theorem 1 assumes that $f$ is $\\mu_x$ strongly-convex $\\mu_y$ strongly-concave. In addition, $\\mu$ is not defined in the theorem.\n\n\n2. The results for both weak primal-dual and strong primal-dual population risks depend on $T$, $\\eta$ and $n$. The authors might give some discussion on the choices of $\\eta$ and $T$, and establish the explicit population rates as provided in [1] and [2], and compare with their results. \n\n[1] Wang, P., Lei, Y., Ying, Y., and Zhou, D. X. (2022). Stability and generalization for markov chain stochastic gradient methods. Advances in Neural Information Processing Systems, 35, 37735-37748.\n\n[2] Lei, Y., Yang, Z., Yang, T., and Ying, Y.(2021). Stability and generalization of stochastic gradient methods for minimax problems. In International Conference on Machine Learning, pages 6175–6186.\n\n3. The experiment setup is unclear, I would suggest the authors add some details in the appendix.\n\nQUESTIONS:\n1. Can the stability and generalization results be generalized to the directed graph?\n\n2. As mentioned in Weakness 2, could the authors provide some discussions or corollaries for discussing the choices of stepsize $\\eta$ and iteration number $T$ to establish the explicit population rates? \n\n3. A small question: in table 2, for fully connected graph, why $\\frac{1}{1-\\lambda}$=0?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyzes the generalization of the Decentralized Stochastic Gradient Descent Ascent (D-SGDA) Algorithm for min-max problem using the algorithm stability framework. Specifically, the paper analyzes the weak and strong primal-dual generalization gap under both convex-concave (C-C) and nonconvex-nonconcave (NC-NC) settings.\n\nThe key result is that the decentralized structure with undirected graph does not harm the generalization upper bound of D-SGDA.\nEmpirical experiments are performed to further validate the theoretical findings.\n\nSTRENGTHS:\n1. The paper is well written and easy to follow.\n\n2. The paper studies an important problem of the generalization of the DSGDA algorithm for minmax problems in the decentralized settings.\n\nWEAKNESSES:\n1. Technical novelty is limited.\n\nThe algorithm stability for decentralized SGD algorithm has been analyzed in prior work [31]. And the algorithm stability for the SGDA algorithm for minmax problem has also been analyzed in prior work [18]. The techniques used in this paper mainly combine the two papers [18, 31].\n\n2. The tightness of the generalization upper bound obtained in this paper is not discussed.\n\n3. Some results are not clearly described. See **Questions**.\n\n\n### Minor:\n\n1. Unify the notations for projection operator. e.g. Algorithm 1 uses $P_{\\cal X}$, while line 564-567 in the appendix uses $Proj_{\\cal X}$.\n\n2. Line 109, \"differential\" -> \"differentiable\".\n\nQUESTIONS:\n1. In Table 1, why only the NC-NC case has a bound depending on $m$, the number of agents while other results do not depend on $m$?\n\n2. Please also include results on the optimization error and population risk in Table 1 or in the appendix.\n\n3. One theoretical result shows that the decentralized structure does not harm stability and generalization of D-SGDA. This is different from the conclusion of [31], which shows the decentralized SGD are adversely affected by an extra non-vanishing term compared to SGD. What leads to the difference in the conclusions? Is it because of specific assumptions on the structure of the undirected graph, or improved / tighter analysis? This needs to be discussed carefully since it is the main contribution of this work.\n\n4. In Theorem 6, what are the step size choices to achieve the corresponding bound? This needs to be stated as in other theorems.\n\nLIMITATIONS:\nThe authors have discussed the limitations in Section 6, conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}