Title: A Near-Optimal Algorithm for Decentralized Convex-Concave Finite-Sum Minimax Optimization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Minimax optimization

decentralized optimization

stochastic algorithm

variance reduction
Abstract: In this paper, we study the distributed convex-concave finite-sum minimax optimization over the network, and a decentralized variance-reduced optimistic gradient method with stochastic mini-batch sizes (DIVERSE) is proposed. For the strongly-convex-strongly-concave objective, it is shown that DIVERSE can achieve a linear convergence rate that depends on the global smoothness parameters, yielding sharper computation and communication complexity bounds than existing results. Furthermore, we also establish the lower complexity bounds, which show that our upper bounds are optimal up to a logarithmic factor in terms of the local incremental first-order oracle calls, the computation rounds, and the communication rounds. Numerical experiments demonstrate that our algorithm outperforms existing methods in practice.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses decentralized convex-concave finite-sum min-max optimization over networks and proposes the DIVERSE algorithm (Decentralized Variance-Reduced Optimistic Gradient with Stochastic mini-batch sizes). While the theoretical analysis is solid and the claimed complexity results improve upon existing works under strongly-convex-strongly-concave settings, the submission falls short in several key areas. In particular, the algorithmic novelty over prior methods is not sufficiently emphasized or clearly explained.  The experimental section is limited and does not adequately support the theoretical claims or demonstrate practical advantages.  Furthermore, the overall presentation lacks clarity, especially in conveying intuition and key technical contributions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n1. Near-Optimal Complexity with Global Smoothness Dependence\n- Achieves tight computation complexity $\\tilde{O}\\left(m n+\\frac{\\min \\{m n L, \\sqrt{m n} \\bar{L}\\}}{\\mu} \\log \\frac{1}{\\epsilon}\\right)$\nand communication complexity $\\tilde{\\mathrm{O}}\\left(\\sqrt{\\chi} \\cdot \\frac{L}{\\mu} \\log \\frac{1}{\\epsilon}\\right)$.  First decentralized minimax method whose complexity depends solely on global smoothness parameters ( $L, \\bar{L}$ ) instead of conservative local bounds ( $L_l, \\bar{L}_l$ ).Matches lower bounds (Theorems 4.2-4.4), outperforming prior art by orders of magnitude under data heterogeneity (e.g., $10 \\times$ speedup when $\\bar{L} / L=\\Theta(\\sqrt{m})$ in Ex. A.1).\n2. Breakthrough Sampling Strategy: Stochastic Mini-Batches\n- Bernoulli-based batch sizing ( $b_{i, j}^k \\sim \\operatorname{Bernoulli}(q)$ ) enables:\n- Node partial participation: Weak nodes can skip iterations dynamically.\n- No synchronized batch sizes: Eliminates computation bottlenecks in heterogeneous systems.\n- Reduces expected per-iteration cost to $\\mathrm{O}(m n p+(1-p) b)$ vs. $\\mathrm{O}(m n)$ in existing methods.\n\nWeaknesses\n- Variance reduction (Section 2.2) and multi-consensus (Algorithm 1) are repurposed techniques \n- Lower bounds (Theorems 4.2-4.4) extend single-machine results vs. introducing new proof techniques\n- Insufficient justification for global smoothness focus (no real-data examples)\n- Parameter choices $p=\\frac{\\bar{L}}{8 L} \\max  \\left( \\frac{\\mu}{L}, \\frac{1}{\\sqrt{m n}} \\right)$ lack intuition\n- Table 1 formatting obscures complexity comparisons\n\nQUESTIONS:\n1. Insufficient Clarification of Algorithmic Novelty​\nThe core contribution—stochastic mini-batch sampling—lacks thorough motivation. The authors should explicitly clarify:\n(a) How the Bernoulli sampling scheme overcomes the \"identical mini-batch size\" limitation (Lines 46–47) and enables partial participation across nodes;\n(b) How this design shifts complexity dependence from local smoothness parameters s ( $L_l, \\bar{L}_l$ ) to global ones ( $L, \\bar{L}$ ), with direct comparisons to methods like OADSVI [37]. This should be emphasized in Sections 1 and 3.1.\n​\n2.Inadequate Experimental Validation​\nExperiments are limited to robust linear regression (quadratic costs, Eq. 52) on a ring network. To demonstrate generality:\n(a) Include complex minimax benchmarks (e.g., GAN training or adversarial learning);\n(b) Test under data-heterogeneity scenarios (Example A.1) to validate superiority over GT-EG/MC-SVRE in practical settings where $\\bar{L} \\gg L$. \n\n3. Ambiguous Parameter Selection Rationale\nCritical parameters in Algorithm 2 $(p, b, \\alpha)$ are presented without derivation (Lemma 3.1). The authors must:\n(a) Justify design choices (e.g., $p=\\frac{\\bar{L}}{8 L} \\max  \\left( \\frac{\\mu}{L}, \\frac{1}{\\sqrt{m n}} \\right)$ ) in the main text or appendix;\n(b) Discuss practical implications of the $\\bar{L} \\leq \\sqrt{m n} L$ vs. $\\bar{L}>\\sqrt{m n} L$ cases on parameter tuning.\n\n4. Experiments Overlook Theoretical Highlights\nExperiments omit key innovations:\n(a) Performance under varying $\\frac{\\bar{L}}{L}$ ratios (to showcase global-smoothness dependency);\n(b) Communication rounds across topologies with different $\\chi$ values (Fig. 1's ring network inadequately validates O $\\left(\\frac{\\sqrt{x} L}{\\mu}\\right)$ complexity).\n(c) The authors fail to provide additional datasets, case studies, and code to support reproducibility and further validation of their approach.\n\n5. Writing and Presentation Issues\n(a) Definition gaps: Asymptotic notation $\\Omega(\\cdot)$ is undefined in Theorem 4.3;\n(b) Formatting inconsistencies: Indentation in Algorithms 1-2 and line breaks in Table 1 (e.g., DIVERSE computation rounds) require alignment;\n(c) Motivation clarity: The introduction inadequately explains the practical significance of global-smoothness dependency (e.g., acceleration in $\\frac{\\bar{L}}{L} \\gg 1$ regimes).\n\nLIMITATIONS:\nAcknowledged but not mitigated: The SC-SC assumption (Sec 2.1) is clearly stated but lacks:\n- Extension plans to nonconvex settings (e.g., GANs)\n- Discussion of relaxations (e.g., weak convexity)", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper designs a variance-reduced optimistic gradient method with stochastic mini-batch sizes for distributed convex-concave finite-sum minimax problem. And the method could achieve a linear convergence rate depends on global smoothness(not local smoothness) in strongly-convex-strongly-concave situation. In addition, experiments show that performance of proposed method is better than existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1 The paper makes a reasonable adjustment to the assumptions, allowing the convergence rate to be analyzed in terms of both the upper and lower bounds with respect to the global smoothness parameter and the mean-squared smoothness parameter. This differs significantly from existing work that relies on local smoothness assumptions, which may be less realistic in heterogeneous data scenarios.\n\n2 The structure of the paper is clear, and both the algorithmic design and the theoretical contributions are well explained.\n\n3 The experiments include a wide range of baselines under comparable settings, and the reported results are convincing.\n\nWeaknesses\n\n1 The experimental section is relatively limited in terms of dataset diversity, which may restrict the generalizability of the results. Additionally, the absence of commonly reported evaluation metrics such as accuracy makes it difficult to comprehensively assess the practical effectiveness of the proposed method. As a result, the experimental validation appears somewhat narrow and could benefit from more thorough benchmarking.\n\n2 The main algorithmic contributions seem to lie in the incorporation of variance-reduction techniques and a modification to the minibatch size used for local gradient estimators, which is fixed in prior methods. While these changes are reasonable, they appear to be incremental improvements rather than fundamental innovations. The overall level of novelty is therefore somewhat limited.\n\nQUESTIONS:\n1 Could you share additional experimental results, such as accuracy with respect to LIFO, communication rounds, and computation rounds for the four datasets already included? If more datasets have been evaluated, including those results would further strengthen the empirical section and support the generality of the proposed method.\n\n2 In the convergence rate analysis, what specific challenges arise from the change in assumptions, and how are they addressed in your analysis? While I believe the modification of the assumptions is meaningful, it is unclear to me whether it introduces significant technical complications or fundamentally alters the existing analytical framework.\n\n3 I have some concerns regarding the novelty of the algorithm. The integration of variance-reduction techniques alone may not substantially deviate from existing methods in terms of analysis. Could you elaborate on the statement at the end of Section 3.1, where you mention that the change to the mini-batch size for the local gradient estimator is one of the key differences from existing work? Specifically, how is this change handled in the convergence rate analysis, and does it pose any unique analytical challenges?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the decentralized finite-sum minimax optimization problem under the strongly-convex-strongly-concave setting. The authors propose a DIVERSE algorithm, which achieves near-optimal complexities for local oracle calls, computation rounds, and communication rounds. The primary contribution is that the derived complexity bounds depend on the *global* smoothness parameters of the objective function, rather than the *local* (and often much larger) smoothness parameters that prior works rely on. The authors provide a rigorous theoretical analysis, establishing both upper bounds for their algorithm and nearly matching lower bounds for a broad class of decentralized first-order methods. The theoretical findings are validated by numerical experiments on a robust linear regression problem, where DIVERSE outperforms existing state-of-the-art algorithms.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- The paper makes a contribution by providing a sharper bound in decentralized minimax optimization.\n    \n\n**Weaknesses:**\n\n- This paper does not provide insight into the algorithm design, rendering it to seem like a simple combination of several existing methods.\n    \n- Even though the theoretical analysis is correct and solid, it is better to highlight the key steps in the proof to provide more information to the audience. For example, the idea of establishing the lower bound.\n    \n- Some technical claims have not been fully justified. See **Question.**\n    \n- The experiments do not report error bars or any other measure of statistical significance.\n\nQUESTIONS:\n1. What do we use a stochastic gradient batch-size? The authors claim, “It is worth noting that existing decentralized minimax optimization methods require identical mini-batch size for all the nodes when constructing the local gradient estimator, which is sample inefficient.” Authors need to justify it.\n    \n2. What are the key techniques to enable a sharper bound?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

comments: SUMMARY:
In this paper, the authors consider decentralized finite-sum strongly convex–strongly concave saddle-point problems. To solve this problem, they propose a new variance-reduced algorithm called __DIVERSE__. The algorithm is a non-trivial combination of OGDA and Snapshot Gradient Tracking. They provide convergence guarantees along with new theoretical lower bounds, which match their upper bounds. To support their theoretical findings, they present experimental results demonstrating that the proposed method outperforms previous approaches.

STRENGTHS AND WEAKNESSES:
__Strengths:__

1. Well-written paper. 

2. Strong theoretical findings: novel lower bounds and a new algorithm. 

__Weaknesses:__

1. The complexity of DIVERSE has an additional logarithmic factor compared to OADSVI. 

2. See the Questions section.

QUESTIONS:
__Questions:__

1.On page 28, right after line 669, there is an equation:
$$\sum_{i=0}^m ||g_i(z^{0})||^2 = \sum_{i=0}^m ||g_i(z^{0}) - g_i(z^{\star})||^2.$$
However, this holds only if each $g_i(z^{\star}) = 0$. I believe this is a mistake in your proof. Could you comment on this observation?

__Recommendations:__

1.It would be better to cite the following two papers on the snapshot technique for gradient tracking:
https://arxiv.org/abs/2110.05282,
https://arxiv.org/abs/2212.05273.

2.To be more precise, DIVERSE is not based exactly on OGDA but rather on the Operator Extrapolation method (https://arxiv.org/abs/2011.02987), which shares similarities in the convergence analysis. Citing this work could help clarify the construction of the method.

3.Please introduce the definition of $\Delta^k$ in the main part of the paper.


I may consider increasing my score if all of my questions are addressed.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-09