Communication-Efficient Decentralized Optimization via Double-Communication Symmetric ADMM
Authors: Jinrui Huang
Xueqin Wang
Dong Liu
Jingguo Lan
Runxiong Wu
Venue: ICLR 2026 Poster (2026)
Keywords: Decentralized Optimization
Symmetric ADMM
Multi-Communication
Primary area: optimization

Abstract: This paper focuses on decentralized composite optimization over networks without a central coordinator. We propose a novel decentralized Symmetric ADMM algorithm that incorporates multiple communication rounds within each iteration, derived from a new constraint formulation that enables information exchange beyond immediate neighbors. While increasing per-iteration communication, our approach significantly reduces the total number of iterations and overall communication cost. We further design optimal communication rules that minimize the number of rounds and variables transmitted per iteration. The proposed algorithms are shown to achieve linear convergence under standard assumptions. Extensive experiments on regression and classification tasks validate the theoretical results and demonstrate superior performance compared to existing decentralized optimization methods.

Published as a conference paper at ICLR 2026
COMMUNICATION- DEEEDECCCCTTTFFFFEEEEEEEEEEEEEAD MIZATION VIA DOUBLE-COMMUNICATION SYMMETRIC ADMM
DECENTRALIZED OPTI-
Jinrui Huang', Xueqin Wang", Dong Liu Jingguo Lan', Runxiong Wu²*
Department of Statistics and Finance, University of Science and Technology of China
2 Department of Industrial & Systems Engineering, University of -isconsin-Madison 3 Department of Operations Research and Financial Engineering, Princeton University
ABSTRACT
This paper focuses on decentralized composite optimization over networks with- out a central coordinator. We propose novel decentralized symmetric ADMM algorithm that incorporates multiple communication rounds within each iteration, derived from new constraint formulation that enables information exchange be- yond immediate neighbors. While increasing per-iteration communication, our approach significantly reduces the total number of iterations and overall com- munication cost. We further design optimal communication rules that minimize the number of rounds and variables transmitted per iteration. The proposed al- gorithms are shown to achieve linear convergence under standard and relatively weak assumptions (e.g., metric subregularity). Extensive experiments regression and classification tasks validate the theoretical results and demonstrate superior performance compared to existing decentralized optimization methods.
INTRODUCTION
The increasing size and complexity of modern machine learning models, combined with the explosive growth of data from sources such mobile devices, sensors, and edge computing platforms, has driven the demand for scalable and privacy-preserving optimization techniques. Among these, decentralized optimization has emerged a powerful approach, particularly when centralized computation is impractical due to concerns of scalability, robustness, data privacy, communicational infeasibility and network connectivity.
Unlike centralized distributed optimization, which still depends on central server to coordinate updates, decentralized optimization involves multiple agents collaboratively solving a global problem by performing local computations and exchanging information only with their neighbors. These agents operate over a connected network typicallyl modeled as a central coordinator. This architecture makes decentralized optimization especially attractive for applications in multiple fields like sensor networks and large-scale machine learning.
wwihout the need for
In decentralized optimization, central challenge lies in reducing the time cost of both local compu- tation and inter-node communication. While existing algorithms differ in their local update strategies, most follow common structural pattern: each iteration is followed by single round of communi- cation. This convention has seldom been challenged, primarily due to the concern that introducing multiple communication rounds per iteration would increase the overall communicational cost. Prior attempts to incorporate fixed multiple communication rounds, such as those gradient- based methods in (Jakovetić et al., 2014; Berahas et al., 2019; Ye et al., 2023; Li al., 2018), achieve this by multi- consensus-rrpeateedde mixing local variables through communication. However, these methods have not demonstrated practical reductions in the total number of communication rounds. As result, the potential for achieving a net communication reduction through non-adaptive nulti-communication algorithms remains largely unexplored.
"Corresponding Author: Runxiong Wu (rwu24 60wi sc. edu)

---

Published as a conference paper at ICLR 2026
Although multi-consensun schemes involve more frequent averaging steps, they do not neces- sarily reduce the overall communication cost. A potential reason is that these repeated consen- sus/communication steps primarily accelerate agreement among local variables, but offer limited improvement to the quality of each iteration. This observation motivates the need for a more principled and dedicated framework for multi-round communication, rather than simply applying multi-consensus in iteration.
In this paper, we explore incorporating multiple communication rounds within a single iteration in an approach orthogonal to existing multi-consensus schemes inspired by this observation. Rather than directly applying multiple mixing steps, we develop our algorithms by introducing linear constraints tailored for ADMM, which naturally embed multi-round communication into each iteration. To further enhance performance, we adopt Symmetric ADMM (He al., 2016) framework to accelerate convergence. Although this design increases the per-iteration communication cost, it enables significantly faster convergence, leading to substantial reduction in the total number of iterations, computations, and overall communication required for convergence.
Our contributions are summarized as follows:
We propose DS-ADMM, a novel Symmetric ADMM-based decentralized composite opti- mization framework that incorporates multiple communication rounds into each iteration, leading to more efficient decentralized training.
We derive optimal communication rules in the proposed algorithms which successfully minimize the communication rounds and the amount of information transmitted per iteration.
We provide rigorous theoretical guarantees for the proposed method, including convergence and convergence rate analysis under standard and strong convexity assumptions.
We conduct extensive numerical experiments that validate our theoretical results and demon- strate the superior performance of our method compared to state-of-the-ar algorithms in decentralized composite optimization by reducing both computational and communicational cost.
To the best of our knowledge, such a SYMMetric-DDMM construction with a fixed multi-round schedule and the associated theory has not been explored in prior decentralized optimization literature, and it successfully reduces the overall communication cost. Our results open a promising direction for decentralized optimization by revealing new trade-off between the number of per-iteration communication rounds and overall convergence speed.
1.1 RELATED WORK
Decentralized Optimization. Decentralized optimization has gained significant attention in large- scale machine learning, particularly in scenarios where data is distributed across multiple agents or devices without central coordinator. common strategy in these methods the use of stochastic or deterministic mixing matrices to perform local averaging of variables across neighboring nodes, facilitating global consensus through communication. Early methods such as Decentralized Gradient Descent (DGD) (Nedic & Ozdaglar, 2009; Jakovetić et al., 2012; Yuan et al., 2016) directly extend classical gradient-based algorithms to networked environments, laying the groundwork for later developments. However, DGD suffers from slow convergence and sensitivity to step size. To address these shortcomings, a line of gradient-tracking based algorithms has been developed, including EXTRA and PG-EXTRA (Shi et al., 2015a:b), NIDS (Li et al., 2019), SONATA (Sun al., 2022), and (Qu & Li, 2018), which incorporate correction terms to estimate the average gradient across the network. Several of these algorithms, including (Shi al., 2015b; Li et al., 2019; Sun al.. 2022) are designed for decentralized composite optimization problems with mmooth-nonsmood structure by embedding proximal gradient steps. And the unifying analysis in (Xu al., 2021) shows that broad family of iethhdd--iclludng (Shi et al., 2015b; Li et al., 2019; Qu & Li, 2018)~can be understood through a single theoretical framework and established linear convergence under strong convexity assumptions. In addition to improving per-iteration convergence rates, many recent methods incorporate acceleration techniques such as Nesterov acceleration, leading to further improvements in both computation and communication complexity including (Jakovetić et al., 2014; Li et al., 2018; Qu & Li, 2020: Ye et al., 2023). In parallel, a different family of methods focuses on dual-based formulations. These include (Uribe et al., 2020; Scaman al., 2017; Lan et al., 2017)
2

---

Published as a conference paper at ICLR 2026
and decentralized adaptations of the Alternating Direction Method of Multipliers (ADMM) (Wei & Ozdaglar, 2013; Shi et al., 2014; Chang et al., 2015; Aybat et al., 2018) which reformulate the problem with consensus constraints and alternate between primal and dual updates. A notable development in this line is (Wang et al., 2018), which construct constraints based on mixing matrix into an ADMM framework for decentralized composite optimization.
The Alternating Direction Method of Multipliers (ADMM) is a powerful algorithmic framework for solving linearly constrained convex optimization problems with separable objective structures, and it does so without requiring smoothness assumptions. This characteristic makes ADMM particularly well-suited for composite optimization tasks involving non- smooth loss functions and regularizers. We refer readers to (Eckstein & Yao, 2015; Boyd et al., 2011; Glowinski, 2014) for comprehensive overviews the classical ADMM framework, and to (Deng & Yin, 2016; He & Yuan, 2015; Han et al., 2018; Yuan et al., 2020) for detailed convergence analyses of ADMM and its various extensions. To improve convergence speed and numerical performance, Symmetric ADMM (S-ADMM) and its generalizations have been proposed in recent years (He et al., 2016; Bai et al., 2018). These methods modify the standard ADMM iteration by introducing symmetric primal-dual update structure, typically involving an additional intermediate update of the dual variable. This symmetric design allows for more balanced update dynamics between the primal and dual variables and often leads to improved practical performance. Convergence analyses for S-ADMM and its extensions have been established in works such as (Bai et al., 2019; Gu et al., 2015).
ADMM and Symmetric ADMM.
Nulti-Communicattion in Decentralized Optimization. Incorporating multiple communication rounds per iteration has been explored in decentralized optimization for different purposes, using either fixed or adaptive strategies. Early work by (Berahas et al., 2019) showed that fixed multi- communication inherits DGD's convergence issues and incur high communication costs, while communication rounds increase periodically-can achieve exact conver-
adaptive strategies- gence, albeit requiring tuning or prior knowledge. Later methods (Jakovetić et al., 2014; Ye et al., 2023; Li et al., 2018) adopted fixed cuulticommunnictton to attain optimal theoretical communication complexity. However, empirical results in (Ye et al., 2023) indicate that fixed multi-round schemes unable to reduce total communication. This limitation motivates algorithmic designs that embed multiicommunication within the problem structure, rather than treating it as an external enhancement.
2 PRELIMINARIES
2.1
PROBLEM SETUP
We consider network of agents collaboratively solving the decentralized composite optimization problem
min F(x) EER
Σ +(()))
where fi is a convex local loss function and is convex local regularizer, both privately held by agent In settings involving a global regularizer g, we distribute it across the agents as 9i = Pig with nonnegative weights satisfying Σ P; 1. This formulation captures a broad range of decentralized machine learning problems. Typical examples of loss functions include least squares, quantile loss, Huber loss, and hinge loss, while common regularizers include the E -norm, f--nrmm and elastic net. Moreover, we denote the proximal operator of function f as
Prox,f (v) = arg min
[(x)
-
1
wll²
=CR
2n
Many commonly used loss and regularization functions admit closed-form proximal operators, which we leverage in the algorithmic development that follows.
2.2 GRAPH TOPOLOGY
We model the communication pattern among agents using an undirected and connected graph
G (V, F.), where V (1,1,...,,) denotes the set of agents and edge (i, F represents
3

---

Published as a conference paper at ICLR 2026
direct bidirectional communication link between agents and The graph structure is encoded by a mixing matrix W E R=**, where We E [0, 1] specifies the communication weight assigned to agent j by agent i. Throughout the paper, impose the following standard assumptions
W: Assumption The mixing matrix W satisfies: (I) W is symmetric; (2) W doubly stochastic, i.e. W1 = 1, where 1 denotes the all-ones vector; (3) Wij for i + j if and and only if (i,3) E, and Wa > for all E V.
An example of such a matrix based on the Metropolis-Hasseee weights (Hastings, 1970) is provided in Appendix A. Under Assumption the graph and its mixing matrix enjoy several well-known properties that follow from the Perron-Frobenius theorem:
Proposition 1. The following properties hold: (1) The eigenvalues of W satisfy (1))
X2(W) 2 ... 2 An (W) > -1, and the spectral gap = max ( (()2(())))))))))))) (W)] is strictly positive; (2) The null space of In W is given by mull(I, n W) span(1); which characterizes
the consensus subspace.
METHOD
This section presents the design of our communication-effiienn decentralized algorithm. We begin by reformulating the consensus constraint to enable multiple rounds of communication within each iteration. We then explain how proximal linearization facilitates decentralized updates, describe our communication scheme that minimizes per-iteration transmission, and finally introduce the full decentralized form of the proposed algorithm.
3.1 SYMMETRIC CONSENSUS CONSTRAINTS
A key requirement for applying symmetric ADMM (He et al., 2016) in decentralized optimization to enforce agreement among agents' local variables. Unlike distributed ADMM (Boyd al., 2011), where a central coordinator explicitly enforces consistency, the decentralized setting lacks global authority and therefore requires a different mechanism for encoding consensus within linear constraints. Our approach is motivated by the spectral properties of the mixing matrix W, in particular the structure of the null space of In W. Let W W @ la, and let (ul ..... ) Rnd denote the stacked vector of local variables. Using the fact that W is symmetric, doubly stochastic, and has positive spectral gap, one can show:
Un is equivalent the existence of auxiliary stacked variable u (v. ..... ) T Rnnd satisfying и We and e Wu.
Proposition 2 (Symmetric consensus constraint). The consensus condition di = U2
This pair of constraints in the above Proposition 2 embeds consensus into a linear, fully decentral- ized structure. Let (u) Σ fi(u;) and g(v) ->= (v:). Using the above constraints, problem equation is equivalently reformulated as
min f(u) + g(v) s.t. Au Bu 0,
a.ngRad
where = (W, Jnd)T and B = (Jnd: WTT. Note that the reformulated problem equation 3 is invariant under the exchange symmetric ADMM iteration balanced and self-adjoint: the two primal blocks enter the augmented Lagrangian in exactly the same way and therefore admit identical ADMM updates. We refer this formulation as the symmetric consensus constraint to highlight this structural novelty.
and (A, B) ++ (B, A). This invariance makes the associated
Remark 1. The ADMM variant proposed in (Wang et al., 2018) uses related but asymmetric
constraint formulation with A = ((Ind - W)1/2, Ind) and B = (0, Ina)T. This design enables single communication round per iteration by absorbing part of the dual update. However, the inherent asymmetry prevents the use of symmetric ADMM, which requires a balanced primal-dual structure.
3.2 GRAPH-AWARE PROXIMAL LINEARIZATION
Let À = (Yi, denote the Lagrange multipliers associated with the two equality constraints in equation 3, let 3 > be the augmented Lagrangian penalty parameter, and let r, 8 be the
4

---

Published as a conference paper at ICLR 2026
symmetric dual update steps. The generalized symmetric ADMM (see, e.g., He et al., 2016; Bai et al., 2018; Deng & Yin, 2016) applied to the reformulated problem equation 3 takes the form
((+1)) arg min f(u) (A Au) 2/ ll Br uERnd
1 2
u 12
1(++)) = ( r3 -Brl().
3 0((+1) = arg min g(v) + (1(++)),, Bv) + 2 [1Aul(tt)) Bull? vERnd
+ + 221l " 10
((+11 = ((+ - s (B (++1) Bb(tt))).
To efficiently linearize the subproblems and remove the quadratic coupling introduced by the terms ||Au - Bb(0 and [| - Bell we introduce a graph-aware positive definite proximal matrix
Q ((1 T W W):
0.
This choice ensures that the augmented Lagrangian becomes separable across agents, enabling fully decentralized updates of both primal variables. Then, by expanding the symmetric ADMM updates in equation 4, the updates can be rewritten in the following decomposable form:
((+1) = arg min f(u) - (WA + >1 + (22 (0) + (1 (1++7 +T) ueRnd
W W Wulo) u)
+ ( ;) |
A1 (++ ( r3 (WalWee (1+1) - x(0)). 2/ ((++)) = 211 - r (3( (1:2)--Wo0 Woll),
((+1)) = arg min g(v) - (3 (2 (c+1) (1+ t) (1) - w Wo(o) - (((+)) WAy ((+ vERna
+ + (1
7) 2 10122
X ((+))
1 (++
sB - - x X2 (1+
- Wi((tt))
(5)
This graph-aware linearization eliminates quadratic coupling, restores decomposability, and yields update rules that are fully decentralized and structurally balanced across the two primal blocks. For detailed derivation the general update form, we refer the reader to Appendix B.
3.3 OPTIMAL DOUBLE- DOUBLE-COMMENNE COMMUNICATION STRUCTURE
We now describe how implement the decomposable symmetric ADMM updates in equation 5 in fully decentralized manner. There are four blocks of variables u, Å1. X2, E Rnn We write
[w] ..... TT, (v ..... ... TT, (AI, ... ATT A2 (AT
AT)
with u; Us. A1. 22, E Rd stored locally at agent i, and communication is restricted to exchanging weighted neighbor-averager values determined by the mixing matrix W. For convenience, for any stacked variable z = (2] ........:.. ",T"" we use the notation z(!) ΣjEN, Wazz
key structural feature of the symmetric ADMM updates is that the quadratic term W T W in equa- tion 5 requires 2-hop neighbor information. Consequently, one full iteration necessarily involves two distinct rounds of communication, and the communication pattern must therefore be scheduled with care. Our design follows two ordered principles: (1) achieve the minimal number of rounds by restricting communication to exactly two rounds per iteration: (2) within each round, minimize the amount of transmitted data.
The first principle is enforced because X (+++) requires aggregated u(*+1, and X (t+1) requires aggre- gated (1+1), fixing the communication rounds between these updates. Following this structure,
5

---

Published as a conference paper at ICLR 2026
organize each iteration into two update groups groups-Group and Group 2 separated by two munication rounds. To satisfy the second principle, we transmit carefully chosen vurl-variable combinations rather than primal variables, reducing the communication in each round only two d-dimensional vectors and achieving the minimal cost compatible with the symmetric ADMM structure. The overall cammuunccatoo-compaaatiin workflow is described as follows:
Group 1 update and Communication 1.
At iteration agent i begins with its current local vari- () and the cached mixed values oft) and 6(6) from the previous iteration. ables (u. . 1,11 2,
(t)
(t-
In Group 1, agent first updates (1) then computes u, (t+1) via the proximal operator of ft. and
finally forms the intermediate dual iterate X (++
At this point, the first communication round occurs. Instead of transmitting all primal variables, each agent constructs the auxiliary message a; (++1) X2, (+++)
+ : (2 - and broadcasts the (++1) to its neighbors. The received messages are aggregated to provide all mixed information needed for the Group updates.
pair (u: (++1) _
Group 2 update and Communication 2. Using प, (++1) and ब. ((++1), agent i performs the Group
update: it updates X b> computes ui (++1) via the proximal operator of gi, and then forms X 1,5
(t-1)
In the second communication round, each agent constructs b((+1) 2x - and broadcasts
(r+
(v. (o(++11)) b (((+))) to its neighbors, and received messages become inputs for the next Group 1 update.
Each iteration of DS-ADMM therefore consists of two local update groups and two neighbor- communication rounds. Information from one group not used directly in its update but enables the update of the other group, creating a feedback structure. This interleaving induces a coupled communication-uppaat mechanism where each block drives progress in the other.
Furthermore, this structure makes symmetric ADMM not just suitable but essential: it accelerates convergence without increasing communication, and leads to clean, symmetric algorithm. To our knowledge, this tightly coupled update-communication framework is novel in the decentralized optimization literature.
3.4 ALGORITHM
The resulting procedure is summarized in Algorithm and illustrated in Figure 1, which we refer CouubleCoommmiccatte Symmetric ADMM (DS-ADMM). The step-size parameters set
0 < r < and 8 1.
1
1 (++ ()
Group
(1+1)
(t+1))
((+1))
1 (4)
(+
A (++1)
Group 1
Figure 1: Illustration of one iteration of the proposed two-group symmetric decentralized ADMM scheme. Solid arrows denote inter-group communication, i.e., information exchanges that require communication across groups, while dashed arrows indicate intra-group communication performed locally within each group.

---

Published as a conference paper at ICLR 2026
Algorithm Double-Commminiattonn Symmetric ADMM (DS-ADMM)
1: Initialize: u
0 for all i E {L.... n.. mixing matrix W E R step sizes > 0, augmented Lagrangian parameter 3 > 0 and proximal term parameter т >> 0.
(0) = u1 x = و
2: repeat
3: [Group 1 update]
22,
(t- 2
sB(
(t)
0;
(1+1) Prox 312- bi 2+5 +11+))u()) + (24750
+ xg?))
";
(1+2)
- r (++1)
2,,
4: [Communication Transmit aj
(++1)
225 + (((())))) 2 and uf (++1)
(e+
5: [Group 2 update]
1, (1+b) 11. rr (1+1)
// (1)
of ((+1) = Prox कहMन 2+r
1
(1)
(1+1) + (1 + T)o(0) - (2+T)Ø (2 (++
+ al(+)))
((+1) 1 Blū;
(t+1)
1/3 1
11.
6: [Communication 2] Transmit 2 1++1 and b. (((+1) = 2x (1++
7: until convergence criterion is satisfied
CONVERGENCE ANALYSIS
In this section, analyze the convergence properties of the proposed decentralized algorithm. As it is a direct application of symmetric ADMM with proximal terms, several convergence results follow from existing literature. We further establish linear convergence under specific conditions that mild yet broadly applicable to machine learning problems.
To facilitate the analysis, we define the block matrix:
Q 1+110
LBB
B
B
(6)
36++
which is positive definite. We also define the concatenated variable 0 = (u (uT,uT, "TTTTTTTTTTT E Rnd
4.1 GENERAL SUBLINEAR CONVERGENCE
Theorems 3.3 and 4.2 of (Gu et al., 2015) imply that the proposed algorithm enjoys general sublinear convergence rate (11/t) without requiring strong assumptions the objective functions.
Theorem 1. Let { be the sequence generated by DS-ADMM. Then {o(t)) converges to solution point 0e, and the following non-ergodic sublinear rate holds:
- | 0(1+1112 BT((+1)
1 1 1-r (10() o(0) 12 + 11o v. (00)))))))
Remark 2. This sublinear convergence result is independent of the underlying communication graph and mixing matrix. Thus, the algorithm inherently robust to different network topologies in decentralized environments.
4.2 LINEAR CONVERGENCE UNDER METRIC SUBREGULARITY
Although various results on linear convergence of ADMM and its variants exist (e.g., (Deng & Yin, 2016; Han et al., 2018; Yuan et al., 2020; Bai et al., 2019; Gu et al., 2015)), none directly apply to our algorithm. Nevertheless, we adapt ideas from these works to establish linear convergence under standard regularity condition known metric subregularity.
Definition (Metric Subregularity). set-valued map : RRn = R2 said to metrically subregular al (z. 7) gph(w) with modulus > if there exists > O such that:
dist(r, v - K dist(y, (xx)), Wo E (T).
7

---

Published as a conference paper at ICLR 2026
We consider the KKT mapping:
(8f(u) A 0g(v)) + B
(KK =
Au Be
and solution set Ω == {0 0 E [K))
Under the above framework, we now in position to state the following linear convergence theorem which established a Q-linear rate of distance the solution set, and a R-linear rate suboptimality. The proof is deferred to Appendix C.
Theorem 2. Suppose TKKY is metrically subregular at (0. 0) with modulus f any O E Dr Then the sequence {0(t)} generated by DS-ADMM converges Q-linearly to D2, i.e. there exist integer > O and constant > O such that for all t T:
dist 2 (0(+1) 22) S +
d (0(),))),
(10)
where
1-r
> 0,
min
23p,
(11)
c²0
B
and
2r +1
6 = max Grr + B2 12 + + ( 3(73)))
(= 3(r +1)
+ +
(12)
Also the suboptimality converges R-linearly, which means there exists 1 > 0:
1 + g - s(u (())))
(13)
I+c
The linear convergence rate clearly depends on the algorithmic parameters т, r, 3, and the structure of the mixing matrix W. In particular, larger value of which reflects better network connectivity, leads to a faster convergence rate.
4.3 SUFFICIENT CONDITIONS FOR METRIC SUBREGULARITY
First we state the important definition of PLQ functions:
Definition 2. A function f IR" + RU }+oo] is piecewise linear-quadratic (PLQ) if is quadratic on a finite union of polyhedral regions:
1
f(x) 5 +cQr+To+tn
(14)
Many loss and regularization terms used in machine learning are PLQ, including the & and &a norm, hinge loss, squared loss, and elastic net.
The following proposition gives characterization the metric subregularity of TKKT- Each case justified by different results from the literature: Theorem 46 and Theorem 60 of (Yuan et al., 2020) support the first condition, Robinson's 's continuity property (Robinson, 198 1) establishes the second, and Lemma 4 of (Latafat et al., 2019) together with Theorem 60 of (Yuan et al., 2020) imply the third.
Proposition 3. The KKT mapping TKKT is metrically subregular at (0, 0) for any Ō E O under any of the following conditions: (i) each fü is smooth and strongly convex, and each gi is PLQ: (ii) all f. and 9: PLQ: (iii) all and gi smooth and strongly convex.
Therefore, AS-ADMM achieves linear convergence across wide range of practical decentralized optimization problems, including Lasso, logistic regression, SVM classification and other models frequently encountered in machine learning.
8

---

Published as a conference paper at ICLR 2026
5 NUMERICAL EXPERIMENTS
We evaluate the proposed DS-ADMM algorithm on two representative decentralized composite optimization tasks: Lasso regression and lg-reuularized SVM classification. All experiments use n 30 agents connected via a random graph with edge probability = 0.5. The Metropolis-Hastinge mixing matrix is constructed following Appendix A, and data are evenly partitioned among agents. The global regularizer is split uniformly as 9: (z) = g(z). Comparisons are made against four representative methods: Decentralized Proximal ADMM (Wang et al., 2018), PG-EXTRA (Shi et al., 2015b), NIDS (Li et al., 2019) and ProxMudag (Ye et al., 2023). All experiments are conducted on a machine equipped with Intel Core 17-1260P CPU and 16GB RAM.
All adaptive parameters of baseline methods are tuned for best performance. For DS-ADMM, we use fixed step sizes (r, s) (0.99, 1) and proximal coefficient 0.01. Suboptimality measured as F(u(t)) F(u"), ), where u" is high-accuracy centralized solution.
Lasso Regression.
Using the a9a dataset from the LIBSVM repository (Chang & Lin, 2011) with
- 111111, == 1
f((z)
|| - bull?
g((x)
2m
SVM Classification. Using ala, each agent solves
= - Σ
À 2n
[(()) JES.
max(0,
byaJ x).
gi())
2 |,
m
Note that ProxMudag is excluded the SVM classification task as its formulation requires globally coupled nonsmooth terms. Across both tasks, DS-ADMM consistently converges faster and requires significantly fewer communication rounds. Additional results and parameter sensitivity analysis are reported in Appendix D.
14
Lasso: iterations
Lasso: communication rounds
SVM: iterations
SVM: communication rounds
Figure 2: Comparison of DS-ADMM with baseline methods on Lasso and SVM
6 CONCLUSION
We proposed DS-ADMM, a fully decentralized algorithm for composite optimization based on the symmetric ADMM framework. The method integrates novel communication structure that structures

---

Published as a conference paper at ICLR 2026
double communication per iteration without increasing overhead, while maintaining symmetric update pattern across variable blocks. Theoretical analysis established both sublinear convergence under standard conditions, with linear rates guaranteed by metric subregularity of the KKT mapping. Our algorithm accommodates general nonsmooth regularization and broadly applicable to various machine learning tasks. Extensive numerical experiments demonstrate that DS-ADMM outperforms existing decentralized methods in both iteration and communication efficiency.
ACKNOWLEDGMENTS
This research is partially supported by the National Natural Science Foundation of China (12231017, 72171216) and the National Key R&D Program of China (2022YFA 1003803). We gratefully acknowledge the insightful comments and suggestions provided by the anonymous reviewers.
REFERENCES
N. S. Aybat, Z. Wang. T. Lin, and S. Ma. Distributed linearized alternating direction method of multipliers for composite convex consensus optimization. IEEE Trunsactions Automatic Control, 63(1))5-20, 2018.
Jianchao Bai, Jicheng Li, Fengmin Xu, and Hongchao Zhang. Generalized symmetric admm for separable convex optimization. Computational Optimization and Applications, 70(1):129-177 2018.
Jianchao Bai, Xiaokai Chang, Jicheng Li, and Fengmin Xu. Convergence revisit generalized symmetric admm. arXiv preprint WyXiv:1906008880 2019.
Albert Berahas, Raghu Bollapragada, Nitish Shirish Keskar, and Ermin Wei. Balancing communi- cation and computation in distributed optimization. IEEE Transactions Automatic Control, 64 (8):3141-3155, 2019.
Stephen P. Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed opti- mization and statistical learning via the alternating direction method multipliers. Found. Trends Mach. Learn., 3:2-122, 2011.
Chih-Chung Chang and Chih-Jen Lin. Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol., 2(3), May 2011.
Tsung- Hui Chang, Mingyi Hong, and Xiangfeng Wang. Multi-agent distributed optimization via inexact consensus admm. IEEE Transactions Signal Processing, 63(2):482-497, 2015.
Wei Deng and Wotao Yin. On the global and linear convergence of the generalized alternating direction method of multipliers. Journal of Scientific Computing, 66(3):889-916 2016.
Jonathan Eckstein and Wang Yao. Understanding the convergence of the alternating direction method of multipliers: Theoretical and computational perspectives. Pacific Journal of Optimization, 11: 619-644, 2015.
Roland Glowinski. On Alternating Direction Methods of Multipliers: A Historical Perspective, pp. 59-82. Springer Netherlands, Dordrecht, 2014.
Yan Gu, Bo Jiang, and Deren Han. A pomii-poxiial-ooo based strictly contractive peaceman-rachiee splitting method. Journal of Computational Mathematics, 41, 2015.
Deren Han, Defeng Sun, and Liwei Zhang. Linear rate convergence of the alternating direction method multipliers for convex composite programming. Mathematics of Operations Research, 43(2):622-637, 2018.
W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1))97-109 1970.
Bingsheng He and Xiaoming Yuan. On non non-ergodic convergence rate of douglas -achfoor alternating direction method of multipliers. Numer. Math., 130(3):567-577, 2015.
10

---

Published as a conference paper at ICLR 2026
Bingsheng He, Feng Ma, and Xiaoming Yuan. Convergence study on the symmetric version of admm with larger step sizes. SIAM Journal on Imaging Sciences, 9()):1667-15011 2016.
Dušan Jakovetić, João Xavier, and José M. F. Moura. Convergence rate analysis of distributed gradient methods for smooth optimization. In 2012 20th Telecommunications Forum (TELFOR), pp. 867-870, 2012.
Dušan Jakovetić, João Xavier, and José M. F. Moura. Fast distributed gradient methods. IEEE Transactions Automatic Control, 5965)1131-1146, 2014.
Guanghui Lan, Soomin Lee, and Yi Zhou.
cnttumiccation-fffiiiee algorithms for decentralized and stochastic optimization. Mathematical Programming, 180, 2017.
Puya Latafat, Nikolaos M. Freris, and Panagiotis Patrinos. A new randomized block-coordinate primal-dual proximal algorithm for distributed optimization. IEEE Transactions Automatic Control, 64(10):4050-400 40605500-46,,.-0 2019.
Huan Li, Cong Fang, Wotao Yin, and Zhouchen Lin. Decentralized accelerated gradient methods with increasing penalty parameters. IEEE Transactions Signal Processing, 2018.
Zhi Li, Wei Shi, and Ming Yan. A decentralized meoximalggradien method with network independent step-sizes and separated convergence rates. IEEE Transactions Signal Processing, 67(17): 4504, 2019.
Angelia Nedic and Asuman Ozdaglar. Distributed subgradient methods for multi-agent optimization. IEFE Transactions Automatic Control, 54(1):48-61. 2009.
Guannan Qu and Na Li. Harnessing smoothness accelerate distributed optimization. IEEE Transactions on Control of Network Systems, 5(3): 2018
Guannan Qu and Na Li. Accelcrated distributed nesterov gradient descent. IEEE Transactions on Automatic Control, 65(6):2566-2535 2020.
Stephen M. Robinson. Some continuity properties of multfuunio,, pp. 206-214. Springer Berlin Heidelberg, 1981.
Kevin Scaman, Francis Bach, Sébastien Bubeck, Yin Tat Lee, and Laurent Massoulié. Optimal algo- rithms for smooth and strongly convex distributed optimization in networks. In 34th International Conference on Machine Learning, pp. 3027-3036, 2017.
Wei Shi, Qing Ling, Kun Yuan, Gang Wu, and Wotao Yin. On the linear convergence the admm in decentralized consensus optimization. IEEE Transactions Signal Processing, 6767),1750-176,, 2014.
Wei Shi, Qing Ling, Gang Wu, and Wotao Yin. Extra: An exact first-order algorithm for decentralized consensus optimization. SIAM Journal Optimization, 25(2):944 966, 2015a.
Wei Shi, Qing Ling, Gang Wu, and Wotao Yin. A proximal gradient algorithm for decentralized composite optimization. IEEE Transactions on Signal Processing, 63((2):6013 (002))601336023, 2015b.
Ying Sun, Gesualdo Scutari, and Amir Daneshmand. Distributed optimization based on gradient tracking revisited: Enhancing convergence rate via surrogation. SIAM Journal on Optimization, 32 (2):354-385, 385,, 2022.
César A. Uribe, Soomin Lee, Alexander Gasnikov, and Angelia Nedić. A dual approach for optimal algorithms in distributed optimization networks. In 2020 Information Theory and Applications Workshop (ITA), pp. 1-37, 2020.
Bin Wang, Hongyu Jiang, Jun Fang, and Huiping Duan. A proximal admm for decentralized composite optimization. IEEE Signal Processing Letters, 25(8): 2018
Ermin Wei and Asuman Ozdaglar. On the o( (1/k) convergence of asynchronous distributed alternating direction method of multipliers. In 2013 IEEE Global Conference on Signal and Information Processing, pp. 551-554, 2013.
11

---

Published as a conference paper at ICLR 2026
Jinming Xu, Ye Tian, Ying Sun, and Gesualdo Scutari. Distributed algorithms for composite opti- mization: Unified framework and convergence analysis. IEEE Transactions on Signal Processing, 69:3555-3570, 2021.
Haishan Ye, Luo Luo. Ziang Zhou, and Tong Zhang. Multi-consensus decentralized accelerated gradient descent. Journal of Machine Learning Research, 24(306):1-50, 2023.
Kun Yuan, Qing Ling, and Wotao Yin. On the convergence of decentralized gradient descent. SIAM Journal on Optimization, 26(3):1835-1855 2016.
Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. Discerning the linear convergence of admm for structured convex optimization through the lens of variational analysis. Journal of Machine Learning Research, 21(83)11-75, 2020.
12

---

Published as a conference paper at ICLR 2026
Appendix
A MIXING MATRIX BASED ON NETROPOLLIS-HASTING WEIGHT
desired mixing matrix can be constructed using Metropolis-Hastings weights, where d, = [Wi]] is the degree of node i:
if (i, E E, if (i,j) & E and j + i,
dd,
Wij-
0,
Σ Wat
if = i,
IEN
B GLOBAL FORM OF UPDATE
According to the update rule of Symmetric ADMM and our linear constraint formulation, global t-th iteration can be written in the below equation:
(1+ = arg min [u) - (A(), Au) + 3 =1 - 10 + _ 2 211 11 wER
2
(((+)) = 1000 -r3(Au((t)) -BulD)
(15)
v(+1) = =arg min g(v) + ()(++)), Bob + 3 || - Bull + 2 - 1110010012 11..3.
vERMa ((+1)= = ((+ 2) (B (1+1) (t+()))).
2
Then we decompose the Lagrange multiplier X, which yields the below equation:
(++1) arg min f(u) - (2 Wu) (259),) u) + 3 " 010 + _ -lu 10 unRnd
+ 2 ||u - u 10)
2 (1+b) 11 rr (W - (()), X (1+b)) =11=-) r18(((+1)) - -Wul
((+1) arg min g(v) + (2) (2((+b) (1+b2)
,)) + (212 (1+b) Ww) + 3
| - =11
veRnd
2
+ 5 [| 11 2 + 1
2 110 - w0() 1/)
s3(1((+1) x((+1) = A (+++) -3 (1+1) - v((+1) ((+1)= = 2 (t+ - WolltD)).
(16)
And it can be transformed into the following version ready for decentralized update:
(1+1) = arg min f(u) - (W>() + >1S + ((2 + (1 + r)u( - W Ww())))) u)
WERRd
T + + - |/
1 ((+)) = (1) - r (++1) -()), 2/ (+++) =x r3( (++1 -191))-77 W.(l)),
(17)
(((+1) = arg min g(b) - (3(2W (++1) + (1 + ()(()) W Wrllt)) - (2 (1(++))+
VERnd
W WX!+ ( + 3(1 _ ||o112
X (1+1) = X (++ }) - 9.3 ve (1+1) - ((++))) A (++1) = X1 (t+ - (3( - (.(t+)))
Rearranging the terms and using the facts that (v(0) w W 3े wa( - () and
(n(+1) - WT ww(0 - WW (1++)
x()) in the above equation, we get:
13

---

Published as a conference paper at ICLR 2026
((++)) = arg min j(()- - (WAf" + +1/0 + +WWDD + + W(S - -1f- > un ueRnd
+ (1 + T
2 211u11
21 x(++)) = ( r3 (++1) - ,())) X/2 > = -a1i - -r3((((t))) -
-
(((+1) = arg min g(0) (8(W (++1) - (1 + T) (2 + +W (e+b)
wERnd
1 W( (2₂ (1 (x+ - X (0) + 3(1 2 2)1lul1
+ 1
1 ((+1)
X (1+ s36 ( u (++1) ,(tt))), 1 \/ (1+ s3( -w((tt))
(18)
By our previous definition of (1)) = x((++t + 1 (2 (c+ 110) and ((t)= = 210 + =2( (t) - 21 (((
we get
u (1+1)) = arg min f(u) - (Wa + -1 + 3(W(()) (1)) + +1+ u) + 3(1 + " "]llu|??
and 2 2(++)) =x(- (Wu - (()), x = -110 -13(u((1)) - -W
c((+1) = arg min g(v) (3(W (++1) + (1 + r))(0) - -(2/ (2/t+b) + +Wbl)), u) + 3(1 + T aERnd
2 ||o0|
1 ((+1)) 1 ((++)) w (++1) - ((tt)), (bt111 = 12 (4+)) +3(1 - (+t))
(19)
C PROOF OF THEOREM? 2
The proof is based on the notations of global form equation 7. First, we denote vectors
(
-AT A T Au -u)
= =
u1 =
V
F(O)- =
(20)
1
((0) = (1+1)
((0) = (1+11)
X(0) = AO - 3 (Aultt))- - -Bottt))
(21)
7(0)
i7(1))
200) =
g0o =
70
: h(z) = f(x) + t(y)
(22)
((0)
((1)
and matrices
Q
("
(1
(=)))
s=
Q +
3BT
B
-TBT
,M =
(23)
B
11
-3B
Q
Q
H
Q+ + +1
3
B
r B
,
Q
(24)
B
B(r41)
1-r
B
It is easy to verify the following properties:
g(++l) = g()) -
(25)
S + ST
- M
H = SM
(26)
The following lemma characterizes the eigenvalue property of H and G:
14

---

Published as a conference paper at ICLR 2026
Lemma 1.
Amax Sc
S:
[G) 2(1 r)p
(27)
Proof.
H = (
Q
Q+ (1-r))3
B
BBBB
x BT
(28)
r+1
-
so it is easy verify that
(rBBT)
(rBB7)
Amaxe H)- S Amax 10 (Q+ B) + Amaxx 3(r+
(29)
--1
while
Amax (Q+(1-r)3B' B) = ( W+(1-r) W)) - (2+T-r)3, (30)
rBBBT
(T3BT) T )= 3(r + 1) Amax Amax ((r3) W W) + 1)
2r +1
Amaxx B(r+
3(r
(31)
combining above, we get Amax S Also, we have
Amin (G) = min(Amin (Q): B 1--) 2 min minpp
1- r = Φ. 3
(32)
This equivalent form of solution set 92" follows from (He et al., 2016) by the definition of subdiffer-
entials:
Ω = n
(0 M²) ((3) + (0 - 0. F(O) 2 0).
(33)
BERtnd
Then give the following important lemma, whose proof follows directly from the proof of Theorem 2 and Theorem in (Bai et al., 2018) and the above form of solution set:
Lemma 2.
g((+1)
0r
| 11
s
g(0)
0-
(00))
g(0)
A0t E Ω²
(34)
11
G
Let us revise the KKT mapping
(0f(w) AT
TKKT =
Ög(v) + T Au Bv
(35)
The following lemma bounds the distance through G-norm. Lemma 3. The sequences {0(0} and {ō(0) satisfy
dist Host?(0,,77 (0, ((xt((()+1)))) < |000
Proof. By Proposition 2.1 in (He al., 2016) and our notations, we derive the following inequality characterizing the subproblems:
f(u) f(ū(1)) + (u (u- -A
- - + Q(7!)) - -n(0)) 20
(36)
s(v) ((0)-((7(0) ) (1))) -rB (2 ( + (Q 3B B)(1 - -(0)) 20 (37)
-
is obvious that
dist. Tk) (0) dist di, Ti (0)) dist (0, T2()) dist dist (O,
(38)
15

---

Published as a conference paper at ICLR 2026
where (0) Of(u) - A T0)) -g(())
A, Tz(f) = Au Bu.
X,
From the above two inequalities, we have the following:
dist(0, (1((e++))) - A (10) - 1((+1))) Q(T( - -((1)//
(39)
= AT [1(10) - -X(1) BB((
b(0) -B)("))))()))
- QQ(T0) -x(0//
where the second equality uses the update rule
((++1) = x B(Au((+1) - By((t))
( -((A((+1) Bblltt) + (,3B(bl)) - v(1-1))
(40)
= A - (r + - -X²) + +BB(((0) (((t))
similarly, we have:
dist (2(((()))) S
B (7(1) - ((++1)) (Q 3B B) - -0(0) + +rB (7(0) |(0)
1 -/00/|
(41)
and finally
(0, ((((t+)))) - - X(0 - B(x(0 1300--3000 -(6(0)=((0)) 200)
1 =
-000
(42)
Substituting equation 39 equation 41 equation 42 into equation 38 while using the definition of A, B, Q, we get
dist? (0, TKKT (0((++)))
s ||AT 1-210- X(0) - 3B(((()) - -b(0)] Q((7(1) - -1(0))' -Q(Z()-((())
1 (1 - - -X00)- - B(o(() - - /
+ ( +1 +
A AT(10- 1910-5700 -²00 +
2
|1:00 -500) /
33.²
+ 338² A ATB(( - (V(((00~a0))) )/
(43)
+2 B(0 - -/0 + ( +3 QQ( -
s (6r²+ 3e | + +11232 + +4+ (T3)3)
|
- 1 1²
+ +3(73)2 (1(0) - -
5 (0(00 - /101/² < (t()) - T(t) 2
Then we are able prove the first part of Theorem 2:
Proof. Because Ω" is closed convex set, there exists 02 E O satisfying
dist = 1000) 05
(44)
16

---

Published as a conference paper at ICLR 2026
Then, by the metric subregularity of the KKT mapping and the global convergence result, there exists > 0, for any t T we have
Φ dist (0, 1(KT((((+1)))
0(0) - x(0)
2V
dist((((+1), ,22)
(45)
c
Φ
2
dist (ist:((0((t))) .2").
(0(++
c20
So, we will have from the above inequality that
(1 + E) dist (0(++1), -2*)=== < [((++))
8.
1
+ e (0(e+1),
< p(+1)) 0g
2
+ |1000 5000 2
(46)
1 (00)) 8:
2
dist (2*)
and the main result is proved.
To prove the R-linear rate of suboptimality, first, from the same approach of proving Corollary 2.1 in (Bai et al., 2019) we are able to prove the R-linear rate of ||0(0) f0 And a simple corollary from Lemma 2 that | g(t)
and Au(+1) Bu(++)) 1) also converges R-linearly.
Then substitute 20 and R into equation 36 and equation 37:
f(u + -(( 2 (( + g( - (xr u - -A)) + Q(T()) =(1)))
(100 0000 7((), - B TT - (2(00 x + (Q BB B) - v10))
(47)
and from the definition of solution set f(u((-1) + g((((+1)) > f(u + g + (100 Au - Bv((+)))
(48)
the above inequalities and the R-linear convergence of ||000) g0 |000 g(0)
|u((+I) - Be(++ lead to the R-linear convergence of suboptimality, which finishes the proof.
D ADDITIONAL EXPERIMENTAL DETAILS
D.1 PARAMETER SENSITIVITY ANALYSIS
We summarize the hyperparameters used in the DS-ADMM algorithm and clarify their respective roles.
Proximal parameter 7. The proximal term ensures that the matrix Q is positive definite, which is required in order to establish linear convergence. Throughout our experiments, we set 0.01.
Augmented Lagrangian parameter 3. The parameter has a major influence on the convergence speed. One may either adopt the adaptive tuning strategy proposed in (Boyd al., 201 1) or evaluate several candidate values and select the one that performs best empirically.
Dual update step sizes (r, s). The convergence region and admissible ranges for the symmetric dual step sizes have been characterized in prior work (Bai et al., 2018). Following these theoretical guidelines, we fix and 0.99, which consistently yield stable and efficient performance.
17

---

Published as a conference paper at ICLR 2026
D.2 ADDITIONAL EXPERIMENTAL RESULTS
We report experimental results 30-agent sparsely connected random graph with edge probability p = 0.2. As expected, algorithms perform worse on this sparse topology, consistent with theoretical predictions. Nevertheless, our proposed DS-ADMM retains the best overall performance.
189
109
Corerin Pourids
Figure 3: Lasso performance on the 30-agent sparse random graph (p = 0.2). Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
39
12"
331
10
50
Commin calir Dounds
Figure 4: SVM performance on the 30-agent sparse random graph (p 0.2). Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
We further evaluate the Lasso problem a 100-agent network under two connectivity levels, with edge probabilities p = 0.5 and p 0.2.
3.7
197
33
10
Figure 5: Lasso performance on the 100-agent random graph (p = 0.5). Left: suboptimality vs. iterations. Right: suboptimality communication rounds.
18

---

Published as a conference paper at ICLR 2026
37
187
13
Contral CHEET Rourds
Figure 6: Lasso performance on the 100-agent random graph (p 0.2). Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
Then we test performance when the dataset is randomly partitioned among agents, using -30 and n 100 with graph edge probability = 0.5.
189
12
300
253
4.00
700
Figure 7: Lasso performance with random data partition on the 30-agent random graph (p 0.5).
Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
109
0,000
10
13
10"
Forrttoo
Pounds
Figure 8: Lasso performance with random data partition on the 100-agent random graph (p = 0.5). Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
Finally, test the performance ring network with 50 agents.
19

---

Published as a conference paper at ICLR 2026
32
13
Coruras CHEDT Rounds
Figure 9: Lasso performance on the 50-agent ring topology. Left: suboptimality vs, iterations. Right: suboptimality vs. communication rounds.
LLM USAGE
We used large language model (LLM) to help polish the writing and enhance readability. The authors are solely responsible for the technical content, analysis, and conclusions.
20


Reviews:
[{"review_id": "si6y6LegeP", "reviewer": "Reviewer_PsdU", "summary": "The paper proposes a decentralized optimization method, DS-ADMM, that embeds two rounds of neighbor communication inside each ADMM iteration, by designing a symmetric pair of linear consensus constraints and then applying S-ADMM with a graph-aware proximal linearization to enable separable subproblems. The algorithm communicates (per iteration) with two d-vectors perr agent in each of two communication rounds by sending cleverly chosen dual combinations a^(t+1) and b^(t+1) rather than two primals, which the authors argue minimizes what must be sent for symmetric ADMM to work. Theoretical results include a non-ergodic O(1/t) sublinear rate and Q-linear convergence under metric subregularity, with explicit constants depending on the spectral gap of the mixing matrix. Experiments on lasso and l2-svm over random graphs with n=30 show faster decrease in suboptimality per communication round than several baselines, including on a sparser graph.", "questions": "- How would you handle smooth but non-proximable f_i (e.g., logistic regression)? Is there a prox-gradient DS-ADMM variant that retains the same communication schedule? \n- Do your conclusions extend to directed or time-varying graphs? What breaks in the proof is W is not symmetric?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(+) The constraint design is an elegant way to bake two-hop information flow into the constraints themselves. This makes the need for two communications per iteration principled rather than an ad-hoc multi-consensus loop. To my knowledge this particular symmetric, two-block dual realization of decentralized ADMM is novel. \n(+) The communication scheduling -- transmitting two surrogates rather than two primals -- is thoughtfully engineered so each block enables the other block's update with minimal payload. That design is interesting and practically motivated.  \n(+) The communication-aware design is reflected in the evaluation. Testing on two graph densities is nice and qualitatively matches the theory.", "weaknesses": "(-) Counting \"rounds\" alone does not equal communication volume here because DS-ADMM transmits two d-vectors per round (four per iteration). Baselines often send a single d-vector per round. A better comparison should report total scalars (or bytes) transmitted per agent to reach a target accuracy. Without this \"net communication reduction\" claim is not fully substantiated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZdIsOFfiuA", "reviewer": "Reviewer_GNZq", "summary": "The paper propose a variant of ADMM algorithm for decentralized consensus optimization which uses so-called double communication strategy", "questions": "> Note that ProxMudag is not included in this comparison because it requires the nonsmooth term to be globally coupled, which is incompatible with this separable formulation (Ye et al., 2023).\n\nCould you please explain the statement?", "limitations": "", "rating": 2, "confidence": 2, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "Due to weaknesses, I can not highlight any substantial strengths of the paper", "weaknesses": "It is stated that\n> To our knowledge, this is the first decentralized optimization framework that achieves a net reduction in total communication by leveraging fixed multi-round communication within each iteration.\n\nbut multi-round schemes showed  communication acceleration in Scaman et al., 2017 and Ye et al., 2023, so the contribution is unclear to me.\n\nThere is no theoretical complexity comparison with SOTA decentralized optimization algorithms such as Mudag or even Symmetric ADMM, from which the algorithm was derived. The only comparison is through numerical experiments but they are not representative.\n\nOverall, the contribution seems to be minor", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XTf9jg8buf", "reviewer": "Reviewer_PbZX", "summary": "The paper considers decentralized symmetric ADMM, an important optimization problem, and develops a variation that allows multi-round communications with each ‘iteration’. This is called decentralized symmetric ADMM (DS-ADMM), a decentralized version of S-ADMM. The key step is to formulate an auxiliary constraint that is embedded. This leads to a 2-round approach per iteration. Convergence is guaranteed analytically under standard assumptions, and numerical experiments illustrate the method.", "questions": "Improved numerical experimentation would strengthen the work.\n\nWhat, if any, are the implications for scalability?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The auxiliary constraint embedding (Prop 2) with proximal linearization is developed to be a useful method for obtaining the desired symmetric ADMM compatibility. \n\nGenerally linear convergence is obtained, while communications overall are reduced because of fast convergence despite the 2-round per iteration needed.", "weaknesses": "The experiments are limited to a single group size, and are rather basic.\n\nRobustness to dropouts or topology changes isn’t clear.  \n\nTuning is required and sensitivity and robustness to topology and number of agents are not explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DH48A0c8zQ", "reviewer": "Reviewer_AAag", "summary": "This paper proposes DS-ADMM, a decentralized optimization algorithm for composite problems that incorporates two communication rounds per iteration. The key innovation is a reformulation of consensus constraints that enables information exchange beyond immediate neighbors through a symmetric ADMM framework. The authors claim this approach reduces total communication cost despite increasing per-iteration communication. They provide convergence analysis showing sublinear rates generally and linear rates under metric subregularity conditions. Experiments on Lasso regression and SVM classification demonstrate superior performance compared to existing methods.", "questions": "Refer to the weaknesses section.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The reformulation of consensus constraints using the symmetric form $u = \\tilde{W} v, \\tilde{W} u = v$ is creative and well-motivated. This differs meaningfully from prior work (Wang et al., 2018) and enables the application of Symmetric ADMM in a decentralized setting.\n- The analysis in Section 3.3 carefully optimizes which variables to transmit and when. The identification that only two communication rounds are necessary and the strategic transmission of dual variables ($a_i^{(t)}$ and $b_i^{(t+1)}$) rather than primal variables shows careful algorithmic design.\n- The paper provides both sublinear (Theorem 1) and linear convergence guarantees (Theorem 2). The characterization of sufficient conditions for metric subregularity (Proposition 4) covering PLQ functions and strongly convex cases is comprehensive and practically relevant.", "weaknesses": "- Limited Experimental Scope: Only two problem types (Lasso and SVM) and two relatively small datasets (a9a, a1a) are tested. Network size fixed at $n=30$ agents is small for decentralized optimization. Only random graphs with $p=0.5$ (and $p=0.2$ in appendix) are considered. No evaluation on structured topologies (ring, grid, etc.). No scalability analysis showing how performance varies with network size or sparsity.\n- Parameter Sensitivity Not Addressed: The choice of $\\tau = 0.01$ appears arbitrary with no justification or sensitivity analysis. The convergence rate depends on $r, \\beta, \\tau$, and $\\rho$ (Theorem 2), but no guidance is provided on selecting these parameters. Different baseline methods may require different tuning efforts, so the fairness of the comparison is unclear.\n- The notation switch between $w$ (dual variable blocks) and $w$ (concatenated variable including primal and dual) in Section 4 is confusing. The communication strategy in Section 3.3 is difficult to follow. Why specifically transmit $a_i^{(t)} = w_{2i}^{(t+1/2)} + 1/r(w_{2i}^{(t+1/2)} - w_{2i}^{(t)})$? The derivation or intuition is missing. Algorithm 1 uses $w_{2i}^{(-1/2)}$ in initialization, but this notation is never explained. The relationship between the proximal parameter Q and the mixing matrix W could be explained more intuitively.\n- How is the optimal solution $u^{\\star}$ computed for measuring suboptimality? What is the convergence criterion in Algorithm 1? Computational cost per iteration is not analyzed—only communication is counted. No discussion of numerical stability or practical implementation challenges.\n- The introduction states that multi-consensus schemes \"offer limited improvement to the quality of each iteration\" but provides no formal analysis or empirical evidence quantifying this claim. The paper would benefit from ablation studies showing why two rounds specifically are optimal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)