# Communication-Efficient Federated Learning with Adaptive Consensus ADMM
## 3 FedACADMM: Adaptive Consensus ADMM for Federated Learning
### FedACADMM Algorithm  
We now turn to the description of a novel ADMM algorithm for federated optimization, and outline the FedACADMM method in Algorithm 1. Then, some discussions are offeredto help throw light on the proposed method. The following straightforward yet crucial changes to FedACADMM lead to notable empirical advancements and also enable us to ensure convergence of the approach. ```
0:\(T\), \(m\), \(\rho^{t}\), \(S^{t}\), \(\theta^{t}\), \(\theta^{t-1}\), \(E\), \(\eta\)
0:\(\theta^{t+1}\)
1:for\(t=0,1,...,T-1\)do
2: server selects a subset \(S^{t}\) of \(m\) clients at random
3: server broadcasts \(\theta^{t}\) and \(\theta^{t-1}\) to all chosen clients
4:for client i \(\in\ S^{t}\)do
5: check the convergence verification and update the variable parameter \(\rho^{t+1}\) as in (8)
6: update client model as \(w_{i}^{t+1}\) for \(E\) epochs as in (9)
7: compute the difference \(\Delta_{i}^{t}\) as in (10)
8: sent \(\Delta_{i}^{t}\) back to the server
9:endfor
10: server calculates average parameter \(\theta^{t+1}\) as in (11)
11:endfor
```  
**Algorithm 1** FedACADMM  
We begin by describing the iterative process in detail. At the start of the \(t\)-th round, a subset of clients (denoted as \(S_{t}\)) is chosen, and each chosen client downloads the current server model \(\theta^{t}\) and the last server model \(\theta^{t-1}\). Confirm whether the convergence verification \(\left\|w_{i}-\theta^{t}\right\|^{2}+\left\|\theta^{t}-\theta^{t-1}\right\|^{2}<\varepsilon\) is satisfied, and stop the loop if it is satisfied. We give the definition of generalized residuals:  
\[ \begin{cases}r^{t}:=w_{i}^{t}-\theta^{t}\\ d^{t}:=\theta^{t}-\theta^{t-1}\end{cases}  \tag{7}\]  
Here, \(r^{t}\) is the primal residual at iteration \(t\), and \(d^{t}\) is the dual residual at iteration \(t\). We enable \(\rho^{t+1}\) to fluctuate within an acceptable range from iteration to iteration:  
\[\rho^{t+1}= \begin{cases}\rho^{t}/(1+\tau_{i}),&\text{ if }\left\|r^{t} \right\|<\mu\left\|d^{t}\right\|\\ (1+\tau_{i})\rho^{t},&\text{ if }\mu\left\|r^{t}\right\|\geq\left\|d^{t} \right\|\\ \rho^{t},&\text{ otherwise}\end{cases}  \tag{8}\]  
with \(\mu\in(0,1)\). This approach speeds up convergence by automating the local penalty parameter for each client, rather than relying on a global parameter for all clients. This approach eliminates the need to estimate a global penalty parameter, resulting in improved performance. For the purpose of updating the client model locally:  
\[\mathcal{L}_{i}(w_{i}^{t},y_{i}^{t},\theta^{t})=f_{i}(w_{i}^{t})+(y_{i}^{t})^ {\top}(w_{i}^{t}-\theta^{t})+\frac{\rho^{t+1}}{2}\left\|w_{i}^{t}-\theta^{t} \right\|^{2} \tag{9}\]  
In the practical case, (9) does not need to be solved exactly, we can calculate the local model \(w_{i}^{t}\) inexactly according to \(\left\|\
abla_{w_{i}}\mathcal{L}_{i}(w_{i}^{t},y_{i}^{t},\theta^{t})\right\|^ {2}\leq\varepsilon_{i}\). Here, \(\varepsilon_{i}\geq 0\) represents the local accuracy. The use of the augmented Lagrangian \(\mathcal{L}_{i}\) for training local model parameters \(w_{i}^{t}\) has proven to be effective. This is due to the fact that a suitable value for the quadratic proximal coefficient \(\rho\) can be determined, which allows the client accuracy level \(\varepsilon_{i}\) to be easily achieved. The parameter difference uploaded to the server is:  
\[\Delta_{i}^{t}=(w_{i}^{t+1}+\frac{1}{\rho^{t+1}}y_{i}^{t+1})-(w_{i}^{t}+\frac{ 1}{\rho^{t+1}}y_{i}^{t}) \tag{10}\]In our scenario, clients decide to perform different workloads to accommodate the heterogeneity of the system based on their local environment. In FedACADMM, each client holds a tuple of \(w_{i}^{t},y_{i}^{t}\), but it only transmits the enhanced model difference \(\Delta_{i}^{t}\). This means that the communication cost per round is the same as in FedAvg and FedProx.