# Communication-Efficient Federated Learning with Adaptive Consensus ADMM
## 2 Related Work  
Since FL was put forward [8], it has been committed to solving the security problems caused by the diversity and complexity of communication networks and the difficulties in obtaining private data [22]. In recent years, the research and discussion on federated optimization have been very active. [PERSON] et al. [23] presented a federated multi-task learning approach for heterogeneity, MOCHA, which takes into account communication cost, stragglers, and error tolerance. This method is not suitable for non-convex models. The objective function of a non-convex model is not a convex function, which is a function where the straight line connecting any two points on the function lies completely above the function. Therefore, non-convex models can have multiple local minima and maximums, and their optimization problems are more complex. In non-convex settings, FedAvg is shown to be effective, but does not perform very well in the real-world situation of device heterogeneity [8]. FedProx modifies FedAvg by adding a weighted proximal term to the local objective function to account for heterogeneity [9]. It might be challenging to choose the weighted proximal term in FedProx for complicated purposes. In addition, subsequent research found that the answer to the question of whether FedProx can improve the speed of convergence is unclear [24]. A communication-efficient method, q-FFL, is inspired by techniques of fair resource allocation in wireless networks, and it alters the objective function of FedAvg by modifying the weights of different devices [25]. It performs well in both convex and non-convex models. Numerous studies have been published that discuss the function of ADMM in distributed machine learning [11]. In [26], this idea derived from the R-ADMM as an application of the Peaceman-Rachford splitting. It demonstrated a virtually certain convergence of the proposed algorithm under broad assumptions about the distribution of communication loss and node activation events by utilizing findings from operator theory. The authors of [27] derived a perturbation strategy for ADMM in which the perturbed term is connected with the penalty parameters; it was shown that this simultaneously increases utility and privacy. The authors of [27] derived a perturbation strategy for ADMM, where the perturbed term is linked to the penalty parameters, which increases the utility and privacy at the same time. The primal-dual methods are also widely used in FL. Based on the Peaceman-Rachford splitting approach, [28] developed FedSplit to solve the distributed convex problem and proved that FedSplit can solve FL problems under convexity only without any additional assumptions about the homogeneity of the system or data. FedSplit is only applicable to deterministic algorithms, excluding the random approximate gradient update method commonly utilized in practice. FedPD is a primitive dual-based meta-learning algorithm, which uses the dual variable to calculate the difference between local and server learning [29]. We note that all clients in FedPD are continuously participating in training. [PERSON]. [30] considered both synchronous and asynchronous settings and applied the non-convex Douglas-Rachford splitting method, a random block coordinate strategy, and asynchronous updating to solve the non-convex composite optimization problem in federated learning. FedDR and asyncFedDR, unlike FedSplit and FedPD, only update a subset of customers at every round of updates and can update synchronously or asynchronously. Our research was inspired by [31; 32]. To increasing the robustness of federated optimization, in [31], FedADMM adds the local objective function with dual and augmented Lagrangian terms. It should be noted that this method works extremely effectively on large-scale networks with diverse data. Ref. [32] concentrated on a useful modification of the decomposition method by [16], allowing the penalty parameter to alter automatically based on specific adaptive principles.