# Communication-Efficient Federated Learning with Adaptive Consensus ADMM
## 1 Introduction
Theoretically, ADMM converges to any constant penalty parameter [12; 13; 14]; however, in practice, the efficiency of an ADMM is highly dependent on the selection of the parameter [15] and can be enhanced using adaptive penalty selection methods [16]. To summarize, taking into account the effectiveness of ADMM in distributed optimization, we have chosen to implement it in the context of federated learning (FL). Several challenges differentiate optimization problems in federated learning, also known as federation optimization, from traditional distributed optimization. These challenges include user privacy, costly communication, and system and statistical heterogeneity [9]. The user privacy challenge arises because although federated learning protects  
Figure 1: A framework for FL. The IoT devices, which act as the clients in federation learning, send the trained model to the central server, rather than the raw data. In this setup, remote devices communicate with the central server periodically to learn the global model. The steps performed in each round of communication are illustrated in the figure. data on devices by sharing model updates, communicating these updates throughout the training process can expose sensitive information to third parties and the central server [17]. The high cost of communication refers to the need for communication-efficient methods in the vast IoT network, where it is necessary to send small messages or model updates iteratively as part of the training process, instead of sending the entire dataset over the network, to adapt the model to the data generated by devices in the federated network [18]. System heterogeneity refers to differences in the storage, computational, and communication capabilities of IoT devices participating in federated learning. Statistical heterogeneity refers to differences in the way devices collect and generate data, as well as differences in the amount of data across devices. Clients participating in federated learning are typically highly heterogeneous in terms of both the size of their local datasets and computational speed [19]. Such challenges are common in real-world Internet of Things application scenarios. For example, there are many edge devices with computing power in the Internet of Things (these devices can be diverse, including wearable sensors, mobile phones, watches, and smart appliances). The data they generate are also diverse, and they are widely distributed across the Internet of Things; although they want to participate in federated learning training, they do not want their privacy to be compromised. Expensive communication represents a significant bottleneck in federated learning, where a considerable number of clients are widely distributed, necessitating local data retention. In actual IoT communication networks, communication can be several orders of magnitude slower than local computation [20; 21], thereby highlighting the urgent need for the development of communication-efficient methods. This approach considers two key aspects: (i) reducing the total number of communication rounds and (ii) decreasing the size of the transmitted information per round. In this work, we propose a new federated optimization algorithm that combines adaptive consensus ADMM. This method successfully overcomes heterogeneity and converges in fewer communication rounds. The main contributions of this paper are summarized as follows. * We propose and demonstrate a communication-efficient federated optimization algorithm, which performs well in heterogeneous settings. The proposed method uses an adaptive consensus ADMM method to select local parameters. Instead of estimating a single global penalty parameter for all clients, different local penalty parameters are generated for each client based on the local gradient of the subproblems. * We propose and adopt a new method similar to the double-rising and multiplier method to update the objective function of the customer. By using this method, the gradients of the server and the individual clients are aligned.