# Communication-Efficient Federated Learning with Adaptive Consensus ADMM
## 5 Experiments
### Experimental Setup  
**Datasets and Models.** In our study, we adopted the standard federated configuration where every client owns their local data and communicates with the central server to transmit and receive information. Our experiments involved handwriting recognition and image classification using three well-known benchmark datasets: MNIST [35], Fashion MNIST (FMNIST) [36], and CIFAR-10 [37]. MNIST consists of handwritten digits from zero to nine. FMNIST includes images of 10 different categories of clothing, such as T-shirts, pants, and sneakers. CIFAR-10 is a dataset composed of 10 different types of objects, including aircraft, cars, dogs, and so on. To conduct these experiments, we utilized two CNN models, each comprising two convolutional layers, two fully connected layers, and a linear transformation layer. The detailed architecture of the models is provided in Table 2.  
**Data Distribution.** To investigate federated optimization and conduct comparative experiments, we adopted the data distribution settings used in the FedAvg approach. Specifically, we conducted experiments using two different data division methods: IID (homogeneous) and non-IID (heterogeneous). Under the IID method, clients receive an equal distribution of data. Under the non-IID method, we arranged the training data by labels and then distributed them evenly among the shards. Each client was randomly and evenly assigned two shards.  
**Validation metrics.** To assess the generalization performance of our approaches, we applied the entire test set in well-known federated datasets. Because convergence speed and final performance are important indicators of federated learning, we computed the performance attained at a specified number of rounds and the number of communication rounds required for the algorithm to reach the test target accuracy. Typically, the ADMM approach converges to a reasonable accuracy within a few tens of iterations. Therefore, we selected one hundred experimental rounds for all datasets after preliminary estimation. We acquired a reasonable evaluation target accuracy displayed in Table 3. For techniques that did not achieve the desired accuracy within the maximum communication round (\(t>100\)), we appended a plus sign (\(+\)) to the communication round. Additionally, we  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Datasets** & **Task** & **\# Parameter** & **Model** \\ \hline MNIST & Handwritten recognition & 1,663,370 & 2-layer CNN + FC \\ CIFAR-10 & Image classification & 1,105,098 & 2-layer CNN + FC \\ FMNIST & Image classification & 1,663,370 & 2-layer CNN + FC \\ \hline \hline \end{tabular}
\end{table}
Table 2: Datasets and models.  
reported the communication round savings of FedACADMM compared to the classical FedAvg algorithm.  
**Implementation Details.** We have implemented FedACADMM in PyTorch, which involves simulating a federated learning network with one server and \(m\) clients. To enable comparisons with other baselines, we reused the implementations of FedAvg and FedProx. All of our numerical experiments were conducted on a cloud computing platform, where we used a virtual machine with the following specifications: 4 vCPUs, 16 GB memory, and an NVIDIA RTX A4000 GPU. The operating system utilized for the training was Ubuntu 18.04 LTS, and we employed PyTorch 1.8.1 as our deep learning framework for training.  
We set the global learning rate to 1 for all methods and used a client subset size equal to 10% of the total number of customers (\(C=0.1\)). For all algorithms, we utilized the SGD optimizer with a learning rate of 0.1 for local updates.