# FedCO: Communication-Efficient Federated Learning via Clustering Optimization +
## 4 Proposed Approach
It is also possible to assign explicit weights to the clusters representing their relative

Figure 3: Flowchart depicting Cluster optimization algorithm. importance, and calculated based on their size. Our future plans include the investigation of an optimized version of the FedCO algorithm where the importance of clusters will be considered in the aggregated model. ## 5 Datasets and Experimental Setup

This section describes the datasets, the distribution of the data across the edge nodes, the model selected and related parameters, and the performance metrics used for evaluating FedCO. ### Datasets

We conducted experiments using a wide range of datasets. Firstly, we selected three benchmark datasets widely used for image classification: MNIST [40], Fashion MNIST [41], and CIFAR-10 [42]. * The MNIST dataset contains a 60,000-point training set and a 10,000 point test set with 10 classes. Each sample is based on a grayscale image of handwritten digits with a size of 28 \(\times\) 28 pixels. * The Fashion MNIST dataset comprises a 60,000-point training set and a 10,000 point testing set of images of fashion items with 10 different classes. Each image has dimensions of 28 \(\times\) 28 in grayscale. * The CIFAR-10 dataset consists of a 50,000-point training set and a 10,000-point testing set with images of objects from frogs to planes, where each image is 32 \(\times\) 32 pixels in 10 classes. Secondly, we considered two LEAF datasets [43], an open-source benchmark for Federated Learning. * FEMNIST for 62-class image classification, which serves as a more complex version of the popular MNIST dataset [44]. * CelebA for determining whether the celebrity in the image is smiling, which is based on the Large-scale CelebFaces Attributes Dataset. ### Data Distribution

In an FL context, the performance is affected by the distribution of the training data stored on the various workers. Interestingly, unlike other FL studies using clustering techniques, different degrees of non-IID data do not affect the clustering results, as FedCO clustering occurs based on the model parameters and not on the data themselves. In order to assess the impact of different data distribution scenarios, we generated two experimental datasets for each dataset introduced above:

* The IID dataset: Each worker holds the local data equal in size and label distribution. * The Non-IID dataset: Each worker holds different data distributions in size and label distribution compared to the global dataset. ### Model Selection and Parameters

We have compared the proposed FedCO algorithm against the FedAvg, CMFL, and FedProx algorithms using the Convolutional Neural Network (CNN) classifier as a training model. The CNN model we used consists of two 5 \(\times\) 5 convolution layers with a ReLU activation and a final softmax output layer. The baseline configuration parameters' values listed in Table 3 are shared among the four compared algorithms. ### Performance Metrics

FL typically relies on a large number of edge devices, sometimes in the magnitude of millions, and due to the limited computing capabilities of those devices, decreasing the communication rounds or communication overhead is crucial during the training process. Hence, the performance metrics selected are the _Number of Communication Rounds_, the _Communication Overhead_, and the model _Accuracy_. The _Communication Overhead_ is defined in [9] as

\[(N\times\mid W_{s}\mid)\times(2\times T+1),\]

where \(N\) is the size of the trained model in bytes, \(\mid W_{s}\mid\) is the number of selected workers, and \(T\) is the total number of training rounds. We assume the size of the model updates to be fixed. However, other communication costs are negligible. It is worth mentioning that the total communication overhead of FedCO can be calculated as the summation of the communication costs of the initialization stage and the iteration stage together.