# FedCO: Communication-Efficient Federated Learning via Clustering Optimization +
## 2 Related Work  
This section mainly reviews the published research works aimed at reducing communication overheads in FL. In general, Federated Learning requires massive communication between the central server and the workers to train a global model [6]. Such an overhead is imputed to the size of the model exchanged and to the number of rounds to converge. Many works aim at reducing communication costs; e.g., HeteroFL [12] utilizes models of different sizes to address heterogeneous clients equipped with different computation and communication capabilities, while the work in [13] uses decentralized collaborative learning in combination with the master-slave model.  
Among many of the published FL solutions, there are few existing FL works that use clustering techniques [14; 15; 16; 17; 18]. For example, in [14] the study proposes clustering algorithms based on clients' similarities. The authors have tried to find a cluster structure of data to collect clients with similar data distributions and to perform baseline FedAvg training per cluster. In [15], the authors introduce clustering techniques to partition the clients with similar data distribution using a measure of distance between the weight updates of the clients. A dynamic clustering through generative adversarial network-based clustering (GAN) is designed to obtain a partition of the data distributed on FL clients in [16]. The authors in [17] introduced a new framework, namely the Iterative Federated Clustering Algorithm (IFCA), in which clusters of users also aggregate their data with others in the same cluster (the same learning task) and optimize model parameters for the user clusters via gradient descent. Finally, [PERSON] et al. [18] present clustering algorithms to cluster the heterogeneous data across clients into various clusters to participate in global model learning. The authors grouped the data after reducing its dimensions using PCA, and they measured the similarity of local updates.  
Although the studies discussed above [14; 15; 16; 17; 18] have applied clustering techniques to FL scenarios, all of them have clustered the clients based on the distribution of their own data, while our proposed technique partitions the clients based on their training model parameters, i.e., in a way that ensures that each cluster will contribute to the model by learning different aspects (different model parameters' values) of the studied phenomenon. Evidently, our solution for mitigating communication costs of FL is conceptually different from the approaches discussed above, despite it also being based on clustering.  
The majority of the studies in the field of resource-aware FL can be distributed into two main categories: a reduction in the total number of bits transferred, and a reduction in the number of local updates. Table 1 summarizes the techniques proposed by the research community, classifying them according to the categorization mentioned above.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Categories** & **Existing Studies** & **ML Model Used** & **Datasets** \\ \hline \multirow{8}{*}{**First category**} & [PERSON] et al. [19] & CNN, LSTM & MNIST, HAR \\ \cline{2-4}  & Fed-Dropout [20] & DNN & CIFAR-10, MNIST, EMNIST \\ \cline{2-4}  & [PERSON] et al. [21] & CNNs, RNNs & Cifar10, ImageNet, \\  & & & Penn Treebank \\ \cline{2-4}  & STC [22] & VGG11, CNN & CIFAR-10, MNIST \\ \cline{2-4}  & PowerSGD [23] & ResNet-18, LSTM & CIFAR10, WIKITEXT-2 \\ \cline{2-4}  & FedOpt [24] & NN, LM & CIFAR10, MNIST \\ \cline{2-4}  & FEDZIP [25] & CNN, VGG16 & MNIST, EMNIST \\ \cline{2-4}  & FetchSGD [26] & NN & CIFAR-100, CIFAR-10, \\  & & & FEMNIST \\ \cline{2-4}  & T-FedAvg [27] & MLP, ResNet-18 & MNIST, CIFAR-10 \\ \hline \multirow{2}{*}{**First category**} & \multirow{2}{*}{CNN, Logistic} & CIFAR-10, Fashion-MNIST, \\  & & & Sentiment140, FEMNIST, \\ \cline{2-4}  & & & Reddit \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of recent studies to minimize communication overhead in FL.