# FedCO: Communication-Efficient Federated Learning via Clustering Optimization +
## 2 Related Work
### Reduction of the Total Number of Bits  
The first category incorporates works that reduce the total number of bits transferred for each local update through data compression. [PERSON] et al. [19] propose an enhanced Federated Learning technique by introducing an asynchronous learning strategy on the clients and a temporally weighted aggregation of the local models on the server. Different layers of the deep neural networks are categorized into shallow and deep layers, and the parameters of the deep layers are updated less frequently than those of the shallow layers. In addition, a temporally weighted aggregation strategy is applied on the server to make use of the previously trained local models, thereby enhancing the accuracy and convergence of the central model. [PERSON] et al. [20] design two novel strategies to reduce communication costs. The first relies on lossy compression on the global model sent from the server to the client. The second strategy uses Federated Dropout, which allows users to efficiently train locally on smaller subsets of the global model and reduces client-to-server communication and local computation. [PERSON] et al. [21] propose Deep Gradient Compression (DGC) to significantly reduce the communication bandwidth. [PERSON] et al. [22] introduce a new compression framework, entitled Sparse Ternary Compression, that is specifically designed to meet the requirements of the Federated Learning environment. [PERSON] et al. [24] implement a Federated Optimization (FedOpt) approach by designing a novel compression algorithm, entitled Sparse Compression Algorithm (SCA), for efficient communication, and then they integrate the additively homomorphic encryption with differential privacy to prevent data from being leaked. [PERSON] et al. [25] develop a novel framework that significantly decreases the size of updates while transferring weights from the deep learning model between the clients and their servers. A novel algorithm, namely FetchSGD, that compresses model updates using a Count Sketch and takes advantage of the mergeability of sketches to combine model updates from many workers, is proposed in [26]. [PERSON] et al. [27] present a federated trained ternary quantization (FTIQ) algorithm, which optimizes the quantized networks on the clients through a self-learning quantization factor. [PERSON] et al. [23] design a PowerSGD algorithm that computes a low-rank approximation of the gradient using a generalized power iteration. A novel Federated Learning method, entitled FedAT, with asynchronous tiers under Non-IID data, is presented in [28]. FedAT synergistically combines synchronous intra-tier training and asynchronous cross-tier training. By bridging the synchronous and asynchronous training through tiering, FedAT minimizes the straggler effect with improved convergence speed and test accuracy. Our research does not consider methods that leverage data compression techniques because of \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Categories** & **Existing Studies** & **ML Model Used** & **Datasets** \\ \hline \multirow{8}{*}{**Second category**} & CMFL [11] & CNN, LSTM & MNIST, NWP \\ \cline{2-4}  & FedMed [29] & LSTM & PTB, WikiText-2, Yelp \\ \cline{2-4}  & CEEP-FL [30] & CNN & MNIST, CIFAR-10 \\ \cline{2-4}  & FedCS [31] & NN & CIFAR-10, FashionMNIST \\ \cline{2-4}  & FedPSO [32] & CNN & MNIST, CIFAR-10 \\ \cline{2-4}  & AdaFL [33] & MLP, CNN & MNIST, CIFAR-10 \\ \cline{2-4}  & MAB [34] & NN, CNN & MNIST, Video QoE \\ \cline{2-4}  & FedAtt [35] & GRU & WikiText-2, PTB, Reddit \\ \cline{2-4}  & FedPAQ [13] & CNN, Logistic & MNIST, CIFAR-10 \\ \cline{2-4}  & [PERSON] et al. [36] & CNN, Logistic, RNN & Synthetic, EMNIST, Shakespeare \\ \hline \multirow{2}{*}{**C**A-FL [9]} & SGD & mHealth, Pamp2 \\ \cline{2-4}  & Proposed (FedCO) & CNN & MNIST, Fashion-MNIST, CIFAR-10, FEMNIST, CelebA \\ \hline \hline \end{tabular}
\end{table}