# FedCO: Communication-Efficient Federated Learning via Clustering Optimization +
## 4 Proposed Approach
Therefore, at each iteration, we can calculate the global update by aggregating the local updates by using those cluster representatives, which have reached a high accuracy level at this iteration phase. Two scenarios are considered to this end: (i) a global model is trained by FedAvg based on updates made by randomly selected workers, regardless of their accuracy value; (ii) a global model is trained by applying FedCO, and in that way, at each training round, only workers (cluster representatives) that have achieved the highest accuracy values are used. Let us briefly summarize the working mechanism of the proposed FedCO algorithm. In the \(t\)th global training iteration, each worker involved (\(w_{i}\in W_{t}\)) calculates the average \(g_{t}^{i}\) gradients using the optimization algorithms in the local dataset in the current global model

Figure 4: The clustering updates in the first five global communication rounds of the proposed FedCO algorithm applied on the Non-IID FashionMNIST dataset. Notice that the number in the circle represents the cluster label. \(\mathcal{M}_{t}\). Note that according to Equation (5), high-quality data and a high accuracy of the workers' models can lead to a faster convergence of the local loss functions (Equation (3)) and the global loss function (Equation (4)) [45]. Both the local model update \(\mathcal{M}_{t}^{i}\) of the worker in Equation (1) and the shared global model update \(\mathcal{M}_{t+1}\) in Equation (5) can be more quick to converge to the target value with fewer iterations. Consequently, the training time of a worker in a global iteration is decreasing. Therefore, highly accurate workers' models can significantly improve the learning efficiency of Federated Learning; e.g., it can ensure less training time [31, 46]. This process is iterative until a global accuracy \(\epsilon\) (\(0\leq\epsilon\leq 1\)) is achieved. Specifically, each update of the local model has a local accuracy \(w_{i}^{e}\) that corresponds to the local quality of the worker \(w_{i}\) data. A higher local accuracy leads to fewer local and global iterations [46, 47]. FedCO uses an iterative approach that requires a series of communication rounds to achieve a level of global accuracy \(\epsilon\). Server and representative communications occur during each global round of the iteration phase. Specifically, each representative minimizes its objective \(F_{i}(\mathcal{M}^{i})\) in Equation (3) using the local training data \(n_{i}\). Minimizing \(F(\mathcal{M})\) in Equation (4) also requires multiple local iterations up to a target accuracy. Then, the global rounds will be bounded as follows:

\[\frac{\mathcal{O}(\log{(\frac{1}{\epsilon})})}{1-w_{i}^{e}}\]

Thus, the global rounds are affected by both the global accuracy \(\epsilon\) and the local accuracy \(w_{i}^{e}\). When \(\epsilon\) and \(w_{i}^{e}\) are high, FedCO needs to run a few global rounds. On the other hand, each global round consists of both computation and transmission time. Our primary motivation in this work is to consider the communication overhead, discussed and analyzed in detail in Section 6.3. The computation time (\(w_{i}^{cmp}\)), however, depends on the number of local iterations. When the global accuracy \(\epsilon\) is fixed, the computation time is bound by \(\log(\frac{1}{w_{i}^{e}})\) for an iterative algorithm to solve Equation 1; here, (SGD) is used [46]. Therefore, the total time of one global communication round for a set of representatives is denoted as

\[T^{com}=\sum_{w_{i}\in W_{t}}\log(\frac{1}{w_{i}^{e}})w_{i}^{cmp}+w_{i}^{com},\]

where \(w_{i}^{com}\) represents the transmission time of a local model update. As a result, a high local accuracy value of \(w_{i}^{e}\) leads to fewer local iterations \(w_{i}^{cmp}\) and eventually to lower global communication rounds \(T^{com}\). Unlike FedCO's convergence rate, FedAvg does not necessarily guarantee a faster convergence speed. This is because FedAvg uses a much larger number of workers compared to the FedCO model.