# FedCO: Communication-Efficient Federated Learning via Clustering Optimization +
## 4 Proposed Approach
\(|\ C_{tj}\)\(|>1\)do
5:\(s(C_{tj})\leftarrow\)SilhouetteCluster(\(C_{tj}\), \(W_{t}\))
6:while\(s(C_{tj})<0\)do
7:\((C_{tj}^{1},C_{tj}^{2})\leftarrow\)Kmedoids(\(\{\mathcal{M}_{t+1}^{i}\mid w_{i}\in C_{tj}\},k=2\))\(\triangleright\) run 2-medoids to generate two new clusters
8:\(\tilde{C}_{t}\leftarrow\{C_{t}\setminus\{C_{tj}\}\}\cup\{C_{tj}^{1},C_{tj}^{2}\}\)
9:\(s(\tilde{C}_{t})\leftarrow\)SilhouetteScore(\(k_{t}\), \(C_{t}\), \(W_{t}\))
10:if\(s(\tilde{C}_{t})>s(C_{t})\)then
11:\(C_{t}^{\prime}\gets C_{t}^{\prime}\cup\tilde{C}_{t}\)
12:\(k_{t}\gets k_{t}+1\)
13:endif
14:endwhile
15:endfor
16:return(\(C_{t}^{\prime}\), \(k_{t}\))
17:endprocedure
18:functionSilhouetteScore(\(k_{t}\), \(C_{t}\), \(W_{t}\))\(\triangleright\)Silhouette score of whole cluster solution \(C_{t}\)
19: Compute \(s(w_{i})\) between each \(w_{i}\in W_{t}\) and each medoid \(c_{tj}\in C_{tj}\) (\(j=1,2,\ldots,k_{t}\)) according to (6)
20: Compute the average Silhouette score over all representatives \(w_{i}\in W_{t}\) according to (7)
21:return\(s(C_{t})\)
22:endfunction
23:functionSilhouetteCluster(\(C_{tj}\), \(W_{t}\))\(\triangleright\)Silhouette Score of cluster \(C_{tj}\in C_{t}\) (\(j=1,2,\ldots,k\))
24: Calculate Silhouette score \(s(w_{i})\) for each \(w_{i}\in C_{tj}\) according to (6)
25: Compute the mean over Silhouette scores of all cluster members \(\{s(w_{i})\mid w_{i}\in C_{tj}\}\) according to (8)
26:return\(s(C_{tj})\)
27:endfunction
```

**Algorithm 2** ClusteringOptimization

### Initialization Phase

1. At time \(t=0\), the Server initializes the inputs for the FedCO algorithm (Algorithm 1). These are the model \(\mathcal{M}_{0}\), the set of representative workers \(W_{t}\), the number of clusters \(k_{t}\), and the number of iterations \(T\). \(t=0\) (line 1 in Algorithm 1). 2. A central Server transmits the initial global model \(\mathcal{M}_{t}\) to a set of workers \(W_{t}\) (\(W_{t}\subset W\)). These are selected to be used for initial training in round \(t=0\) of Federated Learning (lines 3 in Algorithm 1). 3. Each worker \(w_{i}\in W_{t}\) receives the global model \(\mathcal{M}_{t}\) and optimizes its parameters locally; i.e., the \(\mathcal{M}_{t}^{i}\) initial update is produced and sent back to the Server (Equation (1)) (lines 4-6 and lines 38-45 in Algorithm 1). 4. The Server aggregates the parameters \(\{\mathcal{M}_{t}^{i}\mid w_{i}\in W_{t}\}\) uploaded by the selected workers \(W_{t}\) to update the global model \(\mathcal{M}_{t}\) through the FedAvg algorithm (Equation (2)) (line 7 in Algorithm 1). 5. The local updates \(\{\mathcal{M}_{t}^{i}\mid w_{i}\in W_{t}\}\) of the workers in \(W_{t}\) are analyzed by using the \(k\)-medoids clustering algorithm (functionKMedoids, line 8 in Algorithm 1)). As a result, \(k_{t}\) clusters of workers with similar updates are obtained; i.e., an initial clustering \(C_{t}=\{C_{t1},C_{t2},\ldots,C_{k_{t}}\}\) of the workers in \(W_{t}\) is produced. ### Iteration Phase

1. At each iteration round \(t\) (\(t\geq 0\)), the Server evaluates each local update \(\mathcal{M}_{t}^{i}\), \(w_{i}\in W_{t}\) by using an evaluation measure that is suitable for the task under consideration. It ranks the workers in each cluster \(C_{tj},j=1,2,\ldots,k_{t}\) with respect to their evaluation scores and selects the top-ranked worker, i.e., the representative (function SelectTopRanked, line 11 in Algorithm 1). The selected representatives form a new set of workers \(W_{t+1}\), where \(\mid W_{t+1}\mid=k_{t}\) and \(k_{t}<<\mid W_{0}\mid\). Each selected worker \(w_{i}\in W_{t+1}\) will check in with the Server. 2. The Server sends the global model \(\mathcal{M}_{t}\) to each representative \(w_{i}\in W_{t+1}\) (line 12 in Algorithm 1). 3. Each representative \(w_{i}\in W_{t+1}\) receives the global model \(\mathcal{M}_{t}\) and optimizes its parameters locally; i.e., the \(\mathcal{M}_{t+1}^{i}\) update is produced (Equation 1) and sent back to the Server (lines 13 and 15 in Algorithm 1). 4.