# On a Framework for Federated Cluster Analysis
## 2 Materials and Methods
### The Federated Fuzzy Clustering Framework
#### 2.2.1 Federated Fuzzy \(c\)-Means with \(k\)-Means Averaging  
Our proposed federated fuzzy \(c\)-means algorithm (FedFCM) is an extension of the iterative fuzzy \(c\)-means algorithm to the federated learning setting similar to [20; 21], but with a different take on the global cluster center calculation. The global cluster calculation is similar to the one proposed in [22], where it is applied in the context of federated one-shot \(k\)-means. This idea was first mentioned and discussed in our preliminary work [37]. In the federated scenario, the data are not stored in a centralized database, but distributed among multiple clients. The goal is to learn a global clustering that is similar to the clustering of the centralized data while the data stay private. The general procedure is as follows: Each client runs a number of fuzzy \(c\)-means iterations locally, and sends the resulting cluster centers to a central server. The central server is responsible for calculating meaningful global clusters from the local learners' results. After calculating the global centers, they are shared with the clients that use them to recalculate their local centers, which in turn are shared with the central server, and so forth. That procedure is repeated until the global centers remain stable. The creation of a global model from clients' local model updates was first introduced by [38] and is known as federated averaging (FedAvg). Our averaging method is a \(k\)-means averaging that was independently developed and applied in the context of federated one-shot clustering in [22]. Let data \(X\) be distributed among \(P\) parties (clients), i.e., \(X=\bigcup_{l=1}^{P}X^{(l)}\). The protocol reads as follows:The central server initializes \(K\) global cluster centers \(c_{1},\ldots,c_{K}\). The central server shares \(c_{1},\ldots,c_{K}\) with the clients. Client \(l\) calculates membership matrix \(U_{l}^{l+1}\) according to Equation (5) and generates local cluster centers \(c_{1}^{(l)},\ldots,c_{K}^{(l)}\) according to Equation (6) (\(l=1,\ldots,P\)) and repeats until local convergence. Client \(l\) shares \(c_{1}^{(l)},\ldots,c_{K}^{(l)}\) for \(j=1,\ldots,K\) and \(l=1,\ldots,P\). The central server updates \(c_{1},\ldots,c_{K}\) by applying an averaging function \(avg([c_{1}^{(l)},\ldots,c_{K}^{(l)}]_{l=1}^{p})\). The central server checks a convergence criterion. If not converged, go back to step 2. Since the central server has only access to the local cluster centers \(c_{k}^{(l)}\), the previous convergence criterion can not be applied. As an alternative, we check whether the cluster centers changed by less than \(\epsilon\) between two iterations. Let \(c_{k_{t}}\) be the global cluster center \(k\) after time step \(t\). Then, the convergence criterion can be formulated as follows: \(\sum_{k=1}^{K}||c_{k_{t}}-c_{k_{t+1}}||\leq\epsilon\). Note that this new criterion might lead to different cluster centers than in the previous formulation. It might let the centers move closer to the center of mass even though the assignments might have stabilized already. In order to find meaningful global clusters, it is essential to find a good averaging function \(avg(\cdot)\) used in step 5 of the framework outlined above. To address this challenge, we apply a \(k\)-means averaging function, similar to the one in [22]:  
\[ \begin{split} avg:\mathbb{R}^{P\times d\times K}& \rightarrow\mathbb{R}^{d\times K}\\ avg([c_{k}^{(l)}]_{k,l})&:=kmeans([c_{k}^{(l)}]_ {k,l}),\\ &=[c_{k}]_{k},\end{split}  \tag{14}\]  
where \(kmeans(\cdot)\) denotes a function that applies the \(k\)-means clustering algorithm and outputs the \(k\) cluster centers it converged to. This averaging function applies the \(k\)-means algorithm to all reported local cluster centers to find new global cluster centers. It does introduce increased complexity compared to federated fuzzy \(c\)-means in [20; 21], but we observed robust results in preliminary experiments [37].