# On a Framework for Federated Cluster Analysis
## 3 Results
### Experiments
#### 3.2.2 Evaluation on Benchmark Data  
We test our framework on cluster benchmark sets from an online repository ([[http://cs.uef.fi/sipu/datasets/](http://cs.uef.fi/sipu/datasets/)]([http://cs.uef.fi/sipu/datasets/](http://cs.uef.fi/sipu/datasets/)), accessed on 17 March 2022) [41]. In particular, the G2 sets were introduced in [42] and each set was generated by drawing 2048 samples from two Gaussian distributions with different means, i.e., each set contains two ground truth centers. The Gaussians are centered at \(\mu_{1}=(500,500,\dots)\) and \(\mu_{2}=(600,600,\dots)\) with standard deviations \(\sigma\in\{10,20,\dots,100\}\) and dimension \(D\in\{2,4,8,\dots,1024\}\). In total, there are 100 sets with varying dimension and standard deviation. In order to evaluate the federated clustering framework, we randomly (but uniformly) distribute the points among \(P\in\{2,5,10\}\) clients. Note that the number of samples is fixed to be 2048 such that with an increasing number of clients, each clients has fewer samples. Each experiment is repeated 20 times and the averages over the runs are reported. The first step is to determine the correct number of clusters for each G2 set. We follow the procedure described in Section 2.2.3 for \(K\in[2,\dots,6]\) and choose the minimum as the framework's guess. An overview of the results can be found in Table 3 and Figures 2 and 3 for the framework with \(k\)-means averaging. Moreover, the complete framework is evaluated with the federated averaging method for comparison. Results are summarized in Table 3 as well. For the calculation of the correct number of cluster guesses metric in Table 3, we only consider datasets with clustering results with a federated fuzzy Davies-Bouldin index below 1.3. Through exploratory analysis we found that a higher Davies-Bouldin index often shows that the algorithm could not converge (which we also discuss later in this section). In such cases, the framework's cluster number guess is meaningless, because the clustering itself is not meaningful. Note that the value of 1.3 coincides with the 75% quantile in all scenarios (central, \(k\)-means averaging, and federated averaging) such that the number of considered datasets is similar for all evaluations. We want to demonstrate the effect of sparsity by taking a closer look at the full G2 set for the \(k\)-means averaging. The performance generally worsens with increasing \(\sigma\) (Figure 3; averaged over all \(D\)), independent of the number of clients. For \(\sigma\in\{10,20,30\}\), the detection rate is close to one. For \(\sigma\in\{40,50,60,70,80\}\), the detection rate is between \(0.8\) and \(0.9\) with lower numbers for higher number of clients. Finally, there is a noticeable performance drop when \(\sigma\in\{90,100\}\) with the steepest decline when \(P\in\{5,10\}\), where in the majority of cases, the correct number of cases is not detected. Moreover, we also observe detection rates varying across dimensions (Figure 3; averaged over all \(\sigma\)). For \(D=2\), the correct number of clusters is only detected in \(0.25\) to \(0.3\) of all cases. Then, it peaks with a detection rate close to \(1\) for \(D=8\) before decreasing and stabilizing at \(D=64\) and being constant thereafter. The trend is similar for all \(P\), with small exceptions for \(P=2\). However, the level of detection rate is smallest for \(P\in\{5,10\}\), with the exception of \(D=2\), where it is even slightly higher than in the central case. Second, we report the results of the clustering itself in terms of (federated) fuzzy Davies-Boulding index and knowledge gap. Overall, we see that the results are mostly similar in the central case and in the federated scenarios for \(P\in\{2,5,10\}\) for either clustering method: The \(0.25\), \(0.5\), and \(0.75\) quantiles and the minimum values for both metrics are similar. However, the maximum value for the federated fuzzy Davies-Bouldin index shows some variation (Tables 4 and 5).