# Heterogeneous Distributed Big Data Clustering on Sparse Grids
## 6 Results
### Node-Level Performance and Performance-Portability  
The Figures 6 and 7 show the runtimes of the node-level experiments. For more consistent results, the runs were repeated four times and the measurements averaged. The 1M-10C dataset could be processed on a Tesla P100 in less than 20 s. Processing the 1M-100C dataset is more time-consuming and required 248 s again using a Tesla P100. The main reason for the time increase is because the density estimation requires more time due to a larger sparse grid and a smaller \(\lambda\), which leads to more CG iterations. The experiments with the 10 million data points datasets are shown in Figure 7. Due to the increased size of the datasets, the \(k\)-nearest-neighbor graph creation takes up the largest fraction of the runtime in both experiments. This illustrates that for large datasets, because of its quadratic complexity, the \(k\)-nearest-neighbor graph creation step will dominate the overall runtime. In these two experiments, increasing the number of clusters has only a small effect on the runtime. Mainly, because in both cases a sparse grid with level \(l=7\) was used. On the P100 platform, the experiments with the 10M-100C dataset took 1162 s. The other hardware platforms took longer, proportional to their lower raw performance. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Name** & \(\lambda\) & **Threshold \(t\)** & **Level** & **Grid Points** & **CG \(\epsilon\)** & \(k\)** & **ARI** & **Type** \\ \hline
1M-10C & 1E-5 & 667 & 6 & 76k & 1E-2 & 6 & 1.0 & node-level \\
1M-100C & 1E-6 & 556 & 7 & 0.4M & 1E-2 & 6 & 0.85 & node-level \\
10M-10C & 1E-5 & 1167 & 7 & 0.4M & 1E-2 & 6 & 1.0 & node-level \\
10M-100C & 1E-6 & 1000 & 7 & 0.4M & 1E-2 & 6 & 0.90 & node-level \\
10M-3C & 1E-6 & \(0.7\cdot\max(\mathbf{\alpha})\) & 7 & 0.4M & 1E-3 & 5 & - & distributed \\
100M-3C & 1E-6 & \(0.7\cdot\max(\mathbf{\alpha})\) & 8 & 1.9M & 1E-3 & 5 & - & distributed \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The parameters used for configuring the clustering algorithm and the adjusted Rand index (ARI) for the node-level experiments. In the distributed runs, the threshold was specified as a fraction of the maximum surplus of the density function. The node-level runs used an absolute threshold value. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Name** & **Size** & **Clust.** & \(\sigma\) & **Dim.** & **Dist.** & **Noise** & **Type** \\ \hline
10M-3C & 10M & 3 & 0.12 & 10 & \(3\cdot\sigma\) & 0\% & distributed \\
100M-3C & 100M & 3 & 0.12 & 10 & \(3\cdot\sigma\) & 0\% & distributed \\
1M-10C & 1M & 10 & 0.05 & 10 & \(7\cdot\sigma\) & 2\% & node-level \\
1M-100C & 1M & 100 & 0.05 & 10 & \(7\cdot\sigma\) & 2\% & node-level \\
10M-100C & 10M & 100 & 0.05 & 10 & \(7\cdot\sigma\) & 2\% & node-level \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The Gaussian datasets for the distributed and the node-level runs. Table 5 shows the performance achieved in the node-level experiments. It displays the performance in GFLOPS and the achieved fraction of peak performance. The achieved fraction of the peak performance relative to the instruction-mix-based limit is displayed as well. These results were calculated from the runs with the 10M-10C dataset as specified in Table 4.  
Figure 6: The duration of the node-level experiments with one million data points. Because the 1M-100C dataset requires a larger grid, the density estimation takes up most of the overall runtime.