# Optimization Algorithms for Scalable Stream Batch Clustering with k Estimation
## 6 Conclusions  
In this work, six algorithms designed to cluster scalable stream batches and estimate the number of clusters were experimentally compared. SESC was the fastest algorithm due to its summarizing power, while SPKMS achieved high-quality results for most data sets due to its goodness-of-fit tests over projections, a heuristic for seeding prototypes, and partition refinement by \(k\)-means. Moreover, ISESC-CD and ISESC stand out for their quality results, while ISESC-CD achieved competitive processing time. Considering the algorithms based on goodness-of-fit tests, SGMS was the fastest, obtaining quality results for the normally distributed data sets. Additionally, the evolutionary algorithms obtained the best results, in terms of quality, for real data sets. Such results show that our proposed algorithms can complement each other for various application scenarios. The experiments support our hypothesis that the specialist approach for clustering data sets with normal distribution could improve the accuracy in scenarios of scalable batch streams, as the most accurate results for the normal distribution data sets were reached by SGMS and SPKMS. As a general guide, a suitable algorithm may be selected according to the distribution of the data set and the available computational resources. The goodness-of-fit-based algorithms (SGMS or SPKMS) should be preferred if normality is expected in the data set. SGMS is faster and can handle scenarios where the available computational resources render the SPKMS unfeasible. However, the latter obtained the results with the best quality. If the clusters are not expected to fit a normal distribution or their distribution is unknown, the evolutionary algorithms, SESC or ISESC-CD, are preferred due to their average quality. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **SESC** & **ISESC** & **ISESC-CD** & SGMS & **SPGMS** & **SPKMS** \\ \hline
100K5-2 & **0.78 (0.02)** & 26.26 (3.04) & **5.97 (0.63)** & 8.76 (0.45) & \begin{tabular}{c} 60.74 \\ (17.23) \\ \end{tabular}  & 33.14 (7.46) \\
1M5-2 & **1.03 (0.01)** & 40.69 (6.97) & **12.11 (1.51)** & 25.59 (1.22) & \begin{tabular}{c} 162.97 \\ (11.75) \\ \end{tabular}  & 151.79 \\
1M10-5 & **2.75 (0.44)** & \ \begin{tabular}{c} 89.73 \\ (12.52) \\ \end{tabular}  & **20.89 (2.38)** & 35.70 (3.04) & \begin{tabular}{c} 284.68 \\ (32.12) \\ \end{tabular}  & 316.05 \\  
SESC is fast and recommended for scenarios with limited computational resources, while ISESC-CD has the best quality, requiring more computational resources. According to the experiments, ISESC and SPGMS did not stand out during the experiments. Although ISESC reached quality results close to ISESC-CD, the former is considerably slower. SPGMS showed that adapting PG-means for distributed and parallel batch streams is burdensome and sometimes unfeasible, as the expectation-maximization step needs to access the data multiple times. However, it inspired SPKMS, the adaptation of SPGMS that uses k-means evaluated by silhouette, which is faster and achieves the best-quality results, one of the main findings of this paper. Concerning arbitrary-shaped clusters, we intend to investigate how to scale density-based clustering algorithms for this task in future work. Conceptualization and methodology, P.G.L.C., J.A.S., E.R.F. and M.C.N. ; software, P.G.L.C. ; validation, J.A.S., E.R.F. and M.C.N. ; formal analysis, J.A.S. and E.R.F. ; investigation, P.G.L.C. ; resources, M.C.N. ; data curation, P.G.L.C. ; writing--original draft preparation, P.G.L.C. and M.C.N. ; writing--review and editing, J.A.S. and E.R.F. ; visualization, P.G.L.C. ; supervision, M.C.N. ; project administration, M.C.N. ; funding acquisition, M.C.N. All authors have read and agreed to the published version of the manuscript. Not applicable. The real datasets experimentally compared here are public domain and made available by their original developers.