Cluster algorithms are gaining in popularity due to their compelling ability to identify discrete subgroups in data, and their increasing accessibility in mainstream programming languages and statistical software. While researchers can follow guidelines to choose the right algorithms, and to determine what constitutes convincing clustering , there are no firmly established ways of computing a priori statistical power for cluster analysis. Here, we take a simulation approach to estimate power and classification accuracy for popular analysis pipelines . We systematically varied cluster size, number of clusters, number of different features between clusters, effect sizewithin each different feature, and cluster covariance structurein generated datasets . We then subjected these datasets to common dimensionality reduction approaches (none, multi-dimensional scaling, or uniform manifold approximation and projection ) and cluster algorithms (k-means, hierarchical agglomerative clustering with Ward linkage and Euclidean distance, or average linkage and  cosine distance, HDBSCAN). Furthermore, we simulated additional datasets to explore the effect of sample size and cluster separation on statistical power and classification accuracy . We found that clustering outcomes were driven by large effect sizes or the accumulation of many smaller effects across features, and were mostly unaffected by differences in covariance structure. Sufficient statistical power can be achieved with relatively small samples (N=20 per subgroup), provided cluster separation is large ({\Delta}=4). Finally, we discuss whether fuzzy clustering (c-means) could provide a more parsimonious  alternative for identifying separable multivariate normal distributions, particularly those with  lower centroid separation  } .
Cluster algorithms are increasingly popular in biomedical research due to their compelling ability to identify discrete subgroups in data, and their increasing accessibility in mainstream software. While guidelines exist for algorithm selection and outcome evaluation , there are no firmly established ways of computing a priori statistical power for cluster analysis. Here, we estimated power and accuracy for common analysis pipelines through simulation. We varied subgroup size, number , separation (effect size), and covariance structure . We then subjected generated datasets to dimensionality reduction (none, multidimensional scaling, or UMAP ) and cluster algorithms (k-means, agglomerative hierarchical clustering with Ward or average linkage and Euclidean or cosine distance, HDBSCAN). Finally, we compared the statistical power of discrete (k-means), "fuzzy" (c-means), and finite mixture modelling approaches (which include latent profile and latent class analysis) . We found that outcomes were driven by large effect sizes or the accumulation of many smaller effects across features, and were unaffected by differences in covariance structure. Sufficient statistical power was achieved with relatively small samples (N=20 per subgroup), provided cluster separation is large ({\Delta}=4). Fuzzy clustering provided a more parsimonious and powerful alternative for identifying separable multivariate normal distributions, particularly those with slightly lower centroid separation ( \Delta}=3). Overall, we recommend that researchers 1) only apply cluster analysis when large subgroup separation is expected, 2) aim for sample sizes of N=20 to N=30 per expected subgroup, 3) use multidimensional scaling to improve cluster separation, and 4) use fuzzy clustering or finite mixture modelling approaches that are more powerful and more parsimonious with partially overlapping multivariate normal distributions .
R
gaining in popularity
increasingly popular in biomedical research
D
programming languages and statistical
R
researchers can follow guidelines to choose the right algorithms, and to determine what constitutes convincing clustering
guidelines exist for algorithm selection and outcome evaluation
R
take a simulation approach to estimate power and classification accuracy for popular analysis pipelines . We systematically varied cluster
estimated power and accuracy for common analysis pipelines through simulation. We varied subgroup
R
of clusters, number of different features between clusters, effect sizewithin each different feature, and cluster covariance structurein generated datasets
, separation (effect size), and covariance structure
R
these datasets to common dimensionality reduction approaches
generated datasets to dimensionality reduction
R
multi-dimensional
multidimensional
R
uniform manifold approximation and projection
UMAP
R
hierarchical agglomerative
agglomerative hierarchical
D
linkage and Euclidean distance,
A
Euclidean or
R
Furthermore, we simulated additional datasets to explore the effect of sample size and cluster separation on statistical power and classification accuracy
Finally, we compared the statistical power of discrete (k-means), "fuzzy" (c-means), and finite mixture modelling approaches (which include latent profile and latent class analysis)
D
clustering
D
mostly
R
can be
was
R
Finally, we discuss whether fuzzy clustering (c-means) could provide
Fuzzy clustering provided
A
and powerful
A
slightly
A
(
A
\Delta
A
=3). Overall, we recommend that researchers 1) only apply cluster analysis when large subgroup separation is expected, 2) aim for sample sizes of N=20 to N=30 per expected subgroup, 3) use multidimensional scaling to improve cluster separation, and 4) use fuzzy clustering or finite mixture modelling approaches that are more powerful and more parsimonious with partially overlapping multivariate normal distributions