Title: t-SNE Exaggerates Clusters, Provably
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: nonlinear dimension reduction, data visualization, t-SNE

Abstract: Central to the widespread use of t-distributed stochastic neighbor embedding (t-SNE) is the conviction that it produces visualizations whose structure roughly matches that of the input. To the contrary, we prove that (1) the strength of the input clustering, and (2) the extremity of outlier points, cannot be reliably inferred from the t-SNE output. We demonstrate the prevalence of these failure modes in practice as well.

TL;DR: t-SNE visualizations can overemphasize clusters and suppress outliers significantly