Question: What is the real benefit compared to t-SNE or UMAP?
Evidence: Dimensionality reduction (DR) and visualization of high-dimensional data is of theoretical and practical value in machine learning and related fields. In theory, there exists an intriguing, non-intuitive discrepancy between the geometry of highdimensional space and low-dimensional space. Based on this discrepancy, we propose a novel DR and visualization method called Space-based Manifold Approximation and Projection (SpaceMAP). Our method establishes a quantitative space transformation to address the "crowding problem" in DR. With the proposed equivalent extended distance (EED) and function distortion (FD) theory, we are able to match the capacity of high-dimensional and low-dimensional space in a principled manner. To handle complex high-dimensional data with different manifold properties, SpaceMAP makes distinctions between the near field, middle field, and far field of data distribution in a data-specific, hierarchical manner. We evaluated SpaceMAP on a range of artificial and real datasets with different manifold properties, and demonstrated its excellent performance in comparison with classical and state-of-the-art DR methods. In addition, the concept of space expansion provides a generic framework for understanding nonlinear DR methods including t-distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP).
We applied the proposed SpaceMAP to a variety of datasets and computed quantitative metrics to evaluate the performance of DR. The method is benchmarked by other classical or state-of-the-art methods including PCA, Laplacian Eigenmaps, t-SNE, and UMAP.
We tested on a large range of high-dimensional datasets, including the standard MNIST, Fashion-MNIST, Swiss Roll (Van der Maaten et al., 2007) (Here we artificially created a hole on the Swiss Roll to challenge the DR methods if they can preserve the local property on a continuous manifold),, RNA-seq, and an experimental cardiac MRI dataset from three different vendors. Examples of cardiac MRI are given in Fig.. The details of the chosen datasets are in Appendix A.6. Here the original data is simply vectorized and taken as input of DR. We also tested all methods on the Google-News Word2Vec 3 Million dataset. For quantitative evaluation, we computed the 20-fold cross-validated k-nearest neighbor classifier accuracy (KNN accuracy), trustworthiness, continuity, Shepard goodness, and normalized stress to evaluate both the local and global structure preservation of the datasets. KNN accuracy measures the local structure preservation along different neighborhood scale k. Trustworthiness and continuity evaluate the local pattern of the embedding by calculating the true neighbor rate and the missing neighbor rate. The Shepard goodness and the normalized stress are two measurements of the goodness in global structure preservation. The detailed introduction of these evaluation metrics is in Appendix A.7.
Our method establishes a quantitative space transformation to address the “crowding problem” in DR. With the proposed equivalent extended distance (EED) and function distortion (FD) theory, we are able to match the capacity of high-dimensional and low-dimensional space in a principled manner.****For quantitative evaluation, we computed the 20-fold cross-validated k-nearest neighbor classifier accuracy (KNN accuracy), trustworthiness, continuity, Shepard goodness, and normalized stress to evaluate both the local and global structure preservation of the datasets (Espadoto et al., 2021; Nonato & Aupetit, 2019).
Answer: We have emphasized this point in the revision, in both Introduction and Discussion part. In short, the SpaceMAP has two major benefits: (1) SpaceMAP provides an explicit space transformation between LD and HD, while in contrast t-SNE or UMAP make highly implicit transformation, which may not fit the data (Figure 1 as an example). The explicity of space transformation by SpaceMAP, derived from geometrical measure, potentially better preserves the relative distance in the LD map and makes it possible to enforce data priors (i.e. when the intrinsic dimensionality in some cases is known, such as Swiss Roll). (2) The hierarchical formulation of SpaceMAP takes into consideration the local and global structure of the manifold, and is compatible to a wider range of datasets (disjoint or continuous, e.g. both MNIST and Swiss Roll). Furthermore, we have added more description to interpret the experiment results as suggested.