###### Abstract  
Advances in next-generation sequencing have provided high-dimensional RNA-seq datasets, allowing the stratification of some tumor patients based on their transcriptomic profiles. Machine learning methods have been used to reduce and cluster high-dimensional data. Recently, uniform manifold approximation and projection (UMAP) was applied to project genomic datasets in low-dimensional Euclidean latent space. Here, we evaluated how different representations of the UMAP embedding can impact the analysis of breast cancer (BC) stratification. We projected BC RNA-seq data on Euclidean, spherical, and hyperbolic spaces, and stratified BC patients via clustering algorithms. We also proposed a pipeline to yield more reproducible clustering outputs. The results show how the selection of the latent space can affect downstream stratification results and suggest that the exploration of different geometrical representations is recommended to explore data structure and samples' relationships. UMAP; dimensionality reduction; clustering; embedding geometry; RNA-seq; breast cancer; tumor stratification; reproducibility +
Footnote †: journal: MedicalOf these, uniform manifold approximation and projection (UMAP) [12] has been widely used in several tumor stratification and RNA-seq analyses [13; 14; 15; 16]. Unlike other popular dimensionality reduction methods, such as principal component analysis (PCA) [17], UMAP is nonlinear and emphasizes local data structures well. It is therefore used in a preprocessing step to reduce the dimensionality of a genomic dataset, either before implementing a full ML model [18] or before visualizing complex expression profiling data [19]. A recent comparative study [20] showed how UMAP can also considerably improve the performances of clustering algorithms. In general, users tune the number of neighbors, i.e., the main UMAP hyperparameters, to project local data structures in a low-dimensional Euclidean space. Nevertheless, several other tuning parameters can be specified. In particular, users can choose the embedding metric onto which UMAP projects the data. By default ([[https://umap-learn.readthedocs.io/en/latest/parameters.html](https://umap-learn.readthedocs.io/en/latest/parameters.html)]([https://umap-learn.readthedocs.io/en/latest/parameters.html](https://umap-learn.readthedocs.io/en/latest/parameters.html)), accessed on 20 April 2022), it is a Euclidean distance. Basically, the more the geometry of the latent space matches the structure of the input data, the more important the quality of the embedding representation [21]; therefore, different metric spaces (onto which to project the data) might reveal different sample relationships and structures. Recently, [PERSON]. [22] proposed a deep generative model to embed cells into low-dimensional hyperspherical or hyperbolic spaces, providing excellent visualization for data exploration. In another study, [PERSON]. [23] demonstrated how a hyperbolic embedding can reveal meaningful hierarchies among samples starting from pairwise similarity information. Hence, when we cannot know the geometry of the latent space a priori, different types of metrics should be explored. To the best of our knowledge, the literature contains no reports on investigating different curvatures of the UMAP embedding space. Given the above, here we evaluated how non-Euclidean UMAP embeddings can impact the results of a breast cancer stratification analysis. We used UMAP on RNA-seq data with three different output metrics: Euclidean, spherical, and hyperbolic. Then, we stratified the samples via clustering algorithms applied on each UMAP embedding. We then compared the results in terms of clustering accuracy. For the present study, we used the RNA-seq breast cancer data downloaded from The Cancer Genome Atlas (TCGA) website. Details of the dataset are presented in the Methods section.