DREAMS: Preserving both Local and Global Structure in Dimensionality Reduction
Authors: Noël Kury
Dmitry Kobak
Sebastian Damrich
Venue: Accepted by TMLR ()

Abstract: Dimensionality reduction techniques are widely used for visualizing high-dimensional data in two dimensions. Existing methods are typically designed to preserve either local (e.g., $t$-SNE, UMAP) or global (e.g., MDS, PCA) structure of the data, but none of the established methods can represent both aspects well. In this paper, we present DREAMS (Dimensionality Reduction Enhanced Across Multiple Scales), a method that combines the local structure preservation of $t$-SNE with the global structure preservation of PCA via a simple regularization term. Our approach generates a spectrum of embeddings between the locally well-structured $t$-SNE embedding and the globally well-structured PCA embedding, efficiently balancing both local and global structure preservation. We benchmark DREAMS across eleven real-world datasets, showcasing qualitatively and quantitatively its superior ability to preserve structure across multiple scales compared to previous approaches.

Published in Transactions on Machine Learning Research (01/2026)
DREAMS: Preserving both Local and Global Structure in Dimensionality Reduction
Noël Kury
noel-elia noel-elia.krry@ästueeee.. kwry@@sttuentt..m/tuueiinnnn...
Hertie Instätute for AT in Brain Health University of Tübingen, Germory
Dmitry Kobak
Hertie Institude for AI in Brain Health University of Tübingen, Germany
Sebastian Damrich
Hertie Institute for Al in Brain Health University of Tübingen, Germany
Reviewed on OpenReview: /ttps://openpeee opwrreveee.........om net/ forum? id- 2p cho
Abstract
Dimensionality reduction techniques are widely used for visualizing high-dimensional data in two dimensions. Existing methods are typically designed to preserve either local (e.g., I-SNE, UMAP) or global (e.g., MDS, PCA) structure of the data, but none of the es- tablished methods can represent both aspects well. In this paper, we present DREAMS (Dimensionality Reduction Enhanced Across Multiple Scales), a method that combines the local structure preservation of t-SNE with the global structure preservation of PCA via simple regularization term. Our approach generates spectrum of embeddings between the locally well-structured t-SNE embedding and the globally well-structured PCA embedding, efficiently balancing both local and global structure preservation. We benchmark DREAMS across eleven real-world datasets, showcasing qualitatively and quantitatively its superior ability to preserve structure across multiple scales compared to previous approaches.
PCA
DREAMS
E-SNE
Inhibitory
Inhibitory
neurons
neurons
NO
66
excitatory
NP Ls CT
LSII
L6 NP
L6
LA 12 IT
L5 NP
12a
L4
non-neurans
PC1
neuronss
Figure 1: PCA, DREAMS, and I-SNE embeddings of the Tasic et al. dataset illustrate how DREAMS preserves the global organization seen in the PCA embedding - such as the separation of non-neurons, inhibitory, and excitatory neurons - while also capturing the local cell-type structure that is present in the t-SNE embedding.

---

Published in Transactions on Machine Learning Research (01/2026)
1 Introduction
Real-world data often exhibit complex structures, making their effective and interpretable visualization a crucial step in exploratory data analysis. Dimensionality reduction methods serve this purpose by projecting high-dimennional data into more interpretable low-dimensional representations while preserving meaningful structures (de Bodt et al., 2025). Among dimensionality reduction methods, principal component analysis (PCA; Hotelling [933) and i-distributed stochastic neighbor embedding (I-SNE; van der Maaten and Hinton, 2008 have emerged two widely used methods. PCA excels at capturing global structures by projecting data onto a lower-dimensional subspace in directions that maximize variance, providing broad overview of the datasct's structure. In contrast, t-SNE is maighbo--embedding method with the objective to map data points that are nearby in high-dinensional space close lo one another in the low-dimensional embedding. Its focus on neighborhood preservation makes t-SNE particularly effective in preserving local structures. When applied individually, both methods suffer from limitations: PCA often overlooks fine local relationships, while t-SNE distorts global structures in favor of local neighborhood preservation.
However, in many real-world datasets, e.g., single-cell transcriptomic datasets, both local and global struc- ture is meaningful. Local relationships can reveal microscopic data patterns such as small clusters and cell types. For example, the Tasic et al. dataset features 122 fine clusters that represent cell types, which are clearly separated in the t-SNE plot (Figure right). Conversely, the global structure reflects macroscopic data patterns such as separations between broad cell classes (Tasic et al. 2018) or developmental trajecto- ries (Kanton et al., 2019) across an entire cell population. In the Tasic et al. dataset, this level of structure separates individual cells into non-neurons, excitatory, and inhibitory neurons. This global structure is not evident in the t-SNE plot, but is prominent in the PCA visualization, which in turn fails to separate the finer clusters [Figure 1 left). Since both local and global structure carry essential and often complementary information, neglecting either structure scale can result in incomplete or biased interpretations, highlighting the importance of preserving both scales simultancously.
Our proposed method DREAMS combines the interpretability and global structure preservation of PCA with the local sensitivity of t-SNE in simple, yet effective way. On the TTasic et al.] data, it maintains PCA's global arrangement of non-neurons, inhibitory, and excitatory neurons, while also revealing the finer cluster structure within each broad group, similar to t-SNE (Figure 1 middle). DREAMS integrates global structure preservation into the I-SNE objective by a PCA-based regularization term applied throughout the entire optimization process. Varying the regularization strength allows DREAMS to transition from the locally well-structured I-SNE embedding to the globally well-structurel PCA embedding. Along this spectrum it trades off local and global strueture more favorably than its competitors, resulting in improved qualitative and quantitative structure preservation, particularly on datasets with hicrarchical organization that exhibit both prominent local and global patterns.
We chose PCA the default global method in DREAMS due to its simplicity, inherent interpretability (PCA is lincar) and fast runtime. That said, DREAMS also offers regularizing with other global embeddings such as multidimensional scaling (MDS).
In summary, we introduce the new method DREAMS making the following contributions:
1. suggest simple but effective regularization strategy to combine t-SNE's local with PCA's global quality;
2. present spectrum of visualizations with stale-of-the-art trade-off between local and global structure;
3. perform benchmark of ten algorithms on eleven real-world datasets;
4. provide an open-source implementation of DREAMS based on the openTSNE library.
Our code is available at http://tth....oooooooaoaaaaaaaaa modified оpenTSNE implementation a modified CNE implementation available at http:s//gttaabb..cmm/rrrreaaaa/s
We used a
at
and
2
Related Work
The most prominent and well-established visualization methods either excel at preserving local structure, e.g. neighbor-embedddngg methods I-SNE (van der Maaten and Hinton 2008) and UMAP (MeInnes et al.
2

---

Published in Transactions on Machine Learning Research (01/2026)
Hotelling 1933) and MDS Kruskal 1964). but not both simultaneously. Therefore, many recent efforts have aimed at producing visualizations with faithful local and global structure.
2018], or global structure, c.g., PCA Pearson 1901
Most of them attempt to improve the global structure preservation of neighbor embeddings. One way to make neighbor embeddings more sensitive to the global structure of a dataset to widen the set of points that are deemed similar beyond a small set of nearest neighbors. For instance, increasing the perplexity parameter in t-SNE effectively increases the number of considered nearest neighbors but consequently leads to increased runtime (Kobak and Berens, 2019 Lee et al., 2015 De Bodt et al. 2020). Similarly, several modifications of UMAP, e.g., PaCMAP (Wang et al. 2021) and TriMAP (Amid and Warmuth| 2019), do not only use nearest neighbors, but also consider more distant points in their optimization. In addition to attraction between nearest neighbors, PaCMAP employs weak attraction on mid-near points, while TriMap tries to also preserve the order of similarities in random triplets. Another triplet-basod method, ivis (Szubert ct al. 2019). strives to balance local and global structure with a parametric encoder and margin loss on triplets of embedding distances. Changing the similarity of input points more drastically, EmbedOR (Saidi et al. 2025) computes neighbor embedding from distance matrix based on the curvature, and thus geometry, of the k-ncarest neighbor graph.
A different strategy to improve the global structure in neighbor embeddings is to initialize them with a global embedding, which can improve the global structure of the final embedding, despite no further steps for preserving global structure during optimization (Kobak and Linderman, 2021 Wang et al. 2021). UMATO (Jeon et al., 2025) first computes a skeletal layout of only the most densely connected points to capture the global layout and then adds the remaining points in second optimization phase.
A prominent way to trade off local and global structures within the neighborr-mbbeedling framework is the attraction-repullion spectrum (Böhm et al. 2022 Damrich et al.) 2023). Along this spectrum methods with stronger between-neighbor attraction tend to focus on more global structure. UMAP and t-SNE both lie on this spectrum with UMAP having more attraction. The most global method on this attraction-repullion spectrum is Laplacian Figenmaps (Belkin and Niyogi 2003).
Recently, hybrid methods were proposed that combine elements of neighbor embodding methods and global methods during optimization. Several of these are variational autoencoders with 2D latent space and regularized ELBO maximization. The method sevis (Ding et al. 2018) uses Gaussian latent prior and adds a variant of the t-SNE objective to the ELBO. Its successor VAE-SNE (Graving and Couzin 2020 employs a more flexible Gaussian mixture prior. Instead of a regularizer promoting local structure proservation, ViVAE Novak et al.] 2023) adds a stochastic MDS regularizer, but denoises the high-dimensional data basod on k-nearest neighbor relations.
More similar to our approach tions. local-to-Global Structures (Miller et al., 2023), method for generic graph drawing, applies MDS-like stress minimization to shortest path graph distances among pairs of points that are strongly connected, while repelling non-neighboring pairs. Cluster and Embed (Coda et al. 2025) embeds clusters separately, focusing on their local structure. These fixed cluster embeddings then get arranged into a full embedding by rigid transformations that minimize the overall stress like in MDS. The MadaddDD--bbbrrt (Lambert et al. 2022) interpolates between the objectives of t-SNE and MDS. For MDS it uses the stochastic quartet framework of Lambert et al.] (2022). To make both objectives more compatible, SQuadMDS-hybrid normalizes their gradients before blending them together. The hybrid method StarMAP (Watanabe et al. 2025) changes the attractive force in the UMAP objective. Instead of pulling only ncarest neighbors together, it also pulls points towards the PCA coordinates of their k-means cluster centroid. This combination of neighbor em- bedding with PCA is similar to our DREAMS. But instead of UMAP, we use I-SNE, due to its better local structure preservation. Moreover, we propose a simpler objective that avoids StarMAP's clustering step, leaves the saighbor-embeddingg objective intact, and simply pulls each embedding point towards its own PCA position. We found that this simpler approach leads to a better local-global trade-off. Additionally, sctting the regularization strength in DREAMS to or to 1 allows to recover standard t-SNE and PCA, while the StarMAP framework cannot fully recover PCA.
non-paramttric hybrid methods that directly optimize the embedding posi-
3

---

Published in Transactions on Machine Learning Research (01/2026)
Different from neighbor-enbeeddnng inniiieeddddddidddd methods, PHATE MMoon al. 2019 is diffusion- based method that tries to balance local and global structure preservation. Like neighbor embeddings, it starts with the k-nearest neighbor graph of the high-dimensional data. It then integrates this local information into a global graph distance. PHATE uses potential distance, a variant of diffusion distance that focuses more on global structure. To visualize this global distance metric in 2D, PHATE uses MDS. Geomelry--eguularivel autoencoders Duquu et al. 2022) regularize their 2D latent space with a procomputed PHATE embedding. DREAMS also uses reference embedding for regularization, but is non-parametric, uses PCA MDS, and employs the t-SNE loss instead of the reconstruction loss.
3 Background
In dimensionality reduction, we aim to represent a high- dighnsimensio dataset X = [I1,,..., Fn] E F with n observations in an m-dimensional space by a lower-dimensional embedding Y = [M],..., yo]] E Raxd where d < m. The objective is to construct Y such that meaningful relationships among the observations are preserved in the lower-dinensional space. In this section, we outline the two dimensionality reduction methods that DREAMS builds upon PCA and t-SNE and show how they approach this goal from complementary perspoctives.
3.1 Principal component analysis (PCA)
linear transformation that projects the data onto coordinate system aligned with the directions of maximum variance. PCA seeks linear, orthogonal mapping W E Rmxx that projects the data into lower-dimensional space Y = XW, where the directions in W capture the maximal variance across the entire dataset X. This ensures the preservation of the macroscopic, global structure since distances along directions with high variance are preserved in the projection (Huang et al. 2022). However, because orthogonal projections can map distant points to similar locations, PCA performs poorly at preserving local structures (Huang et al. 2022 Wang al. 2023
Principal component analysis (PCA; Pearson 1901; Hotelling, 1933)
3.2 d-distributed stochastic neighbor embedding (t-SNE)
t-distributed stochastic neighbor embodding (t-SNE; van der Maaten and Hinton, 2008 is a widely used neiglbor-cmbedding method that espocially effective preserving local similarities (Espadoto al. 2019). By transforming Fuclidean distances between points into pairwise similarity probabilities, I-SNE constructs probability distribution P, based the high-dimensional observations X, and probability distribution Q, based on the low-dimensional embeddings Y. The distribution P = {piy |? encodes the neirest structure in the high-dimensional space via
Pjja + Pijj where Pate Σxys exp -0, xx 1x (2/)))
exp (-1|z; x;,13/(207))
Pij
if + j and Paje 0.
2n
The width 0; of the Gaussian kernels is adaptively chosen for each data point to ensure the same effective neighborhood size, i.e., the number of points j for which Pylı > O. Due to the exponential decrease of the Gaussians, most Pıj are close to zero, and are treated as exactly zero in most implementations.
The similarity probability distribution of the low-dimensional embedding points Q = K the Cauchy kernel:
based on
(1 + |1 1),l3)
ExSl (1+ 1lykk 3i(12) -I if H and qi 0.
The objective is to arrange the dov-dimensional embedding Y such that the low-dimensional similarities Øo match the similarities P:y as measured by the Kullback -eibbe divergence
LYSNE(Y) = KL(P || Q) = -
Pij log
Pij
(1)
43
i.j

---

Published in Transactions on Machine Learning Research (01/2026)
which is minimized via gradient descent with respect to the embedding positions Y.
The similarity probabilities act as kernels centered around the data and embedding points, assigning high similarity values only to close neighbors, while distant points have little impact on the loss function. This makes neighbors the high-dimensional space remain neighbors in the low-dimensional embedding space, ensuring the preservation of local structure. In contrast, due to the weak influence of distant points, global structure can be misrepresented (Huang et al., 2022 Wang al. 2023
4 Methods
4.1 Regularizing with precomputed global embedding
In DREAMS, we first precompute the PCA positions Ý R as the reference embedding for the global structure of the data. To combine the local structure preservation of t-SNE with the global structure preservation of PCA, we augment the t-SNE loss function by a regularization term that penalizes embedding points Y deviating from their PCA positions Y, yielding the loss
((Y) = (1 (NN))) + 1| - 18
(2)
Since a PCA embedding scales linearly with the original data scale, while I-SNE embedding does not, we rescale Y to match the scale of Y during each gradient-descent iteration by computing scalar
a= |
(3)
This scaling encourages the reference embedding to match the current scale of the t-SNE embedding during optimization, making the global and local objectives more compatible. Note that a was treated as constant in each gradient descent iteration. We discuss alternative scaling options in Appendix B
The first term of Equation 2 enforces the preservation of local structure in the embedding by minimizing the t-SNE loss (Equation 10, while the regularization term ensures the preservation of global structure by encouraging the embedding to resemble the (scaled) PCA embedding Y. This setup allows for local adjust- ments by the t-SNE loss, while the quadratic penalty prevents large deviations that would distort the global layout. The regularization strength À controls the impact of the PCA embedding on the final embedding, thereby enabling a trade-off between local and global structure preservation. For X = 0, DREAMS produces a standard t-SNE embedding since the loss term (Equation 2) reduces to the t-SNE objective. In the other limiting case of regularization strength = 1, the objective becomes the regularizer without the I-SNE loss whose optimum a (scaled) PCA embedding. For intermediate (0, 1) the objective mcan between the t-SNE objective and the regularizer. Empirically, found X 0.15 to be a suitable value for combining the individual strengths of global strueture preservation of PCA with the local structure preservation of ISNE, without substantially compromising either aspect.
weighted
This framework is not limited to using the PCA embedding as the global reference embedding Y. Depending on the characteristics of the dataset, other methods with good global structure preservation, such as MDS, can also be used as an effective choice for Y. We will refer to the version of DREAMS that uses the SQuadMDS embedding for regularization as DREAMS-MDS. By default, we use PCA instead of MDS because PCA is faster and is inherently interpretable due to its linearity.
Our implementation is based on the openTSNE library (Polićar et al. 2024), where we added the regularization term to the gradient-descent optimization. We used default openTSNE iyperparameters and initialized the embedding with the regularization embedding Y,, which was rescaled so that its first dimension had a standard deviation of 10-4 (as default in openTSNE).
The oSenTSNE library implements Barnes Hut t-SNE (Yang et al. 2013 Van Der Maaten 2014 and FIt-
SNE (Linderman et al. 2019) approximations having runtime complexity Onn and ((n) respectively.
5

---

Published in Transactions on Machine Learning Research (01/2026)
4.2 Linear decoding regularization
An alternative approach lo using precomputed PCA embedding is using linear decoder. Equivalent to variance maximization, PCA can also be obtained by minimizing the reconstruction error
W = arg min || XWW [| subject to W W Id-
(4)
W WeRxx
Since the PCA embedding is given by y XW, the DREAMS loss with linear decoding regularizer becomes
C(Y D) (1 A)CL.NNE X ||| (YD +b]l
with Rmxd being a trainable linear decoder and b R" a trainable bias term that is added row-wise and allows to handle uncentered embeddings. In this sctup, by minimizing the reconstruction error, the decoder is responsible for the global structure preservation by pushing the embedding towards the PCA structure that has the minimal reconstruction error. If D were constrained to have orthogonal columns (D D Ta):
the optimum of the regularization term would be = W (Plaut 2018 Nazari et al. 2023 Although we do not explicitly enforce orthogonality, we observed that the learned linear decoder D naturally tends to have
approximately orthogonal columns.
We based our implementation on InfoNC-I-SNE (Damrich et al. 2023 a GPU-based contrastive learning approximation of t-SNE with the InfoNCE loss, implemented in PyTorch as CNE (contrastive neighbor embedding) package. This allowed us to add the decoding regularizer based on a linear PyTorch layer. We increased the number of negative samples to 500 to improve the local structure preservation and approx- imate 1-SNE more closely. For the same reason, we ran the optimization for 750 epochs. The remaining hyperparameters were kept at default values. For the regularization term, we used a linear layer mapping the low-dimensional embedding to the original feature space. The weights were initialized using the first two principal components of the data (the bias term was initialized with zero). We will refer to this version DREAMS-CNE--ecode
For comparison, we also implemented a version of DREAMS using the CNE backend with precomputed PCA regularization as in Section 4.1 We will refer to this version as DREAMS-CNT Here the gradient with respect was computed using autodifferentiation, including the ||||| contribution a.
5
Experimental setup
5.1 Datasets and performance metrics
To validate our method experimentally, we used eleven real-world datasets, all but one containing both prominent local and global structures (Table 1). We measured the embedding quality using two established metrics quantifying local and global structure preservation (Kobak and Berens 2019
KNN The k-nearest neighbor recall (KNN) is the fraction of k-nearest neighbors in the high-dimensional data that are preserved as k-nearest neighbors the low-dimensional embedding. We used 10 throughout all experiments (for results with different choices of k, see Figure S5). The final metric is given the average across all data points. KNN quantifies the preservation of local structure in the embedding.
CPD
The correlation of pairwise distances (CPD) is the Spearman correlation between the pairwisc dis- lances in the high-dimensional space and in the embedding. We computed pairwise distances among 1 000 randomly chosen data points. CPD quantifies the preservation of the global structure in the embedding.
We also evaluated an aggregated local-global score s. For each dataset, we normalized both metrics basod on the minimum and the maximum values observed across all methods. Let KNN and CPD denote the lowest (worst) KNN and CPD scores across all methods for given dataset, and KNN max and CPDmax

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Datasets used in our experiments with a description of their local and global structure. For MNIST and Fashion MNIST, only used the training set.
Name
Description
Global structure
Local str.
Tasic ct al.
scRNA-soq of mouse cortex major cell classes
cell types
23 822
Macosko al.
scRNA-seq of mouse retina Kanton et al.
major cell classes
cell types
44 808
scRNA-seq of human brain
developmental trajectory
cell types
20 272
Wagner et al.
scRNA-seq of zebrafish embryos scRNA-seq of C. elegans
developmental trajectory developmental trajectory continental ancestry
cell types
63 530
Packer et al.
cell types
86 024
T000 Genomes
human whole-genome sequencing 3D Mammoth skeleton
populations
450
Mammoth
overall body shape soil color shoes /bags/ none
bone structure
50 000
Satellite
satellite image crops
soil types
435
FMNIST
images of fashion items
shoes/bags/garmentt garments
10 classes
60 000
MNIST
hand-written digits vehicle and animal images
digits 0-9
60 000
CIFARIO
vehicles vs animals
10 classes
60 000
denote the highest (best) scores. Given embedding with specific KNN and CPD values, we define the aggregated local-global score as
KNN - KNN min
CPD CPD
2 KNN max KNNNinn
CPD max - CCD
This score ranges from 0 (embedding has the worst local and the worst global scores) to 1 (embedding has the best local and the best global scores) and allows direct comparison of embedding methods regarding their combined local and global structure preservation. Note that it only produces a relative score as depends on the considered competitors. For an intuitive understanding of the local-global score, see Figure S4
These metrics rely solely the high-dimensional data X and its corresponding embedding Y and do not make use of any metadata listed in [able 1 In contrast, metrics like classification accuracy, or class separation measured via the Silhouette score, agreement between clusters and classes measured via the adjusted Rand index, all require class labels.
5.2 Comparison methods
We validated DREAMS against several baseline and hybrid methods (always using the default hyperparam- eter settings):
i-SNE (van der Maaten and Hinton 2008 using the openTSNE implementation Poličar et al. 2024
and UMAP (MeInnes et al. 2018 as basclines for local structure preservation;
PCA and MDS using the SQuadMDS implementation [Lambert et al. 2022 as baselines for global structure preservation;
TriMap (Amid and Warmuth 2019). PacMAP WWng et al. 2023), and PHATE Moon et al. 2019
as methods that strive preserve both local and global structure;
and hybrid approaches SQuadMDS-hybrit (Lambert al. 2022) and StarMAP (Watanabe al, 2025), which, like DREAMS, mix the local structure preservation of neighbor embeddings with global methods.
We always report means and uncertaintios (often barcly perceptible) across four random seeds. All experi- ments were conducted on a single Intel Xeon Gold 6226R CPU 2.90 GHz (16 cores, 32 threads), an NVIDIA RTX A6000 GPU (48 GB VRAM, CUDA 12.7), and 377 GB system RAM, running on a Limux environ- utilized the GPU. Without parallelization, the runtime of all experiments was about week. For runtime analysis of DREAMS and its competitors, see Figure $2
ment. Only experiments including DREAMS-CNT CNE or

---

Published in Transactions on Machine Learning Research (01/2026)
1-SNE
DREAMS
A=0.0
A 0.05
= TriMap
= StarMAP
>=1.0
SQuadMDS-hybrid
UMAP
d
900 Inhibitorr neurons
Excitatory neurons
Non-neural cells
Figure Embeddings of the Tasic et. al.] datasel. a: Spectrum of DREAMS embeddings for different values of regularization strength A. b-c: Embeddings obtained by some of the competing methods. For all embeddings, see Figure S6
6
Results
6.1 DREAMS successfully combines the strengths of (-SNE and PCA
We first illustrate DREAMS using the Tasic et al. dataset. When adjusting the regularization strength DREAMS generates continuum of embeddings between the extremes of the locally focused I-SNE bedding and the globally focused PCA embodding (Figure 2p). For small À values, the resulting embedding resembled the (-SNE embedding, effectively capturing the local data structure and highlighting fine-grained clusters and cell types. For larger values, the embedding progressively shifted towards the PCA em- bedding, emphasizing higher-level groupings, such as broad cell classes anniiitooyycxxcitattonn non-neuronal cells). At intermediate regularization strengths, DREAMS integrated both local and global structures without visibly compromising either aspect the data. Competing methods often missed much e). e).
ncurons and
of the global structure Figure 2p,c) or showed less local structure (Figure
The quantitative metrics corroborated the ability of DREAMS to maintain both local and global structure (Figure 3). On all datasets but Mammoth, 1-SNE achieved the highest KNN value, providing the best locally structured embedding. In contrast, PCA and MDS maintained global structure best, as reflected in their CPD values being the highest. For nearly all regularization parameters E [0, 1] DREAMS yielded embeddings with better local or global structure preservation than its competitors. On most. datasets, DREAMS with its default regularization strength (A = 0.15) simultancously achieved KNN close t-SNE's and CPD close to PCA's. Across all datasets, DREAMS preserved the local structure much better than StarMAP, which also combines neighbor embeddings with PCA. This is likely because DREAMS relies t-SNE, which preserves local structure better than UMAP, which is used in StarMAP.
UMAP generally performed poorly in our metrics, often preserving not only the local but also the global structure worse than t-SNE, as measured by the CPD metric. This may be due to different default initializa- tions of openTSNE and UMAP (PCA and Laplacian Eigenmaps, respectively). That said, note that UMAP is known to perform well in clustering- based metrics (Espadoto et al., 2019 Xiang et al., 2021 Lause al. 2024), which do not use for evaluation in this work.
8

---

Published in Transactions on Machine Learning Research (01/2026)
Tasic et al.
b Macosko et al.
c
Kanton et al.
d Wagner et al.
e
Packer et al.
f 1000 Genomes
KNN (local)
0.4
0.3
0.2
0.1
0.0
0.7
1.0
0.65
0.80
0.95
0.6
0.8
1.0
0.50
0.75
1.00
0.5
0.7
0.9
0.30
0.65
1.00
Mammoth
h
Satellite
FMNIST
MNIST
k
CIFAR10
0.8
KNN (local]
0.4
0.2
0.0
1.0
0.70
0.85
1.00
0.6
0.8
1.0
0.25
0.50
0.75
0.60
0.90
CPD (global)
CPD (global)
CPD (global)
CPD (global)
CPD (global)
DREAMS
DREAMS-MDS (A=0.15)
openTSNE
SQuadMDS
TriMap
PHATE
DREAMS
(A=0.15)
PCA
UMAP
SQuadMDS-hybrld
StarMAP
PaCMAP
Figure 3: Quantitative evaluation of local and global structure preservation of different methods across multiple datasets [Table 1p. Spearman correlation of pairwise distances (CPD, global metric) is plotted against ANN recall (KNN, local metric). For improved visual clarity different markers were used.
TriMap performed similarly well StarMap (note that both use PCA initialization), but still fell short compared to DREAMS in terms of local structure preservation. PHATE and PacMAP yiclded worse CPD and worse KNN scores compared to DREAMS all datasets.
On the MNIST dataset DREAMS was not able to maintain high global and local structure at the same time, likely because this datasct does not have prominent global structure in the first place (CPD values were comparatively low for all methods including PCA and MDS). Similarly, on the pixel-valued CIFAR10 dataset, DREAMS incurred a clear trade-off between local and global structure. On this challenging dataset all mothods severely struggled (Figure S16). Nevertheless, even in these two cases, DREAMS was on par or outperformed most other methods (Figure 3).k).
Using aggregated local-global score, we could directly compare DREAMS to other methods along single dimension (Table 2). The highest local-global score every dataset was achieved by either DREAMS or DREAMS-MDS, depending whether PCA or MDS was capturing global structure more effectively in terms of the CPD metric. Following DREAMS, SQuadMDS-hybrid was consistently the next best method. It achieved scores comparable to DREAMS on three datasets, but on remaining datasets, its score was lower by least. 0.06 than the best performing DREAMS variant. On the datasets where DREAMS achieved good balance between local and global performance (all but MNIST and CIFAR10), all other methods had scores lower by least 0.09 than the best performing DREAMS variant.
6.2 DREAMS provides the best local-global spectrum
Next, we compared the local-global embedding spectrum of DREAMS to the local-global spectra of existing methods and variants of DREAMS.
In SQuadMDDShyybll the spectrum of embeddings is obtained by specifying the learning rates of t-SNE and MDS. For StarMAP, varying the regularization strength allows trading off local and global structure. Furthermore, increasing the exaggeration parameter in openTSNE can emphasize global structure preservation and also yields local-global continuum of embeddings (Böhm al. 2022

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: Aggregated local-global score. For each dataset, the methods within 0.05 of the highest score are highlighted in bold. The order of the datasets is in Table 1
Tas
Mac
Kan
Wag
Pac
1kG
Mam
Sat
FMN
MN1
C10
DREAMS
0.88
0.85
0.88
0.84
0.62
0.89
0.95
0.95
0.83
0.61
0.66
DREAMS-MDS
0.87
0.78
0.90
0.90
0.83
0.93
0.95
0.94
0.87
0.64
0.62
SQuadMDS-hybrid
0.71
0.61
0.77
0.76
0.64
0.88
0.94
0.84
0.81
0.61
0.58
StarMAP
0.68
0.57
0.66
0.59
0.39
0.66
0.68
0.70
0.48
0.26
0.16
PHATE
0.25
0.42
0.40
0.39
0.29
0.35
0.01
0.47
0.14
0.19
0.03
PaCMAP
0.45
0.34
0.42
0.39
0.36
0.51
0.70
0.35
0.12
0.20
0.15
TriMap
0.66
0.55
0.56
0.58
0.51
0.55
0.68
0.58
0.24
0.10
0.25
UMAP
0.26
0.46
0.26
0.19
0.40
0.31
0.68
0.39
0.18
0.19
0.16
t-SNE
0.57
0.50
0.60
0.61
0.50
0.64
0.86
0.50
0.55
0.61
0.56
PCA
0.50
0.50
0.48
0.46
0.25
0.43
0.60
0.48
0.44
0.35
0.32
MDS
0.48
0.47
0.52
0.51
0.51
0.55
0.50
0.54
0.51
0.50
0.50
We observed that these spectra offered noticeably worse trade-offs than DREAMS, as illustrated on the Tasic et al. dataset in Figure 4h, where PCA produces the best global layout. Similarly, DREAMS-MDS achieves better trade-off than SQuadMDSShybrrdd, demonstrated the Packer et al. dataset. ( FFiuure 4p), where MDS produces the best global layout.
Moreover, while DREAMS includes standard I-SNE and PCA/MDS as its corner cases for À = 0 and X = (marked with stars in Figure 4), neither SQuadMDS-hhyln nor StarMAP could reach MDS and PCA, respectively, in their most global sctting. Additionally, SQuadMDS-hybrid underperformed compared to I-SNE in its most local configuration (Figure 4p,b). This oulcome expected for StarMAP, as ils objective continues blend aspects of UMAP and PCA even at its highest regularization strength. For SQuadMDS- hybrid, the reason likely the normalization of the t-SNE and MDS gradients before combining them.
Switching the neighbbo-eenbeeddnng backend of DREAMS from openTSNE to contrastive neighbor embed- dings (CNE; Damrich et al. 2023) with the InfoNCE loss decreased the KNN score, proportionally to the difference in KNN score between I-SNE and its InfoNCE version InfoNC-L-SNE Figure 40). Using the CNE backend, we compared regularization using a precomputed PCA embedding (DREAMS CNE) with linear decoding regularization (DREAMS--NE-Deeoder CNE-Decoder) and observed only marginal improvements with the de- coder approach (Figure S3). The MNIST dataset was the only one where the decoder approach performed much belter in terms of our metries, but this did not translate into visible improvements of the embedding structure. Although the decoder could reconstruct the PCA embedding and enhance global structure, it introduces additional randomness and computational complexity. Moreover, with precomputed global em- bedding, we are more flexible to adjust the global layout, e.g., by switching from PCA to MDS. For these reasons and especially due to the better local structure of openTSNE than InfoNC-t-SNE, we prefer the simple regularization using a precomputed embedding as in the openTSNE-baaset DREAMS variants.
7 Discussion
In this work, we introduced DREAMS, a dimensionality reduction method that adds a regularization term to the t-SNE objective. This regularization term penalizes embedding points that deviate from their PCA positions and thereby encourages global structure preservation throughout the optimization process. Our approach addresses critical shortcoming in conventional dimensionality reduction techniques, which often prioritize one structural scale over the other. Through this simple yel. effective regularization term and a tunable hyperparameter X, DREAMS allows for a continuous spectrum of embeddings that transition smoothly from the locally faithful structure of t-SNE the globally coherent and interpretable structure of PCA. Furthermore, with its default regularization strength (A 0.15), DREAMS provides an embedding that successfully balances local and global structure preservation, outperforming competing methods.
10

---

Published in Transactions on Machine Learning Research (01/2026)
Competing methods (on lasic et al.)
MDS hybrids (on Packer et al.)
DREAMS variants (on Tasic al.)
K-SNE
0.4
InfoNC- C'SNE A-O
0.3-
0.2-
0.1-
DREAMS
hybrid
DREAMS
StarMAP
DREAMS
DREANS-NDA SEAANS-NDSS
apenTSNE
exaggeration
NDS
A=1
MDS hybrid
PCA
DREAMS-CNE-Decodk
DREAMS-CH ONE
MDS
Aw1
A=1
0.55
0.75
0.96
0.40
0.65
0.90
5
0.75 CPD (global)
0, 96
CPD (giobal)
CPD (global)
Figure 4: Tradc-offs between local and global structure preservation in different methods. Spearman cor- relation of pairwise distances is plotted against kNN recall. a: Performance of DREAMS compared with other local-global spectra. b: Comparison of DREAMS-MDS and SQuadMDS-hybrid, c: Performance of different DREAMS variants. Panels a and с show results on the Tasic et al. dataset, while panel b is based on the Packer al. datasct.
Across eleven real-world datasets, DREAMS DREAMS-MDS with default À = 0.15 consistently achieved the best performance in terms of the local-global score. This highlights its robustness and wide applicability, especially to hierarchical datasets with multiple inherent structure scales. While multiscale structure is common in real-world datasets, DREAMS less useful in datasets without it, such as the MNIST dataset, as there is less global structure to preserve. In particular, we do not have theoretical performance guarantees ensuring that DREAMS will balance local and global structure well. The additional time compute the regularizer and its gradient led to DREAMS having slower runtime than most competitors (Figure S2) Moreover, DREAMS introduces an additional hyperparameter, the regularization strength X, whose optimal value can depend on the specific dataset and the intended use. Consequently, different datasets may require tuning À to achieve the optimal balance between local and global structure preservation. That said, we found that A = 0.15 worked well across all datasets.
Because the regularization enforces alignment with a predefined structure, any biases or limitations in the PCA embedding would propagate into the final embedding. As demonstrated in cases where the MDS embedding capturcs global structure more effectively than PCA, it can sometimes be beneficial to regularize towards the MDS embedding, as in DREAMS-MDS. Thanks to its flexible and simple design, DREAMS accommodates such substitutions, making it adaptable to the specific characteristics of different datasets.
Our alternative implementation FREEAM--NEE-Dccoo to the lower local quality of sampling-based InfoNC-I-SNE. Therefore, we prefer our openTSNE- based imple- mentation, even though the PyTorch implementation may be preferable some use cases.
performed worse than the default DREAMS due
In conclusion, DREAMS successfully combines the local structure preservation strength of i-SNE with the global structure of PCA. This makes visualizations of high-dimensional data more faithful and interpretable, particularly for datascts with hicrarchical structure.
Acknowledgments
We thank Pierre Lambert and Cyril de Bodt for their help with SQuadMDS, Koshi Watanabe for making the StarMAP code available, and Alex Diaz-Pap for providing the preprocessed 1000 Genomes data.
This work was funded by the Gemeinnützige Hertic-Stiftung. DK is a member of the Germany's Excellence cluster 2064 "Machine Learning New Perspectives for Science" (EXC 390727645). The National Institutes of Health (UMIMH130981 funded initial phases of this work. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
11

---

Published in Transactions on Machine Learning Research (01/2026)
References
Ehsan Amid and Manfred K. Warmuth. TriMap: Large-scale dimensionality reduction using triplets. ar preprint Xix::19910 00204, 2019.
Mikhail Belkin and Partha Niyogi. Laplacian Eigenmaps for dimensionality roduction and data representa- tion. Neural Computation, 15(6):1373-138) 2003.
Jan Niklas Böhm, Philipp Berens, and Dmitry Kobak. Attraction-repulsion spectrum in neighbor embed- dings. Journal of Machine Learning Research, 23(95):1 2022.
Elizabeth Coda, Bry Arias-Castro, and Gal Mishne. Cluster and then Embed: A modular approach for visualization. arXiv preprint arxi::25090034 03373, 2025.
Sebastian Damrich, Niklas Böhm, Fred A Hamprecht, and Dmitry Kobak. From t-SNE to UMAP with contrastive learning. The Eleventh International Conference on Learning Representations, 2023.
Cyril De Bodt, Dounia Mulders, Michel Verleysen, and John Aldo Lec. Fast multiscale neighbor embedding.
IEEE Transaclions on Neural Nelworks and Learning Systems, 33(4):1546 1560, 2020.
Cyril de Bodt, Alex Diaz-Papkovich, Michael Bleher, Kerstin Bunte, Corinna Coupette, Sebastian Damrich, Enrique Fita Sanmartin, Fred A Hamprecht, Emöke-Agnes Horvát, Dhruv Kohli, et al. Low-dimensional embeddings of high-dimensional data. arXiv preprint arx 15929, 2025.
Alex Paak--ap,,,,ch, Luke Anderson-TTroomé, Chief Ben-Eghan, and Simon Gravel. UMAP reveals cryptic population structure and phenotype heterogeneity in large genomic cohorts. PLoS Genetics, 15(11):
e1008432, 2019.
Alex Diax-Papkovich, Shadi Zabad, Chief Ben-Eghan, Luke Anderson-Trocnee Georgette Femerling, Vikram Nathan, Jenisha Patel, and Simon Gravel. Topological stratification of continuous genetic variation in large biobanks. bioRrie, 2023.
Jiarui Ding, Anne Condon, and Sohrab P Shah. Interpretable dimensionality reduction of single cell tran- scriptome data with deep generative models. Nature Communications, 9(1):2002, 2018.
Andres F Duque Sacha Morin, Guy Wolf, and Kevin R Moon. Geometry Regularized Autoencoders. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):738147394, 2022.
Mateus Espadoto, Rafael M Martins, Andreas Kerren, Nina ST Hirata, and Alexandru C Telea Toward quantitative survey of dimension reduction techniques. TEEE Transactions Visualization and Compuler Graphics, 27333225332213, 2019.
Colin Goodall. Procrustes methods in the statistical analysis of shape. Journal of the Royal Statistical Society; Series B (Methodological), 53(2):285 321, 1991.
John C Gower. Generalized procrustes analysis. Psychometrika, 40(1):33 51, 1975.
Jacob M. Graving and Iain D Couzin. VAE-SNE: deep generative model for simultancous dimensionality reduction and clustering. BioRwiv, pages 2020 07, 2020.
Harold Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6):417, 1933.
Haiyang Huang, Yinglan Wang, Cynthia Rudin, and Edward P Browne. Towards a comprehensive evaluation of dimension reduction methods for transcriptomic data visualization. Communications Biology, 5(1):719,
2022.
Hyoon Jeon, Kwon Ko, Soohyun Lec, Jake Hyun, Tachyun Yang, Gychun Go, Jacmin Jo, and Jinwook Soo. UMATO: Bridging local and global structures for reliable visual analyties with dimensionality reduction. IEFE Transactions on Visualization and Computer Graphies, 2025.
12

---

Published in Transactions on Machine Learning Research (01/2026)
Sabina Kanton, Michael James Boyle, Zhisong He, Malgorzata Santel, Anne Weigert, Fátima Sanchis- Sanchis-Calljja, Patricia Guijarro, Leila Sidow, Jonas Simon Fleck, Dingding Han, et al. Organoid single-cell genomic at uncovers human-specific features of brain development. Nature, 574(7778)::18 422, 2019.
Dmitry Kobak and Philipp Berens. The art of using t-SNE for single-cell transcriptomics. Nature Commu- vicalions, 10(1):5416, 2019.
Dmitry Kobak and George C Linderman. Initialization is critical for preserving global data structure in both t-SNE and UMAP. Nature Biotechnology, 39(2):156-157, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009.
Joseph B Kruskal. Multidimensional scaling by optimizing goodness of fit to nonmetric hypothesis. Psy- chometrika, 29(1):1 27, 1964.
Pierre Lambert, Cyril De Bodt, Michel Verleysen, and John A Lee. SQuadMDS: A lean stochastic quartet MDS improving global structure preser neighbor embedding like t-SNE and UMAP. Neurocom- puting, 503:17 27, 2022.
Jan Lause, Philipp Berens, and Dmitry Kobak. The art of seeing the elephant in the room: 2d embeddings of single-cell data do make sense. PLOS Computational Biology, 20(10)::1012403, 2024.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basse learning applied to document recognition. Proceedings of the JEEE, 86(11):2278 22222222222222, 1998. doi: 1.L100/5..76777
John A Lee, Diego H Peluffo-Ordáiezz and Michel Verleysen. Multi-scale similaritics in stochastic neighbour embedding: Reducing dimensionality while preserving both local and global structure. Neurocompuling, 169:246-25 261, 2015.
George C Linderman, Manas Rachh, Jeremy G Hoskins, Stefan Steinerberger, and Yuval Kluger. Fast citrpollatonnbbaete t-SNE for improved visualization of single-cell RNA-seq data. Nature Methods, 16(3): 243-245, 2019.
Evan Z Macosko, Anindita Basu, Rahul Satija, James Nemesh, Karthik Shekhar, Melissa Goldman, Itay Tirssh, Allison R Bialas, Nolan Kamitaki, Emily M Martersteck, et al. Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets. Cell, 161(5):1202 1214, 2015.
Leland Mclnnes, John Healy, and James Melville. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint Xiv:11880. 03426, 2018
Jacob Miller, Vahan Huroyan, and Stephen Kobourov. Balancing between the local and global structures (T.GS) in graph embedding. In International symposium graph drawing and network visualization, pages 263 279. Springer, 2023.
Kevin R Moon, David Van Dijk, Zheng Wang, Scott Gigante, Daniel B Burkhardt, William Chen, Kristina Yim, Antonia van den Elzen, Matthew Hirn, Ronald R Coifman, et al. Visualizing structure and transitions in high-dimensional biological data. Nature Biotechnology, 37(12):1482 1492, 2019.
Philipp Nazari, Sebastian Damrich, and Fred A Hamprecht. Geometric autoencoders: what you is what you decode. In Proceedings of the 40th International Conference Machine Learning, pages 25834 25857, 2023.
Max Noichl. Examples for umap-dimensionality reduction using 3D models of prehistoric animals. https: //4ol.org/40.5555/eeeooooo.
www.///////....... 2tt. 17290165 2025. Version 0.0.1 [Datasct]. Zenodo.
David Novak, Cyril de Bodt, Pierre Lambert, John Loe, Sofie Van Gassen, and Yvan Sacys. Interpretable models for scRNA-seq data embedding with multi-scale structure preservation. bioRrio, pages 2023-11 11, 2023.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Jonathan S Packer, Qin Zhu, Chau Huynh, Priya Sivaramakrishnan, Elicia Preston, Hannah Dueck, Derek Stefanik, Kai Tan, Cole Trapnell, Junhyong Kim, et al. A lineage-resolved molecular atlas of C. elegans embryogenesis at single-cell resolution. Science, 365(6499) 2019.
Karl Pearson. Liii. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11):559 572, 1901.
Elad Plaut. From principal subspaces to principal components with linear autoencoders. orXiv preprint 1XX::1804.1125.. 2018.
Pavlin G Poličar, Martin Stražar, and Blaž Zupan. openTTNN: a modular Python library for I-SNE dimen- sionality reduction and embedding. Journal of Statistical Software, 109:1-30, 2024.
Tristan Luca Saidi, Abigail Hickok, Bastian Rieck, and Andrew J Blumberg. EmbedOR: Provable
cluster-preserving visualizations with curvature-bast based stochastic neighbor embeddings. arXiv preprint arXiv:2509.0377 2025.
Smithsonian Institution. Mammuthus primigenius (Blumbach) 3D model]. http:////dd.sieeudoooeeeee 2020.
3d/mammuthus-prrttceo pwrilts--iimimiiiiiien prr.mmggenuus--uubbbaaa blumbachh f967 11110-4500--------5---
Ashwin Srinivasan. Statlog (Landsat Satellite). UCT Machine Learning Repository, 1993. DOL:
/ttp:///dbi.orgg//24433//// C55887.
Benjamin Szubert, Jennifer Cole, Claudia Monaco, and Ignat Drozdov. Structure-preservingg visualisation of high dimensional single-cell datasets. Scientific Reports, 911):9914, 2019.
Bosiljka Tasic, Zizhen Yao, Lucas T Graybuck, Kimberly A Smith, Thuc Nghi Nguyen, Darren Bertag- nolli, Jeff Goldy, Emma Garren, Michael N Economo, Sarada Viswanathan, et al. Shared and distinct transcriptomic cell types across ncocortical arcas. Nature, 563(7729):72-78, 2018.
The 1000 Genomes Project Consortium. A global reference for human genetic variation. Nature, 526(75)1): 68 74, 2015.
Laurens Van Der Maaten. Accelerating t-SNE using tree-based algorithms. The Journal of Machine Learning Research, 15(1):3221 3245, 2014.
Laurens der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, ((Nov))2579-2600) 2008.
Daniel E Wagner, Caleb Weinreb, Zach M Collins, James A Briggs, Sean G Megason, and Allon M Klein. Single-cell mapping of gene expression landscapes and lincage in the zebrafish embryo. Science, 360(6392): 981 987, 2018.
Kaiwen Wang, Yuqiu Yang, Fangjiang Wu, Bing Song, Xinlei Wang, and Tao Wang. Comparative analysis of dimension reduction methods for cytometry by time-of-flight data. Nature Communications, 14(1):1836, 2023.
Yingfan Wang, Haiyang Huang, Cynthia Rudin, and Yaron Shaposhnik. Understanding how dimension reduction tools work: an empirical approach to deciphering I-SNE, UMAP, TriMAP, and PaCMAP for data visualization. Journal of Machine Learning Research, (2201)): 2021.
Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa, and Miki Haseyama. StarMAP: Global neighbor em- bedding for faithful data visualization. arXiv preprint arXiv:2502. 03776, 2025.
Ruizhi Xiang, Wencan Wang, Lei Yang, Shiyuan Wang, Chaohan Xu, and Xiaowen Chen. A comparison for dimensionality reduction methods of single-cell rna-seq data. Frontiers in Genetics, 12:646936, 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. Xi preprint WNi::700.00777 2017.
Zhirong Yang, Jaakko Peltonen, and Samuel Kaski. Sealable optimization of neighbor embedding for visu- alization. In Proceedings of the 30th International Conference on Machine Learning, 2013.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Appendix
A Datasets
All scRNA-soq datasets (except Packer al.) were preprocessod as Böhm et al. (2022) and Kobak and Berens (2019). After selecting the 1000 (3000 for Macosko et. al.) most variable genes, we normalized the library sizes to the median library size in the dataset, log- rngttrassfomee the normalized values with (x) and finally reduced the dimensionality to 50 via PCA. The Packer ct al.] dataset was alroady preprocessed to 100 principal components of which we used the first 50. The original data was downloaded following links in the original publications.
The raw 1000 Genomes Project data (The 1000 Genomes Project Consortium 2015) available at https: //ftp.1000gge. 10000e0omeeee...mmmm ebi ac uk This dataset contains 3 350 human genotypes. We got the preprocessed data from Diaz-Papkovich et al. (2019 2023); this is in integer-valued data matrix with 53 999 features, containing values 0, 1, and representing the number of alleles differing from a reference genome. We used PCA to reduce the number of features to 50.
We used 50 unprocessed random samples of the Mammoth dataset (Smithsonian Institution 2020 Noichl 2025) and likewise used the umprocessed Landsat Satellite (Srinivasan 1993) dataset. For the Mammoth dataset, cluster labels (solely used for coloring the visualizations) were oblained by applying Agglomerative Clustering with 11 classes the original data.
For the MNIST (Lecun et al. 1998.. Fashion MNIST (Xiao al. 2017 and CIFAR10 (Krizhevsky et al. 2009) datasets (downloaded using the torchvision API), we used the first 50 principal components. For Fashion MNIST, we additionally rescaled the original data to the interval [0, 1] before computing the first 50 principal components.
B Role of global embedding scaling in the DREAMS loss
In the regularization term
1 -1 a
R(Y) =
(6)
the scaling parameter a used match the scale of the global reference embedding Ý with that of Y. We considered two scaling strategies:
|||r |
(1,Y)r o 1
an
By the Cauchy Schwarz inequality, 0n 2 ap, with equality and only Y is a positive rescaling of Y. The first option, On, rescales Y to match the scale of Y, preventing the regularization term from imposing an artificial scale onto the scale-sensitive t-SNE embedding. The second option, ap, is the a that minimizes the regularization term R(Y) for a given Y and Y and inspired by Procrustes analysis GGoodall 1991 1975).
Gower
Experimentally, we found that ap performed slightly worse than on on the Tasic et al. dataset, leading to reduced local structure preservation as measured by smaller KNN value (Table While the regularization term was, by construction, smaller with ap, the KL divergence was slightly higher and the scale of the final embedding was further from t-SNE's than with an (Figure S1}. Consequently, we chose use the scaling with On and conducted all subsequent experiments with this scaling method. Importantly, both scaling strategics worked much better than not using scaling at all and fixing a 1 (Figure Si Table S1).
Treating a as constant during optimization yields the gradient
2 ( (V=)
VyR(Y))
(8)
15

---

Published in Transactions on Machine Learning Research (01/2026)
Table S1: Comparison of DREAMS performance across various alignment approaches for matching Y and Y on the Taste et al.] dataset. We compare default DREAMS with DREAMS using lower scaling factor Op, additionally translationally aligned the global reference Y to Y (treating the shift constant), and full Procrustes analysis, which also contains a rotational alignment. The smaller scaling with Op was detrimental, while translational and rotational alignment had no effect as the t-SNE loss is invariant to them.
Metric
DREAMS (an)
ap
ap + translation full Procrustes
no scaling
KNN
0.37
0.35
0.35
0.35
0.27
CPD
0.89
0.89
0.89
0.89
0.53
which pulls Y lowards the scaled global reference embedding aY. In contrast, using On scaling and allowing the gradients to propagate through a, yields the gradient
VyR(Y) 2 2 o8
2 -2( (Y,Y) n | -
-1);
(9)
with an additional second term that corresponds to a shrinkage of towards the origin, since YY As mentioned above, since the t-SN loss is scale-sensitive, do not want the regularizer to have bearing the scale of Y. Nevertheless, in the CNE variant of DREAMS, and still obtained a good trade-off between the InfoNC-t-SNE and PCA performance (Figure 4). We also verified that treating a as a constant during optimization the CNE version did not lead to any noticeable difference performance.
|| S
propagated the gradient through a
Regularization error
divergence
Scaling parameter
Embedding scale
150
450
21 k
100
G En-sne sne m
300 k.
14 k-
a, scaling
RM
dp scaling scaling
50
3-
150
k
no regularizer
1
250
750
750
250 500 Iteration
250 500 750
Iteration
Iteration
Iteration
Figure S1: Regularization error, KL divergence, scaling parameter and embedding scale of DREAMS during optimization using different or no scaling methods. Results are reported as the mean over four random seeds on the TTasic et all dataset.
C Runtime analysis
DREAMS' adds a small overhead for computing the gradients of the regularizer compared openTSNE. As result, it runs bit slower than openTSNE and similarly fast MhhnadMDS-hybrit The other methods were faster than DREAMS (Figure S2
16

---

Published in Transactions on Machine Learning Research (01/2026)
PCA 0.01 0.01
SQuadMDS
openTSNE
TriMap
StarMAP
SQuadMDDShybrid
DREAMS-MDS
DREAMS
100
150
Time (seconds)
Figure S2: Runtime comparison. The bars indicate the mean runtime and standard deviation across four random seeds the Tasie al.] dataset. For DREAMS and DREAMS-MDS, the reported times also include the runtime of the respective global reference embeddings (PCA and MDS).
D DREAMS-CNE and PREAMS-CNE-DE -ecoodr
Tasic et al.
Macosko et al.
Kanton et al.
Wagner at.
Packer et all
1000 Genomes
KNV (lcca))
0.3-
0.2
0.0
0.35
0.95
0.83
0.92 0.82
0.92
0.6
0.9
0.55
0.7
0.55
0.85
Mammoth
Satellite
FMNIST
CIFAR10
0.8
KNN (occal)
0.6-
0.4
0.2
(giobal)
CPD (global)
CPD (global)
(global)
CPD (giobal)
DREAMS-CNE
DREAMS-CNE-Decoder
DREAMS-CNE 10-0.00 0.0005)
DREANS-CNE-Decoder (A 0.005)
InfoNC-t-SNE
PCA
Figure S3: Spearman correlation of pairwise distances (CPD, global metric) is plotted against ANN recall (local metric). The figure shows the spectrum DREAMS-CNE using regularizer with precomputed PCA embedding, and
using a regularizer with a linear decoder, across all eleven datasets. and SEFEAMS-CNF--ecco
The bigger scatter points display regularization strengths (which achieved the highest average local-global score across all data sets) and local (InfoNC-L-SNE) and global (PCA) reference methods. Here, InfoNC-I-SNE is used as the (-SNE backbone and corresponds to the regularization strength of À = 0 while PCA corresponds to the maximal regularization strength of 1. In panel j (MNIST) the scatter points of PCA and DREAMS CNE--ecoder lic exactly on top of each other.
with their respective default
17

---

Published in Transactions on Machine Learning Research (01/2026)
E Intuition of local-global score
Tasic et al.
Packer et al.
0.5
1.00
0.4
SQuadMDS-hybrid
DREAMS-MDS
TriMap
0.3
3ed.00
2w0
PCA
StarMAP
TSen
PHATE
KNN
0.2
1w0
UMAP
PaCMAP
SQuadMDS
0.1
1=0.0
0.0
0.45
0,70
0.95
0.5
0.7
0.9
(global)
(global)
Figure S4: Spearman correlation of pairwise distances (CPD, global metric) is plotted against kNN recall (KNN, local metric) for different methods with marked local-global score spectrum. The figure shows the structure preservation results on the Tasic al. and Packer et al. dataset.
F Sensitivity of ANN recall to choice of k
0.6
0.8
0.5
local-global score
DREAMS
SQuadMDS -hyrrd
KNN (local)
DREAMS-MDS
TriMap
StarMAP
openTSNE
PHATE
0.3
0.4
UMAP
PaCMAP
0.2
SQuadMDS
0.0
0.0
20
100
200
1
20
100
200
k
Figure S5: Local structure preservation metric and combined local-global score for different numbers of neighbors & in the &NN recall. Displayed are the results on the Tasic et al.] dataset. Except for SquadMDS. hybrid, the ranking of methods in terms of INN recall insensitive the exact value of
F.1 Visualizations of all datasets
See next pages.
18

---

Published in Transactions on Machine Learning Research (01/2026)
UMAP
InfoNC-I-SNE
Embedding legend
Inhibitory neurons
Excitatory
Non neural cells
KNN: 0.46
KNN: 0.25
KNN: 0.36
CPD: 0.56
CPD: 0.50
CPD: 0.59
DREAMS
DREAMS MADSMS-DDS
DREAMS-CNE
DREAMS-CNE DNE-eecder
KNN: 0.37
KNN: 0.42
KNN: 0.31
KNN: 0.33
CPD: 0.89
CPD: 0.84
CPD: 0.90
CPD: 0.90 0.84
0.88
s: 0.87
0.82
SQuadMDS
SQuadMDS-hybrid
KNN:
0.50
S:
S:
TriMap
PHATE
PaCMAP
0.5-
0.3
KNN
0.2
KNN: 0.22 CPD: 0.86 0.66
KNN: 0.19 CPD: 0.55
KNN: 0.21
CPD: 0.69
0.25
0.45
0.0
0.35
0.65
0.95
CPD (global)
Figure S6: Visualizations of the Tasic el al. dataset with all considered methods.
19

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
Embedding legend
cones
Bipolar cells
Amacrine cells
Horizental cells
Ganglion
Non-neural cells
KNN: 0.18
KNN: 0.04
KNN: 0.10 CPO: 0.84
CPD: 0.68
0.50
DREAMS
DREAMS-MDS
DREAMS-CNE
CAAMM--CNE-Deoode
KNNGO 14 CPD: C199
K&N: 0.14 CPD: 0.85
KNN 0.08 CR
KNN: 0.10 CPD30.88
0.85
0.78
0.68
S: 0.71
PCA
SQuadMDS
HQuadMDSShyybre
StarMAP
CPD: 0.91
CPD: s: 0.47
CPD: 0.76
CPD: 0.89
s: 0.50
s: 0.61
s: 0.57
TriMap
PHATE
PaCMAP
0.20
KNN (loca))
0.05
KNN: 0.04 CPD: 0.89
KNN: 0.03
KNN: 0.03
CPD:
0.00
0.65
0,30
0.95
CPD (global)
Figure S7: Visualizations of the Macosko el al. dataset with all considered methods.
20

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
Embedding legend days
4 days
days
1 month
2 month months
KNIN 0.38
KNN: 0.21
DREAMS
DREAMS-MDS
DREAMS-CNE
DREAMS-CNE-Decoder
KNIN O
KNN: 0.35 CPD: 0.87
KNN 0.27 CPO: 0.86
KNN: 0.29 CPD: 0.80
CPD: 0.86
0.88
0.90
0.77
0.70
SQuadMDS
SQuad MDS-hybrd
StarMAP
KNN: O
KNN:
CPD:
CPD: 0.91
CPD: 0.78
CPD: 0.86
s: 0.48
s: 0.52
s: 0.77
s: 0.66
TriMap
PHATE
PaCMAP
0.4
KNW (ocal)
0.2
KNI: 0.17
KNN: 0.14
KNN: 0.18
CPD: 0.82
CPD: 0.75
0.0
0.55
0.35
0.95
CPD (global)
Figure S8: Visualizations of the Kanton el. al. dataset with all considered methods.
21

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
Embedding lagand
hp
hpf
hpf
10
14 hpf
18 hpt
24 hpt
KONNNN 0.35 CPD: 0.62 0.61
KNN: 0.14 CPD: 0.54
KNN: 0.24 CPD: 0.64
DREAMS
DREAMS-MDS
DREAMS-CNE
SLEAMSSCNE-Decodee
RNN: 0.28 GED: 0.87
KNN: 0.31
KNN: 0.21 CRD: 0.86 5: 0.72
KNN: 0.23 GFD: 0.87
CPD: 0.87
0.84
0.90
0.77
PCA
SQuadMDS
SQuadMDS-hybrie
StarMAP
KNN:
EPD: 0.88
CPD: s: 0.51
CPD: 0.77
CgD:
0.46
s: 0.76
s: 0.59
TriMap
PHATE
PaCMAP
0.35
0.25
KNN (local)
0.15
KNN: 0 @PD: 6197 0.58
KNN: 0.09 CPD: S:
KNN: 0.11
CPD: 0.72
0.05
0.00
0.50
0.33
0.95
CPD ababall
Figure 89: Visualizations of the Wagner et al. dataset with all considered methods (hpf = hours post fertilization).
22

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
VKNN: oh
S:
DREAMS
DREAMS-MDS
DREAMS-CNE
DREAMS-CNE-Decoder
KNN: 0.29
KNN: 0.30
KNN: 0.22 CPD: 0.67
KNN: 0.25 CPD: 0.66 0.55
CPD: 0.66
CPD: 0.79
0.62
0.83
0.52
PCA
SQuadMDS
SQuadMDS-hybrid
StarMAP
KNN: 0.01
KNN:
KNN: 0.11
CPD: 0.69
CPD: 0.86
CPD: 0.69
CPD: 0.68
s: 0.25
S: 0.51
S: 0.64
S: 0.39
TriMap
PHATE
PaCMAP
0.4
S0N [local]
0.2
KNN: 0.11
KNN: 0.09
KNN: 0.12
CPD: 0.76
CPD: 0.63
CPD:
05
0.7
0.9
CPD (global)
Figure S10: Visualizations of the Packer el. al. dalaset with all considered methods.
23

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
Embedding legend
African
European
South Asian
East Asian
Admixed American
KNN: 0.25
KNN:
S:
S: 0.66
DREAMS
DREAMS-MDS
DREAMS-CNE
DREAMS-CNE-Decoder
KNN: 0.35
KNN: 0.35
KNN: 0.30
KNN: 0.32 CPD: 0.79
CPD: 0,78
CPD: 0.84
CPD: 0.78
0.89
0.93
0.81
0.84
SQuadMDS
SQuadMDS-hyyan
StarMAP
KNN: CPD: 0.79 S: 0.88
KNN: 0.21
CPD: 0.81 s: 0.43
CPD: 0.89
CPD: 0.79
5: 0.55
S: 0.66
TriMap
PHATE
PaCMAP
0.4
KNN (local]
0.2
KNN: 0.21
KNN: 0.20
KNN: 0.19
CPD: 0.66
CPD: 0.45
CPD: 0.65
0.0
033
0.6 CPD (global)
0.9
Figure S11: Visualizations of the 1000 Genomes Project dataset 2015) with all considered methods.
The 1000 Genomes Project Consortium
24

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
Embedding legend
Cluster gnmeet 3D data
KNN: 0.76
KNN: 0.62
KNNN: 0.70
CPD: 0.75
CPD: 0.68
CPD:
DREAMS
DREAMS-NE MDS
DREAMS-CNE
DREAMS DREAMS-CNE-DeSS CNE--DDN-DD
KNN 0.74 CPD: 0.97
KNN: 0.74
KNN: 0.69
KNN: 0.70
CPD: 0.97
CPD: O.
CPD: 0.98 0.91
0.95
0.9.95
0.90
PCA
SQuadMDS
SQuadMDS-hyarid
StarMAP
KNN: 0.38 CPD: 0.99
KNN: 0.29
KNN: O. 7.78 CPD: 0.88
CPD: 0.99
CPD: 0.98
s: 0.60
S: 0.50
s: 0.94
s: 0.68
TriMap
PHATE
PaCMAP
0.8
0.7
KNN (local)
0.6
0.5
0.4
KNN: 0.51 CPD:
KNN: 0.56
CPD:
0.3
(giobal)
Figure S12: Visualizations of the Mammoth dataset (Smithsonian Institution, 2020 Noichl 2025) with all considered mots Colors represent. cluster assignment. of original data and solely used for visual clarity but do not correspond to any ground truth labels.
25

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-E-SNE
Embedding legend
red catton crop
grey soil
damp grey
soll with vegetation stubble
very damp grey soil
KNN: 0.47
KNN: 0.32
KNN: 0.41
CPD: 0.72
CPD: 0.77
CPD: 0.81
0.50
0.39
0.59
DREAMS
DREAMS-NE MEEEMSS-DSS
DREAMS-CNE
KNN: 0.46 CPD: 0.95
KNN:
KNN: 0.40
KNN: 0.42
CPD: 0.95 S: 0.93
CPD: 0.96
CPD: 0.9
0.93
S: 0.86
S: 0.87
SQuadMDS
SQuadMDS-hybrid
KNN: 0.13
0.44
KNN:
s:
TriMap
PHATE
PaCMAP
0.5-
0.3
KNN
KNN: 0.24 CPD: 0.94 0.58
KNN: 0.23 CPO: 0.89 0.46
KNN: 0.28
CPD: 0.79
0.34
0.1
0.70
0.85 CPD (global)
1.00
Figure S13: Visualizations of the Landsat Satellite dalaset Srinivasan
1993 with all considered methods.
26

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
Embedding legend
T-sirt/top
Trouser
Shirt
Pullover
Sneaker
Dress
Bag
Coat
boot
KNN: 0.39 CPD: 0.66
KNN: 0.15
KNN: 0.28 CPD: 0.74
CPD: 0.64
0:5.
0.1.18
0.52
DREAMS
DREAMS-MDS
DREAMS-CNE
DREAMS-CNE-Decoce
KNN: 0.38 CPD: 0.82
KNN: 0.38
KNN: 0.26 CPD: 0.83
KNN: 0.28 CPD: 0.74
CPD: 0.84 0.83
S: 0.80
S: 0.64
S: 0.53
SQuadMDS
SQuadMDS-hyorid
StarMAP
0.44
S:
S: 0.81
S: 0.49
TriMap
PHATE
PaCMAP
0.4
0.3
KNN (local)
0.2
KNN: 0.10 CPD: 0.71
KNN: 0.11 CPD: 0.64 0.14
KNN: 0.11 CPD: 0.63
0.24
0.12
0.0
0.6
0.8 (giobal)
1.0
Figure S14: Visualizations of the Fashion MNIST dataset (Xiao et al. 2017 with all considered methods.
27

---

Published in Transactions on Machine Learning Research (01/2026)
openTSNE
UMAP
InfoNC-t-SNE
Embedding legend
8
KNIN: 0.39 0.35
KNN: 0.12 CPD: 0.30
KNN: 0.26
DREAMS
DREAMS-MDS
DREAMS-CNE
DREAMS-CNE-Decoder
KNN: 0.32 CPD: 0.44 0.61
KNN: 0.29 CPD: 0.49 0.64
KNN: 0.19 CPO: 0.41
KNN: 0.01
CPD: 0.56
0.41
0.35
SQuadMDS
SQuadMDS-hybrid
StarMAP
0.55 5: 0.35
CPD: 0.68
S: 0.50
S: 0.61
s: 0.26
TriMap
PHATE
PaCMAP
0.4
KNN (local)
0.2
KNI: 0 09 CPD: 0.26 0.10
KNN: 0.09
KNN: 0.09
CPO: 0.33
CPO: 0.34
0.0
0.25
0.48
0.70
CPD (global)
Figure S15: Visualizations of the MNIST dataset Lecun et al. 1998 with all considered methods.
28

---

Published in Transactions on Machine Learning Research (01/2026)
UMAP
InfoNC-t-SNE
Embedding legend
airpiane ane
dog
frog
bird
horse
cat
ship
deer
truck
KNN: 0.20
KNN: 0.02 CPD: 0.69 0.16
KNN: 0.13
CPD: 0.66
CPD: 0.66
0.56
0.38
DREAMS
DREAMS-MDS
DREAMS-CNE
DREAMS-CNE-Decoder
KNN:
KNN:
KNN:
CPD: 0.71
CPD: 0.71
CPD: 0.68 S: 0.33
CPD: 0.77
S: 0.64
S: 0.62
s: 0.50
PCA
SQuadMDS
MQu-hhihd
StarMAP
KNN:
KNN: 0.01
KNN: 0.15
KNN: 0.02
CPD:
CPD: 0.86
CPD: 0.73
CPD: 0.69
PHATE
PaCMAP
0.25
0.20
0.15
ONN
0.10
KNN: 0.01
KNN: 0.02
KNN: 0.02
CPD: 0.74 0.25
CPD: 0.64 0.03
CPO: 0.69
0.15
0.00
0.60
0.75 CPD (global)
0.90
Figure S16: Visualizations of the CIFAR10 dalasel (Krzhevsky el al. 2009 with all considered methods.
29


Reviews:
[{"review_id": "IL8hb6oalN", "reviewer": "Reviewer_gRWA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**\n\nThe paper introduces DREAMS, a dimensionality reduction approach that augments the t-SNE objective with a regularization term penalizing deviations from PCA positions. By encouraging the embedding to PCA, the method aims to preserve global structure during optimization, addressing a common limitation of conventional techniques that tend to emphasize either local or global structure but not both.\n\n**Strengths**\n\n1. The paper is well written and has a complete framework.\n2. The proposed algorithm has been applied to seven real-world datasets with multiple baselines.\n3. The proposed algorithm outperforms the baselines in terms of the trade-off between preserving local and global structure.\n4. The visualization is clear, and the plots are informative.\n\n**Weakness**\n\n1. The theoretical guarantee is unclear. Please refer to the comments in the next section.\n2. Most of the chosen datasets focus on a specific area, i.e., scRNA-seq, and I am unsure whether diversity of datasets is needed for evaluation in this area.", "claims_and_evidence_yn": "No", "claims_explanation": "Empirically, the proposed algorithm appears to balance local and global structure well, and the embedding visualizations look compelling. I have a couple of clarification questions.\n\nRegarding the aggregated local–global score: is this score commonly used in the literature? If not, a further justification of its design and interpretation would be helpful. Also, is the KNN metric you use different from the “KNN Accuracy” in the paper Watanabe et al. 2025 about STARMAP? And is the evaluation sensitive to the choice of $k$?\n\nFor the theory, could you clarify how $\\alpha$ operates at each iteration and why it can be treated as a constant? When minimizing the modified objective function, is there a theoretical guarantee that both local and global structures are preserved in a balanced manner governed by $\\lambda$?", "audience_yn": "Yes", "audience_explanation": "Given the competitive performance on seven real-world datasets relative to other baselines, I think the proposed algorithm will be of interest. However, as mentioned above, the diversity of datasets, the evaluation metrics, and the theoretical component warrant further discussion.", "requested_changes": "1. Please clarify the points mentioned above.\n2. It would be interesting to see results on other commonly used datasets for dimensionality reduction.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rkSOWQaz5g", "reviewer": "Reviewer_zedR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new method called DREAMS to simultaneously preserve both “local structure (e.g., neighborhood relationships)” and “global structure (e.g., the overall arrangement of classes)” in the two-dimensional visualization of high-dimensional data—a classic challenge. This is achieved by adding a regularization term derived from PCA to the objective function of t-SNE. Specifically, it introduces weighted regularization that penalizes the squared sum of the difference between the embedding and a pre-obtained global reference embedding (primarily PCA, or MDS in some cases) against the KL loss of t-SNE, aiming to minimize L.\n\nStrengths\n- The objective function extension is simple and easy to implement. The ability to obtain a continuous local-global spectrum via lambda is practically useful.\n- The experimental design, encompassing not only PCA but also DREAMS-MDS using MDS as a reference, demonstrates the possibility of selecting a “good global” spectrum tailored to data characteristics.\n- By placing indicators (KNN/CPD) on both axes and visualizing the trade-offs of each method in Figures 3 and 4, the differences are easily conveyed to readers.\n\nWeaknesses\n- Theoretical implications are shallow: The treatment of alpha, the equivalence of solutions when lambda (perfect PCA recovery), and its inverse remain at the level of empirical rules. It would be better to state that they are mathematically based and strictly consistent.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The systematic visualization and quantitative comparison across seven datasets largely supports the main claim that the proposed method maintains both local and global properties well. The direct comparison of competing methods' spectra via lambda sweep curves is also persuasive. While experimentally promising, theoretical grounding is necessary.", "audience_yn": "Yes", "audience_explanation": "TMLR readers include many practitioners and researchers in dimensionality reduction for representation learning and data visualization, where balancing local and global properties remains a longstanding practical challenge. Findings showing superior trade-offs over existing hybrids with simple regularization are noteworthy.", "requested_changes": "- Explicitly describe mathematically how PCA and t-SNE are each restored depending on lambda.\n- Clarify the treatment of alpha and geometry. State the motivation for fixing α at each iteration and provide stability analysis (presence/absence of contribution to gradients). Present comparisons (at least ablation studies) with introducing Procrustean alignment (scale + orthogonal rotation) into regularization, including rotational degrees of freedom. The current approach uses scale only and may not be invariant to rotation.", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}, {"review_id": "whQMfwstml", "reviewer": "Reviewer_yrhb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Summary  \nThis paper presents DREAMS (Dimensionality Reduction Enhanced Across Multiple Scales), a novel dimensionality reduction method designed to bridge the gap between local and global structure preservation. To achieve this, DREAMS introduces a straightforward regularization term that fuses the strong local structure retention of t-SNE with the robust global structure preservation of PCA. Specifically, this term penalizes deviations of low-dimensional embeddings from precomputed PCA positions—effectively addressing the long-standing trade-off in traditional methods, where tools like t-SNE excel at local structure but struggle with global patterns, and PCA prioritizes global structure at the cost of local detail. The authors conduct rigorous evaluations of 10 dimensionality reduction algorithms across 7 real-world datasets, providing compelling evidence for DREAMS’ effectiveness in balancing local and global preservation.  \n\n\n## Strengths  \n1. DREAMS outperforms competing methods by effectively retaining both local and global structure. Notably, on most datasets, it achieves KNN scores (a local structure metric) comparable to t-SNE, while its CPD scores (a global structure metric) remain close to those of PCA—demonstrating a rare ability to excel at both scales.  \n2. DREAMS is not limited to PCA as a global reference; it can integrate other global structure methods (e.g., MDS) to generate variants like DREAMS-MDS. This adaptability makes it well-suited for datasets where MDS (rather than PCA) more accurately captures global patterns, expanding its practical utility.  \n\n\n## Weaknesses  \n1. While DREAMS is simple and effective, its core innovation is limited: it primarily combines the use of PCA and t-SNE rather than introducing a fundamentally new methodological framework. Additionally, its two-step workflow—first preforming PCA, then optimizing t-SNE with the regularization term—may introduce higher computational costs compared to one-step methods.  \n2. The regularization term anchors DREAMS’ embeddings to a precomputed global reference (e.g., PCA or MDS). Consequently, if the reference embedding has inherent biases or limitations (e.g., PCA’s tendency to distort non-linear global patterns), these flaws directly propagate to DREAMS’ final output, undermining its performance in such cases.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, the claims in the submission are supported by accurate, convincing, and clear evidence. The paper validates DREAMS’ ability to balance t-SNE’s local and PCA’s global structure preservation through both qualitative (embedding visualizations across 7 datasets, e.g., Tasic et al.’s clear separation of cell classes and fine clusters) and quantitative metrics (KNN for local, CPD for global, and aggregated scores), where DREAMS or DREAMS-MDS consistently outperforms 10 competing methods. It further confirms flexibility (supporting MDS as a global reference) and reproducibility (detailed experimental setups, open-source implementations, and results averaged across 4 random seeds), ensuring the evidence is rigorous and credible.", "audience_yn": "Yes", "audience_explanation": "Yes, at least some of TMLR’s audience will be interested in this paper’s findings. TMLR focuses on machine learning research, and the paper addresses a long-standing challenge in dimensionality reduction—balancing local and global structure preservation—which is critical for tasks like high-dimensional data visualization (e.g., single-cell transcriptomics, genetic data) that many readers (e.g., researchers in ML for biology, data visualization, or applied ML) work on.", "requested_changes": "The paper does not evaluate DREAMS on datasets where neither PCA nor t-SNE performs effectively (e.g., highly non-linear datasets with complex, multi-scale structures). It would be valuable to assess how DREAMS performs in these challenging scenarios, as this would clarify its robustness beyond cases where its reference methods already work well.", "broader_impact_concerns": "No ethical concerns are noted.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)