t-SNE Exaggerates Clusters, Provably
Authors: Noah Bergam
Szymon Snoeck
Nakul Verma
Venue: ICLR 2026 Poster (2026)
Keywords: nonlinear dimension reduction
data visualization
t-SNE
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: t-SNE visualizations can overemphasize clusters and suppress outliers significantly

Abstract: Central to the widespread use of t-distributed stochastic neighbor embedding (t-SNE) is the conviction that it produces visualizations whose structure roughly matches that of the input. To the contrary, we prove that (1) the strength of the input clustering, and (2) the extremity of outlier points, cannot be reliably inferred from the t-SNE output. We demonstrate the prevalence of these failure modes in practice as well.

Published as a conference paper at ICLR 2026
T-SNE EXAGGERATES CLUSTERS, PROVABLY
Noah Bergam', Szymon Snoeck", & Nakul Vermat
CComputer Science Department; AApllied Mathematics Department Columbia University
(noah. bergam, sgs2179) co uumbi a edu, vermaêcs col umbia a edu
ABSTRACT
Central to the widespread use of t-distributed stochastic neighbor embedding (t-SNE) is the conviction that it produces visualizations whose structure roughly matches that of the input. To the contrary, we prove that (1) the strength of the input clustering, and (2) the extremity of outlier points, cannol always be inferred from the t-SNE output. We demonstrate the prevalence of these failure modes in practice as well.
INTRODUCTION
I-SNE (Van der Maaten & Hinton 2008) and related data visualization methods have become sta- ples in modern exploratory data analysis. They just seem to work: practitioners find that these techniques reliably tease out interesting cluster structures in datasets. Consequently they are used ubiquitously in a wide array of fields, ranging from single-cell genomics to language model interpretability (Kobak & Berens
2019 Petukhova al., 2025). The practical success of these techniques has naturally piqued interest in the theoretical community as well.
Existing analysis of t-SNE has established that, given high-dimensional data with spherical, well- separated cluster structure, I-SNE outputs visualization which preserves that cluster structure (Arora al. 2018: Linderman Steinerberger 2019). In other words, t-SNE is provably good at generating true positives in its visualization of clusters.
Curiously, theoretical investigation of t-SNE's potential to generate false positives (clustered plots despite un-clustered input) and false negatives (un-clustered plots despite clustered input) has re- mained elusive. One should note that this not a purely academic curiosity, since the interpretation of t-SNE outputs have important consequences downstream in the sciences, influencing hypothesis generation, experimental design, and scientific conclusions.
As an illustration of t-SNE's potential to mislead, consider the visualizations produced by three example inputs in Figure ! In the left column, a t-SNE plot depicts two salient clusters, despite the pairwise distances in the input being nearly uniform. In the middle column, t-SNE plot depicts an unstructured blob, whereas its corresponding input is easily partitioned into two salient clusters (with the exception of single adversarially placed point). In the right column, t-SNE plot shows two well-separated clusters, completely misrepresenting the outlying third of the input dataset.
Our work formalizes these and other limitations of t- t-SNE in terms of faithfully depicting distance- based information in the input. Our theoretical analysis, suffused with experiments, shows that one should take t-SNE visualizations with grain of salt. Our contributions as follows.
(i) Misrepresentation of cluster salience: We prove that any well-clustered t-SNE visualiza- as well as arbitrarily weakly- separated clustered datasets, see Theorem g and Corollary 4 Moreover, a slight distance perturbation of inputs can lead to vastly distinct visualizations, see The- orem 5 We identify the property of t-SNE that explains these peculiar behaviors, and use this understanding to design an adversarial attack that disrupts cluster structure in the output by moving as little one point, see Figure
tion can produced identically by both a strongly-separatee
prove that even

---

Published as a conference paper at ICLR 2026
Output: 2 salient clusters
Output: 1 cluster
Output: 2 salient clusters
Output t-SNE Plot
Input: nearly uniform distances
Input: clusters adversarial point
Input: clusters outliers
Input Pairwise Distance Matrix
34
34
92
Figure 1: Some examples of how t-SNE may misrepresent cluster information in the input dataset.
(ii) Misrepresentation of outliers: We prove that, regardless input, the resulting t-SNE output is incapable of depicting extreme outliers, in the sense of depicting point as substantially far away from all the others, see Theorem 9 In practice, on both synthetic and real datasets, we observe often subsumed into the cluster structure the bulk of points, see Figures 5 and
concerning phenomenon that faraway outliers
While there has been some work investigating the shortcomings of t-SNE in various practical settings (see Section 2.2 for detailed discussion of the relevant literature), to the best of our knowledge this the first work which theoretically analyzes some of the key limitations of t-SNE.
RELATED WORK
Confidence in the data visualizations produced by t-SNE and related methods is contentious subject in data science MMaxx 2024 de Bodt et al. 2025 Some argue that these methods have merit in of preserving cluster structure and therefore aid in exploratory data analysis, while others warn us about the distortions introduced by these methods.
terms
2.1 PERFORMANCE GUARANTEES AND ANALYSIS OF T-SNE
Shaham & Steinerberger (017 were among the first to provide a guarantee on the visualization pro- duced by optimal SNE embeddings of well-clustered data. Works by Linderman & Steinerberger (019)) and Arora et al. 2018) refined and extended this analysis, showing that t-SNE outputs pro- duced using gradient descent yield well-clustered visualizations so long as the input is sufficiently well-clustered. The latter work established this guarantee in considerable generality, including cases where the input sampled from mixture of well-separated log-concave distributions.
Along with these algorithmic performance guarantee results, there is line of work that seeks to establish a more fundamental understanding of t-SNE as an optimization problem. Cai & Ma 2022). for instance, characterized the distinct phases of gradient-based optimization of t-SNE, and proved an asymptotic equivalence between the early exaggeration phase of t-SNE and spectral clustering. Auffinger & Fletcher 2023) proved consistency result for continuous analogue of S-NNE, viewing the optimization problem as producing a map between distributions rather than just point sets. Jeong & Wu (2024) and Weinkove (2024) studied the gradient flow of t-SNE. The former showed mild
2

---

Published as a conference paper at ICLR 2026
assumptions under which optima exist, and the latter showed that, in cases where the gradient flow diverges the relative interpoint distances stabilize in the limit.
2.2
WEAKNESSES AND CRITICISMS
Bunte al. 2012 were among the first to investigate the potential shorteomings of using KL- divergence in a t-SNE visualization and proposed a generalization to other divergences that may be better suited for specific datasets and user needs. Building upon the precision-recall framework of Venna et al. 2010 Im et al. 2018) extended this result and explored specific intrinsic structures within data that may be less suited for t-SNE. They concluded that while t-SNE is more attuned to reveal intrinsic cluster structure, it usually fails to reveal intrinsic manifold structure.
In terms of analyzing cluster structure specifically, Yang et al. (2021) provided empirical evidence that t-SNE visualizations are prone to false negatives. They presented selection of well-clustered real-world datasets which t-SNE embeddings, even with reasonable parameter-tuning, do not seem to represent faithfulll. They also showed that these practical datasets do not abide by the theoretical cluster separation conditions that are required by Arora et al. (2018)'s 's analysis. [Chari Pachter 2023) argued that t-SNE and UMAP are unreliable tools for exploratory data analysis. Taking single-cell genomic data as an important world example, they provided systematic empirical evidence that these embeddings suffer high distortion, and often misrepresent neighborhood and cluster structure. Curiously, to the best of our knowledge, there is no systematic theoretical study investigating the failure modes of (-SNE.
More recently, Snoeck et al. (2026) provided theoretical evidence that, not just t-SNE, but any em- bedding technique that attempts visualize data in constant dimensions is bound misrepresent neighborhood structure in almost all inputs. This result makes general assertion that misrepresenta- tion is geometrically inevitable; the current work investigates how such misrepresentations manifest precisely in t-SNE on a variety of inputs of interest.
3 PRELIMINARIES
Given an input dataset X ........ In C RD, the goal of t-SNE is to come up with an embed- ding Y (y.,...,, Vn Rdd (where d < D. typically d 2 that approximately maintains the neighborhood structure in X. t-SNE accomplishes this by assigning affinities to input data points which encode how likely input point
be a neighbor configuration of the embedded points Y that induces similar neighborhood affinity. Specifically, let P P= P(X) Rax" and Q Q(Y) E R be the input and embedded affinity matrices describ- ing the pairwise neighborhood similarities in the input and output, respectively. t-SNE constructs P by first computing neighborhood affinities for each point i defined as (for any f
given point. The goal then is find
exp(- [z, (20?)
P3)((ioo))== Eng exp( |, xx, 10,,///,1210000000 (2(o))
Prt(X;o))= :=
(1)
where o; 2 0 encodes the (point-dependent) neighborhood scaling When X and of are clear from context, we will often drop it from the notation. It is worth noting that P.je is a valid probability distribution over [n). The matrix P is then constructed based on a crucial parameter called the perplexity (denoted p and taking values in [1, n 1)), as follows:
(1) For each E [n], select the (unique, see Lemma 14p neighborhood scale of that minimizes the gap between the entropy of P. (X;()) and logz P.
1 (2) Define P = [Palsssee where Ps := 2n (P)1( + P))();))
To avoid the so-called crowding problem (see Van der Maaten & Hinton (2088) for details), the output affinity matrix Q is computed based on a t-distribution. Specifically, for i i
(1 11 y3 )
(2)(): = ER.NK + 1 wll-)-1
Qa(Y) : 0.
(2)
I With a slight abuse of subset notation, duplicates are allowed in X and Y.
II a; = 0, define P)){(X;0) 0) lim +o (o)))
3

---

Published as a conference paper at ICLR 2026
As indicated before, the objective then is minimize the gap between the imput and output affinities P and Q. This is accomplished by minimizing relative entropy (KL-divergence) between the P and Q affinities (viewed probability distributions).
(25(2)
minimizey Lx(Y) == KL(P(X))I(()) = P()(X) log
(r)())
This highly non-convex objective usually optimized by initializing at a good starting point via an early exaggeration phase, followed by performing standard gradient descent methods and returning an embedding Y that corresponds local minimum of the objective. Our central task is study the nature of the these (local minimum) embeddings returned by t-SNE and their relation to the space of input datasets.
Definition 1. For an n-point datase X C Rn-1 and perplexity parameter [1, 1}, define
t-SNE,(X) = (Y c R² Vy = -}
as the set of outputs Y C. Rd that stationary to the t-SNE objective a given input X. Further- more, for a set of n-point datasets Xuo we define:
(Xn)
U
I-SNE
XEX,
If Xn is the set of all n-point datasets, we denote t-SNE,((K,) as Im(t-SNEpaa) indicate the entire image of the t-SNE map.
All the supporting proofs for our formal statements can be found in the Appendix, and the code related to our experimental demonstrations is available on Github at https: http://tttttutttee
thub. http://co///nermnm.o.mmmm htt.///rrrreeeeeeeeeeeeeereeeeee custerss
MISREPRESENTATION OF CLUSTER SALIENCE
Previous works by Linderman & Steinerberger 2019) and Arora et al. 2018) have identified that clustered inputs induce clustered t-SNE visualizations in the sense that sufficiently well-separated Gaussian-shaped clusters in the input must produce corresponding well-separated clusters in the output visualization. key question for practitioners left unanswered by these analyses is: when does a clustered output imply clustered input? More generally, what information can be deduced about the input given a visualization? We begin to answer this question by proving that the strength of cluster separation in the input cannot be reliably inferred from the low-dimensional visualization.
To quantify the strength cluster separation in a dataset, we employ well-known distance-based cluster indices such the average silhouette score (Rousseeuw 1987), the Calinski-Harabasz index (Caliski Harabasz] 1974p. and the Dunn index [Dunn] 1974). For sake of readability, we focus on presenting results with respect to the average silhouette score. Our results hold identically for the other indices as well (see Appendices A.4 and A.5).
[n] of points {I1,..., Enl X into Z 2 parts, the silhouette score of point 24, denoted S(i). is the normalized difference between the average within- and the closest across-cluster distances from xy
Definition 2. Given a partition C1 L C2 U...
b(i) - (i)
I - x,
| - x,y1
S(i) = max maa)
a(i)
Σ JEC(O)
b(i) min
C -1
1Cml
Cm ((() JECm
where (10) is the cluster to which belongs. Note that if ((+) 1, then S(i) is defined to be zero. The average silhouette score then is simply the average across all points in X:
S(X; CmE(()
S(i). sE(m)
Observe that the (average) silhouette score ranges from -1 to 1 with higher scores reflecting large separation between clusters relative to cluster diameter. A score of zero reflects minimal separation between clusters, while negative values reflect cluster overlaps.
Without loss of generality, shall that the input dimension D
4

---

Published as a conference paper at ICLR 2026
4.1 DIFFERENT INPUTS, SAME OUTPUT
Defining the strength of a clustering with respect to the silhouette score, we show that any stationary I-SNE output (including outputs with arbitrarily well-separated clusters) can be produced by input with minimal distance separation between the clusters:
Theorem 3. Fix any k > 1, and n-point dataset X C with partition C₁ U.....LU L [n] such that mmmE > 1 and S(X; mmE(k) is well defined. For all 0 <e S 1, there exists n-point
dataset X. IR" such that
S(Xei CmElq) =
- SS((; =mm(X))m)))
yet, for any E [1, 1- 1],
I-SNE,,(X) I-SNE;((.).
important understand the implications of this result. For any nigh-dimensional dataset X (that contains well-separated clusters), we can always find an impostor dataset X, with minimal cluster separation such that all t-SNE (local as well as global) stationary points of X and X, match perfectly! In other words it is impossible to distinguish between X and X based on the low- dimensional t-SNE visualization.
As a consequence, the same visualization of salient clusters can be produced by sequence of impostor datasets containing clusters ranging from well-separated to minimally separated.
Corollary 4. For all 2 4 even, and partition C1 Li C2 [n] such that |Ci = |C2l = 2. There exist sequence of n-poinl datasets in IRn,1 {Xe)oeesı, with
S(X.; C1,
C2) =
such that for any E [1, 1], we have Y E nocegi (-SNE,(X.) with C2)
The above shows that Y, perfectly clustered visualization according to silhouette score, is local (and global, see the proof in Appendix) minimizer for any member of a set of inputs of arbitrary silhouette score. Thus, even from a visualization which is perfectly clustered, the strength of the input's cluster structure cannot be inferred.
Note that the existence of an impostor X, is not just theoretical; it can be constructed practically as an explicit construction). Hence this phenomenon can be demonstrated in real-world scenarios, see Figure - In this case, select preprocessed version of the well- known PBMC3K single-cell genomics dataset (2638 points, 50 dimensions; TÜx Genomics (2019) as X. We show that there an "impostor dataset" X. that is essentially indistinguishable from the real dataset in terms of its 2D t-SNE visualization, yet has much weaker cluster separation than the original dataset, The difference between impostor and original can be quantified by silhouette score and visualized in terms of the interpoint distance matrix and dendrogram.
well (see Appendix
worth emphasizing that while the distance information in the original and impostor dataset is dramatically different, the ordinal relationship between neighbors is identical between the datasets, as demonstrated by the corresponding structure the dendrograms.
4.2 DIFFERENT OUTPUTS, SIMILAR INPUTS
The previous section established that inputs with qualitatively distinct metric structure can yield the exact same (-SNE output. We continue with a complementary result: that near-identical inputs can yield very distinct outputs.
4.2.1 INSTABILITY UNDER SMALL DISTANCE PERTURBATIONS
Theorem S. Fix any n 2 and E [1, 1}. For all e > 0 and all Y,Y' E Im(t-SNF p.), there
exists n-point datasets = {x1,..., In) and X' = {zf...., d' C gn-1 such that Vi $
I; - x, 12
1 1-c5 <
< 1 11c
1 x 1
yer Y (X) and Y' (-S).,
5

---

Published as a conference paper at ICLR 2026
PBMC3K Single Cell
L-SNE, original
distance matrix, original
dendrogram, original
.429
.198
t-SNE, impostor
distance matrix, impostor
dendrogram, impostor
.462
.0001
Visualizations of single-cell data (top row) versus impostor dataset with arbitrarily small cluster separation (bottom row). Based the 2D t-SNE visualization (left column), it is difficult to distinguish which dataset (real or impostor) may have produced the plot. Plotting the input interpoint distance matrices (middle column) suggests that the clusters in the impostor dataset are significantly less separated than in the original datasct. The corresponding dendrograms (right column, produced using the Ward's method) further clucidate the relative strength of pairwise distances. It is worth emphasizing that the impostor dataset retains the relative rank ordering of the pairwise distances (and therefore the ordering of nearest neighbors), and only distorts the distances to make the differences much finer. Note that the color coding in of the scatterplots corresponds to a stable cut in the hierarchical clustering given by the dendrogram of the original dataset, and the bottom left labels correspond to silhouette scores with respect that clustering.
Figure 2:
Thus arbitrarily minor interpoint distance perturbations of the input dataset can develop into massive changes in the visualization. Figure 3 demonstrates this phenomenon quite clearly. We start with dataset X that is a regular unit simplex (all pairwise distances are unit length). By systematically perturbing the input X ever so slightly (c < 0.01), t-SNE produces strikingly different outputs.
The key observation behind our main Theorems 5 and 5 is the simple yet counter inntter-intuitive fact that t-SNE is not only invariant under multiplicative scaling of the input distances, but also additive shifts
thereof property also investigated independently by Lee & Verleysen (201 2014). Specifically
given dataset X {II ........ kn). for any dataset X" = { { Jn and C' I such that,
lz! 22,1' 1| ||x; - 22 + > for i of j. we have t-SNE = t-SNE,(X)) (see Lemma for formal statement, and Lee & Verleysen (20IT), Section 4). As consequence, for any input dataset, we can simply pump up the interpoint distances and construct an impostor dataset which has the same visualization profile but is arbitrarily close regular simplex (and hence is arbitrarily unclustered fl This observation also leads to the following seemingly bizarre fact.
Lemma 6. Fix any > 2 and p E [1, 1]. For any >> O, define the set of e-perturbations of unit simplex as A. == [X- =
C 1n-1, Vi + j. ||= - 13112 E [1 - 6, + e]}. Then,
for all é >
Im(t-SNE,..) = I-SNE,
*See Algorithm 1 for formalization of this process.

---

Published as a conference paper at ICLR 2026
Figure 3: Various different 2D I-SNE visualizations produced by adversarial perturbations of 200-point unit regular simplex. Each pair of perturbations satisfies the conditions of Theorem S for 0.01.
In other words there a set of datasets A, arbitrarily close to a regular unit simplex that generates all possible stationary t-SNE outputs! This result indicates that t-SNE outputs are highly unstable on near-simplex inputs (cf. Figure 3). which has real-world consequences since many high-dimensional datasets fall into this regime (Beyer al. 1999 Aggarwal al. 2001) due the concentration of measure phenomenon (Ledoux) 2001).
4.2.2 INSTABILITY UNDER INSERTION A SINGLE POINT
The previous lemma tells us that on intrinsically high-ding gigh-dimensionll data, small perturbations of all the interpoint distances can have outsize effects on the t-SNE output. We observe that such datasets are susceptible much simpler adversarial attack: namely, the insertion of single data point.
Consider a dataset X sampled from a mixture two high-dimensional Gaussians. t-SNE, as ex- pected, reveals the two underlying clusters (see Figure first panel). However, we can add just single "poison point" to X and destroy the clustered visualization (see Figure $ second panel; see also Figure T0 This failure mode of t-SNE is also observed a real high-diiensional datasets (see Figures and left vs. center).
How single point do much damage? The answer lies in how t-SNE registers neighborhood information and its potential consequences in high dimensions. In the case of Figure the poison point is placed at the mean of the dataset (this captured clearly by the PCA plot). Due to the high dimensionality of the configuration (indeed, this mixture of two Gaussians approximately regular simplex) the poison point is the nearest neighbor most of the points in the dataset. This significantly changes the input affinity matrix, downweighting the intra-cluster affinities and upweighting affinity for the poison point, resulting in a visualization which keeps points in tight proximity to their nearest neighbor (namely, the poison point).
This disproportionate amount of affinity towards the poison point is modulated by the perplexity value. As decreases, the proportion of affinity placed on the poison point increases. In the limiting case, when p = all affinity assigned to the poison point. We can formalize this case as follows:
Theorem 7. Take 2 4 even. There exists two n-point datasets such that:
* Xo C 2- has silhouette score 0 with respect to all partitions, and
X, C R has silhouette score /3-1 0.42 with respect to some partition of points,
v3
yet there exists R- such that (for 1)
I-SNE, Uzo) = t-SNE, Um)).
Therefore the poison point attack can make it impossible to distinguish between a well-clustered
and a completely un-clustered input based on their t-SNE loss landscapes (this is experimentally
corroborated in Figure П.p Our practical demonstrations (e.g. Figures g I0 [5 indicate that
similar level of degradation persists for typical settings of p in the usage of I-SNE.

---

Published as a conference paper at ICLR 2026
t-SNE, original
t-SNE, 1 paison point
original
1 poison point
Figure 4: L-SNE versus PCA's radically different responses the injection of single "poison" point in the input dataset. The original dataset, visualized in panels and 3, consists of 400 points sampled from a mixture of two Gaussians in 22000 The poison point is then placed at the mean of the previously sampled points; the resulting 401 point dataset is visualized in panels 2 and 4. Note that this behavior persists for larger datasets, see Appendix B.1
5 MISREPRESENTATION OF OUTLIERS
Most analysis of t-SNE, including the previous section, is concerned with whether it faithfully de- picts global structure, specifically cluster structure. In this section, we consider how t-SNE repre- sents points that drastically deviate from the global structure: namely, outliers. It is natural to hope that data visualization methods can enable the identification of outliers. Unfortunately, we find that I-SNE may arbitrarily suppress the severity of outliers present in the input dataset.
This phenomenon has been observed empirically in prior work, though to our knowledge we are the first to formalize it (Schubert & Gertz 2017). An intuitive explanation of t-SNE's response to outliers can be made based on the asymmetry of the input and output affinity matrices of t- SNE. Roughly speaking, the input affinity behaves like normalized, symmetrized nearest neighbor graph, whereas the output affinity behaves more like a radius neighbors graph. This means the output affinity optimized represent the outlier point in close proximity with least some points, even if it was extremely far from those points in the input.
To begin to formalize this observation, we provide a geometric definition of an outlier. Definition 8. Fix X DRD and E Rzo- We say X is an (o,,o))outie configuration if there exists a hyperplane separating To and X {2o} with margin width at least
a max (1,x(11,dann diamiX {200))-
Define the outlier number of dataset, denoted xxX), as the largest o for which there exists To E X such that X is an (o,zo)-outlier configuration.
This definition can be generalized to accommodate more than one outlier. Note that is defined with respect to a thresholded data diameter i.e. max(1, diami(X \ {xo})) to maintain a suitable notion of outlier even in the extreme case of diam(X {2o}) = 0.
Our main theorem establishes that any stationary t-SNE output, regardless of its input, is incapable of depicting extreme outliers.
Theorem 9. Fix 2 and E [1, 1]. Let Y E Im(t-SNE be any stationary 1-SNE embedding. Then we have:
a(Y) < 3.266 + (1).
The result is proven via analysis of the t-SNE gradient: we argue that if the outlier is too far away, its gradient is nonzero, thus violating stationarity. Key to this analysis is comparison between the aggregate behavior of the outlier point's 'ss affinities in the input versus the output; in other words, the comparison between Σm= Pi and Σn= Qs0- This is where the fundamental asymmetry of t-SNE comes in. While the latter is dependent former has lower bound of 1/ 1/(2n) due to the normalization of the conditional affinity probabilities.
the position of the outlier point yo, per Lemma 21 the
'The choice of fixed threshold is arbitrary and does meaningfully affect the results.
8

---

Published as a conference paper at ICLR 2026
Credit Card Fraud Detection
t- ENE
PCA
odINPUT) vs a(OUTPUT)
T-SNF
PCA
identity
6
alphsINPPUT
Synthetic Data: Gaussian plus outlier(s)
L-SNE, a(INPUT)
PCA. @(INUUT)
D(INPUT) a(OUTPUT)
t-SNE
PCA
identity
e(pha((NPUT)
t-SNE, +1 outlier
+10 outliers
+100 outliers
PCA, +1 outlier
+ +10 outl
100 outliers
Figure 5: t-SN's response to a-outliers, compared with PCA. Top row: given data monitoring financial activity (n 5050, D = 30) where one percent of users are committing fraud, PCA succeeds and t-SNE fails representing the fraudulent users as outliers. Note that all of the fraudulent users register as (a > -ouuulee with respect to the regular users: the top right we show how I-SNE and PCA represent those a-values their output. Middle row: a similar analysis on a synthetic dataset comprised a Gaussian sample plus an outlier. Bottom row: mixture of two Gaussians plus 1, 10, and 100 outliers. t-SNE shows the outliers are essentially part of the cluster structure, while for PCA the outliers overtake the structure of the embedding.
The input-agnostic nature of this result is striking: even if the input is an extreme outlier configura- tion, no stationary t-SNE output can depict its extremity past roughly a ss 3.6. This behavior stands in stark contrast to that of principal component analysis (PCA), as shown in Figure 5 on both real and synthetic data models. PCA tends to roughly preserve the outlier number, while (-SNE seldom depicts outliers past >> 0.2 in practice, and sometimes even depicts them as within the convex hull of the rest of the points (yielding a 0). Furthermore, when faced with multiple outliers, (Figure bottom) t-SNE gracefully accommodates them into the global structure of the bulk of the data, in contrast to PCA, which depicts the outliers as randomly scattered.
5
Our result suggests that t-SNE is the wrong tool to use in situations involving outlier detection. Consider, for instance, a dataset of financial transactions where the goal to detect fraudulent users, studied by Pozzolo et al. (2015). In this dataset, only 0.172% percent of the points (492 out of 284, 807) are fraudulent and by many standard statistical metrics register as outliers. Comparing the t-SNE and PCA plots fraudulent), we see that t-SNE mixes the frauds with the bulk of the points while PCA keeps them separated for the most part, Figure top row.
random representative subset of this data (5050 points, of which 50 are

---

Published as a conference paper at ICLR 2026
BBC News (transformer word embeddingg))
I-SNE
+10% poison points
+50% far outliers
399
196
.118
Figure 6: t-SNE's susceptibility poison points in contrast with its muted response outliers, the BBC news article classification dataset. The bottom left label denotes silhouette score of the original points (without the injected points) with respect to the true labels (business, entertainment, politics, sport, tech).
Finally, note the distinction between t-SNE's muted response to outliers and its dramatic sensitivity to poison points. We illustrate this distinction on dataset of BBC news articles which cluster into five topics (Greene& & Cunningham 2006). see Figure C Given RoBERTa (Liu et al. 2019) sentence 1024), we find that injecting 220 poison points, i.e. roughly 10% (see Appendix [.. for the explicit construction) can halve the silhouette score of the t-SNE embedding with respect to the ground-truth labelling, whereas injecting 1100, i.c. roughly 50%, a-outliers for E (0.8, 1) does not degrade the cluster structure.
embeddings of these articles (n 2225,
DISCUSSION
Our study of t-SNE has established in considerable generality that one cannot infer the degree of cluster salience or the extremity outliers from a t-SNE plot, see Theorems 3 and The proofs and intuitions behind these statements guided us to surprising empirical observations that one cannot even infer the existence of clusters or outliers in some cases. In particular, the injection of small subset of adversarially chosen points can largely mask the cluster structure, while sizable injections of outlier points are reliably masked within the cluster structure, see Figures
and 15
We have identified two properties of t-SNE that give rise to these idiosyncratic behaviors: (1) ad- ditive invariance with respect to the squared interpoint distances, and (2) the asymmetry between the input and output affinity matrices. While have uncovered significant failure modes that arise from these properties, we cannot completely rule out their utility. For instance, though additive in- variance may lead to exaggerated clusters, prior work has indicated that it may be useful in filtering out high-dimensional noise that may be present the imput, see e.g. Figure 9 and Lee & Verleysen
2011 2014):
Karoui 2010); Karoui & Wu 2015): Landa & Cheng (023
t-SNE belongs to a wide selection of data visualization techniques that are yet to be understood
fully Mclnnes et al. 2018: Jacomy al. 2014 Tang et al. 2016 Amid & Warmuth 2019). Our
preliminary experiments (see Appendices A.3 and B.2) suggest that the failure modes discussed this paper may extend to other force-based dimension reduction techniques. Our hope is that this work inspires the reader to explore this fascinating landscape further and pursue the essential question: what can be provably deduced from a visualization?
ACKNOWLEDGMENTS
The authors would like thank the anonymous reviewer for their helpful feedback and suggesting improvements to the manuscript. N.B. was supported by the ONR grant N0001424SB001 and The Herbert and Florence Irving Institute for Cancer Dynamics during the writing of this paper.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
10x Genomics. PBMCs from C57BLJ6 mice (v1, 150x,50), Single Cell Immune Profiling Dataset by Cell Ranger v3.1.0. https://www wwxgenomicss.ccccmmomoo..o....se com/ 2019.
Charu C. Aggarwal, Alexander Hinneburg, and Daniel A. Keim. On the surprising behavior of dis- tance metrics in high dimensional space. International Conference on Database Theory (ICDT),
pp. 420-434, 2001.
Ehsan Amid and Manfred Warmuth. TriMap: Large- scale dimensionality reduction using triplets. Computing Research Repository (CoRR), abs/1910.00200 2019.
Sanjeev Arora, Wei Hu, and Pravesh K Kothari. An analysis of the t-SNE algorithm for data visual- ization. Conference on Learning Theory (COLT), pp. 1455-1462, 2018.
Antonio Auffinger and Daniel Fletcher. Equilibrium distributions for t-distributed stochastic neigh- bour embedding. Computing Research Repository (CoRR), abs/2304.03727, 2022,
Kevin Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. When is "nearest neighbor" meaningful? International Conference on Database Theory (ICDT), pp. 217-235, 1999.
Kerstin Bunte, Sven Haase, Michael Biehl, and Thomas Villmann. Stochastic neighbor embedding (SNE) for dimension reduction and visualization using arbitrary divergences. Neurocomputing, 90:23 45, 2012.
Tony Cai and Rong Ma. Theoretical foundations t-sne for visualizing high-dimensional clus- tered data. Journal of Machine Learning Research (JMLR), 23(1): 2022.
T. Caliński and J Harabasz. A dendrite method for cluster analysis. Communications Statistics, (1):1-27, 1974.
Tara Chari and Lior Pachter. The specious art of single-cell genomics. PLOS Computational Biology, 19(8), 2023.
Cyril de Bodt, Alex Diaz-Papkovich, Michael Bleher, Kerstin Bunte, Corinna Coupette, Sebastian Damrich, Enrique Fita Sanmartin, Fred A. Hamprecht, Emöke Ágnes Horvát, Dhruv Kohli, Smita Krishnaswamy. John A. Lee, Boudewijn P. F. Lelieveldt, Leland McInnes, Ian T. Nabney, Maxim- ilian Noichl, Pavlin G. Poličar, Bastian Rieck, Guy Wolf, Gal Mishne, and Dmitry Kobak. Low- dimensional embeddings of high-dimensional data. Computing Research Repository (CoRR), abs/2508. 15929, 2025.
C Dunn. Well sepa-sepa clusters and optimal fuzzy partitions. Journal of Cybernetics, 4(1): 95-104, 1974.
Derek Greene and Pádraig Cunningham. Practical solutions to the problem of diagonal dominance in kernel document clustering. International Conference Machine Learning (ICML), 2006.
Daniel Jiwoong Im, Nakul Verma, and Kristin Branson. Stochastic neighbor embedding under f-
divergences. Computing Research Repository (CoRR), abs/1811 0tss/1811.01244 1.07 2018.
Mathieu Jacomy, Tommaso Venturini, Sebastien Heymann, and Mathieu Bastian. ForceAtlas2, continuous graph layout algorithm for handy network visualization designed for the Gephi soft- ware. PloS One, 96)xc98679, 2014.
Seonghyeon Jeong and Hau-Tieng Wu. Convergence analysis of t-SNE as gradient flow for point cloud on a manifold. Computing Research Repository (CoRR), abs/2401.17675, 2024.
Noureddine El Karoui. On information plus noise kernel random matrices. The Annals of Statistics, 38(5), 2010.
Noureddine El Karoui and Hau-Tieng Wu. Connection graph laplacian methods can be made robust to noise. The Annals of Statistics, 2015.
Dmitry Kobak and Philipp Berens. The art of using t-SNE for single-cell transcriptomics. Nature Communications, 10(1 )0511,, 2019.
11

---

Published as a conference paper at ICLR 2026
Boris Landa and Xiuyuan Cheng. Robust inference of manifold density and geometry by doubly stochastic scaling. SIAM Journal on Mathematics of Data Science. 5(3):589 614, 2023.
Michel Ledoux. The concentration of measure phenomenon. Number 89, American Mathematical Society, 2001.
John A. Lee and Michel Verleysen. Shift-invariant similarities circumvent distance concentration in stochastic neighbor embedding and variants. Procedia Computer Science, 2011.
John A. Lee and Michel Verleysen. Two key properties of dimensionality reduction methods. Com- putational Intelligence and Data Mining (CIDM), 2014.
George C Linderman and Stefan Steinerberger. Clustering with t-SNE, provably. SIAM Journal on Mathematics of Data Science (SIMODS), 1())3113-32, 2019.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pre- training Approach. Computing Reseurch Repository (CoRR), abs/1907.11692 2019.
Vivien Marx. Sceing data as t-SNE and UMAP do. Nature Methods, 21(6):930-933 2024.
Leland MeInnes, John Healy, and James Melville. UMAP: Uniform manifold approximation and projection for dimension reduction. Computer Research Repository (CoRR), 10s//880..3426, 2018.
Alina Petukhova, Joao P Cartal-a and Nuno Fachada. Text clustering with large language model embeddings. International Journal of Cognitive Computing Engineering, 6:100-108, 2025.
Andrea Dal Pozzolo, Olivier Caclen, Reid A. Johnson, and Gianluca Bontempi. Calibrating prob- ability with undersampling for unbalanced classification. IEEE Symposium Series on Computa- fional Intelligence, pp. 159-166, 2015.
Peter Rousseeuw. Silhouettes: graphical aid the interpretation and validation of cluster analy- sis. Journal of Computational and Applied Mathematics, 20:53-65, 1987.
Isaac Schoenberg. Metric spaces and completely monotone functions. Annals of Mathematics, 39 (4):811-841, 1938.
Erich Schubert and Michael Gertz. Intrinsic t-stochastic neighbor embedding for visualization and outlier detection: A remedy against the curse of dimensionality? International Conference Similarity Search and Applications, 2017.
Uri Shaham and Stefan Steinerberger.
Stochastic neighbor embedding separates well-separated
clusters. Computing Research Repository (CoRR), abs/1702.02670, 2017.
Szymon Snoeck, Noah Bergam, and Nakul Verma. Compressibility barriers to neighborhood-
preserving data visualizations. Algorithmic Learning Theory (ALT), 2026.
Jian Tang, Jingzhou Liu. Ming Zhang, and Qiaozhu Mei. Visualizing large-scale and high- dimensional data. International Conference on World Wide Web (WWW), pp. 287-297, 2016.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE Journal of Machine Learning Research (JMLR), 9(1)), 2008
Jarkko Venna, Jaakko Peltonen, Kristian Nybo, Helena Aidos, and Samuel Kaski. Information retrieval perspective to nonlinear dimensionality reduction for data visualization. Journal of Ma- chine Learning Research (JMLR), (2(,), 2010.
Ben Weinkove. Stochastic neighborhood embedding and the gradient flow of relative entropy. Com- puting Research Repository (CoRR), abs/2409. 16963,, 2024.
Zhirong Yang. Yuwei Chen, and Jukka Corander. t-SNE optimized reveal clusters in data. Computing Research CepsstonnyCootl..aaa20000133.3.0 2021.
12


Reviews:
[{"review_id": "lDLx41AxM5", "reviewer": "Reviewer_1Dsn", "summary": "This paper introduces a novel theoretical framework for understanding how t-distributed Stochastic Neighbor Embedding (t-SNE) can fabricate or distort clusters and outliers—a phenomenon that has been largely underexplored. The authors substantiate their claims with rigorous theoretical analysis and extensive empirical studies.", "questions": "1. Could the authors provide more intuitive interpretation of additive invariance?\n2. Is the theoretical analysis applicable to variants of t-SNE, such as FFT-t-SNE?\n3. Since the misrepresentation of cluster structure does not occur for PCA in the comparison experiments, could the authors provide further theoretical explanation for this phenomenon?\n4. There are other popular dimensionality reduction techniques, such as UMAP. Can the proposed theoretical framework be extended to cover these methods as well?\n5. In the preliminaries section, the authors assume that the input dimension $D = n - 1$. However, in high-dimensional statistics, it is more common to consider $D \\gg n$. In practical applications such as few-shot learning in image processing, the input dimension is often much larger than the sample size. Could the theoretical analysis be extended to cover these scenarios?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper addresses an interesting problem that t-SNE can misrepresent the cluster structure and ignore outliers of the original data, which is relatively underexplored before.\n2. The authors provide rigorous theoretical analysis and extensive empirical studies to support their claims.", "weaknesses": "1. **Limited scope:** The study focuses only on t-SNE’s misrepresentation of cluster structures and the treatment of outliers. However, the authors do not propose potential solutions to mitigate these challenges or develop a theoretical framework that could be extended to broader dimension reduction techniques or visualization methods.\n2. **Limited theoretical novelty:** The theoretical analysis mainly relies on the exponential operations used in t-SNE, involving only relatively simple algebraic calculations. This results in limited analytical depth and weak theoretical contribution, which may hinder the development of a deeper understanding of the observed empirical phenomena.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jzKZ6Mozm2", "reviewer": "Reviewer_wvWe", "summary": "The paper tackles an important but often overlooked issue with t-SNE, that is, whether the visual clusters we see actually reflect any true structure in the data. The authors provide both theoretical and empirical results showing that t-SNE can produce identical, strongly clustered embeddings even from unclustered data, and that it fails to represent extreme outliers. The analysis attributes these effects to two intrinsic properties: additive invariance of squared distances and asymmetry between input and output affinity matrices.", "questions": "Lemma 15 is missing in the current manuscript.\n\nDoes a single poison point still break the structure for $n=2000$ or $n=10000$? Otherwise, the example in Figure 3 might simply reflect finite-sample instability rather than a general property.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The focus on false positives in cluster visualization is fresh and genuinely useful.\n- The theoretical results (especially Theorems 3, 5, and 8) are carefully stated and, to my reading, technically sound. They connect interestingly with known properties of t-SNE optimization, like the concentration of measure effect in high dimensions.\n- The idea of constructing “impostor” datasets that generate the same clustered t-SNE plots is a good demonstration.", "weaknesses": "- In the paper, the empirical results are shown in $d=2$. This is understandable for visualization, but it limits the evidence for the general claims. It remains unclear whether the same cluster exaggeration or outlier suppression occurs when t-SNE embeds into 3D or slightly higher dimensions (where some of the “crowding” artifacts might diminish). The additive-invariance argument may hold mathematically, but its practical severity could depend on $d$. The author could think of a way to quantify the severity, e.g., by visualization (for 3D) or KNN accuracy measure.\n\n- The same figure contrasts t-SNE’s collapse with PCA’s stability, but PCA’s robustness is expected because it is linear and insensitive to single points at the mean. In contrast, t-SNE minimizes KL divergence on a normalized graph, so central points have a disproportionate effect by design. The figure thus compares incomparable objectives. A fairer baseline would be UMAP or TriMap, which share t-SNE’s nonlinear affinity structure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7uMr3gVFuZ", "reviewer": "Reviewer_9aD7", "summary": "The paper analyzes the t-SNE algorithm in scenarios in which the output does not preserves the clusters within the data. Specifically, the authors consider the Silhouette score as a way to quantify the clustering property of the dataset. They then show multiple scenarios in which this property is not preserved during the embedding process, which may lead to incorrect interpretation of the data.", "questions": "I wanted to emphasize again that the results are interesting, but I do have some concerns. I will be happy to update my score based on your responses.\n\n1. The authors examine a framework in which the number of samples n and dimension d satisfy d=n-1.\n1A. Could the authors elaborate more on why this framework is interesting? Isn't this regime too restrictive?\n1B. What was the framework that other works used to tackle related questions with respect to t-SNE?\n1C. A common practice in the field is to apply PCA on the data before applying t-SNE. Are your results transferable to such a case?\nThe paper [Kobak et al], which you cited in your work, states in its abstract: \"(we) develop a protocol for creating more faithful\nt-SNE visualizations .... it includes PCA initialization\". \nAdditionally, the standard preprocessing for single-cell PBMC data, such as the one shown in Figure 1, includes PCA.\n\n2. In Theorem 5, I am missing a relation between epsilon and the norm of the points within X and X'. If the norm of the points is much smaller than epsilon, the implications of this theorem may be weak.\n\n3. Your finding about the \"Additive invariance with respect to the pairwise squared distance\" requires clarification. Throughout the paper the authors indicate that they have identified this property, but while this is a straightforward property of normalized kernel, there are multiple works that uses this fact in their proofs already. \nFor example, there are papers that study effect of homoskedastic noise on kernel matrices (See [El Karoui]) and normalized Laplacians (See [El Karoui et Al]) in high-dimensional settings. While the first paper shows that that the noise results in global multiplicative factor to the kernel matrix (much like adding a global constant to the pairwise distances), the second shows how the normalization helps removing or reducing the noise, and uses a similar observation throughout its proof to the one made in this paper. \nFurthermore, this was noted in Section 1.3 \"The influence of noise\", of [Landa et Al.], using a similar description to the one used in the current paper, via a global constant that is added to the squared pairwise Euclidean distances and removed through the normalization.\n \n\nMinor adjustments: \nA. Figures captions. The figure in page 1 and the figure on the top of page 2 are missing the captions. I am not sure whether this is valid.\nB. It will be good to refer to Appendix A in Table A (page 1) as you did in the second paragraph of Section 4 (page 4).\n\n\nReference:\nDmitry Kobak, and Philipp Berens. The art of using t-SNE for single-cell transcriptomics. Nature Communications, 2019.\nNoureddine El Karoui. n information plus noise kernel random matrices. The Annals of Statistics, 2010.\nNoureddine El Karoui, and Hau-Tieng Wu. \"Graph connection Laplacian methods can be made robust to noise. The Annals of Statistics, 2016.\nBoris Landa, and Xiuyuan Cheng. Robust inference of manifold density and geometry by doubly stochastic scaling. SIAM Journal on Mathematics of Data Science, 2023.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper shows interesting theoretical results relating to the pitfalls of t-SNE, when applied without applying any preprocessing beforehand.", "weaknesses": "1. The example given at the beginning is not clear. It is hard to understand the structure of the data, and the distance matrix of the original data is non-informative with respect to the data's structure.  \n\n2. The authors do not discuss the pitfalls of the Silhouette score (and the other scores mentioned in the Appendix). For example, given a dataset composed of two curvy lines, where each of these lines is curled around itself, the Silhouette score will be high. However, a possible embedding that preserves all the structural and cluster information can be of the form of two nearby straight lines around the origin, which may result in small Silhouette score. I am not sure that this embedding is non-informative.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Yf4LM1gSB8", "reviewer": "Reviewer_WD8K", "summary": "This paper is about the faithfulness of t-SNE visualizations to high-dimensional data. In the first part, the authors illustrate counter-intuitive phenomena of high-dimensional data and discusses their effects on t-SNE visualizations. They also present the novel notion of poison points that drastically alter the t-SNE plot. In the second part, they discuss how t-SNE is unable to place outlier points far away in its visualization. Several of the statements are backed by formal proofs and the findings are qualitatively illustrated on both synthetic and real data.", "questions": "Q1: Poison points drastically change the t-SNE output, but t-SNE's response to outliers is \"muted\" (line 458). But are poison points, especially the mean point in Figure 3, not also outlier points? When does the t-SNE embedding change and when is it barely affected?\n\nQ2: How does the explanation in line 339 to 344 explain the embedding change? (a) the absolute \"gap\" between intra- and inter-distances remains the same ($2\\varepsilon$), the ratio of gap to intra-distances is smaller, though. (b) Granted that \"effective\" intra- and inter-distances are more similar in some sense with the poison point, I do not see why this has to result in such a striking change in the embedding.  \n\nQ3: Should the term $2u\\cdot t$ not be added rather than subtracted in the denominator in line 955? And if so, why can this positive term be dropped when constructing the lower bound in the next step?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- S1 The paper is well-written and for the most part easy to follow. Formal statements are well explained in natural language. Most examples are well-chosen to illustrate the findings.  \n- S2: The fact that a high-dimensional dataset can be arbitrarily close to a simplex, while still encoding any structure is interesting and deserves broader dissemination.  \n- S3: The striking effect of poison points (Fig 3) is also very interesting and novel.  \n- S4: It is impressive that the authors manage to formally back many of their statements, given that t-SNE and related methods are not easy to analyze theoretically.   \n- S5: The authors provide code, facilitating reproducibility.", "weaknesses": "**Major**  \n*W1 Presentation:* While the paper is accessible, I think the interpretation of high-dimensional data structure needs to be more nuanced. In Table 1 and related figures as well as Fig 1,2 the authors convey that t-SNE can produce structured visualizations (e.g. clusters, or astronaut shapes) of data that does not actually carry this structure (because it has low clustering scores, has nearly uniform distances, and has similar pairwise distances as datasets whose t-SNE embeddings are very different). I think it is critical to discuss in what way this high-dimensional data is structured in more detail.\n\nCurrently, the discussion focuses exclusively on the value of the pairwise distances, which are all nearly the same, suggesting a lack of structure. However, other perspectives, e.g., the kNN graph of the data (used by many visualization tools also including PHATE, clustering methods like modularity clustering, and much general manifold learning) would tell precisely the opposite story: The kNN graph would likely encode structure very similar to what t-SNE (which performs a layout of this graph) shows. \n\nThe kNN graph is multiplicatively and additively invariant. So the kNN graphs of the 2D embeddings in Fig 2 and of $n-1$-dimensional configurations obtained by Alg 1 from these 2D embeddings are exactly the same (despite these distances satisfying Thm 5 at arbitrarily small $\\varepsilon$). So from this point of view, the high-dimensional point clouds of Fig 2 are neither similar nor unstructured, but (a) carry distinct kNN graph structure that is (b) very different from each other.\n\nThis perspective could also be made quantitative by popular clustering metrics, like the modularity of the kNN graph, its conductance, or its spectral gap, or by e.g. the performance of a kNN classifier on the high-dimensional data. For instance, on the data with two clusters for Table 1, the modularity of the kNN graph is likely very close to its maximal value of 1. From this perspective, t-SNE does *not* exaggerate clusters. Given the different behavior of distances in high-dimensional spaces, which the authors mention in several places, I wonder if metrics like Silhouette score are even the appropriate choices. \n\nThis tension between different interpretations of high-dimensional structure is interesting (as is the authors' proof that one can have essentially *any* kNN-graph structure in an $\\varepsilon$-ball around the regular simplex). But currently, the discussion in the paper is very one-sided, only focusing on the distance perspective.\n\nIn addition, the above is more about the interpretation of high-dimensional data in general and less about a specific issue with t-SNE. The paper's storyline (including the title) should reflect this. As mentioned above, any method relying on the kNN graph has additive and multiplicative invariance, e.g., Leiden clustering. But would the authors similarly claim that Leiden clustering \"exaggerates clusters\" by finding the partition that realizes the kNN graph's high modularity?\n\nThere is some discussion in this direction starting in line 478, but this is insufficient. The framing is my main issue. I am happy to increase my score if it gets suitably addressed. \n\n*W2: Need for high-dimensions.* The process in Alg 1, that makes pairwise distances arbitrarily similar, requires embedding dimension $n-1$ in general. This is somewhat glossed over in footnote 1. In real-world settings, this is typically not the case. At least the intrinsic dimensionality of a dataset is typically much lower than $n-1$. Moreover, many pipelines (e.g. popular visualization pipelines for single-cell data) explicitly have a PCA step that reduces dimensionality to tens of dimensions, much lower than the dataset size, before applying t-SNE. The authors should at least comment on this. Ideally, the central statements like Thm 3, 5 could make the achievable $\\varepsilon$ dependent on a given dimensionality $2 \\ll d \\ll n-1$. \n\n*W3: Discussion of poison points.* I much appreciate the finding that poison points exist for t-SNE. But it think its current treatment is suboptimal. The bulk of Sec 4 is on $\\varepsilon$-distance perturbations, but at the end the conceptually different perturbation of adding a poison point is introduced without even starting a new paragraph or a subsection. I also think the argument on ranges of distances and \"effective\" distances could benefit a lot from a graphical depiction of the respective (effective) distances ranges with(out) poison point. \n\nWhen reproducing the experiment using the provided code, I noticed very brittle behavior with respect to the random seed. Sometimes the embedding was as in Fig 3 with the two clusters falling into two half-discs. Sometimes it had the same shape, but the clusters were well-mixed in the disc. Other times, there was a well-mixed disc and the poison point was a very far outlier. It would be good to at least mention this brittle behavior, or even show it in additional panels.\n\nIn contrast to much of the rest of the paper, the explanation for t-SNE's behavior in the presence of a poison point remains rather vague (see also question 2). I wonder if not an integral part of that explanation should be that the poison point will act as a hub in the kNN graph (since every other point has it as its nearest neighbor). I expect that it is this affinity of the entire dataset to a common point that prevents any visual separation. However, given the random seeds for which the poison point is not embedded in the convex hull of the rest of the embedding, this is not the full story either.\n\nFinally, the paragraph speaks of distances, not squared distances, in which case the distance to the poison point should be $1/\\sqrt{2} \\cdot (1-\\varepsilon)$.\n\n*W4: Proof of Lem 19:* I wonder if the proof of Lemma 19 is correct, see question 3.\n\n\n**Minor**  \n\n*W5: Outlier section.* I think the novelty of section 5 is lower than that of section 4. The fact that t-SNE pulls outliers onto the main embedding is not new, e.g. [a]. While Thm 8 is new and interesting, its practical usefulness is limited, I think, as a point with separation of three times the embedding diameter clearly looks visually like an outlier. (In practice, outliers in t-SNE are of course much closer). I also wonder how the bad situation in Fig 5 right panel really is. The outliers are clearly much sparser than the dense clusters. Detecting them, e.g., with HDBSCAN on the 2D embedding should be straightforward. \n\n*W6: Analysis of additive invariance.* Contrary to footnote three, additive invariance was extensively discussed (as \"shift-invariance\") by Lee and Verleysen [b] in the context of SNE. \n\n*W7: Verification of visual hypotheses:* Since the authors mention that t-SNE is routinely used for hypothesis generation, it might make sense to also mention that it is common knowledge that these visually generated hypothesis need to be independently verified (e.g. by running a clustering method on the high-dimensional data) [c].\n\n*W8: Presentation of proofs:* The proof of Lem 19 can be improved by mentioning that $v$ is a normalized normal vector of the hyperplane as in Definition 7 (this becomes clear later in the proof, but Def 7 does not speak of $v$ or normal vectors). Moreover, in the proof of Lemma 6, there should also be a case distinction on $D_{\\text{max}}= D_{\\text{min}}$ as the last two lines of the inequality becomes undefined if the embedding is a regular triangle.\n\n*W9: Abstract* The abstract is extremely short. When reframing the storyline, this could be expanded.\n\n**Typos:**\n\nT1 $x_0 \\in X$ not $x_0 \\in\\mathbb{R}^D$ in Def 7\n\nT2: Upper summation limits in proof of Lem 18 should be $n-1$ not $n$.\n\nT3: There are two identical equations in the proof of Thm 8. Moreover, the gradient of the loss misses a factor -4. \n\n**References**\n\n[a] Schubert, E., & Gertz, M. (2017). Intrinsic T-stochastic neighbor embedding for visualization and outlier detection: A remedy against the curse of dimensionality?. In International Conference on Similarity Search and Applications\n\n[b] Lee, J. A., & Verleysen, M. (2011). Shift-invariant similarities circumvent distance concentration in stochastic neighbor embedding and variants. Procedia Computer Science, 4, 538-547.\n\n[c] de Bodt, C., Diaz-Papkovich, A., Bleher, M., Bunte, K., Coupette, C., Damrich, S., ... & Kobak, D. (2025). Low-dimensional embeddings of high-dimensional data. arXiv preprint arXiv:2508.15929.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sn1lGf42bR", "reviewer": "Reviewer_cJrt", "summary": "This paper theoretically demonstrates that the low-dimensional visualizations produced by t-SNE may not faithfully reflect the underlying high-dimensional structure. Bu using quantitative measures such as the Silhouette score, the authors present \"existential\" cases that contradict common intuitions about cluster representation. The work raises the need for further discussion on the applicability and reliability of t-SNE in practical data analysis.", "questions": "- It would significantly enhance the value of the paper if the authors could revisit each of the points discussed in the \"Weakness\" section from the perspective of how the theoretical findings could inform or be applied during practical exploratory data analysis.\n In particular, clarifying how practitioners might leverage these results when interpreting or performing t-SNE analyses would make the paper's contribution more actionable and impactful.\n\n<to be corrected>\n- P8, l418: In Theorem 8, the expression $3+o(1)$ is presented without specifying the variable with respect to which the asymptotic term is defined. While it is likely meant as $n\\to \\infty$, an explicit statement would improve clarity, as multiple asymptotic regimes appear in the paper.  Clarifying this would strengthen the mathematical rigor of the main result.\n\n- P15, l763: \\max_{i\\neq j} → \\max{i,j\\in[n], i\\neq j}\n\n- If you put figures to the paper, they should be titled and captioned.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper provides a rigorous theoretical analysis of the stationary points of the t-SNE algorithm, which is widely used for visualizing global data structures.\nIt demonstrates that counter-intuitive mappings can arise during dimensionality reduction. The arguments are logically consistent and mathematically precise. In particular, the paper gives a clear theoretical foundation for phenomena that had previously been observed only empirically—namely, that t-SNE may exaggerate apparent cluster separations.\nThis theoretical grounding represents a significant contribution to the understanding of t-SNE’s limitations.", "weaknesses": "* The paper presents a rigorous theoretical analysis of t-SNE’s limitations but provides limited practical guidance for practitioners. It focuses on stationary embeddings $t\\text{-SNE}_{\\rho}(X)$, while in practice, t-SNE is often terminated before full convergence. The paper does not examine whether the reported phenomena (e.g., exaggeration or misrepresentation) persist under early-stopping conditions. Including numerical experiments with partially converged optimization would strengthen its practical relevance.\n* Although the authors combine theoretical reasoning with synthetic and some real-world datasets, the paper lacks concrete guidance on how users should interpret or adjust their use of t-SNE in light of these findings.\n     Readers may find the results intellectually compelling but unclear on how to apply them safely in practice.\n* In Section 4 (“Misrepresentation of Cluster Structure”), the paper shows that cluster separability may not be preserved between high- and low-dimensional spaces, yet it does not discuss how distance metrics or preprocessing might affect this issue in exploratory data analysis.\n* In Section 5 (“Misrepresentation of Outliers”), the paper explains how additive invariance leads to distorted proximity perception when outliers are present, but it offers no suggestions on how users might mitigate or account for such effects in practice. Moreover, it would be valuable to discuss how such distortions could affect subsequent analytical tasks—for example, whether applying t-SNE before an outlier detection step could degrade detection accuracy.\n     If such degradation occurs, should practitioners avoid using t-SNE for preprocessing in anomaly detection pipelines, or are there possible adjustments or alternative approaches that could mitigate this effect?\n     Clarifying these implications would enhance the paper’s practical significance and provide more concrete guidance for real-world use cases.\n* Overall, while the theoretical results are clear and compelling, the paper leaves a gap between formal analysis and practical application.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)