Token Embeddings Violate the Manifold Hypothesis
Authors: Michael Robinson
Sourya Dey
Tony Chiang
Venue: NeurIPS 2025 poster ()
Keywords: token embedding
manifold hypothesis
statistical hypothesis test
point cloud
Primary area: deep_learning
TL;DR: The token embeddings for LLMs are unlikely to be manifolds with low curvature, based upon a novel rigorous statistical hypothesis test.

Abstract: A full understanding of the behavior of a large language model (LLM) requires our grasp of its input token space.
If this space differs from our assumptions, our comprehension of and conclusions about the LLM will likely be flawed.  We elucidate the structure of the token embeddings both empirically and theoretically. We present a novel statistical test assuming that the neighborhood around each token has a relatively flat and smooth structure as the null hypothesis. Failing to reject the null is uninformative, but rejecting it at a specific token $\psi$ implies an irregularity in the token subspace in a $\psi$-neighborhood, $B(\psi)$. The structure assumed in the null is a generalization of a manifold with boundary called a \emph{smooth fiber bundle} (which can be split into two spatial regimes -- small and large radius), so we denote our new hypothesis test as the ``fiber bundle hypothesis.'' By running our test over several open-source LLMs, each with unique token embeddings, we find that the null is frequently rejected, and so the evidence suggests that the token subspace is not a fiber bundle and hence also not a manifold. As a consequence of our findings, when an LLM is presented with two semantically equivalent prompts, if one prompt contains a token implicated by our test, the response to that prompt will likely exhibit less stability than the other.

Token Embeddings Violate the Manifold Hypothesis
Michael Robinson
Sourya Dey Galois, Inc. Arlington, VA, USA souryadgalois.. com
Mathematics and Statistics American University Washington, DC, USA michhelr@american.co edu
Tony Chiang
Department of Mathematics, University of Washington,
Seattle, WA, USA
baangdmattwwwsshnttoon edu
Abstract
A full understanding of the behavior of large language model (LLM) requires our grasp of its input token space. If this space differs from our assumptions, our comprehension of and conclusions about the LLM will likely be flawed. We elucidate the structure of the token embeddings both empirically and theoretically. We present novel statistical test assuming that the neighborhood around each token has relatively flat and smooth structure as the null hypothesis. Failing to reject the null is uninformative, an irregularity in the token subspace in a v--neighborhood, B(b)). The structure assumed in the null is generalization of a manifold with boundary called smooth fiber bundle (which can be split into two spatial regimes small and large radius), so we denote our new hypothesis test as the "fiber bundle hypothesis." By running our test over several open-source LLMs, each with unique token embeddings, we find that the null is frequently rejected, and so the evidence suggests that the token subspace is not fiber bundle and hence also not manifold. As consequence of our findings, when an LLM is presented with two semantically equivalent prompts, if one prompt contains token implicated by test, the response to that prompt will likely exhibit less stability than the other.
rejecting it at a specific token v implies
Introduction
core area for Artificial Intelligence (AI) research is understand the central components of
modern learning systems, e.g. large language models (LLMs), assuming certain regularity conditions.
Extensive research efforts have been mobilized to tackle the explainability of Al question (Gunning
et al. 2019)), leveraging ideas from computer science (Gordon [224]), Wu et al. [2022] Traylor
al. 2021). ), information theory ( TTan al. [2024), Wang et [2024] ). and theoretical physics
(Chen et al. 2025), Geshkovski et al. [2023] Lu et al. 2019) ), to mathematical insights from
high-dimensional probability theory (Huang et al. 2024 Asher al. (202]), and algebraic and
differential topology (Bradley and Vigneaux (2255 Rathore et al. [202]] Bradley et al. [022]]
either to prove mathematical guarantees or to elucidate model behavior.
In many-if not most-Al research papers, a manifold hypothesis is tacitly assumed, that the data are concentrated near a low curvature manifold without boundary. This assumption drives the research just as assumption of Gaussianity drives classical approaches to data modeling. For instance, the Uniform Manifold Approximation and Projection (UMAP) algorithm (MeInnes al. (2018)) explicitly assumes that data are sampled near low dimensional manifold and cautions the reader that
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

the algorithm may not be reliable if this assumption is violated. Yet many researchers make inferences from UMAP, rarely checking that this assumption holds, and even arguing it is unnecessary. This
argument has been routinely rejected in the literature (Jeon et al. (2255] Chari and Pachter (2023 )
and so it is important to test this kind of assumption before data modeling. One common testing strategy is simply "plotting the data" to ascertain the empirical distribution. Plotting high-dimensional
data, however, is non-trivial, but there are statistical tests to help us understand the underlying distribution (Mann- (Mann-Whitney, Kolomogorov-Smimoo etc.). In this paper, we provide an analogous test for the regularity of token embeddings.
The first layer of most LLMs is usually comprised of the token embeddings, which are vector representations of tokens (fragments of texts). Thus, a good first step to explain the behavior of an LLM is to understand its initial token subspace where these embedding vectors lic. For the remainder of this paper, the term "token"" used in the context of the token subspace will refer to this embedding.
Numerous papers (for instance, see Verma et al. [202]] He et al. [2024] Vargas et al. (to appear), [L] et al.] 2024). Battle and Gollapudi] [2024], Valmeekam et al. 2023]. Sclar al. 2023 Chao et al. [2023] have pointed to unexpected behaviors exhibited by LLMs that hinge on subtle changes in wording and text layout between apparently similar prompts, suggesting that certain semantically similar tokens may have dramatically different topological neighborhoods in the token subspace.
Linguistically, these irregularities may correspond to polysemy or homonyms (Jakubowski et al. (0200).
The presence of irregularities imply small changes in language can drive large variation in the token subspace. Moreover, if the token subspace of an LLM is, in fact, singular, hence not manifold, irregularities can persist into the output of the L.M,, perhaps unavoidably and potentially even because of its architecture. This is because the input and output of LLMs are a fixed finite set of tokens yet the internal architectures of the model assume a structure resembling a Riemannian manifold: a metrizable space that locally preserves inner products. Not accounting for singularities in the token subspace may thereby confound our understanding of the LLM's behavior.
1.1 Contributions
Our contributions are: (1) two novel statistical tests (for manifolds and for fiber bundles; see below), (2) evidence to reject both hypotheses for several LLMs (Section(5.2) and therefore (3) the identification of otherwise unreported structure in the embeddings for these LLMs. We emphasize that while this paper demonstrates the manifold and fiber bundle tests on token embeddings, our main contribution is primarily methodology for hypothesis testing on embedded points.
Because connected manifolds have a unique dimension, and because the dimension of fiber bundles are characterized by Theorem [ our two tests address the following hypotheses given in Table [
Table 1: Hypotheses we test. We assume the token subspace has reach 7 > 0 (see definition in Sec. 2 and estimate dimension in the ball centered token with radius r T.
Manifold test
Fiber bundle test
The dimension at & in a ball of radius r
Ho
There a unique dimension at var
does not increase as r increases.
H1
There is not a unique dimension at g/
The dimension at @ increases at some r
Both tests are implemented as Algorithm I The embedded reach 7 defines an assumption on the curvature of the underlying space. Low reach corresponds to high curvature and vice versa (Berenfeld et al. 2022]). Classical Lagrange interpolation asserts that any finite sample of points can be fit exactly by manifold. The resulting manifold may have extremely high curvature, which is undesirable because it results in numerical instability (Isaacson and Keller (2012)).
To demonstrate our method, Figure 3 (aa (c) shows the application of the manifold test to several
synthetic examples. In Section 5.2 we then provide empirical evidence that the token embedding function obtained from each of four different open source LLMs of moderate size GPT2 (Radford
ct al. 2209)). ), Llemma7B (Azerbayev et al. (2024), ). Mistral7B (Jiang et al. (2023), ), and Pythia6.97 (Biderman et al. 2023 cannot be manifold due to non-constant local dimension as well as tokens
(points) without an intrinsic dimension. When these tests are rejected at token tb. this implies that
d is less stable under mathematical transformations than other tokens. In Section 3.2 we show the

---

endemic nature of these instabilities by proving that context cannot resolve singularities persistently. And finally, we provide conditions (Theorem 2 for when singularities can propagate into the output of an LLM for generic transformers.
1.2 Background and related work
Manifold learning takes its roots in nonlinear dimension reduction of high dimensional data into low dimensional representations of smooth manifolds. While these algorithms have played an important role in modern data science especially in domains such as cosmology (Park et al. (0224 D, astrophysics (Makinen et al. (2024), Einig et al. (2023). dynamical systems (Masuda and Kundu [2022], Parsons and Rogers| 20155) ) and quantum chemistry [Hemmateenejad et (2004). D. the main utility of these methods have been for data visualization. For instance, even though UMAP provided an optimal algorithm for fitting sampled data to a manifold, the authors themselves concede that "if the global structure is of primary interest then UMAP may not be the best choice for dimensionality reduction". manifold learning algorithm will find some smooth structure fit the data rather than "error out" when the data is not sampled from manifold as it is not a hypothesis test. Grigsby et al. [2023] showed the dimension of ReLU-Nets are inhomogenous, and so when algorithms fit manifold to data optimally, for instance, the found manifold may have extremely high curvature or dimension. Both of these are undesirable properties in a model, and both implicitly suggest that a manifold would be a poor choice for representation [Isaacson and Keller (2012)).
To address this concern, Fefferman et al. 2 presented test for whether data lie on a manifold with a specified embedded reach, but did not present the values of constants that are critical for practical implementation. More recently, [Lim et al. [2023] Von Rohrscheidt and Rieck [2023] present tests for the manifold hypothesis, with promising results on low dimensional datasets. Our paper remedies both issues: we construct a practical implementation of hypothesis tests for manifolds and fiber bundles with a specified embedded reach.
The token input embedding matrix defines the salient subspace of the latent space for LLM,, by specifying where the tokens are located. There is no a priori reason to suspect it is a manifold. It has already been shown that local neighborhoods of each token have salient topological structure (Rathore et al. (2023). ). and one of the most basic parameters is the dimension near any given token in this space. Higher dimension tokens have less density (Jakubowski et al. (2220). Intuitively, densely populated neighborhood (b²) will generically occupy higher dimension than otherwise.
token v means that & has greater density while lower dimensional
Numerics, months, days of week, cardinal directions
probability density
Single English words
0.02
Mostly word fragments and a few single words
0.01
0.00
200
400
600
800
dimension
Figure I: The histogram of local dimensions estimated near tokens in GPT2 (Robinson et al. 2024 This histogram shows a mixture with at least three distinct peaks.
Assuming that word embeddings yield manifolds, some researchers have used global dimension estimators on token input embeddings and word embeddings (Kataiwa et al. 20225.. Gromov et al.
[2024), Tulchinskii et al. (2023). ). By using local (not global) dimension estimator, Robinson et al.
(2024) presented the first (to our knowledge) direct test of whether the token subspace manifold
for the token input embeddings for several LLMs, although rigorous hypothesis test was presented.
3

---

The subspace of tokens has multimodal distribution of dimension estimates (Figure D so unless the space is highly disconnected, cannot be a manifold.
Mathematical background
A manifold is space in which every point has Euclidean neighborhood, and the space can be covered by taking finitely many of these neighborhoods at each point. follows that a manifold is connected, then the dimension of each of these Fuclidean neighborhoods is the same; call this the dimension of the manifold. Figure 3 (a) shows the hollow sphere, which is an example of a 2- 2-dimensional manifold subset of TRa.
Given an embedding dimension É, embedded tokens lie on subspace X Euclidean space R', with the topology and geometry inherited from R Equivalently, one can consider an embedding function e: X + RR, which one-to-one and has full rank matrix partial derivatives (Jacobian matrix). Embedding functions therefore preserve topological structure. The inherited information includes metric (the Euclidean distance) and a notion of volume. The volume form on a subspace is induced by the Euclidean volume. If the subspace is manifold that is compatible with the inherited metric and volume form, we call the subspace a Riemannian manifold. For a 2-dimensional Riemannian manifold, the volume form is the familiar notion of area. In general, if space is Riemannian manifold of dimension d, then the volume v of a ball of radius r centered at any point is given by an equation of the form Gray 1974).
V Krd + (correction terms),
(1)
where K > 0 is constant depending d. If the correction terms are large, the manifold is said to have high curvature.
The definition of manifold is global: if even one point fails to have Euclidean neighborhood, then the space cannot be a manifold, and such a point is called a singularity. One commonly encountered singularity is boundary, in which point has neighborhood that is topologically equivalent to a half space. Spaces whose only singularities are boundaries are called manifolds with boundary. For instance, the solid ball in R is manifold with boundary. Its set of singularities is the hollow 2-dimensional sphere.
Classifying the types of singularities up to topological equivalence of their neighborhoods is in general an open problem, though some types are common. Figure 3b) shows a space with cusp and boundary singularities. Figure (( shows a singularity associated with change in dimension.
In order handle singularities, we also consider generalization of a manifold called a fibered mani- fold with boundary (fiber bundle for short). This is a space in which every point has a neighborhood that can be written as a Cartesian product boundary. We call B the base space and F the fiber space.
F, where B is a manifold and F is manifold with
Curvature (the correction terms in Equation ( does not generalize cleanly to spaces with singulari- ties. Instead, the embedded reach plays similar role in what follows. The embedded reach (simply reach) for space X R$ the largest distance 2 that point y E RC can be from X, yet still have a unique closest point on X. For convex subspaces, = co. Conversely, spaces with a cusp (like Figure (b)) have 0. Manifolds with small reach have high curvature, and vice versa (Berenfeld et al. 2022
3 Theoretical insights
Theorem T asserts that for a fiber bundle, the number of tokens within a ball of a given radius centered at a fixed token is piecewise log-log-linear linear function of radius, in which the slopes correspond to dimension, and the slopes cannot increase as r increases.
3.1 Volume versus radius for fiber bundles
The learned token embedding vectors in an LLM each have the same number of elements, which we refer to as f. Thus, the explicit representation of the token subspace T used by an L.M arises by an embedding function T + E into the latent space RR. Since embeddings are one-to-one, each

---

token / E T corresponds to unique element e(()) E R in the latent space. With a slight abuse of notation, we can write v for both.
Theorem 1. Suppose that T a compact, finite citte-dimensooall Riemannian manifold with boundary, with volume form v satisfying v(T) < oo, and let p T - S be a fiber bundle. If e T + R is smooth embedding with reach T, then there is a function p e(T) + [0, r] such that if e(T), the induced volume (e,v) in Th! satisfies
(eLv) (B.( - {
[0 10 (1ddin7)
if0 Sr S p());
(
(Eav) (Brro(ti) + + ((r- p( (()))d)))
if pbb) Sr,
(2)
where B, (1b) the ball of radius r centered 1b, and the asymptotic limits are valid for small r.
Proof. See Section A.5 for full details; sketch follows. Taking the logarithm of both sides of Equation T yields linear equation, in which d is the slope.
For the case of the fiber bundle, Equation T must be modified to handle the case where the ball of radius r "sticks out" of the space. This causes a reduction in volume, hence a smaller slope. 0
The probability distribution of tokens in T can be modeled by the volume form u. If fiber bundle is the correct representation of the space of tokens, Theorem characterizes the resulting probability distribution in F' using parameters (the exponents in Equation (2) that can be estimated from the token input embedding. These parameters are bounded by the dimensions of S and T. In Algorithm| [ Monte Carlo estimates of volume are used, simply by counting tokens within a given radius. Although the convergence of Monte Carlo estimates is a strong assumption, is also extremely well-established in the literature.
Violuee
Vislates liber
manfold
bundle
but not (iber bunille
volume (log scale)
hypothesis
hypothesis
(dimension change the space)
(bomadary present in the space) volume (log scale)
daye
Siee
Reject tiber bundl?
isaae increase)
Reject monileed! (ákppe changs)
Dree's
radius (lng scale)
radius (log scale)
Figure 2: Interpretation of the manifold and fiber bundle tests: the manifold test detects any slope changes in the log volume versus log radius curves, while the fiber bundle test only detects slope increases.
In short, if we take the logarithm of both the volume and radius in Equation 2 we obtain linear curve for manifold. Abrupt slope changes imply that the space is not a manifold. For fiber bundle, we obtain piecewise linear curve, in which the slopes are given by dim S dim T, and the slopes must decrease through a discontinuity as the radius increases, as shown in Figure 2
3.2 Context does not persistently resolve singularities
The remaining question whether singularities can persist into the output of the LLM. Can the user elicit the instabilities caused by singularities? We provide sufficient conditions for unstable outputs:
Theorem 2.
Let Z be a d-dimensional bounding manifold the token subspace, such that T C Z. Consider an LLM with a context window of size w, which the latent space of tokens is Rk! and we collect m tokens as output from this L.M.
Suppose the following, (enough tokens are collected from the response) m > 2wd E but (the context window is longer than the number of tokens we collected) w 2 m. Under these conditions, a generic set of transformers yields topological embedding of T T x . into the output of the LLM.
Proof. See Section A.5
Since T' C f is always contained within such bounding manifold 2 C 1
5

---

As a consequence, there always exists such an m satisfying the hypothesis 2d < é, i.. the bounding manifold has dimension less than half the latent space dimension. If our hypothesis tests show that the token subspace contains singularities and the hypotheses of Theorem 2 are satisfied, singularities will persist into the output of the L.M for generic transformers even for large context window sizes.
Methods
The tests we propose implemented via Algorithm 1 The coordinates of the points (token embedding vectors, in Section are fixed when we analyze the data, and they do not change from Algorithm[]] is fully deterministic, there is requirement of multiple runs. Centered at a given point, the sorted list of distances to all other points contains the radii at which the Monte Carlo estimate of the volume of ball increases. The corresponding volumes are simply the indices of this list. Both tests estimate dimension as slope in the log-log progression of the entries versus the indices (3-point centered difference estimates via numpy- gradient O), and detect slope changes.
Algorithm 1 Manifold and fiber bundle tests
Require: -1,-... In E Re: coordinates for each point
Require: Umin and Umar: minimum and maximum number of tokens in neighborhood
Require: W: sliding window size
Require: a: significance level
Ensure: pr: set of p values for manifold hypothesis
Ensure: p2: set of p values for fiber bundle hypothesis
Ensure: Set of dimension estimates
1: procedure
Umin Umaz- W)
pairwise distance matrix D between all tokens 3: for Each column of D) do
2: Compute
>> Columns correspond to token indices Now row indices of distance matrix are volumes, entries radii
4: Sort the column
5: Retain rows Umin through Umar
6:
Compute log-log slopes (= dimension estimates) along the column
7:
Run two sample Teest along adjacent sliding windows of size W with level a:
Manifold test: Append to p1: the p-value for the hypothesis that the slope is constant
Fiber bundle test: Append to p2: the p-value for the hypothesis that the slope decreases with row index
8: Store both values and slope with corresponding token (column index)
9:
end for
10:
Apply Holm-Bonferroni multiple test correction to both sets of p-values
end procedure
5 Results
To demonstrate the validity our tests practice, we demonstrate them on three synthetic datasets in Section 5.T shown in Figure (c)- We then turn to the main thrust of the paper, testing LLM token embeddings in Section 5.2 Source code is available at Robiinsn 2025).
5.1 Synthetic datasets
3 a) shows the result of applying manifold hypothesis test to the unit sphere in R3, Since the sphere is manifold, the p value for every point tested is close to 1, and no rejections of either hypothesis occur.
Figure
Figure 3b) shows the result of applying the manifold test portion of the surface given by
8 + 2 2² O, where x² + y² < 1.
This surface is a fiber bundle in which the base space is simply the origin, and the fiber spaces are the radial line segments tracing out from the origin to the edge of the disk. a manifold with boundary (not manifold), and the origin y 0 is cusp singularity. The number of points

---

Pinch point
(a)
cusp
(b)
(d)
singularity
Pinch point ly
Cusp point ember
pvalue
0.0
Cusp point
boundary
dimension
02
point
change
ISNE courdi
(c)
Zoomed in region
PC2
ISNE coord L-SNE plot tokens in Mistral
Figure 3: This figure shows the application of this paper's manifold hypothesis test on three synthetic datasets: (a) manifold. the unit sphere in F.3 (b) fiber bundle, the cusp surface 28 x? y² in R2, (c) neither, a "lollipop" space, and (d) neighborhood of the token ember in Mistral7B. Points are colored by the p values of the manifold hypothesis test; darker colors mean reject the manifold hypothesis.
participating in the cusp and boundary is small (41 points) compared with the total number of points the dataset (1200 points).
The manifold hypothesis test strongly detects the cusp singularity in Figure 3(b). with p < 5 10 The manifold test fails detect the boundary (p 0.001). While using larger Umar in Algorithm improves the test's s sensitivity to the point where the boundary is detected, it also reduces the specificity of the test for identifying which points are singularities. On the other hand, the fiber bundle test is not rejected at all, which is consistent with the space being a fiber bundle.
Finally, FigureBic) shows sample of 3000 points drawn from space shaped like a candy "lollipop." hollow sphere (the
constructed by attaching a line segment (the "stick") to a point on the surface "candy"). The lollipop is neither manifold nor fiber bundle. On the interior of the stick, the space a I-dimensional manifold. The candy is 2-dimensional manifold, except where the stick attaches. The entire space has exactly two singular points. At the very end of the stick is singular point (a boundary point). Where the stick attaches to the candy is another singularity; it is transition from the 1-dimensional stick the 2-dimensional candy.
Running the manifold hypothesis test on the lollipop reveals low p-values on points near the boundary of the stick (75 points) and at the transition (58 points) (both have < 10-5). Due to random sampling, the exact points singularity are not actually present the dataset; these detections are near the two singularities. None of the points near the boundary point of the stick cause the fiber bundle hypothesis to be rejected with p < 10 while 2 points near the transition cause the fiber bundle hypothesis to be rejected with p 10 Since even rejection is sufficient, the tests correctly identify that the lollipop is neither a manifold nor a fiber bundle.
5.2
LLM token embeddings
We applied our Algorithm to the token subspaces of the four open source LLMs introduced in Sec. [.1] GPT2, Llemma7B. B. Mistral7B, and Pythia6.9B. In each LLM we tested, Table exxhibits evidence that the token subspace not globally a manifold because the local dimension is highly variable ( ("dim." columns). Moreover, can infer that they are either not fiber bundles have high curvature locully; either case-violation the hypotheses or having high curvature- model behavior near certain tokens will be highly unstable (low p-values in the "rejects" columns). Tables
7

---

i in the Supplementary material list every violation we found. Additionally, Figure shows the application of the manifold test the tokens of Mistral7B in the vicinity of the token ember. The manifold hypothesis is rejected at various tokens (low p-value, shown in darker color).
For each LLM studied, chose two pairs of Umin and Umaz parameters in Algorithm 1 one yielding a small radius neighborhood and one yielding larger radius neighborhood. These parameters were chosen by inspecting a small simple random sample of tokens, with the aim of identifying where changes in dimension were likely to occur. See Section A for details. This configuration results in three tests: single test for the manifold hypothesis, the fiber bundle test for small radius, and the fiber bundle test for large radius.
Table 2:
Dimensional data and number of tokens rejecting the manifold and fiber bundle hypotheses two slope changes. is the number of tokens (vocabulary size) in each model. The p-values displayed report the minimum p-value for each test, accounting for multiple testing. Quartiles (Q2 the median) are shown for the distribution of the dimensions estimated from the slopes of the log-volume versus log-radius plots aggregated over all tokens. The "rejects" columns list the number of tokens rejecting each hypothesis. Tables I in the Supplementary material show each of the manifold and fiber bundle hypothesis violations we found.
Fiber bundle
Manifold rejects
Model
Smaller Radius
Larger radius
dim.
rejects
dim.
rejects
Q1: 20
Q1: 8
GPT2 50257
66
12
7
Q2: 389
Q2: 14
p 3 X 10-8
p x 10-6
p 3 10-8 8
Q3: 531
Q3: 32
QI: 4096
Q1: 8
Llemma7B n = 32016
33
1
Q2: 4096
Q2: 11
p RJ 5 X 10
N/A
p 3 X 10
Q3: 4096
Q3: 14
Q1: 9
Q1:
Mistral7B 32016
40
1
2
Q2: 48
Q2: 6
= 3 X 10
Q3: 220
p =
X 10
Q3: 9
p 8 X 10 10-5
Q1:
Q1:2
Pythia6.9B n = 50254
54
0
0 N/A
Q2: 108
Q2: 5
p Rc 2 x 10-7
N/A
Q3: 235
Q3: 145
Table 2 shows the results of our hypothesis tests for the four models we analyzed. The models consequently have token input embeddings with vastly different topology, and all of them exhibit highly significant rejections of the manifold hypothesis. GPT2, Llemma7B and Mistral7B also reject the fiber bundle hypothesis. The rejections of the fiber bundle hypothesis are more frequent in the larger radius region than the smaller radius region, which is consistent with the polysemy interpretation oakuboowsii et al.] (2020).
The "rejects" columns of Table 2 give the number of tokens that fail our test (in the top rows of each cell). The listed p is the overall y-value after Bonferroni correction. The "dim." column shows the quartiles for the estimates of dimensions, at least for the tokens for which dimension can reliably be ascribed. Note for each of the tokens listed in Tables 10 no dimension estimate is possible.
Since there are many rejections of the manifold hypothesis, we summarize some general trends, obtained by manual inspection of Tables 5 10 in the Supplementary material, to which we direct the reader. The GPT2 tokens at irregularities are tokens that can only appear at the beginning of words. The Pythia6.9B tokens at irregularities
nearly all word fragments or short sequences of text that are quite meaningless on their own. The Llemma7B and Mistral7B tokens at irregularities a combination of the previous two: either they can only appear the beginning of words or they are word fragments.
While most of the tokens are not shared between the LLMs, Llemma7B and Mistral7B have identical token sets. The fact that Table [ablee]]sone shows significant differences between these two models indicates that the structure of the irregularities for these two models is quite different. This alone implies that their response to the same prompt is expected to be markedly different, even without considering the
More views of Figure d) are shown in Figure in the Supplement.
8

---

differences in their respective transformer stages, corroborating the results shown by Vargas et al. appear
Table Testing the hypotheses of Theorem Will singularities generically persist into the output?
See statement of Theorem 2 for variable definitions. "Small" and "Large" refer to the small and large
radius estimates of dimensions from Table
Model
Latent dim
Bounding dim.
Context w
Min. output tokens
Satisfied?
m such that
w 2
m?
Small
Large
Small
Large
Small
Large
GPT2
768
389
14
1024
1038
38
No
Yes
Llemma7B
4096
4096
11
4096
8193
23
No
Yes
Mistral7B
4096
48
6
4096
97
13
Yes
Yes
Pythia6.99 4096
108
5
4096
217
11
Yes
Yes
Estimates of the bounding manifold dimension used in Theorem for the four LLMs we studied are the "dimension" columns of [ with the most stringent being the dimension for smaller radius. Table 3 shows the results of testing the hypotheses of Theorem 2 using the medians (Q2). Mistral7B and Pythia6.9B 9B satisfy the hypotheses of Theorem 2 using the small radius dimension estimates, but GPT2 and Llemma7B do not. All four LLMs satisfy the hypotheses of Theorem if we instead use the larger radius dimension estimates. This means that it is likely that the transformers of Mistral7B and Pythia6.9F are unable resolve the singularities that show exist in the token subspace, so singularities will persist in the output of the model. If the true bounding manifold for GPT2 Llemma7B is smaller than the small radius estimate suggests, these too will likely produce singularities in their output.
6 Discussion
The large distribution of local dimension (Figure I and Table 2 rejects that the token embedding is a globally smooth manifold for each of the LLMs we studied. Beyond non-constant dimension, we also uncovered token neighborhoods without well-defined intrinsic dimension, structure not previously reported in these embeddings. These irregular structures suggest that the topology of the embeddings are more complicated than generally assumed. Consequently, all four token subspaces cannot locally be manifolds with large reach, and three cannot be fiber bundles with large reach. Given the aggregation of training data across domains (and even across natural languages), the potential for internal contradictions (contronyms) and confounders (natural language, tweets, source code), it is not surprising to see that our hypotheses are rejected.
The semantic role of tokens likely contributes to LLM sensitivity of prompts that has been observed in the literature (Schulz et al. (2025). Cui et al. [2025] Zhao et al. [2025a) Zilberman ( [2023]. Chao et al. [2023] and may underscore why elucidating LLM behavior remains challenging and open. The manifold hypothesis gave both an intuitive and mathematically grounded explanation for the behaviors of learning machines (Liu et al.) [2024), Brahma et al. [2015], Narayanan and Mitter (2000) yet the instability LLM outputs strongly suggested that these models were different. Our work gives conclusive empirical evidence against the manifold hypothesis for token embeddings and may offer novel ways to interpret LLM behavior as we are not regular setting.
Sclar et al.)
Such irregularities are noticeable in a principal components analysis (PCA) projection of the token subspace (Figures d). 7 and 8) but are extremely hard interpret priori. Mathematicians are still classifying singularities in 1 /Zhu et al. (2024)) implying that the classification problem is far from complete (even )) Intuitively, this means that the use of inner products to perform token ne-embeddings by orthogonal projections may not be accomplishing the intended objectives. Even small perturbations within the latent space may result in instabilities these projections, providing potential explanations for the instabilities in LLM outputs. In addition to token instability, the fact that token subspaces are not manifolds and may have small reach means that the geodesic distance between tokens can also be very unstable. While the distance along geodesics can be defined, it may not correlate with any sense of semantic distance between tokens. Furthermore, as RRobinson et al. 2024) found, in most of the models, there are tokens with dimension 0 neighborhoods. These tokens

---

are theoretically isolated, implying that the token subspace is disconnected and hence intrinsically infinitlly far from other tokens.
Singularities may arise either as artifacts of the training process or from features of the languages being represented. Consistent with the idea that polysemy may yield singularities (Jakubowski et al. (2020), several of the tokens where the fiber bundle hypothesis is rejected are clear homonyms. For instance, both "affect" and "monitor" can be used either nouns or verbs, and their meanings are different in these two roles. A token may also correspond to a pair of homonyms after the addition of a prefix or suffix. token like "aunder" can be prefixed to yield the word "launder", word with multiple meanings of opposite sense. Specifically, one can "launder" clothing (positive connotation) or "launder" money (negative connotation). Several other tokens where the fiber bundle hypothesis is violated form words with substantially different meanings or grammatical roles upon adding prefix or suffix. For instance, "wins" can function as a noun, as a verb, or form the adjective "winsome".
The differences in how the manifold and fiber bundle hypotheses are rejected across different L.MSs suggest that the training methodology for each model leaves an indelible fingerprint (or gDNA coined by Vargas et al. [to appear)). Making general assertions about LLMs without consideration of their training and evolution likely fraught. Even between Llemma7B and Mistral7B, which have identical token sets, prompts likely cannot be "ported" from one LLM to another without significant modifications if they contain tokens near irregularities, likely a consequence of the invariance of domain theorem. This may have serious ramifications for interacting LLMs such as those powering
multi-agentic frameworks being deployed today (Gu et al. [025)_ Jin et al. [2025) Qian et al. (025]).
Tokens that begin a word or are a word fragment are often located at an irregularity. Additionally, in LB (but not Mistral7B) and Pythia6.9B, the tokens with unusually low fiber dimension often contain non-printing or whitespace characters. This suggests that these models are quite sensitive to text layout, perhaps the exclusion of more semantically salient features in the text. Given our findings, future experiments can be run to explore the impact of irregular tokens on the variability of responses produced by different LLMs.
A natural question is we can find a token embedding that is everywhere regular. One way to approach an answer is to view the number of tokens in a vocabulary a hyperparameter. There might he a certain vocabulary size that minimizes the number of singularities given the corpus, architecture, etc. Even the best vocabulary might lead to space that could still be singular, for reasons inspired by Dakuubowskii et al.] [2020)- Consider the * symbol, which happens to be singular for Mistral7B (See Table number 29). In natural language, it has very distinctive meaning in terms of calling out footnotes, while in other contexts it can be used for multiplication or marking text as bold (in Markdown). Therefore, this character connects these different semantics. Suppose that each language were in fact well-described by a manifold. Then these manifolds would be joined together in the entire latent space at the (and likely other points). As a result, this token with high likelihood would be a singularity because the dimensions for different languages are almost surely different (Tulchinskii et al. (2023). In reality, tokens are neither in 1:1 correspondence with words nor with grammatical symbols. Not only do parts of words map onto a single token, but tokenization is a many to one map across the board. How well context / attention transformers can unfurl these mappings is a big part of the mystery of how these models work. Identifying and analyzing the irregularities may help us understand the approximate limits of aatenneenttransfommerss architectures.
As we attempt to model natural phenomena particularly in our physical world, we know singularities exist (e.g. phase transitions), and so assuming regularity limits our understanding of the natural world. Our work extends this notion to the linguistics of natural language where its richness and complexity do not appear to manifest in a smooth representation.
7 Limitations
While topological theory motivates our statistical test, only the slope changes that occur for radii less than the embedded reach correspond to rejections of the manifold or fiber bundle hypotheses. If the reach of a manifold is small, then it likely has high local curvature, and measuring distances on that surface will be numerically unstable. And lastly, we emphasize that these tests allow us to reject the structure assumed in our null hypothesis (smooth and regular), but do not permit us to infer an alternative. While rejecting regularity does imply singularity, it does not afford us any evidence as to the type of singularity. That must be determined post hoc if possible.
10

---

Acknowledgments and Funding Disclosure
The authors would like to thank Mohammed Abouzaid, Anand Sarwate, Andrew Lauziere, and Andrew Engel for helpful suggestions on a draft of this manuscript.
This material is based upon work partially supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR001 17000114400 124C0319. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Defense Advanced Research Projects Agency (DARPA). Distribution Statement "A" (Approved for Public Release, Distribution Unlimited) applies to the portion of the work funded by DARPA.
The authors have competing interests declare.
References
David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf. and Guang-Zhong Yang. XAI-Explainable artificial intelligence. Science Robotics, 4(37):caay 120, 2019. doi: 10.1 1 aay 1200. URL https: bttpps:/wwww.or http::/wwww.sieeeeee wwwwwsscencc.... wttp/////////////////////ooooaet 1.00..122 1126/ scirobotics a2y7120
Colin S. Gordon. The linguistics of programming. In SIGPLAN symposium on New ideas, new paradigms, and reflections on programming and software, 2024. URL https://apt
semanticscholar orggooogggo www.:ola....rrrroprrr:::///////0 CorpusID:2222597288
Zhaofeng Wu, William Merrill, Hao Peng. Iz Beltagy, and Noah A. Smith. Transparency helps reveal when language models learn meaning. Transactions of the Association for Computational Linguis- tics, 1 1::17-633, 2022. URL :ttpps://aaa //ppi.ppp.../// www.nticscoooeee.....ooooocccc
axg/CorpusID- CorpusIDD25220722:5 252907248
Aaron Traylor, Roman Feiman, and Ellie Pavlick. And does not mean or: Using formal languages to study language models' representations. Annual Meeting of the Association for Computational Linguistics, 2021. URL https://api. wwm.nticscoollrr.......occcc.aa. wwwpCorrggIDDaa.. : 236459821
Zhiquan Tan, Chenghai Li, and Weiran Huang. The information of large language model geom- etry. ArXiv, abs/2402.03471, 2024. URL https://api. semanticscholar wrgp/Crppss/DDDDD 267499620
Xuhong Wang, Haoyu Jiang, Yi Yu, Jingru Yu, Yilun Lin, Ping Yi, Yingchun Wang, Yu Qiao, Li Li, and Fei-Yur Yue Wang. Building intelligence identification system via large language model watermarking: A survey and beyond. ArXiv, abs/2407.111000 2024. URL https
seapi.semanticschollrr Drg/CorpusID- : 271218007
Shi Chen, Zhengjiang Lin, Yury Polyanskiy, and Philippe Rigollet. Quantitative clustering in mean-field transformer models. arXiv preprint arXiv:2504.1469) 2025.
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. A mathematical per- spective on transformers. ArXiv, abs/2312.10794 2023. https://:// semanticscholas
org/CorpusII:2200 :2: 266359450
Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang. and Tie- Yan Liu. Understanding and improving transformer from a multi partiiee dynamic system point of view. ArXiv, htss1996.00766,, 2019. URL httph://appp semanticscholis oro//oorpull
174801126]
Yi-Chong Huang, Xiaocheng Feng, Baohang Li, Yang Xiang, Hui Wang, Bing Qin, and Ting Liu. Ensemble learning for heterogeneous large language models with deep parallel collaboration. In Neural Information Processing Systems, 2024. URL https://app semanticscholar org/
CorpusID: : 269282634
Nicholas M. Asher, Swarnadeep Bhar, Akshay Chaturvedi, Julie Hunter, and Soumya Paul. Lim- its for learning with language models. ArXiv, abs/2306. 12213, 2023. URL httpps: :estps://api semanticscholar.. org/CorpusID:259000 2592211162

---

Tai-Danae Bradley and Juan Pablo Vigneaux. The magnitude of categories of texts enriched by language models, 2025. URL https://arxiv org/ htggabb/2501..6655 06662
Archit Rathore, Yichu Zhou, Vivek Srikumar, and Bei Wang. Topobert: Exploring the topology of fine-tuned word representations. Information Visualization,
2023.
Tai-Danae Bradley, John Terilla, and Yiannis Vlassopoulos, An enriched category theory of language: from syntax to semantics. La Matematica, 122::511-580, 2022.
Leland McInnes. John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint 180. 08.80.0..226,.... 2018.
Hyeon Jeon, Jeongin Park, Sungbok Shin, and Jinwook Seo. Stop misusing t-sne and umap for visual analytics, 2025. URL htt.::ttpp://ttttae org/ abaaabaa//265
Tara Chari and Lior Pachter. The specious art of single-cell genomics. PLOS Computational Biology, 19(8);e 1288, 2023.
Mudit Verma, Siddhant Bhambri, and Subbarao Kambhampati. On the brittle foundations of react prompting for agentic large language models. ArXiv, 0es/24055.13666 2024. URL https
V/api.semanticschollaa wrwp:/Corr/ssss:::29930//3
Jia He, Mukund Rungta, David Koleczek, Arshdeep Sekhon, Franklin X Wang, and Sadid A. Hasan. Does prompt formatting have any impact on Ilm performance? ArXiv, abs/2411.10541, 2024. URL
https://api semanticschon
org/ htt://oopuI:::241331..
Max Vargas, Reilly Cannon, Andrew Engel. Anand D Sarwate, and Tony Chiang. Understanding generative ai content with embedding models. Science Advances, (to appear) 2025.
Yuxi Li, Yi Liu, Gelei Deng, Ying Zhang, Wenjia Song, Ling Shi, Kailong Wang, Yuekang Li, Yang Liu, and Haoyu Wang- Glitch tokens in large language models: Categorization tax- onomy and effective detection. Proc. ACM Softw. Fin.,, 1:2075-2097, 2024. URI. https
/aapi.sseanticcccoolar org/ www./orggg.a.eeeeooo I
Rick Battle and Teja Gollapudi. The unreasonable effectiveness of eccentric automatic prompts, 2024. 10949
BRLLhttss:// //atpi///ariv... org/
Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, and Subbarao Kambhampati. On the planning abilities of large language models- a critical investigation. ArXiv, abs/2305. 2023.
URL httpp::/tppppp semanticscholarr set.attttcccool...oooooooooe/oaa
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt format- ting. ArXiv, abs/2310.11322 2023. URL htttt::///aappp semanticscholar. wtw.chooaa...r//oosussllll. org/Corpus
262172710
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.0841,, 2023.
Alexander Jakubowski, Milica Gasic, and Marcus Zibrowius. Topology of word embeddings: Singularities reflect polysemy. In Iryna Gurevych, Marianna Apidianaki, and Manaal Faruqui, editors, Proceedings of the Ninth Joint Conference Lexical and Computational Semantics, pages 103-113, Barcelona, Spain (Online), December 2020. Association for Computational Linguistics. URL https::/aclaathhlllee
org 2020 sww.ssa-aaessaa 1.11
C. Berenfeld, J. Harvey, M. Hoffmann, and K. Shankar. Estimating the reach of manifold via its convexity defect function. Discrete and computational geometry, pages 403-438, 2022.
Eugene Isaacson and Herbert Bishop Keller. Analysis of numerical methods. Courier Corporation, 2012.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
12

---

Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics, 2024. htR.:ttps://arxi org/ abs/ 10631
Albert Q- Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Amin Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William El Sayed. Mistral7b, 2023. URLhttpp //axivv org/ abs/2310. 06825
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O' D'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models across training and scaling, 2023, URL
Minsu Park, Marco Gatti, and Bhuvnesh Jain. Dimensionality reduction techniques for statistical inference in cosmology. Physical Review D, 2024. pttppppppp//pppi.smamtiicooooooo org/ CorpusID:272367783
T. Lucas Makinen, Alan Heavens, Natalia Porqueres, Tom Charnock, Axel Lapel, and Benjamin D. Wandelt. Hybrid summary statistics: neural weak lensing inference beyond the power spec- trum. ArXiv, abs/2407. 18909, 2024. URL https://pp/ semanticscho.. wwwccholar.org///// Corg/CrrsssID
271516383
Lucas Einig. J' J'erome Pety Iram, Lerma, Antoine Roueff, Pauline Vandame, Jocelyn Chanussot, Maryvonne Gerin, Jan H. Orkisz, Pierre Palud, M. Santa-Maria, Victor de Souza Magalhaes, Ivana Bevsli',, Sébastien Bardeau, Emeric Bron, Pierre Chainais, Javier R. Goicoechea, Pierre Gratier, Viviana Guzman Veloso, Annie Hughes, Jouni Kainulainen, David Languignon, Rosine Lallement, Franccois Levrier, Dariusez C. Lis, Harvey S. Liszt, Jacques Le Bourlot, Franck Le Petit, Karin Danielsson Oberg, Nicolas Peretto, Evelyne Roueff, Albrecht Sievers, Pierre-Antoine Thouvenin, and Pascal Tremblin. Deep learning denoising by dimension reduction: Application to the orion-b line cubes. Astronomy & Astrophysics, 2023. (tthttts:://app ptttps::/apl.ssmmatiiccchhe semanticscholarr org/CorpusIIl 260154950
Naoki Masuda and Prosenjit Kundu. Dimension reduction of dynamical systems networks with leading and non-leading cigenvectors of adjacency matrices. Physical Review Research, 2022.
URL https: /api. semanticscholar wwwpcchholarrrr//ooooooo :0t.:/oo.......roroorooa2a222110 2477611661
Todd L.
Parsons and Tim Rogers. Dimension reduction via timescale separation in stochastic dynamical systems. arXiv: Probability, 2015. URL htttps://appi semanticscholar org/
CorpusID: 17056583
Bahram Hemmateenejad, Mohammad Safarpour, Ramin Miri, and Fariba Taghavi. Application of ab initio theory to qsar study of -e-rihhhddooyrddin---eeed calcium channel blockers using ga-mir and pe-ga-ann procedures. Journal of Computational Chemistry, 25, 2004. URL https //api.semanticschon
Drg/CorpusIDD:. 344448333
Elisenda Grigsby, Kathryn Lindsey, and David Rolnick. Hidden symmetries of ReLU networks. In Andreas Krause. Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 11744-17760. PMLR, 23-29 Jul 2023. URL
mlr. www.ppressvvvee v202/ griggriggson gsay23a...a..... htmll
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 2944):93-1109, 2016.
Uzu Lim, Harald Oberhauser, and Vidit Nanda. Hades: Fast singularity detection with local measure comparison, 2023. URI. htttp:://axxi..rrrrrssss httpg//abs22111.04111 04171
Julius Von Rohrscheidt and Bastian Rieck. Topological singularity detection at multiple scales. In
International Conference Machine Learning, pages 35175-35197. PMLR, 2023.
13

---

Michael Robinson, Sourya Dey, and Shauna Sweet. The structure of the token space for large language models, 2024. URL httttt////axxii org/aogg////.o.... ab////bs211..... 08993
Takuya Kataiwa, Cho Hakaze, and Tetsushi Ohki. Measuring intrinsic dimension of token embeddings. 2025.
2503 02142
Vasilii A. Gromov, Nikita S. Borodin, and Asel S. Yerbolova. A language and its dimensions: Intrinsic dimensions of language fractal structures. Complexity, 2024. URL https://doi. org 1tgg/10.15 1155/
2024/8863360
Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Serguei Barannikov, Irina Piontkovskaya, Sergey Nikolenko, and Evgeny Burnaev. Intrinsic dimension estimation for robust detection of Al-generated texts, 2023. Uttttttss:///aat wwtp://////bb////////// 04723
Alfred Gray. The volume of small geodesic ball of Riemannian manifold. Michigan Mathematical Jourmal, 20(4))329 344, 1974. doi: 10. (0.1307/mmi//0020011555 URL https:///oooooorr///
#507/mm///102900113
Michael Robinson. Testing for manifolds and stratified manifolds, 2025. URI httphhttps://gthhut
Kasimir Schulz, Kenneth Yeung, and Kieran Evans. Tokenbreak: Bypassing text classification models through token manipulation, 2025. http:://///rr
07948
Yu Cui, Yujun Cai, and Yiwei Wang. Token-efficient prompt injection attack: Provoking cessation in Ilm reasoning via adaptive token compression, 2025. URL htRL:ittpp://arxxi.orr/bbbbb/// 20493
Yulai Zhao, Haolin Liu, Dian Yu, Sunyuan Kung, Meijia Chen, Haitao Mi, and Dong Yu. One token to fool llm-as-a-judge. 2025a. URL hRtps:/::://att http://axxvvorgg///// abs/ 2507.08794
Omer tion
Zilberman.
Unicode
exploits
compromising applica-
security. 2025.
URL
https: / /wwww www.pppompp.co seromp..securttybble..
walccoeeeppoitttt are- core-comrromising-- application conppromssinn--ppppcccii-------- securityy
Toni JB Liu, Nicolas Boullé, Raphaël Sarfati, and Christopher J Earls. Llms learn governing principles of dynamical systems, revealing an in-context neural scaling law. arXiv preprint arXiv:2402.00795,
2024.
Pratik Prabhanjan Brahma, Dapeng Wu, and Yiyuan She. Why deep learning works: A manifold disentanglement perspective. IEEE transactions on neural networks and learning systems, 27(10): 1997-2008, 2015.
Hariharan Narayanan and Sanjoy Mitter. Sample complexity of testing the manifold hypothesis. Advances in neural information processing systems, 23, 2010.
Mengting Zhu, Huisha Yang, Yanlin Li, Rashad A. Abdel-Baky, Awatif AL-Jedani, and M. Khalifa Saadd, Directional developable surfaces and their singularities in euclidean 3-space. Filomat, 38(32):pp. 11133311134,, 2024. ISSN 03545180, 24060933, httLhttss:/// //wp:: jstor org/ sstable/73388000
Yuan Gu, JiaLe Han. HaoWen Wang, Xiang Li, and Bo Cheng. Explain-andi analyze-generatee: A sequential multi-agent collaboration method for complex reasoning. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend AI-Khalifa, Barbara Di Eugenio, and Steven Schockaert, editors, Proceedings of the 31 International Conference Computational Linguistics, pages 7127-7140, Abu Dhabi, UAE January 2025, Association for Computational Linguistics. URL https://aclanthology org/ /ttg/2025.coom c1155.coling c1125--ooinn 475/
Can Jin, Hongwu Peng, Qixin Zhang, Yujin Tang, Dimitris N. Metaxas, and Tong Che. Two heads are better than one: Test-time scaling of multi-agent collaborative reasoning, 2025. URL
https://arxiv.orm org
09772
Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Scaling large language model-based multi-agent collaboration, 2025. ht //wxxiv/.vvv///////////ee org/ wttp////..00..0 07155
14

---

Benjamin Schweinhart. Fractal dimension and the persistent homology of random geometric com- plexes. Advances in Mathematics, 372, 2020.
Elizaveta Levina and Peter J. Bickel. Maximum likelihood estimation of intrinsic dimension. In Proceedings of the 18th International Conference Neural Information Processing Systems, NIPS' 04,S'04,BB page 777-784, Cambridge, MA, USA, 2004. MIT Press.
Elena Facco, Maria d' Errico, Alex Rodriguez, and Alessandro Laio. Estimating the intrinsic dimen- sion of datasets by minimal neighborhood information. Sci Rep, 7(12140), 2017.
Andrew Lee, Mclanie Weber, Fernanda Vićgas, and Martin Wattenberg. Shared global and local abaaabbb////////////////
geometry of language model embeddings, 2025. URL https://///xx
2503. 2503.21073
Yize Zhao, Tina Behnia, Vala Vakilian, and Christos Thrampoulidis. Implicit geometry of next- token prediction: From language sparsity patterns to model representations, 2025b. URI. https
7/arxiv. hwtgg//bb/24408.... 15417
J. Lee. Smooth Manifolds. Springer, 2003.
Michael Robinson, Sourya Dey, and Taisa Kushner. Probing the topology of the space of tokens with structured prompts. Mathematics, 13(20):3320, Oct 2025.
Michael Rawson and Michael Robinson. Instability of manifold reach and a stable combinatorial reach. In Online International Conference Computational Harmonic Analysis, 2021.
Eddie Aamari and Clément Levrard. Nonasymptotic rates for manifold, tangent space and curvature estimation. Ann. Statist., 1771):177-204, 2019.
15

---

A Supplementary Material
A.1 Additional discussion of the literature
Although there a number of mathematically distinct definitions of dimension, they coincide for manifolds. Intrinsic dimension of point cloud has been widely studied in the literature SSchweinhart (020). All of the papers that address practical estimation of intrinsic dimension from data do so under the assumption that the data lic on manifold. For instance, [evina and Bickel [2004] provides maximum likelihood estimates, operating under the assumption that data "can be efficiently summarized in a space of much lower dimension, such as nonlinear manifold." Moreover, Facco
et al. [2017) provides a surprisingly effective approach using extremely small neighborhoods, a fact which underlies our use of our test "small radius neighborhood" in Section 5.2
Even though we ultimately reject the manifold hypothesis, the token subspace has been studied under the assumption that it is a manifold. For instance Lee et al. [2025] observe that,"Iaan "Language models often construct similar local representations as constructed from local linear embeddings." " We note that most of the tokens in a typical vocabulary fail to reject the manifold hypothesis, which means that without the deeper analysis we pursue, it is easy to overlook important differences. Furthermore, our Table 2 shows significant topological differences in the local representations between models.
we mentioned that Lee et al. 2025 show few things that are consistent with our findings. They note that "Token embeddings exhibit low intrinsic dimensions," which agrees with our in Table at least in the large radius neighborhood. Their estimates of dimension for GPT2 is within the contidence interval we computed. They do not recognize that there might be a fiber bundle structure in play, so Lee et al. [2025] does not estimate dimensions from a smaller radius neighborhood.
As introduced in Section I.2]
Relevant to Theorem 2 there is possibility that the implicit space of possible activations can be inferred from the behavior of the model Zhao et al. [2025b]. Since Theorem represents the context window as product of copies of the latent space, a projection of this implicit structure is the token subspace. In this vein. Zhao et al. 2025b) is not inconsistent with other findings within the literature, for instance Robinson et al. 2024]] This latter paper shows that the token subspace can be inferred (up to topological equivalence) from the responses of the LLM to single-token prompts. Taken together. Zhao et al. [2025b], Robinson al. [2024] and the present one have pretty significant implications. Even though the token subspace may have about 0. % of tokens that are singular, we suspect that a user will be encountering singular tokens on a regular basis when using L.MMs (which seems empirically plausible). Estimating the rate of occurrence of a singular token remains an open question.
A.2 LLM Background
At an abstract but precise level, an LLM consists of several interacting processes, outlined in Figure An LLM implements a transformation of a sequence of tokens (the query) into a new sequence of tokens (the response). Formally, if each input token is an element of a metric space T, then the LLM is transformation Tw - T where w is the number of tokens in the query and m is the number of tokens in the response. This transformation is typically not a function because is stochastic-it involves random draws.
To operate upon tokens using numerical models, such could be implemented using neural networks, we must transform the finite set of tokens T into numerical data. This is typically done by way of a pair of latent spaces X Re and Y 12, The dimension q of Y is chosen to be equal to the number of elements T, so that elements of Y have the interpretation of being (unnormalized) probability distributions over T.
The transformation T T is constructed in several stages.
Input tokenization : Each token is embedded individually via the token input embedding function e: T X. As whole, X is called latent window.
Transformer blocks The probability distribution for the next token is constructed by a continuous function f : X + Y. This is usually implemented by one more transformer blocks.
16

---

Query Response
Query token
T
T
T
Output tokens
sequence
Outside the LLM
random draws
token input
Y
Next token distributions
Inside the LLM
embedding
e"
X
X
X
Context windows
initial
transormer block(s)
context
window
Figure 4: Data flow in a typical LLM. sequence of tokens forming the query is converted via the token input embedding ex into the initial context window, point in the latent space X". Each of these windows the latent space are converted, token-by-token, into probability distributions via f into the single token latent space X. From these, each token presented in the output (in the set Y) is obtained via a random draw. These output tokens are then used for subsequent windows.
Output tokenization Given the output of one of the transformer blocks f. one can obtain an output token in T by random draw. Specifically, if (t/1, ...,, thw) is the current window in X", then the next token is drawn from the distribution given by f(11, ,
Next window prediction Given that token was drawn from the distribution, the next latent window itself is constructed by transformation F X - X which advances the window as follows:
F F(1/1 2/,, .2,,..,,,............... vw)) (12, (1/2...., ... ww: t).
Note well: the function e. 7 + coordinates in the latent space merely the first stage in the process. As such, e is distinct from the embedding of sequences provided by many LLM software interfaces. That sequence embedding is properly an intermediate stage within the function f.
X= which transforms a sequence of individual tokens into
The focus of this paper is specifically upon the structure of the token imput embedding e T > X = =R. Since the token set 7 is finite, can be stored corresponds to an element of T, and thereby ascribes a vector of numerical coordinates to each token. By replacing the last layer of the deep neural network f X - Y, vector of probabilities for the next token is obtained from the activations of the last layer. One can therefore interpret the probabilities as specifying token output embedding. Both the tokenization and the transformer stages are learned during training, and many strategies for this learning process are discussed extensively in the literature. These two stages interact when they produce the LLM output, it important to understand the lineage of given tokenization as being from particular LLM.
matrix. In this matrix, each column
A.3
Computational resource requirements
Applying our entire method (starting with the token embedding matrix and ending with the p-values for each of the three tests at every token) to each model took approximately 12 hours of wall clock time (per model) on an Intel Core i7-3820 with 32 GB of CPU RAM and no GPU running at 3.60GHz. Almost all of the runtime was spent computing the token pairwise distance matrix D in Algorithm I The distance matrix calculation was performed using scipy spatial distance. _iceee_a_tiii__ () which produces dense matrix, by computing the distance between all pairs of tokens in the latent space. Because the relatively low runtime requirements, we did not feel the need use sparsity- enhancing techniques to accelerate the distance matrix calculation. The subsequent stages, including the three hypothesis tests run in only a few minutes on the same machine.
A.4 Interpretation of fiber bundles in terms of spatial variability
It is usual to describe measurements exhibiting the combined effect of multiple spatial scales. One way this might arise we could express each measurement as being an ordered pair (signal, noise), with the short spatial scales correlating to noise and the longer spatial scales correlating to the signal.
17

---

With this interpretation, if the space of all possible signals is S and the space of all noise values is V. we could represent the space of all possible measurements as the cartesian product E = V. This cartesian product representation can arise for noiseless data as well, as it merely captures multiple spatial scales. what follows, we will call S the base space and V the fiber space.
log(area)
Base direction
Fiber direction
disk
entirely inside
slope as 00 Sharp corner
strip
R
R₂
Subtle comer
slope 2
disk peeks out both sides
log(r)
R1 R₂
Figure 5: Our method applied to a fiber bundle in RR2 The vertical direction is the base space (long spatial scale), while the horizontal direction represents the fibers space (short spatial scale). Gray points on the right frame show estimates from random sampling of points in the strip; the solid line shows the theoretical area versus radius curve.
Figure Jsswws an example of this situation. consists of a 1-dimensional base space (the large radius neighborhood) and 1-dimensional fibers (the small radius neighborhood), which in this case forms narrow strip in the plane. Volumes (areas, in this case) of balls of small radius scale quadratically (slope 2 in log-log plot), but scale asymptotically linearly (slope in a log-log plot) for large radii. The transition between these two behaviors is detectable by way a corner in the plot. This situation is easily and robustly estimated from the data; the gray points in Figure 5 (right) are derived from random sampling of points drawn from the strip.
To test for multiple spatial scales, we propose a signal model that is mathermaticlll represented by a fiber bundle, In a fiber bundle, the long spatial scales are modeled by a space S, but all possible measurements are modeled by function p : E + S. The idea is that fibers p (b) are still cartesian products: pairs of both spatial scales, and these are all identical up to smooth changes of coordinates. Our method relies upon particular geometric property of fiber bundles: can identify if the fibers are not all identical according to when the conclusion of Theorem T is violated.
One possible way that fiber bundle might arise is if the underlying noise model is homoscedastic, which is say that the dimension of the noise near token does not depend on that token. In this case, the fibers (small radii) mostly correspond to noise, and the base (large radii) correspond mostly to signal. Nevertheless, a fiber bundle may arise with noiseless data as well; the presence of multiple spatial scales is what matters.
On the other hand, if the underlying noise model is heteroscedastic-it depends on the token in question-then the fiber bundle model should be rejected. Figure shows situation that is not fiber bundle, since there is change in the dimension of the fiber. the left portion of the figure, the fiber dimension is 0 while in the right portion the fiber dimension is 1. This is detectable by looking the volume versus radius plots for two samples. While both samples show corners in their volume versus radius plots, Theorem I establishes that the slopes always decrease with increasing radius for a fiber bundle. This is violated for the sample marked th, so we conclude that the space is not a fiber bundle. On the other hand, because the sample marked 1/2 does not exhibit this violation, it is important to note that if a sample yields data consistent with Theorem we cannot conclude that the space is a fiber bundle.
18

---

A.5 Mathematical proof of Theorems 1 and
This section contains mathematical justification for the fiber bundle hypothesis proposed earlier in the paper and the proof of Theorems I and The central idea is the use of special kind of fiber bundle, namely a fibered manifold with boundary.
By the submersion theorem LLee 2003 I) if the Jacobian matrix of a fibered manifold p : S every point has rank equal to the dimension of S, then the preimages p (xb C T of each point & E T are all diffeomorphic to each other. These preimages form the fibers discussed in the earlier sections of the paper.
As a consequence, each point y in the base space S has an open neighborhood where the preimage p-1 (U) is diffeomorphic to the product U x xpp xp-(vv) which is precisely the base-fiber split discussed in Section A.4 Specifically, the base dimension is simply the dimension S, whereas the fiber dimension the (dim dim
The notion of a fibered manifold p- : : +
forms the intrinsic model of the data, which only implicit in an LLM. The tokens present in given LLM can be thought of as sample from a probability distribution on T, which can taken to be the Riemannian volume form on T normalized so that (1") = 1.
Definition 1. If e: T R is a continuous map and is a volume form on T, then the pushforward defined by
(C) = (v))
for each measurable set V.
It is a standard fact that if e a fibered manifold or an embedding, then e.v is also volume form.
Proof. (of Theorem 1 If e is assumed to be a smooth embedding the image of e is manifold of dimension dim T. The pushforward of volume form a contravariant functor, so this means that e, is the volume form for a Riemannian metric on e(T). Using this Riemannian metric on e(T"), then Gray 1974. Thm 3.1] implies that for every & E e(T'). if & T, then
(e.v) (B)) = O (ndimT)
Since is compact, S is also compact via the surjectivity of p. This implies that there a maximum radius 1 for which ball of this radius centered point on e(T) is entirely contained within e(T'). Also by compactness of S, there is minimum radius such that ball of radius r2 centered on a point v e(T') contains point outside of e(T').
Since e is assumed to an embedding, by the tubular neighborhood theorem (Lee be that 22 < Define
2003 must
p()) == argmax (B) S {((T))
from which it follows that <i S p( r2 < T. As result, Equation (3) holds for all p(v).
which is also the first case listed in Equation (2)
If r is chosen such that p(t)) < than what is given by Equation 3 namely
T, the volume of the ball centered on ip of radius r will be less
(B,
(B, < dim T).
Since is a volume form, its pushforward (p.v) onto S is also volume form. Moreover, via the surjectivity of p.
(e.r)(B,()) vle (le
(Be
$ O(
From this, the second case of Equation 2 follows by recentering the asymptotic series on p())
'Smoothness is not a strong constraint here. Any continuous function (such as ReLU) can be approximated well enough by smooth map.
19

---

Notice that the second case in Equation 22 may be precluded since while it holds for small r, it may be that p(() may not be sufficiently small. As consequence, the second case only occurs when both and p(t)) are sufficiently small. In the results shown in Section 5.2 both cases appear to hold frequently.
We now address Theorem 2 Informally, we cannot get rid of all stochasticity during training (due to finite precision, stochastic gradient descent, etc.). This ensures that the transformers in practice are always in general position, hence generic.
Given a context window of (w-th) token. This gets added to the context window and the process continues. (See Section [A.2 for details.)) In this section, provide mathematical argument showing that the use of context is insufficient to resolve singularities present the token subspace for a generic transformer in a persistent manner. Again, denote the token subspace by T, noting that Section 5.2 shows that ' is unlikely to be a manifold for typical LLMs. Let us begin with lemma:
tokens, LLM uses its learned parameters to predict the next
Lemma 1. If X a manifold, Y an arbitrary topological space, and x Y a manifold, then Y must also be manifold.
Proof. Observe that Y is homeomorphic (X Y)/X, which is the quotient of two manifolds via the projection map, which is surjective submersion. Hence Y is a manifold.
Consequently, the contrapositive of Lemma states that if T is not manifold but Tw-1 is manifold, it follows that 7", the space of sequences of tokens, cannot be a manifold. At best, the latent space for every other context window size can be manifold. For every proposed context window length, longer window will exhibit singularities. short, lengthening the context window does not persistently resolve singularities in the token embedding.
For instance, if use the embedding interface provided by the LLM to embed sequence of w tokens, we will obtain a vector in the latent space R On the other hand, if we embed each of the tokens individually yet sequentially, we obtain a vector in Rexw, The transformer uses the vector in I2 e to determine the vector in R'. Theorem 2 states that the latter (R" is crucial in understanding the output of the L.M.. (See Section A.2 further details.)
Proof. (of Theorem We need to manipulate the inequalities in the hypothesis to show that they satisfy [Robinson al. 2025 Thm. 1]. Once the hypothesis is satisfied, [Robinson et al. 2025 Thm. 1] establishes that a generic set of transformers will embed T into the output of the LLM. Notice that [Robinson et al. 2025 Thm. I] considers the response of the LLM to a single token, though this can be easily extended by enlarging the bounding manifold dimension. With this in mind, the hypothesis to be satisfied is that
2wd < ml < wé.
The right inequality is clearly satisfied if context window is longer than number of output tokens collected, namely w m. The left inequality is satisfied if
2d < m
w
which can be easily rearranged to the form in the hypothesis.
A.6 Case study of some tokens in GPT2
Figure 6 demonstrates our method on three tokens used by GPT2. The tokens $ and play an important role in many programming languages, whereas & does not. (Note that these tokens were chosen for illustrative purposes. The p-value for rejecting the hypotheses is larger than a 10-3 used in Section 5.2. so these particular tokens do not appear in any of the tables in the following sections.) The token does not show any rejections of our hypotheses. While this does not allow one to conclude that the vicinity of # is a manifold or fiber bundle, we can use Theorem I to estimate the dimension from the slope of the curve (approximately 53, once logarithms of the values on both axes are taken). The token & exhibits rejection of the manifold hypothesis, but not the fiber bundle hypothesis. We can conclude that there a irregularity at a distance of roughly 3.9 units of &. The curve for $ exhibits two slope changes at radii 3.8 and 4.0. The smaller radius slope change represents
20

---

Fail to reject
Token (GPT2)
#
c
1000
Reject manifold
volume (log scale)
Reject
fiber bundle
100-
(large radius)
Reject
manifold
radius (log scale)
(small radius)
Figure 6: Application of the tests to real data consisting of the tokens #, 4. and with GPT2's token input embedding. Notice that in the case of $, the first slope change decreases the slope; the fiber bundle test fails the slope increases.
violation of the manifold hypothesis, while the large radius slope change represents a violation of the fiber bundle hypothesis.
Given the presence of multiple slope changes, is necessary to select the parameters Umin and Umaz in Algorithm [ carefully. Ideally, aims capture at most slope change between Urain and Umaz- This is not specifically required by Algorithm [ since it uses a sliding window, multiple rejections may be detected.
A.7 Case study of a neighborhood of ember in Mistral7B
F and the application of this paper's ss manifold hypothesis test to neighborhood of the token ember in Mistral7B. Since the latent space of Mistral7B 4096, we have projected the coordinates of each token shown to 3 using PCA. The inset shows all of the tokens in Mistral7B (rendered with t-SNE), with erosshairs showing the location of the zoomed-in region. Each point shown in Figure is a token used by Mistral7B, colored based upon the p-value for the manifold hypothesis test. Darker colors correspond to lower p-values, which are less consistent with the hypothesis that the token's 's neighborhood is manifold with low curvature.
Given that the we have established that manifold rejections occur in Section we can attempt to understand possible causes. Intuitively, several non-manifold features are visibly presen in Figure 7 The tokens "ember" and "p" appear to be both putative examples cusp points, places where the space exhibits a sharp exposed point. Manifolds cannot have cusp points, so attempting to approximate a cusp with a manifold-for instance, with neee-nncessttates high curvature. The tokens ":" and "ly" appear to be pinch points-like what are present in chain of Lausages-whie again cannot be approximated by a manifold unless has high curvature.
A.8
Table of fiber bundle rejections in all four LLMs
See Table 4
A.9
Tables of manifold rejections in all four LLMs
See Tables 5 10
"The coordinates, token text, p-vuuues, and a video of the space with commentary are available in the ZIP file Supplement. Due to size requirements for the coordinates of the entire vocabularies for all four LLMs, we will upload the coordinates public repository.
21

---

Pinch point
Pinch point
Sharp cusp point ember
1y
PC3
Sharp cusp point
P
ISNE -204
Sueb dass
Zoomed in region
ISNE cound
t-NN plot of all tokens in Mistral 7B
Figure 7: This figure shows the application of this paper's manifold hypothesis test to neighborhood of the token ember in Mistral7B. (The fiber bundle tests are not shown due to space considerations.) Since the latent space of Mistral7B has dimension & = 4096, we have projected the coordinates of each token shown to using PCA. The inset shows all of the tokens Mistral7B (rendered with t-SNE), with crosshairs showing the location of the zoomed-in region. Each point shown in this figure is a token used by Mistral7B, colored based upon the p-value for the manifold hypothesis test. Darker colors correspond to lower p- values, which are less consistent with the hypothesis that the token's ' neighborhood is a manifold with low curvature. Putative geometric structures are labeled for certain tokens.
A.10 Limitations of theory
Because our approach requires the distances between all pairs of tokens, to use the hypothesis tests presented in this paper, one needs have access to the token embedding vectors. This why our paper considers four open source models of moderate size GPT2 (Radford et al.) (2019)). Llemma7B (Azerbayev et al. (2024)) ), Mistral7B (Jiang et al. (2023) ). and Pythia6.99 (Biderman et al.. (2023). While Robinson et al. [2025] does provide an approach to extract the token embeddings using systematic prompting, which would allow the application of our tests to proprietary models, their method requires exquisite control of the prompting and is computationally expensive.
The reach itself is difficult estimate accurately from sampled data (Berenfeld al. [2022] Rawson and Robinson [2021), Aamari and Levrard (2019). Our implementation tests the radius below an estimated reach, determined by an inspection the tokens. If the true reach is smaller than our estimates, the curvature of the manifolds that fit the data well, found by any method, will be high.
plots of random sample of
22

---

Figure 8: Additional rotational views of the PCA of the neighborhood of ember in Mistral7B shown in Figure Each point shown is a token. Colors are p-values for the manifold hypothesis, with darker colors corresponding to lower likelihood that that point's 's neighborhood is manifold with low curvature.
23

---

Table Violations the fiber bundle hypothesis
Model
Token ID
Token
Slope change
p-value
Comment
GPT2
47623
laughable nuance dt
Smaller radius
9 10
Must start word
GPT2
47128
Smaller radius
2 x 10-4
Must start word
GPT2
28664
Smaller radius
2 X 10
GPT2
37031
Mesh affect
Smaller radius
2 10
GPT2
2689
Smaller radius
3 X 10
Must start word
GPT2
48387
Thankfully
Smaller radius
3 x 10
GPT2
44284
swat
Smaller radius
6 10 10m4
Must start word
GPT2
30031
Malaysian Palestinian
Smaller radius
6 X 10
Must start word
GPT2
8501
Smaller radius
7 x 10-1
Must start a word
GPT2
7864
wins
Smaller radius
8 10
Must start word
GPT2
46086
hedon
Smaller radius
9 10
GPT2
17052
donor
Smaller radius
9 x 10
Must start a word
GPT2
47482
Xan
Larger radius
3 x 10
Must start a word
GPT2
21118
aunder
Larger radius
2 2 10
GPT2
20564
Dri
Larger radius
2 X 10
GPT2
1681
ney
Larger radius
3 X 10
GPT2
2076
rodu
Larger radius
3 10
GPT2
44402
Insert
I Larger radius
4 x 10-1
GPT2
38833
Ying
Larger radius
4 x
Must start word
Llemma7B
16352
pax
Larger radius
3 10
Mistral7B
25219
änge
Smaller radius
8 8 10
Mistral7B
9009
monitor
Larger radius
8 X 10
Must start word
Mistral7B
4104
HO
Larger radius
5 X 10-4
24

---

Table 5: Rejections of the manifold hypothesis for GPT2 (1 of 2)
Number
Token ID
Token
p-value
Comment
47482
Xan
0.0000000020
Must start word
28899
chemist
0,0000000602
3
32959
fry
0.00000564
Must start word
4
47623
laughable 0.00000807
Must start word
29884
Retro
0,0000108
Must start word
48387
Thankfully
0,0000339
20619
intoler
0.0000514
Must start word
8
18533
assumes
0.0000533
Must start word
45916
handwritten
0.0000783
Must start word
10
17615
MY 0.00008 1
Must start word
47128
nuance
0.000105
Must start word
12
24817
playground
0.0001 10
Must start word
28664
dt
0.000124
14
21118
aunder
0.000132
37031
Mesh
0.000135
16
37338
Disney
0.000139
23780
McCI
0.000155
Must start a word
18
20564
Dri
0.000196
19
17886
Destiny
0.000206
Must start word
20
1681
ney
0.000206
21
21904
juvenile
0.000209
Must start a word
22
19980
advent
0.00021 1
Must start word
23
8939
perpet
0.000213
Must start word
24
38908
otally
0.000223
25
2689
affect
0.000226
Must start word
26
16814
efficient
0.000238
27
2076
rodu
0.000242
28
14770
Kl
0.000260
Must start word
29
2359
duct
0.000284
41890
frail
0.000292
Must start word
31
28654
alle
0.000304
Must start word
20703
Cha
0.000322
Must start word
33
6684
00
0.000333
25

---

Table 6: Rejections of the manifold hypothesis for GPT2 (2 of 2)
Number
Token
Token
p-value
Comment
34
44402
Insert
0.000336
35
38833
Ying
0.000357
Must start a word
36
41072
Georgia
0.000373
37
48679
flo
0.000384
38
32745
uddled 0.000413
39
6399
subsequ
0.000421
Must start a word
40
26106
lig 0.000439
Must start word
41
13272
Environmental
0.000456
Must start word
42
14840
Vladimir
0.000466
Must start word
43
16721
Defence
0.000510
Must start word
44
44284
swat
0.000558
Must start a word
45
28336
registering
0.000564
Must start word
46
30031
Malaysian
0.000569
Must start a word
47
45047
trough
0.000575
Must start word
48
11368
ears
0.000615
Must start a word
49
37070
Meh 0.000637
Must start word
50
8501
Palestinian
0.000648
Must start a word
51
22053
reckless
0.000654
Must start word
52
42656
Wax 0.000658
Must start a word
53
4307
Dist
0.000673
Must start word
54
20648
Quad
0.000725
Must start a word
55
39468
relations
0.000749
56
7864
wins
0.000781
Must start a word
57
42839
KER
0.000785
58
50057
gist
0.000786
Must start a word
59
46086
hedon
0.000834
60
19055
MMA
0.000839
Must start a word
17052
donor
0.000865
Must start word
62
32100
aleb 0.000873
18932
Asked 0.000873
64
29756
Isaiah
0.000940
Must start word
65
13084
Survey
0.000947
Must start a word
66
40944
youngster
0.000987
Must start word
26

---

Table 7: Rejections the manifold hypothesis for Llemma7B
Number
Token ID
Token
p-value
Comment
24812
clojure
0.00000000434
10617
agu
0.000000846
24391
0.000000938
Cyrillic; Must start word
6341
custom
0,00000163
26226
kW
0.00000496
Must word
19579
|[|
0,00000833
17485
porque
0.0000128
Must start word
23146
Leopold
0.0000135
Must start a word
5240
Rem
0.0000232
Must start word
10
9317
Ern
0.0000265
Must start word
2466
though č
0.0000540
Must start
12
3413
0.0000619
Must start word
13
18659
endl corrected
0.0000683
Must start a word
14
24114
0.0000738
Must start word
3252
tw 0.0000788
Must start a word
16
5908
ères
0.0001 15
17
22385
grep
0.000151
18
2924
kind
0.000202
Must start word
19
16352
pax
0.000243
20
8602
Tri
0.000250
Must start word Must start a word Must start word d with underline
21
18681
spielte
0.000258
22
22977
Heaven
0.000262
23
31946
0.000289
24
7949
0.000320
Cyrillic
25
9160
Card
0.000335
Must start a word
26
14406
Wall
0.000351
Must start word
27
29591
periodic
0,000359
Must start word
28
10236
ucht
0.000463
29
18308
jeu
0,000553
Must start word
10029
slightly
0.000807
Must start word
31
15935
passion
0.000823
Must start word
5322
Charles
0.000833
Must start word
33
15354
victory 0.000906
Must start word
27

---

Table 8: Rejections of the manifold hypothesis for Mistral7B
Number
Token ID
Token
p-value
Comment
20639
CV ember
0.000000241
Must start a word
2
1314
0.00000199
3
31684
0.00000224
Korean
4
31567
i
0.00000770
5
8418
rise
0.0000283
Must start word
6
20508
poured
0.0000305
Must start a word
7
17939
aimed
0.0000364
Must start word
8
21637
allocate
0.0000553
Must start a word
9
13215
UINT
0.0000684
10
9009
monitor
0.0000767
Must start a word
11
25867
0.000105
Cyrillic
12
30314
\u0006
0.000132
13
31517
0.000133
Chinese
14
7402
Lat
0.000139
Must start a word
15
24351
Vel
0.000148
16
10369
challenges
0.000178
Must start a word
17
30085
0.000200
Chinese
18
11424
Od
0.000222
Must start word
19
26983
neys
0.000244
20
23851
selon
0.000244
Must start a word
21
10582
measures
0.000268
Must start a word
22
27460
assumes
0,000321
Must start word
23
13542
Et
0.000338
Must start a word
24
13722
Rot
0.000438
25
31097
0.000538
Chinese
26
25244
olutely
0.000540
27
4104
HO
0.000567
28
24806
Solutions
0,000598
Must start word
29
29587
0.000658
30
12862
Condition
0.000665
31
3903
zt
0.000677
32
3903
iced
0.000696
33
5343
URL
0.000722
34
21518
SEO
0.000728
Must start a word
35
19929
weigh
0.000728
Must start a word
36
19929
änge
0.000765
37
19929
Vic
0.000777
Must start word
38
7566
apparent
0.000780
Must start a word Chinese
39
31780
0.000896
40
230
<0xE3>
0.000921
28

---

Table 9; Rejections of the manifold hypothesis for Pythia6.9B (1 of
Number
Token ID
Token
p-value
Comment
49503
psychotic WJ(
0,00000016 161
Must start a word
2
49503
0.00000147
3
13435
1990
0,00000231
31918
tiene
0.00000282
Must start word
5
12202
ucky
0.0000134
25260
Hebrew
0.0000141
Must word
7
46631
mess 0.0000184
8
5272
reasonable
0.0000300
Must start word
9
25389
collector
0.0000311
Must start word
24898
embodiments
0.0000428
Must start a word
11
47769
Inhibition
0,0000593
Must start word
12
41264
Hers
0.0000691
Must start a word
13
13676
odge
0.000120
49669
0.000144
Turkish
15
16287
0.000165
Turkish
16
26397
198
0.000170
Must start a word
17
30272
scare
0.000192
Must start word
18
41305
0.000202
19
25666
LIMITED
0.000212
Must start word
20
5184
hop
0.000232
Must start a word
21
39253
BIO
0.000246
22
48089
raison
0.000252
Must start a word
23
40084
inside
0.000254
24
296
st
0.000261
25
34475
waving
0.000286
Must word
26
21753
Austria
0.000287
Must start a word
27
5134
ny
0.000313
29

---

Table 10: Rejections of the manifold hypothesis for Pythia6.9B (2 of
Number
Token ID
Token
p-value
Comment
28
46728
bags
0.000326
29
28672
medium
0.000372
30
15210
incent
0.000378
Must start a word
31
21742
carboh
0.000407
Must start word
32
4279
secret
0.000451
Must start word misc Unicode
33
50231
0.000468
34
40146
0.000519
Turkish, Must start word
35
8193
120
0.000536
36
47334
wastes
0.000565
Must start word
37
32232
ressor
0.000570
38
22609
Lip
0.000597
Must start word
39
42150
preview
0.000650
40
19886
redund
0. 0.000653
Must start word
41
41962
ridic
0.000673
Must start a word
42
48444
Ramirez
0.000698
Must start word
43
8977
omatic
0.000707
44
46972
Gloria 0.000733
Must start word
45
18807
eton
0.000780
46
16782
chairman
0.000789
Must start word
47
6413
Supreme
0.000859
Must start a word
48
42351
grazing
0.000890
Must start word
49
5743
activation
0.000892
Must start a word
50
34282
imming
0.000914
51
46513
radiological
0.000931
Must start word
52
11363
branc
0.000941
Must start word
53
47183
SWN
0.000942
Must start word
54
25409
DEFAULT
0.000992
30

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims in the abstract and introduction are about applying rigorous statistical tests to verify the structure of the token subspace. We present three such tests in detail, prove that they are correct mathematically, and then demonstrate them on four LLMs. We obtain and report p-values for all such tests.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We include detailed Limitations section of the paper, and further discussion in the Supplement.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
31

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The paper contains detailed mathematical proofs that include all assumptions, as well as citations into literature for supporting results. Statements the theorems and assumptions appear in the main body of the paper, while the proofs are in the Supplementary portion.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Although the actual code and data involved are not included, both due size and limitations applied by one of our sponsors, we have given full details of the statistical tests. Furthermore, in the Supplementary data, provide enough information that Figure can be examined in full by the reader.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
32

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Source code is available at http:://ggchubb.cc///ddddd///r/e which mentioned in the main text of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis..ca
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ctipss.c/puullc//uuii
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: However, note that we are simply using standard pre-trained LLM embeddings, with specific citations to the original papers in which they were announced.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: ABSOLUTELY! This is essentially a statistical paper, and we report all relevant valuue and statistical test details.
Guidelines:
33

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative етror rates).
If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: This is noted in the supplementary. But basically standard laptop (not necessarily equipped with GPU acceleration) suffices for everything.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://euurpp:: httpp//ppp..pppppp.ia.ccccc.caaa
Answer: [Yes]
Justification:
We believe that we are compliance.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
34

---

To the extent that we can, our Introduction and Discussion sections cover this matter. However, we note that one way read our paper is calling out others for failing to do this important work by highlighting gap in the literature.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
We are analyzing existing, pre-trained models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Since we are only using open source pre-trained models, we have cited the original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
35

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thcode com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets are released.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Not applicable.
Guidelines:
36

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
This paper is about LLMs but did not use them as a tool.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM for what should or should not be described.
37


Reviews:
[{"review_id": "cmAoBYFzKL", "reviewer": "Reviewer_MkzF", "summary": "Motivated by the widespread but implicit assumption that token embeddings in large language models (LLMs) lie on a smooth manifold, this paper investigates whether that assumption holds in practice. It introduces a statistical testing framework to detect geometric irregularities in the embedding space, based on the notion of local neighborhoods deviating from smooth fiber bundle structures.\n\nThe main methodological contribution is a local hypothesis testing procedure that assesses whether individual token neighborhoods exhibit manifold-like behavior. The authors demonstrate that many tokens reside in locally irregular regions, that is, their neighborhoods violate the smoothness and flatness assumptions central to the manifold hypothesis.\n\nThe authors show that these irregularities are not mitigated by context window: even when used within context, such tokens introduce instability in model behavior. As a result, semantically similar queries may yield significantly different outputs depending on subtle syntactic differences, with potentially serious implications for robustness and interpretability in LLMs.\n\nThe authors test their methodological/theoretical approach on four open-source models: GPT‑2, Llemma7B, Mistral‑7B, and Pythia‑6.9B.", "questions": "# Major Questions/Comments\n\nI will consider improving the clarity score and overall score if the following points are addressed:\n\n---\n\n## 1. Structure\n\n**a)** Section 1.1: Please enumerate the contributions (statistical test, context, propagation, etc.) with reference to the specific parts of the text.  \n\n**b)** Please consider restructuring Section 2. As mentioned in the \"Weaknesses\" section, it is a bit difficult to start discussing the methods and theorems, which are said to be explained in the next section, even though they have only been briefly introduced by that point (e.g., in Figure 2), and lack accompanying explanation. Theorem 1 is referenced but not yet formulated until the following section. One suggestion is to merge Sections 2 and 3.1, but this is not the only possibility.  \n\n**c)** Please consider adding a subsection after line 184, as this is where you begin discussing singularity propagation through the transformer.\n\n---\n\n## 2. Mathematical formalism\n\nI am aware that it is a non-trivial task to align a notion of a *“minimal but sufficient set of definitions”* with the expectations of the reader. That said, I have listed several terms that I believe are important to define explicitly. Terms such as *embedded reach*, *curvature*, *volume form*, *radius*, *pushforward*, *homeomorphism*, and *diffeomorphism* are used without explanation. \n\nAdditionally, it would be helpful to formally define *cusp point* and *pinch point* (unless they are supposed to be used informally), in which case I would appreciate a brief explanation.\n\n---\n\n## 3. Hypothesis test formulation\n\nWhile the testing framework is described, clarity would be improved by explicitly stating the hypotheses in standard form:\n\n- Null hypothesis (H₀): …  \n- Alternative hypothesis (H₁): …\n\nIdeally, this should be added at the end of Section 3.1, just before the discussion on limitations of the contextual window to solve irregularity. It would also be beneficial to clearly formulate the computed statistics (you may use additional algorithmic clauses if needed). The main point is to provide a clear and concise test description.\n\n---\n\n## 4. Visual illustration design\n\nSchematic diagrams or 3D (compared to 2D) illustrations of manifolds and fiber bundles (with small and large radii) could significantly enhance material accessibility and clarity. Within the same figure, you could also visually explain concepts such as *base space*, *radius*, *volume*, *cusp point*, *pinch point*, etc. \n\nThis would help readers build intuition, which can be leveraged when interpreting Figure 1. Additionally, this visual could potentially replace Figure 3a (left). While I acknowledge the authors’ effort in the current manuscript, I personally found the visuals somewhat difficult to interpret and believe they reduce overall clarity. \n\n---\n\n## 5. Results interpretation\n\nSome results and conclusions would benefit from more detailed explanations and clarification:\n\n**a)** Line 236: How were these general trends obtained? Were they based on manual inspection? Please elaborate and consider updating the manuscript.  \n\n**b)** Lines 200–306: Similarly, how were these results derived? Were they also from manual inspection? Please clarify and consider updating the manuscript.\n\n---\n\n## 6. Methodological clarity\n\nThe authors introduce a new test that has no existing history of successful application in prior work and apply it directly to real-world data (i.e., the embedding space of open-source LLMs), effectively without rigorous control over the data.\n\nWould it be better to include a small experimental section using synthetic manifolds or fiber bundles, where the underlying structure is known by controlled generation? This would allow you to validate the test in a controlled setting and make your claims more convincing. I would like authors to consider incorporating such experiments, or to know their opinion on why they might disagree.\n\n\n\n# Minor Questions/Comments\n\nQuestions/comments that are minor compared to the previous set, but I encourage the authors to consider them as they can further improve the quality of the paper:\n\n1. Line 12: “Failure to reject null hypothesis…” is a repetition of a very similar phrase above.\n\n2. Line 29: I wonder if this statement can be supported by prior work. While it intuitively seems to be true, it would be interesting to know if anyone has studied and quantified it.\n\n3. Line 59: Define the neighborhood  $B(\\psi)$.\n\n4. Line 71: Should it be “global token embedding space” instead of “global token embedding”? Otherwise, please ensure that the term *embedding* is not used interchangeably to mean a single instance, an embedding function, or an embedding space.\n\n5. Figure 1: Is there any particular reason why t-SNE is used for the large-scale view and PCA for the fine-grained structure? Why not build a three-dimensional embedding with either t-SNE or PCA and then zoom in? I could not find clarification.\n\n6. Line 99: “In our test case, the basic assumption…” — I agree that LLMs operate in a token subspace rather than the full latent space. However, what *test case* is referred to, and why is this assumption made? I believe this is just perhaps inaccurate phrasing, so please resolve the confusion.\n\n7. Figure 1 reference: The first reference to Figure 1 appears in the text at Line 215 (three pages later). Consider either referencing it earlier or moving the figure closer to where it is discussed.\n\n8. Figure 2: Why does the pipeline at the top not differentiate between small and large radii?\n\n9. Figure 3b: Should the label be “Reject fiber bundle (small radius)” instead of “Reject manifold (small radius)” for the first slope change in the green curve?\n\n10. Figure 3b: The blue curve appears to more closely resemble the top image in Figure 3a, which would suggest rejecting the fiber bundle as the slope increases. Why is that? Please make the figures coherent or provide an explanation.\n\n11. Figure 3a (left side): This part of the figure is a bit counterintuitive. Please consider improving the visualization (e.g., as suggested in pt. 4. Visual Illustration design in Major Comments section above) to better show the fiber bundle and connect it to the slope change dynamics in volume/radius. Additionally, even in this 2D depiction, $r_3$ seems off — it's unclear what is being represented.\n\n12. Line 142: There is ambiguity in the notation/terminology. Is $T$ the embedding or the embedding subspace?\n\n13. Line 145: Am I correct that $\\psi$ is used solely as an element of $\\mathbb{R}^l$, and not in relation to $T$? Otherwise, please resolve the confusion.\n\n14. Theorem 1: There is a bracket inconsistency in the LHS (induced volume of $B_r(\\psi)$).\n\n15. Notation $T^w$: Please briefly introduce this notation. You use $T^{w-1}$ first, and only later define it as a sequence of tokens.\n\n16. Line 201: Please check whether the reference is correct. Section 4.1 is the “Limitations” section, which seems like a mistake.\n\n17. Line 218: What does “chain of sausages” mean?\n\n18. Lines 221–224: Consider moving this part to a footnote.\n\n19. Table 1: Could you please elaborate on why all quartiles for LLaMA-7B are exactly 4096? Is there an explanation for these results?\n\n20. Line 257: It is unclear what is meant by “dependency between large and small scale variability.”\n\n21. Line 270: It would be helpful to have a formal definition of *singularity* if this is a specific type of irregularity. In Line 51, it is stated that the token subspace in LLMs is “singular,” meaning not a manifold. However, the term “singular” and its variants are later used without clarification.\n\n22. Line 323: Would it be possible to at least list possible *post hoc methods* and discuss the implications of determining that?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths** \n\nFrom my perspective, the paper has the following strengths:\n\n- **Significance:** The paper presents a well-defined motivation, namely, irregularities in the embedding space can lead to unstable model responses, which is undesirable. This justifies the need for statistical testing to identify tokens that lead to such unstable responses, especially given the current scale of LLM usage.\n\n- **Originality:** To the best of my knowledge, the paper offers an original perspective on the problem (subject to the presence of prior work and the distinctions discussed by the authors in Section 1.2). In particular, I appreciate the authors’ effort to emphasize the distinction of their approach from dimensionality reduction techniques, which could otherwise raise questions about overlap.\n\n- **Soundness:** The authors take a reasonable and systematic approach by examining the structural properties of the embedding space in practice. The experiments are logically structured, intuitive, and well-designed to support the methodological results of the paper.\n\n**Weaknesses**\n\nBelow, I provide a high-level summary of weaknesses. Please refer to the “Question” section for a detailed list of comments, where I distinguish between major and minor questions/comments.\n\nWhile somewhat subjective, I found that the paper occasionally **lacks clarity**, largely due to its narrative structure and presentation style:\n\n- **Mathematical formalism**:  I appreciate the formal approach taken in the theoretical sections. However, the paper would benefit from more consistent and clearly defined mathematical terminology. Some readers may not have sufficient background in topology, differential geometry, or related areas. Given space constraints, some definitions could be moved to an appendix with appropriate references from the main text.\n\n- **Structure**:  The structure of the paper is somewhat convoluted. There are abrupt transitions and forward references without sufficient contextual introduction, which makes the logical flow harder to follow.\n\n- **Hypothesis test formulation**:  Although the main methodological contribution is a statistical test, it is not clearly or explicitly formulated. A standard hypothesis testing format, when authors are stating the null hypothesis (H₀) and alternative hypothesis (H₁), would significantly improve clarity.\n\n- **Visual representation**:  While I appreciate the theoretical focus, the paper would benefit from additional and/or clearer/more intuitive visualizations. For instance, schematic or 3D representations of manifolds and fiber bundles (with both small and large radii) could enhance understanding. The existing figures are a step in this direction, but they remain difficult to interpret.\n\n- **Results interpretation**:  Some of the results and conclusions could be explained in greater detail. Certain claims would benefit from additional justification or clarification.\n\nI would like to emphasize that these weaknesses do not indicate any fundamental flaws. It is evident that the authors have already considered many of these points in preparing the submission. Please find my precise questions/comments/requests (both major and minor) in the section “Questions.”", "was_revised": true, "final_justification": "The authors diligently addressed all of my concerns during both the main rebuttal and discussion phases. I am satisfied with their responses and have accordingly increased my clarity score and overall rating.", "rebuttal": "Thank you very much for the extremely detailed review!  We should be able to implement each of your suggestions without any trouble at all, as detailed below.\n\nWe agree that the exposition could have been more organized.  We were struggling with the page limit and hastily moved some of the explanatory material to the supplement.\n\nNB: Action items for the authors are marked with a \"+\" in the margin!\n\n+We will edit Sec 1.1 as follows:\n\nManifold test: A 2-sided hypothesis test where the null is defined as\nH_0: The token $\\psi$ is sampled from a manifold embedded within $\\mathbb{R}^\\ell$ with reach $\\tau$.\nand the alternative is defined as\nH_1: The token $\\psi$ is not sampled from such a manifold (ie. we reject the null if there is any change in the slope of the log volume v. log radius curve)\n\nTwo runs for the fiber bundle test: a 1-sided test for the following hypothesis:\nH0': The token $\\psi$ is sampled from a fibered manifold with boundary that is embedded within $\\mathbb{R}^\\ell$ with reach $\\tau$.\nH1': The tokens are not such a sample (ie. we reject the null if there is a slope INCREASE in the slope of the log volume v. log radius curve)\nWe performed the volume v. radius estimation twice: once for larger radii, and once for smaller radii.\n\nFor almost all the tokens, we can estimate a dimension.  But for the remaining tokens--those that cause the hypotheses above to be violated--the dimension at that token is not well-defined at all!\n\nYour request for a visual schematic of what our test does is a great idea!  Various schematic figures were included in an earlier draft of the paper, where we had included \"toy\" examples of our test being run on \"noisy\" manifolds and stratified manifolds. However, we had to cut these examples out of the submission due to space constraints.\n\n+we will include these examples in our final version!\n\nFor the purpose of visualization, think of a \"lollipop\".  This shape has exactly two important singular points.  On the interior of the stick, the space is a 1-dimensional manifold.  At the very end of the stick, though, is a singular point (the boundary).  Where the stick attaches to the \"candy\" part is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy part.  This visual example was not too difficult to construct, so your suggestion of including pictures showing cusp points, pinch points, etc. is easily implemented.\n\nResponses to specific points that you raise in the Major Questions/Comments section of your review:\n\n> Please enumerate the contributions ... with reference to the specific parts of the text.\n\n+Sure!\n\n> Please consider restructuring Section 2...One suggestion is to merge Sections 2 and 3.1...\n\n+Section 2 indeed needs some help, but your suggestions (along with those of the other reviewers) are very do-able.  We will also move material from A.3 and A.4 into Section 2 as well for completeness.\n\n> Please consider adding a subsection after line 184, as this is where you begin discussing singularity propagation through the transformer.\n\n+Good idea!\n\n> I am aware that it is a non-trivial task to align a notion of a “minimal but sufficient set of definitions” with the expectations of the reader.\n\n+The things that absolutely need to be defined (most in Section 1.2) are:\n\n* manifold\n* singularity (and separately \"singular\")\n* embedding\n* embedded reach\n* volume form\n* radius\n* fibered manifold\n* fiber bundle\n* p-value\n* curvature\n* pushforward\n* homeomorphism\n* diffeomorphism\n\n> Additionally, it would be helpful to formally define cusp point and pinch point (unless they are supposed to be used informally), in which case I would appreciate a brief explanation.\n\nWe meant these informally, and in that sense, they're best understood visually.  The \"toy examples\" mentioned above can do this easily. \n\n+We'll mention at least the following in our revision.\n\n1. A \"cusp point\" is something that looks like the graph of $y^3 + x^2 = 0$ at the origin.  \n2. A \"pinch point\" looks like the graph of $z^{2}+x^{2}-y^{2}=0$  (check this out on any 3d graphing software!).\n\n> Re: Fig 3: I personally found the visuals somewhat difficult to interpret and believe they reduce overall clarity.\n\n+With the \"lollipop example\" added, we will alter Fig 3(a) so that it is based on that example.  It will be considerably clearer\n\n> Line 236: How were these general trends obtained?\n\nThey were obtained from a manual inspection of Tables 4-9, which are present in the Supplementary Material due to space constraints.\n\n> Lines 200–306: Similarly, how were these results derived? Were they also from manual inspection?\n\nLines 200-206: This follows simply the fact that Tables 2-9 are nonempty; rejections were obtained.  \n+We will mention this!\n\nLines 207-214: Inspection of Table 1.  The fact that the rejection rates and the quartiles for the dimensions are non-overlapping.\n+We will mention this!\n\nLines 215-224: Manual inspection.  This should be taken as suggestive.  Given that the previous paragraphs established that manifold rejections occur, we are pivoting to understand possible causes.\n+We will explain that this is less about drawing conclusions about the specific reasons and more about developing the hypotheses/intuition for future study.\n\n> Would it be better to include a small experimental section using synthetic manifolds or fiber bundles, where the underlying structure is known by controlled generation?\n\n+We did run such experiments already and can include them in the revision!  Specifically, we've run spheres, planes and tori (of various dimensions), and a handful of stratified spaces/fiber bundles.\n\nWe can include a toy stratified space to elucidate singular v regular points in the paper and put the rest of the examples into the supplement (and we will include all of the code publicly).\n\n> Line 29: I wonder if this statement can be supported by prior work. \n\nWe haven't found anything that says this directly, but here are a few examples that are mathematically assuming manifolds:\n1. Inner products\n2. Cosine Similarity (which is just inner products on n-spheres which has even more assumptions about curvature especially negative curvature)\n3. Manifold learning (Any sort of dimensionality reduction, etc)\n\nIn reality, the structure of LLMs (token in -> token out) implies that the token space itself is closed under these operations.\n\n> Line 59: Define the neighborhood \n\n+This is a subtle typo.  It should be \"whether *any* neighborhood of the token contains an irregularity\"\n\n> Line 71: Should it be “global token embedding space” instead of “global token embedding”?\n\nIf we say anything other than \"embedding function\", it's a typo!  In this case, we mean the \"global embedding function\".\n+We will check closely!\n\n> Figure 1: Is there any particular reason why t-SNE is used for the large-scale view and PCA for the fine-grained structure?\n\nWe tried various \"looks\" and this was the clearest.\n\n> Line 99:\n\n+Oops.  \"In our test case\" should be deleted.\n\n> The first reference to Figure 1 appears in the text at Line 215...\n\n+Yup. Thanks.\n\n> Figure 2: Why does the pipeline at the top not differentiate between small and large radii?\n\n+We just run the process twice.  We'll edit Fig 2 accordingly.\n\n> Figure 3b: Should the label be “Reject fiber bundle (small radius)” instead of “Reject manifold (small radius)” for the first slope change in the green curve?\n\nNo.  The first slope change decreases the slope; the fiber bundle test fails if the slope *increases*.\n\n> Figure 3b: The blue curve appears to more closely resemble the top image in Figure 3a, which would suggest rejecting the fiber bundle as the slope increases. Why is that?\n\nNo. The top curve on Fig 3(a) has a slope increase through the slope discontinuity, whereas the blue curve in Fig 3(b) has a slope decrease.\n\n> Line 142: There is ambiguity in the notation/terminology. Is the embedding or the embedding subspace?\n\nThe issue is the grammar.  The embedding function is $e$, and the token subspace is $T$.  The function $e$ is \"embedding\" the set $T$ into the latent space. \n+We'll fix.\n\n> Line 145: Am I correct that $\\psi$ is used solely as an element of the latent space?\n\n+Yes!\n\n> Theorem 1: There is a bracket inconsistency\n\n+Thanks!\n\n> Please briefly introduce $T^w$. \n\n+It's a sequence of tokens of length $w$\n\n> Line 201: Please check whether the reference is correct.\n\n+Good catch!\n\n> Line 218: What does “chain of sausages” mean?\n\nIt's an intuitive attempt at capturing the notion of a \"pinch point\", i.e. two spheres glued together at the north pole of one and the south pole of the other.  See above!\n\n> Lines 221–224: Consider moving this part to a footnote.\n\n+Sure.\n\n> Table 1: Could you please elaborate on why all quartiles for LLaMA-7B are exactly 4096? Is there an explanation for these results?\n\nIt appears to be due to quantization.  We don't want to speculate too much, but it looks like many of the tokens are exactly equally spaced in some parts of the token subspace, almost like they were \"snapped to grid\".  \n\n> Line 257: It is unclear what is meant by “dependency between large and small scale variability.”\n\nEuclidean space is \"regular\" in the sense that translations are homeomorphisms, so then a singular space cannot exhibit this kind of self-similarity.  \n+We were being a bit vague and can clarify.\n\n> Line 270: It would be helpful to have a formal definition of singularity if this is a specific type of irregularity. ...\n\nThis is an *entire* branch of mathematics!!  Our test is not strong enough to make this classification. Maryam Mirzakhani and Heisuke Hironaka got Fields Medals for proving partial results in this direction!\n\n> Line 323: Would it be possible to at least list possible post hoc methods and discuss the implications of determining that?\n\nWe can speculate, and are happy to do so, but given our response to your query immediately above, it's really, really hard mathematics!"}, {"review_id": "QmvAL25j6q", "reviewer": "Reviewer_MDro", "summary": "This paper focuses on the structure of the token embedding space used as input for Large Language Models (LLMs). It demonstrates that this space does not possess the smooth structure typically assumed by the manifold hypothesis or the more general fiber bundle hypothesis.\n\nThe authors propose a novel statistical test to detect irregularities in the neighborhood of specific tokens and report that such irregularities are frequently observed for many tokens.\n\nFurthermore, the paper emphasizes that its primary contribution is a \"methodology for hypothesis testing on embedding points.\"", "questions": "Much of language data often possesses semantic hierarchy within its context, and understanding how this hierarchy is represented in latent space is considered a crucial research topic. I believe Robinson+2025 is one significant prior work in this area; would it be possible to conduct an analysis and discussion regarding hierarchy similar to what is done there?", "limitations": "The actual code and data are not publicly available. This makes it impossible to quickly verify the effectiveness of this method.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n\nThis paper proposes a new statistical test, the \"fiber bundle hypothesis,\" to elucidate the structure of the token embedding space in Large Language Models (LLMs).\n\nIt offers the direct test of whether token subspaces are manifolds, a topic for which prior research had not presented rigorous hypothesis testing.\n\nThe authors conducted tests on four different open-source LLMs and found \"strong evidence that token subspaces are not manifolds.\" They also reported results indicating that tokens exhibiting irregularities often include word beginnings, word fragments, non-printable characters, and whitespace characters.\n\n\nWeakness\n\nAs the authors state, if the main point of this paper is the proposal of a \"methodology for hypothesis testing on embedding points,\" then in addition to the proofs of Theorems 1 and 2, experimental validation is needed to demonstrate the validity of this method. \nAlthough experimental results using the proposed method are shown, there is no verification of whether these results are \"valid.\" A comparative evaluation with results from other analysis methods is necessary. Alternatively, analysis using, for example, synthetic data where the ground truth is known, is also an option.\n\nWhile rejecting regularity does imply singularity, it does not afford us any evidence as to the type of singularity.", "was_revised": true, "final_justification": "Based on discussions with reviewers, we determined that the potential significance is high and raised the Significance rank by one level.\nThere is no change to the overall assessment.", "rebuttal": "Thank you for your review and making us think about a topic we had yet to consider...semantic heirarchy.\n\nNB: Action items for the authors are marked with a \"+\" in the margin!\n\nFor our rebuttal, here's a good starting point: For many tokens, we can estimate a dimension, which is at least understandable because manifolds have a well-defined dimension.  But for a few tokens--those that cause the hypotheses above to be violated--the dimension at that token is not well-defined at all!\n\nResponses to your specific concerns are below:\n\n> As the authors state, if the main point of this paper is the proposal of a \"methodology for hypothesis testing on embedding points,\" then in addition to the proofs of Theorems 1 and 2, experimental validation is needed to demonstrate the validity .... Alternatively, analysis using, for example, synthetic data where the ground truth is known, is also an option.\n\nYour request for a visual schematic of what our test does is a great idea!  Various schematic figures were included in an earlier draft of the paper, where we had included \"toy\" examples of our test being run on \"noisy\" manifolds and stratified manifolds.  \n\nHowever, we had to cut these examples out of the submission due to space constraints...  \n\n+we will include these examples in our final version!  \n\nSince we are not permitted to include graphics in our rebuttals, for the purpose of visualization, one example we plan to include is a \"lollipop\": a candy ball attached on the end of a short stick.  This shape has EXACTLY two important singular points.  On the interior of the stick, the space is a 1-dimensional manifold.  At the very end of the stick, though, is a singular point (the boundary).  Where the stick attaches to the \"candy\" part is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy part.  This visual example was not too difficult to construct, so your suggestion of including pictures showing cusp points, pinch points, etc. is easily implemented.\n\n> While rejecting regularity does imply singularity, it does not afford us any evidence as to the type of singularity.\n\nCorrect. This is an entire branch of mathematics and a complete solution to your question would be a breakthrough of tremendous significance (Maryam Mirzakhani and Heisuke Hironaka got Fields Medals for proving partial results in this direction!).  Given the current state of this branch of mathematics, we can offer intuitive explanations for two types of singularities.  \n\n(Check these out on any graphing software!)\n\n1. A \"cusp point\" is a sharp point. It looks like the graph of the function $y^3 + x^2 = 0$ near the origin;  \n2. A \"pinch point\" looks like the graph of $z^{2}+x^{2}-y^{2}=0$. \n\n> Much of language data often possesses semantic hierarchy within its context, and understanding how this hierarchy is represented in latent space is considered a crucial research topic. I believe Robinson+2025 is one significant prior work in this area; would it be possible to conduct an analysis and discussion regarding hierarchy similar to what is done there?\n\nCould you please clarify which reference you are referring to via \"Robinson+2025\"?  The paper with that citation in our present submission -- [Robinson et al, Probing the topology of the space of tokens with structured prompts, arxiv:2503.15421, 2025] -- does not establish anything hierarchical according to our interpretation.  If it does in your interpretation, we would request you to clarify how.\n\nThe fact that ontologists organize language into semantic hierarchies suggests that the idea of embedding tokens into a single, common space, might not be the most effective representation.  In that vein, it is entirely possible that the appearance of singularities is an indication that the idea of using an embedding representation is ill-conceived.  \n\nWhile an intriguing possibility, an alternative token representation has not been discovered in the literature.\n\nHere is another line of thinking, inspired by [Jakubowski et al, Topology of Word Embeddings: Singularities Reflect Polysemy, in Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, 2020].  Instead of an hierarchy, it's a link of sausages of many different dimensions. Let's take the * symbol, which happens to be singular for Mistral 7B (See Table 7, number 29). In natural language, it has a very distinctive meaning in terms of calling out footnotes, or marking text as bold in Markdown, but in programming it has mathematical meaning as multiplication.  Therefore, this character connects these different senses of natural language with programming languages.  Suppose that each domain/language were in fact well-described by a manifold.  Then they would be joined/glued together in the entire latent space at the * which with high likelihood would be a singularity because the dimensions for different langauges are different [Tulchinskii et al, Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts, arxiv:2306.04723, 2023].\n\nSo there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined. This implies something deeper like pinch points that link spaces of varied dimension. So in the lollipop example, the link point between the candy and the stick is singular because if we are at that point and look left, we appear to be in 1D while if we look right, we appear to be in 3D. The inner product that is being leveraged is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected. In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).\n\n> The actual code and data are not publicly available. This makes it impossible to quickly verify the effectiveness of this method.\n\n+We will make the code publicly available once released by our government sponsor. There have been some delays due to unforeseen events; however, the sponsor did clear the present submission for release, so we do not anticipate there to be any issues with clearing the code release eventually.\n\n> The caption for Table 1 and its explanation in the main text are unclear. A more easily understandable explanation of what each numerical value in Table 1 signifies is needed.\n\nYes, there is a lot of content in that table.  \n\n+In the revision, we can certainly explain it in detail.  The following will be inserted into Section 3.3, probably around line 225.\n\nFirst of all, there are two main tests we run on the data (See lines 62-26): one test against manifolds, and two versions of a test for a fiber bundle\n\nThe \"rejects\" columns give (top rows of each cell) the number of tokens that fail our test.  This is also the number of rows in Tables 3-9.  The listed p value is the smallest such p-value, which is the overall p-value for the whole test.\n\nThe \"dim.\" column shows the quartiles for the estimates of dimensions, at least for the tokens for which a dimension can be reliably be ascribed.  Note that the whole point of our test is that for each of the tokens listed in Tables 3-9, **no dimension estimate is possible**.  Quartiles are a nice way to summarize a distribution.  Usually\n\n- Q1 = 25-th percentile, so 25% of the dimensions estimated are below this quantity\n- Q2 = 50-th percentile, i.e. the median\n- Q3 = 75-th percentile, so 25% of the dimensions estimated are above this quantity\n\n> NeurIPS guidelines encourage providing a \"short proof sketch for intuition\" in the main paper when proofs are included in the supplementary material.\n\nWe will provide a proof sketch for both Theorems in the main body, in Section 3.1 and Section 3.2.  \n\n+A sketch for Theorem 1:\n\nIf the assumed space is a manifold of dimension $d$, then the volume $v$ of a ball of radius $r$ centered at any point is given by an equation of the form\n$$\nv = K r^d + (\\text{correction terms})\n$$\nwhere $K>0$ is some constant (involving $\\pi$ and $d$), and the correction terms involve curvature and reach $\\tau$.  Taking the logarithm of both sides yields a linear regression problem for $d$\n$$\n\\log v = K + d \\log r + (\\text{correction terms})\n$$\nwhere the dimension $d$ forms the slope of the curve.  \n\nFor the more general case of a fiber bundle, the equations above must be modified to handle the case where the ball of radius $r$ \"sticks out of\" the space, and so the slope of the curve must decrease in that case.\n\n+Theorem 2's proof is already quite short, so we might (depending on space constraints) include it in full, or simply state that it is a matter of simply satisfying the bounds from [Robinson et al., 2025]."}, {"review_id": "MRBL4HfWUC", "reviewer": "Reviewer_XXVi", "summary": "This work investigates the presence of irregularities and singularities in the structure of token embedding spaces used by large language models. Treating the embedding space as a geometric object, the authors evaluate its conformity to the manifold hypothesis and the more general fiber bundle structure. They introduce statistical tests to identify tokens whose embeddings violate these hypotheses, revealing cases where the space is locally inconsistent or singularities. The authors also provide a theoretical justification for why such singularities can persist through the model, even with extended context, potentially affecting downstream behavior. Finally, they analyze patterns of geometric irregularity across multiple models, identifying patterns among tokens which violate the hypothesis.", "questions": "Q1: Could the authors point to works in literature which previously suggested adding more tokens to the context help reduce singularities ? \n\nQ2: Could you please elaborate on the significance of Fig 2 bottom ? \n\nQ3: There are a number of works which have discussed a souce of potential geometric similarities across models to be due to shared dataset and next token distributions. Some references are Lee et al  2024 (Shared Global and Local Geometry of Language Model Embeddings) and  Zhao et. al 2024 (Impliciteometry of Next Token Prediction) . Do the authors have any thoughts on this, specially regarding how some of the singular tokens are shared among models such as Mistral and GPT ? Could this all be an artifact of the training data and the next token distribution ?", "limitations": "Addressed", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strengths\nThe paper presents an original and novel perspective on the geometry of token embedding spaces in large language models. By framing the embedding space through the lens of the manifold hypothesis and evaluating deviations via both manifold and fiber bundle tests, the authors introduce a methodology for identifying structural irregularities in token representations. This line of inquiry opens up potential avenues for systematically characterizing LLM embedding geometry and I appreciate the use of empirical analysis in topological and geometric theory is commendable.\n\nWeaknesses\nWhile the contribution is conceptually intriguing, the paper suffers from several issues that limit accessibility and clarity. Many of the key mathematical terms — such as manifold, fiber bundle, and singularity — are introduced without sufficient formal definition or intuitive explanation. This makes it difficult for readers unfamiliar with differential geometry (such as myself) to follow the central arguments. Additionally, some theoretical results, particularly Theorem 1 and Theorem 2, are introduced and referenced before they are clearly stated.\n\nFrom a presentation standpoint, several figures are difficult to interpret. I'm having a hard time understanding the significance of Fig1 and Fig2 bottom, the flowcharts in Fig2. \n\nFinally, a central concern is the scale and significance of the results. If interpreted table1 correctly, fewer than 70 tokens (out of ~30K–50K vocabulary) fail the manifold hypothesis test, and the number of fiber bundle violations, while higher, still represents a relatively small fraction (again if I understand table 1 correctly). While the authors argue these irregularities are meaningful, it's unclear whether these patterns are consistent across training runs or simply artifacts of randomness in high-dimensional embedding spaces. This raises questions about the practical implications and generality of the findings.\n\nI would like to continue this conversation to clarify some of the points in the rebuttal phase.", "was_revised": true, "final_justification": "Apologies for submitting this late, I thought it was only for cases where the score changes. I see the merit in the work and I understand its potential. The authors do a great job at responding to some of the points raised by other reviewer and myself. In particular I believe their final point regarding the impact of the singular tokens on ntp probability is quite impressive. I believe the work is acceptable but I believe that the work requires more clarity in its presentation and needs to include some of the points discussed during this review process for me to raise my score. Therefore, I am keeping my original rating.", "rebuttal": "Thank you for your review and pointing out that we were not clear on the novelty and significance of the singular tokens.\n\nNB: Action items for the authors are marked with a \"+\" in the margin!\n\n> I would like to continue this conversation to clarify some of the points in the rebuttal phase.\n\nGreat!  We are delighted to discuss this with you!  We appreciate your effort in reading our paper.\n\n+We will define in Section 1.2\n\n* manifold\n* singularity (and separately \"singular\")\n* embedding\n* embedded reach\n* volume form\n* radius\n* fibered manifold\n* fiber bundle\n* p-value\n\n+In the appendix\n\n* curvature\n* pushforward\n* homeomorphism\n* diffeomorphism\n\n> Additionally, some theoretical results, particularly Thm 1 and Thm 2, are introduced and referenced before they are clearly stated.\n\n+We will move up some of the material from A.3 and A.4 into Sec 2 so that that is rather more clear.  (see also the response to Reviewer TdY4)\n\n>If interpreted table1 correctly, fewer than 70 tokens (out of ~30K–50K vocabulary) ...\n\nWe point out that even though they are quite few in number, these singular tokens could be of extreme operational importance.  Individual tokens are already known to be the root cause of certain vulnerabilities.  For instance, [arXiv:2507.08794] explains that some very specific tokens correspond to major vulnerabilities in LLMs (called the \"master-key\" in unlocking LLM prompt injections).  These single-token attacks are becoming more widely known and prevalent [arXiv:2504.20493], [Lakshmanan, \"New TokenBreak Attack Bypasses AI Moderation with Single-Character Text Changes\", 2025], [Zilberman, \"Unicode Exploits Are Compromising Application Security\", 2025].\n\nIt is plausible that singularities might be the root cause of these vulnerabilies.  Moreover, these singular tokens might be the cause of other vulnerabilities not yet discovered. (That's future work!)   We would argue that since the singular tokens are distinctly different than the others, and that there are not that many of them, finding the singular tokens is potentially of great value to be community.\n\n+We will insert the above text (after editing) into the Introduction Sec 1 or the Discussion Sec 4.\n\n> While the authors argue these irregularities are meaningful, it's unclear whether these patterns are consistent across training runs\n\nNote that the entire process in our paper is deterministic, so there is no requirement of multiple runs.  What we are presenting is an analytic tool (hypothesis test) for pre-trained models, that moreover can be applied to any point cloud within a latent space.  Note that the models we are considering are published/deployed models, where the token embedding vectors are frozen after training. In other words, the token embedding vectors are fixed when we analyze the model, they do not change from run to run.\n\n+We will add the above text (suitably edited) into the Background Sec 1.2, and reiterate it in our Limitations Sec 4.1 (since although it is not a limitation, the Limitations section is where readers with similar questions might look.)\n\n> Q1\n\nWe have actually never found this discussed in the literature at all.  Indeed, we found (and already cited) only a few papers that mention singularities at all.  What we are saying is that we suspect one of the motivations for the trend toward increasing context window size might be a response to the presence of singularities.\n\nWe also stress that our work shows that if the initial token embedding contains singular tokens, they will persist in the final embedding regardless of the number of tokens (i.e. the context window size) by Thm 2. Our result is not a contradiction of prior work but simply a result that stands on its own.\n\n+Assuming the above paragraph clarifies your question, we could add it to the Discussion Sec 4.\n\n> Q2\n\n+Fig 2 bottom is the histogram of the estimates of dimensions for GPT2 (=slopes of the log radius versus log volume plots), at least for when the tokens have a well-defined dimension.\n\n+We will update the caption of Fig 2 to say the above explicitly.\n\nSo there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined (these are our highlighted singular tokens). This implies something deeper like pinch points that link spaces of varied dimension. Imagine glueing a line segment to a ball (lollipop); the link point does not have a well-defined dim because it joins 1D to 3D. The inner product that is being leveraged at this link is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected.  In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).\n\n> Q3\n\nWhile there might be good alignment between models over much of the token subspace (which would account for the broad geometric similarities), there *simply cannot be* alignment between the singular tokens *if the singular tokens differ* or if the dimensions differ.  Since we show (Tables 3-9) that the singular tokens between the four models we studied differ, there *cannot* be alignment between these models in a fully global sense.\n\n> Some references are Lee et al 2024 (Shared Global and Local Geometry of Language Model Embeddings) \n\nThe authors of this paper show a few things that are consistent with our findings:\n\n+We will discuss this paper explicitly in Sec 1.2, as below:\n\n1. Language models often construct similar local representations as constructed from local linear embeddings\n\nThe \"often\" part here is important.  Our finding is that most of the tokens in the vocabulary fail to reject the manifold hypothesis, which means that we are able to estimate some local structure.  However, for the singular tokens this is simply impossible without deep investigation of the local neighborhoods.  In [Lee 2024], they are not looking at the singular tokens in the way that we are here.\n\n2. Token embeddings exhibit low intrinsic dimensions\n\nThis agrees with our findings in Table 1, at least in the lower radius regime.  Their estimates of dimension for GPT2 is within the confidence interval we computed.  They do not recognize that there might be a fiber bundle structure in play, which is our \"larger radius regime\".\n\n3. Tokens with lower intrinsic dimensions form more semantically coherent clusters.\n\nWe agree, though since they did not perform a test for whether a given token is singular, we are concerned that they may have estimated dimensions for tokens that do not have a well-defined dimension!\n\nNevertheless, Figure 1 seems fairly consistent with their findings.\n\n4. Tokens have similar dimensions across language models\n\nThe hypothesis that \"Tokens have similar dimensions across language models\" is a null hypothesis.  It can be rejected (simply by exhibit two models whose dimensions disagree), but never accepted as true since *all* models cannot be tested.  Our Table 1 shows that this hypothesis must be rejected.  It is not that the dimensions estimated by [Lee 2024] are wrong.  We and they both looked at four models (with one in common, GPT2).  It is highly likely that they might have--by chance alone--picked models that are similar to one another and thereby did not find that their dimensions could differ.\n\n> Zhao et. al 2024 (Implicit geometry of Next Token Prediction)\n\nThis paper is about the transformer structure (our Section 3.2), but not so much about the token embeddings explicitly, so it is less directly relevant to our present submission, but see below.  \n\n+We will include the following two paragraphs (perhaps summarized after discussion with you) in Section 1.2.\n\nSpecifically, they show that the implicit space of possible activations can be inferred from the behavior of the model.  A *slice* of this implicit structure is the token subspace.  We note that in this vein, [Zhao 2024] is not inconsistent with other findings within the literature, for instance [Robinson et al, Probing the topology of the space of tokens with structured prompts, arXiv:2503.15421, 2025].  This latter paper shows that the token subspace can be inferred (up to homeomorphism, aka topological equivalence) from the responses of the LLM to single-token prompts.\n\nTaken together, the above papers and the present one have pretty significant implications.  Even though the token subspace may have about 0.1% of tokens that are singular, we suspect that a user will be encountering singular tokens on a regular basis when using LLMs (which seems empirically plausible).  Estimating the rate of occurance of a singular token is hard, and very much future work!\n\nHowever, this matters in two distinct ways:\n1. Prompting from the user.  If the user hits a singular token, both [Zhao 2024] and [Robinson 2025] indicate that the model's behavior is likely to go badly from that point onward.  The current \"fix\" for this problem is prompt engineering.\n\n2. As part of how they work, the transformers sample the next token, and almost surely as the number of samples gets very large, the LLM will indeed pick a singular token. There is no prompt engineering fix in this case.\n\n> Could this all be an artifact of the training data and the next token distribution ?\n\nWe are not entirely sure about the specific meaning of your question.  Please clarify?"}, {"review_id": "5EBEWWM16z", "reviewer": "Reviewer_TdY4", "summary": "The authors propose three hypothesis tests designed to assess the geometric structure of embedding subspaces associated with individual tokens. Specifically, they aim to determine whether a subspace can be characterized as (1) a manifold, (2) a fibered manifold, or (3) a fiber bundle. These tests are grounded in Theorem 1, which describes how the volume of a ball of radius $r$, centered at a point $\\psi$, changes for small versus large $r$. The authors apply their methodology to analyze token embeddings produced by large language models (LLMs).", "questions": "# Major Comments\n\n**1. Presentation and Organization**\nThe paper lacks clear structure, which significantly hinders readability. For example, Figure 3 is said to encapsulate the core methodology, yet its interpretation is non-trivial without additional context. The necessary intuition is relegated to the appendix, where it is unlikely to be seen early by readers. I strongly recommend relocating this explanation to the main text and integrating it into the broader narrative.\n\n**2. Volume Estimation in Figure 3b**\nIt is unclear how volume is defined or computed in Figure 3b. Lines 120–125 suggest that it may correspond to the number of tokens within a ball. If this is correct, the authors appear to be using token count as a proxy for manifold volume. This is a strong and potentially unjustified assumption. Can the authors provide a theoretical or empirical rationale for this estimation method?\n\n**3. Slope Thresholding**\nThe hypothesis tests seem to hinge on detecting slope changes in volume scaling. Does this involve setting a specific threshold? If so, how was this threshold determined? Why was this particular value chosen over others? The paper should clarify the methodology for threshold selection and discuss its sensitivity.\n\n**4. Experiments on Specific Token Subsets**\nThe authors’ conclusion---that LLM embeddings do not lie on a single coherent manifold---is intriguing, but more granular analysis is needed. For example, are certain semantic subsets (e.g., years from 1900–2000, color names, or month names) manifold-like? Exploring such structured subsets could strengthen the argument and help determine whether the negative results reflect global structure or just aggregate noise.\n\n**5. Comparison to Other Embedding Methods**\nThe empirical evaluation is limited to LLM embeddings. However, other widely used embedding techniques, such as Word2Vec and GloVe, also produce token representations. Applying the proposed tests to these methods could provide important comparative insights and help clarify whether the results are specific to LLMs or generalizable across embedding strategies.\n\n**6. Test Validity and Robustness**\nThe paper lacks validation of the proposed tests on synthetic data. For example, if the authors apply their methodology to points sampled from known manifolds (e.g., spheres, tori), do the tests recover the expected structure? How do noise and sampling density affect test accuracy? Exploring such controlled settings would provide valuable evidence for the tests’ robustness and reliability.\n\n---\n\n# Minor Comments\n\n* **Equation Formatting**: Equation (1) contains an unclosed parenthesis: $(e_*v)(B_r(\\phi)\\color{red})\\color{black}=\\dots$", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 3, "originality": 4, "strengths_and_weaknesses": "## Strengths\n\n* The paper addresses a timely and relevant problem.\n* The theoretical framework appears sound and well-motivated.\n* The conclusions challenge prevailing assumptions in the community.\n\n## Weaknesses\n\n* The paper's presentation is disorganized and often unclear.\n* The experimental section lacks detail and depth.\n* The empirical analysis is limited in scope.\n\n## Review Summary\n\nThis work introduces a novel and theoretically grounded approach for testing the manifold hypothesis in token embeddings derived from LLMs. The conclusions are intriguing and potentially impactful, as they challenge commonly held beliefs about the geometry of learned representations.\n\nHowever, the paper suffers from several major weaknesses. The organization and clarity of presentation are lacking---key intuitions and methodological explanations are buried in the appendix rather than integrated into the main narrative. Moreover, the experimental validation is limited both in scope and detail. The authors focus exclusively on LLM embeddings, ignoring other widely used embedding techniques such as Word2Vec or GloVe. Furthermore, while the proposed tests appear general, their empirical application is shallow, raising concerns about their validity and robustness.\n\nDespite the paper’s potential, I do not recommend it for publication in its current form. I suggest a **borderline reject**.\n\nThat said, I acknowledge that I may have missed or misunderstood certain aspects of the paper, and my confidence in this assessment is low. I would be open to revising my recommendation if the authors are able to address the concerns outlined below.\n\nRegardless of the final decision, I can see the potential of this work and I encourage the authors to continue their research in this direction.", "was_revised": true, "final_justification": "The authors have addressed my main concern by outlining a clear and reasonable revision plan. Additionally, they have provided satisfactory responses to my other comments. I have decided to raise my score.", "rebuttal": "Thank you for your review and positive encouragement for the findings in our work.\n\nNB: Action items for the authors are marked with a \"+\" in the margin!\n\n> That said, I acknowledge that I may have missed or misunderstood certain aspects of the paper, and my confidence in this assessment is low. I would be open to revising my recommendation if the authors are able to address the concerns outlined below.\n\nLet us lead off by saying that we are happy to engage in discussion!  \n\nMathematically, a manifold has a singular dimension (pun intended), so the variability of dimension estimates is emblematic of manifold violation.  If the estimates of dimension were concentrated with a small variance around an integer, then we might conclude that the estimates were simply noisy.  Our results shown in Fig 2 and Table 1 show that this is not the case.  \n\nNote that the variance is across all tokens for which intrinsic dimension can be estimated, which is a quite large number of them, so these estimates are reliable.  This is not in contradiction with the fact that for some of tokens, dimension cannot be estimated at all.\n\nSo there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined. This implies something deeper like pinch points that link spaces of varied dimension. Imagine glueing a line segment to a ball (lollipop); the link point does not have a well-defined dim because it joins 1D to 3D. The inner product that is being leveraged at this link is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected. In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).\n\n> The organization and clarity of presentation are lacking\n\n+We can bring material from A.3 and A.4 up into the main body Sec 2 without too much effort, especially because the final version permits an extra page.  (Do also have a look at our response to reviewer MkzF, wherein we have committed to a detailed revision plan.)\n\n> For example, Fig 3 is said to encapsulate the core methodology, yet its interpretation is non-trivial without additional context. The necessary intuition is relegated to the appendix, where it is unlikely to be seen early by readers. I strongly recommend relocating this explanation to the main text and integrating it into the broader narrative.\n\nThat is a great idea.  Thanks!  We will also incorporate some text like what we say below (concrete actions marked)...\n\n> Volume Estimation in Fig 3b It is unclear how volume is defined or computed in Fig 3b. Lines 120–125 suggest that it may correspond to the number of tokens within a ball. If this is correct, the authors appear to be using token count as a proxy for manifold volume. This is a strong and potentially unjustified assumption.\n\nWe are treating the tokens as a sample from the assumed space (either a manifold or a fiber bundle, depending on the test). We are estimating the volume of the ball of a given radius around a given token by counting the number of tokens within that radius. This is nothing other than a classic Monte Carlo estimate of volume. It is a strong assumption, and is inherent in the \"volume form\" in the statements of our Theorem 1, but it is also extremely established in the literature.\n\n+We will include the above paragraph (suitably edited) in Sec 2\n\n> Slope Thresholding The hypothesis tests seem to hinge on detecting slope changes in volume scaling. Does this involve setting a specific threshold? If so, how was this threshold determined? Why was this particular value chosen over others? The paper should clarify the methodology for threshold selection and discuss its sensitivity.\n\nThank you for your careful read of the paper!\n\nFor the manifold test, which is a little simpler, we collected dimension estimates using the standard 3-point centered difference estimate via `numpy.gradient()` at each radius under test.  These dimension estimates were then binned into a pair of adjacent sliding windows.  The estimates in neighboring bins were compared using the standard unpaired two-sample T-test with the stated alpha level for decision.  This test yields a Bonferroni-corrected p-value for the token.  If the p value is less than alpha, then there is not a well-defined dimension for that token, so the manifold test fails at that token.\n\n+We will include the above paragraph (suitably edited) in Sec 2\n\n> The authors’ conclusion---that LLM embeddings do not lie on a single coherent manifold---is intriguing, but more granular analysis is needed.\n\nThat is a fascinating idea, but really ought to be future work.  We need to systematically collect data to conduct such an experiment.  Thank you for the suggestion!\n\n+We can list your suggestion as future work in Sec 4.\n\n> Exploring such structured subsets could strengthen the argument and help determine whether the negative results reflect global structure or just aggregate noise.\n\nThey are not multiple runs, as the entire process in our paper is deterministic.  What we are presenting is an analytic tool for pre-trained models, that moreover can be applied to any point cloud within a latent space.  In other words, the token embedding vectors are fixed when we analyze the model, they do not change from run to run.\n\nA natural question is therefore if we can find a token embedding that is everywhere regular. One way to approach an answer is to view the number of tokens in a vocabulary as a hyperparameter.  There might be a certain vocabulary size that minimizes the number of singularities given the corpus, architecture, etc.  Even the best vocabulary might lead to a space that could still be singular, for reasons inspired by [Jakubowski et al, Topology of Word Embeddings: Singularities Reflect Polysemy, in Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, 2020].  Let's take the * symbol, which happens to be singular for Mistral 7B (See Table 7, number 29). In natural language, it has a very distinctive meaning in terms of calling out footnotes, while in other contexts it can be used for marking text as bold in Markdown, or in programming it has the mathematical meaning of multiplication.  Therefore, this character connects these different semantics: i.e. natural language with programming languages and markdown.  Suppose that each language were in fact well-described by a manifold.  Then these manifolds would be joined together in the entire latent space at the * (and likely other points).  As a result, this token with high likelihood would be a singularity because the dimensions for different \"languages\" are almost surely different [Tulchinskii et al, Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts, arxiv:2306.04723, 2023].\n\n+We can include the above discussion in the introduction (space permitting) or Discussion Sec 4.  \n\n+We will also mention something about your question about randomness in our Limitations Sec 4.1 (since although it is not a limitation, the Limitations section is where readers with similar questions might look.)\n\n> The empirical evaluation is limited to LLM embeddings. However, other widely used embedding techniques, such as Word2Vec and GloVe, also produce token representations. Applying the proposed tests to these methods could provide important comparative insights and help clarify whether the results are specific to LLMs or generalizable across embedding strategies.\n\nThis is a fair point, and it's entirely sensible to run our test(s) on these other embeddings.  The method applies to any point cloud in Euclidean space, and LLMs are mostly a convenient test case. We opted for LLM embeddings because they are currently the most widely deployed models where we could easily make some high level comparisons. Comparing across model embeddings is complicated by potential confounding effects, e.g. corpus, cost function, etc. This is certainly future work.\n\n> The paper lacks validation of the proposed tests on synthetic data. For example, if the authors apply their methodology to points sampled from known manifolds (e.g., spheres, tori), do the tests recover the expected structure? How do noise and sampling density affect test accuracy? Exploring such controlled settings would provide valuable evidence for the tests’ robustness and reliability.\n\nRegarding a synthetic example, in an earlier draft of the paper, we had included \"toy\" examples of our test being run on \"noisy\" manifolds and stratified manifolds.  However, we had to cut these examples out of the submission due to space constraints...  \n\n+we will include these examples in our final version!  \n\nSince we are not permitted to include graphics in our rebuttals, for the purpose of visualization, one example we plan to include is the \"lollipop\": a candy ball attached on the end of a short stick (mentioned before).  This shape has EXACTLY two singular points.  On the interior of stick, the space is a 1-dimensional manifold.  The end of the stick is a singular point/boundary.  Where the stick attaches to the \"candy\" is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy.\n\n> Equation Formatting: Equation (1) contains an unclosed parenthesis: \n\n+Oops.  Thanks for catching that!"}, {"review_id": "i2SSu2EhqV", "reviewer": "Reviewer_ZHkg", "summary": "The manuscript  introduces an analysis aimed at verifying if the points of a dataset can be meaningfully considered as contained in a manifold or in a fiber bundle. The statistical test is applied for each points separately, and is based on a null model in which the manifold is locally flat and uniformly populated. The approach is applied to the analysis to token emebeddings in four large language models, and it allows identifying, for each model, between 40 and 70 tokens which violate the manifold hypothesis with high statistical confidence. A smaller number of tokens violates even the fiber bundle hypothesis.", "questions": "1) how does the intrinsic dimension  (ID) vary across the dataset?\n2) how does the values of the ID obtained by a linear fit compare with those obtained by other ID estimators? For example you should consider the Danco Estimator, or the approach https://www.stat.berkeley.edu/~bickel/mldim.pdf\n3) can you exclude that  the violations of the manifold hypothesis are due to variations of the density?\n4) if one would eliminate from the vocabulary the (few) \"singular\" tokens, substituting those by synonyms, would the data manifold become everywhere smooth?", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: focusing on the geometric properties of the data set in the neighborhood of each point separately is interesting and brings insight on the properties of each single token. Indeed, the tokens violating the manifold hypothesis are listed in Supp Mat, and one can attempt to interpret why these specific tokens correspond to \"singularities\"\nWeaknesses:\n1) (minor) the geometric criterion at the basis of the approach (eq 1) is very similar to the approaches used from the seventies onward to estimate the fractal dimension of a dataset (for example the correlation dimension). It is well known that one can obtain  deviations from  a linear behavior of the log-log plot of the volume versus radius just because the probability density of the data on the manifold varies quickly. \n2) (major): the number of violations detected by the approach is very small, of the order of 0.1 % (according to table 1). Therefore, these results do not seriously challenge the idea that data representation are contained in a relatively low-dimensional manifold, also because, to the best of my knowledge,  nobody ever claimed that this manifold is a clean and smooth \"Riemanian\" variety. The fact that this manifold is not smooth everywhere is not a surprise.  \n3) (major): The first claim of section \"Experimental results\" is that  \"the local dimension is highly variable\".  Leaving aside the fact that even this result is not really novel, one  would like to see some numerical data supporting this claim.", "was_revised": true, "final_justification": "I changed my score from 2 to 3 since in my first review I did not notice that the value of the local dimension is reported in fig 2. My main remaining concern  is that the number of violations of the manifold hypothesis is really small. The authors use the lollypop as an example to explain why a small number of violations implies that a geometrical object is not a manifold. This is for sure correct, but, in my opinion, a bit off-topic in the contest of the analysis of representations in DNNs. Most (if not all) the approaches I am aware of do not assume that those representations  are GLOBALLY contained in a genuine manifold, as it would be intended in mathematical physics, and the fact that the intrinsic dimension is highly variable is well known.", "rebuttal": "Thank you for your review for it made us realize that we had not been clear about the novelty and significance of the singular tokens.\n\nNB: Action items for the authors are marked with a \"+\" in the margin!\n\nWe point out that even though they are quite few in number, singular tokens could be of extreme operational importance.  If we ignore/remove/substitute the singularities to \"force\" the token subspace to a manifold, we risk losing our ability to understand the behavioral features of the entire model.\n\nIndeed, after our present submission, this paper [Zhao et al, One Token to Fool LLM-as-a-Judge, arXiv:2507.08794, 2025] explains that some very specific tokens correspond to major vulnerabilities in LLMs.  These single-token attacks are becoming more widely known and prevalent [Cui et al, Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compression, arXiv:2504.20493, 2025], [Lakshmanan, \"New TokenBreak Attack Bypasses AI Moderation with Single-Character Text Changes\", published on The Hacker News website, June 12, 2025], [Zilberman, \"Unicode Exploits Are Compromising Application Security\", published on the Prompt.Security website, April 30, 2025].\n\nIt is more than plausible that singularities might be the root cause of these vulnerabilities, and possibly the cause of other vulnerabilities not yet discovered. (That's future work!)\n\n+We will mention the above in our discussion Section 4.\n\nYou are also correct that in a variety of settings, estimates of local dimension (which, depending on the author, has been called \"intrinsic dimension\"=\"ID\", \"local dimension\", among various other terms) can vary quite a bit.  Also, you are correct that this variability has been reported by a number of others.  Our submission cites a few of these already [Fefferman 2018, Gromov 2024, Tulchinskii 2023, Robinson 2024], and the paper that you list is indeed another one of those.  Thanks for that!  \n\n+We will include the paper you cite in our list of citations, mentioning it in Section 1.2\n\n+Also, since writing we found another good summary to mention in Section 1.2: [Schweinhart, Fractal dimension and the persistent homology of random geometric complexes, Advances in Mathematics, vol. 372, 2020]; we can include that paper in our citations to benefit the enterprising reader.\n\nMathematically, a manifold has a singular dimension (pun intended), so the variability of dimension estimates is emblematic of manifold violation.  If the estimates of dimension were concentrated with a small variance around an integer, then we might conclude that the estimates were simply noisy.  Our results shown in Fig 2 and Table 1 show that this is not the case.  \n\nNote that the variance is across all tokens for which intrinsic dimension can be estimated, which is a quite large number of them, so these estimates are reliable.  This is not in contradiction with the fact that for some of tokens, dimension cannot be estimated at all.\n\nSo there are two ways our test has shown a violation of the manifold hypothesis. The large distribution of local dimension (as seen in the histogram in Fig 2 and Table 1) rejects that the token embedding is globally a manifold. This does not preclude that the embedding space is a disjoint union of smooth manifolds of different dimensions (which would still be problematic since LLMs leverage mathematical machinery such as inner products). Our test also rejects this possibility because we find tokens where a local dimension is simply not well defined. This implies something deeper like pinch points that link spaces of varied dimension. So in the lollipop example, the link point between the candy and the stick is singular because if we are at that point and look left, we appear to be in 1D while if we look right, we appear to be in 3D. The inner product that is being leveraged is on the extrinsic space and not the actual embedding space, which implies that the model is not operating on the token we would have wanted/expected. In fact, an inner product is not defined at this point (at least not in the usual way) and so we would have to characterize the singular point and find the correct mathematical machinery (if it exists).\n\nMoreover, in response to Q1, we do report the variability local dimension estimates in two places.  The histogram in the lower frame of Figure 2 shows the distribution of local dimension estimates for GPT-2.  Table 1 shows quartiles for the empirical distribution of local dimensions for each of the models we tested.  The fact that the interquartile range is large suggests that the local dimension estimates do indeed vary substantially.\n\n+We will rewrite Figure 2's caption to call out the fact that it is the distribution of dimension estimates, at least for the tokens whose dimensions can be estimated.\n\nSpecifically, our primary/novel contribution is a *test*, not an *estimator*.  Our test asks whether there is a single reasonable estimate for dimension at each token.  If our test fails, then there is at least one token for which a single dimension does not make sense.  If our test fails, then any and all estimators (including the one that the referee cites) are not valid for use at that token, because the assumed relationship between volume and radius is not log-log linear.\n\n+We will insert the above two paragraphs into Section 1.1 so that there is no confusion\n\nThere are a number of cases where the community implicitly assumes the data lies on a manifold, of the kind that formally satisfies the mathematical definition.  \n\n+For avoidance of doubt, we will include the formal definition of a manifold in our paper, along with quite a few others that we mistakenly left out.  (See our response to Reviewer XXVi) \n\nHere are a few examples of when the manifold hypothesis is (strongly) assumed...using:\n1. Inner products (which, in fact, requires a Riemannian manifold)\n2. Cosine Similarity (which is just inner products on n-spheres which has even more assumptions about curvature especially negative curvature)\n3. Manifold learning (Any sort of dimensionality reduction, etc)\n\nThe key takeaway is that there are mathematical machinery for singular spaces...in practice, the community simply doesn't say the word \"manifold\" because we have always implicitly assumed we have a manifold, and often assumes a convex one at that. And so, an implication of our work is that there might be straightforward explanations when the model behaves erratically since we are using mathematical tools when we ought not. Without a Riemannian manifold as the embedding, it should not be surprising that applying inner products (loosely speaking) will generate unstable results. \n\nIt is very much the case that *samples* drawn from a manifold under noisy conditions are also the norm.  That provides no issues for our results, as it is the formal setting of our paper, as noise is (explicitly!) incorporated into the volume form.  Indeed, this is why we use the standard hypothesis testing framework -- to handle noise in a statistically proper way.\n\nIn the case of common manifolds, such as a sphere or torus, our manifold test never fails. In an earlier draft of the paper, we had included \"toy\" examples of our test being run on \"noisy\" manifolds and stratified manifolds.  However, we had to cut these examples out of the draft due to space constraints...\n\n+We will definitely include these examples (see below for some details) in our final version!\n\nThe example of a stratified manifold is perhaps most important for the overall importance of our story.  As a simple example, consider a space that has the shape of an \"X\", like {(x,y) : |x| = |y|} in the 2D plane.  This space is not a manifold because it has exactly one singularity at the origin (0,0).  Local dimension at every other point is 1, but at the origin, the space does not have a well-defined dimension.  One can certainly remove the singularity to obtain a 1-dimensional manifold (with four connected components).\n\nAnother example we plan to include is a \"lollipop\": a candy ball (hollow sphere) attached on the end of a short stick.  This shape has *exactly* two important singular points.  On the interior of the stick, the space is a 1-dimensional manifold.  At the very end of the stick, though, is a singular point.  Where the stick attaches to the \"candy\" part is another singularity; it is a transition from the 1-dimensional stick to the 3-dimensional candy part.\n\nHowever, removing the singularity, even though it accounts for 0\\% of all points in the space, destroys the entire structure of the space!  The case of the token subspace is similar; even though the singularities constitute a vanishingly small number of tokens compared to the total vocabulary, they might be similarly important in interpreting the global structure of the token subspace."}]

Author rebuttal:

Meta-review: This paper considers a simple question: Do token embeddings satisfy the manifold hypothesis? The manifold hypothesis is a popular explanation for the success of deep learning, particularly in vision applications. But whether or not language models exploit a similar structure is (surprisingly) unknown. The authors adopt a rigorous approach to study this problem and show that in fact, token embeddings *do not* satisfy a smooth version of the manifold hypothesis. This accomplished by formulating the problem as a hypothesis test, and developing the necessary theory to develop a method for testing this hypothesis.

Reviews were split and there was extensive discussion on this submission (almost twice as many comments as average). Reviewers all agreed that the problem is relevant and the formulation provided here is correct and appropriate. There was some disagreement on the level of formalism used, which is a matter of taste and subjectivity. Given that the manifold hypothesis invokes many mathematical nuances that may be unfamiliar to some audiences, this paper does an admirable job of traversing the landscape. In particular, the role of regularity and smoothness in formulating this hypothesis is carefully laid out for readers. At the same time, one reviewer commented that such regularity may not always be necessary, and the authors should clarify this point in the camera ready. Furthermore, during the discussion, the authors helpfully clarified several points and promised to include more exposition and discussion in the final version.

Ultimately, this paper tackles an important problem in a novel way, provides a new framework for studying it, and contributes both rigorous theory and a practical method. Experiments on real LLMs (GPT, Llama, Mistral, Pythia) demonstrate the usefulness of the results.
Decision: Accept (poster)