Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scaling
Authors: Federico Tiblias
Irina Bigoulaeva
Jingcheng Niu
Simone Balloccu
Iryna Gurevych
Venue: Accepted by TMLR ()

Abstract: The linear representation hypothesis states that language models (LMs) encode concepts as directions in their latent space, forming organized, multidimensional manifolds. Prior work has largely focused on identifying specific geometries for individual features, limiting its ability to generalize. We introduce Supervised Multi-Dimensional Scaling (SMDS), a model-agnostic method for evaluating and comparing competing feature manifold hypotheses. We apply SMDS to temporal reasoning as a case study and find that different features instantiate distinct geometric structures, including circles, lines, and clusters. SMDS reveals several consistent characteristics of these structures: they reflect the semantic properties of the concepts they represent, remain stable across model families and sizes, actively support reasoning, and dynamically reshape in response to contextual changes. Together, our findings shed light on the functional role of feature manifolds, supporting a model of entity-based reasoning in which LMs encode and transform structured representations.

Published in Transactions on Machine Learning Research (02/2026)
Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scaling
Federico Tiblias 1,23344
federico. tibliasttttdddrsstttaa de
Irina Bigoulaeva 1,2,3
irina. vrina.bigoulaeeeo Gtv Stw-darmstadt. de
Jingcheng Niu 12,3
jöngcheng viu wtu waw.-dammsadd...m de
Simone Balloccu
simone. simone.ballocuue balloccu@@taddarssstaae de
Iryna Gurevych
sryma.gareencch@tta wtw www.shodtdddddddddttttta.m de
Deportment of Computer Science, Technical Universily of Darmstadt Ubiquitous Knowledge Processing Lab (UKP Lab)
National Research Center for Applied Cybersecurity ATHENE, Germany
Zuse School ELIZA, Technicol University of Dormstadt
Reviewed on OpenReview: http:://pperrrvveeeeeeee//rrrrtr
Abstract
The linear representation hypothesis states that language models (LMs) encode concepts as dircctions in their latent space, forming organized, multidimensionall manifolds. Prior work has largely focused on identifying specific geometries for individual features, limiting its ability to generalize. We introduce Supervised Multi-Dimensional Scaling (SMDS), model-agnostic method for evaluating and comparing competing feature manifold hypotheses. We apply SMDS to temporal reasoning as a case study and find that different features instantiate distinct geometric structures, including circles, lines, and clusters. SMDS reveals several consistent characteristics of these structures: they reflect the semantic properties of the concepts they represent, remain stable across model families and sizes, actively support reasoning, and dynamically reshape in response to contextual changes. Together, findings shed light on the functional role of feature manifolds, supporting model of entity-based reasoning in which LMs encode and transform structured representations. Our code is publicly available at: httpps///gtthii wttp:/wwt/////ctt////./ wwwpsa//mmm222--mmmeem fttpaaaammmmmmm222mmmmmooooooolo manifold-analysiss
1 Introduction
There is increasing evidence from recent work in mechanistic interpretability that language models develop structured representations of entities in their latent space. Notably, Heinzerling Inui (2024) find that numerical entities (e-g., Karl Popper was born in 1902) are represented in a monotonic, "pseudo-linear" fashion. Incrcasing or docrcasing specific neuron activations can lead the model output a higher or lower value. More recently, Engels et al. (2025) discover non-linear modes of structural entity representation, which form strikingly interpretable patterns. They show that days of the week (Sunday, Monday) and months (December, January), for example, form circular structure. Concurrent work by Modell et al. (2025) provides formal definitions of these feature manifolds and explores how they arise in LMs.
Nevertheless, several fundamental questions remain unanswered: we do not know if and how LMs make use of these manifolds during reasoning, or how reliably detect their presence (Engels et al. 2025 Modell et al. 2025). Answering these questions can help improve LMs and how we control them. This is particularly important in light of current limitations of LMs, such poor temporal reasoning (uuan et. al. 2023

---

Published in Transactions on Machine Learning Research (02/2026)
Method
Findings
munth
Plaus
ganmer
Circular
Cluster
Auturn
Spring
SMDS
45%
Yoniggh
Winter
Accuracy
Accuracy
Linear
Semicire
The only one born in gummer Allie
Speacman p: O 513
Peurson 0.560
0.01
Enables locating feature subspuces matching user-de user-defined geometric assumptions.
Supports comparison of geometries via gocdness-offfit metric to identify the best match.
Feature manifolds of temporal entilies adapt response to lask, prompi, and stage of d'reasoning.
Goodness-offB fit metric significantly correlates with reasoning performance.
Figure 1: Our Main Contributions. Supervised Mulli-Dimensional Scaling is novel dimensionaliiy reduction method for identifying and testing subspaces with known geometry (left). Using it, we show that temporal entities in LMs form task- and prompt-dependent feature manifolds that support reasoning (right).
et al. 2024), difficulty in alignment (Wang et al. 2023 bias (Gallegos al. 2024). and vulnerability distraction (Shi et al. 2023 Niu et al. 2025).
In this paper, we address these questions by introducing Supervised Multi-Dimensiion Scaling (SMDS), novel method to systematically analyze feature manifolds. Commonly used dimensionality reduction methods typically enforce fixed structural assumptions, making it difficult to compare results across alternative geometric hypotheses. SMDS complements existing methods by providing a unified way lo specily arbilrary geometric assumptions and a quantitative metric to evaluate their fit. SMDS reframes the identification of a feature manifold as model selection problem, and thus offers quantitative support for claims about the underlying structure of learned representations. Moreover, this method enables observing how a feature manifold evolves across different layers and reasoning steps.
We focus on temporal reasoning through short-form QA tasks, such identifying recency, ordering events and estimating durations, as we consider them an ideal test bed for manifold analysis. This choice is motivated by three factors: (1) LMS display poor performance in such tasks (Yuan et al. 2023; Huang et al. 2023 Niu
et al. 2024); (2) initial evidence has found temporal feature manifolds to vary widely across tasks (Heinzerling
& Imui 2024 Engels et al. 2025); and finally, (3) there is gap in analyses targeting the atomic structures of temporal reasoning from mechanistic standpoint.
Our main findings can be summarized as follows:
Fi: Temporal entities form feature manifolds with intuitive structures, and this pattern is consistent across model architectures and sizes. We find that the manifolds associated with various temporal concepts (e-g,, days of the year, hours, durations, and historical events) align with interpretable geometries such as circles, lines, and clusters, thus substantially extending Engels et al. (2025) findings. Our SMDS experiments cover over sixty thousand recovered manifolds and confirm that the identified feature structures are robustly shared across different model sizes and architectures.
F2: Feature manifolds are dynamically adjusted depending the task. SMDS enables to compare manifold structures across different token positions. We analyze prompts that share the same context but differ in their final completion cuc, and find that LMs alter feature manifolds based on the eue and task in an intuitive way.
Fa: Feature manifolds actively support reasoning. We find that LMs actively utilize ura manifolds to perform reasoning tasks, supported by two pieces of crucial evidence. First, perturbing hanifold-alignee subspaces consistently impairs reasoning performance, while equivalent noise applied to random subspaces has negligible effect. Second, we observe that manifold quality significantly correlates with downstream performance.
2

---

Published in Transactions on Machine Learning Research (02/2026)
When combined with previous results on the binding problem (Feng & Steinhardt 2023 our findings suggest explanation for the mechanism by which LMs perform reasoning. We hypothesize an entity-based reasoning pipeline in LMs that:
Represents entity properties in coherent locations on manifold within the residual stream;
2. Applies a transformation to this manifold, guided by the question or task context;
Selects an appropriate output based on the transformed representation.
Finally, we extend our analysis beyond mono-dimensional temporal features into two separate experiments (5.44) the first is an entity-based reasoning task on geography that similarly uncovers manifold structures shared across models; the second studies a pair of temporal features to locate a multidimensional manifold. These experiments show our analysis can be extended beyond the temporal domain and to higher-dimensional features. Overall, these results suggest that feature manifolds play an important role in how LMs represent and reason about entities. We view this work as step towards better understanding the mechanisms behind reasoning in modern language models.
Contributions We first present a survey of previous feature manifold analysis methods and the types of geometric structure they are able to capture ($2). We then introduce the novel SMDS method in $ Next,
we present our results in $5 where we outline three major findings: ($5.) manifold geometry for the same type of entity is shared across models; ($5.2) LMs adapt structures in context for different tasks; and ($5 (.3)
LMs actively use feature manifolds for reasoning. Moreover, we show that our approach extends to other
domains and to multidimensional manifolds ($5.4). Finally, we conclude the paper with a discussion
2 Feature Manifold Analysis
(a)
Podel
Bob bazn the 10th Hacch.
Bob
Charles was born on the 29th of dune.
Dimensionality reduction methods commonly used in manifold analysis typically rely on fixed structural assumptions about the data, which makes it difficult to quantitatively compare alternative geometric hypotheses. This gap motivates us to introduce our SMDS method in Section 3 In this section, we set up the problem with relevant background and survey existing feature manifold analysis methods.
the oldeat is...
Activation
Others
Ours
(b)
(c)
LDA
PCA
PLS
SMDS
Tople dute wone
with
Figure 2: Comparison of structural assumptions in LDA, PCA, PLS and SMDS with a circular geometry specified as the hypothesis.
Preliminaries We illustrate our method us- ing temporal reasoning task as a running example (Figure 2b). Performing temporal rea-
soning requires model to understand both explicit mentions of temporal expressions (Jia et al. 2018b) and implicit knowledge of temporal calculus (Allen 1981. Our analysis focuses on how LMs process temporal expressions, which are central to temporal reasoming and define precise, measurable quantities that can reveal underlying feature manifolds. Temporal reasoning also offers good diversity: different types of temporal expressions demand different reasoning skills (o.g., comparing frequencies, ordering events, or identifying recency) and models vary widely in how well they handle these tasks (Chu al., 2024).
In particular, we seek to start from confirming Engels al s (2025) finding that LMs tend to represent calendar dates in a circular topology, placing December near January in their latent space. Consider a prompt comprising several sentences following the template "<name> was born on the <day> of <month>." When asked "The oldest is," the task is answered correctly if the model uses contextual information to produce the correct. answer <name>.
By querying the LM with several such prompts varying the reference date, we elicit internal representations that collectively reside on the feature manifold of calendar dates. In this case, our quantity of interest. is
3

---

Published in Transactions on Machine Learning Research (02/2026)
the birthday of the correct person (e.g., Bob's birthday: 10th of March in Figure 2n), which collectively represent set of labels y. We map these labels onto the [0, 1] interval, where 0 corresponds to Jan 1st and 1 to Dec 31st. We then extract the hidden states corresponding to the last token of the date (e.g,, the "<month>" token : yielding a collection of hidden states X E R=*d with n number of samples and the hidden size of the LM. Next, we use dimensionality reduction to project the high-dimensional hidden states X onto an interpretable, low-dimensional space.
Existing Methods We identify three primary methods used in previous works: PCA. LDA, and PLS (Wold et al. 2001 Park et al. 2024a El-Shangiti et al. 2025 Modell et al. 2025 inter alia) From observing the visualizations in Figure 20, we see that each method highlights different aspects of the underlying data structure but none can readily recover arbitrary geometries such as the circular one we seek. LDA finds interpretable clusters but has no notion of order; PCA fails to identify feature subspaces if they are not aligned with the dircctions of maximum variance; and PLS is limited to linear features unless a suitable transformation is applied to the data (AlquBoj el al. 2025). Moreover, without quantitative metrics to assess the goodness-offfe offteess-offfii fit across different methods, it is unclear which of the manifolds best reflects the original reprosentations.
3 Supervised Multi-Dimensional Scaling
To address this, we introduce Supervised Multi-Dimensional Scaling (SMDS), a method for testing and comparing user-specified geometric hypotheses about representation structure. extends classical Multi- Dimensional Scaling (MDS: Ghojogh et al., 2020) by incorporating supervision, under the assumption that, labels can parametrize the underlying feature manifold formed by the model's hidden states. The method first uses MDS to build an idcal geometry representing the manifold, and then learns lincar mapping from model embedding to this manifold structure. SMDS is flexible, as varying the assumption enables recovering multiple different structures, and provides a common basis to quantify their fit and thus identify preferential one.
Formally, we assume that activations X forming the feature manifold can be located using labels y that represent a numerical property. SMDS first computes ideal pairwise distances d(y,, yj) between yr, Vo E y that encode the geometry of the desired manifold (e.g., circular, linear, or clustered). It then finds linear projection W E Rm such that the Euclidean distances between projected points W we and Wx, best match d(y,, Uj), with 2y E X. SMDS minimisos the loss:
C= E (1lW(,;-;))) - d(y). d(y).,,)))² v2)
(1)
d(yt, Uj) is tesk-depn and implicitly defines the hypothesis structure. For example,
d(y, }1) :2 sin (i min (d.y. 1- 6,,)) 8y = 1yl-
(2)
these two formulas represent the chord distance between two points on a unit circle, thereby defining a circular structure. As shown in Figure 2 SMDS finds clear circular projection of calendar dates, consistent.
with Engels et al. (2025) findings.
We assess the quality of recovered projection W trained on activations X by computing a variant of normalized stress (Amorim et al. 2014), adapted for a supervised task. In particular, we compute stress over held-out set of points X, 0 and corresponding ideal distances d, = d( iy):
s= Σ [ W2yll d:j] 2
(3)
This metric measures how well distances in the recovered projection align with those of hypothesized manifold. High-dimensions activations that originally exhibit a particular geometric structure can often
For readability, omit space tokens in the examples. Tokenization is still performed usual.
We provide a review of relevant works in A
4

---

Published in Transactions on Machine Learning Research (02/2026)
be projected into low-dimensional space that preserves this geometry, thereby yielding low stress. By comparing stress values across multiple distance functions, one can identify the best-fitting manifold. In practice, however, selecting a single best hypothesis can be challenging, as stress scores for different manifolds frequently cluster closely together. In such cases, additional evidence needed to discriminate among candidates, for example via statistical lesting, as performed in $5
We propose a set of distance functions for SMDS to detect a heterogeneous variety of manifolds. Seminal works have shown several instances of the idiosyncratie structure of feature manifolds. Notable examples include:
Distance Functions
Cyclical features form ring shape in the latent space (Engels et al.
2025
Numbers are compressed according to logarithmic progression { Alquibe Boj al. 2025
Years of the 20th century form a U-shaped structure (Engels et al.
2025
Modell et al. 2025
Categorical features visually form clusters corresponding to the vertices of a polytope (Park et al. 2024a);
Lastly, Gurnee Tegmark (2023) have extracted multidimensional manifolds representing features such as latitude and longitude.
Therefore, as listed in Table we parametrize shapes such as circles, semicircles, lines, logarithmic lines and clusters so that the resulting manifold interpretable. The manifolds we define are categorized based on their topology: linear, where concepts follow continuous, monotonic progression; cyclical, where the progression is continuous but wraps around to the starting point, forming loop; and categorical, where concepts occupy discrete, equidistant regions without inherent ordering.
In the following sections, we use this collection of distance functions to identify feature man- ifolds for several tasks and different stages of the reasoning process.
Table 1: Collection of distance functions used throughout our study. colors denote manifold topology: linear, cyclical categorical. di == y yj. M max(y).
4 Experimental Setup
Distance Function d(y,, Uz)
Resulting Manifold
||:|||
linear
Data & Prompt Setup Based on the TIMEX3 specification (Pustejovsky et al., 2010), we create five synthetic datasets and three variants, probing precise aspects of tem- poral understanding over a variety of numeri- cal quantities (Table 2p. All sentences across datasets have a similar format: they describe an action performed by three individuals, the action is associated with a temporal expression, and a continuation cue is attached to elicit temporal reasoning. The right answer is always one of the three names mentioned in the context. We randomize the names, actions, and temporal expressions to increase robustness but keep the same structure across all samples. Temporal expressions are sampled uniformly across a given range, but respecting some plausibility constraints (c.g. "once per year" is never associated with common actions such as "takes a shower"). We also make sure names are always tokenized as a single token for all models. The three variants seae_se date_temperan tee_teemrerrure, and time_of_dee _f share the same context and range with their main counterparts, but ask different question that requires a diffcrent type of reasoning (e.g. "The only person born in spring is"). Overall, our data exhibits greater variability than similar datasets used in previous literature. See Appendix B for an extended discussion on our temporal taxonomy, datasets and for the variability analysis.
log y: - log My
log_linear
2 sin 1 (6g))
semicircular
2 sin I log - log 1)
log_semicircular
s ( min(16.), (6g)))
circular
min((o:)]: M 1 16
discrete_circulat
0 if y = yj. otherwise
cluster
LM Selection The bulk of our analysis is performed on three models from different families: Qwen2.5 Instruct Team et al. 2025), Llama-3.2-3B-Instmee (Grattafiori al.] 2024).. gemma-2-2b-it (Gemma et al., 2025). We also study what impact instruction tuning has on these representations by comparing these models with their base versions. For the Llama family, also study larger models to observe whether the manifolds
5

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: Tasks and Corresponding Prompts. Variants date_season, date_ cest__temeraturee and time_of_ _ime_of_day___ee phase are omitted for brevity and are detailed in Appendix [B] colors represent templates: blue denotes names, orange denotes actions, red denotes the corresponding continuations, green denotes temporal expressions, and black denotes expressions that do not change throughout the dataset.
Dataset
Context
Continuation
Expression Range
date
Anna took bus on the 16th The first person that took 01/01 31/12 bus was
January.
duration
Neil is starting workshop the The person whose workshop 01/01 31/12
11th of January lasting day. ends first is
day 4 years
notable
Emma was born on the day Pius X The oldest is became Pope.
1900 2000
periodic
Kevin waters the plants every day.
The person who waters the daily every years plants more often is
time_of_day
Lucy naps 16:15.
It is 19:37. The last 00:00 23:59 person who napped is
identify persist at scale: (lama-3.1-$B-Instrvee and 1lama-3.1-70B-Instuul Due to computational constraints, we run these models using 4-bit quantization.
Generalizing Manifold Analysis We generalize our study by analyzing activations across all layers and at different positions along the sentence. In particular, we consider three sites: (i) the final token of the temporal expression (e.g., "on the 16th of January," abbreviated as TE); (ii) the final token of the prompt (e.g., "The first person that took a bus was," abbreviated as LP); and (iii) the token corresponding to the generated answer (i.c., one of the contextual names, abbreviated as A).
To systematize the hypothesis selection process, we drop any assumption about which manifold should correspond to which feature and instead rum grid search over all defined distance functions. We fit instance of SMDS for each dataset and layer and then compare recovered manifolds using stress (Eq. 3 Throughout the study, we choose m = 3 as it is the minimum number of dimensions required to represent all our hepot manifolds (1D) for most linear manifolds, 2D for some linear and cyclical ones, and 3D for clusters, which form a tetrahedron in 3D space). Increasing dimensionality yields similar results, which are discussed in Appendix D.4 Unless stated otherwise, all manifolds visualized in the study show the first two components identified by SMDS for the best-scoring layer, computed with a 50/50 train/test split.
To demonstrate the robustness of our analysis, we conduct statistical significance study using 10-fold cross- validation repeated 5 times across all datasets, models, and manifolds. Firsi, group observations by dataset, model, and manifold rank, and perform a Friedman test. Then, for all groups that achieve statistical significance (p < 0.05), we perform a post-hoc Nemenyi test to evaluate the significance of manifold ranks on a given dataset. To break any ties in manifold rank, we additionally perform bootstrapping with 500 iterations, followed by the same Friedman-Nemeent guarantees for the identified hypotheses. We define preferential manifold as one that (1) attains the best average score and (2) is statistically significantly different from all alternative hypotheses. Further details on statistical significance are provided in Appendix D.3
protocol as before. This yields even stronger statistical
5 Experiment Results and Analysis
We present experiment results around the three major findings in this section.

---

Published in Transactions on Machine Learning Research (02/2026)
Qwen/Qwen22.n 5-3 Insstruttt
poogle/gemma-22221 -2-bb-it
meta -ta.llam/aaaaaaaaaa -amam-...----nstteeoe
tng. Treer
SQ. SRBEE
Tgg semidrruiar
Irear
date season
semicinuu
samicroular
Strcllor
2.5 5.0
mo
Iog CANSICIIII
droularr
Stustar
liner
periodic
Sicular
Figure 3: Feature manifolds retrieved from the site. We can observe that models represent features in a similar way, and the resulting manifolds interpretable and match intuitive progression (lincar, circular categorical) of the underlying features. The scatter plots the left show the first two components of SMDS dimensionality reduction; the bar plots on the right depict scoring of different manifolds on the given activations. Scores displayed are computed as log S to emphasize the difference between values; error bars are shown in black. Bar plot color reflects manifold topology: linear; cyclical; categorical;
5.1 Temporal Entities Share Intuitive Manifold Structures Across Models.
Figure 8 and Table 3 show the best-scoring manifolds across models and tasks. We first observe that all manifolds identified this way are not only interpretable, but also match prior research (Engels et al. 2025 Park et al. 20244a AlquBoj et al. 2025). Their topology always matches meaningful properties of the feature they explain: monotonic features are represented by lincar topologies, cyclical features wrap around in loops, and categorical features map to cluster structures. Notably, the best manifold shape is consistent across all observed model familics as well as most of the hon-instruction-tuned counterparts. Morcover, this pattern persists at scale, with all three observed sizes (3B, 8B, 70B) creating coherent shapes between them. This suggests there are preferential ways to encode the same knowledge, and all language models eventually converge to similar structures, providing further proof of hypotheses formulated in previous literature Huh
et al.
2024
Previous work has shown that LMs encode numerical quantities in a logarithmically compressed way [AlquBoj ct al. 2025). Our work extends this finding to temporal reasoning for the first time: in both the duration and periodic tasks, time intervals such as days to weeks, weeks to months, and months to years are preferentially represented with roughly uniform spacing, indicating logarit compression of temporal magnitude.
7

---

Published in Transactions on Machine Learning Research (02/2026)
Critical difference diagrams across tasks
periodic - TE site Layer? 7
date
daily
1
3
5
four times,a a weeks
00
date_seasee
1
3
5
7
every days
five times,a month
notable
tthree times month
3
4
5
6
7
every weeks
six times year
every & months
four four times year
periodic
3
5
every years
circ
disc_circ
log_lin lin log_ pg__emic
semic
clust
lin
Figure 4: 3..-33..-----trrr on the periodic task. Events display logarithmic compression in their fre- quency: long intervals (e.g., months, years) are rep- resented with the same granularity as shorter ones (e.g., days, weeks).
Figure 5: Avg. rank of manifolds across all models over 500 bootstrapping iterations. Horizontal bars show groups of statistical equivalence. Best-ranking manifold is always statistically different from others.
This pattern (Figure 4) though not directly comparable, bears superficial resemblance to the logarithmic compression described by the Weber Fechner law (Dehaene 2003). We note that the labels we use are themselves logarithmically spaced. A deeper study into temporal understanding is therefore needed to clarify whether the compression we observe is a genuine emergent. behavior or an artifact of our synthetic dataset.
Many tasks exhibit high scores for more than one topology. The cross-validation setup is not sufficient to break all ties on manifold rank, therefore we perform bootstrapping with 500 iterations. Figures 5 and 17 show the overall ranking of manifolds across tasks confirming the results obtained in the previous analysis and establishing clear best-ranking hypothesis for cach task.
This analysis both validates SMDS as a manifold analysis tool and underscores the difficulty of identifying a single best-ranking manifold for a given problem. One plausible explanation that, while preferential manifolds do exist, models often construct multiple valid representations whose disentanglement requires
Table 3: Best-scoring manifold, corresponding average stress, and accuracy for different models and tasks in 10-fold setting with 5 repetitions. Standard error shown in grey.
date
date season
date temperature
duration
log
Manifold
Ace
log
Manifold
Acc
log S
Manifold Acc
log
Manifold
Aoc
Llama-3.1-70BIIT
2.909
10.07
cire
0.93
6.047
10 091
clust
0.83
6.219
10.
log semic 0.82
3. 3.194
1: 025
log lin
0.62
1--8-IT
2.483
020
0.39
6.628 030
clust
0.66
6.018 #0.0as
clust. 0.49
2.990
+0.044
log lin
0.09
Llams-3. 2-3B-IT
2--B-IT
2.502
00.022
cire
0.81
7.031
+0.029
clust
0.74
6.525
20.000
clust
0.51
3 12124 20.00.
log
lin
0.30
Qweu2.5-3B
03:011
0.36
8.502
20-66
clust
0.26
7.518 20.019
log semic
2.805
100000000000
log
lin
0.32
gemma---22b.
2.969
cire
0.38
7.408
1o
clust
0.29
7.207 20.00s
clust 0.36
3.390
log lin
0.26
Llama-3, 3,2-3B
2.116.10.0 022
disc circ
0.39
6.780
20-05
clust
0.53
6.702220.0.0
log semic
0.37
2.879
1:0 029
log lin
0.21
Qwen2.5-3B
2.797 10 059
cire
0.21
8. 8.43
oo 041
clust
0.55
7.254_o..00 10.050
log lin
0.23
2.959
L. o2s
log lin 0.17
gemma-2-20
1412+ 142+0.080
0.31
7.358
023
clust
6.918 =0.060
clust
2.721 +0.004
log lin
0.11
notable
periodic
time day
time
day phase
log S
Manifold Ace
log S
Manifold Acc
log 5
Manifold
Acc
log S
Manifold Acc
Llama-31170B-IIT
2.63840.m 071
semic
0.18
3.858
o52
log lin
0.60
1.482 20.015
circ
0.53
6.182 070
clust 0.73
Llama-3.1-8BB
2.270
semic
0.31
3. 882 +0.030
log lin
0.28
1.298
1.298400.08
+0.018
cire
0.12
6.731
+0.020
clust 0.69
Llama-3.2-3B.
2.192 022
semic
0.49
20.0222
log.
1.278
20.014
circ
7.072
220.22
clust
0.66
Qwen2.5-3BBTT
1.96440.or 40.020
semic
0.32
3. 3.80440..00
log__in
lin
0.32
1.140
#0.014
cire
0.19
8.803
+00
clust
0.30
gemma-2-2b-IT
2. 2.036 02.
aemic
0.58
2..929...
20.041
log
0.14
1.256=0.000
20.032
semic
0.07
7.589
+-a oss
clust
0.29
Llama-3.2-3R
1. 1.10340.111
dise__circ cire
0.01
3.600
:::00=0..00 oas
log lin
0.33
I. 1.284
=0.000
cire
0. 10
7.061
+0.02s
clust
0.64
Qwen2.5-3B
1.76440.04s
0.08
3.436 0327
log
0.18
1.287
20.012
cire
0.24
8.711
+50000
clust
0.45
gemma-2-2b
1.423 215
log lin
0.01
3.7669 77691...000
log
lin
0.30
20.011
cire
0.18
7.599
2000000
clust
0.60
8

---

Published in Transactions on Machine Learning Research (02/2026)
TE Zit See Sere
Structure layers
Allre wee bate the 18th of May.
Target Taken
Ansave
Last of temp scor
Last of prompt
The oldent foee tottr
bem
oild sarth
Lope1
34
06
66
Normllized Layer Depth
Figure 6: Feature manifolds of Llamma3.2-3B-Instruu on the date task and its variants. Best-scoring layers shown, identified in Section | Different continua- tions produce drastically different topologies.
Figure Manifold quality at different layers and po- sitions in the sentence. Information transits from its injection point (orange) to the answer token (blue).
substantial computation. This representational polymorphism is not an artefact of SMDS: control tasks with randomized labels exhibit consistently high stress, confirming that SMDS does not simply overfit a hypothesized manifold. We provide a more detailed discussion in Appendix D.7
5.2
LMs Adapt Structures In-Context for Different Tasks.
This section describes two observed phenomena in which LMs reshape manifolds across tasks and depth.
Figure shows how the LM adapts the TE site feature manifold to different structures the LP site, depending the question prompt. Tasks date, date_season and date_ sate_ttemerature all start from the same context but result in strikingly different final structures: in date, circular structure is required to account for the looping nature of dates in year, while in the other two tasks inputs are mapped to linearly separable clusters. This can be interpreted as the model internally performing regression or classification to solve the task.
When comparing the location in the sentence where the structure is located, models exhibit a form of information flow between entities, which can strengthen certain manifold structures, degrade others, or even drastically change their shape as observed earlier. Figure 7 shows how to detect this flow with stress. In initial layers, the TA site is highly structured. As layers progress, this structure disperses into later tokens, such the LP loken and the A token. This process is not perfect: duplicated manifolds on LP and display noticeably higher stress than the ones found at the TE site, A possibility is that later tokens in the same sentence accumulate more contextual information than carly ones, thus resulting in noisier manifolds. Our results extend previous findings on the existence a binding mechanism in LMs (Feng & Steinhardt 2023 Dai et al. 2024]] ; we show that not only vectors, but entire feature manifolds are preserved and propagated between entities.
5.3
LMs Actively Use Feature Manifolds for Reasoning.
Here prosent two causally relevant lines of evidence that LMs actively use the structure of their represen- talions to perform temporal reasoning.
Located subspaces are causally relevant to noise perturbation. To demonstrate that feature manifolds are utilized by LMs in their reasoning process, we perform causal intervention by adding noise to the manifold subspace and measuring downstream accuracy. We inject Gaussian noise ~ W/(0, o² (m)) into the first layer at the TE site. Given a hidden state z e Rd, the perturbation is applied as = x + W-1e, where e an m-dimensiooal noise vector projected back into the original space. Subspaces of dimension m are located via SMDS in the usual way, and overfitting is prevented by training and evaluating the SMDS on 50/50 split. We select the top three task-model pairs achieving the best. accuracy the original task,
9

---

Published in Transactions on Machine Learning Research (02/2020)
Post-intervention Accurar date
Manifolds before and after intervention
Layer
Layer
0.8
0.7
0.6
Intervention Type
SMDS
SMDS 5500)
0.5
Random (m m
Random 500!
(m 3072)
0.4
Ns Intervention
0.5
1.0
5.0
10.0
$0.0
1:tt lan
33st Dec
Figure 8: Downstream accuracy of Llama-3.2-31B Instruct at increasing levels of noise. Error bars represent standard error. Intervening on low- dimensional manifold subspace is just as effective at degrading model performance as perturbing much larger activation spaces.
Figure 9: Llama-3.2-2 In--unst on the date task. Latent space of the LP token before and after applying noise on the TE token (top and bottom respectively). Interventions on carly tokens cause disruptions to the manifolds of later ones.
these will be the scttings where a disruption will be more noticeable: date, date_season and time_of_dee -aalood_aaaaaaa day_p on Llama-3.2-3B-Innttruu
Across all tasks, performance gracefully degrades as the noise scale is increased (Figure) 8) Crucially, we observe degradation for low suggesting that temporal features concentrated in very small yet highly informative regions of the activation space. We perform two other types of intervention in which we inject noise in the full latent space and in a random subspace, respectively. Affecting the full latent space achieves much more destructive effect for low values of o². On the other hand, disrupting random subspace has no detectable effect on performance for subspaces of size < 100. The addition of noise also results in the disruption of structures located at subsequent tokens and layers (Figure D). Interestingly, later layers are still able form vaguely organized shape, meaning information is partially being propagated reconstructed. Our choice of perturbing the first layer is empirically motivated by the fact that intervention later layers did not show strong effect. We hypothesize this is because information propagates quickly across tokens and layers, therefore the model is able to reconstruct. manifold from context tokens even if its source token has been disrupted. Overall, our experiments confirm that SMDS-located subspaces are critical for temporal understanding.
Manifold quality significantly correlates with model performance. We find a significant positive correlation between downstream accuracy and the ability of models to form well-organized manifolds, as quantified by log S (Spearman's p = 0.513, p 0.0174; Pearson's relationship emerges only for models that attain above-chance accuracy, specifically, Llamm--3.23I--Instrudd Llama-3.1-8D --nsttsrttt, and Llama-3.1-70B- Instruut. The results suggest that while feature manifolds tend to emerge naturally in LMs, a critical factor for strong performance lies in how effectively the model utilizes them during reasoning.
0.560, p 0.0083). Notably, this
Table 4: Stress for the duration task at the LT site evaluated with the linear manifold. Standard error shown in grey. Differences with a control task with randomized labels are all statistically significant.
Table 5: Stress values for the cities task at the RC site. The highest-scoring manifold is always a spherical onc.
Manifold log S flat
Model
Best Layer
log S
Control log
Model
Acc
cylinder
geodesic
sphere
Llama-3 -lama-3.2-BB-it 2-B-iit
0.549
2.071
1.931
2.118
2.285
20
2.173.+0.114
0.460 020
0.031
Qwen2.5-3B.cr
1.975
B-3BII
2.585 +0.047
0.506-2..22
gemma-2-25.5 2b-it
0.493
2.070
1.947
2.073
2.248
gemmm-22-2bit
2.595 26.
0.51910.011
0.031
10

---

Published in Transactions on Machine Learning Research (02/2026)
2D Manifold Durations
2D Manifold
Dates
150
22 months
Decambere
150
$12 months
Decembera
Defombore 36 morchs
September
Dočambers
Saptember
100
days
100
36 month?
days
Component
50
0
21yeang 2el
lungh NoVembbrr
March
days*
50
24years
June November
March days
AAgri April 25 days? days March 211dgys
April April
-0
rebnuary
550
February angll 1
Siddys days
woth 18 manths
Marchi
-100
-100
21yayys
-150
-150
-200
-20
-600
-400
-200
200
400
-600
-400
-200
0
200
400
Component
Figure 10: Multidimensional manifold constructed by Llama-3.2-31 2-3t--ru on the duration task. Component 0 is proportional to the duration, component 1 to the day of the year.
5.4 Generalizing SMDS Across Domains and Feature Types
Defining manifolds through distance functions enables extending SMDS beyond the mono-dimensional case. This sections provides two such examples.
2D Manifold We analyze the duration task in more detail as cach sentence contains two temporal expressions. We use both the duration and starting day to define a 2D label, and run SMDS with the linear hypothesis as it is the only distance function supporting multidimensional labels. Doing so reveals 2D manifold that displays properties from both features. Table presents Wilcoxon p-values obtained by comparing the model to control task, showing they are significant across models. We hypothesize the creation of such manifolds happens during the information flow discussed earlier: features are retrieved from multiple locations and combined. The recovered manifold is shown in Figure 10
Spatial Reasoning Domain To demonstrate the versatility of SMDS beyond temporal reasoning, we apply it to task grounded in geographic knowledge. We construct dataset of prompts referencing various cities around the world and use their latitude and longitude to compute pairwise distances and reconstruct a manifold. While Gurnee & Tegmark (2023) deemonstrate that geographic location is decodable from LMs' hidden states, their analysis is limited to a planar projection. We extend this by evaluating spherical, cylindrical, and geodesic- based geometries, and find that a spherical manifold best captures the structure of the representations. This again highlights how feature manifolds align with the true geometry of the underlying domain. Further details are provided in Appendix D.6
6 Discussion & Conclusion
This paper introduces SMDS, a model-agnostic method for identifying and testing subspaces with a specified geometry via a linear projection. SMDS is not intended as a replacement for existing dimensionality reduction techniques; rather, it addresses a specific gap in their use: the ability to isolate subspaces which representations conform a hypothesized geometric structure. In SMDS, the target structure is specified through a user-defined distance function, in contrast to methods such as LDA PCA where geometric patterns arise indirectly from the optimization objective. As result, the method tests candidate geometric hypotheses rather than discovering entirely new ones, and its offectiveness depends on the quality and breadth of the typotheses considered. By evaluating a sufficiently broad set of candidate structures, however, SMDS can help narrow down plausible manifold hypotheses. As a supervised approach, SMDS additionally requires labeled data and the ability to define a mcaningful geometric structure over those labels. Finally, as with other supervised methods, care must be taken mitigate overfitting, for example through appropriate regularization and well-motivated selection of candidate geometries.
11

---

Published in Transactions on Machine Learning Research (02/2026)
Our study establishes connection between the geometry of representation manifolds and the causal language modeling process, demonstraaing that a structured organization of knowledge is not only present but beneficial for model reasoning. By analyzing the persistence of these structures across tokens particularly from the injection point to the answer-we we provide compelling evidence that feature binding operates through continuous, lask-relevant manifolds in the latent space. The persistence of manifolds across tokens suggests that language models transfer not just vectors, but structured representations, reinforcing the presence of a binding mechanism and extending prior evidence to more diverse tasks (Dai et al. 2024).
Although our experiments center temporal reasoning, the proposed method extends to any task involving structured features on which a distance function can be defined, as we demonstrate in $D.6 Starting from hypothesis manifolds inspired by prior work, we obtain consistent, interpretable results, effectively reframing manifold analysis as a model selection problem. A compelling direction for future research is understanding how individual features combine into multidimenniooaal manifolds. While we present initial evidence of composition, more expressive manifold hypotheses could offer deeper insights. SMDS lays the foundation for such investigations
Our stress metric often yields tightly clustered scores. The approach followed in this study is to drastically increase experimental observations via repetition or bootstrapping to hone in on single leading manifold for each task. Future studies could adopt different approaches: the development. of more discriminative metrics, the use of larger and more varied datasets, and complementary intervention experiments such as the one in $5.3 There is a final option, however, which is to reassess the assumption that a single preferential manifold exists. An intriguing hypothesis is that models instead adopt multiple equally valid representational geometries and dynamically select them based on task context. Failure to isolate a single best manifold should signal that new hypotheses must be formulated, potentially accounting for the coexistence of multiple representational geometries.
In the scope of model reasoning, hypothesis-driven manifold analysis can serve as a basis for several lines of future work. For instance, combining SMDS with circuit discovery (Conmy et al. 2023) could help identify which operations LMs use to transform information throughout reasoning. Another promising direction is model steering (Park et al.) (024bb), where knowledge of feature manifolds could inform methods that leverage these structures directly. Finally, systematically studying the rolc of noise in feature manifolds across layers, and whether miligating it improves reasoning, offers another rich line of inquiry.
In sum, shape happens. Our work lays the foundational ground for interpreting and comparing representations in LMs through geometric structures. This invites further exploration into how manifold shapes are formed, combined, functionally employed in downstream reasoning, and how knowing about them could improve existing models.
Limitations
Our use of language models trained on predominantly English corpora introduces an inherent bias toward the cultural norms of the Anglosphere. This is reflected in several design choices: the reliance on the Gregorian calendar for date expressions; the selection of names that are tokenized as single units, which tends to privilege Anglo-American names; and assumptions about seasonal properties (e.g., associating December with cold weather), which implicitly expects the location to be country in the northern hemisphere, with a temperate or continental climate. The high accuracy and well-formed manifolds observed in these settings can therefore be seen as indicators of such biases. SMDS could find use diagnostic tool, uncovering how underlying representations reflect these biases.
In our work, we omit fuzzy expressions for which it is not possible to define precise temporal pointers (e.g., "in the morning," " "later," and "next week") and therefore an exact location on a feature manifold. As Kenneweg ct al. (2025) show, fuzziness a temporal expression is a key factor in performance degradation. Future works could better characterize the interplay between fuzziness in temporal expressions and the quality of feature manifolds.
12

---

Published in Transactions on Machine Learning Research (02/2026)
Acknowledgments
Federico Tiblias is supported by the Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA) through the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research.
This work has been funded by the LOEWE Distinguished Chair "Ubiquitous Knowledge Processing, LOEWE initiative, Hesse, Germany (Grant Number: LOEWE/4a//519 005/00000000 (00.002(0002)881) by the German Federal Ministry of Education and Research, and by the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Rescarch Center for Applied Cybersccurity ATHENE.
We thank Alireza Bayat Makou, Andreas Waldis and Hovhannes Tamoyan (UKP Lab, TU Darmstadt) for their feedback on an early draft of this work. We also thank Marco Nurisso (Politecnico di Torino) and Anmol Goel (UKP Lab, TU Darmstadt) for the many insightful discussions that helped shape this paper.
References
Guillaume Alain and Yoshua Bengio. Understanding Intermediate Layers Using Linear Classifier Probes. The International Conference on Learning Representations (ICLR), 2017. URL https://opeerevieeee net/
Forum?id=HJ4-raVLL
James F. Allen. An Interval-Based Representation of Temporal Knowledge. In International Joint Conference
Artificial Intelligence (IJCAI), LJCAI'1, pp. 221 226, 1981. URL https://www..smantiicco Bappr/AAfSSfo66666////6662022000
V. AlquBoj, Hilal AlQuabeh, Velibor Bojkovic, Tatsuya Hiraoka, Ahmed Oumar El-Shangiti, Munachiso Nwadike, and Kentaro Inui. Number Representations in LLMs: A Computational Parallel to Human Perception, 2025. URL https://arrii.org//b///2222
Elisa Amorim, Emilio Vital Brazil, Luis Gustavo Nonato, Faramarz Samavati, and Mario Costa Sousa. Multidimensional Projection with Radial Basis Function and Control Points Selection. In IEEE Pacific Visualization Symposium, pp. 209-20 296216, 2014. URL https://ieeexxplore ieeeeerreeeee.ee.eee.eeeeeeeeeee ieee. www.::eeeeeeeeeoeeoooooeee ortp//ccummn///77719999
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. On the Cross-lingual Transferability of Monolingual Representations. In Annual Meeting of the Association for Computational Linguistics (ACL), 2020. URL https://aclaanhologyy org 2020 maammmii..2...
Yonatan Belinkov. Probing Classifiers: Promises, Shortcomings, and Advances. Computational Linguistics, 48(1):207-215 2022. URL https:://claantolloo org/
2022. c1-1.7/
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. What Do Neural Machine Translation Models Learn about Morphology? In Annual Meeting of the Association for Computational Linguistics (ACL), pp. 861-872, 2017. URL http:///accaantooooooor///////
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Halffeld-Dodds.. Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards Monosemanticity: Decomposing Language Modcls with Dictionary Learning. Transformer Circuits Thread, 2023. URL https://transformer--iin wtt.iiiiisssuuteree..ece
Tyler Chang, Zhuowen Tu, and Benjamin Bergen. The Geometry of Multilingual Language Model Repre- sentations. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 119-136,
2022. URL. https://aclanthollle org. wwnlp-main main.9
David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, Satvik Golechha, and Joseph Bloom. A Is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders, 2025. URL
https://arriv.org/abb///aa 14507
13

---

Published in Transactions on Machine Learning Research (02/2026)
Wenhu Chen, Xinyi Wang, William Yang Wang, and William Yang Wang. A Dataset for Answering Time-Sensitive Questions. In Newral Information Processing Systems Track on Dalasets and Benchmarks (NeurlP' D&B), volume 1, 2021. URL http://s//aatttseeeeeeeeaaaaasss
neurips.cc/paper. pdf
Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang. Ming Liu, and Bing Qin. TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models. In Annual Meeting of the Association for Computational Linguistics (ACL), pp. 2008, 2024. URL https://aclanthoooogy wtw.///.........o// hrg../...........1ooo..........
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tai, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, et al. Scaling Instruction- Finetumed Language Models. J. Mach. Learn. Res. (JMLR), 25(1):70:331170:3333 2024. ISSN 1532-4435.
URL https://doi.orgg/
48550/arXiv 2210. 11416
Arthur Commy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alouse Towards Automated Circuit Discovery for Mechanistic Interpretability. In Conference on Neural Information Processing Systems (NeurIPS), 2023. URL https://doi. org/ /0.. 48550/arXiv 2304. 14997
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. Emerging Cross-lingual Structure in Pretrained Language Models. In Annual Meeting of the Association for Computational Linguistics (ACL), pp. 6022-6034, 2020. URL httpss//aclaanthoooog.ooo orgp//00....//00 acl. mollmainn.. 536
Qin Dai, Benjamin Heinzerling, and Kentaro Tnui Representational Analysis of Binding in Language Models. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 17468 17493, 2024.
URL
wwtp2222.222.....mmmmmmmmm www.44mmmmmppmm....mmmmmm main.9677
Stanislas Dehacne. The Neural Basis of the Weber-Fochht Law: A Logarithmic Mental Number Line. Trends in Cognitive Sciences, 7(4):145 147, 2003. ISSN 1364-6613. URL https///www.ccieeeddreee... com/
Ahmed Oumar El-Shangiti, Tatsuya Hiraoka, Hilal AlQuabeh, Benjamin Heinzerling, and Kentaro Inui. The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Sub- spaces. In Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), pp. 350-561, 2025. ISBN 979-8-89176-190-2 URL (tLLhttps: / https///aaaaanolooorrrgg.. wwwpooollooo.og//22000 naacl-short.47 47/
Joshua Engels, Eric J Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. Not All Language Model Features Are Die-Dimensionally Linear. In The International Conference on Learning Representations (ICLR), 2025. URL
Foin Farrell, Yeu-Tong Lau, and Arthur Conmy. Applying Sparse Autoencoders to Unlearn Knowledge in Language Models. In Neurips Safe Generative Al Workshop, 2024. URL httpps://oeenreiiew.eet wwwpwviiiiiiieeooouun
Bahare Fatemi, Mchran Kazemi, Anton Tsitsulin, Karishma Malkan, Jinycong Yim, John Palowitch, Sungyong Seo, Jonathan Halcrow, and Bryan Perozzi. Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning. Tn The International Conference on Learning Representations (ICLR), 2025. URL
Jiahai Feng and Jacob Steinhardt. How Do Language Models Bind Entities in Context? In The Interna- tional Conference on Learning Representations (ICLR), 2023. URL httppss//oggrrev.ee..ee///ooooo zb3b6oK077
Lisa Forro, Laurie Gerber, Inderject Mani, Beth Sundheim, and George Wilson. TIDES: 2003 Standard for the Annotation of Temporal Expressions. Technical report, 2003. URL httt::::/aapps.dic.mmm/ece//ee
Erecms/AD1110911/
14

---

Published in Transactions on Machine Learning Research (02/2026)
Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mchrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. Bias and Fairness in Large Language Models: A Survey. Computational Linguistics, 50(3):1097 1179, 2024. URL https:/aallanhhooggg.oo
Gemma, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vicillard, Ramona Morhej, Sarah Perrin, Tatiana. Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean-bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Tvo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, et al. Gemma 3 Technical Report, 2025. URL
http://wwwwggbb//////ea/ 19786
Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, and Mark Crowley, Multidiimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey, 2020. URL https://arxxiv.or//bbb//
08136
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, and Dinesh Manocha. A Closer Look at the Limitations of Instruction Tuning. In International Conference on Machine Learning (ICML), pp. 15559 15589, 2024. URL https: org/ 10. 48.4 http://axxxv22222.05555 2402.05119
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Alunad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Authony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, et al. The Llama 3 Herd of Models, 2024. URL
21783
Wes Gurnee and Max Tegmark. Language Models Represent Space and Time. In The International Conference on Learning Representations (ICLR), 2023. URL https:://penrreview.. www.:neet/formmiid-jeaaaaaa
Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. Finding Neurons in a Haystack: Case Studies with Sparse Probing. Transactions on Machine Learning Research (TMLR), 2023. ISSN 2835-8856. URL
Jiaojiao Han, Wujiang Xu, Mingyu Jin, and Mengnan Du SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Models, 2025. URL htttps://axxi..ogg/bb//// 20820
John Healy and Leland McInnes. Uniform Manifold Approximation and Projection. Nature Reviews Methods
Primers, 4(1):82, 2024. ISSN 2662-8449. URL http:///www..rrrrooooooooo////00
wwwp::/wattte.cccooo 040-2/ees//44336--2---033333333 0024-003333
Benjamin Heinzerling and Kentaro Inui. Monotonic Representation of Numeric Attributes in Language Models. In Annual Meeting of the Association for Computational Linguistics (ACL), pp. 175-195, 2024.
URL https:://rxiin
John Hewitt and Perey Liang. Designing and Interpreting Probes with Control Tasks. In Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing FEMNLP-IJCNLEP pp. 2733-2743, 2019. URL http://///aaaaathooooooooo//////
John Hewitt and Christopher D. Manning, A Structural Probe for Finding Syntax in Word Representations. In Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 4138, 2019. URL htttp:://aclantholooe
1419
Yihuai Hong, Meng Cao, Dian Zhou, Lei Yu, and Zhijing Jin. The Besooning-Memmrizzateie Interplay in Language Models Is Mediated by a Single Direction. In Findings of the Association for Com- putational Linguistics (Findings of ACL), pp. https:///cclantholog. org/ wwt.:o////222..nnnnnnn-lllll Findings-acl.1111))
2025. ISBN 979-8-89176-256-5, URL
Quzhe Huang, Yutong Hu, Shengqi Zhu, Yansong Feng, Chang Liu, and Dongyan Zhao, More than Classification: A Unified Framework for Event Temporal Relation Extraction. In Annual Meeting of the Association for Computational Linguistics (ACL), pp. 8631-9648 2023. URL Ittpss//aclanthholey..rrg 2023. acl-aalllonnn -lng. 536
15

---

Published in Transactions on Machine Learning Research (02/2026)
Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse Autoencoders Find Highly Interpretable Feat in Language Models. In The International Conference on Learning Representations (ICLR), 2023. URL httpps://ppenreviews http:///pppmeiiieeeeeeefrrrriiio www.:e//oorummiiifffeeBLL..L
Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The Platonic Representation
Hypothesis. In International Conference on Machine Learning (ICML), pp. 20617 20642, 2024. URL
httpss//proceddngs.m... mlr...mmlr.... www..//.........eee////////..a..
Ganesh Jawahar, Benoit Sagot, and Djamé Seddah. What Does BERT Learn about the Structure of Language? In Anmual Meeting of the Association for Computational Linguistics (ACL), pp. 3651-3657, 2019. URL
https://aclanthollee htwph/oooooooooggggggg.. wwtg/P19-122 1356
Zhen Jia, Abdalghani Abujabal, Rishiraj Saha Ray, Jannik Strötgen, and Gerhard Weikum. TEQUILA:
Temporal Question Answering over Knowledge Bases. In ACM International Conference on Information and Knowledge Management (CIKM), pp. 1807-1810, 2018a. URL
org/10. 1155/326226
3269247
Zhen Jia, Abdalghani Abujabal, Rishiraj Saha Roy, Jannik Strötgen, and Gerhard Weikum. TempQuestions: A Benchmark for Temporal Question Answering. In Companion Proceedliggs of the The Web Conference (WWW), pp. 1057 1062, 2018b. URL https://ddi..rrg/10 014.11.111111311455555 3191536
Zhen Jia, Soumajit Pramanik, Rishiraj Saha Roy, and Gerhard Weikum. Complex Temporal Question Answering on Knowledge Graphs. In ACM International Conference on Information & Knowledge Management (CIKM), pp. 792-802, 2021. URL https::///ii.org/// / 0T0//4456677.884444
Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers? In International Conference on Computational Linguistics (COLING), pp. 558-573, 2025. URL https:///cclanthologg org/2025,rr cwwinn-maii.iinn3n
Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda. Are Sparse Autoencoders Useful? A Case Study in Sparse Probing. In International Conference on Machine Learning (ICML), 2025. URL /ttp:///////ooooeeeeeee///////e/
Svenja Kenneweg, Jörg Deigmöller, Philipp Cimiano, and Julian Eggert. TRAVELER: A Benchmark for Evaluating Temporal Reasoning across Vague, Implicit and Explicit References, 2025. URL https:
01325
Amit Arnold Levy and Mor Geva. Language Models Encode Numbers Using Digit Representations in Base 10. In Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), pp. 385-395, 2025. ISBN 979-8-88176-190-2. URL httpssss/aaaaaaaoooooyoooroor/ org/2222 naacl-short.33)
Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Emergent World Representations: Exploring a Sequence Model Trainod on Synthetic Task. In The International Conference on Learning Represenlations (ICLR), 2022. URL https://oppnrevieww net/ forum?id=De607_Tccl
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg, Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. In Conference on Neural Information Processing Systems (NeurIPS), 2023. URL https://open:: eeeeppenrrview..mm.. wtt.iinnet/forrmmi-aaalllypssss
Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yojin Choi. The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning. In The International Conference on Learning Representations (ICLR), 2023. URL https: //openreview.n www.:ewwee//orruue dwww.oee
16

---

Published in Transactions on Machine Learning Research (02/2026)
Raja Marjich, Veniamin Veselovsky, Thomas L. Griffiths, and Ili Sucholutsky. What Is a Number, That Large Language Model May Know Tt?, 2025. URL http:///aaxxi...ggabb////// .0540
Samuel Marks and Max Tegmark. The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of Frue/Falls, Datasets. In Conference on Language Modeling (COLM), 2024. URL
Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models. In The International Conference on Learning Represenlations (ICLR), 2024. URL https://openreview... net/
forrm?Pid11e822IIDY
Gouki Minegishi, Hiroki Furuta, Yusuke Iwasawa, and Yutaka Matsuo. Rethinking Evaluation of Sparse Autocncoders through the Representation of Polysemous Words. In The International Conference on Learning Representations (ICLR), 2024. URL http:://ppeeeeeeeeee.eeeerrrrrrs
Alexander Modell, Patrick Rubin-Delanchy, and Nick Whiteley. The Origins of Representation Manifolds in Large Language Models, 2025. URL hnttss://ariii httpss///arx..oor//b///5555 18235
Jingcheng Niu, Wenjie Lu, and Gerald Penn. Does BERT Rediscover Classical NLP Pipeline? In
International Conference on Computational Linguistics (COLING), pp. 3143-3153, 2022. URL https:
//aclanthology. org wwwgg//22..oco coling-1.22
Jingcheng Niu, Saifei Liao, Victoria Ng, Simon De Montigny, and Gerald Penn. ConTempo: A Unified Temporally Contrastive Framework for Temporal Relation Extraction. In Findings of the Association for Compulational Linguislics (Findings of ACL), pp. 1533, 2024. URL https::/aclaadee httph//acaaaatooogy.yooooooo..// .gggg org/ 2024. 1.44.finding findings-acl. acl. 89/
Jingcheng Niu, Xingdi Yuan, Tong Wang, Hamidreza Saghir, and Amir H. Abdi. Llama See, Llama Do: A Mechanistic Perspective Contextual Entrainment and Distraction in LLMs. In Annual Meeting of the Association for Computational Linguistics (ACL), 2025. URL https://doi. org/10. wrgg//..4550/ark/t.. 2505 09338
Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. The Geometry of Categorical and Hierarchical Concepts in Large Language Models. Tn ICML, Workshop on Mechanistic Interpretabilil, 2024a. URL
Kiho Park, Yo Joong Choc, and Victor Veitch. The Linear Representation Hypothesis and the Geometry of Large Language Models. In International Conference on Machine Learning (ICML), volume 235 of ICML 24, pp. 39643 39666, 2024b. URL https: http:///doooor//0.../ 48550/arXiv.... 2311.03658
Qiwei Peng and Anders Søgaard. Concept Space Alignment in Multilingual LLMs. In Conference on Empirical Methods Natural Language Processing (FMNLP) pp. 5511 155526, 2024. URT. https:///aclantholegg org/2024. 11:.24..mnlp-mam comnlp-main.33 315
James Pustejovsky, Kiyong Lee, Harry Bunt, and Laurent Romary. ISO-TimeML: An International Standard for Semantic Annotation. In International Conference on Language Resources and Evaluation (LREC'10), 2010. URL http://ww..recc conf org. www.://ewwc....oororrrreeen////e prrcceddings//eee200///////////
Roser Saurí, Jessica Moszkowicz, Bob Knippen, Rob Gaizauskas, Andrea. Setzer, and James Pustejovsky. TimeML Annotation Guidelines Version 1.2.1. 2006.
Andrea Setzer. Temporol Information in Newswire Articles: An Annotation Scheme and Corpus Study. PhD dissertation, University of Sheffield, 2001.
Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeffrey Wu, Lucius Bushnaq, Nicholas Goldowsky- Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Isaac Bloom, Stella Biderman, Adrià Garriga-Alonso, Arthur Conmy, Neel Nanda, Jessica Mary Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, William Saunders, Fric J. Michaud, Stephen Casper, Max Tegmark, David Bau, Eric Todd, Atticus Geiger,
17

---

Published in Transactions on Machine Learning Research (02/2026)
et al. Open Problems in Mechanistic Interpretability. Transactions Machine Learning Research (TMLR), 2025. ISSN 2835-8856. URL
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. Large Language Models Can Be Easily Distracted by Irrelevant Context. In International Conference on Machine Learning (ICML), pp. 31210 31227, 2023. URL https://proceedings.. wwl.gggs...r.... press//
v202/shi23a, html
Ju Juan Luis Suárez-Diaz, Salvador García, and Francisco Herrera. A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms, Experimental Analysis, Prospeets and Challenges (with Appendices on Mathematical Background and Detailed Algorithmss Explanation), 2020. URL https://arxiv.org/abl
1812.05944
Varshini Subhash, Anna Bialas, Weiwei Pan, and Finale Delei-Velez. Why Do Universal Adversarial Attacks Work on Large Language Modols?: Gcometry Might Be the Answer. The Workshop on New Frontiers in Adversarial Machine Learning, 2023. URL
Qingyu Tan, Hwee Tou Ng, and Lidong Bing. Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models. In Annual Meeting of the Association for Computational Linguistics (ACL), pp. 14820 14835, 2023. URL https::/aclanthhlleee wwwp://...gg./222..... acw.:22....cloooon.. 828/
Qwen Team, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, et al. Qwen2.5 Technical Report, 2025. URL https::/aara https:///axxvv.rr/aaaa..a atgggabb//212.111111
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Danicl Freeman, Thcodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Somnet. Transformer Circuits Thread, 2024. URL 2024/scaling wtw.riinnnoooontniiiiittottyooie wwwpttttiiiniiiine html
lan Tenney, Dipanjan Das, and Ellie Pavlick. BERT Rediscovers the Classical NLP Pipeline. In Annual Meeling of the Association for Compulational Linguisties (ACL), pp. 4593 4601, 2019. URL https:
/aaclanthology.oo www.tthoooogyooog/P1111 p
Nya. Trofimov, Daniil Cherniavskii, Eduard Tulchinskii, Nikita Balabin, Evgeny Burnaev, and Serguei Barannikov, Learning Eypology-Preserinng Data Representations. In The International Conference net
Learning Representations (ICLR), 2022. URL
Eduard Tulchinskii, Daria Voronkova, 1lya Trofimov, Evgeny Burnaev, and Serguei Barannikov, RTD-Lite: Scalable Topological Analysis for Comparing Weighted Graphs in Learning Tasks. In The International Conference on Artificial Intelligence and Statistics (AISTATS), 2025. URL hnttps://openret net/ forum?id==j0T]O0633
Laurens van der Maaten and Geoffrey Hinton. Visualizing Data Using I-SNE. Journal of Machine Learning Research (JMLR), 9(86):2579228) 2605, 2008. ISSN 1533-7928. URL http:///mmr.dt org/ org/papers//
w9/vandermaatend0a.htt
Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. Aligning Large Language Models with Human: A Survey, 2023. URL https:
//arxiv.org/abs//300.. 12966
Yuqing Wang and Yun Zhao. TRAM: Benchmarking Temporal Reasoning for Large Language Models. In Findings of the Association for Computational Linguistics (Findings of ACL), pp. 6389 6415, 2024. URL
https://aaclantoollgg..oor//// findings-ach 382.38//
18

---

Published in Transactions on Machine Learning Research (02/2026)
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetumed Language Models Are Zero-Shot Learners. In International Conference on Learning Representations (ICLR), 2021. URL /ttuttssss//peeeeeeieee/////rorr htt.:///ooooooiiiiorrrroooooooo
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Fd H. Chi, Quoc V. Lc, and Denny Zhou. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In International Conference on Neural Informalion Processing Systems (NewrIPS). pp. 24824 24837, 2022.
Svante Wold, Michael Sjöström, and Lennart Eriksson. PLS-regression: A Basic Tool of Chemometrics. Chemomelrics and Intelligent Laboratory Systems, 58(2):109 130, 2001. ISSN 0169-7439. URL https:
//ww.ssciencedrect..comm wttp://ecct.cccceiiccee//eeeeec/ htt.://cw...ccc.cccccccc.cccac/a
Xuansheng Wu, Wenlin Yao, Jianshu Chen, Xiaoman Pan, Xiaoyang Wang, Ninghao Liu, and Dong Yu. From
Language Modeling to Instruction Following: Understanding the Behavior Shift LLMs after Instruction Tuning. In Conference of the North American Chopler of the Associalion for Compulational Lánguistics (NAACL), pp. 2341-2369, 2024. URL htttp:://allattiooggg.o/////// -or-o//.........noogoggg 130/
Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. In International Conference on Machine Learning (ICML), 2025. URL https: net/forumPiddk22ckkJJyy
Chenhan Yuan, Qianqian Xie, and Sophia Ananiadou. Zero-Shot Temporal Relation Extraction with ChatGPT. Tn Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pp. 92-102, 2023.
URL httpss:/aaclanttoooogo.rrr////// wrw./o/g....///2333//3 http://oog....////222...oo...... bionlp-1.7]
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi BERTScore: Evaluating Text Gencration with BERT. In International Conference on Learning Representations (ICLR), 2019. URL
https://openrevieww.rr wwtp://wwww....toeooooieeieeeace
Ben Zhou, Daniel Khashabi, Qiang Ning, and Dan Roth. "Going on a Vacation" Takes Longer than "Going for a Walk": A Study of Temporal Commonsense Understanding. In Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019. URL https://eclaattoooogyyor/////
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: Less Is More for Alignment. In Conference on Newral Information Processing Systems (NeurIPS), 2023. URL https://openrevieew 2okKokkk22
19

---

Published in Transactions on Machine Learning Research (02/2026)
Contents
Introduction
1
2
Feature Manifold Analysis
3
3
Supervised Multi-Dimensional Scaling
4
Experimental Setup
5
5
Experiment Results and Analysis
6
5.1
Temporal Entities Share Intuitive Manifold Structures Across Models.
7
5.2
LMs Adapt Structures In-Context for Different Tasks.
9
5.3
LMs Actively Use Feature Manifolds for Reasoning.
5.4
Generalizing SMDS Across Domains and Feature Types
Discussion & Conclusion
11
A Extended Literature Review
21
B Temporal Taxonomy Datasets
23
B.1 Data Variability
25
C Supervised Multi-Dimensional Scaling
26
D Supplementary Experiments
28
D. Model Performancd
28
D.2
Additional Observations on Manilold Analysis
28
D.3
Establishing Statistical Significance
32
D.4
Sensitivity Number of Components m
34
D.5
Additional Observations Intervention
35
D.6
Identifying a Spatial Manifold
35
Feature Manifolds are not Artelacts of SMDS
36
D.8
Exploring the Impact of Instruction Tuning
37
20

---

Published in Transactions on Machine Learning Research (02/2026)
A Extended Literature Review
Linear Representations & Feature Manifolds The linear representation hypothesis proposes that language models encode interpretable features directions in their latent space, with concepts expressed
sparse linear combinations of these directions (Park et al., 2024b Modell et al. 2025 Recent work extends this view, revealing that related features tend to organize into structured manifolds. For example, ring-like ling-like structures [Engel et al., 2025), logarithmic progressions (AlquBoj et al. 2025). U-shaped curves (Engels)
et al. 2025 Modell et al.) 2025 clusters organized around the vertices of geometric polytopes (Park et al,
2024ap, and higher-dimensional surfaces (Gurnee & TTegmark] (003)). Other studies on multilingual LMS have also investigated structures in the latent space and consistently found shared representations across languages (Peng& & Søgaard 2024 AAtetxe et al. 2020 Chang et al.) 2022 Conneau et al., 2020 inter alia).
Beyond static feature geometry, recent work has shown that such linear and manifold structures also underpin model behaviors, including truthfulness (Marks & Tegmark 2024) and the interaction between reasoning and memorization (Hong el al. 2025
Existing Dimension Reduction Methods Several linear dimensionality reduction techniques have been applied to recover structure from language model representations: Principal Component Analysis (PCA) identifies dircetions of maximal variance in the embedding space (Gurnee & Tegmark 2023 Modell et al.
2025); Linear Discriminant Analysis (LDA) finds directions that best separate labeled categories (Park el al,
2024a]; Partial Least Squares Regression (PLS) identifies components that most strongly covary with target
labels Wold ct al.] 2001 El-Shangiti et al., 2025 Hcinzerling & Inuil 2024]; and Dillii-Dimensionall Scaling
(MDS) seeks low-dimensional embeddings that preserve pairwise distances from the original space (Marjich
2025). In addition to these linear methods, some non-linear techniques such as t-SNE and UMAP have also been applied (van der Maaten & Hinton 2008 Healy & Mclnnes 2024 Subhash et al. 2023). Other dimensionality reduction techniques may also yield promising insights, but to our knowledge have not yet been systematically applied to probing LM representations (Trofimvv el al. 2022 Tulchinskii et al. 2025).
et al.
Probes have become a widely used tool for analyzing the internal representations of language models. Typically, a probe is a simple classifier trained to predict specific linguistic or conceptual property from a model's hidden states. They have been employed to study range of linguistic features, including morphology and syntax [Belinkov et al. 2017 Hewitt & Manning, 2019), well broader aspects of neural network behavior (Alain & Bengio| 2017 Jin et al.] 2025). Other works has used probes to examine the sparsity of feature representations Gurnee et al. 2023), detect model truthfulness (Li et al. 2023 Marks & Tegmark 2024), and
uncover the representations of concepts such as world locations, temporal quantities and numbers (Gurnee & Tegmark 2023 Engels et al.) 2025: Levy & Geva 2025). However, probe-based interpretations remain debated as probe performance does not necessarily imply mechanistic use (Jawahar et al. 2019 Tenney et al.
2019 Niu et al.) 2022), a limitation less applicable representation-ggomety approaches that incorporate
causal interventions (Heinzerling & Inui 2024).
Another prominent technique used in interpretability works is the Sparse Auto Encoder (SAE), a neural network building a mapping from the dense activation space of a LM to a high-dimensional, sparse, latent space such that single neurons of a SAE represent atomic concepts (Bricken et al. 2023 Huben et al.) 2023 SAEs have been successful at recovering vast collections of monosemantic, interpretable features at scale (Templeton et al.] (2024), but have also found usage in unlearning (Farrell et al. 2024), detecting internal causal graphs (Marks et al. 2024) and identifying circuits (Minegishi al. 2024). Recent work has also explored post-hoc interpretation of SAE features themselves, for example through agentic explainer frameworks such as SAGE (Han et al. 2025). While SAEs have shown promise for LLM interpretability, they face substantial critiques and limitations that challenge their effectiveness and reliability. Representations ident by SAEs may fall victim of "feature absorption," complicating the disentanglement of atomic features (Chanin et al. 2025). In model steering, simple baselines have been observed outperforming SAEs (Wu et al. |225 Kantamneni et al.) 2025). Lastly, SAEs are expensive to construct as they require extremely large dimension of their latent space and necessitate in some cases billions of tokens for training. Their construction also makes them model-specific, preventing transferability (Sharkey et. al. 2025).
21

---

Published in Transactions on Machine Learning Research (02/2026)
In this work, primarily compare SMDS with other linear dimensionality reduction techniques. While non-linear methods and SAFs may offer valuable insights, we do not focus them here due to their high computational demands. Our goal is to enable scalable and systematic exploration of feature manifolds across models and tasks. This requires lightweight methods that can be efficiently applied in closed form, making linear approaches better suited to the scope of our investigation.
Temporal Reasoning Temporal reasoning refers to the ability to interpret and manipulate expressions that describe temporal informatior in order to determine when events occur how they relate temporally (Jia al. 2018a). Such reasoning tasks often require composing multiple temporal expressions to answer nuanced, time-sensitive questions.
Several datasets exist that seek to benchmark LMs across different facets of temporal reasoning. Some evaluate factual recall over time [Jia ct al. 2018b Chen et al. 2021 Jia et al. 2021), others focus on temporal understanding of real-world scenarios (Zhou al.
Fatemi et al. 2025 and yet others probe the temporal arithmetic capabilities of LMs (Tan et al. 2023). Lastly, some works have aggregated existing
2019
benchmarks in order to evaluate broader capabilities such symbolic, commonsense, and event reasoning
Wangg& & Zhao 2024
Chu et al. 2024
Existing benchmarks primarily assess overall performance complex tasks involving multiple temporal expressions and reasoning types. Simpler tasks focusing on specific types of temporal expressions, despite being foundational to temporal understanding, remain underexplored. To enable a mechanistic investigation of how language models process temporal information, homogeneous datasets that isolate specific facets of temporal expressions are required.
For example, dates (e.g. "May 2010"), times ("9 pm"), or temporal relations ("before," " "in the morning")
22

---

Published in Transactions on Machine Learning Research (02/2020)
Table 6: Possible temporal expressions for each task.
Dataset
Temporal expression set
cities
Uniformly sampled based on location from the World Cities Database, considering only prominent cities cities with 10.000 inhabitants for US and Canada.
date, date_season, date_ tempertreer
Uniformly sampled from all 365 days of non-leap year.
duration
way date, durations uniformly sampled from fixed set: 1 day, 2 days, 3 days, days, days, 6 days, 7 days, 8 days, 9 days, 10 days, week, weeks, 3 weeks, 4 weeks, days, days, 14 days, 21 days, 25 days, 30 days, month, 2 months, 3 months, 4 months, 6 months, months, weeks, weeks, weeks, 10 weeks, year, years, 3 years, 4 years, 12 months, 18 months, 24 months, 36 months.
Dates sampled in the
notable
Uniformly sampled from a fixed set, extracted from Wikipedia. Omitted from brevity, full dataset available in the code repository.
time_of _day,, time_of. daadddyppaaeee.
Action time uniformly sampled from all hours :00, :15, :30, :45. Reference time sampled uniformly from all times of the day.
B
Temporal Taxonomy & Datasets
This section describes the synthetic datasets have generated to probe atomic aspects of temporal understanding.
Taxonomy Various annotation schemes have been developed to characterize temporal expressions such as TIMEX3 (Pustejovsky et al. 2010), TIMEX2 (Ferro el al. 2003), TIMEX (Sttzrr 2001 and TimeML (Sauri et al. 2006), as well as several variants. We take inspiration from TIMEXI-3 to construct several synthetic datasets. Each one covers a specific family of temporal expressions (Table 2
date: Refers to a specific calendar date. To explore periodic reasoning, we omit the year;
time_of_day _ay: Specifies precise moment in the day;
duration: Defines a duration and its starting point;
periodic: Refers to events that recur with given frequency;
Contains an indirect but precise reference to an event taking place in a given moment in time.
notable:
The taxonomy has been defined in such way that temporal expressions have a unique, precisely-definel associated numerical quantity. We have chosen to omit fuzzy expressions for which it is not possible to define precise temporal pointers (e.g. "in the morning", "later", "next week") and therefore an exact position in a feature manifold.
Dataset Creation We build each sentence in the dataset by combining three contextual sentences and a termination that elicits reasoning. Each sentence contains name, action and temporal expressions which are all uniformly sampled from a given set. Names and actions have been generated via ChatGPT and checked manually to be consistently formatted and the resulting sentences grammatically correct. Names have been chosen so that they are not broken up into separate tokens. Temporal expressions of the notable task have been obtained from Wikipedia and have been rewritten via ChatGPT and checked manually to ensure consistence. For datasets, each sentence contains exactly one temporal expression. We chose not to
htttps://en... www.pp/eea...////ii///eitt...
23

---

Published in Transactions on Machine Learning Research (02/2026)
Table 7: Additional examples for each task.
Dataset
# Samples
Examples
cities
2000
Luke lives Boston. William lives in Toronto. Michael lives Cancún. The person who lives closest to Luke is
Mark lives in Leuven. Jack lives in Heidelberg. Dallas lives in Messina.
The person who lives closest to Mark is
date
1992
Brandon donated clothes the 29th of September. Bob donated clothes on the 31st of August. Jerry donated clothes on the 27th of September. The first person that donated clothes was
Matt visited new city the 22nd of February. Josh visited new city on the 14th of February. Frank visited a new city on the 1st of March. The first person that visited new city
date_season
2000
Emily mowed the lawn on the 8th of December. Blake mowed the lawn on the 30th of April. Walker mowed the lawn on the 27th of June. The only person that mowed the lawn fall is
Rose painted mural on the 16th of June. Robert painted a mural on the 13th of July. Martin painted mural on the 27th of July. The only person that painted a mural spring
tete_temperature
2000
Richard left for vacation on the 25th of June. Neil left for vacation on the 22nd of December. April left for vacation the 22nd of August. The only person that left for vacation in a cold month is
Jason returned from vacation on the 12th of February. Connor returned from vacation on the 21st March. Rachel returned from vacation on the 19th of October. The only person that returned from vacation in warm month
duration
3000
Maria starting their internship on the 15th of December and is set to run for 25 days. George is starting their internship on the 13th of December and is set to run for 14 days. Laura is starting their internship on the 3rd of December and is set to run for 1 week. The person whose internship ends first is
Hunter runs festival booth on the 27th of December staying open for 10 days. George rums festival booth on the 12th November staying open for 9 days. Connor runs a festival booth on the 20th of December slaying open for days. The person whose festival booth ends first
notable
2000
Robert was born on the day the MV Doña Paz sank. Maria was born on the day the independent State of Palestine born on the day the Dayton Accords were signed. The oldest is
proclaimed. Andrew was
Neil was born on the day Herbert Hoover was inaugurated as President. Leon was born on the day James Joyce published Ulysses. Alice was born the day Mandatory Palestine established. The oldest is
time_of_day
3000
Steve watches movie at 23:15. April watches movie at 11:45. Charlie watches a movie at 7:15. now 2:58. The last person who watched movie is
Charlie watches TV 4:45. Richard watches TV at 12:15. Steve watches
TV at 4:30. It is now 14:42. The last person who watched TV is
Lime__o__da_phaae__
2000
Leon goes for a walk at 6:15. Brandon goes for walk at 18:30. Matt goes for a walk at 5:00. The only person that goes for walk in the morning is
John writes in a journal at 4:45. Matt writes in a journal at 5:00, Luke writes in journal 20:45. The only person that writes in journal in the evening is
24

---

Published in Transactions on Machine Learning Research (02/2026)
include more, using composite expressions, to obtain cleaner feature manifolds. The only exceptions are the duration and _ime_of_day datasets that contains two. This was necessary in order to formulate non-trivial questions that require reasoning across time spans. The notable task not only requires comparing different expressions but also involves factual recall of events from paramctric memory. Variants date_season, date_tempent see__tepperatrre, and time_of oa_ day_p contain the same contextual sentences as the original lasks but a different termination that clicits a classification- based form of reasoning. Finally, we note that the number of examples per dataset varies. This is necessary ensure that, after filtering for correctly answered instances, sufficient number of activations remain for SMDS training. For consistency, require minimum of 500 correctly classified examples model-task pair and cap the number of activations used in manifold search at this threshold. See Table D and Table for a more extensive collection of templates and examples.
Table 8: BERTScore- variability statistics for our datasel compared to prior datasets commonly used in studies of representational geometry and the linear representation hypothesis. Our dataset exhibits substantially higher variability across similarity pairs.
Dataset
Mean Similarity
Std
Our datasets
cities 3way
0.8170
0.0217
date_3way season
0.8306
0.0244
date dlate_3way___ Sewway_tepeeraurrr
0.8480
0.0240
date 3way
0.8218
0.0250
duration 3way
0.7975
0.0311
notable owaable_3way
0.8099
0.0281
periodic 3way
0.8023
0.0302
time_ of day 3way_phase
0,8169
0.0265
time day 3way
0.8174
0.0257
Heinzerling Inui (004)) and
el-Shangiti et al.
(2025)
P569 birthyear
0.8582
0.0444
P570 death year
0.8485
0.0517
P625.lat latitude
0.8289
0.0489
P625.long longitude
0.8598
0.0409
P1082 population
0.8544
0,0430
P2044 elevation
0.8410
0.0445
Engels et al
2025)
days of week
0.9627
0.0171
month of year
0.9617
0.0144
B.1
Data Variability
Overall, as shown in Table g the datasct we created demonstrates higher variability compared to datasets curated by prior work related areas. In particular, we compute several BEITTScore-based (Zhang et al., 2019) variability metrics across all splits of our dataset and compare them against the dataset used in Heinzerling & Tnui (2024) and [El-Shangiti et al. (2025), well the dataset in Engels et al.) (2025), works that established the study of representational geometry and the linear representation hypothesis. We measure variability by computing BERTScore F1 for every possible pair of texts in dataset split, using the model's 's rredicced-reference pairs formed via 5000 randomly sampled combinations. The mean similarity is the average of these pairwise Fl scores, while the std is the sample standard deviation of the same set of scores.
25

---

Published in Transactions on Machine Learning Research (02/2026)
C Supervised Multi-Dimensional Scaling
In this section provide further details the dimensionality reduction method use throughout the paper, as well as highlight its differences and similarities with other techniques which served as inspiration.
Description SMDS is based on the assumption that points X E Raxd in the residual stream roughly lie on a leature manifold that can be parametrized with labels E Y with E [0. 1]. Distances on this ideal feature manifold are assumed similar to Euclidean distances in the residual stream. Formally, given activations X E Rx two samples Fa, 33 E X and lincar projection W Rmxd from the full space the manifold subspace, we assume that d(yi, U5) se [a; x1)]l- To find W, we can minimize Eq. 1 reported here for readability:
C= Σ (1IW(2,-) - xz)l1 d(yr.V,)))
The problem is solved as follows. First, ideal distances d(vi, Uj) between labels are computed and the squared distance matrix is defined as:
Dy = d(y,, vz)
(4)
Then, classical MDS is performed. Double centering applied:
H=I- 1-1--117 11 B 1 - n
(5)
B is eigen-decomposed and low-dimensional embodding Y is obtained:
B = VAV
(6)
Y = V
1/2 R"*"
(7)
with Vn the top cigenvectors and Am the corresponding cigenvalues. The embeddings Y represent the locations of data points in the parametrized approximation of the manifold such that - Yjll ~ d(yi,
The following steps perform regression to find a mapping from datapoints X to this subspace. We center and by subtracting their mean:
X =Y=Y-Y.
(8)
Substituting Eq.
I.WTWW W T
(9)
At the optimum W we get that X. W. Y.. Solving Eq. I is expensive, however we can approximate W by solving proxy problem:
W arg min
X Yall-
(10)
W
this is the same formulation as a linear probe, but with Y being computed from the label using MDS. A
solution is easily found:
W Y. X (X. X.)
(11)
A regularization term can be added to Eq 11 make the resulting projection more robust:
W = y X X X + +al) x
(12)
In all our experiments, set 0.1.
The embeddings Y represent the locations of data points in the parametrized approxination of the manifold (Figure 2 In principle, if such embeddings are already known, the preceding sleps can be skipped entirely. Computing arbitrary d(y,, 31) just gives more flexibility, By using SMDS to perform a search across several candidate hypotheses, the problem of identifying a manifold can be reduced to one of model selection: one only needs perform SMDS on several metries d(yi, y₁) or parametrized manifolds Y, and compare them using a quality metric like stress.
26

---

Published in Transactions on Machine Learning Research (02/2026)
Comparison to other methods SMDS is extension of MDS and uses it as part of the procedure. There is, however, a key difference in its use case: while classical MDS is unsupervised and only learns a lower-dimensional mapping that preserves Euclidean distances, SMDS first builds a distance matrix from labels and then uses to learn the actual projection via rogression. There are also differences in the stress metric we use (Eq. 3 classical normalized stress (Amorim al.) 2014) evaluates the error between distances in the original and lover-dimensional space; our formulation effectively does the same, but between the projected and the ideal subspace Y.
The first term in Eq. can be reformulated:
[[(a, xz)11 = (W x;²)) (W(ss, zz)), (z. - xz)) TT]]TTT T W(x; - -z)); (m--x;) - M(: xz),
with M E R=x" being positive semi-definite matrix. This is the squared Mahalanobis distance, widely used in Distance Metric Learning. In fact, many other dimensionality reduction techniques can be described as Distance Metric Learning algorithms (Suárcz-Diaz et al. 2020
Li al, Gurnee & Tegmark] 2023 inter alia). Some have successfully employed circular probes recover feature manifolds (Engels et al. 2025) and study other cyclical patterns such number encodings (Levy & Geva 2025 but to the best of our knowledge prior works have used MDS to build probes of arbitrary
SMDS is closely related to probes, which have been extensively used in prior works (Belinkov 2022
2022
shape.
27

---

Published in Transactions on Machine Learning Research (02/2026)
Model Accuracy
Model Accuracy
Accuracy
Sre. of city plane
Fime of day plaso
dee CANDON
Sine, of plone
CAM <RARST
DUR
Uree
DMC
Sinc dia
dat: laTRENaaC
day
largo SAFS
day
DOT porperdarc
paToci
Scrabon
perko::
dustoon
percoli
Vilolaratiirn Insract
Figure 11: Accuracy temporal tasks. Accuracy is low across the board, with only Llama models achieving above-chance accuracy, The 3B Llama variant is also shown outperforming the 8B one.
Time of Day Manifolds Qwen2.5-3B-Instrue.
Site: TE
Site: LP
Site: A
Figure 12: Circular manifolds on the time_of_day task. SMDS cannot find any structure on LP and A despite one being present at the TE site.
D Supplementary Experiments
D.1 Model Performance
We evaluate exact match accuracy across all tasks and models, finding that performance is generally very low, with only instruction-tumed models from the Llama family outperforming random chance. This is notable because, despite poor task performance, the models still produce well-defined feature manifolds. Among Llama models, we find the more recent Llama-3.2-3B 3.2--BB-nsttrue outperforms its 8B counterpart, while the 70B version displays stronger performance in almost all tasks (Figure I1I.
While stress is good indicator of performance, discussed in Appendix 5.3 we observe no significant correlation for the three models of the main analysis (Spearman's p - 0.034). We hypothesize that while most LMs effectively structure knowledge internally, some struggle to leverage it during generation. This might explain their lower performance. Another possibility that the specific wording of the prompt does not allow LMs to effectively recover information from context. In that case, chain-of-thought prompting (Wii ct al. 2022 may improve performance.
D.2 Additional Observations on Manifold Analysis
Manifold analysis reveals consistent patterns across model scales. As Figure 14 shows, larger models from the same family tend to converge to the same internal representations as their smaller counterparts. This suggests architecture and pre-training data play a pivotal role determining LMs representations.
We observe two instances where manifold analysis exhibits unexpected behaviors. On the time_of_day task, SMDS is unable to recover well-organized manifolds at the LPP site despite clear, preferential circular manifold
28

---

Published in Transactions on Machine Learning Research (02/2020)
Owenn/Qwen2.--38-nntttt
google/gemma-2222b-t
meta-llama/Llama-3. 2.33-------t
Semmi
tularr
Iinerr
Innear
Log linerr
Srialar
lncalar
Slaster
day
Tinear
Trew
lineer
sidrestar
dirculae
of day paa_ddaapssssppa phase
semitreetar
semicinestr
log wog.mnidrsitrr
Sig. TgL.enicrrsslrr
Tog. wog.anssdrrsaar
0.0
2.5
5.0
Figure 13: Additional manifolds for different tasks and models. Continued from Figure
3 error bars are
shown in black. Manifold topology: linear; cyclical; categorical;
being present at the TE site (Figure 12 noting that time_of_da sentences contain two temporal expressions in the same format instead of one. The two representations may interfere destructively, preventing their recovery. When also considering findings from DD5 is also possible that, for this specific prompt, the site is not storing any semantically relevant information. Future works could start from tasks such as this to characterize how multiple feature manifolds combine, and in which token is this information encoded.
The lack of transferability between the two sites can be explained by
On the date_teeperaauuee task (Figure 13) the clusters are correctly identified but the scoring yields unreliable values. This is expected when considering how distances are computed in the binary cluster scenario: two clusters can be modelled correctly by any hypothesis manifold, as there no order that can be enforced. This signals caution and suggests reverting to simpler probes evaluate binary features.
29

---

Published in Transactions on Machine Learning Research (02/2020)
meta-lamaaLiama-3.1
meta-llama/Lima--3
meta-llma//lama-3 2----mm--------nssttuu
Timee
date
circulor
circubrr
CICCUbT
3.0
2.5
9.2
5.0
date Tee.eeeeeee Temperature
circular
semicircalan
SIICINLL GROSE
crutt circuler
crett circuine
0.0
2.0
25 5.0
0.0
3.2
log SToEr
Iog Inner CURIRIC MERIIIC
notable
1og
lirear
periodic
time of day
time of day phose
THE
cluster
Cluster
Cluster
Figure Feature manifolds for models at different sizes.
The preferential manifold is consistent across categorical;
scales. Error bars are shown in black. Manifold topology: linear; cyclical;
30

---

Published in Transactions on Machine Learning Research (02/2020)
Owen/Qwen2.5
goagle/germa
meta -aaalaaaaaaaaaaaaa 332-3B
dnte
semicirsuler
Rurd deen
suckdean
date SEnSon
5.0
Bud Idean
sclidean
maad Meen
sumidrcus
saniddraater
dato
FIEE direudr
SrOTO circiarr
HEl direter
0.0
25
25 5.0
0.2
2.5
5.0
PUCO
notable
The circularr
0.5
0.5
0.0 3.5
penodic
time of day
circular
circular
00
5.5
1.5
3.5
1.0
time of day phase
2.5
Figure 15: Feature manifolds for base models. Geometries are consistent with the instruction-tumel counter- parts in most cases. Manifold topology:
linear; cyclical; categorical;
31

---

Published in Transactions on Machine Learning Research (02/2026)
1st ranked manifold score (aggregated across all models)
Obsert (-tog10 p]
notable
periodic
time, of day
time, of wwy.oaaadyyaooooooo paaydp
0.00 525
0.50
0.75
1.00
3,25
1.50
3.00 0.25
0.50
0.75
1.00
1.25
1.50
0.00
0.25
0.50
0.75
1.00
3.25
1.50
0.00
0.25
0.50
0.75
1.00
1.25
1.50
Expected (-log10
Figure 16: Q-Q plot of p-values obtained from a Nemenyi lest comparing the stress scores of the first ranked manifold with all others for a given dataset. For all datasets except for date tete_teeper,,uee, the best identified manifold scores significantly higher than the alternative starting, on average, from the 3th-4th one.
D.3 Establishing Statistical Significance
SMDS tends to return tightly clustered stress scores across different hypotheses, making it hard to identify a single best manifold. To overcome this issue, we resort to statistical testing to isolate a manifold that is statistically better than the alternatives.
First, we turn to a 10-fold cross-validation setup with repetitions in which SMDS is trained on folds and evaluated the 10th. We group observation across dataset, model, and rank of the manifold, and perform Friedman test. For all groups that achieve statistical significance (p 0.05) perform a post-hoc Nemenyi test to evaluate the significance of manifolds ranks on given dalaset. The resulting Q-Q plot in Figure 16 shows that for most tasks the manifolds ranked 1st performs comparably to manifolds ranked 2nd to 4th, and significantly better than the rest. For duration, periodic, and time_of_day this further reduces to manifolds 2nd to 3rd. The binary nature of temperaure produces very homogeneous scores across all datasets and therefore we are not able to verify any significance. Results are summarized Table 3 showing the best-scoring manifold for each task and model.
To further separate results, employ bootstrapping. We run 500 independent bootstrap iterations, each time training dataset obtained by sampling with replacement from the original data and evaluating performance on the corresponding out-of-bag samples. We perform Friedman and Nemenyi tests as before and finally draw critical difference diagrams across all datasets. We perform this analysis over all models save for Elama-3.1-70BB-
which we exclude due to compute limitations. As Table 9 shows, confidence intervals bccome very narrow and thus provide statistically significant results for all models and tasks. Figures 5 show the overall ranking of manifolds across tasks confirming the results obtained in the previous analysis and establishing clear best-ranking hypothesis for each task, the only exception once again being the date_temperature dataset.
Taken logether, The repeated cross-validatiion and bootstrap analyses provide consistent evidence that our conclusions are stable across data splits. Across both settings, statistical tests yield convergent significance patterns and identify the same top hypotheses for each task (with the expected exception of aate_temperature. where scores are inherently homogeneous). This agreement across two complementary resampling protocols supports the claim that SMDS manifold selection is robust to variation in the underlying data split.
32

---

Published in Transactions on Machine Learning Research (02/2026)
Table 9: Manifold scores for each model and dataset in a bootstrap setting with 500 iterations. Standard error shown in grey, best manifold bolded.
Dataset
date
date season date tempersture
duration - log S
notable log S
periodic loog S
time of dayltime of day phase
Manifold
S S
log S
- log S
log S
log S
Llama- 1aama-8.1-8--11
1-8B-IT
lin
1.733=0.c00
6.128
+0.007
15. 1.703-.0 007
1.626-0..om
1.912+0..04
2.137
=0.000
0.837 +0.000
6.05 +00007
log_ lin
1.310
6.060
00.007
5.703
2.708 000007
1,849
3.696 ::
0.679 66799+0.004 000
0.132 +:0 007
semic
2.005
6.288
20.006 000
5.703 1. 007
20-077
2.071 no4
1.250_o.o0s
1.075 10075:1000 003
6.353 3:00 006
log somic
1,879g
6.217 1
5.703
1.796
2.013
2.487
0.959 10
6.260 1:00
cire
2.210
004
5.98
+0,000
0.000
+:0 ooo
1.575
1.669
1.893
20..000.000 .000
1.248 =0 002
6.365 #:0 000
disc circ
2.032+00
0.002 3.048
5.703
007
1.349
1.876+
1.377-0.018
1.178.+0
3.032 +-0 002
clust
0.459 2. .002
6.421
00.005
5.703:2
1: 007
1.105
0.619 1:01992..000 0.002
1.127 20.002
0.886 003
6.539 0..39000000
Llama-3.9-33
3.--3B112
lin 1.8511g..oo
6.310
2.310.00.000
0.00
6.306 007
1.806-20.0s
1.808
1.805
20.050
0.720
6. 6.150.to. 007
log lin
0.000
6.226..0
100007 000
6.306
0 007
2.943 10.010 0..004
1.743g..000
0.0000
3.573 1. 0n5
0.399 1.0 003
6.463 1.0 007
semic
2.146.0.000 0.006
6.550 1:0 000
6.306 0 007
2.103 0.00
004 1.985
1.833_0
000
0.963 002
6.791 4:0 000
log_ semic
1.956--0.006
6.425
+0.000
6.306
2.345
1.921
+0.004
2.020
0.841 +0.002
6.638 (638++0.007
cire
2.268 :: ooa
6.515 g0.005
0.000
0.000
1.865 20.012
2..00
1.782=00.01
1.210
=0. 002
6.937 00s
disc cire 2.211 1.
2.916
1.0
6.306
1.726 10-019
1.78111..0m
1.321 10
1.138
2.955
clust
0.462 0..2.00.00 0011
6. 6.846
0..000 00.
6.306
1. 007
1.159 10.003
0.584 000
1.120 20.00000000 0.000
0.892 002
6.979 000000
Qwen2.5-P5 9-
lin 7.343 2.395
7.093
1.994=0.00s
1.715
1.859
0.557 003
7.816.to
log_lin lin
0.903
7.360 7.360+0.013 +0.011
7.093
2.719 00000
1.632+00.04
3.576 00000
0.532
+0.02+0.00 005
7.804=000030
semic
2.736=0.011
7.072 ..072+0.010
7,093-:0 010
2.097
1.890
004
1.920=0.011
0,813
8.1944+0. 009
log semic
2.0
7.534
7.093_0.010
2. 2929 10.008
1.822
2.131 20.010
0.713 003
8.06-1go. 009
cire
2.860
010
7.793
10 010
0.000 ooe
1.604 10.019
003 1.578
1.478
000
1.134 1o 002
8.250 1:0 00s
dise circ.
2.347 +0.00..00 0.011
2.893
50,002
7.093 2..3.00.010 010
1.24 +0.026
1.561
+:0.000
0.725
00.25
0.972 00G
2.858 4:0000000000 600001
clust
0.508+0.002
7 7.958 0.009
7.093 + 02.
1.147 +.002
0.620+0.000
1.023
1003--0.002
0.895
0.895++0.002
8.416
-0.00
gemmoo-28 8T
lin 2.007
6.408
10.000
6. 6.726 0.010
10.00
1.650 000
1.945
0.867 005
6.476 1:0 000
log lin
1.597:10.008
6.2419
:00000
6.726=00.10
3. 3.269 000005
1.661
3.655 :0:5::0.00 007
0.616 006
6.5:12+0.00 1.5-12+000000
semic
2.376g0.011
6.598
1.599810.000
6.726 _. 010
2.047 10-005
1.871
2.020
1.091
6.8-45 1:4.5_10.000
log semic
2.226.0.000 0.007
6.468 1.0 oos
0.726620.000 010
2.392 10.22 ons
0.0004
2.252=0.00 000
0.946 005
6.703-10.000
cire
2.605
.oo
6.490
0.000
1.48 4
1.616
1.782 0.027
1.184 1084-o..0 00.
6.92 +00000000 000
dise cire
2.078
o..00
2. 2.83/4
0..00
6.726=00.00
1.281 20000
1.601
:..000
1.523
1.523=0.044
1 1.023-:0.000 0.000
2 2.851 +:00 002
clust
0.589
o00
6.847 .ooo
6.726 0.10
1.189 10.002
0.768
2. 003
1.556
10.000000000 0.0003
1.114gg o03
7.036 g.
Llama-9.8--
1.451 +0.000
6.035 +0.008
6.401 +0..0o00 0..0TT
1.800 =o.os
0.521 +0.011
1.560
-0.00
+0.02
6. 1..4444.0000 009
log lin
0.952
5.909
6.401::0
2.717
0.0004
0.409
3.429
00.0008
0.639
6.232 ::0 030
semic
1.7112.0.005
6.303
2.303300.07 007
6.401 1:0 007
10-00100005
0.673
1.856
0.971 1: 003
6.54 1:0 005
log semie
1.533+00.00
6.1344+0. 000
6.401
1100++0..oo o.00
2.209 =o.o0s
0.574400008
2.041
0.008
0.84440.000
6.398 +-0 000
cire
1.862 :: 000
6,282,
10.000
0.000 20000.0.00
1.882 20.00s
0.879
1.760 =0
1.194 =o 002
6.676 ::0
disc
1.841g
1. oos
2.943
1.0 002
6.401
0 oor
1.727 100
0.696_0.000
1.291
g 012
1.155
1 002
2.9/3 1.0 001
clust
0.489
6.619 .0.0
6.401 oor
1.159
0.804 0..04=00.07 0.007
1.133
1.004
6.787 000077
Qwem22.5-B
lin 2.079.2.000
1..24go.00s
6.868 06.8200000
1.792.20...
20.006
1.160.go.bos
L8806_0.012
0.675
0.675-20.003 003
7. 7.835 -:3551:000 009
log lin
1.579_0.000
7.388 10.08s
6.86881
2.823 Lo
1.449
3.200
0.511 10
7.921 1:0
semic
2.383 0.000
7.705+0.000 007
6.868 1.68@.0.0 .0.0
2.138 10.0ns
1.592 007
1.403=0.001
0,916 003
8. 8.180 007
log semic
2.240.0
7.578
6,868 in one
2.318 1o-005
1.551
1.451
0,805
8.069 1:0 008
cire
2.522 1 005
7.682
1.000.00000 007
0.000 0.0 000
ono 1.873
1.340 2.0 004
0.646
00.
1.220 _o 002
8.204 1.0 00G
disc circ
2.003 0.000
2.855 #0.001
6.868 0.000
1.613
1.506
0.488
=0.026 .20
1.028440.00
2.862 1462+00000 000001
clust
0,518
7 7.935 0.007
6.868 6.8688.0.o
1.157 00.002
0.6-12
1.642220..000
1.306
0.937 +:
8.371
gemman-2226
lin 2.353
6.176_1o.010
6.449 0.011
1.627_0.006
0.609_0.020
1.601
_. 00.009
0.874
003
6.449 009
log__in lin
1.622200.008
6.064400.00
6.449 2.001
2.61
1-155000000
0.497 +0.06
3.462 +0 10.000
0.726 0.726.00.00
6.617400.00
semic
2.741::0.00
6.424440.000
6..49.0..011
1.856 :0.0s
0.8334000011
1.837=0.000
1.172200.000
6.807 008
log_somic
somic
2.543
6.274_00.000
6.449
011
2.090
0.675
2.109
1.032
6.725
cire
2.810 007
6.649 00.
0.000 ...00+0..00
1.501+0.006
0.842
1122220.013 013
1.319-0.000
1.404-0.00
0.0004
6.88 +-0 008
disc circ
2.191
2. 2,826-2.0oo 001
6.1119 011
1.287 0177:00000 20.009
0.680 0200
0.65720.01e
1.159
2.832 ::0 001
clust
0.518 oo2
6.907
1. 007
6.449 1.449..000 011
1.282 1. 003
0.796_.0.om
1.222 20.000 000
0.977 1. 002
7.024 L. 007
33

---

Published in Transactions on Machine Learning Research (02/2026)
Critical difference diagrams across tasks
date temperature
5
6
duration
4
5
6
time of day
5
time of day phase
5
cire
disc
log Jin log semic
somic
clust
ln
Figure 17: Additional critical difference diagrams showing avg. rank of manifolds across all models over 500 bootstrapping ilerations. Horizontal bars show groups of statistical equivalence. Best-ranking manifold is always statistically different from others, the only exception being the date_ temperaure dataset.
Strees dimensionaliit manifolds
Strees dimension ality models manifald)
24-
2.4
Manifold
Model
drouar
Qwen2.533
duiker
2.5
gemma:2 2b
2.
Owe<2 5-344T
sudtieean
gemma-2-2-I
22- semicrcular
treer
Loma 3.1
(og so.ssiicrrulrrr
Lamb 3.2 38
L.mma-32:33 38-81
Manifoid dimensionality [m)
Manifold dimensionallty (m)
Stress as a function of subspace dimension- ality m, grouped by manifold. The cluster manifold displays an upward trend while all others are stable.
Figure 18:
Figure 19: Stress as a function of subspace dimen- sionality m, grouped by model. Scores remain largely stable across models and dimensionalities.
D.4 Sensitivity to Number of Components m
We analyze the effect of the recovered subspace dimensionality m on the stress score by varying m e 32, 4, 50) and mcasuring stress across all tasks and models. Figure 18 shows that stress is largely stable dimensionality increases for nearly all manifold hypotheses. The main exception is the cluster hypothesis, whose stress increases with m. This behavior follows from its distance definition, which enforces equal pairwise distances between the |y| clusters: in m dimensions, at most m + 1 clusters can be embedded with equal separation, so tasks with more labels necessarily incur higher stress at fixed dimensionality. In contrast, the other manifold hepotheses do not depend on label cardinality and therefore show near-invariant stress across dimensions. This pattern is mirrored across models Figure 19 where stress remains consistent once the cluster case is excluded.
Based on these observations, we fix the SMDS dimensionality to = 3 throughout the study. This is the smallest value sufficient to represent all considered manifold hypothescs: lincar and cyclical structures require at most two dimensions, while cluster structures in tasks such as date_season and time_of_das of__d___a_aaa_______a_____aaaa_ involve four labels (lyl = 4) and thus require three dimensions to be embedded with equal separation. Increasing m beyond this threshold yields only minor changes in stress for all but the cluster manifold. We recommend choosing a single shared dimensionality to ensure consistent comparisons across manifold types and model settings.
34

---

Published in Transactions on Machine Learning Research (02/2026)
Post-intervention Accuracy date_ season
Post-intervention Accuracy time of _a_aaaphaae ppaase
0.0
0.7
Accuracy
Intervention Type
Type
SMDS (m 2]
SMDS (m 21
0.5
Random (m 2]
0.5
Random im 2)
Random
Random 500)
Full (m 3072)
Full (m 3072)
No intervention
0.4
No Intervention
0.5
1.0
5.0
10.0
50.0
0.5
1.0
5.0
10.0
50.0
(logzo scale)
(logus scale)
Figure 20: Additional accuracy plots from the intervention experiment. Error bars represent standard error. The time_of_eeee day task is the least alfected by all forms of intervention.
Cities (Site: reference city, Layer: 24)
South
horth Amoiaa
Curope
Arkk
Ocaana
Figure 21: SMDS of gemma-2-2b-it on the cities task. The recovered projection shows the relative position of continents. For the sake of clarily, the flat manifold is shown instead of the best-scoring spherical one.
D.5 Additional Observations on Intervention
Figure 20 shows how the time_of_day is the least affected by intervention, when perturbing the full latent space. We believe this is due to the specific formatting of time used: expressions such as 19:37 are tokenized as 19, :, 37, with the TE site corresponding to the minute part of the expression. For most examples, the hour sufficient to determine the right answer, and since that information is left untouched, the model is able to contimue with minimal disruption.
D.6 Identifying a Spatial Manifold
To show the flexibility of SMDS, we extend our analysis a spatial reasoning task. In the same vein Appendix B we build sentences composed of three statements "<name> lives in <city>." Then, prepend continuation "The person who lives closest to <name> is" to elicit reasoning. Names are sampled from the usual set, while cities are obtained from the World Cities Database We select only prominent cities as they are more likely to be present. in the model's memory. For the US and Canada we instead select cities with > 10.000 inhabitants, since following the provided labels results in severe undersampling. We then uniformly sample citics basod on location.
35

---

Published in Transactions on Machine Learning Research (02/2026)
Each city is characterized by its latitude and longitude coordinates C (lat,, From these, project cities on various shapes and compute the relative distance function. We investigate a flat plane, sphere, a cylinder, and a complex geometry defined by the geodesic distance between cities. The flat manifold is computed simply as the Euclidean distance between the two coordinates, same as the linear metric used before. For the sphere manifold, we convert each coordinate into a 3D point on sphere of radius as follows:
Φ radt,, A radians(lon.),
27 cos( cos(Ar),
D/ = cos(o)))in(())) sin(A:);
2; = (oin
Then the distance between two cities is the Euclidean chord length:
| (z. - x2)² + (ys - y,) + (z+ - 22)²
For the cylinder manifold, we map latitude to vertical height and longitude to angle around a cylinder of radius r. Each point is embedded as:
h = radians(lat,) 8, X; = radians(lon.),
22 = cos(Ar),
y; = r siin();)
2; = h4-
The chord distance is again computed as the Euclidean distance in 3D. For the geodesic manifold, we compute the great-circle distance between two cities i.e., the shortest path along the surface of sphere. We first convert latitude and longitude to radians and compute the differences:
Φ radians(lats); A; radians(lons).
AΦ Φ; O3.
AA = X Xy-
We then use the Haversine formula:
AΦ
AA
= sin 2
+ cos( cos(dr) sin?
|8yl r 2arcsin (Va).
This corresponds to the true surface distance between two points on the Earth, assuming a perfect sphere. addition to the usual LP and A sites, we analyze two more locations: the correct city (CC), corresponding to the final token of the city where the correct person lives, and the reference city (RC), referring the final token of the city where the person in the question lives. Both cities are drawn from the context statements.
Table 5 shows that the manifold achieving the closest fit is spherical one across all models. In Figure 21 visualize the projection recovered by SMDS and find clear clusters around the shapes of continents. Their relative position is consistent with their real-life location, but projecting a spherical manifold onto plane inevitably distorts their real position.
D.7 Feature Manifolds are not Artefacts of SMDS
In this section we validate the robustness of SMDS and confirm that the feature manifolds recovered are indeod consistent and not an artefact of overfitting a projection. Primary evidence is provided in the main analysis of $ the error bars produced by cross-validation are narrow for almost. all datasets, confirming that activations for given feature do have preferential manifold.
The second piece of evidence is obtained by designing control tasks following Hewitt & Liang 2009 We build control variants for all tasks by shulling the labels. This should make it impossible for SMDS to identify a structure and we should observe significant increase in stress. For each model-task pair, we evaluate the best manifold identified $5 As in the main experiment, we perform a 5-fold validation on the dataset. Table shows the results: absence of structure causes a sharp increase in stress (and corresponding drop in log S). This is evidence that SMDS does not force structure when underlying manifold exists.
36

---

Published in Transactions on Machine Learning Research (02/2026)
Table 10: Stress values for control tasks. Absolute difference with the base task is shown in red. SMDS consistently produces low scores if no structure present.
Baama-3.2-3B-Instrre
2wen2.5-3B-Instrre
gemma-2-2lh
Dataset
Best shape - -log S
Best shape
llog S
Best shape
-log S
date
circular
0.995 0.932
circular
0.855
circular
0.806 1.563
dlate_season
cluster
4.509
cluster
0.956 7.205
cluster
0.858 6.308
date temperature
cluster
0.376 5.669
cluster
0.332
cluster
0.211 -6.111
duration
log linear
0.513
log linear
0.519-2.063
log linear
0.445-2.059
notable
semicircular
0.722 1.248
semicircular
0.740 -0.801
semicircular
0.482 1.168
periodic
log linear
0,523
log linear
0.502 33110
log linear
0.235
time day
circular
0.987 -0.7-...20 0.223
circular
0.978 00.006
semicircular
0.683 00.468
time of day phase
cluster
0.961 447100
cluster
1.000
cluster
0.902 6.4--66477
D.8 Exploring the Impact of Instruction Tuning
Since we exclusively use instruction-tuned models in our main experiments, we are interested in whether instruction luning impacts the feature manifolds and accuracies of our models. Instruction luning is post-training method that is widely believed to enhance models' generalization and task-solving capabilities (Wei et al. 2021 Chung et al.) 2024), however not all of the potential changes induced in a basc model by instruction Luning have been explored. Various prior works suggest that instruction luning mainly impacts stylistic output tokens rather than changing the model's parametric knowledge [Zoou et al.) 2023 Ghosh
jet al. 2024 Lin et al. 2023), and that additionally causes models to rotate the basis of their representation space to adapt user-oriented tasks (Wu et al.) 2024). However, these works deal primarily with token probability distributions and do not explore feature manifolds.
To explore the impact of instruction tuning in experimental setup, conduct main experiments the base versions of three of our models: Llama-3.2-3B, Qwen2.5-3B, and Gemma-2-2B. We report the stress
values in Table and the feature manifolds in Figure 15
Overall, across our three models, we find that instruction tuning did not substantially alter the optimal structure. For all lasks except notable, periodic, and duration, the structure that was optimal for the instruction-tuned model tended remain in the top-3 optimal structures for the base model as well. We note that the three outlier tasks have monotonic topologics, while the rest have cyclical or cluster-like structures.
For some tasks, we observed that the feature manifolds for base models tended to be more scattered than models have tighter, well-formed ring structures than the manifolds the base models (Figure However, this was not the case for all tasks: for example, the periodic manifolds for both base and Instruction-tunel models showed a clear separation of clusters that remained consistent within particular model architecture.
those of the instruction-tuned models. For example, the date manifolds for all instruction-tun
(Figures 3 13 14h
Surprisingly, we found that the accuracies varied unpredictably between the base and instruction-ture models. For Llama, all base accuracies were much lower than the instruction-tuned accuracies, while with the Qwen and Gemma models, base models sometimes markedly outperformed the instruction-tuned models. This could possibly be due to differences in the instruction tunninntttnnnggnniin methods of these models.
Overall, our results suggest that the impact of instruction-tuning on fcature manifolds will depend on the task, model architecture, as well the specifics of the instruction-Luning process. A detailed exploration of this is promising avenue for future work.
37


Reviews:
[{"review_id": "M94ukvevZK", "reviewer": "Reviewer_RQND", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper uses an extension of Multi-Dimensional Scaling (MDS) called Supervised MDS, to automatically discover the best manifold representation with specific geometric properties for a language modeling task of temporal question answering. The experiments show that across different model sizes and architectures (tested with Qwen, Gemma, Llama family of models), these representations have consistent structures. The experiment also finds that language models utilize different structures when tasked with different types of reasoning. Perturbation analysis on these feature manifolds further show that the language models indeed utilize these feature manifolds for reasoning tasks.\n\nStrengths:\n- Clear set of contributions and well-organized content\n- Detailed description of experimental setups\n- Supporting experimental results\n\nWeaknesses:\n- In Figure 5, the standard in which the visualized layers are selected among all other layers is not clear. If they are cherry-picked, further justification is needed. \n- In Section 5.3, the justification behind the injection of noise into the first layer is lacking.", "claims_and_evidence_yn": "Yes", "claims_explanation": "All main claims are mostly supported by the experimental results. \n\n- The first claim (F1) is supported by the experiments on Gemma, Llama, Qwen family of models in Figure 3 and section 5.1.\n- F2 is supported by Figure 5 mainly (although a bit more details on the layers visualized may be needed). \n- F3 is supported by the perturbation analysis in Section 5.3 (although further justification of the noise choice may be needed).\n\nPoints two and three are further mentioned in Requested Changes; other than these the claims are supported by the experimental results provided.", "audience_yn": "Yes", "audience_explanation": "Individuals interested in mechanistic interpretability and geometric structures of manifolds learned by LLMs may be of interest. Temporal reasoning is the task of interest in this paper, so researchers interested in extending to more general reasoning tasks may also be interested in this direction. Also compared to more “famous” methods like SAEs, this method is much more compute efficient, so interpretability researchers with stricter compute budgets may be interested.", "requested_changes": "- Critical: as mentioned in the Weaknesses part, more justification behind the specific choice of layers visualized in Figure 5 is needed. Are they selected based on the stress value, like done in Figure 3? What do the manifolds in other layers actually look like?\n- Critical: as mentioned in the Weaknesses part, more justification behind the choice of noise injection layer as the first layer is needed. Why not inject it to the layer that has the best stress value?\n- Recommendation: The concrete results of section 5.4 are all essentially presented in the Appendix. For the paper to be more self-contained, a suggested edition would be to either (1) move the entire section to the appendix and briefly mention the extension to multi-dimensional case in the main part of the paper, or (2) include all figures in the main body of the section so that the results and figures can be better referred to with the main text.\n- Recommendation: I think the train/test split details for F1 and F2 is missing. How many data points were held out to compute the stress?", "broader_impact_concerns": "No impact or ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YoUNJHEwr1", "reviewer": "Reviewer_q9Y9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel interpretability method named SMDS (Supervised Multi-Dimensional Scaling) for automatically discovering feature manifolds in the activation space of Large Language Models (LLMs). SMDS frames manifold discovery as a supervised learning problem, learning a linear projection that maps high-dimensional activations to a low-dimensional space to match a pre-assumed geometric distance (e.g., \"circular\" or \"linear\").\n\nStrengths:\n\nQuantitative Comparison:\nThe main contribution of this paper is providing a unified framework and a quantitative metric (Stress value) that allows researchers to compare the goodness-of-fit of different geometric assumptions. This differs from previous methods that rely on single, fixed assumptions (e.g., PCA, LDA).\n\nCausal Evidence ($\\mathcal{F}_{3}$):\nThe paper’s most compelling finding is in Section 5.3. Through causal intervention (injecting noise into the specific low-dimensional subspace found by SMDS), the authors demonstrate that the LLM relies on these minimal subspaces for reasoning, whereas perturbing random subspaces has little or no effect.\n\nDynamic Analysis ($\\mathcal{F}_{2}$):\nThe paper presents suggestive evidence that manifold structures may be dynamically adjusted depending on the task (e.g., date vs. date_season).\n\nWeaknesses:\n\nLimited External Validity:\nThe study depends heavily on synthetic datasets with highly uniform syntactic structures. This limitation makes it unclear whether the discovered manifolds reflect abstract temporal concepts or merely overfitting to specific syntactic templates.\n\nMethodological Simplification:\nSMDS functions more as a “hypothesis-driven” linear probe—it tests pre-defined manifold assumptions rather than discovering them automatically. It also struggles with less clearly defined temporal concepts (such as “morning”), as acknowledged in the limitations section.\n\nData Design Effects:\nSome findings, such as “logarithmic compression,” may partially arise from the experimental data design rather than from the model’s internal representations. For example, the label categories (“daily,” “weekly,” “yearly”) are themselves logarithmically spaced.", "claims_and_evidence_yn": "No", "claims_explanation": "Several of the paper’s key claims appear to rely on experimental designs that may introduce confounding factors, leading to conclusions that are less convincing than intended.\n\n1. Limited Generalization Testing\n\nFixed Syntactic Templates:\nAll experiments are based on synthetic data with very similar formats (e.g., [Name] [Action] on [Date]).\n\nConcept vs. Template:\nThe claim that models “converge to similar structures” ($\\mathcal{F}_{1}$) does not yet demonstrate convergence to general conceptual representations. Instead, it may reflect that when all models are tested on nearly identical templates, they naturally converge to similar internal shortcuts.\n\nThe paper does not yet provide evidence showing whether these manifolds persist under varied syntactic structures (e.g., “The day ... was January 16th”).\n\n2. Methodological Simplification and Over-interpretation of Results\n\nTreating “seasons” as independent clusters may represent a simplification that overlooks their intrinsic cyclical nature. The date_season task currently treats seasons as uncorrelated clusters, ignoring the cyclic order (Spring → Summer → Autumn → Winter). While this design simplifies the experiment, it may also weaken the correspondence between task structure and the intended goal of identifying intuitive and meaningful properties.\n\nInsufficient Evidence for 2D Manifold (Fig. 14):\nSection 5.4 claims the discovery of a 2D manifold for “start date” and “duration.”\n\nMethodological Clarification:\nThe paper would benefit from a clearer explanation of how the “cyclical” and “log-linear” features were combined into a single distance function.\n\nThe evidence in Figure 14 appears to challenge the stated interpretation: the relationship between Component 0 and “duration” seems closer to inverse than proportional, with considerable variance in the data points (e.g., “3 days” appears left of “7 days”).\n\nConflating Memorization and Reasoning:\nThe “spherical manifold” observed in the cities task may primarily reflect retrieval of memorized geographic information rather than abstract reasoning.\n\n3. Narrative Inconsistency in $\\mathcal{F}_{2}$ Findings\n\nThe $\\mathcal{F}_{2}$ finding contains an internal inconsistency.\nOn one hand, Figure 5 suggests manifolds are “reshaped” (e.g., “circle” becomes “clusters”).\nOn the other hand, Figure 6 suggests they are “preserved and propagated” (e.g., “circle” becomes a “blurred circle”).\nThe discussion appears to merge two distinct mechanisms—“transformation” and “propagation”—that might be analyzed more clearly as separate phenomena.\n\n4. Contextual Interference in TE Activations\n\nThe analysis relies on extracting activations from the TE (Temporal Expression) position to study “information flow.” However, in Transformer architectures, the activation at the TE position inherently integrates contextual information and may not represent purely temporal content. This contextual mixing makes the interpretation of “manifolds being preserved and propagated” ambiguous.\nThe “blurriness” of the manifolds in Figure 6 (higher Stress) could arise from stronger contextual interference during later-stage aggregation. The authors could discuss this issue more explicitly.", "audience_yn": "Yes", "audience_explanation": "Yes. Mechanistic interpretability remains a central topic in machine learning research.\nThe paper’s question—how LLMs represent abstract concepts internally—is of strong interest to the TMLR audience.\n\nThe causal intervention approach ($\\mathcal{F}_{3}$) is technically rigorous and could inspire future work.\nDespite certain methodological limitations, the SMDS framework as a “quantitative probe” and the phenomena it highlights are likely to spark useful discussion and follow-up research.", "requested_changes": "Critical:\n\nSyntax Generalization Test:\nTo strengthen the main claim, the authors are encouraged to test whether the identified manifolds remain stable when using sentences with different syntactic structures but the same temporal meaning. This would directly assess generalization beyond the fixed templates.\n\nRe-evaluate the “Season” Manifold:\nRe-analyze the date_season task using a more appropriate distance function (e.g., discrete_circular) to reflect the cyclical order of seasons, rather than treating them as unordered clusters.\n\nClarify the 2D Manifold Methodology:\nProvide a clearer explanation in Section 5.4 of how the “cyclical” and “log-linear” dimensions were combined into a single 2D distance function. Without this clarification, the results in Figure 14 remain difficult to verify.\n\nStrengthen:\n\nClarify the $\\mathcal{F}_{2}$ Narrative:\nResolve the apparent contradiction in Section 5.2—does the model “propagate” or “reshape” the manifold? Treating these mechanisms separately could improve clarity.\n\nTone Down Claims:\n\nThe finding from the cities task might partly reflect knowledge retrieval or memorization rather than genuine reasoning. Testing with fictional or inconsistent city names could help clarify this.\n\nIt would also be useful to note explicitly in the main text (not only in the limitations) that SMDS currently cannot capture fuzzy or context-dependent temporal concepts, which limits its scope.\n\nThe considerable variance among data points in Figure 14 should be discussed, even if it arises from contextual ambiguity.\n\nFinally, the authors should acknowledge that the labels used are themselves logarithmically spaced, meaning the observed “logarithmic compression” effect is likely an artifact of the experimental setup rather than an emergent model behavior.", "broader_impact_concerns": "The experimental design implicitly reflects a Northern Hemisphere bias (e.g., associating December with “cold”), as noted in the Limitations section.\n\nIt would be valuable to expand this discussion in the main text and to emphasize that SMDS could serve as a diagnostic tool to analyze how models encode such cultural or environmental priors.", "additional_comments": "This paper addresses an important and timely problem in LLM interpretability.\nThe strongest component is the causal intervention experiment ($\\mathcal{F}_{3}$), which compellingly shows that information may be represented in very low-dimensional subspaces within the LLM—a contribution of genuine independent value.\n\nHowever, the reliance on synthetic data with fixed syntax substantially limits the generalizability of the findings.\nA promising future direction would be to integrate techniques from Topological Data Analysis (e.g., persistent homology) to discover manifold structures in an unsupervised and assumption-free manner, rather than verifying pre-defined shapes such as 2D circles.", "extra_scores": {}, "extra_text": {}}, {"review_id": "YPty8nHqbn", "reviewer": "Reviewer_eyL6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors propose a targeted variant of multidimensional scaling (MDS) to discover submanifolds with specific geometric and topological properties consistent with input data, which they call supervised MDS (SMDS). The authors outline specific target properties encoded by various choices of distance functions to test for circular, linear, and categorical concepts to see if these can be identified and correlated to practically analogous concepts in the data. Basing their study on findings in previous works, the authors focus almost exclusively on demonstrating time-based concepts, in addition to a longitude-latitude concept, across what amounts to three model families (Llama, Qwen, Gemma) with some variety in capacity (2-3B, 8B, 70B parameters).\n\n### Note on Structural Concern\n\nBefore addressing the scientific claims, this reviewer notes a structural concern: while the paper is formatted as a <12 page short work, the core quantitative evidence supporting the main findings is located in the appendix (Tables 5, 8; Figures 10, 11, 12). Hence the supplementary ceases to act as additional results and instead contains all the primary evidence behind the work. The main text relies predominantly on visual/exploratory presentations that cannot be properly evaluated without cross-referencing extensive appendix content.\n\nFor TMLR, which offers flexible paper length, this reviewer questions whether artificially constraining the paper to 12 pages is appropriate when it prevents integrated presentation of the core results. This is an editorial choice to provide for the need for fast review cycles for conference-length papers. In this case, the reviewer believe the work is presented as a shorter work to adhere to the page limit for shorter review cycles. This exaggerates the load of the review, and makes it more challenging to assess whether the quantitative evidence is sufficient to support the claims. \n\nBy choosing this format, the authors implicitly signal that the appendix material is supplementary rather than central to evaluating the claims. This review will therefore weight the main text evidence more heavily in assessing whether the findings are adequately supported.\n\n\n### Findings\n\nThe authors present three claimed findings;\n\n1. Temporal concepts / \"entities\" share an intrinsic manifold structure across different models. The temporal understanding of different models hence seem to mirror across independently trained models. The authors provide quantitative evidence in Figures 3, 10, 11 and Tables 5, 8. The same geometry consistently ranks 1st across models for most tasks (e.g., circular for dates in all 3 models, cluster for date_season in all 3 models). While this seems to outline a general pattern, the evidence lacks statistical significance tests to show that the 1st-ranked geometry is significantly better than the 2nd-ranked. There is no reporting on difference in scores (effect size) nor confidence intervals. While some scores are provided in figures, without quantification of the margins between geometries and their statistical reliability, it's difficult to assess whether the preference for specific geometries is strong or marginal.\n2. Language models adapt structure \"in-context\" for different tasks, such that circular properties are adapted in temporal concepts where appropriate, such as the circular structure of months, and linear structure is applied with perceived temperature (hot / cold). The authors provide evidence through Figure 5 (visualization), Table 5 (showing different best geometries for date vs. date_season vs. date_temperature), and Figure 6 (showing information flow across layers). Overall, the trend seems to be that tasks with the same context but different continuations prefer different geometries at the LP site (circular for date, cluster for seasonal tasks). This holds consistently for Llama and Qwen, though Gemma shows a different pattern for date_temperature. However, the evidence for in-context adaptation (that the model actively transforms one representation into another based on the prompt) is primarily illustrative rather than quantitative. While Figure 5 shows different shapes at TE vs. LP sites, there is no measurement of how different these manifolds are or whether the transformation is statistically significant. Moreover, the evidence doesn't distinguish whether models actively reshape a single representation versus maintaining separate task-specific representations throughout processing. \n3. The third finding is that feature manifolds actively play a role in reasoning for language models. The authors provide two pieces of evidence: (a) perturbations to manifold subspaces degrade reasoning performance more than random perturbations (Figure 7); (b) manifold quality (measured by stress) correlates with task accuracy (Spearman's ρ = 0.513, p = 0.0174). Of the three findings, this provides the strongest quantitative evidence in the main text, including the only reported p-values for any of the main claims. The correlation between manifold quality and accuracy provides statistical support for the functional relevance of discovered manifolds. However, the causal intervention experiment (Figure 7), while showing clear degradation patterns, lacks error bars or statistical tests comparing the SMDS perturbation condition against random perturbations. Without significance tests, it's unclear whether the difference between perturbing SMDS-identified subspaces versus random subspaces is reliably greater than noise. Additionally, the intervention is only demonstrated on three task-model pairs selected for high baseline accuracy, severely limiting the generalisability of this finding.\n\nOverall, the findings the authors present are not wholly unconvincing, but the lack of statistical testing and rigour severely lessens the evidence for the claims made in the paper. The fact that the central quantitative results are relegated to the supplementary material also affects the way the manuscript is interpreted and parsed by the reader. In this case, the paper would have benefited from a longer presentation to fully back up the weight of the claims made by the authors.", "claims_and_evidence_yn": "No", "claims_explanation": "The authors rely heavily on qualitative visualisations and exploratory data analysis to make their claims, as supposed to more rigorous quantitative hypothesis testing. Given the strong claims, the findings need to be backed up with more rigour than visualisation to make the claims. If these claims hold, there should be a clear way to do this testing and provide statistical test results to verify the claims made beyond visual inspection. Moreover, given that the authors have opted to present the main quantitative results as supplementary to keep the main page count low, the evidence for the claims made in the paper is in this reviewers opinion therefore insufficient.\n\n**In summary**: the claims are supported by suggestive patterns but lack the statistical rigor required for convincing scientific evidence. The work would benefit from formal hypothesis testing throughout, multiple testing correction, effect size reporting, and either integration of appendix results into the main narrative or reframing of claims to match the level of evidence provided.\n\n### Issues with Claimed Findings\n\nFor F1; the qualitative pattern seems clear, as the same geometries tend to rank first across models. However, without hypothesis tests, effect sizes, or confidence intervals, we cannot determine if these preferences are statistically meaningful or within measurement noise.\n\nIn the case of F2, the evidence is primarily illustrative. While visualizations show different geometries at different processing stages, there is no quantification of manifold transformations or statistical tests showing these changes are systematic rather than artifacts of visualization.\n\nF3 represents the strongest claim, with correlation statistics (p = 0.0174) and intervention experiments. However, even here, the intervention lacks error bars and significance tests comparing conditions, and is demonstrated on only three carefully selected task-model pairs.\n\n### Issue on Temporal Focus\n\nThe paper focuses almost exclusively on temporal reasoning because *\"initial evidence has found temporal feature manifolds to vary widely across tasks\"* (page 2), citing Engels et al. (2025) and Heinzerling & Inui (2024). While this initially seemed like a strong motivation, it also creates potential circularity: \n- prior work found circular structures for temporal concepts,\n- SMDS is designed to detect circular structures,\n- the paper tests primarily on temporal tasks,\n- circular structures are found. \n\nIn other words, the authors are implicitly looking to confirm rather than test for existing findings relating to submanifold structures. This further highlights the need for explicitly defining the hypotheses of the work, and ensuring the reader that the scientific methodology of the paper is sound and unbiased.\n\nWhile the authors add an additional test on spatiotemporal structure, this too has been largely confirmed in previous works. To strenghten the study, additional concepts with specific structures could be tested to verify these structures to confirm that the structures appear outside of previously discovered concepts. This would improve the novelty and contribution of the work.", "audience_yn": "Yes", "audience_explanation": "Overall, the reviewer finds the scope of the research in the manuscript both interesting and novel. The paper presents a valuable new approach through SMDS, and observes highly interesting patterns while proposing novel findings that corroborates with existing works. Generally, these types of studies have the potential of providing a lot of value to the research community, and this reviewer does not doubt that if methodological an presentation issues are sufficiently addressed, the work has potential to serve as an interesting and impactful read that would interest a significant majority of the journals intended audience.\n\nTo improve on the study, the authors are encouraged to determine some concepts independently of previously discovered concepts to investigate whether the discovered submanifold structures agree with intuition. While this is not required, it would greatly add to the existing study, further demonstrating the applicability of the method to discover new structures instead of just confirming existing works.", "requested_changes": "### Main concerns\n- Add formal statistical hypothesis testing for all three main findings, and clearly outline the framework and hypotheses in the study. This is required for recommending acceptance.\n- Integrate the central quantitative results into the main body of the paper. If the statistical tests are performed, and corroborate the findings in the work, these should be sufficient to include in the main body, and the current details can remain in the supplementary material as additional evidence. However, without these tests, the central evidence for the claims should not be placed in the supplementary. This is required for this reviewer to recommend acceptance.\n- Quantify cross-model consistency with explicit metrics (e.g., rank correlation of geometry preferences, top-k agreement rates across models) rather than relying on visual inspection. Given the nature of the claims as highly general findings relating to the geometry of concepts in language models, showing this explicitly is required to back up the claims made in the paper.\n\n### Optional Addendums\n- It would be interesting to conduct additional experiments for at least one circular and log linear concept, independent of previous works. One example that springs to mind is testing for colour hue, which humans interpret as circular. An example of a log-linear magnitude type concept is easier to determine, e.g. physical size (animals, common objects, physical structures or buildings).", "broader_impact_concerns": "As a largely theoretical work on machine learning, the ethical implications of the work does not warrant a broader impact statement.", "additional_comments": "To reiterate on the structural concern, we highlight that the paper is currently 11 pages, with a 12 page supplementary material section. The issue the reviewer has with this is that the central quantitative results made to support the claims in the paper is almost exclusively placed in the supplementary section, which makes it difficult for readers to engage with the work as intended. \n\nWhile this reviewer expects that this is likely due to a wish for a shorter review cycle, it is not a practice that should be encouraged. TMLR supports works that are longer than standard conference papers, but with the additional caveat that reviewers and ACs are provided enough time to meaningfully engage with the work.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)