On The Geometry and Topology of Representations: the Manifolds of Modular Addition
Authors: Gabriela Moisescu-Pareja
Gavin McCracken
Harley Wiltzer
Colin Daniels
Vincent Létourneau
Doina Precup
Jonathan Love
Venue: ICLR 2026 Poster (2026)
Keywords: mechanistic interpretability
representation learning
geometry
topology
manifolds
universality
platonic representation hypothesis
Primary area: interpretability and explainable AI
TL;DR: We quantitatively and qualitatively show that the manifolds learned by neural networks trained on modular addition are universally the entire input space manifold or projections of it.

Abstract: The Clock and Pizza interpretations, associated with architectures differing in either uniform or learnable attention, were introduced to argue that different architectural designs can yield distinct circuits for modular addition.
In this work, we show that this is not the case, and that both the uniform and trainable attention architectures implement the same algorithm via topologically and geometrically equivalent representations.
Our methodology goes beyond the interpretation of individual neurons and weights.
Instead, we identify all of the neurons corresponding to each learned representation and then study the collective group of neurons as one entity.
This method reveals that each learned representation is a manifold that we can study utilizing tools from topology.
Based on this insight, we can statistically analyze the learned representations across hundreds of circuits to demonstrate the similarity between learned modular addition circuits that arise naturally from common deep learning paradigms.

Published as a conference paper at ICLR 2026
ON THE GEOMETRY AND TOPOLOGY OF REPRESENTA- TIONS: THE MANIFOLDS OF MODULAR ADDITION
Gabriela Moirescu-Pareja" McGill University, Mila
Gavin McCracken" McGill University, Mila
Harley Wiltzer
McGill University, Mila
Vincent Létourneau Université de Montréal, Mila
Colin Daniels Independent
Doina Precup
McGill University, Mila, Google DeepMind
Jonathan Love Leiden University
ABSTRACT
The Clock and Pizza interpretations, associated with architectures differing in ei- ther uniform learnable attention, were introduced argue that different archi- tectural designs can yield distinct circuits for modular addition. In this work, we show that this is not the case, and that both uniform attention and trainable atten- tion architectures implement the same algorithm via topologically and geometri- cally equivalent representations. Our methodology goes beyond the interpretation of individual neurons and weights. Instead, we identify all of the neurons cor- responding each learned representation and then study the collective group of neurons as one entity. This method reveals that each learned representation is a manifold that we can study utilizing tools from topology. Based on this insight, we can statistically analyze the learned representations across hundreds of circuits demonstrate the similarity between learned modular addition circuits that arise naturally from common deep learning paradigms.
INTRODUCTION
As deep neural networks (DNNs) scale and begin to be deployed in increasingly high-stakes settings, will be imperative to develop a concrete understanding of how these models perform computa- tions and ultimately make decisions. Towards this end, research in mechanistic interpretability has focused on identifying sub-structures of these nedels-referred to as circuils-and understanding the function and formation of these circuits the subtasks that they responsible for. In order to extract generalizable understanding of circuits, researchers have formulated a key hypothesis universality (Li et al. 20155] Olah al.) 2020),, which suggests that similar networks trained on similar data will form similat lar circuits. On the other hand, the manifold hypothesis (Bengio et al.) 2013 Goodfellow al., 2016), suggests that representation learning consists of finding lower- dimensional manifold for the data. If these hypotheses were to be false, the task of interpreting large scale models becomes dire-there would be little hope of identifying common patterns across initializations, architectures, and datasets.
Yet, recent work claimed totally disparate and disjoint circuits were learned by DNNs trained on the exact same data using modular addition (a b) mod sults seemingly provide a counter-cxample to the universality hypothesis, thus suggesting the pursuit of learning interpretable principles that generalize across tasks may be doomed. In fact, this sug- gests that the identification of simple circuits larger neural networks could be combinatorially difficult-a if different small-scale models trained on task can learn totally different cireuits with no commonality, then large language models (LLMs) may learn many disjoint circuits for a task within their weights simultaneously.
(Zhong al. 2023). Their
"Equal contribution.
gavin htt.://mmmm.mmmmmmm.m...

---

Published as a conference paper at ICLR 2026
Our primary contribution to interpreting modular addition, f.e. the dataset (a b) mod n c is the resolution the seemingly disparate circuits found by Zhong et al. (2023).
1. We show that, under certain conditions on the structure of learned embeddings, all networks studied by Zhong al. (2023) and McCracken et al. (2025) learn preactivations having equivalent geometry and topology, which can be expressed in closed form;
2. These closed-form equations allow us to rigorously claim that all architectures we study universally make use of the same class of manifolds;
3. We introduce new tools, leveraging topologiaal data analysis, to empirically validate these aforementioned conditions, providing extensive empirical evidence of shared representa- tion geometry across networks.
Altogether, our results restore the possibility that the universality hypothesis is true, as Zhong al. 2023 's architectures are no longer shown to be a counter-exampee
2
RELATED WORK
Deep learning (DL) research increasingly turns to mathematical tasks as controlled settings for in- vestigating learning phenomena and studying the fundamentals of DL. (Ghosh) (025). Such tasks provide opportunities to (i) derive exact functional forms that yield theoretical insights beyond em- pirical studies (McCracken, 2021), (ii) analyze how agents discover novel algorithms such as matrix multiplication or sorting (Fawzi et al. 2022 Mankowitz al. 2023), and (iii) develop methods to better interpret learned policies (Raghu et al. 2018). From toy models that capture superposition (Elhage et al. 2022) formal analyses of in-context learning (Lu et al. 2024 2025), training on math tasks has become a productive way to study DL fundamentals. Within this agenda, mechanistic interpretabiliiyy has produced especially influential results. By reverse-engineering networks trained on group-theoretic problems-SSe -ucsssssschhhh as modular addition (Nanda et af.] 2023) Chughtai et al. 2023
Gromov
2023
Morwani al. [2024] McCracken et | [2025]
Yip al., [024] He al. 2024 2024), and dihedral multiplication
Tao et al. 2025
Doshi et al. 2023), permutations (Stander et al.
McCracken al.
researchers have uncovered mechanisms that speak to core hypotheses about
representations (Huh et al. 2024), universality (Olah et al.) 2020 Li et al. 2015). and algorithmic
structure DL Eberle et al. (025).
As the datasets of group multiplication aren't linearly separable and modular addition (Cyclic group multiplication) very well studied, has become a standard testbed for toy interpretability research. It is ideal for asking: "What exactly do neural networks learn, and how is that computation repre- sented internally?". ?", Two influential works stand out. First, Nanda et al. (003)) reverse- enggnee-ngieeeeeed transformers trained on modular addition and described their internal computations to illuminate the grokking phenomenon (Power et al. 2022), giving progress measures to predict it. Building on this, Chughtai al. (2023) claimed that the algorithm generalized to all group multiplications. Second, Zhong et al. (2023) modified the transformer from Nanda et al. (2023) by interpolating between uniform and learnable attention. They claimed their networks learned two distinct circuits, either Pizza or Clock (described by Nanda et al. (2023)), proposing metrics to separate the two disjoint
circuits.
Recently however, McCracken et al. 20255 showed via abstraction, that across both MLPs and transformers, models converge to one unifying divide-and-conquer algorithm that approximates the Chinese Remainder Theorem and matches its logarithmic feature efficiency. They found first-layer neurons are best fit by degree-1 trigonometric polynomials, with later layers requiring 2 in contrast to the interpretation of Nanda et al. (0033) which modeled neurons as degree-2 trigonomet- ric polynomials all layers.
While many works have focused on reverse-engineering specific algorithms in modular addition, rel- atively little has been done to systematically compare neural representations themselves. Tools from other domains-such complementary ways to characterize representations and may enrich mechanistic interpretability. For instance, distributional methods such as maximum mean diserepancy (MMD) Gretton et al. (2012) are rarely used in mechanistic interpretability, though widely applied elsewhere to quantita- tively measure similarity, align domains Ghifary al. (2014); Zhao et al. (2019) and test fairness
distributional hypothesis testing and topological data analysis (TDA)-
2

---

Published as a conference paper at ICLR 2026
Topological data analysis (TDA) offers comple- mentary view: Shahidullah (2022) used persistent homology to track how network layers preserve or distort imput topology, and Ballester et al. (2024) surveyed TDA tools such as persistent homology and Mapper for analyzing architectures, decision boundaries, representations, and training dynam-
Deka & Sutherland (2023): Kong al. (2025)
ics.
3 SETUP AND BACKGROUND
The learning task we are interested in is the operation of the cyclic group, modular addition (a, b) + + mod for a, E Zn our experiments we have used neural network architectures but we will mostly refer to them by the name that was given to an interpretation of their neuronal operations. All architectures begin by embedding the inputs a, to vectors Eg. E, R 28 using shared (learnable) embedding matrix. The architectures differ in how the embeddings are then processed: MLP-Add immediately passes E, + Eg through an MLP, MLP- Concat immediately passes the concatenation E, E,, R256 through MLP. Learnable attention (claimed to learn Clocks) and uniform attention (claimed to learn Pizzas) transformers, introduced by Zhong al.] (2023). pass Eg: E, through self-attention layer before the ML.. Specifically: uniform attention is constant attention matrix, and trainable attention (Nanda et al. (023) uses the standard learnable softmax attention. We refer to trainable attention architectures as Attention 1.0 and constant attention architectures as Attention 0.0 since Zhong et al. (2023) used parameter to switch between them.
59. We consider various
3.1 PREVIOUS INTERPRETATIONS OF MODULAR ADDITION IN NEURAL NETWORKS
Prior works have all reported disjoint circuits of different "frequencies" f are Iearned. We adopt the notation of Zhong et al. (002)), each frequency f associated with circuit, and an abstraction can be given for the values the embedding vectors Eg and E, associated with that frequency,
Eg
f/n.a//nn...)). sin(2nfa/n))
Eg [2 fb/ //),, sin fb/n)].
angle summation
What distinguishes them is how the embeddings are transformed post- attention. Treating the attention as a blackbox and looking at its output Esb, Zhong et al. (2023) make the following two claims. Clock circuits compute the angle sum (Figure D.
1.0
0.5
Eab = [2o (a + sin ((a + b)/n))
(2)
-05
encoding the modular sum on the unit circle, which needs second-order interactions (c.g., multiplying embedding components via sigmoidal at- tention). In Pizza circuits, Eas adds the embeddings directly as E, + Es, E.6 = (co/(2 + cos(2n fb/ m)m sin(2n + fin(2ffb/m))
-1.0
-1.0
-0.5
0.0
0.5
1.0
vector addition
1.0/
0.5
yielding vector addition on the circle (Figure [ which is entirely linear in the embeddings. Zhong et al. (2023) gave metrics distance irrelevance and gradient symmetricity to distinguish networks having learned the clock vs. pizza circuit (see Appendix E).
0.0
005
Definition 3.1 (Simple Neurons). For (a, E Zp x Zp. a simple neuron neuron that has pre-activation
1.0
-0.
0.0
0.5
1.0
(a, b) = cos(2nfa//) + Φ²) cos ( bb/ 3²),
(4)
(mod 59)
where f E Z, a frequency and ot RR E [0, 2x) are phase shifts.
Figure 1: Clock and Pizza's analytical forms visualized. with frequency assumed to be f = for simplicity. Each point corresponds to pair (a, b) after being trans- formed by the correspond- ing analytical form and is colored by its sum (a + b) mod 59.
This form of the neurons being linear superposition of sinusoid in a and sinusoid in b corroborated in Gromov (2023); Morwani et al. 2024); Doshi et al. (2023): McCracken al. (2025) Li et al. (2025).
Empirical Fact 3.2 (MCCacken et al. (2025)). For Attention 0, At- tention 1, and MIPss (with learnable one-hot encoded embeddings) architectures, first layer neurons are well approximated by simple neu- rons. Later layers can encode combinations of 1 and 2 sinusoids.
3

---

Published as a conference paper at ICLR 2026
3.2
REPRESENTATION SIMILARITY MEASURES
We use two representation similarity measures: centered kernel alignment (CKA) Kornblith et al., 2019) and the representational similarity matrix (RSM) (Kriegeskorte et al. 2008
3.3 TOPOLOGICAL DATA ANALYSIS
We use Betti numbers from algebraic topology to distinguish the structure of different stages of circuits across layers. The k-th Betti number P.. counts dimensional holes: 30 counts connected components, 81 counts loops, 82 counts voids enclosed by surfaces. For reference, a disc has Betti numbers (3u. $1. (1, 0, 0). circle has (1, 1, 0), and 2-torus has (1, 1).
4 CANONICAL MANIFOLDS
We will now focus on networks with a single learnable embedding matrix, matching the setups of Nanda et al. )2023); Zhong et al. (2023); McCracken al. (2025). Our analysis will center on the representation manjfols in frequency cluster f coming from the preactivations h f (a, b) at layer
l and the logits (a, b). The corresponding representation manifolds are, explicitly,
Ma == [a, b) (a,b) E 2 C R and M := {1((a,b) b): (a, b) E 2Z CR",
where de is the number of neuron in the frequency cluster layer f. Our thesis is that under the simple neuron model of squationāiinie introduucd by McCracken et al. (2025) and a simple application of symmetry corresponding to the interchangeability of a, in a + mod n, the exact structure of the M manifolds and how they are mapped from inputs a, b can be revealed. Particularly, we will show that under this model, M always encodes the torus T vector addition disk of Figure D that is, the pizza.
The remainder this section will proceed by formalizing this result in $4.1
4.1 SIMPLE NEURON PHASE DISTRIBUTION DICTATES REPRESENTATION MANIFOLD
Under the simple neuron model, for any frequency cluster f. the only degrees of freedom in the resulting preactivations lie the maps b) - ( for a, E Zn learned by neural net- works. Given that modular addition is commutative, with respect to Φ² ΦB Particularly, one might expect that o² = o for all a, (since swapping the inputs should have effect on the output), or the very least that the random variables &² o identically distributed for A, B ~ Uniform(Z,,) It turns out, as we show in the following theorem (whose proof given in Appendix B). that the resulting manifold MM takes an easily characteriz- able form almost surely in this event. We devote $6 to validating that the phase maps indeed satisfy these properties in thousands of trained neural networks.
might expect to a form of symmetry
us to easily analyze the geometry of representations across
Before stating the theorem, let us introduce some notation that will be useful. Under the simple neuron model, neuron indexed belonging neuron cluster with frequency f maps (a, b) Zz to cos(f, + ) + cos(f, + ( where 0₂ = 2 fa/p. The notation Q is meant to evoke that we model these phases random variables; these are random due to random initialization and random gradient updates. The joint distribution of ( 4 denoted pt A100, 2m))).
Theorem 4.1. Let f for p 2 3, and consider the frequency cluster at layer Let m denote the number of neurons in this cluster, and assume m 2 2. Define the matrix X E TRy² xm according to ((a.)). = cos(O,, + + + cos(O, + of)), denoting the simple neuron preactivations. Assume oy. 4/..b are identically distributed for each neuron i E {l....,m) in this cluster, and that the support of o. has positive (Lebesgue) measure. Then the following hold almost surely:
1. (Perfect phase correlation) If T/.00 and qf are perfectly correlated, in the sense that L.D.b. = R. R.b then X has a rank-2 factorization X = V W with disc E 1RPxx2 x2 satisfying
Velisc (cos 02 cos 0b, sin O2 sin sin
(5)
4

---

Published as a conference paper at ICLR 2026
2. (Phase independence) Otherwise, X has rank-4 factorization X Vtoria W with ytorus 12t x4 given by
tarus
(cos 0ar sin Oar cos 0o sin 0b) (a.b)
(6)
Geometrically, the disc can be viewed as projection of the torus: 43, Tta + (x) + 33, 22 + T.). Thus, the torus structure generalizes the vector-addition disc.
Having established this theorem, it is worth stepping back to contextualize its consequences. As shown by McCracken et al. (2025), first layer preactivations are dominantly simple neurons. Theo- rem 4.1 shows that, under the symmetry properties of T and R2 posited above, the preactiva- tions have simple, low-dimensional structures: in the case of perfect phase correlation, the represen- tation manifold can be compressed to Vdise, which is precisely the vector addition disc of Figure | In the case of phase independence, the representation manifold can be compressed Vtorus which exactly encodes the torus T
Remark 4.2. It is noteworthy that the Clock representation from Zhong et al. (2023 cannol oceur under the hypotheses of Theorem 4.1] The remainder of the paper demonstrates that these hypothe- ses are satisfied empirically with overwhelming probability. Thus, while the Clock eircuit of Zhong et al. (2023) is theoretically plausible, it does not occur naturally in practice. On the other hand, the possibility of the torus representation has not previously been identified in the literature.
A notable consequence of this result is that the geometry and topology of representation manifolds can be characterized by simply investigating the distributions μi a.b of the learned phases. As we describe in [5) this can done quantitatively. allowing us to derive statistical likelihoods of neural circuits arising over thousands of initializations across architectures.
4.2 QUALITATIVE ANALYSIS OF INTERMEDIATE REPRESENTATIONS
This section presents the experimental observations that support the predictions of section 4. Given that learned embeddings are expected to be points simple baselines capturing the closed form of the above manifolds: MLP-Add should add two points on a circle, giving the vector addition disc (Figure T and MI.P-Concat should concatenate two points on a circle, giving the torus T
circle, consider two MLP-modee as
In all networks, we cluster neurons together and study the entire cluster at once. This is done by constructing an n X n matrix, with the value in entry (a, b) corresponding to the preactivation value on datum (a, b). A 2D Discrete Fourier Transform (DFT) of the matrix gives the key frequency for the neuron. The cluster of preactivations of all neurons with key frequency I is the n² x Jcluster fl matrix, made by flattening each neurons preactivation matrix and stacking the resulting vector for every neuron with the same key frequency. We call this matrix the neuron-cluster of preactivations matrix.
Principal component analysis (PCA). To probe the representational geometry of the first-layer neurons, we perform PCA on the neuron-cluster matrix of pre-activations. This layer is where dif- ferences between Clock and Pizza architectures are expected to emerge, since their outputs diverge only after attention.
Figure 2 shows the PCA embeddings for MLP-AAd, MLP-COncat, Pizza, and Clock models. Each input pair (a, 6) has been remapped following (McCracken et al. 2025) and coloured by a+b mod n, so that points close with respect the modular addition task are visually grouped.
The results align strikingly with the predictions Theorem 4.1 For MLP-Add, Pizza, and Clock, the first two principal components explain more than 99% of the variance, yielding 2D disc- like structure (matching the vector addition geometry of Figure []. For MLP-Concat, the first four principal components each explain about 25% of the variance, producing toroidal structure.
Surprisingly, the Clock same 2D disc structure as Pizza and MLP-Add.
never previously described as having a "pizza disc" exhibits the
Distribution ReLt activations.
We next examine the strength of activations across neu- rons within cluster. For each neuron, construct heatmap of its Dost-Rel.U activation
5

---

Published as a conference paper at ICLR 2026
MLP-Add
PCA
Clock PCA
MLL-Concaat
a+b mod 59
a+b mod 59
43
g-
PC2
23
a+
Figure 2: PCA of neuron pre-activations for a single frequency cluster across architectures: MLP-Add (ʃ 27). Pizza (f 17), Clock ( 21), MLP-Concat (f 22). Each point is an input (a, b). colored by (d + d b) mod 59 corresponding the network's output (see Section 3.1 Pizza and Clock nearly identical cach other and to MLP-Add, but differ strongly from MLP-Concat. Note that the trained pizza and clock models being PCA'd are downloaded directly from Zhong et al. 2023 's Github: p9xzdpe99zdp. and model. wa..kkkkhzzoo hhzciux.pt.
Normalized sum of post-ReLU activations in single cluster each architecture Attention
Attention
MLP.Concat
50
0.8
Normalizee
30
0.8
20
10
0.2
40
Figure 3: MLP-Add(f=2D) Pizza(f=17), Clock(f=21). MLP-Concat((=22) Normalized sum of post-activations in clusters in each architecture all (a, 6). Clusters in MLP vector add, Attention 0.0 and 1.0 activate strongest on (a, b) close activation strength decreases with distance from a Clusters MI.P-Concat activate almost equivariantly.
values over all input pairs (a, b), after applying the remapping procedure of (McCracken et al., 2025). Each heatmap thus reflects the activation profile of single neuron across the input space.
To summarize activation patterns at the cluster level, we sum the heatmaps of all neurons in the cluster. Although the network never performs this sum directly, provides compact visualization of where activation strength is concentrated across the input domain.
Figure shows that in MLP-Add, Clock, and Pizza, activations are sharply concentrated along the = b diagonal. This indicates that cosine responses peak when a b. consistent with neurons having equal phases. Moreover, the smooth falloff of activations away from the diagonal implies that representations also encode the distance la - 6). Because downstream weights from neurons to logits are fixed across inputs, this structure implies that logits systematically depend both on (a, b) and on the separation b.
Surprisingly, this diagonal dependence previously claimed by Zhong al. (223) to be the deñin ing characteristic of Pizza-appears not only in Pizza networks but also in MLP-Add and Clock. We return to this point the experiments section.
Linking PCA geometry and activation distributions. The PCA results and activation heatmaps reflect the same underlying phenomenon: principal components encode activation strength. In Fig- 2 MLP--Ad, Pizza, and Clock show negligible activation near the origin (0, 0) of the PCA plane, since neither principal component contributes strongly there. By contrast, for MI.P-Concat the 4D embedding contains datapoints
(0, O, 0, 0). indicating that the cluster activates roughly equally across all inputs. This observation supports the view that the torus-to-cirele map is the natural map for MLP-Concat (Appendix HH.
More generally, these findings suggest that activation strength determines how inputs are arranged in PCA space. For MLP-Add. Pizza, and Clock, the dependence geometry at the logits rather than a perfect circle (Figure 4.. For MLP-Concat, the absence of dependence yields representation closer true circle, consistent with the eircular structure
5 produces annular

---

Published as a conference paper at ICLR 2026
Embeddings
Layer
Layers 2
Logits
MLP-Add Attenton IIL--Coneat
Figure 4: Different factorizations of the torus-to-circle map. We find first-layer intermediate representations to be either torus or disc (resembling vector addition on the circle). Later layers can construct circle, and the logits approximate circle. See Appendix H regarding the torus-to to-cirele factorization.
given in the torus-to-circle map (Appendix H and illustrated in Figure 1 Figure confirms this distinction: MLP-Concat exhibits a torus with a nearly uniform circular projection, while MLP- Add, Pizza, and Clock exhibit vector addition disc (pizza) with annular structure at the logits.
Finally, Figure shows the possible manifolds we see in networks across layers.
METHODOLOGY
Phase Alignment Distributions. Given Theorem 4.1 we can classify representation manifolds from phase statistics. Thus, we propose yet another representation: the Phase Alignment Distri- bution (PAD). To a given architecture, a PAD is a distribution over Zn x Zn. Samples of this distribution are drawn as follows:
1. Sample a random initialization (e.g., random seed) and train the network.
2. From the resulting trained network, sample a neuron uniformly.
3. Return the pair (a, b) E En, Zn that achieves the largest activation in the resulting neuron.
A PAD illustrates, across independent training runs and neuron clusters, how often activations are maximized on the inspecting the proximity of samples to this diagonal, we propose to compare the PADs of architec- tures according to a metric on the space of distributions over Z,, x Zn, giving an even more precise comparison. In the following section, we will provide estimates of the PADs for the aforementioned architectures, as well PAD distances under the maximum mean discrepancy (Gretton al., 2012 MMD> family of metrics with tractable unbiased sample estimators.
diagonal that is, depicts how often learned phases align. Even beyond
Following prior work on one-layer networks (Nanda al. 2023 Zhong et al. 2023). ) building on the empirical validation of the simple neuron model (Eq. in McCracken et al. (2025), we restrict our PAD analysis to gagle-hidden-layeee architectures. In practice, raw activations are often noisy, which makes direct sinusoid fitting unreliable. Instead, we assume the simple neuron model and extract the learned phase pair (Ø", R) using one of two procedures: (i) by taking the point of maximal activation, or (ii) by computing the activation's 's center of mass. Both estimators yield qualitatively similar PADs; we report any differences between them in the Results section.
Betti number distribution. Now turn to multi-layer networks. We estimate the distribution over Betti number vectors corresponding to the set of neurons within a cluster given layer (or logits) to distinguish the structure of the layers: that is Me, and Miggisy- This helps us identify when the underlying structure resembles disc, torus, circle. We compute these using persistent homology with the Ripser library (Bauer 2021 de Silva et al. 2011 Tralie et al. 2018) For more details see Appendix A.3

---

Published as a conference paper at ICLR 2026
Histogram distributions of: neuron max activation (top): neuron activation center of mass (bottom)
MLP-Add
Allention
Allention 1.0
MLP*Concal
loolgeuro
Maligganti
30
1ong 00
Lorg 5.8
Targ 8a
Tang 3
0
MLP-Add
Atlerdion
Atlention
MLP-Concal
logizeenti
30
20ng 50
22ng 2-18
Lang 4.50
Tang
0
40
20
b
Figure 5: Log- dog-itnsity heatmaps for the distribution of neuron maximum activations (top) and activation center of mass (bottom) across 703 trained models. Attention 0.0 and 1.0 architectures exhibit modest off off-diagonal spread compared to MI.-Add, but remain constrained by architectural bias toward diagonal alignment. The maximum discrepancy scores between Attention 0.0 and 1.0 are 0.0237 and 0.0181 in 1 and respectively, indicating they are very similar distributions.
6 RESULTS
6.1 MLP-ADD, ATTENTION 0.0 AND 1.0 (PIZZA AND CLOCK) ARE nearly THE SAME
PAD results. We study 703 trained one-hidden-layer networks drawn from our four architectures:
MLP-Add, Attention 0.0 (Pizza), Attention 1.0 (Clock) and MLP-Concat.
Figure [ shows PAD plots across architectures. We note that MLP-Add, Attention 0.0, Attention 1.0 very concentrated on the diagonal, while MLP-Coneat is not. To further quantify this, propose the torus distance, which is the discrete graph distance from a point (a, b) on the torus to the line. We see that
b line. Figure guantifies this with histogram of torus distances to the Attention 0.0 and T.0 are almost indistinguishable and both very similar to MLP-Add; morcover, our metric successfully discerns these models from MLP-Concat.
Table 4 shows the PAD distances under the MMD distance. All comparisons are statistically signif- icant (p-values 0). We see that Attention 0.0 and 1.0 are extremely close each other, MLP-Add lies moderately close to both, and MLP-Concat is strongly separated from all others. We also intro- duce and study the torus distance metric Appendix D
Previous metrics. Appendix (particularly E.2 and distance irrelevance of Zhong et al. 2023
we evaluate the metries gradient symmetricity
CKA and RMS results. We report similarity using centered kernel alignment (CKA) and repre- sentational similarity matrix correlation (RSM). Linear CKA is bounded in [0.1], where indicates identical representational geometry (up to orthogonal transformation and isotropic scaling) and 0 indicates no linear similarity. RSM values correspond the correlation between pairwise similarity matrices and are bounded in [-1,1] Thus, values approaching reflect strong alignment between learned representations and the reference manifold, while values near zero indicate weak or absent geometric similarity.
We report results for I-layer networks compared to reference manifolds predicted by Theorem 4.1 Table 6. shows that first-layer representations strongly reflect the inductive bias of each architec- ture: MLP-Concal aligns almost entirely with the torus manifold, whereas MLP-Add aligns with the disc (vector-addition manifold. Both attention models (Attention 0.0 and Attention 1.0) likewise exhibit representations that are predominantly disc-like. In contrast, Table shows that the logits across models overwhelmingly aligned with the circle (clock) manifold.
8

---

Published as a conference paper at ICLR 2026
Table 1: Layer I representations: representation similarity measures compared specific reference manifolds (mean std).
Reference Manifold
Metric
MLP-Concat
MLP--Ad
At00.0.0
At0n
CKA
0.707 + 0.012
0.998 + 0.017
0.988 + 0.041
0.974 + 0.028
DISC (VECTOR ADD)
RSM
0.578 = 0.015
0.998 - 0.018
0.986 + 0.044
0.972 + 0.034
CKA
0.994 + 0.011
0.706 0.012
0.699 + 0.029
0.689 + 0.020
TORUS
RSM
0.991 + 0.016
0
0.576 + 0.022
0.581 + 0.014
CKA
1.60e-11
-3.84e-12
-1.01e-10
0.012 0.010
CIRCLE (ClOCk)
2.12e-10
3.00e-10
7.60e-10
RSM
0.112 + 0.003
0.109 0.002
0.109 + 0.004
0.126 0.014
Table 2: Cluster contributions to logits: representation similarity measures compared specific reference
manifolds (mean + std).
Reference Manifold
Metric
MLP-Concat
MLP-Add
Attn.0.
Attn_1.0
CKA
4.61e-4 +
0.037 + 0.094
0.002 + 0.005
0.002 + 0.004
DISC (VECTOR ADD)
0.001
RSM
0.110 + 0.006
0.300 + 0.148
0.258 + 0.047
0.209 + 0.060
CKA
0.001 + 0.001
0.026 0.066
0.001 + 0.003
0.002 0.003
TORUS
RSM
0.115 + 0.003
0.139 + 0.056
0.115 + 0.007
0.115 + 0.008
CKA
0.986 0.025
0.926 0.064
0.940 0.071
0.941 0.078
CIRCLE (CLOCk)
RSM
0.981 + 0.033
0.881 0.097
0.917 + 0.086
0.925 0.091
Betti number results. It's the case that the homology of what networks learn gives that MLP-Add, Attention 0.0 and Attention 1.0 architectures While the MLP-Concat model appears to be different, it's in fact just more efficient, which results from the torus already having the holes necessary to accurately project the correct answer onto the logits after just one non-linearity (see Fig. 66. It's worth noting that while discs appear to be learned in the logits, all the cases checked by hand the dises were caused by limitations of persistent homology, which struggles to find a hole of small radius
all making topologically equivalent computations.
7 DISCUSSION, LIMITATIONS, AND CONCLUSION
This work introduces new lens under which circuits for modular addition may be compared. We argue that networks trained on modular addition tend to learn a common logit manifold, and that representations at intermediate layers are dictated by the alignment of learned phases for the in- puts in the first layer preactivations. Studying the distribution over phases, we identify that networks learn torus or vector-addition disc manifolds (known colloquially pizzas) in the first layer, and proceed by iteratively applying rotations and linear projections to these manifolds before ultimately arriving logit annulus. We argue our work recovers the universality hypothesis by showing that the counter-exante example architectures of Zhong et al. (2023). and even our torus-learning MLP-Concn architecture all use the same torus logits map. The transformers and MLP-Add architecture sim- ply learn lower rank manifolds of the same class of torus manifolds. This follows rigorously and directly from our closed-form equations for the torus and pizza discs: the pizza dises are linear projection of the torus. Thus, we posit "is the nature of universality that DNNs recover either universal manifold, or linear projections of it in order to data?
Notably, this conclusion was reached by examining networks from a different perspective, which al- lowed us to mathematically predict the structure of representations, and also to test these predictions precisely at large scale using tools from topological data amalysis. In particular, work identifies that the joint distribution over simple neuron phases both determines the geometry of representa- tions, and permits efficient quantitative evaluation.

---

Published as a conference paper at ICLR 2026
logils
layer
leyer 2
layer 3
logils
73%
1.09%
2.5%%
23
74
Attention 0.0
.N.
Attention
1.04%
75.3%
94.5%
96%
1.96%
8.5%
22.35
1.47%
1.31%
111.8%
1.44%
77.4%
88.0%
SRL 9%
92.9 92.1%
1.35%
2.23%
1.38%
18.0%
24.
19 6%
34ayer
65%
1.18%
73 89
1.62%
78.5%
87.3%
0.949%
96. 96.7%
0.137%
1.9%
J.11%
1.53%
1,369
MLP-Add
25.8%
MIP- MLP-Coonct
1.74%
12.1%
100%
97%
0.54%
1,23%
0.631%
1.07%
0.38%
0.705%
8.67%
1.92%
22.4%
2-lyar
0.847%
76.9%
99.1%
0.7185
98.1%
8%
91%
5.68%
3.5%
1.95%
2.17%
3..4%
2.13%
1.07%
1.56%
11.3%
14.8%
6.52%
13.8%
100%
92.21
94.
15%
23.2%
21. 3%
Figure 6: Betti number distributions across layers for 1-, 2-, and 3-layer models (100 seeds for each model). In layer 1, MLP-Add, Attention 0.0, and Attention 1.0 mostly yield disc-like representations, while produces torus. From the second layer onward, MLP-Add and both Attention variants converge either disc or circle: the circle reflects the logits topology (correct answer), while the dise is an intermediate that can persist in later layers. MLP-Concat instead transitions directly to the circle. Across depth, Attention 0.0 and 1.0 are nearly identical with the latter having fewer transient discs.
notable limitation of our work is that, its present form, it treats only circuits modular addition. To interpret scage-scale neural networks, it will be necessary derive strategies for characterizing circuits across domains. While our precise methodology is particular to the analysis of modular addition networks, believe the underlying strategy of isolating the degrees of freedom in the ge- ometry of hidden representations and estimating the statistics of their corresponding learned values can be generalizable approach characterize circuits more complex domains.
Beyond this, our results also suggest that the universality property is likely to hold in modular addition. It also draws connections between universality and the manifold hypothesis, given the correspondence we see between the low-dinensional representations formed throughout the layers of the networks we observed. Ultimately, we believe the tools we introduced in this work can inform formal hypotheses about universality and manifold continuity that can be assessed quantitatively.
REPRODUCIBILITY STATEMENT
We have provided code in supplementary.zip and experimental settings in Appendix A We have also provided GPU optimized procedures for all computations we perform including the repro- duction of Zhong et al. (2023)'s 's metries to make repeating tests more accessible.
REFERENCES
Rubén Ballester, Carles Casacuberta, and Sergio Escalera. Topological data analysis for neural net- work analysis: A comprehensive survey, 2024. URL
05840
Ulrich Bauer. Ripser: efficient computation of Vietoris-Rip persistence barcodes. J. Appl. Comput. Topol., 5(3):39 423,, 2021. ISSN 2367-1726. doi: 10. 1688007//41448-021--00000 URL https: 7/doi 1ttp://110100071111111---1-88111
10

---

Published as a conference paper at ICLR 2026
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1828, 2013.
Bilal Chughtai, Lawrence Chan, and Neel Nanda. A toy model of universality: Reverse engineering how networks learn group operations. In International Conference on Machine Learning, pp. 6243-6267, PMLR, 2023.
Vin de Silva, Dmitriy Morozov, and Mikael Vejdemo-Iohansson. (co)homology. Inverse Problems, 27(12):12403,, November 2011. ISSN 1361-642) doi: 10.
Dualities in persistent
URL htttp://dd doi http.///.//..///0066-------00-10 1000010000000000666661////12
124003
Namrata Deka and Danica J Sutherland. Mmd-b-fair: Learning fair representations with statisti- cal testing. In International Conference on Artificial Intelligence and Statistics, pp. 9564-9576.
PMLR, 2023.
Darshil Doshi, Aritra Das, Tianyu He, and Andrey Gromov. To grok or not to grok: Disen- tangling generalization and memorization on corrupted algorithmic datasets. arXiv preprint arXiv:2310.13061, 2023.
Oliver Eberle, Thomas McGee, Hamza Giaffar, Taylor Webb, and Ida Momennejad. Position: We need an algorithmic understanding of generative ai. arXiv preprint arXiv:2507.0754: 2025.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauma Kravec, Zac Hatfield-Dodds Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposi- tion. arXiv preprint arXiv:2209. 2tt.::22009005522 /0652, 2022.
Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Moham- madamin Barekatain, Alexander Novikov, Francisco R. Ruiz, Julian Schrittwieser, Grzegorz Swirszez, et Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 6 61077930):47-53, 2022.
Muhammad Ghifary, W Bastiaan Kleijn, and Mengjie Zhang. Domain adaptive neural networks for object recognition. In Pucific Rim international conference on artificial intelligence, pp. 898-904. Springer, 2014.
Sourangshu Ghosh. Mathematical foundations of deep learning. 2025.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT press Cambridge, 2016.
Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola.
A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
Andrey Gromov. Grokking modular arithmetic. arXiv preprint arXiv:2301.02677 2023.
Tianyu He, Darshil Doshi, Aritra Das, and Andrey Gromov. Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks. arXiv preprint arXiv:2406.02550 2024.
Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis, 2024. URL TRL/https:/// htttt::////xiii....
079877
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.6980, 2014.
Insung Kong, Kunwoong Kim, and Yongdai Kim. Fair representation learning for continuous sen- sitive attributes using expectation of integral probability metrics. IEEE transactions on pattern analysis and machine intelligence, 2025.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International conference on machine learning, pp. 3519- 3529. PMLR, 2019.

---

Published as a conference paper at ICLR 2026
Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. Representational similarity analysis- connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2:249, 2008.
Chenyang Li, Yingyu Liang, Zhenmei Shi, Zhao Song, and Tianyi Zhou. Fourier circuits in neural networks and transformers: A case study modular arithmetic with multiple inputs, 2025. URL Sittps:/arxxs org/ abo22402. 09469
Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson. and John Hoperoft. Convergent learning: Do different neural networks learn the same representations? arXiv preprint 151005544, 2015.
Yue Lu, Mary Letey, Jacob A Zavatone- Veth, Anindita Maiti, and Cengiz Pehlevan. In-context learning by linear attention: Exact asymptotics and experiments. In NeurlPS 2024 Workshop on Mathematics of Modern Machine Learning, 2024.
Yue M. Lu, Mary Letey, Jacob A Zavatone- Veth, Anindita Maiti, and Cengiz Pehlevan. Asymp- totic theory of in-context learning by linear attention. Proceedings of the National Academy of Sciences, 122(28):2202599112 2025.
Daniel Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco Selvi, Cosmin Paduraru, Edouard Leurent, Shariq lqbal, Jean-Baptin Lespiau, Alex Ahern, et al. Faster sorting algorithms discovered using deep reinforcement learning. Nature, 618(79641:257--23, 2023.
Gavin McCracken. Using Exact Models to Analyze Policy Gradient Algorithms. McGill University (Canada), 2021.
Gavin McCracken, Sihui Wei, Gabriela Moisescu-Pareja. Harley Wiltzer, and Jonathan Love. The representations of deep neural networks trained on dihedral group multiplication. In NeurIPS 2025 Workshop on Symmetry and Geometry in Neural Representations.
Gavin McCracken, Gabriela Moisescu- Pareja, Vincent Letourneau, Doina Precup, and Jonathan Love. Uncovering a universal abstract algorithm for modular addition neural networks, 2025.
URL httpt:::////at org/ 2255 18266
Depen Morwani, Benjamin L.. Edelman, Costin-Andrei Oncescu, Rosie Zhao, and Sham M. Kakade. Feature emergence via margin maximization: case studies in algebraic tasks. In The Twelfth In- ternational Conference on Learning Representations, 2024. /tt//ttt/////oeereeeee net/
forum?id=ii<DX8650
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress mea- sures for grokking via mechanistic interpretability. In The Eleventh International Conference Learning Representations, 2023. /ttphtttpp//oppereevieee
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5(3)xe000240001, 2020.
Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Gen- eralization beyond overfitting on small algorithmic datasets, 2022. URL https::/arxivv..rggg
abs/2201.
Maithra Raghu, Alex Irpan, Jacob Andreas, Bobby Kleinberg, Quoe Le, and Jon Kleinberg. Can deep reinforcement learning solve cedos-eelfridge-spenne games? In International Conference on Machine Learning, pp. 4238-4246 PMLR, 2018.
Archie Shahidullah. Topological data analysis of neural network layer representations, 2022. URL https://arxiv.orge abs 2200/ 06438
Dashiell Stander, Qinan Yu, Honglu Fan, and Stella Biderman. Grokking group multiplication with cosets. In Forty-first International Conference on Machine Learning, 2024.
Tao Tao, Darshil Doshi, Dayal Singh Kalra, Tianyu He, and Maissam Barkeshli. (how) can trans- formers predict pseudo-random numbers? In Forty-second International Conference on Machine Learning, 2025. URL
12

---

Published as a conference paper at ICLR 2026
Christopher Tralie, Nathaniel Saul, and Rann Bar-On. Ripser.py: A lean persistent homology library for python. The Journal of Open Source Software, 3(29):925, Sep 2018. doi: 10.21 105/joss. 00925. URL https: httrg//....0///ssssssooooo 00925
Chun Hei Yip. Rajashree Agrawal, Lawrence Chan, and Jason Gross. Modular addition without black-boxes: Compressing explanations of mlps that compute numerical integration, 2024. URL
wtt.oorr//abb////e/e /
Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. International conference on machine learning, pp- 7523-7532. PMLR, 2019.
Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas. The clock and the pizza: Two stories in mechanistic explanation of neural networks. In Thirty-seventh Conference on Neural Informa- fion Processing Systems, 2023. URI. htt.:rrrpp///pppeeeeeee.eeee/sss
A ADDITIONAL EXPERIMENTAL SETUP DETAILS
A.1 TRAINING IYPERPARAMETERS
All models are trained with the Adam optimizer Kingma & Ba (2014). Number of neurons per layer in all models is 1024. Batch size is 59. Train/test split: 90% 10%.
Attention 1.0
Learning rate: 0.00075
L2 weight decay penalty: 0.000025
Attention 0.0
Learning rate: 0.00025
L.2 weight decay penalty: 0.000001
MLP-Add and MLP-Concat
Learning rate: 0.0005
1.2 weight decay penalty: 0.0001
A.2 CONSTRUCTING REPRESENTATIONS
In all networks, we cluster neurons together and study the entire cluster at once McCracken et al. 2025). This done by constructing an X n matrix, with the value entry (a, b) corresponding to the preactivation value datum (a, b). A 2D Discrete Fourier Transform (DFT) of the matrix gives the key frequency f for the neuron. The cluster of preactivations of all neurons with key frequency is the n² |eluster | matrix, made by flattening each neurons preactivation matrix and stacking the resulting vector for every neuron with the same key frequency.
A.3 PERSISTENT HOMOLOGY
We compute these using persistent homology. applied point clouds constructed from intermedi- ate representations at different stages of the circuit, as well as the final logits. This yields a compact topological signature that captures how the geometry of these representations evolves across layers, helping us identify when the underlying structure resembles a disc, torus, or circle. We use the Ripser library for these computations Bauer (221) de Silva et al. (201T) Tralie et al. (2018).
For our persistent homology computations, we set the k-nearest neighbour hyperparameter to 250.
Our point cloud consists of 592 3481 points.
13

---

Published as a conference paper at ICLR 2026
A.4 REMAPPING PROCEDURE
Neuron remapping (McCracken et al., 2025). For a simple neuron of frequency f, we define canonical coordinate system via the mapping:
(a, b) + (a d, b d), where = ge
mod
(7)
ged(f,
This inverse is the modular multiplicative inverse, i.e. for any Z5 let x E Zg. Its inverse x= exists if god(x:, k) 1 and gives x x 1 mod k. This normalizes inputs relative to the neuron's periodicity and allows for qualitative and quantitative comparisons.
B PROOF OF THEOREM 4.1
The proofs of both cases of Theorem 4.1 follow the same pattern: apply an angle sum formula the entries of the pre-activation matrix, realize this matrix as a product of low-rank matrices and use the assumption of uniformity of the phase variables deduce full rank of the composition.
For integers 2 and m 2 2, consider the p² x m data matrix of the pre-activations of the model network with simple neurons (sece equation and
((a,b), = cos(1, + ( + (0, + (b and using the identity cos(æ y) cos cos sin sin y, we have
02 2tl (a, E {O.....P 1}2.
P
(a.,)), c0s( cos( ① ! - sin()a)) sin ( (.)) + cos(0,) cos( $ R.b)) sin(O)) sin (2.))) (8)
Next, we show the details specific to each of the cases: disc and torus.
Proof of Theorem 4.1 (Disc)
Proof. By assumption, $/.a R.b = o, for all Then, equation 8 becomes
X ((ab),. (cos 0g + cos 0b) cos (sin 0. sin 0,) sin d;
Notice then that X VW for the matrices V and W defined by the following row and column vectors respectively
Va,,),: = [cos(0,) + cos(Or), sin(O.) + sin(Os)]
(10)
cos( W = - sin(
(11)
Now we show they both have rank 2 and the kernel of W intersect the image of trivially. The rank 2 of V follows from the independence of cos and sin and the rank of W is true almost surely following the the independence of cos and sin and the hypothesis that & L.o and R.bb have uncountable support.
Suppose (Vi,,)).; W.,) for some (a, and all that means cos(O. 6.) cos(O, + Φi) for all From the assumption the random variables 6² and St are not discrete, this event has probability 0, so the kernel of W intersects the image of V trivially and X = VW has rank 2.
Proof of Theorem 4.1 (Torus)
Equation shows X VW for the matrices V, W defined by rows and columns respectively
Va,,, = (cos(fa)), (l.)
(12)
cos($ L,a
sin $ L.0
W
R.b
(13)
cos +
sin R2.0
The proof that has rank the same as the one for the respective statement in theorem (uni- formity of phases give the rank of V and W and the independence of the image of V and kernel of W).
14

---

Published as a conference paper at ICLR 2026
C STATISTICAL SIGNIFICANCE OF MAIN RESULTS
C.I FIGURE 5
We trained 703 models of each architecture, being ML. vec add, Attention 0.0 and 1.0, and MLP concat, and recorded the locations of the max activations of all neurons across all (a, b) inputs to the network. We also computed the center of mass of each neuron as this doesn't always align with the max preactivation (though it tends to be close).
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0,0
0.0968
0,0000
Moderate difference; highly significant
MLP vec add vs Attention 1.0
0.1239
0.0000
Clear difference; highly significant
MLP vec add vs MLP concat
0.2889
0.0000
Very strong difference; highly significant
Attention 0.0 vs Attention 1.0
0.0338
0.0000
Subtle difference; highly significant
Attention 0.0 vs MLP concat
0.1987
0.0000
Strong difference; highly significant
Attention 1.0 vs MLP concat
0.1723
0.0000
Strong difference; highly significant
(a) Row I: Max activation
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0.0
0,0583
0.0000
Small difference; highly significant
MLP vec add vs Attention 1.0
0.0689
0.0000
Moderate difference; highly significant
MLP vec add vs MLP concat
0.2614
0.0000
Very strong difference; highly significant
Attention 0.0 vs Attention 1.0
0.0210
0.0084
Subtle difference; highly significant
Attention 0.0 vs MLP concat
0.2126
0.0000
Strong difference; highly significant
Attention 1.0 vs MLP concat
0.1947
0,0000
Strong difference; highly significant
(b) Row 2: Center of mass
Table 3: Gaussian-kemaa Maximum Mean Discrepancies (MMD) Gretton et (012)) and permutation p-
values between the empirical distributions shown in Figure 5 For each architecture comparison, we sampled 20,000 points from each empirical distribution (derived from histogram-based neuron statistics), then computed the unbiased Gaussian- -eursian-kernel MMD with bandwidth chosen via the pooled median heuristic. Significance assessed using 50,000 permutation tests per comparison.
D TORUS DISTANCE METRIC
We introduce and study the torus distance metric.
Figure 7 We trained 703 models of each architecture with 512 neurons in its hidden layer (MLP vec add, Attention 0.0 and 1.0, and MLP concat), and recorded the a, value where the max activation of neuron takes place across all (a, b) inputs to the network and all neurons. We also computed the (a, b) values for the location of the center mass each neuron as this doesn't 't always align with the max preactivation (though it tends to be close). Then compute the shortest torus distance from the point of the max activation or the center of mass, to the line = b.
E PREVIOUS INTERPRETABILITTY METRICS (ZHONG ET AL. 2023
E. DEFINITIONS OF GRADIENT SYMMETRICITY AND DISTANCE IRRELEVANCE
Gradient symmetricity measures, over some subset of input-output triples (a, b, c). the average co- sine similarity between the gradient of the output logit Q(a,b.e) with respect to the input embeddings of a and b. For a network with embedding layer E and a set S C CZ of input-output triples:
@QQae aQube
$
sim
JE OE
(a.b.)ES
where sim(u, v)
Tw]l[Te| is the cosine similarity. It is evident that Sy [[-1,]].
15

---

Published as a conference paper at ICLR 2026
Shortest torus distance from diagonal (a=b) to: max activation (row 1); center of mass (row 2)
MLP-Add aegi 0.00
Allention 0.0 aug: 4.88
Allention 1.0 5.81
MIP-Connaal aeg: 14,
250k
25k
Mex scllestoo
300k
200k
200k
20k-
courge
200k
150k
150k
13k
100k
100k
10k
50k
50k
5
10
15
20
25
5
10
15
20
25
5
10
15
20
25
5
10
15
20
25
distance
distance
distance
distance
MLP-Add avg: 0.00
Allention 0.0 avg: 2.77
Allention 1.0 avg. 3.80
MLP-Coneal avg: 13:48
200k
Center of mese
160k
oounte
200k
100k
100k
100k
O0K
50k
distance
distance
distance
distance
Figure 7: Histograms of torus-distance from each neuron's 's phase to the diagonal a = b, across 703 trained mod- els. MLP-Add neurons align perfectly with the diagonal, Attention 0.0 and 1.0 show increasing off ofagona spread, and
exhibits broadly distributed activations the torus.
Table 4: Graussian-kema Maximum Mean Discrepancies (MMD) Gretton et al. 2012) and permutation p-
values between the empirical distributions shown in Figure 7 For each architecture comparison, we sampled 2000 points from each empirical distribution (derived from histogram-haset neuron statisties), then computed the unbiased Gaussian- kernel MMD with a bandwidth chosen via the pooled median heuristic. Significance was assessed using 5000 permutation tests per comparison.
(a) Row I: Max activation
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0.0
0.3032
0,0000
Strong difference; highly significant
MLP vec add vs Attention 1.0
0.3888
0.0000
Very strong difference; highly significant
MLP vec add vs MLP concat
0.9508
0,0000
Extremely strong difference; highly significant
Attention 0.0 vs Attention 1.0
0.0705
0.0000
Moderate difference; highly significant
Attention 0.0 vs MLP concat
0.6323
0,0000
Very strong difference; highly significant
Attention 1.0 vs MLP concat
0.5695
0.0000
Very strong difference; highly significant
(b) Row
Center of mass
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0.0
0.7727
0,0000
Extremely strong difference; highly significant
MLP vec add vs Attention 1,0
0.7517
0.0000
Extremely strong difference; highly significant
MLP vec add vs MLP concat
0.9148
0,0000
Extremely strong difference; highly significant
Attention 0.0 vs Attention 1.0
0.0520
0.0006
Moderate difference; highly significant
Attention 0.0 vs MLP concat
0.7022
0,0000
Very strong difference; highly significant Very strong difference; highly significant
Attention 1.0 vs MLP concat
0.6391
0,0000
Distance irrelevance quantifies how much the model's 's outputs depend on the distance between and b. For each distance d, we compute the standard deviation of correct logits over all (a. b) pairs where a - b = d and average over all distances. It's normalized by the standard deviation over all data.
Formally, let Ta.j = Qij.it be the correct logit matrix. The distance irrelevance q is defined as:
1 Edez, std( (TT,tal E Zp))
Vi E z,))
where q E [0, with higher values indicating greater irrelevance to input distance.
E.2 EVALUATING GRADIENT SYMMETRICITY AND DISTANCE IRRELEVANCE METRICS
Figure 8 shows the mean and standard deviation of the gradient symmetricity and distance inrele- vance metrics from Zhong et al. (2023). Unlike Zhong et al. (2023), who report symmetric- ity results over randomly selected subset of 100 input-output triples (a, b, c) Z3. compute
16

---

Published as a conference paper at ICLR 2026
Per-architecture gradient symmetricity & distance irrelevance
std gradient symmetricity
std distance irrelevance
MLP-Add
0.4
0.5
Attention 0.0
0.3
0.4
Attention 1.0
0.3
MLP-Concat
0.2
0.2
0.1
0.1
0
0 0.2 0.4 0.6 0.8 avg gradient symmetricity
0.2 0.4 0,6 0.8 1
avg distance irrelevance
Figure 8: Evaluation of gradient symmetricity (left) and distance irrelevance (right). Each point shows the average (avg) standard deviation (std) one trained network. MLP-Add and MLP-Coneat lie at nearly opposite extremes, while attention 0.0 and 1.0 overlap substantially. Gradient symmetricity separates Attention 1.0 better, but neither metric always distinguishes between Attention 1.0 and 0.0.
the metric exhaustively across all 593 205, 370 triples add accuracy. See Appendix G for the GPU-optimized procedure.
MLP-Add and MLP-Coneee cluster on opposite extremes, implying the metrics just identify whether neurons have phases df * &R. MLP-Add models have high gradient symmetricity and low distance irrelevance and MIPP-Cnnat models have low gradient symmetricity and high distance irrelevance. Attention 1.0 models span wide range between these extremes depending two factors: 1) how well the frequencics they learned intersect and 2) how well neurons are able to get their activation center of mass away from the d² = & line. Attention 0.0 is closer to MLP-Add than Attention 1.0 because it's harder for this architecture to learn &h * oR_ Notably, failure cases exist using both: neither metric distinguishes between Attention 1.0 and 0.0 models.
MMD results for these two metrics are reported below, again showing that the distance between attention 0.0 and attention 1.0 models is small. This is the case even those these metrics were chosen to differentiate between the two architectures.
Using just the x-axis (since the y-axis those plots is the std dev) MMD results presented next.
We can conclude that the attention transformers are far from vector addition, and very close to each other under all metries.
17

---

Published as a conference paper at ICLR 2026
[h]
Table 5: Permutation test MMDs the empirical gradient symmetricity and distance irrelevance distributions
across all architcctures, All p-values are < 10 (reported as 0.0000).
(a) Gradient symmetricity (2-D: avg and std)
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0.0
1.2725
0.0000
Extremely strong difference; highly significant
MLP vec add vs Attention 1.0
0.9688
0.0000
Extremely strong difference; highly significant
MLP vec add vs MLP concat
1.3471
0,0000
Extremely strong difference; highly significant
Attention 0.0 Attention 1.0
0.7750
0.0000
Very strong difference; highly significant
Attention 0,0 vs MLP concat
1.3503
0,0000
Extremely strong difference; highly significant
Attention 1.0 MLP concat
1.2360
0.0000
Extremely strong difference; highly significant
(b) Distance irrelevance (2-D: avg and std)
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0.0
0.7534
0.0000
Very strong difference; highly significant
MLP vec add vs Attention 1.0
0.7079
0,0000
Very strong difference; highly significant
MLP vec add vs MLP concat
1.2488
0.0000
Extremely strong difference; highly significant
Attention 0.0 vs Attention 1.0
0.2078
0,0000
Moderate difference; highly significant
Attention 0.0 MLP concat
1.2255
0.0000
Extremely strong difference; highly significant
Attention 1.0 vs MLP concat
1.0990
0,0000
Extremely strong difference; highly significant
[h]
Table 6: Permutation-test MMDs on scatter-plot averages only (1-D). All p-vauues are < 10-b, so every difference is "highly significant." Note that the distance between attention 0.0, attention 1.0, and MLP vec add large, implying they are not performing vector addition.
(a) Row 3: Gradient symmetricity (avg only)
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0,0
1.2755
0,0000
Extremely strong difference; highly significant
MLP vec add vs Attention 1,0
0.9842
0.0000
Extremely strong difference; highly significant
MLP vec add vs MLP concat
1.3833
0,0000
Extremely strong difference; highly significant
Attention 0.0 vs Attention 1.0
0.7726
0.0000
Extremely strong difference; highly significant
Attention 0.0 MLP concat
1.3802
0,0000
Extremely strong difference; highly significant
Attention 1.0 vs MLP concat
1.2559
0,0000
Extremely strong difference; highly significant
(b) Row 4: Distance irrelevance (avg only)
Description
MMD
p-value
Interpretation
MLP vec add vs Attention 0.0
0.7739
0.0000
Extremely strong difference; highly significant
MLP vec add vs Attention 1.0
0.7268
0.0000
Very strong difference; highly significant
MLP vec add vs MI.P concat
1.2501
0.0000
Extremely strong difference; highly significant
Attention 0.0 Attention 1.0
0.2109
0.0000
Strong difference; highly significant
Attention 0,0 vs MLP concat
1.2443
0,0000
Extremely strong difference; highly significant
Attention 1.0 MLP concat
1.1093
0.0000
Extremely strong difference; highly significant
F ADDITIONAL COMMENTARY ON THE RESULTS
F.1 THE ATTENTION 1.0 MODEL IS USING ATTENTION AS A WEAK NON-LINEARITY.
It's noteworthy that some neurons in the first MLP layer of transformers have learned one frequency f. and a sum of two different sinusoidal features (i.e. a superposition via linear combination). These
18

---

Published as a conference paper at ICLR 2026
of one first-order sinusoidal feature (the simple neuron model) sin (0(22//)) + sin( (2//b) and also a second term, being second order and cos( flatb-e), ), with both terms having the same f value.
This is why the sum of neuron-cluster post-activations plot (Figure 3) has bit of "off diagonal patchy-spread" that runs from the top left to the bottom right at 45 degrees in only the attention 1.0 model. These small cloudy patches are caused by few neurons activating cos( (la++==)).
Thus, it's the case that for modular addition, the attention layer and its sigmoidal non- con-linear is being used like weak non-linear layer. Sigmoidal attention isn't strong enough non con--innerrty for the network to have all neurons with frequency learn the second order cos( ((a+b-e) that's typically seen in the second and third layers. Indeed, a past work, McCracken et al. (2025), utilized rigorous empirical framework to inspect this over layers, learning rates, and 12 weight decay hy- perparameters. They showed that across these settings, the best fit in the first layer in all neurons in transformers still comes from the simple neuron model.
G GPU-OPTIMIZED COMPUTATIONS
G.1 GPU-OPTIMIZED CENTER-OF-MASS IN CIRCULAR COORDINATES
Let be the grid size and for each neuron 1, ..... N we have pre-activation map
(i.j = O.....P 1).
Define nonnegative weights
(n) = Di
(n)
k.
Let fn E {l,..., ] be the dominant frequency for neuron n, and let
fm be the modular inverse of fn modulo p. fn fn = 1 (mod p).
Convert the row index and column index j into angles ('un-wrapping by f(²)::
(n)
2m p
oj
(n)
2x
J.
p
Form the two complex phasor sums
5(())
exp(i
Wi
p-1
s = ΣΣ
w o
The arguments of these sums give the circular means of each axis:
(!()) arg ((()),
He (m)
arg(s!"))
where arg returns angle in (-T, T.. To ensure nonnegative result, normalize into [0, 2m):
(μ 2T) mod 2T.
Finally, map back from the angular domain to grid coordinates:
CoM(") a
P
(m)+
CoM
P (n)+
1
2n 44b
2T
This handles wrap-around at the boundaries automatically and weights each location (i,j) by [x(])] producing smooth, circularly-awar center of mass. All tensor operations-ang computation, complex exponentials, and weighted sums are expressed as parallel array primitives that JAX can JIT-compile and fuse into single GPU kernel launch, eliminating Python-level overhead. By pre- computing the angle grids and performing the phasor sums inside one jitted function, this imple- mentation fully exploits GPU parallelism and memory coalescing for maximal throughput.
19

---

Published as a conference paper at ICLR 2026
G.2 GPU-VECTORIZED DISTANCE IRRELEVANCE OVER ALL n² INPUT PAIRS (a, b)
Let
I = ((a a, e to,......-1... 1)}.
and order its elements lexicographically:
(ao. bu), (a1, b1), .... (, x=²x²
X
n² single batched forward pass on the GPU computes
Logits
Transformer (X) IR n2'
producing all n² n output logits in parallel. We then extract the "correct-class" logit for each input:
yk Logits k (ox+by) mod n
0, n²
Next we reshape into an n x n matrix L by
Lij = yi where i= (ax + bb) mod n, j (ax by) mod n.
All of the above: embedding lookup, attention, MLP, softmax and the advanced indexing is imple- mented as two large vectorized kernels (the batched forward pass and the gather), so each of the n? inputs is handled in (11) time but fully parallel on the GPU.
Finally, define
T Fglobal
V n² -( ==
1 n²
Σ
Lig.
W
and for each "distance") j
Lis - Ln)
L== = n
Σ
Lijs
global
We report
n-1
7 Σ j=0
std(1)
Σ j=0
G.3
GPU-OPTIMIZED GRADIENT SYMMETRICITY OVER ALL n TRIPLETS (a, bc)
Let
E nxd with d = 128
be the learned embedding matrix, and denote by
Q Eb)
the scalar logit for class e obtained by feeding the pair of embeddings (Eg, Ex) into the model. We define the per-triplet gradient cosine-similarity
b(a,b,c) = <VE, Q Eb)e. VE, Q(E,,
V E. Q(E,, Eo)o E.)o V E, Q(E, Eb)o
for all (a, h,) (0...., n- 1133
To compute (S(a, c)} over the full n² grid in one fused GPU kernel, we first form three index tensors
Aij.A i,, Bija = C...A k. i, 0, ... 1, then flatten to vectors vec(A), b vec(B), c vec(C) {0. (O...., 13nd We gather the embeddings
emb, F(aa] x emb, Ebb T xd
20

---

Published as a conference paper at ICLR 2026
and in JAX compute
= vmap( (ea, eb. c) + V Qlea (emb) emb;, c),
g vmap (Ca, cb. c) + E. Q(ea, (emb), emb; c),
each producing an (n3 x d)-shaped array. Finally the similarity vector
S
R
ga
gbb
and we report
1
S
S
0g
S)
i=1
Runtime. Because express Ba; Bh and the subsequent dot-and-norm entirely inside single ejax.jit+ vmap invocation, XLA lowers it to one GPU kernel that processes all n² triplets in parallel. The kernel dispatch cost therefore O(1), and each triplet's gradient and cosine com- putations are fused into vectorized instructions with constant per-element overhead. Although the total arithmetic work On23), the full Kata-parallel execution means the wall-clock latency grows sub- liberly in n and the per-triplet overhead remains effectively constant.
H HYPOTHESIS: MODULAR ADDITION AS A FACTORED MAP FROM THE TORUS TO THE CIRCLE
Modular addition the function Z, Z, - Zn sending the pair (a. = mod n.
Geometrically, we may embed E On on the unit circle R² via E, (cos(2Tra/)) /n), sinn)nt//)). So the product space En x Zn embeds into R4 as a discretized torus, parameterized by
(a. b) + (cos u, sin u, cos u, sin v),
2na/n, v 2nb/n.
In this embedding, modular addition corresponds to the following map from the torus the circle:
(I,, Zz, 24) + (æ L2 22 21 + 22Tz).
Parameterizing by angles, this becomes the familiar trigonometric identity
(cos u, sin u, cos e, sin + (cos(u + v), sin(u +)).
We claim that networks we study are approximating this specific geometric map from torus in Ra circle in R In Section 3.1 we saw that the "clock" and "pizza" interpretations included learned embeddings of the form of E. Taken together, these embeddings define torus T as the input representation space. Our hypothesis is that architectures (MLP-Add. Attention 0.0 and 1.0, MLP- Concat) do not learn fundamentally different solutions; rather they factor the same torus-to-circle map via different intermediate representations. Fig. shows factorizations of this map.
21


Reviews:
[{"review_id": "YsVl1zAvij", "reviewer": "Reviewer_yR58", "summary": "This work proposes a topological approach to interpretability in modular addition. Under the assumption of simple neurons, it is theoretically established that activations lie on a disc (when phases are in sync) or on a torus (when phases might be independent). The authors then perform experiments to shed light on the geometry of the activations. Based on their empirical analysis, they argue that previously identified mechanisms Pizza and Clock are in fact the same algorithm, restoring the universality hypothesis.", "questions": "1. What exactly is meant by pre-activations at layer1? Is it the vectors from eqs 2 and 3?\n2. Exactly what amount of the variance is described by the top-2 principal components for each network?\n3. What do the plots looklike if we were to study the activations projected to the row-space of the unembedding matrix?\n4. What do the 3rd and 4th principal components look like?\n5. How much do the principal components 1-4 align with the unembedding matrix?\n6. Does the torus topology of MLP-cat undermine the universality hypothesis?\n7. Are the authors willing to tone down the claim that \"the clock and pizza are the same algorithm\", for example, to something like \"topological similarities between the clock and the pizza\"?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The question of universal representations is important and modular addition is a good setup to study this question.\n- The topological/geometric approach is interesing and sheds light on the representations learned by neural networks.\n- The paper studies a diverse and comprehensive set of four architectures: MLP-add, MLP-concat, Attention 0.0 (transformers with constant attention), Attention 1.0 (transformers with learnable attention).\n- The paper combines theoretical analysis with a comprehensive suite of experiments.\n- The theoretical analysis appears correct (although the assumptions are up for debate).", "weaknesses": "1. The paper uses ambiguous language that might lead the reader to confuse neural network architectures and the mechanisms learned by these networks. For example, in lines 116-118, the terms Clock and Pizza are introduced as two different architectures, when in fact they are different mechanisms learnable by the same architecture (the standard transformer block: MLP + Softmax Attention). In lines 151-154, the authors are making a statement seemingly about archictures, when in reality they are refering to specific learned networks. I would suggest that the authors renouce the term \"architecture\" in favour of just \"network\", while clarifying that they are refereing to a specific mechanism learned by a specific architecture.\n2. The numbers in the terms \"Attention 0.0\" and \"Attention 1.0\" probably reffer to the $\\alpha$ constant introducted by Zhong et al. (2023), but this is not described here. I would recommend renaming \"Attention 0\" and \"Attention 1\" to \"Constant-Attention\" (or \"Uniform-Attention\") and \"Learnable Attention\", respectively.\n3. The use of the letter E in eqs 2 and 3 is confusing by suggesting an embedding vector, or at least an activation, but after reading the paper several times this appears to be just a high level explanation (at least for the clock).\n4. Most importantly, the idea that the clock algorithm can be reduced to the same \"pizza disk\" is very problematic. The Clock algorithm uses the attention mechanism to perform trigonometric operations. However, these operations require a linear attention regime (not softmax). Therefore, it is expected that most attention scores would concentrate closely around zero, where the application of softmax is approximately linear. However, this also implies that the output of attention would be approximately an average sum (as in the pizza), with the entire clock mechanism is encoded in the variance from this average. Therefore, it is not surprising that a PCA of the Clock reveals a pizza disk. However, a closer look at the plots reveals that the Clock PCA is not identical to the Pizza PCA: there is a lot of \"noise\" (hidden information) in the Clock plot. The same can be said about the sum of post-relu activations: we can see in Figure 3 that Attention 1.0 contains an additional (albeit weak) signal that is not present in the Attention 0.0. It is exactly this faint signal that is essential to the Clock algorithm. I strongly suspect that all of these apparant similarities would vanish if we considered the activation vectors projected into the row-space of the unembedding matrix.\n5. The claim that \"the Pizza and the Clock algorithms are actually the same algorithm\" is **misleading**. Setting aside the weaknesses of the empirical analysis, the pizza and the clock are well understood mechanisms that are known to perform different computations. This papers reads to me like an argument that *quick sort* and *merge sort* are actually the same algorithm because they share some similarities. I believe this claim is greatly weakening this paper. The paper would be more interesting if the topological approach was used instead to shed light on the similarities and differences of the clock and the pizza. For example, can we characterize the complete topology of the clock algorithm beyond simple neurons?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m0HijZreTn", "reviewer": "Reviewer_3muT", "summary": "This paper revisits the modular addition benchmark, showing that models previously thought to learn distinct “Clock” and “Pizza” circuits in fact converge to geometrically and topologically equivalent manifolds. Using the simple neuron model (from McCracken et al., 2025), the authors derive closed-form expressions for layer-1 preactivations and prove that under mild assumptions on phase distributions, all networks learn either a disc (vector-addition) or torus manifold. They then verify these findings empirically via Phase Alignment Distributions (PAD) and Betti number analyses across hundreds of networks, concluding that the supposed circuit diversity of Zhong et al. (2023) arises from superficial differences.\nThe work further argues that the universality hypothesis—that similar networks learn similar circuits—remains valid, thus unifying previously divergent mechanistic interpretations of modular arithmetic.", "questions": "How does your explanation differ from https://arxiv.org/pdf/2311.07568, which showed that $\\cos(2\\pi f(a+b))$ can emerge from linear mixing of $\\cos a$ and $\\cos b$ with activation function? Maybe activation function in real model can also get  $\\cos(2\\pi f(a+b))$. Hence it may not contradict to the clock algorithm.\n\nIn https://arxiv.org/pdf/2402.09469, attention outputs are sparse in Fourier space—does this align with your observations? Are the attention outputs closer to $\\cos a + \\cos b$ or to $\\cos(a+b)$?\n\nHave you examined whether similar disc or torus topologies appear in pretrained language models, such as those analyzed in https://arxiv.org/abs/2406.03445?\n\nBeyond applying standard Betti-number analysis, what new theoretical insight or interpretability perspective does your TDA pipeline provide?\n\nWhat motivates including MLP-Concat, given that it trivially yields a torus (four degrees of freedom) and is not common in modern architectures?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Theoretical clarity: Derivation of closed-form preactivation manifolds (Theorem 4.1) provides a clear and elegant link between Fourier structure and representation geometry.\n\nUnified interpretation: The paper offers a coherent view showing that previously “distinct” circuits (Clock vs Pizza) are mathematically equivalent, reinforcing the universality hypothesis.\n\nMethodological novelty: Introduces quantitative topology tools (Betti distributions, PAD + MMD metrics) for large-scale circuit comparison.\nEmpirical rigor: Evaluates 703 trained models across multiple architectures, giving the conclusions strong empirical grounding.\n\nConceptual impact: Connects manifold geometry, phase statistics, and interpretability—potentially inspiring similar analyses for other arithmetic circuits.", "weaknesses": "Missing connection to real LLMs: https://arxiv.org/abs/2406.03445 shows that similar Fourier-modular features emerge in large pretrained transformers, yet this paper does not discuss whether such real representations share the same torus or disc topology. Integrating this perspective could help demonstrate broader relevance beyond synthetic tasks.\n\nInsufficient engagement with prior findings: https://arxiv.org/pdf/2311.07568 and https://arxiv.org/pdf/2402.09469 already show that linear superposition of $\\cos a$ and $\\cos b$ can yield $\\cos(a+b)$ without attention and also characterize attention outputs in Fourier space. A deeper comparison or reconciliation with these works would make the analysis more convincing.\n\nLimited methodological novelty: The TDA methods (Betti numbers, persistent homology) are established tools; the contribution mainly lies in their application to this particular setting.\n\nQuestionable realism: The MLP-Concat baseline, while illustrative, trivially produces a torus and is rarely used in realistic architectures. Clarifying its motivation would strengthen the empirical design.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QckPuQyLKt", "reviewer": "Reviewer_9zL4", "summary": "The paper investigates internal representations of networks trained to perform modular addition. Prior work claims that networks either learn a Clock or Pizza circuit. The paper shows that for a number of networks, the manifold at early layers is a torus which slowly morphs to be more circle-like in later layers (including a disk) although not completely a circle. This is shown first theoretically, then empirically through a number of analyses.", "questions": "- Please clarify sections 4.1 and 5\n- See minor points above", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The paper's key strength is the depth of the empirical analysis. The authors consider a number of techniques to probe the representations learned by the networks including PCA, activation strengths, PAD and Betti numbers. This is all done for a number of different architectures. The theoretical analysis also seems sound and is convincing.\n\nOverall, the paper would likely be of significance to those in mechanistic interpretability, particularly those studying modular addition.", "weaknesses": "In my view, the paper doesn't have many weaknesses. My main critique is on clarity: sections 4.1 and 5 could benefit from more intuition provided for the theorem and proposed analyses. For instance, it's not immediately obvious what the factorization structure of X in Theorem 4.1 has to do with geometry. Similarly, in the description of PAD, the significance of a strong diagonal is never explicitly explained. I would encourage the authors to use their extra page in the revision to more slowly walk through these harder-to-understand areas of the paper.\n\nMinor points:\n- \\mathbf Z and \\mathbb Z appear to be used inconsistently in places (as well as for T in the appendix)\n- It's unclear why bold font is used for E and Z but not for any other symbols\n- The color scheme for visualizing numbers from 0-58 is not cyclic (it looks like viridis); I recommend using a cyclic color scheme\n- Figure 6 text is a bit small, especially for the legend", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)