In-Context Algebra
Authors: Eric Todd
Jannik Brinkmann
Rohit Gandikota
David Bau
Venue: ICLR 2026 Poster (2026)
Keywords: Interpretability
In-Context Learning
ICL
Algebra
Grokking
Symbolic Reasoning
Primary area: interpretability and explainable AI
TL;DR: Transformers can infer relational structure and learn to manipulate symbols in-context without needing to refer to a token's underlying meaning.

Abstract: We investigate the mechanisms that arise when transformers are trained to solve arithmetic on sequences where tokens are variables whose meaning is determined only through their interactions in-context. While prior work has studied transformers in settings where the answer relies on fixed parametric or geometric information encoded in token embeddings, we devise a new in-context reasoning task where the assignment of tokens to specific algebraic elements varies from one sequence to another. Despite this challenging setup, transformers achieve near-perfect accuracy on the task and even generalize to unseen groups. We develop targeted data distributions to create causal tests of a set of hypothesized mechanisms, and we isolate three mechanisms models consistently learn: commutative copying where a dedicated head copies answers, identity element recognition that distinguishes identity-containing facts, and closure-based cancellation that tracks group membership to constrain valid answers. Our findings show that the kinds of reasoning strategies learned by transformers are dependent on the task structure and that models can develop symbolic reasoning mechanisms when trained to reason in-context about variables whose meanings are not fixed.

Published as a conference paper at ICLR 2026
IN-CONTEX: ALGEBRA
Eric Todd Jannik Brinkmann 1.2
Rohit Gandikota
David Bau
Northeestern University
2TU Clausthal
ABSTRACT
We investigate the mechanisms that arise when transformers are trained to solve arithmetic on sequences where tokens are variables whose meaning determined only through their interactions in-context. While prior work has studied trans- formers in settings where the answer relies on fixed parametric or geometric infor- mation encoded in token embeddings, we devise new in-context reasoning task where the assignment of tokens to specific algebraic elements varies from one sequence to another. Despite this challenging setup, transformers achieve near- perfect accuracy on the task and even generalize to unseen groups. We develop targeted data distributions to create causal tests of set of hypothesized mecha- nisms, and we isolate three mechanisms models consistently learn: commutative copying where dedicated head copies answers, identity element recognition that distinguishes adentity-containeg group membership to constrain valid answers. Our findings show that the kinds of reasoning strategies learned by transformers are dependent on the task struc- ture and that models can develop symbolic reasoning mechanisms when trained to reason in-context about variables whose meanings are not fixed.
facts, and closure-based cancellation that tracks
INTRODUCTION
The hallmark of abstract reasoning is the ability to work with words and symbols whose meaning is unknown ahead of time (Newell, 1980; Lampinen et al., 2024). Yet much of the performance of language models (LMs) can be attributed to the power of the token embedding, for example pre-encoding the attribute female in the embedding for "Queen," (Mikolov et al., 2013) or pre- encoding divisible-by-twe within the embedding of the token "1 "1" (Zhou et al., 2024; Hu et al., 2025; Kantamneni & Tegmark, 2025; Nikankin et al., 2025). What mechanisms can a transformer language model employ if it is unable to pre-encode solutions in the embeddings of the words?
In this work, we devise a simple in-context learning setting where tokens serve as pure variables, acquiring meaning only through their interactions within each sequence. This allows us to ask: What computational strategies do transformers develop when deprived of meaningful embeddings?
We adopt familiar arithmetic problem setting, training small transformers to predict answers to arithmetic problems sampled from finite groups. What makes our setting unique is that each token is variable that represent any algebraic element: the meaning of each token is only fixed within single sequence. Complementary to previous studies of emergent arithmetic reasoning (Power et al., 2022: Zhang et al., 2022; Nanda et al., 2023; Zhong et al., 2023), solving problems in this setting forces models to infer structure solely from observations of contextual relationships (Piantadosi & Hill, 2022) rather than relying fixed information encoded within token embeddings.
Surprisingly, we find that models trained on this task develop fundamentally different reasoning strategies than those previously observed when LMs solve arithmetic in fixed-token settings. Rather than learning geometric representations of a Fourier basis, we find that models acquire symbolic reasoning mechanisms based on sparse relational patterns (Zucchet al., 2025). We identify three primary algorithmic strategies models employ beyond verbatim copying: commutative copying, identity element recognition, and closure-based cancellation. These findings suggest that the kinds of reasoning strategies learned by transformers are dependent on the task structure, with symbolic, context- based strategies emerging when tokens earry no consistent meaning across contexts.
"Correspondence todd.er@e northeasterm.edu Open-source code and data available wlgghra.baulab.i.. info.

---

Published as a conference paper at ICLR 2026
Sampled Groups Ga1
C3
C5
C4 0
(b)
00
7
1 9 3
Variable
Sampled Facts
12
83
Assignment
Sequence 51
pi = ku
j.jd
g.
j.jj
S8i
A E i +
m i
Predicted Next Token
Vocabulary
V
D
PP2
9
0
W
1
21
13
20
06
22
Ad
b, 0.g
oh
R: pb
83
R oe
c. p.pp
Ie
ap
Figure I: An overview of the data generation process. (a) Variable Assignment: We sample a set of finite groups and assign the elements of each group a non-overlapping set of vocabulary symbols. (b) Sequence Generation: Sampled facts are converted into variable statements via the latent map- ping iPs and concatenated together to form a sequence. constructed by sampling a new set of groups, defining a new latent mapping, and sampling new string of facts. The vocabulary symbols are assigned specific meanings within individual sequences, but can take very different meaning across sequences.
(c) Sample Diversity: Every sequence is
TASK DESCRIPTION
In this section, we describe our in-context learning task. At high level, our task involves simulating mixture of finite groups.' Each task sequence presents several examples of products between elements in a group and the model is trained on the ordinary next-token language modeling objective, with the goal that it will learn predict the outcome of unseen group products (Figure 1).
More formally, we have a set of m groups g [G1. G.,..., Gm that the model trained to simulate. Recall that for any finite group G, the product of two elements y G is written as 2 x y E G. We call each such product of & facts written using vocabulary of variable tokens U; E V whose meaning may vary between sequences. In practice, the vocabulary is small, with N = IV| < Σ; G A typical sequence takes the form shown in Equation where individual facts consist of four tokens.
y = z" a fact. Training data consists of sequences
Uy1 Usi, U222ly2 U22, UzkU U2k"
We describe the positions of fact with the following terminology: The first element Uni, occupies the "left-slot"; Uye the "right-slot"; is the "predictive token"; and Ugg is in the "answer-slot".
To generate a training sequence s, we first sample² a set of groups Go from & whose total number of elements is less than or equal to the number of variable tokens N. We define the set of all sampled group elements to be H, UG., where |H. S N. We then construct a one-to-one latent mapping 9. H, + V that randomly assigns all elements of H. to distinct tokens in V. We ensure that each group in G. is assigned a non-overlapping set of variables so that the meaning of each variable within given sequence is determined by the underlying group structure (Figure la).
Given this latent mapping, we then assemble s by sampling facts from the groups in G., converting them to variable statements via Pss and concatenating them together (Figure lb). The statement "s,, (x)))()))) = (x)" only appears in when there is a corresponding valid fact "x y = z" among the sampled groups in g.. Importantly, while the mapping ys fixed within a sequence, varies between sequences, ensuring that vocabulary tokens e; E act as variables without fixed global meaning (Figure Ic).
'hiie not imperative for understanding our task sctup, we provide a hrief review of relevant topics from group theory in Appendix A. Forr more details about how we sample groups to construct G., Appendix C.2.
2

---

Published as a conference paper at ICLR 2026
Held-Out Ott-Outt Accuracy
Held-Out Accuracy
Held-Out Accuracy
1
0.8
0.8
0.8
0.6
Cyclic
0.6
0.6
Cx C-xxx
0.4
Cyclic 6
0.4
0.4
Cyclic 8
0.2
0.2
0.2
Cyclic
Semigroup
0
50
100 150
40k
80k
50
100
150 200
Number of Facts
Training Steps
Number of Facts
(a) Performance increases with context length. Accuracy in- creases monotonically with the number of in-context facts. Groups of higher order require in-
Phase transition on non-
copyable facts. Accuracy on
(c) Generalization across alge- braic structure. The model gen- eralizes to unseon groups order and also achieves non-trivial hold-
queries where copying is impossi-
ble remains low early in training
but then rises abruptly, indicating
context facts achieve near-
that the model learns to generalize
out accuracy on semigroups.
perfect performance.
beyond simple copying strategies.
Figure 2: In-context algebra performance.
3 CAN TRANSFORMERS LEARN IN-CONTEXT ALGEBRA?
We train transformer models on the in-context algebra task ($2) and evaluate both their in- distribution performance and their ability to generalize across contexts. We report results for one representative model throughout, but observe qualitatively similar patterns across multiple training runs (see Figure in Appendix C). Our main model is a 4-layer autoregressive transformer with 8 attention heads per layer and hidden size 1024, trained with next-token prediction sequences of k=200 algebra facts (~10000 tokens). The training distribution G = {C3..... Cio. D3. D1, Ds} includes cyclic (C,) and dihedral groups (D;) of up order 10, with sequences written using -=16 variable tokens plus the special tokens "=' and Because group-to- -o-varabbee assignments are ran- domized per sequence, tokens act as placeholders whose meaning must be inferred from context.
Performance increases with context length. Accuracy increases monotonically with the number of in-context facts k, but the rate of improvement depends the group order (Fig. 2a). Smaller groups (e.g., C4. Co) reach high accuracy with only few facts, whereas larger groups (e.g., Cs. C1o) require substantially more context achieve similar performance (see Fig. 10 in Appendix C.3).
Phase transition on non-copyable queries. At large k, many queries are trivially solvable by copy- ing previously seen fact (about 90% of queries are copyable at k=200 versus 45% k-50). To address this, we evaluate the model with held-out data where the final fact "zy=" and its commu- tative pair "yz=" never appear elsewhere in the sequence. In this setting, the model still achieves near perfect accuracy, and we observe an abrupt improvement during training (a phase transition) on non-copyable queries (Fig. 2b), suggesting the emergence of strategies beyond verbatim retrieval.
Generalization across algebraic structure. The model also transfers to unseen groups: on the com- plete set of order-8 groups (for groups excluded from training), the model also achieves near- perfect performance (Fig. 2c). Interestingly, hold-out performance is good for non-group structures such as semigroups, but is worse for quasigroups and collapses on magmas (Fig. 1 lb in Appendix C.3). The model still achieves non-trivial accuracy for quasigroups, particularly cancellation data (Fig. 12 in Appendix C.3), though generalization remains consistently stronger for groups than non-groups.
HYPOTHESIZING MODEL MECHANISMS
When analyzing a random in-context algebra sequence, it is possible that multiple algorithms could theoretically produce correct predictions. That can make challenging to identify which mecha- nisms the model actually implements. Consider the sequences shown in Equation 2 and Equation that differ only by which fact is bolded. The model could correctly predict "dp=p" by either copying the answer from the duplicate fact appearing earlier (i.e., dp=p) by recognizing d is an identity
3

---

Published as a conference paper at ICLR 2026
Pereen Data Data Explazzed (%)
1.0
Data
k: = 50
k 100
Deapy
100.0%
100.0%
D Dommuut
98.0%
99.0%
Pidentity 99.0%
100.0%
0.2
D Desociate
60.2%
56.5%
D oxneel
97.0%
92.0%
0.0
25
75
100
150
175
200
25
50
73
100
150
175
200
Number Facts
Number Facts
(c)
Copy
Commute
Ideutity
Cancel
Associatc
Model Accuracy
Figure 3: Algorithmic coverage: (a) the percentage of training data that can be solved by cach mechanism: copying (green), commutative copying (purple), identity recognition (yellow), closure- based elimination (red), associativity (blue), compared to the empirical model performance (black). The gray shaded region represents unexplained performance. (b) Coverage of sequences where neither form of copying is possible. Identity recognition solves 28.7% of the problems (yellow), closure-based cancellation can solve an additional 39.1% 1% (red) and associativity solves 16.9% (blue). Model performance on hold-out sequences is shown in black. (c) The model achieves high accuracy on almost all algorithmic distributions (97-100%), except for associative composition (60%).
element (i.e., de=c) and applying the identity rule.
"kb = i, dc = c. cl p. 1, dp P. en = bb nI = 1,
"kb = =, c, p.jp 1,
c, bb pj
To disambiguate between potential mechanisms, we design five targeted data distributions to test specific algorithms that can solve algebra sequences when a corresponding set of facts is present in the context. We describe each data distribution and the hypothesized algorithm it tests below:
1. Verbatim Copying (Deopy). This data tests whether the model copics exact facts from the text. We construct sequences 8 E 'Dooyy to contain at least one duplicate of the final fact.
Commutative Copying (Pcommute). ). In many groups, knowing that ab=c often implies that ba=c. This data tests whether the model copies these commutative facts from the context. We construct s E Devennntte to contain at least one instance of the commutative fact (c.g., yz=z for final fact ry=), and no duplicate facts.
3. Identity Element Recognition (Didentity). This data tests whether the model can recognize and apply the identity rule. We construct E Detemiiy such that the final fact contains an identity element (e.g., ry=x) and that at least prior fact in the context reveals the identity element (e.g., 2y=2). We remove any duplicate or commutative facts.
Associative Composition (Dassociate). This data tests whether the model can chain fact results to- gether answer new fact via associativity. Given a final fact zy so that contains a minimum set of facts that would enable a solution via association. For exam-
construct D associate ple, the three facts s.g==, so==/gddyy fdd= can be composed (i.e., (xgg)d=fd to compute =y= We make sure to only use triples without duplicate or commutative facts.
((gd)==>> = zy==)
Clssede-Based Cancellation (Deancel). This data tests whether the model can track group mem- bership and appropriately apply the cancellation law to climinate invalid answers (e.g.. zb=g eliminates as an answer to "xy="). Given a final fact ry=2, we construct & E Deancel by in- cluding all the facts that share x in the left-slot (e.g., xb=g) y in the right-slot (e.g., cy=e), and removing duplicate and commutative facts.
4.1 MEASURING COVERAGE AND PERFORMANCE ON TARGETED DISTRIBUTIONS
We seck to answer two questions: (I) What fraction of in in-context algebra sequences can theoretically be solved by these hypothesized algorithms? and (2) Does the model successfully solve sequences that algorithmic strategies can solve when presented with the appropriate facts in-context?
Algorithmic Coverage. To understand the breadth of data that our hypothesized mechanisms might explain, we implement Python equivalents all five algorithms (Appendix G) and measure their coverage, i.e., the percentage of sequences they theoretically solve. We apply the algorithms
4

---

Published as a conference paper at ICLR 2026
sequentially in the following order: verbatim copying. commutative copying, identity recognition, dlosure-based cancellation, and associative composition, where each algorithm is only applied to sequences unsolved by previous mechanisms. We compute algorithmic coverage over both random training sequences (Figure 3a) and random hold-out sequences where neither form of copying is possible (Figure 3h), using 2000 sequences for each evaluation.
We find that verbatim copying can solve a large percentage of the training data, with its area under the curve (AUC) being 67.9% (Figure 3a, green). Commutative copying accounts for an additional 12.1% 1% of cases (purple), with the identity sol solving 4.2% (yellow), closure-based cancellation solving 2. 7% (red), and associativity solving 3.6% (blue) for total coverage AUC of 90.4%. contrast, the model accuracy (black) achieves an AUC of 92.4%. While the hypothesized algorithms can explain most of the model's empirical training performance, they do not explain everything the model has learned (~2.0% AUC, gray); there may be other interesting mechanisms this analysis misses.
When a sequence cannot be solved via copying or commutative copying, we see a very different trend (Figure 3b). In this more challenging setting, the model achieves a slightly lower AUC of 87.3% (black). Identity recognition is able to solve 28.7% of hold-out cases (yellow), closure-based cancellation can solve another 39.1% (red), and associativity solves 16.9% (blue) bringing the total hold-out coverage AUC to 84.7% 7%. Here, the AUC gap between the model's empirical performance and our algorithmic coverage is 2.6% (gray), and is primarily for algebra sequences with fewer facts.
Model Performance on Subdistributions. We evaluate the model on 400 sequences sampled from each distribution D; and report results at i 50 and A: 100 facts (Figure 3c), with more results in Appendix C. We find the model gets near perfect performance on four of the five data distributions that we test: verbatim copying (100.0%), commutative copying (99.0%), identity element recogni- tion (100.0%), and closure-based cancellation (97%). However, model performance on sequences that test associative composition is worse (60.2%), suggesting only partial learning of this property.
CAUSAL VERIFICATION OF LEARNED MECHANISMS
Based on the results in Section 4.1, perform causal interventions understand how the model mechanistically implements the algorithms with stronger empirical evidence: (1) verbatim copying, (2) commutative copying, (3) identity element recognition, and (4) closure-based cancellation.
5.1 CAUSAL INTERVENTIONS
In order to understand the internal computations underlying the model's capabilities, we use causal interventions ( (Vig et al.. 2020; Meng et al., 2022; Geiger et al., 2025) to verify how the model implements the targeted behavior. This is typically done by implicating model components such as attention heads or directions a model's activation space (Wang et al., 2023; Geiger et al., 2024; Mueller et al., 2025). Similar prior work, we quantify the importance of a component via its indirect effect (IE; Pearl, 2001). We compute IE as the change in probability of the target variable token Umrget under some intervention across pair of algebra sequences that differ in a meaningful way (Setean- scoerop)) Equation 4 shows an example of computing IE for an attention head ((.5)) at layer I, head h, by patching its activations from Selean into Scorrupt
IB(1, h) = P( P(Vtargel (a
(L.h) - sccerupe) P PMtarger Scomppt) Beesn
(4)
where d'sieen > corrupt indicates activations (I.,)) are being patched (or replaced) from Selaan into the
(1.h)
same location in Scomupt The average indirect effect (AIE) can be computed over dataset D as:
AIE(D,I,M) =
Σ (IE(1)))) D
(5)
D
5.2 COPYING AND COMMUTATIVE COPYING
In this subsection, we investigate how the model implements verbatim and commutative copying (see also Appendix D). As shown in Section 4.1, a large percentage of our training data (~80%) can either be solved by verbatim copying or commutative copying (Figure 3a), and the model achieves high performance (07-100%) when either form of copying possible (see Figure 3c).
5

---

Published as a conference paper at ICLR 2026
(a)
.9a-1. 1so-i.ku-e., fomoee..on.... 1 (.00--rr_----- lonk
.ckef porrio.-ii... ag0....00.00 bbjcc
16-
16
(1)
keoko,kesk, bfak.e.....
.cbse,bcse,ks
Verbation Copying
Logits
1, kg-e, 1b-
(b)
[ 1a0.-bb,cc0,000 b
.e000. besk.bfak wgg.k........................... kko joso.bosf.....
.bae. kcc
Commusaiiv Copying
.cbee, be (c)
.cecc 10-30.90.,__..... ob- 0.
.00-3. fa-e. -b-c.
10
18 (h)
.eame, www.kkkkkkkk....... genkggnn.... kkro. kjka
Copying
Logits
.900., .g-e kc- 10-.
og-3, b-c..1-ff.cm 1ook
(d)
.cbse.bere,ke kkmo.
Corying
Vocubulary
Vocabulary
Figure 4: An analysis of copying ($ 5.2). Attention patterns (a-d) and direct logit contributions (e-h) of the copying head (layer 3, head 6) across variations of the same algebra sequence. (a) When verbatim copying is possible, the head attends to the answer-slot of the previous fact "ke = f and (e) directly promotes that token's 's logit (green). (b) When the exact fact is absent, the head's attention shifts to the answer-slot and predictive token of the commutative fact "ck f" and (f) promotes that token (purple). Note this fact was also in (a) but not attended to, indicating exact facts take precedence over commutative ones. (c) When both exact and commutative facts are absent, the head often self-attends and (g) no longer strongly promotes one token. (d) When injecting matching "corrupted" fact with an incorrect answer (*Ke j", red), the head attends each answer-slot and (h) promotes both variables (green, red).
Verbatim Copying, We search for attention heads responsible for copying the correct answer from the context by computing the average indirect effect (AIH) of each attention head (1.b) (layer 1, head h). We patch the activations of each head a (1.b) from the final predictive token in Sclemns taken from Deopy. into the same token position in coompp randomly sampled sequence where copying is not possible, and measure its IE. We compute AIE(Doopy: I, h) over 200 samples from Doopy.
We find single attention head (layer 3, head 6) with high AIE (0.91) that is primarily responsible for copying, with no other head having an AIE higher than 0.08 (Figure 13a). We visualize the attention patterns of this copying head in Figure 4a, and find that it attends the answer-slot of duplicated facts (shown in green), much like the n-gram heads observed in Akyürek et al. (2024). On Daupy, head 3.6 strongly promotes the logit of the attended-to token (Figure 4e) which can be by applying the model's unembedding matrix to the attention head output U (a((A)) This allows us understand its output contribution terms of vocabulary tokens (Nostalgebraist, 2020; Elhage et al., 2021; Dar et al., 2023). The top logit consistently matches the attended-to token (Figure 14),
Commutative Copying. In many groups, knowing that ab will also imply that ba c. To investigate how the model implements such commutative copying, we compute the indirect effects of patching attention head activations from sequences Selcan E D commutee non-copying sequences Scomupi where neither verbatim nor commutative copying are possible. We find the same pattern: head 3.6 is again the only head with strong AIE (0.48) for commutative copying (Figure 13b). In the absence of duplicate facts, head 3.6 attends to the predictive token and answer-slot of the commutative fact (Figure 4b) and similarly promotes the attended-to token (Figure 4f).
Non- Con-Coping Sequences. When neither copy-inducing fact is present in the context, head 3.6 often self-attends (Figures 4c), not strongly promoting any token (Figure 4g). However, when the query contains identity fact, we find head 3.6 has interesting identity demotion behavior ($ 5.3).
Corrupt Copying. While copying the answer-slot a commutative fact can solve facts for abelian groups, this doesn't work for non-commutative facts. When analyzing the copying behavior of head 3.6 on cyclic and dihedral groups separately, we find that more than 97% the time promotes the token it attends to, even if that token is the wrong answer (see Figure 14b). We illustrate this behavior in Figure 4d, where we inject a duplicate fact with an incorrect answer and show that head 3.6 attends both duplicates (red, green) and promotes both of their logits (Figure 4h).

---

Published as a conference paper at ICLR 2026
PCA: Layer Amerton
Ippi: (ffff (C00)) ba-c, 60
Head 5.1
Hiad 3.6
(addef,
(0) PCA Steering (OP%
(30-f, 05-4....
(iO QP Demote d:
Bondo
15 1i 20-7 Cp-c, Dec 00-d..........
(1) QP
Tiely cipe, -1-
Ideatity Non Identity Facts
Query Promocion (QP)
Idencit Demotion (ID)
Steering identiny Pacts
(a)
(b)
(c)
Figure 5: Identity Recognition. (a) PCA decomposition of fact hidden states at the final attention layer reveals clear separation of identity facts (blue) and non-identity facts (red). (b) Head 3.1 promotes the logits of both variables in the query (a and c), while head 3.6 demotes the logit of the identity variable, e. (c) PCA steering on its own can induce identity behavior, but it promotes both variables in the query to have near-equal logits. Inserting a false identity fact for either query variable triggers identity demotion, which, along with PCA steering, achieves cleaner identity control.
5.3 IDENTITY RECOGNITION
Our coverage analysis has revealed that when verbatim and commutative copying are no longer allowed, the identity algorithm can solve close 30% of all hold-out problems. In this section, we use data from Diteatity to study how the model solves sequences where the query is an identity fact. Recall that an identity element e E G' satisfies e.x. = x e = for all elements x G, so that one variable in the question is known to be the identity, the answer is equal to the other variable.
Our experiments suggest that identity recognition emerges from the interaction of two complemen- tary mechanisms: query promotion, that elevates both variables in the question as potential answers, and identity demotion that suppresses the known identity element. When both mechanisms activate simultaneously, the non-identity token is correctly selected.
Structure from PCA. First, we note that our transformer's representations reveal a strong signal correlated with the presence of an identity element in the question. To analyze this, we use PCA to plot final-layer attention outputs at the predictive token position (the "=" symbol) just before the model predicts answer. There is a clear separation between facts containing identity elements (blue) and non-identity facts (red) along the first PCA dimension (Figure 5a). This separation is invariant to the specific variables in the fact or the underlying group. This suggests the model has learned to recognize and solve identity facts differently from those without an identity element.
Query Promotion and Identity Demotion. To analyze the role of the final layer attention at pre- dicting identity facts in Figure 5b we use the logit lens (Nostalgebraist, 2020) and find two heads whose logits correlate strongly with identity variables. Head 3.1 promotes both variables in a given fact, serving as a "query promotion" mechanism. This strategy of predicting that the answer is equal to the question is appropriate for problems in which one of the factors the identity, although on its own it would have the undesirable effect of promoting the identity element itself as the answer.
On identity fact sequences. head 3.6 acts as an "identity demotion" mechanism, attending to previous identity facts in the context and suppressing the identity token's logit (Figure 5b, pink). Combined with the previous strategy, this serves to leave only the non-identity factor as the promoted answer.
Causal Verification. Our experiments suggest that the dominant PCA direction in representation space controls the query promotion submechanism. To understand the causal effects, we perform representation steering experiments along this learned direction (see Appendix E for details). When we intervene on the final layer attention output of a non-identity fact and steer toward the identity cluster, the model begins producing equal logits for both query tokens (Figure Sc: i vs. fi).
In addition to query promotion, we can also manipulate the model's identity recognition by introduc- ing false identity facts influence the identity demotion signal. When inject a fact incorrectly suggesting one of the query tokens is an identity element, the identity demotion head (3.6) responds by suppressing that token and causes cleaner identity prediction (Figure 5c, iii, iv).
On the other hand, when the model is presented with a false identity fact in-context while the query is non-identity fact, it typically confuses the prediction. However, if we steer in the negative PCA
7

---

Published as a conference paper at ICLR 2026
direction (away from the identity cluster), the model recovers and correctly predicts the non-iden non-identity answer. These steering experiments demonstrate that the learned PCA direction has causal influence over the model's s identity reasoning, enabling to both induce and suppress identity predictions.
5.4 CLOSURE- BASED CANCELLATION
The closure-based cancellation algorithm is a combination of two key submechanisms: (i) tracking which variables belong to the same group (i.e., the closure), and (ii) systematically eliminating invalid answers using the cancellation law, which implies that for elements z, y, E G. if + then xy æz and ye # z (see Appendix A).
We hypothesize the algorithm can be understood at a high level as computing the difference of two sets: Seloarre Seanet-- Consider the sequence sampled from Deancel shown in Equation 6. For the final query pe =, the closure contains all elements that have previously appeared in facts involving p or e, i.e., Seloaure = {p,e, e, f, a. ra). The cancellation law then eliminates candidates from facts that share a variable in the left- or right-slot: (from pf = p). (from EE n)). f (from ae = f), and (from pp = e). leaving a as the only valid answer (i.e., Scanel {p. n. fe))).
"pf = ee Ee = n. pf = p, pf = p, ae = f. pp = e, pf = pn = pp e, pe=pp= = We use causal interventions to determine how the model implements these two submechanisms. Our analysis reveals evidence of both a closure subspace, that promotes the logits of variables in the same group, and an elimination subspace that demotes answers based on facts present in the context.
(6)
Closure Submechanism. The closure submechanism emerges naturally from autoregressive train- ing: when predicting the right-slot of a fact like zy =, the model must identify which variables could plausibly follow x. These are precisely the elements that belong to the same group (i.e., the closure). In fact, when we analyze the model's predictions at left-slot positions, we find nearly uniform logits across all elements previously associated with that variable, confirming the model has learned how to compute group closure (see Figure 15, Appendix F). Inspired by previous work showing subspaces can encode high-level causal variables (Geiger et al., 2024; Prakash et al., 2025), we aim to identify subspace W that captures the model's 's representation of the closure set, Setosure- We construct counterfactual pairs (s, s') from Deancel that have different closure and elimination sets (S,, S() such that under intervention, the expected counterfactual answer corresponds to a modified set difference: UCF = Selouure Seancel where the closure set comes from s and the elimination set comes from s'. We perform subspace-level patching from into s' and train W to maximize the likelihood of producing the expected counterfactual output UcF (Equation 7). If the intervention causes the model to predict UCF. we take this evidence that the subspace W correctly represents the hypothesized closure set.
(Wver[(Wa! + +(T (T- W))')) + aby))
We can measure its accuracy as how often the model's predicted answer under intervention matches the expected counterfactual target (Figure 17). We train a 16-dimensional W on the model's final layer attention output a', and find that it can achieve good intervention accuracy (99.8%) after a few epochs of training on 2000 data pairs (Figure 18). More details are in Appendix F.2.
For the closure subspace, we train probes (Alain & Bengio, 2017; Belinkov, 2022) to understand what the subspace has learned, and how it represents variables. We train each probe on the subspace to detect whether a variable is in the group closure or not. We find probes are able to identify when a variable is in the closure subspace with high accuracy (Figure 20), and that these veriable-level probes partially align with the model's unembedding matrix (Figure 21), furthering evidence that the closure subspace promotes variables it has seen before in the context.
Cancellation Submechanism. To understand the cancellation submechanism, we train a subspace using a similar construction to the above, but vary the patching setup. If this new subspace W' cap- tures the climination set, then it should generate the counterfactual answer arising from the opposite set difference, where the closure comes from the corrupt sequence s', and the climination set comes from the clean sequence s. giving: (S Sunnce) t'C--
We similarly train this subspace and find it also achieves high intervention accuracy, indicating successfully represents the elimination set (Figure 19). Intervening in this subspace causes variables to be less likely to be predicted by the model. Both subspaces are trained on a single attention layer, and we find they capture partial contributions from several attention heads (Appendix F.1).
8

---

Published as a conference paper at ICLR 2026
Training Loss
2.0
(
Training (n-4)
(c)
1.5
(d)
1.0
1.0
0.8
Accuracy
0.6
0.4
Structure Tokon
Cancel
0.2
Closure Top-k
Identity
Copy
Associate
Commute
0.0
2000
4000
6000
8000
10000
12000
14000
Training Steps
Figure 6: Dissecting Phase Transitions. (top) Average training loss of transformer models broken into stages of learning. (bottom) We track metrics corresponding acquires throughout training. (a) The first sharp drop in loss corresponds to learning to predict struc- tural tokens: "=' and (Ø, gray). (b) Next, the model begins to learn how to predict group closure (2, orange), and also learns the identity's query promotion submechanism (8, yellow, 5.3). (c) This sharp drop in loss corresponds to the model learning how to copy answers verbatim from context ( green), along with commutative copying (G, purple). (d) After learning to copy. the model quickly improves on cancellation (@, red) and identity sequences (Z, yellow) in parallel. We hypothesize this joint improvement corresponds to the fact that both tasks share similar "demo- tion" submechanism identity demotion and cancellation. These complement the closure and query promotion submechanisms learned previously. (e) Accuracy associative sequences increases last (@, blue), after all other mechanisms have been learned.
different skills the model
PHASE TRANSITIONS CORRESPOND TO LEARNING OF DISCRETE SKILLS
We find that models undergo distinct phase transitions during training (Figure 6; see also Ap- pendix C). Across seeds and configurations, the same sequence of stages marked by drops or plateaus in loss recurs (Figure 7). We study the training loss by computing several metrics at each model checkpoint. For each hypothesized mechanism ($4), we evaluate the model using 128 randomly sampled data points from the corresponding datasets described in Section 4. For structural tokens, we compute the model's accuracy of predicting '=" and tokens across a batch of 128 prompts. For computing group closure, measure the top-K matching accuracy the left-slot position (more details are provided in Appendix F).
The earliest ability emerge is prediction of structural tokens: "=' and (Figure 6a, gray). This is followed closely by group closure ($ 5.4): the model learns that combining two elements always yields another valid group element (Figure 6b, orange). This ability appears in left-slot predictions of facts, where the model distributes probability nearly uniformly across all valid candidates (Ap- pendix F). At the same time, the model learns the query promotion submechanism (Figure 6b, yel- low), achieving around 50% on identity sequences ($5.3). The next sharp drop in loss corresponds to the model learning contextual copying ($ 5.2), first reproducing facts verbatim (Figure 6c, green) and then extending to commutative copying (Figure 6c, purple).
Later mechanisms emerge more gradually. The model develops identity recognition, steadily im- proving on identity-related facts and acquires elimination reasoning in parallel, applying cancel- lation laws and closure constraints to rule out inconsistent candidates. Unlike closure and copying, these abilities do not show sharp transitions but appear jointly, suggesting they build top of copy- ing: once the model can retrieve and recombine facts, can also infer identities and apply elimina- tion strategies (Appendix B). We hypothesize these learned jointly because the identity demotion

---

Published as a conference paper at ICLR 2026
mechanism ($5.3) and the elimination subspace ($5.4) perform similar functions, and their "promo- tion" submechanism counterparts are learned at similar times earlier in training. Finally, models begin to solve some associative sequences, after all other mechanisms have been learned.
7 RELATED WORK
Arithmetic a testbed for interpretability. Arithmetic tasks have long served controlled set- tings for studying and interpreting transformers (Liu et al., 2023). Small transformers trained on modular arithmetic exhibit "grokking" where they first memorize training data before converging to interpretable, generalizing solutions with periodic embeddings (Power et al., 2022; Liu et al., 2022; Nanda et al., 2023; Zhong et al., 2023; Stander al., 2024; Morwani et al., 2024). Pretrained LLMs exhibit similar periodic structure in their number embeddings (Zhou et al., 2024; Hu et al., 2025: Kantamneni & Tegmark, 2025; Nikankin al., 2025). enabling modular arithmetic without explicit training. Deng et al. (2026) find that arithmetic- siriii-fffttttedddd LMs rely on symbolic subgroup patterns, instead of using partial products, but Bai et al. (2025) show that implicit chain-of-thoup training does induce partial products and Fourier number representations. More closely related to our setting, He et al. (2024) show that transformers trained on permutations one group develop hi- erarchical "circle-of-circles representations, and Zhong & Andreas (2024) demonstrate that models with trained embeddings, but otherwise frozen random weights can still implement familiar geomet- ric solutions. While these works study arithmetic settings where tokens have some fixed structure, our work examines a complementary setting where we remove fixed meanings of tokens altogether, requiring models solve problems where token referents vary arbitrarily between sequences.
Mechanisms of in-context learning. The ability of transformers to learn from demonstrations has been attributed to several mechanisms. Early work identifies induction heady that underlie copying (Elhage et al., 2021: Olsson et al., 2022: Feucht et al., 2025), while theory frames ICL as Bayesian inference (Xie et al., 2022: Akyürek et al., 2023; Wurgaft et al., 2025) or gradient-descent-lik adaptation (Von Oswald et al., 2023). More recent studies show LMM representations capture task- level structure (Todd et al., 2024; Hendel et al., 2023; Yin & Steinhardt, 2025; Minegishi et al., 2025), and token representations flexibly adapt to context (Park et al., 2025a; Marjieh et al., 2025).
Symbolic reasoning and causal interpretability. Neural systems have long been studied as poten- tial mechanisms for symbol manipulation, from tensor product (Smolensky, 1990) and holographic reduced representations (Plate, 1995) to recent cognitive-science studies of emergent symbolic rea- soning in modern networks (Swaminathan et al., 2023; Yang et al., 2025). More recently, mecha- nistic interpretability has started mapping internal symbolic reasoning circuits in transformers (Li et al., 2023; Brinkmann et al., 2024; Prakash et al., 2024; Saparov al., 2025; Wu al., 2025; Li et al., 2025), using causal intervention techniques (Mueller et al., 2025; Geiger et al., 2024; 2025).
Variables versus value processing in LMs. A few works have tried to disentangle the ability of LMs to solve math abstractly from their ability to perform arithmetic computation. Cheng et al. (2025) find that LMs better at abstract variable-based formulation of solutions compared to meric computation of the same word problems, while Calais et al. (2025) find that in other problem settings LMs struggle with textual comprehension more than equation solving. Mirzadeh et al. (2025) similarly find that LMs lack robustness to changes in numeric values of math problems.
8 CONCLUSION
We have studied LMs trained on a focused algebra task designed to isolate abstract in-context rea- soning behavior in the absence of Exed-meaning symbols. Our findings suggest that the kinds of reasoning strategies learned by transformers are dependent on the task structure. In our in-context algebra setting, where tokens carry no fixed meaning, we have analyzed the mechanisms learned by transformer LMs in detail and found that models develop symbolic mechanisms instead of the familiar parametric or geometric strategies found in settings where tokens do have fixed meanings. We have seen that transformers can learn to manipulate symbols in-context without needing to re- fer to their underlying meaning, similar to the way that high-school algebra students learn to solve math problems by manipulating letter variables without constantly thinking about the values they might contain (Usiskin, 1988). Understanding when and why models choose different computa- tional strategies remains an important open question for future interpretability work.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This paper aims to advance the foundational understanding of in-context learning and transformers. While such research may influence future model development and deployment, cannot meaning- fully anticipate these downstream impacts within the scope of this work.
ACKNOWLEDGMENTS
The authors would like to thank Chris Wendler, Natalie Shapira, Nikhil Prakash, Arnab Sen Sharma, Freya Behrens, and Dana Arad for advice on various stages the project and also Grace Proebsting and Michael Ripa for feedback the paper. We are grateful for the generous support of Coefficient Giving (ET, RG, DB) and the National Science Foundation (Grant No. 2403303; RG, DB).
REFERENCES
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models. In The Eleventh International Conference on Learning Representations, 2023, URL httpt /oppneeveeeeeeeeee net/ forum?
1d=0g0X4HByN41.
Ekin Akyürek, Bailin Wang. Yoon Kim, and Jacob Andreas. In-context language learning: Architec- tures and algorithms. In Forty-first International Conference on Machine Learning, 2024. URL
https: htttps://openrvvatt
openreview net forum?id=333 323121255...
Guillaume Alain and Yoshua Bengio. probes, 2017. URL https:
Understanding intermediate layers using linear classifier httppt/://opeeeeeeeeee..t.t http:////peeeeeeee.e.ee.mm wttp:/eeeeeee/toooml/////r//o
Xiaoyan Bai, Itamar Pres. Yuntian Deng. Chenhao Tan, Stuart Shieber, Fernanda Viégas, Martin Wattenberg, and Andrew Lee. Why can't transformers learn multiplication? reverse-engineering reveals long-range dependency pitfalls, 2025. URL https:://arxiii wrtp://////////////// 00184.
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computa-
fional Linguistics, 48(1):207-119 March 2022. doi: 10.1 10.1162/coli URL https:
//aclanthology. wwtp///g/222//////// cl.1.7/.
Lukas Biewald. Experiment tracking with weights and biases, 2020. URL httph:tt///tttt //wt:://ww.. wardb. com/. Software available from wamdb.com.
Jannik Brinkmann, Abhay Sheshadri, Victor Levoso, Paul Swoboda, and Christian Bartelt. A mechanistic analysis of a transformer trained on symbolic multi-step reasoning task. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Compu- tational Linguistics: ACL 2024, pp. 4082-4102, Bangkok, Thailand, August 2024. Associa- tion for Computational Linguistics. doi: 10.1855// htt.://........ssaaaaaee. findings-acl.242. URL https: / //acllace /aclanthologyy 2224 findings-ade 242/.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle. M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances Neu- ral Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020. URL bttps::/proceeetee www.iiinnnnn.suuseeeeeee neurips ce cc/paper. http:/ppe___feleeeeeeeeeeeeeeeep www.fflleaaaaeee filess/pperr/220. file/1 htleee/5570ddbfffcbb a2e
pdf..
Pedro Calais, Gabriel Franco, Zilu Tang, Themistoklis Nikas, Wagner Meira Jr., Evimaria Terzi, and Mark Crovella. Disentangling text and math in word problems: Evidence for the bidimen- sional structure of large language models' reasoning. In Wanxiang Che, Joyce Nabende, Ekaterina

---

Published as a conference paper at ICLR 2026
Shutova, and Mohammad Taher Pilehvar (eds.). Findings of the Association for Computational Linguistics: ACL 2025, pp. 12671-12688, Vienna, Austria, July 2025. Association for Compu- acl.666. URL
tational Linguistics. ISBN 979-8-99176-256-5. doi: 10. 10.18653 httpss: accl//ccllaaa wnt.an wttg//22255222 f indiinnnnnenn acl 656/-
Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew L Leavitt, and Naomi Saphra. Sud- den drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in MLMs. In The Twelfth International Conference on Learning Representations, 2024. URL https: / /poenrevvie. net. Forum? d=MO5P 1INOO5PPKHHER KHELW.
Ziling Cheng, Meng Cao, Leila Pishdad, Yanshuai Cao, and Jackie CK Cheung. Can LLMs rea- son abstractly
math word problems without CoT? disentangling abstract formulation from arithmetic computation. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 14300-11333, Suzhou, China, November 2025. Association for Com- doi: 10. (0.18653/11/2225.mmml wat.:5.emnmmmmmi.222.2. URL caammnppmmiin...mmmmmmmm main. 723/.
putational Linguistics. ISBN 89977-839877-7-666
https: http://aaaaaattlhaeccee
Qin Dai, Benjamin Heinzerling, and Kentaro Inui. Representational analysis of binding in lan- guage models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp- 17468-17493, Miami, Florida, USA, November 2024. Association for Computational Linguis- tics. doi: 10. 10.18653/1120244 10..6653..2222...mmmmmm...... emr.ppmaa ma main. 967/.
URL https: htaaaaaaaaanllloooooeeaec org/2024/
Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. Analyzing transformers in embedding space. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pp. 16124-16170, Toronto, Canada, July 2023. Association for Computational Linguis- tics. doi: 1/28653//12022..alllooooo 1o3g.893. URL https: acl-long. 893.
2r2/2023.
Chunyuan Deng, Zhiqi Li, Roy Xie, Ruidi Chang, and Hanjie Chen. Language models are symbolic learners in arithmetic. Transactions on Machine Learning Research, 2026. ISSN 2835-8856.
URL /ttttsss///peneeeettttttt /ttp//ppervviieeeeeeeeeeeeeeeeee net forum? iit::dddissbbbli (el==0sbblllll. xUM.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A survey in-context learning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 1107- 128 Miami, Florida, USA, November 2024. Association for Computational Linguisties. doi: 10. /1183/////2224emlllnmmm/ 64. URL https: : tttttp:://ccaaattttn aclanthoooghyy org/ 2024. sanip-mminn... 64/.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Gan- guli, Zac Hatfield-Dodds Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei. Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. URL https::/t/// tttaassaassss//// former- cuiits pub/ /tcp///211/// ft021/ffrmewar fwa.wwork/ index. html.
Jiahai Feng and Jacob Steinhardt. How do language models bind entities in context? In The Twelfth International Conference on Learning Representations, 2024. URI. ht / //wp//eeeeeeeeeeee//e/ee/e..e..e
net fett/formmiin id- 2b3b60K077.
Sheridan Feucht, Eric Todd, Byron C Wallace, and David Bau. The dual-route model of induction. In Second Conference Language Modeling, 2025. URL https: httpss//ppeeeeveeeeeeeeeee net/ forum? d=bNTTKggnG9.
Jaden Fried
Alexander Russell Loftus, Eric Todd, Jannik Brinkmann, Koyena Pal, Dmitrii Troitskii, Michael Ripa, Adam Belfki, Can Rager, Caden Juang, Aaron Mueller, Samuel Marks, Amab Sen Sharma, Francesca Lucchetti, Nikhil Prakash, Carla E. Brodley, Arjun Guha, Jonathan Bell, Byron C Wallace, and David Bau. NNsight and NDIF: Democratizing access
12

---

Published as a conference paper at ICLR 2026
open- open- weight foundation model internals. In The Thirteenth International Conference on Learn- ing Representations, 2025. URL https: owwnreview. arXiv:2407.14561.
Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah Goodman. Find- ing alignments between interpretable causal variables and distributed neural representations. In Francesco Locatello and Vanessa Didelez (eds.). Proceedings of the Third Conference on Causal Learning and Reasoning, volume 236 of Proceedings of Machine Learning Research, pp. 160-187. PMLR, 01-03 Apr 2024. URL https: /tt::://roccceee ngs mlr. press/v236/ geiger24a html.
Atticus Geiger, Duligur Ibeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and Thomas Icard. Causal abstraction: A theoretical foundation for mechanistic interpretability. Journal of Machine Learn- ing Research, 26(83):1-64, 2025. URL http:// http:t/pggppppppppeeeemme html.
org/ pappps//p2a 2ttp/////22/000--2 2050
Tianyu He, Darshil Doshi, Aritra Das, and Andrey Gromov. Learning to grok: Emergence of in- context learning and skill composition in modular arithmetic tasks. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, 2024. URL htth:::: https:://opeeeevaiiiie opppreveee...ee.eeeeee net/ forum? id=aVh 9KRZdRk.
Roce Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 9318-9333, 2023. URL https: /aaaaaaaaannnooooooocoooo org. 2023. findings emnlp. 624.
Jesse Hoogland, George Wang, Matthew Farrugia-Roberts Liam Carroll, Susan Wei, and Daniel Murfet. Loss landscape degeneracy and stagewise development transformers. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https: /ttp:::///ppeneeee http//eeeneeeeeeeeeeeeeeeeeeeeee netttreeeieweee/eeeeeeee. net/
Forum? 1irrmmid==55JyBBBBB
Alston S. Householder. Unitary triangularization of nonsymmetric matrix. Journal of the ACM,
5(4):339-342, October 1958. ISSN 0004-5411. doi: 10.11455320941.3220z URL https: ///pi. 1ttp///...11///911.13332223
Xinyan Hu, Kayo Yin, Michael Jordan, Jacob Steinhardt, and Lijie Chen. Understanding in- context learning of addition via activation subspaces. arXiv preprint arXiv:2505.05145, 2025.
URL http:p:s//axxv..rgg///o wtw.:iv.rrr//b/2555... 05145.
Mark T. Jacobson and Peter Matthews.
Generating uniformly distributed random
latin squares. Journal of Combinatorial Designs, 4(6):405-437, 1996. doi: https:
http://wwwww..ooooooooo////.oaaa 281 281996329483A65 09219662242263300033333341----00
URL https:
CO$352-J.
Sara Kangaslahti, Elan Rosenfeld, and Naomi Saphra. Hidden breakthroughs in language model training, 2025. URL httpp:://aaae
org/ abs/2506 15872.
Subhash Kantamneni and Max Tegmark. Language models trigonometry do addition, 2025. URL /tops::/arxivve org/ abs./02..22222...// 00873.
Andrej Karpathy. NanoGPT. httpp:///ittii htttps///stthb..ccmm/aaaattttttt 2022.
2022.
Jaeyeon Kim, Sehyun Kwon, Joo Young Choi, Jongho Park, Jaewoong Cho, Jason D. Lee, and Ernest K. Ryu. Task diversity shortens the in-context learning plateau. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https:/ wwwp/eweee.w..eee/ee/ee net.:iiwwwweeeeeeeeeeeeeeee.eeee forum? 5DzaaJDD.
Louis Kirsch, James Harrison, Jascha Sohl-Dickstein and Luke Metz. General-purpose in-context learning by meta-learning transformers, 2024. URL https: arxix. wwtpa///.rrgaa///////oo/osssss/ 2212.. 04458.
13

---

Published as a conference paper at ICLR 2026
Andrew K Lampinen, Ishita Dasgupta, Stephanie C Y Chan, Hannah R Sheahan, Antonia Creswell, Dharshan Kumaran, James L McClelland, and Felix Hill. Language models, like humans, show content effects reasoning tasks. PNAS Nexus, 3(7))pgae233, 07 2024. ISSN 2752- 6542. doi: 10. /a.00///aasxxxxs/gaa223.3 URIhttps: petptts::/doi.ooeooo http:/ooo.oooog//// http:/..rrrr....10/////mmmeserer pgae2333.
Andrew Kyle Lampinen, Stephanie C. Y. Chan, Aaditya Singh, and Murray Shanahan. The broader spectrum of in-context learning, 2025. URL https://arxixl httpa//vwrr//////////a/aa 03782.
Belinda Li, Zifan Carl Guo, and Jacob Andreas. (how) do language models track state? In Forty- second International Conference Machine Learming, 2025. URL htttt:::///pneevieww net. fest/forumt d=8SXXsAVIFH.
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wat- tenberg. Emergent world representations: Exploring sequence model trained on synthetic task. In The Eleventh International Conference on Learning Representations, 2023. URL
https: http:///ppeeeeeeeeeeeeeeeeeeeeee net/ /tt.://ooooommmo--oooT
Bingbin Liu, Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang. Transformers learn shortcuts to automata. In The Eleventh International Conference Learning Representa- tions, 2023. URL https:://// hwtpe/eppeeeeeeeeeeeeeeeeeeeeeep weepereeewweeeeeeeeeeeeeeeeeeeee wwt.:/eeeet/.ttt/////e//meee ftrumorrmmmm De
Ziming Liu, Ouail Kitouni, Niklas S Nolte, Eric Michaud, Max Tegmark, and Mike Williams. To- wards understanding grokking: An effective theory of representation learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems, volume 35, pp. 34651-34663. Curran Associates, Inc., cc / paper files/ [iles/paper/202//
2022. URL File/dfc310es he Coneeeeeeeeeeeeeeeeceeeeeeeeeee
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Confer- ence Learning Representations, 2019, URL https: orto
BkG6RicqY7.
Raja Marjieh, Veniamin Veselovsky, Thomas L. Griffiths, and Ilia Sucholutsky. What a number, that large language model may know it?, 2025. URL https://aaxii/ wtt..rggg/bb/////////////////// 01540.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations gpt. D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Process- ing Systems, volume 35, pp. 17359-17372. Curran Associates, Inc., 2022. URL nwwrips co cw/paper.pppppp.pppeep...eeee..e https::/proceet (rt.ga/5.222711800066333322-22-2
S. Koyejo, S. Mohamed, A. Agarwal,
file//
Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Talking heads: Understanding inter-layer communication in transformer language models. In A. Globerson, Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Pro- cessing Systems, volume 37, pp. 61372 61418. Curran Associates, Inc., 2024. doi: 10.52202/ neurips ca/ wctpaper_rieeeseeeaeeeeeeeeeee 079017-1962. URL https: / Prt.:a//-SSa1011100000-----rrrrr
Conference .df..
Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ondicj Čertík, Sergey B. Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rath- nayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta, Shivam Vats, Fredrik Johansson, Fabian Pedregosa, Matthew J. Curry, Andy R. Terrel, Štěpán Roučka, Ashutosh Saboo, Isuru Fernando, Sumith Kulal, Robert Cimrman, and Anthony Scopatz. Sympy: symbolic computing in python. PeerJ Computer Science, 3:e103, January 2017. ISSN 2376-5992. doi: 10.77 1t.7717/pe 103. URL httpst:a:: /ooi org /0. 771 http/peer-eeee 103.
14

---

Published as a conference paper at ICLR 2026
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed represen- tations of words and phrases and their compositionality. Advances in neural information pro- cessing systems, 26, 2013. URL https: httpt:p:s/pprcceedinnge neurips cctpccc/ppcee.ee papppppr..... files/ paper/2013 / 2b
223ce Pww.--.p................p.......
Gouki Minegishi, Hiroki Furuta, Shohei Taniguchi, Yusuke Iwasawa, and Yutaka Matsuo. Beyond induction heads: In-context meta learning induces multi-phase circuit emergence. In Forty-second International Conference on Machine Learning, 2025. URL htttps://ppperreeeeeee net/ Forum?id=Xw011
Seyed Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. GSM-symbolic: Understanding the limitations of mathematical reasoning in large language models. The Thirteenth Internationall Conference on Learning Representations, 2025. URL https:
Depen Morwani, Benjamin L. Edelman, Costin-Andrei Oncescu, Rosie Zhao, and Sham M. Kakade. Feature emergence via margin maximization: case studies in algebraic tasks. In The Twelfth International Conference on Learning Representations, 2024. URL https: /oppneeveee....
fort/form?? i 9wDX850
Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, Eric Todd, David Bau, and Yonatan Belinkov. The quest for the right mediator: Surveying mechanistic interpretability for nlp through the lens of causal mediation analysis. Computational Linguistics, pp. 1-48, 09 2025. ISSN 0811-2017. doi: 10.1162/COLL..72. URL 572.
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretabilityy. In The Eleventh International Conference on Learn- ing Representations, 2023. URL https: /ttp://///e/eeeeeeeeeeeeeeeeeeee http://eeeeeeeeeeeeee fort:mooummm/ 9wwp:mmmdddddddedeeeeeee
Allen Newell. Physical symbol systems. Cognitive Science, 4(2):135-183, 1980. doi: https://doi. org/10.1207/s1551111 10711000007 URL httt:s://onlineeibbatt wiley com/dot abs/10. 1200011117//010 htbp/100..227//1155111100coooo.
Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. Arithmetic without algo- rithms: Language models solve math with a bag of heuristics. In The Thirteenth International Conference on Learning Representations, 2025. URL
URLittps://openeeeiee http:////pppeeeeeeeeeeeeeeeeeeee forum?
id=09YTt26I2P.
Nostalgebraist.
Interpreting GPT: The logit lens. http::/ww..essssseeeee com/ -osee-logiteeee lens, 2020.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-oodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Ka- plan, Sam McCandlish, and Chris Olah. In-context learning and induction heads. Trans- former Circuits Thread, 2022. URL / reuttrrrsstroreerrrrtttttttto.t. reuits. rcuitss.ubb2222// in context-
pub/2022/
and- inducti in mreads/indexx html.
Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Yongyi Yang, Maya Okawa, Kento Nishi, Martin Wattenberg, and Hidenori Tanaka. ICLR: In-context learning of representa- tions. In The Thirteenth International Conference on Learning Representations, 2025a. URL https: https///eeperreeee......m net/ forum?id=px1mmmm
Core Francisco Park, Ekdeep Singh Lubana, and Hidenori Tanaka. Competition dynamics shape al- gorithmic phases of in-context lcarning. In The Thirteenth International Conference on Learning Representations, 2025b. URL https: wwwooeeevvvveeeeeeeeeeeeeeeeeeee inwereeieew.... net/ forum? id=xgH1 FHSSBB
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
15

---

Published as a conference paper at ICLR 2026
Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth Conference Uncer- tainty and Artificial Intelligence, 2001, pp. 411-420. Morgan Kaufman, 2001.
Steven T. Piantadosi and Felix Hill. Meaning without reference in large language models, 2022.
URL pettps://azxxn org/ absa 2208 02957.
Tony A. Plate. Holographic reduced representations. IEEE Transactions on Neural Networks, 6(3):
623-641, 1995. doi: 10.1109/72.377968.
Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Gener- alization beyond overfitting small algorithmic datasets, 2022. https:/ /ttp:///axxivoooooee abs/2201. 02177.
Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. Fine-tuning enhances existing mechanisms: A case study on entity tracking. In Proceedings of the 2024 International Conference Learning Representations, 2024. URL https:: /ttp:///ppeeeeeeeeeeeeeee net/forum id hid.:mmmmidddssaaaaF0F0 arXiv:2402.14811.
Nikhil Prakash, Natalie Shapira, Arnab Sen Sharma, Christoph Riedl, Yonatan Belinkov, Tamar Rott Shaham, David Bau, and Atticus Geiger. Language models use lookbacks to track beliefs, 2025.
URL httph:tp/////m /aaxiviviiv./////a//aaaa///aaava org/ abs/ /205 14685.
Allan Raventos, Mansheej Paul, Feng Chen, and Surya Ganguli. Pretraining task diversity and the emergence of non-bayesian in-context learning for regression. In Thirty-seventh Conference Neurul Information Processing Systems, 2023. URL https:/ ctttts::/openreeee// /ttp///wwpeeeee.e/////e/e net forum?
id-BtAZ4a5xD0
Jie Ren, Qipeng Guo, Hang Yan, Dongrui Liu, Quanshi Zhang, Xipeng Qiu, and Dahua Lin. Iden- tifying semantic induction heads to understand in in-context learning. In Lun-We Ku, Andre Mar- tins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACI. 2024, pp. 6916-6932, Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 101286653/112224 h/ndininnniin/././sssaaaaaa ac1.412. URL https: / aclannthooogy org/2024, findings - acl 412/.
Abulhair Saparov, Srushti Ajay Pawar, Shreyas Pimpalgaonkar, Nitish Joshi, Richard Yuanzhe Pang, Vishakh Padmakumar, Mehran Kazemi, Najoung Kim, and He He. Transformers struggle to learn to search. In The Thirteenth International Conference on Learning Representations, 2025. URL https: http///peeeeeweeneeeeeeee net wttpr//ewwwe///ountttee forum?id- 3dd.m1111-9d11t.
Aaditya Singh, Ted Moskovitz, Felix Hill, Stephanie C.Y. Chan, and Andrew Saxe. What needs to go right for an induction head? mechanistic study of in-context learning circuits and their formation. In Forty-first International Conference on Machine Learning, 2024. URL https: / ///ppperevie..../////////e htwpeevwwwwweeetttuo/ieiieee.e.e fww.mfrrumomoo wo-oommmmddorrri11111
Paul Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1):159-216, 1990. ISSN 0004-3702. doi:
https://doi.org/10 101 370000007702(909000007
Dashiell Stander, Qinan Yu, Honglu Fan, and Stella Biderman. Grokking group multiplication with cosets. In Forty-first International Conference on Machine Learning, 2024. URL https: / //openreview. net wttpie/wwwn//foomm//eeeeee wtw.:m/icdccccaaatna
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.
Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju, Murray Shana- han, Miguel Lazaro-Gredilla, and Dileep George. Schema-learning and rebinding as mechanisms of in-context learning and emergence. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.). Advances in Neural In- Curran Associates, Inc.,
formation Processing Systems, volume 36. pp. 2023. URL https: https::/proceeee/neeeee neurips.eeipp...e. hwwrips.cc/pppe.. files/ files/paper/202/ .ddf.
16

---

Published as a conference paper at ICLR 2026
Eric Todd, Millicent Li, Arna Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. Function vectors in large language models. In The Twelfth International Conference on Learn- ing Representations, 2024. htttttss//p/erreeeee .nw.e/wwweeeeeeeeeeeeeeeeeeeeeee wtt.:/nee//ooomdirrieeerra yMwaG.
arXiv:2310.12113.
Zalman Usiskin. Conceptions of school algebra and uses of variables. In The Ideas of Algebra, K- 12, the 1988 Yearbook of the National Council of Teachers of Mathematies. National Council of Teachers of Mathematics, Reston, VA, 1988.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio. H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL pttpp:://proceedingg wnw.eeeedddnnn...ninise..o....e. www.iiiiss... ca hwppppape__.. http:/pppefee/ppaaaeaaaa files/papeer/0117 File/3f5ee24365 4ddeee2244d4dee99 9 1 c4a845aa
.pdf.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 12388-12401. Curran Associates, Inc., 2020. URL https::/proceeee::ee htppp/pooeednnggg// nwwrips...... mereriiesssnnns..nessses...s.... wwwpurrpssccpppppppeeeeeee cctpccppppppllccccepee paper/2020/FF wttee//ffl//////// /ttp//265bb2222222 22t.2221171111111111117110022220
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, Joao Sacramento, Alexander Mordvint- sev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient de- scent. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learn- ing. volume 202 of Proceedings of Machine Learning Research, pp. 35151-35174 PMLR, 23-29 .tmll.
Jul 2023. URL https: http///poceeegs...ooo////o mlr press wttp://///////////////
Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Reprexentations, 2023. URL https:
Yiwei Wu, Atticus Geiger, and Raphaël Millière. How do transformers learn variable binding in symbolic programs? In Forty-second International Conference on Machine Learning, 2025. wwtprreeiieweeeeeeeeeeeee htt.//orrmmnuttyrrrr
URL https: /
Zhengxuan Wu, Atticus Geiger, Thomas Icard, Christopher Potts, and Noah Goodman. In- terpretability scale: Identifying causal mechanisms in alpaca. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 78205-78226. Curran Associates, Inc., cc wwtppr/filessppprr////ae/aa Confreerooneeeeneeeeeee pdf.
2023. URL htttps:/pproceedingg.
file / /ttt:///000eeeeeeeeeaaaasee----0
Daniel Wurgaft, Ekdeep Singh Lubana, Core Francisco Park, Hidenori Tanaka, Gautam Reddy, and Noah Goodman. In-context learning strategies emerge rationally. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL httph:t///// https://openrevvie. /twp//eveenee...eee
met/forune http///oouuuuuu//OUOOOroOO d-bBUUOOIONN
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. In International Conference Learning Representations, 2022. URL https: www.oeeiew.nneeeeeeeeeeeeeeeeeee forum? inm CH-.RREUUUHUHUUUA
Yukang Yang, Declan Iain Campbell, Kaixuan Huang, Mengdi Wang, Jonathan D. Cohen, and Tay- lor Whittington Webb. Emergent symbolic mechanisms support abstract reasoning in large lan- guage models. In Forty-second International Conference on Machine Learning, 2025. URL https: https:///ppeeeeieee net wtwriiiw.ne/fooummie-eeeeee
Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. Physics of language models: Part 2.1, grade-school math and the hidden reasoning process, 2024. URL https: / warxivvooorg.................... abs/2407. 20311.
17

---

Published as a conference paper at ICLR 2026
Kayo Yin and Jacob Steinhardt. Which attention heads matter for in-context learnin?? In Forty- second International Conference on Machine Learning, 2025. URL httttt:://opeereiiiieee/ net bett/foo wtt
Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, and Tal Wagner. Unveiling transformers with lego: synthetic reasoning task. arXiv preprint arXiv:2206. 220x::2200.04001,, 2022. URL https: //tp://aavvvvv/// http:////wwwww...aaaaa/ wwg/abs//////..oooooo......o...a 2ttbbs//22..002 04301.
Ziqian Zhong and Jacob Andreas. Algorithmic capabilitties of random transformers. The Thirty- eighth Annual Conference Neural Information Processing Systems, 2024. URL https: //
openreview.oo net/ forum? wtt.ummmdddppllgkeeeee.
Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas. The clock and the pizza: Two stories in mechanistic explanation of neural networks. In Thirty-eventth Conference Neu- ral Information Processing Systems, 2023. URL /btern//DDD...uuttttt//////eeees S5wmboclwe.
/open/evvee..netttteteeee nete fontt/foumm?cc
Tianyi Zhou, Deqing Fu, Vatsal Sharan, and Robin Jia. Pre-trained large language models use fourier features to compute addition. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, 2024. URL https: http:/pppnrrewwe.eeeeeee
Nicolas Zucchet, Francesco Angelo, Andrew Kyle Lampinen, and Stephanie C.Y. Chan. The emergence of sparse attention: impact of data distribution and benefits of repetition. In The Thirty-ninth Anmual Conference on Neural Information Processing Systems, 2025. URL https:
//ooenreeiew..e net/
www.netett/ttteeemmmoo forum? d== JMMRRbb4475..
18

---

Published as a conference paper at ICLR 2026
A GROUP THEORY
In this section, we review relevant terms from group theory that are used in our analysis.
A group (G, -) is non-empty G' equipped with binary operation the following properties:
G G - G that satisfies
Associativity, For all elements æ, y, 2 E G: (x - y) - x (y.
Identity. There exists an identity element E G such that . - g. g for all g G Invertibility. For each element g G, there exists inverse element g 1 G such that g g 9 -
The order of a group is the number of elements contained in the set G, and we denote it as [G]. For notational convenience, we refer to group (G, simply as G.
group G is called abelian (or commutative) for all v; y E G, the following holds: x y- = y x.
Our training data consists of two main families of groups: cyclic groups and dihedral groups ($ 2).
eyclic group of order n, denoted Cn. consists of all powers of single generator element: C, fe, g.g"....... 9"= } where g" = e is the identity. Every eyclic group C, has the same structure as (i.e., is isomorphic to) doing arithmetic modulo n. For example, in C5. multiplying group elements (e.g., Equation 8) works exactly like adding numbers mod 5 (e.g., Equation 9):
g² gg" = g²
(8)
3 +
(mod 5)
A dihedral group Dn is the group of symmetries of a regular n-gon (square, hexagon, etc.), with order Dx1 2n. Its elements consist of rotations and n reflections. We note that while all cyclic groups are abelian, dihedral groups non-abelian for 2 3.
An important consequence of group structure is the cancellation law, which states that we can "cancel" common terms in equations. Specifically, for any group G with elements x, y,
* Left cancellation: If xy zz, then y =
Right cancellation: If yz z then y =
Equivalently (by contrapositive): if y * 2, then xy + x2 and yz * zr. This rule guarantees that distinct group elements produce distinct products which helps ground our understanding of the closure-based cancellation mechanism described in Section 5.4.
For completeness, we briefly describe other algebraic structures we test on that lack some subset of group properties:
A semigroup a with an associative binary operation, but does not require an identity element or inverses.
A quasigroup is set where equations as = and yo = b always have unique solutions for any a, b, but the operation need not be associative. Finite quasigroups are equivalent to Latin squares (Jacobson & Matthews, 1996).
A magma is simply a set equipped with a binary operation, with no other required structural properties.
B DISCUSSION: CONTEXTUAL. REASONING IS RICHER THAN JUST COPYING
In this section, we provide additional context about how our findings relate prior work investigat- ing the mechanisms underlying in-context learning.
Since the introduction of in-context learning (ICL: Brown et al., 2020). understanding the ICL pabilities of LLMs has become a major area research (Dong et al., 2024; Lampinen et al., 2025). Seminal work by Elhage et al. (2021) and Olsson al. (2022) studied ICI. through the lens of
19

---

Published as a conference paper at ICLR 2026
context-dependent loss reduction and identified induction heads that copy tokens from earlier in the context. Beyond literal copying, Olsson et al. (2022) hypothesized that much of in- in-oonextt learning could be explained as "in-context nearest neighbors" or "analogical sequence copying" via induction heads that operate on higher-level abstractions beyond literal tokens.
Subsequent research has revealed increasingly sophisticated variations of induction-like behavior showing models transport abstract contextual information such as multi-token concepts (Feucht al., 2025), few-shot task representations (i.e., function vectors) (Todd et al., 2024; Hendel al., 2023; Yin & Steinhardt, 2025), syntactic and semantic relations (Ren et al., 2024), or abstract vari- ables (Yang al., 2025). At the implementation level, these mechanisms all involve attention heads moving information between token positions. However, each mechanism differs substantially in the kind of information they transport and how it is processed.
Thus, it is unclear from prior work whether all forms of contextual learning are best characterized as some form of "fuzzy copying" where characterizing them by the level of abstraction at which they operate is enough or if qualitatively different computational strategies emerge. For instance, when studying how LLMs perform tasks that require entity binding, several prior works have observed models transport token positional information pertaining to where tokens appear in sequence (Feng & Steinhardt, 2024; Dai et al., 2024; Prakash et al., 2024; 2025; Wu et al., 2025). While it is still "coppied" forward via attention, this form of contextual reasoning seems fairly distinct from traditional or "fuzzy" induction as it is related to a token's relative position instead of its meaning.
Studying this question is challenging in pretrained LLMs because token hidden states often reflect both immediate context and parametric knowledge encoded during pretraining. This entanglement makes it difficult to isolate pure contextual reasoning. To study this in more controlled setting, we design a contextual reasoning setting that eliminates parametric knowledge by removing fixed token meanings entirely. Our in-context algebra task assigns tokens to algebraic elements randomly within each sequence, forcing models to infer all structure from observed relationships alone ($ 2).
We analyze the mechanisms that transformers consistently develop when trained on our in-context algebra setting and find that they still learn both induction-style copying and "fuzzy" commutative copying ($5.2). This is expected, as copying can solve a large portion of the training data on its own ($4.1, Figure 3a). However, we also identify two additional mechanisms that seem qualita- tively different from the fuzzy copying strategies seen in prior work: identity element recognition, and closure-based cancellation. At the level of individual operations, these are not entirely unprece- dented; inhibition signals or demotion of tokens through attention heads have been documented before (Wang et al. 2023; Merullo et al., 2024), and closure-based cancellation relies similar elimination submechanism.
What seems distinetive is not the operations themselves but what they operate over: rather than suppressing based on fixed syntactic or semantic token information, both mechanisms are driven by relational structure derived entirely in-context. Identity recognition works by observing which element leaves others unchanged across multiple facts, then applying both query promotion and a conditional suppression rule to select the non-identity variable (see $5.3 and Appendix E). Closure- based cancellation infers group membership to construct a set of valid candidates, then systemati- cally eliminates those ruled out by the cancellation law ($5.4, and Appendix F).
These mechanisms are examples of how models can reason about tokens as variables by manipulat- ing references in-context without knowing their underlying meaning. Taken together, our findings provide additional evidence that the contextual strategies that transformers can develop look increas- ingly different from mere copying. Understanding when and why models learn different contextual strategies remains an important open question for future work.
C MODEL ARCHITECTURE AND TRAINING
In this section, we provide more details about our training setup.
Model Training Details. We train autoregressive transformer models (Vaswani et al., 2017), on in-context algebra sequence data sampled as described in Section 2. with batch size of 128, and sequences with 200 facts. Our vocabulary consists of 16 variable tokens, a predictive token, and a separator token (N- 18 in total), and each fact is made up of 5 tokens. As an additional hold-out
20

---

Published as a conference paper at ICLR 2026
3.5
0.8
Traliing Los
0.6
Evaluation
1.5
0.2
1,0
20000
40000
60000
20000
40000
60000
Training Steps
Truining Steps
Seed 42
Seed 6048
Sead 7277
Seed 9791
Figure 7: Consistency across runs. We observe qualitatively similar patterns across multiple train- ing runs, both in terms of (a) phase transitions in the loss curves and (b) corresponding "grokking" increases in hold-out evaluation performance.
scheme, we select one variable token (e.g., a) that is never assigned as an identity element during training. This ensures that we can test both per-sequence hold-outs (where copying is not possible), and variable-assignmen hold-outs where the exact assignment of elements to variables is unseen.
Tooling and Compute. Our transformer implementation is based on code adapted from nanoGPT (Karpathy, 2022); we use Rotary Positional Embeddings (RoPE) (Su et al., 2024) in- stead of learned positional embeddings, We use the A optimizer (Loshchilov & Hutter, 2019) with a learning rate of 10 and 1000 warmup steps. The primary model we study in the main paper has 4 layers, with 8 attention heads per layer, and hidden state dimension 1024. As shown in Figure 7, observe qualitatively similar patterns across different seeds, both in terms of phase transitions in the loss and hold-out evaluation accuracy. We usually see convergence (eval. accuracy 99%) between 30,000 and 75,000 steps and save the checkpoint with the best evaluation accuracy.
We train our models with either NVIDIA A 100 80GB GPUs or NVIDIA A6000 48GB GPUs. Train- ing statistics are logged using Weights and Biases (Biewald, 2020). Experiments are implemented using NNsight Frotto-Kaufman et al., 2025) and PyTorch (Paszke et al., 2019), and run on work- stations with NVIDIA A6000 48GB GPUs We use SymPy (Meurer et al., 2017) for simulating various group structures for our in-context algebra setting and use a custom implementation for magmas, semigroups, and quasigroups, with quasigroup generation based on Jacobson & Matthews (1996)'s 's method for Latin squares.
C.1 MODEL ARCHITECTURE HYPERPARAMETERS
In this subsection, we study the effect of three hyperparameters that govern model capacity: (1) num- ber of layers, (2) number of attention heads per layer, and (3) hidden state dimension. The training loss and evaluation accuracy of each hyperparameter sweep shown in Figure 8, and the corre- sponding breakdown of model performance by metric and hyperparameter configuration is shown in Table I, where results are reported using the checkpoint with the highest evaluation accuracy.
For each hyperparameter setting, there are noticeable drops in loss throughout training which cor- respond to the learning of discrete skills relevant for the task ($ 6), and is consistent with similar findings of phase transitions and skill-learning prior work (Olsson al., 2022: Nanda al., 2023; Singh et al., 2024: Chen et al., 2024; Kangaslahti et al., 2025; Kim et al., 2025; Hoogland et al., 2025). general, models with more capacity (i.e., more layers, larger hidden dimension, or more heads) learn the task more quickly and have shorter loss plateaus. Having fewer layers corresponds to longer loss plateaus, and delayed generalization (Figure 8, left). Models with smaller model di- mensions (~dd < 256) fail to generalize well when trained on the same amount of data (Figure 8, middle). Using hidden size = 128 achieves accuracy near random guessing and d = 256 only achieves ~60% accuracy while the models with hidden size d 2 512 achieve 95% evaluation accu- racy or above. Models trained with only two attention heads per layer exhibit delayed generalization compared to four eight heads per layer (Figure 8, right).
21

---

Published as a conference paper at ICLR 2026
Nomber of Layers
Model Dimension
Number of Heads
15
30
3.0
Traisin Lan
Traiding Lawe
2,3
2.5
12
1.5
1006 20000
2006 46093 3906
60093
1930
10000
2006
20008
-000
5900
10000
1009]
2000
3938 40003 3060
6800 16900
Training Steps
Training Stugo
Trakiing Stgg
Nomber of Layers
Model Dimension
Number of Heads
1.0
0.5
Byskatiin AOUPAA
Brokkattvn Acueaary
Evchatton
10068 2009)
wide 8691 saine 6000
2006
11693 2003)
16300
19300
9300 (603)
11691 2006 Véd0 cbin Vides co5a) 2069
Training Steps
Training Stepe
Tribiig Staps
Figure 8: (top) Training loss training steps for various architecture hyperparameters:: number of layers, model hidden dimension, and number attention heads per layer. (bottom) Evaluation accuracy for each hyperparameter sweep. With more model capacity (more layers, larger hidden size, or more heads), models achieve better performance and converge more quickly. There are some cases for model dimension where the model only partially converges (or does not converge all), suggesting the model needs sufficient capacity to solve this task.
Table 1: Effect of model architecture hyperparameters: layers, heads, and hidden size. For each figuration, we report metrics at the training step with maximum evaluation accuracy (up to 60,000 steps). Scores under 90% are highlighted red, indicating poor performance. Results show that models require sufficient hidden size (dimension 2 512) to learn the task effectively. In general, more capacity yields better evaluation performance. Associativity scores show high variance (and consistently lower scores) across all configurations despite consistent evaluation accuracy. Corre- sponding training curves are shown in Figure
Configuration
Evaluation Metries
# Layers
# Heads
Dim.
Eval. Acc.
Copy
Commute
Identity
Associativity
Closure
Sweep I: Number Layers
1024
93.5%
100.0%
98.4%
100.0%
78.1%
100.0%
1024
99.4%
100.0%
100.0%
100.0%
85.9%
100.0%
1024
98.6%
100.0%
100.0%
96.9%
59.4%
100.0%
1024
99.4%
100.0%
100.0%
100.0%
51.6%
100.0%
10
1024
98.8%
100.0%
100.0%
100.0%
75.0%
100.0%
Sweep 2: Number Attention Heads
1024
99.2%
100.0%
100.0%
100.0%
59.4%
100.0%
1024
95.9%
100.0%
100.0%
98.4%
64.1%
100.0%
1024
99.4%
100.0%
100.0%
100.0%
85.9%
100.0%
Sweep 3: Hidden State Dimension
128
13.3%
9.4%
17.2%
62.5%
17.2%
51.6%
256
61.3%
100.0%
87.5%
85.9%
56.2%
97.7%
512
96.9%
100.0%
95.3%
100.0%
82.8%
100.0%
1024
99.4%
100.0%
100.0%
100.0%
85.9%
100.0%
2048
98.6%
100.0%
100.0%
100.0%
87.5%
100.0%
22

---

Published as a conference paper at ICLR 2026
1.
3.5
1.0
3.0
0.8
0.8
Evaluation Accuracy
Training Loss
2.5
0.6
0.5
2.0
0.4
0.2
0.2
1.0-
0.0
10000
20000 30000
40000
50000
60000
70000
10000
20000
30000
40000
50000
60000
70000
Training Steps
Training Steps
mix_0. 0.0
mix 013
mix 0.5
mix 0.7
mix 1.0
Figure 9: Effect of group sampling probability Pmix- We train five models with the same seed (42) and architecture (4 layers, heads, 1024 hidden size), but vary the group sampling probability Pmix {0.0, 0.3, 0.5, 0.7, 1.03. (a) The training loss curves for different values of Pmix follow
consistent pattern: lower values of Pmix have steeper early drops. but longer plateaus that follow. Higher values of Pmix have shorter loss plateaus. (b) Evaluation accuracy for different values of Pmix- Training runs with higher values of Pmix tend to achieve better held-out evaluation performance.
C.2 GROUP SAMPLING PROBABILITY AND TASK DIVERSITY
To generate an in-context algebra sequence, we first sample a set of groups from the training distri- bution (see Section 2). In this subsection, we provide more details about our sampling procedure, and investigate the effect of the group sampling probability Pmix as training hyperparameter.
When sampling mixture of groups G.. for a sequence s, the first group is sampled uniformly from G. After an initial group is chosen, additional groups are iteratively sampled with replacement with probability Pmix. continuing while the total order is less than or equal to the number of variables [ = N or random draw from the interval [0, 1] exceeds Pmix. A new group is added to G, only the total order of 9, would remain less than or equal to N. Thus, choosing Pmix sequences containing exactly one group, while Pmix 1 maximizes the number of groups mixed within each sequence, up to total order VV.. Thus, Pmix can be thought as a measure of in-context task diversity. The algebra model we study in the main paper Pmix 0.7.
results in
In Figure 9a, we show loss curves and evaluation accuracy for transformer models trained with the same seed but different values of Pmix E {0.0, 0.3, 0.5, 0.7, 1.0]. Training loss curves follow a consistent pattern: models trained with lower values of Paux have steeper early drops, but longer loss plateaus. Higher values of Pmix correspond to shorter loss plateaus, but higher training loss. However, lower train loss does not necessarily correspond to higher evaluation accuracy (Figure 9b).
Recall that evaluation data excludes copying and commutative copying sequences (Section 3)- We find that models trained with higher values of Pmix tend to achieve better held-out evaluation accuracy, even though they have higher training loss (Figure 9b). One reason for this might be that sequences generated using higher values of Pmix have more groups per sequence, and thus more in-context task diversity. This aligns with findings from previous work showing that higher task diversity leads to more robust generalization (Raventos et al., 2023; Kirsch et al., 2024; Ye et al., 2024: Park al., 2025b; Wurgaft et al., 2025). Similarly, since repetition is more likely to happen in sequences with fewer groups (i.e., lower values of Pmix), models trained with lower sampling probabilities have lower task diversity (e-g.. copying much more common as a possible solution).
An additional benefit of training with higher mixing probabilities (more groups per sequence) is that models tend to achieve high evaluation accuracy (generalize) faster than lower mixing probabilities. This was initially surprising, but is consistent with Kim et al. (2025) who show that increased task diversity actually shortens loss plateaus. While having more groups in a sequence is more difficult quickly. Using mixing probability of Pmix 0.0, where only a single group is sampled per sequence, has the slowest time to held-out generalization, while higher values of pmix begin to generalize sooner.
problem, Figure 9b shows this configuration learns
23

---

Published as a conference paper at ICLR 2026
(a) Cyclic Groups
(b) Dihedral Groups
1.
0.8
0.8
Accuracy
Accuracy
0.6
0.6
C2
C2
0.4
Cs
0.2
Cs
C
0.2
Ds
Ds
C. Cio
D4
0.0
Number of Facts
Number of Facts
Figure 10: Heldout performance in-distribution groups. (a) Model accuracy cyclic groups generally increases with context length, except for very small groups which tend to degrade in performance with longer contexts. The model needs more facts achieve the same performance with larger groups. (b) Dihedral groups follow a similar trend. Larger groups get better with more facts. Ds, which has 10 elements, reaches near-perfect accuracy around 200 facts, while smaller dihedral groups converge earlier (around 75-100 facts).
(a) Train Accuracy
(b) Holdout Accuracy
1.0
0.8
0.8
Accuracy
Accuracy
0.6
0.4
0.4
0.2
0.2
0.0
0.0
50
100
150
200
50
100
150
200
Number of Facts
Number of Facts
C4x(2
C2xCxxC2
Quasigroup
Magma
a
Semigroup
Figure 11: (a) Performance on algebraic structures unseen during training (where copying is pos- sible). This includes the three unseen groups of order C₁ x Cz. Qs- and C2 x C2 x C2 (also called z3), and 3 non-group structures: semigroup, quasigroup. and magma. The model achieves comparable performance on the unseen groups as does to the in-distribution order groups, while quasigroups and magmas have worse accuracy. (b) Model performance unseen algebraic structures. The hold-out performance of the model is surprisingly good for all groups, well the semigroup. However, holdout performance on the quasigroup is poor, only achieving max of 50% at 200 facts and the model performs even worse on the magma (near zero).
held-out sequences for
C.3 PERFORMANCE ON GROUPS AND NON-GROUP ALGEBRAIC STRUCTURES
In this section, we compare the model's performance in-distribution to the model's performance on groups not seen during training, as well as non-group structures. Figure 10 shows the model's performance in-distribution cyclic and dihedral groups. Performance typically increases with the number of facts in the sequence, and groups with more elements take longer to achieve perfect accuracy. For C3. the performance actually begins to decreases after 25 facts.
24

---

Published as a conference paper at ICLR 2026
0.8
Accuracy
0.6
0.4
D4
Quasigroup
0
50
100
150
200
Number of Shots
Figure 12: Model performance on cancellation-basee subset quasigroup sequences. Although the overall hold-out performance of the model on quasigroups is poor (S 50%) (see Fig. 11b), we find that performance is much better on quasigroup sequences where cancellation gives a unique answer, getting up to 100% around 50 shots, and doing cancellation just well in-distribution groups. This suggests the closure-based cancellation mechanism learned by the model a general- izable symbolic mechanism that does not depend on the specific algebraic structure of the data.
For unseen group structures of order the model still performs very well (Figure 11). The holdout accuracy is similar to that of in-distribution groups. The model also able to solve the semigroup with near-perfect accuracy with enough facts in the sequence, while hold-out performance (where copying is not possible) on quasigroups and magmas is significantly worse
Finite quasigroups naturally solvable via the cancellation law, thus evaluate a subset of quasigroup sequences where cancellation can solve the problem. We find that on this subset, the model does much better than the overall hold-out accuracy previously reported, providing evidence that some mechanisms (i.e.. mossure-based cancellation) learned by the model are generalizable sym- bolic mechanisms that not depend on the specific algebraic structure the data sampled from, as long as the data possesses that property.
C.4 ADDITIONAL PERFORMANCE DATA SUBSETS
We extend Figure 3c to show performance data subsets for varying number of facts (Table 2).
Table 2: Model performance on different data subsets from The model gets near-perts near-perfect
curacy (97 100%) on almost all sequences, except for those solved via associativity, on which it maxes 65% for 5-fact sequences.
Number of Facts
Key
5
10
25
50
75
100
150
200
D copy
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
commute
92.0%
89.0%
95.0%
98.0%
98.0%
99.0%
100.0%
99.0%
D idnntity
94.0%
97.0%
99.0%
99.0%
100.0%
100.0%
100.0%
98.0%
D Dasociate
65.0%
62.0%
66.0%
60.2%
62.0%
56.5%
50.0%
40.0%
D Denncel 57.0%
75.0%
94.0%
97.0%
94.0%
92.0%
81.0%
76.0%
D ADDITIONAL RESULTS ON COPYING
In this section, provide additional experimental details and results related to the copying and commutative copying mechanisms. Figure 13a shows a heatmap the average causal effect of patching from verbatim copying sequences into no-copy sequences for each attention head in the
25

---

Published as a conference paper at ICLR 2026
1.0
1.0
Layer
0.5
0.5
0.0
0.0
0
2
3
4
5
6
1
2 3
5
7
Head Index
Head Index
Figure 13: (a) Average causal indirect effect (Equation 5) for each attention head when patching from copying sequences into non-copying sequences, where darker green indicates a stronger change in probability. A single head (layer head 6) is strongly implicated verbatim copying behavior (AIE=0.91). (b) The same head is implicated when performing patching from commutative copying sequences into non-copying sequences, though the causal effect slightly weaker (AIE=00479),
1.0
1.0
Copy Match Pereentage
Copy Match Pereentage
0.8
0.6
0.6
0.4
0.2
0.2
0.0
0.0
3.0
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.0
3.1
3.2
3.3
3.4
3.5
3.6
3.7
Head Index
Head Index
Figure 14: Decoding the output of each attention head at the final layer via the model's unembedding matrix reveals how often an attention head's highest logit matches the correct answer on copying sequences. (a) For cyclic groups, we see one head stand out: head 3.6's highest decoded logit matches the correct answer more than 99.5% of the time on sequences where verbatim copying is possible (green), and 97% of the time for commutative copying sequences (purple), while almost never promoting the correct answer on non-copying sequences (black). (b) For dihedral groups, where not facts are commutative, we see a similar same trend for exact copying sequences (green, Deopy), while for commutative copying head 3.6 only matches the correct answer 32.5% of the time (purple, Dcommuse). However, if we instead measure whether the highest decoded logit matches the most attended-to token, this happens 97% of the time (shown in red).
model, computed as AIE (Deooy. fi) for an attention head at layer I and head index h. Figure 13b shows a similar heatmap of average causal effects for each attention head when patching from com- mutative copying sequences into no-copy sequences, similarly computed as AIE(D commutes L h) for each layer 7 and head index h. each case, identify single attention head (layer 3, head index 6), which has a much higher AIE than other heads and is primarily responsible for copying behavior.
To understand the behavior of these heads under various data settings, we characterize each head's output using direct logit attribution, or the "logit lens" (Nostalgebraist, 2020; Elhage et al., 2021). We apply the unembedding matrix to each attention head output (i.e., (t.(k))) and compute how often the token with the highest decoded logit matches the target token.
Figure 14a shows how often each attention head promotes the correct answer token when using only cyclic groups to generate copying sequences. This computed using 200 prompts for each prompt distributions: sequences where verbatim copying is possible (s Dcopy), sequences where commutative copying possible (s E D Dcoommme), and sequences where neither form of copying is possible. The highest logit promoted by the copying head (layer 3, head 6) almost always matches
26

---

Published as a conference paper at ICLR 2026
the target answer for both verbatim (green) and commutative copying sequences (purple), but almost never on non-copying sequences (black).
However, performing this same analysis sequenees sampled using only dihedral groups yields slightly different result (Figure 14b). When verbatim copying is possible, we still see head 3.6's top logit matches the correct answer token more than 99% of the time (green), as expected. However, on sequences sampled from Doommute- this value drops to 32.5% (purple). If we instead measure whether the highest decoded logit matches the most attended-to token, this happens 97% of the time for head 3.6 (shown in red). This is curious because for these sequences, head 3.6 seems to be "blindly" copying the symbol it attends even though is not the correct answer. While this strategy would solve any commutative pair of facts, it cannot solve con-cmmmutative facts found in dihedral groups. Figure 4d, we show a related behavior where head 3.6 will attend to and promote the answers of injected, incorrect facts in addition to correct ones.
E IDENTITY RECOGNITION DETAILS
In this section, we provide additional details about the steering experiments mentioned in Sec- tion 5.3. We use them study whether the separation of identity and non-identity facts seen via PCA in Figure 5a is causally relevant to downstream predictions.
To extract the PCA steering direction, we sample 1000 sequences, ensuring that cach sequence has 200 facts, and gather final attention layer outputs the predictive token position (the "=" symbol) for each sequence. We then perform PCA on the activations, and find that the variation along the first principle component separates identity facts from non-identity facts (Figure 5a). We denote this PCA direction as Vi, and evaluate its role through several experiments. In each experiment we use a sample of 5000 non-identity sequences, each containing 200 facts.
PCA Steering Controls Query Promotion. We first test the effect of steering non-identity se- quences towards the identity cluster along V1. Concretely. this done for non-identity sequences by adding 5v, to the final attention layer output at the final prediction token. While the default be- havior of the model to predict an answer that is not present in the query fact, we find that steering towards the identity cluster using V, causes the model to predict query variables 99 99.2% of the time over the 5000 non-identity sequences we evaluate on. Upon examining the model's output logits, we find that both query variables are strongly promoted as likely answers (with near-equal logits). This suggests that V1 encodes a signal for "query promotion". Interestingly, query promotion by itself is sufficient criteria to distinguish between identity and non-identity facts, since only identity facts will have one of the query symbols as an answer.
Adding False Identity Facts Controls Identity Predictions. While steering non-identity sequences with V1 can promote both query variables, it does not reliably promote one over the other. Recall that the other submechanism identified as important for solving identity facts was identity demotion. Next, we study whether we can cause the model to treat arbitrary non-identity facts as identity facts through a combination of PCA steering and injecting false identity facts into the context.
Concretely, given a non-identity sequence with a query xy=, we replace a random fact in the se- quence with a false fact indicating that either the left-slot (x) or right-slot (y) should "act" as an identity element (e.g., =?=?, or ?y=?, where "?' could be any variable). We then steer the model towards the identity cluster using V1 as before, and measure whether the model predicts the non- identity variable (e.g., y or I:) under this counterfactual setup.
We find that using PCA steering and injecting a false left-slot identity fact causes the model predict the right variable 96.1% of the time. And similarly, PCA steering along with injecting a false right- slot identity fact causes the model to predict the left variable 79.6% of the time. These results show that we control whether model treats fact as an identity fact (i.e.. uses its identity mechanisms) through a combination of two simple interventions: PCA steering along v1 to induce query promotion, and false identity fact injection to suppress specific variable.
Steering Away from the Identity Cluster. Injecting a false identity fact into non-identity sequences confuses the model, causing it wrongly predict one of the two query symbols. We find that steering away from the identity cluster (subtracting v, instead of adding) can crase this confusion, causing the model to predict the correct non-identity answer 97. 97.0% of the time.
27

---

Published as a conference paper at ICLR 2026
Sequence:' hp=e, A1=i, no=c, pttp_.....cc.ppprrrr.eeppppppppp pa=h, eh=n, la=a, pp=f, fe=h, th=e, il=i, ef=h, hh=f, ep=h, pe=h, hh=f, en=e, bi=l, (hc=/bi=)
10-
I.
Logits
5-
Logits
5
0-
0-
e & keiijkilnn kOmop
h @bodefghijklma
b
Figure 15: Closure submechanism ($5.4). When predicting the right-slot of a fact, the model pro- duces nearly uniform logits over all variables previously associated with the left-slot in the context. Here, we show the logits the left-slot for the same sequence that differs only in the final query fact (he=, vs. bi=). For different left-slot variables (h vs. b), the model produces higher logits over either (a) the six elements connected to {c, e, J, h, n. p) (shown red), or (b) the four variables associated with b: {a, b, (shown in blue).
F CLOSURE AND ELIMINATION SUBSPACES
In this section, we provide additional details about results related to the closure and elimination subspaces described Section 5.4.
At the left-slot of given fact (c.g., /c=),,=)--0 the "goal" of the model is to predict any variable that could be associated with the left-slot variable (e.g., h). This requires identifying all variables previously connected to h in the context. This set of variables is precisely what we call the "closure" of the group. We find the model is very good this task, producing near-uniform logits over previously seen elements of the query's group, example of which is shown in Figure 15.
We quantify the model's ability to compute the closure by measuring the top- K matching accuracy at the left-slot position. We identify the K variables with the highest predicted logits. Top- K matching accuracy then computed as the proportion of these top- predictions that correspond to variables from the corresponding group Gi that have appeared in the context so far. Perfect performance means the model assigns the highest logits exactly to the K group members seen in context, regardless of their relative ordering. We also report top-1 accuracy, which is whether the highest logit is one of the variables in G. Over a batch of 2000 randomly sampled algebra sequences with 200 facts, we find that our model gets 100.0% top-1 accuracy, and 100.0% top-K matching accuracy, indicating it has correctly learned how to compute within-group closure.
F.1 HOW ARE CLOSURE AND CANCELLATION SETS COMPUTED?
In this section, we investigate how the set difference operation introduced in Section 5.4 is imple- mented by the transformer model. For given query zy=, recall that the closure set Sekouuee contains all elements that have previously appeared in the context associated with y, and the cancellation Seaneet the set of answers from previously seen facts that share either its left-slot y its right-slot. Upon examining attention patterns and attention head outputs via the logit lens (Nostal- gebraist, 2020), we find evidence that these two sets are built up from contributions across several attention heads. We find that the closure computation is reused across token positions, and also describe a few heads implicated constructing the cancellation set in more detail below.
Closure Computation. We find that the same circuitry used to compute group closure is reused across token positions: both at the left-slot prediction, and at the predictive token ("='). While measure the model's ability to compute closure using facts' left-slot predictions, that group closure dominates predictions at the predictive token during early phases of training producing near-equal logits of group symbols until other strategies learned (Figure 6b, orange).
also find
28

---

Published as a conference paper at ICLR 2026
(b)
(a)
Head 3.2
k ac=h, cc=n, nk=a chcck Re=h bb=a, hcck, anp=,
nnann, a acch, nb-cc, ec-n,, ec
.ac=h,
(c)
Head 3.4
(d)
.nkka, kk==a,kk=B.. ac= ec=n, cc--nncc-n...............-- ce-.nncccec....eee.... bockk bc- ac-aahce bhck an=t, anst,
ac=h, nb ec=n, n ec==n,ecen, kce, maan ne=h acchh
Figure 16: Cancellation Set Construction. Several attention heads in the model work together to build the cancellation set and contribute the cancellation subspace. (a) Typical attention pattern of Head 3.2, which primarily attends to the answer-slot of facts that share the same symbol as the query's 's left-slot. (b) The attended-to tokens (e.g., {a, h, e, n)) have their logits demoted from head 3.2's output contribution (red). (c) Typical attention pattern of Head 3.4, which primarily attends to the answer-slot of facts that share the same symbol as the query's right-slot and (d) similarly demotes the attended-to token's 's (e.g. {e, h, n, k, aa) logits (red).
Without additional structural information (e.g., that variables represent group elements), randomly predicting symbol from the closure is near-optimal strategy. But as data-specific structure is learned, the closure prediction is built upon by other mechanisms such as the elimination subspace.
Cancellation Set Construction. A few attention heads at the final predictive token exhibit atten- tion patterns that are suggestive of partial cancellation law behavior. For example, head 3.2 primarily attends to answer-slots of facts that share the same symbol its left-slot as the query (i.e., facts of the form 2? ?, where ? can be any variable token, see Figure 16a). We find that head 3.2 places an average of 74.4% of its attention probability mass on answer-slots of facts that share the same left-slot symbol the query (averaged over 200 prompts). After attending to these tokens, head 3.2's attention contribution subsequently demotes the logits each answer token (Figure 16b). We find another attention head (layer 3, head index 4) that primarily attends answer-slots of facts that share the same symbol in its right-slot as the query (i.e., facts of the form ?y = ?, see Figure 16c), doing 57.1% % of the time. Similarly, head 3.4 demotes the logits of the answer-slot tokens it attends to (Figure 16d). These examples show how multiple attention heads help identify set of tokens that should be eliminated answers, and find that learning low-dimensional subspace over the attention layer can cleanly capture the corresponding cancellation subspace.
F.2 SUBSPACE CONSTRUCTION
Here we describe how we construct a learnable subspace that can characterize multi-dimmensionll high-level variables such the closure and cancellation sets described in Section 5.4.
We learn a set Householder unit-vectors {uz E IRd, ||>;|==1} (where is the model's hidden dimension), to construct a series of Householder matrices, H, = 2mzu?" that are composed to form an orthogonal matrix Q = Hg ... Hl E Rddd (Householder, 1958). The first columns of Q, denoted Qk dd form an orthonormal basis for our intervention subspace. We construct our subspace projection as W = Qu and perform subspace-Level interventions by mixing information between activations of the model on sequences and s' as shown in Equation 10:
Wh, + W h2
(10)
where ht represents an activation taken from the model under sequence s, h&g represents an activa- tion taken from the same location under sequence s', and means the activation hy is replaced with the intervened representation Wh, + (I W'h... While we use h, to denote "activation" here, it could be representation taken from any location in the model.
29

---

Published as a conference paper at ICLR 2026
1.0
LO Atin
0.00
0.00
0.01
0.03
0.03
0.08
0.12
0.15
0.13
0.13
LO MLP
0.00
0.00
0.00
0.00
0.00
0.01
0.02
0.02
0.02
0.02
LO Block
0.00
0.00
0,00
0.01
0.02
0.03
0.05
0.06
0,06
0.07
0.8
LI Attn
0.00
0.01
0.03
0.05
0.08
0.12
0.14
0.17
0.17
0.17
Intervention Accuracy
LI MLP
0.00
0.01
0.02
0.04
0.05
0.08
0.09
0.11
0.12
0.11
0.6
L1 Block
0.01
0.03
0.07
0.11
0.18
0.19
0.26
0.27
0.29
0.31
L2 Attn
0.01
0.03
0.04
0.07
0.10
0.14
0.19
0.23
0.26
0.27
L2 MLP
0.02
0.02
0.05
0.07
0.08
0,15
0.18
0.22
0.24
0.22
0.4
L.2 Block
0.02
0.06
0.13
0.17
0.27
0.37
0.55
0.66
0.64
0.70
L3 Attn
0.07
0.30
0.53
0.76
0.88
0.95
0.99
1.00
1.00
1.00
0.2
L3 MLP
0.00
0.00
0.01
0.01
0.03
0.03
0.04
0.04
0.04
0.04
L3 Block
0.00
0.00
0.01
0.02
0.02
0.03
0.01
0.02
0.02
0.02
0.0
4
6
8
10
12
16
20
24
32
Subspace Dimension
Figure 17: Identifying the closure subspace. determine the intrinsic dimension and location of where the closure set is represented by the model we train closure subspaces, sweeping over model components (attention layer output, MLP layer output, and full layer output) and over the number of dimensions (A E (2, 4, 6, 8, 10, 12, 16, 20, 24, 32)). For each configuration, we report the validation accuracy of intervening with that subspace. We find that the closure set computation best captured (2 99%) in the final layer attention output with an intrinsic dimension of 166 which is exactly the number of variable tokens in the model's vocabulary.
F.2.1 SUBSPACE TRAINING DETAILS
We train closure and elimination subspaces over different model components (attention layers, MLP layers, full layers), and sweep over different values for the subspace dimension k {2, 4, 6, 8, 10, 12, 16, 20, 24, 32). We measure the interchange intervention accuracy (Wu et al., 2023; Prakash et al., 2025) for each trained subspace and report the results in Figure 17.
We find that the closure set computation is best represented in the final attention layer and that 16-dimensional subspace sufficient to achieve near-perfect intervention accuracy (99.9%), and that it converges pretty quickly during training (Figure 18a). Subspaces trained at most locations unable to meaningfully influence the construction of the closure set. It can be partially controlled (~70%) by subspace trained on the input to the final attention layer (i.e., the layer 2 output is the same location as the layer attention's input), but requires more dimensions and doesn't 't achieve as clean of performance as the final attention layer (Figure 17).
For the elimination subspace, we find that training on counterfactual sequence pairs similar to those used for the closure subspace (d 16) results in subspace that plateaus around ~75% intervention accuracy, even after extended training (Figure 18b, red). Analyzing the subspace intervention failure cases reveals that 99.9% of these sequence pairs fit into one of two categories: either (i) at least one sequence ends in an identity fact, or (ii) the counterfactual target renders the query identity fact under intervention. Because of this, we choose exclude sequences that end in identity facts (and counterfactually-ipee
identity facts) from the training distribution of the elimination subspace. We find that without identity sequences, the elimination subspace quickly achieves near-perfect intervention accuracy (Figure 18b, blue).
The inability the elimination subspace to capture the model's predictions on identity sequences suggests that the mechanism the model uses to solve identity sequences is distinct from the closure- based cancellation mechanism under study here. The identity mechanism explored in more detail in Section 5.3. The fact that identity facts easily separated from other facts under PCA might explain some of the reason why training the elimination subspace fails to converge on identity se- quences, though this distinction doesn't matter for training the closure subspace (Figure 1 18a, red).
30

---

Published as a conference paper at ICLR 2026
(a) Closure Subspace (d=16)
(b) Elimination Subspace (d=16)
1.0
1.0
Val. Intervention Accuracy
Intervention Accuracy
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
50
100
150
200
250
50
100
150
200
250
Training Steps
Training Steps
Exclude Identity Facts
Include Identity Facts
% Non-Identity Facts
Figure 18: Subspace Intervention Accuracy. We train subspaces to represent the closure set and cancellation used by the model ($ 5.4). (a) The closure set promotes the variables of particular group in a sequence can be well-approximated by a 16-dimensional subspace (which the same the number of variables in the model's vocabulary), and achieves very good intervention accu- racy both when identity facts are excluded (99.9%, blue) or included (97.5%, red) in its training data. (b) The cancellation set eliminates possible answers via the cancellation law and can also be well-approximated by 16-dimensional elimination subspace if identity sequences are excluded from its training data (blue). When identity facts are included, the trained subspace only achieves 75% intervention accuracy, suggesting that identity facts are solved differently. Interestingly, the percentage the holdout data distribution that are not identity sequences is 71.1 ($ 4.1), shown as dotted black line, and this roughly aligns with the intervention plateau of the elimination subspace.
LO Attn
0.00
0.00
0.01
0.03
0.04
0.07
0.13
0.13
0.13
0.14
LO MLP
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
LO Block
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.8
LI Attn
0.00
0.01
0.03
0.05
0.06
0.11
0.13
0.17
0.16
0.17
Intervention Accuracy
LI MLP
0.02
0.02
0.03
0.03
0.03
0.06
0.07
0.08
0.09
0.09
0.6
L1 Block
0.04
0.08
0.07
0.12
0.13
0.16
0.17
0.18
0.21
0.20
L.2 Attn
0.01
0.01
0.04
0.07
0.10
0.12
0.21
0.21
0.27
0.28
0.4
L2 MLP
0.01
0.01
0.03
0.03
0.04
0.06
0.08
0.09
0.12
0.13
L2 Block
0.06
0.10
0.15
0.16
0.24
0.38
0.44
L3 Attn
0.06
0.29
0.57
0.77
0.88
0.92
1.00
1.00
1.00
1.00
0.2
L3 MLP
0.01
0.06
0.14
0.14
0.13
0.13
L3 Block
0.00
0.01
0.01
0.01
0.01
0.02
0.01
0.01
0.01
0.00
0.0
4
8
10
12
16
20
24
32
Subspace Dimension
Figure 19: Identifying the elimination subspace. We also train elimination subspaces, sweeping model components (attention layer output, MLP layer output, and full layer output) and over the number of dimensions (k: E {2, 4, 6, 8, 10, 12, 16, 20, 24, 32}) to determine where the cancella- tion is represented by the model. For each configuration, we report the validation accuracy of intervening with that subspace. We find that the cancellation set computation is also best captured
(2 99%) in the final layer attention output with an intrinsic dimension of ~6. the same as the number of variable tokens in the model's s vocabulary.
31

---

Published as a conference paper at ICLR 2026
Einieriie Prokes
14
(A52
0.30 0AA
0:0
0.45
6-4500
8:6
647
(44 0:00
0:6
0:00 0,30
0:00
044
0.2
0.51 1650
0.00
0:47 1:48
0:45
1:45
0.49
0:47
455
+40.30
6:5 0.88 6:40
0:48
4:49 0,90
0:50
0.50
646
0.90
0:51 9.50
0:51
0.45 a
0:45
0:66
0.50
(-0.5) 0:8
0.4
0.51
IUS0
648
0.51
0.00
0.50
1.50 0.49
0.53
9:8
9:8
(V:0 6:13 4.51 0/b (:51
0.50
1-4,50 8.49 947 844 0.4 0.52 0.3 WA0 8.49 0.30
044 0:00
040
0,50
0.49
4.44 2.46 1:48
8.50
0.45
0,45 6.47 3:40
6:4
0.53 0.48
0.52
0.47
1.0.50 6:58 0ut9 a:de 0.51 0:00 5.000550000 0.50 651 0.51 6:46
0.90
0:50
2-4655 4T
0.84
4: 0.66
6:45
6:47
0.66
10.50 948 0.51 648 BA0 2.52 0.50
0:5
2:49 0:51 0.4
0:51
0.32
2:50
6-77
0.47 6.51
0.45
6:47 0.48
-43
1-500 9:59
0,50
N.99
3:50
6:49
0.1
0.50
948
3.53 0,07
0.93
0.51 51
0.52 0.00 0.49
0.51
Tooied Systel
Toired
Figure 20: Closure and elimination probe generalization accuracy. We train individual probes (col- umn) on the closure and elimination subspaces for each symbol (row) to test for the presence of each variable in the closure set or cancellation set. (left) Probe are accurately able predict closure set membership within the closure subspace. Probes achieve between 91-96% accuracy for the symbol they are trained on, and chance performance on all other symbols. (right) Probe accuracy for pre- dicting cancellation set membership using the elimination subspace. Overall accuracy is lower than for the closure probes. but still decent overall. The probe for a achieves 77% accuracy, perhaps due to the fact that during training was never seen as the identity element, but all other probes classify ciiiinatioo-mmmbrrhhee with 81 -7% accuracy.
We also perform sweep over layers and dimensions for the elimination subspace and find that the cancellation set is best represented in the final attention layer with a subspace of ~1 dimensions (Figure 19). While it is the same size the closure subspace, we find these subspaces are distinct from each other, as they are unable to produce valid counterfactuals in the opposite setting.
F.2.2 PROBING THE CLOSURE AND ELIMINATION SUBSPACES
We train probes to better characterize how the trained closure and elimination subspaces represent the closure set and cancellation set. We train individual probes to detect whether variable is in the group closure or cancellation set using both the closure and elimination subspaces (Figure 20). We find the probes can accurately predict with high accuracy whether variables are in the closure set (91-96% (Figure 20, left), and with good, but slightly worse accuracy for predicting whether a variable in the cancellation set (77-87%) (Figure 20, right).
We compare the directions of the learned probes with the unembedding directions of the model to check how aligned they are with the output space. We find that the closure probes show a weak positive alignment (average cosine similarity of +0.38) with the unembedding vectors with the corresponding variable (Figure 21, left). This matches our intuition that the closure subspace does in fact promote (increase the probability of) the variables that are present in the group closure.
In contrast, the elimination probes show weak negative alignment (average cosine similarity of -0.17) with their corresponding unembedding vectors and near-zero similarity with off-diagonal entries (Figure 21, right). This suggests the elimination subspace "demotes", or lowers the probabil- ity of variables that are present in the cancellation set, again matching our intuition from Section 5.4.
We also compute the cosine similarity between the trained probes for both subspaces and find that for each subspace the probe for one symbol is independent of those for other symbols (Figure 22).
32

---

Published as a conference paper at ICLR 2026
Clesao Prebos
Hlnnmnbon Proles
0.45 0.22 0.8 0.60 0Lt5 0uz2 0.85
1.00
19.0000002 2.81
9.75
4-0000 04-330
170.83
0.33 ALLP 4120-1101
0.90
0.34 0181
0.25
4225
0.35 0.83
0.02
0U24 0.54 2-011-3
AL26-0
01
01 0.24 0.02 /3.5 0.22
0.6 0.40 0.36 -1.33- 02
0,06 0.02
2.83-0 00.0000..00000000000 00.0000000000 00-003333300 -2900-2..330000
0181
00003 0.81
0.1 0:11
040.01 281
0222 0.56 02300223000000
Prob: Tnonntte Symbel
Figure 21: Cosine similarity between learned probe directions (columns) and model unembedding vectors (rows) for the closure and climination subspaces. (left) The probe directions in the clo- sure subspace weak positive alignment with the model's unembedding direction for their respective variable with an average cosine similarity of +0.38, suggesting the closure subspace does indeed promote those tokens when present. (right) The probe directions in the elimination subspace show weak negative alignment with the model's 's unembedding direction for their respective variables, with an average cosine similarity of -0.17, suggesting the elimination subspace demotes the those tokens when present, though the effect is less strong than seen for the closure subspace.
Closue Probes
1.30
Bleniascis Proles
1.00
075
A75
Lote- 10-0011-0
5.00
4.11
0.54
1/0
6:0
(111 0/6
6-61 601
4:02
11
05-0009
8.0
8/14
005 815 4-961.31 0.16 9
18-400
8
oup
0.12 00--001-440
0011 0101
0009 8:00
0.000000
8.00
2.57 4u2 0:0
0.01
4.00
A0D 8:0 8:22 0.15 0.0 8.008
0.04 0.04 6.0-1004
1:00
OVD 4
06-0000000
8.04
422
OHIER Sealiit
J.ITAIT 4444445550000
116
0.00 8:44 4.00 0.01 6.11 0.11-0000000
1e -0--0-2--11 124:11 11111-1111--
12443
0.6
016-446
100
2.10
0.10
ADC
31 nal 6:11 om 426.4
A.een21 861 866 0.6 4
0.10 6655
0.00
141112 40inas n
12-420-115
1-460 0115811 4.12
883 4015-
15-011-600
0.94- 4 6.11 1.5 0.94
8:14
022 015 0.06 6.11 035
100
DO ALL1 11 A000- 89-4 16-U29 A1 24
02 4001
1.
564
40.2 6,16 0.06 48 101-919- 4.0 4.00 4002
850 40.11
-034
1.0
1.9
12 ALB5- 4000- 16 500 3.83 2 10-
14.00
1TTT 0.64 8.11 0.2 0.0 500 025 0.10 O07 8.18 0:07 4.00
609
1.50
Figure 22: Cosine similarity between learned probe directions for closure (left) and elimination (right) subspaces. Each probe is trained to detect specific symbol in the respective set. High diagonal values indicate probes learn distinct, symbol-specific directions, while off-diagonal values near zero suggest orthogonality between different prohe directions. The elimination subspace probe for the a variable shows a distinct trend from other probes - it has mostly negative cosine similarity with the other climination probes which might be explained by the fact that a was not seen as an identity element during training.
33

---

Published as a conference paper at ICLR 2026
G DATA COVERAGE PSEUDOCODE
In this section, we provide some pseudo-code examples showing how we check the coverage of each algorithm hypothesized in Section 4. Optimized code implementations can found through project website: algebra.baul baw.bb..b......
def check sheec_copyable :sequence):
2
sequence (stel: A sequence of consecutive algebra facts.
3 ex: " ",Ik=1, kn-g, oded, kheo, inwo, ntwh, ogeg, gt=o, 1dmh, ogeg, dÉ-g'
5 facts sequence. appini (G)))
quary facta(-1]
7
return any ([fact atpiit (1-r) 13, -- query.split ( (1-11
fact in facts:: 11:)
Code Block 1:
Verbatim Copying. A python implementation to check if verbatim copying could solve the given algebra sequence.
1 def MMeck_reverse stweerse_copbblllc (sequence):
.n* sequence (ste): A sequence consecutive algebra
3 ex: ", knng, cded, khme, nêmh, egmg, iéme, idmb, egrg, dÉmg'
*.*
5 facts sequence. apl (t))
6 quary fecta[-1]
return ( capl (faf) [9] query.eplyy (1 for fact In [acts
Code Block 2: Commutative Copying. Python implementation check if commutative copying could solve the given algebra sequence.
Ser chack Bonck__ddenttte tyysoltyyyoloooo vabll (sequence):
2
sequence (stel: à sequence of consecutive Lie algebra facts, ka=ki,
idrh, ex: kamg, oded, khee, na=e,nE-h, egmg, if-n, if-e, egmg, dE-g'
4
sequence, attpnnnce..pltt(CC (f,
[acta[-1]
left_identity [0] [-ct|--]]
query.split (")))
[acts[1: 1]]
right_identite [fact[l] [-ct[-1] and fact[0] aulry. [0; fact facta[1:-1]]
return any (leCt__deettig sight_identiigy
Code Block 3: Identity Recognition. Python implementation to check if identity recognition could solve the given algebra sequence.
der check Soock_olosure__oo wttp__lloureellllleee eminnat son__on sonsslll sol vab (sequence):
*** sequence (stel: & sequeace of consecutive algebra facts.
3 ex: 1,fk=i, kang. khee, nE-h, egmg, iê=n, idrh, egmg, dEmg' .n.
5 facts sequance.c .ppl (t,)
quary fects[-1]
share_symbol [fact foz fact in [Ic-a query[0] in fact or query[1] in ract]
chare_a_on__ie [fact for fact in fasts[l:-1] if fac:(0) query[0]] aharg_b_on_right (fact for fact in facta[1:-1) is fact query(1];
chare_symbol_oleee thare_a_on_left
shara_b_on_righ:
def oel_closuze_set (facts):
return set setin".join for in facts]) repplace'"""
eet_ollaure get_ol answer_closure clt_uuooure_set ([x[-1] for in Share_symbol_elode
pet_allourr__se osure_set (shara_ (aymbol includes
return len (set_clouuuee answer_closuret sequeace 1]
and (set_olosure anawer_closure)
Code Block 4: Closure-based Cancellation. Python implementation to check if closure-based climination rule could solve the given algebra sequence.
34

---

Published as a conference paper at ICLR 2026
def check_associative (sequence)
sequence istr): sequence of consecut Ive algebra cacta. ",fk=i, sdmg,, kh inme, nf-h,, gf-e, dE-y'
facts sequence.oplat((".
query = *sctal-1]
tripleta cntermine_aasoce at parpaara (queryl
1s_associative-Rsles for triplet in tripicts:
=ll_"acta_oxlat for fact in triplet:
fact not [1L_
each fact associative triplet
break if all_facts_exist If there's triplet of Sacts that compose to solve the query i5 jss_asociativv=-rre
is_associative
Code Block Associativity. Python implementation to check if composition of facts via associativity could solve the given algebra sequence.
H USE LARGE LANGUAGE MODELS
As per the ICLR 2026 author guidelines, provide details about use of large language models (LLMs) in the preparation this manuscript. LLMs were primarily used as general general-purpose purpose tool to aid and polish writing, both the sentence level (e-g-- grammar re-wording sentences), and the paragraph). When considering L.LM suggestions,
paragraph level (e.g.. re-organizing sentences the resulting text went through many subsequent editing rounds. LLM use did not contribute in any way that we would consider equal to the level of an author.
35


Reviews:
[{"review_id": "48AHxkLrby", "reviewer": "Reviewer_Cayz", "summary": "This paper examines how transformers solve algebraic problems. Unlike the standard setting, however, the meaning of the symbols are not fixed, and must be inferred from their context. In this setting, and unlike previous mech interp work that showed that models used geometric embeddings (famously the Fourier basis) to do arithmetic, models use symbolic reasoning in this setting. The authors use e.g. causal ablations to show that models develop three mechanisms to do this: attention heads to verbatim/commutative copying, a mechanism for identity element recognition, and something they term “closure-based cancellation” where models track groups membership to constrain valid answers.", "questions": "- How generalizable are the symbolic mechanisms might the symbolic mechanisms identified here? For example, how might these interact with the geometric mechanisms found robustly in prior work? In a mixed task with both fixed-symbol and variable-symbol arithmetic, would the model would the geometric strategy dominate?\n- It is maybe an over-claim that the PCA direction \"causally controls identity recognition”? Namely, the described modification changes the \"query-promotion\" component and must be paired with a second, separate intervention (inserting a false identity fact) for the full identity-demotion step.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "- There are a large number of experimental results. The models design a novel in-context algebra task, and perform lots of experiments. Unlike previous settings, this task isolates in-context arithmetic. It elicits symbolic mechanisms; the authors very convincingly show this.\n- Likewise, the authors bring a number of methods to bear to prove out the mechanisms described above, including computing the indirect effects at the level of attention heads.", "weaknesses": "- This task is novel, however it strikes me as contrived and very toy. I struggle to see (a) how these findings will generalize to more interesting settings, (b) what this tells us about models that we did not already know, or (c) how this work convincingly tests interpretability methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ep10Dn7UbW", "reviewer": "Reviewer_XJFb", "summary": "This work explores how transformers learn to perform certain types of symbolic manipulations, where tokens have no fixed meaning but rather have meaning that is context-dependent. This is done through an in-context learning algebra task where the model is prompted with a sequence of $k$ facts from a set of groups whose elements are randomly mapped to tokens in the vocabulary, then asked about some product in one of the groups. This design makes it so that the learned embeddings for each token cannot themselves represent the identities of the group elements. Instead, the model must learn the corresponding algebra in-context. The authors identify five simple strategies (verbatim copying, commutative copying, identity element recognition, associative composition, and closure-based cancellation) that account for some of the model's performance. They then identify how the model implements four out of five of these mechanisms in causal intervention techniques.", "questions": "- Is the task designed to be solvable for all numbers of facts $k$? I.e., does there exist a unique solution for each prompt sequence? From the random nature of the data generation process described, it seemed like the answer is no. If so, it would be interesting to compare the model performance to the ideal algorithmic performance. How close does the transformer model get to this optimal level of performance?\n- Why is the \"associative composition\" strategy missing from Figure 3?\n- What was the purpose of having the task be composed of multiple groups? I can imagine an alternative (simpler) task where, in each sequence, a group structure is randomly generated with group elements being randomly assigned tokens in the vocabulary. What would the difference be between these two tasks for the purposes of the investigation?\n\nMinor comments:\n- the bold \"=\" sign does not render very nicely (the spacing is a bit off) in Eqs (2) and (3). It makes the two equations appear a bit out of alignment.\n- In Fig. 2a, you can use different colors rather than line styles for different lines to improve readability.\n- The \"Task Description\" section was a bit unclear on the first read. I'd suggest revising it. For example, the sentence in lines 95-96 was phrased in a confusing way (needed to look at the figure to understand). I'd also recommend explicitly mentioning that the mapping $\\phi_s$ is one-to-one and that $|H_s| < |V|$", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The problem is interesting and well-motivated. The paper is generally well-written and clearly presented. \n- The methodology is sound, and the conclusions are moderate and well-supported by the empirical evidence.\n- The in-context algebra task provides a reasonable testbed for studying symbolic reasoning, which ablates the effect of learned geometric embeddings.\n- The analysis of the closure-based cancellation mechanism is technically solid and one of the paper’s strongest contributions.", "weaknesses": "- The task probes a narrow type of symbolic reasoning. The five hypothesized strategies are relatively simple and do not fully capture the richer “meaning-free” symbolic manipulation motivating the study. The contribution would be stronger if the task enabled the discovery of more complex or novel mechanisms.\n- The causal analysis fails to identify a concrete mechanism for associative composition, arguably the most conceptually interesting of the five. The mechanisms that are identified for the other strategies are similar to transformer circuits that were identified in previous work.\n- The five hypothesized mechanisms do not fully explain model performance, especially for contexts with fewer facts $k$ (which is the more challenging and more interesting case)\n\nSee also the questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CSWzIAjmay", "reviewer": "Reviewer_sQQh", "summary": "Overall, this work is thorough, well-written, well-motivated, and is strongly situated in the literature. The causal experiments broadly support the claims made in the paper. While the scope of transformer models is limited (I would like to see how and if model depth changes the ability of the model to make completions of different type), the paper offers a deep dive into how transformers accomplish in-context reasoning in this scenario. This paper would be useful and interesting to those in the mechanistic interpretability field, and it should be accepted to ICLR.\n\n\nThe authors design a novel experiment in which small transformer models learn to complete algebraic facts where the meaning of individual symbols is new (and only defined by the context) each time. This is significant because it helps researchers understand mechanisms by which transformers learn from context rather than from pre-determined world knowledge or structure when completing tasks. The authors report that small transformers can achieve near-perfect accuracy on the task and generalize to new samples. Most of the accuracy can be attributed to five known strategies for the completions, but a small percentage of performance remains unexplained. They find three causal mechanisms by which transformers complete the task: identity element recognition, commutative copying, and closure-based cancellation. The causal experiments involve measuring IE when patching specific attention layers from completions where the strategy is possible to completions where the strategy is not possible. They find specific attentions are responsible for implementing specific strategies. These attention heads tend to deactivate (attend to themselves) when the strategy is not possible. The authors describe a clear multi-step method by which identity facts are recognized and promoted to form the completion. Lastly, the authors investigate mechanisms for closure-based cancellation, leveraging probes to track the model’s knowledge of candidate completion elements given incomplete information and subspace models which generate counterfactual answers. The mechanisms are verified with causal experiments.\n\n\nThe work is well-motivated and grounded in relevant literature on mechanistic interpretability and LLM arithmetic problem solving. \n\nMinor Comments:\nLine 409 “can encoding” -> “can encode”\nLine 481 typo \"transformner\" -> \"transformer\"", "questions": "How does transformer size impact the mechanisms learned? How does size (mostly depth) impact the performance on this task?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The work is well-motivated and grounded in relevant literature on mechanistic interpretability and LLM arithmetic problem solving. \n\nThe claims regarding learning learning mechanisms are largely validated using causal experiments.\n\nThe experiment design is thorough and sound. There is sufficient reproducibility information.\n\nThe paper is well-written and easy to follow for those in the field.", "weaknesses": "The scope of transformer model is limited to one architecture and size. It would be interesting to see how the results change as different numbers of layers (but also attention heads, representation dimension) are considered. One might predict that transformer layer depth allows for more complex sequential operations. For example, depth might lead to better performance on associative composition, which did not appear to benefit from more in-context examples.\n\n\nThere is limited to no discussion of ways in which the results *might* extend beyond this abstract problem setting", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZWDEUxf5UB", "reviewer": "Reviewer_9LQ5", "summary": "The paper investigates the mechanisms transformers learn to solve arithmetic problems defined by finite algebraic groups. Critically, the task is set in an in-context learning format where the meaning of tokens (their assignment to specific group elements) is variable and defined only within the current sequence. This contrasts with prior work (e.g., on \"grokking\" modular arithmetic) which used fixed token-to-value mappings and found the emergence of geometric, Fourier-based representations.\n\nThe authors find that in their variable-token setting, the model achieves high accuracy and generalizes to unseen groups. Through causal analysis (activation patching, subspace analysis), they identify a different set of mechanisms: not geometric embeddings, but \"symbolic\" operations. These include a dedicated head for verbatim and commutative copying, a two-part mechanism for identity element recognition (query promotion + identity demotion), and a subspace-based mechanism for closure-based cancellation (tracking group membership and eliminating invalid answers). The paper concludes that geometric representations may be an artifact of fixed-symbol tasks and that models default to symbolic reasoning when meanings are context-dependent.", "questions": "- The model's difficulty with \"Associative Composition\" (Fig 3c) is very interesting, as this is a core algebraic property. I was surprised to see it wasn't included in the causal analysis. Could the authors elaborate on why this was omitted, and perhaps share any preliminary findings on why the model struggles here? This seems like a fascinating area for future work.\n- Could the authors provide more mechanistic detail on the \"Closure-Based Cancellation\" (Sec 5.4)? The text describes it as a set difference $S_{closure}$ - $S_{cancel}$. How are these sets represented in their respective subspaces, and how is the \"set difference\" operation mechanistically computed by the model? The high-level concept is clear, but a deeper dive into the \"how\" would be fantastic.\n- The model's ability to generalize to unseen groups (Fig 2c) while failing at associativity is a curious result. Does this suggest to the authors that the model has learned general heuristics (copying, identity, cancellation) that work for many group-like structures, rather than a deep, rule-based understanding of group axioms? The generalization to non-groups (semigroups, magmas) seems to support this, and I'd be interested in the authors' take.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The \"in-context algebra\" task is a novel and interesting probe for mechanistic interpretability. By forcing token meanings to be sequence-local, it creates a clean setting to study how transformers perform reasoning when they cannot rely on pre-trained, fixed-meaning embeddings.\n\n- The paper goes beyond simple performance metrics and attempts a rigorous mechanistic decomposition of the learned algorithm. The use of causal interventions (patching) to isolate specific heads (e.g., Head 3.6 for copying) and the PCA/subspace analysis to understand identity and closure mechanisms are commendable.\n\n- The finding that mechanisms are multi-part (e.g., identity recognition being a combination of \"query promotion\" and \"identity demotion\") is insightful and adds a layer of depth beyond just \"this head does X.\"", "weaknesses": "- The paper's central thesis is that its contribution is \"challenge the hypothesis that geometric embeddings are the primary mechanism\" for algebraic problems. This framing may be slightly marginal for a conference. The prior work on geometric/Fourier-basis representations documented the phenomenon specific to a task with fixed token meanings. It might be more constructive to frame this paper's findings as a complement to that work, showing an expected and important contrast: when the pre-conditions for geometric embeddings are removed (i.e., by making token meanings variable), models develop a different, symbolic-like strategy. This distinction is a valuable contribution in itself, showing how task design dictates the learned mechanisms.\n\n- *Weak Experimental Analysis*: The paper's mechanistic analysis is a great start, but feels incomplete in a few areas that would make the claims more robust.\n  - **Omission of Associativity**: The authors hypothesize five mechanisms (Sec 4) but find their model performs poorly on \"Associative Composition\" (60.2% accuracy at k=50, Fig 3c). Associativity is arguably the most fundamental property of a group. The paper would be much stronger if it included a causal analysis of this failure. Why does the model fail at associativity? What partial or heuristic-based mechanism does it learn instead? Analyzing this failure mode could offer even deeper insights than analyzing the successes.\n  - Because the analysis (understandably) focuses on the learned mechanisms, it's less clear what generalizable insights this task provides. The mechanisms (copying, identity, elimination) are powerful heuristics, but they aren't unique to group theory. It would be great if the authors could further discuss whether the model has learned \"algebra\" or a set of clever, task-specific heuristics.\n\n- *Ablation Studies*: The analysis is based on a single \"toy\" model (4-layer, 8-head) trained from scratch. The findings would be more generalizable if the authors could provide some ablation studies. For example, how do these mechanisms change with model depth/width, or other hyperparameters? This would help confirm that these mechanisms (e.g., Head 3.6) are a general solution rather than an artifact of one specific architecture.\n\n- \"From Scratch\" vs. Pre-trained: Relatedly, the \"from scratch\" setting is interesting, but a very relevant follow-up would be to explore how a pre-trained LLM tackles this task. Would it develop these same mechanisms, or would it repurpose existing in-context learning circuits? Exploring this could bridge the gap between toy models and real-world LLMs.\n\n- **Tokenization Details**: A small but important detail that would be good to clarify is tokenization. The paper implies each variable is a single token. It would be helpful to explicitly confirm this and perhaps discuss how the task difficulty might change if variables were multi-token strings, which is a common scenario in natural language.\n\n- I also notice authors may also may be great mention some relevant work like [Language Models are Symbolic Learners in Arithmetic](https://arxiv.org/abs/2410.15580) which they use subgroups to derive symbolic behaviors in LM and a concurrent work [Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls](https://arxiv.org/abs/2510.00184) (concurrent). Situating this paper's findings relative to these would be very interesting for the community.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)