Function Induction and Task Generalization: An Interpretability Study with Off-by-One Addition
Authors: Qinyuan Ye
Robin Jia
Xiang Ren
Venue: ICLR 2026 Poster (2026)
Keywords: interpretability
language models
task generalization
induction heads
Primary area: interpretability and explainable AI
TL;DR: We interpret how language models perform off-by-one addition (1+1=3, 2+2=5, 3+3=?), uncover a function induction mechanism, and show that it enables broader task-level generalization.

Abstract: Large language models demonstrate the intriguing ability to perform unseen tasks via in-context learning. However, it remains unclear what mechanisms inside the model drive such task-level generalization. In this work, we approach this question through the lens of off-by-one addition (i.e., 1+1=3, 2+2=5, 3+3=?), a two-step, counterfactual task with an unexpected +1 function as a second step. 
Leveraging circuit-style interpretability techniques such as path patching, we analyze the models' internal computations behind their performance and present three key findings. First, we identify a mechanism that explains the model's generalization from standard addition to off-by-one addition. It resembles the induction head mechanism described in prior work, yet operates at a higher level of abstraction; we therefore term it "function induction" in this work. Second, we show that the induction of the +1 function is governed by multiple attention heads in parallel, each of which emits a distinct piece of the +1 function. Finally, we find that this function induction mechanism is reused in a broader range of tasks, including synthetic tasks such as shifted multiple-choice QA and algorithmic tasks such as base-8 addition.
Overall, our findings offer deeper insights into how reusable and composable structures within language models enable task-level generalization.

Published as a conference paper at ICLR 2026
FUNCTION INDUCTION AND TASK GENERALIZATION: AN INTERPRETABILITY STUDY WITH OFF-BY-ONE AD- DITION
Qinyuan Ye Robin Jia' Xiang Ren'
University of Southern California Salesforce Al Research (qinyuany, robinjia, xiangren]@usc.ede
ABSTRACT
Large language models demonstrate the intriguing ability to perform unseen tasks via in-context learning. However, it remains unclear what mechanisms inside the model drive such task-level generalization. In this work, we approach this question through the lens of off-by-one addition (ie., 1+1=3, 2+2=5, 3+3=?), two-step, counterfactual task with an unexpected +1 function as second step. Leveraging circuit-style interpretability techniques such as path patching, we analyze the models' internal computations behind their performance and present three key findings. First, we identify a mechanism that explains the model's s generalization from standard addition off- off-by- off-by-one addition. It resembles the induction head mechanism described in prior work, yet operates at higher level of abstraction; we therefore term it "function induction" in this work. Second, we show that the induction of the +1 function is governed by multiple attention heads in parallel, each of which emits a distinct picce of the +1 function. Finally, we find that this function induction mechanism is reused in a broader range of tasks, including synthetic tasks such as shifted multiple-chen choice QA and algorithmic tasks such as base-8 8 addition. Overall, our findings offer deeper insights into how reusable and composable structures within language models enable task-level generalization.
INTRODUCTION
As the capabilities of language models (LMs) continue to grow, users apply them to increasingly challenging and diverse tasks, accompanied by evolving expectations (Zhao et al., 2024; Tamkin al., 2024; Kwa et al., 2025). Consequently, it becomes impractical to include every task of interest in model's training prior to deployment. this context, task-level generalization the ability of a model perform novel tasks at inference becomes highly erucial and valued.
Prior work shows that LMs already exhibit this capability to a significant extent through in-context learning (Brown al., 2020; Chen et al., 2022; Min et al., 2022a). The underlying mechanisms of this behavior are being actively investigated, with work on induction heads (Oisson et al., 2022) and function vectors (Hendel et al., 2023; Todd al., 2024) offering substantial insights on pattern matching tasks (i.e., [A][B].. [A] + [B]) and mapping-style tasks (e.g.. France: Paris, Australia: + Canberra). However, our understanding is still limited, especially regarding more complex generalization scenarios involving multi-step reasoning or newly-defined concepts in the task.
In this work, we aim to enhance our understanding how models handle novelty and unconvention- ality with one counterfactual task: off-by-one addition (i.e. 1+1=3, 2+2=5, 3+3=?). For humans, this task consists of two sequential steps: standard addition, followed by an unexpected increment of one to the sum. When language model is prompted to perform this task with in-context learning, we anticipate two possible outcomes: (1) the model acquires the intended +1 operation and thus outputs 7, or (2) it adheres to fundamental arithmetic rules and outputs 6.
We begin our study by evaluating six contemporary LMs on off-by-one addition. Our findings indicate that all evaluated models consistently demonstrate the first outcome, effectively leveraging in-context
Code: IIN.-USCf/uuntiion--nduucoooo

---

Published as a conference paper at ICLR 2026
examples; furthermore, performance increases consistently more shots are used. Motivated by these observations, we seek a more comprehensive understanding of how models perform off-by-onn addition, and in particular, the +1 step of the task. To this end, we employ mechanistic interpretability and path patching techniques (Wang et al., 2023), which enables us to trace the model's output logits specific set of attention heads and their interconnections responsible for +1 behavior.
Our analysis with Gemma-2 (9B) (Gemma Team, 2024) reveals that the model's computation of +1 is mainly mediated by circuit involving three groups of attention heads. Notably, two of these groups and their connections resemble the structure of the induction head mechanism described in prior work (Olsson et al., 2022) This observation leads us to hypothesize that the circuit implements a case of function induction inductive reasoning that transcends token-level pattern matching and operates at the function-levve Our analysis also reveals that the function is transmitted along six (or more) paths in the model's computation graph; in each path, an attention head writes a distinct fraction of the function, whose aggregate effect yields the complete +1 function.
We further validate the universality of our findings across models and tasks (Olah et al., 2020; Merullo et al., 2024). Regarding models, we repeat our analysis on Mistral-v0.1 1 (7B) (Jiang et al., 2023), Llama-2 (7B) (Touvron et al., 2023) and Llama-3 (8B) (Grattafiori et al., 2024), confirming the existence of the function induction mechanism, though in slightly varied forms. Regarding tasks, extend our analysis with four task cairs-ooffby-A addition, shifted multiple-choice QA, Caesar Cipher, and base-8 addition deettine to replace sub-steps in off-hy-one addition with substantially different operations. We demonstrate the reuse of the same mechanism in these task pairs.
Overall, our results advance our understanding of important language model capabilities such as in-context learning and latent multi-step reasoning. They highlight the flexible and composable nature of the function induction mechanism we have characterized, and provide substantive insights into how models may generalize when encountering novel task variations.
2 LMS LEARN OFF-BY-ONE ADDITION IN CONTEXT
Off-by-one addition is a synthetic, counterfactual task involving two steps. The first step is standard addition, and the second, unexpected step is +1 function. In this work, are interested in whether and how the model can perform this task with in-context learning. We provide concrete 4-shot examples of standard addition and off-by-one addition in Table 1. In this section, we first evaluate contemporary language models on this task and describe our observations.
Base Task
Standard Addition
4+3=7 m32==5 6+0=6 33:3= 1+0=
Contrast Task
Off-by-One Addition
4+3=8
h342-6 66:--763+3777mmm
Table 1: Example Prompt of Standard and Off-by-One Addition. Red is used to mark the base prompt and answer. Orange is used to mark the contrast prompt and answer.
Data. To create the evaluation data, we randomly sample 100 test cases, each with 32 in-context examples (a; + b; = c.) and one test example (Atest + biest = (test). We sample a, b, from the range of [0,999], and restrict that for all i, Cteat A c;- This is to make these test cases evaluate models on inducing +1 function, instead of copying and pasting the answer token (Crest) from the previous context (cz).
Models. We evaluate six recent LMs on this task: Llama-2 (7B) (Touvron et al., 2023), Mistral-v0.1 (7B) (Jiang et al., 2023), Gemma-2 (9B) (Gemma Team et al., 2024), Qwen-2.5 (7B) (Yang et al., 2024a), Llama-3 (8B) (Grattafiori et al.. 2024) and Phi-4 (14B) (Abdin et al., 2024). These models were developed by different organizations, employ different number tokenization methods, and were released in different years, thereby providing a diverse and representative sample. Please refer to Table 4 for details of these models.
IInduction heads (Olsson et al., 2022) facilitate a language model's token copying behavior in sequences like [A][B]... [A] + [B] by directly copying token [B] from the context. Our work aims to explain more abstract, liectonn-eel behavior- how models induce the function f(r) = J. + 1 from sequences like
[A] f([B) ... [C] f[D]) (e.g.. 1+1 = 3 =+ See $6 and $A for further details.
2

---

Published as a conference paper at ICLR 2026
Evaluation Results. In Fig. 1, report the accuracy when different numbers of in-context examples are used. All evaluated models exhibit non-trivial performance on this task, demonstrat- ing that this behavior is pervasive. Additionally, performance always improves as the number of shots increases, indicating effective utiliza- tion of the in-context examples. Notably, more recent models like Llama-3 (8B) and Phi-4 (14B) achieve the strongest performance, with near perfect results in 8-shot experiments. More details of our evaluation (e.g. reporting accuracy on standard addition, using a smaller number range like [0,9], removing the restric- tion of Crest o C;) are deferred to $B.
Llama'2 2 (7B)
Mistral (70))
Gemma (9B)
Qwen: 2.5 (71)
Phi-t (:4BB
100%
Costrest Accuracy
80%
60%
20%
2 24
8
16
32
Number ICL Examples
Figure I: In-context Learning Performance of Off-by-One Addition.
INTERPRETING THE OFF-BY-ONE ADDITION ALGORITHM
Off-by-one addition likely unseen or highly underrepresented in the pre-training data, yet as Fig. 1 shows, contemporary language models can effectively induce the +1 operation with in-context learning. Intrigued by these observations, we aim to interpret the model's internal computation behind this behavior. $3.1 provides brief overview of mechanistic interpretability and path patching, a line of methods that we find highly suited to our investigation. We further formalize our notation this section. In $3.2 we describe circuit discovery process and findings.
We choose Gemma-2 (98) as the default model based on our preliminary experiments ($B), and use 1I+1=3mn2224 mm+===m++== experiments below use 100 off-by-one addition test cases using numbers in the range of [0,9].²
running example in the following. Unless specified otherwise, all
3.1 BACKGROUND: MECHANISTIC INTERPRETABILITY AND PATH PATCHING
Mechanistic interpretability is subfield of interpretability that aims to reverse-engineer model computations and establish "correspondence between model computation and human-understandable concepts." (Wang et al., 2023) A transformer-based language model can be viewed as a computation graph M, where components like attention heads and MLP layers serve as nodes, and their connections as edges. We use M (y/z) denote the logit of token y when using as the input prompt. A circuit C' is subgraph of M that is responsible for certain behavior. In our study, the behavior of interest the induction and application of the function in off- off-by-one addition.
The specific method we rely on is path patching (Wang et al., 2023), which is built on activation patching (Meng al., 2022) and causal mediation (Vig et al., 2020) methods from prior work. the past, such technique has supported interpretability findings on wide range of model behaviors (e.g.. Hanna et al., 2023; Stolfo al., 2023; Prakash al., 2024b; et al., 2025).
Extending path patching to our case, we first run forward passes both the base prompt Thase (1+1=2 2+2=4 n ((]aFaase) and (. We will then (1) replace part of the activations in (-(xconn) with the corresponding activations target nodes (e.g., output logits, query of specific head) in the graph; (3) replace the activations of the target nodes in (.[Ieoont) with the activations obtained in (2). The computation graph after such replacement is denoted as M'. If such replacement alters the model's output of "3+3=7" back to "3+3=6", we would believe that the part has contributed to the computation of the +1 function.
in and contrast prompt Zoont (1+1=3) n mn+=5, to obtain the logits
(- ); (2) let the replaced activations propagate to designated
To simplify the notation, define F(C, as the logit difference between Ybase (6) and yoont when prompted with x and using the circuit C' while knocking out nodes outside C' in the computation graph, i.e,, F(C,2)) Clybose (x) C'ycona xr). Following Wang al. (2023), we quantify the effect of replacement by first computing F FMM', Iconnt) and then normalize it by the logit difference
TTo accommodate our computational resources, circuit discovery experiments ($3.2) were conducted with 4 shots (3c3uracy333%) while circuit analysis experiments ($4) were performed with 16 shots (accuracy=86%)
3

---

Published as a conference paper at ICLR 2026
(a) Patching to Outpat Logits
(a) Patching H39.7 Value
Figure 2: Circuit Discovery with Gemma-2 (9B). Top:
20%
Patching Re- sults on Selected Target Nodes.
100%
Logit DEt Vartation
Head Index
(a) We identify Group 1 heads and Group 2 heads that directly influence the output logits.
0%
12
-10%
(b) We identify Group heads that write the value of H39.7.
-20%
Bottom: Attention Pattern of Se- lected Heads. We use 4 ICL examples in the format of "a+b=c'un", Causally- marked in pink.
24
28
32 Layer
36
40
32
36
40
Layer
(c) B41 4 Att.. Pattern
(d) H39.7 Attn. Pattern
(e) 1138.7 Attn. Pottern
relevant positions (c) Group heads mainly attend to the current token and <bos>.
Querry
(d) Group 2 heads attend to the answer tokens (c:) of previous ICI. examples at the position of "=".
(e) Group heads attend to the preced- ing "=" at the position of
Key
Key
Key
F(M.acont) (buae) See $C.1 for its expansion and explanations. The resulting ratio r, which we refer to relative logit difference, will typically fall in the range of [-100% 0%). with -100% representing the model favors ybese (i.e., the model losts its ability on ooe- addition after replacement), and 0% representing the model favors Ycont.
before intervention, Le., F(M' (cont) F Rcont)
3.2 CIRCUIT DISCOVERY
Patching to the Output Logits. Our investigation begins by setting the output logits as the target node, effectively asking "which attention heads directly influence the model output?" The results, visualized in Fig. 2(a), highlight 10 attention heads with a relative logit difference |r| > 2%.
We further investigate the attention pattern of the highlighted heads and categorized them into two groups. Group 1 heads appear exclusively in the last two layers of the model, and mainly attend to the current token and the <bos> token at each position (Fig. 2(c)). Group 2 heads present periodical patterns consistent with the ICL examples in the prompt (Fig. 2(d)). Specifically, at the position the last "=" token, where the model is expected to generate the answer as the next token, these attention heads will attend to the answer tokens (c₂) in previous ICL. examples (a, = c;).
We additionally conduct path patching using the value of Group heads as the target node, revealing that Group 2 heads also write the value of Group heads which then influence the final output logits. Combining these findings, we hypothesize that Group heads are responsible for finalizing and aggregating information, while Group 2 heads are responsible for carrying the +1 function from the in-context examples to the test example.
Patching to the Value of Group 2 Heads. To further trace down the origin of the +1 function, we set the value of each head in Group 2 as the target node for path patching. For example. H39.7 (Head 7 in Layer 39) is a representative head in Group 2 with a relative logit difference r of 227 when patching to the final output. When setting H39.7's value as the target node and performing path patching, three heads are highlighted (Fig. 2(b)) and all of these heads follow the pattern of attending the previous token certain positions (Fig. 2(e)). In particular, the answer token e; in each in-context example, these head attend to the "_" token immediately before e, We repeat this procedure for remaining heads in Group 2 and identify more attention heads with the previous-token attending behavior. We collectively refer to them as Group heads.
Our subsequent path patching attempts do not uncover any new attention heads leading to significant logit differences, thus we conclude the algorithm at this point.
'See $D.1 for additional analysis using smaller threshold values. 'The chosz-attending behavior is commonly interpreted as no-op and is prevalent in transformer-based language models (Barbero al., 2025). See SC.3 for extended discussion.
4

---

Published as a conference paper at ICLR 2026
The Function Induction Hypothesis. Fig. 3 provides overview of the circuit identified, illustrating the connections of the three head groups and highlighting the token positions they operate The comprchensive list of heads in each group be found in $C.2.1 and Fig. 22(b).
We find it particularly intriguing that the structure of the circuit, in particular Group 2 and Group 3, resembles the structure of induction heads (Olsson al., 2022), known mechanism responsible behavior. In the induction head mechanism, a previous token head
for language model's s "copies information from the previous token the next token", and induction head "uses that information to find tokens preceded by the present token." (Olsson et al., 2022)
Comparing these two mechanisms, induction heads could be seen as inducing constant (zeroth- order) function f=o)tput([B]), whereas in our mechanism, a first-order function fx) + is induced. Based on this intuition, the three groups of attention heads cooperate as follows:
legend
Nexi loken pred.
key/value
output
6
abvl
Head
query
Group
Within an ICL example, at the "=" token (e.g., "I+1="), the model initially drafts answer via early-layer computations (e.g., "2"), and anticipates to generate as the subsequent token. However, at the answer token posi- tion cs, the model encounters an unexpected answer (e.g., "3"). Consequently, heads in Group 3 register this discrepancy at the posi- tion of cç- Given their previous-token attend- ing behavior, we name heads in Group 3 as previous token (PT) heads.
Group
Group 3
Early layers
Figure 3: Overview of the Identified Circuit.
In the test example portion of the prompt (e.g., "3+3="), Group 2 heads retrieve the information registered by Group heads at the "=" token, and subsequently writes the +1 function. We name Group 2 heads as function induction (FF) heads as their operation resembles that of standard induction heads but applies to arithmetic functions rather than tokens.
Lastly, refer to Group 1 heads as consolidation heads, hypothesizing their role in finalizing the next-token output by synthesizing information from various sources.
CIRCUIT VALIDATION AND ANALYSIS
Previously, we constructed the function induction hypothesis based on our path patching results and its structural similarity to that of the induction heads mechanism. In this section, dive deeper into the identified circuit, aiming to provide a more granular understanding.
Ablating FI Heads. We begin our investigation with head ablation, a common tech- nique to validate head's involvement in a specific model behavior (Halawi et al., 2023; Wu et al., 2025). Here, focus on FI heads and "ablate" head by replacing its output in the forward pass on Foome with the corresponding head output in the forward pass on Tbase. As shown in Fig. 4(a), the complete, unablated model achieved an accuracy of 86% on 16-shot off- by-one addition. Upon ablating the six Fl heads, the model's behavior switched back to standard addition, resulting in 100% accuracy standard addition and 0% on off-by-one addition. For a controlled compar- ison, also ablated six randomly selected heads; these showed minimal influence on either the base or contrast accuracy. This set of results provides preliminary evidence that the six FI heads are necessary in off-by-ont addition.
Initial Validation:
Base Ace
Contrast Acc
Prompt
Contrest Prompt
Contrast Prompt (Ablate FT Eleads)
Controst Prompt (Abiate Rand. Heads)
100%
Accuracy
Figure 4: Head Ablation Results.
Further Validation: Measuring the Causal Effect of FI Heads. In our hypothesis, Fl heads are responsible for writing the +1 function to the residual stream the "_" token. This behavior is highly

---

Published as a conference paper at ICLR 2026
H39.7
H39.12
H36.7
H32.6
0.50
0.25
Output
Output
Output
Ourput
Oatpat
-0.00
H25.13 (*)
(H32.44)
(H28.6)
(H24.9)
(Normalized)
lcput
-0.25
-0.50
5
0
5
5
Output
Output
Output
Ourput
Catput
Figure 5: Individual and Overall Effect of Identified Heads. Each head writes out different information, which aggregates to implement the function of [(x) + 1 (bottom-right panel). (*) Effects H32.6, H25.13, and H32.4 are rescaled to [-0.15, 0.15] to make the patterns more readable.
relevant to recent work (Todd et al., 2024; Hendel et al., 2023) which indicates that small number of attention heads (i.e. function vector heads; or FV heads) effectively transport task representations (i.e., function vectors) in in-context learning. The FI heads identified align with this description, and moreover, uncover novel instantiation of the mechanism that operates within multi-step tasks.
The notion of function vectors inspires effect on a naive prompt Fnaives e.g., "2=2'm3=?" for which the model is expected to assign a high probability to "3". If FI head indeed writes out the function, adding its output to the residual stream at the final "=" token should cause the model to increase its probability of generating "4".
further validate the role of FI heads through their causal
Concretely, construct the naive prompt "(x-1)=(x-I) (n (x]=?" for E [0, 9j. and track the model's 's logits for tokens [0, 9] both before and after adding the FI head output to the residual stream the corresponding layer. This leads to a 10 10 heatmap, where the value at cell (Fenputs youtput) represents the change in logits for token when the function vector is added.
In Fig. 5, we present these heatmaps for each of the six FI heads identified in $3.2. We include three additional heads (H32.4, H28.6 6, H24.9) that, while showing modest effects (1% < |r| < 2%) in $3.2, contribute meaningfully the +1 function as revealed by this analysis. We find that heads work collaboratively each of them contributes a distinct piece to the overall +1 function. For example, with an input æ, H39.7 promotes x + H28.6 6 suppress x 1, H32. promotes digits greater than H24.9 suppresses When the outputs of these nine heads added to the final residual stream altogether, their combined effect implements the +1 function, as depicted in the last panel of Fig. 5.
Universality of Function Induction. To investigate the universality of our findings across models, we repeat the path patching experiments with Llama-3 (8B), Llama-2 (7B), and Mistral-vo (7B). We identified all three groups of heads across these models, except that the two consolidation heads identified in Mistral-vo. 1 display weaker and less consistent signals. Still, these observations provide promising evidence that the function induction mechanism we've found is general and consistently emerges across various language models. See SC.2 for more details.
FI Heads (Ours) and FV Heads (Todd al., 2024) are Two Disjoint Sets of Heads. While our analysis demonstrates that Fl heads transport task representations similarly to the FV heads described in prior work, direct comparison with Llama 2 (7B) reveals important distinctions. Todd al. layers of the model (before layer 20), whereas
(2024) reported that FV heads appear in early-midde our Fl heads are located in late layers of the model (layer 29-31). There is no overlap between the two sets of heads, suggesting that our work presents distinct, previously undocumented finding. We hypothesize that heads can seen as an instantiation of the broader FV head mechanism, but are only triggered in multi-step tasks when late layers used to perform the late steps. See $C.2.4 for the full list FI/FV heads and $6 for further discussion on their differences.
Additional Analysis. Due to space limits, we defer various supporting evidence to the appendix. We conduct a rigorous evaluation our circuit using the faithfulness, completeness, and minimality criteria introduced in Wang et al. (2023). Our circuit mostly satisfies these criteria, and we discuss the results in $D. We deliberately focus on Fl heads in $4 given the interesting insights from these results. We provide further validation and analysis of consolidation heads and previous token heads in $E.

---

Published as a conference paper at ICLR 2026
5 TASK GENERALIZATION WITH FUNCTION INDUCTION
Our investigation far suggests that function induction is the key mechanism enabling the model to generalize from standard addition and manage the unexpected +1 step in off- aff-by-one addition. Given the importance task generalization for capable AI systems, we aim to explore the broader usage of this mechanism. In this section, we investigate how the identified mechanism extends to a broader set of synthetic and algorithmic tasks. Specifically, $5. introduces the four task pairs examined, and $5.2 presents the overall findings and additional analyses for two of these pairs.
5.1
TASKS
(a) Off-hy-k Addition
(c) Caeser Cipher
Standard
4+3=7 o3+2=5 0 344=m 1+0=
ROT-D
Off
4+3=9mn3 h3+2=7 +=0==8 3+3=8 1+0=
ROT-2
C <->em
>> 7
(b) Shifted MMLU
(d) Base-k Addition
Standard
[...] Answer. I[..)
Answer: (A)
Basc-10
25+16=41
60+16=76 13+35=48
52+17=
Shift-by-One
I... Answer: (C) m|....] J...]in Answer: (B)
Base-8
25+16=43 0004443360066-660 66=16=76
13435=50
71
Table 2: Task Pairs Used Task Generalization Experiments. Red is used to mark the base prompt and answer. Orange is used to mark the contrast prompt and answer.
(a) Off-by-k Addition. One extension of off off-by-on by-one addition is changing the offset to other values. representative case be reported in the main paper. Results and analysis on the other offsets are deferred to $F.
Here, we consider offsets & E {-2, -2, 2}. We use
(b) Shifted Multiple-choice QA. We consider going beyond arithmetic tasks and replace steps in off-by-one addition with substantively different steps. The base task is chosen to be multiple-choice QA questions on selected subjects the MMLU dataset (Hendrycks et al., 2021). The contrast task created with additional step to shift the answer choice letter by one letter, e.g. A-BB B-C.
(c) Caesar Cipher. One realistic task that leverages shifting functions is Caesar Cipher. During encoding. letter is replaced by the corresponding letter fixed number of positions down the alphabet (Wikipedia contributors, 2025). This task is also commonly used to evaluate a language model's 's reasoning capabilities (Prabhakar al., 2024). Here we consider single-characte Ceaser Cipher with different offsets l: E (-12, - - 11,....,0, O, .,..., 12, 13). We use k = 0 as the base task, and k 2 as the representative contrast task.
(d) Base-k Addition, Lastly, we consider the task of base-k addition, which was used by Wu et al. (2024) assess the model's memorization versus generalization. Prior work (Ye al., 2024) suggests that LMs may formulate shortcut solution for base-8 addition by interpreting it as "adding 22 to the sum" from in-context examples; our interpretability analysis helps further investigate this observation. We consider two digit base-10 addition as the base task, and base-k addition as the contrast task, with (6, 7,8, 9}. We use 8 as a representative case in the main paper.
5.2 RESULTS AND ANALYSIS
FI heads are reused in a wider range of tasks. Using the four task pairs introduced above, we examine the role of the function induction mechanism we discover with head ablation experiments, similar the one done in Fig. 4. We run forward passes on both the base task and the contrast task. We then replace the Fl heads outputs in M (-]2eont) forward pass with the corresponding head outputs in the M [base) forward pass.
We report results of the representative cases in Fig. 6. In all four task pairs, we first see non-trivial performance on the contrast task, indicating effective generalization. Upon ablating the six FI heads, we observe consistent trend: the model's contrast accuracy substantially decreases; the base accuracy increases and often returns to level comparable to that achieved with the base prompt. These findings suggest that the mechanism identified with off-by-one addition is largely reused in
7

---

Published as a conference paper at ICLR 2026
ligh:: Politiis
Addition
(d) Adddtin
Base Fromutt
Contrnlt Prompt
Contrest Prompt (Ablate P1 Iteeds)
Controet
Contrist (Abinte 6 Hand, (tends)
100% 6%
50%
100%
Accurscy
Accuracy
Accuracy
Accuracy
Figure 6: Task Generalization with FI Heads. (d), base-8 addition has non-zero accuracy with the base-10 prompt, because in these test cases the base- 10 answers happen to be correct in base-8.
0.6
Input
1139.7
Output
Output
Output
Output
Input
H25.13
5
0.0
0
0
0
Output
Output
Output
Output
Figure 7:
Effect of Two FI heads When Using Different Offsees in Off-by-k Addition.
these task pairs, which share a similar underlying structure but also represents substantially different sub-steps. This strongly demonstrates the mechanism's flexibility and composability.
also observe that in (b) Shifted MMLU and (c) Caesar Cipher, the model has non-zero contrast accuracies when the FI heads are ablated. This implies that the six FI heads found with off- off-by-one addition are useful, but not complete for these task pairs. See $F for additional discussion.
F Function vector analysis with off-by-k addition. We revisit the function vector style analysis done in Fig. 5, but this time considering different offsets le E 11, 1,2). Results on two representative heads (H39.7 and H25.13) shown in Fig. with other heads deferred Fig. 24-26.
We find that the effect of Fl heads varies meaningfully with the offset kt demonstrating their generality and consistency with the hypothesized functionality. For the selected heads in Fig. find that each of them has their own "specialty". For example, the heatmap for H25. 13 suggests its primary responsibility for writing out 2:2 functions. While its effect is stronger when the offset +22 still contributes in the case of k = +1 by suppressing the original output x.
Models struggle in base-8 addition due under- over-generalization. may sound unintuitive why the induction mechanism specialized in shifting functions could facilitate base-8 addition. One possible explanation that the model initially performs standard base- 10 addition with early layers, and apply minor adjustments when necessary. This adjustment step is possibly handled by the function induction mechanism late layers.
Following this intuition, propose one possible algorithm for two-digit base-8 addition in Listing 1. No adjustment needed when there no carrying over from the unit digit (Case 1), e.g., 60 76 correct in both base-8 and base-10. 10. When carry-over occurs, two separate considered. In Case 2, both the unit and the eight's place digit require adjustment, e.g., 13g + 35g = 50g and 1310 3510 4810. both and in 4810 need to be adjusted. In Case 3, only the unit digit needs adjustment, e.g., 25g + 16g = 43g and 2510 + 1610 = 41
needs to be
We randomly sample 100 32-shot prompts for each of these three cases, and track the model's behavior on the unit and eight's s place digit. We report the results in Table Case 1, digits are adjusted unnecessarily in 7% (=6%++1 %) of instances, suggesting meerraenerliizttion Conversely,
8

---

Published as a conference paper at ICLR 2026
in Case 2 and 3, digits were not adjusted as expected in 84% (=68%+16%) and 83% of instances, suggesting under-generalizationn
Overall, this evidence suggests that while the model can induce simple functions like +2 to some extent, it struggles with more complex situations where +2 should be only be triggered under certain conditions. Alternatively, if the induction of these conditions is viewed an additional step in multi-step reasoning. the model investigate may not yet be capable of two-step induction in a three-step task, thereby limiting their performance in base-8 addition.
deff baseSadditian b):
Full Model
Ablate Fl Heads Neither
perform base-10 addition basel maseelaaddiiion(,, b) # case
Case
Neither
c[0]
<[1]
Both
(2) apply adjustments
Case 93
6
1
0
100
1f <= a[a] + b[0] 18: # case 2
Case 2
68
0
16
16
100
c[a] [c[e] 2) x 10
Case
83
14
0
0
100
c[1] c[1]
elif a[o] + b[e] >= 18: a case
c[0] c[o]
Table 3: Error analysis for two-digit base-8 addi-
10
return c
tion. We use 100 examples for each case specified in Listing 1. The correct behavior is marked in green "Neither" suggests the number of times that neither c[0] or c[1] is adjusted, which is anticipated in
Listing 1: One possible algorithm for two-digit base-8 addition. This algorithm divides all scenarios into three cases. c[0] represents the unit digit and c[1] represents the tens/eights digit in two-digit number
Case 1. "c[o]" suggests that only c[0] is adjusted. "Both" suggests both digits are adjusted.
RELATED WORKS
Mechanistic Interpretability. The field of mechanistic interpretability aims to reverse-engineer complex neural networks into uannan--dddrttadablll algorithms (Bereska and Gavves, 2024; Sharkey al., 2025), enhancing our understanding of a wide range of model behaviors, including long-context retrieval (Wu et al., 2025), and chain-of-thought reasoning (Cabannes et al., 2024). A common methodology involves analyzing their computation graphs of specific task, as exemplified by studies on indirect object identification (Wang et al., 2023), "greater than" operation (Hanna et al., 2023), and entity tracking (Prakash et al., 2024a). Following this, our work begins with the off-by-one addition task, and showcases the broader applicability of our findings with various task pairs.
Induction Heads in LMs. Induction heads, described in Elhage et al. (2021) and Olsson al. (2022), is a fundamental mechanism in language models that facilitate its pattern-matching behavior in sequences like [A][B]... .. [A] - [B]. This could be seen as inducing a zeroth-order, constant function f=output( (B]), whereas our work identifies circuit for inducing first-order, linear function f(z) = x + 1, effectively generalizing the finding from token-level to function-level. Sharing our motivation of going beyond token-copying behavior, Minegishi et al. (2025) explored training two- layer transformers on carefully designed non-copying based ICL tasks and investigated the circuit emergence. Ren et al. (2024) introduced the concept of semantic induction heads, which handles higher-level information processing such as syntactic dependencies and entity relationships in context.
Function Vectors in LMs. Recent work has characterized in-context learning in language models as the compression of in-context examples into a single task or function vector, which is subsequently transported to the test example to trigger the model apply the function (Todd et al., 2024: Hendel et al., 2023; Yin and Steinhardt, 2025). These studies present strong evidence pertaining to single-step, mapping-style tasks like country-to-capital and English-Frence translation. Our work is inspired by this line of research, yet with two key differences: (1) We focus on off- off-by-one bo--ooyoeeeeeee addition, a multi-step arithmetic task, where the learning of the second step depends on the results of the preceding step. (2) We provide a finer-grained interpretation on how function vectors, sent out by different attention heads, vary content but collaborate to form a complete function. In concurrent work, this latter aspect was also explored by Hu et al. (2025), who investigate the task of add-k (i.e., "5-+8, 1-+4, 2-?")) using subspace decomposition.
Latent Multi-step Reasoning and Structural Compositionality in LMs. Various studies investigate whether and how models perform latent multi-step reasoning, typically via multi-hop factoid QA tasks (Yang et al., 2024b; Wang et al., 2024). Our work demonstrates that LMs can dynamically infer the second step in a two-step problem from in-context examples, a process representing a novel, flexible and composable form of latent multi-step reasoning. More broadly, our findings relevant

---

Published as a conference paper at ICLR 2026
to research investigating structural compositionality (Lepori al., 2023) (i.e., breaking down complex tasks into subroutines) in language models.
7 CONCLUSION
In this work, we present interpretability study the off-by-one addition task, with the broader goal of investigating how language models handle unseen tasks using in-context learning. Our analysis uncovers a case of function induction mechanism that captures the key "twist" required generalize from seen to unseen tasks. This discovery extends and generalizes previous interpretability findings on induction heads and function vectors. We further show this mechanism is broadly reused beyond off-by-one addition, notably in realistic algorithmic tasks like Caesar Cipher and base-8 addition. Collectively, these observations deepen our understanding of what language models are capable of with in- in-context learning and multi-step reasoning, and how models generalize to novel tasks and situations. Moreover, our work provides compelling evidence that language models may develop composable and general mechanisms for handling ever-changing task variations, suggesting one possible pathway toward explaining and perhaps further enhancing model capabilities.
Implications for LLM Development and Applications. While our work focuses very specific tasks and mechanisms, we believe the findings could further guide LLM development and application.
Evaluation: In $5, we found that models achieve non-trivial accuracy on base-8 addition by relying on an unintended shortcut algorithm. This result strongly suggests that accuracy- -based evaluation may disguise flawed reasoning processes inside the model. Complementing accuracy- based evaluation with interpretability analysis can help reveal the model's 's true capabilities and whether it has learned the intended reasoning process.
Pre-training: Our analysis reveals that models perform multi-step reasoning latently and reuse shared mechanism across tasks, remarkable emergent structure given that these models are typically trained end-to-end on next-token prediction. This observation could inform the design of pre-training data mixtures or curricula that enhance multi-step compositional reasoning. For example, it may be beneficial to first train the model on single-step tasks (e.g.. standard addition) before exposing to multi-step tasks (e.g., off-by-one addition), thereby encouraging the development of function induction mechanisms.
Model Behavior and Alignment: Recent work has identified concerning behaviors in language models, such as sycophancy (Sharma et al., 2025), agreement bias (Andrade et al., 2025), and susceptibility to belief shifts (Geng et al., 2025), which impact their reliability in real-world applications. We hypothesize that these behaviors may share structural similarities with the function induction mechanism we identified. Specifically, models may induce "belief-modifying functions" from context that drive their output generation. Investigating this connection could inspire future methods that better ensure reliability of language models.
Future Work. Beyond the discussion above, we believe the function induction mechanism itself is an intriguing interpretability finding that opens up many new research questions. In particular, investigating the emergence and formation of this mechanism during pre-training and attributing it to specific training instances could be an interesting future direction. For instance, one could test whether exposure to structurally related tasks (e.g., puzzles riddles) contributes to the emergence of the circuit. Additionally, Yin and Steinhardt (2025) discovered that function vector heads may have evolved from standard induction heads during pre-training. would be interesting to investigate if such an evolutionary process applies to function induction heads in this work.
LIMITATIONS
Regarding circuit discovery experiments (53, $C and (D), the identified circuit is limited as it does not perfectly satisfy the faithfulness and completeness criteria, even with our best efforts. This challenge arises because achieving simultaneous satisfaction of faithfulness, completeness, and minimality is difficult, as these criteria often regulate each other. Moreover, number tokens often mapped into sinusoidal (Fourier) feature space rather than a linear space in language models (Nanda et al., 2023; Zhong et al., 2023; Zhou al., 2024), which further complicates our interpretability analysis.
10

---

Published as a conference paper at ICLR 2026
Regarding circuit analysis ($4 and ŞE), mainly used causal intervention methods and examine the causal effect of attention heads on naive prompts. Future work could provide deeper mechanistic insights by analyzing the query-key and output-value circuits within these heads (Elhage et al., 2021), or by investigating the role of ML.P layers in the overall mechanism.
Regarding task generalization experiments ($5 and $F), our current scope limited two-step tasks where the second step involves shifting-related function. We view our results as identifying a case of function induction in language models, and we do not intend claim universal mechanism that applies to any function. That said, we anticipate that function induction, as a conceptual framework, may extend a broader class of functions. Such behavior could be implemented by shared circuit or by multiple specialized circuits; we leave systematic investigation of this possibility to future work. Finally, the task pairs we investigated are synthetic or algorithmic; further exploration of the role of function induction heads on naturally occurring texts would be highly valuable.
REPRODUCIBILITY STATEMENT
Our code and data are released at 2 ww--UuCCffucciionn inniuuiiiioonn and uploaded as supplementary material. Specifically, we include: (1) the datasets and code used to generate datasets; (2) the code for the main experiments reported in the paper; and (3) the code for creating the result figures. To further support reproducibility, we also provide environment configuration instructions and interactive notebooks that serve as guided walkthrough. See $H for additional reproducibility details.
ACKNOWLEDGMENT
We thank the anonymous reviewers for their helpful comments and feedback, which greatly strength- ened our work. We thank Xisen Jin, Ting- Yun Chang, Lorena Yan, Yuging Yang. Deqing Fu, Ollic Liu, Daniel Firebanks- Quenked,, Johnny Wei. well as members of USC INK Lab and Allegro Lab for insightful discussions. QY and XR were supported in part by the Office the Director National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022222072200000000 2207220000,, the Defense Advanced Research Projects Agency with award HR001 2220046, and NSF IIS 2048211. QY also supported by a USC Annenberg Fellowship. RJ was supported in part by the National Science Foundation under Grant No. IIS-2403436. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
REFERENCES
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang. Phi-4 technical report, 2024. URL httph:t wwwpriivv.ooggaaeee hatrg//bs//ae 241..08905
Moises Andrade, Joonhyuk Cha, Brandon Ho, Vriksha Srihari, Karmesh Yadav, and Zsolt Kira. Let's think in two steps: Mitigating agreement bias in milms with self-grounded verification, 2025. URL hittps://arxiv abgg///bs//50)) 11662.
Federico Barbero, Alvaro Arroyo, Xiangming Gu, Christos Perivolaropoulos, Petar Veličković, Razvan Pascanu, and Michael M. Bronstein. Why do LLMs attend to the first token? In Second Conference on Language Modeling, 2025. URL https::/ www.eeenne/frrrmmiitt tu4dFUsW5z.
Leonard Bereska and Stratis Gavves. Mechanistic interpretability for Al safety review. Transac- fions on Machine Learning Research, 2024. ISSN 2835-8856. URL Forum?id==PUVetPPuu Survey Certification, Expert Certification.
net/
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler,
11

---

Published as a conference paper at ICLR 2026
Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Ben- jamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc., 2020. URL Pataa paper_ ott.r//ese.pppperereoeee//ee/2/0 http:///e.....tooooooaaaaaaaaaaa
wwwp:eceeeeddnns.nnesssrs..... neurips.cc/
Vivien Cabannes, Charles Arnal, Wassim Bouaziz, Alice Yang, Francois Charton, and Ju- lia Kempe. Iteration head: A mechanistic study of chain-of-thought. In A. Globerson, I.. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 109101-109122. Curran Asso- ciates, Inc., 2024. URL 150f8180ef3440 0eeeee/eee00000eeeeeeeeeeeee2220
neur cc/ sec/pape__ss paaperffllee fwt.s//paper/222//11 Coneer/reeaeeeeeeeeeeeeeeeeeee pdf.
Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. Meta-learning via language model in-context tuning. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 719-730, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 18.18653/v wttp:///1//2222llllll -ong.............. URL https: http//aaaanttooo...oooooo org/2022. aaw-aaclllooooo 1ong--oon.... 53/.
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. What does BERT look at? an analysis of BERT's attention. In Tal Linzen, Grzegorz Chrupata, Yonatan Belinkov, and Dieuwke Hupkes, editors, Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276-286, Florence, Italy, August 2019. Association for Computational Linguistics. doi: 10.18653/1//1191000 W19-4828/.
URL https: https///aaaaaaolggaaogee org/
Nelson Elhage, Neel Nanda. Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. mathematical framework for transformer circuits. Transformer Circuits Thread, 1(1):12, 2021.
Javier Ferrando and Elena Voita. Information flow routes: Automatically interpreting language models at scale. Arxiv, 2024. URL https://arxivv org/abs//aal 00824.
C. Daniel Freeman, Laura Culp, Aaron Parisi, Maxwell L Bileschi, Gamaleldin F Elsayed, Alex Rizkowsky, Isabelle Simpson, Alex Alemi, Azade Nova, Ben Adlam, Bernd Bohnet, Gauray Mishra, Hanie Sedghi, Igor Mordatch, Izzeddin Gur, Jaehoon Lee, JD Co-Reyes, Jeffrey Penning- ton, Kelvin Xu, Kevin Swersky, Kshiteej Mahajan, Lechao Xiao, Rosanne Liu, Simon Kornblith, Noah Constant, Peter J. Liu, Roman Novak, Yundi Qian, Noah Fiedel, and Jascha Sohl-Dickstein. Frontier language models are not robust to adversarial arithmetic. or "what do need to say you agree 2+2-57, 2023. URL https: //ttxiaaavvvv///////a/ org/ abs/2311 07587.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at practical size. arXiv preprint arXiv:2408.00118. 2024.
Jiayi Geng, Howard Chen, Ryan Liu, Manoel Horta Ribeiro, Robb Willer, Graham Neubig, and Thomas L. Griffiths. Accumulating context changes the beliefs of language models, 2025. URL https:///axxiw.or//bbba 110251111111....1
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint 1xXXi:2407.21788. 2024.
Xiangming Gu, Tianyu Pang. Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. When attention sink emerges in language models: An empirical view. In The Thirteenth International Conference on Learning Representations, 2025. URL. htt.ttt:////peeeeeeeee net/ 7oNN
Danny Halawi, Jean-Stanisla Denain, and Jacob Steinhardt. Overthinking the truth: Understanding how language models process false demonstrations, 2023. URL https:/ http::/:///peeeiie.w..... net/ forum?i d= Gvxa.
12

---

Published as a conference paper at ICLR 2026
Michael Hanna, Ollie Liu, and Alexandre Variengien. How does GPT-2 compute greater-thant Interpreting mathematical abilities in pre-trained language model. shirty Conference Neural Information Processing Systems, 2023. URL p4PckNOR8K.
Roce Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors. In Houda Bouamor, Juan Pino, and Kalika Bali, editors. Findings of the Association for Com- putational Linguistics: EMNLP 2023, pages 9318-9333, Singapore, December 2023. As- sociation for Computational Linguistics. doi: 2018653/v1/202: 1/2.://///.....nnnng-s..a. settinns-emmll.624. URL https://aclantholoogy http://cc.nntoooooy..rrrr/////// org/2023, modings--mmll.665 624/.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference Learning Representations, 2021. URL
3GmQ.
Xinyan Hu, Kayo Yin, Michael Jordan, Jacob Steinhardt, and Lijie Chen. Understanding in-context learning of addition via activation subspaces, 2025. URL. https: :tttt:://aat /ttp://rii..rr//bb/2/........ 05555
Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tim Rocktäschel, Edward Grefenstette, and David Krueger. Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks. In The Twelfth International Conference on Learning Representations, 2024. URL https::////. https::/ /ttp////eeeeeeeeeeeeeeeeeee foet//oormmiddAAHKKK
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Ana Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL abs/2310. 06825..
Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Harold Miles, Seraphina Nix, Tao Lin, Neev Parikh, David Rein, Lucas Jun Koba Sato, Hjalmar Wijk, Daniel M. Ziegler, Elizabeth Barnes, and Lawrence Chan. Measuring ability complete long tasks, 2025. URL httpp:://axkee orgooogg//ooo. abs/ 14499.
Michael Lepori, Thomas Serre, and Ellie Pavlick. Break it down: Evidence for structural positionality in neural networks. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 42623-42660. Curran Associates, Inc., 2023. URL httpss:/pproceeddngs.no wwwpeeeeedeeineeeeeieeeeeeeeeeee ctterrissccc/////ee/e cc/paper_ hit1s//pppr/2222//11///555811111 orr.://05000000000e06600ee6607-0 .df.
Belinda Z Li, Zifan Carl Guo, and Jacob Andreas. (how) do language models track state? arXiv preprint arXiv:2503.02854, 2025.
Ziqian Lin and Kangwook Lee. Dual operating modes of in-context learning. In Forty-first Inter- wttpe/ewwwwwwweefeeeeeeeeeee
national Conference Machine Learning, 2024. URL https://openrevieww net. net./orummiid E1VHUWyL3n.
Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, and Hannaneh Hajishirzi. Z-ICL: Zero- shot in-context learning with pseuudodemmonstratinns.. In Anna Rogers, Jordan Goyd-Graber, and Naoaki Okazaki, editors. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2314-2317, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/ 122. URL https://aclanthon
www.oorgg......eooooo acl- acl-long. 129/.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 17359-17372. Curran Associates, Inc., 2022. URL https: /ttoce/ee/o/ccccoeeeeoooooeco/se neurips. wwaper_files/pppeee 2022/file/ 2222:/11661116222220000633332222
pdf.
13

---

Published as a conference paper at ICLR 2026
Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Circuit component reuse across tasks in transformer language models. In The Tweifth International Conference on Learning Representations, 2024.
URL https::/opeereviieeee net
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. MetalCL: Learning to learn in context. In Marine Carpuat, Marie-Catherinn de Marneffe, and Ivan Vladimir Meza Ruiz, editors, Proceedings of the 2022 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies, pages 2791-2809, Seattle, United States, July 2022a. Association for Computational Linguistics. doi: 10. 10.18653/v1/2022..mm naacl-maii 201. URL htt https: / hatpaaaaaaaancoooo org/ 2022. saaacl-maiin 201/.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? In Yoav Goldberg. Zormitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing, pages 11048-1 1064, Abu Dhabi, United Arab Emirates, December 2022b. Association for Computational Linguistics. doi: 10. 10.58553/v1 2022.emnlp-main.1 7599 URL https: ptttps://aclantthoogg. .rt.orgg///20 2022. mmnlp-main..
Gouki Minegishi, Hiroki Furuta, Shohei Taniguchi, Yusuke Iwasawa, and Yutaka Matsuo. Beyond induction heads: In-context meta learning induces multi-phase circuit emergence. In Forty-second International Conference on Machine Learning, 2025. URL http://///opereeeeeeeeeeoooo/o id=Xw01 vF1 3aV.
Neel Nanda and Joseph Bloom. Transformerlens, TransformerLens, 2022.
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference Learning Representations, 2023. URL https:// htttps://pperrvieev www.net/ooruumiiddgffssooooo
nostalgebraist. interpreting gpt: the logit lens. https://www. wttps//wwww....ooggg...o.oo com/ posts/ ackk@swwpdaa@vrrr//nnterrrttdddd taaiee---ttt---ee----ee-eeee 2020. Accessed: 2025-06-17 17.
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. doi: 10.23915/distill.000224
Zoom in: An introduction to circuits. Distill, 2020.
URL
https://distill. wttpssttll..pbb2e/ pwpppppbbbbpppppe http://02//crriit//oooooo rcuittszoom-iin.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads. arXiv preprint arXiv:2209.1 1895, 2022.
Akshara Prabhakar, Thomas L. Griffiths, and R. Thomas McCoy. Deciphering the factors influenc- ing the efficacy of chain-of-thought: Probability, memorization, and noisy reasoning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Com- putational Lingwistics: EMNLP 2024. pages 3710-3722 Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. (0.18853/v11 1/2024. 1/2024.indings org/2024 findings-ennen 212/
URL
Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. Fine-tuning enhances existing mechanisms: A case study on entity tracking. In Proceedings of the 2024 International Conference Learning Representations, 2024a. arXiv:2402. 14811.
Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. Fine-tuning enhances existing mechanisms: A case study on entity tracking. In The Twelfth International Conference on Learning Representations, 2024b. URL https::/openrev///ee wtw.://wwweeeoouumiiiiii. BsKCAWOF2D.
Jie Ren, Qipeng Guo, Hang Yan, Dongrui Liu, Quanshi Zhang, Xipeng Qiu, and Dahua Lin. Identify- ing semantic induction heads to understand in-context learning. In Lun Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 6916 0-6992, Bangkok, Thailand, August 2024. Association for Computational Lin- guistics. doi: 10. 10.18653/v11 10.5///1//02...// http///222..iinssaaeeea acl.112. findings-acl. 4112
URL https: haap:ss/acllnhhhlogyyy or2/2024.
14

---

Published as a conference paper at ICLR 2026
Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq. Nicholas Goldowsky-Din Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, Stella Biderman, Adria Garriga-Alonso, Arthur Conmy, Neel Nanda, Jessica Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, Eric J. Michaud, Stephen Casper, Max Tegmark, William Saunders, David Bau, Eric Todd, Atticus Geiger, Mor Geva, Jesse Hoogland, Daniel Murfet, and Tom McGrath, Open problems in mechanistic interpretability, 2025. URL https://aaxi: wwtprriwwww.oraeeeeeeeeeeeooeee org/abs/ 2501. 16496.
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. Towards understanding sycophancy in language models, 2025, URL https://arxet 2331/ 1 13548.
Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing, pages 7035-7052 Singapore, December 2023. Associ- ation for Computational Linguistics. doi: 10.18663/w1/2233.cmml //aclanthology. org/2023. emnlp-main 435/.
URL https:
Alex Tamkin, Miles McCain, Kunal Handa, Esin Durmus, Liane Lovitt, Ankur Rathi, Saffron Huang, Alfred Mountfield, Jerry Hong, Stuart Ritchie, Michael Stern, Brian Clarke, Landon Goldberg, Theodore R. Sumers, Jared Mueller, William McEachen, Wes Mitchell, Shan Carter, Jack Clark, Jared Kaplan, and Deep Ganguli. Clio: Privacy-prees preserving insights into real-world ai use, 2024. URL https: /awpvv/rxivvvvv/aa////// org abs/ 2412. 13678.
Eric Todd, Millicent Li, Amab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. Function vectors in large language models. In The Twelfth International Conference on Learning Representations, 2024. URL httpps://ppnret www.eewweeeeeeeeeeeeeeeeeeeeeeee wwt.//orutmteeeeoemm AwyytyyyyMaaaaa
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprini arXiv:2307.09288 2023.
Igor Tufanov, Karen Hambardzumyan, Javier Ferrando, and Elena Voita. Lm transparency tool: Interactive tool for analyzing transformer language models. Arxiv, 2024. URL https://arxiv. org/ 24004 07004.
Jesse Vig and Yonatan Belinkov. Analyzing the structure of attention in a transformer language model. In Tal Linzen, Grzegorz Chrupata, Yonatan Belinkov, and Dieuwke Hupkes, editors, Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. pages 63-76, Florence, Italy, August 2019. Association for Computational Linguisties. doi: 10. 10.18653/11/W11-48888 URI. http:///aaaaantooo.ororro/////// -198////
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analy- sis. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 12388-12400 Curran Asso- neur fttllppppplllle//ll/pllppppppelp wwt.le///pper//22//11eeee pdf.
ciates, Inc., 2020. URL 2711
Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. Grokking of implicit reasoning in transformers: A mechanistic journey to the edge of generalization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httph::t////ett //tptrv///peeevvew... net net/forumiid= D4QgSWxiOb.
Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, 2023. URL https:
//
15

---

Published as a conference paper at ICLR 2026
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma. Larger language models do in-context learning differently, 2024. URL httpp::/ /ttps//s////eeeeeeeee//////////t
Wikipedia contributors. Caesar cipher Wikipedia, the free encyclopedia, 2025, URL https: /enn wikipedia, .rg arg/w/indexx wttppppppi..eeeeeee..e.eeeeeeaea 12-June-20251
[Online; accessed
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac. Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: Sfate-of-the-ar natural language processing. In Qun Liu and David Schlangen, editors, Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pages 38 45, Online, October 2020. Association for Computational Linguisties. doi: 10. 11665///12220emmmmmmmeeeeee URL httph:tts///tt. /tatts::/accaane http:///olaanoooogyyyyoooo htwpthhooogooooog/g/oee org/2020. emnlp-demos. 6/-
Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. Retrieval head mechanisti- cally explains long-context factuality. In The Thirteenth International Conference Learning Representations, 2025. URL https:// evwewwwweeww..... met//Forummiid-EyBBB0BB1
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang. Najoung Kim. Jacob Andreas, and Yoon Kim. Reasoning reciting? exploring the capabilities and limitations of language models through counterfactual tasks. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pupers), pages 1819-1862, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/v 2tt/533/1//222.maaael naacl-long. 102. URL htwps:ttt//oo /atp//aaaaaaannoooaaaaaaoa org. 2024/2224..2 naacl-long 102/.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representutions, 2024. URL https://openreviiiii net deet/Forumiid=NG7SS33
An Yang, Baosong Yang, Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, al. Qwen2.5 technical report. arXiv preprint 15 2024a.
Sohce Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning? In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10210-10229, Bangkok, Thailand, August 2024b. Association for Computational Linguistics. doi: 10. 11116553//1/2222al-lloooooo //aclanthology. org/2024 acl- acl-long. 550/.
URL https:
Qinyuan Yc, Mohamed Ahmed, Reid Pryzant, and Fereshte Khani. Prompt engineering prompt engineer. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 355-385, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 122853/1/22024.11. fin.24..inninn-aac.21 URL https: findings-acl.22 21/.
//acl www.aaanthooooorrrrrooooo
Kayo Yin and Jacob Steinhardt. Which attention heads matter for in-context learning?, 2025. URL 14010.
https:/ www.w.wwwwww...////oorr
Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Wop Lee, Sang-goo Lee, and Taeuk Kim. Ground-truth labels matter: A deeper look into input-label demonstrations. In Yoav Goldberg, Zoritssa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2422-2437, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational l Linguistics. doi: 10. 18653/v1/2022 http/2222eemmp2nng monnp-mainnI1e main. URL / h/tp///aaaaanntog....a/co org/2022 wwtp:222...mmmmmmmmemmmmmmmmmp mmnlp-main... 155/.
Lifan Yuan, Weize Chen, Yuchen Zhang, Ganqu Cui, Hanbin Wang, Ziming You, Ning Ding, Zhiyuan Liu, Maosong Sun, and Hao Peng. From f(x) and g(x) to fg(())): Llms learn new skills composing old ones, 2025. htt.:: wwwpiii.org//b//////ea 25123.
by
16

---

Published as a conference paper at ICLR 2026
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: Im chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations, 2024. URL
Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas. The clock and the pizza: Two stories in mechanistic explanation of neural networks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL htwph: https://openrreiew. wttpeerreeiieeeeeeoouuttt..eeee mett/forumiid-SSmmbbe Te..
Tianyi Zhou, Deqing Fu, Vatsal Sharan, and Robin Jia. Pre-trained large language models use fourier features to compute addition. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024, URL https: /tpprr/veeeeeeeeew.ee// net forum?i 4MutM2TZb.
17

---

Published as a conference paper at ICLR 2026
A INDUCTION HEAD MECHANISM AND FUNCTION INDUCTION MECHANISM
legand
Next token pred.
Next token
key/vollue
OLtput
50%6
20%
Head
query
Induction Heads
Group
Previous
Token Heads
Group 3
Early layors
evaluation
Figure 8: Comparing Induction Head (Left) and Function Induction (Right). See Fig. 9 for an annotated version of this figure.
Comparing Induction Head and Function Induction. Fig. 8 provides side-by-side visualization of the induction head mechanism (Olsson et al., 2022) and the hypothesized function induction mech- anism ($3.2), demonstrating their structural similarity and explaining the basis for our hypothesis.
To provide more concrete example how induction heads work, consider the hypothetical scenario where a language model is completing the prompt: "Llama 0 was released in 2022. This paper presents extensive evaluation of Llama ..." When the model first encounters uncommon phrase, e.g., "Llama 0", a previous token head will attend to "Llama" and register the information that "Llama appears before 0" the position of "0". Later on, when "Llama" appears in the context again, an induction head will retrieve this piece of information from position of "0" and increase the likelihood of generating "0" the next token. This induction head mechanism informs our hypothesis on function induction in $3.2 2 and the collaborative interaction between previous token heads and function induction heads in Fig. (Right).
We also provide an additional figure (Fig. 9) that is annotated with the hypothesized roles of query, key, value and output representations.
Relevance to In-context Learning with False Demonstrations. Various prior works investigate how language models handle false, random, or perturbed demonstrations in in-context learning (Min et al., 2022b; Yoo et al., 2022; Freeman al., 2023; Wei et al., 2024; Lyu al., 2023; Lin and Lee, 2024). Notably, Halawi et al. (2023) adopted an interpretability approach, observing the overthinking behavior of models (i.e., models draft truthful answers at early layers and flip them to untruthful answers at late layers), and identified false induction heads that are responsible for copying the untruthful answers from the ICL examples.
Our analysis of off-by-one addition was largely motivated by these studies. Here we revisit the findings of Halawi et al. (2023) along with ours, using unified view of two-step tasks, i.e., (g(()). In Halawi et al. (2023), the first step, = g(x) is typically text classification task, e,g., news topic classification, and the second step, f(y is permutation of the labels, e,g.,
CerTecch-Worlle World->Sort, Sports Sportss+Businne Busiiesss. In our work. y = g(x) is standard addition, and f(y) is function.
In this view. our findings with off- off- addition are consistent with those in Halawi et al. (2023), while also advancing the understanding in several aspects: (1) In both cases, language models decompose the task into two steps, and induce the second step based on the results of the first step. The second step could be either a conditional copy-paste function, e,g., a permutation of labels, or an algorithmic function, e.g., +1 function. The latter represents novel finding of this study, demonstrating that the second step can exhibit forms more complex than copy-paste operations. (2) Our path patching procedure has led us to identify two additional group of heads (consolidation heads and previous token heads) that are involved handling false demonstrations. (3) Our work also suggests that the strategy improve truthfulness by zeroing false induction heads or function
18

---

We find that the base accuracy consistently decrease with Published as a conference paper at ICLR 2026
legend
Next token
token
ken//salaa
output
50%
80%
Head
R0N
Ab%
query
Group
aspeas
Induction Heads
Apply fodrow)
Something in offar
Group 2
Previous
Taken Heads
Gurren bósen
Group 3
Curient tokio to Next J0kin is
Current "Uaama"
Current "Lana"
Early layers
released
evaluation
Figure 9: Comparing Induction Head (Left) and Function Induction (Right). this figure, we've annotated the hypothesized roles of query, key, value, output representations of involved heads.
induction heads may have unintended consequences on models' positive capabilities, given their positive contributions the cipher task and the base-8 addition task.
Related to the view of two-step tasks, Jain et al. (2024) demonstrate that models learn a "wrapper" function g over existing function f in sequential fine-tuning setting. Very recently, Yuan et al. (2025) show that models can learn to chain atomic functions into compositional functions during reinforcement learning. Our work and Halawi al. (2023) suggest that language models demonstrate simple forms of this behavior with in-context learning as well.
work and sharing the same motivation of investigating complex model behaviors in in-context learning, Minegishi et al. (2025) presents an in-depth study on training language models to perform in-context learning tasks and interpreting the mechanism. We discuss how our findings relate to theirs below.
Relevance to Minegishi al. (2025). Highly relevant
In terms of the study design, both work study how transformer models perform in-context learning, using group of task variants. Minegishi et al. (2025) designs group of non-copying based classificatoon-style tasks, while we focus on algorithmic tasks. Additionally, Minegishi et (2025) trains small transformer models from scratch, enabling discovery of three-phase circuit formation process. We instead interpret larger, off-the-shelf language models, which are more closely aligned with real-world applications and demonstrate strong capabilities across diverse tasks.
Regarding the findings, Minegishi et al. (2025) identifies a two-head circuit whose attention patterns and connections align with the previous token head and function induction head identified in our work. Both works find that models execute individual tasks through multiple parallel pathways and observe that models can adopt shorteut solutions for certain tasks by leveraging existing mechanisms. Our unique contributions are two-fold. First, we demonstrate that the mechanism we identify operates in two-step tasks, showing that models perform latent multi-step reasoning. Second, find that this mechanism is reused across many other tasks, suggesting broader compositional principles in model behavior.
B OFF-BY-ONE ADDITION EVALUATION
Models. In $2 we evaluated six recent language models on the task of off-by-one addition. In Table 4 provide details of these models.
Reporting base and contrast accuracy. Previously in Fig. 1, we reported the accuracy of off-by- one addition (i.e., the percentage of time that the model outputs 7 when given 3+3). In Fig. 10(a) we additionally report the accuracy of standard addition (e.g.. "3+3=6"), when the models are given the contrast prompt (e.g., "I+1=3in2+2-5 more in-context learning examples. Fig. 10(c), we show that models may also output numbers that incorrect either in standard addition off-by-one addition (i.e., neither "6" or "7").
19

---

Published as a conference paper at ICLR 2026
Tokenization 0-999
Model Name
Huggingface Identifier
Reference
Llama-2 (7B)
meta--Llamma/Lmma----hhhhtt
Touvron al. (2023)
Mistral-v0.. (7B)
mistralai/Miittrll-8--0
Jiang et (2023)
Gemma-2 (98)
google/gemma-2-96
Gemma Team (2024)
Qwen-2.5 2.5 (7B)
Qwen/Qwen2.
Yang et al. (2024a)
Llama-3 3 (8B)
me arta-llama/Meta-Llamss
Grattafiori et al. (2024)
Phi-4 (14B)
mi microsoft/phi-4
Abdin et al. (2024)
Table 4: Models Evaluated on Off-by-One Addition. "0-9" means the model uses digit-level tokenization for numbers, e.g., "123" is tokenized into ["'"""2" "2"" "3"], "0-999" means all numbers smaller than 1000 considered one single token, e.g., "123" is tokenized into ["123"].
(b) Contrast Acc.
100
1.00
100
20
842
200
Lima 2 (7B)
Accursay 196)
Portuon (84)
60
60
60
Qwen 2.5 (7B)
Mastralvt (78)
40
Libma (BB)
PA14 (145)
2.4
16
32
2.4
8
16
32
2.4
16
32
Number 6fTCL Exomples
Nunber ICI Examples
Number ofDCL Exanples
Figure 10: Off-by-One Addition Evaluation, Reporting Base Accuracy.
Results in smaller number range. Previously in Fig. 1, we reported results when the operands were sampled from the range of [0,999]. In Fig. 1, we additionally report results when sampling from the range of [0,9] and [0,99]. For two models using 0-9 tokenization (Gemma-2 (9B) and Qwen-2.5 (7B)). the performance drops with larger number ranges. For the remaining models, the performance remains stable regardless of the number ranges.
Results with/without the constraint of Cteat * C₁. Previously in $2 we deliberately impose the constraint that Vi, Creat * q. This is to rule out the possibility that language models perform off- one addition via copying Ctest from previous contexts. In Fig. 12, we compare the results of two additional sampling strategies: (1) constraint Cteat and c;: (2) 3i, Gtest C-- By comparing Fig. 12(b) and (c) we see that for Mistral-v0.1 (7B) and Gemma-2 2 (9B), the accuracy is higher when 3i, Ctest C₁. This observation implies that these two models leverages copy-paste induction more than function induction in performing off-by-one addition, though more rigorous analysis is required draw a conclusion.
Results with off-by-k addition. In Fig. 13-14, we present 32-shot off-by-k addition results with various offsets using Gemma-2 (9B) and Llama-3 (8B) respectively." One consistent trend that models struggle more with offsets & of larger absolute values. While Llama-3 (8B) generally outperforms Gemma-2 (9B), Gemma-2 (9B) demonstrates strong performance when potentially due to its adoption of 0-9 tokenization. An additional observation reveals that Gemma-2 (9B) typically achieves stronger performance with even values of compared to odd values.
+10,
"We chose Gemma-2 (9B) the default model in study because (1) focused on the range of [0.9] in early stage of this work to prioritize simplicity, and Gemma-2 (9B) performs competitively in this setting; (2) Qwen-2.5 2.5 (7B). Llama-3 (8B). Phi-4 (14B) were not released or integrated into transformer- -eneerr-eeeseeeen that time. We acknowledge this experimental design limitation and address it by interpreting Llama-3 3 (8B) and Mistral-v0. (7B) in SC.
'The visualization of Fig. 13-14 inspired hy Prahhakar al. (2024).
20

---

Published as a conference paper at ICLR 2026
(a) Range [0.9]
Raaae [0,99]
Range [0,998]
100
100
100
Aearc.y (50
60
Acsioway (h)
60
90
Owen 3.5 (7B)
Gemma-d (901
Mistru vo.1 (78)
40
UJamedl (til)
Phi-d (14B)
0
Figure 11: One- Addition Evaluation, Using Smaller Number Ranges.
(a)
100
100
100
Accunay (h)
Acturaty (he
Gemna (28)
Qween223 (7B)
Mistral vo.1 (78)
60
40
40
UJam-d3 (5B)
Phi-i (14B)
Number ICL. Trompiine
Number Esomplexx
Number Doompleee
Figure 12: Off-by-One Addition Evaluation, Different Sampling Constraints
C CIRCUIT DISCOVERY
C.I RELATIVE LOGIT DIFF
$3.1 introduced r, the relative logit difference, to measure the effect of replacement during circuit discovery. We now elaborate on this formula to enhance clarity.
F FMM Fernnt F F(M, Foons)
(1)
F F(M, Teont) - F(M, Tbase)
[M' (Ybase (cone) - M' (yoont (cont) }]
[M (Ubasel (eont) M (Noont [cont]]
(2)
[M (yhase (oome) (Yoont [eomt)]
M (yhase (baan) (Yeent Thase)
C.2 IDENTIFIED HEADS
In the main paper, we focus on interpreting Gemma-2 (9B). To explore the universality of the mecha- nism, we additionally conduct path patching with Llama-3 (8B), Llama-2 (7B) and Mistral-v0. (7B). We list the identified attention heads below.
C.2.1 Gemma-2 (9B)
Gemma-2 (9B) has 42 layers and 16 heads per layer. Path patching experiments were conducted with 4-shot off-by-one addition with numbers sampled from range [0,9].
Consolidation Heads: H41.4, H41.5, H40.11, H40.12;
Function Induction (FI) Heads: H39.7, 139.12, H36.7, H32.1, H32.6, H25.13:
Previous Token (PT) Heads: H38.6, 6, H38.7, H38.9, H35.14, H35.9, H31.4, H31.5, H29.5.
C.2.2 Llama-3 (8B)
Llama-3 (8B) has 32 layers and 32 heads per layer. Path patching experiments were conducted with 4-shot off- off-by-one addition with numbers sampled from range [0,999]. We visualize the path patching results in Fig. 15.
Consolidation Heads: H31.1, H30.25, H29.11, 11, H29.10, H28.16, H28.17 H28.18;
21

---

Published as a conference paper at ICLR 2026
Performance with different offset k, Gemma-2 (9B), 32-shot
Contrial Anc.
Base Acc.
Accurary
0.4
Range [0,9]
Range
0.2
Range (0,999)
0.0
Figure 13: Off-by-k Addition Evaluation, Gemma-2 (9B)
Performance with different offset K. Llama-3 (8B), 32-shot
Contrast Acc.
Accuracy
Base Acc.
0.4
Range [0.9]
Range [0,99]
0.2
Range (0.999)
0.0
-10
-9
10
Figure 14: Off-by-k Addition Evaluation, Llama-3 (8B)
Function Induction (FI) Heads: H26.2, H23.13, H23.15;
Previous Token (PT) Heads: H24.10, H24.11, H22.25, H22.27, H21.7.
C.2.3 Mistral-v0.1 (7B)
Mistral-v0.1 (7B) has layers and 32 heads per layer. Path patching experiments were conducted with 4-shot off--byonne addition with numbers sampled from range [0,9]. We visualize the results in Fig. 16. For the two consolidation heads in the list below, they show weaker effect and attend to both the current token and some other tokens, which slightly deviates from our findings with Gemma-2 (9B). Apart from this, the results using Mistral-vO. 1 are consistent with other models.
Consolidation Heads: (H31.10), (H31.1)
Function Induction (FI) Heads: H30.2, H30.3, H30.4, H30.8, 10, H30.18, 18, H31.2
Previous Token (PT) Heads: H29.4, H29.6, H29.7.
C.2.4 Llama-2 (7B)
Llama-2 (7B) has 32 layers and 32 heads per layer. Path patching experiments were conducted with 4-shot off-by-one addition with numbers sampled from range [0,9]. We visualize the results in Fig.
Two Variations. All three groups of heads present in Llama-2 (7B). However, notice two small variations compared to the circuit in Gemma-2 (9B). (1) We identified H16.24 that achieves r 2.12%, but its attention pattern doesn't fit that consolidation head or function induction head. Since 2.12% is slightly above the 2% threshold we set, we consider this noise; (2) One previous token head (H29.1) 1) no longer attends to the "=" immediately before the answer token e the token G. Instead, attends to the "=" token one ICL example away.
Comparison with Todd et al. (2024). To discuss our findings with those of Todd et al. (2024) with common ground, extract the list of FV heads by selecting the 10 heads with the highest absolute
22

---

Published as a conference paper at ICLR 2026
Patebiig Output Logits
(b) Patching to H26.2 Valbe
20%
6
10%
Difi
Hoad Index
16-
0%%
220
10%
28
-20%
24
20 28
Layer
Lager
(c) #28.16 Attn. Patter
(d) H26.2 Athn. Pattern
(e) H24. 10 Attm. Pattem
Quary
Key
Key
Key
Figure 15: Circuit Discovery with Llama-3 (8B). Causally-relevant positions are marked in pink. Results are consistent with those with Gemma- 2 (9B) in Fig. 2.
(a) Patching to Output Logits
(b) Patching to H30.4 Value
20%
10%
DIT.
Head Indox
12
(1%
20
24
10%
26
24
24
Layer
Layer
(c) 1131. Attn. Pattern
(d) 1130.4 Attn. Pattern
(e) B29..7 Attn, Pattern
Quary
Key
Key
Key
Figure 16: Circuit Discovery with Mistral-v0.1 (7B). Causally-relevant positions are marked in pink. Results are mostly consistent with those with Gemma-2 (9B) in Fig. 2, with the exception of the consolidation heads showing weaker signals.
average indirect effect (AIE) from Fig. 19 in Todd et al. (2024). These heads are concentrated in early-middle layers (before layer 20), whereas our heads appear in late layers (layers 29-31). There is no overlap between the two sets.
Consolidation Heads: H31.28, H31.10, H30.3;
Function Induction (FI) Heads: H31.30, H31.4 14, H29.26, H29. 16, H30.26, H30.3;
Previous Token (PT) Heads: H30.13, 129. H28.5, H28. 10, H28.16, H28.24, H27.31;
Miscellaneous Heads: H16.24;
Function Vector Heads (Todd et al., 2024): H9.25, H11.2, H11.18, H12.15, H12.18, H12.28, H13.7, H14.1, H14. 16, H16.10.
23

---

Published as a conference paper at ICLR 2026
Patebiig Output Logits
(b) Patchinng to H29.16 Value
20%
6
10%
Difi
Head Index
12.
16
H01
0%
20
10%
28
-20%
24
20 28
Layer
Lager
(c) #31.1D Attn. Patser
(d) H29.16 Attm. Patter
(e) H28.16 16 Attm. Pattem
Query
Key
Key
Key
Figure 17: Circuit Discovery with Llama-2 (7B). Results are mostly consistent with those with Gemma-2 (9B) in Fig. 2. Causally-relevant positions marked in pink. In 129.16 and H28.16, the first f- token receives significant attention. This may represents an approximate "no-op", similar to typical <bos>-attending behavior in language models (SC.3).
C.3 <BOS> ATTENDING BEHAVIOR OF IDENTIFIED HEADS
By visualizing the attention patterns of the heads in the function induction mechanism, we found that many heads attend to the <bos> token. In most cases, this happens at positions not causally relevant to our tasks, hence, we defer discussion here in the appendix.
Attending to <bos> is a prevalent behavior in language models. Barbero et al. (2025) showed that "almost 80% of the attention is concentrated on <bos> token" in Llama-3 (405B). This phenomenon is sometimes referred to as "attention sink," (Xiao et al., 2024), and has attracted lot of interest in the research community. A common interpretation in the literature is that attending to <bos> represents
approximate "no-op" "resting" operation (Gu al., 2025; Barbero et al., 2025; Clark et al., 2019; Vig and Belinkov, 2019). Since attention weights must sum to 1 due to the softmax operation, the model learns attend <bos> when attention to other tokens needed in the current context.
C.4 ADDITIONAL INTERPRETABILIT ANALYSIS
C.4.1 LOGIT LENS ANALYSIS
In this section, we apply logit lens (nostalgebraist, 2020), a widely-adopted interpretability method, to off-by-one addition. This involves directly computing the logits from intermediate layer represen- tations using the final layer norm and the final unembedding layer.
We use Gemma-2 (9B) and 100 16-shot examples this set experiments. In Fig. 18, we report the logits of the base answer yhose (i.e., model outputting 3+3=6), the contrast answer yeont (i.e., model outputting 3+3=7) and their differences, computed using the contrast prompt Zcont (i.e., 1+1=3) as model input. In Fig. 19, repeat the experiments using Tbase (i.e., 1+1=2) as the input prompt.
By comparing Fig. 18(a) and Fig. 19()), we find that the curves in the two subplots begin to diverge notably after layer 25. This supports claim that the model performs standard addition in the early layers and apply the +1 function in late layers.
Additionally. by comparing Fig. 18(b) and Fig. 19(b), find that the logit diff decreases sharply after layer 38 in Fig. 18(b), a phenomenon absent in Fig. 19(b. This is consistent with our findings that H39.7 and H39.12 contribute significantly writing out the +1 function to the residual stream.
24

---

Published as a conference paper at ICLR 2026
(a) Logits
(b) Legit Dift, Hoggltyy
Log Lugetyrael
20-
Loght Deer
Laget
-10
20
25
Figure 18: Logit Lens Results when Using Tcont as the Input Prompt.
(a) Logits
(b) Logit Diff, Logittyyood alllidyyrr
25
Logit/baee Lugit
20
20-
Logst
10
Loge DNT
35-
5-
-10
15
2.0
25
30
35
40
5
1.0
15
20
25
30
35
Figure 19: Logit Lens Results when Using Tbase as the Input Prompt.
C.4.2 ACTIVATION PATCHING ANALYSIS
Residual Stream
(b) Attention Outpat
MLP Output
25%
05
25%
100
20
30
40
10
2.
30
4.0
1.0
20 Lagee
30
40
Eager
Layer
Figure 20: Activation Patching By Token.
Output
Query
(c) Key
(c) Value
(e) Atth Patters
50%
5-
10
25%
15
0%
5
10
15
10
15
3
10
15
50%
Figure 21: Activation Patching By Head.
this section, we apply activation patching (Meng et al., 2022) to off-by-one addition. We performed this analysis in the early stages of work to gather initial intuitions and signals for our problem, before transitioning to path patching for a more fine- imn-nnri understanding of the model's 's internal computation.
We use Gemma-2 2 (9B) and 100 4-shot examples in this set of experiments. First, we run forward passes for both the base prompt Thase and the contrast prompt Tcont- We store the activations and
25

---

Published as a conference paper at ICLR 2026
then replace the activations in the cont forward pass with corresponding activations in the Pbase forward pass. We consider activation patching by token (Fig. 20) and by head (Fig. 21). We report the ratio r' + FIM F #basel in these figures following previous works, We scaled the colormap in the figures to the range of [-50%, 50%] for clear visualization.
F(M F(M,)
Fig. 20(a) visualizes the information flow from in-context examples to the residual stream of the last " =" token. Additionally, Figure 20(b) highlights several layers, specifically layers 32, 36, and 39 at the last "=" token, and layers 35 and 38 at the answer tokens e; in the in-context examples. This aligns with the FI heads (H36.7, H39.7, H39.12 and PT heads (H35.9, H35.14, H38.6, H38.7. 7, H38.9) identified in $3.2. Figure 20(c) further reveals that MLP layers also play critical roles at certain positions. It is possible that FI heads write the function to the residual stream, with subsequent attention and MIPP layers involved in the execution of the +1 function. This hypothesis is inspired by prior observations of how MLP layers in transformer models are involved in arithmetic operations (Nanda al., 2023; Stolfo et al., 2023). In this work, we limit our focus to attention heads, deferring further analysis of MLP layers to future work.
Results in Fig. 21 guide and complement our path patching experiments in $3.2. The identified PT heads (H35.9, 9, H38.6, H38.7, H38.9) are highlighted in Fig. 21(b) and the FI heads (H36.7, H39.7, 77 H39.12, H32.1, H25.13) are highlighted in Fig. 21(d).
C.4.3 ALTERNATIVE HEAD ABLATION METHODS
In the main paper, we "ablate" or "knock out" a head by replacing its output in the Eoont forward pass with the corresponding head output in the Tbase forward pass. This instance-specific ablation approach is adopted to better isolate the +1 function computation in each instance. However, this differs from ablation methods commonly used in interpretability work, such as zero ablation (Halawi et al., 2023) or mean ablation (Wang et al., 2023).
To demonstrate the consistency of our findings across different ablation settings, we repeated the experiment in Fig. 4 using zero ablation and mean ablation. For mean ablation, we averaged head outputs at the final "=" token from 100 standard addition examples. We found that all ablation settings (zero ablation, mean ablation, and instance-specifie ablation), the contrast accuracy reduced to 0% and the base accuracy increased to 100% after ablation.
D CIRCUIT EVALUATION
Completeness
Minimality
40%
Orenty Steech
Suntioe Set
Change In Logit Differveton
Consclaiation Hend
Puno: Induction Basd
Outsojiation Hood
Tamply
30%
Presiex Tolenn Hoad
Funo. Induction Hood
FUM K3
Previcus Tolas Head
20%
-2
F(C
Figure 22: Circuit Evaluation.
In $4, we primarily validated the identified circuit using head ablation experiments and causal effect visualizations. Wang et al. (2023) proposed more rigorous framework for circuit evaluation, based on faithfulness, completeness, and minimality.
the following, we evaluate the identified circuit according these metrics. Note that focus on interpreting the "off-by-one component of the task, rather than the standard addition component. Hence, these circuit evaluation metrics are adapted accordingly to use F(M, Zbase) reference point.
The faithfulness metric measures whether a circuit C has a similar performance to the full model M, Le. whether FCC, Feout) is close to F (M, Feont), with F(C, 2r) defined earlier in $3.1. We find that
26

---

Published as a conference paper at ICLR 2026
F(M, Tbase) 7.17, F(M,Iccont) Fcone) = -1.26, and F(C,Iconl) 0.56, suggesting that C' recovers 7.17-0.36
7.17-(-1.26] = 78.4% of the performance of M.
The completeness criterion evaluates whether for each subset K S C. the difference between F(C',K Feons) and
is small. In the following, will omit the Toonz term for brevity. We use various different sets (e.g., randomly or greedily selected) as K and report the results in Fig. 4(b). We find most points representing (F) ((M) fall slightly below the line, while maintaining a monotonic trend, suggesting that the circuit C' is partially complete. This represents the best we can achieve with our current methodology. We also find that when K the set of all PT heads or all Fl heads, both f(C) K)) and f(M, K) are high, suggesting that the model favors ybase in next-token generation (i.e., 3+3=6) and switches back to standard addition under these ablation conditions. These observations are consistent with our function induction hypothesis.
Lastly, the minimality criterion measures whether each head v in C' necessary, by seeking a subset K S CMv that has high score of [FC U {u))) P(C),K)) We manually constructed the K sets for this purpose. As shown in Fig. 4(c), each head in is relevant to the task and has a non-trivial effect (>2%) in performing off-by-one addition.
D.1 IMPROVING THE IDENTIFIED CIRCUIT WITH LOCAL SEARCH
Our circuit evaluation above is based on the circuit discovery procedure described in $3.2, where we used 2% threshold to highlight relevant heads and ensure that the number of heads involved is manageable. This evaluation reveals that circuit C achieves faithfulness score of 78.4%, recovering substantial portion of the model's behavior but suggesting we may have missed several heads responsible for the remaining 22%.
To address this, we employ local search method. Specifically, we enumerate heads not in C' (i.e., to form new circuit = C'U {u}. and evaluate the faithfulness score of
v E C),, add each C". We identify 7 heads that produce faithfulness score changes greater than 1.5%, which we report in Table 5. These include 6 heads (H37.6, H38.8, H40.0, H37.4, H24.9, H32.4) that increase the faithfulness score and head (H36.6) that decreases it.
Including the score-ine inoe--ncreasing heads in C' (denoted C&) raises the faithfulness score to 89%. Including all 7 heads (denoted C7) reduces the faithfulness score to 82%, which still exceeds the 78% achieved by the original circuit C.
We further examine the attention pattern of these heads. H37.6, H24.9, H32.4, and H36.6 exhibit patterns consistent with FI heads, while H38.8 fits the description of consolidation heads. However, H40.0 and H37.4 4 do fit any head group in our function induction hypothesis. At tokens. H40.0 attends to itself in the forward pass of Thase but not in the forward pass of &cont, likely suppressing the standard addition answer (ybose) by weakening its consolidation. H37.4 attends to "=" tokens in previous ICL examples at each "=", likely retrieving the consolidation signal from earlier ICL examples.
A Faithfulness (%)
Rel. Logit Diff (%)
H37.6
3.13
0.88
H38.8
2,33
0.55
H40.0
2.29
-1.26
H37.4
1.96
0.39
H24.9
1.86
1.64
1132.4
1.67
1.29
H36.6
-3.44
-1.15
Table 5: Identifying Missing Heads with Local Search. The "Rel. Logit Diff r" columns reports the relative effect of these heads when patching to the final logits ($3.2). Previously we set a threshold of 2% and hence these heads were identified.
27

---

Published as a conference paper at ICLR 2026
E CIRCUIT ANALYSIS
Due to space limit, we mainly perform circuit analysis on function induction (FI) heads in the Gemma-2 (9B) model and present the most notable findings in the main paper ($4). In this section, we discuss remaining findings on FI heads in 1. We also present additional analysis on consolidation heads $E.2 and previous token (PT) heads in $E.3.
E.1 FUNCTION INDUCTION (FI) HEADS
H39.7
H39.12
H36.7
H32.1
H32.6
0.50
0.25
Output
Output
Output
Output
Output
(H24.9)
Heads (Normalized)
-0.25
-0.50
0
5
5 Output
5 Output
Output
Output
Output
Figure 23: Individual and Overall Effect of Identified Heads (Standard Addition).
H33977
H39.12
H36.7
H32.
H32.6
0.50
Input
0.25
Output
Oulput.
Oulpul
Output
Output
0.00
H25.13
(H33..4)
(H28.6)
(H24.9)
Heads (Normalized)
Input
-0.25
-0.50
0
0
5
0
5
Output
Output
Output
Output
Output
Figure 24: Individual and Overall Effect of FI Heads in Off-by-k Addition, k = -2
What do FI heads write out in standard addition? Our function vector style analysis in $4 primarily focuses on what FI heads write out in off-by-one addition. However, these heads may also assume roles in standard addition. To investigate this, we add the head outputs in the M (rase)) to the naive prompt Tnaive. and visualize the effect in Fig. 23. By comparing Fig. 5 and Fig. 23, observe that most FI heads contribute meaningful but distinct information in standard addition, with H39.12 being exception given its minimal effect in standard addition. The aggregated effect in the bottom-right panel in Fig. 23 suggests that Fl heads collectively suppress r and promote x in standard addition.
One possibility is that FI heads sharpen the answer x, or double-check it by performing (x - 1) + 1 in standard addition. In contrast, during off-be-one addition, the standard addition answers first
28

---

Published as a conference paper at ICLR 2026
H39.7
H39.12
H36.7
H32.1
H32.6
0.50
Input
0.25
Output
Output
Output
Output
Output
0.00
H25.13
(H13244)
(H28.6)
(H24.9)
Heads (Normalized)
-0.25
-0.50
0
5 Output
5 Output
5 Output
Output
Output
Figure 25: Individual and Overall Effect of FI Heads in Off-by-k Addition, k
H39.7
H39.12
H36.7
H32.1
H32.6
0.50
Input
0.25
Output
Output
Output
Output
0.00
H25.13
(H32.44)
(H28.6)
(H24.9)
9 Heads (Normalized)
Input
-0.25
-0.50
0
0
5
0
5
Output
Output:
Output:
Output
Output.
Figure 26: Individual and Overall Effect of FI Heads in Off-by-k Addition, h: 2.
"locked in" after early layers, and the Fl heads are repurposed to perform +1. We leave further investigation of this phenomenon to future work.
What do heads write out in off-by-k addition? Previously in Fig. 7, we demonstrated how the effect of H39.7 and H25.13 13 changes with respect to different offset In Fig. 24-26 we report the effect of all nine heads when & = -2 1, We find that for some heads (e.g., H32. and H24.9), their effect of suppressing remains consistent across different values. For other heads (e.g., H39.7, H39. 12, H25.13), their effect changes accordingly with k.
Causal Effect of FI heads in Other Models. Our previous analysis focuses on the causal effect of FI heads Gemma-2 (9B). To further demonstrate the universality of our results across models, we repeat the initial validation (head ablation) experiments ($4; Fig. 4) with Llama-3 (8B), Mistral-v&, (7B), Ll ama-2 (7B). Due the different tokenization methods of these models, we use addition in the range [0,9] for Mistral-ve 1 (7B) and Llama-2 (7B); [0,999] for Llama-3 (8B). We visualize the results in Fig. 27. Similar to the observations in Fig. 4, the models achieve non-trivial performance off-by-one addition, but switch back to perform standard addition when Fl heads were ablated.
In addition to the initial validation (head ablation) experiments, also repeated the further validation (causal analysis) with these three additional models. The results were visualized in Fig. 28-30. Our claims made with Gemma-2 (9B) hold across these three models: each Fl head sends out
29

---

Published as a conference paper at ICLR 2026
distinct signal, and their aggregated effect implements the +1 function. Please refer to the captions of Fig. 28-30 for detailed discussions on the role of each head.
One mixed result have is that in Llama-2 (7B), the oldest model among the four investigated, the function induced by FI heads is closer to closer to f(x) I than f(x) = a + 1. This
suggests that 1ama-2's FI heads may not be fully formed yet, which in turn explains Llama-253 weaker performance on off-by-one addition (6% accuracy on the range [0,9]).
(a) Liama 3 (8B), 3 Fl Heads
(b) Mistral-v0.1 (7B). Heads
Liama 2 (7B), 6 Fl Heads
Prompt
Contraat: Prompt (Ablate Fl Heads)
Contrast Prompt (Ablate Band. Heads)
100%
50%
Accuracy
Accuracy
Ancuracy
Figure 27: Head Ablation Experiments, Heads, Three Additional Models. In the random head ablation experiments, the number of ablated heads is equivalent to the number of FI heads identified in the model. The findings are consistent with those reported with Gemma-2 (9B) in Fig.
H26.2
H23.13
H23.15
Heads (Normalized)
1.0
0.5
Input
10
0.0
-0.5
-1.0
10
20
0
20
10 Output
10 20
Output
Output
Output
Figure 28: Individual and Overall Effect FI Heads in Off-by-k Addition, k 1,, Llama-3 (8B). While Llama-3 (8B) uses [0.999] tokenization, we visualize results in the range of [0,29] for readability. H26.2 promotes x + and sometimes z + 2; H23.13 and H23.15 promotes x 1 and They collectively contribute to promoting x + the output.
H30.2
H30.3
H30.4
H30.8
0.50
Input
0.25
Output
Output
Output
Output
H30.10
H30.18
(H31.2)
Heads (N (Normalized)
Input
-0.25
-0.50
Output
Output
Output
Output
Figure 29: Individual and Overall Effect FI Heads in Off-by-k Addition, k Mistral--v.
(7B). H30.3 promotes x and + 1; H30.4 and H30.10 promote digits larger than æ; H30.8 and
H31.18 suppresses æ. They collectively contribute to a function that's close f'xx) = x + 1.
30

---

Published as a conference paper at ICLR 2026
H31.30
H31.4
H30.26
H30.3 (*)
0.50
Inpat
5
5
0.25
5 Output
Output
Output
Output
H2
H29.16
Heads (Nermalized)
Imput
0..3
-0.50
5 Output
5 Output
Output
Figure 30: Individual and Overall Effect Heads in Off-by-k Addition, & = 1, Llama-2
(78). H31.30 promotes numbers greater than H31.4 suppresses z, H30.26 suppresses x, H30.3 suppresses r and promotes z + 1, H29.26 and H31.30 suppresses x, H29. 16 suppresses æ 1, x - and promotes
Their combined effect approximates f(x) + though it leans toward f(x) = x + 2, which may account for Llama-22 2's weaker performance on this task. (*) Effects of H30.26 and H30.3 are rescaled make the patterns more readable.
31

---

extract the PT head outputs at the Published as a conference paper at ICLR 2026
E.2 CONSOLIDATION HEADS
We repeat our function vector style analysis from $4, but this time use the consolidation heads as the subject. Concretely, we patch the outputs of these heads from the last token residual stream in off-by-one addition (e.g., 144-==3m222=5mmm33 report the effect of this intervention on the output logits in Fig. 31.
to the naive prompts (e.g.. "2=2m3=??"). We
We observe that three of these heads (H41.4, H40.11, 11, H40.12) are suppressing answers other than and one head (H41.5) ..5) is promoting answers other than æ. Their aggregated effect leads to promoting and suppressing + 1, which counters the effect brought by Fl heads discussed in Fig. 5.
H41.4
H41.5
H40.11
H40.12
Heads
0.5
Input
0.0
-0.5
5
5
0
5
0
5
Output
Output
Output
Output
Output
Figure 31: Causal Effect of Consolidation Heads. The aggregated effect of consolidation heads counter the effect of heads by promoting x and suppressing æ + 1.
Discussion. We name these heads as "consolidation heads" based three observations: (1) they appear in the final two layers; (2) they attend exclusively to the current token and the <bos> token, suggesting that they mainly process information locally at the current token; (3) causal analysis in Fig. 31 shows that some them are promoting 3+3=6 and some are promoting 3+3=7 in off-by-one addition, suggesting that they are weighing the two possible outputs collaboratively.
Despite these observations, our understanding on the exact role of these heads, and why they emerge, remain limited. We believe it relates to the broader phenomenon of "negative" behavior in language models, which has been noted as a challenge for current interpretability methods (Sharkey et al., 2025). We hope future work will present finer-grained interpretation of these heads.
E.3 PREVIOUS TOKEN (PT) HEADS
Head Ablation Experiments. To validate the role of previous token heads, first repeat the head ablation experiments in Fig. 4, but we ablated previous token heads instead. We consider both off-by-one addition and off-by-two addition, and 16-shots in the prompt. We report the results in Fig. 32. Ablating the previous tokens heads almost completely restores the model's 's default behavior. This supports our hypothesis that these heads are critical for inducing the function.
(a) km1
(b) k
Base Prompt
Contrast Prompt
Contrast Prompt (Ablate 8 Heads)
Contrast Prompt (Ablate 8 Rand. Heads)
100%
50%
100%
Accuracy
Accuracy
Figure 32: Head Ablation Experiments, Previous Token Heads, Gemma-2 (9B).
Causal Effect. To further investigate the causal effeet of previous token (PT) heads, adapt causal analysis method previously used for heads and consolidation heads. For example, consider the off-by-one addition prompt,
32

---

Published as a conference paper at ICLR 2026
H38.9 (*)
H38.6
H38.7
H33.9
H35.14 (*)
0.2
Input
0
0
5
Output
Output
Output
Output
Output
0.0
H29.5 (*)
Heads
Input
5
-0.2
Output
Output
Output
Output:
Figure 33: Causal Effect of Previous Token Heads. (*) Effects of H38.9, H35.14 and H29.5 are rescaled to [-0.02, 0.02] to make the patterns more readable.
tokens marked in brown, average the outputs over the two tokens (3 and 5), and add them to the forward pass of the naive prompt "2=2'n3=?"" at the token 2 marked in brown. In our experiments, we scaled this explanatory example to 100 4-shot examples of off-by-one addition to extract the PT head outputs.
We report the causal effect these PT head outputs in Fig. 33. Similar to the findings with heads, find that each PT head conveys distinct information. For example, H38.7 promotes x: + 1 and x + 2, H35. promotes x ++++ and suppresses x - 1. Collectively, these heads suppress x and promote + 1, which aligns with the hypothesized role of PT heads. Unexpectedly, these heads also promotes 1. We attribute this to the task shift from off- by-one addition and the naive prompt, and the token-level averaging operation we employ which may cause loss of information.
Pairs of Heads with Countering Effect. We notice that two pairs of PT head (H38.6/7 and H31.4/3 demonstrate opposing patterns, and they happen to in the same group group query attention. Similarly, two consolidation heads (H41.4/5, Fig. 31) have similar countering effect. Hence we hypothesize that group query attention may help these heads develop countering or hedging behaviors.
F TASK GENERALIZATION
F.1 TASKS AND DATA PREPARATION
In this section we describe the task pairs we used in $5 with more details.
Off-by-k Addition. For experiments in the range of [0.9], consider k E (-2, 11,,11,1,)) For experiments in the range of [0,99] and [0,999], we consider E have reported the results Fig. 13-14 incorporating the range and offset information. We use 16 shots in the experiments in Fig. 6(a).
-1, 1, 2..,,...)) We
Shifted Multiple-choice QA. We focus on 6 subjects the MMLU dataset (Hendrycks al.,
2021): high school government and politics, high school US history, US foreign policy, marketing, high school psychology, sociology. We downloaded the MMLU dataset from O hendrycks/ test. We chose these subjects because Gemma-2 (9B) achieves 90% accuracy with 5 shots them. For subjects where Gemma-2 (9B) achieves lower accuracies, tracking and analyzing performance on Shift-by-On MMLU becomes challenging, because the model could score points by random guessing. We use shots the experiments in Fig. 6(b), where the shots combine "validation" and "dev" examples from the original MML.U dataset.
33

---

token in Gemma-2's Published as a conference paper at ICLR 2026
Caesar Cipher. We adopted cyclic approach where "a" is considered the next character after "z". We also included both lower-case or upper-case examples, e.g., "c-> and -C D" are both valid examples in ROT-1. We use 16 shots in the experiments in Fig. 6(c).
In the early stages of this work, we experimented with multi-character Caesar cipher. To prevent single unit (e.g., "ew"
multiple characters from being tokenized tokenizer), we used preceding whitespace (_) before each character, formatting it as "_e_"" so that "_e" and "_w" became separate tokens. However, we ultimately focused one-character Caesar cipher in the experiments because Gemma-2 (9B) has insufficient performance on the multi-character version. The tokenization-awe aware formatting was retained. The actual model input will be "_ _d" for the example "c d=
Base-k Addition. We sampled two-digit addition problems using a procedure similar to off-by-k addition, with one additional constraint that the sum number has two digits in both base- 10 and base-k. We use 32 shots in the experiments in Fig. 6(d). For the base-8 addition analysis in $5.2 and Table 3, examples for Case 1-3 were resampled.
F.2
RESULTS
Full Results using Different Offsets and Bases. Previously in Fig. 6, we report results on repre- sentative cases, e.g., & in shifted MMLU. In Fig. 34-36, we report results of the full list of offsets and subjects.
off-by-k addition, the subject of "high school government and politics"
We observe that some of these task variants exceed Gemma-2 (9B)'s capabilities. For instance, Gemma-2 2 (9B) has notable performance on cipher when k E {-2, 1, 1, 2,3, 13] but shows insuf- ficient performance in other settings. Similarly, it only exhibits non- non-trivial performance on certain subjects of Shifted MMLU. However, when models do have non-trivial performance, consistently see the involvement
heads, evidenced by the decreased contrast accuracy after ablating them.
(3) X-
k- -1
k-)
lblK R=
Fromutt
Costrast Frompt
Contrnat Prompt
Controat Prompt (Abinte 6 Bund. Haoda)
0%
50%
100%
0%
50% Anturacy
100%
0%
50% Ancuracy
100%
05%
50% Accurayy
100%
Acouracy
Figure 34: Task Generalization with Heads, Off-by-k Addition. We consider addition in range of [0.9] and & E 1,,
34

---

Published as a conference paper at ICLR 2026
High School
Goverment and Politics
(h) High School US History
(6) US Forsign Policy
Rosa Prompt
Contrast Prompt
Contrast Prompt
(Abiato
Escads)
100%
50%
100%
100%
Accuracy
Accuracy
Accuracy
(d) Marketing
High School Paychology
Sociology
Bosn Prompt
Contrast Prompt
Contrast Prompt (Ablate F1 Heads)
Contrast Prompt
50%
100%
50% Accuracy
100%
50% Accummy
100%
Accurany
Figure 35: Task Generalization with FI Heads, Shifted MMLU.
k
(b) =
(ct k
k
Frompt
Costrast Frompt
Costrest Frompt
Contrast (Ablate 6 Rand, lioads)
0%
50%
100%
0%
50%
100%
0%
50% Ancuracy
100%
06
50%
100%
Acmuracy
Anturacy
Accarary
Figure 36: Task Generalization with Fl Heads, Base-k Addition. We consider A E (6, 7, 7,8,9). The dashed lines represent the base prompt's 's contrast accuracy, emphasizing the delta in contrast accuracies between rows.
35

---

Published as a conference paper at ICLR 2026
1b02
01
Now Preee!
(Maa
Cestrut
1304
26% Acsumdy
100%
65%
100%
Ataray
Acarity
Astinyy
Castruat Proegt
Castrou Obbar 6. Sovel
50%
35%
52%
50%
10.2
50.2
Bose Fraent
Odiir Tank Sindel
50h
1300
50% Acturscy
50/
100%
53%
100%
50%
Accarayy
Astancyy
Castruat Preeg!
Obaas 9: Sionkl
Castrul
2022-10
191-13
Preer!
Castrutt
Castrul
Obteic Jants Hintal
1075 Arouray
1364 05
166%
50/ Acarucy
100%
64%
100% 4%
50%
106%
Actumayy
Aczunayy
Figure 37: Task Generalization with FI Heads, Caesar Cipher. We consider k 1-12,-11, -1} and 21, ..., 13}. In this figure, we ablate 6 heads plus 3 additional FI heads (discussed in $4 and Fig. 5), yielding clearer pattern than ablating 6 heads alone.
36

---

Published as a conference paper at ICLR 2026
ligh: Politiis
Addition
(d) Adddtin
Base Fromutt
Contrnlt Pronpt-
Contrest Frompt (Ablate 9 Iteeds)
Controet
Contrist (Abinte 9 Hand, (tenss)
100%
50%
100%
Accurscy
Accuracy
Accuracy
Accuracy
Figure 38: Task Generalization with FI Heads, Ablating 9 FI Heads. We repeat the experiment in Fig. 6, this time ablating three additional FI heads (H32.4, H28.6 6. H24.9) which showed weaker effect (1% < |r| 2%) during circuit discovery on off-by-one addition.
Ablating three additional FI Heads. Previously in Fig. 6, ablate the 6 FI heads identified in $3.2 by setting a threshold of [r] >> 2%. In $4 and Fig. we showed that additional Fl heads with weaker effect (1% < |r| 2%) also contribute meaningfully off-by-one addition. Here consider repeating the experiments on task generalization in Fig, 6 and ablating the 9 heads together. We report the results in Fig. 38.
We find that the weaker heads contribute meaningfully to the Shifted MMLU, causing its contrast performance drop to near 0% when all heads are ablated (Fig. 38()), contrasting with 12% when 6 heads are ablated (Fig. 6(b)). We have similar observation with Caesar Cipher (k = 2). where contrast accuracy drops to 0% (Fig. 38(c)), contrasting with 36% when heads are ablated (Fig. 6(c)). These observations suggest that the 3 heads may specialize in letters more than numbers. Understanding these detailed specializations will be interesting direction for future work.
G INSIGHTS FROM TRAINING TOY MODELS
Our work mainly focuses on interpreting off-the-shelf large language models, which is a common practice in works like Wang al. (2023); Hendel et al. (2023); Todd et al. (2024). One alternative and promising research methodology is to train smaller transformer models from scratch, which allows precise control of the training data and more likely lets isolate the circuit. This methodology is exemplified by works like Olsson et al. (2022); Nanda et al. (2023); Minegishi et al. (2025). We present small preliminary study in this direction below.
Specifically, we trained a ranttnly-inittalizzd,, standard transformer model to perform addition (in the range of [0,99] with 5-shot input examples. The transformer has Naayer 3. Thead 4, dhead 128, d, 512. We use the Adam optimizer, with its weight decay set to 0.0001. We use an initial learning rate 0.001, reduce the learning rate by half when the validation accuracy does not improve after 10 epochs, and stop training when the validation accuracy does not improve after 50 epochs.
We consider three different settings for the training data. We summarize the results in Table 6 and discuss our findings below.
Trained on Test on
k=0
k=2
k=0
98.3 + 1.0
0.7 0.4
0.0 = 0.0
k=0 and k=1 (50%/50%)
53.7
5.4
42.3
5.5
1.7 1 3.2
k= and (50%/50%)
1
75.7
4.2
10.6
Table 6: Results Training Toy Transformer Models on Off-by-k: Addition. We report mean and standard deviation over 5 runs.
Trained standard addition (k=0): The model achieved 98.3 accuracy k=0, and near-zero accuracy on off-by-one addition (k=1).
Trained on k=0 and k=1, 50%/50% Mix: The model achieved around 50% test accuracy on both k=0 and k=1, suggesting that it still could not infer the task in-context at the end of training. We tried several adjustments, such as increasing the number of layers or changing the mixing ratio of the training data, but none of these yielded improvements.
37

---

Published as a conference paper at ICLR 2026
Trained on k=0 and k=2, 50% Mix: We explored whether this would enable the model to generalize to k=1. The model reached 16.0% accuracy on k=0, 10.6% on k=2, but surprisingly 75.7% k=1. It appears the model averages the two training tasks: when trained on 1+1=2 and 1+1=4, tends to output 1 1+1
Overall, these results suggest that training a toy model to perform off-by-one addition is non-trivial. It likely requires specific changes to the data distribution or training curriculum. The search space is large and requires a separate study to address fully. We will leave this as future work.
H REPRODUCIBILITY
Code Release.
Our code and data are released at Q INN-USCffunntiion--niiccio...
Frameworks. We primarily the 7 transformer--ens (Nanda and Bloom, 2022) library for model inference and interpretability analysis. This library is built on the 2 transformers (Wolf al., 2020) library. We have also used the O 1mm-trraspareecc-ttoll (Ferrando and Voita, 2024; Tufanov et al., 2024) for early exploration.
Hardware. All experiments were conducted with one NVIDIA RTX A6000 GPU (48GB). Path patching experiments involving 100 4-shot examples and iterating over all attention heads for a given target node will typically take hours.
I LARGE LANGUAGE MODEL USAGE
After writing initial draft, we refined using language models to correct grammar, enhance clarity, and improve overall presentation. We also consulted language models for experiment implementation and matplotlib questions.
38


Reviews:
[{"review_id": "6XPlyqIuzf", "reviewer": "Reviewer_CMXz", "summary": "This paper investigates the mechanistic underpinnings of task-level generalization in LLMs through in-context learning. The authors use the novel, counterfactual task of off-by-one addition as a probe to understand how models adapt to new rules. This task requires a two-step process: performing standard addition and then applying an unexpected +1 function. Using circuit-style interpretability techniques, primarily path patching on the Gemma-2-9B model, the authors trace the internal computations responsible for this behavior. The authors find the identification of a circuit that generalizes the well-known induction head mechanism from the token level to the function level. Instead of just copying a token, this mechanism induces and applies an entire arithmetic function (+1) learned from the in-context examples. The circuit is composed of three distinct groups of attention heads: Previous Token heads, Function Induction heads, and Consolidation heads. Moreover, the authors show that this function induction mechanism is not specific to off-by-one addition. They provide evidence through head ablation experiments that the same core mechanism is reused by the model to solve a diverse range of other tasks requiring in-context rule adaptation, including off-by-k addition, shifted multiple-choice QA, Caesar ciphers, and base-8 addition. This suggests it is a general, flexible, and composable capability for task generalization.", "questions": "Please also see **Weaknesses** section above.\n\n[Q1] How should these findings suggest the way we evaluate, train, prompt, or otherwise use LLMs in practice? This paper may benefit more from an extensive discussion of how the function induction circuits on these tasks could be relevant for practical uses of LLMs in the real world.\n\n[Q2] Could we expect/predict how these explanations transfer to larger models (more than 10B/100B/MoE, etc) well?\n\n[Q3] Minegishi et al. (https://openreview.net/forum?id=LNMfzv8TNb) have studied the extension of induction heads similarly; the mechanism not only copying necessary tokens, but inferring the task from the context tokens and predicting the next-tokens corresponding to the query. Could you discuss the relevance and difference between them?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "[S1] This paper is well written and easy to follow even if without sufficient prior knowledge on this domain.\n\n[S2] The experiment and analysis seem to be simple yet have effective coverage and depth.\n\n[S3] The paper demonstrates analyses in the tasks where function-induction heads appear, suggesting that the function-induction heads in LLMs might be responsible for the capability to identify the off-by-one function and other new functions in in-context learning.", "weaknesses": "[W1] It is unclear how these results could generalize well and be universal across different models. Most analyses focus on the Gemma2-9B model, with function-induction heads only identified in other models but not sufficiently validated (we can see the results of Mistral/Llama in Appendix D.2) as done in Gemma2.  Moreover, usually the lines of mechanistic interpretability research (esp. induction heads) have started with simple toy models (e.g., a few layer attention networks) to clearly point out the target circuits in the model, and then expanded the analysis to LLMs. In contrast, this work just began with LLMs, where we could not exactly figure out the circuits. I think we might not be able to have higher confidence on the conclusion than the previous works.\n\n[W2] It's not clear what aspect of function induction heads could be attributed as a generalization of induction heads. While induction heads could have a clear explanation about the role of key/value/query in the attention, the counterparts in function induction heads are unclear.\n\n[W3] From the attention analysis in Figure 2 and Appendix D.2, I think it is quite challenging to distinguish among Group-1/2/3 heads at a glance. They all looked similar. Also, the analysis in the main paper is only about Group-2 heads (Figure 5/7). It may be necessary to clearly explain how we can classify those heads into Group-1/2/3 in the main text.\n\n[W4] Limitation is not discussed in the main paper. Because the mechanistic interpretability may not perfectly explain how LLMs work in practice, the extensive discussion should be included.\n\n[W5] While the results are interesting and validated to some extent, it is not really discussed how this finding could contribute to the progress of how LLMs are developed in practice. It is not clear what the takeaways from this analysis can be.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iCgRMzlsf3", "reviewer": "Reviewer_nVof", "summary": "This paper studies in-context learning through the lens of a non-typical logic task: off-by-one addition. The idea is that inputs are two number sums (3 + 4) and outputs are the sum plus one (8). They use empirical mechanistic interpretability techniques to try to understand how the LLM solves the problem. In their results, they find a more abstract version of the induction head, they show multiple heads solve different parts of the task, and they show the same mechanism works with other, similar tasks beyond off-by-one addition. \n\nIn more detail, the main results are as follows:\n\n- Off-the-shelf LLMs solve the off-by-one addition task in context and perform better with more examples.\n- Path patching can be used to identify which parameters are contributing to different parts of the task. Path patching works by taking activations from a base task (regular addition) and replacing them in a network prompted to do the contrast task (off-by-one addition). If, after the patch, the network solves the base task, this is evidence that the patched part of the weights were key to the contrast part of the task (the +1 operation, for example). In this paper, all patches are the weights for particular attention heads. They identify several heads (they call them function induction heads), for which indeed replacing weights of the contrast task with the base task leads the network to revert to base task behavior (100% accuracy on the base task, 0% accuracy on the contrast task).\n- They explore several other tasks besides off-by-one addition, and show similar phenomena.", "questions": "What is the FV head and how is it different than the FI head?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The path patching results are quite compelling. It is very surprising to me that replacing the heads from the contrast task with those of the base task leads to any sensible behavior at all, much less reversion to correct base task performance.\n- I appreciate that the authors presented several other tasks in section 5. The results are a bit more mixed on the other tasks, but are nonetheless consistent with the narrative presented in the paper.", "weaknesses": "- There was not enough description of the differences between the FI head and the FV head. All that is reported is that the heads appear at different layers. The paper should describe conceptual differences between the solution concepts. If there aren’t major conceptual differences, this is a concern. The fact that the heads appear at different layers could be explained by other factors, like different models.\n- I would quibble with “Off-by-one addition is likely an unseen task to these language models and represents a novel challenge”. Model producers and benchmarks for in-context learning often include synthetic tasks that don’t correspond to typical learning tasks. I think it is likely safe to assume models are trained on this kind of thing as well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qZsoAavvrs", "reviewer": "Reviewer_xswB", "summary": "This paper investigates how large language models (LLMs) generalize to unseen tasks in-context by analyzing a counterfactual arithmetic task called off-by-one addition.\nUsing mechanistic interpretability techniques, particularly path patching, the authors identify a novel computational structure, the Function Induction (FI) mechanism, which generalizes the known “induction head” mechanism from token-level copying to function-level generalization.", "questions": "In Figure 2, Group 3 indeed behaves like a previous-token head, but it also shows strong attention to the very first token (e.g., `<bos>`). Why does it attend so strongly to the first token? What role does this attention play in the function induction process?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The off-by-one task is simple yet counterfactual, enabling precise mechanistic tracing. The experiments are carefully designed to isolate the function-induction process.\n\n- The analysis spans six modern LLMs (Gemma-2, Llama-2/3, Mistral, Qwen-2.5, Phi-4), confirming the generality of findings.\n\n- The introduction of Function Induction (FI) heads extends prior induction-head results to function-level abstraction, a conceptually and methodologically significant advance.\n\n- The paper provides one of the most concrete demonstrations of compositional and reusable circuits in LLMs, aligning with recent interpretability goals.", "weaknesses": "- Ambiguous motivation:\n\nIn Line 41, the paper states that “our understanding is still limited, especially regarding more complex generalization scenarios involving unexpected elements or newly defined concepts in the task.” However, the actual experiments focus on a very simple and synthetic task (off-by-one addition). It is therefore difficult to claim that the study meaningfully addresses “more complex generalization scenarios.”\n\n- Unclear mechanism of Group 1 (consolidation heads):\n\nThe discovery of the Function Induction (FI) heads is an excellent contribution, but the role of Group 1 heads, those that most directly influence model outputs, remains vague. These heads attend strongly to the first token (e.g., `<bos>`), yet the reason such attention is crucial is not sufficiently explained.\n\n- Limited scope and real-world implications:\n\nThe paper’s experiments are all conducted on toy arithmetic or algorithmic tasks. While this design is justifiable for achieving clear mechanistic insights and is a well-accepted approach in the mechanistic interpretability community, the paper would be stronger if it discussed how the identified mechanisms might translate to more realistic or linguistically complex tasks.\n\nDespite these limitations, I found this paper highly compelling overall. The experiments are exceptionally clear, the evidence is convincing across multiple models, and the identification of FI heads represents a meaningful step forward in mechanistic interpretability research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SX7JP6cw3y", "reviewer": "Reviewer_Bkv2", "summary": "The paper introduces an in-context learning task called \"off-by-one additio\", where examples like `2+2=5`, `3+3=7`, and `4+4=?` require the model to infer the hidden “+1” rule. Using mechanistic interpretability, the authors identify a three-part circuit of attention heads:\n\n1. Group 3 – Previous-Token heads: detect local patterns by attending from each answer to the preceding “=”, capturing the off-by-one discrepancy.  \n2. Group 2 – Function-Induction heads: attend from the test “=” to previous answers, encoding the learned “+1” transformation.  \n3. Group 1 – Consolidation heads: attend to the current token and `<bos>`, integrating signals to produce the final output.\n\nAblation studies show that Group 3 → Group 2 → Group 1 forms a causal chain implementing the “+1” rule.  \nThe same mechanism generalizes to other tasks — off-by-*k* addition, shifted QA, Caesar ciphers, and base-8 arithmetic — revealing a reusable **function-induction circuit** within transformers.", "questions": "See Weaknesses", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clearly written and easy to follow, with a well-organized presentation of methods and results.  \n2. The evidence and ablation studies are convincing, providing solid empirical support for the proposed mechanisms.  \n3. The authors present an interesting and novel mechanistic explanation of in-context learning behavior in language models.", "weaknesses": "1. It is unclear how much of the observed flexibility of these attention heads originates from pretraining data. The paper assumes that the off-by-one addition task is novel, but it is plausible that similar unconventional arithmetic examples (e.g., math puzzles or riddles) appear in the pretraining corpus.  \n2. This raises two key questions:  \n   i. If the model were trained only on standard addition examples (e.g., `2+2=4`, `5+5=10`), would it still perform well on the off-by-one addition task?  (let's say we train them in an in context manner like the sequence of summations.)\n\n   ii. If not, what aspect of the pretraining process enables these heads to adapt and represent such novel generalization?\n\n3. These tasks appear quite simple; one would expect that even a relatively small transformer model, with significantly fewer layers, should be able to infer the in-context logic. With this in mind, I have the following questions:\ni. If I replace the embeddings corresponding to these heads with zeros or random vectors, why can’t other heads—perhaps in later layers—compensate for that and still recover the in-context logic?\nii. What determines the specific role of each head? Is this role fixed during training?\niii. what happens if only use first few layers say 6 out of 24/36, can the model still recover the logic? Is the model uses other heads for doing so(if the answer is yes)?\n\nOverall, I think the paper has a nice observation, but it is quite random to me that why certain heads are very important if we corrupt them other heads cannot compensate for that, and how much of all these in context learning capabilities is still linked to the pre-training phase.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)