Language Models Use Lookbacks to Track Beliefs
Authors: Nikhil Prakash
Natalie Shapira
Arnab Sen Sharma
Christoph Riedl
Yonatan Belinkov
Tamar Rott Shaham
David Bau
Atticus Geiger
Venue: ICLR 2026 Poster (2026)
Keywords: Mechanistic Interpretability
Belief Tracking
Theory of Mind
Primary area: interpretability and explainable AI
TL;DR: We study how language models represent and track characters’ beliefs in a story.

Abstract: How do language models (LMs) represent characters’ beliefs, especially when those beliefs may differ from reality? This question lies at the heart of understanding the Theory of Mind (ToM) capabilities of LMs. We analyze LMs' ability to reason about characters’ beliefs using causal mediation and abstraction. We construct a dataset, CausalToM, consisting of simple stories where two characters independently change the state of two objects, potentially unaware of each other's actions. Our investigation uncovered a pervasive algorithmic pattern that we call a lookback mechanism, which enables the LM to recall important information when it becomes necessary. The LM binds each character-object-state triple together by co-locating their reference information, represented as Ordering IDs (OIs), in low-rank subspaces of the state token's residual stream. When asked about a character's beliefs regarding the state of an object, the binding lookback retrieves the correct state OI and then the answer lookback retrieves the corresponding state token. When we introduce text specifying that one character is (not) visible to the other, we find that the LM first generates a visibility ID encoding the relation between the observing and the observed character OIs. In a visibility lookback, this ID is used to retrieve information about the observed character and update the observing character's beliefs. Our work provides insights into belief tracking mechanisms, taking a step toward reverse-engineering ToM reasoning in LMs.

Published as a conference paper at ICLR 2026
LANGUAGE MODELS USE LOOKBACKS TO TRACK BELIEFS
Nikhil Prakash 9, Natalie Shapira Arnab Sen Sharma Christoph Riedl
Yonatan Belinkoy Tamar Rott Shaham 2 David Bau Atticus Geiger*
Northeastern University
Technion
MIT CSAIL
Goodfire iPr(Ai) Group
ABSTRACT
How do language models (LMs) represent characters' beliefs, especially when those beliefs may differ from reality? This question lies at the heart of under- standing the Theory of Mind (ToM) capabilities of L.s.. We analyze LMss' ability to reason about characters' beliefs using causal mediation and abstraction. We construct dataset, CausalToM, consisting of simple stories where two characters independently change the state two objects, potentially unaware of each other's s actions. Our investigation uncovers pervasive algorithmic pattern that call a lookback mechanism, which enables the LM to recall important information when it becomes necessary. The LM binds each character-objeec-state triple together by co-locating their reference information, represented as Ordering IDs (Ols), low-rank subspaces of the state token's residual stream. When asked about a character's ' belief's regarding the state of an object, the binding lookback retrieves the correct state Ol and then the answer lookback retrieves the corresponding state token. When we introduce text specifying that one character is (not) visible to the other, we find that the LM first generates visibility ID encoding the relation between the observing and the observed character Ols. In visibility lookback, this ID is used to retrieve information about the observed character and update the observing character's 's beliefs. Our work provides insights into belief tracking mechanisms, taking a step toward eegiine-nginn ToM reasoning in LMs.
INTRODUCTION
Theory of Mind (ToM), the ability to infer others' mental states, is essential aspect of social and collective intelligence (Premack & Woodruff, 1978; Riedl et al., 2021). Recent studies have established that LMs solve some tasks requiring ToM reasoning (Street et al., 2024; Strachan et al., 2024a; Kosinski, 2024), while others have highlighted shortcomings (Ullman, 2023; Sclar et al., 2025; Shapira et al., 2024, inter alia). Previous studies primarily rely on behavioral evaluations, which do not shed light on the internal mechanisms by which LMs encode and manipulate representations of mental states to solve (or fail to solve) such tasks (Hu et al., 2025; Gweon al., 2023).
In this work. we examine how LMs internally represent and track beliefs of characters, a core aspect of ToM (Dennett, 1981; Wimmer Perner, 1983). classic example the Sally-Anne test (Baron-Cohen et al., 1985), which evaluates ToM in humans by assessing whether individuals can track conflicting beliefs: Sally's 's belief, which diverges from reality because of missing information, and Anne's belief, which is updated based on new observations. Our goal is to determine whether LMs learn a systematic solution to such tasks or rely on superficial statistical association.
We construct CausalToM, a dataset of simple stories involving two characters, each interacting with an object to change its state, with the possibility of observing one another. We then analyze the internal mechanisms that enable L1ama-3-77 -1--33-----Isttucee Llama-3.
and Qwen2. . 5-1 - B--nnstre ruct (Grattafiori et al., 2024; Bai et al., 2023) to reason about and answer questions regarding the characters' beliefs about the state of each object (for sample story, see Section 3 and for the full prompt refer to Appendix A).
Correspondence prakash. nf www.rrteesterr..c.

---

Published as a conference paper at ICLR 2026
Our findings provide strong evidence for system- atic solution to belief tracking. We discover that LMs pervasive computation, which refer to as the lookback mechanism, for belief tracking. This mechanism enables the model to recall impor- tant information at later stage. In a lookback, two copies of single piece of information are trans- ferred to two distinct tokens. This allows attention heads at the latter token look back at the ear- lier one when needed and retrieve vital information stored there, rather than transferring it directly (see Fig. 1).
Payload
Address
We identify three key lookback mechanisms that collectively perform belief tracking: 1) Binding lookback (Fig. 3()): First, the LM assigns ordering IDs (Ols; Dai et al. 2024) that encode whether character, object, or state token appears first or sec- ond. Then, the character and object OIs are copied to the corresponding state token and the final token residual stream. Later, when the LM needs to an- swer a question about a character's beliefs, it uses this information retrieve the answer state OI. 2) Answer lookback (Fig. 3(i)):: Uses the answer state Ol from the binding lookback retrieve the answer state token value. 3) Visibility lookback (Fig. 7): When a visibility condition between characters is mentioned, the model employs additional reference information called the visibility ID to retrieve infor- mation about the observed character, augmenting the observing character's awareness.
Source
Pointer
Source
Recalled
Lookback Token
Token
Token
Figure 1: The lookback mechanism performs conditional reasoning: The source token con- tains reference information that is copied into two instances, creating pointer and address. Next to the address in the residual stream is a payload. When necessary, the model retrieves the payload by dereferencing the pointer. Solid lines represent information flow, while the dot- ted line indicates the attention "looking back" from pointer address.
Overall, this work only advances understanding of the internal computations in LMs that enable ToM but also uncovers a pervasive mechanism that plays a foundational role for in-context reasoning. All code and data supporting this study are available https //beliet ie baulal ab. info.
2 THE LOOKBACK MECHANISM
Our investigation uncovers a recurring pattern of computation that we call the lookback mechanism. In lookback, a source reference is copied (via attention) into address copy in the residual stream lookbuck token that occurs later
of the recalled token and pointer copy in the residual stream in the text. The LM places the address alongside payloud in the recalled token's 's residual stream that can be brought forward to the lookback token necessary. Fig. shows a generic lookback.
That is, the LM can use attention dereference the pointer and retrieve the payload present in the residual stream of the recalled token (which might contain aggregated information from previous tokens), bringing it to the residual stream of the lookback token. Specifically, the pointer at the lookback token forms an attention query vector, while the address at the recalled token forms a key vector. The pointer and address not necessarily exact copies of the source reference, but they do have a high dot product after being transformed by a query or key attention matrix, respectively. Hence, OK-cireuit (Elhage al., 2021) established, forming bridge from the lookback token to the recalled token. The LM uses this bridge to move the payload that contains information needed to
Althhouhh this mechanism may resemble induction heads (Elhage al., 2021: Olsson al., 2022), differs fundamentally. In induction heads, information from a previous token occurrence is passed only to the subsequent token, without being duplicated to its next occurrence. In contrast, the lookback mechanism copies the same information not only to the location where the vital information resides but also to the target location that needs retrieve that information.
2

---

Published as a conference paper at ICLR 2026
80
Character
60
Object
Layen
State
40
11A
20
story.
Bob Maggy and
Carla
are working 1
busy
restauran
an
order
Bob - Maggy grabs
opaque
bottle + mug and fills
miikkk
Then
Carla
grabs
and
opaque
cup
and
fills
with
coffee
Question
what does Bob bel believe Live
the bo bottle contains
Answer
comp
Figure 2: Tracing information flow of crucial input tokens using causal mediation analysis.
complete the subtask through the OV-circuit. See Appendix C for background on the residual-stream framework well on QK- and OV-eircuits.
To develop an intuition for why an LM would learn to implement lookback mechanisms, consider that during training. LMs process in sequence with no foreknowledge what might come next. Instead of trying to resolve every possible future question about the current context, it would be useful to place addresses alongside payloads that might useful to remember the future when performing a variety of downstream tasks. In our setting, the LM constructs representation of story without any certainty about the questions may later be asked about that story, so the LM localizes pivotal information in the residual stream of certain tokens, which later become payloads and addresses. When the question text is reached, pointers are constructed that reference this crucial story information and dereference it to find answer to the question.
3 EXPERIMENTAL SETUP: DATASET, MODELS, AND METHODS
3.1 DATASET
Existing datasets for evaluating ToM capabilities of LMs are designed for behavioral testing and lack counterfactual pairs needed for causal analysis (Kim Sundar, 2012). To address this problem, we construct CausalToM, structured dataset of simple stories, where each story involves two characters, each interacting with distinet object causing the object to take unique state. For example: "Characterl and Character2 are working in busy restaurant. To complete an order, Characterl grabs an opaque Object1 and fills it with Statel. Then Character2 grabs ancther opaque Object2 and fills it with State2." " We then ask the LM to reason about one of the characters' beliefs regarding the state of object: "What does Character1 believe Object2 contains?" We analyze the LM's ability to track characters' beliefs in two distinct settings. (1) No Visibility, where both characters unaware of each other's actions, and (2) Explicis Visibility, where explicit information about whether character can/cannot observe the other's 's actions is provided, c.g., "Bob can observe Carla's actions. Carla cannot observe Bob's 3 actions." We also provide general task instructions (e.g., answer unknown when character is unaware); refer to Appendices & for the full prompt and additional dataset details. All subsequent experiments are conducted 80 samples that the model answers correctly. We also demonstrate generalization the mechanism to Bog dataset (Gandhi et al., 2024) in Appendix M.
3.2 MODELS
Our experiments analyze Qwen2. 5-14--11-nnse
Llama 3-70---------- Itusatut and Llama-3 1- 1-405BBInnstrucs models in FP32, FP16, and INT8 precision, respectively, us- and be found in Appendix N. We selected these models due to their strong behavioral performance under both no-visibility and explicit-visibility conditions. Additional details be found in Appendix D.
ing NNsight (Fiotto-Kaufmann et al., 2025). Results for 5-122.5-14B-nns 1--.:3..1-00--nntttee
3

---

Published as a conference paper at ICLR 2026
Layers
coffee
60
(ii)
Answer lookback
oofffer
Answer
50
Answer State O
(i) Binding lookback
40
30
Character1 Ol
Character2 OI
State1 O
Object2 OI
State2 OI
Object1
20
10
To complete
urdec
Bob
an
apseee bottle
and
fills it
wich
beez
mhn Carla gr grabs
another
opsque
cup
and
Eills
with
coffee
Coestion
What
does
Carla
believe
the
cup
containa
?
Answer
Figure Belief Tracking with no visibility between characters. We hypothesize that the LM tracks beliefs using two lookback mechanisms. First, in (i) Binding lookback, LM binds together each character-ohject-staee triple in the state token residual stream. When asked about specific character-objer pair, the LM looks back to the corresponding Ols to retrieve the correct state OL. Second, in (ii) Answer lookback, LM dereferences that state OI (used as a pointer) to retrieve the token value of the correct state. Colors indicate information type, shapes indicate role of information in lookback (see Fig. 1), e.g., state Ol is a payload (A) in (i) and a pointer-address (O) in (ii).
3.3 CAUSAL MEDIATION ANALYSIS
Our goal is develop mechanistic understanding of how LMs reason about characters' beliefs and answer related questions (Saphra & Wiegreffe, 2024). A key method for conducting causal analysis is interchange interventions (Vig et al., 2020; Geiger al., 2020; Finlayson al., 2021), in which the LM is run on paired examples: an original input o and a counterfactual input and certain internal activations the LM run on the original input are replaced with those computed from the counterfactual, process also known as activation patching. We begin our analysis by tracing information flow from key input tokens to the final output, by performing interchange interventions on the residual vectors. Specifically, we construct an intervention dataset where contains question about the belief character not mentioned the story, while the story in c includes the same queried character, shown in Fig. 2. The expected outcome of this intervention is change in the final output o from unknown to a state token, such as beer. We conduct similar interchange interventions for object and state tokens (refer to Appendix E for details).
Figure 2 presents the aggregated results of this experiment for the key input tokens Characterl, Objectl, and Statel. The cells color-coded to indicate the interchange intervention accuracy (IIA; Geiger et al., 2022). Even at this coarse level Causal Mediation Analysis (Mueller et al., 2024; Vig al., 2020; Meng et al., 2022), several significant insights emerge: 1) Information from the correct state token (beer) flows directly from its residual stream to that of the final token in later layers, consistent with prior findings (Lieberum al., 2023; Prakash et al., 2024); Information associated with the query character and the query object is retrieved from their earlier occurrences and passed the final token before being replaced by the correct state token.
Desiderata Based Patching via Causal Abstraction The causal mediation experiments provide a coarse-gre coarse-grained analysis of where information flows, but do not identify what information is being transferred. transformer, the first layer represents the input and the last layer represents the output, but wish know: what represented in the middle? We analyze the internal mechanism
4

---

Published as a conference paper at ICLR 2026
using Causal Abstraction (Geiger et al., 2021; 2024); First, we hypothesize a high-level causal model of the computational steps from input to output (Sec. 4), and then align its variables with the LM's internal activations (Sec. 5). We test the alignment through targeted interchange interventions on causal variables in the hypothesized model and hidden activations in the L.M. If the LM produces the same output the causal model under these aligned interventions, it provides evidence supporting the hypothesized causal model. We quantify this effect using interchange intervention accuracy (IIA; Geiger et al., 2022), which measures the proportion of cases where the intervened causal model and intervened LM agree. See Appendix F for more details.
In addition to measuring IIA on entire residual stream vectors. we also intervene on localized subspaces to further isolate causal variables. To identify the subspace of a specific variable, we employ Desiderata-buse
Component Masking (De Cao al., 2020; Davies et al., 2023; Prakash et al., 2024). This method learns a sparse binary mask over the activation space that maximizes the logit of the hypothesized causal model output. We train a mask to select singular vectors of the activation space that encode high-level variable (see Appendix H for details). Our experiments in Sec. report both interventions on the full residual stream and on the identified subspaces.
HYPOTHESIZED HIGH-LEVEL CAUSAL MODEL OF BELIEF TRACKING
Here start with an overview of our hypothesized causal model of belief trackinn when characters are not aware of each other's actions. The causal model is an algorithmic process that has variables with structural roles that do not refer the details of transformer architecture. Appendix G presents the full pseudocode of the causal model. In Section 5, we will present experiments to verify that the causal model's variables align with representations in the transformer.
Belief tracking begins when the causal model assigns ordering IDs (OIs; O. object, and state token, marking their order of appearance, For instance, in the example in Fig. 3, Bob is assigned first character OI ), and Carla is assigned the second character OI( @ Please refer to Appendix C for more information about OI. Then uses these Ols in two lookback mechanisms:
to each character,
(i) Binding lookback. The causal model creates address copies of each character OI (0) and object Ol (O) that are bound to the state OI (Binding Payload, A), creating a character-object-stae triple. When question is asked about character and object, the causal model creates pointer copies of that character and object OIs (Binding Pointers
and dereferences them to retrieve the state OI.
(ii) A Answer lookback. The causal model creates an address copy of the state O1 (Answer Address O) that is bound to the state token (Answer Payload, A). Through the binding lookback, a pointer copy of this OI (O) is created. The causal model dereferences the pointer to retrieves the correct state token payload as the final output.
VERIFYING THE HYPOTHESIZED CAUSAL MODEL OF BELIEF TRACKING
We test our hypothesized causal model by localizing its variables within the transformer's neural representations. Specifically, we localize the addresses, pointers, and payloads of the (i) binding lookback and (ii) answer lookbacks within the LM's internal activations. In Fig. 3, we show trace of the causal model run on an input overlayed onto a schematic of a transformer architecture. This visualizes the alignment between variables in the causal model and locations in the LM residual stream that the experiments in the remaining of this paper will support. In the binding lookback, the character and object Ol addresses are realized in the residual stream of the state token. The pointer copies are brought forward to the last token residual stream where they are dereferenced via attention to bring forward the correct state Ol payload. In the answer lookback, the address copy of the state Ol is in the state token residual stream while the pointer copy is in the last token residual stream.
Each of the following experiments localizes the presence of specific ordering IDs (OIs) and verifies their roles as hypothesized by our causal model. We do this by targeted interchange intervention experiments on the causal model and the LM. We copy hidden states between identical tokens (for example, replacing the representation of " in context with the representation of ";" in another context, as in Fig. 4). When this intervention causes the LM's have the same output as the causal model under interchange intervention on OI variables, we have evidence that the Ol is carrying
5

---

Published as a conference paper at ICLR 2026
Intervention Input Example
Layer-wise Intervention Results
Connettatal
Casla and Bob are working in busy restaurant. To complete ar: ordez, Carla grabe an opeque cup fille with Then Bob grabs another opaque bottle and fills it with water.
Intervention Accuracy
Full residual
0.8
Subspace
dimension ss
Question: What does Carla beliave cup consaina? Answex: tea
0.6
Subspace
dimension si
Bob Carla working in busy
complet
ar ordes, graba an opaque bottle and fills it with bear.
Than carla grabs another opaque cup and fills it with coffee.
0.2
quest Mhaz does Carla believe the cup consains? Answer: coffee
0.0
0
10
20
30
40 Layers
50
60
70
Interveniion
Pointer (0), Cousal Model Oulpul:
Intervatiin 2: Anewer Psyloud (A), Consal Model Output: tea
Figure 4: Answer Lookback Pointer and Payload: The causal model predicts that we alter the "Answer Payload A of the original to instead take the value of the counterfactual answer payload, the output should change from coffee to tea; the gray curve in the line plot shows this does occur when patching residual vectors at the "." token beyond layer 56, providing evidence that the answer payloat resides in those states, On the other hand the causal model predicts that taking the counterfactual "Answer Pointer O" would change the original run output from coffee to beer-a new output that matches neither the original nor the counterfactual!-ann patching layers between 34 and 52, providing strong evidence that the answer pointer is encoded at those layers. These results suggest the Answer Lookback occurs between layers 52 and 56.
we see this surprising effect, again when
the hypothesized role. Each experiment reports the effects of 80 different cases with the same structure, and the effect is measured at every layer.
Because the last step the causal model easiest understand, we proceed through the experiments in reverse order, beginning with an experiment to verify the final "answer lookback" stage. After this instructive starting point, we work backward to verify the earlier steps of the model. Additional results can be found in Appendix and J.
5.1 STEP ANSWER LOOKBACK RETRIEVING THE CORRECT STATE
Localizing the Answer Payload We first verify the presence of the correct Answer Payload A at the deepest layer representation of final token ":". To do so, we run an interchange intervention experiment shown in Fig. 4a which the counterfactual example e swaps the order of the characters and objects of the original example and also replaces the state (drinks) tokens with new values. If the Answer Payload is correctly localized, swapping it should cause the answer of the counterfactual (e.g.. tea) to replace the answer of the original example (e.g., coffee). The gray line in Fig. 4b shows that this output change is observed in every one of n = 80 cases, both when intervening on the full residual stream and on the identified subspace. However, not at every layer: the information is only present after layer 56, indicating that before this stage, the transformer has not yet retrieved the correct answer payload into the residual stream. That is consistent with our hypothesis that at early steps, the OI has not yet been dereferenced. At an earlier stage, we expect to see an Answer Pointer.
Localizing the Pointer Information To identify the Answer Pointer O before it is dereferenced to bring the payload (state token value), we examine the representations of ":"" at layers earlier than 56. Our causal model provides the hypothesis: if the Answer Pointer is present, then patching the pointer from the counterfactual run into the original run should redirect the LM to attend to the location of the correct counterfactual state and fetch its payload. For example, in Fig. 4a the counterfactual pointer references the first presented state. When we patch it into the original story, we expect the model's answer to change to beer rather than coffee. The colored line in Fig. 4b confirms that this effect is consistently observed when patching any layer between 34- - 52 (both when patching the full residual stream and the identified subspace), supporting our hypothesis that these layers encode the Answer Pointer information at the final token, rather than directly transferring token values.
5.2 STEP I: BINDING LOOKBACK - LINKING CHARACTERS. OBJECTS, AND STATES
Localizing the Address and Payload In this experiment, we verify the presence of the address copies of the character and object Ols as well as the payload (state OI) at the state token residual stream (recalled token, Fig. 3). As illustrated in Fig. 5a, we construct an intervention dataset where

---

Published as a conference paper at ICLR 2026
Intervention Input Example
Layer-wise Intervention Results
Oiignaal Conntrlettl
Carla and Bob working in restaurent.
Then Bob grsiss another opaque bottle and fills it with beer. question: What dees Carla believe the cup contains? Answer: coffee
an order, Carla grabs an opaque cup and fills it with coffee.
complete
Intereentio Accuracy
0.8
Rill residual
dimensi
0.6
Bob and Carla ace working in a busy restauzant, To compet order, Bob grabs opaque bottle fills it with bees
Then Carla grabs anothes opaque cup and fills question: Nhat dees Carla believe the cup containa? Answers coffee
coffee.
0.2
0.0
10
20
30
40
50
60
70
Intervention: Hinding Paylonds and Addresses
Layers
Causal Model Ouiput: beaz
Figure Binding lookback Address and Payload: The causal model predicts that swapping addresses (character and object Ols; O and O) and payloads (state Ols; A) should cause the binding lookback mechanism to attend to the alternate state token and retrieve its state OI. This retrieved state is then dereferenced by the answer lookback, producing the corresponding token as the output (e.g.. beer instead of coffee). The LM's behavior matches this prediction when we perform interchange interventions the state token across layers 33 33-3.. These findings support our hypothesis that both address and payload information are encoded in the residual stream state tokens.
(a) Intervention Input Example
Layer-wise Intervention Results
Ortginal Ounterbett
Carla and Bob are working in busy restaurant. To complete an order, Carla graba an opaqua cup and filla with tea.
Intervention Accurncy
1.0
Then Bob grabs another opaque bottle and filla it with water. questione shat dees Carla beliave the cup contains? Answer:
0.8
Subspace
dimension 167
Bob and carta are working in a busy restaurant. To conglate an orderg Báb grabs an opaque Bottlie and rills it wis vismpoer. Seer.
0.4
Then Carla grabs
epaque cup fills with coffee.
0.2
Queation: What doea Carla believe the cup containa?
Answer: coffee
0.0
10
20
30
40
50
60
70
Intervention: Binding Source
Layers
Frozom:
Rinding Answer Paylkouds A) Causal Model Oulput: beer
Figure 6: Source Reference Information of Binding lookback: The causal model predicts that swapping the source reference information (character and object Ols: O O), while freezing the addresses and payloads of the binding lookback, should cause the binding lookback mechanism to attend to the alternate state token and retrieve its state OI, which would generate alternate state token as the final output via the answer lookback (e-g., beer instead of coffee). The LM's behavior matches this prediction when we perform interchange interventions at the character and object tokens across layers 20-34. These results support our hypothesis that source reference information is encoded in the residual stream of character and object tokens.
each example consists of an original input with an answer that is not unknown and a counterfactual input с where the character, object, and state token values are identical, except the ordering of the two story sentences is swapped while the question remains unchanged. The expected LM's output predicted by our hypothesized causal model the other state token in the original example, e.g., beer. That is because patching the address and payload values at each state token, without changing the pointer, makes the LM dereference the other state token. As result, the model's output should flip to the other state token in the original input.
We perform the interchange intervention experiment layer-by-layer, where we replace the residual stream vector (or the identified subspace) of the first state token in the original run with that of the second state token in the counterfactual run and vice versa for the other state token. It is important to note that the intervention targets state token values instead of their Ols, it should not produce the expected output. (This happens in the earlier layers.) As shown in Fig. 5b, the strongest alignment occurs between layers 33 and 38, supporting our hypothesis that the state token's residual stream contains both the address (character and object Ols) and the payload information (state OI).
Localizing the Source Reference Information Next, localize the source reference information, ie., character and object Ols at their respective token residual stream. As illustrated in Fig. 6a, conduct intervention experiment with dataset where the counterfactual example, e, swaps
7

---

Published as a conference paper at ICLR 2026
Layers
Answer
40
(i) Binding lookback
(iii) Visibility lookback
Observed Info
Parsed Question
30
Character2 Ol
State2 OI
Object2
20
Visibility ID
Bob
graaba
an
Then
Carla
grabs
another
opaque
cup
and
fills
it with
coffee
Bob
can
dbserve
Carla
acLions
Queation
does
Bob
believe
the
cup
containo
Answer
Figure 7: Visibility Lookback When one (observing) character can see another (observed) character,
the LM assigns visibility ID (O) the visibility sentence where this relation is defined. An address copy of this visibility ID remains in the visibility sentence's 's residual stream. A pointer copy of the visibility ID is transferred to the subsequent tokens' residual stream. The LM dereferences this pointer through QK-cireuit, bringing forward the payload ), when processing subsequent tokens. Based on initial evidence, this payload contains the observed character's OI(@). See Appendix K for details. This mechanism allows the model to incorporate the observed character's 's knowledge into the observing character's belief state, enabling more complex belief reasoning.
the order of the characters and objects as well as replaces the state tokens with entirely new ones while keeping the question the same in Under this setup, an interchange intervention on the hypothesized causal model that targets the source reference should propagate changes through both the address and the pointer, leaving the final output unchanged. However, if we instead freeze the state token residual stream, which carries both the payload and the address, the causal model produces the alternate state token (e.g., beer in Fig. 6), as the pointer refers to the other state's address.
In the LM, interchange the residual streams of the character and object tokens ttyer-by-layer while keeping the residual stream the state token fixed. As shown in Fig. 6b, this experiment reveals alignment between layers 20 and 34, indicating that source reference is encoded in the residual streams of the character and object tokens within this layer range. Additional results are provided in Appendix where Fig. 13 shows that freezing the residual stream of the state token is necessary for this alignment to emerge, These findings support our hypothesis that source reference is present in the character and object tokens and subsequently transferred to the recalled and lookback tokens.
Localizing the Pointer Information Finally, we localize the pointer copies of the character and object Ols to their corresponding tokens in the question and to the final token. See Appendices I & J for details of the experiments and results.
In summary, belief tracking begins in layers 20-34, where character and object OIs encoded in their respective token representations. These OIs are transferred the corresponding state tokens in layers 33-38. When question is asked, pointer copies of the relevant character and object Ols are moved to the final token by layer 34, where they are dereferenced to retrieve the correct state OL At the final token, this state Ol is represented across layers 34-52, and between layers 52-56, it is dereferenced to fetch the answer from the correct state token, producing the final output.
IMPACT OF VISIBILITY CONDITIONS ON BELIEF TRACKING MECHANISM
So far, have demonstrated how the LM uses ordering IDs and two lookback mechanisms to track the beliefs of characters that cannot observe each other. Now, we explore how the LM updates the beliefs of characters when one character (observing) observe the actions of the other (observed).
8

---

Published as a conference paper at ICLR 2026
Intervention Inpot Example
Layer-wise Intervention Results
Cazla end Bob are working in busy restaurant
corplete en
Interveation Accurccy
1.0-
order, Carla grabs on epaque cup and tills it with tea. Thor Bob bettle and filla it
condriin
grabs anothez
0.8
Unees Junss vilal Bows Carla Luve 1iw botfle contetry
observe Carla's actiona. Carla can obsarve Bobra actiona.
0.6
Vie by Q on
watez
0.4
complete ozdaR, Karen axabs en epaque flute and. filla it soda. Thes
and
working
buey restairant.
0.2-
Max grebi andi liz peeque jas and fills it with coffee. Mas call
observe Raren's accions. Xaren cannot obcorve Max's actions, Question: What odes Karen be leve tne jar contaime?
0.0
0
10
20
30
40
50
60
70
(aPzzi unxtows
Layers
Intervention I: Source (0), Cansal Model Ouguu: coffee
Intervention Payload d 1. Canval Model Ouppt: coffee
Interveation Addiess Pointer (0), Cansol Model Ouiqut: coffee
Figure 8: Visibility Lookback: We conduct three interchange intervention experiments to support the Visibility Lookback hypothesis: (I) Source Alignment: We align the source reference (0) by intervening on the visibility sentence, replacing it with its representation from a counterfactual run where the visibility sentence causes the queried character become aware of the queried object's contents. We observe that source reference information aligns between layers 10 and 23, after which it splits into separate address and pointer components. (2) Payload Aligument: To align the payload (
). we intervene on all subsequent tokens and observe alignment only after layer 31. (3) Address and Pointer Alignment: When intervening on both the address and pointer information (O), we observe alignment across broader range of layers, particularly between layers 24 and 31, because of the enhanced alignment between the address and pointer copies at the recalled and lookback tokens.
Hypothesized Visibility Lookback Mechanism We hypothesize that the LM uses an additional lookback mechanism, which we call the Visibility Lookback, to integrate information about the observed character when it is explicitly stated that one character can see another's action. As illustrated in Fig. 7, hypothesize that the LM first generates Visibility ID (O) at the residual stream of the visibility sentence, serving as the source reference information. The address copy of the visibility ID remains in the residual stream of the visibility sentence, while its pointer copy gets transferred to the residual stream of the subsequent tokens (lookback tokens). Then LM forms a QK-cireuit the lookback tokens and dereferences the visibility ID pointer to retrieve the payload.
Although our two-character setting is unable to discern the exact semantics of the payload in the visibility lookback, our observations are consistent with the payload encoding the observed character's OI. Our initial observations suggest another lookback where the story sentence associated with the observed character serves as the source reference, and its payload encodes information about the observed character. The observed character's Ol appears to be retrieved by the lookback tokens of the Visibility lookback, with causal effects on the queried character's awareness (see App. for details).
6.1 VERIFYING THE HYPOTHESIZED VISIBILITY LOOKBACK
Localizing the Source Reference In this experiment, we localize the Visibility ID (O), i.c., the source reference of the Visibility lookback. We conduct an interchange intervention experiment where the counterfactual is different story in which the characters' visibility flipped from unobserved to observed (Fig. 8a), and we look for an output change from "unknown" to the answer that would be observed. We intervene on the representation of all the visibility sentence tokens. As shown in Fig. 8b (blue - line), causal effects appear between layers 10 and 23, indicating that the visibility ID remains encoded in the visibility sentence until layer 23, after which it is split into address and pointer copies that must be connected by dereference to have effect. This pattern supports hypothesis that the LM generates a reference to the Visibility ID.
information using the same counter- factual dataset. However, instead intervening on the recalled tokens, we intervene on the lookback tokens, specifically the question and answer tokens. As in the previous experiment, we replace the residual vectors of these tokens in the original run with those from the counterfactual run. As shown in Fig. 8b (red - line), alignment occurs after layer 31, indicating that the information causing the queried character's awareness is present in the lookback tokens after this layer.
Localizing the Payload Next, we localize the payload
Localizing Address and Pointer The previous two experiments indicate the absence of both the source and payload information between layers 24 and 31. We hypothesize that this lack of signal is

---

Published as a conference paper at ICLR 2026
due to a mismatch between the address and pointer information that inhibits a lookback dereference. Specifically, when intervening only on the recalled token after layer 25, the pointer is not updated, whereas intervening only the lookback tokens leaves the address unaltered, a mismatch in either case. To test this hypothesis, we conduct another intervention using the same counterfactual dataset, but this time, we intervene on the residual vectors of both the recalled and lookback tokens, i.e., the visibility sentence, as well as the question and answer tokens. As shown in Fig. 8b (green - line), alignment occurs after layer 10 and remains stable, providing evidence that lookback now occurs between layers 24 and 31. This intervention replaces both the address and pointer copies of the visibility IDs, enabling the LM form QK-eircuit and resolve the visibility lookback.
RELATED WORK
Theory of mind in LMs
Theory of mind in LMs has been widely benchmarked (Le al., 2019;
Shapira et al., 2023; Wu et al., 2023; Kim et al., 2023; Xu et al., 2024; Jin et al., 2024; Chan et al., 2024; Strachan al., 2024b). However, these benchmarks lack adequate counterfactuals for the binding manipulations we need, so we made CausalToM (Section B). Although several methods have been proposed improve ToM performance in LMs (e.g. Sclar al., 2023; Moghaddam & Honey, 2023; Zhou et al., 2023; Wilf et al., 2024; Hou et al., 2024), our goal is not to enhance ToM abilities. Instead, we aim to understand the existing ToM behaviors of LMs by reverse-engineering the underlying mechanisms that support belief tracking.
Entity tracking in LMs Entity tracking and variable binding essential for LMs to support not only coherent ToM reasoning but also broader neurosymbolic inference. A substantial body of work has sought uncover how LMs implement these abilities. Li al. (2021) demonstrated that LMs form internal representations that encode the dynamic states of entities, which can be identified and manipulated using linear probes (Belinkov, 2022). Building on this, Davies al. (2023), Prakash et al. (2024) and Yang et al. (2025) found that modern LMs rely on a small set of attention heads that track an entity's 's positional information to retrieve its corresponding attributes. Complementing these findings, Feng & Steinhardt (2023) showed that LMs encode similar relational information in the form of Binding IDs within their hidden representations. Extending this line of work, Dai et al. (2024) demonstrated that LMs also learn more symbolic representation, referred to Ordering IDs, that capture the positional information in low rank subspace of a token's 's residual stream. Wu et al. (2025) trained Transformer on symbolic programs and showed that variable binding emerges over three distinct training phases, progressing from shallow heuristics to a fully systematic mechanism. Our work builds this growing literature by uncovering the end-to-end mechanism that enables variable binding and by further elucidating how ToM--elated reasoning is implemented through these binding processes within LM internal representations.
Mechanistic interpretability theory of mind Recently, a few studies have begun probing the internal representations underlying ToM abilities in LMs. Zhu et al. (2024) showed that the belief states of different agents can be linearly decoded using simple probes, and that intervening on these representations leads to substantial changes in ToM performance. Building on this, Herrmann & Levinstein (2024) examined how prompt variation and fine-tuning affect the stability and structure of belief representations. Additionally, Bortoletto et al. (2024) proposed adequacy criteria, such as accuracy, coherence, uniformity, and use, for determining when an internal representation in an LLM should count as belief-like. While these works provide valuable insight into how belief information is encoded, they do not illuminate the mechanisms by which LMs actually solve ToM tasks, limiting our ability to understand, predict, and control model behavior.
CONCLUSION
Through a series of interchange intervention experiments, we have mapped the end-to-end underlying mechanism responsible for the processing of partial knowledge and false beliefs in a set of simple stories. We are surprised by the pervasive appearance of a single recurring computational pattern: the lookback, which resembles a pointer dereference inside a transformer. The LMs use combination of several lookbacks to reason about nontrivial belief states. Our improved understanding of these fundamental computations gives us optimism that it may be possible to fully reveal the algorithms underlying not only the theory of mind, but also other capabilities in LMs.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We recognize that ToM reasoning can be misused for deception, persuasion, targeted advertising. For this reason, our work deliberately avoids amplifying such capabilities and instead focuses solely on measuring and characterizing the ToM abilities that models already possess. Our goal is not to accelerate a model's capacity for ToM reasoning, but to understand the underlying mechanisms that would allow to detect ToM-related behavior by directly examining its neural fingerprint.
We believe that this kind of internal neural analysis essential, particularly when there are concerns that models might learn deceive persuade. Because deception and persuasion are, by definition, difficult or impossible to identify from model's outputs alone, it is crucial to understand their neural signatures. Doing so enables to distinguish between surface-level outputs that merely appear to reflect certain behaviors and internal reasoning pathways that genuinely reveal ToM processes.
Finally, acknowledge that research ToM LMs be misinterpreted implying human-like cognition or intentionality. We explicitly caution that our findings describe internal computational mechanisms, conscious reasoning, and should not be taken evidence of sentience moral agency in LMs.
10 REPRODUCIBILITY STATEMENT
To facilitate reproducibility, we release the full CausalToM dataset, including all story tem- plates and the code used to generate the various story instances that serve as counterfactual variants in our experiments. The repository, which can be accessed at https: / baulab. info, contains all scripts required to construct the dataset, extract activations, per- form interchange interventions, and compute causal mediation metrics. along with the hy- perparameters and random seeds used for subspace identification via DCM. All experiments were conducted using publicly available open-weight models (m- -ateee----0-I-ns-tttee 3 1-4058--Instrucee and Qwen2. 1iree2..5144--1 --144--Istttt hosted on Huggingface. Ex- periments with L1 ama 3-70--7000--00 Inst and Qwen2. 1e114--Instruee were carried lo- cally on two 80GB NVIDIA A100 GPUs, while experiments with Ll Llama 3. 1---11---BBB- Seee-Instrrunn ruct were conducted remotely using the NDIF 1llmm-3--70-Inn- ruct and Qwen2. 1iten2.5.. 5-1 - can also be reproduced using NDIF sufficient local computing resources are unavailable.
et al., 2025) service. Results for
11 THE USE OF LARGE LANGUAGE MODELS
We used L.MSs as writing assistant to correct grammatical and typographical errors; beyond this, they did not contribute to any stage of the research.
12 ACKNOWLEDGEMENT
This research was supported in part by Open Philanthropy (N.P., N.S., A.S.S., D.B., A.G., Y.B.), the NSF National Deep Inference Fabric award #2408455 D. the Israel Council for Higher Education (N.S.), the Zuckerman STEM Leadership Program (T.R.S.), the Israel Science Foundation (grant No. 448/20; Y.B.). an Azrieli Foundation Early Career Faculty Fellowship (Y.B.), a Google Academic Gift (Y.B.), and a Google Gemma Academic Award (D.B.). This research was partly funded by the European Union (ERC, Control-LM. 101 165402.. Views and opinions expressed are however those of the author(s) only and do necessarily reflect those the European Union the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.
11

---

Published as a conference paper at ICLR 2026
REFERENCES
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 2023.
Simon Baron-Cohen, Alan M Leslie, and Uta Frith. Does the autistic child have "theory of mind"? Cognition, 21(1):37-46 1985.
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):007-219, 2022.
Matteo Bortoletto, Constantin Ruhdorfer, Lei Shi, and Andreas Bulling. Benchmarking mental state representations in language models. arXiv preprint M.KY::2406.155133 2024.
Chunkit Chan, Cheng Jiayang, Yauwai Yim, Zheye Deng, Wei Fan, Haoran Li, Xin Liu, Hongming Zhang, Weiqi Wang, and Yangqiu Song. Negotiationtom: A benchmark for stress-testine tessinn machine theory of mind on negotiation surrounding. arXiv preprini arXiv:2404. 12644.1332.... 2024.
Qin Dai, Benjamin Heinzerling, and Kentaro Inui. Representational analysis of binding in language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.). Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 17468-17499 1793,, Miami, Florida, USA. November 2024. Association for Computational Linguistics. doi: 10.18653/v1/ 2024.emmlp-ma www.emmmmpmminn66111 967.
URL https::///// http://:///aaaaahoooogsss www./oog..oooooooooooo..oo 2024. emnlp-mmin. m
Xander Davies, Max Nadeau, Nikhil Prakash, Tamar Rott Shaham, and David Bau. Discovering variable binding circuitry with desiderata, 2023. URL https: pttpp:://arivvi http//aaxii....aaaaa... wwt./oog/////ooooooooo.ooo... hat///bb//23000 /22.1///37.///// 23007. 03637.
Nicola De Cao, Michael Sejr Schlichtkrull, Wilker Aziz, and Ivan Titov. How do decisions emerge across layers in neural models? interpretation with differentiable masking. In Proceedings of the 2020 Conference 3255, Online, November 2020. Association for Computational Linguistics. doi: 10.11653/v1/2022 emnlp-main.22 262. URL htwps:ttpstccmm http::::////claahooolll....//// org/ 2020 emnlp cannp-mmiing main.2. 262.
Empirical Methods in Natural Language Processing (EMNLP), pp. 3243-
Daniel Clement Dennett. The Intentional Stance. MIT Press, 1981.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-oodss, Danny Hemandez,, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. hutps:/transformee-ccrcciii pubpll21 /fframewrrk/nnex.....e html.
Jiahai Feng and Jacob Steinhardt. How do language models bind entities in context? arXiv preprint arXiv:2310.11 171911, 2023.
Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann, Stuart M. Shieber, Tal Linzen, and Yonatan Belinkov. Causal analysis of syntactic agreement mechanisms in neural language models. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACI/LICNLPP 2021, (Volume I: Long Pupers), Vir- tual Event, August 1-6, 2021, pp. 1828-1843. Association for Computational Linguistics, 2021. doi: sol-long: 144.
144, URL https: / 105310.111611122222.
Jaden Fried Frotto-Kaufmmn. Alexander Russell Loftus, Eric Todd, Jannik Brinkmann, Koyena Pal, Dmitrii Troitskii, Michael Ripa, Adam Belfki, Can Rager, Caden Juang, Aaron Mueller, Samuel Marks, Arnab Sen Sharma, Francesca Lucchetti, Nikhil Prakash, Carla E. Brodley, Arjun Guha, Jonathan Bell, Byron C Wallace, and David Bau. NNsight and NDIF: Democratizing access to open-weight foundation model internals. In The Thirteenth International Conference on Learning Representations, 2025. URL https: http:///ppeeevv/eeeeeeee net forum 1
12

---

Published as a conference paper at ICLR 2026
Kanishk Gandhi, Jan- Jan-Philipp Fränken, Tobias Gerstenberg, and Noah Goodman. Understanding social reasoning in language models with language models. Advances in Neural Information Processing Systems, 36, 2024.
Atticus Geiger, Kyle Richardson, and Christopher Potts. Neural natural language inference models partially embed theories of lexical entailment and negation. In Afra Alishahi, Yonatan Belinkov, Grzegorz Chrupata, Dieuwke Hupkes. Yuval Pinter, and Hassan Sajjad (eds.), Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP. pp. 163- 173, Online, November 2020. Association for Computational Linguistics. doi: 10. 10.18833/11/2200. URL https: / a anthol ogy org/ 2020. bl blw.kcooocllpppccc.cc p- 16.
blackboxnlp-1.. 1.16.
Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34; Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 9574-9586, 2021. URL ott.eaae777..221110--btttttttttt html. 2021 htt4///ee22222555000000722222223
neurips ccc copppper///////
Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah Good- man, and Christopher Potts. Inducing causal structure for interpretable neural networks. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 7324-7338. PMLR, 17-23 Jul 2022. URL https: http::://proceedings mlm.rrr.... www.mmr...rssse.mmmmmmmmmmmmme.e /v1 /eiiee//ggggeeaaaa.. /tt.a//pepee///////gggggge22 html.
Atticus Geiger, Duligur lbeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and Thomas Icard. Causal abstraction: theoretical foundation for mechanistic interpretability, 2024. URL https:// arxiv. org/ abs
Mor Geva Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall of factual associations in auto-regressive language models, 2023. URL attpps://rrxiaa orgoorgggaoo abs/
2304.14761
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong. Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary. Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lec, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, mmanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeaa Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic. Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishman, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic,
Plawiak,
13

---

Published as a conference paper at ICLR 2026
Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vitor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitey Meers, Xavier Martinet, Xiaodong Wang. Xiaofang Wang. Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, 7oe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape. Bing Liu, Bo Wu, Boyu Ni. Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Shhing-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang. Duc Le. Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni. Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpem, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj. Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean- nan--Batistee Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang. Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang. Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A. Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manay Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari. Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar. Ozlem Kalinli. Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji. Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj. Qian Liang. Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng. Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyighe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kochler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru,
14

---

Published as a conference paper at ICLR 2026
Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang. Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang. Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen. Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang. Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary Vito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The Ilama 3 herd of models, 2024. URL https: wwg.iiii..gg//.... 2407 21783.
Hyowon Gweon, Judith Fan, and Been Kim. Socially intelligent machines that learn from humans and help humans learn. Philosophical Transactions of the Royal Society A, 381(2251):2020048, 2023.
Daniel A Herrmann and Benjamin A Levinstein. Standards for belief representations in llms. arXiv preprint arXiv:2405.2 20024405220000 2024.
Guiyang Hou, Wenqi Zhang, Yongliang Shen, Linjuan Wu, and Weiming Lu. TimeToM: Temporal space is the key to unlocking the door of large language models' theory-of-me mind. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Findings of the Association for Computational Linguistics ACL 2024, pp. 11532-11547, Bangkok, Thailand and virtual meeting, August 2024. Association for Computational Linguistics. URL https: / /aacaattoollooooooooo ogy. orrg/2244 Findings-acl.. 685.
Jennifer Hu, Felix Sosa, and Tomer Ullman. Re-evaluating theory of mind evaluation in large language models. arXiv preprint arXiv:2502.21098. 2025.
Chuanyang Jin, Yutong Wu, Jing Cao, Jiannan Xiang, Yen-Ling Kuo, Zhiting Hu, Tomer Ullman, Antonio Torralba, Joshua Tenenbaum, and Tianmin Shu. MMToM-QA. Multimodal theory of mind question answering. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 16077-16102, Bangkok, Thailand, August 2024. Association for Computational Linguistics. URL https: / /aaaaaaaantttoooooo
org/ 2022002222 acl- -ongg. 851.
Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. Fantom: A benchmark for stress-testing machine theory of mind in interactions. arXiv preprint arXiv:23100 130.5:231,, 2023.
Youjeong Kim and S Shyam Sundar. Anthropomorphism of computers: Is it mindful or mindless? Computers in Human Behavior, 28(1):2411255) 2012.
Michal Kosinski. Evaluating large language models in theory of mind tasks. Proceedings of the National Academy of Sciences, 121 121(45), October 2024. ISSN 1091-6490. doi: 10.1073/pnas. 2405460121. URL http: doi wwwoorrg///.. 10t.1003//ppaa 2405460121.
Matthew Le, Y-Lan Boureau, and Maximilian Nickel. Revisiting the evaluation of theory of mind through question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP),, pp. 1872-5877, 2019.
Belinda 7. Li, Maxwell Nye, and Jacob Andreas. Implicit representations of meaning in neural language models. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pp. 1813 1827, Online, August 2021, Association for Computational Linguistics. doi: 18.58v53/v1/ 2021.acl-lomp 143, URL ht aataaaaaaannooooooooaaaaaa
2ttp:/2222..cc1noneeea 143.
Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla, 2023. URL https:/
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 36, 2022.
Shima Rahimi Moghaddam and Christopher J Honey. Boosting theory-of ofauy-of-minn mind performance in large language models via prompting. arXiv preprint arXiv:2304. 1490, 2023.
15

---

Published as a conference paper at ICLR 2026
Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal. Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Armb Sen Sharma, Jiuding Sun, Eric Todd, David Bau, and Yonatan Belinkov. The quest for the right mediator: A history, survey, and theoretical grounding of causal interpretability, 2024. URL https: / arxiv org/abs absg///b///// 2408 01 01401.66 4.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez. Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse. Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. In-context learning and induction heads. Transformer Circuits Thread, 2022. https:/ttaassfrree-ccccuiii
Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. Fine-tuning enhances existing mechanisms: A case study on entity tracking. In Proceedings of the 2024 International Conference on Learning Representations, 2024. arXiv:2402.145: 14811.
David Premack and Guy Woodruff. Does the chimpanzee have theory of mind? Behavioral and Brain Sciences, (4)):115520) 1978. doi:
Christoph Riedl, Young Ji Kim, Pranav Gupta, Thomas W Malone, and Anita Williams Woolley. Quantifying collective intelligence human groups. Proceedings of the National Academy of Sciences, (18(21):22057737 18, 2021.
Naomi Saphra and Sarah Wiegreffe. Mechanistic? In Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mueller, and Hanjie Chen (eds.), Proceedings of the 7th BlackboxNI.P Workshop: Analyzing and Interpreting Neural Networks for NLP, pp. 480-498, Miami, Florida, US, November 2024. Association for Computational Linguistics. doi: 10. 00.18653/v1/20222 blacco/ooob.
30. URL https: / httptt:p:///caaaattttn /wt.aaaaannnnooooooooooooaoooola org/ 2024 2t00244bbacbbboopp11...333
Mclanie Sclar, Sachin Kumar, Peter West, Alane Suhr, Yejin Choi, and Yulia Tsvetkov. Minding language models' (lack of) theory of mind: A suug-nnd-play multi-character belief tracker. arXiv preprint ar arXivv2306.0099
2023.
Melanie Sclar, Jane Yu, Maryam Fazel-Zarandi, Yulia Tsvetkov, Yonatan Bisk, Yejin Choi, and Asli Celikyilmaz. Explore theory of mind: toggram-guide adversarial data generation for theory of mind reasoning. ICLR, 2025.
Natalie Shapira, Guy Zwim, and Yoav Goldberg. How well do large language models perform faux pas tests? In Findings of the Association for Computational Linguisties: ACL 2023, pp. 10451, Toronto, Canada, July 2023. Association for Computational Linguistics. URL. https: / ////caaannn aclanthologgy org/ 2023. Lindinggs-cc.. 663.
Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. Clever hans or neural theory of mind? stress testing social reasoning in large language models. In Yette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume Long Papers). pp. 2257-2273. St. Julian's s, Malta, March 2024. Association for Computational Linguisties. URL. https:: /ttp:pp:///caaatttiee //t//aaanhhelccoooooo
org/ 2.22 eacl- long. 138.
Paul Smolensky. Neural and conceptual interpretation of PDP models. In James L. McClelland, David F. Rumelhart, and the PDP Research Group (eds.), Parallel Distributed Processing: Explo- rations in the Microstructure of Cognition: Psychological and Biological Models, volume 2, pp- 390-431 MIT Press, 1986.
James W. A. Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, Michael S. A. Graziano, and Cristina Becchio. Testing theory of mind in large language models and humans. Nature Human Behaviour, 17): Jul 2024a. ISSN 2397-3374, doi: 10. 1005//41562-024-118555 URL hwt.t:tpp//......... //tp::::/ddoo /ww.///o.i///// www.ooi.orgo110 1088/ //B///415650 024-0 01 882-2.
James WA Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, et al. Testing theory of mind in large language models and humans. Nature Human Behaviour, 8(7): 1285: 188--1255.. 2024b.
16

---

Published as a conference paper at ICLR 2026
Winnie Street, John Oliver Siy, Geoff Keeling, Adrien Baranes, Benjamin Barnett, Michael McKibben, Tatenda Kanyere, Alison Lentz, Blaise Aguera Arcas, and Robin I. M. Dunbar. Lims achieve adult human performance higher-order theory of mind tasks, 2024. URL tattps://arxtaa org/abs/ 2405. 18870.
Tomer Ullman. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08339, 2023.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 12388-12400 Curran Asso- ciates, Inc., 2020. URL https: httpppppeeppppoceeinnnneeeiiiiii neurips........irs.......... cc/pper files/ /apee// 2020/f 1e/ 22510
Alex Wilf, Sihyun Lee, Paul Pu Liang, and Louis-Philippe Morency. Think twice: Perspective- taking improves large language models' theory-of-man mind capabilities. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 8292-8308, Bangkok, Thailand, August 2024. Association for Computational Linguisties. URL https: hittps://aclantheee org/2024. a acl- llogg 451.
Heinz Wimmer and Josef Perner. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children's 's understanding of deception. Cognition, 13(1):103-128, 1983.
Yiwei Wu, Atticus Geiger, and Raphaël Millière. How do transformers learn variable binding in symbolic programs? arXiv preprint arXiv:2505.20896 2025.
Yufan Wu, Yinghui He, Yilin Jia, Rada Mihalcea, Yulong Chen, and Naihao Deng. Hi-ToM: A benchmark for evaluating higher-order theory of mind reasoning in large language mod- els. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10691-10706, Singapore, December 2023. As- sociation for Computational Linguistics. doi: 10.18653/1/2023. 1/20..333//.2....nnnnng findinng--emnlp.11 emnlp.717. URL https: /toos//aacaan http://aaaaannnncc..aaa..c wttp:////..........g.oooo....... wtw.:r..iiiiinnnemmne... 717.
Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, and Yulan He. OpenToM: comprehensive benchmark for evaluating theory-of-mind reasoning capabilities of large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8593-8623, Bangkok, Thailand, August 2024. Association for Computational Linguistics. URL https: // l/aclanthoooge org/ 2024.0024.. acl-acllnnna 466.
Yukang Yang, Declan Campbell, Kaixuan Huang, Mengdi Wang, Jonathan Cohen, and Taylor Webb. Emergent symbolic mechanisms support abstract reasoning in large language models. arXiv preprint arXiv:2502.20332 2025.
Pei Zhou, Aman Madaan, Srividya Pranavi Potharaju, Aditya Gupta, Kevin R McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh, et al. How far are large language models from agents with theory-of-mind? arXiv preprint arXiv:2310.03051, 2023.
Wentao Zhu, Zhining Zhang, and Yizhou Wang. Language models represent beliefs of self and others. arXiv preprint 2002 18496, 2024.
17

---

Published as a conference paper at ICLR 2026
A FULL PROMPT
No Visibility
Track the belief of each character as deseribed in the story. 2. A character's belief is formed only when they perform an action themselves or can observe the action taking place. 3. character does not have any beliefs about the container and answer the question,
Instruction:
its contents which they cannet chserve. preaict only what is inside the queried container, strictly based on the belief of the character, mentioned in the question. 5. IF the queried character has belief about the container in question, then predict 'unknown'. 6. Dc not predict container character as the final output.
story: Bob and Carla are working in a busy restaurant. To complete an order, Bob grabs an opaque bottle and fills it with beer. Then Carla grabs another opaque cup and fills it with coffee.
Question:
What does Bob believe the bottle contains?
Answer:
Explicit Visibility
Instruction: 1. Track the belief of each character as described in the story. 2. A character's belief is formed only when they perform an action themselves or can observe the action taking place. 3. character does not have any beliefs about the container and its contents which they cannet chserve. 4. To answer the question, predict only what is inside the queried container, strictly based the belief of the character, mentioned in the question. 5, If the queried character has belief about the container in question, then predict 'unknown'.
De not predict container character
as the final output.
Story: Bob and Carla are working in a busy restaurant. To complete an order, Bob grabs an opaque bottle and fills it with beer. Then Carla graps another opaque cup and fills it with coffee. Bob can observe Carla's actions. Carla cannot observe Bob's actions.
Question: what does Bob believe the cup contains?
Answer:
B THE CAUSALTOM DATASET
We needed to construct new dataset because required task that models could reliably solve. In contrast, most existing ToM datasets remain challenging for LMs. Additionally, we needed a dataset which each sample is paired with multiple counterfactuals, enabling causal computations and the extraction of the underlying mechanism. The only dataset that met both criteria was ToM, which we used in our study. However, even ToM was insufficient for investigating the full range of factors influencing the mechanism, such as the relationship between character and their object. Hence, we needed to simplify the task allow for additional counterfactuals. To test the effect of specific element, we required the ability to modify only that element without altering the rest of the story or creating an incoherent scenario. For example, consider a BigToM story where a flood occurs, and opening gate releases the water. In the counterfactual scenario where the gate remains closed, the story's s continuation becomes unintelligible, with the occurrence of a flood.
To address this, we developed CausalToM, which features simple stories accompanied by range of counterfactuals. Key features include: (1) two characters, objects, and states, (2) the ability to modify each of them independently. and (3) control over whether characters witness each other's actions. The dataset comprises four templates, one without visibility statements and three with explicit visibility statements. Each template supports four types of questions (e.g., "CharacterX asked
18

---

Published as a conference paper at ICLR 2026
about ObjectY").. We used lists of 103 characters, 21 objects, and 23 states. For our interchange intervention experiments, we randomly sampled 80 pairs of original and counterfactual stories.
C MECHANISTIC INTERPRETABILITY CONCEPTS
In this section, we summarize the residual-strr streamm framework on transformer architecture as well as describe QK- and OV-cireuits which form the conceptual foundation for the lookback mechanisms analyzed in this work (Elhage et al., 2021).
C.I THE RESIDUAL STREAM FRAMEWORK
Following the standard transformer-circuiss framework (Elhage et al., 2021), we adopt the residual- stream view, in which every layer writes its contributions into shared vector space. For token at position and layer let r!" Rd denote its residual-stream representation. The embedding layer initializes r, (0) and each sub-layer (self-attention and MLP) adds vector:
rt ri (2) Attn 10 MLP((.
(e+1)
This additive structure allows information written at one token position to be recovered many layers later downstream attention heads. Within the residual stream, distinct subspaces encode different types of information; for example, LMs allocate a specific low-rank subspace for representing the OI associated with crucial token.
C.2
QK CIRCUITS: ROUTING ATTENTION VIA POINTERS AND ADDRESSES
Each attention head computes queries and keys from the residual stream:
q Worl",
k, Wxr
and forms attention weights through sealed dot products:
4 kg
Ott>ss O exp
dd
The QK-circuit determines where the model looks. In the context of lookback mechanisms, pointer a component of the residual stream at the lookback token that, after the Wq projection, aligns strongly with address stored in earlier token after its Wx projection. A high dot product between these transformed vectors causes the model to attend to the recalled token, thereby forming QK-cireuit.
C.3 OV CIRCUITS: TRANSFERRING PAYLOADS ACROSS TOKENS
Once the QK-cireuit selects a recalled token, the attention head retrieves the value-vector payload stored in its residual stream, thereby forming OV-cireuit:
V, Wvr
which is then projected through Wo:
Attan (6) Wo Σ as-avs)
This OV-circuit determines what information is transferred.
The QK- and OV-cireuits thus together implement the mechanism by which lookback retrieves its payload: QK selects the source of information, and OV copies the relevant content into the current token's 's residual stream.
19

---

Published as a conference paper at ICLR 2026
C.4 LOOKBACK AS POINTER DEREFERENCE
lookback consists three components:
1. source reference (e.g., an OI) created an early token;
2. address copy of this reference stored in the recalled token's residual stream and placed alongside the payload to be retrieved later;
3. pointer copy stored in the lookback token's 's residual stream.
When the pointer and address align form the QK-eireuit the head dereferences the pointer and uses the OV-circuit to bring the payload forward. This general pattern recurs across the binding lookback, answer lookback, and visibility lookback mechanisms. The residual-stream view makes this structure explicit by enabling precise localization of pointers, addresses, and payloads in the model's internal representations.
C.5 ORDERING ID ASSIGNMENT
The LM assigns Ordering ID (OI; Dai et al. (2024)) the character, object, and state tokens. These Ols, encoded in a low-rank subspace of the internal activation, serve as a reference that indicates whether entity is the first second of its type independent of its token value. For example, in Fig. 3a, Bob is assigned the first character OI, while Carla receives the second. We validate the presence of OIs through multiple experiments, where intervening tokens with identical token values but different Ols alters the model's 's internal computation, leading to systematic changes in the final output predicted by our high-level causal model. The LM then uses these OIs building blocks, feeding them into lookback mechanisms to track and retrieve beliefs.
D MODEL BEHAVIORAL EVALUATION
The section reports the behavioral performance of several LM families the CausalToM task, over 10 independent runs of 100 samples each. As shown Table I, most models, particularly the smaller ones, struggle to perform the task consistently. In contrast, the Llama models, espe- cially Llama
and the Qwen model Qwen2. 1-14B------ Inst mnst achieve over 80% accuracy in both the visibility and no- no-visibility conditions. Motivated by this, we conduct a mechanistic analysis of these pegh-performin models to investigate how LMs represent and track beliefs.
and Llama-3. 1.l.1140BBInaaa
E CAUSAL MEDIATION ANALYSIS
In addition the experiment shown in Fig.9, conduct similar experiments for the object and state tokens by replacing them in the story with random tokens, which alters the original example's 's final output. However, patching the residual stream vectors of these tokens from the counterfactual run restores the relevant information, enabling the model to predict the causal model output. The results of these experiments are collectively presented in Fig.2, with separate heatmaps shown in Fig. 10, 11, 12.
20

---

Published as a conference paper at ICLR 2026
Table Model performance under the visibility and no-visibility conditions of CausalToM (mean standard deviation), computed over 10 independent runs of 100 samples each. Highlighted models achieve accuracy above 80% both conditions.
Model Name
No Visibility (mean + std)
Visibility (mean + std)
7B Models
Llama-2-7b-hf
0.011 + 0.007
0.006 + 0.008
Qwen2.5-7B
0.280 + 0.039
0.061 + 0.017
0.948 + 0.022
0.719 + 0.031
8B Models
Llama-3.1-81
0,446 + 0.046
0.293 + 0.042
Llama-3. Biima-3.1-8B-Insse -.nstretttt
0.722 + 0.044
0.310 0.035
Meta-Llama-3-8
0.297 +: 0.027
0.117 + 0.024
Meta-Llama-3-in -aa-ma------Inttuuu
0.349 + 0.042
0.085 + 0.025
13B Models
Llama-2-13b-hf
0.328 + 0,041
0.117 + 0.027 0.360 + 0.049
OLMo-2-1122-13311 Innsttttttttt
0.522 +: 0.034
14B Models
Qwen2.5-14B
0.865 + 0.038
0.433 + 0.040
Qwen2.5-14P Instructt
0.962 + 0.021
0.912 + 0.022
27B Models
Gemma-3-27ba
0.527 + 0.036
0.388 + 0.034
32B Models
DLMMo-2-032-33BBInnttte
0.814 + 0.033
0.679 + 0.029
70B Models
Meta-Llama- Instruct
0.952 +: 0.020
0.923 + 0.014
40SB Models
Meta- 3.1
0.883 0.041
0.97 + 0.013
DESIDERATE BASED PATCHING VIA CAUSAL ABSTRACTION
Causal Models and Interventions A deterministic causal model M has variables that take on values. Each variable has mechanism that determines the value of the variable based on the values of parent variables. Variables without parents, denoted X, can be thought of as inputs that determine the setting of all other variables, denoted M(x). A hard intervention +- overrides the mechanisms of variable A, fixing it to a constant value a.
Interchange Interventions We perform interchange interventions (Vig et al., 2020: Geiger et al., 2020) where a variable (or set of features) A is fixed be the value would take on if the LM were processing counterfactual input e. We write A +- Get(M(c), A) where Get(.M(c), A) is the value of variable A when M processes input c. experiments, we will feed original inpur o to a model under an interchange intervention Mie)Ge(MMie))))))0
Featurizing Hidden Vectors The dimensions of hidden vectors (Smolensky, 1986), and so it is typical to featurize a hidden vector using some invertible function, e.g., orthogonal matrix, project hidden vector into variable space with more inter-
an ideal unit of analysis
21

---

Published as a conference paper at ICLR 2026
Bob and Carla are working in a busy restaurant. To complete an order, Bob grabs an opaque bottle and fills it with beer. Then Carla grabs another opaque cup and fills it with coffee. Question: What does Bob believe the bottle contains? Answer: beer
Counterfactual
David and Carla are borking in busy restaurant. To complete order, David grabs opaque bottle and fills it with beer. Then Carla grabs another opaque cup and fills it with coffee.
Original
Question: What does Bob believe the bottle contains? Answer: unknown
Causal Model Output: beer
The original example produces the output unknown because Bob is not mentioned in the story, leaving the model without any information about his beliefs. However, when the residual stream vectors corresponding to Bob from the counterfactual run are patched into the original run, the model acquires the necessary information about that character and consequently updates its output to beer.
Figure 9: Causal Mediation Analysis:
80
Character
60
Layes
40
Story:
Beb -agey
Carla
aar
worksng
busy
restaurann at
an order complete
ueb- Haeey grabis
an opaque
fills
with
beer
Then
Carta
grabs
ano que
cup
and
wi
cuf te
question
What diess Bnb
be beliewe
bot ts conta Lins
Answer
Figure 10: Information flow of character input tokens using causal mediation analysis.
Object
Layers
Story
Carla
are workiing
bus restaurant
corplete
an
order r
Bob grabs
an
openue bottt mig
it
with
Then
grabs
opeque cup
and
coff fee
Questtion
Mhat does Bob
bott containg Answer
fil
Car
be
Figure 11: Information flow of object input tokens using causal mediation analysis.
pretable dimensions called "features" "rcatures"(Muellae (Mueller al., 2024). A feature intervention Fn +- f edits the mechanism of a hidden vector h to fix the value of features Fn to
Alignment The LM is low-level causal model C where variables are dimensions of hidden vectors and the hypothesis about LM structure a high-level causal model H. An alignment II assigns each high-level variable A to features of a hidden vector FA. e.g., orthogonal directions in the activation space of h. To evaluate an alignment, we perform intervention experiments to evaluate whether high-level interventions on the variables in H have the same effect as interventions the aligned features L.
Causal Abstraction
We use interchange interventions to reveal whether the hypothesized causal model H is abstraction of an LM L. To simplify, assume both models share input and output
22

---

Published as a conference paper at ICLR 2026
State
Layers
60
Story
and
Carla
are working
bus restaurant rana
To
compl Letete an order r
Bob grabs
an
opaque bottle fills lil
with
Then
Carta
grabs
cup
and
wi
cof
Question
Mhat
does
bell Bob
bot tl contains
Answer
beer-
Figure 12: Information flow of state input tokens using causal mediation analysis.
space. The high-level model H. is abstraction of the low-level model L under given alignment when each high-level interchange intervention and the aligned low-level intervention result the same output. For high-level intervention A aligned with low-level features Fý with counterfactual input с and original input b, we write
GetOutput(Crgg (e) GetOuttpt((pgg--Geece.ppppp (o)) = GetOutput(/Ax--Garreeeeey
(1)
If the low-level interchange intervention on the LM produces the same output as the aligned high-level intervention on the algorithm, this piece of evidence in favor the hypothesis. This extends naturally to multi-variable interventions (Geiger et al., 2024).
Graded Faithfulness Metric We construct counterfactual datasets for each causal variable where base prompt and a counterfactual prompt.. The counterfactual label the expected output of the algorithm after the high-level interchange intervention, i.e., the right-side of Equation 1. The interchange intervention accuracy is the proportion of examples for which Equation holds, i.e., the degree which H faithfully abstracts
an example consists
Aligning Features to Causal Variables (SVD) to featurize residual stream vectors, i.e., features are the orthogonal singular vectors. For given transformer layer and token location, we collect the residual stream vectors across large number of examples and compute the singular vectors. Given singular vector features Fn of hidden vector in the residual stream of the LM C, we select features to align with a causal variable A in causal model H using Desiderata-basee
our experiments, we use Singular Vector Decomposition
Component Masking (DCM) (De Cao al., 2020; Davies et al., 2023; Prakash et al., 2024). Given original input o and counterfactual input c, we train a mask m [0, 1] |xll on the following objective
CE (eet) (b)), GetLogits Ge(H((), (AL (b)))
23

---

Published as a conference paper at ICLR 2026
G PSEUDOCODE FOR THE BELIEF TRACKING HIGH-LEVEL CAUSAL MODEL
Algorithm 2 High-level causal model for the no visibility
1: procedure BELIEITRACKINGGG, 01, $1, C2y 02 82, 9c 9o)
2: Ordering ID assignment
3:
00 S < AssignOIs((e 01, s1l: 01 01 01
4:
02 22 + AssignOIs( (02, 02, 82). 2)
5:
6: Binding lookback mechanism
7:
binding.address, + (copy(cp') copy madiing.dddess, (copy (ep) copy (0g) ))
8:
9:
2 + copy : 00 02 (go))
8 + copy for of 02 02 Hgol)
20
11:
binding pointer + oi (o))))
12:
13:
14:
if hinding_address = binding-pointer then
15:
binding payload
hinding-payload t copy(s) (1)
16:
else if binding-address,,
medding-pointer then < )
17:
18:
end
19:
20:
Answer lookback mechanism
21:
answer_pointer answerl
Londing-payloon
22:
andresssadrress address + sP1
answer2.address 2011
23:
24:
if answerl Ladress = answer_pointer then
answer.payload + $1
26:
else if answer2 address = answer pointer then
answer payload <- 82
28:
end if
29:
return answer_paye payload
30:
end procedure
H RESIDERATA-BASER COMPONENT MASKING
While interchange interventions on residual vectors reveal where causal variable might be encoded in the LM's internal activations, they do not localize the variable to specific subspaces. To address this, apply the Desiderata-based Component Masking technique (De Cao al., 2020; Davies et al., 2023; Prakash et al., 2024), which learns sparse binary mask m over the singular vectors of the LM's internal activations. We first cache the internal activations from 500 samples at the token positions specified in the main text for each experiment. Next, apply Singular Value Decomposition to compute the singular vectors as a matrix V Rdx5000 where is the dimensionality of the residual stream. We then masked this matrix using learnable binary vector m singular vectors
[0, 1] to choose subset of
V Vmasked Vm
The chosen subset of
vectors is used to construct projection matrix W Wproj E Rdxd
V W proj Vmnakea masked
Then, we perform subspace-level interchange interventions (rather than replacing the entire residual vector) using the following equations:
Anear W he + (T W Wprojjl h
24

---

Published as a conference paper at ICLR 2026
where ho is the full residual stream of the original run, he is the full residual stream of the counterfac- tual rum, and knew is the intervened vector where the chosen subspace of ho is replaced with that of he-
The core idea is to first remove the existing information from the subspace defined by the projec- tion matrix and then insert the counterfactual information into that same subspace using the same projection matrix.
In order to find the optimal subspace, optimize to maximize the agreement between the causal model output and the LM's output. To do so, we train the mask for each experiment on 80 examples of the same counterfactual datasets specified in the main text and use another 80 samples as the validation set. We use the following objective function, which maximizes the logit of the causal model output token:
C -lgit wellloggccuaal..oo. http:sss.mmoo..oouuu...eeeeeeeee
+
Σ m
(6)
Where À is a hyperparameter used to control the rank of the subspace and m is the learnable mask. See Appendix F for details on how the causal model output under intervention are computed. We trained m for one epoch with ADAM optimizer, on batches of size 4 and learning rate of 0.01. During training, the parameters of m are continuous and constrained to lie within the range [0, 1]. To enforce this constraint, we clamp their values after each gradient update. During evaluation, we binarize the mask by rounding each parameter to the nearest integer, i.e., or 1.
I ALIGNING CHARACTER AND OBJECT OIs
As mentioned in section 5.2, the source reference information, consisting of character and object OI, is duplicated to form the address and pointer of the binding lookback. Here, we describe another experiment to verify that the source information is copied to both the address and the pointer. More specifically, we conduct the same interchange intervention experiment without freezing the residual vectors at the state tokens. Based on our hypothesis, this intervention will not be able to change the state of the original run, since the intervention at the source information will affect both address and pointer, hence making the model form the original QK-
described in Fig. 6, but
Counerfittal
Carla and Bob are working in a busy rurant.... To complete
ordes, Carla grabs opaque and filla with coffee.
Interveation Accuncy
1.0
Then Bob be another opaque botthe and f11's it with beer.
0.8
Subspace simension
ldeatide: what doea carla beliove
cup contains?
0.6
Answer: : ooffee
0.4
Bob and Cagla
working in Jusy restaurant. To complete ordele, Rob grabs opaque bottle and fills with bear,
Origial
0.2
Then Carla grabs another opaque cup and fills it with coffee.
Question: what does Carla believe the cup contains?
0.0
Answer: coffee
10
60 70
Intervoution:
Binding Source (0,0 0)
Layers
Causal Model Output: beer
Figure 13: Source Reference Information of Binding lookback: In this interchange intervention experiment, the source information, i.c., the character and object OIDs (O, O), is modified, while the address and payload (O, O, A) are recomputed based on the modified source. Since both the address and pointer information are derived from the altered source, the binding lookback ultimately retrieves the same original state token as the payload. As result, we do not observe high intervention accuracy.
In section 5.2, we identified the source of the information but did not fully determine the locations of each character and object OI. To address this, now localize the character and object Ols separately to gain a clearer understanding of the layers at which they appear in the residual streams of their respective tokens, shown in Fig. 14 and 15.
25

---

Published as a conference paper at ICLR 2026
Conntrlatiul
then Carla frabs another apaque cup and fills it with coffee. Gueatidet What ssea Carla belleve the concains? Answer: coffee
Bob and Carla working in restaurant. To complete and fills ordez, Bob grazs opaque
beer.
Interveation Aceuracy
0.8
Rull residual
dimen 23.0
Cazla Bgb ase working
b restaueant. To coplete
0.4
Origind
an ordge carla grabs er opeque di and fills it with (
Then Bolo grabs another opaque bottle and fil's it with water.
0.2
Question:
believe bottle
Answer: unknown
0
10
20
30
40 Layers
50
60
70
Intervention: Charstee (0)
Frozem: Object Ol, Binding and Answer Adddressss-4Ptyyaakkk (0, O.
Cansal Model Outpot:
Figure 14: Character OI: This interchange intervention experiment swaps the character Of (O), while freezing the object OI well binding lookback address and payload (O, O O). Swapping the character Ols in the story tokens changes the queried character OI to the other one. Hence, the final output changes from unknown to water.
Chunestatual
Bob and Carla are working in a busy restaurant. To complete an ordez, Bob grabs opsque bottle and fills it with beer, fills with coffee.
Intervention Accuracy
1,0
residual Subspace dimension $8.00
Then Carla gralos another apaque believs cup contains?
Questiok: what
Answez: coffee
0.6-
Cazla and agb are working in becy restaurant To complete grabs opaque rillf E with Bob graha ang opaque bottle and
0.4
Original
ordge,
0.2
Queat What d0eg Carla belleve the bottle contains?
Answer:
unknown
0.0
10
20
50
60
70
Intervoation: Rinding Source (0, O)
Layers
Binding Answer AckhessosPPyloodd,
Cansal Model Output: tea
Figure 15: Object OI: This interchange intervention experiment swaps both the character and object Ols (O, O). while freezing the address and payload of binding lookhack (O, O. O) as well as queried character Ol (0). Swapping both character and object Ols in the story tokens ensures that the queried object gets the other OI. Hence, the final output changes from unknown to tea.
ALIGNING QUERY CHARACTER AND OBJECT Ols
section 5.2, we localized the pointer information of binding lookback. However, we found that this information is transferred to the lookback token (last token) through two intermediate tokens: the queried character and the queried object. In this section, we separately localize the Ols of the queried character and queried object, shown in Fig. 16 and Fig. 17.
Ouunsttatull
Bob and Carla working in restaurant. order, Rob grabs opsque bottle fills complete beer,
Interveation Accuracy
Then Carla graos another apaque cup and fills it with coffee. Question: what does Carla believe the contains? Answer: coffee
0.8
Subspace
dimension =
busy restaurant. To complete an ordez, Carla grabs ac opaque cup and fills it with tea.
Cazla Bob ase werking
0.4
(Migoal
Then Bob graba another spaque bottle and fil's it with water. What Carla belleve bottle Answer: unknown
0.2
0
10
20
30
40 Layers
50
60
70
Binding
Camuul Model Ouigul: water
Figure 16: Query Character OI: This interchange intervention experiment alters the Ol of the queried character (O) to the other one. Hence, the final output changes from unknown to water.
K SPECULATED PAYLOAD IN VISIBILITY LOOKBACK
As mentioned in section 6, the payload of the Visibility lookback remains undetermined. In this section, attempt to disambiguate its semantics using the Attention Knockout technique introduced
26

---

70 Published as a conference paper at ICLR 2026
Orggind Conneraatoll
and fills then Carla graps another apaque cup and fills it with coffee. Queation: What deea Carla belleve the contains? Answer: coffee
Bob and Carla working in restaurant. To complete beer.
an ordez, Bob grazs opaque
Intervention Accuracy
0.8
Subspace
dimension s7
Cazla Bob ase working in busy restaurant. To complete
0.4
an ordes, Carla grabs er opeque cup and fills it with tea.
Then Bob graba another opaque bottle add fil's it with water.
0.2
Question: what does believe cup contains?
Answer: unknown
0.0
0
10
20
30
40 Layers
50
60
Intervention: Binding (+)
Causal Model Outputt water
Figure 17: Query Object OL: This interchange intervention experiment alters the OI of the queried object (O) to the other one. Hence, the final output changes from unknown water.
in (Geva et al., 2023), which helps reveal the flow of crucial information. We apply this technique to understand which previous tokens are vital for the formation of the payload information. Specifically, we "knock out" all attention heads at all layers of the second visibility sentence, preventing them from attending to one or more the previous sentences. Then, we allow the attention heads attend to the knocked-out sentence one layer at time.
the LM is fetching vital information from the knocked-out sentence, the interchange intervention accuracy (IIA) post-knockout will decrease. Therefore, decrease in IIA will indicate which attention heads, at which layers, are bringing in the vital information from the knocked-out sentence. If, however, the model is not fetching any critical information from the knocked-out sentence, then knocking it out should not affect the IIA.
Layer-wise IIA with Attention Knockouts
Intervention Accuracy
1.0
0.8
0.6
0.4
Second sent sent
First vis sent
0.2
Second sent
0.0
40
60
80
Layer Index
Figure 18: At the second visibility sentence, attention heads are restricted to retrieve information from of three prior contexts: (1) both the second story sentence and the first visibility sentence line), (2) only the first visibility sentence (- line), or (3) only the second story sentence (- line).
To determine if any vital information influencing the formation of the Visibility lookback payload, perform three knockout experiments: 1) Knockout attention heads from the second visibility sentence to both the first visibility sentence and the second story sentence (which contains information about the observed character), 2) Knockout attention heads from the second visibility sentence to only the first visibility sentence, and 3) Knockout attention heads from the second visibility sentence the second story sentence. In each experiment, measure the effect of the knockout using IIA.
Fig. 18 shows the experimental results. Knocking out any of the previous sentences affects the model's ability produce the correct output. The decrease in IIA in the early layers can be explained by the restriction on the movement of character Ols. Specifically, the second visibility sentence mentions the first and second characters, whose character Ols must be fetched before the model can perform any further operations. Therefore, we believe the decrease in IIA until layer 15, when the character Ols are formed (based on the results from Section I), can be attributed to the model being restricted from fetching the character Ols. However, the persistently low IIA even after this layer-especcally when both the second and first visibility sentences are involved indicates that some vital information is being fetched by the second visibility sentence, which is essential for forming the coherent Visibility lookback payload. Thus, we speculate that the Visibility payload encodes information about the observed character, specifically their character OI, which later used to fetch the correct state OI.
27

---

Published as a conference paper at ICLR 2026
L CORRELATION ANALYSIS OF CAUSAL SUBSPACES AND ATTENTION HEADS
This section identifies the attention heads that align with the causal subspaces discovered in the previous sections. Specifically, first we focus on attention heads whose query projections are aligned with the aarracss--caractteriin by the relevant singular vectors scctrs-that contain the correct answer state OI. To quantify this alignment between attention heads and causal subspaces, we use the following computation.
Let Q E
IR Aamasae Xemass denote the query projection weight matrix for given layer:
We normalize Q column-wise:
Q |Q
Q.
for each column j
(7)
Let S E R represent the matrix of i singular vectors (i.e., the causal subspace basis). We project the normalized query weights onto this subspace:
Qn Q
(8)
We then reshape the resulting projection into per- head components. Assuming Q. E Radass x each attention head has dimensionality dn, we write:
and
head
( SV
E
Rd xk for i 1,... Theads
(9)
Finally, we compute the norm of each attention head's projection:
head norm,
for
Phends
(10)
We compute the head norm for each attention head in every layer, which quantifies how strongly a given head reads from the causal subspace present in the residual stream. The results are presented in Fig. 19, and they align with our previous findings: attention heads in the later layers form the QK-cireuit by using pointer and address information retrieve the payload during the Answer lookback.
We perform similar analysis to check which attention heads' value projection matrix align with the causal subspace that encodes the payload of the Answer lookback. Results are shown in Fig. 20, indicating that attention heads later layers primarily align with causal subspace containing the answer token.
M BELIEF TRACKING MECHANISM IN BIGTOM BENCHMARK
This section presents preliminary evidence that the mechanisms outlined in Sections and 6 generalize to other benchmark datasets. Specifically, we demonstrate that Llama 3-70B- 77007000I Instruct answers the belief questions (true belief and false belief) in the Big dataset Gandhi al. (2024) in manner similar to that observed for CausalToM: by first converting token values to their corresponding Ols and then performing logical operations on them using lookbacks. However, as noted in Section NigTToMilike other benchmarks-lack lacks the coherent structure necessary for causal analysis. As result, we were unable to replicate all experiments conducted on CausalToM. Thus, the results reported here provide only preliminary evidence of similar underlying mechanism.
To justify the presence of Ols, we conduct an interchange intervention experiment, similar to the one described in Section J, aiming to localize the character OI at the character token in the question sentence. We construct an original sample by replacing its question sentence with that of a counterfactual sample, selected directly from the unaltered BigToM dataset. Consequently, when processing the original sample, the model has no information about the queried character and, as result, produces unknown the final output. However, if replace the residual vector the
28

---

Published as a conference paper at ICLR 2026
4 proj norms Answer lookback pointer subspace
60
-10
57
54
51
48
Layers
45
42
39
36
33
-2
30
0
10
15.
20
25
30
35
400
45
50
55
60
Attention Heads
Figure 19: Alignment between the Answer lookback pointer causal subspace and the query projection
matrix in L.aama- 3- IOs-Inst
'_proj norms on Answer lookback payload subspace
75
12
70
10
Layers
65
60
55
50
0
10
15
20
35
40
45
50
55
Attention Heads
Figure 20: Alignment between the Answer lookback payload causal subspace and the value projection
matrix in Llama-3 --70---------0---- ruct.
queried character token in the original sample with the corresponding vector from the counterfactual sample (which contains the character OI), the model's output changes from unknown to the state token(s) associated with the queried object. This because inserting the character OI at the queried token provides the correct pointer information, aligning with the address information at the correct state token(s), thereby enabling the model to form the appropriate QK-circuit and retrieve the state's OL As shown in Fig. 21, we observe high IIA between layers 9 28 similar to the pattern seen in CausalToM-Sueee suggessingg that the queried character token encodes the character OI in its residual vector within these layers.
Next, we investigate the Answer lookback mechanism in BigToM, focusing specifically on localizing the pointer and payload information at the final token position. To localize the pointer information, which encodes the correct state OI, we construct original and counterfactual samples by selecting two completely different examples from the Tog dataset, each with different ordered states as the correct answer. For example, as illustrated in Fig.22. the counterfactual sample designates the first state the answer, thrilling plot, whereas the original sample designates the second state, almond milk. We perform an intervention by swapping the residual vector at the last token position from the counterfactual sample into the original run. The causal model outcome of this intervention is that the
29

---

Published as a conference paper at ICLR 2026
Noor is working barista busy coffee shop. Noor wants nake delicious cappeccino foz customer who asked for oat milk. Noor grabs milk pitcher and fills it with oat milk, A coxezxer, who didn't hear the cuatomert request, milk
swaps the Noor attending to another task. Noor see3 her coworker swapping the milk.
pitcher with almond milk while
1.0
Interventin Accuracy
Full residual Subspace
0.8
Question: Does Noor believe the milk pitcher contains oat milk almond
dimension 380
0.6-
Answez:
almond mick
0.4
Tariq is fisherman in a coastal village in Indonesia. He wente catch fisp for his family's dinner using fishing dry tat the net appears to be dry and ready to use. A sudder downpouz soaks the fishing net, making it heavy and difficult use. Tariq rebaces the
0.2
that left hanging
branch, Tariq sees
Orighall
0.0
30
50
60
soaking the fishing net. Question: Does Noor believe fishing net is dry
Layers
soaked?
Anzwer:
unknown
Causal Model Output: soaked
This interchange intervention experiment inserts the first character's OI into the residual stream at the queried character token (O), resulting in the movement of pointer information to the last token that aligns with the address information of binding lookback mechanism. Consequently, the model is able to form the appropriate QK-circuit from the last token to predict the correct state answer token(s) as the final output, instead of unknown.
Figure 21: Query Character OI in BigToM:
model will output the alternative state token from the original sample, oat milk. As shown in 22g this alignment occurs between layers 33 and 51, similar to the layer range observed for the pointer information in the Answer lookback of CausalToM.
Yaul is in a small bookstore browsing through the so encence fiction section. Yael wants to find a book with thrilling plos to read during her upconing vacation. de notices the book cover futuristic cityreape. A sustomer accidentally places book with similar cover, but bland storyline, in the spot where the thrilling novel
nevel displaying
intriquing scene of a
1.0
after browsing it. Yael does not notica the custemer putting the book bacx the wrong spoz.
Interveation Accuracy
Pull residaal
Subspace
dimensin s 149
Does Ysel believe the book with the intriguing thrilling plot bland storyline? Answez: thrilling plot
Question:
0.6-
0.4
Noor i working
barista
busy coffee shop. Noor asked
0.2-
delifilow ceppuccino for for oe: milk. Noor grabe milk pitcher and fills it with
0.0
Origoal
eat mi k. A coworker, who didn't hear the customer's request, milk the pitcher with almond milk while swaps sttending to another seek. Noor sees her coworker
0
10
20
30
40
50
60
70
Layers
swappi g the milk.
Questinis on: Does Noor believe the milk pitcher contains oat
milk almond milk?
Answez:
milk
Causal Madel Output: oat milk
This interchange intervention experiment modifies the pointer information (0) of the Answer lookback, thereby altering the subsequent QK- circuit to attend to the other state (e.g., oat milk) instead of the original one (e.g., almond milk). As a result, the model retrieves the token value corresponding to the other state to answer the question.
Figure 22:
Answer Lookback Pointer in BigToM:
Further, to localize the payload of the Answer lookback in BigToM, we perform an interchange intervention experiment using the same original and counterfactual samples as mentioned in the previous experiment, but with a different expected output-namely the correct state from the counterfactual sample instead of the other state from the original sample. As shown in Fig. 23, alignment emerges after layer 59, consistent with the layer range observed for the Answer lookback payload in CausalToM..
Finally, investigate the impact of the visibility condition the underlying mechanism and find that, similar to CausalToM, the model uses the Visibility lookback to enhance the observing character's awareness based on the observed character's actions. To localize the effect of the visibility
30

---

Published as a conference paper at ICLR 2026
Yael is in small bookstore brewsing through the science #iccion section. Yael wants find book with thrilling plo: to read during her upcomang vacation. de notices the book cover of novel displaying futuristic cityscape. customer accidentally places book similar cover, where the thrilling novel was, after browaing it, Yael doea not notice the customer putting the book back in the wrong apot.
Interveation Accuracy
1.0-
Full residual
0.8
Subspaca
intriquing scene of
dimension
Chuststtatu
bland atoryline,
the apot
0.4
0.2
Queation: Does Yael beliave book with intriguing cover has a thrilling plot or a bland storyline? thrilling plot
0
10
20
30
40
50
60
70
Anawex:
Layers
Noor is working as a barista at a basy coffee shop. Noor wante to make delicious cappuccino for customer who asked Noor
and fiils with A coverker, who didn't hear customer's requzst, swaps oat milk in the pitcher with almond milk while Nooz i: attending to another task. Noor sees her coworker
for
milk
Oetprea
eat
swapping 2g the nilk.
Queation in: Does Nooz beliave
pitcher contains oat
milk of almond milk?
Anawer; almond milk
Consal Model Oulput: thrilling plot
Figure 23: Answer Lookback Payload in BigToM: This interchange intervention experiment directly modifies the payload information (A) of the Answer lookback, which is fetched from the corresponding state tokens and predicted as the next token(s). Thus, replacing its value in the original run, e.g. almond milk, with that from the counterfactual run, e.g. thrilling plot, causes the model's next predicted tokens to correspond to the correct answer of the counterfactual sample.
Noor is working as barista at busy coffee shop. Noor for make delicious ceppucsino for customer who asked pitcher fills eat milk. A cowerker, who didn't hear the customer's s request, swaps the eat milk in the pitcher with almond milk while Noor attending another task.
grabs
Orunteriettl
hez coworker
1.0
swapping
Interveation Accuncy
Vaest Lon:
boes Noor beliave the milk pitcher contasng oat
0.8
Visiblitt condition
mi
almond milk?
0.6
Question
Noor working as a barista at busy coffee shop. Noor wants to nake delicious cappeccino for customer who a asked for milk. Noor grabs malk pitcher and fille with esstomer's request, swaps the eat milk in the pircher with almond milk while Noor is attending to another taak. Noor does retusee her poworker awapping the mikk.
0.4
Misiilitty
condition
+Question
(rnggnal
didn't
0.2
0.0
10
60
Layers
Vgestion: Uoes Roor believe
mask or alnond milk?
AnoNeE: oak milk
Expected Outgut: almond milk
Figure 24: Visibility Lookback in BigToM:: We perform three interchange interventions to establish the presence of the Visibility ID. which serves as both address and pointer information. When intervening at the source (O)-ie., the visibility sentence both the address and pointer are updated, resulting in alignment across layers. Intervening only at the subsequent question tokens leads to alignment only at later layers, after the model has already fetched the payload (Á) ). However, intervening at both the visibility and question sentences results in alignment across all layers, as the address and pointer remain consistent throughout.
condition, we perform an interchange intervention in which the original and counterfactual samples differ in belief
that is, if the original sample involves a false belief, the counterfactual involves a true belief, and vice versa. The expected output of this experiment the other (incorrect) state of the original sample. Following the methodology in Section 6, we conduct three types of interventions: (I) only at the visibility condition sentence, (2) only at the subsequent question sentence, and (3) at both the visibility condition and the question sentence. As shown in Fig. 24, intervening only the visibility sentence results in alignment at early layers, up to layer 17, while intervening only at the subsequent question sentence leads to alignment after layer 26. Intervening both the visibility and question sentences results in alignment across all layers. These results align with those found in the CausalToM setting shown in the Fig. 8.
31

---

Published as a conference paper at ICLR 2026
Previous experiments suggest that the underlying mechanisms responsible for answering belief questions in BigToM are similar to those in CausalToM. However, we observed that the subspaces encoding various types of information are not shared between the two settings. For example, although the pointer information the Answer lookback encodes the correct state's s Of both cases, the specific subspaces that represent this information at the final token position differ significantly. We leave a deeper investigation of this phenomenon-sharte semantics across distinct subspaces in different distributions-for future work.
N GENERALIZZTION OF BELIEF TRACKING MECHANISM ON CAUSALTOM TO INSTRUCT AND QWEN2. 5
LLAMA-3 - -LAMA-3.1-405BIINTTTT -NSSRRRIISSTRR
This section presents all the interchange intervention experiments described in the main text, conducted using the same set of counterfactual examples on Ll Llama-3. 1.taaa...110------- 1-1-005--Instruce and Qwen2. 5- 144 Pes-Instruse ruct, using NDIF Fiotto-Kaufman et al. (2025). Each experiment was performed on 80 samples. Due to computational constraints, subspace interchange interven- tion experiments were not conducted. As illustrated in Figures 26-42, the results indicate that Llama-3.m 3--.--1111-00---s----- mechanism a. - This suggests that the identified belief-tracking mechanism generalizes to not only larger but models in other models families well.
and Owen2. 2-14BB-Irs Instruct employ the same underlying to reason about belief and answer related questions.
Origiel Connerhattl
Carla are working busy restaurant. complete ordez, Carla grabs an opaque cup and fills it with coffee. grabs another epaque What aoes Carla believe the contains?
Intervanton Accuracy
1.0
Full residual
and fills
beer.
Question:
0.6
Answer:
Bob and Carla ase working in a busy restaurant. To conplate peer,
order, Bob grabs opaque bottle fills
carla grabs another opaque cup and filla with coffee.
0.2
Question: what does Carla believe the contains? Answer: coffee
0.0
10
20
30
40
50 60 70 80 90 9010011022
Model Ouiput:
Layers
Figure 25: Payload and address of Binding lookback in Llama-3 1---005--nnstruc.
Counetiatual
Carla Bob are working busy restaurant. To complete an ordez, Carla grabs an opaque cup and fills it with coffee, Then Bob grabs another opaque bottle fills
10
Full residual
bear.
Inteevestivn Accmmey
0.8
Question: what does Carla believe the cup contains? Answer:
Bob and Carla are working in a busy restaurant. To complete opaque bottle and fills Then carla grabs another her opaque cup and filla with coffee.
Oetgiol
Bob
02
Queation: What doea Carla belleve the cup containa? Answer: coffee
Causal Model Output:
0.8
19
20
Layen
30
Figure 26: Payload and address of Binding lookback in Qwen2 1wen2.5-14B-
Connttribctta
an ordez, Carla grabs an opaque cup and filla it with tea.
Carla Bob working in bosy
complet
Interveation Accuracy
1.0
Full residual
0.8
Then Bob gra another opaque bet and fil's with
Queatidd; khat doea Carla beisbve we the cup contains?
0.6-
Answer:
Bob and canja are working in a restaurant. To conc akk ordgs, Bob
0.4
Origual
botgle and fills it with Beer. Carla grads another opaque cup sills with coffee.
0.2
question: what does Carla believe the cup concains?
Answer: coffee
0
1'0
20
30
40
50
60 70 s0 901001
Cousal Model Output: beer
Layers
Figure 27: Source Information of Binding lookback in Llama-3.14 -
32

---

Published as a conference paper at ICLR 2026
Carla and Bob working in restsurant. To complete
Conntrlatial
an ordez, Carla grabs ar opaque cup and fills it wita tea.
Then Bob grabs another
bot and fills with bell the cup concains?
Intervertoon Aromay
888
Quest ie: what does
Anewer:
1.00
Bob and Canja ase working in à guly restaurant. To compeate
0.4
Onginn
an order, Bob grabs opaque bobple and fills it with Boor, another opaque cup filla with coffee.
Carla
02
Quest ion: what does Carla believe the cup concains?
Full res dual
Answer: : coffee
10
Cannal Madel Outgut: beaz
Layen
Figure 28: Source Information of Binding lookback in 5wen225-14BIInstruee
Connectatual
an ordez, Carla grabs ar opeque cup and fills it with coffee. Then Bol arabs another
Carla and Bob working in restaurant.
complete
Interveation Accuracy
Full residual
botele and fills with beer. Quest what does Carla believe the cup concains? Answer:
0.8
coffee
0.4
Original
busy restaurant. To complete an ordse, Bob grais an opaque botele and fills with beer.
Cad/aaria ase working
0.2
Then Cazla graba another opáque and filla it with coffee.
What Carla believa contains?
Anawer:
0
10
20
30
40
50
80 901001 (0 Layers
60 70
Model Oulpul:
Figure 29; Source Reference Information of Binding lookback without freezing address and payload in Ll alama-3. 1.4440--Innsuuuee
O GENERALIZATION OF BELIEF TRACKING MECHANISM ON CAUSALTOM WITH THREE CRREACTER-ORJECTSSTTIE TRIPLES
Experimental results from the main text and the previous section show that the identified belief- tracking mechanism generalizes across both model scale and model family. Although the interchange- intervention experiments on BigToM demonstrate that this mechanism transfers to real-world bench- marks, it remains unclear whether it would still generalize when the prompt includes larger number of characters, objects, and states.
To investigate this, conducted behavioral evaluations of larger Llama models and Qwen models on tasks involving three characters, three objects, and three states, under both no-visibility and visibility conditions. All three models succeeded in the visibiiiyyy condition, but none consistently solved interchangeinttrreniioo the visibility condition. Consequently, we performed interchange intervention experiments only for the no-visibility condition on Qwen2. 1--14-IInntzutt generalizes. As shown in Figures 43-50, Qwen2. ..nnn..55110 5-14B-Instt ruct utilizes the same mechanism involving binding and answer lookbacks solve the no-visibility belief tracking task.
assess whether the mechanism
33

---

Published as a conference paper at ICLR 2026
Carla Bob working in besy
complete
original Conaurlacuul
an ordez, Carla grabs at opaque cup and fills it with coffee. bettle and fills with beer, questide: what does Carla believe the cup contains? Answer:
Then Bob grabs another
Interverion Aremay
coffee
0.6
and
ase working
susy restaurant. Io complete
0.4
an ordas, Bob grabs an opaque bottle and filla it with bear.
Then Carla grabs another opaque and tills it with coffee.
0.2
Carla bellew cup
Anawez:
10
90
Model Ouiput:
Layen
Figure 30: Source Reference Information of Binding lookback without freezing address and payload in Qwen2.
Origiea Countoractel
and Carla are working busy restaurant. To complete an ordez, Bob grass an opaque bottle and filla it with bear.
Interennion Aceuracy
1.0
Full residual
Then Carla grabs another opaque and fills it with coffee. belleve
Queat Iho: what
cup contains?
Anawez:
0.6
Carla and. agb are working in a busy
opaque restaurant. To conglete fills with teg.)
ordge carla
grabs anozher epaque bottle and fills Question: what does Carla believe the bottle contains?
0.2
Answer: unknown
10
20 30
40
50
60 70 so 901000ia20 Layers
Consal Model Ouigut: water
Figure 31: Character OI in Llama-3, -1-tos--Instrrut.
Odigiol Couneeiantal
are working busy restaurant. To complete an ordez, Bob grabs an opsque bottle and fills with beer, Then Carla drads another apaque cup and fills it with coffee. what does Carla believe
10
Full residual
0,
Questibp: Answer:
contains?
Intervetiven
Carla ans agb are working in a bia restaurant. To conf ordge, carla geabs at opaque tup and filla it with tog.)
Then Bob graba another opaque bottle fil's contains?
02
Queation: What doea belleve
Answer: unknown
00
19
20
30
40
Causal Model Output: water
Layen
Figure 32: Character Ol in Qwen2. 5-144-Insstruurt
Cruntefaatta
an ordez, Bob graza opaque
in busy and fills beer.
complete
Intervention Accuracy
1.0
Full residual
Then Caela grabs another opaque cup and #i1ls it with coffee.
0.8
Cheatiih: What doea carla bellavk the contains? Anawer:
ooffee
Carla and agb are working in a oucy restaurant. To corpecte an ordge, carla grabs ar opaque and fills it with teg) another bottle fills believe bottle contains?
0.4
Orighnal
Then Bob
0.2
Question: what
Answer: unknown
0
10
20
30
40
50
60 70 80 9020 Layers
Output: tea
Figure 33: Object OI in Llama-3, 1..maa-.--058------1 114408BInnstruee
34

---

Published as a conference paper at ICLR 2026
Bob
complete an orde", Bob grabs opeque bottle and fills beet,
in
Full residual
Then Cazla grabs another opáque cup and filla it with coffee.
Accrrnc
Quest ibn: what does Carla bel: believy the contains?
0,6
Carla and hgb are working in a pucy restaucant. To corpeet
0.4
(trignal
an ordge, Carla grabs ar opaque and fills it with tel)
Then Bol grabs anothery bottle and fills with contains?
02
Queation: What asea belleve Answer: unknown
19
30
40
Output: tea
Layers
Figure 34: Object Of in Qwen2. -wwn2.5-144--e...... --144-
Connertbacal
an orde", Bob grabs opaque bottle and fills
Bob are working in busy restaurant. To complete
beer,
Interveation Accuracy
1.0
Full residual
Then Carla grabe another opaque cup and fills it with coffee. Question: what does Carla believe the contains?
0.8
0.6
Carla and Bob ase working in a busy rescaurant. To complete ordez, Carla grabs at opaque cup and fills it with tea. grabs another opaque bottle add tills with Queation: What ases believe tha cup consains?
(riginal
0.2
Answer: unknown
10
20
30
40
50
60 70 80 90100110220
Cousal Mudel Oulput: water
Layers
Figure 35: Query Object Ol in -
Bob are
in busy
complete
Counictfictall
an ordez, Bob graiss an opaque bottle and fills with beer,
Full residual
Then Carla grabe another opaque cup and fills it with coffee. Carla believe
Ireeveenton Arommyy
Question:
What
contains?
Answez:
Carla and. Bob are working in a busy rescaurant. To complete ordes, Carla grabs an opaque cup and rilla it with tea. another opaque bottle and tills Oueat ton: what does bel feve the cup containe?
0.4
Orgiiull
Bol
02
Answer; :
unknown
00
10
20
30
Cousal Model Oulput: water
Layes
Figure 36: Query Object OI in -wen2.5-14B-Innteeee
and Carla working in buay restaurant.
complete
1.0
Countrraact
order, Bob grabs an opaque bottle and fills it with beer.
Full resioual
Then Carla grabs another opeque cup and fills is wish coffee. Question: Answer: coffee
believe the cup centains?
0.6
"Carla
working in buay
Interventie
0.4
Origonl
an order, Carla grebs a: opaque cup and fills it with tes.
Then Bob grahs snothar seaque bottle and fil's it with water,
question: What does Carla believe the bottle contains? Answez:
0.0
0
10
20
30
40
50
60 70 80 90100 10120 Layers
Model Output: water
Figure 37: Query Character OI in Llama-3. 1.4aa....1-000--tstuutttt
Bob and
working busy restaurant. an order, Bob grabs an opaque bottle and fills it with beer. Intevattinn Accuasy
complete
Connerattiall
Full residual
Then Carla graba another opaque cup and rilla 12 wish coffee. believe leve cup
Sonatioon what Answer: coffee
Carla and Bob are working in grabs at opaque cup
restaurant. To complete
Orggaa
it with tea. Then Bob grabs another opaque bottle and til.s it with water. Son: What does Carla believe the bottle contains?
Anewer:
20
Cousal Model Oulput:
Layets
Figure 38: Query Character Ol in Qwen2. 5-14B-Instrucc.
35

---

Published as a conference paper at ICLR 2026
Conttotactu
Then Bol grabs another epaque bottle and fills it with water. belleve an ordes, Carla grabs an opaqua cup and filla
Cazla and Bob aze working in a besy restaurant. To complete tea.
Interveation Accuracy
0.8
1.0
Nismor Payload
Binding Payload
Queation: Answer:
What doea tea
cup concains?
Bob and Carla are working in busy restaurant. To complete
0.4
Original
an ordez, Bob graiss opaque bottle and fills it with beer,
"hen Carla
another opeque cup fills it with coffee. believe
0.2
Questign: What does
cup concains?
Answer: coffee
0.0
0
10
20
30
40
50
60 70 80 90100110020
Intervention: Answer Pointer (O), Cousal Model Output: beer Intervention: Payload Cousol Oulput:
Layers
Figure 39: Answer Lookback Pointer and Payload in Llama-3. L-ama-3..--00D-IIsttuutt
Cazla and Bob are workingg in bosy restaurant. тo complete an ordez, Carla grabs ar opaque cup and fills it with tea.
Oounketootul
Then Bob graba another opaque bottle and fil's with water. believe
Intervention Accuracy
0.8
Anceer nonrer
Queation: What dsea Answer: tea
cup soncaina?
0.6
Bob and Carla are working in a busy restaurant. To complete
0.4
Origoal
ordez, Bob grazs opacue bottle and fills it with beer.
Then Carla
another opaque cup fills with coffee.
Queat io:: What doea
belleve
cup concaina?
Answer:
coffee
20
Intervention:
Answer Pointer (0), Causal Model Output: bear Ouiput:
Layers
Answer Payload
Figure 40: Answer Lookback Pointer and Payload in 2 --suut-nsttt..
Carla and Bob working in basy sealausant. To curple.e an
grabs opeque oup Then Bob grabs another opaque bottle and filly it with water. Bob cannot observe Carla's actione, Carla
Intervention Accuracy
1.0
Visibility
Bob's
0.8
actions.
Visibility Question
Cuestion. on: Wast dobs Carla believe the bottle contamne?
0.6
Quanstion
Oragred
an order, Carla grabs an opegue ose and fillx it wil beat.
Bob and Carla are working in a bazy zestaurant. To cappaae
Then Bob yrabs ano
bor and filla Ch coffee.
0.2
Bob cannet observe carla's acti Carla cannok
0.0
Question: Wast
believe
0
10
20
30
40
60 70
901001 2000
Answer: unknown
Layers
Camaal Model Ontput: coffee
Figure 41: Visibility Lookback in Llama-3. 3.05.---n---ttt..
Carla and Bob are worsing in a busy restaurant. To complete cup fills with Then Bob grabs another opaque bottle and fills it with water. Bob cannot. observe Carla's actions. Carla can observe Bob"s actions.
Counettrral
1.0
Visiiiity condition
0.8
Quistiin
Qwalioan: What doks Carla believe the bottle contams?? water
Visiniilty O Quesstton
0.6
Answer:
Organal
and Carla are working ng a brey restasrant To complese an order, Carla geaba an opague oue and fills with bear.
0.4
Then Bob graba anocher onamue bor and tille wich coffee.
0.2
Bob cannct observe Carla's actions. Carla cannck Bohie, actions.
Cuestion:
believe
0.0
20
30
Answez:
Layers
Canaal Model Output: coffee
Figure 42: Visibility Lookback in Qwen2 -5144B-Instruct.
36

---

Published as a conference paper at ICLR 2026
Tom, Bob, and Cazla are working a buay reatausanc- complete order, Ton grabs another
10
residual
and fills it it with
Counterbatee
tea. Then grabs an opaque
0.8
beez. Finally, Carla grazs another apaqus and fills it widp cottee.
0.6
Cueatiook What aoes Carla belleve the contains?
Anawers
Bob, Cariaa, and
wocking
restaurant,
complep- atelas order, grabs opaque bottle and fills
Origial
with bege-. Then Carla grabs another opaque cup and fills it
with coftpe. Finelly, Tom grabs another opaque tun and fills
000
16
50
with
Cuestion:
does believe oup concains?
Answer: tea
Comoat Model Ouiput: beer
Figure 43: Payload and address of Binding lookback with three character-objeec-ssate triples in
Ton, Bob, and Carla working in restaurant. It complete an order, Ton grabs another opaque
1.0
Countrtatul
with tea. Then Bob grabs opaque bottle and fills it with beaz. Finally, cakla grabs another opsque cup and
Inveventoo Acarea
with coffee.
Full residual
Queation: What 6des
belleve cup contains?
Answer:
coffee
0.4
Bob, Carla, and Ton aee wocking in busy restauzant. To complera an order, Bob grabs opaque bottle fills it Than Carla ghabs
02
with 33
it
it with tes.
cotfer. Finally, Tom grabs
epaque
10
Layers
50
Queation: What doea Carla belleve the contains?
Answer: coffee
Output:
Figure 44: Source Information of Binding lookback in three character-ohject-state triples in 5-14B..-14BIIst.ii.
Tom, Bob, and Carla are working in busy restaurans. To complete order, Ton grabs another
Full
Countetooul
tea. Then hrabs an opaque bottle and it
and fills it
Intervestion hesmey
0.8
beer. rinally, Callee grabs another apaque and fills it coffee.
0.6
Question: what does Carla believe the conceins?
Anawer:
busy rescaurant. complete Bob graza opaque bottle, and fills with bear. Than carla shats ancther opaque cup and fills it with coffee. Finally, Tom graba another opaque tun and filla Original
Bob, Carla,
are wacking
0.0
10
30
with
Layen
Queat ton: What doea belleve the cup contains?
Anawer: : coffee
Causal Madel Output: tea
Figure 45: Source Information of Binding lookback without freezing address and payload in three character-ohjeet-staa triples in Qwen2 S-14B-Instruce
Ton, Bob, and Carla working in complete an order, Ton grabs another opaque ton and fills it with tea. Then Bob grabs
10
Full residual
Couttortattal
bottle and fills it with bear. Finally, Cakla grabs another opaque cup and
with coffee.
Ineeventon
Questionn What ddes
believe cup contains?
Answer:
coffee
0.4
Bob, Carla, and Tom aee working in buag custauraan. To completth an order yo grabs opaque bottld id and fills it
02-
Onggiaal
with begg Than Carla ghabs arother opaque 4nd fills epaque
corfer. Finally, Tom grabs it with tea.
16
30
Layen
Queatioon: What doea Carla belleve the tun contains?
Causal Model Output: tea
Figure 46: Character Ol in three character-objeet-state triples in Qwen2.5-1ee. 5-144BIInstruct
37

---

Published as a conference paper at ICLR 2026
Trm, Bob, Carla working busy restaurans. complet eteplee..ae an order, Ton grabs another opaque tun and f11'8 & it Then grabs opaque bottle
10
Full residual
Coumertacie
iiteventonn Acamcyy
8x
beaz. Finally, Calla grabs another opaque cup and #1118 it coffee.
Fon: What ddes Carla believe the cup contains?
coffee
Bob, Carla, and Ton aeo Norking in busy restatrant. To complegs 522 ordes
022
bottle, and fills geabs another opaque cup and it
drabs
Oetpiel
with cotsep. Finally, Tom grabs anothez epaque tun and f1l1s
bese Than
000
19
20
30
it with tea.
Layer
Gueation: What
beliove bottle contains?
Answez:
Output:
Figure 47: Object OI in three character-objeectstate triples in Qwen2.5-14B 5-1uB-Insttuue
and Carla complete an order, Ton graba another opaque tun and fil's it with Then Bob
10-
Full resioual
Countrlattl
opaque bottle filla beer. Finally, Carla grabs another opaque cup and fills it coffee.
Intrvertion Accuayy
08
Question:
What does Carla believe the cup contains?
Answex:
coffee
lob, Cacla, and Tom are wocking in a busy restaurant. 2o complete an order, Bob grabs opaque bottle and filla
02
onggutl
srabs another opsque cup with coffee. Finally, foom grabe another opaque tun and fills
19
20
30
-0
it with tes.
Layer
Question: what does Carla believe the contains?
Answez:
Model Output: tea
Figure 48: Query Character OI in three character-objeet-state
triples in
Tom, Bob, Carla working busy restaurans. complete an order, Ton grabs another opaque tun and fills it then
1.0
Full residual
Couutorbacal
opaque bottle beaz. Finally, Cacla graba another opaque cup and #1118 it coffee.
Inteventonn Acanary
6x
Questiion: what doea Carla belleve the cup contains?
Anawer:
coffee
Bob, Carla, and Tom are working in busy restaurant. To bottle and fills it beaz. Than Carla grabs another opaque cup with coffee. Finally, Tom grabs anothes epaque tun and fil1s Orighaal
02-
complete ordes, Bob
0.8
10
20
30
it with tea.
Layen
Queation: What
believe cup contains?
Answer: unknown
Causal Model Output: tea
Figure 49: Query Object OI in three character-objjectstaate triples in
Qwen2. 1-1.5--11I-ntttie
Orighaal Countehheuul
Tom, Bob, and Carla are working busy restaurans. complete an order, Ton grabs another opaque tun and dills it wine. Than graba an opaque bottle
Intervention Accuracy
1.0
Answer Taylouc
it with
juice. Finally, Carla grabs another opaque and dills it milk.
Question: What does Carla believe the contains?
0.4
Anewer:
Bob, Carla, and Tom working in
restaurant. 20
0.2
comple order, Bob graba opaque bottle filla
with beer. Then Carla geabs another opsque cup and fille it
0.0
with offfee. Finally, Tom grabe another opaque tun and fills it with tea.
10
20
30
40
Layers
Cuestign:
believe cup contains?
Answer:
Interventio:: Answer Pointer (+), Cousal Model Uutpul: tea Intervontion: Answer Paylead Cousal Model Outptt:
Figure 50: Answer Lookback Pointer and Payload in three -haracter-objeetsstate triples in Qwen2.
38


Reviews:
[{"review_id": "9xF0G4jYUG", "reviewer": "Reviewer_dpdh", "summary": "This paper asks a concrete mechanistic question pertaining to ToM: how transformers store, update, and retrieve characters and their states. The dataset is CausalToM, a toy two‑sentence story set with two characters, two objects (containers), and two object states (contents). Each example concludes with a question, such as “What does Bob believe the bottle contains?”, accompanied by optional visibility statements that specify who can observe whom. The authors analyze Llama‑3‑70B‑Instruct using interchange interventions on residual activations (i.e., patching counterfactual activations) to observe how the model’s behavior changes.\n\nThe central finding is a pervasive lookback mechanism. The model “writes” tags (my terminology; referred to as OIs in the paper) for the “first/second” character, object, and state into the residual stream. Further, state tags are bound to the appropriate character/object tags at the state token, and the model then “looks back” from the answer position to retrieve (i) the right state tag and then (ii) the actual state token via attention. The lookback is localized layer-wise (e.g., tags form approximately layers 20–34; binding occurs at the state token, approximately layers 33–38). Figures 1, 3, 4, and 7 illustrate the pointer/address/payload flow and the three lookbacks.", "questions": "Please see weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, the results are surprising and insightful. The analysis is not hand-wavy.\n\n* Strong causal methodology. The authors use interchange interventions (activation patching) with carefully matched counterfactual stories to manipulate specific internal variables and measure IIA layer by layer. For example, patching the final “Answer:” token at mid layers redirects the answer pointer (layers 34–52), whereas patching late layers swaps the answer payload (at layers> 56).\n\n\n\n* Careful dataset design for causal analysis. CausalToM is deliberately simple (two characters/objects/states), so counterfactuals differ in only one factor at a time.", "weaknesses": "1. Analysis restricted to successful cases. All mechanistic experiments are run on 80 correctly answered examples. This risks selection bias: we only study the circuit when it worked. What happens for incorrect cases?\n\n\n2. Scaling beyond “first/second” is unclear. tags/OIs encode first vs. second character/object/state, which is perfect for this dataset, but what happens as you scale up?\n\nA small toy study (e.g., 3+ entities per type) would help address both weaknesses by revealing failure modes and testing whether the mechanism extends beyond binary order.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TeKTrJjzyF", "reviewer": "Reviewer_AD4K", "summary": "This paper investigates the internal mechanisms by which LLMs track characters' beliefs in ToM tasks.  The authors construct CausalToM, employ causal mediation analysis and causal abstraction to identify systematic computational patterns. Three specific lookback mechanisms are identified: (1) binding lookback that links character-object-state triples via ordering IDs, (2) answer lookback that retrieves state token values, and (3) visibility lookback that updates beliefs based on character observability. The mechanisms are validated through interchange intervention experiments on Llama-3-70B-Instruct and Llama-3.1-405B-Instruct models.", "questions": "See Weakness.\n\nThe models used by the authors are 70B and 405B. Would the same phenomena described in the paper also appear if the model size were around 7–32B?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "Unlike previous works in the Theory of Mind (ToM) domain, such as prompt-based (Think twice, TimeToM), tool-based (Social world model), or model-based approaches (Bayesian framework), this paper analyzes the model’s belief reasoning ability from a novel and interpretable perspective. In ToM research, there has long been debate over whether models’ ToM abilities are truly robust, and whether a correct answer to a ToM question genuinely reflects capabiltiy level. Analyzing this issue from the viewpoint of interpretability offers a promising path toward resolving this controversy. The paper presents excellent visualizations and provides a clear description of the research background.", "weaknesses": "The data pattern of CausalToM mentioned in the paper is quite simple.\n\nTheory of Mind (ToM) is a broad framework encompassing various dimensions of mental states, and its scenarios are often diverse and complex. The interpretability analysis in this paper is applied only to a narrow data scope (simple story settings and the belief dimension). When the data scenarios become more complex (e.g., longer narratives or richer social contexts), can this method still maintain good scalability and generalization?\n\nMoreover, the interpretability analysis is conducted only on the LLaMA series models. Will these observed phenomena also appear in other model families?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qa4Wn0ac3h", "reviewer": "Reviewer_zFrv", "summary": "The paper claims evidence for a specific mechanism, the \"lookback\", that allows a transformer architecture to dereference information about an entity in a sequence of tokens that presents a Theory of Mind problem, so as to correctly answer queries about what the entity knows. The paper's central claim is that the encoding of such information in the residual stream is based on the relative ordering of tokens corresponding to entities in a set of relational statements (first character vs. second character, first object vs. second object), not on structures that encode information as attributes of the entity's identity. The authors present a benchmark dataset (CausalToM) to analyze a model's ability to simulate ToM, and show through causal interventions in the layer-by-layer evolution of the residual stream that ordinal position (encoded as \"Ordering IDs\"), as opposed to identity, is how the LM manages information to correctly answer ToM queries.", "questions": "Does reducing ToM to positional bookkeeping (lookup mechanism + Ordering IDs) suggest sophisticated behavioral mimicry (à la Searle's Chinese Room thought experiment) rather than understanding or intentionality? What additional evidence would demonstrate conceptual understanding beyond structural pattern extraction? Are the Ordering IDs grounded in meaningful semantic relations about information access and belief formation, or are they arbitrary indices that correlate with correct answers in this constrained task structure?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "The paper addresses an extremely challenging and important problem in understanding the internal mechanisms by which language models perform Theory of Mind reasoning. \n\nThe methodology used to address this question is very clearly laid out, and to this reviewer's mind well motivated. The paper provides clear and useful graphical presentations of the mechanism proposed and the results of the interchange interventions in both the no-visibility and visibility cases. The contribution of a structured dataset of simple stories to provide a way to effectively elicit interpretable responses to confirm or disconfirm the Ordering IDs hypothesis is also an important contribution. \n\nThe findings demonstrate consistency across models in a given model family (Meta Llama) at multiple sizes (70B and 405B), and preliminary evidence suggests the mechanism generalizes to more naturalistic scenarios (i.e., the BigToM benchmark), strengthening confidence in the robustness of the identified patterns.", "weaknesses": "The paper would be significantly strengthened by addressing the following issues related to soundness and presentation. This reviewer sincerely hopes that these can be satisfactorily addressed in the rebuttal phase. \n\n1) The paper's central claim that models use Ordering IDs rather than identity-based or semantic representations is not adequately distinguished from plausible alternatives. There is only the briefest mention of prior work, and it assumes a great deal of familiarity from the reader with what appears to be a very specific body of work. The paper does not explain how the Ordering ID hypothesis was developed, what alternatives were considered, or whether there was exploratory analysis not reported. This makes it very difficult to assess the proposed mechanism in the context of work in mechanistic interpretability to date. \n\n2) The paper lacks crucial information about computational requirements, experimental iteration (how many analyses were attempted before arriving at reported results), and robustness checks (sensitivity to hyperparameters, sample selection, random seeds). Code availability is not mentioned. \n\n3) The paper uses specialized mechanistic interpretability terminology (\"residual stream,\" \"QK-circuit,\" \"OV-circuit\") without adequate definition, assuming familiarity that may not be universal even among the ICLR community. While core concepts like interchange interventions are explained well, a brief background section defining key architectural concepts would significantly improve accessibility and aid reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a41iwax6n0", "reviewer": "Reviewer_Jrau", "summary": "n/a - see ethics review", "questions": "n/a - see ethics review", "limitations": "", "rating": null, "confidence": 1, "soundness": 1, "presentation": 1, "contribution": 1, "strengths": "n/a - see ethics review", "weaknesses": "n/a - see ethics review", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)