Characterizing and Mitigating Reasoning Drift in Large Language Models
Authors: Yufeng Zhang
Xuepeng Wang
Lingxiang Wu
Jinqiao Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Steering Vector
LLMs Reasoning
Inference-time Scaling
Primary area: interpretability and explainable AI

Abstract: While chain-of-thought prompting enables powerful multi-step reasoning in Large Language Models (LLMs), the stochastic nature of the generation process undermines its reliability. In this work, we first analyze thousands of reasoning paths to identify Reasoning Drift, a key failure mode where models get locked into flawed reasoning patterns. We reveal that the manifestation of drift is a complex interplay between universal functional tendencies and unique, model-specific signatures. Based on the diagnosis, we propose Reasoning-Aware Activation Steering, a novel inference-time intervention method to gently nudge the model's activations away from pathological patterns. We pre-compute a library of vectors from contrastive functional transitions and apply them dynamically. Experiments show that our method effectively mitigates the drift problem and boosts accuracy. Additionally, it generalizes to out-of-distribution tasks, demonstrating a deeper capture of valid reasoning principles.

Published as a conference paper at ICLR 2026
CHARACTERIZING AND MITIGATING REASONING DRIFT IN LARGE LANGUAGE MODELS
Yufeng Zhang' Xuepeng Wangh>; Lingxiang Wu' Jinqiao Wang
"Intitute of Automation, Chinese Academy of Sciences
Sccool of Artificial Intelligence, University of Chinese Academy of Sciences WWhhnn AI Research
(yufeng. zhang, xuepeng. w dia.@aaa.e aclaa.cccr [1 llinxxiang wu, jqwang) [00een1ppppr iaiia.aae ac.ac.om
ABSTRACT
While chain-of-thought prompting enables powerful multi-step reasoning in Large Language Models (LLMs), the stochastic nature of the generation process under- mines its reliability. In this work, we first analyze thousands of reasoning paths identify Reasoning Drift, a key failure mode where models get locked into flawed reasoning patterns. We reveal that the manifestation of drift is a complex interplay between universal functional tendencies and unique, model-specific sig- natures. Based on the diagnosis, we propose Reasoning-Aware Activation Steer- ing, a novel inference-time intervention method to gently nudge the model's 's ac- tivations away from pathological patterns. We pre-compute library of vectors from contrastive functional transitions and apply them dynamically. Experiments show that method effectively mitigates the drift problem and boosts accuracy. Additionally, it generalizes to -iitr---iitiitttion tasks, demonstrating a deeper capture of valid reasoning principles.
INTRODUCTION
Recently, large language models (LLMs) have demonstrated consistently improving performance across wide range of reasoning tasks (Havrilla et al.) 2024 Plaat et al.
2024 Ke et al. 2025
Chen al., 2025). By generating multi-step chains of thought (CoT) (Wei et al.) 2022). LLMs
can provide explicit traces of their reasoning process. However, the generation of these thought
chains is inherently stochastic. growing body of research (Wang et al., 2023 Lightman et al.)
2024 Bigellw al. [2025] Bogdan et al. 2025) has shown that such randomness significantly affect the reliability of reasoning: simply replacing certain sentences, which represent discrete and
human-readable thought steps, along the reasoning path can lead to entirely different outcomes.
To mitigate the negative impact of stochasticity, several strategies have been proposed, such as self- consistency (Wang al. 2023 Wan et al. 2025 Taubenfeld et al. 2025) and path search (Yao et al.
2023 Xie et al. 2023; Leang et al. 2025). These methods typically rely on extensive sampling promising reasoning paths final answers. While these approaches have been shown to improve reasoning accuracy, they face two major limitations. First, they often incur substantial computational cost due heavy dependence on large-scale sampling and state exploration without clear guidance. Second, and more importantly, they treat LLMs largely as black boxes. By merely increasing trials rather than probing the internal mechanisms of reasoning, these methods offer limited insight into how LLMs actually reason, or why they sometimes fail.
and then selecting the
In this work, we aim uncover and analyze the deeper mechanisms underlying LLM reasoning. We begin with a fundamental question: why does resampling individual steps lead to such drastic differences in outcomes? Leveraging the recently released Math-Rollow dataset (Bogdan et al.) 2025), we systematically investigate how substituting different candidate sentences at intermediate steps influences the final success of reasoning, and further explore the underlying transition patterns that most strongly determine reliable outcomes.
"Corresoonding authors.

---

Published as a conference paper at ICLR 2026
Our analysis yields two primary discoveries. First, identify "funneling effect" in the rea- soning process: it begins a state high plasticity where outcomes are volatile, but early-stage choices rapidly solidify the trajectory, drastically reducing the potential for later-stage correction and largely determining the final outcome. Second, we diagnose the underlying mechanism for this phe- nomenon, failure mode term Reasoning Drift. We find that drift is random but is strongly correlated with specific, pathological shifts in the functional behavior between steps. For instance, models are prone drift when they prematurely enter plan generation without adequate problem setup, or exhibit overconfidence by systematically avoiding corrective uncertainty management.
To mitigate the reasoning errors stemming from this drift, we propose a novel intervention inspired by activation steering (Turnrr et al. 2023; Riimsky et al.) [2024] Arditi et al. 2024] Venhoff et al. 2025]. Our method, termed Reasoning-ANe Aware Activation Steering, designed interpretable, and targeted. By contrasting preferred and dispreferred functional shifts from the dataset, we extract a library of fine-grained steering vectors, each capable of gently nudging devi- ating reasoning process back onto productive path. During inference, we compute a real-time drift score to dynamically determine when and how to apply these vectors. Experiments demonstrate that method is both effective in improving reasoning reliability and broadly applicable across tasks.
lightweight,
Our contributions are thus threefold:
We conduct fine-grained analysis of the functional dynamics of reasoning trajectories. From this, we identify and characterize a critical failure mode we term reasoning drift, demonstrating that failures are systematically preceded by specific transitions.
We propose novel, reasoning aware activation steering method. Our approach is lightweight, interpretable, and targeted. It detects the drift likelihood and applies pre- computed steering vectors gently guide the model's activations away from failure modes.
Experimental validation shows that our method can mitigate the drift problem and improve reasoning accuracy. Furthermore, it is generalizable to out-of-distribution datasets.
2 ANALYSIS OF STEP-BY-STEP REASONING
Reasoning in LLMs is fundamentally a neneettemiiiistic process. This inherent stochasticity, while a source of creativity, also undermines their reliability, as a model can generate both valid and fallacious reasoning paths for the very same problem (Wang et al.) 2023 Lightman et al., 2024 Bigelow et al. 2025: Bogdan et al. 2025). Our investigation begins with a fundamental question: why merely resampling a single sentence in reasoning chain can flip the final outcome from correct to incorrect, or vice versa. To quantify this phenomenon, we first establish an analytical framework.
2.1 PRELIMINARY DATASET
Probing the intricacies of model reasoning requires fine-grained comparative analysis of success- ful versus failed trajectories. Such a framework is essential to pinpoint precisely where and how an LLM's thought process diverges under its inherent stochasticity. For this purpose, we leverage the recently proposed Math-Rollout dataset (Bogdan et al. (025)) as an analytical testbed. This dataset provides a curated collection of 40 correct and 40 incorrect solutions mathematical problems, gen- erated by the DeepSeek-RR -1-eeeckk----iitiii D-isstill-Llama-BB and DeepSeek-RR R1-Distill- Setill-Qwen-leee 11B models. Each solution is decomposed into sentence- level reasoning steps.
Crucially, each reasoning step is accompanied by 100 stochastic rollouts (i.c. alternative paths the model could have taken) annotated with their final outcome. This provides rich, counterfactual view into the model's decision space at each step. Furthermore, each step is annotated with one of eight functional categories to delineate its role in the problem-solvine
process: Problem Setup, Plan Generation. Fact Retrieval, Active Computation, Result Consolidation, Uncertainty Manage- ment, Self Checking, and Final Answer Emission. Details of the dataset are provided in Appendix C While the dataset provides this powerful lens, the dynamics of why and how models navigate these steps successfully or fall into failure modes remain unexplored. This is the central investiga- tion of our work.
2

---

Published as a conference paper at ICLR 2026
Analyyis TWo Models
Divergent Sample
Identical Sample
08-
Polarity Changed
Proportions
0.-
68%
76%
88%
Polarity Stable
DeepSeak-R1-
9.3
Liama
DeepSeekk R--
Owen-148
40%
47%
92%
15%
Rulloul Senteriee
Early SLage
Mid Stage
Late Slage
Figure 1: Analysis of rollout stability. The leftmost group shows the proportion of identical versus divergent samples after resampling from the original sentence. The three rightmost groups show the outcome polarity flip rate at the early, mid, and late stages of the reasoning process. For example, for DeepSeek-RR -arecc--R--iisill--laaaa
while only 5% of samples are identical to the original, 48% of resamples in the early stage lead to flip in the final outcome's polarity.
2.2 PROBING REASONING INSTABILITY
We define the Outcome Polarity Flip Rate the probability that rollout's final outcome (success or failure) differs from the original reasoning path's outcome. We then segment the reasoning process into three distinct phases analyze its temporal dynamics: the Early Stage (the first third of steps), the Mid Stage (the middle third), and the Late Stage (the final third).
As illustrated in Figure our first observation is the inherent stochasticity of the models: rollouts are textually identical to the original sentence in only mere 5%~7% of cases. For the vast majority (293%) of rollouts that introduce textual variations, the impact on the final outcome is profound. The Outcome Polarity Flip Rate in the Early Stage is exceptionally high, exceeding 47% for both models. This rate diminishes approximately 30% in the Mid Stage and continues fall thereafter.
This demonstrates a "funneling effectt in the reasoning process. The initial steps are highly plastic and serve as critical inflection points, where even slight variations in expression can have massive, almost foin-flip-like impact on the final trajectory. As reasoning progresses, the path solid- ifies and becomes increasingly deterministic. While this resonates with the micro-level observations Ji et al. 2225)., our analysis highlights this phenomenon at more systemic and global scale. This raises the crucial next question: Are these influential, early-stage variations simply random noise, or do they represent meaningful transitions in the model's underlying reasoning pattern?
2.3 FUNCTIONAL DYNAMICS OF TRANSITIONS
To answer the question posed above, we now shift our focus from textual variation to the underlying transitions in reason- ing patterns. We seek to understand how the functional role of a rollout sentence (at step t) diverges from the role of the preceding step that prompted it (at step f - 1). Our goal is determine how these specific one-step transitions correlate with the eventual success or failure of the reasoning chain.
promising step
S₁t
rale
lower SUDOEES
Erroncous
unpromising step
To enable large-scale, automated analysis of these dynamics, we trained a robust DistilBERT classifier (Sanh et al, (019) on the annotated functional categories from the Math-Rollout dataset (see Appendix Dor training details and performance).
lower suscesa
Higher success rate
We then applied this classifier to annotate every rollout sen- tence with its corresponding functional label. This process transformed our raw text data into structured sequence of steps, enabling
conrective rollouts
construct comprehensive transition ma- trix that quantifies the tendency for the model to move from one functional step to another.
Figure Illustration of different rollouts and steps.
3

---

Published as a conference paper at ICLR 2026
Difference between erroneous drifts and corrective revis
0u4suon
600
a.s
0.00
p03
0.00
smip
600
2.46
668
2.74
4.34
8.00
From Calegery (Step [-1]
10
0eserotcn
Bact
Aerioval
Actvv
5:35
22.99
0.41
Poadt
1.43
3.34
2.71
2.08
1.76
-0
uncerta ligier
0.02
307
12.28
17:47
27.04
9.84
11.04
6.42
146
Chocking
Sclf
0.02
0.28
3.55
1.07
0.85
3.78
-.25
0.78
2.74
Fronecn
0.00
0.00
0.09
2:48
0.33
-0.0
-.00
132
4.01
n8a 0005028
hobilmm Scoip
Ran ocneraocn
PUCE NET1050
hosut CaRsal
Uncortairty Mgm
SuT checkigg
hid ASGACE smisse
computadon
To Category (Step
DeepSeek R-- RI- -isstill -aaill--aam---- 8B
Difference between erroneous drifts and corrective revis ons
I101
0.50
0/7
9.11
0.04
0/00
2/6
0.9
9.00
Sclap
0.22
1.00
222
2.74
1.54
0.29
1.30
043
3.00
From Category (Step E-1)
Gaseranion
Ran
0.0
137
-734
4.30
433
3.30
4.37
141
4.64
Neciowal
0.00
0.64
16.76
1124
7.35
6.44
4.09
Compulation
-0
ConGo.
Lncert rty
checking
Sclf
0.22
0.00
Frad Zrowne
0.32
5.0
hublon
Pun
Nul
Aulice
houk
Se
Ard Anynu
To Category (Step
(b)
Figure 3: Heatmap of step transitions contrasting erroneous drifts and corrective revisions. The ma- trix constructed by subtracting the frequency corrective revisions (S,-1 + T, St unpromis- ing. T; = corrective) from that of erroneous drifts (S,-1 + Te S promising, T; =erroneous). Red cells indicate transitions that are more frequent in failed reasoning paths, whereas blue cells highlight transitions more common in successful reasoning paths. The neon blue and purple out- lines highlight, for each row, the minimum and maximum values, respectively.
Notably, to fully capture the entire reasoning trajectory, we expand the set of eight functional cate- gories (from Section 2. I to nine by introducing a special-purpose "zeroth" category: Initial Ques- tion. This serves as the universal starting point for every reasoning chain, enabling our transition analysis to model the critical move from the problem prompt to the opening line of reasoning.
We now introduce the diagnostic framework used assess the health of each transition within it. Formally, a reasoning path consists of intermediate steps S S2.... S.-1, St,..., SM with final outcome As,- As shown in Figure 2 rollout from step S, means replacing S, with candidate Tę and completing the reasoning until the final answer St. S2.... S.-1. Te. A's Each rollout is evaluated by whether the resulting outcome is successful not:
Corrective Rollout: A rollout sentence T, corrective (green nodes with green borders in Figure 2) if the reasoning path it produces leads to successful final outcome.
Erroneous Rollout: A rollout sentence T; is erroncous (green nodes with red borders in Figure 2 if it leads to failed final outcome.
4

---

Published as a conference paper at ICLR 2026
Then, define the potential of St (blue nodes in Figure 2 by comparing the success rate of keeping St versus replacing it with rollouts Te:
Unpromising Step: A step S, is unpromising if, on average, its rollouts achieve higher success rate than keeping S. this case, resampling from S, beneficial.
Promising Step: Conversely, St is promising if keeping it achieves higher success rate than resampling. In this case, resampling is detrimental.
This diagnostic lens allows us to isolate the most telling events in a reasoning trajectory. Our analysis will concentrate on two opposing dynamics: a) the "the correction scenario." where we study the state transition from a preceding step S to a corrective rollout T, that remedies an impromising step St; and b) the "the drift scenario." where we study the transitions from S that a promising step St derails into an erroneous rollout T₁.
2.4 REASONING DRIFT PHENOMENON
Figure 3 visualizes the heatmap of our difference matrix (Drift Matrix Correction Matrix), for which the individual matrices are provided in Appendix [E] Here, hotter colors signify pathological transitions strongly associated with failure, and cooler colors denote salutary transitions character- istic of correction. This diagnostic map reveals several critical insights into various failure modes, which we collectively refer as Reasoning Drift.
First, at the very beginning of the process, we observe shared failure archetype. The transition from the initial question to plan generation that models often fail by rushing to solution without sufficient problem setup. Furthermore, we find a universal characteristic of failure the avoidance of the uncertainty management step. The strong negative values for all transitions into this step imply that entering step of self-reflection is a powerful corrective mechanism, one that failing trajectories systematically bypass in a display of erroncous overconfidence.
strong indicator of an erroneous path, suggesting
Most strikingly, the heatmap reveals unique, model-specific drift signatures. Llama's 's drift is pre- dominantly inertial: it manifests seen the strong positive values along the matrix diagonal. We term this Inertial Drift. Qwen, conversely, exhibits more chaotic form of drift. Its failure is not characterized by getting stuck, but by making erratic, non-linear jumps, often backtracking to previous functional states. This pattern, visible in the hotter lower triangle of its matrix, we classify as Chaotic Drift.
strong tendency to get stuck in unproductive self- as
In summary, our analysis reveals that reasoning drift is not monolithic phenomenon but manifests in model-specific signatures. Having diagnosed these distinct pathological dynamics, the critical question becomes: can we develop targeted intervention to actively steer the model away from these failure modes and guide back towards a productive reasoning path?
PROPOSED METHOD: REASONING-AWARE ACTIVATION STEERING
To address the challenge outlined above, we propose a novel, inference-time intervention method: Seasoning-Awarg Activation Steering (RAAS). Prior work has demonstrated the efficacy of acti- vation steering across various domains (Subramani et al. 2022: Turner et al. 2023 Rimsky et al., 2024 Han et al. 2024 Arditi et al.) 2024), yet its application to the nuanced, multi-step nature of reasoning remains underexplored. Building upon our fine- mme-grained analysis of reasoning behaviors, adapt this technique to intervene in the subtle and complex transitions that determine reasoning outcomes.
Our method is designed to act as a corrective nudge in the model's activation space, precisely when it begins to exhibit the transition patterns characteristic of a potential drift. Unlike fine-tuning ap- proaches that require extensive retraining, inference-tin -ereeneee-meeeeeee methods like self-consistency and path search that rely on massive sampling, our method offers a lightweight, interpretable, and targeted way enhance reasoning robustness.
5

---

Published as a conference paper at ICLR 2026
Pre-compute Steering Vector Library
previbus stop
col SV Library
Functional Prototypes
activation subtraction
S
preterred step
vector, ..
functional category
row !
aclivation average
dispreferred step ffunctional category /)
Dvift Adpstive Intervention
previous step
current generation
vector sorrection
0.1
0.6
0.2
Biokon
Bupcen
functional category
unknown category
8"gg
B
Figure 4: The overall architecture of reasoning- aware activation steering.
3.1 CONSTRUCTING CONTRASTIVE PAIRS
Our method is based on learning a directional vector from contrastive pairs. Formally, we construct dataset where each entry represents a single reasoning choice, containing shared context Set, preferred next step S+ and a dispreferred next step S The assignment of these roles is determined by the two critical scenarios identified our analysis (Section
The Correction Scenario: This oceurs when original step S. unpromising. this case, goal is to steer the model towards better path. We therefore assign the corrective rollout T; as the preferred next step (S*+), ), and the original unpromising step S, itself as the disprefemed next step )
The Drift Scenario:
This occurs when the original step S, is promising. Here, our goal is steer the model away from the erroneous path drifted into. We therefore assign the promising step S; as the preferred next step (S+), and the erroneous rollout T, as the dispreferred next step (S
Consequently, every entry our final dataset D is triplet of the form (Scr, This structure allows us systematically learn the vector differences between desirable and undesirable reasoning paths, always pointing from a less-pree eeseepreer step to a more- pore-preferred one.
3.2 EXTRACTING THE BEHAVIORAL STEERING VECTORS
Next, we proceed pre-compute a comprehensive library of fine-grained steering vectors, each tailored to counteract specific type of pathological functional transition. This library can be con- ceptualized matrix of vectors, where an element V/=z represents the corrective vector for transition from step of category to an undesirable step of category j. It is crucial to clarify that the steering vector nj-ij is not designed prevent all transitions from category to j. as many such transitions are valid and essential for correct reasoning. Instead, we aim to correct only the specific instances that transition that are showing signs of drifting.
We then iterate through all possible transition pairs (i,j) found in our dataset, where is one of the 9 source categories (defined Section 2.1 and]]33 and j is one (excluding initial question). For each pair, create corresponding subset of data, Di-j. which contains triplets where S,-1 has the label i and S has the label j.
8 possible destination categories
The steering vector U-4j is then computed by averaging the difference between the preferred and dispreferred activations within this specific subset:
U = E(s+ Hist,s-)en, Jen [act((S*) ac](S-)
(1)
where act(S) is the activation representation of sentence S. We define this as the mean-powt moolee vector of its token activations, ar, from the I-th transformer layer:

---

Published as a conference paper at ICLR 2026
18|
act( S) -
ar. (token:)
(2)
S
This systematic process yields library of up to 9 72 unique steering vectors. Each vector, therefore, acts as a gentle nudge to guide a potentially pathological instance of a transition back towards its salutary counterpart. Details of intervention layer selection are in Appendix
3.3 REAL-TIME DRIFT DETECTION
Armed with our library of fine-grained steering vectors, we face the challenge of applying them during sequential generation. While we can classify the preceding step St-1, the functional category of the current, partially-gene geenertdd step S. is unknown. Backtracking or lookahead mechanisms are computationally prohibitive.
To overcome this, we propose Real-time Drift Score, a real-time metric that predicts the model's instantaneous tendency drift towards specific pathological function. To compute this score, first pre-calculate set of Functional Prototypes from our dataset D. The pathological prototype (a,;) the average activation of all failing examples, while the salutary prototype (a,,) is the average of all successful ones:
a = (..S-) ,y |act a = ((st))en, 1y [aac((S*)]
(3)
During inference, the model generates each token for the current step S,, we extract its activation as Awken- The real-time drift score is then formulated to be non-zero only when the token's 's activation demonstrably closer pathological prototype than its salutary counterpart. Formally, the score for each potential destination category j is:
DriftScore(t.j)) (cosine 2.j) cosine ajnken; at,)) cosine Aroken a..))
(4)
where I(-) ) is the indicator function, which returns if the condition is met and otherwise, The score is therefore zero if the token is closer to the "good" prototype. If it is closer to the "bad" prototype, the score's magnitude equal its cosine similarity to that pathological prototype () A higher DriftScore(i..j) thus provides stronger signal that the current generation not only directionally incorrect but also strongly aligned with known failure mode
3.4 INFERENCE TIME ADAPTIVE STEERING
Finally, we integrate our pre-computed components (the steering vector library and the functional prototypes) into a cohesive online algorithm for inference-ting steering. This algorithm provides continuous, fine-graine course correction throughout the generation process.
The process is executed at every token generation step (Wang al. 2024). First, take the completed previous step, St-1 (or the initial question prompt if f 1), and apply our pre-trained classifier (Section 2.3) to determine its functional category, i. Then, using the current token's tivation, we compute the DriftScore(i,5) for all 8 potential destination categories j. as defined in Section 3.3 This score quantifies the model's instantaneous tendency drift towards each patho- logical category. Next, we compute dynamic, context- correction vector, Pconection. by taking a weighted sum of all steering vectors originating from the current context The weights for this sum are the real-time drift scores themselves:
Venmection
DriftScore(i.j)) Ue-oj
(5)
This aggregated correction vector is added to the original token activation to guide the generation:
Tokon Byoken + Ceorrcction
7

---

Published as a conference paper at ICLR 2026
Differences
transition original transition (without Intervention)
Gaeetion
Proflem
0.00
0.53
1.56
2.39
1.89
9.49
962
031
0.1
frem Category (Step t:11)
0
Genoration
0.00
0.95
29.95
17.90
21.99
2.14
4.1
2.00
1.33
10
Reitteval
Actioe
1.42
4.25
2.15
33.00
227
20
Sesut Consol
0.00
2.38
6.05
11.51
0.69
15.45
9.00
13.09
2.13
50
Unicertanty
Hggnt
Cleckking
2./4
5.03
3.13
-30
0.00
0.33
0.14
0.03
0.30
0.17
0.02
4.40
0.33
Probler Serip
Plan Ceneration
Tect Berreual
Active
Datui Connell
Uncerbenyy Hy
Sal thecking
Fiaal Freedonn
Queetio
Corpp
To Category (Step t)
Figure 5: Heatmap of the difference matrix, calculated by subtracting the original transition matrix (without intervention) from the transition matrix of our steered model on RI-Distill-Llam--8 on GSM8K. Positive values (hotter colors indicate transitions that our method promotes, while negative values (cooler colors) signify transitions it suppresses.
4
EXPERIMENTS
In this section, we conduct series of experiments to empirically validate the effectiveness of our proposed online steering method improving reasoning capabilities.
4.1 EXPERIMENTAL SETUP
For our primary experiments, we employ the same base model used for vector extraction, RI-Distill- Llama-80 8B and R1-Distill-QU Siiill--wen--la 14B. To assess the method's 's performance, we evaluate it on suite of challenging mathematical reasoning benchmarks similar in nature to the Math-Rollout dataset: GSM8K (Cobbe et al.] (021). AIME2024 (MAA) (024). AIME2025 (OpenCompass, (025), and
Swen-11B.
GPQA-Diamond (Rein et al. 2024). Details of implemention are in Appendix G Our evaluation structured around the following key research questions (RQs):
RQ1 (Mechanistic Validation): How effectively does method mitigate the specific patterns of the drift problem identified in our analysis?
RQ2 (Performance Impact): Does the mitigation of reasoning drift translate into tangible improvements in final task accuracy?
RQ3 (Generalization): Is our proposed framework for extracting and applying steering vectors a general approach applicable other base models?
4.2 MITIGATION OF DRIFT (RQ1)
To answer RQ1, we first investigate the mechanistic impact of our steering method on the model's reasoning process. Figure presents a heatmap of the difference matrix, calculated by subtracting the original transition matrix (without intervention) from the transition matrix of our steered model on Llama. Results for Qwen are in Appendix H For each problem, we normalize the transitions by type according to their total number within that problem. Positive values (hotter colors) indicate transitions that our method promotes, while negative values (cooler colors) signify transitions suppresses.
The results provide a clear affirmative answer. At the very start of the process, our method acts crucial guardrail, discouraging the model from prematurely jumping from the initial question into plan generation and instead promoting a more deliberate problem setup. Furthermore. the intervention consistently boosts the likelihood of entering the uncertainty management step from all
8

---

Published as a conference paper at ICLR 2026
Table Comparison results (%) across different reasoning tasks. Methods with denote the results are copied from Leang et al. 2025). while those without are reproduced.
Models
Methods
GSM8K
AIME2024
AIME2025
GPQA-
Vanilla"
37.5((11.52) 73.67(10.32)
29.63(10.37)
42.87(+:1.07)
SC
74.01(11077)
38.89(11.67)
25.00(10.37)
42.10(+1..77)
PICSAR
76.42(11.116
47.78(14.01)
33.33(11.1))
47.31(10.17)
R1-Distill-Llame
Vanilla
82.45(+00778)
34.99(#1.67)
25.56(#4.44)
50.50(
CAAum
85.06(+00.6)
45.00(#3.49)
30.00(#2.35)
52.52(
Ours
87.56(+0.30)
55,56(#1149)
31.70(#1.66)
55.52(+0.90)
Vanilla
93.69()00988
54.43(#2.23)
26.67(#3.33)
55.55(+1.01)
RI-Distill D1-Distil-QQwen (wven-14B
CAAum
95.36(+0.38)
53.33(#2.35)
28.89(#2.33)
52.02(++1.63)
Ours
95.60(+10:22
62.49(#1.66)
36.67(
57.57(
100
80
Performance
60
40
20
GSMBK
AIME2024
AIME2025
GPOA-Diamend
Target
Fully Random
Figure Ablation study of steering variants. While partially mismatched vectors (Random Source/Target) offer marginal regularization benefits over the vanilla baseline, fully random inter- ventions severely disrupt reasoning, highlighting the necessity of functional context.
other steps. This demonstrates significant reduction in the model's overconfidence, forcing it to adopt more reflective and cautious reasoning style, benefit that echoes the findings of [Bogdan al. (2025). The heatmap also provides direct evidence of model-specific drift mitigation. While strong inertial patterns persist in the late stages of reasoning (the bottom-right of the matrix remains hot), we see a marked reduction the pathological self-transition loops the early-to-mid stages (cooler diagonals in the top-left and middle). This demonstrates that our steering mechanism is most effective during the early, eeastpplttictty phase.
4.3 RESULT COMPARISON (RQ2)
To answer RQ2, we evaluate our approach against three representative baselines: vanilla CoT, Self- Consistency (SC) (Wang al. 2023), and the PiCSAR (Leang al.) (025) path-selection method. Additionally, we conducted an experiment where we exclusively selected the single vector (Rimsky al. 2024) associated with uncertainty management to investigate whether it could mitigate the model drift caused by insufficient self-reflection, termed CAAum.
Table summarizes the accuracy of all methods across several challenging mathematical reason- ing datasets analogous to Math-Rollout. Overall, the results show that our method consistently improves the reasoning accuracy. The performance gains are particularly pronounced on highly complex problem sets like AIME 2024 and AIME 2025, which demand sophisticated, multi-step logical deduction. Notably, these evaluation datasets are out----distributtion (OOD) within the math domain with respect the data used extract the steering vectors. They demonstrate that fine- grained functional vectors are not merely overfitting to the source distribution but have captured generalizable, fundamental principles of logical reasoning.
To further validate whether our method's success stems from generic activation perturbations, we conduct targeted ablation study. Specifically, when the model is in step and exhibits drift towards j (where the optimal intervention is vi-z)), we intentionally mismatch the contextual components. We introduce three randomized steering variants: I) Random Source (Frand Á) that applies

---

Published as a conference paper at ICLR 2026
vector from randomly sampled source category fraud; 2) Random Target (t + Frand) that applies vector from a randomly sampled target category Jrand: 3) Fully Random (frand + frand) that applies randomly selected vector from the library.
Figure 6 compares the three randomized variants against baselines. Overall, our proposed steering consistently achieves the highest reasoning accuracy. Interestingly, the Random Source and Random Target variants yield marginal improvements over the vanilla baseline. We infer that since all vec- tors are derived from the difference between preferred and dispreferred activations, they inherently encode a general corrective direction. Applying them acts as a beneficial test-time regularization. Conversely, the Fully Random variant severely degrades performance. These results confirm that while a generic corrective nudge offers slight benefits, precise context mapping (i - j) is essential for maximizing reasoning improvements.
4.4 APPLICATION TO OTHER MODELS (RQ3)
Table 2: Results on other models.
To answer RQ3, we directly reuse the vectors learned on the distilled models and apply them to the corresponding non-distilled base models. We are interested in what happens under this zero-retraining transfer scenario. Table 2 presents the performance of our method when applied to Llama 3.1-8B and Qwen 2.5-14B, prior their task-specific distilla- tion. While the performance improvements are smaller compared to our primary model, the method consistently applicable and produces positive effects across all tested models. These results indi- cate that our framework is generally applicable, though the magnitude of improvement depends on model-specifis characteristics such as reasoning capacity and pretraining data.
Model
Method
GSM8K
GPQA-D
Llama3.1-8B
Vanilla
83,91
28.28
Ours
85.22
29.10
Qwen2.5-14B
Vanilla
83.77
34.27
Ours
35.18
RELATED WORK
Chain-of-Thoughh and Advanced Reasoning Models. Chain-of-Thought (CoT) prompting has become the de facto standard for eliciting multi-step reasoning in Large Language Models (LLMs) (Wei et al. 2022). Building upon this, new generation of models exhibiting advanced "thinking" capabilities, such as DeepSeek-RI (Guo et al.) 2025), OpenAl's o3 (o3) 2025), and Anthropic's Claude series (Anthropic 2025). have emerged. The models central our analysis, RI-distill- Ilama- 8B and R R1-distill-qwer 14B are direct distillations from the powerful DeepSeek- R1, inheriting its strong reasoning faculties and making them ideal subjects for our study.
Addressing Stochasticity in Reasoning. Despite these advances two dominant strategies exist: sampling-base methods like self-consistency (Wang et al. 2023] Wan et al. 2025 (Tubeeneldd et al. 2025). which rely on majority voting, and search-based based methods like Tree-of-Thoughtt (Yao
cct al.
2023). ) beam search (Xie et al., 2023), and PICSAR (Leang et al. (025), which explore and
prune multiple reasoning paths. Our work builds upon prior analyses of thinking models (Venhoff
ct al.
2025 Bogdan et al. 2025).
Activation Steering for Process Intervention. We draw inspiration from activation steering 2023; Rimskk et al.) 2024| Arditi et al. 2024: Venhoff et al., 2025). a technique that modifies model behavior by adding a steering vector to its internal activations during inference. This allows us to guide the model away from undesirable states and towards desirable ones, enhancing reliability without the significant computational overhead of massive resampling.
(Turne et al.
CONCLUSION
This paper tackled the challenge of stochasticity in LLM reasoning through a diagnostic lens. By analyzing the functional dynamics of reasoning, we identified and characterized Reasoning Drift as primary failure mechanism. Our findings reveal that models drift into failure via distinct signa- tures. To combat these identified failures, we developed Reasoning-Adaan Adaative Activation Steering, targeted, inference-time method that demonstrates the effectiveness of process-orienter interven- tion. Our results show that this process-oriented approach not only enhances reasoning accuracy but also generalizes to different scenarios.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Key R&D Program China under Grant No. N0.2024FFC315500. National Natural Science Foundation of China (No. 62276260). We sin- cerely thank all reviewers and ACs for their insightful comments, time and efforts.
REPRODUCIBILITY STATEMENT
Our experiments rely on publicly available datasets (Math-Rollout, GSM8K, AIME benchmarks, and GPQA) and open-source models (Llama and Qwen series), all accessible via the Hugging Face Hub. All code, including data preprocessing, classifier training, vector extraction, and evaluation scripts, will be publicly released. Experiments were run with fixed random seeds, and key hyperpa- rameters and environment specifications are detailed in the Appendix.
REFERENCES
Anthropic. Bonnet
Claude 3.7 sonnet. 2025. URL httpss//ww..aattrrppc...ooo
Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by single direction. In The Thirty- eighth Arnual Conference on Newral Information Processing Systems, 2024. URL https:
Eric Bigelow, Ari Holtzman, Hidenori Tanaka, and Tomer Ullman. Forking paths in neural text generation. In The Thirteenth International Conference on Learning Representations, 2025, URL
https::/
net. meetr/forrummii-BBCCaeeeeeee
Paul C Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy, Thought anchors: Which Ilm reason- ing steps matter? arXiv preprint arXiv:2506. 19143, 2025.
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng. Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: survey of long chain-of- thought for reasoning large language models. arXiv preprint arXiv:2503.09567. 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110. 2021.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 2tt.:::2551199:44.: 12948, 2025.
Chi Han, Jialiang Xu, Manling Li, Yi Fung, Chenkai Sun, Nan Jiang, Tarek Abdelzaher, and Heng Ji. Word embeddings are steers for language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 16410-16438 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653w1120224.cllloogg https: antho
URL
8647|
Alex Havrilla, Sharath Raparthy, Christoforus Nalmpantis, Jane Dwivedi- Yu, Maksym Zhuravin- skyi, Eric Hambro, and Roberta Raileanu. Glore: When, where, and how to improve Ilm reasoning via global and local refinements. arXiv preprint arXiv:2402. 2024.
Ke Ji, Jiahao Xu, Tian Liang, Qiuzhi Liu, Zhiwei He, Xiaoyuan Liu, Xingyu Chen, Junying Chen, Benyou Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. The first few tokens are all you need: An efficient and effective unsupervised prefix fine-tuning method for reasoning models. In The Thirty ninth Anmual Conference on Neural Information Processing Systems, 2025. URL ht //openrevien ew.
d= 1CMF

---

Published as a conference paper at ICLR 2026
Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al. A survey of frontiers in Ilm reasoning: Inference scaling, learning to reason, and agentic systems. arXiv preprint arXiv:2504.09337, 2025.
Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema, Sohee Yang. Wai-Chung Kwan, Xuanli He, Wenda Li, Pasquale Minervini, Elconora Giunchiglia, and Shay B Cohen. Picsar: Probabilis- tic confidence selection and ranking. arXiv preprint 2Xiv:2558.22787. 2025.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2024.
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A chal- lenge dataset for machine reading comprehension with logical reasoning. In Christian Bessiere (ed.), Proceedings of the Twenty-Nimth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 1622-3628. International Joint Conferences on Artificial Intelligence Organiza- tion, 7 2020. doi: 10. 119663/jcai.2020/5500 URL htttps://cc.i ott.::..rrr/0....99333. org/10.2499633//....
2020/501 Main track.
MAA.
American
invitational
mathematics
examination
aime.
2024. inerican-invitatten mathemat -ow.emmtticcsseemmmniniiooo----- aime
URL
https: org math compettionntttinnns//
Neel Nanda.
Attribution patching:
Activation patching at industrial scale. 2023.
URL
https://wwt nee.nanda. htw.aa/w.....ccccaaaaaaaateeeeee
attribution. patttttiiinnptccinngn
OpenAl o3. Introducing openai o3 and 04-mini. 2025, URL http:pss//ppnna..commoooo
1atr-ddccinn-33--m----------
OpenCompass. Aime 2025 dataset. 2025, URL httpss//wuggingaacc..//aaattttt
(geencompass/A1E222
Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. Rea- soning with large language models, a survey. arXiv preprint arXiv:2407.11511, 2024.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. URU/attps://ppeneviii net Mert/Eorum?2 71
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 15504-15522, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.186633///2222.llllloo.g www.ooyy..rgg/2222 https::/acclanthoooog.. ww.. www.002....1lloooo 8287]
URL
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint 10Xiv:1910.01108. 2019.
Nishant Subramani, Nivedita Suresh, and Matthew Peters. Extracting latent steering vectors from pretrained language models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics: ACL 2022, pp. 566-581, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 00.18653/v1/2022 findings-acl.de
httpppp://caaattlllooo.o.oo 020/2022 http:///22....aiggaaaaaaaeaeeeea findings-aclla 488 ottp://attttoooo.o.ooooo22222
Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. arXiv preprint arXiv:2310. 2023.
12

---

Published as a conference paper at ICLR 2026
Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves ceffcconsitency in LLMs. In Wanxiang Che, Joyce Nabende, Eka- terina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Compu- tational Linguisties: ACI. 2025, pp. 20090-20111, Vienna, Austria, July 2025. Association for Computational I. Linguistics. ISBN 979-9-999----7 doi: 10. 00.185331112025. www..iinnnnnaa....... httthtttss//accce hattps:://clannthoooyy.
fifdiiignggs. acl. 1030/
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering, arXiv preprint arXiv:2308. 10248, 2023.
Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Commy, and Neel Nanda. Understanding reasoning in thinking language models via steering vectors. In Workshop on Reasoning and Planning for Large Language Models, 2025. URL
id=OwhVwNOBcZ.
Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Reasoning aware self-consistency: Leverag- ing reasoning paths for efficient LLM sampling. In Luis Chiruzzo, Alan Ritter. and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 3613-3635, Albuquerque, New Mexico, April 2025, Association for Computa- tional Linguistics. ISBN 979-8-89176-18944 doi: 18.18653/v 1/2025.aaall-oogg..
URL
2025 1aa.2/222.aaaalooloo naacl-long 1847
Weixuan Wang, Jingyuan Yang, and Wei Peng. Semantics-adaptive activation intervention for llms via dynamic steering vectors. arXiv preprint arXiv:2410. 1it:2999../299, 2024.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowd- hery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, JCLR 2023, Kiguli, Rwanda, May 1-5, 2023, 2023. URL https: /ttps:////peeevieeeeee net. went//foummiii 1PL. NTMMrW
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoe V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Ad- vances in Neural Information Processing Systems, 2022. URL TRRLhttss////pteeeeett forum? Siatmmidd_vijpp LiS.ieee@MBBB_
net/
Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min-Yer Kan, Junxian He, and Qizhe Xie. Self-evaluation guided beam search for reasoning. The Thirty-seventh Conference on Neu- ral Information Processing Systems, 2023. https:/ httpss////peneeeeeeeee.eeeeee fortm/frum??
Bw82hwg5Q3
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In The Thirty-eventth Conference Neural Information Processing Systems, 2023. URL https://
openreview. wtttt//frummidd -xcldddddxcexx1nn
Weihao Yu, Zihang Jiang. Yanfei Dong, and Jiashi Feng. Reclor: reading comprehension dataset requiring logical reasoning. In International Conference on Learning Representations, 2020. URL
net www.ee/oorrrnnddtotttteeeeeee
A LLMS USAGE STATEMENT
Regarding the writing process, our use of LLMs was strictly limited to refining and polishing the manuscript. The core ideas and overall structure of the paper were developed entirely by the authors. The LLM was employed solely for grammatical and syntactical checks, and all revisions were sub- sequently reviewed by the authors to ensure accuracy and fidelity to our original intent. As for the experiments, all code was manually written and executed. The only involvement of LLMs in research was as a subject of the study itself. They were not used in any other part of the experimental process.
13

---

Published as a conference paper at ICLR 2026
Table 3: Statistics of the math-rollout dataset.
Models
R RI-Distill-Llama-8B
R1-Distill-Qwen-
Correct Base Solution
20
# of Avg. Steps in Correct of Total Rollouts in Correct
200.1
147.6
288500
184800
of Incorrect Base Solution # of Avg. Steps in Incorrect # of Total Rollouts in Incorrect
20
174.3
176.1
28
281500
273600
B LIMITATIONS
We acknowledge the following limitations of our current work, which also point possible direc- tions for future research:
First, our methodology models reasoning as first-order process, focusing on the transition from S,-1 to S. This has proven effective for correcting local drifts but may not capture failures caused by longer-range, higher-order sequences of behaviors. Investigating these higher-order relationships would, however, lead to combinatorial explosion in complexity and demand datasets with an even greater scale of rollouts to achieve statistical significance. Exploring more sophisticated sequential models to capture these long-range dynamics remains a direction for future work.
Second, our findings are based on the Llama and Qwen model families. While we uncover dis- tinet, model-specific drift patterns, is open question how these signatures generalize other prominent architectures, especially closed-source models whose internal states are not accessible for study. We believe study provides starting point and methodological blueprint for a broader investigation into the common and unique reasoning behaviors across the entire LLM ecosystem.
C DETAILS OF DATASET
We adopt the taxonomy introduced by [Bogdan et al. 2025). which classifies each sentence in reasoning trace into eight categories:
Problem Setup: Understanding and interpreting the original problem
Plan Generation: Devising reasoning plan based the problem
Fact Retrieval: Incorporating relevant background knowledge to support reasoning
Active Computation: Carrying out explicit calculations or logical operations
Result Consolidation: Integrating intermediate results summarizing progress
Uncertainty Management: Expressing confusion, re-evaluating intermediate steps, or backtracking when necessary
Self Checking: Verifying the correctness of previous steps
Final Answer Emission: Explicitly providing the final solution
This taxonomy provides a structured abstraction of sentence-level reasoning behaviors, allowing capture the fine- fine-grained dynamics of reasoning traces. By distinguishing the functional role of each sentence, it offers principled basis for analyzing, monitoring, and ultimately steering the reasoning process of LLMs. We build this taxonomy to design mechanisms that address reasoning drift through dynamic steering.
Detailed statistics of the dataset are provided in Table 3
14

---

Published as a conference paper at ICLR 2026
Table 4: Classification report for our trained DitilIBERT classifier.
Precision
Recall
F1-Score
Support
Problem Setup
0.89
0.67
0.76
12
Plan Generation
0.75
0.78
0.77
74
Fact Retrieval
0.91
0.88
0.89
207
Active Computation
0.92
0.91
0.91
243
Result Consolidation
0.75
0.74
0.75
82
Uncertainty Management
0.88
0,96
0.92
93
Self Checking
0.74
0.77
0.75
30
Final Answer Emission
0.88
1.00
0.93
7
Macro Avg
0.84
0.84
0.84
748
Weighted Avg
0.87
0.87
0.87
748
Accuracy Trend with Classifier Training Epochs
Class Steering Accurity
Traiiing Eoecha
Figure 7: Impact of different classifier training degrees on the downstream activation steering accu- racy. We select the AIME 2024 dataset as the experimental subject.
D DETAILS OF TRAINED CLASSIFIER
To efficiently obtain the functional category for a large number rollouts, we trained DistilBERT" classifier on the pre-labeled sentences from the math-rollout dataset, chosen for its efficiency and reliability. Specifically, we used a total of 7,481 labeled samples, which were split into a 9:1 ratio for training and validation. The model was trained for 3 epochs with learning rate of 5e-5 and weight decay of 0.01. The performance the trained classifier shown Table 4
To investigate how classifier quality influences final performance, compared classifiers trained to varying degrees of accuracy. As shown in Figure 7 the accuracy of downstream activation steering directly influenced by the classifier's 's training duration. We observed clear trend: classifiers trained for fewer epochs performed worse on both category identification and the downstream task. However, performance degradation due to potential overfitting observed after three epochs of training.
E CORRECTION MATRIX & DRIFT MATRIX
Figures
g and 9 display the original step transition matrices for the Llama and Qwen models,
respectively. The corrective revisions matrix is constructed based (St-1 T. St unpromising, T, = corrective), while the erroneous drift matrix based on (S.-1 - T. S promising, T. erroneous). To analyze the occurrence of drift, subtracted the Corrective Mairix from the Drift Matrix to obtain the difference, as shown in Figure
15

---

Published as a conference paper at ICLR 2026
(a) RI-Distill-Llamm- 8B Corrective Matrix
(b) RI-Distill-Llama--B Drift Matrix
Figure 8: Heatmap of step transitions matrix on RI-Distill-Llama-BB The corrective revisions ma- trix is constructed by (S.-1 T. S. umpromising, T. correclive), while the erroneous matrix is constructed by (S:-1 - T3 S promising, T2 = erroneous).
The coneccvv revilora marro
The eTereRo dill matt
AXC
AXC
011
31%
KX
200
(a) RI-Distill-Qwent Corrective Matrix
(b) RI-Distill-Qwen-14P Drift Matrix
Figure 9: Heatmap of step transitions matrix R1-Distill- Siiill--wen-14BB
F INTERVENTION LAYER SELECTION
The choice of which layer to apply the steering vector intervention to is critical question, inter- vening at layers that are too shallow or too deep may be suboptimal (Rimsky et al. (2024). Inspired by the methodology of activation patching (Syed et al., 2023 Nanda [023] Venhoff et al. 2025), we heuristically select the most impactful model layer for intervention. The core principle of this method is measure the magnitude of change in activations, such as the KL divergence, after re- placing sentence with a counterfactual counterpart. Formally, the patching effect is measured by:
AL L (Xelean do(a appacc)) L (Xeleon)
where a is the original activation, and apatch is the counterfactual activation. We applied this approach across all functional categories to identify a single, comprehensively effective layer for intervention,
illustrated in Figure 10]
Plan_ Generation
Fact_Retrieval
Uncertainty_Marageeenn
Uncertainty_ Management
Result_ Consolidation
Moain KL Divergence
0.0
2.5
10
20
30
10
20
30
10
20
30
10
20
30
Layer
Layer
Layer
Layer
Figure 10: The patching effect across different model layers. The value measures how effective is by applying the corresponding steering vector each layer.
16

---

Published as a conference paper at ICLR 2026
Differences between steered
original transition (without intervention)
H1 al scnup
0.00
0.05
1:12
0.10
0.44
4.00
0.00
0.00
From Category tstep t-1)
Han
900
0.15
0.20
1.03
022
6.53
044
904
040
Computatioon
0/00
0.09
0.33
0.40
2.91
4.16
0.61
006
0.5
Nosalt
0.30
.0.00
4.21
41.00
Uncertainty
Checking
0.00
9-81
0.01
0.18
0.2
6:14
5.37
0.35
0.00
Annnwer
hital Quantion
Problem Setup
nan Caneration
Rott Patneval
Active Compinal
Accult Canwal
Uncortainity Mgmnt
Soff Checking
Foal Answer Perssion
To Category (Step t)
Figure 11: Heatmap of the difference matrix, calculated by subtracting the original transition matrix (without intervention) from the transition matrix of our steered model on R1- R1-Distill-Qwer Soalll-wenn-4444 14B on GPQA-Din -DDa-DDd
G DETAILS OF IMPLEMENTATION
For our experiments, we selected the RI R1-distill-lama--81 and R RI-distill-qwenn-14 14B model which are the same models used for the steering vector extraction. We configured the hyperparameters according the officially recommended settings, with temperature of 0.6 and top-p of 0.95. Instead of using the vLLM framework, we utilized the original model implementation to facilitate the acquisition of activations during the process. All experiments were conducted on four NVIDIA 4090 GPUs
H DRIFT MITIGATION ON QWEN
Figure IT shows the heatmap of the difference matrix calculated by subtracting the original transition matrix (without intervention) from the transition matrix of our steered model on Qwen. Positive values indicate transitions that our method promotes, while negative values sigmify transitions it suppresses.
We make the following observations:
The transition from initial question to plan generation occurs less frequently. Instead, the model shows a preference for approaching the question through fact retrieval.
There an increased frequency of transitions leading to the uncertainty management state, which indicates the model exhibits model to verify its steps more carefully before reaching a conclusion.
uncertainty during its process. This prompts the
The lower triangle of the matrix displays lower values (indicated by cooler colors), im- plying that the reasoning process becomes more linear and contains fewer non-sequential, backtracking jumps.
The findings, together with those of Llama, show that our method can effectively mitigate the drift problem.
EVALUATION ON NON-MATI REASONING TASKS
http://gg.g.naaaooo//////e/aa//e -1--eessoee------iiiiiiii-- -Distill--Llama-SR https://uugiingffccccm http:///w...a.oooooooooo/c///a/a -Dissill---we---1
17

---

Published as a conference paper at ICLR 2026
In primary evaluations, focused mathe- matical reasoning benchmarks. Because our steer- ing vectors were derived from the MolhoRooloutt dataset, these benchmarks served as rigorous out-of- distribution (OOD) tests within the same overarch- ing mathematical domain. However, a fundamental question remains: do the pathological state transi- tions and corresponding steering vectors we identi-
Table 5: Results on non-math datasets.
Model
Method
ReClor
LegiQA
Vanilla
46.39
58.80
RI-Distill-Lamm--8
Ours
48.85
61.00
Vanilla
65.97
RI Dhsiill-Qwen-lie
Ours 66.51
81.40
fied capture domain-aae agnossic principles of logical reasoning, or are they overfitted to mathematical problem-solving?
To investigate the cross-domain applicability our method, we extend our evaluation broader, non-mathematical reasoning tasks. Specifically, we select ReClor (Yu et al., 2020) and LogiQA (Liu et al., 2020p, two widely adopted benchmarks designed assess reading comprehension and logical inference. Unlike mathematical datasets, these tasks require the model to parse complex textual premises and deduce logical validity without relying on formal mathematical operations. By applying math-deriver steering vectors to these datasets, we aim to evaluate the transferability of our reasoning- aware intervention across distinctly different reasoning modalities.
S the performance of our method on the two non mon--athematical reasoning datasets. Across both benchmarks, the application of our math-derived steering vectors yields consistent and stable improvements in reasoning accuracy over the baselines. These results substantiate the hy- pothesis that our steering vectors do not merely overfit to domain-specific mathematical heuristics or vocabulary. By effectively correcting the fundamental structural errors introduced by reason- ing drift, our intervention demonstrates broader applicability to general reasoning tasks beyond the scope of its original extraction data.
J CASE STUDY
J.1 ORIGINAL SOLUTION CASE
Below shows an original solution that leads to failed outcome.
(Initial Question) Solve this math problem step by step. You MUST put your final answer in b) Problem: Steve loves playing video games. His parents get him console along with 5 games for his birthday. He saves up enough money to buy game per month for year, and then the following year he starts buying 2 games a month. For the third year he buys 4 games a month as he has a new part-time job that makes him more money. He also gets 5 games for Christmas every year. How many games does Steve have after 3 years?
Solution: <think>>
(Plan Generation) To determine how many games Steve has after years, let's break down his purchases and the games he receives each year.
(Active Computation) **Yeaa 1:**
** **Mothlly Purchases:** 1 game month x 12 months = 12 games
**Toaa Games in Year 1:** 12 5 17 games
(Active Computation) ** Year 2:
(Active Computation) ## Year 3:
... ...
(Activate Computation) **Total Games After Years:** **
Year 1: 17 games
**
Year * 29 games
** Year 3;* 53 games
**Cuulltiive * 17 +3 = games.
Final Answer Emission) Therefore, Steve has total of 99 games after years.
</think>
18

---

Published as a conference paper at ICLR 2026
J.2 STEERED SOLUTION CASE
Below shows a steered solution that leads to a successful outcome.
(Initial Question) Solve this math problem step by step. You MUST put your final answer in boxed Problem: Steve loves playing video games. His parents get him a console along with 5 games for his birthday. He saves up enough money to buy 1 game per month for year, and then the following year he starts buying 2 games a month. For the third year he buys 4 games month as he has a new part-time job that makes him more money. He also gets 5 games for Christmas every year. How many games does Steve have after 3 years? Solution: <think>
(Problem Setup) Okay, let me try to figure out how many games Steve has after 3 years. (Fact Retrieval) Hmm, he gets the console and games for his birthday. (Plan Generation) First, need to break down what he does each year. (Fact Retrieval) In the first year, he buys game per month for year.
(Active Computation) So for the first year, he gets 12 games from his monthly purchases and 5 games from Christmas.
(Uncertainty Management) But hold on, does that include the initial 5 games from the birth- day?
(Uncertainty Management) But wait, that doesn't 't include the initial 5 games from the birth- day.
(Active Computation) So 5 + 17 + 29 + 53. 5 + 17 is 22, 22 + 29 is 51, 51 + 53 is 104. So
104 games.
(Self Checking) But earlier, thought of 17 + 29 + 53 = 99, which would not include the initial 5. So, perhaps, the correct answer is 104.
(Final Answer Emission) So, I'm confident that the answer is 104.
</think>
19


Reviews:
[{"review_id": "Y6puuA02V9", "reviewer": "Reviewer_kY9T", "summary": "This paper exploits a rich dataset of math reasoning rollouts to trace reasoinng trajectories that cause failure or success. Through defining corrective and erroneous rollouts, it identifies the promising and unpromising steps and studies the transitions of the model between step categories. This gives insight into which transitions are useful and which ones might hurt performance. Interestingly, this reveals different failure patterns in the Qwen and Llama models. Moreover, the paper proposes a method for detecting the undesired drift in reasoning and learns steering vectors that prove to be effective to correct reasoning trajectories in inference time.", "questions": "1. Could you please elaborate on why the improvements for other models (RQ3) is less significant? Are the transition patterns different? \n2. Could you explain the reason behind the definition of DriftScore? Is it just a heuristic or it is well-motivated?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper provides a granular study of the frequent reasoning patterns, which is helpful in studying the failure modes of reasoning in large language models. Through using a rich dataset, it is able to identify model-specific patterns. Then, it extends its study to a practical method that detect drifts in reasoning and corrects them at inference time with interventions with steering vectors. Overall, the work both brings clarity to understanding reasoning behavior of language models and provides practical solutions.", "weaknesses": "The proposed methods, while being effective in experiments, do not seem very well-motivated. It is not clear why defining the DriftScore in the way it is defined is plausible. Moreover, it seems that v_{i->j} can be computed from a_{i, j}^+ and a_{i, j}^-. Finally, the application of the method to other methods RQ3 could be explored more thoroughly to understand the gap between the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rtEQO6Ilum", "reviewer": "Reviewer_hd5F", "summary": "The paper identifies the issue of \"Reasoning Drift\" (when LLMs use unsuccessful reasoning patterns) and propose Reasoning-Aware Activation Steering (RAAS), an approach based on steering vectors to mitigate this issue. They claim that current methods: 1. require lots of model inferences to implement some kind of best-of-N like mechanism, and 2. don't use LLM internals so we don't understand where they go wrong (although this method doesn't help too much with this). The analysis in the paper is done using the Math-Rollout dataset, a dataset of CoT reasoning traces of a DeepSeek R1 distilled Qwen model on common math datasets, where reasoning sentences are labeled to one of 8 different reasoning categories, e.g., problem setup or plan generation, and also sentences are swapped out in various parts of the CoT to study counterfactual generations. The authors use this dataset to train a DistillBERT model to classify the reasoning category of sentences, and then use this to determine which reasoning category transitions on a sentence by sentence level are useful and not useful, e.g., perhaps self checking after problem setup is not very useful. The RAAS algorithm uses these transitions to design steering vectors and then adds a weighted average of these vectors when the reasoning category transition is expected to not be useful. Unrelatedly, they identify a \"funneling effect\", where on this math reasoning dataset, changing sentences early in the CoT have more influence over final answers than later sentences.", "questions": "- How dependent are the identified functional transitions and drift patterns on the specific characteristics of the Math-Rollout dataset and the DeepSeek distillation process? Can it extend to other model families and tasks?\n\n- In Figure 3, what do the neon blue and purple outlines around certain rectangles signify? The caption is incomplete and the text does not clarify this.\n\n- In the introduction, the authors seem to treat the fact that changing sentences in a CoT leads to different outcomes as a problem stemming from stochasticity. Can the authors clarify why this is viewed as a \"negative impact\"? Isn't it expected that different reasoning steps lead to different conclusions?\n\n- Given the very small performance gains on the base models (Table 2), does this suggest that the steering vectors are primarily capturing behaviors specific to the distillation process or the specific dataset, rather than generalizable reasoning principles?\n\n- Writing needs massive improvement. There is a lot of jargon phrases like 'reasoning drift', 'funneling effect', 'universal functional tendencies' and 'functional shifts' to name a few. They don't really mean much and are never formally defined. Also the intro seems to use a lot of such repetitive language but doesn't convey the main results. Also, RAAS uses Reasoning-Aware and Reasoning-Adaptive in different parts.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The analysis of the Math-Rollout dataset to identify the reasoning drift is conceptually interesting. The \"funneling effect\" and the distinction between model-specific failure signatures (Inertial vs. Chaotic Drift) are also interesting.\n\n- Use of activation steering to the process of reasoning (RAAS) at test-time is a creative idea. This makes the method computationally efficient.", "weaknesses": "- The most significant weakness is the paper's heavy reliance on the Math-Rollout dataset and limited evaluation. This dataset is specific to the math domain and derived from only two specific distilled models (R1-Distill-Llama/Qwen). It is unclear if the identified drift patterns, the 8-category taxonomy, or the derived steering vectors generalize beyond this domain and beyond these models. The gains on non-distilled models are too small to justify generalizability.\n\n- The paper suffers from significant clarity issues, making it difficult to follow, and the writing style often lacks precision and lots of loose terminology. Key terms such as \"universal functional tendencies\" and \"functional shifts\" are vague and not formally defined. The introduction frames the problem confusingly. For example, it describes how changing sentences in a CoT can lead to \"entirely different outcomes\" as if this is inherently problematic, whereas this is obvious and expected in a reasoning process. The specific \"negative impact of stochasticity\" they aim to address is not clearly articulated. The figures are confusing and poorly presented.\n\n- Experimental details for the DistilBERT classifier, a critical component of RAAS, are largely missing from the main paper and sparse even in Appendix D.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8htVNxi7K8", "reviewer": "Reviewer_iuYD", "summary": "This paper introduces \"Reasoning Drift,\" a failure mode in LLM chain-of-thought processing where models adopt flawed functional patterns. To address this, the authors propose Reasoning-Aware Activation Steering (RAAS), an inference-time method that uses a pre-computed library of steering vectors to guide model activations away from pathological states. The method is evaluated on several mathematical reasoning benchmarks and demonstrates significant accuracy improvements.", "questions": "The questions refer to the weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1) The paper provides a rigorous, data-driven characterization of \"Reasoning Drift.\" The analysis of functional state transitions offers a valuable new lens for understanding LLM reasoning failures that moves beyond surface-level error analysis.\n2) The proposed RAAS intervention is elegantly designed and directly motivated by the preceding diagnostic analysis. As a lightweight, interpretable, inference-time technique, it presents a practical alternative to computationally expensive methods like large-scale sampling.\n3) The method achieves consistent and substantial accuracy gains across multiple challenging mathematical reasoning benchmarks, including complex, out-of-distribution datasets like AIME.", "weaknesses": "1) The core methodology relies on a strong and unproven assumption. The diagnostic analysis and steering vectors are derived from sentence-level representations (using mean-pooled activations; see Eq. 2), yet the intervention is applied at the token level (see Eq. 4, 6). The paper provides no evidence to validate that a single token's activation can be meaningfully compared to the averaged activation of an entire sentence, calling the intervention's foundational principle into question.\n2) The paper demonstrates a correlation between the RAAS intervention and improved performance but fails to establish causation. The accuracy gains could be a byproduct of an uncontrolled factor, such as a regularization effect from the activation shifts, rather than the specific correction of functional transitions as claimed. The lack of critical control experiments (e.g., applying randomly selected steering vectors) leaves the mechanistic claims unverified. Maybe adopting significance tests (such as bootstrap) can make results more convincing.\n3) The paper's claim of generalization to \"out-of-distribution tasks\" may be misleading. The steering vectors are extracted from a mathematics dataset (Math-Rollout), and all evaluation benchmarks (GSM8K, AIME, GPQA) are also within the mathematical reasoning domain (see Sec 4.1). This constitutes within-domain generalization at best, and there is no evidence to suggest the framework would be effective in other domains such as programming.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j5hgzxv1Zo", "reviewer": "Reviewer_iJ4q", "summary": "This paper first characterizes the failure modes of LLM reasoning patterns, and then proposes to steer the generation with pattern-specific steering vectors.\n\nTo characterize the failure patterns, the paper trains a classifier to label each reasoning step into one of the 9 categories (e.g. \"initial question\". \"Problem setup\", \"Uncertainty management\" etc). It identifies model-specific failure modes, e.g. Llama tends to get stuck in self-loops, and Qwen tends to make erractic jumps.\n- Specifically, the classifier is a DistillBERT trained on the reasoning trajectories from the Math-Rollout datasets and the category labels therein.\n\nBased on these findings, the paper constructs steering vectors by taking the difference in activations between a desired step and an undesired step. At inference time, the model computes per-step steering vector adaptively and adds the steering vector to the activation.\n- The layer to intervene is chosen to be the one that incurs the most change in the activation.\n- The adaptive steering vector is a weighted sum of all steering vectors, where the weights are based on the similarity the current activation and the steering vectors.\n\nThe experiments are based on R1-distilled Llama and Qwen, using 4 math datasets (i.e. GSM8k, AIME2024, AIME2025, GPQA-Diamond). They show that:\n- The steering method effectively changes the transition probabilities among the 9 categories.\n- Steering improves math reasoning performance over baselines (vanilla CoT, self-consistency, and PiCSAR).\n- The steering method is applicable to Llama and Qwen without distillation, though the improvement is much smaller.", "questions": "- To clarify, is it correct that: the trained DistilBERT classifier is for annotating rollouts that are not in Math-Rollouts, and the classifier is used only for analysis and not during inference.\n- line 297: should it be $S^+$ has the label $i$?\n- Sec 4.4 / Table 2: are the steering vectors extracted per-model? What do you think makes the performance gain much weaker now (e.g. is this related to whether the models have gone through distillation)? What are the performance of SC and PiCSAR?\n- How to improve the effect of steering in later stage? For example, is this because one token matters less when the attention is put on more tokens compared to at the beginning? Would additionally biasing the attention weights be helpful?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The analysis on characterizing per-step patterns can be useful for diagnosing models, which helps inform intervention which the paper subsequently explores. \n- The proposed inference-time intervention is lightweight and can be practically useful. Results on the 4 math datasets are promising: the intervention achieves noticeably better accuracy without requiring resampling / parallel sampling.\n  - Notably, the steering vectors computed on Math-Rollout are applicable to 4 datasets which can be considered OOD.\n- The writing is easy to follow.", "weaknesses": "- Sec 4.4 / Table 2: The effectiveness of steering seems to decrease significantly with non-distilled models.\n- The improvement on AIME2025 (i.e. the newest benchmark) is much more moderate than other datasets. This might suggest that the method might be less effective when the applicable is truly OOD -- which is understandable though worth noting.\n- The intervention requires white-box access to the activations, which can be strong for certain practical considerations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)