Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping
Authors: Zhenyu Lei
Qiong Wu
JIANXIONG DONG
Yinhan He
Emily Dodwell
Yushun Dong
Jundong Li
Venue: ICLR 2026 Poster (2026)
Keywords: Mechanistic Interpretability
Model Editing
Circuit Reshaping
Primary area: interpretability and explainable AI

Abstract: Large language models (LLMs) often exhibit flawed reasoning ability that undermines reliability. Existing approaches to improving reasoning typically treat it as a general and monolithic skill, applying broad training that is inefficient and unable to target specific reasoning errors. We introduce Reasoning Editing, a paradigm for selectively modifying specific reasoning patterns in LLMs while preserving other reasoning pathways. This task presents a fundamental trade-off between Generality, the ability of an edit to generalize across different tasks sharing the same reasoning pattern, and Locality, the ability to preserve other reasoning capabilities.
Through systematic investigation, we uncover the Circuit-Interference Law: edit interference between reasoning patterns is proportional to the overlap of their neural circuits. Guided by this principle, we propose REdit, the first framework to actively reshape neural circuits before editing, thereby modulating interference between reasoning patterns and mitigating the trade-off. REdit integrates three components: (i) Contrastive Circuit Reshaping, which directly addresses the generality-locality trade-off by disentangling overlapping circuits; (ii) Meta-Contrastive Learning, which extends transferability to novel reasoning patterns; and (iii) Dual-Level Protection, which preserves preexisting abilities by constraining reshaping update directions and regularizing task-level predictions.
Extensive experiments with Qwen-2.5-3B on propositional logic reasoning tasks across three difficulty levels demonstrate that REdit consistently achieves superior generality and locality compared to baselines, with additional validation in mathematics showing broader potential. Our code is available at https://github.com/LzyFischer/REdit.

Published as a conference paper at ICLR 2026
REFORMING THE MECHANISM: EDITING REASONING PATTERNS IN LLMS WITH CIRCUIT RESHAPING
Zhenyu Lei"; Qiong Jianxiong Dong".
Yinhan He', Emily Dodwell?, Yushun Dong & Jundong Li University of Virginia, AT&& Chief Data Office, Flooida State University {vjd52r, nee7ne, jundong jundong]evirgian eewieevii...eee..ee.ee......eeee edd,, (qw6547, JD612R, ed720d @att. com,
donuunddon@@suu. edu
ABSTRACT
Large language models (LLMs) often exhibit flawed reasoning ability that un- dermines reliability. Existing approaches improving reasoning typically treat it as a general and monolithic skill, applying broad training which is inefficient and unable to target specific reasoning errors. We introduce Reasoning Edit- ing, a paradigm for selectively modifying specific reasoning patterns in LLMs while preserving other reasoning pathways. This task presents fundamental trade-off between Generality, the ability of an edit to generalize across differ- ent tasks sharing the same reasoning pattern, and Locality, the ability to preserve other reasoning capabilities. Through systematic investigation, we uncover the Gircuit-Interference Law: Edit interference between reasoning patterns is pro- portional to the overlap of their neural circuits. Guided by this principle, we pro- pose REdit, the first framework to actively reshape neural circuits before editing, thereby modulating interference between reasoning patterns and mitigating the trade-off. REdit integrates three components: (i) Contrastive Circuit Reshaping. which directly addresses the generality-loaaity trade-off by disentangling over- lapping circuits; (ii) Mela-Contrastive Learning, which extends transferabilityy to novel reasoning patterns; and (iii) Dual-Level Protection, which preserves preex- isting abilities by constraining reshaping update directions and regularizing task- level predictions. Extensive experiments with Qwen-2. 5- on propositional logic reasoning tasks across three difficulty levels demonstrate that REdit consis- tently achieves superior generality and locality compared to baselines, with addi- tional validation in mathematics showing broader potential. Our code is available at https://giituu..cm////yyy
INTRODUCTION
Large language models (LLMs) achieve state-of-the-art performance across various domains such as mathematics (Liu et al. 2023a 2024a). law (Cheong et al. 2024 Sun 2023), and medicine (Zhao al. 2023 Hadi al., 2023). The success arises from their exceptional rea- soning ability when executing complex instructions (Lu et al.) 2023 Villalobos et al. 2022 Wang al.] 2024b). Despite this success, LLMs often produce incorrect or misleading re- sponses (Perković et al., 2024, Huang et al. 2025) driven by spurious reasoning processes, which significantly undermines their reliability and safety. For example, an LLM may correctly encode the fact that "if brain aneurysm is present, bleeding or swelling (A-B OR A-*)"", but still wrongly infer "no bleeding implies aneurysm (-B -A) ". risking harmful medical consequences (Sim & Chen 2024). Addressing such gaps remains a critical challenge for researchers and practitioners alike.
CT scan will show
To strengthen reasoning, researchers typically view it one general, monolithic skill that calls for broad enhancement (Wang et al., 2023] Parmar et al.) 2024 Wan et al. 2024). Standard ap- proaches include fine-tuning on large reasoning corpora (Zhang al. 2024a Kumar et al. 2025
"This work was initiated and completed while Zhenyu was an intern with AT&T CDO. Both Qiong Wu and Jundong Li are the corresponding authors.

---

Published as a conference paper at ICLR 2026
) and sophisticated test-time prompting (Bi et al. 2024 Zhang et al. (022). However, treating the LLM's reasoning monolithic ability has several drawbacks. First, overall reasoning enhancement can be difficult and expensive, demanding extensive human annotation and huge computational bud- gets (Luo et al.] 2024 Lai al. 2025). Second, growing evidence indicates that LLMS' reasoning is not monolithic but can be decomposed into separable patterns (Zhang et al. 2025b) Jiang et al., 2025 Zhang et al., 2025d; Shao & Cheng 2025). Indiscriminately training over every reasoning pattern fails to distinguish between those the model already handles well and those it struggles with, thus leading to inefficient use of resources and suboptimal correction of specific reasoning errors. Therefore, recent approaches have shifted towards enhancement at the level of specific reasoning tra- jectories or intermediate steps, which involve only a handful of reasoning patterns (Cui et al. [025] Havrilla et al. 2024b). However, these methods heavily rely on the model's own self-verification of- ten without the model truly mastering the correct reasoning patterns, thus failing to reliably remedy reasoning errors. As result, how to correct erroneous and inject new reasoning patterns without retraining on the whole reasoning datasets still remains an open problem. Recent work has demon- strated that specific reasoning patterns
reinforcement learning from human feedback (RLHF) (Havrilla al. 2024a Yue et al. 2025
encoded in localized parameters or neural circuits within LLMs (Hong et al. 2024 Kim et al. 2024). mirroring the way factual knowledge is stored in model weights (Meng et al. 2022a| Yao et al.) 2024 Zhang et al. (024c). Given the success of parameter- based methods for editing piecewise knowledge in LLMs (De Cao et al. 2021 Meng et al. 2022b), we propose a natural extension: If knowledge can be edited through parameter modification, can we analogously edit LLMs to correct flawed reasoning patterns or inject new ones?
In this paper, we take an initial step toward reasoning editing, defined as the selective modification of a certain LLM's reasoning pattern while preserving its factual knowledge and other reasoning pathways. To establish rigorous foundation for this investigation, we focus on propositional logic (PL). where reasoning patterns can be precisely defined and systematically evaluated. Although structurally simple, reasoning editing in PL remains challenging due to two fundamental desider- ata (Hua et al., [224: Sun y025): (I) Generality, edits applied to one instance should consistently generalize to all instances with the same reasoning pattern across domains, rather than memorizing surface semantics. For example, editing the transitive rule "A-*B, B-+C in math should also hold in medicine. (2) Locality, edits must remain narrowly scoped, correcting the targeted in- ference rule without impairing the LLMs' performance other reasoning patterns it already handles correctly, For example, editing the spurious rule " --B----AA- A= should not affect modus tollens (A-BB, -B) -~A" ". The two desiderata constitute a trade-off as shown in Section 2.2 whereby enhancing one dimension typically diminishes the other, thus presenting a significant dilemma.
To tackle this trade-off, we first probe the mechanism underlying reasoning edits. Motivated by evidence that reasoning mechanism can be faithfully revealed by neural circuits, we conduct a systematic investigation into the relationship of edit effects and the circuit of reasoning pattern. Through this analysis, we discover a fundamental principle we term the Cireuit-Interference Law: the degree to which an edit to one reasoning pattern affects another is directly proportional to the overlap between their respective neural circuits. Guided by this observation, we introduce REdit, the first framework to actively reshape circuits prior to reasoning editing, enabling controlled mod- ulation of interference among reasoning patterns. REdit employs three key components: At its core, (1) Contrastive Circuit Reshaping directly addresses the cenerality-locallie trade-off by disentan- gling overlapping circuits reduce cross-reasoning pattern interference which improves locality while consolidating pattern-specific circuits to promote within-reasoning pattern generality. Build- ing upon this foundation, (2) Meta-Contrastin Learning enhances transfer broader reasoning patterns beyond those observed during reshaping and (3) Dual-Level Protection safeguards preexist- ing reasoning abilities by constraining reshaping update directions via soft null-space projection and regularizing prediction distributions of reasoning tasks. After reshaping, widely used LoRA-based editing (Ge et al., 2024) suffices to achieve the desired generality and locality. We conduct exten- sive experiments on Qwen-2. 2..nnn2.553333 5B across three propositional-logic difficulty levels, showing that REdit consistently enhances generality while reinforeing locality, surpassing strong baselines. Fur- thermore, additional evaluations in the mathematics domain highlight REdit's potential to generalize effectively to broader reasoning scenarios. Our contributions can be summarized as follows:
Reasoning Editing Paradigm: We introduce the first systematic framework for reasoning editing, extending model editing from knowledge correction to the selective modification of logical inference patterns, and formally identify the generalityloocality trade-off.
2

---

Published as a conference paper at ICLR 2026
Circuit Reshaping Methodology: We pioneer active neural circuit modulation in LLMs, enabling principled and targeted modification of specific reasoning pathways through con- trolled modulation rather than passive circuit analysis.
Novel REdit Framework: We propose a unified approach that synergistically combines contrastive circuit shaping, meta-contrastive learning, and dual-level protection to simulta- neously achieve both broad generality and precise locality in reasoning editing.
Empirical Validation: We demonstrate consistent improvements on propositional logic reasoning tasks across three difficulty levels, showing superior performance in generality and locality compared to existing editing methods.
PRELIMINARIES
2.1 PROBLEM FORMULATION
We study the problem of reasoning editing for LLMs the context of propositional logic. Our goal to enable precise modifications logical rules while preserving its existing correct ones. To formalize this, we first introduce the necessary components of propositional logic reasoning, then define reasoning patterns and their neural approximations, and finally present the reasoning editing problem.
L.M'' reasoning behavior, ensuring it adheres desired
Notations. Let X' = f=i...., .m be a finite set of propositional variables (PVs), each taking a truth value in (TRUE, FALSE). Let denote fixed set of logical connectives (e.g., 7, A, V, -)) A premise set d is collection of well-formed formulas over (X', S) that we assume to be true. A goal a formula over S). We use the standard entailment relation F where P += op means every model that satisfies P also satisfies We write y = {TRUE, FALSE, N/A} for the three-way status labels for G. where N/A means "neither entailed nor refuted."
Definition (Propositional-Loggc (PL) Reasoning) Given premises P and a goal G, infer the sta- G (I) "TRUE" ifP 1 G. (2) "FALSE" ifP -G, and (3) "NIA" " otherwise.
Definition (Reasoning Pattern) Let X be finite of placeholder PVs composed of symbols with no semantic meaning. reasoning pattern is == (P, S), 9(X, S)), where P(X, S) a set of premises comprising placeholders and the connectives in S and (
(2,)) S) the goal.
substitution a: -- X' replaces each placeholder by concrete PV, yielding the instantiated pair
To (Po, Go) ())
where ({8)) denotes the set of ground variables obtained by applying each placeholder. Two instances π. and F.' are said to share the same reasoning pattern exactly when they both derive from the same template under different substitutions a o'. In practice, LLMs internalize reasoning rather than executing explicit symbolic logic, formalized as neural approximation.
Definition 3 (Neural Approximation of PL.) A parameterized language model So approximates PL. reasoning by mapping concrete pair (Po, Ga) to a predicted status, as So (Pa.9o) 7 y.
Problem 1 (Reasoning Editing) Suppose we have a fixed neural reasoner fe with parameters 0, we also possess a finite revision dataset D (p(i), g(o), T( y ( in which each
concrete premise-gont goal pair. glt) fo ((()) (,()), g() the original model's 's prediction on that pair, and (i) is the target status we wish the edited model for to produce instead. Our objective of reasoning editing is to find revised parameter vector 0' that meets below three requirements.
(1) Edit Success. For each sample (p(i), g( g()), y =(() D, the edited model with parameter 6' must predict exactly the desired status, shown as for (p(6), g()) = (()
(2) Generality. Let ((0) denote the underlying reasoning pattern of the example (p(i), G( Once decide to revise I's behavior one specific instantiation of (5), require that the edit extend
3

---

Published as a conference paper at ICLR 2026
all other premise- goal pairs arising from the same abstract pattern. Formally, for any substitution that produces the pair To (Po: 90), ) the edited model must satisfy fo- (1) }
(3) Locality. Finally, let C = (P. 9)
= be the collection of all premise- goaal pairs on which the original model's 's prediction fa (P, already matches the ground truth. We demand that editing 0 into O' does not disturb any of these previously correct predictions. Equivalently, for every (P.9) EC, Jo (P, 9) = So(P, 9).
2.2 PRELIMINARY STUDY
We begin by conducting preliminary experiments on subset of proposi- tional logic dataset ContextHub (Hual
100
Tht Chokkng
Logic Ressoning
Accuracy (15
et al. 2024). where empirically
1ncclity
reveal a generaliryllocality trade-off:
0.6
Gemma 25
40
Gumma-75
simple edit cannot simultaneously maximize both desiderata. Our in- vestigation is motivated by a key ob- servation that LLMs generally lack logical reasoning ability. As Fig- ure Ta shows, the accuracy of LLMs answering propositional logical ques- tions (Reasoning) is on average 10% lower than tasks that merely require recalling the premise from the propositional logic (Fact- Checking). This gap highlights systematic weakness in basic logical inference and motivates direct edits to correct faulty reasoning patterns.
0.4
Unma-7R
20
Qwan-35
0.2
Owen 78
Qwem- 78
Gemmaa 78
(wen
Liama-71 7B
Gemmg 28
Gemm 3B
0.6
0.8
Gonerality
(a) Paet-Checking Reasoning (b) Generality locality trade-off Figure 1: LLM reasoning deficiencies and editing trade-off.
To evaluate whether simple edit can achieve the dual desiderata of generality and locality, conduct experiments to measure the two metrics. Let denote the index set of reasoning patterns. For cach i 11, let S; denote its instance set. Given an instance S, fine-tune the model on the triple Di,- = (P(e), g(+), y (+)) to obtain edited parameters ((1.e), The two metrics are defined as:
Generality
Σ
(1)
E;
S
S {sH
SES,
(P.9)
Locality =
Σ
Σ j
Σ
(f (P.9)
(2)
Σ, S:1
II
SES,
(P,))ES,
In practice we approximate the last summation by randomly sampling a small subset of instances from each Sj instead of evaluating over the entire set for efficiency. We conduct experiments multiple training configurations with learning rates 7) E [1x10-5, 9,2100-1 As shown in Figure Ib increasing improves generality but decreases locality, yielding a trade-off between generality and locality. The remainder of this work therefore proposes framework designed to mitigate the observed trade-off, thus leading to better editing generality while preserving locality.
METHODOLOGY
3.1
CIRCUTT-INTERFEREEN LAW
Prior sections reveal a generality-locality trade-off: edits often fail to generalize within the intended reasoning pattern or inadvertently spill over to other ones. To understand this gap, we turn vestigate the underlying mechanisms of reasoning editing of LLMs. Recent work in mechanistic interpretability suggests that reasoning patterns are implemented by different neural circuits, and that different tasks may recruit shared modular circuits (He et al.). Building on these findings, we conjecture that the degree of overlap or separation among these eircuits may govern whether edits can generalize and remain local. Intuitively, if two reasoning patterns share substantial circuit com- ponents, editing one should also influence the other; their circuits are largely disjoint, edits are expected to remain localized. This motivates central hypothesis: circuit similarity predicts cross- pattern editing effects, with closer circuits yielding stronger interference and more distant circuits preserving locality. To validate this hypothesis, we design four-step experimental procedure.
4

---

Published as a conference paper at ICLR 2026
0.30
Interference
Inlerferesos
Interference
40-
0.25
0.25
0.25
Density
20
0.20
0.20
0.20
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.2
0.4
0.6
0.8
-0.375-00 -0.300 0.50-0325
Distance
Distance
Distance
Pearsen
(a) Edit
(b) Jaccard
(c) OT
(d) Pearson Correlation
Figure 2: Correlation between circuit distance and interference. (a-c) Scatter plots with regression lines show that larger distances consistently correspond to reduced interference across different dis- tance metries. (d) Density plots of Pearson correlations confirm consistent negative associations.
(1) Circuit Attribution via Edge Attribution Patching (EAP) Syed et al. 2023 For each pattern π, sample K instantiations {(P..: Go.))K=1 clean input de and build corrupted input of detailed in Appendix Let sy se(d) denote the log-probability of the ground-truth label y" (d). For edge in the computational graph with activation Ve. its edge attribution for instance is an approximation of the score drop when alone is patched:
EAPx(e) = (V., (o (c ve (e(d
To mitigate instance-specific noise unrelated to the reasoning pattern, average the edge at- tributions across K instantiations, yielding (x(e) - - Σx (A)x(e). We then define the threshold fx (T) = (frantile)-((Wu((ee (e))), and construct the attributed circuit as the top-7 edges: C(T) [(e, Wn (e)) wx(e) 2 (T)).
(2) Circuit Distance. Given two patterns Ti, Ty with attributed circuits c c, we quantify struc- tural dissimilarity using three complementary metries: weighted edit distance dgaa (i, Jaccard distance dJoccera(i, j)), and optimal transport distance dor (i.(i.) detailed in Appendix
(3) Interference from Single-Pattern Pattern Edits. Pick source pattern and small revision set D; ((P(n), g(*), y (x))) N where each (p(x), g(m) is an instance of and y²(x) its ground truth. Obtain edited parameters Pedit()) by fine-tuning mie-tunnng fe on Dr For any target pattern j, define accuracy on its held-out set S, as Acc,(0) and corresponding edit interference from to j as A.-rj.
Acc,(0) |S,1
Σ (P,9)ES,
= r" As-1 AAce((eamts)) - ACc)(0))-
(4) Cireuit-Interferencce Relation. We examine the correlation between interference A j and circuit distance d(i,j) E (dJac, dEdit, dor ). modeled as A,, a+ j) (Figure c). We consistently find and negative Pearson correlations, robust across edit budgets, random seeds, and dataset subsamples as illustrated in Figure 2 We term this finding as Cireuit-Interferene Law, which posits monotone relationship between structural proximity and cross- effects where smaller circuit distance implies larger A, and vice versa.
3.2 REDIT: CIRCUIT RESHAPING FOR REASONING EDITING
The Cineeet-nneerference Law suggests that achieving both generality and locality requires well- structured circuits: representations of the same reasoning pattern should align closely, while those of different patterns should remain distinct. This leads to bold proposition: rather than passively analyzing existing circuits, can we actively reshape them to enforce these properties? In this paper, we take a step that direction with REdit, a framework that reformulates model eircuits through contrastive meta-learning objective with dual-level protection constraints before reasoning editing, enabling more effective and controlled reasoning edits.
Contrastive Circuit Reshaping. Directly reshaping two circuits to make them similar is challeng- ing since (i) circuit structure is discrete and (ii) circuits are not available in closed form. We there- fore adopt the attribution weights defined in Section 3.I as differentiable surrogate. Within each minibatch, we sample multiple instantiations per pattern and compute their weights w,- We then normalize them ल, w>/ //////// w I|2- For each anchor example we construct positive example it
5

---

Published as a conference paper at ICLR 2026
from different group of instantiations of the same pattern, and negatives (i)) from instantiations of other patterns. We then conduct InfoNCE (Oord et al. 2018) over attribution vectors:
exp( W; )/T)
Letr (0) = -- - log exp( (ixs, (T+) + Ejer(s exp( (w;, mTy)TT)
(3)
where T. is temperature. Optimizing equation 3 increases similarity within a reasoning pattern and decreases similarity across patterns, shaping circuits implicitly through their attributions.
Meta-Contrastive Learning. Training only on observed reasoning patterns may hinder transfer to rare unseen ones. To address this, we adopt first-order meta-learning scheme on the contrastive objective inspired by the Contrasseee Network (Lin et al. 2021), adopting a Reptile-like frame- work (Nichol & Schulman 2018) that iteratively samples mini batches, performs several inner gra- dient steps, and updates parameters toward aask-ed weights. By aligning gradients across tasks, this process amplifies updates along shared directions while suppressing instance sasancc--peccfic directions, thereby mitigating overfitting to spurious contrastive relationships between particular reasoning pat- terns and enabling circuits generalize beyond those observed during training. In practice, each meta-iteration, we sample a batch of contrastive tuples B each regarded as a task, perform s inner steps of adaptation, and obtain task task-specific parameters di 0% The outer update then moves the model weights toward the mean of these task-adapted parameters:
Inner: 0,+1 0f VaC (0)), 0f = 0, Outer: 0 < -0+7
Σ YEB
0). (4)
Duel-Level Protection. To preserve the model's original behavior while enforcing correct mecha- nisms, we impose constraints at both the (a) prediction level and the (b) optimization level.
(a) Prediction Distribution Preservation. Given correctness set C and frozen reference model faret (a pre-iteration snapshot of 0), we penalize deviations on C:
(5) (b) Null-Space Protection. At each inner step / of task i, we form an anchor group (3.00, with its instantiations set derived from the anchor. We compute the average prediction loss (f(( = [o(5))) Edectt.y (d(d) and the gradient gi,, = (olo(a((.t). To prevent reshaping from impairing
Cpred (0) E (e)c)ccLL( P.9) fol- P.G)).
reasoning task performance on the anchor, we define the rank-1 projector (1) (9.s) and the soft null-space operator p(i.t) = I - -Pllg... where P [0. 1] controls projection strength and > 0 ensures numerical stability. The inner-loop gradients are then replaced by their projected versions:
(w.g)
V ( (i) (0t) p(i.) Vo Vo (8)).
0- V (e (0
When م = 1, the update is confined to the null space of gi.t: leaving the anchor's 's loss unchanged to first order. While prediction preservation maintains consistency in the model's outputs, null-space protection regulates internal parameter updates, thereby preventing catastrophic drift.
we obtain the reshaped parameters Orsp- To enable fair comparison, we then apply a widely used parameter-efficient editing method LoRA on the revision set D, yiclding the adapted parameters Oedit = millg.p [o] ((P.Gw)) CE(fe.,, (- P, y²), With circuit reshaping, this lightweight edit is expected to achieve improved generality and locality.
LoRA-based Edit. After circuit reshaping,
4 EXPERIMENTAL SETTINGS
Datasets & Metrics. We experiment on CONTEXTHUB (Hua et al. 2024) with details in Ap- pendix A.I We evaluate with the Generality and Locality metrics introduced in Section 2.2
Backbone LLM. We use D0wen2.5-3B-Instruct (Yang et al. 2025) as the backbone LLM for all experiments unless otherwise noted. This model offers competitive reasoning capability at modest parameter scale compared to larger ones, which keeps memory and inference costs manageable.
Baselines. We compare REdit two families of approaches. (i) Model Reforming: (1) BIMT (Liu
al. 2023b) Brain-Inspired Modular Training) encourages functional modularity for MLPs during pretraining: adapt it more complex L.LMs to promote separable circuits for distinct reasoning

---

Published as a conference paper at ICLR 2026
Table I: Main results on ContextHub evaluated with generality and locality metries. The best and second-best scores are highlighted in bold and underlined, respectively. Raw denotes the perfor- mance of the unedited LLM. For BIMT, we apply the same LoRA-based editing method as in REdit.
Dataset
Metric
Raw
BIMT
LoRA
ROME
AlphaEdit
Ours
Generality
60.7 + 2.3
72.2 +1,4
63.8 2.9
67.8 =3.2
67.9 +1.9
74.1 + 1.6
Level
Locality
N/A
61.5 10.7
84.9 1.6
89.8 =3.1
87.0 10.9
94.3 + 0.4
Generality
53.2 + 1.4
63.6
58.4 + 0.1
61.3=11
58.8 #1.5
64.8 + 1.2
Level
Locality
N/A
59.4 #4.1
91.5 0.0
93.1 =0.1
93.3 +0.0
94.3 + 0.5
Generality
45.1 + 1.6
52.6 #0.4
50.1 =0.100 0.8
51.5 333
54.2 #0.8
55.0 + 1.6
Level
Locality
N/A
52.3 #1.0
92.3 2.8
94.6 #2.7
92.2 #0.7
94.4 0.8
patterns, followed by LoRA-based editing. (i) Model Editing: (2) LoRA (Hu et al.
2022 ap-
plies low-rank adapters for carameter-efficient tine-tuning and a widely used and simple baseline in knowledge editing (Wang et al.) 2024c) Jiang al., 2024); (3) AlphaEdit (Fang al. 2024) augments editing with null-space protection reduce collateral changes; (4) ROME (Meng al., 2022a) locates and updates internal representations associated with targeted knowledge. We adapt each method to the PL setting for a fair comparison. All editing methods select the optimal learning rate within the range 5 x 10 and 2 10 For other implementation details, refer to Appendix
5 RESULTS AND ANALYSIS
In this section, address five research questions: RQ1: How does REdit compare with existing baselines? RQ2: What is the contribution of each component within REdit? RQ3: How effectively REdit reshape circuits in LLMs? RQ4: To what extent does circuit reshaping transfer unseen circuits? RQ5: How does REdit perform on other domains compared to baselines?
5.1
MAIN RESULTS
In this section, we address RQ1 and present our findings in Table [ Our analysis yields several key insights: (1) REdit consistently outperforms all baselines, achieving up at most 16.1% im- provements in generality and 12.2% in locality com- pared to LoRA without circuit shaping, and averag- ing 2.0% gains over state-of-the-art methods. (2) REdit's advantage increases as task complexity de- creases, though improvements persist at all difficulty levels. This reflects that simpler tasks have more tractable circuit structures amenable to targeted re- shaping. (3) BIMT achieves strong generality but poor locality due to its disruption of internal mecha- nisms, compromising preservation original capa- bilities. (4) ROME and AlphaEdit exhibit compet- itive locality but inferior generality. ROME's focus on middle-layer MLPs inadequately captures distributed reasoning capabilities, while AlphaEdit's constrained editing directions limit generality enhancement to preserve other knowledge.
Raw
DIMT
ROME
LoRA
AlphaEdst
REáit
75
Score (9))
50
25
Level
Level 2
[evell
Figure 3: Editing success rates across meth- ods ContextHub. REdit achieves suc- cess rates comparable to other approaches, confirming that it does not compromise the model's 's fundamental editing capabilities.
To ensure method does not compromise the model's fundamental editing capabilities the target instances, we evaluate editing success rates in Figure 3 Most methods achieve comparable performance, with ROME as notable exception showing significantly lower success rates. This result further validates that restricting modifications to middle-layer MLPs is insufficient, given that reasoning capabilities in LLMs are distributed across multiple architectural components.
5.2
ADDITIONAL ANALYSIS
Ablation Study. To address RQ2, conduct ablation study with results presented in Table 2 Here, w/o MCL denotes the removal of deta-Contrastive Learning, w/ PDP indicates without Pre- diction Distribution Preservation, and w/o NSP represents without Null Space Protection. We have

---

Published as a conference paper at ICLR 2026
Table 2: Ablation studies on ContextHub evaluated with generality and locality metrics. The best and second best scores are highlighted in bold and underlined, respectively.
Dataset
Metric
Raw
w/o MCL.
w/o NSP
w/o PDP
Ours
Generality
60.7 + 2.3
72.9 0.4
73.3 + 0.2
73.4 0.5
74.1 + 1.6
Level
Locality
N/A
90.7 + 1.8
89.5 + 0.3
90.1 + 2.5
94.3 = 0.4
Generality
53.2 + 1.4
62.5 0.3
62.4 + 1.6
61.3 2.0
64.8 + 1.2
Level 2
Locality
N/A
94.9 + 0.6
93.0 + 1.8
94.0 0.8
94.3 + 0.5
Generality
45.1 + 1.6
53.8 1.3
50.9 + 0.6
51.8 + 0.6
55.0 + 1.6
Level 3
Locality
N/A
93.7 1.3
92.8 + 1.1
92.8 1.2
94.4 + 0.8
0.5
Inter Paltura
0.30
Raw
Intarffreeno
0.4-
Intra-Paitam...
Intectorecce
Intra-Pattern
0.5
wio NSP
MCL
Wif PDP
0.3
Siluuette
0.15
0.3
0.10
0.2
0.05
0.50 Distance
0.75
1.00
0.4
0.8
1.0
0.1
Overall
Distance
Interference
Before reshaping
(b) After reshaping
(c) Cluster silhouette
Cireuit-interfernnce relationship before and after circuit reshaping. (a.b) Scatter plots of intra- and inter-pattern measurements show improved separability in interference and circuit dis- tance. (c) Silhouette scores across reasoning patterns indicate consistent gains in cluster separation.
Figure 5:
the following observations: (1) All proposed components contribute meaningfully to REdit's 's overall performance, demonstrating their individual effectiveness. (2) Removing NSP or PDP substantially degrades performance, particularly in locality metrics, indicating that these protection mechanisms are essential for preserving model capabilities during circuit reshaping. (3) MCL provides modest but consistent improvements, attributable enhanced optimization stability through meta-learning.
To address RQ3, we measure how circuit reshaping alters circuit distances between patterns. We visualize the circuit-interference relationship as described in Section 3.1 distinguishing measurements between circuits from the same reasoning pattern (Intra- Pattern) and different reasoning patterns (Inter-Pattern). Comparing circuit-interference rela- tionship before and after circuit reshaping in Figure more separable in both interference and circuit distance dimensions. The right panel shows silhou- ette scores for the clusters across different reasoning pattern sets, where Overall indicates scores in the 2-dimensional space. Our results demonstrate that REdit and its components consistently improve circuit distance separation between different reasoning patterns while refining interference patterns: increasing intra-pattern interference (enhancing generality) and decreasing inter- mnter-pattern interference (improving locality). This validates both the effectiveness of our circuit reshaping approach and the Inturiirnteeeereece Law.
Reshaping Effect on Circuit Distance.
observe that the two clusters become
95.0
Transferability of Reshaping. To address RQ4, we in- vestigate the transfer of the effect of meta-contrastive cir-
92.5
65
90.0
cuit reshaping unseen reasoning patterns. We apply
60
87.5
Level Level
REdit to partial reasoning patterns (20% 80% ratio) and evaluate generality and locality on the remaining pat- terns. The results in Figure 4 show that while accuracy decreases slightly as the training ratio decreases, REdit consistently outperforms baselines without circuit reshap- ing (0% ratio) in both generality and locality metrics. This demonstrates the effectiveness of meta-contrastive learning transferring learned circuit modifications to previously unseen reasoning patterns.
55
50
Lovel 3
0.2 0.4 0.6 0.8 Ratio
0.0 0.2 0.6 0.8 1.0
Batio
(a) Generality
(b) Locality
Figure 4: Performance on unseen rea- soning patterns after circuit reshap- ing with different ratios for training. REdit consistently outperforms base- lines without reshaping.
8

---

Published as a conference paper at ICLR 2026
Evaluation on Mathematics Tasks To address RQ5, we broaden our evaluation beyond logical tasks by assessing REdit on TemplateGSM. a mathematical reasoning benchmark. TemplateGSM encompasses multiple math templates. where each template represents distinct reasoning pat- tern analogous to propositional logic reasoning patterns (detailed in Appendix A.2). in Figure show that while all methods perform worse on TemplateGSM than on propositional logic reasoning due the intrinsic complexity of math problems, REdit consistently outperforms all baselines, demonstrating its
The results
Generality
Locality
100-
Accuracy
50-
effectiveness on a broader range of domains. BIMT fails
on both generality and locality. indicating its inability to modularize LLMMs for complex tasks. Additionally, AlphaFddit and ROME show limited generality improve- ments, highlighting the constraints of traditional knowl- edge editing methods mathematical reasoning tasks.
Raw
LoRA
BIMT
AlphaEdit
ROME REDI
Figure 6: Evaluation on mathematical
reasoning benchmark TemplateGSM.
6 RELATED WORKS
LLM Reasoning.
Recent advances in LLMs have been driven significantly by their improved rea-
soning ability (Huang & Chang 2022, Yu et al.
2024
Chen et al. 2025a Li et al.
2025 Ferrag
al., 2025 Zhang et al. 2024b] Wang al.
2024d] Zheng al. 2025 He al.) 2025 2026
Zhu et al. 2025 Lei et al. (0255)), which is the capacity for structured, logical thinking to solve
complex problems such as mathematical proofs (Ahn al. 2024 Yang et al.. 2024a). causal infer-
ence (Wang) 2024 Ma 2024), and formal logic (Wan et al. 2024 Parmar et al. (024). Despite their
impressive performance, LLMSs reasoning abilities remain limited, especially with rigorous logical
deduction (Cai al. 2024), multi-hop inference (Yang al. 2024b), and precise symbolic manip- ulation (Sullivan & Elsayed 2024), thus prompting further improvement. Existing approaches often enhance reasoning through global strategies, such supervised tuniit (Kumar et al., 2025 Zhang et al. 2025c Luong et al. 2024) or RLHF (Hou et al. 2025, Yue et al. [025]] Wei et al, 2025). However, these methods treat reasoning monolithic capability rather than decompos- ing it into finer-grained, interpretable patterns (Havrilla et al., 2024b). As a result, they lack the precision target and improve specific reasoning weaknesses (Chen et al. 2024). In this work, we propose more granular reasoning editing paradigm that disentangles reasoning into distinct patterns. This enables targeted, efficient, and adaptive improvements tailored to specific reasoning challenges, moving beyond one-size-fits-a solutions.
Model Editing. Model editing modifies a pre-trained LLM's 's behavior post-hoc (Wang et al.
2024c
Zhang al., 2025a Lei et al.] 2025a), enabling error correction (Chen et al.] 2025b Li et al. knowledge updates (Wang et al. 2024a),, or task adaptation without full retraining QQ al.
2023).
2024
Current techniques fall into several categories: memory-based methods (Liu et al. 2024b
Hu al., 2024 Mitchell et al., 2022), meta-learning approaches (Mitchell et al. 2021] Tan et al. 2023), and localized rank-one updates (Hase et al. [023]] Meng et al. 2022a). These methods have predominantly concentrated on editing factual knowledge, typically represented as structured knowl- edge tuples. In contrast, reasoning editing addresses more complex reasoning processes, which are more intricately encoded within the neural circuits LLMs (Hong et al. [024 Kim et al.] (024)) Conventional knowledge editing techniques often fail in this setting. they struggle to satisfy the dual desiderata of generality and locality. Moreover, no prior work has systematically investigated how to directly manipulate neural circuits to enhance reasoning capabilities. In this work, we bridge this gap by taking the first step toward reasoning editing. We introduce a novel circuit-reshaping framework designed to mitigate the inherent generality-lcality trade-off, thereby enabling more ef- fective editing of reasoning patterns.
7 CONCLUSION
In this work, we present the first systematic study of reasoning editing, extending model editing beyond factual correction to logical inference, which introduces the generality trade-off. Through cireuit-level analyses, we uncover the Siteuit-nteerference Law, showing that interference between reasoning patterns is proportional to their circuit overlap. Inspired by this principle, we pro- pose REdit, framework that reshapes model circuits prior editing to mitigate the trade-off. REdit

---

Published as a conference paper at ICLR 2026
integrates contrastive circuit shaping align within-pattenn circuits while disentangling across- pattern ones, a meta-contrastive objective to enhance generalization, and dual-level protection to preserve both prediction distributions and update directions. Empirical results show that even with simple LoRA editor, REdit consistently outperforms knowledge editing and model reforming base- lines on propositional logic across three difficulty tiers using Qwen-2.5-3BB Additional experiments further demonstrate its potential across different reasoning domains.
ACKNOWLEDGMENT
Zhenyu Lei and Jundong Li are supported in part by the National Science Foundation (NSF) under grants IIS-214422 1442009, IIS IIS-2223769, CNS-2154962, BCS2228534, and CMMI-241 1248; the Office of Naval Research (ONR) under grant N000142412636. Yushun Dong is supported by the National Science Foundation (NSF) under grants OAC-2530786 and GEO-2536578. This project was initi- ated while the first author, Zhenyu Lei, was a summer intern at AT&T CDO. The authors gratefully acknowledge the opportunity to participate in the internship program and appreciate the supervision and guidance provided throughout the project.
REFERENCES
Janice Ahn, Rishu Verma, Renze Lou, Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint w0X10:2402.00157, 2024.
Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. Forest-of-thought: Scaling test-time compute for enhancing Ilm reasoning. arXiv preprint arXiv:2412.09078, 2024.
Chengkun Cai, Xu Zhao, Haoliang Liu, Zhongyu Jiang, Tianfang Zhang, Zongkai Wu, Jenq-Neng Hwang, Serge Belongie, and Lei Li. The role of deductive and inductive reasoning in large language models. arXiv preprint arXiv:2410.02892. 2024.
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng. Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of- thought for reasoning large language models. arXiv preprint arXiv:2503. 095667 2025a.
Qizhou Chen, Taolin Zhang, Chengyu Wang, Xiaofeng He, Dakan Wang. and Tingting Liu. Attri- bution analysis meets model editing: Advancing knowledge correction in vision language models with visedit. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp-
2168-2176, 2025b.
Yulong Chen, Yang Liu, Jianhao Yan, Xuefeng Bai, Ming Zhong, Yinghao Yang, Ziyi Yang, Chen- guang Zhu, and Yue Zhang. See what 11ms cannot answer: A self-challenge framework for un- covering IIm weaknesses. arXiv preprint arXiv:2408.nl 08978, 2024.
Inyoung Cheong, King Xia, KJ Kevin Feng, Quan Ze Chen, and Amy X Zhang. (a) am not a lawyer, but...: engaging legal experts towards responsible IIm policies for legal advice. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pp. 2454-2469, 2024.
Yingqian Cui, Pengfei Hc, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, al. Stepwise thought reasoning in large language models. arXiv preprint arXiv:2502.1322 2025.
guided refinement for efficient chain-of-
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. arXiv preprint arXiv:2104.08166 2021.
Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained knowledge editing for language models. arXiv preprint arXiv:2410.02355. 2024.
Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah. From 1lm reasoning to au-
tonomous ai agents: comprehensive review. arXiv preprint arXiv:2504. 19678, 2025.
10

---

Published as a conference paper at ICLR 2026
Xiou Ge, Ali Mousavi, Edouard Grave, Armand Joulin, Kun Qian, Benjamin Han, Mostafa Arefiyan, and Yunyao Li. Time sensitive knowledge editing through efficient finetuning. arXiv preprint arXiv:2406. 2tt.:2406.04496, 2024.
Muhammad Usman Hadi, Rizwan Qureshi, Abbas Shah, Muhammad Irfan, Anas Zafar, Muham- mad Bilal Shaikh, Naveed Akhtar, Jia Wu, Seyedali Mirjalili, et al. A survey large language models: Applications, challenges, limitations, and practical usage. Authorea Preprints, 2023.
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models. Advances in Neural Information Processing Systems, 36: 172111663-17777 7666-17668, 2023.
Alex Havrilla, Yuqing Du Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi- Yu, Maksym Zhuravinskyi, Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu. Teaching large language models to reason with reinforcement learning. arXiv preprint arXiv:2403.04644 2024a.
Alex Havrilla, Sharath Raparthy, Christoforus Nalmpantis, Jane Dwivedi- Yu, Maksym Zhuravin- skyi, Eric Hambro, and Roberta Raileanu. Glore: When, where, and how to improve IIm reasoning via global and local refinements. arXiv preprint arXiv:2402. 10963, 2024b.
Yinhan He, Wendy Zheng, Yushun Dong, Yaochen Zhu, Chen Chen, and Jundong Li. Towards global-level mechanistic interpretability: A perspective of modular circuits large language models. In Forty-second International Conference on Machine Learning.
Yinhan He, Wendy Zheng, Yaochen Zhu, Zaiyi Zheng, Lin Su, Sriram Vasudevan, Qi Guo, Liangjie Hong, and Jundong Li. Semcot: Accelerating aligned implicit tokens. arXiv preprint arXiv:2510.24940 2025.
reasoning through semantically-
Yinhan He, Yaochen Zhu, Mingjia Shi, Wendy Zheng, Lin Su, Xiaoqing Wang, Qi Guo, and Jundong Li. lapo: Information-aware aware policy optimization for token-efficient reasoning. arXiv preprint arXiv:2602. 2026.
Guan Zhe Hong, Nishanth Dikkala, Enming Luo, Cyrus Rashtchian, Xin Wang, and Rina Panigrahy. How transformers solve propositional logic problems: mechanistic analysis. arXiv preprint arXiv:2411.041 2024.
Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang- Thinkprune: Pruning long chain-of-thought of Ilms via reinforcement learning. arXiv preprint arXiv:2504.01290 2025.
Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Wilke: Wise-layer knowledge editor for lifelong knowledge editing. arXiv preprint axXiv:240..10987 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Wenyue Hua, Kaijie Zhu, Lingyao Li, Lizhou Fan, Shuhang Lin, Mingyu Jin, Haochen Xue, Ze- long Li, JinDong Wang, and Yongfeng Zhang. Disentangling logic: The role of context in large language model reasoning capabilities. arXiv preprini arXiv:2406.02785 2024.
Jie Huang and Kevin Chen-Chuan Chang. Towards reasoning in large language models: A survey. arXiv preprint
2022.
Lei Huang, Weijiang Yu, Weitao Ma. Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions Information Systems, 43(2):1-55 2025.
Gangwei Jiang, Yahui Liu, Zhaoyi Li, Qi Wang. Fuzheng Zhang, Lingi Song, Ying Wei, and Defu Lian. What makes a good reasoning chain? uncovering structural patterns in long chain-of- thought reasoning, arXiv preprint arXiv:2505.2211 22.00..21110 148, 2025.

---

Published as a conference paper at ICLR 2026
Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, et al. Learning to edit: Aligning ilms with knowledge editing. arXiv preprint arXiv:2402.11005, 2024.
Geonhee Kim, Marco Valentino, and André Freitas. A mechanistic interpretation of syllogistic reasoning in auto regressive language models. arXiv preprint arXiv:2408.08590 2024.
Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Fahad Shahbaz Khan, and Salman Khan. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21321, 2025.
Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-rl: Reinforce- ment learning for generalizable medical reasoning in vision-language models. arXiv preprint arXiv:2503.13939 2025.
Zhenyu Lei, Patrick Soga, Yaochen Zhu, Yinhan He, Yushun Dong, and Jundong Li. Moledit: Knowledge editing for multimodal molecule language models. arXiv preprint arXi::251.112770, 2025a.
Zhenyu Lei, Zhen Tan, Song Wang, Yaochen Zhu, Zihan Chen, Yushun Dong, and Jundong Li. Learning from diverse reasoning paths with routing and collaboration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 2832-2845, 2025b..
Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502./741/ 2025.
Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang, Xi Chen, and Huajun Chen. Unveiling the pitfalls of knowledge editing for large language models. arXiv preprint arXiv:2310.02129, 2023.
Yuanze Lin, Xun Guo, and Yan Lu. Self-supervised video representation learning with meta- contrastive network. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8239-82499 2021.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Jiateng Liu, Pengfei Yu, Yuji Zhang, Sha Li, Zixuan Zhang, and Heng Ji. Evedit: Event-based knowledge editing with deductive editing boundaries. arXiv preprint arXiv:2402. 2024b.
Wentao Liu, Hanglei Hu, Jie Zhou, Yuyang Ding, Junsong Li, Jiayi Zeng, Mengliang He, Qin Chen, Bo Jiang, Aimin Zhou, et al. Mathematical language models: survey, arXiv preprint arXiv:2312.07622. 2023a.
Ziming Liu, Eric Gan, and Max Tegmark. Seeing is believing: Brain-inspired modular training for mechanistic interpretability. Entropy, 26(1):41, 2023b.
Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. Are emergent abilities in large language models just in-context learning? arXiv preprint arXiv:2309.01809 2023.
Liangchen Luo. Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint 266552,40. 2024.
Trung Quoe Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Rea- soning with reinforced fine-tuning, arXiv preprint arXiv:2401.08667 3, 2024.
Jing Ma. Causal inference with large language model: A survey. arXiv preprint arXi::449.00982, 2024.
12

---

Published as a conference paper at ICLR 2026
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359-17372, 2022a.
Kevin Meng, Arnb Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory a transformer. arXiv preprint arXivv2210.07222, 2022b.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning, Fast model editing scale. arXiv preprint arXiv:2110.11330 2021.
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory- based model editing scale. In International Conference on Machine Learning, pp. 15817- 15831, PMLR, 2022.
Alex Nichol and John Schulman. Reptile: scalable metalearning algorithm. arXiv preprint arXiv:1803.02999, 2(3):4, 2018.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding, arXiv preprint arXiv:1807.03745 2018
Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra, and Chitta Baral. Logichench: Towards systematic evaluation of logical reasoning ability of large language models. arXiv preprint arXiv:2404.15522, 2024.
Gabrijela Perković, Antun Drobnjak, and Ivica Botički. Hallucinations in Ilms: Understanding and addressing challenges. In 2024 47th MIPRO ICT and Electronics Convention (MIPRO), pp. 2084 2088. IEEE, 2024.
Siyuan Qi, Bangcheng Yang, Kailin Jiang, Xiaobo Wang, Jiaqi Li, Yifan Zhong, Yaodong Yang, and Zilong Zheng. In-context editing: Learning knowledge from self-induced distributions. arXiv preprint arXiv:2406. 1e11111444, 2024.
Raimundo Real and Juan M Vargas. The probabilistic basis of jaccard's index of similarity. System- atic biology, 45(3):380-385 1996.
Jintian Shao and Yiming Cheng, Cot is not true reasoning, it is just tight constraint to imitate: A theory perspective. arXiv preprint arXiv:2506. 02878, 2025.
Shamus Sim and Tyrone Chen. Critique of impure reason: Unveiling the reasoning behaviour of medical large language models. arXiv preprint arXiv:2412.15748, 2024.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on machine learning research, 2023.
Rob Sullivan and Nelly Elsayed. Can large language models act as symbolic reasoners? arXiv preprint arXiv:2410.21400 2024.
Alan Sun. Circuit stability characterizes language model generalization. arXiv preprint arXiv:2505.24731, 2025.
Zhongxiang Sun. short survey of viewing large language models in legal aspect. arXiv preprint arXiv:2303.09136, 2023.
Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. arXiv preprint arXiv:2310 10. 10348, 2023.
Chenmien Tan, Ge Zhang, and Jie Fu. Massive editing for large language models via meta learning. arXiv preprint arXiv:2311.046m 2023.
Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn. Will we run out of data? limits of Ilm scaling based on numan-generaan data. arXiv preprint arXiv:221 1.4325, 2022.
13

---

Published as a conference paper at ICLR 2026
Yuxuan Wan, Wenxuan Wang. Yiliu Yang. Youliang Yuan, Jen-tse Huang, Pinjia He, Wenxiang Jiao, and Michael R Lyu. Logicasker: Evaluating and improving the logical reasoning ability of large language models. arXiv preprint arXivv2401.00757 2024.
Boshi Wang, Xiang Yue, and Huan Sun. Can chatgpt defend its belief in truth? evaluating Ilm reasoning via debate. arXiv preprint arXiv:2305. /3/60, 2023.
Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Wise: Rethinking the knowledge memory for lifelong model editing of large language models. Advances in Neural Information Processing Systems, 37:53764-553 53797, 2024a.
Qineng Wang, Zihao Wang. Ying Su, Hanghang Tong, and Yangqiu Song. Rethinking the bounds of IIm reasoning: Are multi-agent discussions the key? arXiv preprint arXiv:2402. 18272, 2024b.
Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey, ACM Computing Surveys, 57(3): 2024c.
Yiqi Wang, Wentao Chen, Xiaotian Han, Xudong Lin, Haiteng Zhao, Yongfei Liu, Bohan Zhai, Jianbo Yuan, Quanzeng You, and Hongxia Yang. Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey on emerging trends multimodal rea- soning, arXiv preprint arXiv:2401.0680 2024d.
Zeyu Wang. Causalbench: A comprehensive benchmark for evaluating causal reasoning capabilities of large language models. In Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-IO) pp. 143-151, 2024.
Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I Wang. Swe-rl: Advancing Ilm reasoning via rein- forcement learning on open software evolution. arXiv preprint arXiv:2502. 18449, 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 20XY::2505.09388,, 2025.
Kaiyu Yang, Gabriel Poesia, Jingxuan He, Wenda Li, Kristin Lauter, Swarat Chaudhuri, and Dawn Song. Formal mathematical reasoning: A new frontier in al. arXiv preprint 2xXiv2242..16777 2024a.
Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning? arXiv preprint 200::2402.16877. 2024b.
Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. Knowledge circuits in pretrained transformers. arXiv preprint 2rXiv:2405.17963 2024.
Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. Natural language reasoning, survey. ACM Computing Surveys, 56(12):1-39 2024.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does inforcement learning really incentivize reasoning capacity in Ilms beyond the base model? arXiv preprint 2iv::2500. 13837, 2025.
Li Yujian and Liu Bo. A normalized levenshtein distance metric. IEEE transactions on pattern analysis and machine intelligence, 1966): 2007.
Biao Zhang, Zhongtao Liu, Colin Cherry, and Orhan Firat. When scaling meets Ilm finetuning: The effect of data, model and finetuning method. arXiv preprint arXiv:2402.17199 2024a.
Binchi Zhang, Zhengzhang Chen, Zaiyi Zheng, Jundong Li, and Haifeng Chen. Resolving editing- unlcarning conflicts: A knowledge codebook framework for large language model updating. arXiv preprint 2rX1v:2502.00158 2025a.
Lin Zhang, Lijie Hu, and Di Wang. Mechanistic unveiling of transformer circuits: Self-influence as key model reasoning. arXiv preprint 2202:22002.0002... 2025b.
14

---

Published as a conference paper at ICLR 2026
Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E Gonzalez. Tempera: Test-time prompting via reinforcement learning. arXiv preprint arXiv:2211.11890, 2022.
Xinlu Zhang, Zhiyu Zoey Chen, Xi Ye, Xianjun Yang. Lichang Chen, William Yang Wang, and Linda Ruth Petzold. Unveiling the impact of coding data instruction fine-tuning large language models reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 25949-25957, 2025c.
Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. Lim as a mastermind: survey of strategic reasoning with large language models. arXiv preprint arXiv:2404.00 01230, 2024b.
Yifan Zhang. Training and evaluating language models with template-base bassd data generation. arXiv preprint arXiv:241 18104, 2024.
Yufeng Zhang. Xuepeng Wang. Lingxiang Wu, and Jinqiao Wang. Enhancing chain of thought prompting in large language models via reasoning patterns. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 39, pp. 25985-25993, 2025d.
Zhuoran Zhang, Yongxiang Li, Zijian Kan, Keyuan Cheng, Lijie Hu, and Di Wang. Locate-then-edit for multi-hop factual recall under knowledge editing. arXiv preprint
2024c.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223 1(2), 2023.
Zaiyi Zheng, Song Wang, Zihan Chen, Yaochen Zhu, Yinhan He, Liangjie Hong, Qi Guo, and Jundong Li. Corag: Enhancing hybrid cerieval-augmente generation through cooperative retriever architecture. Association for Computational Linguistics, 2025.
Kaijie Zhu, Jiaao Chen, Jindong Wang, Neil Zhenqiang Gong, Diyi Yang, and Xing Xie. Dyval: Dy- namic evaluation of large language models for reasoning tasks. arXiv preprint arXiv:2309. 17167, 2023.
Yaochen Zhu, Harald Steck, Dawen Liang, Yinhan He, Vito Ostuni, Jundong Li, and Nathan Kallus. Rank- grpo: Training Ilm bandbased conversational recommender systems with reinforcement learn- ing. arXiv preprint arXiv:2510.20155 2025.
ETHICS & REPRODUCIBILITY STATEMENT
We use only public, anonymized datasets. No human subjects or sensitive data are involved. The work aims to improve I.IM reliability, aligning with the ICLR Code of Ethics.
For reproducibility, datasets, settings, and metrics are detailed in the paper and appendix. Code and instructions released in the anonymous repository.
THE USE OF LARGE LANGUAGE MODELS
In this study, mainly leverage LLMs to enhance the clarity and polish of the manuscript. All conceptual development and methodology design were conducted by the authors.
A DATASET DETAILS
A.1 PROPOSITIONAL LOGIC: CONTEXTHUE
ContextHub (Hua et al. 2024) is benchmark for propositional logical reasoning, built on top of formal logic templates generated by Val (Zhu et al. 2023). It dynamically instantiated these tem- plates into natural language questions across 11 real-world domains drawn from Wikipedia (e.g., culture, health, technology) along with an abstract form, thereby ensuring both diversity and robust- ness of reasoning scenarios.
15

---

Published as a conference paper at ICLR 2026
Statistics. ContextHub consists of total of 256 formal logic templates, spanning several difficulty levels. Each template is instantiated across domains with variations per domain. This yields 360 samples for level- logic, 600 for level-2 logie, and 2,880 for level-3 logic types. Each sample balanced across the three answer labels (True, False, N/A). In this work, we treat each logic template distinct reasoning pattern.
Example. Table illustrates both an abstract and a contextual instantiation of the same level-1 template. The abstract form substitutes propositional variables with arbitrary character sequences, while the contextual form grounds them in a concrete domain.
Abstract Instance
Contextual Instance
(vxkgr V caunc) + ybyz. Given ybyz is False, what is the value of caunc?
If an area of land has experienced significant uplift been shaped by powerful erosional forces, then the terrain will feature tall, steep mountains. Given that the area does not have tall, steep mountains, can it be determined powerful erosional forces have shaped the land?
Table 3: Level-1 example instantiations in ContextHub.
A.2 MATHEMATICS: TEMPLATEGSM
TemplateGSM Zhang 2024 is a large-scale benchmark for mathematical reasoning, constructed using the Template- based Data Generation (TDG) paradigm. Frontier LLMs (e.g., GPT-4) are em- ployed author parameterized meta-templates, which are then instantiated into natural language problems paired with programmatically verifiable solutions. This ensures not only linguistic and structural diversity but also guarantees correctness scale.
Statistics. TemplateGSMM comprises 7,473 GPT-4-authored templates, instantiated into approxi- mately 7.47 million grade-school math problems spanning arithmetic, fractions, percentages, and elementary algebra. Problem lengths range from 18-636 tokens. In this work, we experiment on curated subset of 600 problems, each restricted to single numerical answer (integer float).
Example. Table illustrates a GPT-4-authored template alongside one instantiated problem, high- lighting how TDG generates diverse mathematical reasoning tasks.
Math Template
Instantiated Problem
[NAME] sold [NUM]] [ITEM]
Rosy Plascencia sold 238 air fryers her friends in April at yoga studio boutique Bracken County, Kentucky. sold 119 air fryers. How many air fryers did Rosy Plascencia sell altogether in April and May?
(teeirhe) friends in April at a [LOCATION] in (COUNTY), [STATE].
May, they
In May. [PRONOUN] sold [NUM2]
[ [ITEM]. How many [ITEM] did [NAME]
sell altogether in April and May?
Table 4: Example instantiations in TemplateGSM.
B CIRCUIT DISTANCE METRIC
Given two patterns Te, n, and their attributed circuits as the sets of top-7 edges ranked by attribution scores: c(F) ((e, wx(e)) wa (e) 2 (x()), we quantify structural dissimilarity between n, and Tj using three complementary metrics.
(a) Weighted Jaccard Distance (Real & Vargas 1996).
Execht ucip min ( (w; (e), wi we)e)
(i.
E Execht ucty max w; (e), (y(e) +e
16

---

Published as a conference paper at ICLR 2026
Raw
REdit (w/o Edit)
60
Score (%)
40
20
Level 1
Level
Level
Figure 7: Comparison of accuracy between the original LLMs (Raw) and the unedited models after REdit circuit reshaping (REdit (w/o Edit)) across three difficulty levels of logical reasoning tasks. REdit consistently provides modest gains, with the most notable improvements at Level
This emphasizes overlap of influential edges in the two attributed circuits.
(b) Edit Distance (Yujian & Bo, 2007).
Eeeclf ws(e) wy(e))
dgait (1,
(8)
Execht ucly) max (w;(e), wy(c) +e
This captures the minimal "edit cost" required to reconcile the two circuits.
(c) Optimal-Transpee (OT) Distance (Cuturi 2013). Normalize edge weights in each attributed circuit to probability masses
Wx(e) (c) Execct Wn (e!)'
e C(T).
Let c(e, e') 2 0 denote a ground cost between edges (e.g., based on layer/head/type and token-span offsets). The optimal transport distance is then
dor(i,1)
min Tell(p:ps)
Σ Σ
Te cle,e)) c(e,e))
eEct?? eeect
(9)
T(pr,P)) - (T20: Tepee((ee) [per=)(e))--
This explicitly accounts for circuit geometry by measuring the minimal mass transport needed to align the two attributed circuits.
C BONUS EFFECT OF REDIT
In this section, we compare the performance of the original LLMs with that of the unedited models after undergoing REdit circuit reshaping in Figure Surprisingly, we observe that without plicit editing, REdit consistently yields modest accuracy gains across three difficulty levels of logical reasoning tasks, with the largest improvements occurring on the easier problems. We attribute this phenomenon to circuit reshaping's 's ability to reorganize the model's s internal mechanisms, where it might suppresses noisy erroneous eircuits while preserving task-critical ones, thereby enhancing the model's overall reasoning performance. We will explore this phenomenon further in the future.
D ALGORITHM
In this section, we provide the algorithm of REdit circuit reshaping in Algorithm 1
17

---

end for Published as a conference paper at ICLR 2026
Algorithm REdit Circuit Reshaping
1: procedure REDIT(O, Uarain 7, at, 8,
2: Input: LLM 0; training patterns Mmann rates 7/; ot steps 8; ratio p.
3: Output: Reshaped LLM 0'
4: Contrastive Circuit Shaping:
5: Derive attribution scores W,r w,/ / w 22 define InfoNCE loss (0) as in Eq. equation
6:
Meta-Contrastive Learning with Dual Protection:
7:
for each meta-iteration do Sample batch B C Lkrann
8:
9:
for each B do Initialize 0: -
Inner loop (equation 4
10:
for 0, 0. do
12:
Compute preservation loss preea (0)) as in Eq. equation Inner objective: c(t) = C + Xppred
13:
Derive gradient of inner objective: gi, V VoL (0))
Form projector p(itt) I pllgue
b Null-space protection
16:
Update gt+1 + f: - p(i.) gi.t
Protected update (6)
end for
18:
Set @; 4 0.
end for
20:
Outer update: 0 + 0 +1
[8| Eieo(o -
Meta update, Eq. equation
21:
22:
return GREdi
23:
end procedure
E IMPLEMENTATION DETAILS
For circuit reshaping, we set the inner learning rate to a = x 10-6 and the outer learning rate to 7 = 10-6, running 200 steps with an inner update step size of s = In each iteration, we sample [B] = 2 contrastive pairs of reasoning patterns in batch. The temperature for contrastive circuit shaping is fixed at T. = The null-space protection coefficient is set to p = 0.5, and the prediction distribution preservation weight is À 0.1. When computing attribution scores,
K = 10 instantiations for circuit distance calculation and K = 2 instantiations for REdit circuit reshaping due to computational restricts. For experiments validating
Law, construct circuits with top-r = 5% edges. During editing, we modify one instance per sample. Unless otherwise specified, all editing methods select the optimal learning rate within the range 5 x 10 and 2 x 10-4 for 10 steps. All experiments are conducted on four A100 GPUs; each REdit meta-iteration consumes 1 minute.
Corrupt Dataset. To construct the corrupt dataset, we modify the final question to query the status of the first propositional variable in the premise (fact-checking), instead of the status of the goal G, while keeping all other components unchanged.
Prompts. For the propositional logic dataset, we append the instruction: (Answer only True, False, or N/A (Neither)). Answer: to each question. For the mathematical dataset, we append: Answer with only the final numeric result. Answer to ensure precise and standardized responses.
F CASE STUDY
In this section, we present a case study illustrating the circuits of two reasoning patterns be- fore and after REdit circuit reshaping. As shown Figure & prior to reshaping, circuits from different instantiations of reasoning pattern I exhibit substantial overlap, though discrepancies remain, most notably around node a23. h4 and the tree structure formed by m21, m22, and m23. Circuits from reasoning pattern II share slight overlap with those of pattern I. particu- larly within the same tree structure. After circuit reshaping, circuits from different instantia- tions of reasoning pattern become more consistent and exhibit stronger alignment, with noisy
18

---

Published as a conference paper at ICLR 2026
Reasoning Pattern
Reasoning Pattern
Before Circuit Reshaping
After Circuit Reshaping
True. False. (SA>
<A> False, <B> <B> > Cc. Deduce the result of =C>.
False, (SA> and
<B>) > <C>, Deduce the result <C*
Figure 8: Case study of circuits from reasoning patterns and before and after REdit circuit reshaping. REdit enhances intra- aitra-pattern consistency while eliminating inter- titee-patter overlap.
nodes and edges effectively pruned. At the time, overlap between circuits of reason- ing patterns I and II almost completely eliminated. This case study highlights the effective- ness of REdit: it reshapes circuits achieve greater separation across different reasoning pat- terns while producing more coherent and centralized structures within the same reasoning pattern.
G GENERALITY-LCALITT TRADE-OFF OF REDIT
REdit
0.8-
In this section, we compare the Ennerality-lccalii trade-off before and after applying circuit reshaping with REdit. As shown in Fig- ure across different learning rates, L.MSs trained with REdit con- sistently achieve superior Pareto frontier compared LLMs, highlighting the effectiveness our approach.
Locality
0.6
0.6
0.7
Generality
H ADDITIONAL EXPERIMENTS
Figure 9: Trade-off of REdit
To strengthen empirical support and demonstrate generality beyond a single architecture and domain, we present two additional sets of experiments.
Experiments on Gemma-3-1BBIT. To verify that REdit's effectiveness is not tied to specific model family, we evaluate all methods using Gemma-3-
as the backbone on ContextHub. As shown in Table REdit consistently outperforms all baselines across all three difficulty levels in generality and achieves competitive locality, confirming that the benefits of circuit reshaping transfer across model architectures. Note that Raw's locality is trivially N/A since edits have been applied.
Experiments on the Date Dataset. To further assess generalization to a qualitatively different reasoning domain, evaluate REdit variant of the commonly used Date Understanding dataset (Srivastava et al. 2023), where each template encodes a temporal reasoning pattern such computing date offset from given reference. T representative instantiation.
19

---

Published as a conference paper at ICLR 2026
Dataset
Metric
Raw
BIMT
LoRA
ROME
AlphaEdit
REdit
Generality
47.1 + 0.1
68.3 0.1
64.8 +: 0.4
57.3 1.1
69.4 0.4
70.8 1: 0.1
Level
Locality
N/A
72.2 0.3
76.3 0.6
76.5
76.3
80.6 0.5
Generality
46.9 + 0.3
60.4 0.1
55.4 1 0.4
56.3 0.4
0.1
69.1 +: 0.1
Level
Locality
N/A
73.8
73.7 + 0.1
76.5 +
73.4 0.1
78.2 0.1
Generality
55.3 + 0.1
62.6 0.1
62.0 + 0.4
59.5 0.4
62.3 0.5
65.6 + 0.2
Level
Locality
N/A
87.6 0.1
87.0 + 0.4
88.0 +
87.3 + 0.2
87.3 0.1
Table 5: Results on ContextHub with Gemma-3-1B-I1 as the backbone. highlighted in bold.
The best scores are
Date Template
Instantiated Question
Today is Christmas Eve of [YEAR] What
Today is Christmas Eve of 1909. What is the date
is the date lomorrow in MM/DD/YYYY? A. [CHOICELA]
tomorrow in MM/DD/YYYY? A. 10/25/1909 B. 12/26/1909
B. [CHOICE.B]
C. 12/28/1909 D. 12/25/1910
[CHOICEC]]]
[CHOICE.D]
12/27/1909
12/25/1909
E. [[CHOCCE.E] E. [CHOICE_F]]
Table 6: Example template and instantiated question from the Date dataset.
As reported in Table 7 REdit achieves the highest generality among all methods and competitive locality this dataset, outperforming most baselines both metrics. This further demonstrates the broad applicability of our approach beyond propositional logic and mathematical reasoning.
Metric
Raw
BIMT
LoRA
ROME
AlphaEdit
REdit
Generality Locality
41.5 + 1.1 N/A
55.3 + 1.0
44.2 - 0.2
49.5 = 0.7
50.8 = 0.4
57.6 0.3
0.3
87.6 + 0.8
91.2 + 1.1
89.2 1.1
90.7 = 0.5
Table 7: Results the Date dataset with Qwen -ween-2.5-33 The best scores are highlighted in bold.
20


Reviews:
[{"review_id": "I0ApJhjZOL", "reviewer": "Reviewer_ro7e", "summary": "The author proposed REdit, which is one of the pioneering works in reasoning editing. The work gives a perspective from the knowledge circuit on the reasoning process, which is inspiring and interesting. Specifically, the author proposed a systematic framework for reasoning editing with the emphasis on neuron circuits, and the Redit framework combines the findings together to achieve the editing.", "questions": "See above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The presentation of the paper is well-organized and convincing\n\n- The perspective of the knowledge circuit and the finding of Circuit-Interference Law is inspiring and novel.", "weaknesses": "- The work lacks sufficient experimental validation. The current experiments in Sec. 4 are conducted on only one model (Qwen-2.5-3B) and one dataset (ContextHub), which limits the generality of the conclusions.\n\n- The choice of base model is inconsistent between the preliminary analysis in Sec. 2.2 and the main experiments in Sec. 4. In addition, the experimental setup in Sec. 3.1 is not clearly described (not sure if this is also based on Qwen-2.5-3B).\n\n- The method appears overly complex, but the final results do not show clear improvement, leaving me unconvinced about its effectiveness and practical value.\n\nOverall, I am very interested in the ideas and perspectives in this work. It would be helpful for enhancing this paper if the author can provide more results on more base models and datasets to show the effectiveness of Redit.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xwrlhjgU1V", "reviewer": "Reviewer_Qw78", "summary": "The article presents the reasoning editing method for point-by-point modification of logical reasoning patterns in language models without losing other skills. The authors strike a balance between generalization and locality, describe the law of circuit interference, and propose the REdit system. Tests on logical and mathematical tasks show that REdit outperforms existing methods, paving the way for more precise control over model reasoning.", "questions": "See weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The shift from knowledge editing to reasoning editing is original and well-motivated. The generality–locality trade-off is crisply formulated and backed by empirical evidence\n* The results include confidence intervals, which make them more reliable and indicate the stability of the findings.", "weaknesses": "* The experiments focus only on propositional logic and structured math tasks, so they don’t fully reflect how reasoning works in more open-ended, real-world settings. It’s still unclear whether the method would hold up beyond these controlled, symbolic cases.\n* While “circuits” are central, empirical evidence that reshaped circuits correspond to interpretable submodules is limited to correlation plots. There’s no qualitative analysis of what circuits actually represent.\n* The approach is evaluated on a single model Qwen-2.5-3B, so it’s unclear whether the results would hold across different architectures or model scales.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QPxUN1EFKJ", "reviewer": "Reviewer_33G2", "summary": "The paper proposes reasoning editing, which targets edits on reasoning patterns using the proposed REdit method. Through experiments, the authors also uncovered the Circuit-Interference Law, which states that interference between circuits is proportional to their overlap.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- The paper's idea is very interesting and shifts attention from factual knowledge toward the logic applied by models, which is a source of many flaws in their performance.\n- The Circuit-Interference Law is a significant novelty for the community.", "weaknesses": "- The datasets and models used are limited. While it does not seem like the conclusions would differ with larger models, the use of the well-controlled ContextHub raises questions about how things might go wrong with wild, real-world data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d0tj8tTpnC", "reviewer": "Reviewer_cGfW", "summary": "This paper introduces REdit, a framework for reasoning editing in LLMs that extends model editing from factual correction to logical reasoning. It proposes the Circuit–Interference Law, linking neural circuit overlap with edit interference, and applies contrastive circuit reshaping with meta-learning and dual-level protection. \n\nOverall, it is well-motivated, technically sound, and creative, addressing how to modify reasoning pathways without interference.", "questions": "Confused on the instance-specific noise. if edge attribution values vary substantially across samples, how are these aggregated? Is the same top-$\\tau$ threshold applied uniformly, or does it adapt to distributional variance across instances?\n\nCould the authors provide specific examples of “reasoning patterns” to clarify how they are defined and represented? Additionally, how do these reasoning patterns change after applying REdit—is there any qualitative or quantitative visualization of this transformation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents a novel perspective showing that modifying internal circuit traces can directly influence the final reasoning behavior of LLMs, which is an intriguing and valuable insight.\n\nThe proposed Circuit–Interference Law provides a principled and empirically grounded explanation connecting neural circuit overlap with editing interference, representing a fresh and meaningful contribution to mechanistic interpretability research.", "weaknesses": "The experimental setting is limited. The authors evaluate only on a single backbone model (Qwen-2.5-3B) and a single dataset, which constrains the generality of the conclusions.\n\nThe reported improvements are modest rather than substantial, leaving some uncertainty about the practical effectiveness and scalability of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)