Precise and Interpretable Editing of Code Knowledge in Large Language Models
Authors: Min Xue
Nikolai Bolik
Lennart Stöpler
Erik Imgrund
Janik Schmid
Artur Andrzejak
Venue: ICLR 2026 Poster (2026)
Keywords: Programming Languages
Code-to-code Translation
Knowledge Editing
code LLMs
Software Engineering
Primary area: interpretability and explainable AI

Abstract: Large Language Models (LLMs) have demonstrated outstanding capabilities in various code-related tasks, including code completion, translation, or summarization. However, these pretrained models are static, posing a challenge to incorporate new knowledge into an LLM to correct erroneous behavior. Approaches such as retraining or fine-tuning demand extensive labeled datasets and might be computationally expensive, while prompt engineering fails to change models permanently. Knowledge Editing (KE) techniques offer a more efficient alternative, enabling model updates with minimal data, even just a single example. Nevertheless, existing KE methods often manipulate parameters within the Transformer's multi-layer perceptrons (MLPs), where neuronal polysemanticity hinders both the precision and interpretability of the edits. To address these limitations, we exploit TransCoder, an MLP-like model component with a wide and sparsely activated hidden feature vector. Specifically, we introduce **TransCoder-based Precise Editing** (**TCPE**), a novel method that leverages the sparsity and monosemanticity of the TransCoder’s neurons for highly localized knowledge editing. TCPE exhibits neuron-level mechanistic interpretability characteristics, revealing the correspondence between the edited neurons and the specific code-related knowledge. Furthermore, we present KECode, a new evaluation benchmark for code-to-code translation based on functional equivalence. Using KECode, we conduct a systematic evaluation of representative KE methods in the context of code-to-code translation. Our experimental results demonstrate that TCPE outperforms existing KE methods, achieving a substantial improvement of translation accuracy of CodeLlama-7b-Instruct from 57.5% to 64.0% in a low-resource scenario of Java-to-D translation.

Published as a conference paper at ICLR 2026
PRECISE AND INTERPRETABLE EDITING OF CODE KNOWLEDGE IN LARGE LANGUAGE MODELS
Min Xue', Nikolai Bolik', Lennart Stöpler', Erik Imgrund?, Janik Schmid',
Artur Andrzejak"
Institute of Computer Science, Heidelberg University, Germany
BIFOLD & TU Berlin, Germany
(min.xue, bolik, lennart.stoepler, janik.schmid, artur @uni- heid-heideeerrg.de
imgrund farrund@tu-bll tt-bberiin-ee
ABSTRACT
Large Language Models (LLMs) have demonstrated outstanding capabilities in various code-related tasks, including code completion, translation, summariza- tion. However, these pretrained models are static, posing a challenge to incorporate new knowledge into an LLM to correct erroneous behavior. Approaches such as retraining fine-tuning demand extensive labeled datasets and might be computa- tionally expensive, while prompt engineering fails change models permanently. Knowledge Editing (KE) techniques (Wang al. 2024) offer more efficient alternative, enabling model updates with minimal data, even just a single example. Nevertheless, existing KE methods often manipulate parameters within the Trans- former's s multi-layer perceptrons (MLPs), where neuronal polysemanticity hinders both the precision and interpretability of the edits. To address these limitations, we exploit TransCoder (Dunefsky et al. 2024), an MLP-like model component introduce with a wide and sparsely activated hidden feature vector. Specifically, Cransoode-bbaeed Precise Editing (TCPE), novel method that leverages the sparsity and monosemanticity of the TransCoder's 's neurons for highly localized knowledge editing. TCPE exhibits neuron-level mechanistic interpretability charac- teristies, revealing the correspondence between the edited neurons and the specific code-related knowledge. Furthermore, we present KECodee, a new evaluation benchmark for code-to-code translation based on functional equivalence (Weil et al. 2025). Using KECode, we conduct a systematic evaluation of representative KE methods in the context of code-to-code translation. Our experimental results demonstrate that TCPE outperforms existing KE methods, achieving substantial improvement of translation accuracy of CodeLllmma-7b-nnee 64.0% in low-resource scenario of Java-to-D translation.
from 57.5% to
INTRODUCTION
Large Language Models (LLMs) have proved highly impactful a multitude of fields within Software Engineering, including code summarization, code completion, code translation, software testing, program repair, and others (Hou et al. 2024; Jiang et al. 2024 Li et al. 2022 [Snn et al.) 2024). For code-related tasks, these models frequently need to be updated with new knowledge to correct erroneous behavior, accommodate changes in APIs or libraries, or align with developer preferences. However, this process is challenging due the large volumes of training data required, high computational costs, and risks such catastrophic forgetting, or loss of model consistency. Setting aside retraining or full-model finetuning (Li et al. 2024a 7hhu et al. 2024 GLM al. 2024), even lightweight fine-tuning techniques such as LoRA still demand thousands of labeled training samples (Hu et al. 2022). Prompt engineering or external memory augmentation can provide superficial improvements but fail to fundamentally alter model behavior at the parameter level (Wang
let al. 2025b Wang & Zhu 2024 Zhang et al. 2025).
In contrast, Knowledge Editing (KE) techniques (Wang et al. 2024) offer updating model knowl- edge with a small amount of data, typically single training examples, and promise precise model modifications, without impacting unrelated knowledge. To leverage this precision, focus here

---

Published as a conference paper at ICLR 2026
Unembed
CBC
ML. outpet
TransCodee
K
dW
MI pulpui
replace
W
Decoder matrix
@ pli (2)
[e0 9 Eo]
Prausforme layer (Ke)
Sparse
[20D TO'D 0 I5O
00 50
Hi MLP
Apply ReL.
MLPP Mippipp
W CIC
Encoder
[1o
Attn
MLP input
W onC
(
neurons
EEmbedling
Provide Feature Interpretations for Active Neurons
Transformer Variant
Update Active Neurons
W
Figure I: Overview of TransCoder- BBseed Precise Editing. The approach consists of: (1) Replacing the standard MLP module a TransCoder in selected layer, yielding Transformer variant, (2) An editing method which updates a minuscule fraction of TransCoder neurons relevant to the target knowledge, (3) Interpretation of neuron-level activations in TransCoder reveal the link between the edited locations and the injected knowledge.
groundtruth-based local modification methods (classification from (Wang 2024), ) such as the popular approaches ROME (Meng et al., 2022) and MEMIT (Meng et al. 2023 proposed in context of Natural Language Processing (NIP).
ROME, MEMIT and related methods (e.g. PMET (Li al. 2024b), FiNE (Pan et al. 2025 perform updates within particular components of the Transformer architecture, the multi-layer perceptron (MLP) by conceptualizing it key-value store. However, these methods frequently face challenges in real-world scenarios due to model collapse (Yang et al.] 2024a b). They also might suffer from limited specificity and poor interpretability. which can be attributed to the polysemantic nature of MLP neurons (Scherlis et al. 2022).
To address the latter challenges, we propose modified Transformer architecture that replaces the standard MLP layer with a TransCoder module (Dunefsky et al., 2024 Kissane et al.) 2024a). A TransCoder is essentially pair of encoder and decoder matrices where the hidden feature vector wider as in MLP and trained to be sparsely activated (e.g.. via L1 regularization). Figure [ gives TransCoder-based Precise Editing (TCPE).
an overview of our approach, which we refer The key idea is to leverage the sparsity and monosemanticity of the TransCoder's activation space to automatically locate and edit neurons associated with target knowledge. This allows for precise updates while also enhancing interpretability, as the TransCoder's sparse activations can be directly linked specific knowledge components.
Our second major contribution is KECode, benchmark specifically designed for evaluating knowl- edge editing in context of code-to-code translation. Existing benchmarks for knowledge editing predominantly focus on natural language-centric metrics such as efficacy, specificity, and reli- ability (Husein et al.) 2025 Wang et al. 2024). These metrics may not be directly applicable or even meaningful in code-to-code translation, here the primary success indicator is the functional
cquivalence (llucksberg 1984; Wei et al. 2025) the original and translated code.
To bridge this gap, propose a benchmark tailored for evaluating knowledge editing capabilities in the code domain (Chen et al., 202T) based on verifying functional equivalence. To this end we have collected dataset of 600 Java-to-D code translation tasks. We have selected D as the target language due to its relative rarity, which allows us to create a low-resource setting for our experiments. Our benchmark comprises a translation step before knowledge edits, where Transformer model is deployed to translate Java functions into corresponding D-language code. We then leverage unit tests provided for each example to check functional equivalence. Potential mistranslations (likely frequent due to the low-resource scenario) are clustered based on semantic similarity of the error messages (Islam & Inkpen 2008). Subsequently, we inject for each error cluster the suitable correction knowledge into the model using the KE technique to be evaluated. Finally, the testing and clustering process repeated to assess the effectiveness of the knowledge editing method according to multiple metrics described in Section 3.2 Overall, our contributions are follows:
We introduce TransCoder-bane based Precise Editing (TCPE), a neuron-level intervention method that leverages the sparsity and mono-semantii property of the TransCoder's is activation space to precisely identify and update active neurons responsible for target knowledge.
2

---

Published as a conference paper at ICLR 2026
We develop KECode, novel functional equivalence- benchmark specifically designed to evaluate knowledge editing capabilities in a low-resource Java-D translation task.
We demonstrate neuron-level interpretability mechanism which effectively indicates the connection between the edited neurons and the inserted knowledge.
We evaluate a collection established knowledge editing methods, including ROME (Meng
et al. 2022), MEMIT (Meng et al. 2023), PMET (Li et al. 2024b), AGRACE (Li et al.
2025 LoRA Hu et. 2022). among others, on the code-related task.
We conduct extensive experiments which show that TCPE outperforms existing knowledge editing methods, significantly improving the translation accuracy of CodeLlama-7 from 57.5% 64.0% in low-resource setting of Java-to-D translation.
The remainder paper is organized as follows. Section 2 introduces fundamental concepts and terms of the domain. We describe the approach in Section Lastly, we conclude this work in Section Appendix comprises related work, additional experimental results, examples of interpretability experiments, and training details.
X and the experimental evaluation in Section 4
PRELIMINARIES
Transformer and MLP Layer. In general terms, an autoregressive Transformer model Vaswani et al., 2017) can be described as a mapping 9 X' + y of an input sequence æ = [a1..... कT| E X over a vocabulary V to a next-token probability distribution y E y C A This mapping is operationalized as iterative transformation of hidden state h. First, the hidden state is initialized the sum of token embedding and possibly positional embedding n1!0) emb((:: + pos(i) E Relase,, where i denotes the token index and dmodel denotes the model dimensionality. Then the hidden state is passed through L consecutive layers. At each layer 1 1,...,,, the hidden state computed as:
h,))) for the i-th token is
m = R. + MLP (T()), n = h n((1))
Σ bead n
attn (n. (1 11
(1)
where attn (t.m) (11 (122)) denotes the output of the re-hh attention head layer given destination token in (1-1) and all preceding source tokens (t-1) The function MLP((() (-) denotes the token-wise
feed-forward transformation at layer I, defined a
MLPW)(h!)) w
(W!!) . (7
where w! E Rehii dhantel and w(() out dhuss dutt are the weight matrices of the two fully connected in transforms the input from the model's hidden dimension dmodel to the
layers in the MLP. w.O)) internal feature dimension dmip- and w!ll) projects back to the original dimension dmodet. Here, 7(-) denotes the layer normalization function, and a(.) is the non-linear activation function. Finally, an unembedding matrix is applied and the resulting logits are projected onto the probability simplex using a softmax function.
Transformer Variant and TransCoder Module. As shown Figure I TCPE extends the Trans- former model G by replacing the MLP at layer 1" with a TransCoder module. We call this modified model variant A. Dunefsky et al. (2024) introduce TransCoder as a sparse approximation of the MLP layer:
12 (T((0) ReLU (w!!) n,(I) TC (T,
((()) dec
(1) TTC (T()).
(4)
where w(O) E Xdusta and w()) de the encoder and decoder weight matrices. However, unlike traditional MLPs the TransCoder module is trained to minimize the approximation error alongside a sparsity loss (see Appendix L). Consequently, for given input only very few elements of 2/0 (h) are non-zero. We refer to these features as active neurons.
Some variants (c.g. RoPE (Su et al. 2024 ) place the positional embeddings inside the attention module. Throughout the paper all hiases omitted for brevity.
3

---

Published as a conference paper at ICLR 2026
3 APPROACH
In this section, we introduce the TCPE method and describe the benchmark KECode.
3.1 TCPE: TRANSCODER-BASED PRECISE EDITING IN THE CODE DOMAIN
Specifying Correction Knowledge in the Code Domain. In code translation tasks, code LLMs map a source code snippet to functionally equivalent target snippet (Galasso et al.. 2022 Wei et al. (025). To support knowledge editing applications, we represent each translation instance as four-tuple (r(2), 8, (,,r(2),)) o). where 8 is the source code snippet (the subject) and o is the functionally equivalent target snippet (the object). The prefix context r(1) includes the code preceding s, such as imports, comments, and prior definitions, and the suffix context (2! contains the code following s and may include the initial portion of o.
Specifically, we define the prompt as p(r(1), s2)) = r())) ① c r((2, where @ denotes string concatenation (Meng et al. 2023). If the predicted o contains syntax errors or violates functional equivalence, we manually correct it o". The resulting tuple (r(1), rr2), o²) is referred as correction knowledge. An example is provided in Appendix (Complementarily, building on the causal intervention method (Meng et al.) [2022), we introduce a fine-grained causal intervention method to further examine the role of the subject s the four-tuple (r(1), s, r r(()) across different programming languages (see Appendix H).)
Neuron-Level Sparse Update Mechanism. Building on ROME (Meng et al. 2022 we model the TransCoder decoder weight wi())
dec E (Ro Xde as a linear associative memory that maps a key doc
k E R a value v E Rdodt via w(( k V. To precisely inject correction knowledge, we propose to only target the active neurons in the Transformer variant A. Given 7 new key-value pairs
uf S,
kj E ER of R.dand
(2) where (k:g. uz) encodes the j-th correction knowledge (r.)) s3:1) .0)). The set S, = {a E [dh] (k7), > >) contains the indices of activation values in k that exceed the threshold т. where (k )a denotes the activation at position a. ( (Appendix (Appendix([laaalyeee anallyees the overlap of S, across different error types under both MLP and TransCoder modules, where TransCoder exhibits low cross-error overlap, indicating specialized neuron activation for distinct error types.)
Following ROME (Meng et al. 2022), for each key-value pair (kj, oj, Sj), we compute the update matrix A (1 = (-- w
(C- We estimate the covariance matrix C' E Rdexde using (C--R E
samples from the "bigcode/the-stael" dataset. Different to standard ROME, we restrict updates to the active neurons indexed by Sy. enabling precise modifications at the neuron level:
dee [. m] = l. m] + AW( m), Vm E Sj
(5)
where each (W is a rank-one update matrix, sparsified via the active neuron index set S,, ensuring that only relevant neurons are updated, thereby enhancing specificity and minimizing interference with unrelated knowledge.
Encoding Correction Knowledge. Unlike knowledge editing in natural language (Meng al.
2022 2023). the generation of uk),2)) in the code domain relies on the knowledge four-tuple (7,)), s, (2), (()),))) For each error type, we encode the correction knowledge (r,5), $3. 0f) into a kaluv pair (kj, uf) through the following two steps.
(2)
Step 1: Generating kj. We define kū the mean post-activation output from the TransCoder encoder at the final token position i of the prompt p(r;") ,8)). Specifically, we construct N input sequences by prepending randomly sampled prefixes {a])
" I==1 to the prompt p(r,)) (1) ss)), where
p(r (1) 53) r) f Sy. For each composite input a. pp(r 85)) we process it through Transformer
(1)
4

---

Published as a conference paper at ICLR 2026
architecture A, and extract the non-linear activation ( from TransCoder encoder the final token
position of p(r : sz). Finally, we compute k3 as the average activation across all N sequences. Formally, the kg is computed as:
N
TTC
h (a)
(a) plrp) ss)))
(6)
Here, hf (aj @ p(r,)) {3)) denotes the attention output (with residual) at layer / for the final token of the input sequence az @ p(r). 8,).
Step 2: Generating vz. We seek to construct value vector vz that encodes the new relation (r.3), attribute of the prompt (r5", To implement this, we introduce minimal perturbation أ_ق to the TransCoder's output. Specifically, the perturbation ò, is added to the output of the TransCoder decoder w dee- at the final token position of the input sequence oj @ p(r5") (1) 8j. (22) guiding the model to predict the new target object oj. Formally, this process is expressed as:
og = TTC + aarg min 6,
E
log PA(Te:0==5))00 aj e plrj.
(7)
where A(TCU) + 6,)) denotes the addition of the perturbation 6, to the TransCoder output TC within the Transformer architecture A. Once the corrected knowledge {(r))) 83, of) is encoded as {kit. vj. 5,)} we apply Equation (5) to selectively update the active neurons the TransCoder decoder layer w!!)
3.2 KECODE: KNOWLEDGE EDITING BENCIMARK LOW-RESOURCE CODE TRANSLATION
Unlike natural language, programming languages require strict syntactic and semantic correctness. Even with successful knowledge injection, generated code may still fail to compile or exhibit functional errors. Therefore, in the code domain, knowledge editing should be evaluated based on functional equivalence rather than superficial textual similarity (Wei et al. 2025).
Dataset Construction: G4GD. Following the principle of functional equivalence, construct the G4GD dataset for the low-resource Java-to-D translation task. We adopt the GeeksforGeeks" dataset provided by CodeGen as foundation, which contains hundreds of Java functions. To support automated evaluation, we collect 10 representative input-output pairs from each Java function and develop corresponding unit tests in the D language. The final G4GD dataset comprises 600 Java functions, each paired with 10 independent D unit tests.
In Appendix I.2 we provide detailed comparison between the G4GD dataset and widely used benchmarks such as HumanEval (Chen et al., 202T) and MBPP (Austin et al., (02T). Appendix [.6] includes the prompt formulation that guides the model in translating the source code into functionally equivalent functions in the target language. The G4GD dataset is publicly available at Hugging Face
Functional Error Clustering Mechanism. We categorize the generated D functions based on the textual similarity of the compilation messages. Specifically, in the Java-to-D translation task, we use each Java function x, from the G4GD dataset to generate corresponding function y, via a Code LLM, yielding 600 translation pairs (I.,, vs). E [1, 600). We then execute unit tests to assess the correctness of each generated function ys. and collect the compilation messages, including runtime error messages (if compilation fails) success indicators (if compilation succeeds). For failed cases, we extract the first six tokens of the error message, denoted as g.. Each message is paired with its corresponding translation pair (z., Vs), resulting in dataset Diul {(x,, Vs, 9x)) 1600
Based on the compilation logs 9a. we partition Diull into three subsets: Osuce (compiles and passes all tests), Crairss (compiles but fails some tests), and Cincoup (fails to compile). Then, encode the error messages 9. using the bage-baae-en--l..ffj model and cluster them via cosine similarity
5

---

Published as a conference paper at ICLR 2026
(threshold 0.9), yielding A error clusters C S Cincomp, i E [1, A]. (Appendix J provides detailed error cluster statistics, intra-cluster examples, and the error message list for CodeLlama- Instruct.)
Evaluation Protocol. To evaluate the performance of knowledge editing in the code domain, designed a functional equivalence- based evaluation framework focusing on three key metrics: Efficacy, Specificity, and Reliability. In particular, using Java functions from the G4GD dataset inputs, we construct pre-edit and post-edit datasets, Diull = {x Va. Ia) [ and Diuii = {I Vaa g') 1,000 The pre-edit dataset Diul is partitioned into three subsets: Caucer CriilPass and a collection of error clusters C Cincomp = {C:) Similarly, the post-edit dataset Diun is divided into succe FailPass and B C inccomp = [C)] د=ز where A and B represent the numbers of distinct eπror clusters before and after editing, respectively. If i = j. then C; and C³ correspond to the same eπror type. Furthermore, based on the target error type(s) of the edit, we define their union as the pre-edit target group Ciarget. and its complement within Diall as the pre-edit non-targel group, Cnon-turget Dian \\
Ciarget- The post-edit target grou Clarge and post-edit non-target group tanee correspond to the same error types as Guurge and non respectively. Based on this, we define the following evaluation metrics.
Efficacy: It measures how effectively the edit corrects targeted errors and comprises two metrics: (I) Generalization (GN), defined as the proportion of samples in the pre-edit target group Guaget that are correctly translated post-editing: Ez,~x (Cumg))
(Cum))) {< V.,g() C'uce), ). where Tx(-) is the projection operator (Codd) 1970) extracting Is from triplets (Ix: y', s¹)). and II(.) is the indicator function. (2) Cluster Drift (CD), which measures the relative change in the cardinality of the target error group after editing, computed as IC"ggel// //
Specificity: It quantifies the extent to which the edit avoids unintended changes and is measured by two complementary metries: (1) Locality (LoC), which evaluates the consistency of error categories within Coon-arge pre- and post-edit: Ex,-Tx (o (Cmm
V/s g') E C'an turget) J- (2) Destructiveness H[ (
(DT) is defined the proportion of originally correct samples that become incorrect after editing: Bex (C.us) T( (I,,, y'a g')) & ).
Reliability (RE): It measures the global impact of the edit on the model's overall accu- racy, defined as the ratio of post-edit accuracy (ACpost) to the pre-edit accuracy (ACpre): Eg, (Piu) (Pim)) (2 v'a ge) E / (DDn)) (I Us) E Csnce..
EXPERIMENTS
In our study, we (i) investigate TCPE's interpretability (Section 4.2 and Appendix (F). (ii) extend mainstream knowledge editing methods to the code domain for comparative evaluation (Section 4.3 and Appendix F. (iii) examine the information-carrying capacity of active neurons substantiate precise editing (Section 4.3 and Appendix E..2). and (iv) broaden our study to general NLP tasks to further probe the origins of low specificity in ROME-based approaches (Appendix E.33 Furthermore, we analyze the effects of TransCoder size and layer positions, as well as overlaps of active neurons, with detailed results provided in Section 4.3 and Appendix G (The code is available the following
4.1 EXPERIMENTAL DETAILS
Base Models and TransCoder Variants. In our work, we adopt Ladee-llma-7b-Instrree and Llama-2-7b (hereafter CodeLlama and Llama2) base models, and construct a series of Transformer
variants with varying TransCoder widths for systematic analysis. We utilize the "Transformerrent" framework to build four CodeLlama variants with varying TransCoder intermediate dimensions
(dw)): LTCmip (dge = dmup 11, 008), LTC4 (de = 4, 09096 * 4), LTC8 (dgc = 4, 09696 * 8), and LTC16 (die 4. 096 16), as well as two Llama2 variants: MTC4 (dg = 4, 096 4) and MTC8 (dhe = 4, 09. Here, each variant is constructed by replacing single MLP layer at / E {10, 19, 23} with the TransCoder. Notably, we designed the TransCoder Adapter to enable fast integration of TransCoder for the above variants in just few seconds. (See Appendices L.3 and L.2 for details.)
"nnwledgge editing may cause certain target error types to be absent in the post-edit (non-)target group. This considered our evaluation design and docs not affect metric validity. http://awwwacaccm/mmmm//////////
hhttps://git httpa//////.ttttooo/////affrrrre htwp:///eeeaaaaoooeeeeoeeennrarr

---

Published as a conference paper at ICLR 2026
Datasets and Evaluation Metrics. We evaluate knowledge editing performance on the KECode benchmark (including the G4GD dataset), HvmanEval (Chen et al. 2021). zsRE (Levy et al. 2017] + and CounterFact (Meng al. 2022). For the G4GD dataset, observe that the dominant error types remain consistent across CodeI.lama, L,C4,, and LTC8, with clusters Co. Cs, Ca. and Co together accounting for 57.68% 58.51% of Cincomp- Accordingly, our editing evaluation focuses on these clusters, assessing performance in both single-error and multi-error scenarios, using a functional equivalence-bor baaeed framework with three key metrics: Efficacy, Specificitty and Reliability, For HumanEval, we leverage it to examine the broader impact of knowledge injection after G4GD-based edits, using the Reliability metric to measure overall model performance (see Section [.2]. For CounterFact and zsRE, we follow the experimental protocols of Pan et al. (2025), focusing on three key metrics: Efficucy, Specificity, and Generalizution (see Appendix D for details).
Baselines. Meng et al. 2022) demonstrate that factual knowledge is primarily stored in the middle MLP layers of Transformers. Using a fine-grained causal intervention scanning all layers and token positions, we confirm that code knowledge is similarly localized in these middle layers (see Appendix H Furthermore, we observe that TransCoder modules with the same width exhibit consistent sparsity patterns (i.e., the average number of activated neurons per token) and achieve comparable performance when replacing MLP layers at different positions (see Section 4.3 and Figures [Orc) and iO(f)) the Appendix). Accordingly, our subsequent TCPE editing experiments primarily target the middle TransCoder modules at layer 19.
To assess the effectiveness of TCPE, compare it against representative knowledge editing baselines: ROME (Meng et al., 2022). MEMIT (Meng et al. 2023), PMET (Li et al. (2024b), FINE (Pan et al.. 2025). Fine-Tuning (FT) (Zhu et al. 2020b), AGRACE (Li al. (025). Few-shot (Parnami & Lee, 2022), WISE (Wang et al., 2025a). and LoRA (Hu et al.) 2022). 2). (Method descriptions and
hyperparameters for TCPE and baselines are provided in Appendices D.3 and K)
4.2 INTERPRETABILIT ASSESSMENT OF TCPE
In this section, first use TCPE to analyze the relationship between injected knowledge and neurons with varying activation levels, and then compare the interpretability of TransCoder and MLP neurons.
Neuron-Level Interpretability in Knowledge Editing. We employ TCPE LTC4 within the G4GD dataset to explore whether active features during knowledge editing align with specific error types, providing insights into neuron-level interpretability. Fo-
Between 3.54 and 4.43: 0.1221%
cusing on a typical D-type conversion error " "Error: cannot implicitly convert expression 'str.length' of type "ulong' to 'inr"", we analyze the interpretability differences between ac- tive and inactive features in the activation k from the LTC4 TransCoder module. Notably, only 57 features are active, rep- resenting 0.348% of the intermediate dimension die- Specif- ically, first record the indices of the top- 10 most active features and 10 randomly selected inactive features in kg dur- ing the injection of correction knowledge for this error type. Then, using all Java samples and D samples in Cg as input, we capture and analyze the patterns of the top-activating examples for both the top-10 active features and 10 inactive features. Figure 2 shows a representative example from the top-10 10 ac- tive features, which consistently respond to key tokens such as 'str', 'string', or == 1ength'. directly related the target error. (More results can be found in Appendix F.2) In contrast, Appendix F3 presents examples from 10 randomly inactivated features, which respond to structural or control-flow tokens like 'if', 'N', 'ps', and ";" Although these inactive features also exhibit stable activation patterns, their captured knowledge is largely unrelated to the target error. This comparison highlights that the highly active features exhibit semantic specificity and show direct correlation with the target error, providing interpretability for knowledge editing.
nun)) [lintlee mos. 1enethh Example 0, token 64
[linelal sta Ll llngthh;
Example 49, token 121
Sint/c>> fi lengthl Example 65, token 27
ate)) [intin Base. lengthi Example 73, token 14
(iiite [ittel Beorner. Heeeth Example 48, token 116
str) [iint Hss... leenghhi
Example 51, token 81
int sta. (e
Example 65, token 28
ste)) [lintn Best. Longthi Example 32, token 121
v Between 2.66 and 3.54: 0.3743%
G.lininn tree. Nleathii Example 34, token 120
comerr) Cintin in Bate. 3 Tmmath Example 48, token 107
s) linin lengthi Example 47, token 101
Lengthi/intin coorerr. leseth: | 12,token 118
ste)) liatna Example 21, token 127
a) a B Lengthh Example 33, token 83
n) Mntiieenaattt [eenghhl Example 19, token 107
For Cint res. Deneth Example 93, token 10
Figure 2: Top-Activating Exam- ples for Active Neurons.
Blind Interpretability Comparison of TransCoders and MLPs. Following the methodology of Dunefsky al. 2024), evaluate the interpretability of TransCoder features compared to the MI.P
7

---

Published as a conference paper at ICLR 2026
features. Here, feature is considered interpretable if it exhibits clear and consistent patterns (e.g.. syntactic or semantic) across the input examples that activate (Kissane et al., 2024b; Bloom 2024).
We randomly selected 50 features from the TransCoder module (Wene) of LTC4 and 50 features from the MLP layer (w/29) of Cama For each feature, we precomputed the top-activating examples and
from pool of 37.055 Java and D code samples sourced from the "semerr/ccodetexttjie Table I: Interpretability Analysis of MLP and TransCoder (LTC4) Features.
"UKPLab/SLTrang" r corpora. To avoid bias, features
were randomly shuffled prior to analysis. Then, we performed a blind manual evaluation to determine whether the top-activating examples exhibited inter- pretable patterns, categorized "uminterpretable", iesssibly-interretablle or "interpretable" (including subset labeled "context-free"" triggered by individual tokens). Examples of each category are shown in Appendix [.T After annotations were completed, the feature source (TransCoder or MLP) revealed. As shown in Table L features from the TransCoder module demonstrate higher interpretability compared to those from the MLP layer, aligned with prior findings on model interpretability by Dunefsky et al. (2024). Furthermore, in Appendix G evaluate the overlap of high-activation neurons across distinct error clusters, revealing that TransCoder neurons exhibit more independent activation patterns than the MLP, thereby providing further support for the above results.
Type
TransCoder MLP
Interpretable
33
Kussibly-Interprettbbe
8
Uninterpretable
41
Table 2: Comparative Performance of Knowledge Editing Methods a Multi-Error Editing Scenario. Score quantifies overall knowledge editing performance and is calculated as Score (GN CD) + (LoC DT) + RE.
Efficacy
Specificity
Reliability
Method
Scoret
GN
CD 1
LoCt
DT
RET
ACpost
ACpre
FT
86.40
7.09
100.00
87.36
6.38
97.97
56.33
57.50
Few-shol
120.20
12.06
87.23
90.20
2.61
107.78
61.33
57.50
WISE
99.74
3.55
98.58
96.51
1.74
100.00
57.50
57.50
AGRACE
99.85
0.71
100.71
99.56
0.00
100.29
57.67
57.50
FINE
144.50
29.08
54.61
81.92
13.62
101.74
58.50
57.50
LoRA
147.27
29.08
58.87
82.57
10.44
104.93
60.33
57.50
ROME
109.60
29.79
80.14
76.47
16,23
99.71
$7.33
57.50
MEMIT
14.96
7.09
124.82
72.99
22.32
82.03
47.17
57.50
PMET
-4.70
12.77
104.26
55.77
38.55
69.57
40.00
57.50
(CTC(L
171.59
30.61
44.90
83.00
6.05
108.93
63.00
57.83
TCPE (LTC4))
171.45
32.86
46.43
82.17
6.86
109.71
64.00
58.33
TCPE (LTC8)
174,82
31.66
49.64
88.50
5.73
110.03
64.00
58.17
4.3
EDITING PERFORMANCE TCPE
Analysis of TCPE and Baselines in Multi-Error Editing Scenario. Table 2 presents compari- son of TCPE and baseline methods in the multi-error editing scenario on the G4GD dataset. TCPE outperforms all baselines across key metrics, including efficacy, specificity, and reliability. Due the inherently strict syntactic and semantic constraints of programming languages, knowledge editing in the code domain is highly sensitive the granularity of interventions. Compared to broader inter- ventions like MEMIT and PMET, single-layer approaches (e.g., ROME) tend to yield more reliable outcomes. In this work, TCPE builds upon these approaches by combining TransCoder's sparse representations with precise neuron-level interventions. This method enables strong generalization while minimizing unintended side effects, as reflected in its high specificity (in terms of locality and destructiveness). Crucially. TCPE surpasses baselines in reliability, achieving significant performance gains through effective knowledge insertion. It demonstrating TCPE's ability to perform effective knowledge edits while preserving model integrity.
Information-Carryin Role of Active Neurons in Supporting Precise Editing. This study investi- gates the information-carrying roles of highly and lowly active neurons during knowledge injection to evaluate the feasibility of precise model editing. We apply TCPE to conduct conventional experiments on high-activation neurons and perform ablation studies on low-activation neurons G4GD dataset.
hhttp:://uugiiggtt http:///iw.aatt.///.aaaee/////ee java
https://huggingffaee htt.://gggg..ccooaaaaaaatsaaaaaa
8

---

Published as a conference paper at ICLR 2026
Table 3: Role of Active TransCoder Neurons in Multi-Error Editing, Neurons are updated if activations aco exceed thresholds {0, 0.001, 0.01, 0.05, 0.08, 0.1, 0.15, 0.2). "U" and "n" denote the union and intersection of updated neurons across multiple error types.
Efficacy
Specificity
Reliability
Neurons
Method
GNT
CD1
LoCt
DT
RE1
AC ACpost
ACpre
U
LTC4
acv>0.2
24.29
59.29
82.39
7.43
105.43
61.50
58,33
34
3
acv>0.15
27.86
61.43
82.61
7.14
107.14
62.50
58.33
42
3
act>0.1
32.86
46.43
82.17
6.86
109.71
64.00
58.33
69
3
acv>0.08
34.29
52.86
81.96
9.43
106.57
62.17
58.33
78
4
act>0.05
35.00
48.57
82.39
8.57
106.86
62.33
58.33
98
4
acv>0.01
33.57
50.00
83.04
8.29
106.29
62.00
58.33
135
4
acv>0.001
32.86
50.71
83.04
8.29
106.00
61,83
58,33
147
5
ace>0
32.86
50.71
83.04
8.29
106.00
61.83
58.33
147
5
LTC8
acv>0.2
29.50
51.80
88.29
6.59
108.31
63.00
58.17
39
act>0.15
30.22
49.64
87.85
6.59
108.88
63.33
58.17
51
2
acvs0.1
32.37
46.76
87.42
6.59
109.74
63.83
58.17
70
3
acv>0.08
31.66
49.64
88.50
5.73
110.03
64.00
58.17
92
3
acv>0.05
32.37
51.08
82.86
9.74
106.59
62.00
58.17
122
4
acv>0.01
31.66
51.08
82.65
10.32
105.73
61.50
58.17
181
5
acv>0.001
31.66
51.08
82.65
10.32
105.73
61.50
58.17
195
5
acvs0
31.66
51.08
82.65
10.32
105.73
61.50
58.17
197
5
Table Ablation on Active TransCoder Neurons in Multi-Error Editing. Neurons are updated if activations below thresholds {0, 0.001, 0.01, 0.05, 0.08, 0.1). Here, "Updated Neurons" denotes the number of neurons that meet each threshold.
Method
Efficacy
Specificity
Reliability
Updated
GNT
CD1
LoCf
DT
RE
AC
AC
Neurons
LTC4
aCu =0
1.43
101.43
98.48
0.57
100.29
58.5
58.33
16,237
50.001
1.43
101.43
98.48
0.57
100.29
58.5
58.33
16,237
acv $0.01
1.43
101.43
98.26
0.57
100.29
58.5
58.33
16,249
50.05
1.43
101.43
98.04
0.86
100
58.33
58.33
16,286
acu $0.08
101.43
98.04
0.86
100
58.33
58.33
16,306
acv <0.1
7.14
92.86
96.09
1.71
102
59,5
58,33
16,315
LTC8
acv =0
1.44
100
99.13
0.29
100.57
58.5
58.17
32,571
$0.001
1.44
100
99.13
0.29
100.57
58.5
58.17
32,573
acv $0.01
1.44
100
99.13
0.29
100.57
58.5
58.17
32,587
<0.05
1.44
100
99.13
0.29
100.57
58.5
58.17
32,646
acv $0.08
1.44
99.28
99.13
0.29
100.86
58.67
58.17
32,676
acv $0.1
1.44
99.28
98.92
0.57
100.57
58.5
58.17
32,698
(1) Active TransCoder Neurons. In the multi-error editing scenario, less than 1% of neurons in the TransCoder decoder are active. To further investigate the information-carrying capacity of active neurons during knowledge editing. we experimented with different activity thresholds (0, 0.001,01111 0.01, 0.05, 0.08, 0.1, 0.15, 0.2}. As shown in Table 3, the overall score remains relatively stable across range of activation thresholds т, demonstrating the robustness of active neurons in supporting knowledge editing. Notably, even when only small subset of highly active neurons (acv > 0.2) is updated, the performance remains competitive. This suggests that highly active neurons tend to carry more essential information during knowledge injection. Moreover, as the threshold is relaxed, the number of neurons participating in the update (U) increases, while the intersection of activated neurons across different error types (N) remains small. This highlights the specificity of TransCoder's s neurons, as each neuron tends to respond to a specific error type, with minimal overlap between neurons involved in correcting different errors.

---

Published as a conference paper at ICLR 2026
(2) Ablation Study of Active TransCoder Neurons. Based the ablation experiments presented in Table [ we provide a more comprehensive analysis of the relationship between neuron activation levels and the effecttivnnes of knowledge injection. As the activation threshold increases from 0 to 0.1, a larger number of low-activation and inactive neurons are included in the update process. Despite broadening the update scope, we observe tangible gains in generalization. In LTT44, the number of updated neurons increases from 16,237 to 16,306 while the score remains 1.43. In LTC8, the updated neuron count reaches 32.698, yet the score remains unchanged 1.44. Meanwhile, although specificity metrics (including locality and destructiveness) and reliability remain numerically high, this outcome primarily results from the model's inability to execute effective edits. Even when all low- activation and inactive neurons are updated, the model still fails to perform meaningful knowledge injection. These results suggest that, in the process of knowledge injection, a few high-activation neurons carry most of the relevant information, whereas the vast majority of low-activation neurons contribute little, providing support for precise knowledge editing.
Effect of TransCoder Size and Layer Position. We evaluate the functional fidelity of TransCoder by examining whether replacing the MLP layer with a TransCoder module across different layers affects model performance. As shown in Table model performance remains stable across different TransCoder widths and positions on the G4GD dataset. The accuracy changes are marginal, with some config- urations even slightly outperforming the original MI.. indicating that TransCoder integration does not degrade model capabilities, In addition to performance metrics, we also analyze the error patterns across different model variants. As shown in Appendix Table [4] the distri- bution of top error clusters remains largely unchanged after replacing the MLP with TransCoder modules. This indicates that the core behavioral char- acteristics of the model are preserved. These results demonstrate the functional compatibility of the TransCoder module with standard MLP layers. It not only preserves predictive performance but also retains error-specific activation patterns, making it a reliable substitute for evaluating and manipulating the model's internal knowledge.
Table 5: Comparison of Overall Accuracy Between Codel and its Variants
Layer
MIP
LTC4
LTC8
LTC16
layer 10
57.50%
56.83%
58.33%
57.83%
layer 19
57.50%
58.33%
58.17%
58.17%
layer 23
57.50%
57.67%
57.33%
56.12%
AVE
57.50% 57.61% 57.94% 57.37%
CONCLUSION
Existing locate-and methods are built upon the ROME approach. Although these methods improve performance, they still lack sufficient interpretabilityy making it difficult to understand how knowledge is injected into the MLP layers. Building on this, we propose TCPE, which combines ROME with TransCoder, revealing clear correspondence between the edited neurons and the injected knowledge, thereby laying the groundwork for interpreting ROME-based methods such as MEMIT and PMET. On one hand, TCPE reveals clear correspondence between the edited neurons and the injected knowledge. Through intuitive visualization, TCPE enables developers to transparently track the location of knowledge injection. On the other hand, TCPE's performance gains on TransCoder suggest that ROME-basee methods' limited specificity stems from the polysemanticity of MLP neurons, as these edits can induce unintended interference beyond the target scope. This suggests a potential direction for future work: identifying and isolating sub-representations within polysemantic neurons that correspond to specific facts may be crucial for enhancing editing specificity.
BIBLIOGRAPHY
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoe Le, and Charles Sutton. Program synthesis with large language models, 2021.
07732
Joseph Bloom. Open source sparse autoencoders for all residual stream layers
http://wwwweeeerreoccccc///////s open- wwwurce-ppaaseeeeeeeeeeeesessess oatpaaaeeeaaatttoooooooooooooorr
of smallsmall, 2024. URL
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri,
10

---

Published as a conference paper at ICLR 2026
Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel WerberrtVosss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021. URL htttps://arxiii org wttp .03744
Colin Clement, Shuai Lu, Xiaoyu Liu, Michele Tufano, Dawn Drain, Nan Duan, Neel Sundaresan, and Alexey Svyatkovskiy. Long-range modeling of source code files with eWASH: Extended window access by syntax hierarchy, November 2021. URL https://////tttttooo emm1p-main. 3877
org/202g
Edgar F Codd. A relational model of data for large shared data banks. Communications of the ACM, 13(6):377-387, 1970.
Jacob Dunefsky, Philippe Chlenski. and Neel Nanda. Transcoders find interpretable IIm feature circuits. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances Neural Information Processing Systems, volume 37, pp. 24375 24410. Curran Associates, Inc., 2024. URL http:: 0tt////4600044000066666666666600
neurips. ccppaper__pppp files/paper/2022 pdf
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-oodde Danny Hernandez, Andy Jones, Jackson Kermion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olahş. A mathematical framework for transformer circuits, 2021. htlpshtps::///c httptsss/rrrraafoomm---rrrreeeee pub/2021/framework/ innkkx/ html.
Jessie Galasso, Michalis Famelis, and Houari Sahraoui. Fine-grained analysis of similar code snippets. In International Conference on Software and Software Reuse, pp. 3-21. Springer, 2022.
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang. Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406. 12793, 2024.
Sam Glucksberg. Commentary: The functional equivalence of common and multiple codes. Journal of Memory and Language. 23(1):100, 1984.
Suriya Gunasekar, Yi Zhang. Jyoti Aneja, Caio Cesar, Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi Li. Textbooks are all you need, June 2023. URL hhtpps: textbooks-areeeeeeeeeoeee a1w--al1yya yoll-you--eed.
Chenlu Guo, Yuan Wu, and Yi Chang. Nlora: Nyström--initiated low-rank adaptation for large language models, 2025. URL
14482
Daya Guo, Canwen Xu, Nan Duan, Jian Yin, and Julian McAuley. Longcoder: A long-range pre-trained language model for code completion, 2023.
Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. Large language models for software engineering: A systematic literature review. 33(8), December 2024. ISSN 1049-331X. doi: 10. 145/3699689 URL
https:///oi org/ 10. 00e.1145.. 1145/369598
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Interna- tional Conference on Learning Representations, ICLR 2022, Virtual Event. April 25-29, 2022. OpenReview.net, 2022. URL http:s/s/poeeeeeeee.ee/e//o/otcs

---

Published as a conference paper at ICLR 2026
Rasha Ahmad Husein, Hala Aburajouh, and Cagatay Catal. Large language models for code com- pletion: A systematic literature review. Computer Standards Interfaces, 92: 103917, 2025, ISSN 0920-5489. doi: https://dbii.or//0..00/00 11tp://////.1.1111..........0.00 wtt2:/...s...22133991999 URL wttp://www.cceeeeeerreetttttttt. http://cceeoeeerrrriee//////2/23
Aminul Islam and Diana Inkpen. Semantic text similarity using corpus-based word similarity and string similarity. ACM Transactions on Knowledge Discovery from Data (TKDD), 2(2):1-25, 2008.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. /ttpsss:s///axxiv... abs/2310. 06825
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation, 2024. URL httttt/s///axxiiii http:/iiiv.rr////////////// 00515
Connor Kissane, Robert Krzyzanowski, Joseph Isaac Bloom, Arthur Conmy, and Neel Nanda. Interpreting attention layer outputs with sparse autoencoders, 2024a. /ttps::/s///aaxxivv... abs/2406. 17759
Connor Kissane, Robert Krzyzanowski, Arthur Conmy, and Neel Nanda. Attention saes scale to gpt-2 small, 2024b. URL
Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven C. H. Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning, 2022.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Roger Levy and Lucia Specia (eds.). Proceedings of the 2 s Conference on Computationnl Natural Language Learning (CoNI.. 2017), pp. 333 333-342, Vancouver, Canada, August 2017. Association for Computational Linguistics. doi: 10. 10.18653/v1/K11-10000 URL https: httpss//aclaathhllggy.. wttp:///y..oo////1144400
Jeffrey Li, Alex Fang, Georgios Smymiss Maor Ivgi, Matt Jordan, Samir Yitzhak Gadre, Hritik Bansal, Etash Guha, Sedrick Scott Keh, Kushal Arora, et al. Datacomp-lm: In search of the next generation of training sets for language models. Advances in Neural Information Processing Systems, 11111420--42222 2024a.
Raymond Li, Loubna Ben allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia LI, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Joel Lamy-Poirien Joao Monteiro, Nicolas Gontier, Ming- Ho Yee, Logesh Kumar Umapathi, Jian Zhu, Ben Lipkin, Muhtasham Oblokulov, Zhiruo Wang, Rudra Murthy, Jason T Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Urvashi Bhattacharyya, Wenhao Yu, Sasha Luccioni, Paulo Villegas, Fedor Zhdanov, Tony Lee, Nadav Timor, Jennifer Ding, Claire S Schlesinger, Hailey Schoelkopf, Jan Ebert, Tri Dao, Mayank Mishra, Alex Gu, Carolyn Jane Anderson, Brendan Dolan-Gavine Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro Von Werra, and Harm de Vries. Starcoder: may the source be with you! Transactions Machine Learning Research. 2023, ISSN 2835-8856. URL https: / /ttp::////ppeeeeieeeeee
Reproducibility Certification.
Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang. Jun Ma, and Jie Yu. Pmet: precise model editing in transformer. In Proceedings of the Thirty-Eighth AAAI Conference Artificial Intelligence and Thirt Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence, AAAAT22/1AAA22///AAA... AAAI doi: 11160///aai.v388877..222.
Press, 2024b. ISBN 1-78----7775-8888 org/10. 1609/aaai v38i17. 29818
URL https://doo
Xiaopeng Li, Shangwen Wang, Shasha Li, Jun Ma. Jie Yu, Xiaodong Liu, Jing Wang, Bin Ji, and Weimin Zhang. Model editing for Ilms4code: How far are we? In Proceedings of the IEEE/ACM 47th International Conference on Software Engineering, ICSE '25, pp. 937-949. IEEE Press, 2025. ISBN 15989.1 doi: 10. 10.1109/1SSE55337.2255 .00049. URL https
//doi.org/11 11t.0/11999111ESEE1 2025 00049
12

---

Published as a conference paper at ICLR 2026
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition--evell code generation with alphacode. Science, 37866624) 1097, 2022.
Jiate Liu, Yiqin Zhu, Kaiwen Xiao, Qiang Fu, Xiao Han, Wei Yang, and Deheng Ye. Rltf: Rein- forcement learning from unit test feedback, 2023. URL httpps////ppp semanticscholar.org CorpusID: 259501013
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning, 2025. URL
https://arxiv.org org. www.ooggg///////oooooo..o...s//s 2308 08747
Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreviert net nett/forumiid-unwslggkS.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 nips. cc/paper__ppppp December 9, 2022, 2022. URL ht http///pppp.ppp//a/eeeeeeeeees
(T1143d55822337888b6666
3a1
html
Kevin Meng. Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in transformer. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL / net/Forum2iddMMocHHYY
Gemma Team Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, L. Sifre, Morgane Rivière, Mihir Kale, J Christopher Love, Pouya Dehghani Tafti, L' L'eonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Am clie H' elion, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Chere Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Christian Muraru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, Justin Mao-Jones, Kather ine Lee, Kathy Yu, Katie Millican, Lars Lowe Sjoesund, Lisa Lee, Lucas Dixon, Machel Reid, Maciej Mikula, Mateo Wirth, Michael Sharman, Nikolai Chinaev, Nithum Thain, Olivier Bachem, Os car Chang, Oscar Wahltinez, Paige Bailey, Paul Michel, Petko Yotov, Pier Giuseppe Sessa, Rahma Chaabouni, Ramona Comanescu, Reena Jana, Rohan Anil, Ross Mcllroy, Ruibo Liu, Ryan Mullins, Samuel L. Smith, Sebastian Borgeaud, Sertan Girgin, Sholto Douglas, Shree Pandya, Siamak Shakeri, Soham De, Ted Klimenko, Tom Hennigan, Vladimir Feinberg, Wojciech Stokowiec, Yu hui Chen, Zafarali Ahmed, Zhitao Gong, Tris Warkentin, Ludovic Peran, Minh Giang, Clément Farabet, Oriol Vinyals, Jeffrey Dean, Koray Kavukcuoglu, Demis Hassabis. Zoubin Ghahramani, Douglas Eck. Joelle Barral, Fernando Pereira, Eli Collins, Armand Joulin, Noah Fiedel, Evan Senter, Alek Andreev, and Kathleen Kenealy. Gemma: Open models based on gemini research and technology. ArXiv, abs/2403.0822 08899, 2024. URL /ttpps//s///ppp seman wwwmanticscoolr..oooooooooo ott.aaattssst.oooooooooo/ooo/200
Clé ment Crepy, Daniel Cer,
Joseph Miller, Bilal Chughtai, and William Saunders. Transformer circuit evaluation metrics are not robust. In First Conference on Language Modeling, 2024. URI. http::tt////p/eereiieee...
Forrum2id=z5f8JJy00
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In The Tenth International Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022a. URL https://openrevview
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory- based model editing at scale. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICMI. 2022,
13

---

Published as a conference paper at ICLR 2026
17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 15817-15831. PMLR, 2022b. URL httpps://proceedingss. mlr. press/v162/ (tt mitchel122a. html]
Niklas Muennighoff, Qian Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. Octopack: Instruction tuning code large language models. In The Twelfth International Conference on Learning Representations, 2024. U httpp//pnnriiieeeeeeeooorrrrriie net/Forrum11dmmIIPPWWWZZ
Haowen Pan, Xiaozhi Wang, Yixin Cao, Zenglin Shi, Xun Yang. Juanzi Li, and Meng Wang. Precise localization of memories: fine-grained neuron-level knowledge editing technique for llms. The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL htthttts///ppereeiiiee wtwperrwwwwweeeeeeeeeeeeeeee nww/torummiieeeo (i=
Archit Parnami and Minwoo Lee. Learning from few examples: A summary of approaches to few-shot learning. arXiv preprint arXiv:2003.04291, 2022.
Baptiste Rozière, Marie Mane Lachaux, Lowik Chanussot, and Guillaume Lample. Unsupervised translation of programming languages. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6- 2020, virtual, 2020. URL hash/ed23fbf
new.ips..crippccccoooo httpippp..ccpppppeeeeeep cctpappr/2022///////eeep html
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Eytimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. Code Ilama: Open foundation models for code, 2024. URL
httpss//arri..or//bb//38... 12950
Adam Scherlis, Kshitij Sachan, Adam S. Jermyn, Joe Benton, and Buck Shlegeris. Polysemanticity and capacity in neural networks. CoRR, abs/2210.01880 812,, 2022. doi: 1048550/ARXIV2220001822. URL httpt:tttt...o.oot org/10 48550//ara..///......... 2210. 01892]
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomput. 568(C), February 2024. ISSN 0925- 2312. doi: mel.0/6/..euccmmm22233.... 127063
URL httpt::tt/ooooooooo org/ 100. 1016/ neucom. 2023
Tao Sun, Linzheng Chai, Jian Yang, Yuwei Yin, Hongcheng Guo, Jiaheng Liu, Bing Wang, Liqun Yang. and Zhoujun Li. UniCoder: Scaling code large language model via universal code. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1812-1824, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 18.18653/v1/ ae.4.cc--ong.... 100.
wwwp://4...2..... acl- 100/
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Wise: rethinking the knowledge memory for lifelong model editing of large language models. NIPS 224 Red Hook, NY, USA, 2025a. Curran Associates Inc. ISBN 9798331314385,
Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey, ACM Comput. Surv. 57(3), November 2024. ISSN 0360-0300. doi: 10.1145/3698590 URL https:////// org/ 10. 1145/3698590
Xiaoyin Wang and Dakai Zhu. Validating Ilm- -lnegreertted programs with metamorphic prompt testing. arXiv preprint AX1v:2406.06864 2024.
14

---

Published as a conference paper at ICLR 2026
Yue Wang, Hung Le, Akhilesh Gotmare, Nghi Bui, Jumnan Li, and Steven Hoi. Codet5+: Open code large language models for code understanding and generation, December 2023. URL emnlp- emn.p-main. 68/
https://ac_aatioooggy.or////
Zora Zhiruo Wang. Akari Asai, Xinyan Velocity Yu, Frank F. Xu, Yiqing Xie. Graham Neubig, and Daniel Fried. CodeRAG-bench:: Can retrieval augment code generation? In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, pp. 3199-3214 14, Albuquerque, New Mexico, April 2025b. Association for Computational Lin- guistics. ISBN 811-8-889166-9-1111 URL https::/aclanthhll.ee org/ 2025. findings-naaci 176/
Anjiang Wei, Jiannan Cao, Ran Li, Hongyu Chen, Yuhui Zhang, Ziheng Wang, Yaofeng Sun, Yuan Liu, Thiago SFX Teixeira, Diyi Yang, et al. Equibench: Benchmarking code reasoning capabilities of large language models via equivalence checking. arXiv preprint arXiv:2502. /2466, 2025.
Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: empowering code generation with oss-instruct. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. 24. JMLR.org, 2024.
Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. The butterfly effect of model editing: Few edits can trigger large language models collapse. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Findings of the Association for Computational Linguistics: ACL 2024, pp. 5419 5-337, Bangkok, Thailand, August 2024a. Association for Computationnl 00533//1/222.inddinnes findings- ac1. 322/
Linguistics. doi:
URL httttpss//acllaattoooooooorrrr wrg.looo..org//00
Wanli Yang. Fei Sun, Jiajun Tan, Xinyu Ma. Du Su, Dawei Yin, and Huawei Shen. Understanding the collapse of lims in model editing, 2024b. URL /ttps:://axxii www.:/ogggg////eeeee 11263
Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. Repocoder: Repository-level code completion through iterative retrieval and generation, December 2023. URL
org 2023. emnlp-main. 1517
Lei Zhang, Yunshui Li, Jiaming Li, Xiaobo Xia, Jiaxi Yang, Run Luo, Minzheng Wang. Longze Chen, Junhao Liu, Qiang Qu, et al. Hierarchical context pruning: Optimizing real-world code completion with repository-level pretrained code 1lms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 25886-25894, 2025.
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. Modifying memories in transformer models, 2020a. URL
abs/2012. 00363
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. Modifying memories in transformer models. ArXiv, abs/2012.00363, 2020b. URL https: /taps:///pp... www.nticscholar...... Seemannt.ccccooo.rrrooooooooooos
Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. Deepseek-coderr-2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:2406. 1931, 2024.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
Appendix Contents
Related Work
16
Limitations and Broader Impact
LLM Usage Statement
18
Benchmarks and Baselines for Knowledge Editing Evaluation
D.
Datasets
D.2
Evaluation Metrics for Knowledge Editing NLP Benchmarks
19
D.3
Baselines
99
TCPE
Performance Results
20
E.1
Impact of Knowledge Editing Methods on Overall Model Performance
20
E.2
Intormation-Carrylin Capacity Active Neurons in Single-Erre Editing Scenario
20
E.3
Understanding Low Specificity in ROME-based Knowledge Editing
22
TCPE
Interpretability Results
F.1
Interpretable, Potentially Interpretable, Uninterpretable, Context-Free Examples
23
F2
Detailed Results of Interpretability Experiments on Active Neurons
24
F.3
Detailed Results of Interpretability Experiments on Inactive Neurons
25
Active Neuron Overlap Analysis
26
Metries for Neuron Activation Overlap Across Error Clusters
26
Analysis of Active Neuron Overlap between MLP and TransCoder
26
Fine-graiee Fine-grained Causal Intervention
Constructing Correction Knowledge Four Tuples
29
G4GD Dataset Details
29
J.1
Rationale for Low-Resource Translation in Knowledge Editing Evaluation Evaluation of G4GD Dataset with MBPP and HumanEval
29
J.3
Complete List of Error Messages by Cluster CodeLlama
J.4
Error Cluster Statistics and
Examples in CodeLlama
1.5
Comparison of Top-8 Error Clusters in CodeLlama, LTC4, and LTC8
J.6
Examples of Prompt and Output
J.7
Example from the G4GD dataset
34
Training Details for Knowledge Editing Methods
K.1
Hyperparameter Settings for Code Task
K.2
Hyperparameter Settings for NLP Task
Training Details For TransCoder Modules
Experimental Configuration
Training Time
TransCoder Adapter: Fast Integration of the TransCoder Module
37
A RELATED WORK
Code Large Language Models. Recent advancements in Code LLMs have yielded substantial progress along four key dimensions: data quality, model architecture, training methodology. post- training refinement, and retrieval-augmmented generation. On the data-centric side, efforts such as StarCoder (Li al. 2023) introduced "The Stack v1.2", high-quality, deduplicated, and permissively licensed code dataset, while Magicoder (Wei et al., 2024) proposed Ins-uuttt to synthesize diverse programming tasks from open-source repositories. Phi-1 (Gunasekar al. 2023) demonstrated that compact yet high-quality synthetic corpora, such as "texibook-style" data, can yield strong model performance. Furthermore, Mistralleerme--odee enhanced the capabilities of Mistral-7B (Tiang et al. 2023) through comprehensive multilingual fine-tuning on over 200,000 diverse code exam- ples. With respect to architectural and training innovations, TransCoder (Rozière et al. 2020) integrated denoising auto-encoding and back-translation, marking the first successful application of unsupervised program translation the code domain. Code Llama (Rozière et al. 2024) incorporated
infilling-awee tokenizers and extends context length through RoPE scaling (Su al. 2024). while
LongCoder((Gung et al. 2023) addressed long-range dependencies via a sliding window attention mechanism (Clement et al. 2021
16

---

Published as a conference paper at ICLR 2026
In terms of post-training optimization, approaches like WizardCoder (Luo al.
2024 employed Iroo-Instruct for iterative instruction tuning, and OctoPack (Muennighoff et al. 2024) generated
realistic development tasks based GitHub commit histories. Reinforcement learning has also proven effective, CodeRL. (Le et al. (022) utilized unit tests and critic scores as reward signals, whereas RLTF fine-tuned models based test pass rates (Liu et al. 2023). Gemma (Mesnard et al.. 2024) combined Supervised Fine-Tuning (SFT) with Reinforcement Learning from Human Feedback (RLHF), enhancing its capability in natural language understanding for code-related tasks. Finally, retrieval-augmented generation techniques have emerged as a powerful paradigm. In this context, RepoCoder (Zhang et al., 2023) and CodeTT++ (Wang et al.) (023) combine retrieval mechanisms with pre-trained code language models to enhance diverse programming tasks.
However, Code L.MSs are inherently static, and updating their internal knowledge to correct er- roneous behavior remains a fundamental challenge. Conventional methods such as full retraining or fine-tuning are computationally intensive and prone to catastrophic forgetting (Luo et al., 2025 2024). Alternatives like prompt engineering or
Li et al. 2024a Zhu et al., 2024 GLM et al.)
memory-basee augmentation offer only superficia fixes, as they do not alter the model's internal
representations (Wang et al., 20256 Wang & Zhu
2024 Zhang al. 2025
Knowledge Editing Technology in NLP. Knowledge editing aims to modify specific factual as- sociations in pre-trained language models without retraining from scratch or degrading unrelated knowledge. Early works, such as MEND (Mitchell et al. 2022a), introduced collection of small auxiliary editing networks for fast localized editing. Similarly, SERAC (Mitchell et al. 2022b
formulated editing as constrained generation and stored edits in an external memory, though it suffered from generalization issues as the memory size grew. Later, ROME (Meng al.) 2022) proposed directly intervening in the Transformer's feed-forward network (FFN) weights by com- puting rank-one updates, achieving strong editing performance with minimal side effects. Building on this, MEMIT (Meng et al.) (023) extended the idea to batched multi-edit settings by applying a series of localized updates to multiple layers, scaling the editing process while maintaining precision. Complementary to these sliting-based approaches, LoRA (Hu et al. (2022) proposed a low-rank adaptation method for fine-tuning LLMs using low-rank matrices. While LoRA is not specifically designed for knowledge editing, it serves parameter-efficien fine-tuning baseline and has inspired hybrid approaches that blend fine-tuning and editing (Guo et al., 2025).
Recent work emphasized fine fine-grained localization and interpretability. PMET et al. 2024b) built on this idea by analyzing the information flow within transformer layers, distinguishing between contributions from Multi-Head Self-Attention (MHSA), FFN, and residual paths (Miller et al. 2024). PMET found that MHSA A primarily encoded seneral-purpon require modification. As result, it restricted updates FFN weights and used their corresponding hidden states for targeted editing. FiNE (Pan et al., 2025) proposed neuron-level editing framework, locating and updating only small number of FFN neurons, which improved the locality and precision of edits. Overall, recent trends in knowledge editing move from coarse- -rasind layer-wiss wise updates to fine-grained neuron-level interventions.
extraction behaviors and thus did not
However, most existing editing methods focus on manipulating FFN weights, where individual neurons often entangle multiple concepts or functions. This polysemantic nature (Scherlis et al. 2022) makes it difficult to interpret the effect of edits and to control them with high specificity (Ellhage cct al. 2021). As a result, even fine-grained interventions at the neuron level may inadvertently affect unrelated knowledge, limiting both interpretability and specificity. Such limitations highlight the importance of structured and interpretable representations as a foundation for precise and controllable model editing.
B LIMITATIONS AND BROADER IMPACT
Limitations. First, TCPE requires architectural modification of the LLM by replacing an MLP layer with a TransCoder. This may not be feasible in closed-source or production models and may introduce subtle behavioral changes. Second, the purpose of TCPE is to serve as a tool for understanding the underlying principles of knowledge editing: it focuses on interpreting the relationship between limited injected knowledge and its corresponding edits, and is not intended as practical method for scrge knowledge updating. Beyond these conceptual limitations, training TransCoders introduces additional memory and compute overhead, and the required sparse autoencoder training
17

---

Published as a conference paper at ICLR 2026
does not trivially scale. This limits its direct applicability large production LLMs. In addition, integrating TransCoder modules increases optimization complexity compared to standard MLP layers; while sparse activation patterns enhance interpretability, they occasionally result in stochastic convergence behavior during gradient-based updates. With respect to empirical evaluation, although TCPE demonstrates consistent improvements in low-resource Java-to-D translation, future work should extend evaluations to other pairs of programming languages and a broader set of code scenarios in order to assess the generalizability and robustness of our results. Such studies would provide deeper understanding of the method's applicability in various scenarios. Finally. TCPE's reliance on maide functional equivalence introduces dependencies on the quality of the test suites, and its behavior under repeated or large-scale edits remains unexamined. Future work should evaluate multi-edit accumulation, and sensitivity to the used set of unit tests.
Broader Impact. From a broader perspective, this work aims to advance safe and transparent knowledge editing for code-oriented large language models. The proposed neuron-level intervention mechanism facilitates the correction of model behaviors and the integration of new programming knowledge without exhaustive retraining, contributing to both open-source research and to practical deployment scenarios. Moreover, by minimizing interference with unrelated knowledge, TCPE provides a more interpretable and reliable model editing approach. To ensure ethical and responsible deployment, future work should establish auditing protocols, verification pipelines, and safeguard mechanisms to prevent unintended consequences of fine-grained model interventions.
C LLM USAGE STATEMENT
A large language model (LLM) was used solely aid in polishing the text, improving phrasing, clarity, and readability. All scientific content, including ideas, experimental design, analysis, and conclusions, was developed entirely by the authors.
D BENCHMARKS AND BASELINES FOR KNOWLEDGE EDITING EVALUATION
D.1 DATASETS
In this section, we provide detailed descriptions of the benchmarks used in our experiments: KFCode (including the G4GD dataset), HumanFival, CounterFact, and zsRE
G4GD dataset is designed to evaluate knowledge editing in the context of low-resource code translation, with functional equivalence as the primary evaluation criterion. Many existing code benchmarks, such as MBPP (Austin et al., 202I). are less suitable for knowledge editing due to their small sample sizes and diverse task types. For example, tasks like "Write a function zip the two given tuples." " are clearly irrelevant to knowledge editing. In contrast, we focus on code translation tasks because they are both a common benchmark in the code domain and provide clear input input-output mapping, which enables more direct observation of how knowledge edits affect model behavior. In this paper, the Java D translation task was selected as an initial test case since D is low-resource language, making the impact of knowledge edits model performance more pronounced. This setup enables us to clearly observe how knowledge editing impacts both the targeted error clusters and the overall model accuracy.
HumanEval (Chen al. 2021) a benchmark for code generation introduced by OpenAT. It consists of 164 hand-written programming problems, each paired with a natural language prompt and a hidden unit test. The dataset specifically designed to measure functional correctness rather than superficial similarity, as solutions are automatically evaluated by executing the generated code against the ground-truft tests. HumanEval has become a standard benchmark for assessing the ability of LLMs to synthesize correct and executable programs from natural language descriptions. In our study, HumanEval is used to assess the overall impact of knowledge edits introduced in G4GD on model performance.
CounterFact (Meng et al. 2022) is large-scale dataset created to test factual knowledge editing in language models. Each instance in CounterFact contains a subject- relation rellaiion-objectt triple, along with alternative target facts, paraphrased prompts, and counterfactual contexts.
18

---

Published as a conference paper at ICLR 2026
The dataset is particularly useful for assessing whether model can not only incorpo- rate newly injected knowledge but also remain consistent when queried in diverse forms. Moreover, CounterFact provides auxiliary contexts that test whether the model can sup- press its original memorized facts in favor of the new edits, making it a challenging and comprehensive benchmark for evaluating editing specificity and generalization.
zsRE (Levy et al. 2017 a quuestion-answerigg dataset commonly used to evaluate the zero- shot generalization abilities of models. Each sample includes a question, an original answer, and a revised target answer, making it suitable for evaluating whether editing methods can effectively modify factual associations in LLMs. Compared with CounterFact zsRE focuses more on the ability of model to adapt to factual corrections in a caessiionanssering setting, where robustness to paraphrasing and generalization across different query formulations key evaluation aspects.
D.2 EVALUATION METRICS FOR KNOWLEDGE EDITING IN NLP BENCHMARKS
For CounterFact and zsRE, we follow the experimental protocols of Pan et al. (025). focusing on three key metrics: Efficacy, Specificity, and Generalization. Let s, denote the subject of a factual triple, rj the corresponding relation, and O the target object. Furthermore, p(sj.Ta) ry) represents the model's prompt constructed from (sj. g denotes the original model, and G' the post-edited model. Each metric is formally defined as follows.
Efficacy quantifies whether the post-edited model produces the expected output and is formally defined as:
Efficacy Re [arg max Porly p((,,r,))
=() =o})
Generalization assesses the model's ability propagate the edited knowledge to related real-world contexts:
Generalization = ((ee),y;o3)>D0 Mfrrg max Po p(,,, o3}
Specificity examines whether the edit affects only the targeted knowledge without influenc- ing unrelated information. It can be quantified as:
Specificity
E ((ssr)) I(arg max Po /y/ (P(s),ry) arg max Poly (y)).
D.3 BASELINES
We compare our method against set of representative knowledge-editing baselines:
ROME (Meng al. 2022) directly intervenes in the Transformer's feed-forward network weights via rank-one updates. By targeting specific neurons, it enables precise modifica- tion of factual knowledge with minimal side effects, particularly effective in single-edit scenarios. However, its specificity can be limited when multiple edits are required due to the polysemantic nature of neurons.
MEMIT (Meng et al. 2023) extends ROME to multi-edit scenarios by applying localized updates across multiple layers. This enables the model to edit several factual associations in a single pass while maintaining high editing effectiveness, making it suitable for large-scale
knowledge updates.
PMET (Li et al. (024b) improves precision in model editing by separating FFN-specific hidden states from those of MHSA, using only the FFN-relevant component to update FFN weights.
Fine- Tane-Tuning (FT) (Zuu et al. 2020b) is a standard approach where part or all of the model weights updated new data. While FT can successfully inject new knowledge, it is intensiive. computationally costly, and prone to unintentional degradation of unrelated knowledge, making it less targeted than specialized editing methods.
19

---

Published as a conference paper at ICLR 2026
AGRACE (Li et al. 2025) is model editing method designed for code LLMs, which leverages an external memory and a contrastive learning mechanism to correct erroneous knowledge actoss multiple editing instances. However, its evaluation relies text- exxt-baeed validation metrics, which do not align with the conventional evaluation principle for code LLMs, namely the functional equivalence principle.
Few-shot (Parnami & Lee 2022 methods rely in-context learning, providing the model with a small set of demonstration examples at inference time to modify its output behavior. While these methods are lightweight and flexible, they do not permanently modify model weights and thus offer limited long-term effectiveness and precision.
FiNE (Pan et al. 2025) targets neuron-level interventions across multiple MLP layers with high granularity, focusing on specific subspaces of active neurons, thereby achieving precise edits while minimizing cross-interference
WISE (Wang et al., 2025a) introduces dual-memory framework, where the main memory preserves pretrained knowledge while a side memory stores edits. router dynamically selects between the two, and a knowledge-sharding mechanism is employed to avoid conflicts during continual editing.
LoRA (Hu et al. 2022) introduces trainable low-rank matrices to adapt pre-trained models efficiently. By freezing the original weights and updating only the lightweight adapters, LoRA achieves substantial reductions in computational and memory costs while preserving model performance. Although not originally designed for knowledge editing, it is widely adopted as a parameter-effice efficceen fine-tuning baseline.
E
TCPE PERFORMANCE RESULTS
E IMPACT OF KNOWLEDGE EDITING METHODS ON OVERALL MODEL PERFORMANCE
Following the injection of Java-D correction knowledge (Section ROME-based methods modify the internal parameters of code LLMs. Table shows how these edits affect the models' over- all performance via HumanEval. The RE reflects the global impact of the edit on the model's
overall accuracy, while ACpost and ACpre de-
Table 6: Effects of Knowledge Editing on Code
note the model's post- and pre-edit functional accuracy, respectively. In Table TCPE main- tains the original functional accuracy (ACpost ACpre), indicating minimal disruption to the model's overall code generation capabilities. In contrast, other ROME- based methods, such as ROME, MEMIT, and PMET, update thousands of neurons (1 1111000-5,,,, 5,40) and exhibit lower reliability with noticeable reductions in post-edit accuracy. These results underscore the advantages of TCPE's 's neuron- approach. By performing highly selective updates on specific neurons, TCPE enables precise knowledge injection while minimizing unintended side effects on the model's broader behavior.
LLMs via HumanEval.
Method
RE
ACpost
AC
Neurons
ROME
93.61
26.83
28.66
11,008
MEMIT
76.45
21.91
28.66
55,040
PMET
53.18
15.24
28.66
55,040
TCPE (LTC4)
100.00
29.27
29.27
69
1ww_jee__ckess2200 2000 pusstr
E.2 INFORMATION-CARYTIO CAPACITY OF ACTIVE NEURONS IN SINGLE-ERROR EDITING SCENARIO
In the single-error editing scenario, we further investigate the information-carryinn role of neurons in knowledge editing to support precise modifications, employing TCPE to conduct conventional experiments on high-activation neurons alongside ablation studies targeting low-activation neurons on the G4GD dataset.
Role of Active TransCoder Neurons in Single-Error Editing Scenario. Based on the results in Table 7 we examine the performance of single error types Co and Cs under different activation thresholds (acv). For both Co and Cs. updating only a small number of highly activated neurons (i.e., acv 0.1) results in noticeable improvements in generalization and reliability. As the threshold is
20

---

Published as a conference paper at ICLR 2026
lowered to > 0.05 to include more neurons, the improvement in generalization and reliability becomes limited. Moreover, when more low-activation neurons (i.e., acv >> 0) are introduced, both generalization, specificity, and reliability experience varying degrees of decline. This emphasizes the importance of focusing updates on neurons with high activation to achieve effective knowledge injection. Consequently, precise targeting of these highly activated neurons is critical for achieving effective and reliable knowledge injection.
Table 7: Role of Active TransCoder Neurons in Single-Error Editing Scenarios: Error Clusters Co and Cg. In LTC4, only neurons with activation values (acu) exceeding the specified thresholds E {0, 0.01, 0.05, 0.08, 0.1} are updated. The term "Updated Neurons" refers to the count of neurons that surpass each threshold.
Method
Efficacy
Specificity
Reliability
Updated Neurons
GNT
CD
LoCt
DTL
RE
AC
AC ACpre
(CC)
0CV 0.1
38.46
51.28
83.24
6.29
103.71
60.50
58.33
29
QCV > 0.08
43.59
41.03
82.35
6.86
103.71
60.50
58.33
32
aCV > 0.05
43.59
41.03
82.17
6.86
103.71
60.50
58.33
41
acv 0.01
41.03
46.15
82.71
6.86
103.43
60.33
58.33
54
acV > 0
41.03
46.15
82.71
6.86
103.43
60.33
58.33
57
(TC4(Ca)
acv 0.1
28.85
28.85
97.45
0.86
104.00
60.67
58.33
acv 0.08
34.62
15.38
95.62
1.43
106.00
61.83
58.33
OCU 0.05
34.62
13.46
95.62
1.43
106.00
61.83
58.33
21
0.01
28.85
13.46
95.07
1.71
105.14
61.33
58.33
29
> 0
28.85
11.54
95.07
1.71
105.14
61.33
58.33
30
Ablation Study of Active TransCoder Neurons in Single-Error Editing Scenario. In this section, we conduct an ablation study to examine the role of active TransCoder neurons in the context of single-error editing. In Table we observe that as highly activated neurons are excluded from the update set, the model's generalization ability completely collapses (GN = 0 across all thresholds). Even when a large number of neurons are updated (e.g., up to 16,370 in Cs), the model fails to exhibit any effective generalization. Although specificity and reliability metrics remain high, this due to the lack of real intervention. These stable values suggest that the model has not made meaningful modifications and has failed to successfully perform the intended edits. Further analysis reveals that even when the number of updated low-activation and inactive neurons increases (c.g., from 16,327 to 16,355 in Co. from 16,354 to 16,370 in C&), improvement in generalization occurs. This suggests these neurons play a negligible role in the learning process. This highlights that crror-correction knowledge is primarily carried by highly active neurons, while large number of low-activation and inactive neurons carry little to no knowledge.
21

---

Published as a conference paper at ICLR 2026
Table 8: Ablation Study on the Role of Active TransCoder Neurons in Single-Error Editing
Scenarios: Error Clusters Co and Cs. In LTC4, only neurons with activation values (acv) below specified thresholds {0, 0.01, 0.05, 0.08, 0.1} are updated. Here, "Updated Neurons" represents the number of neurons that meet each threshold.
Efficacy
Specificity
Reliability
Updated
Method
GNT
CD
LoCt
DT4
RET
AC ACpost
AC Cpre
Neurons
LIC4 (Co)
acv $ 0.1
0.00
100.00
99.64
0.00
100.00
58.33
58.33
16,355
acu & 0.08
0.00
96.30
99.48
0.29
100.00
58.50
58.33
16,352
x 0.00
005
0.00
100.00
98.95
0.29
100.86
58.33
58.33
16,343
100
23 1000 a2D
0.00
100.00
99.65
0.29
100.00
58.33
58.33
16,330
aaD
0.00
100.00
98.54
0.86
99.71
58.50
58.33
16,327
(C3) ACCU
110 22
0.00
100.00
99.64
0.29
99.71
58.83
58.33
16,370
800
aaD 2 0.00
0.00
100.00
98.54
0.86
99.71
58.17
58.33
16,367
acD 2 0.05
0.00
100.00
99.64
0.00
100.00
58.17
58.33
16,363
acv 3 0.01
0.00
96.30
99.48
0.29
100.00
58.17
58.33
16,355
0
0.00
100.00
99.64
0.29
100.00
58.17
58.33
16,354
E.3
UNDERSTANDING Low SPECIFICITY IN ROME-BE BBSSED KNOWLEDGE EDITING
Most existing locate-and-then editing methods are built upon ROME, they are hindered by ROME's limited specificity (Li et al. 2024b). To systematically investigate the underlying causes of this low specificity in ROME-based approaches, other HOME-based methods, including ROME, MEMIT, and PMET, on the NLP benchmarks ZsRE and CounterFact. Since generalization and specificity are inherently trade-offs, in this study primarily focus on the behavior of specificity under the premise of successful knowledge injection, rather than on generalization. As shown in Table 9 TCPE achieves significantly enhanced editing specificity, updating only approximately 0.2% of all neurons. Moreover, PMET demonstrates superior performance compared to ROME and MEMIT because, during knowledge injection, it discards irrelevant or redundant information, thereby reducing interference and improving editing precision. This indicates that updating MLP neurons may cause different concepts or knowledge fragments encoded within the same polysemantic neurons to interfere with each other, resulting in unintended side effects. In contrast, TCPE selectively identifies neurons relevant the target knowledge and modifies only these neurons, thereby minimizing inter- ference and achieving higher specificity. In summary, the low specificity of ROME- methods primarily arises from the polysemantic nature of neurons, which encode excessive information unre- lated to the target knowledge injection. This suggests potential direction for future improvement: identifying and isolating sub-representations within polysemantic neurons that correspond specific facts may be key to further enhancing editing specificity.
conduct comparative study of TCPE and
Table 9: Quantitative Evaluation of TCPE and ROMe-based Methods on NI.P Benchmarks.
Dataset
Metric
TCPE
PMET
MEMIT
ROME
Elf.
97.8
96.9202
97. 220
97.7
Gen. 50.1g11
57.3169
55.9
56.5
ZRRE
Spe.
70.9_aa
64.6110
49.7
Newrons
30--50
55,040
$5,040
11,008
Eff.
98.3ge
98-1g
18.220
99.5gn
Gen. 49.1
51.8
63.1
58.7a1.
CounterFact
Spe.
62.5g11
58.7g07
43.3
48.9
Neurons
30-550
55,040
$5,040
TCPE: It leverages MTCC4 with new 1
22

---

Published as a conference paper at ICLR 2026
F TCPE INTERPRETABILITY RESULTS
F.1 INTERPRETABLE, POTENTIALLY INTERPRETABLE, UNINTERPRETABLE, CONTEXT- EXAMPLES
In this section, we provide some examples to support the analysis in Section
Between 2.44 and 3.59: 0.0000%
Between 2.51 and 3.13: 0.0022%
unn Co., bill 236
Example 22721, token 29
lint
Example 13827, token 31
Example 20762, token 118
Between 1.28 and 2.44: 0.0167%
3f Crame mmc))
Exempic 6588, token 11
Stable ind
int. max
Examplc 12400, token 19
(oel (nne
retum Example 18612, token 11
5k0 NE will Example 16909, taken 73
xx. Im
oJl Example 9856, token 20
Example 19055, token 22
Ciode [b]
=- Example 25127, token 11
INF=Ione
Example 18544, token 37
اα ketumi
Example 7659, tolken 12
IXF= lons.
Example 12073, token Example 12277, token Example 6317, token 23
3111//20 (K
Example 18341, token 59
Lose Loog ma
KU, KOBS
ker Cn.
Inad Inad Ioe Example 18111, token
Example 4459, token 105
Ini.
Looa Losa ima
wate
Example 11945, token 118
v Between 1.88 and 2.51: 0.0707%
long
foreach Example 9119, token 103
keyz TI
Example 14732, token 16
Example 824, token 66
[ill retumm Example 12741, token 26
v Between 0.12 and 1.28: 11.1386%
retum becal Example 5977, token 12
32 Caull 1 of
Example 5751, token 83
(a) 7: lonsuod a Example 20136, token 5
Mi Example t445,token token 10
strtnes: scan Keli Example 9638, token
forcoch CRR
Example 1784, token 96
cap) reserve Loap Example 2877, token 65
5o20 EE
wrlteenn Example 3131, token
sallllaa Example 2419, token 104
31 L 1E CE Example 47, token 37
Inta cost (int)s, Tenathi Example 7833, token 13
( Example token 17
Il Inport atd. stdio, std. cony Example 3012, token
H relam Example 877, token 9
Import atd. stdio, std. comsi Example 131, token 106
High-activation examples of feature labeled as in- terpretable: This feature was annotated as local con- text feature that activates when describing the closing part of paired symbols. For instance, tokens such as "Y", ")", or "I" tend to trigger this feature.
(b) High-activation examples of feature labeled This feature was annotated as a potential local context feature related to the semantic concept of "max". While it shows some consistent ac- tivation patterns, it remains unclear whether it reliahly represents this concept.
Between 2.52 and 4.37: 0.0020%
Between 6.28 and 7.85: 0.0003%
BAY rol Mlssand mi Bi Exa mple 6361, token
B- "Elrad Example 35, token
10011 Example 590T, token 20
auto tli
#nesloog! Example 3597, token 95
[03:1 wate Ioput Example 25014, token 78
IM
nol
MI TIOTT Example 9274, token 82
intobwight ie aed Example 8193, token 99
Din Example token 21
0lll anta [ Example 1604, token 100
[ac|]
aaa Example 12649, token 105
koan Esa Rili Example 5907, token 30
Frad
0:l Tong -x Example 8010, token 125
Between 0.68 and 2.52: 0.8873%
Between 4.71 and 6.28: 0.0004%
Example 11724, token 97
500s
Example 7684, token 20
S Example 11975, taken 114
Example 4287, token 103
lease In
[Example token 114
Example 2217, token 64
8
Example 13058, taken 78
Example33433 token 54
8. [ Cay k/
Example 17601, token 13
Example 143, token 98
Twan mo 7F
Example 22043, token 100
Example 68, token 89
tiriiee Example 11131, token Example 10291, taken
D609
Alesand Example 777, token G4
IR "Mex"." "TII""
C.
Example 1381, token
"ea"" mou"." Example 1238, token 113
Example token
Bii Ii MHRLLL Example 9274,token 84
Between -1.16 and 0.68: 99.0799%
Between 3.14 and 4.71: 0.0018%
Poper Mi:
Example 43, token 10
88 Tt THE n:M Example 11260, token 122
soldeod fret lonext Jons'l Example 123, token 38
elasa EGe Escoptione Throvlabie licbis Example 5857, token 31
20t5 998248 Example 448, taken 22
class ECF Excectionl Throvable lithis Example 25002, token 17
(c) High-activation examples of feature labeled as un- interpretable: This feature deemed uninterpretable because text fragments that trigger show no ap- parent semantic or structural consistency, making its meaning difficult to infer.
(d) High-activation examples of feature labeled as context-frre:: This feature annotated as a single- token feature, specifically activating on the occurrence of "th" in the middle of a word. It appears to fire inde- pendently of the broader linguistic context.
Figure 3: Examples of "feabuuredashhooardd"" used in the feature interpretation experiments.
23

---

Published as a conference paper at ICLR 2026
F.2 DETAILED RESULTS OF INTERPRETABILITY EXPERIMENTS ON ACTIVE NEURONS
In this section, we present detailed results that expand upon Section 4.2 In Figure 4 using all Java and D samples in Cs. show some results from the top-10 most active features by examining their top-activating examples. These features consistently respond to key tokens such as 'str', 'string'. or '=.length', which are directly associated with the corresponding error. (Additional results are given in the supplementary material.)
Between 3.54 and 4.43: 0.1221%
Between 3.30 and 3.96: 0.0000%
Sanill im Someled Example D, token 64
lastil Length Example
Birtnn ate lergthi Example 49, token 121
Between 2.64 and 3.30: 0.0326%
Garsi Lenpth? Example 65,
listi Kass leagth Example 90, token 89
Example 73, token 14
0i0e ase. Temghhe Example 78, token
① lottel Froree logthh Example 48, token 116
Milii Wont Example token
stril Strol Combi leretal: Example token 81
Example taken
Latkotlni Lenghtic Example 65, token 28
Between 1.98 and 2.64: 0.1790%
Example 32,
Between 2.66 and 3.54: 0.3743%
Minni leng Example
inta Cangthi Example token
(starti
Example 23, token 122
in lesglh Example 48, token 107
linte Eornot Deapth Example token 119
kl 2illinta Ba. length? Example 47, soken 101
Bano sur. Lengch30 Example 27, token 98
Ilenatharreee Boomer. leagth: Example 12, token 118
0A Well Example 10, token 60
Stntia Example 21, token
Miiinto bin. length ( Example 42, token 47
K0ll Bergetti Example token
Alaaileestt loathe mos lLouthh:ar Example 43, token
10 Motteeott Terethl Example 19, token 107
Minmtn ste. leng Example 32,
122
Sociltask Kilee, Den Example 93, token 10
Example token 108
aillintk lenethi Example 3, token 61
Mllattosttle 1o Example token 25
MilintelBurraa 1e Example token Between 1.77 and 2.66: 0.5615%
Between 1.32 and 1,98: 0.3906%
kil leanats Example Example 66, token prel lomi D200 Example 87, token 73
lengthi: Example 41, token
[1rat)) llesr
Strol
Example 88, token
looti Bote (0 Example 73, token 15
Mek venl (engthh) Example 93, token 10
@0 (ott msl Deoeth Example 20, token 85
Aliintn-ste ste Denethioint Example 12, token 112
Kabla Delccill Example 65, token 29
Milittted ate. longthilaat Example 13, token
(a) Feature Index: 1787.
(b) Feature Index: 13321.
Between 8.58 and 10.72: 0.1546%
Between 6.88 and 8.60: 0.0488%
Length3l
Example 25, taken 2
Insel in led latenng lini
Example 31, token 7
Cadle [istacelnn] I
Example 48, taken
Celereeeer Strang Dedi leurs Example 18,
N- Rte.
Tengthe Example 29,
Example 9, token
Example 17, token
Srreturreem/ tial serine filledi Example 65, token
Pelock
Inbetcciaal Example 66, token 18
no Dill; Berine csoin" Saa:: Example 85, token 29
[e (at bafles Example 3, token 7
[all strieg & Fillee Example token
Ieblitttte ate Trnehhirrttrr Example 52, token
Between 5.16 and 6.88: 0.1139%
(-)) Example token 46
Liite aurimestel ( (inc Example 77, token 11
(X--)151 [6 Example token
Eharl (1l Cauring kte (ooreci Example 62, 117
Ko [I] Example 84, token 93
Example 78, token 5
v Between 6.43 and 8.58: 0.3418%
1000 steingti led Example 86, token 113
Greing wts [ Example 58, token
[retumeeeee Koring 1
led Example 78, taken
Kmolin sts. PharAall Example 86, token
Di0t steingt Iritted Example 69, token
(praserent Example
Jasid
Example
75
Example token
Between 3.44 and 5.16: 0.2930%
6 (iillee (etrriirn (iras) Example 5, token
<o>steing
Example token
Lonntiiree ste. shrrAeld Example 88, token 23
Example 36, token 125
Illttt ata Uooerhiittt Example 77, token
I (ateing lini Example 62, token 39
f Lttitedd (itatia Example token
- String bod belawol Example 71, token 21
Leod (Btrincae (ilii Example 78, token
H 161 corieastri ktringeoreer Example
Iiliatt sta. lel Example 23, token 18
b (orineett)) ctt Example token 116
v Between 4.29 and 6.43: 0.9277%
(arzl: certtttee""""e Example 33, token 38
r. Lirieelllttiaaat ittillttttttittlllllllaa lilata Example 2, token 100
Bhain oftd anrtneorb lin Example 38, token 68
Illintre ste. Dong Example 75, token 112
ias Cflloo teed, lack
Example 34, token 99
(ailiii (iat Example
booldt]filled (f Example token
(c) Feature Index: 8370.
(d) Feature Index: $579.
Figure 4: Top-Activating Examples for Active Neurons.
24

---

Published as a conference paper at ICLR 2026
F.3 DETAILED RESULTS OF INTERPRETABILITE EXPERIMENTS ON INACTIVE NEURONS
This section presents detailed results that complement the analysis Section 4.2 Using all Java samples and D samples in Cs as input, we capture and analyze the patterns of 10 randomly selected inactive features. As shown in Figure although these features exhibit interpretability due the sparsity induced by the TransCoder, they primarily attend to irrelevant tokens such as 'if', 'N', ";', and "ps", indicating limited relevance the target error cluster Cs. (Additional results are given in the supplementary material. As some inactive neurons fail to respond to relevant examples, only 8 results are included.)
Between 3.17 and 3.97: 0.0326%
Between 2.05 and 3.07: 0.3337%
3- while Example 85, token 39
Hitl Mattes lehalti Example 1amplee6,, 46, token 12
JJ
Example 62, token 30
Dlleve) 1oll--lll Helwe [evel Example 79, token
Elaee
Example 62, token 62
88-00 (o Example token
seturnZelae 1148) 148)-81 Example 78, token 56
Buse lemg (Len)-n Example 19, token 112
('illve te (reel Example 79, token
Between 2.38 and 3.17: 0.2930%
paint tall Crrnl Foan Example 88, token
Falaw [++]):
1-10)
Example 12, token 88
se [I (ooSeeenn) Example 38, token 24
Dizetur Fullon
Example taken 48
05l (erLii Example 69, token 71
11011 1810 Ineturn Example 86, token
lenctä Til whille GiC Example 5, token 26
10 Faoll Example 80, token 112
v Between 1.02 and 2.05: 1.4160% Example token
XH R krmppie (ert Example 22, token
10 intii
Example 61, token 25
Example token 42
(0
Cnee Inttrmjemett [| Example 36, token 114
K
(ouih Example token
True aalei a1 mre Example 41, taken 65
nux
Lengtt)
Clentth Example 43, token
Ek) 1k)mmtmm trin menuri Example 1, token 101
L-
Example token 55
1ED Helsedai Example 17, token 43
(uedll Example
Between 1.59 and 2.38: 1.7090%
(K0
fatrl Ichor Example 78,
[
(EE Example 19, token
اаلل Izturrie Example token
kharki Example 86,
30 B0 (jepen Example 49, token 32
0,1-) 10000 Example 91,
Nil l8 sert /ort Example 33, token 32
Between 0.00 and 1.02: 3.0599%
Example 21, token 44
Seleneitrue:: Chett Example 38, token 35
Azreria) Example 51, token 52
Kel0 CeTRP Example 32, token 5
(a) Feature Index: 3990.
(b) Feature Index:
Between 0.63 and 0.79: 0.0163%
Between 1.16 and 1.45: 0.0081%
Hllte imck- [skastiin Example 6, token 20
(orll UppeeeCoee ietunr Example 25, token
(String liittlee Example 81, token 112
Between 0.87 and 1.16: 0.0163%
Between 0.47 and 0.63: 0.0488%
Mellee ae [ Example 94, token 91
31 Stringd N. (nbbttiing(i
Example 36, token 57
Aalk)) as [R Example 84, token 103
[[Stringds (
Example 36, token 32
Between 0.58 and 0.87: 0.0407%
ARStrl kobbtrinn (a- -hll Example 11, taken
GnTen 2 21aetum Example 58, token 41
kelstrl lobstrind ( kll, Example 45, token 26
Melleek [RDC Example 92, token 10
Toneth (atrine bubatring Example 6, token 39
Iu Example 94, token 63
Lelledd de (etringa)) llstringees Example 86, token 119
sttnnk ps
Example token
Between 0.29 and 0.58: 0.0732% 1ik0 Example 62, 84
Between 0.31 and 0.47: 0.0326%
O1) Example 81, token 118 Dlaat M.
Ter
[ ps [B]] Example 94, token 100
[k
Example 69, token 22
1B') ) inn pud Preeiiiii Example 54, token
TiC x20 Pettuctt Example 58, taken
< [1] auxher (all Example 91, token
Ri Nink Example 94, token 31
Example 84, taken
Between 0.16 and 0.31: 0.0407%
[H]] -eore substrirg (
Example 16, token 21
returnn mun isstttiooet Example 64, token 91
B0111 pos Poderenk chtl Example 54, token 20
zee returni [oo T Example 0, token 39
[0000 Example 94, token 53
[Stringss N. substring Example 36, token 33
Between 0.00 and 0.29: 0.0407%
De Example 60, token
024 3MA Example 27, token 57
Jel Irun mun. lsabstring! Example 64, token 47
bretural Example token 87
(c) Feature Index: 8772.
Feature Index: 9831.
Figure 5: Top-Activating Examples for Inactive Neurons.
25

---

Published as a conference paper at ICLR 2026
G ACTIVE NEURON OVERLAP ANALYSIS
To evaluate the overlap of active neurons (acv > 0) across different enror clusters, we select multiple clusters C., where E {0, 1, 6, 153. For each cluster C, we construct five correction knowledge four-tuple (.))) ((.a): (1.5) (.))), where E [1, 5], and compute the corresponding
(2)
representation vectors (.1) (see Equation 6). Each ((j.s) is associated with prefix set (a().)
[1, 20)), which includes ten prefixes of length 5 and ten of length 10. Based on this, we analyze the overlap of active neurons at layer 19 for the MLP of CodeLlama and the TransCoder modules of LTC4, LTCS, and LTC16.
G.1 METRICS FOR NEURON ACTIVATION OVERLAP ACROSS ERROR CLUSTERS
In this section, we quantify the overlap of neuron activation patterns across different error clusters. We define two metrics: (1) Absolute Overlap (AO): The ratio between the union of all activated neurons across clusters and the sum neuron unions within each individual cluster. higher value indicates greater independence between clusters. (2) Relative Overlap (RO): The ratio between the total union of activated neurons and the average number activated neurons per cluster, reflecting the degree to which active neurons are shared across clusters. Formally, these metrics are defined as:
- U Sin AO
U U Si. 1 N Σ Σグ= |Sa NR
RO
Σ U Sty
Here, St,, denotes the set of activated neuron indices for the j-th prompt in cluster C. In addition, we denote the average number of activated neurons across clusters as A₁, and within clusters as Ag- Their corresponding union sets are U1 (cross-cluster) and U22 (intra-cluster). The intersection of active neurons within cluster is defined Iz.
G.2 ANALYSIS OF ACTIVE NEURON OVERLAP BETWEEN MLP AND TRANSCODER
In Table 1I we show the overlap of active neurons in the k (.,0 generated by multiple correction
knowledge (r (j.s)* ((j.)) (5.1)* ((.3)) (j.)) within the same cluster C. Obviously, the active neurons in the TransCoder module are more concentrated compared those the MLP for the same error type. As shown in Table I0 in the MLP layer, the RO value is 1.97 and the AO value is 0,16, indicating concentrated distribution the TransCoder neurons exhibit stronger independence across clusters, suggesting more distinct representations of different errors. The AO and RO values for the TransCoder module consistently remain high, with AO ranging from 0.41 to 0.47 and RO from 6.67 to 8.02. This consistency reflects a more structured and sparse organization of active TransCoder neurons, demonstrating potential for fine- grainnr knowledge editing.
active neurons with substantial overlap across error clusters. In contrast,
Table 10: Comparison of Cross-Cluster Active Neuron Overlap between MLP and TransCoder Modules. In the MLP layer, lower Absolute Overlap (AO) and Relative Overlap (RO) values indicate more distributed set of active neurons with substantial overlap across eтror clusters. In contrast, the TransCoder layer consistently exhibits higher AO and RO values, reflecting more distinct and functionally independent active neurons across clusters.
layer
A
U,
RO
AO
MLP
5514
10836
1.97
0.16
LTC4
117
848
7.25
0.44
LTC8
132
1059
8.02
0.47
LTC16
448
2987
6.67
0.41
26

---

Published as a conference paper at ICLR 2026
Table 11: Intra-Cluster Active Neuron Counts and Overlaps for MLP and TransCoder Modules. Obviously, the TransCoder modules (LTC4, LTC8, LTC16) exhibit significantly more localized and concentrated active neurons (see A2 and U2) compared the MLP in CodeLlama.
Stats
Co
C₁
C2
C4
C
Cs
C
C15
AVE
6455
7502
9144
9279
9305
7548
8214
7413
8108
CodeLIama
12
4543
3538
1841
1822
1715
3449
2807
3646
2920
U2
5495
5520
5470
5521
5500
5532
5533
5538
5514
A2
138
111
108
116
124
113
109
115
117
LTC4
1
95
51
20
20
25
51
34
64
45
U2
191
185
270
306
339
200
236
202
241
A2
147
131
127
134
138
126
122
131
132
LTCS
1
91
67
25
29
29
61
43
63
51
U2
223
214
328
369
389
232
284
233
284
A2
551
452
427
440
430
424
396
454
447
LTC16
12
236
73
92
198
123
263
181
U2
736
1068
1172
1208
746
885
754
916
H FINE-GRAINED CAUSAL INTERVENTION
Based on the causal intervention technique proposed in ROME (Meng al. 2022). introduce fine graingr causal intervention method for token-by- ba-bbby-tttonee analysis. Specifically. y, we first pass input into the model G and collect all clean activations {h: (8) E [1, [1, L}} across layers. Then, we sequentially corrupt a single token 1" by adding noise at the embedding layer h == h: + E. This yields set of corrupted activations {h [1, [1, b}} throughout the network. Next, at each layer, we restore the clean hidden activation 11 for the selected token, while leaving the rest of the activations corrupted. We measure the difference in model outputs between the restored and non-restored cases, producing a table of size T L, where each cell quantifies the causal effect of restoring the hidden state of a specific token at a specific layer.
Building on this procedure, we leverage the fine-grained causal intervention method to systematically assess the contribution of each token in the four-tuple (r(1), s, r2), o) across various programming language scenarios. As illustrated in Figure our analysis reveals that the subject token exerts substantial causal influence on the prediction of the object token o.
27

---

Published as a conference paper at ICLR 2026
Iredd of ceeiiin cat soseeeed
subject
subject
(a) Case 1.1: Impact of restoring state after corrupted (b) Case 2.1:Impact of restoring state after corrupted input
input
Impan drseeern HMF
subject
subject
(c) Case 1.2: Impact of restoring MLP after corrupted input
(d) Case 2.2: Impact restoring MLP comuupted input
subject
subject
(e) Case 1.3: Impact of restoring Attn after corrupted Case 2.3: Impact of restoring Attn after corrupted input
input
Figure 6: Analyzing Each Token Behavior via Fine-grained Causal Intervention CodeLlama.
28

---

Published as a conference paper at ICLR 2026
I CONSTRUCTING CORRECTION KNOWLEDGE FOUR-TUPLES
In this section, we introduce the process of constructing correction knowledge four-tuples (x(1), s, r,) o) for fixing specific error types in Java-to-D programming language translation. More specifically, the process begins by identifying translation failures in the Java-to-D task. For ex- ample, when translating Java functions to D, the translation fails the unit test. Leveraging the associated error messages,
locate the incorrect code snippet o and construct the initial four-tuples (r(1), ,,, ,,,(),,,))))))))()), o). where is the prediction generated by the model based on the prompt p(r(1), 8, r(2)). We then manually correct o to the correct object o", forming the correction knowledge four-tuple (x(1), s, s,r(?),"))
In Figure we provide an example. the Java-to-D translation, we consider prompt p(r(1) 88 r(2)), where the source Java code snippet is "str. length", and the model generates the D code snippet o "str. 1ength()) Here, the generated is incorrect and does not conform to language rules. We refine into the properly formatted D code snippet o" "cast(int) .enggh thereby constructing the correction knowledge four-tuple (y(1), s,,)
>>> Source Jaya
Knowledge Editing Input
static int [ gold ( String str sounc atrppet
Propmt- -
rs):
int n Str length 1
[<>][[NS]] Trumslate the following Java function to its equivalent
(n++ 1)/2:
function in the D programming language (dlang) Only provide ()) completed function.
java
D Transation [incorrect]:
static gold String Sy reprrssentss ( (n Ploo)
int Tstr length Jenghh
int f gold String str int -fstt length
freturn *(n+ 1)2
(n+ 1):
[IINST] ''d
int f gold String str )
Fixed code [correct]:
int
int [ gold String str) f omor
Original Target Object:
length represents in (2")
int n cast (nt]str. length (n+ 1/22
New Target Object:
cast(int)str length ) represeens o (P')) o*))
Figure 7: Construction of Correction Knowledge Four-Tuple s())), 0²) in Java-to-D Code Translation.
J G4GD DATASET DETAILS
In this section, we provide a comprehensive overview of the G4GD dataset, including its structure, evaluation with standard benchmarks (MBPP and HumanEval), and representative examples. In addition, we present detailed error analysis of CodeLlama in the Java-D translation task, covering complete list of error categories, clustering statistics, intra-cluster examples, and comparisons across model variants.
J.1 RATIONALE FOR LOW-RESOURCE TRANSLATION KNOWLEDGE EDITING EVALUATION
Popular existing datasets, such as HumanEval Chen et al. 2021 and MBPP Austin et al. 202 ), have been widely used evaluate the performance of code generation models. However, due the small sample size and the diversity of task types, these datasets are not well suited for knowledge editing. For example, a task such as "Write python function to identify non-prime numbers." are clearly irrelevant to knowledge editing.
In this paper, we instead focus on code translation tasks, which are widely used benchmarks in the code domain and provide explicit input-output mappings, enabling direct analysis of the effects of knowledge editing. As an initial case study, we adopt the Java - D translation task. Since D is low-resource language, knowledge edits are more likely to produce observable changes, allowing us to precisely assess their influence on both specific error clusters and the overall model accuracy. We develop specialized benchmark dataset, G4GD, designed to evaluate low-resource Java-to-I)
29

---

Published as a conference paper at ICLR 2026
language translation. The G4GD dataset comprises 600 Java functions, each paired with suite of unit tests. Every test suite includes 10 independent test cases designed to assess the functional correctness of the translated D code (see Table 16 for example).
J.2 EVALUATION G4GD DATASET WITH MBPP AND HUMANEVAL
To comprehensively evaluate the G4GD dataset, we compared it with existing standard benchmarks, HumanEval and MBPP. In of sample size, the HumanEval dataset contains only 164 examples (each accompanied by 7.7 automated test cases (Chen et al. (02T)) while the MBPP dataset includes 974 examples (each with automated test cases (Austin al.] (02T)). In contrast, the G4GD dataset includes 600 examples, with richer and more comprehensive set of unit test cases, offering more robust and diverse resource for model training and evaluation. Furthermore, as shown in Figure S the code snippets in the G4GD dataset generally longer, which presents greater challenge in testing knowledge editing methods for handling complex functions. This design enables the dataset to more effectively assess the model's knowledge editing capabilities in more complex scenarios, particularly in the translation of long code snippets and complex functions, highlighting the model's 's robustness and adaptability.
Percentage of the total sample (%)
100
Java- D dst(ours)
Length
G4GD (ours)
MBPP
HumanEval
MBPP
HumanEval
97.67%
88.5 %
92.07
len 30
91.83 %
62.83
82.32
len 40
84.67 %
44.56 %
64.02
len 50
78.17 %
31.83 %
48.78 %
len 2 60
69.17 %
23.20 %
41,46 %
len 2 70
61.83 %
17.76 %
35.98 %
20
len 2 80
51.33 %
13.66 %
26.83
IV
44.00 %
10.88
19.51
len 100
38.50 %
8.52 %
15.24
Length
Figure 8: Comparison of Input Sequence Length Distributions in G4GD, HumanEval, and
MBPP. We compare the input code length distributions of G4GD with widely used benchmarks
HumanEval and MBPP. Obviously, G4GD contains significantly longer sequences, making it better suited for evaluating knowledge editing on more complex functions. Here, the r-axis denotes sequence length ranges, and the y-axis indicates the proportion of samples.
30

---

Published as a conference paper at ICLR 2026
J.3 COMPLETE LIST OF ERROR MESSAGES BY CLUSTER IN CODELLAMA
To characterize failure patterns, we first present a complete error message list, as summarized in distinct type of compilation failure. This
Table 12, where each error cluster C₁ corresponds systematic categorization serves as the basis for subsequent error clustering, wherein all failed translation outputs are grouped according to their respective error types.
Table 12: Complete List of Error Categories for Codellama in Java-to-I) Translation
Cluster
Error Message
Co
Error: instead of C-style syntax, use D-style 'int[]l] mat'
C1
Error: "std.matt..lggehriic.. sqrt' called with argument types '(int)' matches both:
C2
Error: found when expecting ':' following statement 'Set (int)"
C4
Error: style cast illegal, use "cast(int)x"
Cs
Error: 'I ====0 O' must be surrounded by parentheses when next operator &&'
C.
Error: undetined identifier
C7
Error: none of the overloads of template
wwa.gorthh.sorting.ss.
Cs
Error: cannol implicitly convert expression 's.length' of type 'ulong' to 'int'
C.
Error: identifier expected following comma
C10
Error: semicolon expected following auto declaration, not '>'
C11
Erorr: can only
pointer, not 'int' Error: expression expected, not ')'
C12
C13
Error: incompatible types for '(startIndex) + ("to"): 'int' and 'string"
C14
Eroo: property 'substring' for 'first' of type 'string'
C15
Error: variable 'n' cannot be read compile time
C18 Error: template instance 'HashSetlint' template 'HashSet' is not defined
C17
TestRuntimeErroo
C18
Error: invalid array operation
C19
Error: 'switch' statement without 'default"
C20 Error: no identifier for declarator 'char'
C21
Error: missing closing ')'
Cz2
Error: slice 'x" is not mutable
C23
Error: unterminated character constant
C24
Enor: function not callable
C25 Error: template argument expected following "!"
C28 Eror:: integer overflow
C27 Error: '10.0' is not of integral type. it is 'double'
J.4 ERROR CLUSTER STATISTICS AND INTRA-CLUSTER EXAMPLES IN CODELLAMA
In this section, we analyze the distribution and characteristics of compilation failures produced by CodeLlama Java-to-D translation. We cluster all failed translation outputs based on their corresponding error types. The resulting distribution, shown in Figure reveals a highly concentrated pattern: the top four clusters (Co, Cs, C4. CB) account for 58.51 1 of all compilation failures. Table 13 provides representative examples from two of the most prominent clusters: Co and Cg. Here, cluster Co contains violations of D-style array declaration conventions, and C4 captures illegal uses of C-style casts.
Table 13: Examples of Error Clusters "Co" and "C4" in Java-to-D Translation for CodeLlama
Cluster
Error Message
D Translation File
Eror: instead of Cistyle syntax, use Distyle 'in[[]]] mat'
COUNT_SORTED ROWS MATRIX
Error: iostead of C-style syntax, use D-style 'int(n dp
FRIENDS SPAIIDNS BAA.AAINGPRBBLLLNAAA
Co Error: instead of C-style synlax, use D-style fini(n 1] dp"
LEONARDO_NUMERI l.d
Enror: instead of C-style syntax, use D-style 'int]UU mat"
MAXIMUM VALUE_MATRIXA
Enorr instead C-style syntax, use D-style (in[[[[] LCStuff'
LONGEST, LONGEST_COMME COMMON
COMMON_SURSTRRIGE SUBSTRING.
Error: instead C-style syntax, use D-style 'int[n] ugly"
Enror: C style cast illegal, use (oorsiitt)
COUNT_DIGITS, SUUNT_DIGTIS_PACTORRAA FACTORIAL SET
Emor: style illegal,
Cast(int/Mat..sqnttn))
MINIMUM PERIMETER N, BLOCKS.d
C4
Brorr: C style cast illegal, use Maasttnn)Math.squru Masssiin)MaasssttyyuSuaeee)
LATTICEEPPINTSS... CIRCLE_LATTICE POINTS.
Eror:: C style cast illegal, use "cast(int)strrlii""
SUM_TWO SUM_TWO_LARGE LATVVO_ARR__UUUUE NUMBERS
Error: C style cast illegal, use 'cast(int)summ
EVEN__IBOONCCC FIRONACCL NUMBERS SUM.d
EVEN
Error: C style cast illegal, use "cast(int)strii]"
LEXICOGRAPIICAII NEXT_ STRING.
31

---

Published as a conference paper at ICLR 2026
Cluster
Count
Cluster
Count
Cp
50
Cs
Cs
47
C3
2
40
C4
26
Czo
2
Cs
18
C18
2
Cp
17
C17
2
C1
15
C13
C2
14
C11
C15
10
C21
20
C14
C19
C16
6
C24
C10
C22
C12
C27
C23
C26
C25
Csac
345
CaaiPaas
Error Cluster
Figure 9: Error Clusters of Codellama on Java-to-D Translation Tasks. By applying error clustering to these compilation failure samples, we found that a significant portion of errors is concentrated in specific clusters. A notable concentration of errors is observed, with the top four clusters Co. Cs, C4, and C together accounting for 58.50% of all compilation failures.
J.5 COMPARISON OF TOP-8 ERROR CLUSTERS IN CODELLAMA, LTC4, AND LTC8
In this section, we compare the top-8 error clusters generated by CodeLlama and its two TransCoder variants, LTC4 and LTC8, which replace the 19th MLP layer with TransCoder modules of varying widths. As shown in Table [4] the distribution of the predominant error clusters remains largely consistent across all models. This stability suggests that the TransCoder effectively approximates the activation patterns of the original MLP layer, preserving the model's behavior despite architectural modifications.
Table 14: Top-8 Error Clusters for Pure Codel Jama and CodeI Variants. Here, each C; denotes specific error type, with the frequency indicated in parentheses. LTC4 and LTC8 refer to CodeLlama variants where MLP layer 19 is replaced by TransCoder module with 4, 096 4 and 096 respectively.
No
MLP
LTC4
LTCS
Cu (50)
(52)
Ca (52)
2
Cs (47)
Co (39)
Co (39)
3
C4 (26)
C4 (27)
C. (27)
4
Ca (18)
Ca (22)
Cs (21)
5
Cg (17)
Cg (18)
Cg (21)
6
Ci (15)
C1o (12)
Ciu (13)
7
C2 (14)
C1 (11)
C₁ (12)
8
Cis (10)
(10)
C2 (7)
Crattess (14)
Crairaus (10)
CraiPas (10)
10
Cunc (345)
Cane (350)
Cane (349)
32

---

Published as a conference paper at ICLR 2026
J.6 EXAMPLES OF PROMPT AND OUTPUT
We provide a prompt that guides the model to translate code into an equivalent function in target language. Table illustraat a specific example where the model prompted translate Java function into its equivalent implementation in the D programming language. The input prompt, enclosed in an 
... 
 block, specifies the task (i.e., translation from Java to D), followed by the source code. The model's output is the corresponding translation in the target language.
Table 15: Example Prompt and Corresponding Output: Translating Java Function to the D Language. Type
Details
<B> (INST) Translate the following Java function its equivalent
function in the D programming language (dlang) Only provide the completed function.
int solution (String
Prompt &
int result - 0;
s.lenghh 0;
Output
for (int i 0;
6i i++)
for (int J - 1; n; j++)
if (s.charèt (1) s.charAt (1))
result-+;
result;
[/TNST] "'d
int polution(string 5)
int result 0;
int (int) s.length;
for (int - 0, << < 1++)
for (int 1= i;
jt+]
if (s[1] = a11l)
reault++;
return result;
</8>
33

---

Published as a conference paper at ICLR 2026
J.7 EXAMPLE FROM THE G4GD DATASET
To illustrate the structure of our dataset, Table 16 presents a representative example from the G4GD benchmark. Each entry in G4GD consists of Java function paired with 10 independently constructed D unit tests. These tests are designed to evaluate the correctness of translated code across diverse inputs.
Table 16: Example from the G4GD dataset: The final G4GD dataset comprises 600 Java functions, each paired with 10 independent D unit tests.
Type
Details
def I. E_gold
[
or n 1
return
fl f2 f3 =
Java function
while f3 11 f2
f1
f3 fl f2
return f2
import std.stdio; import mtd. import std.conv; import atd.algorithm
D unit tests
//TOFILL//
void main()
[] results 134, 55, 55, 55, 89, 34, 55, 89, 89, 55]; int [1 param0 [54, 71, 64, 71, 96, 43, 70, 94, 95, 691;
int n_success 0;
for (int
paramo. Popramo.lengten length; i++] if (resulta[1] == f_filled (param0 (parad [i])
nasuceess 1;
)]
writefln (" [Resultss:d, $d", n_success, paramb. length);
K TRAINING DETAILS FOR KNOWLEDGE EDITING METHODS
K.1 HYPERPARAMETER SETTINGS FOR CODE TASK
ROME The ROME configuration follows the original configuration proposed in the ROME with learning rate of 0.5, mom2_n n_n samples of 100,000, and slamp_norm factor of and prefix distribution of [[5, 10]. [10, 10]]. Here, total of 20 text segments were sampled, with 10 having prefix length and the other 10 a prefix length of 10. The covariance matrix estimated from samples drawn from the "bigcode/the-stact// dataset. The intervention is applied at ML. layer 19 of Codel
MEMIT, PMET Following PMET LLi et al. 2024b) and MEMIT (Meng et al. 2023). we perform grid search over key hyperparameters identify the configuration for code- related knowledge editing tasks. The hyperparameter sweep included: learning rates 00.5, 0.05, 0.01, 0.005, 0.001]. mom2_update_ seighhteettttttee {100, 500, 800, 10000, 15000, 20000), v_ weightt decay {0.5, 0.1, 0.05, 0.01, 0.005). and clamp norm factor (0.5, 0.75, 2, 2, 10, 15). In addition, the covariance matrices are estimated from the "bigeode/the-stack dataset, with mom2_n_san __mmplesss of 100.000. Following the hyperparameter sweep. both PMET and MEMIT adopt the same final configuration: learning rate of 0.01, a mom2_update, update_weight of 500, a y_weight__ decay of 0.05, and clamp_non norm factor of 2. (The experimental setup for PMET, as reported by Li et al 2024b), closely follows the configurations employed for MEMIT.)
34

---

Published as a conference paper at ICLR 2026
Few-shot Few-shot (Parnami & Lee 2022) constructs prompts by directly providing the error message along with the Java-corrected D function, enabling the model to perform targeted code correction with minimal additional context.
FINE Following defalt experimental setup (Pan et al. 2025), we freeze the final three layers and update the preceding layers. A learning rate of 0.001, determined via hyperparameter sweep, is applied, while all other hyperparameters remain consistent with the original FiNE configuration.
LoRA We target the q proj and proj modules with rank of 8 and learning rate of 0.0001, restricting edits to a Transformer layer 19. All other model parameters remain frozen during training. following the default experimental setup described in the baseline configuration.
WISE We follow the default experimental setup for WISE (Parnami & Lee 2022 restricting updates the designated inner parameters. Training is performed with learning rate of 1.0 under an Loe -norm constraint of 1.0, while all other hyperparameters remain consistent with the original configuration
AGRACE We follow the default experimental setup of AGRACE under CodeLlama (Li et al. 2025 restricting updates the designated inner parameters and maintaining all other hyperparameters as in the original configuration.
FT We restrict parameter updates MLP layer 19, which enables efficient knowledge modification while preserving the model's 'ss overall behavior. We train with a learning rate of 0.005 and impose an Los norm constraint (Zhu et al., 2020a) of 0.1, applying early stopping once the loss falls below 0.01 to prevent overfitting.
TCPE the TCPE method, the hyperparameters for LTC44 and LTC8 are configured as follows: learning rate (0.05, 0.005), mom2_n n_ n_samples of 500,000, clamp_ _aoppp__mm____ facttfccrrrrr {13, 18), and neuron {0, 0.001, 0.01, 0.05, 0.08, 0.1, 0.15, 0.2]. Knowledge injection is applied
activation thresholds at layer 19 of the TransCoder module. In this process, the covariance matrix samples from the "bigcode/the-stack dataset. addition, to construct the random prefix distribution, we use a single prefix of length 10. To prevent overfitting,
early stopping mechanism is employed during training. If the negative log-likelihood loss shows no significant improvement over three consecutive steps, training is halted. The tolerance is set to 0.001.
K.2 HYPERPARAMETER SETTINGS FOR NLP TASK
For experiments knowledge editing in natural language tasks using Llama2, the baseline methods ROME, MEMIT, and PMET adhere to the hyperparameter settings reported by Pan et al. (2025). In the TCPE method, the hyperparameters for the LTC4 variant are set as follows: learning rate of 0.5, mom2_ n samples of 500,000, clamp norm factor of 35, and neuron activation thresholds of 1 10-4 Knowledge injection is applied at layer 19 of the TransCoder module, with the covariance matrix estimated from samples of the Wikipedia dataset. To prevent overfitting, an early stopping mechanism is employed: if the negative log-likelihood loss does not improve significantly over three consecutive steps, training is halted, with a tolerance of 0.001.
L TRAINING DETAILS FOR TRANSCODER MODULES
The training of TransCoder modules is guided by a loss function designed to balance faithful approximation of the original MLP outputs with sparsity of the internal feature representations. Formally, the loss function is defined as:
((200)))))) Lrc MMP""( - (1 ","
II,
+*
(8)
faithfullness loss
sparsity loss
The quality and effectiveness of the TransCoder modules mainly depend on two key hyperparameters: the learning rate and the L, regularization coefficient X. This coefficient balancing faithful
35

---

Published as a conference paper at ICLR 2026
Feature count per bin
2000
sparsity
10
8e3
layer 10
-0.5
layer 19
layar
-1.0
23
sparsity
683
layer
1000
4e3
-2.0
500
Mean
2e3
-2.5
0e3
-3 -1
267
467
6e7
8e7
0e7
2e7
4e7
6e7
8e7
Log10 Feature firing Frequency
Training Tokens
Training Tokens
(a) die 4, 096 4
de 4, 096
di 4, 096
0.0
Feature count per bin
Mean log sparsity
layer 10
laye
layer
15e3
aaer 19
3000
-1.0
layer 23
LO sparsity
layer
10c3
2000
-1.5
1000
5e3
-25-
-3.0
0e3
6e7
Log10 Feature firing Frequency
Training Tokens
Training Takens
(d) du 4, 096 *
(c) du 4, 096
(f) de = 4, 096
Figure 10: Comparison of sparsity characteristics for TransCoder modules with dhe 4, 096
and du = 4, 8. Figures and [Odd] the end of training for TransCoder module at layer 19. Figures TO(b) and I0(e) report the Lo
the logro firing frequency distribution of features
sparsity, i.e., the number of active features per input token for layers 10, 19, and 23. Figures [0(c)
and IO(f) track the mean logio activation sparsity throughout training for the same layers.
approximation of the original MLP's outputs against maintaining high degree of sparsity on its internal feature space TTC
15
L.I EXPERIMENTAL CONFIGURATION
Datasets. The TransCoder is trained on intermediate activations produced by the target LLM during inference, without requiring access to the original pretraining data. As long as the input text can trigger target MIPPs computations, these activations can be collected for training. Such input data can be any publicly available text, eliminating the need to access the pretraining dataset. For Codel.lama, the TransCoder modules were trained on 80 million tokens sampled from the "codeparrot/ gi thut dataset. For Llama2, the TransCoder modules were trained on 10 million tokens sampled from the "skylionn67//opemeeeeex
dataset.
Hyperparameters. TransCoders were trained with learning rates ranging from 5 10-4 to 1 10 and the L1 regularization coefficient was varied between * 10 and * 10-6, This range consistently produced robust results across diverse experimental settings. Regularization values above this range tended to induce excessive sparsity, leading to a substantial number of inactive features that failed to capture meaningful information. Conversely, sparsity constraints below this range resulted in many features activating frequently and indiscriminately across contexts.
All TransCoders were initially trained replace the MLP layer 19 of the target model architecture. Once a viable hyperparameter configuration was identified for a given expansion factor, the same configuration was reused train TransCoders for additional layers order evaluate whether our approach generalizes across the model. Here, hyperparameters selected for layer 19 were reused for layers 10 and 23. After training, we assess the quality TransCoder by examining its sparsity behavior. We present the distribution of feature firing frequencies (Figures) TO(a) [ndd)and]] To(d) for LTC4 and LTC8. A well-formed distribution appears smooth on log scale and avoids heavy tails, i.e., few features should be consistently inactive or always firing, both of which indicate poor utilization. For
To this end, du typically chosen multiple times larger than datr http://///ggggii.e.ee..o///a///s
"https:///huge hhttps://uugiiggface htt.//ttsssssscceeeeooo///aattaa https:://ugging:: http://wwww.....aaaaaaaeaa////aa http://////gggnnoooooooooo/////n
36

---

Published as a conference paper at ICLR 2026
layer 10, feature sparsity increased toward the end of training, with more inactive features observed in the logio firing frequency. This effect aligns with prior findings: early-layer transcoders often show lower quality (Dunefsky et al. 2024), and editing in early layers tends to be less effective (Meng et al. 2022).
L.2 TRAINING TIME
Existing knowledge editing methods, such as ROME, MEMIT, PMET, WISE, and AGRACE, require significant computational resources and hours-to-days to compute covariance matrices or build external memory modules. Notably, this cost is one-time and reusable for multiple edits.
Similarly, in our study, we trained TransCoders on internal cluster with NVIDIA H100 PCIe GPUs (80GB VRAM). For 4, 096 4 took about 2 hours and 20 minutes single GPU. Notably, TransCoder supports one- time training that can be reused across all locate and-edii methods for interpretability analyses. provides intuitive visualizations that reveal the correspondence between edited neurons and injected knowledge, enabling developers to transparently track the knowledge injection process.
-an-staa----nstrettt, training with an intermediate dimension of
L.3 TRANSCODER ADAPTER: FAST INTEGRATION OF THE TRANSCODER MODULE
We design a Transcoder Adapter that wraps TransCoder as a standard PyTorch submodule. This adapter serves as a plug-and- and-play replacement for MLP layers: TransCoder parameters are loaded from its pretraining, while all other model weights remain unchanged from the original pretrained checkpoint. Here, the loading process takes only few seconds. This approach is similar in spirit to the integration of LoRA modules. As illustrated Figure we provide an example of loading and integrating the TransCoder module.
ran multiple experiments on the G4GD dataset using the same device to compare the total runtime of ROME across different architectures (MLP, LTC4, LTC8). The results show that ROME's runtime m 9 ~ 17 m 45 s on MLP, m s 18 m 6 on LTC4, and 17 m2 ~ 17 m 52s on LTCS. The slight variation in runtime arises from differences in model architecture and size. Empirical evaluation on 600 samples demonstrates that integrating the TransCoder module introduces additional inference latency, resulting in total increase of several seconds.
def load model and tokoni zoz (args):
model = AutoMadel Fortausal MartaussalM.... IM. from protral inneraiindd had (Monel,
load Transcoder:
adapter vranscoderAdapter:
(Transcoder path)
model. base model. Layera [layer num]. .ipp adapter
tok AutoTaker izer. wwwnoooretotttteeeeeeeeeee pretraired (MORFT, path)
model, Lok
Figure 11: Efficient Layer Replacement Transformers via TransCoder Adapters.
37


Reviews:
[{"review_id": "pLnB6Z7XsH", "reviewer": "Reviewer_76ke", "summary": "This paper proposes a TransCoder-based Precise Editing (TCPE) method, to edit code knowledge in large language models. It also presents a new benchmark, KECode, for code-to-code translation based on functional equivalence. Experimental results\ndemonstrate that TCPE outperforms existing KE methods, achieving a substantial improvement of translation accuracy of CodeLlama-7b-Instruct from 57.5% to 64.0% in a low-resource scenario of Java-to-D translation.", "questions": "Would you like to enlarge the fonts in Figure 1, Figure 2, Table 3 and Table 4?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. This paper proposes a new knowledge editing method for code LLMs. There is also a new benchmark for evaluating the performance of LLMs on code-to-code translation.\n2. Experimental results show that, TCPE outperforms existing knowledge editing methods with significant margins.\n3. A neuron-level interpretability mechanism is introduced to effectively indicates the connection between the edited neurons and the inserted knowledge.", "weaknesses": "1. Neither codebase nor dataset is provided to confirm the reproducibility. \n2. This paper lacks discussions of limitations and broader impact.\n3. The presentation should be improved. For example, fonts in tables and figures can be larger for better reading experience.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h2J8f7shsl", "reviewer": "Reviewer_VVHx", "summary": "This paper addresses the challenge of precise and interpretable knowledge editing (KE) in Large Language Models (LLMs) for code-related tasks. The authors argue that existing KE methods, which often target standard MLP layers, are hindered by neuronal polysemanticity, leading to imprecise edits and poor interpretability. To solve this, the authors propose TransCoder-based Precise Editing (TCPE). This method involves two key stages:\n1. Architectural Modification: The standard MLP layer in a target Transformer (CodeLlama-7b-Instruct) is replaced by a \"TransCoder\" module—a sparse, wide, MLP-like component from prior work (Dunefsky et al., 2024) that is trained to have more monosemantic neurons.\n2. Editing Mechanism: A ROME-like update is applied, but it is restricted only to the small set of \"active neurons\" in the TransCoder module that are relevant to the knowledge being corrected.  \n\nFor evaluation, the authors introduce KECode, a new benchmark for code-to-code translation (Java-to-D) that uses functional equivalence (i.e., unit test pass/fail) as the success metric. Their experiments show that TCPE on the modified architecture (e.g., \"LTC4\") outperforms baseline KE methods (ROME, MEMIT, etc.)", "questions": "Does the method apply to more applications and datasets?", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Knowledge editing is an important research direction to save computational resources by avoiding retraining.", "weaknesses": "I'm not familiar with this field, so I will give my confidence score to 1. Please lower my score weight for this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PlzFBjFVMV", "reviewer": "Reviewer_sF6r", "summary": "This paper makes two key contributions to improve the precision and interpretability of knowledge editing in code LLMs on code translation domain.\n\nFirst, it proposes TransCoder-based Precise Editing, which is a method that replaces the Transformer’s MLP layer with a sparse TransCoder module, enabling edits that target specific highly activated neurons associated with a particular piece of code knowledge. This design allows localized and interpretable edits, minimizing side effects on unrelated behavior while providing clear neuron-level insight into how knowledge is inserted.\n\nSecond, the authors introduce KECode, a new benchmark for evaluating knowledge editing in code translation. KECode consists of 600 Java-to-D translation examples paired with unit tests for functional correctness verification. Using KECode, the paper shows that TCPE significantly outperforms existing methods such as ROME, MEMIT, and PMET on correcting Java-to-D translation errors.", "questions": "Two questions:\n\n* How were the specific layers {10, 19, 23} selected for replacing MLP layers with TransCoder modules? Were these layers empirically identified or based on prior interpretability insights about CodeLlama?\n* When initializing a TransCoder-modified model from pretrained weights, how are the parameters loaded or transferred to the new architecture? Is there a compatibility or adaptation step between the original MLP weights and the TransCoder modules?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* For originality, TCPE presents a novel mechanism that leverages the sparsity and monosemanticity of TransCoder neurons to perform more precise, localized edits. \n* For clarity, the authors place strong emphasis on interpretability. Because TCPE operates at the neuron level, they can explicitly identify which neurons are edited and link them to corresponding knowledge changes. The empirical finding that “highly active neurons carry more essential information during knowledge injection” is well-supported and insightful, strengthening the paper’s interpretability claims.\n* For quality, the evaluation is methodologically solid. The paper reports multiple complementary metrics, including efficacy, specificity, and reliability, as well as detailed ablation studies and granular analysis tailored to the knowledge editing context.", "weaknesses": "* For scope and generalization, this paper focuses solely on Java-to-D code translation. It is unclear whether the approach generalizes to other software engineering tasks such as code completion, bug fixing, or program repair. The KECode benchmark evaluates functional error correction, which is one specific type of knowledge editing. It would also be interesting to discuss or demonstrate applicability to, for example, inserting new API knowledge or modifying non-functional aspects of code, which surface broader SWE contexts.\n* For practicality and Integration, TCPE requires modifying the model architecture by replacing MLP layers with TransCoder modules. The paper does not clarify whether this change requires retraining the new layers or fine-tuning the entire model. Additional discussion on how TCPE integrates with other architectures (e.g., MoE) would help assess its practical adoption potential.\n* For baseline adaptation, while TCPE outperforms existing NLP-based editing methods, like ROME, MEMIT, etc, these baselines were originally designed for factual knowledge editing in natural language models, not code. Because code has strict syntax and execution semantics, these baselines may be disadvantaged. The paper would benefit from either a stronger code-specific baseline, or a clearer discussion of how baseline implementations were adapted to ensure fairness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BRy8QL6PCH", "reviewer": "Reviewer_FKNG", "summary": "This paper proposes the TCPE model editing method based on TransCoder. This method replaces the original MLP layers with an MLP-like model component that has a wide and sparsely activated hidden feature vector, avoiding the imprecision and uninterpretable edits caused by the neuronal polysemanticity of the original MLP. The paper also introduces KECode, a new code-to-code model editing benchmark. Experiments on KECode and existing model editing benchmarks demonstrate the effectiveness of TCPE.", "questions": "Would the introduction of TransCoders increase inference latency?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Replacing the original MLP with a wider and sparse MLP using TransCoder is novel and interesting.\n* KECode, a code-to-code model editing benchmark, is proposed, providing a valuable resource for the community.\n* The effectiveness of TCPE is demonstrated on the KECode, ZsRE, and CounterFact datasets.\n* An in-depth analysis of TransCoder neurons is conducted, offering valuable insights.", "weaknesses": "* TCPE lacks principled innovation, as both TransCoder and ROME-like editing are existing works.\n* The evaluation of the model editing task seems limited to single-case edits, lacking assessments closer to real-world scenarios such as sequential or batch edits.\n* There is no discussion of TCPE’s scalability; its performance when editing large batches or performing sequential edits remains unknown.\n* TransCoders require additional training, and the training environment appears to be more demanding compared to the baselines. Moreover, it is unclear whether replacing the MLP with TransCoders affects the model’s original capabilities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)