EditLord: Learning Code Transformation Rules for Code Editing
Authors: Weichen Li
Albert Jan
Baishakhi Ray
Junfeng Yang
Chengzhi Mao
Kexin Pei
Venue: ICML 2025 poster (2025)
Keywords: Large Language Models
Machine Learning for Code Editing
Code Generation
Reasoning
Primary area: applications->everything_else
TL;DR: The paper introduces EditLord, a novel framework that enhances LLMs for code editing by introducing discrete and explicit procedural steps, leading to improved performance on various code editing tasks and better robustness.

Abstract: Code editing is a foundational task in software development, where its effectiveness depends on whether it introduces desired code property changes without changing the original code's intended functionality. 
Existing approaches often formulate code editing as an implicit end-to-end task, omitting the fact that code-editing procedures inherently consist of discrete and explicit steps, and thus suffer from suboptimal performance and lack of robustness and generalization. 
We introduce EditLord, a code editing framework that makes the code transformation steps explicit. 
Our key insight is to employ a language model (LM) as an inductive learner to extract code editing rules from the training code pairs as concise meta-rule sets.
Such rule sets will be manifested for each training sample to augment them for finetuning or assist in prompting- and iterative-based code editing.
EditLord outperforms the state-of-the-art by an average of 22.7% in editing performance and 58.1% in robustness while achieving 20.2% higher functional correctness, across critical software engineering and security applications, LM models, and editing modes.

EDITLORD: Learning Code Transformation Rules for Code Editing
Weichen Li! Albert Jan?
Baishakhi Ray? 2
Junfeng Yang?
Chengzhi Mao
Kexin Pei
Abstract
Pre wronladooltteeedee..... (aia() Lok
GPT44 End End Edited Code
uadny nepeaqaes ats; rala))
Code editing is a foundational task in software development, where its effectiveness depends on whether it introduces desired code property changes without changing the original code's ' in- tended functionality, Existing approaches often formulate code editing as an implicit end-to-end task, omitting the fact that code editing proce- dures inherently consist of discrete and explicit steps. Thus, they suffer from suboptimal perfor- mance and lack of robustness and generalization. We introduce EDITLORD, a code editing frame- work that makes the code transformation steps explicit. Our key insight is to employ language model (LM) as an inductive learner to extract code editing rules from the training code pairs concise meta-rule sets. Such rule sets will be man- ifested for each training sample to augment them for finetuning or assist in prompting- and iterative- based code editing. EDITLORD outperforms the state-of-the-art by an average of 22.7% in edit- ing performance and 58.1% 1% in robustness while achieving 20.2% higher functional correctness across critical software engineering and security applications. LM models, and editing modes.
2x fadter but wrong
vestarciato Man)y
the Ciet B1 nt 111) ein P
1013
bestot),
k.end().
greatereintsg)
fut cut cwek das eedly
Weare
seaess
kcpp wedie opinitate
Functional Specifications
Edit-Lord Edited Code
Fasrtioed Specihiatinn Rugut Special Durtia
ysing neseagace atds
and ma-
ChOpppt Speciicaionn
vesterciats 1çh);
Edtting Rules
11- voutch beguee)) e. B.ene)).
SIC CREEEEECCE post cssatruct
enteet
Rule
cnt))
Figure 1: A performance editing example showing how ED- ITLORD differs from the existing approach (Shypula et al., 2024). The editing rule library here is discovered during the induction rule learning phase (Section 2.2) and used to pre- pare the finetuning set or directly prompting (Section 2.4). EDITL-ORD'S edits include removing redundant sort and calling efficient I/O functions (cin to scanf), while Shy- pula et al. (2024) provides suboptimal performance im- provement while breaking the original code's 's input-output functionality.
repair and security hardening (Xia & Zhang, 2024; Fu et al., 2022; Peng et al., 2025a; He & Vechev, 2023; Perry et al., 2023; Bhatt et al., 2023).
1. Introduction
Pre-trained code Language Models (code LMs) have shown impressive performance in automating software develop- ment and substantially improved the developers' productiv- ity in a wide range of programming tasks (Shani & Staff, 2023). Among these tasks, code editing has been a fun- damental building block with broad applications, such as optimizing code efficiency (Shypula et al., 2024; Huang al., 2024; Peng et al., 2025b; Garg et al., 2022), reverse engineering and decompilation (Tan al., 2024; Hu al., 2024; Wong et al., 2023; Xie et al., 2024), and vulnerability
Code editing is often considered more challenging than code generation, as it comes with extra requirements for intro- ducing desired new properties, e.g., improved efficiency, in addition to generating functionally correct code. Such new requirements necessitate diagnosing and localizing specific properties in the original code, determining the necessary changes to edit these properties, and ensuring the edits do not introduce unintended side effects. These abstract editing procedures are often compositional because fulfilling cer- tain code property changes can consist of multiple editing steps at multiple program points. However, they are also modular and reusable, as the same abstract editing steps can be reapplied in diverse code editing scenarios.
'The University of Chicago CColumbia University 2 RRtgeer University. Correspondence to: Weichen Li <ww- ichenli@uchieeee @wwhicugo.ccu... Kexin Pei <kpei @uc.iccgocccu.........
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Unfortunately, existing works often treat the knowledge of code editing as latent and implicit, and resort to foute-force

---

EDITLORD: Learning Code Transformation Rules for Code Editing
finetuning to internalize it as part of the model weights (Shy- pula et al., 2024; Tan et al., 2024). Such approaches could fail to decouple the modular code transformation proce- dures from the specific training samples and thus suffer from suboptimal performance (Section 3.2) and the lack of robustness and generalization (Section 3.5).
security (fewer vulnerabilities).
More formally, given a Language Model (LM) M and a training dataset D = {z,, B:)) S X x Y, where X = (x:)i=1 is the set of pre-edit code samples and Y = {u:) is the set of post-edit code samples. Finetuning M for code editing tasks can be formulated as optimizing the conditional probability PM (Y with respect M.
We introduce EDITLORD, a novel framework for learning explicit meta-program transformation rules for code editing tasks. Instead of directly prompting finetuning LM to perform the editing, EDITLORD first employs an LM to discover concise set of inductive transformation rules from training data and then trains an LM to apply these transformations for each code pair. EDITLORD consists of three key subtasks: discovering edit rules to abstract the editing steps, summarizing functional specifications to maintain the code functionality, and learning the discovered rules for code editing. Figure 1 compares EDITLORD to the finetuned baseline (Shypula et al., 2024).
EDITLORD first creates augmented training set D before finetuning the model to perform rule-based code editing, Specifically, there are three steps: (1) data-driven inductive rule discovery, (2) functional specification discovery, and (3) rule-based code editing. Figure 2 illustrates the high-level workflow of EDITLORD. The first two steps bootstrap the training data D, while the third step finetunes the code LM based on the augmented training set D*
In data-driven inductive rule discovery, we query an LM G to iterate the training set and summarize a meta-rule set R. We will elaborate on the process in Section 2.2. In functional specification discovery, we extract the functional description 8; E S that describes the shared input-output behavior of both x; and y;. Such a behavior is expected be the same because the post-edit code y: should preserve the same input-output semantics of z4. With the meta-rule set R and the functional specifications S, we obtain an augmented training set D C x x R x S.
A key advantage of our approach is that it operates in a struc- tured, discrete transformation space, reflecting the symbolic and compositional nature of code editing tasks. Since our discovered meta-program transformation rules are expressed in natural language, they remain interpretable. Moreover, by exposing the editing steps as explicit modules, EDITLORD enables more precise operations and allows human experts to intervene when necessary.
During finetuning, we train the LM M to predict the per-sample editing rules R, functional specifica- tion st, before predicting the post-edit code y/. all conditioned on the pre-edit code zy. Par (us E,, P(s.lz:))PP(((zz s.) (r;, 8;, R.). Besides finetuning, we also consider other editing modes, e.g., prompting (Section 2.4).
We evaluate EDITLORD on the three critical software engi- neering and security code editing tasks, including optimiz- ing code efficiency, improving the readability of decompiled code, and repairing security vulnerabilities. EDITLORD outperforms the state-of-the-art code editing techniques by 23.3%. 12.7%, and 27.6%, respectively, across multiple code LMs. With explicit editing rules, EDITLORDD signif- icantly improved the baseline in generalization, e.g., by up to 24.87% improvement in length generalization and 58.1% improved robustness against prresrris code transformations. In addition to finetuning, EDITLORD applies to different editing modes, with an average of 56.3% improvement in zero-shot prompting and 5.7% improve- ment for iterative refinement based on execution feedback. As EDITLORI exposes the explicit editing steps, we show that its editing performance can be further improved by up to 35.5% when steered by human experts.
2.2. Data-Driven Inductive Rule Discovery
Transformations required to achieve the desired changes in code property can be formalized as editing rules. Intuitively, these rules represent the explicit knowledge of code edits instead of the implicit weight internalized in the model. Our goal is to learn such an explicit meta-rule set R from the training samples, before learning to perform the code editing guided by these explicit rules for better generalization. In the following, we describe how EDITLORD iterates the training set D to grow R and ensures the rules in R remain modular and reusable.
2. Methodology
2. Problem Statement
Inductive meta-rule set initialization Formally, start by extracting raw editing rules by iterating each training code pair (z), For each (æ,, y:), we query an LM G to generate a per-sample raw editing rule set R, that explicitly describes the code property changes required to transform xi into yi- Each raw rule set R2 is thus defined as R, =
The code editing task aims to transform a given pre-edit code into a post-edit code. The post-edit code must be semantically equivalent to the pre-edit code, i.e., preserving its input-output behavior and possessing the desired new code properties, e.g., improved efficiency, readability, or
2

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Pairs from Training Data
Functional Specifications
Meto Rule Set Synthesis
Pre-Edit Code
Edit Code
Gove tio dioensiccal bluuyy ardd,
per perfortenne
ADD
performanct
petiemance
cenaker cxrestad greupe
Raw Edilting Rule Set
Reftued Editingg Rule
bealee vinired, Lat cow.
Recae 10 paint aread the sigla by axpuclted
pee constract
poil constract
encry IDls SinA Inah Fane
tood RCC
entag
dheck dicections
SCLEIi DES
AstRies Hir
Xirnn banh Tane
son-s, cod. 2002, caln);
// thenk diventtion visited.
vialsellteedda] [xeavey)
Cnpueseety couite,
Pocrype
wourte
andistess
MERGE
Raw Editinn Rule Set
Raw Editiing Set
Refined Editing Rule
lorredaHilyy
desble recasetpolat
Teslabiliy Feiee connintassat p-
per
post
EOCAnAOy Spavinal
per cotplrast
Brcalic spersnel
poi comntrac
Excandeu Dat
canslinat
gesl cssstrud
Becatio BUN
BVS mraeered
8P% sneeeal
2ew.00000,000.0 lonal 18,47a
AREGE pars
201 [0(0)
esestaca cosstasa potated:
per consiruct
pasl consiract
PRUNE
Destdlltana mustre Easitpps
Raw Editiig Rule Sel
Relined Editing Rule Set
Heerty
ecarrt
por conntenct
posl conntrnct
jee
gest
Vering eeceynicoeingg
Hicerty
1uattnd Cat
FeE construct
posl consiract
BATE parsen
Augmenting Training Set
Editing Modes
Finetuning Inferencee
Pre-Edit Code
F Functional Specilications
X Pre-Edit Code z.
secing merrppic Logest
desc, Eneespt
Lageas)
hash airibad and the
Y Functional Specificutions s,
Per Sample Rule R.
Post-Edit Code 36
Meta-Rule Sct
ing string represcuting plain mescage procened
Zero-Shbot Prompting
part conwtract
out; string containing the hasbed vection of the tnpur mcosage.
X
Pre-Edit Code x;
Metu- Rull Set R
hacl Fane
LES
aringg
HPC
Y
:
Functional Specifications s.
Per Sample Rule R.
Post- -ddtt Code 3/
Per Sample Rule R.
Herative Refinement with Execution Feedback
Post-Edit Code V.
peu constrect
post construtt
X
::C Pre-Edit Code 2.
Meta-Rule Set R
Foedback
Lapot)
wosk bash fanz
back fase
return (tRp55((
Y: Functional Spocificationss
Sample Rule
Post-Edit Code yi
Figure 2: EDITLORD workflow. The upper section shows how EditLord takes pre-edit and post-edit training code samples as input and outputs the discovered meta-rule set and functional specifications (dotted-box -box) for each sample. The lower section shows how EDITL.ORD augment the training data. We finally augment each input with the synthesized functional specifications and per sample rule set derived from the meta-rule set. The editing process can then be performed either by querying the finetuned LMs (by augmented training samples) or using the meta-rule set as the prompt to guide the zero-shot prompting and the iterative refinement with external feedback. Note that the rule learning steps for three tasks independent. We put them together just to show that the process is generic for different editing tasks.
G(I,, v.). We then aggregate {Ri,.., Ra,,...,,,, Rn} to initialize the raw editing rule set R = Ui= Rg
To systematically perform the refinement, we define three operations that G can use to update the inductive meta-rule set R.
Iterative meta-rule set refinement While the initial R collects the raw editing rules for each sample, they are either too generic or too specific to provide actionable guidance for an effective editing. For example, they can be as generic as "check for potential vulnerability". or as specific as "switch from a+1" (with very specific variable names). Addi- tionally, multiple editing rules may indicate similar trans- formations, e.g., "switch from cin to scanf" and "switch from std:: : : to s".nf". Given these challenges, we introduce an iterative meta-rule set refinement algorithm to ensure the ultimate R is concise and effective. We start with this initial R and iteratively refine each rule r E R until R converges. Figure 2 illustrates the procedures.
ADD: [ - R. This operation instructs G to directly add the rule r E R; derived from the training sample (I,, y1) the meta-rule set Rt This step is still necessary as MERGE and PRUNE (described below) can be very aggressive and some useful rules have been inadvertently removed.
MERGE: RU @ry) Mrr.ry - R. This operation instructs G to merge two rules ri R by replacing them with updated rule r; $ ry.
PRUNE: R [r) + R. This operation instructs G to remove the rule r from the meta-rule set R.
3

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Algorithm I Iterative Rula Set Refinement
specifications 8; to prepare the augmented finetuning set D ((1,, 82, R2, v:)) (as shown in bottom-left in Fig- ure 2).
input Initial Meta-Rule Set R output Finalized Meta-Rule Set R'
1: R' +- R
The finetuning task can now be formulated as model- ing the following conditional probability for each sample: PMr S1, Rg (zx,) = P(s; ( P.) [2i, 8t, Ri). Note that the LM M to be finetuned may differ from G, which is used construct the meta-rule set R and func- tional specifications S (see Section 3).
2: while I'' not converge do
3: for each r E RUR.U R' do
4: if r well-balanced then
5: if Dr E R' similar r then f Add Rules */
6:
7:
R' + R
8:
else
9:
1 Merge Rules */
Other editing modes Beyond editing the code using the finetuned model, the generic design of EDITLORD also sup- ports other editing modes. For example, with the meta-rule set R, we can simply prompt an LM M to directly generate the edited code or further refine the edited code based on external feedback (e.g., execution information (Peng et al., 2025b)). Figure 2 illustrates other code editing modes, i.e., zero-shot prompting and iterative refinement with execution feedback. We study how EDITLORD assists these additional editing modes in Section 3.4.
10:
+ R' L'R'UUR (rerrr) (rrr)
11:
end
12:
else
13:
1+ Prune Rules */
14:
+' - R' \. }
15:
end if
16: end for
17:
end while
18:
return It'
3. Experiments
Algorithm 1 shows the procedures of updating the editing rule set R. Before adding each (ADD and MERGE) to R. we prompt G to assess whether it is a balanced rule, i.e. neither too generic nor too specific (line 4). If not, we apply PRUNE to discard (line 14). Otherwise, we prompt G again to decide whether we should ADD or MERGE r (line 5). Specifically, G will be prompted to decide whether there exists rule in R similar or identical to We apply MERGE (line 10) if yes and ADD (line 7) if not. Figure 2 and Table 5 include some examples of the learned rules in R after this algorithm completes.
We evaluate EDITLORD on three critical software engineer- ing and security applications that can be formulated as code editing tasks (Section 3.1). We consider inference using the finetuned model as our default editing mode and compare to the state-of-the -hr-t-rtt baselines, which also mostly based finetuning (Shypula et al., 2024; Tan et al., 2024; Fu et al., 2022). In Section 3.4, we also show EDITLORD comple- ments other editing modes. We choose models that can be full-parameter finetuned on our local hardware (2x4 Nvidia L40S GPUs), i.e., open-source DeepSeek-Coder 1.3B and 6.7B, or via an online API, i.e., GPT-4o mini. To gener- ate the functional specifications and editing rules, we use GPT-4o mini (G in Section 2).
2.3. Functional Specification Discovery
We consider functional specification as a high-level nat- ural language description of the program's intended be- havior and input-output constraints. Specifically, for each sample (xt,, 3:) in the training dataset, we define the fune-
3.1. Setup: Tasks, Datasets, and Metries
Performance optimization This task aims to edit a given program to improve its execution efficiency (see prompts in Appendix A.1). Following Shypula et al. (2024), we specifically focus on the execution time speedup and use gem5 CPU simulator (Binkert et al., 201 1) to mitigate the noises introduced by the bare metal. We use the HQ (high quality) dataset from Shypula et al. (2024) for training and evaluation. The dataset consists of 4.085 training (slow and fast code) pairs, 2,544 validation samples, and 978 testing samples.
tional specification s; of the training code pair (I;; y/) as s; {descs, in,, outi), where desc; describes the func- tionality implemented in (I,, yı), and in; and out, specifies the input and expected output constraints of (Ii, Vi), respec- tively. We prompt G to generate St.
2.4. Code Editing Modes
Finetuning for rule-based code editing Once we ob- tain the meta-rule set R. we iterate each training sample (x,, y:), and prompt G to identify R, C R as the par-sample editing rules that transform I; to ys- With the per-sample rules R;, we also incorporate the corresponding functional
We adopt the same metrics as Shypula et al. (2024): Correct@ k: the percentage of problems in the testing set for which the LM generates at least one correct solution out of the k candidates. OPT@ @ k: the percentage of problems in
4

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Ghidra Decompiled Code
Original Source Code
Decompile (Tan et al., 2024). Overall, our dataset set con- sists of 8,567 (machine-decompiled code, original source code) training code pairs, 834 validation samples, and 131 testing samples with test cases.
void funco(
int funco(
char *param
const chart paren string, lntt returnSize) ? meaningful
Int "paran 2){ mon-doseritthe
// ... paren ]
(cVarl '1o')
//
"\O"; i+e)
>
chr = string]txr
else@ {
!/ if (chr ')')
We adopt the same metrics in Tan et al. (2024) and extend its readability measurements. Compilability: the percent- age of problems in the testing set where the model gener- ates the compilable program. Correctness: the percentage of problems in the testing set where the model generates the correct program. Readability: similarity between the ground truth and the edited code at three different levels, i.e., character-, token-, and embedding-level. The similarity is defined as 1 - d. where is the edit distance between the ground truth source code and the recovered original code at the character and token level, and cosine distance between CodeSage (Zhang al., 2024) embeddings of them at the embedding-level.
//
level 1: (level 0) {
1/
if (evart == ') {
all levels
iVrr = i -1;
(ints)realloc(e1) levels,
if (iVar5 == 0]
sizeo 5 (count 1));
1var5 1Var5
all Ievelll
ptr realloo (ptr, ptr,
nax level;
structured type.
(lors) 4); x raw pointer (iint ))((oog)) ptr AU)) iVart;
nax level *
(loog)iVars
ivars 0;
ireturnSize count: return llvele)
eVari "poVar2;
poVar2 = povar2 + 1; while (eVari !- 101);
Simllarity Metric)
"param_a 3 iVars;
Char based: 31.27
return ptr;
Token based: 33,43
Embedding hased: 33.94
Figure 3: The similarity metries between the Ghidra de- compiled code and its corresponding source code are con- sistently low. This outcome is primarily due to the non- descriptive variable names and convoluted data structures, which result in both poor readability and consistently low similarity scores.
Security hardening This task aims to edit a given vul- nerable code into the patched version (see prompts in Ap- pendix A.3). In the vulnerable code, there can be one or more vulnerabilities under different Common Weakness Enumeration (CWE) categories.
We obtain the vulnerable and secure code pairs from SVEN (He & Vechev, 2023) for training and validation, and evaluate EDITLORD on a strictly unseen testing set, CWEval (Peng et al., 2025a). CWEval includes vulner- able code samples covering 31 CWEs. Importantly, we choose CWEval because each of its samples includes both functionality and security tests to automate the evaluation, while the other benchmarks, e.g., (Fu et al., 2022) focus only on security fixes and rely on manual effort to check the functionality.
the testing set that the fastest and correct code among the k LM generated programs is at least 10% faster. Speedup k the average absolute ratio between the execution time re- quired by the given slow code and the fastest and correct code among the k LM-generated programs.
Decompilation This task aims to edit a highly unreadable decompiled code into more readable form (see prompts in Appendix A.2). We obtain these decompiled code samples using the off- thel decompiler Ghidra (Agency, 2019). Figure 3 shows an example of the unreadable decompiled code and its original source code.
Following CWEval (Peng et al., 2025a), we define n as the total number of sampled solutions and a varying k S n. We then evaluate the security repair performance with the following three metrics. Correct@ k: the expectation of any of k LM generated solutions is correct. Security@k, k: the expectation of any of k LM generated solutions is secure. Correct & Security@ k: the expectation of any of l LM generated solutions is both correct and secure
To construct training and validation samples, we follow Tan al. (2024) randomly sample original code snippets from AnghaBench (Da Silva et al., 2021) and construct the cor- responding Ghidra decompiled code samples by compiling the original code and decompiling it. We keep the testing samples strictly non-overlapping with the training by using MinranEval-Decomppi (Tan et al., 2024).
These metrics are calculated through the same formula
where is the number of correct programs, secure programs, and both correct and secure programs, respectively.
Eproblems
Ghidra decompiled code in HumanEval- Decompppeee may contain both syntactic and semantic errors, i.e., the pre-edit decompiled code may not compile or fail the test cases. These samples make it impossible for any code editor to reconstruct the semantically correct code without an ora- cle, which is often not available in typical code editing scenarios (Section 4). Therefore, we only consider the functionally correct decompiled code from the HumanEval-
3.2. Main Results
We compare EDITLORD to zero-shot prompting, chain-of- thought prompting (CoT), and finetuning (state-of-the-an baselines) on three tasks: performance optimization, de-
5

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Table I: Main results for performance optimization, decompilation, and security hardening. "Finetuned" refers to the approach from Shypula et al. (2024), Tan et al. (2024), Fu et al. (2022), respectively.
O 1 Speedup k Conect@k
Readability: char
Performance opt.
Decompilation Compile, Correct,
=1 =8 h =1
à =8
k =1
k
=8
token emb
DeepSeek-Codee 1.3B
DeepSeek-Codee 1.3B
Prompt 4.0
16.2
1.0
1.2
11.3
18.3
Prompt 67.9
65.7
31.4
35.7
31.3
CoT 5.6
15.9
1.0
1.2
10.0
19.3
CoT 10.7
6.9
31.7
35.9
31.2
Finetuned 12.5
47.5
1.5
3,0
18.5
67.5
Finetuned 77.1
38.9
36.6
40.8
37.5
EDITLORD (Ours)
16.4
53.1
1.7
3.8
28.9
72.0
EDITLORD (Ours) 93.1
46.6
44.0
47.6
41.4
DeepSeck-Coke Coderr 6.7B
DeepSeck Coder 6.7B
Prompt 6.0
19.5
1.1
1.2
60.0
89.5
Prompt 83.2
64.9
35.3
39.3
34.3
CoT 4.1
6.8
1.1
1.2
16.0
36.6
CoTT 4.4
4.4
30.8
35.4
32.7
Finetuned 7.8
76.9
2.1
3.1
20.7
87.4
Finctuned 89.6
56.5
42.1
47.8
39.7
EDITLORD (Ours)
11.0
82.5
2.7
4.5
36.0
89.5
EDITLORD (Ours)
90.1
58.8
46.2
49.9
43.3
GPT-4o mini
GPT-4o mini
Prompt 12.2
26.4
1.1
1,3
47.4
71.5
Prompt 61.8
44.3
33.1
37.0
37.9
CoT 10.0
29.1
1.5
45.2
CoT 59.5
34.3
38.3
39.8
Finetuned 28.1
61.5
2.3
3.8
59.0
89.5
Finetuned 86.3
52.7
45.6
49.4
43.6
EDITLORD (Ours)
31.2
72.5
2.9
4.2
62.5
93.5
EDITLORD (Ours)
97.7
67.2
51.4
54.9
48.4
Correct t
Security
Correct Sec@k
Security hardening
10
= 50
k
=
h: 50
10
DeepSeek-Code 1.3B
Prompt 6.7
26.8
40.4
3,0
15.2
30.8
1.0
7.1
15.4
CoT 21.2.44.5
39.7
48. 47.7
6.44
20.2
34.643.8
1.840.8
9,442.3
17.3
Finetuned 24.14
38.7
40.4
9.5
21.5
28.9
5,344.3
13,8
19.243.8
EDITLORD (Ours)
31.2+21.5
49.6
19.6
12.2
28.4
36.5
7.448.4
18.7+11.6
23.1
DeepSeek-Cond Coder 6.7B
Prompt 20.3
31.7
44.2
14.8
29.5
36.5
8.7
17.3
23.1
CoT
13.0
40.3
51.947.7
7.6
32.7+a.2
42.3
8.84u.u
17.8
21.1-1.9
Finetuned 22.4+2.1
45.5
53.9
13.7-1.1
35.0+5.4
44.2+7.7
7.7-1.0
20.643.3
25.0
EDITLOrD (Ours)
25.3
41.3
49.144.9
17.943.1
37.1
50.0
11.8
23.9
30.8
GPT-4o mini
Prompt 21.9
36.2
46.1
18.0
30.2
40.4
13.0
23.1
26.9
CoT
27.4+5.5
43.247.0
53.947.7
23.345.2
35.3
42.3+1.9
18.4-..44
28.1
34.6+7.7
Finetuned 30.1
42.4+6.2
50.043.9
16.5-1.8
28.5-1.7
34.6
13.0+0.0
22.1-1.0
30.8
EDITLORD (Ours) 31.4
49.9 13.8
61.5 15..4
27.6.9..6
4
50.0
21.2..8.2
29.3
36.5
compilation, and security hardening. As illustrated in Ta- ble 1, EDITLORD outperforms the state-of of-te-of-the-at baselines across all tasks and models, with 23.3%, 12.7%, and 27.6% improvements on average performance optimization, de- compilation, and security hardening, respectively.
ability metrics demonstrate that the prompting approach even decreases the original Ghidra decompiled code's s read- ability, i.e., 36, 31.7, and 31.2 for the character-, token- and embedding- readability metrics, respectively (Sec- tion 3.1).
While EDITLORD underperforms the prompting based ap- proach based on DeepSeek-Cond -ooder 1.3B and 6.7B in the decompilation task and DeepSeek-Cood formance task on the functional correctness, it is important to note that this metric alone can often appear overstated by simply keeping pre-edit code unmodified. The read-
3.3. Ablations
6.7B in the per-
We conduct ablation studies evaluate the effectiveness of each design component in EDITLORD. Specifically, we introduce the following variants by incrementally adding the key components (in both training and inference): (1)
6

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Table 2: Ablations on performance optimization, decompilation, and security hardening (based on DeepSeek-Coder 1.3B).
O 1 Speedup@ k
Correct@/: f
Readabilityy
Performance opt.
Decompilation Compile, Correct,
=1 k
8
k
=1
k
8
k
1 k =8
char
token
emb
Finetuned 12.5
47.5
1.5
3,0
18.5
67.5
Finetuned 77.1
38.9
36.6
40.8
37.5
func-spec 12.5
49.2
1.7
3.5
21.4
73.4
+ func-spec 93.9
38.9
38.9
43.6
38.4
+ edit-rule 15.7
51.9
1.8
3.7
26.2
70.0
edit-rule
87.0
29.8
41.5
45.2
39.5
EDITLORD (Ours)
16.4
53.1
1.8
3.8
28,9
72.0
EDITLORD (Ours)
93.1
46.6
44.0
47.6
41.4
Conrect@ 1
Security@k f
Correct and Seo@ l: 1
Security hardening
= 1
k= 10
= 50
= 1
k: = 10
k 50
= 1
h = 10
50
Finetuned
24.1
35.7
40.4
9.5
21.5
28.9
5.3
13.8
19.2
+func-spec
25.5+1.3
41.8+6.1
46.1+5.8
11.5+1.9
27.3+5.6
36.547.7
6.0+0.7
15.8+2.0
+1.9
+edit-rule
25.7+1.8
39.0+a.a
44.2+3.8
11.8
27.3+0.a
38,5+9.6
6.741.3
17.5
23.1 +3.1
EDITLORD (Ours)
31.2+7.11
49.6+13.9
59.6+19.2
12.2
28.4
36.5+7.7
7.4
18.7 +4..
23.1 +3.8
Table Comparing EDITLORD Tan et al. (2024) against semantics- perrtiiiigpeeeviingggg code transformations for robustness and unseen samples with longer lengths for generalization by measuring performance degradation with the original decompilation results (Table 1).
w/o EditLord
w/ EditLord
3.5
2.5
10.0
Compile, Correct, Readability+
06
0:0
R
8:3
R-5
R-1
1
8-3
R-5
8-0
R-1
R-3
R-5
Performance opt. (b) Decompilation (c) Security hardening
char
token
emb
Robustness
Figure 4: EDITLORD with different editing modes under zero-shot prompting (R = 0) and iterative refinement (Rt
Finetuned 7.0
8.6
4.2
3.1
5.5
1, 3, 5) with execution feedback. Here, R is the number of iterations taken to refine the edited code.
EDITLORD (Ours) 1.2
4.5
1.8
1.7
4.3
Generalization
Finetuned 4.3 EDITLORD (Ours)
11.4
7.6
8.9
5.8
3.4
9.5
5.8
8.7
4.3
Compilability for decompilation, and (ecurity@555 @5 for se- curity hardening, EDITLORD obtains the best performance when considering both the functional correctness and editing performance.
Finetuned: neither functional specification nor editing rule learning task is included; (2) func-spec: only the functional specification learning task is included; (3) edit-rule: only the editing rule learning task is included.
3.4. Other Editing Modes
In addition to finetuning, EDITLORD is complementary to other code editing modes. We evaluate the effectiveness of EDITLORD when applied zero-shot prompting and iterative refinement with execution feedback (Peng et al., 2025b; Huang et al., 2024).
As shown in Table 2, adding the functional specification learning task alone outperforms the end-to-end finetuning baselines on the functional correctness metrics across all three tasks by up to 21.8%, while also improving the editing performance, e.g., efficiency, readability, and security, by up to 26.7%. Similarly, training models with the editing rule learning task alone can outperform the end-to-end finetun- ing baselines on the editing performance metrics across all three tasks by up to 33.2%, while improving the functional correctness by up to 17.1%. When combined, EDITLORD can outperform the end-to-end finetuning baselines by up to 34.8% in overall editing performance.
As shown in Figure 4, EDITLORD improves the perfor- mance of zero-shot prompting and iterative refinement with execution feedback by an average of 56.3% and 5.7%, re- spectively. We observe EDITLORD underperforms slightly in the decompilation and security hardening task when R = 3 or R = 5. We suspect this is due to the coarse- grained execution feedback, e.g., compilation error, execu- tion error, etc., while the state-of-the-ar baselines incor- porate more fine-grained information, e.g., per-statement execution profile (Huang et al., 2024) (see Section for the
While LDITLORD underperforms these ablation variants on some specific metrics, i.e., @orrect@8 for performance,

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Table 4: Editing rules augmented by human experts using DeepSeek-Codee 1.3B.
O t
Speedup @ t
Correct@1: f
Readabiliiyy
Performance opt.
Decompilation Correct, Compile-
k =
k =
8
k ==
k
=
8
k =
1
A: 8
char
token
emb
EDITLORD
16.4
53.1
1.7
3.8
28.9
72.0
EDITLORD 46.6
93.1
44.0
47.6
41.4
+ human 17.9
61.4
1.9
3,9
53.0
76.9
+ human 49.9
89.3
47.0
51.3
56.1
Correct@ l
Security @
Correct & Sec@
Security hardening
=
= 10
= 50
=
= 10
= 50
=
k = 10
k: = 50
EDITLORD
31.2
49.6
59.6
12.2
28.4
36.5
7.4
18.7
23.1
human 33.2
53.9 4.433
65.4
13.5
1.3
33.8
45.4
42.3
8.9
6.941.1
1.6
21.9
43.2
28.9
discussion).
are the authors of this paper) to refine EDITLORDD'S gen- erated editing rules and functional specifications for each testing sample, e.g., removing unreasonable rules or append- ing effective new rules.
3.5. Robustness and Generalization
Robustness We compare EDITLORD to the state-of-the- art baseline against semantics- preseervng code transforma- tions (Wang et al., 2023a; Yefet et al., 2020; Gao et al., 2023; Yang et al., 2022; Bielik Vechev, 2020). The transforma- tions include renaming variables and functions to long ran- dom strings and removing comments, whitespaces, unused variables, and unused headers. We observe individual trans- formations alone rarely produce significant output changes. We thus exhaust each transformation (e.g., renaming all variables) and compose all of these transformations together introduce input changes substantial enough.
Table 4 shows that such an augmentation enhances EDIT- LORD with up to 15.6%, 35.5%, and 25. 55.1% % improvement in efficiency, readability in decompilation, and security.
4. Discussion and Limitation
Iterative refinement with execution feedback Exist- ing LM-based code editing approaches often leverage it- erative refinement with execution feedback (Huang et al., 2024; Peng et al., 2025b; Xia & Zhang, 2024; Waghjale et al., 2024), which relies on the availability of test inputs. However, the code to be edited may not always be well- maintained. Therefore, this paper, we do not assume the tests are available by default (Section 3). We also show that EDITLORD is complementary iterative refinement with coarse-grained execution feedback (Section 2.4). We aim to study whether the fine-grained execution feedback, i.e., per-statement execution profile (Huang et al., 2024), can further improve EDITLORD in the future.
Table 3 (left) shows the average performance degradation in functional correctness, comparability, and readability when the input undergoes semantics perericc--preeiiingggg transformations. EDITLORD achieves up to 58.1% less drops in readabil- ity measured by character-level readability compared to the baseline, indicating that producing explicit functional specifications and editing rules helps EDITLORDD stay less susceptible to syntactic code changes.
Functional correctness guarantee In code editing, func- tional correctness is arguably hard constraint that cannot be violated, while the editing goals can be soft objectives. While we ensured that our editing performance is mea- sured strictly on the subset of edited code that must first functionally correct, EDITLORD cannot guarantee all the edited code correct. While this issue can be trivially mitigated by falling back to the original code, it completely fails to introduce any edits. We thus follow the existing approaches (Shypula et al., 2024; Tan et al., 2024) by treat- ing functional correctness as soft constraint. Imposing formal correctness guarantee for the edited code would be extremely valuable for future work.
Length generalization We investigate EDITLORD's gen- cralizability to longer sequences than those seen in train- ing (Anil et al., 2022). Specifically, we select 71 testing samples with lengths over 500 and finetune EDITLORDD only on 6,505 strictly shorter (< 500) training samples.
Table 3 (right) demonstrates that EDITLORD suffers reduced performance degradation on unseen longer code, achiev- ing to 25.86% less drops in readability measured by embedding-level readability (Section 3.1).
3.6. Rule Augmentation by Human Experts
As EDITLORD produces explicit editing steps, it facilitates human intervention to introduce customized editing rules. To assess the effectiveness of EDITLORD augmented by hu- man intervention, we incorporate two human experts (both
8

---

EDITLORD: Learning Code Transformation Rules for Code Editing
5. Related Work
models, and editing modes.
Code LMs have been extensively used to assist developers in editing code (Guo et al., 2025; Li al., 2023: LaBash et al.,
Acknowledgement
2024; Li et al., 2024: Cassano et al., 2024; Chakraborty et al.,
We thank Xiaofei Ma, Jinjun Peng and all the anonymous reviewers for their constructive comments and feedback, which significantly improved this paper. We are also grateful to Jun Yang for his assistance with parts of the experimental setup. This work was supported in part by Columbia Center for AI Technology (in partnership with Amazon), NSF CCF 2313055, CCF 2107405, Chameleon (Keahey et al., 2020) and OpenAl Researcher Access Program (OpenAl).
2020; Liu et al., 2024a; Gupta et al., 2023; Muennighoff et al., 2023; Singhal et al., 2024) to fulfill various goals, such as bug fixing (Xia & Zhang, 2024; Fan et al.. 2023; Liu et al., 2024c), decompilation (Tan et al., 2024; Wong et al., 2023; Xie et al., 2024; Hu et al., 2024), efficiency optimiza- tion (Huang et al., 2024; Shypula et al., 2024; Garg et al., 2022: Peng al., 2025b), vulnerability repair (He & Vechev,
2023; Peng al., 2025a; Fu et al., 2022; Xia & Zhang, 2024; Perry al., 2023; Bhatt et al., 2023), code translation (Pan et al., 2024; Eniser et al., 2024), code refactoring (Shirafuji et al., 2023; Cummins et al., 2024), and more broadly related tasks like generating proofs (Chen et al., 2025; Chakraborty et al., 2025), invariants (Kamath et al., 2023; Chakraborty et al., 2023), or specifications (Murphy et al., 2024; Ma al., 2025).
Impact Statement
Code editing plays a crucial role in assisting developers' daily jobs. While Large language models (LLMs) have demonstrated promising capabilities in automated code transformation, their end-to-end nature often leads to hal- lucinated edits. These hallucinations would be especially harmful when LLMs are used to edit security and safety- critical code. Our paper introduced new approach to im- proving the safety of LLMs when applied to security software engineering applications.
Most existing approaches either adopt finetuning to learn the direct mapping between the code pairs or leverage it- erative refinement with execution or self-generated feed- backs (Huang et al., 2024; Peng et al., 2025b; Huang et al., 2023a; Xia & Zhang, 2024; Chen et al., 2024; Dong et al., 2024; Madaan et al., 2024; Zelikman et al., 2024; Liu al., 2024b). In contrast, EDITLORD complements these ap- proaches by making the intermediate editing steps explicit.
References
Agency, N. Ghidra Disassembler. https://ghidi:: 22e.
EDITLORD shares a similar philosophy to bootstrapping the symbolic reasoning of LMs (Zelikman et al., 2022; Kim et al., 2023; Huang et al., 2023b; Chen et al., 2024; Hsich
Anil, C., Wu, Y., Andreassen, A., Lewkowycz, A., Misra, V., Ramasesh, V., Slone, A., Gur-Ari, G., Dyer, E., and Neyshabur, B. Exploring length generalization in large language models. Advances in Neural Information Pro- cessing Systems, 35:38546-38556, 2022.
et al., 2023; Wang et al., 2023b; Zhang et al., 2023; Light- man et al., 2023; Zhou al., 2024), where the LMs synthe- size the reasoning procedures to self-augment the samples for supervised finetuning or preference tuning. EDITLORD extends the idea by distilling concise and composable editing meta-rule set, such that the code to be edited can share reusable editing steps and benefits from the improved generalization (Section 3.5).
Bhatt, M., Chenmabasappa, S., Nikolaidis, C., Wan, S., Evti- mov, I., Gabi, D., Song, D., Ahmad, F., Aschermann, C., Fontana, L., et al. Purple Ilama cyberseceval: A secure coding benchmark for language models. arXiv preprint arXiv:2312.04774,, 2023.
6. Conclusion
Bielik, P. and Vechev, M. Adversarial robustness for code. International Conference on Machine Learning, pp. 896-907. PMLR, 2020.
We introduced EDITLORD, generic code editing frame- work, by learning the inductive code transformation rules to elicit the explicit code editing steps. Our key approach is to employ a language model (LM) as an inductive learner to distill a concise and composable meta-rule set for code edit- ing from the training code pairs. EDITLLORD substantially outperforms the sate-offthheart code editing techniques in editing performance while enjoying significantly higher functional correctness and improved robustness against semantics-presedt code transformations across multiple critical software engineering and security applications, LM
Binkert, N., Beckmann, B., Black, G., Reinhardt, S. K.,
Saidi, A., Basu, A., Hestness, J., Hower, D. R., Kr- ishna, T., Sardashti, S., et al. The gem5 simulator. ACM SIGARCH computer architecture news, 39(2):1-7, 2011.
Cassano, F., Li, L.., Sethi, A., Shinn, N., Brennan-Jones, A.,
Ginesin, J., Berman, E., Chakhnashvili, G., Lozhkov, A., Anderson, C. J., et al. Can edit? evaluating the ability of large language models to follow code editing instructions. In Conference on Language Modeling (COLM), 2024.

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Chakraborty, S., Ding, Y., Allamanis, M., and Ray, B. Codit: Code editing with tree-based neural models. IEEF Transactions on Software Engineering, 48(4):1385-1399 2020.
Fu, M., Tantithamthavori, C., Le, T., Nguyen, V., and Phung, D. Vulrepair: a t5-based automated software vulnerability repair. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 935-947, 2022.
Chakraborty, S., Lahiri, S. K., Fakhoury, S., Musuvathi, M., Lal, A., Rastogi, A., Senthilnathan, A., Sharma, R., and Swamy, N. Ranking llm-generated loop invariants for program verification. arXiv preprint arXiv:2310.09342, 2023.
Gao, F., Wang. Y., and Wang, K. Discrete adversarial at- tack to models of code. Proceedings of the ACM on Programming Languages, 7(PLDI): 2023.
Chakraborty, S., Ebner, G., Bhat, S., Fakhoury, S., Fatima, S., Lahiri, S., and Swamy, N. Towards neural synthesis for smt-assisted proof-oriented programming. In Proceed- ings of the 47th International Conference on Software Engineering, 2025.
Garg, S., Moghaddam, R. Z., Clement, C. B., Sundaresan, N., and Wu, C. Deepdev-perf: deep learning-basee ap- proach for improving software performance. In Proceed- ings of the 30th ACM Joint European Software Engineer- ing Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), pp. 948-9588 2022.
Chen, T., Lu, S., Lu, S., Gong, Y., Yang, C., Li, X., Misu, M. R. H., Yu, H., Duan, N., Cheng, P., et al. Automated proof generation for rust code via self-evolution. In The Thirteenth International Conference on Learning Repre- sentations, 2025.
Guo, J., Li, Liu, X., Ma, K., Zheng. T., Yu, Z., Pan, D., Li, Y., Liu, R., Wang. Y., et al. Codeeditorbench: Evaluating code editing capability of large language models. ICLR 2025 Workshop on Deep Learning for Code, 2025.
Gupta, P., Khare, A., Bajpai, Y., Chakraborty, S., Gulwani, S., Kanade, A., Radhakrishna, A., Soares, G., and Ti- wari, A. Grace: Language models meet code edits. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Founda- tions of Software Engineering, pp. 1483-1495, 2023.
Chen, X., Lin, M., Schärli, N., and Zhou, D. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, 2024.
Cummins, C., Seeker, V., Armengol- J., Markosyan, A. H., Synnaeve, G., and Leather, H. Don't it transform the code, code the transforms: Towards precise code rewriting using llms. arXiv preprint arXiv:2410.0880 2024.
He, J. and Vechev, M. Large language models for code: Se- curity hardening and adversarial testing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 1865-1879, 2023.
Da Silva, A. F., Kind, B. C., de Souza Magalhães, J. W., Rocha, J. N., Guimaraes, B. C. F., and Pereira, F. M. Q. Anghabench: A suite with one million compilable с benchmarks for code-size reduction. In 2021 IEEE/ACM International Symposium on Code Generation and Opti- mization (CGO), pp. 378-390. IEEE, 2021.
Hsich, C.-Y., Li, C.-L,, Yeh, C.-k., Nakhost, H., Fujii, Y.. Ramner, A., Krishna, R., Lee, C.-Y., and Pfister, T. Distill- ing step-by-step! outperforming larger language models with less training data and smaller model sizes. In Find- ings of the Association for Computational Linguisties: ACL 2023. pp. 8003-8017, 2023.
Dong, Y., Jiang, X., Jin, Z., and Li, G. Self-collaboration code generation via chatgpt. ACM Transactions on Soft- ware Engineering and Methodology, 13(7);1-38, 2024.
Hu, P., Liang, R., and Chen, K. Degpt: Optimizing decompiler output with llm. In Proceedings 2024 Network and Distributed System Security Symposium (2024). hups:/lapi. semanticscholar org/CorpusID, vol- ume 267622140, 2024.
Eniser, H. F., Zhang, H., David, C., Wang, M., Christakis, M., Paulsen, B., Dodds, J., and Kroening, D. Towards translating real-world code with llms: A study of translat- ing to rust. arXiv preprint arXiv:2405. 11514, 2024.
Huang, D., Bu, Q. Zhang, J. M., Luck, M.. and Cui, H. Agentcoder: Multi-agent-baan -ased code generation with iterative testing and optimisation. arXiv preprint a3Xiv: 13010, 2023a.
Fan, Z.,, Gao, X., Mirchev, M., Roychoudhury, A., and Tan, S. H. Automated repair of programs from large language models. In 2023 JEFF/ACM 45th International Conference on Software Engineering (ICSE), pp. 1469- 1481. IEEF, 2023.
Huang, D., Zeng, G., Dai, J., Luo, M., Weng, H., Qing. Y., Cui, H., Guo, Z., and Zhang, J. M. Effi-code: Unleashing code efficiency language models. 2024.
10

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., and Han, J. Large language models can self-improve. In Pro- ceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 1051-1068, 2023b.
Liu, Y., Gao, P., Wang, X., Liu, J., Shi, Y., Zhang, Z., and Peng. C. Marscode agent: Ai-native automated bug fixing. 2024c.
arXiv preprint
Ma, L.,, Liu, S., Li, Y., Xie, X., and Bu, L.. Specgen: Au- tomated generation of formal program specifications via large language models. In Proceedings of the 47th Inter- national Conference on Software Engineering. 2025.
Kamath, A., Senthilnathan, A., Chakraborty, S., Deligiannis, P., Lahiri, S. K., Lal, A., Rastogi, A., Roy, S., and Sharma, R. Finding inductive loop invariants using large language models. arXiv preprint 1exXi:2311.0094444 2023.
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L.. Wiegreffee, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. Self-refine: Iterative refinement with self- feedback. Advances in Neural Information Processing Systems, 36, 2024.
Keahey, K., Anderson, J., Zhen, Z., Riteau, P., Ruth, P., Stanzione, D., Cevik, M., Colleran, J., Gunawi, H. S., Hammock, C., Mambretti, J., Barnes, A., Halbach, F., Rocha, A., and Stubbs, J. Lessons learned from the chameleon testbed. In Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC '20). USENIX Association, July 2020.
Muennighoff, N., Liu, Q., Zebaze, A., Zheng, Q., Hui, B., Zhuo, T. Y., Singh, S., Tang, X., Von Werra, L., and Longpre, S. Octopack: Instruction tuning code large lan- guage models. In NeurlPS 2023 Workshop on Instruction Taning and Instruction Following, 2023.
Kim, G., Baldi, P., and McAleer, S. Language models can solve computer tasks. Advances in Neural Information Processing Systems, (6:39648-39677 2023.
Murphy, W., Holzer, N., Qiao, F., Cui, L., Rothkopf, R., Koenig, N., and Santolucito, M. Combining Ilm code generation with formal specifications and reactive pro- gram synthesis. arXiv preprint arXiv:2410.19736 2024.
LaBash, B., Rosedale, A., Reents, A., Negritto, I.., and Wiel, C. Res-q: Evaluating code-editing large language model systems at the repository scale. arXiv preprint 11X1v,2406.16801, 2024.
OpenAL.
Openai researcher access program. https://openai.ee access ppoograpr..gam/.
Li, J., Li, G., Li, Z., Jin, Z., Hu, X., Zhang, K., and Fu, Z. Codeeditor: Learning to edit source code with pre-trained models. ACM Transactions on Software Engineering and Methodology, 32(6):1-22, 2023.
Pan, R., Ibrahimzada, A. R., Krishna, R., Sankar, D., Wassi, L. P., Merler, M., Sobolev, B., Pavuluri, R., Sinha, S., and Jabbarvand, R. Lost in translation: A study of bugs introduced by large language models while translating code. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pp- 1-13, 2024.
Li, K., Hu, Q., Zhao, J., Chen, H., Xie, Y., Liu, T., Shieh, M., and He, J. Instructcoder: Instruction tuning large language models for code editing. In Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 4: Student Research Work- shop), pp. 50-70, 2024.
Peng. J., Cui, L.., Huang, K., Yang. J., and Ray, B. Cweval: autcome-driven evaluation on functionality and security of Ilm code generation. arXiv preprint arXiv:2501 008000, 2025a.
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
Peng, Y., Gotmare, A. D., Xiong, C., Savarese, S., Lyu, M., and Sahoo. D. Perfcodegen: Improving performance of Ilm generated code with execution feedback. The 2nd ACM International Conference on Al Foundation Models and Software Engineering (FORGE 2025) in ICSE 2025, 2025b.
Liu, C., Cai, Y., Lin, Y., Huang. Y., Pei, Y., Jiang, B., Yang, P., Dong, S., and Mei. H. Coedpilot: Recommending code edits with learned prior edit relevance, project-wise awareness, and interactive nature. In Proceedings of the 33rd ACM SIGSOFT International Symposium Software Testing and Analysis, pp. 466-478, 2024a.
Perry, N., Srivastava, M., Kumar, D., and Boneh, D. Do users write more insecure code with ai assistants? In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 2785-2799, 2023.
Liu, J., Nguyen, T., Shang, M., Ding, H., X., Yu, Y., Kumar, V., and Wang, Z. Learning code preference via synthetic evolution. arXiv preprint arX1v:2410.03837,, 2024b.
Shani, I. and Staff, G. http:::/ggtttbbbloog// ins insights/ httpt//eeeeeeearrreeeeeeeeallaaa impact haaa--teeeeeeelee--eeeeeeeeeeeee 2023.

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Shirafuji, A., Oda, Y., Suzuki, J., Morishita, M. and Watanobe, Y. Refactoring programs using large lan- guage models with few-shot examples. In 2023 30th Asia-Pacific Software Engineering Conference (APSEC), pp. 151-160. IEFE, 2023.
Xie, D., Zhang, Z., Jiang, N., Xu, X., Tan, L.,, and Zhang, X. Resym: Harnessing Ilms to recover variable and data structure symbols from stripped binaries. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pp. 4554-4568, 2024.
Shypula, A., Madaan, A., Zeng, Y., Alon, U., Gard- ner, J. R., Yang. Y., Hashemi, M., Neubig, G., Ranganathan, P., Bastani, O., and Yazdanbakhsh, A. Learning performance-improving code edits. In The Twelfth International Conference on Learning Repre- sentations, 2024. URL httpst/s:/ http///pperrvvveeee ew.ne// Forumpid-ix7rLVHXX>
Yang. Z., Shi, J., He, J., and Lo, D. Natural attack for pre-trained models of code. In Proceedings of the 44th International Conference on Software Engineering, PP- 1482-1493, 2022.
Yefet, N., Alon, U., and Yahav, E. Adversarial examples for models of code. Proceedings of the ACM on Program- ming Languages, 4(00PSLA))1-30) 2020.
Singhal, M., Aggarwal, T., Awasthi, A., Natarajan, N., and Kanade, A. Nofuneval: Funny how code Ims falter on requirements beyond functional correctness. In First Conference on Language Modeling, 2024.
Zelikman, E., Wu, Y., Mu, J., and Goodman, N. Star: Boot- strapping reasoning with reasoning. Advances Neural Information Processing Systems, 35:15476-15488 2022.
Zelikman, E., Lorch, E., Mackey, L.., and Kalai, A. T. Self- taught optimizer (stop): Recursively self-improving code generation. In First Conference on Language Modeling. 2024.
Tan, H.. Luo, Q. Li, J., and Zhang, Y. Lim4decompile: Decompiling binary code with large language models. In Proceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pp. 3473-3487, 2024.
Zhang, D., Ahmad, W., Tan, M., Ding, H., Nallapati, R., Roth, D., Ma, X., and Xiang, B. Code representation learning at scale. In The Twelfth International Conference Learning Representations, 2024.
Waghjale, S., Veerendranath, V., Wang, Z. Z., and Fried, D. Ecco: Can we improve generatee code efficiency without sacrificing functional correctness? In Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 15362-15376, 2024.
Zhang. Z., Zhang, A., Li, M., and Smola, A. Automatic chain of thought prompting in large language models. In The Eleventh International Conference on Learning Representations, 2023.
Wang, S., Li, Z., Qian, H., Yang. C., Wang, Z., Shang, M., Kumar, V., Tan, S., Ray, B., Bhatia, P., al. Recode: Robustness evaluation of code generation models. In Pro- ceedings of the 6lst Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 13818 138433, 2023a.
Zhou, P., Pujara, J., Ren, X., Chen, X., Cheng, H.-T., Le, Q. V. Chi, E. H., Zhou, D., Mishra, S., and Zheng, H. Self-discover: Large language models self-compose rea- soning structures. In Advances in Neural Information Processing Systems, 2024.
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D.. and Hajishirzi, H. Self-instruct: Aligning language models with self-generated instructions. In Pro- ceedings of the sl Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pp. 13484-13508, 2023b.
Wong, W. K., Wang, H., Li, Z., Liu, Z., Wang, S., Tang, Q. Nie, S., and Wu, S. Refining decompiled с code with large language models. arXiv preprint 10.05303 2023.
Xia, C. S. and Zhang, L.. Automated program repair via conversation: Fixing 162 out of 337 bugs for $0.42 each using chatgpt. In Proceedings of the 33rd ACM SIG- SOFT International Symposium on Software Testing and Analysis, pp. 319-831, 2024.
12

---

EDITLORD: Learning Code Transformation Rules for Code Editing
A. Prompt Format
A.I. Performance Optimization
Prompt I: Bascline prompt for performance optimization.
User:
This the slow code:
[SLOW CODE]
{src_code}
[/SLOW CODE]
The corresponding fast code is:
[FAST CODE]
Assistant: (tgt_code)
[/FAST CODE]
Prompt 2: Functional specifications only prompt for performance optimization.
User: This the slow code:
[SLOW CODE]
{src_code}
[/SLOW CODE]
The given code describes the following problem:
The input specification is: finput specification)
The output specification is: (output.specification)
The corresponding fast code is:
[FAST CODEJ
Assistant: ( [tgt_code]
[/FAST CODE]
Prompt 3: Editing rules only prompt for performance optimization.
User: This is the slow code:
[SLOW CODE]
code] [/SLOW CODE]
Following editing rules should be applied: (editing.rule'
The corresponding fast code is:
[FAST CODE]
Assistant: /tgt code)
[/FAST CODE)
Prompt 4: Our prompt for performance optimization.
User: This is the slow code:
[SLOW CODEJ
{sre_code}
[/SLOW CODE]
The given code describes the following problem: shunctiooallspeccifaatoooo
The input specification is: {inpur_spccffiattio))
The output specification is: {output specification
Following editing rules should be applied: (editing.rules]
The corresponding fast code is:
[FAST CODE]
13

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Assistant: [tgt_code)
[/FAST CODE]
A.2. Decompilation
Prompt 5: Baseline prompt for decompilation.
User:
This the decompiled code:
[MACHINE DECOMPILED CODE]
{sre.code) code}
[/MACHINE DECOMPILED CODE]
The corresponding source code is:
[ORIGINAL. SOURCE CODE]
(Igd_co
VORIGINAL.SOURE SOURCE CODE)
Prompt 6: Functional specifications only prompt for decompilation optimization.
User:
This the decompiled code:
[MACHINE DECOMPILED CODE] {sre_code}
|/MACHINE DECOMPILED CODEJ
The given code describes the following problem: (functional specification) }
The input specification is: }pecifispecf
The output specification is: coutput.speciffcctin
The corresponding source code is:
[ORIGINAL SOURCE CODE]
Assistant: [tgt_code] code)
[/ORIGINLL SOURCE CODE]
Prompt 7: Editing rules only prompt for performance decompilation.
User:
This the decompiled code:
[MACHINE DECOMPILED CODE] {sre_code}
[/MACIINE DECOMPILED CODEJ
Following editing rules should be applied: dediting-rules,
The corresponding source code is:
[ORIGINAL. SOURCE CODE]
Assistant: (tgtttcodd code)
[/ORIGINAL SOURCE SOURCE CODE]
Prompt 8: Our prompt for decompilation.
User: This is the decompiled code:
[MACHINE DECOMPILED CODE]
{sre_code}
[MACHINE DECOMPILED CODE]
The given code describes the following problem: [functional specification)
The input specification is: (input.specification
The output specification is: (output specification)
14

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Following editing rules should be applied: (editing_rules)
The corresponding source code is: [ORIGINAL. SOURCE CODE]
Assistant: [ogee |/OIGINAL SOURCE SOURCE CODE]
A.3. Security
Prompt 9: Baseline prompt for security.
User:
This the vulnerable code:
[VULNERABLE CODE]
{src_code}
[/ULNERABLE CODE]
The corresponding secure code is:
[SECURE CODE]
Assistant: [tgt_code}
[/SECURE CODE]
Prompt 10: Functional specifications only prompt for security.
User: This the vulnerable code:
[VULNERABLE CODE]
{src_code}
[/ULNERABLE CODE]
The given code describes the following problem: [functional specification)
The input specification is: speciffspciii
The output specification is: foutput..peccficaain
The corresponding secure code is:
[SECURE CODE]
Assistant: (tgt code)
[/SECURE CODE]
Prompt 11: Editing rules only prompt for security.
User: This is the vulnerable code: [VULNERABLE CODE] {sre_code}
(/UULNERABLE CODE]
Following editing rules should be applied: (editing.rules)
The corresponding secure code is:
[SECURE CODE]
Assistant: (tgtccoeet
|/SECURE CODE]
Prompt 12: Our prompt for security.
User: This is the vulnerable code: [VULNERABLE CODE] [sre_code)
LVULNERRBLE CODE]
15

---

EDITLORD: Learning Code Transformation Rules for Code Editing
The given code describes the following problem:
ffunctional_speccficattonn
The input specification is: spnpuusspccfiicaionn
The output specification is: {output specification)
Following editing rules should be applied: {cditing-rules}
The corresponding secure code is: (SECURE CODE]
Assistant: (tgt )ode)
[/SECURE CODE]
A.4. Meta-Rule Set
Prompt 13: Generic/specific evaluation for editing rule
Please analyze the provided editing rule (in order to improve (task name)) and determine whether broadly applicable across different code snippets (generic) tailored specific code snippet (specific), An editing rule like "(generic rule example]" should be considered as generic rule. While rule like "(specific rule example)" should be considered as specific rule.
Provide your response in the following format: The rule is [generic/specific] because
So, what do you think about the rule "(editing.rule" ? Is it generic or specific?
Prompt 14: Add/Merge rules
Please analyze the provided editing rule (in order to improve [task_name)) and compare it with the existing editing rules in the meta-rule set. it's similar to any existing editing rule, please suggest how it should be integrated into the existing meta-rule set. Specify the one and only one appropriate action from the options below: [ADD]: none the existing editing rules in the meta-rule set is similar to the current one, provide the refined and updated editing rule to be added to the set.
[MERGE]: If the current editing rule is similar to an existing editing rule, indicate which existing meta-rule is similar the current editing rule that they be merged and how they should be merged.
If [ADD] is selected, please provide the refined and updated editing rule to be added to the set directly without any other information. If [MERGE] is selected, please provide exactly the existing meta-rule that is similar to the current editing rule with an updated editing rule.
Please notice that whether you select [ADD] or [MERGE], the editing rule you add or merge into the meta-rule set must adhere to the format "switch from ... to ...". Ensure that you only provide editing rules that transition from a fold property) to {new www.prrperty)....
Here several examples of the output:
[Example Output 1]
[ADD] only the editing rule to be added here [/ADD]
|/Example Output 1]
[Example Output 2]
[MERGE] only the editing rule to be merged and the updated rule, split by semicolon [/MERGE]
[/Example Output 2]
Meta-Rule Set:
{meta rule set}
Editing Rule Requested for Analysis:
fediting-rule)
B. Meta-Rule Set Learned by EDITLORD
Table 5 shows some examples of meta-rules for code editing discovered by EDITLORD. We also calculate the percentage of edited samples (as indicated in the third column) that benefit from the specific rule in each row. For example, the rule
16

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Table 5: Meta-rule examples discovered by EDITLORD for performance optimization, decompilation, and security hardening (Section 2.2). Each rule follows the universal format "switch from [old properties] to [new properties]" The third column shows the percentage of the samples improved by the specific rule.
From
%1 Examples
Performance Optimization
cout
printi
32.4 Figure
cin
scanf
24.8
Figure
multiple nested loops for condition checks
single streamlined loop condition checks and assignments
21.0
Figure
recursive function calls
optimized iterative data handling methods static memory allocation
7.6
Figure
dynamic memory allocation
7.6
Figure
Decompilation
complex if-else structure
simplified conditional logic
11.9 Figure
complex pointer arithmetic
clear variable assignments
10.1
Figure 7, 8
redundant checks
straightforward boolean comparisons descriptive variable names
Figure
cryptic variable names
5.9
Figure
ambiguous function signatures
clear function signatures
5.9
Figure
Security Hardening
no checks function return values
check function return value use of parameterized logic
32.9 Figure
direct SQL string interpolation
10.6 Figure 10
unvalidated input handling
check for buffer overflows on memory accesses check maximum buffer allocation size before allocation
8.9
Figure
unvalidated memory allocation
7.6 Figure 12
direct parsing
implement comprehensive validation checks character handling
6.8 Figure
"switch from cout to printf" applies to 32.4% of the testing samples that have obtained performance improvement.
C. Case Study
C.1. Performance Optimization
We show several examples to demonstrate how EDITLORD improve performance optimization.
17

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given slow code
EDITLORD output
#include <bits/stdctt.h>
The given code describe the following
problem: Find the minimun cost to reach the last index of sequence of integers,
using namespace std;
where the cost defined as the absolute - difference between consecutive integers.
long long n; long long k;
vector<long long> vec; long long meme [100007];
The input specitication is: The programs take two integers and k as input, where n is the sequence - and k is the maximum jump length. Then, n
the number of integers
long long solve (long long index)
if (index=en-l)
- integers are read, representing the
return C;
sequence.
The output specification is: Output the - minimum cost to reach the last index from + the first index.
(meme [index] =--1) return mome [index];
Following editing rules should be applied: switch from maltiple nested loops to single linear iterations
long long ret 100000000000000000 long long for (long long if (index+i<n)) ret abs (vec index]- vec index=li] solve (indexti);
100000000000000000
ji<-k ; i++)
switch from recursive function calls to - optimized iterative data handling methods The corresponding fast code is: [FAST CODE
#include<bits/stteee
sol min (sol, ret);
using namespace std;
return [index] sol;
long long int 1p1100007 [100007]
int main() long long int n, cin>>n>>k; (or(i=0iim);1t)) cin>>a(i];
int (
cin>>n;
cin>>k;
memset (meme,-l sizeof (meme)) )
long long num;
for (long long 1 = 0 isn i++)
dp[0]=0;
cin>>num;
for [1r11;;1cn;;1+
vec.push back (num)
x=10000000000007>
cout ccut<csolve (0) <<endl;
xxmin(x,,ee
return 0;
dp [p[1]=x;
cout <out<<dpliao nn11;;
(/FAST CODE]
Figure 5: Performance optimization example 1.
18

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given slow code
EDITLORD output
#include <bits/stdc++.br
given code describe the following problen: Calculate the mininum cost to reach the last index in 1ist of integers, where each step can jump to k positions.
using namespace std;
int ,k vectorsint>vi;
The input specification is: The progzams take - two integers a and k as input, where n is the number of integers in the list and is the maximum jump length. Then,
long long mem 200005); bool vis vis12000051;
long long solve (int indx )
integers are read, representing the cost - associated with each index.
if (indx -- n-1)
return C;
The output specification is: Output the minimum cost to reach the last index from the first index.
if [vis(indx)]
return mem[indx];
vis [indx] true;
Following editing rules should applied: switch from multiple nested loops for counting clements to direct arithmetic calculation
nom [indx] 1c13;
for (int i = indx+1; i <-min (indx*k
n-1) i+t)(
mem [indx] min (mem [indx]
switch from multiple nested loops to - compact function
- solve + abs (vi [indx]
vilil));
switch from cout to printf switch from ein to scanf switch from dynamic memory allocation
return mem indxl;
+ static memory allocation.
The corresponding fast code is: [FAST CODE]
int (
cin>>r>>>;
$ Linclude<biig tetooissssddrroo
ri.resize (n);
for (int 0; <n ; i++)
using namespace std;
cin>
cout cculveeo (0);
int n, k; long long [p[10005));
return 0;
long long arr [100005];
int main
scanf ("Sdad",sn,dd
for 1=1;1<=0;1++)
scanf ("cnd(("__dd,6aarr1
for (int 1-2:1==0;11
dp[i]=ie18; for (int (ikk mp
(atr)) 11))
printf("&1dd" [n]);
[/FAST CODE]
Figure 6: Performance optimization example 2.
19

---

EDITLORD: Learning Code Transformation Rules for Code Editing
C.2. Decompilation
We show several examples demonstrate how EDITLORD improves the readability of machine-dccompiled code.
Given machine decompiled code
EDITLORD output
void funcã (char *param_1, long
The given code describe the foll following
4
param_2)
problem: Convert string from case
{
-
(upper or lower) to another, handling special characters appropriately.
char oVarl;
-
long 1Var2; ulong uVar3;
The input specification is: The programs take string input, which may contain uppercase and lowercase letters, and a - buffer to store the converted string.
char *poVaré;
-
int ivar5;
bool bVar6;
The output specification is: Output the converted string, where all letters are converted to lowercase, while preserving + the case of special characters.
IVrr2 - peVar4 param_l;
do
Following editing rules should be applied:
if (IVar2 == 0) break;
switch from arbitrary pointer checks to clearer termination conditions switch from complex pointer arithmetic
IVar2 IVar2 -1;
evarl *povar4;
povar4 - povar4 + 1;
- clear variable assignments
while (evarl !- (00);;
switch from indirect variable assignments to direct return statements switch from complex multi-step declarations - to clear single declarations switch from complex logic to simple loops
iVarS = (uint)lVars - 1;
if iVar5)
uVar3 0; do {
eVarl = param_1 [uVar3];
switch from complex complex if-else structure to simplified conditional logic switch from implicit return values - explicit return values
if ((bytc) (eVarl + 0x9fU)
(xla) {
cVarl cVarl + -0x20;
switch from cryptic variable nanes to descriptive variable names The corresponding source code is: IRRIGINAL SOURCE CODE)
else if (byyte) (eVarl (xbiU)
(xla)
eVarl cVarl + 1
static void func0 (char *str, char +buf)
(char +) (param_2 + uVar3)
eVarl;
{
bVar6 uVar3 != (uint) 1Var2
int i, len strlen (str);
2;
for (i 0j
Len; i++)
uVar? = uVar3 1;
if (str[i] >= 'A' & striil <= '2')
while (bVar6);
but [i] strli] + +1"
else if (str[i] >= la" 68 str[i] <= !2!)
(undefined +) 2param_2 iVar5)
buf[il str[i] else buf[i] str
0;
return;
buf [len] '\0';
[/ORIGINAL SOURCE CCDE]
Figure 7: Decomilation example
20

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given machine decompiled code
EDITLORD output
undefired8 funcO (char *param char
The given code describe the following
- .param_2)
S problem: Determine a string
{
S rotation of another string by checking if any substring of the second string is a substring of the first string.
char eVarl;
int iVar2;
uint uVar3;
The input specification is: The programs take - two strings as input: the first string is
char +poVará;
char +pcVar5;
- the original string, and the second string is the potential rotation.
long 1Var6;
size_t _n
The output speci
is: Return boolean - value indicating whether the second
bool bVarl;
- string is a rotation of the first string.
1Var6 = -1;
Following editing rules should be applied: switch from ambi mubig function signatures to
pevar4 param_2;
do
- clear function signatures
if (1var6 -- 0) break;
switch from complex pointer arithmetic to - clear variable assignments
1ar6 1ar6 + -1;
eVarl *pcVar4;
switch from indirect variable assignments to direct return statements switch from complex logic to simple loops switch from complex loops to clear - single-respossbblttt sections switch from complex nested cond't tionals to clearer sequential logic
peVar4 pcVar4
while (eVarl (
uVar3 (uint)
iVar2 = uVar3 1;
pevar4 (char
mamolo (1ong) (int) (uvar3
ivar2));
switch from complex variable operations to - direct assignments switch from redundant checks to w straightforward boolean comparisons The corresponding source code is: (ORIGINAL SOURCE CODE]
if (0 < iVar2)
1Var6 (long) iVar2;
0;
strnopy poovarr,param, _n, 1 _n);
static int func0 (const char *sl, const char *82)
strnepy (pcVar4 (1Var6
param ;
[1Var6 1X0";
char *s; size t len, i;
poVar5
strstr (param 1, poVar4); if (pcVar5 != (char 0x0x free (pcVar4);
len strlen (a2);
malloc (len 2 1);
if (s NULL) return (0); for (i 0,
return 1;
len; i++)
strnepy (s, a2 + i, len - 1)
bVar7 != uVar3 - 2;
strnepy (s len - s2, 1)
s Ilen]] = '\'';
while (bVar7);
if (sl,, 8) NULL) (r)e
free (pcVar4); return 0;
return (1)
frre(s)
return (0)
l/ORIGINAL SOURCE CCDE
Figure 8: Decomilation example 2.
C.3. Security Hardening
We show several examples to demonstrate how EDITLORD hardens the vulnerable code.
21

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given vulnerable code
EDITLORD output
#include <stdiib.h>
The program defines function that al a1looe locates
#include <string.h>
buffer of specified si ze and initializes its contents by writing 'A' characters at both the start and end of the buffer. The goal is to ensure that
char+
allocate buffer and (site buf_size)
the buffer is filled correctly without
char- buf
exceeding its allocated size, thereby - preventing potential buffer overflows.
4 ( (buf_
memset (buf, 'A', 10);
The input specification is: The function
memset (buf + buー_size 10, ' 10);
takes single input paraneter, buf buf_size, which specifies the size of the buffer be allocated. This size determines how
return buf;
- much memory will be reserved for the buffer.
The output speci fication is: The function returns pointer to the al ocaaed and initiali izd buffer.
-
the allocation is successful, the buffer will contain 'A' characters at the specified positions; otherwise, it may return NULL if memory allocation fails.
-
Following editing rules should be applied: switch from no checks on function zeturn values to check function return value The corresponding
code is:
(SECURE CODE]
char+ allocate buffer_ and write (size buf_size) char- buf mehar
if (Ibuf)
return 0;
memset (buf, 'A', 10); memset (buf buf 10, 'A', 10); return buf;
[/SECURE CODE]
Figure Secure hardening example 1.
22

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given vulnerable code
EDITLORD output
#include <stdio.h>
The program aims to retrieve user information
Minclude <stdlib.h>
from SQLite database based on
Ainclude <sglite3.h>
provided username and password. It connects the database, executes
#include <string.h>
query to find the user, and returns the user's information if found. The program ensures that the database interactions secure and that user Input is handled appropriately to prevent SQL injection - vulnerabilities.
char aget_user *db, const
char susername, const char "password)
sqlite3_etmt *stmi;
char (2561;
char -userInto = NULL;
The input specitication is: The program takes three command-line arguments: the database filename, username, + password. These inputs are used to query
impritt(qquery. sizeof (query), + "SELECT userinfo FROM users
RHERE username = 'is' AND password 'Ss'", username, password);
_ the database for matching user
credentials.
The output specification is: The program outputs the user's 's information - matching user is found in the database, or an error message if no user matches the provided credentials or if error occurs during database operations. Following editing rules should be applied: switch from direct SQL string intezpolation
a
if (eclitc3__rcparo__22 (db, query,
-1, &stmt, NULL) != SQLITE_OK)
(steer, (stderr, "Failed
+
prepare statement: $s\n",
- oglite3_erraen (db));
return NULL;
to use of parameterized Logic The corresponding [SECURE CODE]
code is:
if (sqlite3_step (stmt).
char *get_user (sqlite3 +db, const char * *username, const char *password) { sqlite3_stmt *stmt;
- SQLITE_ROW) {
const unsigned char result
char vuserInfo NULL; const char +query "SELECT userInio FROM users WHZRE username = ? AND password ?";
sglliee__clumne (stmt,
+ 0);
userInfo strdup (const char result);
if [sqlite3_prepare__ (db, query, -1, sstmt, NULL) !- SQLITE_OK)
pprint "Failed to prepare - statement: $s\n", [ilite3_errmsq (db) return NULL;
eqlite3_fil finalinlllll iie (stmt)); return userInão;
sqlite3_bind_toxe (stmt, 1, username, -1, SQLITR_STATTC);
@gliite_biiddteee bi___iin_eettttt____ee___e______ (stmt, 2, password, -1, SQU.ITR_STATIC)
(sqlite3_stee __tepe___pp__p_eeee (stmt)
SQLITE ROW)
const unsigned char *result
Bacteee__oomm__t__e text (stnt, 0):
userInfo strdup (const char result);
sqlite3_finalize (stmt);
return userInfo;
/SECURE CODE
Figure 10: Secure hardening example 2.
23

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given vulnerable code
EDITLORD output
#include <string.h>
The program defines function that trims
trailing spaces from given string. It modifies the string in place, ensuring that any whitespace characters the
void trim_spaces (char= str)
int len strlen (str);
while [ler - 1] !)
+ are removed, leaving the rest of the 4 string intact.
str(len - 1] 1\0;;
len--;
The input specification is: The function takes single string input, which is character array that may contain trailing spaces that need to be removed.
-
The output specification is: The function
does not return any value; instead, it modi fies the input string by removing any trailing spaces, effective's shortening the string if necessary.
-
-
Following editing rules should be applied: switch from unvalidated input handling check for NULL pointers before access switch from unvalidated input handling to check for buffer overflows on menory accesses
The corresponding (SECURE CODE]
code is:
void trim_spaces spaces (char* str)
int len strlen (str);
while (len > &6 str [len 11 str 1] 'ND'; len--;
I/SECURE CODE
Figure 11: Secure hardening example 3.
24

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given vulnerable code
EDITLORD output
#include <stdio.
The program
read line of input from the standard input (stdin) into dynamically allocated buffer, ensuring that the input does not exceed the specified buffer size. It is designed to prevent buffer overflow vulnerabilities
#include <stdiib.h>
char* read stdin_tditttee ttcittttouuufeeueeeeeeeeee (int
buf_size)
char- buf
+ (char-)malloce (bui_ gets (buf);
- by limiting the number of characters read
- based on the buffer's capacity.
return buf;
The input specification is: The function takes integer input representing the
maximum size of the buffer, which
-
- dictates how many characters can be
safely read from stdin.
The output speci fiction is: The function returns pointer to the buffer
+
containing the user input, ensuring that the input is properly null-terminated and does not exceed the allocated buffer size.
Following editing rules should applied: switch from unvalidated input handling to check for buffer overflows on memory accesses
handling check for + NULL pointers after memory allocation The corresponding secure code is: (SECURE CODE)
switch from
char= read_stdin_to_ (int buf_size) char- buf (char malooc (buf_aize);
fgets (buf, buf_size, stdin);
return buf;
(/SECURE CODE]
Figure 12: Secure hardening example
25

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Given vulnerable code
EDITLORD output
#include <stdiib.A>
The program detines function that zetrieves the price of an item at speci fied index from an array of prices. The index
int get_price_at__nnee (int prices,
+ int size, char index_str) { int index atoi (index_stee) str); return (index);
+ provided as string, which is converted to an integer. The function returns the price at the corresponding index if it's valid,
if the index is out of
bounds.
The input specification is: The function
takes three inputs: an array of integers representing prices, the size of the
array, and string representing the index of the price to retrieve.
The output specitication is: The function
returns the price at the specified index if it is valid; otherwise, it returns
error message indicating that the index - is out of bounds.
Following editing rules should be applied: switch from direct parsing to implement comprehensive validation checks for + character handling
The corresponding secure code is:
(SECURE CODE]
int ret_price_at_inee (int* prices, int size, char- index_str)
index atoi (index_str]) if (index II index >= size)
return -1; // Error: index out of bounds
return prices [index];
I/SECURE CODE]
Figure 13: Secure hardening example 5.
26

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Table 6: Evaluating DeepSeek-Coder 1.3B on the Code Polish task in CodeLditorBench.
Accuracy OptScoreTime OptScore
Finetuned 0.9%
0.03%
0.09%
EDITLORD (Ours) 23.4%
1.83%
1.19%
Table 7: Unseen CWEs evaluation. We also include evaluation seen CWEs.
Correct@ ls
Sccurity @
Correct & Sec@ lt
1 10
k
50
k
k
10
50
k
k
10
k
50
Finetuned 24.3
38.4
41.7
12.8
44.0
50.0
21.6
25.0
Seen CWEs
EDITLORD (Ours) 36.8
53.3
66.7
12.5
43.3
58.3
8.7
24.7
25.0
Finetuned 24.1
40.0
8.6
14.8
22.5
11.5
17.5
Unseen CWEs
EDITLORD (Ours) 29.6
48.5
57.5
12.1
23.8
30.0
7.0
16.9
22.5
Table Unseen programming languages evaluation. We also include evaluation on seen language.
Accuracy OptScoreTTime OptScore
Finctuned 1.4%
0.02%
0.24%
Language (cpp)
EDITLORD (Ours)
28.3%
3.1%
2,29 %
Finetuned
0.7%
0.04%
0.02%
Unseen Languages
EDITLORD (Ours)
%
1.18%
0.63%
D. Rule Learning Details
Given the meta-rule set (r/r pre construct -> post construct) (Figure 2), manifest these meta-rules each training sample for augmentation. However, directly prompting the LM to infer meta-rules for each pre-edit and post-edit code pair (I;, Vi) often leads to hallucinations. To mitigate this, we propose two-step approach. First, we apply the LM to extract pre construct rules from and post construct rules from ys separately. Then, we identify which of these pre/post construct combinations are included in the meta rule set. We adopt these verified combined rules as the meta-rules for the code pair (x,, (s).
E. Hyperparameters
To finetune DeepSeek-Codee we use default batch size of 32, learning rate of le-5, and 4,000 context lengths for both the input and output tokens. The models are optimized using AdamW and trained for fixed number of 10 epochs, and we use the model checkpoint that achieves the best validation loss for inference. To finetune GPT-4o mini, we train for only one epoch. At the inference stage, we set the temperature to 0.7 and use the model's default window size, i.e., 16K for Coder and 128K for GPT-4o-mini.
DeepSeek- Sedeer
F. Additional Experiments
F11 Extra Well-Known Benchmark
We further evaluated our finetuned DeepSeek- 1.3B on the Code Polish task in CodeFditorBBech (Guo et al., 2025). We follow their metrics by focusing on 1) accuracy: the percentage of problems with correct edits; 2) OptScoreTime Time: the execution time improvement; and 3) OptScore, the improvement computed by the averaged time and memory. As illustrated in Table 6, EDITLORD, even without extra finetuning on this dataset, outperforms the baseline by 22.5%, 1.8%, and 1.1%, respectively.
27

---

EDITLORD: Learning Code Transformation Rules for Code Editing
Table 9: Comparing EDITLORD to the finetuned baseline under varying amounts of training data.
Readability: char
Data Usage
Compile, Correct-
token emb
Robustness
Finetuned
50%
38.9
77.1
36.6
40.8
37.5
50%
41.2
93.1
42.6
46.3
41.4
EDITLORD (Ours)
100%
46.6
93.1
44.0
47.6
41.4
E.2. cot-of-Dommann Generalization
Unseen CWEs. As described in Section 3.1, our training comes from SVEN (He & Vechev, 2023), but our testing is from CWEval (Peng et al., 2025a) with unseen CWEs. We further analyze the performance the baseline and EDITLORD on unseen CWEs. As shown in Table 7, EDITLORD generalizes better on unseen CWEs, outperforming the baseline by 38.1%% 1%.
Unseen languages. We also investigate EDITLORD's generalizability to unseen languages (Python/Java) in performance optimization tasks CodeEditorBench (Guo al., 2025) when training on C++ code only. Specifically, we train on the HQ dataset from Shypula al. (2024), which contains only C++ samples. Table 8 demonstrates that EDITLORD maintains strong generalization unseen languages, outperforming the baseline by 20.2% in accuracy, 1.14% in execution time improvement, and 0.61 % in combined time and memory efficiency improvement.
E3. Data Efficiency
To evaluate how EDITLORD scale with varying amounts of finetuning data, conduct additional experiment on the decompilation task using only 50% of the finetuning dataset. As shown in Table 9, EDITLORD, trained with just 50% of the data, still surpasses the baseline trained on the full dataset by 5.9%. Moreover, EDITLORD achieves significantly better readability, outperforming the baseline by 13.5% and 16.4% on character- and token-level readability metrics, respectively. This highlights the sample efficiency of EDITLORD, requiring less than 50% of training samples while achieving comparable performance.
28


Reviews:
[{"review_id": "ylEQCp6rn3", "reviewer": "Reviewer_raGi", "summary": "Traditionally, for code editing, language models (LMs) are often used to directly generate the output code (or diff) given the input code in a single turn. There have also been approaches that prompt LMs in a CoT-style manner to generate some reasoning before outputting the edited code. Similarly, existing approaches for supervising LMs for code editing focus on directly generating the edited code (optionally augmented with some reasoning).\n\nThis paper offers a fresh perspective on supervising LMs for code editing tasks. Instead of supervising an LM to directly transform the input code to the output code, the proposed approach first produces an understanding (functional specification) of the input code, a set of rules to transform the input code, and finally, the output code conditioned on the functional specification and the set of rules.\n\nThe paper presents experiments considering 3 different code editing tasks and 3 recent language models (DeepseekCoder-1.3B, DeepSeekCoder-6.7B, and GPT4o-mini). Authors compare their fine-tuning approach with naive fine-tuning; zero-shot prompting; and chain-of-through prompting, for each task and for each model. The results indicate that the proposed fine-tuning approach consistently outperforms the standard fine-tuning and other baselines.\n\nAuthors also present ablations demonstrating their approach to be more robust w.r.t. semantics preserving code transformations and length of the input code.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate your time and effort in reviewing our paper and giving us constructive comments! \n\n**Q1: More well-known benchmarks should be included.**\n\nThanks for pointing this out. We originally focused on individual tasks where the corresponding papers also proposed tailored solutions (e.g., PIE) to ensure we compare to the state-of-the-art baselines.\n\nThat said, we evaluated our finetuned DeepSeek-Coder 1.3B on the Code Polish task in CodeEditorBench. We follow their metrics by focusing on 1) accuracy: the %problems with correct edits; 2) OptScoreTime: the execution time improvement; and 3) OptScore, the improvement computed by the averaged time and memory. EditLord, even without extra finetuning on this dataset, outperforms the finetuned model by 22.5%, 1.8%, and 1.1%, respectively.\n\n||Accuracy|OptScoreTime|OptScore|\n|:-|:-|:-|:-|\n|Finetuned|0.9%|0.03%|0.09%|\n|EditLord|23.4%|1.83%|1.19%|\n\n\n**Q2: How does EditLord work on out-of-domain generalization (e.g., unseen vulnerabilities)?**\n\n\nWe ensured our training and evaluation came from two data sources. As described in Sec 3.1, our training comes from SVEN, but our testing is from CWEval with unseen CWEs.\nWe add below a breakdown of the baseline and EditLord’s performance on seen/unseen CWEs. EditLord consistently generalizes better than the baseline, outperforming it by 7.5% and 38.1%, respectively.\nWe also add generalization tests on unseen languages (Python/Java) in performance optimization in CodeEditorBench. EditLord achieves improvement in both seen and unseen languages, outperforming it by 2.05% and 0.61%, respectively.\n\nAlong with the length generalization results in Sec 3.5, our added experiments here show that EditLord maintains strong generalization in various settings. We will include them in the draft.\n\n|Security|Methods|Correct@k|||Security@k|||Correct & Sec@k|||\n|:-|:-|:-|:-|:-|:-|:-|:-|:-|:-|:-|\n|||k=1|k=10|k=50|k=1|k=10|k=50|k=1|k=10|k=50|\n|Seen CWEs|Finetuned|24.3|38.4|41.7|12.8|44.0|50.0|7.7|21.6|25.0|\n|Seen CWEs|EditLord|36.8|53.3|66.7|12.5|43.3|58.3|8.7|24.7|25.0|\n|Unseen CWEs|Finetuned|24.1|35.0|40.0|8.6|14.8|22.5|4.6|11.5|17.5|\n|Unseen CWEs|EditLord|29.6|48.5|57.5|12.1|23.8|30.0|7.0|16.9|22.5|\n\n|Code Polish|Methods|Accuracy|OptScoreTime|OptScore|\n|:-|:-|:-|:-|:-|\n|Seen lang (cpp)|Finetuned|1.4%|0.02%|0.24%|\n|Seen lang (cpp)|EditLord|28.3%|3.1%|2.29%|\n|Unseen lang|Finetuned|0.7%|0.04%|0.02%|\n|Unseen lang|EditLord|20.9%|1.18%|0.63%|\n\n\n**Q3: The granularity of the input code (function/class/file level) for each task is unclear.**\n\n\nThe input code is at the file level for all tasks. This ensures the code can be compiled to measure the functional correctness. Please see detailed examples [here](https://anonymous.4open.science/r/EditLord-9C9B/example.pdf). \n\n\n**Q4: How robust is EditLord to noise in rule manifestations during inference?**\n\n\nGreat point. We observed that learning rules for each sample often leads to noisy and repetitive rules, which can degrade performance. This motivated us to propose Alg.1 to disentangle the meta-rule learning and the per-sample rule manifestation. While there is no formal guarantee on the generated editing rules’ correctness, our evaluation shows our rule set consistently improves editing performance (Sec. 3.2). That said, generating provably correct rules in formal languages with formal verifiers is indeed an exciting future work.\n\nWe include some preliminary results on the robustness of EditLord against randomly shuffled rules for training. The following shows that perturbed rule sets lead to less than 2.4 performance changes in the decompilation task.\n||Correct|Compile|Readability|||\n|:-|-|-|-|:-|:-|\n||||char|token|emb|\n|EditLord|93.1|46.6|44.0|47.6|41.4|\n|w/ shuffle|94.0|47.3|43.1|45.2|39.7|\n\n\n**Q5: How large is the initial rule set G^0, and the final discovered rule set G? Can you share the entire rule set for each task?**\n\n\nThe initial rule set $G^0$ has 2.9K, 1.9K, and 1.2K rules for performance, decompilation, and security hardening, respectively, while the final rule set G has 221, 228, 237, respectively. Table 8 shows the example rules. Thanks for your interest. We will definitely release the full set once the paper is ready to be published.\n\n\n**Q6: Will extending CoT reasoning with their rule set give better results?**\n\n\nYes, we included this study in the paper. As described in Fig.2, the *prompting* editing mode refers to CoT prompting with our meta-rule set. In Fig.4 (R=0 means CoT prompting without iteratively refining the generated code), we compare this CoT setting with zero-shot prompting w/o CoT (i.e., w/ vs w/o EditLord). Including our rules improves the CoT performance by 46% across all the tasks. \n\n\n**Q7: Related work not discussed.**\n\n\nThanks for pointing out these benchmarks. We have included preliminary results on CodeEditorBench (Q1), showing EditLord’s potential to generalize to unseen benchmarks. We will add new results and discussions of these benchmarks in our paper.", "claims_and_evidence": "Claims made in the paper are supported by clear and convincing evidence.\n\nHowever, for further validation of the results, it would be nice to have additional experiments on some other well-known existing code-editing benchmarks. Please see \"Essential References Not Discussed\".", "methods_and_evaluation_criteria": "Proposed methods and/or evaluation criteria make sense for the problem or application at hand.", "theoretical_claims": "There are no theoretical claims in the paper as such.", "experimental_designs_or_analyses": "Experimental designs seem sound and valid. \n\nHowever, for further validation of the results, it would be nice to have additional experiments on some other well-known existing code-editing benchmarks. Please see \"Essential References Not Discussed\".", "relation_to_broader_scientific_literature": "The key contributions are related and relevant to prior literature on using language models for code editing.\n\nThe paper utilizes benchmarks developed by the following prior literature on code editing for evaluating their proposed approach.\n\n[1] Learning performance-improving code edits. arXiv preprint arXiv:2302.07867, 2023.\n\n[2] Llm4decompile:Decompiling binary code with large language models. arXiv preprint arXiv:2403.05286, 2024.\n\n[3] Cweval: Outcome-driven evaluation on functionality and security of llm code generation. arXiv preprint arXiv:2501.08200, 2025.", "essential_references_not_discussed": "The following code-editing benchmarks are fairly well known and should be discussed and included in experiments\n\n[1] [CodeEditorBench: Evaluating Code Editing Capability of Large Language Models](https://arxiv.org/abs/2404.03543)\n\n[2] [NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness](https://arxiv.org/abs/2401.15963)\n\n[3] [Aider Code Editing Benchmark (including aider polyglot)](https://aider.chat/docs/benchmarks.html)\n\n[4] [HumanEvalFix](https://arxiv.org/abs/2308.07124) (a code-editing variant of Human eval)", "other_strengths_and_weaknesses": "Strengths\n\n* Paper should serve as an interesting read for audience interesting in language models for code editing\n* The paper presents experiments considering 3 different code editing tasks and 3 recent language models (DeepseekCoder-1.3B, DeepSeekCoder-6.7B, and GPT4o-mini). Authors compare their fine-tuning approach with naive fine-tuning; zero-shot prompting; and chain-of-through prompting, for each task and for each model. The results indicate that the proposed fine-tuning approach consistently outperforms the standard fine-tuning and other baselines.\n* Authors also present ablations demonstrating their approach to be more robust w.r.t. semantics preserving code transformations and length of the input code.\n\nWeakness\n* While experiments are fairly detailed, the paper currently ignores some well-known benchmarks specifically designed for code-editing. Covering these benchmarks should help strengthen the claims in the paper and add more credibility. (this is the only major weakness I observe in this paper.)\n* The proposed approach assumes a training dataset in the target domain. (What about covering security vulnuerabilities that were not a part of the training dataset?). Investigating out-of-domain generalization of the proposed approach should be interesting.\n* Paper currently does not provide examples corresponding to each of the three datasets. One representative example from each of the three code-editing tasks would be very helpful. It will help us understand the granularity of the input code (function level/ class level/ or file level) for each task.", "other_comments_or_suggestions": "Comments\n* Post-edited code may not always necessarily be equivalent to the pre-edited code in case of bug fixing. Therefore, s_i should be a function of both x_i and y_i and not x_i alone.\n* If possible, the paper should included zero-shot & CoT performance of larger models like GPT4-o as well. It would be interesting to see if fine-tuning GPT4o-mini makes is as good as GPT4-o for code-editing.", "questions_for_authors": "* How robust is the proposed approach to noise in rule manifestations? During training rules are manifested while making use of both x_i and y_i. During testing, we don’t have y_i, so we might end up using irrelevant rules.\n\n* How large is the discovered rule set for different code editing scenarios? Is it possible to share entire rule set for each of the three scenarios/datasets? Additionally, how large is the initial rule set G^0 and the final rule set G?\n\n* Have authors considered extending CoT reasoning with their rule set? Explicitly augmenting CoT reasoning to use rules might give better results?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jrylxn0CUl", "reviewer": "Reviewer_wc39", "summary": "EditLord is a system designed to improve performance on code editing tasks (e.g. performance/readability/security). The proposed pipeline involves a few steps, starting from a training dataset (editing task-specific) with pre/post edit programs. In the first step, a LLM is used to produce a set of editing ‘meta-rules’. In the second step, a LLM is used to describe the transformation that takes places for each training task. Experiments show, across different tasks, that finetuning a LLM with these extra annotations leads to better performance versus finetuning a LLM on just the tasks alone.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your effort in leaving us such encouraging comments. \n\n**Q1. It’s unclear how robust the meta rule creation process is. And there aren’t really any guarantees on the quality without human interaction.**\n\nGreat point. We observed that if we simply bootstrap per-sample rules, the rules often end up being noisy, repetitive, and not generic. This leads the resulting meta-rule set to be too large to fit in the model context, and the resulting per-sample manifested rules become extremely susceptible to the order of training samples. This motivated us to *mitigate* this issue by developing the meta-rule learning algorithm to keep the meta-rule set concise and generalizable. As you have also noted, our evaluation in Sec 3.6 shows that exposing the meta-rule set during inference allows us to invite human intervention to further improve the quality of the edited code.\n\nWhile there is no formal guarantee on the generated editing rules’ correctness, our evaluation shows our rule set consistently improves editing performance (Sec. 3.2). In our response to your Q4, our additional results show that our meta-rule set learning algorithm is relatively robust against the perturbations of the training samples. We do agree that bringing provable guarantees to the editing rules, e.g., by interacting with formal verifiers brought by the compiler techniques, is an exciting future work.\n\n**Q2. It’s unclear how annotation-driven performance improvements scale with varying amounts of finetuning data.**\n\nThis is a great suggestion. We have added an experiment on the decompilation task when using only 50% of the finetuning data. The results show that EditLord, with only 50% of the training set, still outperforms the finetuned baseline with 100% training samples by 5.9%. Its readability results remain 13.5% and 16.4% higher than the finetuned baseline on char- and token-level readability. This result demonstrates that the sample efficiency brought by EditLord, requiring less than 50% of training samples while achieving comparable performance.\n\n|Data Usage|DeepSeek-Coder-1.3B|Correct|Compile|Readability|||\n|:-|:-|-|-|-|:-|:-|\n|||||char|token|emb|\n|50%|EditLord|41.2|93.1|42.6|46.3|41.4|\n|100%|Finetuned|38.9|77.1|36.6|40.8|37.5|\n|100%|EditLord|46.6|93.1|44.0|47.6|41.4|\n\n**Q3. What is the performance of larger, non-finetuned LLMs w/ or w/o rules/specifications appended?**\n\nWe added an experiment exploring the GPT-4o-mini’s (2024-07-18 version) performance when incorporating our meta-rule set and functional specifications. As shown below, simply including them for non-finetuned GPT-4o-mini will improve its correctness and readability by 11.9% and 8.6%, respectively.\n||Correct|Compile|Readability|||\n|:-|-|-|-|:-|:-|\n||||char|token|emb|\n|Prompt|44.3|61.8|33.1|37.0|37.9|\n|w/ EditLord|49.6|57.3|36.2|40.1|41.0|\n\n\n**Q4. How stable is the convergence of the meta-rule set? Will shuffling introduce high variance?**\n\nWe have added an experiment that randomly shuffles the initial dataset before passing it into Alg. 1 and obtaining a pair of meta-rule sets. We then measure the similarity between these two sets. Specifically, for each rule in one set, we compute its average semantic similarity to the top 5 most similar rules in another set and average them again to obtain the overall similarity. The semantic similarity between two rules is computed by the cosine distances of the rule embeddings computed by CodeSage, following the similar setting as in the readability metrics in Sec. 3.1. The resulting semantic similarity between the shuffled rule sets is 0.87, close to 1.\n\nWe also show the end-to-end results on the robustness of EditLord introduced by the resulting meta-rule set. The perturbed rule sets lead to less than 2.4% performance changes.\n||Correct|Compile|Readability|||\n|:-|-|-|-|:-|:-|\n||||char|token|emb|\n|EditLord|93.1|46.6|44.0|47.6|41.4|\n|w/ shuffle|94.0|47.3|43.1|45.2|39.7|\n\n**Q5. What is DIS in Alg. 1?**\n\nDIS is the internal embedding distance calculated by the LLM. We intended to use DIS to describe that we ask the LLMs to discover the rules that share similar semantics to decide whether to directly ADD the current rule to the rule set or MERGE it with an existing rule in the rule set. We acknowledge that this can lead to confusion, and we have updated the algorithm at [link]((https://anonymous.4open.science/r/EditLord-9C9B/algo.pdf)).\n\n**Q6: Essential References Not Discussed.**\n\nThanks for pointing out these related benchmarks. We have added the discussion to our draft. We view code generation tasks as complementary to ours as their input is not the code, but usually natural language specification, similar to our functional specification. \n\nThanks for pointing out skill discovery works, e.g., TroVE and GENOME! We were inspired by TroVE in the very early stage of our project but ended up narrowing down our focus to primarily code-editing works. We should definitely discuss both, and we have added them to the draft.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes they seem to.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experiments seem sound in design.", "relation_to_broader_scientific_literature": "This system works on the problem of code editing/improvement; offering a general method/strategy for improving LLM finetuning for this task.", "essential_references_not_discussed": "References largely seem appropriate.\n\nAlthough not 'essential', I would encourage the authors to add a discussion of works that try to discover similar artifacts as their 'meta rule-sets'. E.g. for program synthesis (not editing) library learning works (DreamCoder, LILO), or for more general LLM tasks skill discovery (TroVE, GENOME). This connection seems useful, and relevant, though I think even including it in the appendix would be fine.", "other_strengths_and_weaknesses": "This is a strong paper. It offers an interesting solution for a hard, well-studied problem, demonstrating consistent experimental improvement over a range of tasks.\n\nThe idea of trying to extract out meta-rules from a corpora of editing tasks is noteworthy and to my knowledge has not be tried before; beyond benefits during finetuning, this also allows for human-in-the-loop improvements through editing / curation of these rule sets (Section 3.6). \n\nThe experimental framework seems robust and quite comprehensive; from my read, the paper does a good job of supporting their claims against prior finetuning alternatives. \n\nThe most significant weakness of the paper is that its unclear how robust this meta-rule creation process actually is; while it certainly proves consistently useful, the machinery to produce these rules is relatively simple from a certain perspective (which in some ways is a positive of the system). There aren't really any guarantees that these meta-rules will be good / interpretable / general (unless the human-in-the-loop machinery is employed), but this can be left as a problem for future work / investigations.", "other_comments_or_suggestions": "One thought that I had while reading the paper: the proposed process basically creates better annotations for LLM finetuning. When trained on the entire dataset, these annotations lead to improved performance, but how does this performance improvement change as a function of the amount of finetuning data used? I would almost expect this method to do 'better' (than the default finetuning strategy) when the amount of finetuning data is small (would be a useful appendix/supplemental experiment).", "questions_for_authors": "Though not critical, it would have been interesting to see how larger LLMs (not finetuned) would perform on these tasks, especially when the rules/specifications are appended to the prompt or not.\n\nHow stable is convergence of meta-rule set? I.e. if you re-order the rules from 2.2 before passing them into Alg 1, do the same types of meta-rules get discovered, or does this have high variance? I think more analysis on both the stability meta-rule set generation process, and how well it matches 'semantics' (human prior, by some measure) would bring the paper from good to great.\n\nClarification question: What is DIS in algorithm 1? Is this some text-embedding distance?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sMsEVgC4fC", "reviewer": "Reviewer_QPpP", "summary": "The paper introduces a method for editing code in a decompositional way, where it extracts editing steps, obtains functional specifications, and performs rule-based code editing by prompting LMs.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your taking the time and effort to read our paper and leave constructive comments. \n\n**Q1. Why extracted editing rules are effective in improving code efficiency or what the rules actually look like.**\n\nThese extracted editing rules serve as editing guidelines that help the model reason about the useful steps and actions to take to generate the edited code. Please refer to Table 8 in Appendix B for some examples of the rules. We have also added the detailed samples to the [anonymous repo](https://anonymous.4open.science/r/EditLord-9C9B/example.pdf).\n\n**Q2. The figures are confusing and do not clearly illustrate the methodology, e.g., the input/output of each subtasks, rule usage, and the editing process.**\n\nWe apologize for the potentially unclear figure presentation. To clarify, we take the performance optimization task as an example in Fig 1. The input is a slow (pre-edit) code, while the expected output is semantically equivalent but faster (post-edit) code. Now consider inferencing by an EditLord finetuned model. Instead of directly generating the faster code (post-edit code), the model will first generate functional specifications and editing rules, and then these generated is in the context, and the model keeps on generating the post-edit (faster) code. \n\nFor the decompilation task, the input is an unreadable, non-idiomatic ghidra-decompiled (pre-edit) code, while the expected output is a more readable, idiomatic (post-edit) code (see Fig 3). \n\nFor the security hardening task, the input is vulnerable code, while the expected output is secure code.\n\nIn the upper part of Fig 2, we illustrate how we prepare the functional specifications and editing rules. We will take the pre-edit and post-edit code pair from the training data as input and use LLMs to annotate the functional specification and the editing rules for each pair. \n\nSince editing rules are often shared among different samples, our Alg. 1 describes how an LM serves as a rule learner by iteratively refining the raw meta-rule set and producing a more concise meta-rule set by using operations ADD, MERGE, and PRUNE. This step is done *independently* for each task. \nWe also state the input and output for each editing mode more clearly in the [updated figure](https://anonymous.4open.science/r/EditLord-9C9B/workflow.pdf) for our improved workflow figure.\n\n**Q3. The dataset used in their experiment is unclear and causes confusion (e.g., HQ, HumanEval).**\n\nWe evaluate different editing tasks with different datasets, as different objectives require different evaluations. Specifically, for performance optimization tasks, we train on the HQ dataset and evaluate on the test split from PIE. For the decompilation task, we train on AnghaBench and test on HumanEval-Decompile. For the security hardening task, we train on SVEN, and test on CWEval. We will update our Section 3.1 to describe the setting more clearly.\n\n**Q4. More benchmarks should be included for better evaluation.**\n\nThanks for pointing this out. Per our response to reviewer raGi, we evaluated EditLord’s finetuned DeepSeek-Coder 1.3B in CodeEditorBench, the benchmark we did not consider in the submission. We follow their metrics by focusing on 1) accuracy: the percentage of problems with correct edits; 2) OptScoreTime: the execution time improvement; and 3) OptScore, the improvement computed by the averaged time and memory together. \n\nSurprisingly, without further finetuning EditLord on this benchmark, but simply running inference using a finetuned EditLord model on a completely different training set, it substantially outperforms the finetuned baseline by 22.5%, 1.8%, and 1.1%, respectively.\n\n||Accuracy|OptScoreTime|OptScore|\n|:-|:-|:-|:-|\n|Finetuned|0.9%|0.03%|0.09%|\n|EditLord|23.4%|1.83%|1.19%|\n\n\n**Q5. Repo seems empty.**\n\nThanks for your interest in our artifact. We would hope to clarify that the repository is not empty. The code and prompts were uploaded along with the paper submission. It is likely that the README is blank, which makes the repo appear to be incomplete. We have already uploaded other important files and updated the README. We hope this addresses your concern.", "claims_and_evidence": "The paper claims that their method improves code efficiency by leveraging the decompositional nature of code editing tasks. However, after reading the full paper, I don’t fully understand why the extracted editing rules are effective in improving code efficiency or what the rules actually look like. Additionally, their figures need significant improvement, as they don’t help in understanding their methodology—such as the input and output of each subtask, or how the rules and editing process work—but instead confuse me.\n\nI also suggest the authors clearly indicate which dataset benchmarks they use in each table of results. From their experiments section, I gathered that they evaluated on one dataset benchmark, the HQ dataset. However, they also mention the HumanEval dataset elsewhere in the paper, which causes confusion.\n\nOverall, the paper’s presentation requires major revisions and improvements to help readers understand how their method works, and more dataset benchmarks should be included for better evaluation.", "methods_and_evaluation_criteria": "See \"Claims And Evidence\"", "theoretical_claims": "Theoretical claims are not required for this work.", "experimental_designs_or_analyses": "The number of compared baselines are enough, but more datasets benchmarks should be included for a more comprehensive comparison.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "See \"Claims And Evidence\"", "other_comments_or_suggestions": "See \"Claims And Evidence\"", "questions_for_authors": "See \"Claims And Evidence\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W3M87V5T9P", "reviewer": "Reviewer_epTy", "summary": "This paper seeks to decompose the traditional end-to-end LLM-assisted code editing task into discrete and step-wise processes. To this purpose, this paper adopted LLM to summarize meta editing rules from 3 editing tasks: optimization, decompilation and security hardening, and augmented LLM performance in a retrieval style. Their proposed approach achieved SOTA performance in the 3 aforementioned tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate your time and effort in leaving constructive comments. \n\n**Q1: The three tasks it focuses on may limit real-world applicability because they are not highly frequent in real-world editing scenarios.**\n\nThe three editing tasks we considered are extensively studied in the literature [1-9], even evaluated on the same datasets as we do. They are broadly applicable already. For example, Scalene, the popular Python profiler, has already integrated GPT to suggest code optimizations for short code snippet [11]; LLMs have been integrated into existing decompilers as plugins [8,9]; Large companies are actively hosting competitions on LLMs to produce secure code [10]\n\n[1] Tan, Hanzhuo, et al. \"LLM4Decompile: Decompiling Binary Code with Large Language Models.\" Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.\n\n[2] Hu, Peiwei, Ruigang Liang, and Kai Chen. \"Degpt: Optimizing decompiler output with llm.\" Proceedings 2024 Network and Distributed System Security Symposium. Vol. 267622140. 2024.\n\n[3] He, Jingxuan, and Martin Vechev. \"Large language models for code: Security hardening and adversarial testing.\" Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 2023.\n\n[4] Peng, Jinjun, et al. \"CWEval: Outcome-driven Evaluation on Functionality and Security of LLM Code Generation.\" arXiv preprint arXiv:2501.08200 (2025).\n\n[5] Shypula, Alexander, et al. \"Learning Performance-Improving Code Edits.\" ICLR. 2024.\n\n[6] Huang, Dong, et al. \"Effilearner: Enhancing efficiency of generated code via self-optimization.\" Advances in Neural Information Processing Systems 37 (2024): 84482-84522.\n\n[7] Peng, Yun, et al. \"PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback.\" arXiv preprint arXiv:2412.03578 (2024).\n\n[8] aiDAPal: IDA Pro plugin that uses a locally running LLM that has been fine-tuned for Hex-Rays pseudocode to assist with code analysis, https://github.com/atredispartners/aidapal\n\n[9] GhidraAssist: An LLM extension for Ghidra to enable AI assistance in RE, https://github.com/jtang613/GhidrAssist\n\n[10] Amazon Nova AI Challenge accelerating the field of generative AI (LLM for secure code generation), https://www.amazon.science/amazon-nova-ai-challenge-accelerating-the-field-of-generative-ai\n\n[11] Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals, https://github.com/plasma-umass/scalene\n\n**Q2: Despite the manual inspection, the quality of these rules has not been formally verified.** \n\nThis is a valid concern. While there is no formal guarantee on the generated editing rules’ correctness, our evaluation shows our rule set consistently improves editing performance (Sec. 3.2). We included additional results to show that our meta-rule set learning algorithm can bring improved robustness against the perturbations of the meta-rule set (see below). We do agree that further interacting with formal verifiers to guarantee the correctness of editing rules can be an exciting future work. \n\n||Correct|Compile|Readability|||\n|:-|-|-|-|:-|:-|\n||||char|token|emb|\n|EditLord|93.1|46.6|44.0|47.6|41.4|\n|w/ shuffle|94.0|47.3|43.1|45.2|39.7|\n\n**Q3: In practical software engineering, people apply code edits in a repository. In addition, the code edits are triggered by an issue (see GitHub issue). How can this research push forward in this direction?**\n\nEditLord focuses on file-level local edits rather than repo-level editing. File-level edits are already valuable in many scenarios during software development. For example, the developers frequently refactor their code at the local context, e.g., by editing the function they have just written to make it more efficient. While we do believe repository-level editing is indeed an exciting work, we do not attempt to overclaim that EditLord is readily useful for repository-level edits. Instead, we would like to emphasize that the tasks we considered in this paper are already nontrivial and useful, as we have also argued in our response to Q1.\n\n**Q4: Essential references not discussed.**\n\nThanks for pointing out these related works. We found the main difference between EditLord and many existing editing works (e.g., CoEdPilot, Grace, and CODIT) is that these works mainly focus on repairing functionality, while our tasks focus on preserving original functionality while introducing changes in other dimensions, e.g., performance and readability. That said, these are all very related and should be discussed. We have added discussions in the related works of these papers in our draft.", "claims_and_evidence": "The claim of “The post-edit code must be semantically equivalent to the pre-edit code, i.e., having the same functionality, and possessing the desired new properties” raises concerns about the proportion of such edits in real-world scenarios.", "methods_and_evaluation_criteria": "Code editing tasks are usually treated as translation tasks, and the idea of transforming such an abstract process into a chain-of-edit process is interesting and promising. Meanwhile, the writing style of this paper is easy to follow. However, I have some concerns about this work.\n\nThe first concern is the limited scope of the implementation, as it focuses only on three tasks: optimization, decompilation, and security hardening. My intuition is that these tasks are not highly frequent in real-world editing scenarios, which may limit the broader impact of this approach. I would suggest conducting an empirical study to quantify the actual proportion of such edits in real-world settings, providing a clearer picture of the practical significance and applicability of this work. \n\nThe next concern is that the lack of quality assurance of the rule set or edit.  Despite the manual inspection, the quality of these rules has not been formally verified. For example, it remains unclear whether the retrieved rules can always be correctly applied to the samples, consistently leading to accurate edits. \n\nTo summarize, I encourage the authors to extend the editing scenario to project-wise editing, which holds greater practical significance in software development. Additionally, incorporating methods such as symbolic reasoning to formally verify the rules and assess their applicability would enhance the reliability and robustness of the approach.", "theoretical_claims": "NA, there is no theorical claim in the submission.", "experimental_designs_or_analyses": "Yes, all experimental designs.", "relation_to_broader_scientific_literature": "NA", "essential_references_not_discussed": "[1] Liu, Chenyan, et al. \"CoEdPilot: Recommending Code Edits with Learned Prior Edit Relevance, Project-wise Awareness, and Interactive Nature.\" Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 2024.\n\n[2] Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, and Ashish Tiwari. 2023. Grace: Language Models Meet Code Edits. FSE\n\n[3] CODIT: Code Editing With Tree-Based Neural Models. IEEE Transactions on Software Engineering 48, 4 (2022), 1385–1399.", "other_strengths_and_weaknesses": "NA", "other_comments_or_suggestions": "NA", "questions_for_authors": "In practical software engineering, people apply code edits in a repository. In addition, the code edits are triggered by an *issue* (see github issue). How this research can push forward this direction.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The manuscript proposes a novel approach to improving the code-editing abilities of code language models. The authors propose a simple and intuitive formulation of the editing problem -- as one of infering edit rules given training (before, after) code pairs, and then using the inferred rule sets to improve the editing accuracy of LMs. The proposed solution is elegant and the writing brings out the novelty, technical contributions, as well as the merits of the proposed method. 

I am delighted to see that the authors have shown great rigor in their experiment design, ablations, comparisons with competing methods and baselines. I'm also impressed with the experimental validation on metrics/scenarios beyond standard functional correctness, including implementation efficiency and security.   

Overall, I believe the paper makes good contributions to an important software engineering problem, has strong technical contributions, and a thorough set of evaluations backing the proposed method. I recommend accept.
Decision: Accept (poster)