Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization
Authors: Mingzhe Du
Anh Tuan Luu
Yue Liu
Yuhao QING
Dong HUANG
Xinyi He
Qian Liu
Zejun MA
See-Kiong Ng
Venue: NeurIPS 2025 poster ()
Keywords: Code Generation
Software Engineering
Primary area: applications
TL;DR: This paper introduces an iterative optimization framework that leverages online execution feedback to improve code efficiency.

Abstract: Large Language Models (LLMs) generate functionally correct solutions but often fall short in code efficiency, a critical bottleneck for real-world deployment. In this paper, we introduce a novel test-time iterative optimization framework to address this, employing a closed-loop system where LLMs iteratively refine code based on empirical performance feedback from an execution sandbox. We explore three training strategies: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization~(GRPO). Experiments on our Venus dataset and the APPS benchmark show that SFT and DPO rapidly saturate in efficiency gains. In contrast, GRPO, using reinforcement learning (RL) with execution feedback, continuously optimizes code performance, significantly boosting both pass@1 (from 47% to 62%) and the likelihood of outperforming human submissions in efficiency (from 31% to 45%). Our work demonstrates effective test-time code efficiency improvement and critically reveals the power of RL in teaching LLMs to truly self-improve code efficiency. We released our code and data at https://github.com/Elfsong/Afterburner.

Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization
Mingzhe Du
Luu Anh Tuan!
Yue Liu²
Yuhao Qing'
Dong Huang
Xinyi He*
Qian Liu
Zejun Ma² See-kiong Ng²
Nanyang Technological University NNational University of Singapore 'The University of Hong Kong *Xi'am an Jiaotong University "TikTok
(mingzhe001, anhtuan) @ @ntu. edu.s. fyliu, dhuang, seekiong) edu.sg, yhqing@ess hkk.kk hxyhxy @ stu. www.xxt...du.c.. {qian. 1iu, mazej jnn @tiktok.com
Abstract
Large Language Models (LLMs) generate functionally correct solutions but often fall short in code efficiency, critical bottleneck for real-world deployment. In this paper, we introduce a novel test-time iterative optimization framework to address this, employing a closed-loop system where LLMs iteratively refine code based empirical performance feedback from an execution sandbox. We explore three training strategies: Supervised Fine-Tuning (SFT), Direct Preference Optimiza- tion (DPO), and Group Relative Policy Optimization (GRPO), Experiments on our Venus dataset and the APPS benchmark show that SFT and DPO rapidly saturate in efficiency gains. In contrast, GRPO, using reinforcement learning (RL.) with execution feedback, continuously optimizes code performance, significantly boost- ing both PASS @ 1 (from 47% to 62%) and the likelihood of outperforming human submissions in efficiency (from 31% 1 to 45%). Our work demonstrates effective test-time code efficiency improvement and critically reveals the power of RL in teaching LLMs to truly self-improve code efficiency. We released our code and data /tttp:://githb: thub. com/ Elfsong////// Af terbuurerr
1 Introduction
Large Language Models (LLMs) and agent frameworks are catalyzing a profound transformation in software engineering 77 47
24 76), significantly improving the functional correciness of their code generation and starting rival human engineers in certain tasks [7] 69 28 However, this focus on correctness often overshadows another critical dimension of software quality: com- putational efficiency. In real-world systems, where latency and memory budgets are paramount, code that is merely correct but inefficient can precipitate severe performance bottlenecks, leading to inflated computing costs and system-wide latencies. This chasm between functional correctness and computational efficiency represents formidable challenge to deploying automatic code generation in cussion-critical tasks. This challenge has also spurred the development of code efficiency benchmarks. For instance, EffiBench [26] introduces a relative performance metric against reference solutions, while PIE4PERF [60] utilizes system simulation to meticulously assess the impact of optimizations across a vast corpus of C++ code pairs. Moving beyond pairwise comparisons, Mercury [16] employs percentile ranking against human solutions to highlight the efficiency disparity, and EVALL PERF (44 categorizes generated solution efficiency against reference solutions. These benchmarks consistently point out that despite their prowess in generating correct code, current LLMs often produce solutions
Coorrsspnndigg Authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Reinforcement semiing
Supendsed Tuning
Human Submessen Fercentee "Tmme (%)
48
Superieed Fine-Tuning Renforcemmnt Leaming
Problem, Given drray integers suns and integer target indlccs of numbers
sild tanget.
Pees (4g
60
def DvcSunl petr, nuacs iintl,, targeti
int
Listlinch:
in cnunerste(nunsll
comp leaett Target 1um coop lonent wit: out (mue) -
[]
30
def teoSunise1f, nuns: Listrant), targets intt Listfant): out Tohinuns)
60
60
for in rangeld, 31:
17 (ouns::1l nuns (2-22731
nut append(()
Number of Ciptiiization theaiions
Number Optmiiatioon lterations
While the passgil and efficie ncy gains of supervised finc-t.ning plateau, faciitates improving code efficiency optimization during Inforence.
Comparison of iterative optimization performance between SFT model and a RL model on Venus. While the correctness and efficiency gains of the SFT model plateau, the RL model facilitates iterative optimization during the inference time effectively.
Figure 1:
with suboptimal efficiency [59]. Initial attempts to address this gap, such as Chain-of-Thought Thought [66] in PIE [60], self-optimization in Effilearner (25). or fine-tuning LLMs on an efficiency-oriee dataset (27,, have yielded limited success, often failing to instill the adaptive knowledge for robust efficiency improvements.
In this work, we introduce a novel iterative optimization framework (IOF) designed to enhance LLM-generated code efficiency through a closed-loop system of generation and evaluation, driven by Afterburner and Monolith. As shown in Figure 2 Afterburner takes the original code as input and generates an improved one for the subsequent optimization, where Monolith evaluates the improved code and feeds the empirical code performance back to Af Aerburne... The process mirrors how human developers often optimize code through trial and feedback.
Our extensive experiments the novel Venus benchmark and the widely-used APPS 22 benchmark demonstrate the varied learning dynamics of different optimization strategies within IOF. While Supervised Fine-Tuming (SF7) (38) offers initial efficiency gains in the first few iterations, it quickly saturates and struggles with sustained improvement. Direct Preference Optimization (DPO) [56] consistently performs better than SFT but has the same trend as SFT. In stark contrast, Group Relative Policy Optimization (GRPO) [57) continuously refines code performance. As illustrated in Figure I it boosts PASS @ from 47% 62% and significantly elevates all efficiency metrics, for instance, increasing BEYOND-I from 31 % to 45%. We attribute these divergent behaviors to the fundamental nature of what each method tends to capture: SFT tends capture superficial patterns from mimicking examples. DPO internalizes static preferences based on pairwise comparisons from offline data. In contrast, through online interaction with execution feedback, GRPO cultivates an adaptive proficiency in code efficiency optimization, which enables it to explore and exploit the solution space effectively within an iterative, test-time optimization process. Our key contribution not only lies in demonstrating effective test-time improvement of code efficiency but, more critically, in dissecting how different strategies contribute to this iterative optimization and highlighting the superior adaptability of online feedback-driven RL approaches in efficient-oriented code generation.
2 Related Work
LLMs for Code Generation LLMs have demonstrated remarkable progress in code generation, fueled by extensive training on vast code corpora [2] 41 [1 47). Building upon foundational models such as Llama [62] and Qwen [70). subsequent efforts have specialized these models for coding tasks, yielding variants like StarCoder 447. QwenCoder 34 and OpenCoder [33]. These models excel in
diverse applications, including code completion g 38 12 program repair [50] 45 73, 46), and unit
test generation 29 3. Despite their success in generating, functionally correct code, evidenced by
benchmarks like HumanEval [ LiveCodeBench (36], and BigCodeBench (77), the computational efficiency of the generated code remains less explored frontier.
Code Efffciency Evaluation Addressing this gap, recent work has focused on quantitatively assessing the efficiency of L.MM-generaedd code [71 38
10] EffiBench [26] collects 1000 efficiency-eritical Python problems, evaluating code via an efficiency ratio against reference solutions. PIE4Effi [60] emphasizes the importance of reliable measurement. It utilizes system simulator for
54
23

---

code execution and contributes a dataset of over 77,000 C++ efficiency preference pairs. Deviating from pairwise comparisons, EVALPERF 44 introduces Differential Performance Evaluation (DPE) on 121 performance-ccalllegging tasks, categorizing generated solution efficiency against reference implementations. Mereury [ measures efficiency by percentile rank against a substantial corpus of human-written solutions. More recently, ENAMEL [55] proposed an unbiased estimator eff@@ k for time efficiency. These benchmarks reveal that current LLMs still significantly struggle to produce code that consistently matches expert-level computational efficiency. Building these efforts and inspired by Mercury [16], our work introduces the Venus dataset, which expands upon existing resources with more tasks and solutions to facilitate more rigorous efficiency assessment.
Preference Alignment in Code Generation While functional correctness is paramount, code efficiency is critical yet often overlooked preference in LLM-based code generation. Initial attempts to steer LLMs towards efficiency via prompt engineering, such as Chain-of-Thouge [66] in PIE [60] or self-optimization in Effilearner [25). Subsequent instruction tuning methods have predominantly
aimed at enhancing functional correctness 48 66, 67]. Although some recent works like Swift-
Coder [27] and PIE4PERF 60 used efficiency- focused datasets for model fine-tuning, their reliance on cross- cross-entropy loss hindered the direct instillation of nuanced efficiency preferences. To achieve finer- ainer-grineed preference alignment, RL has emerged as a powerful paradigm for code preference alignment [65]. Initial methods like CodeRL. 40) use code execution outcomes as feedhack. More recent approaches such as StepCoder [14, RLEF 18 and Focused-DPO 72 have significantly advanced functional correctness by leveraging execution feedback. However, these RL methods have largely neglected computational efficiency as a primary optimization target, with existing execution environments typically providing only correctness-based rewards. To enable RL-based optimization for code efficiency, our work introduces Monolith, a high- fidelity sandbox that delivers real-time efficiency metrics, thereby fostering deeper preference for performant code.
3 Iterative Optimization Framework
While current LLMs can produce viable solutions, these often fall short of the performance standards required in resource-constrained or time-sensitive applications [16 555. To bridge this gap, we introduce the Iterative Optimization Framework (IOF), a novel approach designed to enhance the efficiency of I.I. -egeerrree code, As illustrated in Figure [ IOF employs closed-loop system where code is progressively refined through cycles of forward generation and backward evaluation.
Central to IOF are two synergistic components: Afterburner, model suite that proposes targeted efficiency improvements, and Monolith, robust code execution sandbox that provides precise, real-world performance metrics. The interplay between these components drives each optimization iteration: commencing with an original code and an efficiency instruction, Afterburner takes the inputs to generate an improved code alongside its reasoning content. This improved code is subsequently executed within Monolith, yielding empirical efficiency feedback to guide the subsequent optimization iteration. The sections detail the mechanics of Afterburner and Monolith 1ith, and the overall iterative workflow as formalized in Algorithm 1
3.1 Afterburner: Code Efficiency Optimization Models
In the realm of aviation, an afterburner is secondary combustion system integrated into jet engines, designed to provide a significant thrust augmentation 78). While this surge in power comes at the cost of considerably higher fuel consumption, it serves as critical mechanism for scenarios demanding peak performance. Drawing a parallel to this concept, our Afterburner aims to push the code to the maximum. Instead of consuming more fuel, Afterburner
efficiency of leverages the inference-time scaling law [w[688] and the execution feedback from the Monolith sandbox to iteratively refine generated code. For the i-th iteration, the process can be formalized as:
Coutt Afterburner (P, I,C S.R,,,, ,M:"),
where P is the problem description, I E { ('time-efficient' 'memory- -efirrrrffiiient'... integral--eficiien denotes a specific efficiency instruction (e.g., minimizing execution time, reducing peak memory usage, optimizing the integral score). C.n denotes the input solution for the current iteration, and M = Monolith(((m) is its performance metric corresponding to objective I. The refined candidate code Cout is then evaluated to obtain its performance metric, Most Monolith(C.
3

---

Code Update
Problem
Original Cade
Improved Code
Description
Afterburner
Generation Forward
Efficioncy
Original Performance
SFT / DPO GRPO
Monolith
Improved Performance
Instruction
Evecution Sondbox
Venus Dataset
Evaluation Backward
Figure Inference Workflow of the Iterative Optimization Framework (IOF). the forward generation (blue lines), Afterburner takes problem description, efficiency instruction, original code (optional), and original performance as input. then produces reasoning content and improved code in a designated format. For the backward evaluation (green lines), the original code and original performance are updated with the improved versions. The detailed pipeline defined Algorithm
For the subsequent iteration, we select the best-pe rest-performing code via a greedy approach:
(Cput, .M?ut) if M? > M,"
(C
(Cin Mi") otherwise
where Mout MM indicates that the performance of Crut is superior that of C with respect to the objective I. The iterative process continues for predetermined number of iterations Niter.
3.2 Monolith: Code Execution Sandbox
Monolith ihhhr is a catalyst of IOF, which executes generated code and provides the empirical performance feedback to the iterative optimization. Since the efficacy of RL and preference optimization methods hinges on the quality and consistency of the feedback signal [57 18], Monolith prioritizes the consistent measurement in its design. While theoretical complexity analysis (c.g., Big O notation) offers high-level insights into algorithmic scalability (1I). it often fails to capture the nuances of real-world performance. A return signal without discrimination may cause the optimization algorithm to lose the optimization gradient [20]. Moreover, Constant factors, specific implementation details (such as language runtime, library choices, and compiler optimizations), and hardware interactions (CPU architecture, memory hierarchy) significantly influence actual execution time and memory consumption [44] Therefore, for the Afterburner models to learn to generate genuinely efficient code, they require empirical metrics from Monolith that reflect these practical realities.
(passed, time, memory. integral] Monoli thoode,, test_esstttttt cesssccssss,
where code is the code, passed is a boolean value indicating whether code is passed all test cases. time, memory, and integral denote the absolute execution time, peak memory usage, and the integral score of the code, respectively. We will explain how to measure these metrics in Section
4 Code Efficiency Optimization
Data Preparation Recent initiatives like Mercury 16.. EffiBench 26 and EVALPERF 44
have made important strides in evaluating code efficiency (see Table 5). but persistent limitations remain. To address these shortcomings, while also building upon these foundational efforts, we introduce Venus, new dataset designed for rigorous code efficiency assessment: (1) Inspired by Mercury [16] and EVALPERF 44, it computes percentile ranks against diverse distribution of reference solutions, unlike methods relying on single, potentially biased baselines [26] 60). (2)
Venus provides substantially larger set of solutions, averaging 106.6 per task by expanding upon EffiBench (66] and Mercury [16]. This is a significant increase from the fewer than 20 solutions found in existing Python efficiency benchmarks as listed Table ensuring more stable and reliable percentile calculations. (3) It offers holistic assessment by evaluating execution time, memory usage, and their combined impact. As shown in Table 7 Venus Python set includes 2, 2,181 training and 300 test tasks. From this data, we derived training subsets for various optimization methods:

---

SFT Dataset. For Supervised Fine- Sine-Tuning, DS is constructed by sampling pairs of functionally correct solutions for tasks from train where the solution exhibiting inferior computational efficiency is designated C- and the superior one is C- DSSFT comprises 58,833 training instances, with 19,611 instances generated for each of the three targeted efficiency instructions.
DPO Dataset. Each instance in the preference dataset DSDPO consists of prompt IP, and a pair of responses ), where we randomly sample three solutions from Venus assigning the best code as C+ and worst as C=, and the mediocre Chaseline as the baseline, according to their efficiency performance M with respect to the objective I. Averaging approximately 13.3K instances per efficiency instruction type, DS contains 90,864 training instances.
Cold Start Dataset. This dataset designed rapidly adapt Afterburner models to the expected response format. DSCOLD is constructed using tasks from VenuStrain. for which initial responses were generated by Gemini 2.5 Pro. From an initial collection of 3,392 raw responses with the fattinking><sluuttiinn format, we filter and construct DSCOLD with 2,071 instances.
GRPO Dataset. Since AfterburnerGuPO learns from code execution feedback, the DSCRPO training dataset does not require ground -ruth responses. Each instance herein is prompt structured as (P, M). DS employs all 984 distinct tasks in Venustrain-
Supervised Fine- Fine-Tuning SFT is the most intuitive approach to imbue LLMs with an initial understanding of code efficiency. Its core idea is to expose the model to the inefficient code paired with the optimized code, thereby teaching it to learn the patterns that transform suboptimal solutions into more performant ones. The Af Afterburnergr takes a prompt X' = (P. I. C- CIM and the training objective is to minimize the cross cross-entropy loss for generating the expected response C+:
(FF)(FE) = -E 1r__ctt-c-uu MM-)r [log To (C+(2))
where Te (C+ (2) is the likelihood of generating the optimized code C+ given the prompt X. It impels LLMs to learn the mapping from inefficient code to their more efficient counterparts.
Direct Preference Optimization While SFT provides a strong baseline, DPO offers more direct way to align LLMs with efficiency preferences offline, without the need for explicit sampling from a reference model during the training. DPO directly increases the likelihood of positive responses C+ and decreases that of negative ones C thereby tuning the model to inherently generate more efficient code according the specified efficiency objective I. Its key advantage is directly optimizing for the preference objective. The Afterburner pPO loss function is formulated as:
(5) where my is the target model, Tre is a reference model (we use the ahove AfterburnersPT model as the reference). X' = (P, I, Chaseline, .M) is the input prompt. B is a hyperparameter controlling the deviation from the reference model, and is the logistic function.
CDro fref) (x,,,) ,c >~DSoPO [oog a (Blog
se(C+ X) (CT](C)[2)
log
Be(C (X) x.. (C (2) (2)
)]
Group Relative Policy Optimization Building upon the principles of preference- learning, GRPO (57) extends the pairwise offline comparison of DPO to a group-wiss online ranking scenario. For given prompt, GRPO generates multiple roll-outs and learns the relative advantage amongst these roll-outs. Inspired by recent works [20 18). we explore whether it can enhance the code efficiency. As depicted in Figure we first SFT the base model on DSCOLD to align it quickly with the designated response format, thereby providing a well-aligned foundation for AfterburnerGRPo.
Reward Functions. We encourage AfterburnercRPo to think about how to improve the efficiency before generating correct and efficient code. Therefore, the reward function comprises three parts: format control reward, functional correctness reward, and computational efficiency reward:
Format Control Reward. This reward component encourages the model structure its out- put in predefined format. Specifically, Afterburner models are expected to have a thinking phase encapsulated in Boankingg..</thinkking
tags, followed by the actual code within <solu- tion>.. </solutitt tags, Eq. (6) defines the reward as I when the model response matches the regex pattern (See Appendix D.5), otherwise, the reward will -1.
{
if Cout matches the pattern otherwise
Reormar (Cout)
(6)
5

---

Functional Correctness Reward. Ensuring the generated code is functionally sound is paramount. We define a boolean Monolith(C; (C,, test _cs__ccssssss___________ indicate whether the provided code C passes all test cases, where test_cases is set of test cases. Reorreat is defined as:
1.0
if A and A O (upgrade)
0.5
if out and A = 1 (maintained passing status) (7)
Revereer ( (hin, (out)
0.5
if A 0 and A O (maintained failing status)
1.0
if A 0 and A 1 (downgrade)
Efficiency Improvement Reward. Given the efficiency instruction I, this reward measures the relative improvement in the corresponding performance metric & E {time, memory, integral) of roll-out code compared to the baseline input code. Here, & = Monolith(C, test_cases, and Eupper are the absolute performance value and the upper limitation with respect to respectively.
in Clip
E out
Refficiecy tanii(gaaie)
E
Eciip - clip(E 0,0, Eupper):
(8)
E in clip
+
Final Reward. We apply an additive reward to combine all rewards comprehensively. 8,, Se, and Be are weight hyperparameters to each corresponding reward competent.
Rimmal = Bj Rformut + Be Reoorect + Be Refficiency
Objective. GRPO leverages a policy gradient approach to optimize the target policy Tg based on the old one ua The training objective encourages the policy to favor generated candidates that not only possess high intrinsic quality but also demonstrate superior performance relative to their peers within the same generation group for given prompt. This objective is formalized as:
LGRPO Ne: n -xx DO:1F Mo (O:ja) [min(W; clip(W,, 1 + t,, c) A:)], (10)
x= (P,I,C), W; = xx(0,1x)
Raa (O]x) A 7R.-meant R
(11)
(R.
where is the input prompt, { is the roll-out group with the size G. W; denotes the policy ratio comparing how the new policy Ty prefer generation against the old policy Teaa To prevent drastic W; updates, we clip the ratio within the interval [1 [1-66 e, + c]. Finally, A, is computed on the reward score of each roll-out R2, to show the relative advantage in the same roll-out group.
5 Experiment Setup
Dataset Recipe Venus Python subset contains 2, 181 algorithmic problems, each accompanied by validated test case generator and an average of 106.6 human solutions, enabling robust empirical analysis of code efficiency beyond functional correciness. For each L.M--generteed test case input, follow the paradigm of Mercury [16), where we execute them through all collected solutions from LeetCode, and only keep those cases having consistent outputs over all correct solutions [30]. Based Venus, Section 4 introduces several datasets for Afterburner training, including DSsFT, DSpPo. DSCOLD, and DSGRPo- APPS is a widely recognized benchmark for evaluating the functional correctness of code generation models (22. While its original design, with 21.2 test cases and 23.4 solutions per problem, focuses on correctness, we integrate into our efficiency evaluation pipeline as auxiliary benchmark (see Appendix B).
Functional Correctness Ensuring functional correctness is prerequisite for code genera- tion models. Following the evaluation paradigm in Codex [9) we employ the PASS@ N Npassed/ / score to assess the global func- tional correctness, where N Npassed is the number of passed generations and Niotal is the total num- ber of test tasks.
Memory Usage (on)]
m2(() df
Integrat (3)
Running Time (FL)
PR(x, D)
| EoED 1 2 x]. (12) Figure 3: Illustration of task-level efficiency metrics.
PR (D. M...D.D... (.( Vieael
BEYOND- (T, M, 1)
(13)

---

Computational Efficiency Following Mercury [16] and EffiBench [26], we avoid employing absolute efficiency metrics because they are highly sensitive to hardware configurations and operating systems. For each task in Venus test set U): Viest, we instead compute percentile ranks of an absolute performance E relative to the distribution DE collected from corresponding reference solutions St- Except the execution time (rx")) and peak memory (max(m sen (t))), we also consider using the integral
gen score k - mig (t) df as comprehensive efficiency metric, where m (t) is the instantancous memory footprint time To compute relative efficiency metries, we establish reference distributions 1
of execution time overhead D = (r [Sa] memory overhead D = {m |Saa]
n=1 )
and integral
efficiency D {&g /n=1- where r.. m.g and gg are the absolute execution time, memory usage, and integral score of the n-th collected solution & Sk, respectively. Based on these distributions, we can calculate the task-level efficiency percentile-rank of the generated solution in Eq. (12). The global efficiency metrics are computed as the average of all task-level percentile-ranks in Eq. (13). Higher scores indicate that the generated code outperforms a larger fraction of the reference solutions, reflecting stronger code efficiency.
Implementation Details Afterburner models trained on single node with eight H100 GPUs.
We utilized Alama-Factory [75] for SFT and DPO training phases, and Verl (58) for GRPO training.
Dataset construction details can be found in Section For inference acceleration, we use vLLM [39].
Comprehensive details regarding the training pipeline (as shown in Figure 4 and hyperparameters
are provided in the Appendix D Monolith configuration can be found in Appendx H
AfterburneIsFF
DNoro
Afterburner
Base Model
AfterburneICSs
Afterburner
DBaaro
Figure Illustration of the training pipeline of Afterburner models.
Discussion and Key Takeaways
6.1 How about the Code Efficiency Performance of Vanilla LLMs?
Our baseline evaluation of diverse LLMs on the Venus and APPS benchmarks (Tables) I and reveals critical performance limitation: Despite achieving high functional correctness (PASS @ 1), vanilla models generate code with strikingly inferior computational efficiency compared to human For example, OpenAl 04 mini, top-performing model with 89.1 1% PASS @ on Venus, produces code whose runtime efficiency (BEYOND- T) surpasses only 56.85% human solutions (and merely 40.07% on APPS), with similar disparities observed for other leading models and across all efficiency metrics. While stronger (bigger) models exhibit marginally better code efficiency, this is insufficient to overcome the fundamental gap. This pervasive efficiency deficit in LLM--genrrted code clearly motivates the development of dedicated optimization frameworks, such as Af Afterburner, enhance code generation in real-world applications.
solutions 44 (55)
6.2 Does Iterative Improvement Framework Work?
The foundational hypothesis of the Afterburner framework is that iterative refinement, driven by execution feedback, can progressively enhance code efficiency. This section investigates the effectiveness of such iterative self-optimizatiinn and how the choice of underlying optimization strategy impacts learning dynamics and outcomes across successive iterations. Notably, the prompt placeholder original_code is left empty for the initial code generation (see Section F..
SFT Memorized Superficial Patterns. SFT primarily learns to mimic transformations from less more efficient code based on its training data. In the model training phase, Af Afterburner SFT updates these leamed patterns. Initial gains are possible if the input code matches known suboptimal patterns. However, SFT's capacity to generalize to novel inefficiencies or explore fundamentally different algorithmic solutions is inherently limited, as it lacks deep understanding of why pattern is efficient beyond its training data co-oceurrence. Consequently, as seen Figure SFT often quickly exhausts its applicable patterns in iterative optimization.
7

---

Table 1: Comparison of Vanilla Efficiency Performance between Open- Open-Source and Closed-Source Models on the Venus Benchmark. Parentheses denote 95% CI. The top score for each metric is highlighted in bold. Afterburner uses 'both time and memory efficient" instruction in the generation.
Model Name
PASS@1 1
BEYOND-T'S
BEYOND-M f
BEYOND-1 1
Open-source Models
Owen 2.5 3B
27.99
12.40 (12.35, 12:45)
13.24 (13.21. 13.28)
10.29 (10.24, 10.34)
Owen 2.5 Coder 7B
52.21
20.66 20.71)
25.21 (25.16, 25.26)
16.78 (16.74, 16.83)
Qwen 2.5 7B instruct
60.78
27.67 27.61,27.73
29.79 (29.73, 29.85)
21.02 (20.98, 21.07)
Llama 4 Scout DeepSeck V3
62.82
33,10 33.03,33.100
38.22 (38. 38.26)
26.91 (26.86, 26.95)
86.33
48.66 (48.57.48.75
51.20 (51.15,
39.20 (39.13, 39.26)
QwQ 32B +
83.09
51.09 (51.03, 51.16)
45.22 (45.16, 45.27)
41.66 (4161, 41.70)
Closed-soure Models
OpenAl 4o
82.26
38. 38.22 (38.15, 38.29)
42,09 (42.04, 42.15)
28. 89.89 (28. 84, 28.95)
Claude 3.5 Haiku
66.45
38.82 (38.75, 38.89)
37.77 37.71,37.882
30.15 15 (30.10, 30.20)
Claude 3.7 Sonnel
86.52
52.19 (52. 10,,10,55 52.27)
49.86 (49.81, 49.92)
40.49 (40.43, 40.55)
OpenAl 04 mini
89.11
56.85
(6.97))56)33
53.41 (53.35, 53,46)
45.71 (45.66, 45.77)
Our Afterburner Tuned on Qwen 2.5 3B at Iteration
AfterburnersFT 48.67
26.78 (26.72, 26,91)
25.30 (25.25, 25,41)
22.50 (22.41, 22.67) 38.95 (38.89, 39.17)
AfterburneIGRPO 61.67
45.17 (45.08, 45,30)
48.05 (47.96, 48.26)
DPO Realized Static Preferences. DPO internalizes preferences for more efficient solutions from ranked pairs. This allows Afterburner DOO to make more nuanced judgments than SFT, guided by characteristics correlated with better performance under the objective I. Iteratively, DPO can steer code towards these preferred traits. However, since DPO is typically an offline method, does not learn from its own generations without retraining. Thus, its exploration is still bounded by the diversity of its initial preference dataset. improvement than SFT, but also tends to plateau once its learned preferences are fully exploited.
shows DPO may offer more consistent
GRPO Cultivated Adaptive Proficiency. GRPO utilizes an online reinforcement learning ap- proach. In the training phase, Afterburnercrro generates multiple candidates, which are evaluated by Monolith. The resultant empirical feedback directly updates the policy no to fa- vor strategies yielding more efficient code for objective I. This online learning is pivotal for iterative self-improving optimization. Rather than merely static patterns or preferences, GRPO develops a deeper proficiency in code optimization. By actively exploring the solution space and receiving direct feedback, AfterburneIGRPO continuously refines its generation strategy, adapts to groblem-specffic nuances, and uncovers sophisticated optimization policy over iterations. The group-wise ranking further enhances its fine-grained understanding of relative efficiencies. This adaptive capability, evident in Figure 5 allows GRPO to achieve sustained and superior performance improvements, continually pushing its optimization boundaries.
Model Generalization. To verify whether our models can generalize out-of-distribution questions, we evaluated Afterburner on the APPS [22]. As illustrated in Appendix Figure g Af terburnr demonstrates a similar pattern of performance improvement, confirming its effec- tiveness on problems with distinct data distribution.
6.3 Why GRPO Can Iteratively Enhance Code Efficiency?
Generation diversity foundational to its iterative capability, By unleashing the KL divergence restriction in the training phase, AfterburnergRee inherently explores multiple potential opti- mization pathways without the ground-truth. This diversity ensures that AfterburneIGHrO is not confined to local optima. Moreover, GRPO gains experience improving code from what it gener- ated through the iterative refinement loop. It does not just generate code, but executes to gather concrete feedback on its real-world performance, effectively learning from its successes and failures continuous cycle. As the model identifies more efficient code structures training, becomes progressively better at producing them in inference. Ablation studies (Table confirm that remov- ing the feedback mechanism or original code context significantly diminishes AfterburnerGRro performance, an effect not always as evident in Af AfterburnerSFT AfterburneI pPO.
8

---

Pa Rerations
Iterations
PAN 186)
Beyondd 1960
7905
1210
m1
2Mth
XBC
2016
1828
1826
20
11315
120mh
1300
1920
1280
11
10
Beyond M M..s. Iterations
Beyand Iv.s. Iterationn
Doyond (A
55
18
1000
1110
1321%
Beyond
700m
21.75
11.00
10200
11
10
10
10
Base Model
Afterburner SFT
Afterburner DPO
Afterbumer GRPO
Figure 5: Iterative Optimization with an Efficient Instruction 'both time and memory efficient'.
Table 2: Performance of Afterburner models at Iteration 4 with removing execution feedback and original code input, respectively. Bracketed values represent the change in performance compared to the baseline: red indicates degradation, and green indicates improvement.
Model/Method
PASS@1
BEYOND-T
BEYOND-M
BEYOND-1
Afterburner-SFT
48.33
26.61
24.39
22.25
Remove Feedback
46.33 (-2.00)
25.41 (-1.20)
24.70 (+0.31)
21.43 (-0.82)
Remove Original Code
45.33 (-3.00)
25.64 (-0.97)
26.17 (+1.78)
20.08 (-2.17)
Afterburner-DPO
51.67
28.45
28.03
27.89
Remove Feedback
50.33 (-1.34)
27.33 (-1.12)
26.73 (-1.30)
25.68 (-2.21)
Remove Original Code
47.33 (-4.34)
25.32 (-3.13)
24.17 (-3.86)
22.01 (-5.88)
Afterburner-GRPP
57.00
40.81
40.68
33.51
Remove Feedback
52.51 (-4.49)
34.15 (-6.66)
34.49 (-6.19)
29.87 (-3.64)
Remove Original Code
54.17 (-2.83)
32.17 (-8.64)
33.25 (-7.43)
24.24 (-9.27)
6.4 Can Afterburner Generate Code Surpassing Human Efficiency?
While LLMs excel generating functionally correct code, often by imitating human- examples in their training data, a key question remains: Can they produce solutions exceeding the code efficiency of this best human-written code? To investigate this, compare the efficiency of model-generated code against human solutions from Venus. As presented in Table 3 reasoning models such as QwQ 32B and OpenAl 04-mini exhibit higher ability to occasionally generate super- human solutions. Crucially, our proposed Af teteburner yields the highest B% scores across all evaluated metrics after iterations: TIME (8. (8.00%). MEMORY (7.00%), and INTEGRAL (5.33%). This demonstrates that Af AfterburnerGRPO moves beyond merely replicating common patterns observed during pre-training. By actively exploring the solution space through RL, it discovers highly optimized implementations that are often structurally different from canonical human approaches. However, this enhanced exploration entails trade-off: Afterburnercnro also generates larger fraction of solutions that are less efficient than the human baseline.
7 Conclusion
We introduced an iterative optimization framework designed to enhance the computational efficiency of LLM- --geerrrtde code. Central this framework are the Afterburner models, which are critically guided by real-time efficiency feedback from the Monolith sandbox. Our comparative analysis of distinct optimization strategies revealed that SFT primarily learned superficial code optimization patterns, while DPO internalized efficiency preferences. In stark contrast, by leveraging online RL with direct execution feedback, GRPO achieved superior and sustained improvements in code efficiency across multiple iterations.

---

Table 3: Model vs. Human on Venus. Bold indicates the top performance per column and model category. B%, M%, W,, and F% denote percentages of solutions: Better than all human, Within mediocre human range, Worse than all human, or Failed to pass all test cases, respectively.
Time
Memory
Integral
Model Name
B%
M%
W%
F%
B%
M%
W%
F%
B%
M%
W%
F%
Qwen 2.5 3B
0.67
27.00
0.33
72.00
0.33
27.33
0.33
72.00
0.67
26.67
0.67
72.00
Qwen 2.5 Coder 7B
1.33
50.67
0.33
47.67
0.67
30.67
1.00
47.67
1.33
50.67
0.33
47.67
Qwen 2.5 7B Instruct
1.67
58.33
0.67
39.33
1.00
58.33
1.33
39.33
1.33
58.00
1.67
39.33
Llama Scout Instruct
3.00
59.33
37.33
60.67
0.33
37.33
1.67
60.67
Deepseek
5.33
80.67
0.67
13.67
3.33
82.67
0.33
13.67
3.00
81.67
13.67
QwQ 32B
6.67
76.00
0.33
17.00
2.33
79.67
1.00
17.00
3.33
79.00
1.00
17.00
GPT-4o
2.33
79.00
17.67
79.00
1.67
17.67
1.33
79.67
Claude 3.5 Haiku
4.67
61.67
0.33
33,67
64.00
0.33
33.67
2.67
63.33
0.67
33,67
Claude 3.7 Sonnet
5.67
80.67
0.33
13.33
2.67
83.33
0.33
13.33
3.33
82,00
1,00
13.33
O4-mini
7.00
82,00
0.00
11.00
3.33
85.33
0.67
11.00
4,00
84.33
0.67
11.00
Afterbureercsro
8.00
46.33
7.33
38.33
7.00
44.33
10.33
38.33
5.33
46.00
10,00
38.33
8
Limitations
While Af Aferburr demonstrates effective efficiency optimization for competition-level program- ming tasks, its extension larger, real-world software engineering projects warrants further inves- tigation. These projects often entail greater complexity in their code context, diverse efficiency criteria beyond algorithmic performance (e.g., library interactions or I/O operations), and may require sophisticated strategies for task decomposition, which are outside the scope of the current work.
Moreover, iterative optimization framework inherently requires more inference time during the code generation phase compared to single-pasy methods. We argue that this upfront investment in optimization be offset by significant cumulative runtime savings when the highly efficient code deployed in production, especially for frequently executed or performance-critica modules. Nonetheless, this trade-off between the optimization cost and long-term execution benefits needs to be carefully evaluated based on specific application requirements and deployment scenarios.
Acknowledgment
This research is supported by DSO grant DSOCL223216. This research also supported by ASTARR CISCO Systems (USA) Pte. Ltd and National University of Singapore under its Cisco-NUS Acceler- ated Digital Economy Corporate Laboratory (Award 121001
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tlge Akkaya, Florencia Leoni Aleman, Diego Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08770 037777774 2023.
[2] Loubna Ben Allal, Raymond Li, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Car- los Munoz Ferrandis, Niklas Muennighoff, Mayank Mishra, Alex Gu, Manan Dey, et al. Santacoder: don't t reach for the stars! arXiv preprint woxiv:2301.039000 2023.
[3] Saswat Anand, Edmund K Burke, Tsong Yueh Chen, John Clark, Myra B Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, Phil McMinn, Antonia Bertolino, al. An orchestrated survey of methodologies for automated software test case generation. Journal of systems and software, (948))1978-20011 2013.
[4] Anthropic. Introducing claude 3.5 sonnet, 6 2024.
[5] Anthropic. Claude 3.7 sonnet and claude code, 2025.
[6] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry Quoc Le, et al. Program synthesis with large language models. arXiv preprint 07X1v:2108.07732, 2021.
[7] Sahil Chaudhary. Code alpaca: An instruction llama model for code generation. https://githe: cww/ waahhh288111/coeeeaeeeee
2023.
[8] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, llya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained code. ICSE, 2021.
[9] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03377 2021.
[10] Simin Chen, Pranav Pusarla, and Baishakhi Ray. Dynamic benchmarking of reasoning capabili- ties in code large language models under data contamination. arXiv preprint arXiv:2503.044 149, 2025.
[11] Ian Chivers, Jane Sleightholme, Ian Chivers, and Jane Sleightholme. An introduction to algorithms and the big o notation. Introduction to Programming with Fortran: With Coverage of Fortran 90, 95, 2003, 2008 and 77, pages 359-364, 2015.
[12] Jianbo Dai, Jianqiao Lu, Yunlong Feng, Dong Huang, Guangtao Zeng, Rongju Ruan, Ming Cheng, Haochen Tan, and Zhijiang Guo. Mhpp: Exploring the capabilities and limitations of language models beyond basic code generation, 2024.
[13] Inc Docker et Docker. linea]. de 2017]. Disponible en: https://www. docker. com/what- docker, 2020.
[14] Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, et al. Stepeoder: Improve code generation with reinforcement learning from compiler feedback. arXiv preprint 23X 2024.
[15] Mingzhe Du, Anh Tuan Luu, Bin Ji, and See- Kiong Ng. From static to dynamic: knowledge metabolism for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 23784-23788 2024.

---

[16] Mingzhe Du, Luu Anh Tuan, Bin Ji, Qian Liu, and See-Kiong Ng. Mereury: A code efficiency benchmark for code large language models. Advances in Neural Information Processing Systems, 37, 2024.
[17] Bradley Efron and Robert Tibshirani. An introduction to the bootstrup. Chapman and Hall/CRC, 1994.
[18] Jonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella, Quentin Carbonneaux, Taco Cohen, and Gabriel Synnaeve. Rlef: Grounding code IIms in execution feedback with reinforcement learning. arXiv preprint arXiv:2410.02089, 2024.
[19] Ruihan Gong, Yue Liu, Wenjie Qu, Mingzhe Du, Yufei He, Yingwei Ma, Yulin Chen, Xiang Liu, Yi Wen, Xinfeng Li, et al. Efficient reasoning via chain of unconscious thought. arXiv preprint arXiv: 19756, 2025.
[20] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Iims via reinforcement learning. arXiv preprint arXiv:2501. 2025.
[21] Xinyi He, Qian Liu, Mingzhe Du, Lin Yan, Zhijie Fan, Yiming Huang, Zejian Yuan, and Zejun Ma. Swe-perf: Can language models optimize code performance on real-world repositories? arXiv preprint arXiv:2507. 12..25507..215,,, 2025.
[22] Dan Hendrycks, Steven Basart, Sauray Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Measuring coding challenge competence with apps. NeurlPS, 2021.
[23] Wenhao Hu, Jinhao Duan, Chunchen Wei, Li Zhang, Yue Zhang, and Kaidi Xu. Dynacode: dynamic complexity code benchmark for evaluating large language models in code generation. arXiv preprint 250t::25003.04552 2025.
[24] Dong Huang, Qingwen Bu, Yuhao Qing, and Heming Cui. Codecot: Tackling code syntax errors in cot reasoning for code generation, 2024.
[25] Dong Huang, Jianbo Dai, Han Weng, Puzhen Wu, Yuhao Qing, Heming Cui, Zhijiang Guo, and Jie Zhang, Effilearner: Enhancing efficiency of generated code via self-optimization, Advances in Neural Information Processing Systems, 37:84482-84522, 2024.
[26] Dong Huang, Yuhao Qing, Weiyi Shang, Heming Cui, and Jie Zhang. Effibench: Benchmarking the efficiency of automatically generated code. Advances in Neural Information Processing Systems, 37:1 177:1506-111 11544, 2024.
[27] Dong Huang, Guangtao Zeng, Jianbo Dai. Meng Luo, Han Weng, Yuhao Qing, Heming Cui, Zhijiang Guo, and Jie M Zhang. Effi-code: Unleashing code efficiency in language models. arXiv preprint arXiv:2410. 10209999,,00 2024.
[28] Dong Huang. Jie M Zhang, Qingwen Bu, Xiaofei Xie, Junjie Chen, and Heming Cui. Bias testing and mitigation in I1m- basddedd code generation. ACM Transactions on Software Engineering and Methodology, 2024.
[29] Dong Huang, Jie M Zhang, Mingzhe Du, Mark Harman, and Heming Cui. Rethinking the influence of source code on test case generation. arXiv preprint arXiv:2409. 0994099444 2024.
[30] Dong Huang, Jie M Zhang, Mark Harman, Mingzhe Du, and Heming Cui. Measuring the influence of incorrect code on test generation. arXiv preprint 2Xii:24409009644 2024.
[31] Dong Huang, Jie M Zhang, Mark Harman, Qianru Zhang, Mingzhe Du, and See- bee-Kiong Ng. Benchmarking llms for unit test generation from real-world functions. arXiv preprint 2:Xi:2558.000008 2025.
[32] Dong Huang, Jie M. Zhang, Michael Luck, Qingwen Bu, Yuhao Qing, and Heming Cui. Agentcoder: Multi-agent- based code generation with iterative testing and optimisation, 2024.
12

---

[33] Siming Huang, Tianhao Cheng, J. K. Liu, Jiaran Hao, Liuyihan Song. Yang Xu, J. Yang, J. H. Liu. Chenchen Zhang. Linzheng Chai, Ruifeng Yuan, Zhaoxiang Zhang, Jie Fu, Qian Liu, Ge Zhang, Zili Wang, Yuan Qi, Yinghui Xu, and Wei Chu. Opencoder: The open cookbook for top-tier code large language models. CoRR, abs/2411 1.04905, 2024.
[34] Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jia- jun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprini arXiv:2409. 1266,,22206666 2024.
[35] Md. Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapcoder: Multi-agent code generation for competitive problem solving, 2024.
[36] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint urXiv:2403.0773 2024.
[37] Bin Ji, Huijun Liu, Mingzhe Du, and See-Kiong Ng. Chain-of-thougee improves text generation with citations in large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18345-18353, 2024.
[38] Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint arXiv:2406.0055 2024.
[39] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[40] Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 55:21314 1111412-2222 21328, 2022.
[41] Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode. Science 378(6624): 1099- 2022.
[42] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint 194377 2024.
[43] Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[44] Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang Evaluating language models for efficient code generation. arXiv preprint arXiv:2408.064 2024.
[45] Yue Liu, Thanh Le- Le-Cong, Ratnadira Widyasari, Chakkrit Tanttthamthhvorr, Li Li, Xuan-Bach D) Le, and David Lo. Refining chatgpt-generater code: Characterizing and mitigating code quality issues. ACM Transactions Software Engineering and Methodology, 33(5):1-26, 2024.
[46] Yue Liu, Shengfang Zhai, Mingzhe Du, Yulin Chen, Tri Cao, Hongcheng Gao, Cheng Wang, Xinfeng Li, Kun Wang, Junfeng Fang, et al. Guardreasoner-vl: Safeguarding vims via reinforced reasoning. arXiv preprint arXiv:2505.1 /049, 2025.
[47] Anton Lozhkov Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Noua- mane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, al. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402. /9/73, 2024.
[48] Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306 08568, 2023.
13

---

[49] AI Meta. The Ilama 4 herd: The beginning of a new era of natively multimodal ai innovation. https://ai. meta. www.biloollama---mmiiioooooo -im-uiiiooaa---elllinneee///// checked on, 4(7):2025, 2025.
[50] Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, and Shayne Longpre. Octopack: Instruction tuning code large language models. arXiv preprint wrXiv2308.07722. 2023.
[51] Erik Nijkamp, Bo Pang. Hiroaki Hayashi, Lifu Tu, Huan Wang. Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203. 13474, 2022.
[52] OpenAl. Introducing openai o3 and o4-mini, 4 2025.
[53] Yuhao Qing, Boyu Zhu, Mingzhe Du, Zhijiang Guo, Terry Yue Zhuo, Qianru Zhang, Jie M Zhang, Heming Cui, Siu-Ming Yiu, Dong Huang, et al. Effibench-x: A multi-language benchmark for measuring efficiency of llm-generann to--generatedd code. arXiv preprint 2rXiv:2505.1300 13004, 2025.
[54] Yuhao Qing, Boyu Zhu, Mingzhe Du, Zhijiang Guo, Terry Yue Zhuo, Qianru Zhang, Jie M. Zhang. Heming Cui, Siu-Ming Yiu, Dong Huang, See-Kiong Ng, and Luu Anh Tuan. Effibench- A multi-language benchmark for measuring efficiency of llm-generated code, 2025.
[55] Ruizhong Qiu, Weiliang Will Zeng, James Ezick, Christopher Lott, and Hanghang Tong. How efficient is Alnegenertted code? a rigorous & high-standard benchmark. arXiv preprint arXiv:2406.066477 2024.
[56] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-5311 2023.
[57] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.033300 2024.
[58] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang. Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: flexible and efficient rihf framework. arXiv preprint arXiv: 19256, 2024.
[59] Jieke Shi, Zhou Yang, and David Lo. Efficient and green large language models for software engineering: Literature review, vision, and the road ahead, arXiv preprint arXiv:2404.04566 2024.
[60] Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob Gardner, Milad Hashemi, Graham Neubig. Parthasarathy Ranganathan, Osbert Bastani, and Amir Yazdanbakhsh. Learning performance-improvee code edits. arXiv preprint arXiv:2302.07867, 2023.
[61] Ali TehraniJamsaz, Arijit Bhattacharjee, Le Chen, Nesreen K Ahmed, Amir Yazdanbakhsh, and Ali Jannesari. Coderosetta: Pushing the boundaries of unsupervised code translation for parallel programming. arXiv preprint arXiv:2410.20527. 2024.
[62] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1 13771, 2023.
[63] Due Anh Vu, Cong-Dup Nguyen, Xiaobao Wu, Nhat Hoang, Mingzhe Du, Thong Nguyen, and Anh Tuan Luu. Curriculum demonstration selection for in-context learning. In Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing, pages 1004-1006, 2025.
[64] Siddhant Waghjale, Vishruth Veerendranath, Zora Zhiruo Wang. and Daniel Fried. Ecco: Can we improve model-generated code efficiency without sacrificing functional correctness? arXiv preprint arXiv:2407. 14044, 2024.
[65] Junqiao Wang, Zeng Zhang, Yangfan He, Yuyang Song, Tianyu Shi, Yuchen Li, Hengyuan Xu, Kunyu Wu, Guangwu Qian, Qiuwu Chen, et al. Enhancing code lmms with reinforcement learning in code generation. arXiv preprint arXiv:2412.20367. 2024.
14

---

[66] Jason Wei, Maarten Bosma, Vincent Y Zhao. Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021.
[67] Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Empow- ering code generation with SSS-instruct In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 52632-52657. PMLR, 21-27 Jul 2024.
[68] Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprini wwXi::2408.00724. 2024.
[69] Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models follow complex instructions. 2023.
arXiv preprint
[70] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.1 15115, 2024.
[71] Tong Ye, Weigang Huang, Xuhong Zhang, Tengfei Ma, Peiyu Liu, Jianwei Yin, and Wenhai Wang. Llm4effi: Leveraging large language models to enhance code efficiency and correctness. arXiv preprint arXiv:2502. 11..22222.88480 18489, 2025.
[72] Kechi Zhang, Ge Li, Jia Li, Yihong Dong, and Zhi Jin. Focused-dpo: Enhancing code generation through focused preference optimization on error-prone points. arXiv preprint arXiv:2502.11475, 2025.
[73] Quanjun Zhang, Chunrong Fang, Yang Xie, YuXiang Ma, Weisong Sun, Yun Yang, and Zhenyu Chen. A systematic literature review on large language models for automated program repair. arXiv preprint arXiv:2405.n 0466, 2024.
[74] Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong. Shan Wang. Yufei Xue, Lei Shen, Zihan Wang, Andi Wang. Yang Li, et al. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 5673-5684, 2023.
[75] Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhanghhi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguisties.
[76] Li Zhong, Zilong Wang, and Jingho Shang. Debug like human: large language model debugger via verifying runtime execution step-by-step. 2024.
[77] Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. Bigcodebench: Bench- marking code generation with diverse function calls and complex instructions. arXiv preprint a406 15877, 2024.
[78] Edward E Zukoski and GC Oates. Afterburners, Aerothermodynamics of aircraft engine components, 1985.
15

---

A
Model Details
Model Name
Model Size
URI.
kttps: http:///bgggggggggeeee wttpse/co//vee//eee22222 5-38
Quen 2.5 Coder
X0
7B
ELtpe 77 //ugg mnd Boe Uveei Lven2 S-Coder- 7oeeeer------ Iestruct
Quen 2B
7B
Llama 4 Scout 17B 16E Instrnct
17B
Bctps /tt////gggggngc//c norneta- Lama/ Lan///Lamm Jaan 4--ccot- 1eeoatt-78---00 Instruct
QwQ 32B 70
32B
kttps
GPT-4o 01
Unknown
RtLps //s/////11111 opents C98/ doce/ cocce//odellae nodela/got-40
Claude 3.5 Hoiku
Unknown
httpa /7982 Antil
ha
Unknown
Rttps://99t anthropic.................. con.iccccc...................... clap///aaadddeeeee
DeepSeck V3 +
Unknown
BC http:::///ww..epppe. despaeek con7
04-mini 52
Unknown
httpa://plattorm.opmm wttt://ccc/ccccccc/cocooocc http://afaaa.pppooccccc/////////
Table 4: Model list with their size, reasoning ability, and model URL. denotes reasoning model.
B Dataset Curation and Statisties
Table 5: Overall Statistics of Representative Function-level Code Generation Benchmark Datasets. indicates the datasets designed for functional correctness solely, while ? indicates the datasets designed for code efficiency. We list the average number of solutions per problem for each dataset. The detailed definition of each metric can be found in Section indicates that there some that extend the original datasets to more diverse programming languages.
works 74
Datasel
Tasks
Test Cases
Solutions
Metries
Languages
Source
HumanEval
8
164
8.1
1.0
Pass@k
Python
Crowdsource
MBPP 6
257
3.0
1.0
Pass@k
Python
Crowdsource
22
10,000
21.2
23.4
Pass@k
Python
CodeForces
BigCodeBench
77
1,140
5.6
1.0
Pass@lk
Python
Synthesis
EFF Effnch
26
1000
100
14.6
NET/ NMU
Python
LectCode
Mercury [66
1,889
+-00
18.4
Pass/ Beyond
Python
LeetCode
ENAMEL
142
20
Effösk
Python
HumanEval
EVALPERF
44
1,474
DPS
Python
6 #3
PIE 600
1,889
104
80.6
%0pt &cCorret/ Speedup
CPP
CodeNet
ECCO 64
48
20
16.5
Time/Memory
python
CodeNet
Venua (ours)
8,598
+000
79.3
Pass/ Time! Memory/ Integral
Multilingual
LectCode
B.1 Venus Dataset
We constructed the Venus benchmark through multi-stage filtering pipeline, illustrated in Figure 6 Beginning with 3,535 problems from LeetCode we first removed paid-only questions to adhere to fair-use principles, retaining 2,821 freely accessible problems. We then filtered for algorithmic problems, discarding other categories such as Database or Shell. To ensure reliable efficiency distribution in the evaluation, runtime. Problems with fewer than 16 solutions passing all test cases were further excluded. This resulted in curated set of 1,284 high-quality problems. Finally, split the dataset into training set of 984 problems and a held-out test set of 300 problems, forming the complete Venus dataset.
executed all available solutions using the Monolith
Multilingual Scope. Transcending the prevalent Python focus of prior benchmarks g 16 21), Venus offers robust support for a multilingual scope as listed in Table 7 Since the test case generator is rooted standard I/O-based case interaction (3I), Venus can further support multilingual code generation benchmarking and training.
26
Language-Agnostic Test Cases A significant challenge in benchmarking code generation models, especially for efficiency, is the availability extensive and diverse test cases. Most online judge platforms do not disclose their test suites, and existing benchmarks often provide a limited number of test cases (Table (), which may be insufficient for robust efficiency profiling. To address this issue, we propose novel approach to generate a large-scale, language-agnostic test case dataset. The process involves:
16

---

Table 6: Definitions of the fields within Venus datasets.
Column Name
Description
problem_id
Unique identifier for each problem (int64)
title
Title of the problem (string)
question_comie content
Full text of the problem statement (string)
difficulty
Difficulty level (categorical)
tags
List of associated tags (sequence)
code_prompt
Prompt used for solution generation (string)
test_case__enerattn
Code generating test cases (string)
Code evaluating test case outputs (string)
cest_case_runners
Code executing solutions with test cases (string)
solutions
Human- suumat-sbmmitted solutions from LeetCode (list of strings)
Paid Only 714
Database 303
LeetCode 3535
Free 2821
Insufficient Pass Solutions 1199
Algorithm 2483
Text Ser 500
Sufficient Solutions 1284
Train Set 984
Figure 6: Pipeline for constructing the Venus dataset. We start from 3,535 LeetCode problems and apply series of quality-contre and de-duplication filters, retaining 1,284 high-quality problems in the Venus benchmark.
Automated Test Case Generation: For each problem Venus, a dedicated test case generator program is synthesized by GPT-40 based on the given problem description. These generators are designed to produce a virtually unbounded stream of diverse and valid inputs.
The validity of each generated test case is paramount. Before being used for evaluation, candidate test case is run against all collected canonical human solutions for that problem. Only test cases for which all canonical solutions produce consistent outputs are accepted. This ensures that the test cases are unambiguous and accurately reflect the problem's 's requirements as understood by proficient human programmers.
Rigorous Validation:
Standard I/O for Language Agnosticism: lies in its interaction protocol with the code being tested. All solutions, irrespective of their programming language, interface with the test harness exclusively via standard input (stdin) and standard output (sidout). Test inputs are provided as text streams via stdin, and the solution's 's output is captured from stdout. This based I/O mechanism decouples the test data from the specifics of any programming language.
The key to the multilingual capability of Monoli i
This design allows the same set of validated test cases to be used for evaluating solutions written in any of the languages supported by Venus (Python, C++, Go, Java, JavaScript, etc., shown in (7). This aegguage-agnostii approach not only broadens the applicability of our framework but also simplifies its extension new programming languages in the future, new test case generators are not required for each language. The common testbed ensures fair and consistent efficiency comparisons across different languages and models.
Venus Justification. Multilingual Coverage. A primary contribution is that Venus is the first multilingual code efficiency benchmark, covering six languages. Prior work, including Mereury [16] EffiBench (26). and EvalPerf [44), focuses exclusively on Python. EffiBench-X (53) is multilingual
17

---

Table 7: Breakdown of Venus dataset by programming language. For each language we list the total number of tasks and the average number of human submission per task.
Language
Python
C++
Go
Java
JavaScript
Total
Train Tasks
2,181
2,183
866
1,358
704
7,298 1,300
Test Tasks
300
300
200
300
200
Avg. Solutions
106.6
112.2
33.6
69.6
74.4
79.3
Problee 10
Titte
Difficulty
enllansieguctee sutput aes, progret loc unoe conerationn Jualtes seiis
292
nin-gane
easy
IF Tocaets
Labd's unds undty
Tags
"nsth", "braintesserr" "pare-theorr"
<tIf "trtsr'
2014.
clac
Code Prompt
rause Valuete oe tevtvatt of Easter striag 1o0tr
class Solution:
1r2
cankinkinn(sll nt int) >> bool:
copetial deseric Line aed toweeteeedddd.. perpet Eniores cesriaticn Eadl pnngringgeee.... 241p
PEore
juit case sinertor
CON't 1000
ligar: car
Urrl, Husee, Pligent leart
Juinn hoal' Chapet "Matrers loaeets Tour i,
(10017
"unie
rakity "rioes Seluriers real aettt
sharn frar, Coan Traete Soturt Siart, Faaurr meeas
panu "aaaurr
setecter
Liuut atr 1014 will
sasen learl Sutert Huare,, trar thnd
wted's moatann baterts
eetoe Te
wheth Couels "eeerrrr Saberty fooe
Figure 7: An Example in Venus Python Subset.
efficiency benchmark at the same time. Solutions Diversity. Venus offers a significantly larger and more diverse set of reference solutions. While EffiBench uses single baseline and Mercury averages 18.4 solutions per problem, Venus provides an average of 79.3 solutions per problem for each language. This massive increase directly contributes to the statistical diversity of performance metrics, as shown in the time and memory distributions in Appendix Figure 7 Evaluation Dimensions. As a direct extension of Mercury, which only measures execution time, Venus evaluates execution time, memory usage, and their integral, providing a more holistic assessment of code efficiency. To better illustrate the Venus dataset, we provide complete instance from Venus shown in Figure 7
B.2 APPS Dataset
APPS is a widely recognized benchmark for evaluating the functional correctness of code generation models (2). While its original design focuses on correctness, we integrate it into efficiency evaluation pipeline as an auxiliary benchmark. It consists of 10,00 10,000 Python programming problems, where each problem is accompanied by an average of 21.2 test cases and 23.4 solutions.
Table 8: Definitions of the fields within APPS datasets.
Column Name
Description
title
Title of the problem (string)
question_ content difficulty
Full text of the problem statement (string)
Difficulty level (categorical) Hlman--ubbiice solutions from LeetCode (list of strings)
solutions
test_cases
Test cases (list of strings)
18

---

OC1
Test Set 5000
CodeForces 10400
APPS 10000
Insufficient Solutions
2197
Train Set 5000
Sufficient Solutions 2803
300
Train Set
2503
Figure 8: Selection procedure for the APPS subset used in our benchmark. Beginning with the official APPS training split (5,000 problems), we discard problems that lack sufficient number of accepted reference solutions, yielding 2,803 problems in the final dataset.
Table 9: Comparison of Efficiency Performance between Open-Source and Closed-Source Models the APPS Benchmark. Parentheses denote 95% confidence intervals. The top score for each metric is highlighted in bold, while the leading score within each model category is underlined.
Model Name
Pass
Beyond-T
Beyond-M
Beyond-I
Open-source Models
Qwen 2.5
9.67
5.10 (5.06, 5.13)
5.48 (5.45, 5.51)
3.80 (3.78, 1.83)
Qwen 2.5 Coder 7B
16.00
9.30 (9.25, 9.344
8.79 (8:75, 8.83)
6.81 (6.78, 6.85)
Qwen 2.5 instruct
17.00
9.65 (9.59, 9.70)
9.33 (9:29, 9.37)
7.28 (7.24,
Llama 4 Scout
47.67
24.85 (24.78, 24.92)
27.96 (27.90, 28.03)
18.22 (18.17, 18.28)
QwQ 32B o
69.62
35.59 (35.51, 35.68)
37.84 (37.35, 37.93)
27.32 (27.26, 27.39)
Closed-source Models
GPT-4o
46.23
26.30 (26.24, 26.37)
26.47 (26.41, 26.53)
19.76 (19.71, 19882)
Claude 3.5 Haiku
36.67
13.63 (13.57, 13.69)
20.96 20.91,21.011
10.74 (10.70, 10.77)
Claude 3.7 Sonnet
45.63
21.68 (21.61, 21.75)
25.76 (25.70, 25.81)
15.75 (15.69, 1581))
DeepSeek V3
56.63
31.58 (31.49, 31.66)
31.42 (3135, 31.49)
23.20 (23.13, 23.27)
OpenAl 04 Mini
78.81
40.07 (39.98, 40.15)
41.00 (40.33 41.07)
28.76 (28.69, 28.82)
C Iterative Efficiency Optimization Procedure
The Iterative Efficiency Optimization Procedure, detailed in Algorithm is designed systematically enhance source code performance. Given a problem description P, an efficiency instruction (e.g.. targeting time memory), and set of test cases Teases, the algorithm iteratively refines code. begins with an initial code version C which generated by the Afterburner component not provided, and its initial performance M is evaluated by the Monolith component. Over Nater iterations, new code versions are proposed by Afterburner based on the current code and its metries, and then evaluated by Monolith. If newly generated version Cout exhibits improved performance Mout according to the criterion when compared to the current iteration's input performance -,,". it is adopted as the input for the subsequent iteration; otherwise, the previous code is retained. The procedure concludes by returning the best- pest-performing code Cin. Niter found after
Nater iterations, along with its corresponding performance metrics Min
D Model Training Details
D.1 Training Pipeline.
As shown in Figure we explore different optimization strategies to train the Afterburner models. Initially, Afterburner sFr models are trained using the DSSFT dataset. For Afterburner models, we initialize them from the checkpoints of the corresponding Af AfterburnerspT models and
19

---

Algorithm Iterative Efficiency Optimization Procedure
Input: Problem description P, Efficiency instruction I {time, memory, integral), Set of test cases T cases. Original code Co (optional), Number of iterations Nater
Output: Improved code Coue, Improved code performance Mo
if not Cir then Co <- Afterburner I,,,,,,,.,,, None, None)
Initial code generation.
end if
M -- Monolith(Co" Teases)
b Initial code evaluation.
for +- Niter do
Cout + Afterburner (P, I,,,", M
Code optimization.
Mo f- ( (Cout :Toases)
> Code evaluate.
if Mout MM then
Compare the performance concerning J. Update with the better performing candidate.
C+.1MMH < (MM
else
1 M s+1 < (Q
Otherwise, retain the current best.
end if
end for return (CN M
Nator)
Return the best code found after Niter iterations and its metrics
subsequently finetune them on the DSpPO dataset. The training process for AfterburnercHPo models involves two steps: first, base model is finetuned on the DSoold start dataset ensure adherence to the required response format; thereafter, these models are trained on DSGRPO.
D.2 Details of Afterburner SFT
Training. We fine-tune 5etmm//wwn225-3B-nnsitt using Low-Rank Adaptation (LoRA). The model trained for one epoch on DS Key hyperparameters include learning rate of 3e-5, managed by a cosine scheduler with 200 warm-up steps, an effective batch size of 64 (per-device batch size of 4 with 16 gradient accumulation steps), and the adamw torch optimizer. For LoRA, the rank 8 and alpha is 16. The training uses BF16 precision, and gradients are clipped at norm of 1.0.
D.3 Details of Afterburner DPO
Training. Af poo is trained from the checkpoint AfterburnersFT utilizing LoRA for one epoch of DSp dataset. Key hyperparameters include: Dearning_rate-ien with cosine scheduler and 300 warm-up steps, an effective batch size of 16 (per-device batch size of 2 with gradient accumulation steps), and the adamw optimizer. LoRA A parameters are set to rank 16, alpha 16. and dropout of 0.05. DPO-specific settings include beta of 0.1 and sigmoid loss function. with pref ftx (SFT loss component) set to O. The training uses BF16 precision, and gradients are clipped at norm of 1.0.
D.4 Details of Afterburner Code Start
Model Response Collection. We collect the model response from the system prompt shown in Section and the Afterburner inference prompt as shown in Section We only keep the responses that can pass the response regex filter as shown in Section D.5
using
Training. We conduct SFT base model @wen2.5 5-3e 3---B--Issttre using the LLaMA Factory frame- work [75]. The model undergoes full fine-tuning on an epoch of the DSCOLD dataset, with input sequences processed up maximum length of 32,768 tokens. Key hyperparameters included learning rate=5e-5, managed by a cosine scheduler with 50 warm-up steps, an effective batch size of 4, and the adamw_ 8bit optimizer.
D.5 Details of Afterburner GRPO
Training. Afterburnercrro is trained on Verl [58 and initialized from Afterburnercs: The GRPO training runs for 20 epochs DSGRPo. Since executing generated code and computing
20

---

its efficiency metrics are time-conssue consuminn,,, we use batch reward function to accelerate the re- ward calculation parallel manner. Key hyperparameters include: actor learning rate=le-6. Saarrningraat=ee-66. ppo_mini ppo_mini_butch___ size=32 (4 per-GPU micro-batch). During the roll-outs, 16 responses are generated per prompt using vLL.M [39 with inference remperature= I.0. KL. loss for actor updates disabled, and the entropy coefficient is O. For the reward weights, we set 3 = 0.2, 0.3, 9. = 0.5. Note that Refliciecyy set to if Cpass pass = Cupper is set to 90, 1048576, 94371840, respectively, which aligns with our timeout (90s) and memory (IGB) limitation. In our experiment, we observed that dominant weight for functional correctness (3, 2 0.5) is essential for stable training, as values below this threshold often led to training crash. To balance our objectives,
fixed Be 0.5 while gradually increasing the code efficiency weight Be from 0.3 to 0.5. This strategy ensures the model first learns to produce correct code before optimizing for efficiency.
Format Regex. Inspired by recent works
37
X 19,, we encourage our model to generate
the reasoning content before the code solution. The designated response format: "<thinking> thing_coe coaingncntenttt </thinking> <solution> colution_contert </solution>".
Afterburner Format Regex
import re def mangle_thinknnge wtw.nniinn.ouuttiooooooree (text: str) -> bool:
pattern = re. compile(
r"nn
\AS* # optional leading whitespace
<thinking>
(?:((]<thhinknn))))
</thinking>ls* end <thinking> <solution>
10
(?: ktinkttttngngg>noonnooao
11
</solution> end <solution> \z
12
re. DOTALL re. VERBOSE,
15
16
return bool (pattern. fullmatch (text))
Reward Function Design for Enhanced Code Generation The efficacy of our Group Relative Policy Optimization (GRPO) framework, particularly for a task as nuanced as code generation, heavily relies on well-designed reward function. Our objective is guide the AfterburnercRRo model not merely towards syntactically valid code, but towards solutions that are functionally correct, computationally efficient, and adhere desired structured output format that includes explicit reasoning phase. To this end, our final reward Rymal is carefully weighted composite of three distinct components, each targeting critical aspect of code quality.
Format Control FFormat) We first incentivize adherence to a predefined output structure, which mandates thinking phase encapsulated in <thinking>... ... </thinking tags followed by the code within <solution> </solution> tags. As defined in Eq. (6). R provides a strong binary signal (+1 for compliance, -1 otherwise). This not only ensures predictable and parsable outputs for automated assessment but also explicitly encourages the model to engage in a "thought process" prior generating the final solution, step believe is crucial for complex problem-solving.
Functional Correctness (Rwrreat) Ensuring functional soundness is paramount. However, a simple binary pass/fail reward for the current generation C' can be sparse and inefficient signal. Instead, Reorrect (Eq. evaluates C" in comparison to baseline attempt C.. assigns the highest positive reward (1.0) for an "upgrade" (i.e., C" passes tests while C' fails) and the largest penalty (-1.0) for "downgrade" (C" fails while C passes). Maintaining a passing or failing status yields intermediate rewards (0.5 and -0.5 respectively). This relative assessment provides a more nuanced gradient, strongly favoring improvements and robustly penalizing regressions.
Efficiency Improvement (Refficioncy. Beyond correctness, generating efficient code is our key objec- tive. Reficiency (Eq. (8) is designed to reward relative improvements computational performance (c.g., time, memory). The core of this reward is egain. which measures the normalized improvement
21

---

of the current solution C over baseline C after clipping efficiency metrics to sensible range [0, Cupeer] to handle outliers. Crucially, we apply the hyperbolic tangent function (tanh) égain. This bounds the reward component within (-1,1), 1), providing a smooth, scaled signal that is sensitive to gains but diminishes returns for extremely large improvements or degradations, thereby stabilizing the learning process. A small in the denominator of Egain ensures numerical stability. Furthermore, the inherent stochasticity often present in empirical efficiency measurements (e.g., due to minor system-level variations non-deterministic aspects of complex code execution) means that Refficiency naturally introduces a degree of noise. This moderate, implicit stochasticity can beneficial for GRPO, as it helps maintain variance in reward signals across roll-outs. This, in turn, can prevent the advantage term A, (Eq. (IT) from prematurely collapsing or vanishing, thereby fostering continued exploration and more robust policy updates.
Iterations
Time% Iterations
16
Fass03 (15)
2mm
Turee PSL.
127%
100
10
2
Base Model
Afterourner SFT
10
Afterburner DPO
Afterbumer GRPO
Figure 9: Iterative Optimization Performance on APPS.
E
Model Prompts
Baseline Prompt
You are an expert competitive programmer who excels at solving algorithm problems in multiple programming languages. Your task to implement (efficiency (efficcieny,instrciiion instruction solution the following problem (target.lang).
## Problem Description {question)
## Output Format Provide the complete solution code in **on markdovn code block** with appropriate language identifier Implement the function with the exact signature (name, parameters, etc.) specified in the starter code.
Afterburner System Prompt
conversation between User and Assistant. The User asks a question and provides an original solution, then
the Assistant improves it.
The assistant first thinks about the reasoning process in the mind and then provides the user with the improved solution. solution are enclosed within <thinking > </thinking> and <solution> </solution> tags, respectively. prrocees</thinkinn </thinking> <solution>
The reasoning process
For example,
improved o
22

---

Afterburner Prompt Template
## Instructions
Your task is to implement a solution to the folloving problem in (target_lang).
## Problem Description <problen_description
## Original Solution {original_solution)
10 ## Original Performance
Passed: loriginal.passed] / Time: foriginal_time) / Memory: { original.meoory / Integral: (original_intega)
12
13 ## Output Format
Provide the complete solution code in one markdown code block with appropriate language identifier.
15 Generate the initial solution code directly Original Solution is empty.
16 Fix the original solution if it was not passed. Optimize the { efficiency. efficiency. inscccciistttttoo performance if the original solution passed.
F
Would the History-Aware Model Perform Better?
We attribute the perfr-performance of history-aware model in Table T0 to two primary factors: 1) Distribution Shift: The performance drop is likely caused by distribution shift, as the training format (single-tur)) differs from the multi-turn, history-aware format used during inference. 2) Training Challenges: While fine-tuning the model with multi-turn format could mitigate the distribution shift, this approach introduces the significant challenge of long-term credit assignment, well-known difficulty in Reinforcement Learning 15
Table 10: Performance Single-ture tur Loop (Afterburner) and Multi-tun Loop (History-awarg
Iteration
7
10
Single-turn Loop
Pass %
47.33
50.33
52.00
58.17
60.50
61.67
Beyond-I
18.24
24.81
29.44
35.48
38.01
38.95
Multi-turn Loop (History-aware)
Pass %
45.00
46.33
46.33
46.33
46.67
46.67
Beyond-I %
15.10
15.10
15.10
15.10
15.10
15.10
The above results show that applying the history-aware method directly to a vanilla model yielded no performance improvement over the single-turn baseline. Despite instructions to avoid repetition, the model predominantly replicated existing solutions from the prompt. The few novel solutions it generated, while claiming to be superior, demonstrated empirical performance gains. This observation aligns with our core hypothesis: vanilla models can generate correct code but lack an intrinsic awareness of code efficiency.
G Uncertainty in Code Efficiency Measurement
Quantifying code efficiency is nuanced challenge. Theoretical efficiency, typically expressed via asymptotic notation (e.g., (nlogg)), offers high-level algorithmic understanding but often neglects constant factors, compiler optimizations, and hardware-specific impacts (like cache performance or instruction-level parallelism) crucial for real-world performance. Consequently, provides an incomplete picture for comparing concrete code implementations.
23

---

Table 11: A summary of different training and inference schemes.
Training/ Inference Single-Turn
Single- Turn
Multi-Ture
Afterburner
Training Waste
(P) Consistent distribution. (P) Stable
(C) wastes the model's trained abil-
RL training. (P) Avoids credit assign-
ity.
ment. (C) Cannot handle history.
Multi-Turn
Distribution Mismatch
Ideal History-Aware
(C) Critical mismatch between train-
(P) Creates a truly context-aware agent.
ing and inference data leads to poor performance.
(C) Suffers from sparse rewards and unstable training. (C) Difficult long-
distance credit assignment.
On the other end of the spectrum, simulation and low-level statistics (c.g.. cycle-accurate simulations, CPU performance counters) can provide extremely detailed data [60]. However, these methods often introduce substantial complexity in setup and interpretation, may have limited scope in accurately modeling all modern system intricacies, or can be overly specific to particular hardware configura- tion, making generalization difficult. For our purposes, the granularity and setup overhead of such approaches outweigh their benefits.
We therefore opt for empirical performance measurement, directly observing execution metrics like runtime and memory usage. This approach holistically captures the interplay of algorithm, code structure. compilation, and the underlying hardware. While direct, empirical results are subject inherent system noise and run-to-run variability. To rigorously address this and derive stable performance indicators, robust statistical techniques are indispensable, leading to our choice of bootstrapping for uncertainty quantification.
G.1 Details of Bootstrapping Evaluation
To quantify the statistical uncertainty of our efficiency metrics, we employ a bootstrapping proce- dure [17]. Task-level efficiency metrics are first grouped by their respective IDs to ensure independent sampling for each task. We generate B 128 bootstrap replicates. Each replicate is constructed by sampling
solutions for every unique problem (in our settings, we repeatedly evaluate each then calculate the Average BEYOND-TI
generated code 16 times). For each of these B replicates, BEYOND-M. and BEYOND-I. Finally, we report the mean of each these four metrics across all replicates, along with their corresponding 95% confidence intervals, to offer a robust evaluation of model performance.
H Monolith Implementation
Code Execution Environment. We deploy a code execution environment on a GCP n2-highepu-96 instance (96 vCPUs, 96 GB Memory) with 81 Monolith workers. Each worker operates within dedicated Docker container [13), which is allocated vCPU, GB of memory, and provided with an isolated temporary directory. To ensure a pristine execution environment for each evaluation, containers are created anew for every task. CPU affinity for each worker was set to 100% to minimize performance variability during measurements. Execution time and peak memory overhead were measured using the 'time -v' command. To gather instantaneous memory usage and calculate the integral score, we sampled the 'VmRSS' field from the process status file (/proc/pii//status) To accelerate model inference, we use the batch inference feature on Neibus Tabbusforr for all available models listed Table For proprietary models, we call their provided APIs. For those models without an online inference point, we host vL..M 39 inference service locally. Further details on the execution environment are available the Appendix H
Runtime. The monolith's 's runtime environment is standardized using Docker containerization to ensure consistency and portability across different programming languages. Each language service
https://stuuio..nbbuu..

---

within the monolith operates within a specific, pre-defined Docker image. Table 12 details the official Docker images utilized for various supported programming languages.
Table 12: Programming Language Docker Images
Language
Image
Python
python:3.9. 19-bullseye
Java
openjdk: 1 1.0. w...1-jjdkbbuusseeeeeee
Javascript
node:22-bullseye
Cpp
gee: 1 2.0-buulsseye
Go
golang: 1.7.0--bullseye
Ruby
ruby:3.0.1 1.t.3300.2bullsey..
Rust
1 1.....-8ulllsyy..
I Symbol List
Symbol / Term
Description
LLM
Large Language Model
SFT
Supervised Fine- Fine-Tuning
DPO
Direct Preference Optimization
GRPO
Group Relative Policy Optimization
RL
Reinforcement Learning
IOF
Iterative Optimization Framework (the proposed framework)
Afterburner
Code optimization models (trained via SFT, DPO, GRPO)
Monolith
A high-fidelity code execution sandbox for performance feedback
Venus
A dataset with human solutions, curated for efficiency benchmarking
APPS
An existing dataset for code generation, also used for evaluation
DSSFT
Dataset constructed for Supervised Fine-Tuning
DSpPO
Preference dataset constructed for Direct Preference Optimization
DSes
Cold Start Dataset used for initial format alignment GRPO models
DSGRPO
Dataset used for Group Relative Policy Optimization training
P
Problem description
I
Efficiency instruction
C
A code solution. Variants:
Cin
Input code solution for iteration i
Cont
Output (improved) code solution from Afterburner at iteration A more efficient/prefeerred code solution
C+
C-
A less efficient/dis prefferrrreeed code solution A baseline code solution for comparison (in DPO context)
Chaseline
M
Performance metric(s) of a solution. Variants: Performance metrics of Cin Performance metrics of Cout
M
Mant
Relation indicating superior performance (c.g., Mout > M
Nater
Total number of optimization iterations
Input prompt to a model (often includes P, Cin (MM
The policy (language model) being trained, parameterized by 0
Tref
Reference policy (e.g.. in DPO, SFT model)
LsFT
Loss function for Supervised Fine- tine-Tuning
LDPO
Loss function for Direct Preference Optimization
Lenro
Loss function for Group Relative Policy Optimization Weights for reward components (3y, (e,
B
o())
The logistic function
Continued on next page
25

---

Symbol / Term
Description
Formatt
Reward component for adhering to the specified output format
Reorrect
Reward component for functional correctness
Reffidency E
Reward component for improvement computational efficiency
An absolute code efficiency value (e.g., execution time, peak memory) Normalized relative gain in an efficiency metric e
Egain
Edip
Clipped value of an efficiincy metric
Eupper
Upper limit for clipping an efficiency metric
R final
The final combined reward signal for GRPO
O
The i-th generated output (rollout/candidate solution) in a GRPO group Size of the rollout group in GRPO
G
W;
Policy ratio (importance weight) for rollout O in GRPO, (0,
To (0.
A;
Advantage of rollout O; within its group in GRPO
A small constant
@ 1
Percentage of the first generated solution passes all test cases
Percentile Rank: fraction of items in distribution 1) that greater than or equal to (for efficiency, lower is better, so - PR or adjusted PR is used implicitly if higher means better)
PR(x, D)
Global efficiency metric: average percentile rank of generated code's execution time relative human solutions. Higher is better.
BEYOND-T
Global efficiency metric: average percentile rank of generated code's memory usage relative human solutions. Higher is better. Global efficiency metric: average percentile rank of generated code's integral score relative to human solutions. Higher better. Absolute execution time, peak memory usage, and integral score of the generated solution for task k Distributions of execution times, memory usages, and integral scores from reference human solutions for task Percentage of generations that are better than all human solutions Percentage of generations whose efficiency falls within the range of human solutions
BEYOND-M
BEYOND-I
gen l
gen gen
2
DT.
D D/
3%
M%
W%
Percentage generations that are worse than all human solutions Percentage of failed model generation
F%
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitations of the work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
27

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All necessary information reproduce the experimental results are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
28

---

Answer: [Yes]
Justification:
We released the data and code to reproduce our main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
cesssnnnoiccyy lc for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cpippi.ccppullc//uuuii wwwpc//ggue////eeeeeoooc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specified all the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes. We report CI 95%.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
29

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes] We provided information the computer resources.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //tp:r/:///uips...eee https//ccpppcccccccccccceeeeeeee
Answer: [Yes]
Justification:
The research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
30

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cited the original paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whworee com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
31

---

Answer: [Yes]
Justification:
The newly introduced dataset is well-documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
32

---

The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non standad components.
Please refer to our LLM policy hattps://neurips... 200ps//ee///0ooeeeeeee////////// for what should or should not be described.
33


Reviews:
[{"review_id": "HmChulYqgm", "reviewer": "Reviewer_ovks", "summary": "This paper proposes a test-time computation method to iteratively improve the efficiency of code snippets generated by LLMs. Specifically, the authors apply reinforcement learning to fine-tune a Qwen-2.5 3B model. Once the model is sufficiently fine-tuned, it is used to optimize code snippets by leveraging execution feedback. To evaluate the effectiveness of the proposed approach, experiments are conducted on the Venus Python dataset. The authors define three metrics: PASS@1, BEYOND-T, BEYOND-M, and BEYOND-I. \nThe results demonstrate that the method significantly enhances code efficiency. Moreover, the results also demonstrate that LLM-generated code may be more efficient than human-written code.", "questions": "1. Does the proposed dataset contain ground truth code snippets? If not, the reliability of the LLM-generated test cases may be questionable.\n\n2. Are all the collected code snippets correct? For example, if some incorrect snippets simply return without executing any statements, they could achieve perfect efficiency but be meaningless.\n\n3. After fine-tuning on the Venus dataset, can the fine-tuned model generalize and perform well on other datasets?\n\n4. Did you comapre the proposed approach with native \"SFT\" baseline, which just finetune the model to generate most efficient correct code snippets.\n\n5. Can your proposed algorithm generalize to other base model architectures, such as Llama?\n\n6. Where do the test cases used in the iterative improvement stage come from?\n\n7. When computing the efficiency metric during evaluation, did you filter out incorrect code snippets, or did you only consider correct ones?", "limitations": "1. **Evaluation Metric Rationale**: The rationale behind the chosen evaluation metric requires further discussion. The authors argue that absolute efficiency metrics are avoided due to their sensitivity to hardware configurations and operating systems. However, this justification is unconvincing for several reasons: (1) their method still relies on comparing absolute efficiency metrics; (2) when measured on the same hardware and operating system, efficiency metrics can be fairly compared, and system noise can be mitigated through multiple runs; and (3) the proposed metric heavily depends on the distribution of efficiency in human-written code. I suggest that the authors also present results using absolute efficiency metrics.\n\n2. **Source of Inference-Time Test Cases**: The origin of the inference-time test cases is unclear. The proposed method relies on these runtime test cases to provide correct and efficient feedback for self-improvement. However, it is not specified where these test cases come from: Are they the same as the evaluation test cases, or are they generated by GPT at runtime? If they are generated by GPT at runtime, there is a risk of intelligence leakage. In other words, if GPT is used to generate test cases, why not use GPT to generate correct or efficient code directly, especially since the results show GPT may outperform the proposed method?\n\n3. **Generalizability of the Proposed Method**: The generalizability of the proposed method is unclear. (1) It is not demonstrated whether the method can generalize to other model architectures, such as Llama. (2) Although Table 1 shows the proposed method outperforms baseline methods, all baselines have been fine-tuned specifically on the proposed dataset. It remains unclear whether the fine-tuned model can generalize to other datasets. I recommend that the authors conduct dynamic evaluation [1, 4] or test on other datasets [2, 3] to better demonstrate generalizability.\n\n4. **Missing Baseline**: Why was a simple baseline—fine-tuning the base model with only the most efficient code snippets—not considered?\n\n5. It is unclear whether the efficiency metric is measured only on the correct code.\n\n[1] DyCodeEval: Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination. ICML 2025\n\n[2] Effibench: Benchmarking the efficiency of automatically generated code. Nips 2024.\n\n[3] Evaluating language models for efficient code generation. COLM 2024.\n\n[4] DynaCode: A Dynamic Complexity-Aware Code Benchmark for Evaluating Large Language Models in Code Generation", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength:\n\n+ The paper is well-written and easy to follow.\n+ The problem addressed is both significant and important.\n+ The evaluation results are promising.\n\n\nWeakness:\n\n– The rationale behind the evaluation metric requires further discussion.\n\n– Some details about the proposed dataset are missing.\n\n– The generalization ability of the proposed model is unclear.", "was_revised": true, "final_justification": "Based on the rebuttal, I updated my score. However, the way authors evaluated (i.e., using the same set of test cases for improving the model and evaluating them) is still very problematic to me. All my other points are address by the authors during the detailed rebuttal period.", "rebuttal": "Dear Reviewer ovks,\n\nThank you for your constructive and insightful feedback. We are encouraged that you found our work valuable. Below, we address per questions and suggestions raised.\n\n> Q1 & Q2: Solution Correctness in Venus\n\nAll Venus code snippets are exclusively collected from accepted (including the official ground truth) solutions on Leetcode, meaning they passed all test cases. We agree that this validation process is a critical detail. Due to space constraints, we included the full pipeline in Appendix C.1. For improved clarity, we will add a brief explanation and a direct reference in Section 5.1 Dataset Recipe:\n\n*“Venus Python subset contains 2,181 algorithmic problems, each accompanied by a validated test case generator and an average of 106.6 validated human solutions... For each LLM-generated test case input, we follow the paradigm of Mercury, where we execute them through all collected solutions from LeetCode, and only keep those cases having consistent outputs over all correct solutions. More details can be found in Appendix C.1 Rigorous Validation.”*\n\n> Q3: Model Generalization\n\nWe evaluated our Venus-tuned model on another dataset APPS [1]. Unlike existing code efficiency benchmarks such as Mercury [2], EffiBench [3] and Venus, which consist of LeetCode-style questions, the APPS dataset features distinct problem formats. As shown in the tables below, Afterburner demonstrates a consistent performance improvement pattern on the APPS dataset. Please refer to Appendix Figure 9 for a complete visualization. We also conducted an additional experiment on LiveCodeBench [8] to demonstrate the consistent performance gain on the contamination-free benchmark.\n\n| Afterburner Base on APPS Iterations | 0 | 1 | 3 | 5 | 7 | 10 |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| Pass | 10.67 | 11.67 | 13.67 | 15.00 | 15.33 | 15.67 |\n| Beyond-T | 4.14 | 4.14 | 4.14 | 4.23 | 4.53 | 4.53 |\n\n| Afterburner GRPO on APPS Iterations | 0 | 1 | 3 | 5 | 7 | 10 |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| Pass | 13.00 | 15.00 | 20.33 | 24.00 | 28.00 | 31.67 |\n| Beyond-T | 4.69 | 9.04 | 12.36 | 14.85 | 16.01 | 16.18 |\n\n| Afterburner Base on LiveCodeBench (release_v6) Iterations | 0 | 1 | 3 | 5 | 7 | 10 |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| Pass % | 5.49 | 6.59 | 6.59 | 7.14 | 7.69 | 7.69 |\n\n| Afterburner GRPO on LiveCodeBench (release_v6) Iterations | 0 | 1 | 3 | 5 | 7 | 10 |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| Pass % | 5.49 | 7.69 | 9.89 | 11.54 | 12.09 | 13.19 |\n\nWe acknowledge that we missed the reference to the APPS results in the main body. Thank you for pointing it out! We will add [5,6] into related work and append the following statement in Section 6.2 to reference these findings in the main text:\n\n*“...To verify whether our models can generalize to out-of-distribution questions, we evaluated Afterburner on the APPS dataset [1]. As illustrated in Appendix Figure 9, Afterburner demonstrates a similar pattern of performance improvement, confirming its effectiveness on problems with distinct data distribution.”*\n\n> Q4 & L4: Native \"SFT\" Baseline\nWe agree that a comparison with a native SFT baseline is valuable. We interpret \"native SFT\" in two paradigms and address both below:\n\n- **Code Refinement SFT:** trains a model using (problem_description, original_code) to generate an improved_code. This is conceptually very similar to our Afterburner SFT baseline, which augments the input with performance feedback from the original code. We provide its results in Figure 5.\n- **Direct Generation SFT:** trains a model using (problem_description) to directly generate the improved_code. *We assume your question refers to this paradigm.* We initially omitted this baseline because it lacks a performance feedback mechanism, making a direct comparison with our feedback-driven method is unfair. Furthermore, as noted by prior work Mercury [2], this direct approach can be susceptible to catastrophic forgetting, especially on smaller models.\n\nWe agree including the native SFT baseline in this work could serve as a straightforward baseline. Prompted by your suggestion, we have conducted this additional experiment to fine-tune the 'Qwen2.5-3B' model using the same hyperparameters as our main experiments. The results are presented in the table below. We observed model collapse on the native SFT baseline.\n| SFT Paradigm      |  Iteration |   0   |   1   |   3   |   5   |   7   |   10  |\n|--|:---:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| Code Refinement   |   Pass %   | 46.00 | 46.00 | 47.00 | 48.33 | 48.67 | 48.67 |\n| | Beyond-I % | 21.01 | 21.09 | 21.88 | 22.31 | 22.44 | 22.50 |\n| Direct Generation |   Pass %   | 21.67 | 21.67 | 22.33 | 22.33 | 22.67 | 22.67 |\n| | Beyond-I % |  5.14 |  5.76 |  6.47 |  6.63 |  6.63 |  6.75 |\n\n> Q5 & L3: Model Generalization\n\nDue to the rebuttal time limitation, we conducted additional experiments on another model ‘Qwen2.5 7B’ and 'GPT-4o'.  As the data shows, larger models do achieve better performance through the IOF iterations. However, the improvement in these vanilla models is significantly less than that of our Afterburner models. This demonstrates that while the IOF framework itself is beneficial, our proposed training paradigm are the primary drivers of the substantial performance gains we report. For other model architectures, such as Llama, we will complete its experiment results in the revision.\n\n| **Qwen2.5 7B Iteration** | 0 | 1 | 2 | 5 | 7 | 10 |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Pass** | 60.67 | 65.67 | 69.33 | 74.67 | 79.67 | 83.67 |\n| **Beyond-T** | 27.67 | 32.87 | 34.22 | 39.50 | 41.91 | 42.86 |\n| **Beyond-M** | 29.79 | 31.55 | 36.48 | 40.97 | 43.26 | 44.51 |\n| **Beyond-I** | 21.02 | 28.10 | 33.85 | 37.54 | 37.54 | 38.66 |\n\n| **GPT-4o Iterations** | 0 | 1 | 2 | 4 | 8 |\n|---|:---:|:---:|:---:|:---:|:---:|\n| **Pass %** | 82.26 | 84.00 | 84.33 | 84.33 | 85.33 |\n| **Beyond-T %** | 38.22 | 40.16 | 42.02 | 44.16 | 46.17 |\n| **Beyond-M %** | 42.09 | 43.28 | 44.71 | 46.20 | 48.01 |\n| **Beyond-I %** | 28.89 | 30.60 | 32.55 | 35.02 | 41.29 |\n\n> Q6 & L2: Where do the test cases used in the iterative improvement stage come from?\n\nWe use the same pre-generated test cases for both iterative improvement and evaluation. This approach is valid because the model only receives high-level execution feedback—such as pass status, running time, and memory usage—without ever seeing the specific contents of the test cases. This design prevents information leakage and ensures a fair evaluation.\n\n> Q7: When computing the efficiency metric during evaluation, did you filter out incorrect code snippets, or did you only consider correct ones?\n\nDuring the efficiency evaluation, we didn’t filter out incorrect code. The monolith sandbox will return the default maximum metric values (time: 90 s, memory: 1,048,576 kb, integral: 90 * 1,048,576 = 94371840), so their percentile ranks will be 0. You can consider these efficiency metrics (time, memory, integral) as efficiency-weighted functional correctness (pass). \n\n> L1: Evaluation Metric Rationale. \n\nThank you for this insightful suggestion! We agree that using absolute efficiency metrics has its advantages, especially 1) it is easy to calculate; 2) it doesn't rely on the distribution of human solutions; 3) it may work if we evaluate all models on the same machine.  We appreciate your perspective and we have revised the paper to clarify our rationale per your suggestion:\n\n- **Benchmark Standardization**: We follow the relative efficiency metrics used in most established code efficiency benchmarks, such as Mercury [2], EffiBench [3], EvalPerf [4] and ENAMEL [7].\n- **Result Comparability:** Since Venus is a code efficiency benchmark, we hope the reported score is comparable. While absolute efficiency metrics may be infeasible to compare across machines, relative metrics can handle this issue. For example, a solution that is faster than 80% of human solutions on one machine is likely to be similarly performant relative to the same set of solutions on another machine.\n- **Performance Normalization:** Absolute runtimes can vary by orders of magnitude across different test cases. This large range means that a model's aggregate score can be dominated by a few long-running outliers, masking subtle but significant improvements. Our relative metric normalizes performance against the distribution of human-written code, providing a more stable and fine-grained measure of efficiency improvement.\n- **Training vs. Evaluation:** You correctly note that we use absolute metrics internally. We do this specifically during model training, where the raw execution time and memory provide simple and effective reward signals. Since training occurs in an isolated sandbox on a single hardware setup, comparability is not a concern. Furthermore, for reinforcement learning, an unbounded absolute metric allows the model to explore solutions that surpass the efficiency of any known human code. However, for the final evaluation, the need for a normalized, comparable, and robust benchmark makes the relative metric the superior choice.\n\n**We hope our explanation can address your concern. Please ask follow-ups if you have any other questions. Thank you!**\n\n### Reference\n\n[1] Measuring coding challenge competence with apps. \n\n[2] Mercury: A code efficiency benchmark for code large language models.\n\n[3] Effibench: Benchmarking the efficiency of automatically generated code.\n\n[4] Evaluating language models for efficient code generation. \n\n[5] DyCodeEval: Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination.\n\n[6] DynaCode: A Dynamic Complexity-Aware Code Benchmark for Evaluating Large Language Models in Code Generation\n\n[7] How efficient is llm-generated code? a rigorous & high-standard benchmark.\n\n[8] LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code."}, {"review_id": "uP0Dm8phY7", "reviewer": "Reviewer_xncp", "summary": "This paper introduces Afterburner, an iterative optimization framework designed to improve the computational efficiency of LLM-generated code. The framework employs a closed-loop system where LLMs iteratively refine code based on empirical performance feedback from \"Monolith,\" a high-fidelity execution sandbox that provides real-time efficiency metrics. The authors compare three training strategies: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO), finding that while SFT and DPO quickly plateau in efficiency gains, GRPO using reinforcement learning with execution feedback continuously optimizes code performance. Experiments on the newly introduced Venus dataset and the APPS benchmark demonstrate that GRPO significantly improves both functional correctness (PASS@1 from 47% to 62%) and efficiency compared to human solutions (BEYOND-I from 31% to 45%). The work addresses a critical gap in LLM-generated code, where models often produce functionally correct but computationally inefficient solutions, and shows that online reinforcement learning with direct execution feedback enables models to achieve sustained improvements in code efficiency through iterative test-time optimization.", "questions": "- Could the authors provide a pointer to the reproducible artifact of Venus, Afterburner, and Monolith?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strength\n\n+ This paper is among the earliest works that aim to improve the \"efficiency\" of LLM-generated code, not only its correctness. Efficiency is an important yet under-explored perspective of LLM-generated code\n+ This paper implements a close-loop pipeline that can automatically generate code, reliably collect the efficiency feedback, and iteratively refine its solutions with Afterburner and Monolith\n+ This paper releases the first dataset, Venus, that can be used for training LLMs for efficiency optimization, further boosting the future research of improve the efficiency of LLM-generated code.\n+ This paper comprehensively and systematically studied the impacts of SFT, DPO, and GRPO for code efficiency refinement training, providing practical insights for the efficiency optimization training recipe.\n+ This paper successfully trained Afterburner, which has only 3B parameters while matching much larger models, such as Qwen-2.5-7B-instruct, and Llama-4-Scout, in code efficiency.\n\n## Weaknesses\n\nOverall, I am very positive about this paper, so I do not have much to complain about. One minor thing is that, while the paper replied \"Yes\" in their checklist about the release of their data and code, I could not find a link to these artifacts. This discrepancy might downgrade the transparency and reproducibility.", "was_revised": true, "final_justification": "I am glad to see that the authors commit to releasing their data and artifacts, which was my major concern. I am keeping my score to 5 to support this paper's acceptance.", "rebuttal": "Dear Reviewer xncp,\n\nThank you for your positive feedback!\n\nWe have prepared the artifacts of Venus, Afterburner, and Monolith. Due to the new rebuttal policy this year, we cannot share external links at this time. We are committed to reproducibility and will update these artifacts in the final version immediately after the review process.\n\nWe hope it can address your concern. Thank you!"}, {"review_id": "voR9bTlWnU", "reviewer": "Reviewer_z3gd", "summary": "The paper introduced AfterBurner, an iterative optimization framework to address the problem of LLM failing to generate efficient code despite their functional correctness. AfterBurner forms a closed loop: an LLM proposes an updated solution, the Monolith sandbox runs it and returns the runtime, memory and time measurements, and the best candidate becomes the seed for the next round. The authors explored three strategies:  Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and a reinforcement-learning variant Group Relative Policy Optimisation (GRPO) that leverages Monolith feedback as reward. To evaluate efficiency, the authors built Venus, a new dataset designed for rigorous code efficiency assessment, with 2181 training and 300 test tasks. The metrics they used is BEYOND-T/M/I, measuring percentile rank of runtime, peak memory and their integral against the human distribution. Experiments show that on Venus, GRPO improves pass@1 from 47% to 62%, BEYOND-I from 31% to 45%. Ablation shows that execution feedback and oracle context are essential.", "questions": "1. How can the proposed strategy handle API-dependent real-world code? For example, does the proposed method still work in software engineering benchmarks like SWE-bench or Defects4J? I noted this was explained in the Limitations section, but I'm still curious about the authors thoughts. It's okay to leave this unaddressed for this paper, but then I will also keep my concerns about the relatively narrowed scope.\n2. The final reward has a combination of hyperparameters. How sensitive is it when given different values?\n3. Could you elaborate on how you measure the statistical significance of your results?\n4. Does the same conclusion hold when APPS is changed to LiveCodeBench with no problem contamination?\n\nIf these questions are properly addressed, I'm happy to raise the score.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n* The experimental design is solid. It compares three optimisation paradigm and provides ablations and confidence intervals. The BEYOND metrics can mitigate some hardware variance.\n* The reward design and overall technique flow is well presented.\n* The paper tackles an under-explored aspect in code generation, i.e. efficiency. It matters a lot for production deployment. It also introduces a sizeable benchmark that others can reuse.\n* Combining RL with execution-based rewards for efficiency rather than correctness is a novel angle\n\n### Weaknesses\n\n* There are many reported improvements, but their statistical significances are unknown.\n* Some typos like \"paassed\" in line 135.\n* There are no decontamination processes mentioned in the paper. APPS is also not a \"live\" benchmark in comparison to LiveCodeBench.\n* The novelty mainly lies in the application of RL to code efficiency. GRPO itself extends as an existing algorithm.\n* The focusing on just python and file-level coding makes the scope of the paper narrow. In practice, developers care more about real-world code living in real software.", "was_revised": true, "final_justification": "The paper has made solid contributions and experiments. While my concern about its generalizability to open-ended software tasks is not addressed, the authors provide clear explanations and have addressed most of my other concerns. I'm giving an Accept.", "rebuttal": "Dear Reviewer z3gd:\n\nThank you for your constructive and insightful reviews! We hope we can address your concerns as follows:\n\n> Q1: Scaling to API-dependent Real-world Code\n\nThank you for this insightful question. While this paper focuses on establishing a robust method for function-level optimization, we agree that scaling to repository-level benchmarks like SWE-bench is a critical next step. We view our current work as the foundational layer for this larger goal and introduce the following strategy to address the main challenges:\n\n- **API Dependencies:** To handle API-dependent code, we may integrate Retrieval-Augmented Generation (RAG) and static analysis. This allows our model to dynamically pull in relevant API definitions, usage patterns, and code snippets from the broader repository, providing the necessary context for effective optimization.\n\n- **Context Management:** Real-world repositories are too large for current model context windows. We could employ a divide-and-conquer approach. This involves identifying and isolating the most relevant functions and modules for optimization, making the task computationally tractable.\n\n- **Test Generation:** Verifying changes across an entire repository is notoriously difficult. Our strategy would focus on generating targeted unit tests specifically for the modules undergoing optimization. This ensures the correctness and efficiency of our changes in a localized, verifiable, and recursive manner.\n\nIn summary, while scaling our method to repo-level presents a significant research challenge, we hope that structured approach provides a clear path forward. \n\n> Q2: Hyperparameters Sensitivity\n\nIn our experiment, we observed that a dominant weight for functional correctness (beta_c​ ≥ 0.5) is essential for stable training, as values below this threshold often led to training crash. To balance our objectives, we set a fixed beta_c ​​= 0.5 while gradually increasing the code efficiency weight (beta_e) from 0.3 to 0.5. This strategy ensures the model first learns to produce correct code before optimizing for efficiency.\n\n> Q3: Statistical Significance\n\nWe employ bootstrap sampling to construct 95% confidence intervals for our efficiency metrics. Here is the detailed procedure:\n1. **Data Collection:** For each individual problem, we generate 4 unique solutions (temperature=1) and measure the performance of each one 16 times. This results in a performance set of 64 measurements (4 solutions×16 measurements).\n2. **Bootstrap Sampling:** We then create 128 bootstrap samples. Each sample is formed by drawing one measurement with replacement from the original performance set.\n3. **Distribution Generation:** We generate an empirical sampling distribution for each efficiency metric by aggregating the 128 bootstrap samples over all problems.\n4. **Score Reporting:** We report the mean of these bootstrap distributions along with their 95% confidence interval.\nThis method allows for a robust estimation of efficiency performance and its statistical uncertainty. We provide a detailed explanation in Appendix G.\n\n> Q4: Contamination-free Evaluation\n\nWe evaluate our methods on LiveCodeBench and report the pass rate below.\nThe results confirm that our conclusions hold. Afterburner-GRPO shows substantial and consistent improvement as the number of iterations increases, while Afterburner-SFT plateaus quickly. This trend is consistent with our findings on the APPS and Venus benchmarks, demonstrating the general effectiveness of our approach.\n\n| **Afterburner-SFT Iteration** | 0 | 1 | 2 | 5 | 7 | 10 |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Pass %** | 5.49 | 6.59 | 6.59 | 7.14 | 7.69 | 7.69 |\n\n| **Afterburner-GRPO Iteration** | 0 | 1 | 2 | 5 | 7 | 10 |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Pass %** | 5.49 | 7.69 | 9.89 | 11.54 | 12.09 | 13.19 |\n\n> Q5: Some typos like \"paassed\" in line 135.\n\nWe will fix this typo in the revision. \n\n**We hope our explanation can address your concern. Please ask follow-ups if you have any other questions. Thank you!**"}, {"review_id": "zRzeN1OfMx", "reviewer": "Reviewer_UfKJ", "summary": "This paper introduces an Iterative Optimization Framework (IOF) for improving code efficiency at test time using feedback-driven refinement. In each iteration, a language model receives the current version of the code and execution-based feedback (e.g., peak memory usage, latency, total memory) from a sandboxed runtime environment, then generates an improved version of the code. The authors evaluate various training strategies—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO)—for their ability to adapt under the IOF setting. They further release Venus, a new dataset consisting of programming problems and associated solution variants with varying efficiency profiles.", "questions": "- See Weaknesses section\n- How does IOF performance vary when initialized with code that spans the efficiency spectrum (from highly inefficient to highly efficient)? Are GRPO-trained models more robust to such variation?\n- Have you conducted ablations on the number of iterations (e.g., convergence rate) or observed performance ceilings?", "limitations": "Yes, the authors have discussed their Limitations but I would suggest it to move it from the Appendix to the Main Body of the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nClear Framework Design: The paper presents a well-defined framework for feedback-driven code-efficiency refinement. The iterative nature of the process and the way it leverages runtime metrics are clearly explained.\n\nClear Dataset Description: The Venus dataset and associated training data are systematically constructed and properly explained. This is a valuable resource for the community focused on efficiency-aware code generation.\n\nEmpirical Gains from GRPO: The GRPO-trained Qwen2.5 3B model demonstrates significantly better test-time efficiency improvements over baseline and SFT/DPO-trained models within the IOF, showcasing the method's potential.\n\n**Weaknesses**\n\n1. Incomplete Evaluation of IOF Design Choices:\nThe current implementation generates improved code based only on the most recent iteration. Would a history-aware model  (i.e., $C_{0:i-1}, M_{0:i-1}$) perform better? Using all previous iterations could be useful to break out of local minima where $M_{i}^{out}=M_i^{in}$ despite code differences $C_{i}^{out} != C_i^{in}$?\n\n2. Lack of Scaling Studies:\nAll key results focus on the Qwen2.5 3B model. It's unclear whether GRPO and IOF yield consistent improvements for larger-scale models (e.g., 7B, 13B). This is critical to assess the generalizability and practical relevance of the framework.\n\n3. Limited Baseline Comparisons in IOF Iterations:\nTable 1 compares performance across different models only at iteration 0, but does not show how baseline models (e.g., vanilla GPT, LLama4, Claude, etc.) evolve under IOF. This obscures how much of the final performance gain comes from model training (e.g., GRPO) versus the framework itself. Showing relative improvement over iterations (e.g. $C_0$ against $C_{i_{best}}$) for different base models would help isolate the benefits.\n\n4. Insufficient Justification for Venus Dataset:\nThe paper lacks a clear comparison with existing datasets such as Mercury, EffiBench, and EvalPerf. What are the limitations of those datasets that Venus addresses? What is the statistical diversity (e.g., memory, time distributions) of reference solution variants in Venus compared to prior work?\n\n5. Missing Benchmark Analysis on Existing Datasets:\nThere is no empirical evaluation of IOF and GRPO-trained models on established efficiency-focused code benchmarks like Mercury, EffiBench. This limits the ability to compare against prior art and undermines claims of generalizability.", "was_revised": true, "final_justification": "The authors have addresses all my concerns with adequate experiments and justification. While using iterative feedback to improve LLM responses is not completely novel, using it for improving code efficiency is an interesting use case. Thus I am increasing my score from Reject to Borderline Accept.", "rebuttal": "Dear Reviewer UfKJ,\n\nThank you for your constructive and insightful reviews. We hope we can address your concerns as follows:\n\n> Q1: Would a history-aware model perform better? \n\nWe conduct the experiment as you suggested, and show results in the following table. The results below show that the history-aware model did not outperform the single-turn model in the current setting.\n\n| Single-turn Loop Iteration | 0 | 1 | 2 | 5 | 7 | 10 |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| Pass %  | 47.33 | 50.33 | 52.00 | 58.17 | 60.50 | 61.67 |\n| Beyond-I %  | 18.24 | 24.81 | 29.44 | 35.48 | 38.01 | 38.95 |\n\n| Multi-turn Loop (History-aware) Iteration | 0 | 1 | 2 | 5 | 7 | 10 |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| Pass %  | 45.00 | 46.33 | 46.33 | 46.33 | 46.67 | 46.67 |\n| Beyond-I %  | 15.10 | 15.10 | 15.10 | 15.10 | 15.10 | 15.10 |\n\nWe attribute the underperformance of history-aware model to two primary factors: **1) Distribution Shift:** The performance drop is likely caused by a distribution shift, as the training format (single-turn) differs from the multi-turn, history-aware format used during inference. **2) Training Challenges:** While fine-tuning the model with multi-turn format could mitigate the distribution shift, this approach introduces the significant challenge of long-term credit assignment, a well-known difficulty in Reinforcement Learning. We agree that history-aware modeling is a valuable and promising research direction. We appreciate the insightful feedback and plan to explore this more thoroughly in our future work.\n\n> Q2: Model Scale Generalization\n\nWhile this work focuses on exploring the methodology of iterative efficient optimization rather than the scalability of larger models, we do agree that it is critical to evaluate our frame on larger models. Due to the rebuttal time limitation, we conducted additional experiments on Qwen2.5 7B as you suggested. The result demonstrates our iterative optimization framework can be generalized to larger models. \n\n| **Iteration** | 0 | 1 | 2 | 5 | 7 | 10 |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Pass %** | 60.67 | 65.67 | 69.33 | 74.67 | 79.67 | 83.67 |\n| **Beyond-T %** | 27.67 | 32.87 | 34.22 | 39.50 | 41.91 | 42.86 |\n| **Beyond-M %** | 29.79 | 31.55 | 36.48 | 40.97 | 43.26 | 44.51 |\n| **Beyond-I %** | 21.02 | 28.10 | 33.85 | 37.54 | 37.54 | 38.66 |\n\n> Q3: Baseline Comparisons in IOF Iterations\n\nWe thank the reviewer for this insightful suggestion. To better isolate the benefits of our framework versus our model training approach, we have conducted new experiments applying IOF to several baseline models as requested. Specifically, We ran the IOF process for 8 iterations on *the vanilla Qwen-2.5-3B, Qwen-2.5-7B, and OpenAI GPT-4o models*. The results are presented below:\n\n| **Qwen2.5 3B Iterations** | 0 | 1 | 2 | 4 | 8 |\n|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Pass %** | 27.99 | 28.33 | 28.67 | 29.00 | 29.33 |\n| **Beyond-T %** | 12.40 | 12.74 | 13.08 | 13.42 | 13.69 |\n| **Beyond-M %** | 13.24 | 13.73 | 14.22 | 14.71 | 15.19 |\n| **Beyond-I %** | 10.29 | 10.77 | 11.25 | 11.72 | 12.07 |\n\n| **Qwen2.5 7B Code Iterations** | 0 | 1 | 2 | 4 | 8 |\n|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Pass %** | 52.21 | 54.00 | 54.33 | 56.00 | 58.33 |\n| **Beyond-T %** | 20.66 | 21.47 | 23.03 | 23.47 | 26.26 |\n| **Beyond-M %** | 25.21 | 25.72 | 26.48 | 29.93 | 31.20 |\n| **Beyond-I %** | 16.78 | 17.69 | 20.91 | 23.21 | 27.14 |\n\n| **GPT-4o Iterations** | 0 | 1 | 2 | 4 | 8 |\n|---|:---:|:---:|:---:|:---:|:---:|\n| **Pass %** | 82.26 | 84.00 | 84.33 | 84.33 | 85.33 |\n| **Beyond-T %** | 38.22 | 40.16 | 42.02 | 44.16 | 46.17 |\n| **Beyond-M %** | 42.09 | 43.28 | 44.71 | 46.20 | 48.01 |\n| **Beyond-I %** | 28.89 | 30.60 | 32.55 | 35.02 | 41.29 |\n\nAs the data shows, larger models do achieve better performance through the IOF iterations. However, the improvement in these vanilla models is significantly less than that of our Afterburner models. This demonstrates that while the IOF framework itself is beneficial, our proposed training paradigm are the primary drivers of the substantial performance gains we report. \n(Note: As closed-source models have not disclosed their training details for, we cannot determine if similar reinforcement learning techniques were already incorporated into their development pipeline.)\n\n> Q4: Venus Justification\n\nWe recognize the importance of a clear comparison with prior work and will clarify this in the revised paper.\n- **Multilingual Coverage:** A primary contribution is that Venus is the first multilingual code efficiency benchmark, covering six languages. Prior work, including Mercury [1], EffiBench [2], and EvalPerf [3], focuses exclusively on Python.\n- **Solutions Diversity:** Venus offers a significantly larger and more diverse set of reference solutions. While EffiBench uses a single baseline and Mercury averages 18.4 solutions per problem, Venus provides an average of 79.3 solutions per problem for each language. This massive increase directly contributes to the statistical diversity of performance metrics, as shown in the time and memory distributions in Appendix Figure 7.\n- **Evaluation Dimensions:** As a direct extension of Mercury, which only measures execution time, Venus evaluates execution time, memory usage, and their integral, providing a more holistic assessment of code efficiency.\n\nWe will incorporate this detailed comparison, currently summarized in Appendix C, into the main body to make the justification for Venus more clear.\n\n> Q6: IOF Efficiency Spectrum \n\nIt's a good suggestion! We conduct the experiment to start Afterburner SFT and Afterburner GRPO with the efficiency spectrum. Since we are not allowed to upload images in the rebuttal response, we present iterative Beyond-I results beginning from the inefficient (0% Beyond-I) to the efficient (50% Beyond-I) solutions in the following table. I will update the complete efficiency spectrum figure in the revision.\n\n| **Less Efficient Start Iterations** | 0 | 1 | 2 | 4 | 8 |\n|---|---:|---:|---:|---:|---:|\n| Afterburner SFT | 3.76 | 4.74 | 5.24 | 7.31 | 11.50 |\n| Afterburner GRPO | 3.76 | 4.28 | 5.21 | 8.60 | 13.85 |\n| **More Efficient Start Iterations** | 0 | 1 | 2 | 4 | 8 |\n| Afterburner SFT | 51.44 | 51.44 | 52.93 | 54.21 | 56.73 |\n| Afterburner GRPO | 51.44 | 53.70 | 55.12 | 58.21 | 62.17 |\n\nWe argue that a key advantage of RL is exploring more sufficiently in the exploitation space than SFT or DPO. Therefore, it is more robust if the inference distribution aligns with the training distribution.\n\n> Q7: Performance Ceilings\n\nYes, In our experiments, performance ceilings are observed from IOF round 7 on both the Venus and APPS benchmarks, as shown in Figures 5 and 9. We hypothesize that these ceilings result from the trade-off between task complexity and the model capability. \nWe conduct an additional experiment on three difficulty categories. For a model with a fixed capability, more difficult tasks require more iterations to converge on an optimal solution, whereas on simpler tasks, performance saturates and hits this ceiling much earlier. The following table shows that the iteration of performance ceilings (less than 5% efficiency gap with the final iteration) on Venus and APPS:\n\n|  | **Easy** | **Medium** | **Hard** |\n|---|:---:|:---:|:---:|\n| **Venus (Beyond-T)** | 3 | 6 | 7 |\n| **APPS (Beyond-T)** | 4 | 5 | 6 |\n| **Venus (Beyond-M)** | 3 | 5 | 8 |\n| **APPS (Beyond-M)** | 2 | 4 | 9 |\n| **Venus (Beyond-I)** | 4 | 5 | 8 |\n| **APPS (Beyond-I)** | 3 | 5 | 9 |\n\n> L1: Limitation Location\n\nThank you for your suggestion. We will move it from the appendix to the main body.\n\n**We hope our explanation can address your concern. Please ask follow-ups if you have any other questions. Thank you!**\n\n[1] Mercury: A code efficiency benchmark for code large language models.\n\n[2] Effibench: Benchmarking the efficiency of automatically generated code.\n\n[3] Evaluating language models for efficient code generation."}]

Author rebuttal:

Meta-review: The paper proposes a test-time iterative optimization framework for code generation, where an LLM refines code in a closed-loop system using empirical performance feedback from an execution sandbox. Three training strategies are explored: SFT, DPO, and GRPO. The GRPO variant with execution feedback yields the strongest gains, improving both pass@1 and the likelihood of outperforming human submissions in efficiency.

Strengths:
- Addresses an important and impactful problem (ovks, xncp, z3gd)
- Implements a closed-loop pipeline that integrates code generation, efficiency feedback collection, reward formulation, and iterative refinement (xncp, z3gd)
- Promising evaluation results with comprehensive experiments across training strategies (xncp, ovks, z3gd, UfKJ)

Weaknesses:
- The rationale behind the evaluation metric needs further discussion, though the authors clarified during rebuttal (ovks)
- Some dataset details were missing and the authors reasonably addressed in rebuttal (ovks)
- Open questions remain on scaling the method to larger models (UfKJ)

This work presents a well-motivated and carefully evaluated approach for integrating performance/efficiency feedback into code generation via iterative optimization. While some methodological details and scaling considerations remain open, the contribution is solid and the findings are empirically supported.
Decision: Accept (poster)