Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learning
Authors: Haolin Pan
Hongyu Lin
Haoran Luo
Yang Liu
Kaichun Yao
Libo Zhang
Mingjie Xing
Yanjun Wu
Venue: NeurIPS 2025 poster ()
Keywords: LLM
Compiler Tuning
Reinforcement Learning
Agent
Compilation Pass
Primary area: machine_learning_for_sciences

Abstract: Compiler auto-tuning optimizes pass sequences to improve performance metrics such as Intermediate Representation (IR) instruction count. Although recent advances leveraging Large Language Models (LLMs) have shown promise in automating compiler tuning, two significant challenges still remain: the absence of high-quality reasoning datasets for agents training, and limited effective interactions with the compilation environment. In this work, we introduce Compiler-R1, the first reinforcement learning (RL)-driven framework specifically augmenting LLM capabilities for compiler auto-tuning. Compiler-R1 features a curated, high-quality reasoning dataset and a novel two-stage end-to-end RL training pipeline, enabling efficient environment exploration and learning through an outcome-based reward. Extensive experiments across seven datasets demonstrate Compiler-R1 achieving an average 8.46\% IR instruction count reduction compared to opt -Oz, showcasing the strong potential of RL-trained LLMs for compiler optimization. Our code and datasets are publicly available at https://github.com/Panhaolin2001/Compiler-R1.

Compiler-RI: Towards Agentic Compiler Auto-tuning with Reinforcement Learning
Haolin Pan 1,23,, Hongyu Lin Haoran Luo+1, Yang Liu Kaichun Yao², Libo Zhang?, Mingjie Xing Yanjun Wu
'Hangzhou Institute for Advanced Study, UCAS, China
Institute of Software Chinese Academy of Sciences, China University of Chinese Academy Sciences, China *Nayyang Technological University, Singapore
(hongyu2021, miiggjieniiiiie @iscas. ac cn, haoran. www..uuueeeee... org panhaolin21 Gmails ucas. ac.
Abstract
Compiler auto-tuning optimizes pass sequences improve performance metries such as Intermediate Representation (IR) instruction count. Although recent ad- vances leveraging Large Language Models (LLMs) have shown promise in automat- ing compiler tuning, two significant challenges still remain: the absence of high- quality reasoming datasets for agents training, and limited effective interactions with the compilation environment, In this work, we introduce Compiler-R1 1, the first re- inforcement learning (RI))drivee framework specifically augmenting I.L.M capabil- ities for compiler auto-tuning -uting Compiler-R1 features curated, high-quality reason- ing dataset and a novel two-stage end-to-ens RL training pipeline, enabling efficient environment exploration and learning through an outcome-based reward. Extensive experiments across seven datasets demonstrate Compiler-RI achieving an average 8.46% IR instruction count reduction compared to opt -Oz, showcasing the strong potential of RL-trained LLMs for compiler optimization. Our code and datasets are publicly available at httpt:: htttt::t///iiiiiii //tp:::///hthtt httpa////aaaaaaaaooooo0/////ooo
Introduction
Compiler auto-tuning 5 focus on automat- ically select and order compilation passes, modu- lar optimization or analysis steps in modern com- pilers like LLVM 20 improve program per- formance. A typical tuning objective is reducing the Intermediate Representation (IR) instruction count while preserving program correctness, as depicted in Figure However, this task presents inherent challenging due to the combinatorial explosion of possible pass sequences and the intricate interactions between individual passes.
PlezA PeEsC Huge Search Space
pass8
[pass,, passB...)
D000! Autofuner
Optimal Pass Seq
IR
0
DOLO Machine
LLVM IR
Compiler
Redacing
Instruction
Figure 1: Overview of compiler auto-tuning task.
Recent years have witnessed significant advance- ments in compiler auto-tuning methods, evolving through three major generations: Heuristic-based Methods: Early work in iterative compilation [3 establish fundamental search strategies
"These authors contributed equally to this work. CCoresponding author(s).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

such as genetic algorithm and simulated annealing. Machine Learning (ML))enhanced Methods: Subsequent approaches 6 [3 26 leveraged ML models to improve tuning efficiency on unseen benchmarks. However, these traditional approaches, including both heuristic and MI. methods, often suffer from inherent inefficiency and limited generalization. Large Language Models (LLMss)- based Methods: With the advent of LLMs, recent efforts 17 have explored their application to compiler auto-tuning tasks, mitigating some of the aforementioned limitations.
Although recent LLMs M [8] exhibit remarkable reasoning and problem-solving capabilities, directly applying them to compiler auto-tuning faces two critical challenges. First, there an absence of high-quality training datasets tailored for auto-tuning. Effective interactive learning for LLM agents requires datasets that include Chain-of of-n--of-Tohghhh (CoT) reasoning, external tool integration. and environment feedback, which are currently lacking in compiler tuning domains. Second, existing methods based on Supervised Fine- Tuning (SFT) exhibit limited interaction with the compilation environment, restricting the model's ability to autonomously explore and adaptively learn from feedback, thus resulting in poor generalization to unseen programs.
To address these challenges, we propose Compiler-RI the first reinforcement learning (RL)-based framework specifically designed to enhance LLM capabilities in compiler auto-tuning. Compiler-RI introduces two main innovations: (1) A high-quality auto-tuning reasoning dataset: We curate comprehensive dataset comprising 19,603 carefully constructed samples, encompassing diverse and representative tuning scenarios, enabling effective interactive learning for compiler optimization tasks. (2) A two-stage end-to-end RL training framework: We develop novel RL framework that equips LLM agents with tool interfaces for autonomous environment exploration and leverages an outcome- based reward function to guide learning- This approach enables the agent capture both individual pass characteristics and their compositional interactions, significantly enhancing generalization capability and tuning efficiency.
We validate Compiler-RI I using various open-source LLM backbones on seven representative bench- marks. Experimental results indicate that Compiler-RI consistently achieves average IR instruction count reduction of 8.46% compared to the baseline method (opt -Oz), demonstrating the consider- able promise of RL-trained LLMs for advancing compiler auto-tuning research.
Our principal contributions are summarized as follows:
We construct a comprehensive. high-quality auto-tuning reasoning dataset by rigorously selecting and integrating multiple authoritative compiler datasets, encompassing diverse and representative tuning scenarios.
We propose Compiler-R1, the first RL-based two-stage training framework specifically designed to augment LLMs with tool-invocation and ad advanced reasoning capabilities tailored to compiler auto-tuning tasks.
We demonstrate through comprehensive evaluations that Compiler-RI consistently outper- forms other baseline methods, highlighting the substantial promise of RL-enhanced LLM approaches for advancing compiler optimization.
2 Related Work
2.1 Rule-based Reinforcement Learning with Large Language Models
Reinforcement Learning (RL) enables agents to perform sequential decision-making by interact- ing with environments maximize cumulative rewards. Recent advancements in rule-based RL frameworks, such as DeepSeek-R1 R1 (1I), have notably enhanced the reasoning and decision-making capabilities of Large Language Models (LLMs). This approach has effectively been extended to vari- ous domains, including question answering (5], knowledge retrieval [19 24), logical reasoning 29], mathematical problem-solving 288, code generation 23) and operating system [22]. Nevertheless, the application of rule-based RL to compiler auto-tuning tasks remains largely unexplored.
2.2 Compiler Auto-tuning
Compiler auto-tuning aims to automatically identify optimal pass sequences to enhance program performance. Traditional compilers like GCC and LLVM utilize fixed optimization pipelines designed

---

by domain experts, which often do not adapt well to specific programs or hardware. Iterative
compilation methods 13 employing heuristics such as genetic algorithms incur substantial
overhead. Supervised learning approaches, like coreset-NVP [21], rely heavily on labeled optimal
sequences, which are challenging to obtain. RL--based methods, such AutoPhase [15], dynamically adjust optimization strategies but have limited generalization. Recently, LLM-based approaches (8]
have shown potential by directly generating pass sequences from source code; however, they typically
lack sufficient environmental interaction, constraining their reasoning and adaptability.
3 Methodology: Compiler-R1
We present Compiler-RI R1, novel framework that introduces large language model (LLM) com- piler auto-tuning, incorporating both supervised learning and reinforcement learning (RL) techniques. As illustrated in Figure Compiler-RI represents the first application of reinforcement learning to compiler optimization, enabling models to move beyond imitation and autonomously learn optimization strategies through structured interaction with a simulated compilation environment.
Question
Given Autophase features (autophase_featiee featuees_s_eueese_eeeeeeee and on initial instruction count
W.WE Find Synengy Pain
(initial _arll__sttt_ttteet cust__cout), find on LLVM pess sequence outperforming -Oz in instruction count reduction. First, evaluate the "Initial Candidate Pass Sequence" with instrcount against -Oz. If improves, recommend If not, use find best__poss_ sequence to search. If this search finds improved sequence, recommend Otherwise, recommend ['-Oz'] Document your reasoning with ethinko, tool interactions <tool_call>> hat_lc/llll/ooo///ee ptepp/oosoossssssse.....eo and the finol sequence <onswer>.
pass8), (peest peasb)...
(passc,
Synergy
P_opt OK?
Pass
X
CoT Sample Pipline
Yes
OOO 80
LLM Agent
P_opt
FBPS
Select the Optinal Sequence
FBPS OK?
Instrcount Tool
State
Reward
[p0AsA pose,, prseC...]
Yes
No...
Compiler Autotuning Envirorment
Astoshena Facttruu
Seq
(Oz)
$
Foundation Dataset
SFT Samples
Cold Start SFT
Reinforcement Learning
Figure 2: Compiler-RI: A two-stage LLM training framework for compiler auto auto-tuning. Stage 1 (SFT): A foundation dataset derived from synergistic pass analysis is used to create SFT samples. These samples guide the LLM through CoT process, teaching interaction with instrcount and f find_ best_ best_passsseeuence sequence (FBPS) tools. Stage (RL): The SFT-initialized LLM agent interacts with the compiler autotuning environment, receiving state feedback and rewards learn optimal policy for pass sequence generation.
3.1 Foundation Dataset Construction
To enable effective compiler auto-tuning, we construct a high-quality dataset tailored for LLM training, derived from empirical compiler tuning workflows (e.g.. CFSAT (26] and enhanced with structural insights. Our process involves the following pipelines:
Raw LLVM IR Dataset Preparation. We aggregate multiple public datasets from CompilerGym 10 applying coreset-inspired sampling [2]] to ensure representational diversity. Programs with more than 10,000 IR instructions are excluded for computational feasibility. The training set comprises six uncurated datasets, while four curated benchmarks (chench-vl, chstone-vO, vO, mibench-vi. 1, and npb-v0 [6 served for out-of-distribution evaluation.
12 14 3

---

Synergy Pass Pair Identification. For each training program P. we derived its high-qualig, pass sequences Sp by collecting synergistic optimization pass pairs (A, O O satisfying the following criteria:
Sp ((A, B) (Count(ApplyCCC B)) Count(P))A
(I)
(Count(Apply
B)) < (ount(ApplyyP,.B))
where Apply(P, (B) denotes applying pass B to program P. and Count(-) the instruction count. A pair (4, B) is considered if B alone reduces the instruction count and A further reduces it when applied before B. These synergistic pairs indicate potential beneficial interactions between passes on a per-program basis.
Global Synergy Graph Construction. We aggregate these per-program synergistic pass pairs construct Global Synergy Graph, where nodes represent passes and edges denote pairs (A, B) found be synergistic for at least one program. This global perspective helps identify recurring beneficial pass interactions across the dataset.
Graph-Guidel Optimal Sequence Selection. Drawing from the Global Synergy Graph, we generate 100 candidate pass sequences (Scana(P)) per program using random walks that prioritize synergistic combinations. Each candidate Scand P)) is evaluated by its relative improvement the Oz baseline, quantified as:
OverOz(s, Count(Apply(P, Oz)) - Count(Apply(PP.s)
(2)
Count Count(Applye (Apply(P, Oz))
The sequence maximizing OverOz is selected as the optimal pass sequence PassSca This empirical, graph-informed method, validated against strong baseline, provides high-quality op- timization examples for training. The derived PassSe<tor (P) for each program then serves as the target solution used construct the Simulated LLM Thought and Action Trajectory for the SFT stage of our training pipeline.
Feature Extraction and Representation. To address LLM context limits, compress programs representations using AutoPhase's 56 statistical features [5]. capturing structural properties such as type distributions and control-flow properties. The extracted AutoPhase features, along with the program's 's initial instruction count, form the core components of the "question" part of our SFT samples, representing the program to the LLM.
3.2 Training Pipeline
propose two-stage training pipeline designed to progressively enhance LLMSs' capabilities in auto-tuning task. The first stage employs supervised fine-tuning (SFT) to establish fundamental reasoning and tool invocation skills. The second stage utilizes applies RL to refine responses based on performance feedback.
3.2.1 Stage 1: Supervised Fine-Tuning for Cold Start
The SFT stage addresses the cold-start challenge by teaching the LLM how to reason over program features and interact with external tools to construct optimized pass sequences. raining data is structured to emulate interactive problem-solving, following sooughttaatioon-feebaee loop that mirrors realistic tool use.
Core External Tools for SFT. Two key tools are integrated into the SFT process:
Evaluates a candidate pass sequence by computing its improvement over the standard Oz optimization, based on IR instruction count reduction.
instrcount:
f sind_best_par blest_pass_seeuencee sequence: Invoked when the initial candidate is ineffective (OverOz S 0), this tool searches for better sequence using the guided search algorithm.
SFT Sample Construction. SFT training samples are designed to simulate interactive problem- solving sessions, each comprising a structured input-output pair. The input represents the problem formulation, including the target program's 's AutoPhase features, clear task instruction (e.g., "find an IR-reducing pass sequence"), and the initial IR instruction count. The output models the L.M''s step- by-step reasoning and decision-makinn process, structured as a Chain-of- Thoouhh (CoT) trajectory.

---

Simulated LLM Thought and Action Trajectory: This part serves as the model's "answer," demon- strating step-by-step reasoning process. It begins with a <think> block where the LLM analyzes the optimal pass sequence PassSegop (P) and plans to verify its effectiveness using the instr count tool. A corresponding stool_calll taining execution status and the OverOz metric. The LLM then processes this feedback in another <think> block to decide whether to accept the candidate or explore alternatives. This entire reasoning flow, structured with tags like <think>, stool_call>, stool_response>, and <answer>, forms a complete and learnable SFT sample.
issued, followed by simulated <tool_ responsss con-
Training Prompt for Compiler-R1
Act a compiler optimization expert finding an optimal pass sequence for LLVM IR, aiming to reduce the total instruction count. The LLVM IR code is represented by statistical features. The initial statistical features are: cettophsseffatureeee (initial inst couss_____...._ooo Note: When calling the 'instrcount" and 'find_best Bacstt_pss_________ sssuuscce tools, use the exact filepath: [filepath)
Initial instruction count:
Your task is to: Provide and evaluate the Initial Candidate Pass Sequence using the instrcount tool determine its instruction count improvement compared to the default -Oz optimization. If the initial sequence provides positive improvement (improvement, _eeetttooerrrz O), recommend it as the final answer. If the initial se- quence does not provide a positive improvement (improvement_over) _eeeettttveereeeee <= 0), use the sead__eet__patt pee__paassseeuence sequence tool to search for a better sequence. the search finds a sequence with positíve improvement (improvements wetreeeent__erceettte percentage > O), recommend that sequence. If the search tool fails to find a sequence with positive improvement, recommend the default ['-0z'] sequence the safest option.
Present your reasoning step-by-step using <think></think> tags and tool interactions using <tool mool_cal>><//oo) call> and <tool_ <tool_respoolt responss><</tol retprooss>>ee>es>>>>>>ee structure, concluding with the final recommended sequence in an <answer></answe>> tag.
3.2.2 Stage 2: Reinforcement Learning for Policy Optimization
After the SFT-based initialization, we further optimize the LLM's policy through reinforcement learning (RL). This stage aims move beyond the imitative behavior learned via SFT, enabling the LLM--now acting as an agent-to autonomously discover more efficient optimization strategies through trial-and-error interactions with simulated compilation environment. The agent's actions include generating internal reasoning traces, determining when and how to invoke external tools, and producing the final optimization sequence.
Agent-Environment Interaction. The LLM interacts with the environment by emitting token sequences that correspond to high-level actions, such <think> statements, <tool_calll invoca- and final answers enclosed in <answer>
tions (c.g., instrcount, find_best___ wes__bessssss___eeeeeeeeeeeesese tags. The environment executes these ciooos--mmulatnn compilation tasks tool invocations- and returns feedback in the form of updated state information and scalar rewards.
Reward Design. To guide the learning process, we define composite reward function Rinal W Riormat + W. Ranswer where w and Wa control the relative importance of each component.
Format Reward: Peormat encourages well-structured and logically coherent trajectories. A full reward is given if the interaction trace (including thoughts, tool usage, and answer) adheres to a predefined valid path (e.g.. directly correct guess, or fallback via tool search); otherwise, no reward is assigned:
1.5, if the format is correct
Rjormat 0, if the format is incorrect
Answer Reward: Ranswer OverOrig quantifies the effectiveness of the final sequence Passsee in reducing the IR instruction count compared to the original unoptimized program Poriginal, where a scaling factor:
OverOrig Count(Porginn)) Count AAsssA(PPpPPPsSSeeepppo PassSerranser))
Count (aanttPongia))) Pongimal)
5

---

Unlike OverOz, which measures improvement over compiler baseline, OverOrig provides denser and more consistent reward signal, thereby facilitating stable RL training.
Learning Algorithm. We apply on-policy RL algorithms, specifically PPO 22 GRPO, and REINFORCE++ (RPP) [30], to update the LLM's policy. These algorithms are well-suited to LLMs, handling large action spaces and long-hone Long-horizon credit assignment. Training is performed iteratively using collected trajectories, progressively refining the policy to generate higher-quality optimization sequences while maintaining training stability.
4
Experiments
This section presents the experimental evaluation of Compiler-RI. We begin by describing the common experimental setup, followed by three main experiments: (1) a performance comparison between Compiler-RI and various baselines; (2) an analysis of factors affecting task success rate, which reflects effective environment interaction and highlights differences in sampling needs between interactive and non-interactive models; and (3) case study investigating the impact of input feature representations.
4.1 Experimental Setup
Datasets. Experiments are conducted using aggregated LLVM IR dataset. Training is performed on six CompilerGym datasets filtered to contain programs with fewer than 10k IR instructions. Evaluation is conducted seven test suites: blas, cbench, chstone, mibench, npb, opencv, and tensorflow.
Baselines. We compare Compiler-RI against classical compiler-tuning methods, including Open- Tuner [ GA [3]. TPE 4, RIO [7], CompTuuner 31 BOCA (6) and AutoPhase with PPO-LSTM and IPO-noL.STM variants.
Optimization Space and Robustness. All evaluated models operate within fixed optimization space comprising 124 LLVM 10.0.0 opt passes and the -Oz preset (125 total actions). Robust evaluation protocols ensure reliability: Compiler-RI defaults to opt -Oz if critical interaction failures occur (e.g., unparsable outputs or missing results). Non-interactive SFT models revert similarly upon invalid predictions. Compiler-RI evaluates all successful interactions, whereas SFT models select the best valid sequence from N inference attempts.
Evaluation Metries. We define the following metries:
Optimization Performance (OverOz%) Average percentage IR instruction reduction relative to opt -Oz (Experiment 1).
Success Rate (%): Proportion of test programs (335 total) for which models successfully execute the interaction protocol (e.g., correct tool invocation, output formatting). Applicable only to interactive models (Experiment 2).
Input Feature Impact (OverOrig): Average IR instruction reduction relative the original, unoptimized program to evaluate the impact of feature representations (Experiment 3).
Models and Training, All experiments were conducted on Intel Xeon Gold 6430 servers (128 cores, ITB RAM) with NVIDIA H100 GPUs (4x80GB HBM3).
Compiler-R1 (Interactive Framework): We Qwen2.5-Instruet models (1.5B, 3B, 7B). Training involves 800 supervised fine-tuning (SFT) samples for protocol initialization, followed by reinforcement learning (GRPO, PPO, RPP) 19k interactive episodes, updating over 40 steps.
SFT-Only Baselines: Non-interactive Qwen models (1.5B, 3B, 7B), fine-tuned to directly predict optimal pass sequences. Experiment 1 reports their best performance from N 40 inference attempts per program; Experiment 2 examines their sensitivity by varying N from to 50.
Additional Ablation Baselines (Experiment 2):

---

SFT-only (Cold-start): Qwen models (1.5B, 3B, 7B), trained only with the 800-sample interac- tion protocol initialization.
RL-only (No SFT): Qwen-1.5B model trained exclusively via GRPO on the full RL dataset without protocol initialization.
4.2 Experiment 1: Optimization Performance Comparison
This experiment compares the average OverOz% performance Compiler-RI variants with SFT- only models (which directly predict pass sequences) and traditional auto-tuning baselines. SFT-only models (SFT-Qwen- (.Been are evaluated using N = 40 inference attempts per program, reporting the best-performing result per instance. Results from traditional methods and AutoPhase variants (PPO-LSTM and PPO-no-LSTM) are also included. [ summarizes the findings.
Table I: Comparison of Optimization Performance (Average OverOz%) and Time Cost. For SFT-
Qwen models, OverOx% results are from N=40 inference attempts.
Method
blas (%)
cbench
chstone
mibench
nph (%)
opency (%)
tf (%)
Average (%)
Time (avg/prog)
(%)
(%)
(%)
Compiter-RI (Interactive)
GRPO-1.5B (Ours)
1.26
4.33
5,00
1.98
8.42
4.28
1.85
3.87
20
GRPO-3B (Ours)
3.14
3.77
2,97
3.84
15.42
3.77
2.96
5.12
GRPO-7B (Ours)
5.27
5.52
9.08
6.67
22.44
4.52
5.72
8.46
PPO-1.5B (Ours)
2.21
4.44
1.90
2.75
6.48
3.45
1.53
3.25
20
RPP-1.5B (Ours)
0.24
2.96
3.84
0.83
4.43
4.32
2.52
2.73
20
SFT-only (Direct Pass Prediction, N=40 attempts for OverOz5k
SFT-QWen-L.55 1.5B (N=40)
0.90
1.18
0.42
1.43
11.24
3.17
3.69
3.15
29
SFT-Qwen-3B (N=10)
0.78
0.62
0.88
0.68
7.00
2.62
1.15
1.96
SFT-Qwen-7B (N=40)
2.29
0.40
6.31
4.63
12.89
3.78
2.31
4.66
Traditional Autotuners
Opentuner
1.60
6.46
3.33
26.19
1.76
1.29
6.09
GA
-1.91
1.99
6,51
0.90
25.63
1.76
1.29
5.17
561
TPE
-2.44
0.97
7.60
0,20
24.62
1.46
1.23
4.83
812
RIO
-2.02
0.24
4,98
3.47
23.87
0.79
1.23
4.65
CompTuner
-0.45
22.99
BOCA
-2.36
-0.16
3.18
-0.69
22.87
1.13
1.22
3.60
2760
AutoPhase (PPO-LSTM)
-1.12
5.60
4.49
4.41
-4.67
-0.09
0.05
1.24
2.2
AutoPhase (PPO-noLSTM)
-4.77
-79.69
-80.90
-107.33
-76.69
-2.32
-0.76
-50.35
1.8
Compiler-Ri achieves consistent and substantial improvements across all benchmarks. Notably, GRPO-7B attains the highest average improvement of 8, 8.46% OverOz, significantly outperforming the strongest SFT-only baseline (SFT-Qwen-7B 4.66%) despite using fewer inference attempts and less runtime. Lower-scale variants (GRPO-1.5B and GRPO-3B) also show competitive performance, further demonstrating the efficacy of RL-based interaction.
Compared to traditional autotuners, Compiler-RI achieves comparable or better optimization quality with markedly lower runtime. GRPO-7B completes each program in 26 seconds, outperforming OpenTTuerr (200s), GA (561s), and CompTuner (9800s+). This efficiency makes Compiler-RI viable for practical deployment.
Although AutoPhase variants benefit from extremely fast inference ( (2s), their optimization quality notably limited. In contrast, Compiler-R1 balances both performance and efficiency: even the lightweight GRPO-1.55 1.55 achieves 3.87% in just 20 seconds, surpassing SFT-Qwen- (3.15% in 29s). These results affirm Compiler-R11 -R's strong potential in real-world compiler optimization pipelines.
4.3 Experiment 2: Task Success Rate and Interaction Efficiency
This experiment evaluates the ability of different models to follow the predefined interaction protocol, quantified by the task success rate on 335 test programs. This metric reflects the agent's capacity to comretlly execute the thought-tool-anse interaction. We examine the effects of RI. algorithms, model scale, training strategy (SFT-only, RI.- only, SFT+RL), and repetition penalty. Additionally, compare the interaction efficiency of
trajectory, serving proxy for effective environment

---

Compiler-Ra RI against SFT-only models under varying inference budgets N. Success rate results are summarized in Table while Figure visualizes the sensitivity of SFT-only models to N.
Table 2: Impact of Configuration and Repetition Penalty on Average Task Success Rate (%) on 335 test programs. Model group abbreviations: Comp.-RI Compiler-RR RI (SFT Cold-start RL), SFT (Abl.) = SFT-Qwen (Ablation), RL (Abl) = RL-only GRPO (Ablation).
Comp.-RI
SFT (AbL)
RL (AbL)
GRPO
PPO
RPP
GRPO
GRPO
SET
SET
SFT 7B
RI.
Rep. Pen.
1.5B
1.5B
1.5
3B
7B
1.5B
3B
1.5B
1.05
83,36
65.97
71.04
45.56
51.92
26.98
4.93
36.85
22.55
1.10
79.69
60.95
60.45
42.61
96.71
41.08
2.72
9.50
8.92
As shown in Table 2 Compiler-B R1 models consistently achieve high task success rates, with GRPO- 1.5B reaching 83.36% and GRPO-7B attaining 96.71%. These results highlight the effectiveness of RL in enabling agents to reliably execute the thought-tool-answe interaction protocol. This effective interaction caaaattionn-caaaatteriiiee by precise tool usage and sound decision-making- facilittssssss the robust optimization performance reported in Experiment through guided, single-traeectory search process.
trajectory mandated by the
Interestingly, GRPO-3B achieves a higher OverOz (5.12%) than GRPO-1.5B (3,87%) despite a much lower success rate (45.56%). This discrepancy illustrates that the success rate captures strict protocol compliance, whereas OverOz reflects end-to-end optimization effectiveness. GRPO- 3B occasionally circumvents protocol checks (e.g., instrcount validation) by directly invoking f hind_best____ee__eet
While this results in protocol violations and lower success scores, still yield high OverOz if effective sequences faithfully follows protocol, likely due to its smaller model capacity and greater adherence to SFT- initialized patterns. GRPO-7B strikes a favorable balance, combining protocol compliance and optimization quality through improved capacity and hyperparameter tuning.
discovered. In contrast, GRPO-1.5B more
SFT-only models, lacking interaction feedback, require extensive sampling to perform well. As shown in Figure 3 SFT-Qwen-7B achieves 14.778% OverOz at N but improves to 4.66% at N 40, demonstrating their reliance on stochastic inference rather than adaptive refinement. Ablation results also confirm that neither SFT-only nor RL-only models are sufficient alone: only the two-stage SFT+RL pipeline consistently achieves high interaction reliability. Additionally, tuning repetition penalty proves crucial for success rate stability and overall interaction behavior.
In summary, Compiler-RI exhibits strong interactive robustness across configurations and outperforms non-interactive models both reliability and efficiency, validating the importance of protocol- training for compiler optimization agents.
Table 3: Comparative performance (Avg. Max OverOrig) of AutoPhase features versus raw LLVM IR as input representations for direct com- piler pass sequence prediction using SFT with Qwen- models.
Avg OverOz%
SFT:4wen-113
SFT-Qwen-3B
Decoding / N
AutoPhase
LLVM IR
-5
SFT-Qwen-7B
GRPO-1.58
Greedy (N=1)
0.4314
0.4321
GRPO-3R
Sampled (N=1)
0.3609
0.3921
GRPO-7B
Sampled (N=10)
0.4620
0.4643
-15
Sampled (N=20)
0.4693
0,4713
20
30
40
Sampled (N=30)
0.4690
0.4690
N Attempts
Sampled (N=40)
0.4837
0.4795
Figure 3: Average OverOz% of SFT-only models N attempts. Dashed: GRPO Ref.
Sampled (N=50)
0.4819
0.4810
8

---

4.4 Experiment 3: Input Representation AutoPhase vs. Raw LLVM IR
This experiment evaluates the impact of input representation on the effectiveness of compiler pass sequence prediction. Specifically, compare AutoPhase features against raw LLVM IR inputs to SFT models based on Qwen-1.5B, using OverOrig as the evaluation metric.
Two Qwen-1.5B models were trained on over 10,000 LLVM IR functions, with a held-out test set of 100 functions, each containing 1,800-3,000 IR tokens. One model was trained using 56-dimensional AutoPhase features; the other used tokenized raw LLVM IR. We evaluate each model under two decoding strategies:
Greedy Decoding (N 1): Deterministic decoding using argmax each step.
Sampled Decoding (N 1-50): Nucleus sampling to generate up 50 diverse candidate sequences per input, reporting the best OverOrig among them.
T the results, showing that AutoPhase features yield performance closely comparable raw IR inputs.
Discussion: AutoPhase provides a compact and semantically rich input representation that performs comparably to raw LLVM IR in supervised pass sequence prediction. Under N 40 sampling, the AutoPhase-basse model achieves a OverOrig score of 0.4837, slightly higher than the 0.4795 attained using IR. This result suggests that AutoPhase captures sufficient structural and semantic information to guide effective optimization, while offering substantial advantages in terms of input length and interpretability.
One potential explanation is that AutoPhase abstracts away non-essential syntactic variations (e.g.. variable names, function ordering), which can obscure semantic similarity in raw IR and lead to inconsistent model behavior. Moreover, performance generally improves with larger N, saturating around N 30-50, which aligns with expectations under stochastic decoding. Notably, greedy decoding (N = 1) consistently outperforms single-shot sampling, highlighting the stability of deterministic inference for this task.
In addition, AutoPhase's fized-size, 56-dimensional feature representation may contribute to gen- eralization across programs of varying sizes. Because program size only influences the numerical values within this compact vector rather than the input dimensionality, the model less affected by context length limitations that typically constrain raw-code-based methods. By relying on structural and proportional characteristics (e.g., the ratio of branch to memory instructions) rather than absolute code length, the model may learn scale-invariant patterns that transfer across different program complexitics. While further empirical validation on larger program benchmarks necessary, this feature-based representation appears provide a practical foundation for scalable and potentially more robust learning.
In summary, AutoPhase serves strong, compact alternative to raw IR for LLM-base compiler tuning, particularly in scenarios constrained by context window length or requiring interpretable feature inputs.
4.5 Summary of Experimental Findings
Our experiments highlight Compiler-R1 effectiveness in compiler auto-tuning.
the GRPO-7B achieves strong optimization results, with an average OverOrig gain of 8.46% and a task success rate of 96.71%. %. These results confirm the benefits of our two-stage interactive RL framework, which enables efficient environment interaction and high-quality pass sequence discovery. In contrast, SFT-only models require extensive sampling (c.g.. N 40) reach comparable performance due the lack of feedback-driven refinement.
First, Compiler-R1-especiinn
Second, ablation studies demonstrate the necessity of two-stage training. The initial SFT phase is essential for learning interaction protocols that RL alone fails to acquire, while RL fine-tuning further improves efficiency and generalization.
Finally, AutoPhase features offer a compact and effective alternative to raw LLVM IR, delivering similar performance in sequence prediction with reduced input length and better robustness to irrelevant syntactic variations.

---

Together, these results position Compiler-Ri. Minp tool-augmented, two-stage LLM framework as scalable and efficient solution for modern compiler optimization tasks.
5 Limitations and Future Work
This work examines the use of reinforcement learning to enhance LLM-based compiler auto-tuning While the results show potential, several limitations remain and suggest directions for future study.
Instruction Count as a Proxy for Performance We use the IR instruction count as the main optimization target. Although this is a common proxy for code size reduction, it does not always correlate with runtime improvements. Cache effects, instruction-level parallelism, and hardware- specific behavior can cause shorter instruction sequences to run slower. Future work could explore integrating direct metrics, such as execution time or cycle counts, into reward function. This would increase measurement cost but better align the optimization objective with real performance.
Semantic Information Loss in Program Representation We adopt the 56 statistical features from AutoPhase to represent programs. This compact form fits within the LLM's context but abstracts away structural and semantic details, such control and data flow. This loss may limit the model's 's ability to capture complex optimization patterns. Future work could explore richer representations that preserve semantics while keeping manageable size, such graph encodings embeddings.
Reliance External Tools Compiler-RI relies external tools evaluate and search for optimization pass sequences. While this design is practical and aligns with how compilers are typically used, it also means that the model primarily learns to follow a procedural protocol rather than developing a deeper understanding of why a given sequence effective. Future work could explore curriculum learning related staged training strategies, where the agent is exposed to progressively more complex optimization tasks. This might help improve its general reasoning ability and reduce reliance on external evaluation tools.
The compiler ecosystem evolves over time, with op- timization passes being added, removed, or modified between releases. The current dataset and training setup are tied fixed compiler version. When applied different version, the framework currently defaults to using the intersection of known passes, which may be conservative. Improving version adaptability remains an open problem. Possible directions include developing methods for automatic adaptation to unseen compiler versions, such as zero-shot or few-shot generalization to new passes, or incremental updates to datasets and reward functions maintain compatibility with evolving compiler toolchains.
Cross-Vert Version Compiler Compatibility
Conclusion
We present Compiler-R1, a tool-augmented framework for compiler auto-tuning with LLMs, ad- dressing the lack of comain-specifie training data and the limited adaptability of non-interactive methods. Compiler-R1 introduces high-quality reasoning dataset combining thain-of-thouuht and tool-use paradigms, and adopts two-stage training pipeline: SFT for initialization, followed by RL for outeome-driven policy optimization. Experiments show that Compiler-R1, particularly GRPO-7B variant, achieves average 8.46% IR instruction reduction and 96.71% % task success rate, outperforming SFT-only baselines and rivaling traditional autotuners with greater efficiency. Ablation studies highlight the importance of combining SFT and RL. Our results demonstrate the potential of LLM-based agents trained with reinforcement learning approaches in complex, interactive compiler optimization tasks.
Compiler-R1 showcases the profound potential of tool-augmented LLM agents trained via RL to revolutionize compiler auto-tuning, validating the effectiveness of an agentic LLM+RL paradigm for complex compiler auto-tuning tasks. Future research directions include leveraging LLMs to improve the interpretability of auto-tuning processes, further developing LLM-RI. agents to explicitly understand and exploit synergies between compiler passes, and exploring the efficacy of interactive, turn-by-ture turt pass application dialogues that could simulate traditional RL auto-tuning paradigms.
10

---

7 Acknowledgements
This work is supported by the National Key R&D Program of China, Grant No. No.2022YFB440144
References
[1] Jason Ansel, Shoaib Kamil, Kalyan Veeramachaneni, Jonathan Ragan-Kelley, Kelley, Jeffrey Bosboom, Una-May O' O'Reilly, and Saman Amarasinghe. Opentuner: An extensible framework for program autotuning. Proceedings of the 23rd international conference on Parallel architectures and compilation, pages 303-316, 2014.
[2] Amir H Ashouri, William Killian, John Cavazos, Gianluca Palermo, and Cristina Silvano. survey on compiler autotuning using machine learning, ACM Computing Surveys (CSUR), 51(5):1-42, 2018.
[3] David H Bailey, Eric Barszez, John Barton, David Browning, Robert Carter, Leonardo Dagum, Rod A Fatoohi, Paul O Frederickson, Thomas A Lasinski, Rob S Schreiber, al. The nas parallel benchmarks. The International Journal of Supercomputing Applications,
5(3):63-73, 73, 1991.
[4] James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper- parameter optimization. Advances in neural information processing systems, 24, 2011.
[5] François Bodin, Toru Kisuki, Peter Knijnenburg, Mike O'Boyle, and Erven Rohou. Iterative compilation in a non-linear optimisation space. In Workshop on profile and feedback-directee compilation, 1998.
[6] Junjie Chen, Ningxin Xu, Peiqi Chen, and Hongyu Zhang. Efficient compiler autotuning via bayesian optimization. In 2021 IEEE/ACM 43rd International Conference Software Engineering (ICSE), pages 1198-1209, IEEE, 2021.
[7] Yang Chen, Shuangde Fang, Yuanjie Huang, Lieven Eeckhout, Grigori Fursin, Olivier Temam, and Chengyong Wu. Deconstructing iterative optimization. ACM Transactions on Architecture and Code Optimization (TACO), 9(3):1-30, 2012.
[8] Chris Cummins, Volker Seeker, Dejan Grubisic, Mostafa Elhoushi, Youwei Liang, Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Kim Hazelwood, Gabriel Synnaeve, et al. Large language models for compiler optimization. arXiv preprint arXiv:2309.07062) 2023.
[9] Chris Cummins, Volker Seeker, Dejan Grubisic, Baptiste Roziere, Jonas Gehring, Gabriel Synnaeve, and Hugh Leather. Meta large language model compiler: Foundation models of compiler optimization. arXiv preprint arXiv:2407.02524 2024.
[10] Chris Cummins, Bram Wasti, Jiadong Guo, Brandon Cui, Jason Ansel, Sahir Gomez, Somya Jain, Jia Liu, Olivier Teytaud, Benoit Steiner, al. Compilergym: Robust, performant compiler optimization environments for ai research. In 2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), pages 92-105. IEEE, 2022.
[11] DeepSeck-aa et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. 2025.
[12] Grigori Fursin. Collective tuning initiative: automating and accelerating development and optimization of computing systems. In GCC Developers' Summit, 2009.
[13] Unai Garciarena and Roberto Santana. Evolutionary optimization of compiler flag selection by learning and exploiting flags interactions. In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, pages 1 1159-11666 2016.
[14] Mathew R Guthaus, Jeffrey Ringenberg, Dan Ernst, Todd M Austin, Trevor Mudge, and Richard B Brown. Mibench: A free, commercially representative embedded benchmark suite. In Proceedings of the fourth annual IEEE international workshop on workload characterization. WWC-4 (Cat. No. 01 EX538), pages 14. IEEE, 2001.

---

[15] Ameer Haj-Ali, Qijing Jenny Huang. John Xiang, William Moses, Krste Asanovic, John Wawzzynek, and Ion Stoica. Autophase: Juggling his phase orderings in random forests with deep reinforcement learning. Proceedings of Machine Learning and Systems, 2:70-81, 2020.
[16] Yuko Hara, Hiroyuki Tomiyama, Shinya Honda, Hiroaki Takada, and Katsuya Ishii. Chstone: A benchmark program suite for practical c-based high-level synthesis. In 2008 IEEE International Symposium Circuits and Systems (ISCAS), pages 1192-1199 1195. IEEE, 2008.
[17] Davide Italiano and Chris Cummins. Finding missed code size optimizations in compilers using lmms. arXiv preprint arXiv:2501.00655 2024.
[18] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, al. Openai ol system card. arXiv preprint arXiv:2412. 2024.
[19] Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Za- mani, and Jiawei Han. Search-rl: Training lims to reason and leverage search engines with reinforcement learning, 2025.
[20] Chris Lattner and Vikram Adve. Llvm: A compilation framework for lifelong program analysis transformation. In International symposium on code generation and optimization, 2004. CGO 2004., pages 76. IEEE, 2004.
[21] Youwei Liang, Kevin Stone, Ali Shameli, Chris Cummins, Mostafa Elhoushi, Jiadong Guo, Benoit Steiner, Xiaomeng Yang, Pengtao Xie, Hugh James Leather, et al. Learning compiler pass orders using coreset and normalized value prediction. In International Conference on Machine Learning, pages 20746-20762. PMLR, 2023.
[22] Hongyu Lin, Yuchen Li, Haoran Luo, Kaichun Yao, Libo Zhang, Mingjie Xing, and Yanjun Wu. Os-rl: Agentic operating system kernel tuning with reinforcement learning, 2025.
[23] Jiawei Liu and Lingming Zhang. Code-rl: Reproducing rl for code with reliable rewards. arXiv preprint arXiv:2503. 18470, 3, 2025.
[24] Haoran Luo, Haihong E, Guanting Chen, Qika Lin, Yikai Guo, Fangzhi Xu, Zemin Kuang, Meina Song, Xiaobao Wu, Yifan Zhu, and Luu Anh Tuan. Graph-r1: Towards agentic graphrag framework via end-to-end reinforcement learning, 2025.
[25] Haoran Luo, Haihong E, Yikai Guo, Qika Lin, Xiaobao Wu, Xinyu Mu, Wenhao Liu, Meina Song, Yifan Zhu, and Luu Anh Tuan. Kbqa-ol: Agentic knowledge base question answering with monte carlo tree search, 2025.
[26) Haolin Pan, Yuanyu Wei, Mingjie Xing, Yanjun Wu, and Chen Zhao. Towards efficient compiler auto-tuning:
Leveraging synergistic search spaces. In Proceedings of the 23rd ACM/IEEE International Symposium on Code Generation and Optimization, pages 614-627, 2025.
[27] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint vrXiv:1707.06411 2017.
(28) Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Y Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.
[29] Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong. Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing Ilm reasoning with tule-based reinforcement learning, 2025.
[30] Junzi Zhang, Jongho Kim, Brendan O'Donoghue, and Stephen Boyd. Sample efficient reinforce- ment learning with reinforce. In Proceedings of the AAAI conference artificial intelligence, volume 35, pages 10887-10895, 2021.
[31] Mingxuan Zhu, Dan Hao, and Junjie Chen. Compiler autotuning through multiple-phan learning. ACM Transactions Software Engineering and Methodology, 33(4):1-38, 2024.
12

---

Appendix
A AutoPhase Feature Set
As referenced in Feature Extraction and Representation, our framework utilizes the 56 statistical features from AutoPhase [3] represent programs compactly for the LLM. These features capture
various aspects of the program's 's static structure and instruction mix. Table 4 provides complete list of these features.
Table 4: List 56 AutoPhase Features Utilized.
Index
Feature Description
Index
Feature Description
BBs: total phi args a
28
Number of And insts
BBs: total phi args [1,5]
BBs: instruction count [15,500]
2
BBs: count with predecessor
30
BBs: instruction count < 15
3
BBs: count with predecessor and successor
31
Number of BitCast insts
BBs: count with predecessor and successors
32
Number of Br insts
BBs: count with successor
33
Number of Call insts
BBs: count with 2 predecessors
Number GetElementPir insts
7
BBs count with 2 predecessors and I successor
35
Number of ICmp insts
8
BBs: count with 2 predecessors and successors
36
Number of LShr insts
BBs: count with 2 successors
37
Number of Load insts
BBs: count with >2 predecessors
Number Mul insts
11
BBs: Phi node count in range (0,3) per BB
39
Number of Or insts Number of PHI insts
12
BBs: count with more than 3 Phi nodes
40
13
BBs: count with no Phi nodes
41
Number of Ret insts Number of SExt insts Number Select insts
Number Phi-nodes beginning
Number branches
43
16
Number of calls that return an int
44
Number of Shl insts Number of Store insts
17
Number of critical edges
45
18
Number of edges
46
Number of Sub insts Number Trunc insts
Occurrences 32-bit integer constants
20
Occurrences of 64-bit integer constants
48
Number of Xor insts Number of ZExt insts
21
Occurrences of constant 0
49
22
Occurrences of constant
50
Number of basic blocks
23
Number of unconditional branches
Number of instructions (all types)
Binary operations with constant operand
Number memory instructions
25
Number of AShr insts
53
Number of non-external functions
26
Number of Add insts
54
Total arguments to Phi nodes
27
Number of Alloca insts
55
Number of Unary operations
B Synergy Pass Pair Algorithm Details
Algorithm iimplements the synergy pass pair identification methodology described Section 3.1 The procedure operates as follows:
C LLVM Optimization Passes
As detailed in Section 4.1 our Compiler-Ri framework and baseline models operate within an action space comprising 124 individual LLVM 10.0.0 opt transformation passes, augmented with the -0z. This results in a total 125 distinct actions available to the optimization agent. Vable(3jenumerater these passes and the -Oz along with their corresponding indices used within system. These flags can typically be obtained from the CompilerGym LLVM environment.
13

---

Algorithm Finding Synergy Pass Pairs for Program
Require: Program P, set of compiler transformation passes O
Ensure: Set of chained synergy pass pairs S +-
1: Compute Vp, the IR instruction count of the P
2: for each optimization pass B O do
3: Apply pass B to P and obtain the IR instruction count Vp
4: if Vpa < Vp then
5:
for each optimization pass A E O do
6:
Apply pass followed by pass to and obtain the IR instruction count Vr
compared to Oo
7:
V Vpg then S S SU (AA, B))
8:
9: end if
10: end for
11: end
12:
end for
13:
return
Table 5: List Utilized LLVM Compiler Passes and -Oz with Corresponding Indices.
Index
Flag
Index
Flag
Index
Flag
aid-discriminators
42
globalsplit
84
lower-expect
adce
43
guann-widening
85
swwer-guard-intrinsse
aggressive-instcomee
hotcoldsplit
86
lowerinvoke
alignment-front wawpieent--oo---ssseeeee commasssumttios
ipconstprop
87
lower-matri-iintiniics
always-inline
ipscop
lowerswitch
argpromotion
indvars
lower-widenabll-condiie
attributor
48
irce
90
memepyopt
barrier
49
infer-address- spaces
91
mergefunc
bdee
50
inferattrs
92
mergeiemps
treak-criteddge
51
93
midst-motion
10
simplifyefg
52
instsimplify
94
sancov
callsite-spliting
53
instcombine
95
name-amon-glohals
called-valte-propaaatie
instnamer
96
nary-rcassociate
canonicalize-aliaes
jump-threading
newgyn
consthoist
56
lessa
pge-memop-opt
constmerge
57
liem
purtial-inliner
16
consiprup
58
libcalls-shrinkwrag
100
17
coro-cleanup
59
101
post-inlin-ce-instrumee
18
coro-carly
60
hop-data-prefetchh
102
functionattrs
19
corn-elide
61
loop-deletion
103
mem2reg
20
coro-split
62
loop-distribtte
104
prune-ch
21
correlated-propagaaiion
63
loop-fusion
105
reassociate
cross-dso-cfi
hop-guard-wideningg
106
selundant-dbb-insseee
deadargelim
loop-idiom
107
rpo-funciinaatrs
dee
coop-intssimplffy
108
rewrite-statepon
die 26
67
soppiterrchanee
109
sccp
dse
68
loa-lood-elim
110
slp veevverorr
27
reg2mem
69
nop-predication
111
sroa
28
div-rem-pairs
70
loop-reroll
112
scalarizer
29
carly-cse-memssa
71
loop-rotate
113
30
carly-cse
72
loop-simplifyctg
114
simple-loo-unswitch
31
elim-avail-extem
73
loup-simplify
115
sink
co-instrument
loop-sink
116
speculative-cxceution
flattenefg
loop-reduce
117
floatZint
loop-unroll-and-jng
118
trip-ddeadprooooyyee
35
forceatirs
77
loup-unroll
119
atip-debug-lecllan
36
inline
78
loop-unswitch
120
strip-nondebug
37
grs-pticoo--rottilng
79
loop-vectorize
121
strip
38
gon-hoist
80
loop-versioning- liem
122
tailcallelim
39
gvn
81
loop-versioning
123
mergereturn
40
globaldce
82
lowerutomic
124
-Oz
globalopt
83
Dnwer-constantinnriiniit
14

---

D Illustrative SFT Answer Trajectories
D.1
Answer 1: Initial Pass Sequence Outperforms Oz
<think>
[Initial Pass Sequence Analysial
provided autophase features. try following pass sequence: [*--dse", co-rrelatdd prapadd-ppaagiionn....
First, I'11 use instrcount tool check this sequence provides improvement -0z optinization.
If iaprovenent_over_co
0, I'll use this sequence as ny ansver.
If improvanent.over.co <= 0, 1'll call find_bast.pasee bind_bast.pass_aquen tool find better seq. </thinx>
<tool_call>
"""am"":"inatronn"
"aaarcoonnt,"rrgnnnttttt "argunents ("" ( eexaplee..
coptinization__lagae"" ("--dd=e""""""" correlated www.olaateed-roppattiio.. propagation', ....]})
</tocl_csll>
<taol_response>
'statua": "success",
"iaprovenent, 0.089965511241339311
</tool_response>
<think>
[Result Analysis]
0.96896551724137988 Since this value is positive, the pass sequence provides better optinization than 02 pass sequence as ny ansuer
instrcount_ tool reporta improvement,
I'11
</think>
<anever>
"=.'do",""..corre" pww.paatddddrrpppaiio......
</ansear>
D.2 Answer 2: Initial Sequence Underperforms, FBPS Finds Better Sequence
<chink>
[Initial Pass Sequence Analysis]
provided autophase features, I'l1 try the following pass sequence: f---jump-threadiin "t-nn""",""-="""" "--_rr""""",""".......
Based
instrcount over -0z optinization.
I'll
sequence provides improvement
If inprovement oveor.
I'1l use this sequence as ny anaver.
If inprovenent_over.cg < 0, I'll call find_beest.pass_eequeee tool find better </think>
<taol_call>
""aat":""nntrcouan"" "argunents" ; ""allppat""""xxapp"e"p 11",
"optinization_flagas ("-ag=s"""--"e""""" -jupp-tbraading''
-.-mmenree" ....]}}
</tool_call> <tool_.esponee>
'status": "success",
"iaprovanent_over.
40.0426540384351198
</taol__esponae>
<think>
[Finding Better Pass Sequence]
The
sequence did provide improvement =zz.
Calling band__bett.pass_seeeeee
tool to find better sequence.
</thinx>
<tool_call>
("nane":"find_" siinddbest_paaae aequence", ""apuunce"""rrgumeeee""""""" "aargmeett"" ("""iiepat"" "exanplee..
</tool_call>
<tool_response>
"atatus": "success", -xppectt --tt
"appetreaat.pprcenttge::: 1.9
</tool_response>
<think>
[Pinal Decision Found Improved Saquence]
Found pass aequence with positive inprovement: 1.9.
Using sequence
final
</think>
(*--lover-eppect "a.peett"""--aete"" r==ressscccist"""
</anseer>
15

---

[Recoznended paes sequence goee D.3 Answer 3: Initial Sequence and FBPS Underperform, Fallback to Oz
<think>
(Initial Sequence Analysis]
Based the provided autophase features, 1'11 try the following sequence: ("--nerorreuurn' stnpllylttyye ,...]
First, I'1l the instrcount toel check if this provides improvenent ever -02 optinization.
improvenent_ over,o, If smproveneett_veer.00 <= 0, I'll call find_ tool to find better </think>
sequence ny
<taol_call>
("name":" "iattr=""nstrrouutt"" "argunente (""illepath" : "exampee3.11 11", "optinization_flags="" mergerrrrurr""" "inpesettur....--eepptttteeeee inpplfycdig......
</tocl_call>> <tool_responee>
'status":
"inprovonent.over.com -0. 00.05664102564102666
</tool_response>
<think>
[Finding
Sequence]
The previcus sequence did not provide improvement over -0z.
Calling pand_beat.pass_sqquee toal to sequence.
</think>
<toel_call>
""tilaatttttlltt""""""""""a""" )1*))
</tocl_cell> <tool_response>
'atatua": "success",
post__pas__equeee P=-siipliffccg', "--men2reg", "aprorenent_pereentte"" 0.0
</spol__espone>>
[Final Decision Fallback to -02]
After attempting with find_ best_best_paas aequueee tool, no sequence with positive improvement found.
Falling back to the standard -0z optimization as the final ansver </think>
<ansver>
['-0z'] </anaver>
E Experiment Prompt Templates
E.1 AutoPhase Features Model
PROMPT:
Act as compiler optinization expert rinding an optinal paas sequence for LLVN IR. aiming reduce the total instruction count.
The LLVX IR cada is represented by autophase featurea, the initial autophase features are: (Autophase Features)
Initiel instruction
(initisl_inet cout_
AKSWER TEMPLATE:
<ansver>
</anaver>
E.2 Raw LLVM IR Model
Act compiler optinization expert finding optinal for LLVN IR, aining to reduce the total instruction count.
The LLVX IR code is: (ll_code)
Initial instruction count for this code: (initial connt)ount)
ANSWER TEMPLATE:
<ansver>
[Reconnended paes sequence goes here]
</ansver>
16

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the
relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
general, answering
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the contributions of Compiler-RR R1, including the construction of high-quality reasoning dataset and novel two-stage RL training pipeline. The claims are supported by experimental results in Section 4.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
17

---

Justification:
The paper discusses limitations in Section 5, such as the need for further
research on interpretability and the exploration synergies between compiler passes.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper focuses on empirical methods and does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Section 4 provides detailed experimental setups, including datasets, baselines, and evaluation metrics. The code and datasets are publicly available (link provided in the abstract).
18

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The abstract mentions that the code and datasets are publicly available at https://anonymous http:///ooonmmooo..ooooo....e/ie Hoppensciennecr/Coomllee RICCCCC
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ phiips.cc//ublic//uuu
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
19

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 4.1 describes the datasets, baselines, optimization space, and evaluation metrics in detail.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Table reports average performance metrics (OverOz%) across multiple benchmarks, and Section 4.2 discusses task success rates with statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section 4.1 mentions the use of the computer resources.
Guidelines:
20

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:///waapppccccooo/o/o/cccaaa
Answer: [Yes]
Justification: The research conforms the NeurlPS Code of Ethics, as it involves compiler optimization without ethical risks.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e-g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Section discusses potential positive impacts (e.g., advancing compiler optimization) and acknowledges the need for further research on interpretability. which could mitigate unintended negative effects.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
21

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The research does not involve high-risk models or datasets requiring safe- guards.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes)
The paper cites datasets like CompilerGym and AutoPhase, and the public release of code/data implies adherence to licensing terms.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces the Compiler-Ri dataset and framework, documented in Sections and 4, with public release details in the abstract.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
22

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The research does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research does involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration LLM usage
Question: Does the paper describe the usage of LLMSs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The LLM was used only for writing or editing purposes and does not affect the core methodology or scientific rigor of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer LLM policy for what should or should not be described.
23


Reviews:
[{"review_id": "M0tcInCUkA", "reviewer": "Reviewer_Wbgx", "summary": "The paper proposes a LLM-based algorithmic framework (Compiler-R1) for program optimization. The primary contributions of the paper are algorithmic and empirical. Specifically, the system uses a two-stage training pipeline (SFT followed by RL) to train a LLM to generate an optimized sequence of compiler operations (passes) by leveraging standard compiler optimization and program analysis tools in order to reduce the instruction count of the compiled program. The paper proposes a method to construct a dataset for the Stage 1 (SFT) pipeline. Experiments on CompilerGym tasks show that Compiler-R1 is reasonably effective at reducing instruction size while running faster than strong baselines.", "questions": "1. Please enumerate the novel contributions of the paper wrt the dataset, algorithmic ideas and empirical (instruction count) improvements over previous SOTA on CompilerGym.\n\n2. How well does Compiler-R1 generalize to programs outside its training distribution? E.g., large programs.", "limitations": "- Somewhat. The paper could do a better job of discussing ways in which the empirical results might be invalid or performance might drop. Examples include cases instances where Compiler-R1 might fail to generalize due to OOD inputs as well as the potential for test dataset leakage during training of the base LLM.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The paper studies an interesting and important problem. Automated improvements to program compilation tasks is an important area and progress here is likely to have significant impact.\n\n- The paper is well written. Key ideas are explained clearly and I found it easy to follow. The paper appendix includes a good amount of empirical details, which, in conjunction with released code and datasets, will aid reproducibility.\n\n- The proposed algorithmic approach is intuitively clear. The two-stage pipeline (SFT followed by RL with GRPO) using AutoPhase features makes a lot of sense. The dataset construction approach for the SFT stage seems like a standalone contribution in itself (as far as I can tell). While the algorithmic novelty is a bit low in that the components are all off-the-shelf, the combination of existing ideas is novel, to my knowledge. (The paper could do a better job of making its novel contributions clearer.)\n\n- The experiments show that Compiler-R1 is generally effective at reducing instruction counts while running faster than most baselines. It strikes a nice balance between AutoPhase (higher instruction count but fast) and Opentuner (lower instruction count but slower). The experiments are richly detailed and contain a good amount of insight into the underlying factors driving performance gains. In particular, the importance of RL training and scaling model size comes through clearly.\n\n**Weaknesses**\n\n- The paper could do a better job of describing its novelty and contributions relative to the larger body of work on compiler optimization. For example, how novel is the induced dataset generated for the SFT training to the community? How important or impactful is the improvement over traditional autotuners in Table 1 (8.46 vs 6.09)? A more detailed discussion of the novelty and impact aspects of the dataset and empirical results (perhaps in the appendix) would significantly improve the paper.\n\n- The paper would be strengthened with a detailed discussion of Compiler-R1's generalization to programs outside CompilerGym (i.e., OOD). For example, what would happen on larger programs (> 10K instruction count which were filtered out) or those significantly different from the tasks in CompilerGym? Put differently, in what scenarios might Compiler-R1 fail?\n\n- The above are relatively minor. Overall, I didn't find any major issues in the paper and I'm open to increasing my score with a better understanding of the empirical contributions.", "was_revised": true, "final_justification": "I thank the authors for their detailed response, which have alleviated my main concerns about the approach. After reading the other reviews and comments, I continue to recommend acceptance of the paper, maintaining my current score.", "rebuttal": "Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.\n\n---\n\n### W1 & Q1: Contributions of the paper wrt the dataset, algorithmic ideas and empirical improvements over previous SOTA on CompilerGym.\n\nThank you for this question. We are happy to clarify our novel contributions:\n\n* **On the Dataset**: We present the **first high-quality, reasoning-centric dataset** tailored specifically for **compiler auto-tuning tasks**. Our key innovation lies in the systematic methodology used for constructing the dataset:\n\n  - We curate a **global knowledge graph** by identifying **synergistic pass pairs**, which then serve to guide the generation of optimal pass sequences.\n  - We introduce a **Chain-of-Thought (CoT) framework**, teaching the model not only what the answer is but also how to reason and use tools effectively to derive it.\n\n  This structured approach addresses the long-standing lack of reasoning data in complex, tool-intensive domains like compiler auto-tuning.\n\n* **On Algorithmic Ideas**: The core innovation of our work is the introduction of **Compiler-R1**, **the first LLM-based agent specifically designed for compiler auto-tuning**. While the two-stage approach of SFT followed by RL is not novel in isolation, our **application of this combination within the domain of compiler optimization** is the novelty. Traditional approaches to compiler pass optimization typically rely on **heuristics** or **static strategies**, whereas our agent uses **interactive reinforcement learning** to autonomously refine optimization strategies, thus moving beyond imitation learning. This makes **Compiler-R1 the first interactive agent** in this domain, marking a significant leap forward.\n\n- **On Empirical Improvements**: Our results demonstrate **state-of-the-art performance** for **LLM-based auto-tuning**. Specifically:\n  - **Compiler-R1** (GRPO-7B) achieves an **8.46% average reduction** in IR instruction count compared to the **opt -Oz** baseline. This performance **significantly surpasses** the **strongest SFT-only baseline** (4.66%) and **traditional autotuners** like **OpenTuner** (6.09%).\n  - We also highlight **the computational efficiency** of our framework: Compiler-R1 completes the task in **~26 seconds**, much faster than traditional methods that can take over **200 seconds**.\n\nThese contributions not only stablish **Compiler-R1** as a leading framework in **LLM-driven compiler auto-tuning**, but also provide a blueprint for creating **interactive agents** capable of solving complex, tool-assisted tasks in diverse domains.\n\n------\n\n\n### W2 & Q2: How well does Compiler-R1 generalize to programs outside its training distribution?\n\nThank you for this excellent question, which addresses a critical aspect of our work. Our design choices have been made to ensure Compiler-R1 is inherently more generalizable, even to large programs, and we believe our approach offers strong potential for generalization outside of its training distribution.\n\n#### **1. Rationale for Focusing on Programs Under 10k Instructions**\n\nFirst, it’s important to clarify our decision to focus on programs with fewer than 10k instructions. This was primarily driven by the need for **a fair and feasible comparison** with traditional auto-tuning baselines such as **BOCA** and **CompTuner**. These methods often require extensive time to tune even a single moderate-sized program, and the time cost becomes prohibitive when scaling to larger programs. Limiting the size to **<10k instructions** allowed us to perform a **rigorous and direct comparison** across all methods, ensuring the **comparability** of results. However, we acknowledge the importance of **testing on larger programs** and are exploring this as a next step.\n\n#### **2. Generalization to Larger Programs with AutoPhase Features**\n\nSecond, **Compiler-R1** is uniquely positioned to handle larger programs, thanks to its use of a **fixed-size, 56-dimensional AutoPhase feature vector**. This design choice has profound implications for **generalization** and scalability, offering two key advantages:\n\n- **Decoupling Program Size from Input Size**: Whether a program has **1,000 instructions** or **1,000,000 instructions**, it is represented by the same compact **56-element vector**. The program’s size only affects the numerical values within this vector, meaning the model does not rely on processing raw code directly. This decoupling **bypasses context window limitations** that typically hinder models based on raw code representation, enabling **better scalability**.\n- **Encouraging Scale-Invariant Learning**: The model learns to make decisions based on **structural patterns** and **proportional relationships** reflected in the features, such as the **ratio of branch instructions to memory instructions**. These structural signatures tend to be shared across programs of different sizes. By focusing on the **\"shape\"** of the program rather than its absolute length, the model can **learn generalizable optimization heuristics**, which makes it more robust to varying program sizes and complexities.\n\nIn essence, the task of generalization is simplified from understanding **arbitrary-length code** to **mapping points within a structured, low-dimensional feature space**. This approach inherently promotes **generalizable and scalable learning**, and while we acknowledge the need for **empirical validation** on larger program benchmarks, we are confident that this feature representation will provide a **strong foundation** for **out-of-distribution robustness**.\n\nWe will add this clarification to our paper to emphasize how our **feature-based design** enables **generalization** and why we believe it positions **Compiler-R1** well for scalability to larger programs.\n\n---\n\nWe hope this response addresses your concerns and demonstrates the novelty and robustness of **Compiler-R1**. We truly value your feedback and will incorporate your suggestions to further improve our paper. Thank you once again for your careful and constructive review!\n\n---"}, {"review_id": "AFnEDk1PZg", "reviewer": "Reviewer_9h7w", "summary": "This paper introduces Compiler-R1, a tool assisted system that uses RL to learn better compiler optimization sequences. Instead of relying on fixed optimization strategies, the authors frame the problem as a decision-making task, where an agent selects a sequence of optimization passes tailored to a specific program. The contributions include construction of a high-quality reasoning dataset and a novel two-stage RL pipeline which is first trained with SFT and then improved using tool assisted feedback based on how well the resulting code is optimized in terms of reducing the number of intermediate instructions. The authors evaluate Compiler-R1 on a diverse benchmark of programs and show that it consistently outperforms traditional methods and existing tools in reducing the number of IR. They also study how different ways of representing the input program affect the agent’s performance and perform ablations on how it is important to do both SFT and RL.", "questions": "1. The environment relies on external tools like instrcount and find_best_pass_sequence, which may introduce non-determinism or fragility (e.g., if tool crashes or gives misleading feedback) Please describe how errors or noise in tool execution are handled. Are they dropped, or is the agent penalized?\n2. SFT models are evaluated with N=40 samples, whereas RL agents use single interaction traces. Could you include a comparison with matched performance (e.g., how many SFT samples are needed to match GRPO-1.5B’s performance),or evaluate both SFT and RL under the same inference budge.\n3. The study focuses on LLVM 10 and specific passes. While sufficient for benchmarking, this may limit real-world relevance. Can the system adapt to newer LLVM versions (e.g., v17)?\n4. Currently, all programs are treated equally during training. This could slow down learning or introduce instability early in RL training.\nHave the authors considered curriculum learning, e.g., starting with short or low-complexity programs? Even a brief analysis of performance across complexity levels could reveal useful insights here.\n5. The paper focuses solely on IR instruction count as a proxy for optimization. it's unclear how well this correlates with runtime performance or power/energy efficiency.", "limitations": "The authors have briefly discussed the potential negative impacts of their work including how they can use LLMs to improve interpretability of auto tuning processes and further develop agents to exploit synergies between compiler passes. They have not gone into the details of the societal impact of this work, especially if there is a correctness aspect of discovering the compiler passes, if not properly sandboxed or verified, this could lead to vulnerabilities in deployed software. failures in learned optimization sequences (e.g., those introducing subtle bugs or non-deterministic behavior) could lead to safety risks especially in critical domains.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- As noted by the authors, Compiler-R1 is the first framework to integrate LLMs with RL for automatically tuning compiler pass sequences to minimize LLVM IR instruction count. This makes it an interesting application of tool assisted SFT+RL reasoning in a new domain. The authors also built a reasoning dataset designed for compiler pass optimization. \n- The system delivers practical improvements: it reduces LLVM IR instruction count by about 8.5% compared to the standard -Oz optimization. they evaluate on a bunch of different datasets in Compiler Gym and against classical compiler-tuning methods, including Open Tuner, GA, TPE, RIO, Comp Tuner, BOCA, and Auto Phase, showing robustness.\n\nWeaknesses:\n- There is not much novelty in terms of training, this paper is mainly applying standard SFT+RL training techniques to a new domain. It would be interesting to see more exploration of tree search, memory-augmented, or curriculum learning for compiler pass discovery.\n- Reward design is not deep: The paper optimizes only for IR instruction count reduction. This does not necessarily reflect runtime, energy use, or correctness guarantees. It is not clear how optimizing for IR instruction would generalize to diverse compiler objectives or hardware targets. There's no multi-objective formulation or discussion of real-world utility tradeoffs.\n- It would be nice to see more details about the RL training pipeline, like how many steps are needed for convergence; some examples of failure analysis of the tool-assisted pipeline.", "was_revised": true, "final_justification": "I am maintaining my score of 4 (Borderline Accept) after carefully considering the authors’ rebuttal. The paper is technically solid and brings an interesting application of LLMs with RL to compiler optimization, a relatively unexplored domain. The rebuttal provided clarifications on error handling in the RL loop and outlines convergence and penalization strategies. The current reward design lacks multi-objective consideration, the system is evaluated on IR count across benchmarks, but there’s no analysis of generalization to runtime or across complexity levels. The paper contributes an interesting and timely application but falls short on broader generalization and deep technical novelty.", "rebuttal": "Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.\n\n---\n\n### **W1: Novelty in terms of training**\n\nWe appreciate the reviewer’s recognition of our work. While the combination of SFT followed by RL is not novel in isolation, the novelty in our paper lies in the integration of these techniques within the specific domain of **compiler auto-tuning**, a domain where such methods have not been explored previously. Traditional methods for compiler pass optimization rely on **fixed heuristics** or simple rule-based systems. Our framework leverages **RL-driven LLMs**, moving beyond static strategies to **adaptive, autonomous, context-aware optimizations**.\n\nThe **novel contributions** of our work can be summarized as:\n\n- **Creation of a high-quality reasoning dataset**: Our dataset is specifically curated for **compiler auto-tuning** tasks, combining **Chain-of-Thought (CoT) reasoning** and **tool-assisted feedback** for effective LLM training. This dataset represents the first attempt to build a comprehensive reasoning dataset for this domain.\n- **First end-to-end RL framework for compiler pass optimization**: While the combination of SFT and RL has been applied in other areas, our work is the first to integrate **RL with LLMs** in **compiler auto-tuning**tasks, offering an innovative framework to **autonomously discover optimized pass sequences** beyond static or heuristic-based methods.\n\nWe believe that the **combination of these components** provides a fresh perspective and contributes significantly to **the advancement of compiler optimization**. To address your suggestion, we will make the **novelty of our work** more explicit in the revised manuscript.\n\n---\n\n### **W2: Reward design is not deep**\n\nThank you for highlighting this aspect. While **IR instruction count** serves as a useful proxy for optimization and is widely used in compiler optimization research, we fully acknowledge the importance of a **broader reward design** for real-world relevance.\n\n- **Current reward design**: Our choice of **IR instruction count** is intentional due to its **deterministic** nature and **wide acceptance** in compiler research. It provides a clear, reproducible metric for assessing the optimization quality of the agent’s output.\n- **Future plans for multi-objective optimization**: We recognize the importance of **multi-objective optimization**, and as part of ongoing work, we are exploring how to integrate additional metrics such as **runtime**, **energy consumption**, and **code correctness** into our framework. These can be easily incorporated by **adjusting the reward function** and replacing the current tools with ones that track the desired metrics.\n\nBy incorporating these objectives, we aim to improve the real-world **applicability** of **Compiler-R1** and provide a more **comprehensive optimization framework**.\n\n---\n\n### **W3: Details about the RL training pipeline**\n\nWe appreciate your request for further details about the training pipeline. We are happy to provide more details of the **two-stage training** process. As outlined in **Figure 2,** we first use **SFT** to initialize the model’s reasoning ability. This is followed by **RL**, where the model further refines its strategy based on feedback from interactions with the compilation environment.\n\nKey aspects of the RL training include:\n\n- **RL algorithms**: We use **GRPO**, **PPO**, and **RPP** to fine-tune the model. These methods are applied to iteratively improve the model’s policy, enabling more efficient optimization strategies.\n- **Training Convergence**: Our RL training generally converges within **40 steps**, showcasing the **efficiency** of the pipeline. We will provide **convergence metrics** and additional details on the **training dynamics** in the revised manuscript.\n\nWe will also include a more **detailed analysis of the training dynamics**, and other important metrics in our revised manuscript.\n\n---\n\n### **Q1: How errors or noise in tool execution are handled**?\n\nThank you for raising this crucial point. The handling of errors and noise in tool execution is a vital component of ensuring the robustness of **Compiler-R1**.\n\nOur approach to error handling is as follows:\n\n- **Error Detection**: If a tool (e.g., `instrcount`) encounters an invalid pass sequence, it returns an **\"error\" status**. The model is trained to recognize this and take appropriate corrective action.\n- **Penalization for errors**: Instead of ignoring errors, the **RL reward function** penalizes any trajectory that leads to errors or tool failures. This **negative feedback** helps the model avoid repeating mistakes and promotes more **robust interactions** in future iterations.\n\nRather than dropping errors, we treat them as **negative feedback** and use them to **improve the robustness** of the agent’s decision-making process. We will add a more detailed explanation of this error-handling mechanism in the final version paper.\n\n------\n\n### **Q2: Comparison of matched performance between SFT and RL**\n\nWe acknowledge the importance of a fair comparison between **SFT** and **RL** models. To address the concern, we refer to **Figure 3** in our paper, which shows that RL models significantly outperform SFT models:\n\n- **RL models** (e.g., **GRPO-1.5B**) achieve higher **OverOz%** improvement compared to **SFT models** when both are allowed only a **single interaction trace** (as shown in **Figure 3**, at **N attempt** = 1), demonstrating the effectiveness of RL-based exploration.\n- As highlighted in **Figure 3**, the **SFT models have already converged**, yet they still underperform compared to RL models with just one interaction round, indicating that **further gains from SFT are unlikely**, while RL continues to improve through active exploration.\n\nIn summary, our results highlight the **clear advantages** of **RL models** over **SFT models**, achieving better optimization with fewer interactions. As shown in **Figure 3**, **RL-based exploration** is more efficient and effective in optimizing compiler pass sequences, underscoring the value of **RL** in complex tasks like compiler tuning.\n\n------\n\n### **Q3: Can the system adapt to newer LLVM versions (e.g., v17)?**\n\nThank you for your query regarding **forward compatibility** with newer LLVM versions. We are confident that Compiler-R1 can be adapted to **newer LLVM versions**, such as LLVM v17, with minimal effort.\n\nHere’s how we ensure **adaptability**:\n\n- **Version-Agnostic Principles**: The fundamental \"phase-ordering problem\" we solve persists and even grows in complexity with newer compilers. Our framework's core value—learning program-specific strategies to outperform standard heuristics like -Oz—remains highly relevant and is not tied to a specific LLVM version.\n- **Modular design**: Our framework is designed for modularity. To adapt to a newer version like LLVM 17, the only required step is to **update the underlying toolchain**. We would simply replace the opt binary in our execution environment with the new one. The core learning pipeline remains unchanged and can immediately leverage the updated compiler, including its new passes and improvements, without modification.\n\nWe will **emphasize the modularity** and **discuss the adaptability** of our framework in the final version paper to highlight these strengths.\n\n---\n\n### **Q4: Consideration of curriculum learning**\n\nWe appreciate the suggestion to explore **curriculum learning**. While our use of **AutoPhase features** mitigates some of the complexity variations across programs, we acknowledge that **curriculum learning** could be useful, particularly in early RL training stages.\n\n- **Current approach**: The **AutoPhase features** provide a **uniform input representation**, reducing the need for curriculum learning.\n- **Future work**: We plan to investigate the use of **curriculum learning** to better handle **low-complexity programs** in future experiments.\n\nWe will include this idea as a **future direction** in the revised manuscript.\n\n---\n\n### **Q5: IR instruction count as a proxy for optimization**\n\nThank you for raising this important point. We agree that IR instruction count as a sole metric has limitations. We chose it because it is a **deterministic, fast-to-measure, and standard proxy** in compiler research, enabling rapid and reproducible experiments.\n\nHowever, a key strength of our framework is its **flexibility regarding the optimization objective**. Compiler-R1 is not fundamentally tied to IR count and can be readily adapted to target metrics like runtime or power efficiency. This adaptation requires three straightforward changes:\n\n- **Re-construct the Dataset:** Generate new target sequences for the SFT data based on the new objective (e.g., best runtime).\n- **Update the Reward Signal & Tools:** Replace the instrcount tool with a runtime measurement tool and adjust the RL reward accordingly.\n- **Modify the LLM's Prompt:** Change the agent's instructions to reflect the new goal (e.g., \"minimize execution time\").\nThe core agentic framework and its two-stage training pipeline remain unchanged, highlighting its versatility.\n\nWe acknowledge the importance of real-world performance, and in **future work**, we plan to extend our framework to include **multi-objective optimization**, incorporating **runtime and energy efficiency** alongside instruction count.\n\n---\n\nWe hope these clarifications may help to address your concerns. At last, we sincerely appreciate your valuable feedback, and we will carefully consider all your suggestions to further improve our paper. We would be deeply grateful if you could kindly reconsider raising the score. Thank you very much!\n\n---"}, {"review_id": "fBg5bufiOo", "reviewer": "Reviewer_EFb8", "summary": "The paper introduces an LLM-based approach for reducing IR instruction count for compiler auto-tuning. They introduce a new autotuning dataset for kicking off training of reasoning models, and then an RL framework. The paper performs several ablations in their training pipeline, and also compares to an existing suite of auto-tuners. They find that their method is in general more capable of reducing instruction counts compared to other traditional methods.", "questions": "1. It is surprising to me that with a very simple SFT data, the model is capable of thinking via RL to a fairly large improvement over just SFT. What do you think accounts for this? And could you share some example model thoughts from the full model? I only saw SFT data in the appendix. \n\n2. Per mentioned in the limitations & weaknesses section, could you make a comparison between your compiler-R1 and traditional autotuners with respect to FLOPs?\n\n3. What are the takeaways from this work? Perhaps you could describe more the value of auto-tuning, especially in reducing instruction counts? I think many of the future directions you mentioned are more on the AI-for-this-problem side, but when it comes to the problem itself,  where might be valuable targets for real world impact?", "limitations": "I think the related works section is a bit slim. I think for the audience of NeurIPS the AI side is fairly well known, but many folks may not have much knowledge about compilersand I think writing more there would be valuable. I also think that it would be good to link the results obtained to how it might inform actual program performance, as mentioned in the introduction.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The idea of using LLMs and reasoning models in particular for optimizing IRs has been a popular topic. An example I remember is from DeepSeek-R1 optimizing some GPU kernels. The topic of this paper is closely related and seems to target a problem that has tangible real world impacts. I appreciate the number of ablations that the authors have performed for their main method, and it does seem that the method has value in reducing instruction counts. \n\nThe main weaknesses I can identify with the paper are the following:\n1. The paper focuses on reducing instruction count, but how desirable of a quality is this? As I understand it, -0z is also for reducing instruction count but not necessarily optimizing for speed, which seems to be more valuable. How correlated is this?\n2. The comparisons in Table 1 do not seem entirely fair. Traditional autotuners are indeed given more time, but how much does the *compute* differ? Even sampling once from a 7B model involves billions of floating point operations, not to mention that to get a good output you may need to generate thousands of tokens. By contrast, traditional autotuners probably use much less? I think it is important to clarify this, because any downstream real-world impact this work may only be available to people who have access to GPUs who can call these models, for instance.", "was_revised": true, "final_justification": "The authors addressed my questions adequately. I am not putting a higher score because it is still unclear to me how much the large reasoning models are necessary here because they are not fully utilized for their thinking for this task. But I think this serves as valuable work for this community, albeit I am not too familiar with the field.", "rebuttal": "Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.\n\n---\n\n### **W1: The paper focuses on reducing instruction count, but how desirable of a quality is this?**\n\nThank you for this insightful comment. We agree that IR instruction count does not always correlate directly with runtime or energy efficiency. However, it remains a **widely used optimization target**with practical value, as reducing instruction count often leads to **leaner code, faster compilation**, and **better optimization opportunities downstream**:\n\n- **Code Size Reduction**: This is crucial for **resource-constrained environments** such as embedded systems, mobile devices, and IoT applications, where minimizing code size can save memory and storage, as well as reduce the cost of **over-the-air updates**.\n- **Compile Time**: A smaller IR typically leads to faster compilation times, which can significantly improve **developer productivity** in large-scale projects.\n- **Performance Proxy**: Although it is not a perfect metric, **reducing instruction count** often leads to **simpler control flow** and **fewer redundant computations**, which in turn frequently improves the **runtime performance**.\n\nWe will clarify this in the revised manuscript by discussing the **empirical correlation** between instruction count reduction and runtime performance. Additionally, we plan to extend our future work to consider **multi-objective optimization**, where runtime, energy efficiency, and correctness will be incorporated into our reward design.\n\n---\n\n### **W2: The comparisons in Table 1 do not seem entirely fair. Traditional autotuners are indeed given more time, but how much does the compute differ?**\n\nThank you for bringing up this point. We understand that the comparison between **Compiler-R1** and traditional autotuners, especially in terms of **time and compute**, requires further clarification. Here are the key aspects of the computational cost comparison:\n\n- **Compute Costs**: Our method, Compiler-R1, relies on inference with LLMs. A single forward pass with a **7B model** involves dense matrix multiplications on GPUs, which, while computationally intense, is **massively parallelizable** and completes in **seconds**.\n- **Traditional autotuners**, in contrast, use **heuristic search algorithms** on CPUs. While the per-compilation cost is relatively lower, these autotuners perform **hundreds or thousands of evaluations** sequentially, leading to much **longer total runtimes** (e.g., 200 seconds for OpenTuner, and over 9000 seconds for CompTuner in our experiments).\n\nWe will update the paper to **clarify this difference** in computational paradigms and discuss how **time-to-solution** is often the most critical metric for developers. While **FLOPs** comparison is difficult, our approach delivers an **order-of-magnitude faster time-to-solution** using **parallelizable GPU computations**.\n\n---\n\n### **Q1: It is surprising to me that with a very simple SFT data, the model is capable of thinking via RL to a fairly large improvement over just SFT.**\n\nThank you for this excellent and nuanced question. You are right to point out that the SFT data appears structurally simple, but we'd argue this simplicity is by design and is crucial for the overall success of the framework.\n\nThe perceived simplicity lies in the consistent think-tool-answer format. This structure is intentionally kept straightforward to ensure the LLM can reliably learn the fundamental **interaction protocol**. The true complexity and value of our SFT data lie not in the format, but in the **reasoning logic embedded within the trajectories**. Each target sequence is derived from our **synergy-graph analysis**, representing a high-quality, non-obvious optimization solution. More importantly, the data explicitly teaches the agent a critical, non-trivial skill: **how to interpret tool feedback to make a conditional decision**—either accept its own solution or escalate to the FBPS tool.\n\nTherefore, the SFT stage provides a **robust foundation**, not a simplistic one. It solves the difficult \"cold-start\" problem by equipping the agent with a solid grasp of the task mechanics and a strong baseline reasoning strategy.\n\nWith this foundation in place, the **RL stage can focus entirely on refining and transcending that strategy**. The agent is no longer struggling to learn how to interact; it is now learning how to optimize better. It explores variations of the strong patterns learned during SFT and fine-tunes its policy based on live feedback, for instance, learning to be more confident in its initial guesses for certain program types. This focused, high-level learning is what accounts for the significant performance improvement over SFT alone.\n\nIn response to your request for an example of model reasoning, we will include an example of the **RL-tuned model's thought process** in the appendix of the final version of the paper. This will illustrate how RL improves the model's reasoning beyond simple imitation.\n\n```python\n\nAssistant: \n<think>\nBased on the provided autophase features, I will try the following pass sequence:\n['--loop-simplifycfg', ...]\n\nI'll use the instrcount tool to check if this sequence provides better optimization than -Oz.\n</think>\n\n\n[Tool Call] Function: instrcount\nArguments:\n{\n  \"filename\": \"xxx\",\n  \"optimization_flags\": [\n    \"--loop-simplifycfg\",\n    ...\n  ]\n}\n\nTool: {\"filename\": \"xxx\", \"status\": \"success\", \"improvement_over_oz\": -8.695652173913043%}\n\nAssistant: <think>\nThe pass sequence did not provide better optimization than -Oz, as improvement_over_oz is negative (-8.695652173913043%).\n\nI will now call the find_best_pass_sequence tool to find a better sequence.\n</think>\n\n\n\n[Tool Call] Function: find_best_pass_sequence\nArguments:\n{\n  \"filename\": \"xxx\"\n}\n\nTool: {\n  \"status\": \"success\",\n  \"best_pass_sequence\": [\n    \"--loop-interchange\",\n    ...\n  ],\n  \"improvement_percentage\": -0.79%\n}\n\nAssistant: <think>\nThe find_best_pass_sequence tool returned a new pass sequence, but the improvement_percentage is -0.79%, indicating a decrease in performance.\n\nSince neither the initial sequence nor the newly found sequence provide positive improvement over -Oz, I'll recommend the standard -Oz optimization as my final answer.\n</think>\n<answer>\n['-Oz']\n</answer>\n\n```\n---\n\n### **Q2: Could you make a comparison between your Compiler-R1 and traditional autotuners with respect to FLOPs?**\n\nThank you for this sharp and important question. A direct FLOPs comparison is exceptionally challenging due to fundamentally different computational paradigms. Here is our analysis of the cost structures:\n\n- **Difficulty of Direct Comparison:** A precise, apples-to-apples FLOPs count is difficult. Our method's cost is dominated by a few highly parallelizable LLM forward passes on a GPU. Traditional autotuners, conversely, perform thousands of less-intensive but often sequential program compilations on a CPU. These distinct processes do not map cleanly for a direct numerical comparison.\n- **Contrasting Computational Patterns:**\n  - **Compiler-R1 (Inference-Heavy):** Executes a predictable burst of billions of FLOPs that completes in seconds on modern hardware.\n  - **Traditional Autotuners (Search-Heavy):** Their cumulative, latency-bound search process, involving thousands of evaluations, results in very long runtimes (200s to over 9000s), despite lower per-evaluation intensity.\n- **A More Holistic View of \"Cost\":**\n  - **Time-to-Solution:** From a developer's perspective, our method offers an order-of-magnitude speedup, which is often the most critical factor.\n  - **Amortized Cost:** Our model has a high one-time training cost but becomes a reusable asset for fast inference. Traditional tuners incur their high search cost for every new program.\n  \n\nWhile our method requires GPU access, the rapidly improving accessibility of LLM inference makes this approach increasingly viable. In conclusion, Compiler-R1 makes a favorable trade-off, leveraging a higher burst of parallelizable computation to achieve a dramatically lower and more practical time-to-solution. We will add a detailed discussion of this cost analysis to the paper.\n\n---\n\n### **Q3: What are the takeaways from this work?**\n\nThank you for this excellent question. Here are the key **takeaways** from our work:\n\n1. **Reframing the Phase-Ordering Problem**: We present a novel approach to compiler optimization by framing the **phase-ordering problem** as a **reasoning task** solvable by an **interactive LLM agent**. This represents a paradigm shift from traditional search-based methods to **adaptive, intelligent decision-making**.\n2. **Value of Instruction Count Reduction**: While not a perfect measure, **reducing IR instruction count** offers several real-world benefits:\n   - **Smaller binary size** for resource-constrained systems (e.g., IoT, mobile devices).\n   - **Reduced compile time**, improving developer productivity.\n   - **Potential runtime improvements**, as reducing instruction count often leads to more efficient program execution.\n3. **Practical Impacts of Auto-Tuning**: We demonstrate that **auto-tuning** is not only valuable in terms of performance but also in terms of **developer efficiency**. Our framework offers **faster time-to-solution** and can be adapted to other **multi-objective optimization** goals in future work.\n\nWe will also add a discussion of **real-world applications** and **future directions** in the revised paper to clarify the broader impact of our work.\n\n------\n\nWe hope these clarifications may help to address your concerns. At last, we sincerely appreciate your valuable feedback, and we will carefully consider all your suggestions to further improve our paper. We would be deeply grateful if you could kindly reconsider raising the score. Thank you very much!\n\n---"}, {"review_id": "57gYLdhWtm", "reviewer": "Reviewer_oQVW", "summary": "This paper proposes an SFT + RL strategy for training LLMs for compiler autotuning (pass sequence optimization). Using synergistic pass analysis and a find-best-pass-sequence tool, an initial SFT dataset is created, demonstrating a fixed decision tree strategy. In the RL training stage, the LLM is then allowed to propose other pass sequences and deviate from this strategy to maximize a reward term composed of both instruction count reduction and adhering to the expected interaction and tool calling format. Several model sizes (1.5, 3, 7B) and RL algorithms are compared (GRPO, PPO, REINFORCE++) using IR instruction count reduction over an -Oz baseline as the main performance metric. AutoPhase features are proposed as a compact yet sufficient representation of programs in the LLM prompts.", "questions": "1. How do the trained GRPO models compare to the initial decision tree strategy used in SFT dataset construction in OverOz and time?\n\n2. How do the trained GRPO models compare to SFT and RL ablations in OverOz and time?\n\n3. Some experimental details need to be clarified. The repetition penalty is not explained anywhere in the paper. Is it applied at the level of tokens or some higher level? How were the weights of the reward components (1.5 and alpha) set and how sensitive is the method to them?\n\n4. Are the RL-only baselines instructed (prompted) with the same interaction protocol? Do the RL-only baselines collect their own on-policy data?\n\n5. Why not simply include the FBPS results in the initial prompt to the LLM rather than require a separate tool call?", "limitations": "Limitations are briefly mentioned in the conclusion.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n\nAs a fully simulated environment, compiler optimizations based on the compiled programs' performance are a scalable source of ground truth rewards for RL training, and therefore a good application domain for R1-like training strategies.\n\nOn held-out test suites, the interactive Compiler-R1 models outperform direct pass prediction SFT models as well as several traditional autotuners. They also significantly outperform traditional autotuners in execution speed.\n\n**Weaknesses**:\n\nThis work is in large part an application of DeepSeek-R1 like training paradigms to the domain of compiler auto-tuning. It may provide limited methodological novelty or insight relevant to a general ML / NeurIPS audience outside of its application domain.\n\nMoreover, the analysis of the GRPO-trained policies’ behavior is lacking: the source of the improvements is not analyzed and no qualitative examples of the learned trajectories are provided. The key decision points in the trajectory should be made explicit to clarify the need for RL training, especially given that only 2 tools are available, one of which (find best pass sequence) deterministically provides a single pass sequence recommendation per program and the other of which only evaluates pass sequences without proposing modifications. Does the main source of improvement come from the policy proposing better initial pass sequences while otherwise following the demonstrated decision tree strategy, or do the models learn to deviate from the strategy towards being more persistent with proposing multiple improvements over the FBPS suggestion? How do the models compare to the performance of the SFT data? Do the models show within-trajectory in-context adaptation based on previous attempts’ evaluation results, or is the improvement due to more generally becoming better at guessing relevant pass sequences thanks to RL training, than the SFT models? Do the models leverage the <think> blocks productively to derive insights from previous attempts, or do they stick to the SFT initialization’s templated monologue style?\n\nThe authors also consider adherence to the thought-tool-answer format as success rate, yet GRPO-3B seems to output good solutions without following the format. What is then the justification for requiring this format? Given this, reporting only success rates and not OverOz% for the SFT and RL ablations is not giving the full picture. \n\nDespite saying so in the NeurIPS checklist, the paper’s experiments are not reported with error bars.\n\n**Minor**:\n\nThe paper includes many grammatical issues, especially in the introduction.  \nThe appendix sections are not referred to anywhere in the main text.", "was_revised": true, "final_justification": "I increase my score to 4 as several of my concerns regarding ablation metrics and experimental details were adressed during the rebuttal period, and these additional comparisons have improved the paper. The main remaining issue is that the source of improvement of the proposed method has not been clarified to a satisfactory degree, especially given its high computational cost relative to simpler strategies or models. This includes the other issues highlighted in Q1 of my latest review comment, specifically, that RL training of an LLM may be an overcomplicated solution given the limited toolset and two-level decision process.\n\nA more expressive toolset or demonstrations of within-episode adaptive reasoning would better highlight the advantage of LLM RL training for this use case.", "rebuttal": "Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.\n\n----\n\n### **Q1: How do the trained GRPO models compare to the initial decision tree strategy used in SFT dataset construction in OverOz and time?**\n\nThank you for this insightful question. The trained **GRPO models** significantly outperform the initial **decision tree strategy** used during the **SFT dataset** construction, both in terms of **OverOz** (IR instruction count reduction) and **time efficiency**.\n\nKey differences:\n\n- **OverOz Improvement**: The **GRPO models**, trained with **Reinforcement Learning (RL)**, adapt and optimize the pass sequence based on feedback from the environment, whereas the decision tree strategy is deterministic and lacks dynamic exploration. As a result, the GRPO models, such as **GRPO-7B**, achieve an **8.46% average reduction** in IR instruction count, significantly outperforming the decision tree-based approach, which has more limited optimization capabilities.\n- **Time Efficiency**: The **GRPO-7B model** is able to complete each program in approximately **26 seconds**, making it considerably more time-efficient than the decision tree strategy and traditional autotuners. This demonstrates that the GRPO models are not only better at optimizing pass sequences but also faster in execution, thanks to their dynamic optimization approach.\n\nThis comparison clearly illustrates the advantages of **GRPO models** in both **optimization performance** and **computational efficiency**.\n\n------\n\n### **Q2: How do the trained GRPO models compare to SFT and RL ablations in OverOz and time?**\n\nThank you for this important question. The **GRPO models** consistently outperform both **SFT-only** and **RL-only** models in terms of **OverOz** (optimization performance) and **time efficiency**.\n\n- **OverOz Comparison**:\n  - The **GRPO-7B model** achieves an **8.46% average OverOz improvement**, significantly outperforming the best-performing **SFT-only model** (which achieves up to 4.66% OverOz improvement).\n  - **RL-only models**, without the SFT initialization, show weaker performance because they lack the structured reasoning framework provided by SFT.\n- **Time Comparison**:\n  - The **GRPO-7B** model completes each program in about **26 seconds**, which is comparable to the runtime of **SFT-only models** (29-55 seconds), and is much faster than traditional autotuners like **OpenTuner** (over 200 seconds).\n  - The **RL-only models**, due to the lack of SFT pre-training, require more time to converge as they rely solely on trial-and-error interactions, which makes them less time-efficient.\n\nIn conclusion, the **GRPO models** offer a balanced trade-off, delivering both **high optimization performance** and **low runtime**, making them **highly efficient** for practical compiler auto-tuning tasks.\n\n------\n\n### **Q3: Some experimental details need to be clarified. The repetition penalty is not explained anywhere in the paper. Is it applied at the level of tokens or some higher level? How were the weights of the reward components (1.5 and alpha) set and how sensitive is the method to them?**\n\nThank you for the opportunity to clarify these details:\n\n- **Repetition Penalty**:\n  - The **repetition penalty** is applied **at the token level** during the generation of pass sequences in the **GRPO model**. This penalty discourages the model from generating repetitive or redundant tokens, promoting more **diverse and effective optimizations**. By applying the repetition penalty, we prevent the model from getting stuck in suboptimal cycles and encourage exploration of a broader range of possible solutions.\n- **Reward Component Weights**:\n  - The **reward function** is a **composite function** consisting of two components: the **format reward** (Rformat) and the **answer reward** (Ranswer).\n  - The weight for the **format reward** is set to **1.5**, which prioritizes the correct reasoning format and tool-invocation structure. This weight was empirically chosen to ensure the model learns to correctly structure its reasoning and interactions.\n  - The weight for the **answer reward** is represented by **alpha**, which controls the importance of **IR instruction reduction** in the reward function. **Alpha** is set to **30**, reflecting the emphasis on optimization quality (instruction reduction) while balancing the importance of reasoning and tool usage.\n- **Sensitivity to Reward Component Weights**:\n  - Our experiments show that the method is **relatively stable** to small changes in the weights of the reward components. We conducted sensitivity analyses by adjusting **1.5** and **alpha** within a reasonable range (±20%) and found that the model's performance remained robust. However, large deviations from the chosen values led to slight performance degradation, highlighting the importance of these weights in shaping the agent's behavior without making the model overly sensitive to them.\n\nWe believe that these clarifications provide a deeper understanding of our experimental setup and the factors that contribute to the effectiveness of **Compiler-R1**.\n\n------\n\n### **Q4: Are the RL-only baselines instructed (prompted) with the same interaction protocol? Do the RL-only baselines collect their own on-policy data?**\n\nThank you for these excellent questions. We are happy to clarify the specifics of our **ablation studies**:\n\n1. **Interaction Protocol**:\n   - Yes, the **RL-only baseline** was prompted with the exact same **instruction prompt** as our full **Compiler-R1 models**. This prompt defines the task, the available tools (instrcount, find_best_pass_sequence), and the required **think -> tool_call -> response** interaction protocol. We ensured that the **RL-only agent** had all the necessary information about the interaction framework from the start.\n2. **On-Policy Data Collection**:\n   - Yes, the **RL-only baseline** follows the standard **on-policy reinforcement learning** procedure. The agent generates **trajectories** based on its current policy, and this data is used to **update the policy**. This process is identical to the **RL stage** of our full **Compiler-R1** framework.\n\nThe **poor performance** of the **RL-only baseline**, despite being prompted with the correct instructions and following the on-policy learning loop, highlights the **difficulty** of learning both **complex interaction protocols** and **semantic reasoning** for compiler optimization simultaneously. The agent struggles to find a meaningful policy due to the **large, sparse-reward search space**. This underscores the importance of **SFT pre-training**, which provides a **\"warm start\"** for the model, making the subsequent **RL fine-tuning** much more tractable and effective.\n\n------\n\n### **Q5: Why not simply include the FBPS results in the initial prompt to the LLM rather than require a separate tool call?**\n\nThank you for this excellent question. The suggestion of pre-computing **FBPS results** and including them in the prompt is a valid approach, but we believe our **interactive framework** offers significant advantages:\n\n- **Efficiency**: Pre-computing FBPS results for every program would enforce a rigid workflow and potentially lead to inefficient use of resources. Our approach allows the agent to **dynamically decide** when to invoke FBPS, ensuring that this computationally expensive step is only used when necessary.\n- **Flexibility and Extensibility**: Our framework is designed to be **extensible** to include multiple optimization tools in the future. The FBPS tool is just one example, and we envision a future where the agent can choose from a variety of tools based on the **specific characteristics** of the program being optimized. This would make the system more **generalizable** and **powerful** for a wide range of optimization tasks.\n\nWe aim to create an **adaptive, problem-solving agent** that can intelligently select the right tool for the task at hand, rather than simply relying on pre-computed results. This flexibility is key to making our approach more robust and capable of handling complex, real-world optimization tasks.\n\n------\n\nWe hope these clarifications may help to address your concerns. At last, we sincerely appreciate your valuable feedback, and we will carefully consider all your suggestions to further improve our paper. We would be deeply grateful if you could kindly reconsider raising the score. Thank you very much!"}]

Author rebuttal:

Meta-review: The paper designs and studies a compiler optimization approach based on using LLMs with the SFT+RL paradigm. 

This is a strong application-oriented contribution and could influence the compiler optimization community. It is also a non-trivial instantiation of the SFT+RL paradigm that could be beneficial as a schema for researchers with analogous problems. 

There was significant interaction with the authors during the rebuttal phase and most concerns were address. The major concern included:

- Limited insights into why we are seeing improvements given that there are few options available to the model.

- Questioning the necessity of the LLM here compared to something simpler.

While these questions are not fully resolved for all reviewers, the authors have indicated solid ways they can adjust the paper to improve in those directions. 

Overall, this is a solid contribution.
Decision: Accept (poster)