Title: Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM

Compiler Tuning

Reinforcement Learning

Agent

Compilation Pass
Abstract: Compiler auto-tuning optimizes pass sequences to improve performance metrics such as Intermediate Representation (IR) instruction count. Although recent advances leveraging Large Language Models (LLMs) have shown promise in automating compiler tuning, two significant challenges still remain: the absence of high-quality reasoning datasets for agents training, and limited effective interactions with the compilation environment. In this work, we introduce Compiler-R1, the first reinforcement learning (RL)-driven framework specifically augmenting LLM capabilities for compiler auto-tuning. Compiler-R1 features a curated, high-quality reasoning dataset and a novel two-stage end-to-end RL training pipeline, enabling efficient environment exploration and learning through an outcome-based reward. Extensive experiments across seven datasets demonstrate Compiler-R1 achieving an average 8.46\% IR instruction count reduction compared to opt -Oz, showcasing the strong potential of RL-trained LLMs for compiler optimization. Our code and datasets are publicly available at https://github.com/Panhaolin2001/Compiler-R1.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an SFT + RL strategy for training LLMs for compiler autotuning (pass sequence optimization). Using synergistic pass analysis and a find-best-pass-sequence tool, an initial SFT dataset is created, demonstrating a fixed decision tree strategy. In the RL training stage, the LLM is then allowed to propose other pass sequences and deviate from this strategy to maximize a reward term composed of both instruction count reduction and adhering to the expected interaction and tool calling format. Several model sizes (1.5, 3, 7B) and RL algorithms are compared (GRPO, PPO, REINFORCE++) using IR instruction count reduction over an -Oz baseline as the main performance metric. AutoPhase features are proposed as a compact yet sufficient representation of programs in the LLM prompts.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\nAs a fully simulated environment, compiler optimizations based on the compiled programs' performance are a scalable source of ground truth rewards for RL training, and therefore a good application domain for R1-like training strategies.\n\nOn held-out test suites, the interactive Compiler-R1 models outperform direct pass prediction SFT models as well as several traditional autotuners. They also significantly outperform traditional autotuners in execution speed.\n\n**Weaknesses**:\n\nThis work is in large part an application of DeepSeek-R1 like training paradigms to the domain of compiler auto-tuning. It may provide limited methodological novelty or insight relevant to a general ML / NeurIPS audience outside of its application domain.\n\nMoreover, the analysis of the GRPO-trained policies’ behavior is lacking: the source of the improvements is not analyzed and no qualitative examples of the learned trajectories are provided. The key decision points in the trajectory should be made explicit to clarify the need for RL training, especially given that only 2 tools are available, one of which (find best pass sequence) deterministically provides a single pass sequence recommendation per program and the other of which only evaluates pass sequences without proposing modifications. Does the main source of improvement come from the policy proposing better initial pass sequences while otherwise following the demonstrated decision tree strategy, or do the models learn to deviate from the strategy towards being more persistent with proposing multiple improvements over the FBPS suggestion? How do the models compare to the performance of the SFT data? Do the models show within-trajectory in-context adaptation based on previous attempts’ evaluation results, or is the improvement due to more generally becoming better at guessing relevant pass sequences thanks to RL training, than the SFT models? Do the models leverage the <think> blocks productively to derive insights from previous attempts, or do they stick to the SFT initialization’s templated monologue style?\n\nThe authors also consider adherence to the thought-tool-answer format as success rate, yet GRPO-3B seems to output good solutions without following the format. What is then the justification for requiring this format? Given this, reporting only success rates and not OverOz% for the SFT and RL ablations is not giving the full picture. \n\nDespite saying so in the NeurIPS checklist, the paper’s experiments are not reported with error bars.\n\n**Minor**:\n\nThe paper includes many grammatical issues, especially in the introduction.  \nThe appendix sections are not referred to anywhere in the main text.\n\nQUESTIONS:\n1. How do the trained GRPO models compare to the initial decision tree strategy used in SFT dataset construction in OverOz and time?\n\n2. How do the trained GRPO models compare to SFT and RL ablations in OverOz and time?\n\n3. Some experimental details need to be clarified. The repetition penalty is not explained anywhere in the paper. Is it applied at the level of tokens or some higher level? How were the weights of the reward components (1.5 and alpha) set and how sensitive is the method to them?\n\n4. Are the RL-only baselines instructed (prompted) with the same interaction protocol? Do the RL-only baselines collect their own on-policy data?\n\n5. Why not simply include the FBPS results in the initial prompt to the LLM rather than require a separate tool call?\n\nLIMITATIONS:\nLimitations are briefly mentioned in the conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an LLM-based approach for reducing IR instruction count for compiler auto-tuning. They introduce a new autotuning dataset for kicking off training of reasoning models, and then an RL framework. The paper performs several ablations in their training pipeline, and also compares to an existing suite of auto-tuners. They find that their method is in general more capable of reducing instruction counts compared to other traditional methods.\n\nSTRENGTHS AND WEAKNESSES:\nThe idea of using LLMs and reasoning models in particular for optimizing IRs has been a popular topic. An example I remember is from DeepSeek-R1 optimizing some GPU kernels. The topic of this paper is closely related and seems to target a problem that has tangible real world impacts. I appreciate the number of ablations that the authors have performed for their main method, and it does seem that the method has value in reducing instruction counts. \n\nThe main weaknesses I can identify with the paper are the following:\n1. The paper focuses on reducing instruction count, but how desirable of a quality is this? As I understand it, -0z is also for reducing instruction count but not necessarily optimizing for speed, which seems to be more valuable. How correlated is this?\n2. The comparisons in Table 1 do not seem entirely fair. Traditional autotuners are indeed given more time, but how much does the *compute* differ? Even sampling once from a 7B model involves billions of floating point operations, not to mention that to get a good output you may need to generate thousands of tokens. By contrast, traditional autotuners probably use much less? I think it is important to clarify this, because any downstream real-world impact this work may only be available to people who have access to GPUs who can call these models, for instance.\n\nQUESTIONS:\n1. It is surprising to me that with a very simple SFT data, the model is capable of thinking via RL to a fairly large improvement over just SFT. What do you think accounts for this? And could you share some example model thoughts from the full model? I only saw SFT data in the appendix. \n\n2. Per mentioned in the limitations & weaknesses section, could you make a comparison between your compiler-R1 and traditional autotuners with respect to FLOPs?\n\n3. What are the takeaways from this work? Perhaps you could describe more the value of auto-tuning, especially in reducing instruction counts? I think many of the future directions you mentioned are more on the AI-for-this-problem side, but when it comes to the problem itself,  where might be valuable targets for real world impact?\n\nLIMITATIONS:\nI think the related works section is a bit slim. I think for the audience of NeurIPS the AI side is fairly well known, but many folks may not have much knowledge about compilersand I think writing more there would be valuable. I also think that it would be good to link the results obtained to how it might inform actual program performance, as mentioned in the introduction.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Compiler-R1, a tool assisted system that uses RL to learn better compiler optimization sequences. Instead of relying on fixed optimization strategies, the authors frame the problem as a decision-making task, where an agent selects a sequence of optimization passes tailored to a specific program. The contributions include construction of a high-quality reasoning dataset and a novel two-stage RL pipeline which is first trained with SFT and then improved using tool assisted feedback based on how well the resulting code is optimized in terms of reducing the number of intermediate instructions. The authors evaluate Compiler-R1 on a diverse benchmark of programs and show that it consistently outperforms traditional methods and existing tools in reducing the number of IR. They also study how different ways of representing the input program affect the agent’s performance and perform ablations on how it is important to do both SFT and RL.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- As noted by the authors, Compiler-R1 is the first framework to integrate LLMs with RL for automatically tuning compiler pass sequences to minimize LLVM IR instruction count. This makes it an interesting application of tool assisted SFT+RL reasoning in a new domain. The authors also built a reasoning dataset designed for compiler pass optimization. \n- The system delivers practical improvements: it reduces LLVM IR instruction count by about 8.5% compared to the standard -Oz optimization. they evaluate on a bunch of different datasets in Compiler Gym and against classical compiler-tuning methods, including Open Tuner, GA, TPE, RIO, Comp Tuner, BOCA, and Auto Phase, showing robustness.\n\nWeaknesses:\n- There is not much novelty in terms of training, this paper is mainly applying standard SFT+RL training techniques to a new domain. It would be interesting to see more exploration of tree search, memory-augmented, or curriculum learning for compiler pass discovery.\n- Reward design is not deep: The paper optimizes only for IR instruction count reduction. This does not necessarily reflect runtime, energy use, or correctness guarantees. It is not clear how optimizing for IR instruction would generalize to diverse compiler objectives or hardware targets. There's no multi-objective formulation or discussion of real-world utility tradeoffs.\n- It would be nice to see more details about the RL training pipeline, like how many steps are needed for convergence; some examples of failure analysis of the tool-assisted pipeline.\n\nQUESTIONS:\n1. The environment relies on external tools like instrcount and find_best_pass_sequence, which may introduce non-determinism or fragility (e.g., if tool crashes or gives misleading feedback) Please describe how errors or noise in tool execution are handled. Are they dropped, or is the agent penalized?\n2. SFT models are evaluated with N=40 samples, whereas RL agents use single interaction traces. Could you include a comparison with matched performance (e.g., how many SFT samples are needed to match GRPO-1.5B’s performance),or evaluate both SFT and RL under the same inference budge.\n3. The study focuses on LLVM 10 and specific passes. While sufficient for benchmarking, this may limit real-world relevance. Can the system adapt to newer LLVM versions (e.g., v17)?\n4. Currently, all programs are treated equally during training. This could slow down learning or introduce instability early in RL training.\nHave the authors considered curriculum learning, e.g., starting with short or low-complexity programs? Even a brief analysis of performance across complexity levels could reveal useful insights here.\n5. The paper focuses solely on IR instruction count as a proxy for optimization. it's unclear how well this correlates with runtime performance or power/energy efficiency.\n\nLIMITATIONS:\nThe authors have briefly discussed the potential negative impacts of their work including how they can use LLMs to improve interpretability of auto tuning processes and further develop agents to exploit synergies between compiler passes. They have not gone into the details of the societal impact of this work, especially if there is a correctness aspect of discovering the compiler passes, if not properly sandboxed or verified, this could lead to vulnerabilities in deployed software. failures in learned optimization sequences (e.g., those introducing subtle bugs or non-deterministic behavior) could lead to safety risks especially in critical domains.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a LLM-based algorithmic framework (Compiler-R1) for program optimization. The primary contributions of the paper are algorithmic and empirical. Specifically, the system uses a two-stage training pipeline (SFT followed by RL) to train a LLM to generate an optimized sequence of compiler operations (passes) by leveraging standard compiler optimization and program analysis tools in order to reduce the instruction count of the compiled program. The paper proposes a method to construct a dataset for the Stage 1 (SFT) pipeline. Experiments on CompilerGym tasks show that Compiler-R1 is reasonably effective at reducing instruction size while running faster than strong baselines.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper studies an interesting and important problem. Automated improvements to program compilation tasks is an important area and progress here is likely to have significant impact.\n\n- The paper is well written. Key ideas are explained clearly and I found it easy to follow. The paper appendix includes a good amount of empirical details, which, in conjunction with released code and datasets, will aid reproducibility.\n\n- The proposed algorithmic approach is intuitively clear. The two-stage pipeline (SFT followed by RL with GRPO) using AutoPhase features makes a lot of sense. The dataset construction approach for the SFT stage seems like a standalone contribution in itself (as far as I can tell). While the algorithmic novelty is a bit low in that the components are all off-the-shelf, the combination of existing ideas is novel, to my knowledge. (The paper could do a better job of making its novel contributions clearer.)\n\n- The experiments show that Compiler-R1 is generally effective at reducing instruction counts while running faster than most baselines. It strikes a nice balance between AutoPhase (higher instruction count but fast) and Opentuner (lower instruction count but slower). The experiments are richly detailed and contain a good amount of insight into the underlying factors driving performance gains. In particular, the importance of RL training and scaling model size comes through clearly.\n\n**Weaknesses**\n\n- The paper could do a better job of describing its novelty and contributions relative to the larger body of work on compiler optimization. For example, how novel is the induced dataset generated for the SFT training to the community? How important or impactful is the improvement over traditional autotuners in Table 1 (8.46 vs 6.09)? A more detailed discussion of the novelty and impact aspects of the dataset and empirical results (perhaps in the appendix) would significantly improve the paper.\n\n- The paper would be strengthened with a detailed discussion of Compiler-R1's generalization to programs outside CompilerGym (i.e., OOD). For example, what would happen on larger programs (> 10K instruction count which were filtered out) or those significantly different from the tasks in CompilerGym? Put differently, in what scenarios might Compiler-R1 fail?\n\n- The above are relatively minor. Overall, I didn't find any major issues in the paper and I'm open to increasing my score with a better understanding of the empirical contributions.\n\nQUESTIONS:\n1. Please enumerate the novel contributions of the paper wrt the dataset, algorithmic ideas and empirical (instruction count) improvements over previous SOTA on CompilerGym.\n\n2. How well does Compiler-R1 generalize to programs outside its training distribution? E.g., large programs.\n\nLIMITATIONS:\n- Somewhat. The paper could do a better job of discussing ways in which the empirical results might be invalid or performance might drop. Examples include cases instances where Compiler-R1 might fail to generalize due to OOD inputs as well as the potential for test dataset leakage during training of the base LLM.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}