Title: QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM

Code Generation

Verilog
Abstract: Large language models (LLMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automatically generating hardware description languages (HDLs) like Verilog from natural-language (NL) specifications, however, poses three key challenges: the lack of automated and accurate verification environments, the scarcity of high-quality NL-code pairs, and the prohibitive computation cost of RLVR. To this end, we introduce CodeV-R1, an RLVR framework for training Verilog generation LLMs. First, we develop a rule-based testbench generator that performs robust equivalence checking against golden references. Second, we propose a round-trip data synthesis method that pairs open-source Verilog snippets with LLM-generated NL descriptions, verifies code–NL–code consistency via the generated testbench, and filters out inequivalent examples to yield a high-quality dataset. Third, we employ a two-stage "distill-then-RL" training pipeline: distillation for the cold start of reasoning abilities, followed by adaptive DAPO, our novel RLVR algorithm that can reduce training cost by adaptively adjusting sampling rate. The resulting model, CodeV-R1-7B, achieves 68.6 \% and 72.9 \% pass@1 on VerilogEval v2 and RTLLM v1.1, respectively, surpassing prior state-of-the-art by 12$\sim$20 \%, while even exceeding the performance of 671B DeepSeek-R1 on RTLLM. We have released our model, training code, and dataset to facilitate research in EDA and LLM communities.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ReasoningRTL, a Reinforcement Learning with Verifiable Reward (RLVR) framework designed for generating Register-Transfer Level (RTL) hardware descriptions. The proposed framework comprises three main components:\n- A rule-based testbench generator that produces test cases to verify the correctness of RTL code generated by LLMs;\n- A natural language description generator that converts RTL code snippets into natural language descriptions to serve as inputs for model training;\n- A training framework that integrates cold-start SFT and adaptive DAPO to enhance the model's RTL generation capabilities.\n\nThe framework is used to train a 7B-parameter LLM, which is then evaluated on multiple RTL generation benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\ntrengths:\n+ The paper is generally well-written and well-structured.\n+ The natural language description generator addresses a key bottleneck in RTL generation: the scarcity of high-quality, task-specific training data.\n+ The approach is relatively cost-effective, requiring only 2656 A100-GPU hours for training, which is competitive for LLM development.\n\nWeaknesses:\n- The evaluation results require further clarification. The proposed model shows a notable improvement over existing methods on the RTLLM dataset, yet achieves similar or even worse performance on the VerilogEval benchmark. This raises questions about dataset-specific characteristics and should be more thoroughly discussed. What makes RTLLM particularly amenable to improvement under this framework?\n- Definition 2.1 and Theorem 2.1 are confusing. Given the inherently probabilistic nature of LLMs, the conditions in Definition 2.1 are unlikely to hold in practice, which undermines the applicability and significance of Theorem 2.1. Moreover, they appear to have limited relevance to the rest of the paper.\n- The natural language description generator component would benefit from a deeper analysis. While the appendix includes some illustrative examples, they are limited to short RTL code snippets. In practice, generating natural language descriptions for complex RTL modules, especially those involving non-trivial sequential logic, is a challenging task. Without access to simulation traces, inferring sequential behavior from code alone is difficult even for human experts. This raises concerns about the generator's ability to generalize to realistic and more complex RTL designs.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces ReasoningRTL (R2TL), , a framework to train RTL generation LLM via reinforcement learning with verifiable reward. The framework consists of 3 main contributions: a rule-based testbench generator that performs robust equivalence checking against\ngolden references, a round-trip data synthesis method that creates a high-quality dataset via validating equivalence between original and regenerated code, a two-stage training pipeline for supervised distillation and an efficient RL algorithm (adaptive DAPO). Evaluation shows it surpasses prior state-of-the-art by 12∼21%.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Applying RLVR to RTL generation domain is a new yet important scope.\n- The rule-based test-generation is novel and achieves 35.5 % fewer false negatives and 62.5 % more injected errors detection in fuzzing tests.\n- The round trip data synthesis method is novel and ensures a high-quality training data.\n- The adaptive DAPO leads to 1.25x speedup for RL\n- The framework shows 12-21% improvement over prior SOTA with a 7B model, even better than 671B DeepSeek-R1 in some testing datasets.\n\nWeaknesses:\n- The rule-based test bench generation may not be good enough for complex or real-world Verilog design, requiring manual maintaining with more expert heuristics.\n- While the method outperforms SOTA on RTLLM (v1.1 and v2), it underperforms on VerilogEval, I believe the RTLLM has fewer tests and is relatively easier than VerilogEval.\n- 3.1k data is small (while demonstrated high-quality), which may affect the generalization and robustness of RL training, even induce overfitting to the dataset distribution. This is not well verified and discussed in the paper.\n\nQUESTIONS:\n- How could the R^2TL be potentially combined with agentic methods? Will it still outperform large size model? e.g, MAGE: A Multi-Agent Engine for Automated RTL Code Generation (DAC 2025), which demonstrates  95% pass rate for VerilogEval dataset.\n- What is the complexity of the RTL code (#tokens) obtained from GitHub? Any case with a very long context length that fails with the 16k context length? \n- How does the rule-based test bench generation can be generalized to larger-scale or complex modules/ hierarchical modules? \n- What could be some potential solutions to further reduce the 2.7% wrong flags?\n- How does the method can be potentially generalized to other HDLs like VHDL, System Verilog?\n\nLIMITATIONS:\nSee weakness", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces R2TL, a tool that uses reasoning to generate RTL. The paper goes into rigorous details on how they generate RTL, assess it's quality, and measure the performance of the code generated quality and the code generate performance.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper addresses an important problem, namely RTL code generation. RTL code generation is challenging because LLMs typically don't have enough examples to \"naturally\" produce RTL, plus there are some precision requirements where an LLM cannot make mistakes in (e.g., clk and reset configuration). This makes it a challenging problem, and the paper attempts to address this via reasoning.\n\nI particularly enjoyed the plethora of different ML-based techniques for this system's level problem. This includes the adaptive DAPO for reducing cost, distillation, and focusing on high quality data over too much data. While not particularly novel, the application here to RTL generation is nice, plus the results are decent. Further, the proofs in the appendix, and the data distribution measurements are well received by this reviewer, and hoping more papers employ such rigor in their evaluation.\n\nWhile not a true weakness: the results aren't mindblowing compared to state-of-the-art methods. So the question then becomes is such a complex approach necessary, or will future reasoning model releases potentially outperform R2TL? I still did appreciate the thorough evaluation, but as a user, would the author recommendation be to use R2TL right now, or wait for Deepseek-R1+N? That said, it seems like future R2TL can benefit from a vanilla DeepSeek reasoner and/or coder model (qwen2.5), so perhaps there is lots of merit to R2TL. Could the authors comment a bit about such a scenario?\n\nQUESTIONS:\n+ R2TL versus future reasoners/coder models? Fundamentally, what does R2TL bring to the table beyond SOTA models, and how can it use new models to improve RTL code generation?\n\nLIMITATIONS:\nN/A.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}